I get why people are paranoid about ISPs blocking content and net neutrality, but let's not cry wolf prematurely. The technical details here strongly suggest a bug rather than intentional blocking of 1.1.1.1 DNS traffic.
CF CEO tweets that 1.0.0.1 is also blocked.
https://twitter.com/eastdakota/status/991718955021623296
Others have confirmed that the ipv6 address belonging to CF appears to be blocked.
""With the recent launch of Cloudflare's 1.1.1.1 DNS service, we have discovered an unintentional gateway IP address conflict with 1 of their 4 usable IPs and are working to resolve the issue,"
https://arstechnica.com/information-technology/2018/05/att-i...
A few of you will be disappointed to know its not a evil attempt to block you from using it. Same way they have literally never blocked the ability to use any other DNS service before.It's simply a bug caused by the way the BGW-210, and Pace 5268AC operate and make use of 1.1.1.1 internally in some way and it will be fixed with a firmware update.
Maybe the firmware update has a bug, but it's very suspiciously timed. Notice that the OP is dated April 2, while 1.1.1.1 was released April 1.
I have ATT U-verse internet service and use their Arris BGW210-700 gateway
One interesting thing is that if I go to the gateway management page, and use their diagnostic tools, I'm able to ping / traceroute the address - but I can't from any devices connected to the gateway
From gateway diag page:
PING 1.1.1.1 (1.1.1.1): 56 data bytes 64 bytes from 1.1.1.1: seq=0 ttl=64 time=0.568 ms 64 bytes from 1.1.1.1: seq=1 ttl=64 time=0.156 ms 64 bytes from 1.1.1.1: seq=2 ttl=64 time=0.164 ms 64 bytes from 1.1.1.1: seq=3 ttl=64 time=0.144 ms
--- 1.1.1.1 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 0.144/0.258/0.568 ms
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 38 byte packets 1 1dot1dot1dot1.cloudflare-dns.com (1.1.1.1) 0.285 ms 0.177 ms 0.090 ms
The times on the pings make it look like its hitting a loopback address instead. Pings to 8.8.8.8 from the diagnostics page take about 23 ms. No way 1.1.1.1 is completing in under 1ms haha
They had the choice of "fix the whole backend" or "block 1.x on the user end".
Guess we know which one was easier. If all this wild speculation is true, maybe they're working on a fix to the root cause and will roll back the patch when complete.
This would make the situation both due to incompetence and intentional.
Maybe stupid question, but why would AT&T block it?
Back in 2010 there were problems that came up when IANA started allocating out of 1.0.0.0/8 (e.g. [1]). Things that were once assumed to be unused started being used, leading to strange issues.
Also, why on earth would AT&T block 1.1.1.1 and not Google DNS and OpenDNS?
[1] https://bgpmon.net/issues-with-allocating-from-1-0-0-08/
Also- If this was intentional- I'm betting they'd filter it for the mobile network as well. This has got to be a fuck-up.
Just like how only the true messiah denies his divinity, it doesn't give innocent bugs much of a chance.
In fact, now we can show that all bugs are suspicious, with apologies to the interesting number paradox:
The least intentional looking bug is the most effectively hidden, and therefore should probably be suspected of being intentional. Since it's now suspect, it's longer the least intentional looking bug, so the next least suspicious bug suddenly deserves a bit more scrutiny, and so on.
D
Blocking 1.1.1.1 and 1.0.0.1 -> what are the odds here?
Remember to scrub EXIF data!
- If the action was malicious, the people involved in writing this code are likely okay with it and not likely to leak details of it.
- If the issue is a bug, the people involved in writing this code are probably working to fix it, and not likely to leak details of it.
- People not involved with making it would likely leave an internal access trail (independent of EXIF data) when they access that code.
Which is to say, expecting an Ed Snowden every time a company does something unethical is kinda silly, otherwise we'd have Google's search algorithm by now.
Lots of cisco example config use 1.1.1.1 for router internal identifier / DHCP server / OSPF dummy network .
Not suprised if it break anything.
AT&T routers also don't let you use a 10.x address at home (possibly to prepare for carrier grade NAT, although there is an official 100.x address reserved for that; so fuck you ATT).
I'm so sick of my AT&T router/modem for various other reasons. I hate how you are required to use it for many of their offerings (including Fiber to the home).
There are a number of tools out there for putting their router behind your Linux box. Most of them configure ebtables or use scripts to forward the 802.1q authentication packets to/from the router.
Hanlon's razor: https://en.wikipedia.org/wiki/Hanlon%27s_razor
But hey, if you have advanced needs, no problem, let me refer you too our Gaming Provider and Streaming Provider subsidiaries.
Oh you need actual technical access to the internet because you write your own software? Tricky, but I'm sure our Business Technology Services Provider subsidiary will have the service you need. (You do have a business, right?)
I really love your idea.
Also see the UK as well for an example of how previously unregulated speech has become regulated because the authorities have pushed over and over again, backing off every time there's a loud enough protest, but trying again after a short time.
It's likely incompetence, not malice. If they didn't want people using other DNS, and were willing to fuck with ip addresses they don't own to accomplish that, they'd be blackholing google's and opendns's public caching nameservers too.
It might even have been a conscious decision. Even though it's horrible and the people involved in developing the firmware need re-education. The decision probably went like this: we need an internal address to do something. We can't use 10, 172.16, or 192.168 ranges because those might conflict with internal LANs. 1.x is safe because we all know nobody uses them. The correct decision obviously would have been to get at&t corporate to commit to never using some tiny corner of their address space, and use that. Or 127.a.b.c if that works on the OS. Those options are only needed if they really need an extra IP address. They might not need one after all if they designed their firmware better.
There is not enough data to attribute this to malice yet, but it does not look good (see CloudFlare's tweet).
I think they're blocking 1.1.1.1 because customers are now using DNS that isn't them, which deprives them of valuable data on which domain names their customers go to, which they can sell to advertisers. Yes, there's other ways to get that information but the DNS server is an easy one.
What's the theory exactly? What would be the benefit for AT&T to block a new 3rd party DNS? Did they do similar things in the past for other 3rd party DNSs such as OpenDNS, Quad9 or Google's? Seems odd to target this one service in particular.
The ship may have sailed on blocking 8.8.8.8 at this point; some things _hard-code_ it.
[1] According to google, it's defined as:
"the principle that Internet service providers should enable access to all content and applications regardless of the source, and without favoring or blocking particular products or websites."
Well, since the formal repeal of net neutrality has been delayed, I think it technically is a violation of the no-blocking rule.
OTOH, it's not like the FCC is enforcing net neutrality while delaying the official effect of its repeal.
Clearly, they’re discriminating against certain client devices, and were under the Obama administration too.
However, the documentation says it should work, and AT&T won’t provide support.
They’ve been getting away with this for years, so I guess plausible deniability (it is “just a bug”) can work wonders in this space.
Still, it looks more like malice since there are other addresses besides 1.1.1.1 that are also blocked.
By their own admission CF receives a ridiculous amount of garbage traffic at this IP, it was not absurd for AT&T engineers in the past to thing "well, we need an IP that we can be reasonably sure nobody is going to use and is never going to conflict with anything on any network, 1.1.1.1 seems reasonable". Seeing everybody in this thread jumping into conspiracy theories instead of the much more likely configuration issue is a bit disappointing for a community that's supposed to understand technology.
This caused issues where my phone would try to get on wifi, but the DHCPACK would be sent along on the existing interface rather than the one coming up. So the wifi icon was continually bouncing back and forth. My only solution was to go into airplane mode and bring the cellular down before bringing the wifi up. I don't think Android ever addressed this issue, and I had to switch around the entire subnet to avoid the conflicts.
If I knew enough about how Android worked, I'd write a patch to have all android interfaces in their own linux netns, with the dhcp client exec'd in that netns, that way you'd never have to worry about this sort of conflict.
stupid to think using the IP was a good idea
malice to break my device in order to paper over their stupidity.
And from a telecom no less - of all people, they should know better.
Technological websites noted that by using 1.1.1.1 as the IP address for their service, Cloudflare created problems with existing setups. While 1.1.1.1 was not a reserved IP address, it was and is used by many existing routers (mostly those sold by Cisco Systems) and companies for hosting login pages to private networks, exit pages or other purposes, rendering the use of 1.1.1.1 as a manually configured DNS server impossible on those systems. Additionally, 1.1.1.1 is blocked on many networks and by multiple ISPs because the simplicity of the address means that it was previously often used for testing purposes and not legitimate use. These previous uses has lead to a huge influx of "garbage" data to Cloudflare's servers.
A wake-up call for all those (ab)users of public address space is also desperately needed. All IPv4 addresses will soon be allocated. Failure to use only private address spaces will cause problems, very soon.
CloudFlare likely did this on purpose, because so many people can't get their heads out of their own asses and follow spec. Now there's a big spotlight on the people purposefully breaking the network. And it will be fixed, eventually, whereas previously, AT&T would have just said "take a hike".
RFC 5735:
> 192.0.2.0/24 - This block is assigned as "TEST-NET-1" for use in documentation and example code. It is often used in conjunction with domain names example.com or example.net in vendor and protocol documentation. As described in RFC5737, addresses within this block do not legitimately appear on the public Internet and can be used without any coordination with IANA or an Internet registry.
> 198.51.100.0/24 - This block is assigned as "TEST-NET-2" for use in documentation and example code. It is often used in conjunction with domain names example.com or example.net in vendor and protocol documentation. As described in RFC5737, addresses within this block do not legitimately appear on the public Internet and can be used without any coordination with IANA or an Internet registry.
> 203.0.113.0/24 - This block is assigned as "TEST-NET-3" for use in documentation and example code. It is often used in conjunction with domain names example.com or example.net in vendor and protocol documentation. As described in RFC5737, addresses within this block do not legitimately appear on the public Internet and can be used without any coordination with IANA or an Internet registry.
But CloudFlare has addressed this with lots of vendors. This update is a new one.
Really the only place I saw 1.1.1.1 regularly though is to set the router ID, and making a loopback address is not the best way to do that to begin with.
I've been using 1.1.1.1, and today went to the library for a quick work break. I pulled out my laptop and tried to connect to the wifi, and it wasn't working. After a few minutes of troubleshooting, I tried deleting my custom DNS entry in my network settings and that did the trick.
I guess the library uses AT&T routers.
Same goes for https handshakes leaking your target domain(otherwise SNI wouldn't work), so DNSSec alone is fairly pointless for regular web traffic obfuscation; and of course the IP is in each TCP frame regardless.
It becomes more a matter of are they doing it yet(re:DNS monitoring in this manner); with enough people using third party resolvers(I'd argue google's public DNS already has enough usage to warrant it) they will be.
Optimally you'd VPN at all times to a provider you trust or one you've setup yourself.
What it all really boils down to though is that the populace simply can't be trusted(nor should they need to be) to make themselves acceptably secure from third party monitoring. We need to have much more discussion around data privacy and retention for ISP's.
It's not a matter of if the data will be misused, it's truly a matter of when and it's not fair to the general public.
It's not a good solution for me, however, because I run PFSense, which is FreeBSD-based and lacks the PF_RING socket support to filter out those EAP packets. As far as I know, PFSense's PF packet filter cannot strain them out, either. Traditional libpcap is available on FreeBSD (slow) and netmap (fast), too. I looked into writing an EAP proxy in Go using a special netmap-enabled libpcap but it was way too much yak shaving and I eventually gave up. I should take another look, or maybe learn enough C to do it natively with netmap. My goal is native EAP proxy support for PFSense that can support filtering EAP out of a wirespeed gigabit fiber connection.
What I don't understand is how this really helps user privacy much. If AT&T, Comcast, etc want to know your browsing habits, can't they still see the IP addresses you're browsing and figure out the URL from the IPs? I can't see that as too big an impediment, but maybe someone with more knowledge can share.
Regarding privacy, Cloudflare are at least saying they aren't spying on you. Your ISP may not even be saying this. Also, Cloudflare don't necessarily have access to your name and address, whereas your ISP does. Also, many different sites can be hosted on the same IP address, so merely tracking the IP addresses a client is connected to won't necessarily tell you what sites they're visiting.
FWIW, it’s possible to bypass AT&T’s router:
https://github.com/jaysoffian/eap_proxy
That said, I tried 1.1.1.1 and found I had to switch back to Google DNS since Cloudflare intentionally doesn’t support EDNS Client Subnet which was causing my AppleTV’s to have trouble loading content.
Also I've heard that their routers report your entire network topology back when they phone home.
The best you get is a "passthrough" mode where a router behind it will get assigned the public-facing IP so it doesn't ultimately behave like double-NAT, but the gateway still maintains an internal NAT table for everything going across.
You can't just use your own VDSL modem or plug your own router into the fiber ONT, as AT&T uses 802.1x auth and the key is burned into the gateway hardware.
It works fine when I disable WiFi on my phone (Verizon).
Hanlon's razor as lots of DNS services are available on not as vanity IP space, and there is no evidence of blockage.
I have one of those routers, and I couldn't use 1.1.1.1 because it was routing to an internal interface on the router. I confirmed this with ping, I was getting microsecond response times from 1.1.1.1.
Under the new firmware, 1.1.1.1 is just dead. So it's probably still connected to the local interface, and nothing is listening.
Cloudflare DNS seems to be down for couple of major ISP's in India as well according to CF forums -
[ACT] https://community.cloudflare.com/t/cloudfare-dns-blocked-wit...
[Airtel] https://community.cloudflare.com/t/cloudflare-dns-not-workin...
Frankly, NEVER paying the bill is an option, too. Downloading Netflix is sweet, maybe you can pool with your neighbor? that's another topic
It's expensive to enforce payment.
If you've never been in collections, it's an experience you might enjoy for sport.
If you live in fear of not being able to get a cheap interest rate on a loan for some shit you don't need... well, maybe you'd better not take part in that type of protest.
Try to avoid the cheap bundled cable/fibre/DSL routers that ISPs "throw in" with their plans/packages.
Disable the remote management/update/TR-069/CWMP/SSH/etc if you can. You don't wanna trust someone else to secure your home.
You can turn off the Wi-Fi on AT&T's gateway and run your own router behind the AT&T hardware. But since your router is behind the gateway everything still goes through it and AT&T still can do all the CWMP stuff to their gateway.
First time I saw it was 8.8.8.8.
I personally had one had in my head from the 80s 128.252.120.1. bit it is obviously not a special one.
They had acknowledged to themselves going into it that the IPs weren't "normal". They could have easily chosen a safer range if that was a priority.
"AT&T firmware update blocks access to 1.1.1.1"
would be more accurate IMHO.
Comcast will happily block ports to “protect” me:
https://www.xfinity.com/support/articles/list-of-blocked-por...
# traceroute 1.0.0.1
traceroute to 1.0.0.1 (1.0.0.1), 30 hops max, 60 byte packets
1 45-18-124-1.lightspeed.austtx.sbcglobal.net (45.18.124.1) 59.462 ms 61.348 ms 63.373 ms
2 71.149.77.208 (71.149.77.208) 1.304 ms 1.695 ms 1.957 ms
3 75.8.128.136 (75.8.128.136) 1.329 ms 1.682 ms 1.393 ms
4 12.83.68.145 (12.83.68.145) 2.673 ms 2.661 ms 2.648 ms
5 12.123.18.233 (12.123.18.233) 8.877 ms 12.753 ms 8.800 ms
6 192.205.36.206 (192.205.36.206) 6.663 ms 6.375 ms 6.680 ms
7 66.110.56.158 (66.110.56.158) 6.885 ms 6.725 ms 6.436 ms
8 1dot1dot1dot1.cloudflare-dns.com (1.0.0.1) 6.855 ms 6.557 ms 6.662 ms
# traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets
1 45-18-124-1.lightspeed.austtx.sbcglobal.net (45.18.124.1) 163.322 ms 163.927 ms 174.243 ms
2 71.149.77.208 (71.149.77.208) 1.346 ms 1.779 ms 2.035 ms
3 75.8.128.136 (75.8.128.136) 1.215 ms 1.214 ms 1.564 ms
4 12.83.68.137 (12.83.68.137) 1.495 ms 12.83.68.145 (12.83.68.145) 2.289 ms 12.83.68.137 (12.83.68.137) 2.283 ms
5 12.123.18.233 (12.123.18.233) 7.783 ms 11.766 ms 11.757 ms
6 192.205.36.206 (192.205.36.206) 6.163 ms 6.160 ms 6.202 ms
7 66.110.56.158 (66.110.56.158) 6.909 ms 6.931 ms 6.423 ms
8 1dot1dot1dot1.cloudflare-dns.com (1.1.1.1) 6.922 ms 6.492 ms 7.075 ms
; <<>> DiG 9.9.5-9+deb8u14-Debian <<>> cloudflare.com @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15100
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1536
;; QUESTION SECTION:
;cloudflare.com. IN A
;; ANSWER SECTION:
cloudflare.com. 53 IN A 198.41.214.162
cloudflare.com. 53 IN A 198.41.215.162
;; Query time: 7 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Thu May 03 13:40:52 UTC 2018
;; MSG SIZE rcvd: 75
; <<>> DiG 9.9.5-9+deb8u14-Debian <<>> cloudflare.com @1.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61685
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1536
;; QUESTION SECTION:
;cloudflare.com. IN A
;; ANSWER SECTION:
cloudflare.com. 66 IN A 198.41.214.162
cloudflare.com. 66 IN A 198.41.215.162
;; Query time: 7 msec
;; SERVER: 1.0.0.1#53(1.0.0.1)
;; WHEN: Thu May 03 13:40:39 UTC 2018
;; MSG SIZE rcvd: 75
Doesn't look like it's anything onn AT&T's internal network.
You can also do essentially the same thing with a userspace 802.1x proxy like this one: https://github.com/SeanMollet/1x_prox
Bypassing the router ensures that stupid router firmware does not do stupid things to my packets, such as special handling of public IPs.
The abortive never-implemented two-year dalliance with Title II doesn't count.
... that said, I'm going to fall in the camp of stating that this is likely an unintentional bug. If they truly wanted to block 1.1.1.1 (and it's backup), doing so via firmware would seem to be the most difficult and unreliable way of doing so. The benefits of doing so are also limited: (a) If the motivation was to avoid losing the ability to spy on their customers via DNS requests, well ... they can still do that. Yes, Cloudflare supports encrypted DNS, but the half of one percent of folks who have this set up wouldn't be worth the effort[0]. (b) If there was some other reason to want customers using their DNS (i.e. redirection to advertising pages when lookup fails), they could simply do packet rewrites (of non-encrypted DNS lookups) to send them over to AT&Ts infrastructure -- the benefit of doing this is that it would be more likely to go unnoticed[1]. (c) There have been several other, far more popular and just as well publicized public DNS services that they haven't messed with -- why pick on a new entrant -- why not break 8.8.8.8 or OpenDNS?
More likely is the explanation that 1.1.1.1 was being used as a defact-o 10.x.x.x address for other purposes. It had a few benefits -- it was far less likely to be used as an internal address for customers (being ... not a traditional non-routable address) and up until recently, it was unlikely to be used for legitimate services. Or ... it's something else. Firmware bugs are everywhere and having had their service and the particular brand of modem they're using, I'm not the least bit surprised. I had to root my modem to make my service work reliably[2]. Heck, I worked for a telecom for 17 years, and the first half of that, the guy who set our network up used 1-10.x.x.x as internal addresses.
[0] It's not terribly difficult to do, but few take the effort. I've got an internal DNS server configured (for AD purposes) which forwards to another internal DNS server that makes all DNS requests out to cloudflare via encrypted DNS. It was a 5 minute change to my internal setup, a lot of which was the time it took to download the container, reboot the host for testing purposes and validation of everything.
[1] It probably would have managed to be hidden an entire minute longer than this debacle.
[2] On their DSL (re-labeled U-Verse despite it having nothing to do with their U-Verse TV/Internet -- it's the old DSL limited to 12Mb down if you're lucky), my modem would randomly display the "Internet is down" page for all requests despite everything being fine. I forgot, exactly, what I had to do to resolve it, but it required hitting their ping page to trigger a buffer overflow, allowing me to get console access and running some command. I also wanted to be able to ping the modem remotely (something they disable with no customer-facing option to correct) to correlate it with weather so as to prove to customer service (...and at least a little to myself) that this bizarre happenstance wasn't all in my head. My next-door neighbors also had this problem, so I suspected it was something in the wiring (expansion/contraction-like) up the street, but it was hard to track down where because all but two people on that street (including us) used those homes as summer vacation homes and were rarely there in the winter -- many didn't have service and those who did were unlikely to be around when the weather hit about 40 degrees, so AT&T wasn't getting reports of outages in enough frequency to do anything about it. Two years ago, they sent a truck, took everyone down and re-did a pole 8 houses down. Since then, the problem hasn't happened.
Cloudflare is using a conventional IP, you are the one that isn't.
>We talked to the APNIC team about how we wanted to create a privacy-first, extremely fast DNS system. They thought it was a laudable goal. We offered Cloudflare's network to receive and study the garbage traffic in exchange for being able to offer a DNS resolver on the memorable IPs. And, with that, 1.1.1.1 was born.[0]
[0]https://blog.cloudflare.com/announcing-1111/
It's not a reserved address like 192.168.0.0/16 or 10.0.0.0/8[1][2], nor is it one of the other reserved addresses for documentation or testing. So I think people using it before as test or LAN addresses are actually in the wrong here. This kind of "tradition" in networking is wrong. That's what things like RFC's are for.
[1]https://tools.ietf.org/html/rfc5735
[2]https://www.iana.org/assignments/iana-ipv4-special-registry/...