Earlier today several people noticed network reachability problems for networks such as Twitter, Google and others. The root cause turned out to be another BGP mishap.
— Fusl Neko Shy Dash (@OhNoItsFusl) October 21, 2017
Between 11:09 and 11:27 UTC traffic for many large CDN was rerouted through Brazil. Below an example for the Internet’s most famous prefix 22.214.171.124/24 (Google DNS)
At 2017-10-21 11:09:59 UTC, AS33362, US based ISP saw the path towards Google’s 126.96.36.199/24 like this:
33362 6939 16735 263361 15169
This shows the US based network AS33362, would have sent traffic to Google via 6939 (HE) to 16735 (Algar Telecom, Brazil), to 263361 infovale telecom which would have tried to delivered it to Google. The successful delivery of packets would have been unlikely, typically due to congestion which would have been the result of the increase in attracted traffic or an ACL blocking the unexpected traffic.
In this case 263361 should have never sent this BGP announcement to 16735 and 16735 should have not passed this on to 6939. It’s a clear example of a BGP leak.
Let’s look at another example for Netflix’s prefix 188.8.131.52/24 which appears to be hosting nflxso.net DNS services. This example AS path was observed by a Canadian network AS14442:
14442 174 16735 263361 2906 40027
Or another example as seen by AS1126 based out of the Netherlands:
1126 6939 16735 263361 2906 40027
In both cases the path unexpectedly traverses 263361 infovale telecom and 16735 (Algar Telecom, Brazil). The Canadian network learned this Brazilian path via Cogent, where the Dutch network saw it via HE (6939).
These are just two examples, there are plenty of other examples involving many other prefixes and ASns. In most cases the root cause appears to be related to the ‘16735 263361′ relation where 263361 announced peer routes to Algar AS16735, which then passed that along to its peers and even transit providers such as Cogent.
So if you are wondering why your tweets didn’t load, or had issues resolving DNS names, or other reachability issues between 11:09 and 11:27 UTC, this could very well have been caused by this accidental reroute through Brazil.
Read more here:: BGPmon