You can’t get there from here

Cogent and Telia are having a lover’s quarrel and, as a result, the Internet is partitioned. That means customers of Cogent and Telia cannot necessarily reach one another. This was not due to a configuration error or a physical cable break. This is the way the Internet works and sometimes doesn’t work. If the businesses that run the show don’t play nice with one another, their customers can pay the price of being cut off from parts of the ‘net. At least when Pakistan mistakenly hijacked YouTube, the matter was sorted out in hours and did not require the cooperation of Pakistan. The Cogent/Telia tiff has been going on for 4 days now and only they can resolve their differences. The rest of the world can only hope for full connectivity to be restored.

When relationships end, it’s often hard to figure out the one thing people really want to know: who dumped whom? The politically correct thing to say is that the two parties had irreconcilable differences or that they had simply grown in different directions. That all sounds nice, but in fact, it’s typically the case that one party called it quits, leaving the other broken-hearted. Although routing data don’t show who pulled the plug, the word on the street is that Cogent jilted Telia last Thursday, ending their long-term peering relationship. The result is that the Internet is now partitioned. Downstream customers of Telia and Cogent can only reach each other if they also have providers other than these two. If not, they are out of luck as Cogent plays the Internet equivalent of chicken with Telia.

So let’s look at the facts. On 13 March 2008 at 22:03 UTC, we saw the link between Cogent (AS 174) and Telia (AS 1299) disappear. The way this looks in the routing announcements is that all AS paths with “174 1299″ or “1299 174″ on them vanished, meaning there is no longer a physical link between these two providers. At this point, Telia lost direct access to 4474 prefixes (networks) transited via Cogent, whereas, Cogent lost 1633 Telia networks. Now, you might be thinking, “So what? Won’t the Internet just route around the problem, finding alternate paths?” Well, yes if there are alternate paths to be found and if the players actually allow traffic to flow via them. For around 12 hours, most Telia customers did access Cogent via Verizon (56% of the 4474 networks), Level 3 (16%), AT&T (6%) and others, but then that abruptly stopped. We’re guessing it’s because Cogent eventually slammed the door shut on these alternate paths to their network from Telia, since none of Cogent’s customers accessed Telia via alternate routes during this time. Like divorce court, depeering is supposed to be painful, otherwise you might not get what you want. You only hurt the ones you love.

So now we have a bit of a problem. Customers who ultimately rely solely on Cogent for transit cannot get to Telia, likewise for customers downstream of Telia. So where are these customers exactly? The following tables show the geo-location of those networks downstream of Cogent that cannot reach Telia and vice versa for Telia.

Telia cannot reach Cogent
Country # Prefixes
US 1868

Canada 232

France 98

Spain 41

Germany 31

UK 27

Others 86
Cogent cannot reach Telia
Country # Prefixes
Sweden 444

Finland 322

Russia 153

Poland 113

US 73

Latvia 62

Bulgaria 52

Spain 40

Denmark 35

Norway 30

Others 249

Given the markets Cogent and Telia operate in, these lists are not too surprising. What is surprising is that networks in the US are actually cut off from each another, since a largely US provider is playing hardball with a largely European one. For the Internet to be whole again, Cogent and Telia need to kiss and make up. No one can force either one to carry traffic destined for the other. But my guess is that Telia is hearing more grief from Scandinavian customers not being able to reach US content than Cogent is hearing from US customers cut off from Northern Europe.

Of course, the list of impacted networks is too long to be included here, but they include a wide range of commercial, educational and government clients. On the Telia side, the victims include the Swedish Defense Data Agency, the Finnish State Computer Center, and broadband customers in St. Petersburg. With regard to Cogent, Blue Cross and Blue Shield of Delaware, Kansas State University and Reuters America were all collateral damage.

But people can and do de-peer all the time for business reasons without blowing holes in the Internet along the way.Early on 14 March 2008, Flag (AS 15412) and SingTel (AS 7473) parted ways, but almost all of the few thousand networks carried between the two managed to find alternate paths via their peers or providers. That’s because neither tried to “stick it” to the other and allowed Internet routing to do what it does best, find another way to get from here to there.

10 comments
Mike
Mike

Is there a listing of sites/networks affected broken down to the ISP level? I have searched but have not found any such information. I work for an ISP in the US that contracts through other companies and we have seen some email and connection issues with certain customers around the US.

JZP
JZP

Thanks for the analysis of the results. Regarding the cause, I find it hard to believe (well, much of om malik speculation) that Cogent would be the one considered 'at fault' by anyone other than those with no experience peering with them. At my previous employer, they were always quite quick to upgrade when needed, and push us to upgrade if we were being slow or upgrading inconsistently. Historically, the roots of the Cogent dispute with many european carriers (resulting in a partition with FT) were based upon Cogent's low-cost entry into their assumed-impervious home markets of those PTTs. Might there not be a similar motive on Telia's side? Sadly, I do not have an online reference for what I believe to have been a presentation from the then-Telia peering manager talking about his system for tracking peers and triggers on upgrades and such. I believe it was at the TerreBit Peering Forum (a Terrmark-hosted event, prior to the age of co-operative multi-vendor peering events) in early 2004, and if memory serves the thresholds discussed then were quite high for "efficient link utilization" (often telco-speak for 'running hot'). If I turn up a copy on media will post URLs.

doug
doug

Cogent seems to break the net a lot lately.

unknown
unknown

i wonder if anyone knows when or if they will fix the problem? Thanks

daniel
daniel

thanks for a great post. I have a question about possible outcomes of this, what if telia and cogent can't agree on peering terms, will cogent let telia traffic in from other carriers instead of filtering the traffic, and if not, is that really a viable business model for any ISP, to shut out any carrier that won't accept your peering terms?

queuefull blog
queuefull blog

Cogent - Telia Depeering Apparently there is a peering dispute between Cogent (AS174) and Telia (AS1299). This has caused a partition between their networks, such that Cogent and Telia customers can’t talk to each other directly. As expected, the Renesys blog provides go...

Alex
Alex

Interestingly, I know somebody whose business is affected; he needs to ftp some heavy video files from a production company whose server is in a hostfarm behind Cogent. And he's in Sweden. He's been seeing disruption since last week. There are probably quite a few people in a similar position, as Cogent markets itself heavily to big content replicators.

Marcus
Marcus

Good post, thanks.

CH Internet Szene
CH Internet Szene

Wer hat den l�ngeren Schnauf: Cogent oder Telia? von Fredy K�nzler Das Armdr�cken zwischen Cogent (AS174) und Telia (AS1299) geht munter weiter ... schon sind mehr als 30 Stunden seit meinem Blogeintrag vergangen und es tut sich nichts in Sachen Re-Peering. Daf�r wird in Blogs spekuliert, und man wer

Data Center Knowledge
Data Center Knowledge

Cogent-Telia Peering Dispute Widely Felt Renesys has some additional details on the impact of the peering dispute between Telia and Cogent.