November 2008 Archives

The Internet is in for interesting times. Previously, I wrote about the engineering issues and about the policy issues facing us over the next five years. But there is at least one large issue still lurking. Most of you will not be surprised to learn that almost all of these issues are outgrowths of a single factor: money. The core of the Internet still doesn't have a sustainable business model.

Many people are getting rich on the Internet, and almost none of them are spending money to keep the interconnection infrastructure (the "Inter" in "Internet") growing and expanding. Look at it from a massively oversimplified perspective: Google make their money from the advertising they sell to search audiences. Comcast make their money by offering TV and Internet access on their local cable infrastructure. Amazon make money selling books and other stuff (including servers and storage space). Most datacenter companies make their money selling space and power inside of their buildings. Spammers make money filling up your inbox with useless crap. Organized crime makes money by launching attacks against profitable companies if they don't pay extortion. DNS squatters make money registering thousands (or millions) of domain names and sitting on them until someone else is willing to pay. And almost none of this helps the core of the Internet.

Look to the wholesale carriers if you want to see an income statement wasteland. Level 3 lost $1.1b last year. They lost $120m in the most recent quarter alone. Cogent is thrilled because they reported a tiny, tiny positive net income last quarter on top of a yearly loss of $30m in 2007. Global crossing lost $300m in 2007 and $88m in the last quarter they're reporting, which doesn't include much of the recent downturn. Other wholesale networks are in the same boat. Dan Golding suggested that it's more important to look at net cash flows rather tha income, but the result is pretty much the same: almost no one is making any money. The only wholesalers who do make money make it on other service offerings: wireless service, metro Ethernet services, VPNs, local phone service, video services and so on. Are there sustainable Internet backbone business models? Does anyone have one?

An open market for buying and selling IPv4 Addresses is coming. Soon. As I wrote previously, IANA is running out of unallocated IPv4 addresses. Estimates vary, but by 2010 (or 2012 at the latest) the world will be out of unallocated IPv4 addresses.

Sometimes it is hard for the general public to understand what this might mean. Essentially, after 2010 or so, if you want to start a new company and get connected to the Internet or just are growing and have more devices that need to have IP addresses, things won't be the same as they are now. Right now what happens is that you go to ARIN, if you're in North America and document your need for IP addresses, you pay a modest administrative fee, and then they allocate them to you. If you grow and you need more, you document how you've used up the ones that you have, and they give you more of them.

All of this assumes that you want your own IP addresses that are not tied to any particular provider (this is an important point that we'll get back to). But even if you get your IP addresses from some provider, they have to get them from somewhere. If you want to be reachable from the Internet, you need an IP address—an IPv4 IP address in particular. And very shortly those are going to get much harder to get.

So let's talk about what happens after the IPv4 addresses are all "used up."

In a few weeks, I will be leaving Renesys, a company I have been associated with for over five years. I moved from New Hampshire (where Renesys is headquartered) to Pittsburgh, PA, over the summer, and I've decided to work a bit closer to my new home.

Before I go, there is work yet to be done. The Renesys blog has become an important place for Internet engineers, managers, developers and salespeople to seek unbiased information about what is happening on the backbones. I have enjoyed contributing to it over the years, and I have enjoyed watching some of my colleagues (most actively Earl Zmijewski and Martin Brown) take the helm more recently. Before I ride off into the sunset, there are at least two things I'd like to contribute to this forum:

1. A clear assessment of where we are with this whole Internet project
2. A good guess about where we're going

At the end of the next series of posts by me, you should either be very, very worried or convinced that I'm very, very wrong. The Internet is facing a confluence of engineering, financial and policy storms that have some small potential to completely derail it. These tempests have a high likelihood of marking a sharp departure from several characteristics once considered fundamental to the the Internet.

If we get through the next five years, I'm sure everything will be fine. Today, I'll tackle the technology and engineering issues. In my next post, I'll address financial issues, followed by policy issues. At the end of this torrent of pessimism, I'll try to point to some plausible ways out of the mess that we have gotten ourselves into.

Sprint re-enabled the connection between Sprint and Cogent at 21:00 UTC (16:00 EST) on Sunday, 2 Nov, 2008. Sprint issued a hastily prepared statement about the reconnection (the HTML is a cut-and-paste job from "IP/MPLS Products from Sprint"), explaining their position. Cogent hasn't commented yet.

The connection appears to be routed much as it was before Oct 30. Previously, we saw Sprint selecting 2700-2900 prefixes from Cogent (that is, picking Cogent as the best path for that many network prefixes). We saw Cogent selecting about 7500-8000 prefixes from Sprint. Now that they have reconnected, Sprint is selecting 2538 prefixes from Cogent and Cogent is selecting 7016 from Sprint. So down slightly, but not appreciably. The link is up.

The fact that Sprint has reconnected this indicates clearly that they intend to fight this battle in court rather than in the routing tables or in the court of public opinion. This fact alone makes this likely to be one of the more interesting peering disputes of the last few years. But the resolution may take months or years, given the speed with which the courts move.