Another possibility is a simple configuration error. They may have changed a router config (to shut down access) but not saved those changes to memory. The next time that router rebooted (say, for early morning network maintenance or generator testing), it would have come back without those changes until someone noticed and re-applied them.
The Battle for Tripoli’s Internet
We’re still piecing together the data that can confirm or deny much of what’s been reported overnight, but one thing is clear: something very strange was going on with Tripoli residents’ Internet access. Service was restored suddenly in Tripoli, flickered on and off for a couple of hours, and then died, with the majority of the country’s international BGP routes withdrawn from service for good measure. Today the routes are back in Tripoli, but ADSL service isn’t. This morning we’re looking back at this curious overnight episode, and speculate about what might have happened.
As dawn broke in Libya on the morning of Sunday 21 August, it appeared that the battle for control of Tripoli was underway. Throughout the night, a steady stream of tweets and retweets emerged from Libyan sources, painting a confusing, often contradictory picture of the evolving situation.
For several months, our picture of the Libyan Internet has been essentally static. The national connection to the Internet consists of 16 blocks of IP addresses, each routed to the outside world through Libyan Telecom and Technology. That basic routing footprint has been advertised to the world, with few interruptions, since the end of the March Internet blackout.
But average people in Tripoli still haven’t had much access to the Internet, because local DSL service has been largely unavailable for the last three to five months, depending on where you live. The few people who did retain their official Internet service continued to access Google and YouTube, as measured by Google’s excellent Transparency Report. When LTT’s international Internet connection started to show sporadic signs of failure a couple weeks ago, it only affected 11 of 16 blocks, leaving intact the neighborhoods who appear to have been generating the majority of the country’s surviving Google traffic.
With so much of Libya’s Internet disconnected in the last mile, there are few alternatives, other than outlawed satellite phones or international dial-up. These are dangerous, slow, expensive ways to get the news out.
Operation “Mermaid Dawn”
Nonetheless, it became apparent from the Libyan Twitterstream over the last couple days that things were about to heat up in Tripoli. It seemed likely that mobile networks, and perhaps the entire phone system, could be shut down within the capital, as the government attempted to prevent the Tripoli uprising from self-organizing. There were sporadic tweets about phone calls not completing, but the expected telecoms shutdown never came.
Instead, Al Jazeera began to report that Tripoli residents were receiving mobile phone text messages, urging them to take to the streets (typically, in the fog of war, it seems unclear whether they were being exhorted to support the government or the rebels). And early Sunday morning, the Twitterstream suddenly began reporting something that seemed, on the face of it, totally improbable: the Internet had been turned back on.
Why would the government turn the Internet back on in the middle of an armed uprising? To get people to stay at home and catch up on five months of email? It seemed preposterous. But clearly, as more and more people realized, it had happened. Bandwidth was scarce, but DSL service was back. People started Skypeing with friends and relatives, some reporting hearing live gunfire in the background as their VoIP calls began to connect.
And then, as suddenly as it had come, Tripoli’s Internet access stopped working again. For a total of perhaps an hour and a half of uptime, spread out in bursts between the hours of 2:00am and 4:30am, local time, the Internet had been functional again. Who was responsible? Would it come back?
Forty-five minutes later, at 5:14am local time, we may have received an answer of sorts. Eleven of LTT’s 16 international routes to the Internet were withdrawn from the global routing table: the same eleven routes that had been sporadically unstable two weeks ago. (Note the drop at the right edge of the plot, below.) They stayed down for the next four hours, and came back up at 9:22 local time. While these routes were down, the up-or-down fate of the corresponding local DSL or Wimax services became essentially irrelevant; their plug had been pulled at the international border, and nobody in those local networks could exchange traffic with the world outside Libya, until LTT chose to reestablish routing for them.
Now, even though the routes are back in place, local Internet service appears to be down again — the status quo ante for the last five months. We note that LTT’s website is still down. Did the brief Tripoli Internet flicker represent a sign of conflict within the phone company itself, with someone struggling to reactivate service at the neighborhood level, only to have it switched off again at the national level? Or was the overnight routing failure just another in a sequence of (probably power-related) outages for LTT’s outlier networks? The people without Internet access in Libya have a lot of questions at this point, and we don’t have enough data yet to give them a satisfactory answer.
Update (12:00am Monday in Tripoli): LTT website (http://ltt.ly) is back online, and the Arabic crawl at the bottom says “Congratulations, Libya, on emancipation from the rule of the tyrant.”