Lights Out in Rio

| 1 Comment | 1 TrackBack
When the power goes out to a large part of Brazil, as happened last night shortly after 10pm, it's going to have an impact on telecommunications.

When power drops, there's an immediate shock to the Internet routing system. You can see it clearly in this plot, which shows the number of South American networks affected by serious instability or unreachability in the previous 15-minute window.

sa2-ren_wm.jpg

By compiling the routing traffic we collected from all over the world during this period of time, it's possible to pinpoint the time of the failure in the Brazilian electric grid (10:26pm, or 00:26 UTC). From an essentially negligible background level, the number of affected networks vaults to more than 150 within seconds. Following the classic pattern of widespread power failures, the number of freshly affected networks drops over time as backup power kicks in, and then there are 'echoes' of new failures as the backup power suffers its own problems.



br-earth-ren_wm.jpg   ihp-sa-ren-names_wm.jpg

These are some screenshots from our KML-based regional outage visualizer, and our Internet Health Portal tool. As you can see, Brazil took the largest hit, but we also saw Paraguayan and Uruguayan networks out as a result of the largest regional power outage to hit Brazil and its neighbors in several years.

Inevitably, speculation will fly fast and furious about the cause. Media reports this week suggested that hackers may have taken down the Brazilian grid in 2005 and 2007. Brazil says it simply wasn't so. Was tonight a sardonic repeat performance, in response to Sunday's 60 Minutes coverage?

1 TrackBack

This post was mentioned on Twitter by abnerg: Nice view of network traffic when power went out in Rio from the Renesys Blog http://j.mp/3zr6lX Read More

1 Comment

Thanks for the analysis.

I use rv2 dataset to analysis the impact of the power outage, and the result shows that outage was way more high than your chart above. I guess maybe it's because rv2 has only some limited view.

About the Renesys Blog

Our weblog is written by a variety of Renesys employees. They run the gamut from senior execs and engineers to sales guys. Anyone who has something to say that could be informative or of interest to our customers and visitors, says it here.

About this Entry

This page contains a single entry by James Cowie published on November 11, 2009 1:43 AM.

Staring Into The Gorge: Router Exploits was the previous entry in this blog.

IP Backbone: Hard sell, not so much is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Archives

Pages