Two weeks ago, I had the great pleasure of visiting Medellin, Colombia to present at the Latin American and Caribbean Network Operators Group (LACNOG) portion of LACNIC 19. Medellin is a vibrant place, recently recognized as the world's most innovative city by the Urban Land Institute due to the city's investments in public infrastructure and civic spaces.

Perhaps equally innovative is Colombia's Internexa, which in recent years has been building the region's first international terrestrial telecommunications network. Meanwhile, another remarkable regional story is the exponential growth of the domestic Internet in Brazil — especially when contrasted with the stagnation in Mexico. While government initiatives in Brazil, the region's largest economy, were able to foster much of its recent growth, the current regulatory overhaul in Mexico hopes to achieve something similar in the region's second largest economy.

In the presentation, I discussed where we did and did not see Internet growth (crecimiento in Spanish) in the Latin American region. To measure growth, I borrowed a metric, domestic ASN growth, from my colleague Jim Cowie's recent presentation on the growth of the Internet of the Middle East at MENOG 12 in Dubai.

For each country, we counted up the number of domestic Autonomous Systems (ASNs), where an ASN is considered domestic in a country when at least 70% of its customer base is in that country. Of course, ASNs can vary dramatically in size, from large companies to small non-profits. But regardless of size, our metric accurately captures the "number of Internet players" in a country. It is the relative change in this count over time that is most informative, as shown in the following table for a subset of Latin American countries:

Our data illustrates very robust growth in Brazil, Argentina and Costa Rica since January 2010. During this period, Brazil grew by 340% and now exceeds the size of the rest of the region combined. By this metric, Brazil adds the equivalent of two Mexicos to its domestic Internet each year.

Dramatic growth of the Internet in Costa Rica is a result of the telecommunications regulatory reform of 2008. The reform was a requirement of the CAFTA-DR trade agreement, which mandated that the government of Costa Rica end the monopoly of ICE, set up a telecoms regulator Sutel, and allow new competition. Although Costa Rica was one of the smaller countries in our analysis, it had the largest relative growth in the region. New players in Costa Rica include new telecommunications providers as well as new enterprises operating in the country.

Brazil

Brazil's tremendous growth has been a result of a national program of infrastructure investment and government incentives, which was described at LACNIC 19 by Artur Coimbra of the Brazilian Ministry of Communications. In addition to upgrading and expanding the domestic backbone of Telebras, Brazil has established 22 municipal Internet exchange points called PTTs (Pontos de Troca de Tráfego) to facilitate domestic exchange of Internet traffic. In his presentation, Wardner Maia, President of the Association of Brazilian Internet Providers (ABRINT) discussed the recent explosion of small regional Internet service providers serving parts of Brazil that hadn't previously had access to the Internet. On average, he said, the Brazilian government was issuing over 700 licenses per year to these new providers, 78% percent of which are considered small companies. When we compared our data, it was clear that the growth of domestic ASNs matched the growth in licensing that his organization was witnessing.

We're not the only ones witnessing Brazil's growth. In their most recent State of the Internet report, content provider Akamai noted the largest year-to-year increase in unique IP addresses reaching their servers was from Brazil (a 33% increase).

Argentina

Like Brazil, Argentina's growth was due to infrastructure development. A relatively competitive market like Brazil, Argentina also had a program of developing regional IXPs to foster domestic connectivity. Argentina's CABASE program has established 10 municipal IXPs and is facilitating the growth we're seeing in that country. This growth has vaulted Argentina past Mexico in the number of players in the domestic Internet. Argentina has the third largest economy in Latin America, behind Mexico and Brazil.

Mexico

On the other end of the spectrum is Mexico, where we have observed negligible growth in its domestic Internet.

One of the least competitive markets in Latin America, Mexico is dominated by its incumbent Telmex. Last year, the OECD concluded that, "inefficient telecommunication markets impose a significant cost on the Mexican economy and the welfare of its population" to the tune of US$129Bn per year or 1.8% of the GDP of Mexico.

The lack of competition is such a major problem that newly elected Mexican President Enrique Peña Nieto has made telecom reform one of the key features of his Pacto por México. Among other measures, the legislation would cap market share at 50% and remove the limit on foreign ownership of telecommunications companies. Once instituted, the market share cap may require the break-up of Telmex and pit Peña Nieto against Telmex's owner and CEO, billionaire Carlos Slim, the richest man in the world.

As of this writing, the Mexican telecom reform has passed both houses of congress and has been ratified by a majority of Mexican states. America Movil, the parent company of Telmex, has already warned shareholders that the new legislation could have a material impact on its Mexican business.

In separate development, several mid-sized Mexican telecommunications companies have united to establish the first domestic Internet Exchange Point (IXP) in Mexico, the only country in the OECD not to currently have one. However, without Telmex on board, the impact on Mexican domestic connectivity as a whole may be limited.

The regulatory reform in Mexico won't take effect until 2014. If Costa Rica can serve as a model of what to expect in Mexico, it may take a couple of years until we observe a meaningful increase in growth. However, given the size and richness of the Mexican economy, perhaps there is reason to believe that a long-awaited expansion could occur at a much faster rate once facilitated by this reform.

In many parts of the world, the promise of regional terrestrial connectivity that directly connects neighboring countries has proven elusive. The Program for International Development in Africa (PIDA) has been pushing for this in Africa, while LIRNEasia has been advocating for the same thing for South and Southeast Asia. The GCCIX project has begun to address this in the Middle East. Without terrestrial connectivity, countries in these regions are reliant on submarine cables to carry their traffic to neighboring countries. This implies that Internet traffic must often traverse long distances to reach geographically close locations, resulting in longer latencies and poorer performance. In addition, submarine cables occasionally suffer faults and undersea repairs take significantly more time than those on land.

Until recently, South America could also be defined much in the same way, but that is changing. Miami is still the common international connection point for Internet providers in Latin America (Los Angeles and Dallas for Mexico). However, Colombian company Internexa is leveraging the international network of power lines owned by its parent company ISA to build the region's first multi-country terrestrial network (pictured at right). At LACNIC 19, Internexa presented statistics about how much regional Internet traffic they are currently carrying that no longer needs to leave the continent in order to reach its destination. They currently have direct terrestrial connections between Brazil, Argentina and Chile as well as between Venezuela, Colombia, Ecuador and Peru. Ultimately, their goal is to build a terrestrial network spanning all the countries of South America to provide the fastest and most direct service in the region. In my talk, I used the screenshots below from our upcoming product, Internet Business Intelligence, to illustrate which Internet provider combinations provided the lowest latency paths from the city hosting the conference (Medellin, Colombia) to nearby Quito, Ecuador. By eliminating the need to send Internet traffic out of the region, Internexa was the fastest option to Ecuador. When looking at performance over time in the graphic below and to the right, we can see that the Ecuadorian government-owned provider Corporacion Nacional de Telecomunicaciones (CNT) experienced some brief performance issues, while Internexa's subsidiary in Ecuador, Transnexa, provided the most stable low-latency path between these two cities during this time period.

Looking to the Future

Developments such as Internexa's terrestrial network and Brazil's amazing Internet boom have been fascinating to observe. However, the most important story for the region in the coming years may be whether or not Peña Nieto is successful at breaking up Carlos Slim's Telmex and its grip on the Mexican telecom market. Solo el tiempo lo dirá — only time will tell.

In January, we reported the news that the ALBA-1 submarine cable connecting Cuba to Venezuela had started carrying Internet traffic two years after its construction, answering the question of what happened to the mystery cable to Cuba.

In the last week, we have observed a second non-satellite connection established for Cuban state telecom, ETECSA. This time a different segment of the ALBA-1 submarine cable is being used to connect Cuba to the neighboring island nation of Jamaica. At 15:04 UTC on 13 May 2013, we observed ETECSA beginning to receive international Internet service through Cable & Wireless Jamaica.

Update (15:26 UTC, 15 May): Routes to Syrian networks have been restored, at 18:26 Damascus time. Outage duration: 8h25m

 

 

 

 

Update (14:20 UTC, 15 May): Plot of latency measurements to Syrian hosts from various locations, indicating that replies stopped returning shortly after 7am UTC, aligned with the withdrawal of routes to Syrian networks. (Click image for details)

Update (07:30 UTC, 15 May): Syrian Internet down again since 07:01 UTC (10:00 Damascus time), Wednesday, 15 May 2013. Syrian news agency reports that they're working to fix. Potentially related to forthcoming UN decision today?

 

 

 

 

 

---

Older Update: Syrian Internet has returned. Outage lasted 19.5 hours, from 18:45 UTC May 7th to 14:13 UTC May 8th.

As we write, the Syrian people are still disconnected from the global Internet at the most fundamental level, nearly all of their paths withdrawn from the global routing table. Since 18:45 UTC on May 7th, Renesys hasn't seen a flicker of activity. We haven't been able to successfully send a ping or a traceroute to any host inside Syria. Government websites, universities, domain name servers, core infrastructure routers, banks, businesses, DSL customers, smartphones: all silent.

]]> http://www.renesys.com/blog/2013/05/syrian-internet-fragility.shtml http://www.renesys.com/blog/2013/05/syrian-internet-fragility.shtml Engineering Wed, 08 May 2013 10:03:00 -0500 Sometimes, it takes a real disaster to create something genuinely new. March 2013 was a month of disasters in the Middle Eastern, South Asian, and East African Internet, with major submarine cable cuts affecting SMW3, SMW4, IMEWE, EIG, SEACOM, and TE-North.

One of the "genuinely new" Internet traffic paths that emerged in response is a counterintuitive terrestrial route, linking the ancient Indian Ocean trade empire of Oman with the Internet markets of Western Europe, by way of Iran, Azerbaijan, and the Russian Caucasus. As we'll see, its effects are now being felt across the region, from Pakistan, to Gulf states like Bahrain and Oman, to Kenya.

The EPEG (Europe-Persia Express Gateway) consortium was actually born in June 2011, as an alternative to the congested, politically uncertain Suez transit corridor. EPEG links together existing fiber routes from the Iranian, Azeri, and Russian incumbents, connected to Cable and Wireless's network to approximate a Great Circle route to Frankfurt. With the aid of one final submarine hop across the Strait of Hormuz to Muscat, EPEG promised to deliver a major new low-latency, high-capacity terrestrial route to carry the Gulf states' traffic to Europe.

EPEG was scheduled to begin service in the summer of 2012, but delays on the final submarine links between Iran and Oman caused the start date to be pushed, and pushed again. When Vodafone acquired Cable and Wireless in July 2012, one of the attractions of the deal may have been the prospect of participating in EPEG — when the project reached completion, Vodafone/C&W would sit astride the first low-latency terrestrial link connecting the Gulf region to the financial and content markets of Western Europe.

We reported in February 2013 that in the wake of Russian service interruptions in the Black Sea, Iran's DCI had begun receiving Internet transit from Omantel, indicating that the missing link in the southern path was alive at last. We began watching intently for signs of end-to-end service activation on EPEG.

Last month, we saw them.

 

 

---

]]> http://www.renesys.com/blog/2013/04/gulf-states-iran.shtml http://www.renesys.com/blog/2013/04/gulf-states-iran.shtml Internet Mon, 22 Apr 2013 16:00:00 -0500 It has been a rough few weeks for the global Internet, given numerous submarine cable failures and the largest DDOS attack ever reported. While we're hard-pressed to find evidence of the purported global Internet slowdown due to the DDOS attack, the dramatic impacts of yesterday's SMW4 submarine cable cut were profound. Recent reports that the cable break was the result of sabotage make the incident even more intriguing. In this blog, we detail what happened to some of the providers in four countries along the route of the cable: Egypt, Saudi Arabia, Pakistan and India.

]]> http://www.renesys.com/blog/2013/03/intrigue-surrounds-smw4-cut.shtml http://www.renesys.com/blog/2013/03/intrigue-surrounds-smw4-cut.shtml Thu, 28 Mar 2013 10:04:54 -0500

On Friday, we published a blog confirming a disruption in Internet connectivity affecting the Internet of North Korea.

This morning, South Korean authorities reported that they have been the victims of a cyber attack which impacted TV News organizations as well as banking institutions.

Renesys can confirm that at least some of today's incidents escalated to the point of global visibility, as both South and North Koreans networks experienced actual disconnections. We note similarly timed outages affecting South Korea's largest natural gas company.

]]> http://www.renesys.com/blog/2013/03/more-outages-in-koreas.shtml http://www.renesys.com/blog/2013/03/more-outages-in-koreas.shtml Wed, 20 Mar 2013 10:39:00 -0500

Earlier this morning, North Korea accused the United States of conducting a cyber attack that disrupted their Internet connectivity. While the details remain unknown, we can confirm that, in the last two days, North Korea's sole Internet provider has had ongoing problems staying connected to the global Internet. We'll summarize some of our evidence in this blog entry.

]]> http://www.renesys.com/blog/2013/03/north-korea-suffers-outage.shtml http://www.renesys.com/blog/2013/03/north-korea-suffers-outage.shtml Fri, 15 Mar 2013 14:47:28 -0500

Last month, Wired.com's fascinating geological sciences blog, Eruptions, cast doubt on the purported cause of the December 23, 2012 failure of the Georgia-Russia submarine cable. That is, the author of the Eruptions blog post thought it unlikely to have been due to an undersea volcanic eruption. Without weighing in on the likelihood of active volcanoes in the Black Sea, we tweeted about some of the Internet impacts of this incident, although in 140 characters, we could only scratch the surface. We'll take a more in-depth look in this blog, noting shifts in traffic as far away as Oman, more than 3,000 kilometers distant!

]]> http://www.renesys.com/blog/2013/02/faraway-fallout-from-black-sea.shtml http://www.renesys.com/blog/2013/02/faraway-fallout-from-black-sea.shtml Submarine Cables Thu, 14 Feb 2013 10:19:00 -0500

The Internet of Bangladesh has been connected to the world by a single submarine cable, Sea-Me-We 4 (SMW4), since this 18,800 kilometer-long optical-fiber system made its landing at Cox's Bazar in 2006. However, in the nearly seven years since SMW4's activation, national Internet outages have plagued Bangladesh with some regularity. When their portion of this system is sabotaged, suffers a failure or is down for maintenance, virtually all Internet bandwidth for the 7th most populous country in the world disappears, forcing local providers to fall back to slow and expensive satellite services or to simply wait for restoration.

However, recent national outages due to planned SMW4 maintenance have revealed that some Bangladeshi providers have now activated a long-awaited second connection to the Internet via a terrestrial link to India. We'll examine this new development here and highlight those providers who can now offer fault-tolerant Internet service for the first time in Bangladesh.

]]> http://www.renesys.com/blog/2013/01/bangladesh-connects-via-india.shtml http://www.renesys.com/blog/2013/01/bangladesh-connects-via-india.shtml South Asia Tue, 29 Jan 2013 09:02:31 -0500

For more latest information, please follow us on Twitter: @Renesys

Cuban Internet connectivity continues to evolve by the hour, with a new, faster mode of operation in evidence as of this morning. Our measurements from around the world suggest that Cuban technicians may have completed the work they began a week ago, creating the first bidirectional Internet paths that are free of satellite connectivity.

]]> http://www.renesys.com/blog/2013/01/cuban-fiber-completo.shtml http://www.renesys.com/blog/2013/01/cuban-fiber-completo.shtml Tue, 22 Jan 2013 14:00:01 -0500 Recent developments captured in a new blog here. (January 22, 2013)

In February 2011, the first submarine cable connecting the island nation of Cuba to the global internet (by way of Venezuela) landed on Siboney beach, Santiago de Cuba. In the two years since, the fate of the cable has been a mystery for Cuba observers. In the past week, our global monitoring system has picked up indications that this cable has finally been activated, although in a rather curious way, as we explain below.

]]> http://www.renesys.com/blog/2013/01/cuban-mystery-cable-activated.shtml http://www.renesys.com/blog/2013/01/cuban-mystery-cable-activated.shtml Caribbean Sun, 20 Jan 2013 09:00:03 -0500 It is an annual tradition at Renesys to provide a year-end review of how the Internet providers at the top of our Market Intelligence global rankings fared over the previous year. The Internet remains a huge blind spot for many organizations that are buying Internet access. Market Intelligence provides the insight into who the leaders in the Internet transit marketplace are today and how they have changed over time. Back in 2008, we chose to look at the 13 providers that spent at least some time in the Top Ten that year, hence the name "Baker's Dozen". We looked at the top players again in 2009, 2010 and 2011. A lot has changed over the years and for 2012, we welcome two new members to this exclusive club, PCCW and XO. As predicted last year, we also say good-bye to a declining AT&T and Savvis. While AT&T's departure from the top of the global stage may be surprising to some, Savvis really hasn't left as it is now part of CenturyLink, which also owns Qwest. And while Qwest did leave our top global rankings in 2011, they have now returned as part of a reinvigorated CenturyLink.

As you read this blog, keep in mind that all of the rankings we discuss are relative to IPv4, the Internet protocol carrying over 99% of all Internet traffic. (For example, compare total traffic to IPv6 traffic at the very busy Amsterdam Internet Exchange.) While we did also review IPv6 rankings last year, so little has changed that we'll just refer you to that blog or, for more current information, our Market Intelligence product offering which covers both IPv4 and IPv6 in detail. So let's dive in and highlight a few of the trends and changes we observed in 2012.

]]> http://www.renesys.com/blog/2013/01/a-bakers-dozen-2012-edition.shtml http://www.renesys.com/blog/2013/01/a-bakers-dozen-2012-edition.shtml Thu, 17 Jan 2013 12:58:43 -0500 Renesys confirms a largely complete restoration of the Syrian Internet this morning, starting at 14:32:10 UTC (16:32 local time in Damascus).

Transit providers for the full prefix set do not appear to be significantly changed, with Internet service being provided post-restoration by Telecom Italia, Tata Communications, Turk Telecom, and PCCW.

Here's a view of live Syrian prefix counts during the outage and restoration, from 29 November to 1 December:

]]> http://www.renesys.com/blog/2012/12/restoration-in-syria-1.shtml http://www.renesys.com/blog/2012/12/restoration-in-syria-1.shtml Middle East Sat, 01 Dec 2012 11:24:57 -0500 {Updated; see below for lists of countries for easier reference, and a special offer for WCIT attendees. —jim}

How hard is it to disconnect a country from the Internet, really?

That's the number one question we've received about our analysis of the Egyptian and Syrian Internet blackouts, and it's a reasonable question. If the Internet is so famously resilient, designed to survive wars and calamities, how can it fail so abruptly and completely at the national level?

The key to the Internet's survival is the Internet's decentralization — and it's not uniform across the world. In some countries, international access to data and telecommunications services is heavily regulated. There may be only one or two companies who hold official licenses to carry voice and Internet traffic to and from the outside world, and they are required by law to mediate access for everyone else.

Under those circumstances, it's almost trivial for a government to issue an order that would take down the Internet. Make a few phone calls, or turn off power in a couple of central facilities, and you've (legally) disconnected the domestic Internet from the global Internet. Of course, this level of centralization also makes it much harder for the government to defend the nation's Internet infrastructure against a determined opponent, who knows they can do a lot of damage by hitting just a few targets.

]]> http://www.renesys.com/blog/2012/11/could-it-happen-in-your-countr.shtml http://www.renesys.com/blog/2012/11/could-it-happen-in-your-countr.shtml Internet Fri, 30 Nov 2012 11:32:57 -0500 Click for latest update: 01:00 GMT Friday.

Starting at 10:26 UTC on Thursday, 29 November (12:26pm in Damascus), Syria's international Internet connectivity shut down. In the global routing table, all 84 of Syria's IP address blocks have become unreachable, effectively removing the country from the Internet.

We are investigating the dynamics of the outage and will post updates as they become available.

]]> http://www.renesys.com/blog/2012/11/syria-off-the-air.shtml http://www.renesys.com/blog/2012/11/syria-off-the-air.shtml Internet Thu, 29 Nov 2012 08:31:09 -0500