Two weeks ago, I had the great pleasure of visiting Medellin, Colombia to present at the Latin American and Caribbean Network Operators Group (LACNOG) portion of LACNIC 19. Medellin is a vibrant place, recently recognized as the world's most innovative city by the Urban Land Institute due to the city's investments in public infrastructure and civic spaces.

Perhaps equally innovative is Colombia's Internexa, which in recent years has been building the region's first international terrestrial telecommunications network. Meanwhile, another remarkable regional story is the exponential growth of the domestic Internet in Brazil — especially when contrasted with the stagnation in Mexico. While government initiatives in Brazil, the region's largest economy, were able to foster much of its recent growth, the current regulatory overhaul in Mexico hopes to achieve something similar in the region's second largest economy.

In the presentation, I discussed where we did and did not see Internet growth (crecimiento in Spanish) in the Latin American region. To measure growth, I borrowed a metric, domestic ASN growth, from my colleague Jim Cowie's recent presentation on the growth of the Internet of the Middle East at MENOG 12 in Dubai.

For each country, we counted up the number of domestic Autonomous Systems (ASNs), where an ASN is considered domestic in a country when at least 70% of its customer base is in that country. Of course, ASNs can vary dramatically in size, from large companies to small non-profits. But regardless of size, our metric accurately captures the "number of Internet players" in a country. It is the relative change in this count over time that is most informative, as shown in the following table for a subset of Latin American countries:

Our data illustrates very robust growth in Brazil, Argentina and Costa Rica since January 2010. During this period, Brazil grew by 340% and now exceeds the size of the rest of the region combined. By this metric, Brazil adds the equivalent of two Mexicos to its domestic Internet each year.

Dramatic growth of the Internet in Costa Rica is a result of the telecommunications regulatory reform of 2008. The reform was a requirement of the CAFTA-DR trade agreement, which mandated that the government of Costa Rica end the monopoly of ICE, set up a telecoms regulator Sutel, and allow new competition. Although Costa Rica was one of the smaller countries in our analysis, it had the largest relative growth in the region. New players in Costa Rica include new telecommunications providers as well as new enterprises operating in the country.

Brazil

Brazil's tremendous growth has been a result of a national program of infrastructure investment and government incentives, which was described at LACNIC 19 by Artur Coimbra of the Brazilian Ministry of Communications. In addition to upgrading and expanding the domestic backbone of Telebras, Brazil has established 22 municipal Internet exchange points called PTTs (Pontos de Troca de Tráfego) to facilitate domestic exchange of Internet traffic. In his presentation, Wardner Maia, President of the Association of Brazilian Internet Providers (ABRINT) discussed the recent explosion of small regional Internet service providers serving parts of Brazil that hadn't previously had access to the Internet. On average, he said, the Brazilian government was issuing over 700 licenses per year to these new providers, 78% percent of which are considered small companies. When we compared our data, it was clear that the growth of domestic ASNs matched the growth in licensing that his organization was witnessing.

We're not the only ones witnessing Brazil's growth. In their most recent State of the Internet report, content provider Akamai noted the largest year-to-year increase in unique IP addresses reaching their servers was from Brazil (a 33% increase).

Argentina

Like Brazil, Argentina's growth was due to infrastructure development. A relatively competitive market like Brazil, Argentina also had a program of developing regional IXPs to foster domestic connectivity. Argentina's CABASE program has established 10 municipal IXPs and is facilitating the growth we're seeing in that country. This growth has vaulted Argentina past Mexico in the number of players in the domestic Internet. Argentina has the third largest economy in Latin America, behind Mexico and Brazil.

Mexico

On the other end of the spectrum is Mexico, where we have observed negligible growth in its domestic Internet.

One of the least competitive markets in Latin America, Mexico is dominated by its incumbent Telmex. Last year, the OECD concluded that, "inefficient telecommunication markets impose a significant cost on the Mexican economy and the welfare of its population" to the tune of US$129Bn per year or 1.8% of the GDP of Mexico.

The lack of competition is such a major problem that newly elected Mexican President Enrique Peña Nieto has made telecom reform one of the key features of his Pacto por México. Among other measures, the legislation would cap market share at 50% and remove the limit on foreign ownership of telecommunications companies. Once instituted, the market share cap may require the break-up of Telmex and pit Peña Nieto against Telmex's owner and CEO, billionaire Carlos Slim, the richest man in the world.

As of this writing, the Mexican telecom reform has passed both houses of congress and has been ratified by a majority of Mexican states. America Movil, the parent company of Telmex, has already warned shareholders that the new legislation could have a material impact on its Mexican business.

In separate development, several mid-sized Mexican telecommunications companies have united to establish the first domestic Internet Exchange Point (IXP) in Mexico, the only country in the OECD not to currently have one. However, without Telmex on board, the impact on Mexican domestic connectivity as a whole may be limited.

The regulatory reform in Mexico won't take effect until 2014. If Costa Rica can serve as a model of what to expect in Mexico, it may take a couple of years until we observe a meaningful increase in growth. However, given the size and richness of the Mexican economy, perhaps there is reason to believe that a long-awaited expansion could occur at a much faster rate once facilitated by this reform.

In many parts of the world, the promise of regional terrestrial connectivity that directly connects neighboring countries has proven elusive. The Program for International Development in Africa (PIDA) has been pushing for this in Africa, while LIRNEasia has been advocating for the same thing for South and Southeast Asia. The GCCIX project has begun to address this in the Middle East. Without terrestrial connectivity, countries in these regions are reliant on submarine cables to carry their traffic to neighboring countries. This implies that Internet traffic must often traverse long distances to reach geographically close locations, resulting in longer latencies and poorer performance. In addition, submarine cables occasionally suffer faults and undersea repairs take significantly more time than those on land.

Until recently, South America could also be defined much in the same way, but that is changing. Miami is still the common international connection point for Internet providers in Latin America (Los Angeles and Dallas for Mexico). However, Colombian company Internexa is leveraging the international network of power lines owned by its parent company ISA to build the region's first multi-country terrestrial network (pictured at right). At LACNIC 19, Internexa presented statistics about how much regional Internet traffic they are currently carrying that no longer needs to leave the continent in order to reach its destination. They currently have direct terrestrial connections between Brazil, Argentina and Chile as well as between Venezuela, Colombia, Ecuador and Peru. Ultimately, their goal is to build a terrestrial network spanning all the countries of South America to provide the fastest and most direct service in the region. In my talk, I used the screenshots below from our upcoming product, Internet Business Intelligence, to illustrate which Internet provider combinations provided the lowest latency paths from the city hosting the conference (Medellin, Colombia) to nearby Quito, Ecuador. By eliminating the need to send Internet traffic out of the region, Internexa was the fastest option to Ecuador. When looking at performance over time in the graphic below and to the right, we can see that the Ecuadorian government-owned provider Corporacion Nacional de Telecomunicaciones (CNT) experienced some brief performance issues, while Internexa's subsidiary in Ecuador, Transnexa, provided the most stable low-latency path between these two cities during this time period.

Looking to the Future

Developments such as Internexa's terrestrial network and Brazil's amazing Internet boom have been fascinating to observe. However, the most important story for the region in the coming years may be whether or not Peña Nieto is successful at breaking up Carlos Slim's Telmex and its grip on the Mexican telecom market. Solo el tiempo lo dirá — only time will tell.

Update (15:26 UTC, 15 May): Routes to Syrian networks have been restored, at 18:26 Damascus time. Outage duration: 8h25m

 

 

 

 

Update (14:20 UTC, 15 May): Plot of latency measurements to Syrian hosts from various locations, indicating that replies stopped returning shortly after 7am UTC, aligned with the withdrawal of routes to Syrian networks. (Click image for details)

Update (07:30 UTC, 15 May): Syrian Internet down again since 07:01 UTC (10:00 Damascus time), Wednesday, 15 May 2013. Syrian news agency reports that they're working to fix. Potentially related to forthcoming UN decision today?

 

 

 

 

 

---

Older Update: Syrian Internet has returned. Outage lasted 19.5 hours, from 18:45 UTC May 7th to 14:13 UTC May 8th.

As we write, the Syrian people are still disconnected from the global Internet at the most fundamental level, nearly all of their paths withdrawn from the global routing table. Since 18:45 UTC on May 7th, Renesys hasn't seen a flicker of activity. We haven't been able to successfully send a ping or a traceroute to any host inside Syria. Government websites, universities, domain name servers, core infrastructure routers, banks, businesses, DSL customers, smartphones: all silent.

Sometimes, it takes a real disaster to create something genuinely new. March 2013 was a month of disasters in the Middle Eastern, South Asian, and East African Internet, with major submarine cable cuts affecting SMW3, SMW4, IMEWE, EIG, SEACOM, and TE-North.

One of the "genuinely new" Internet traffic paths that emerged in response is a counterintuitive terrestrial route, linking the ancient Indian Ocean trade empire of Oman with the Internet markets of Western Europe, by way of Iran, Azerbaijan, and the Russian Caucasus. As we'll see, its effects are now being felt across the region, from Pakistan, to Gulf states like Bahrain and Oman, to Kenya.

The EPEG (Europe-Persia Express Gateway) consortium was actually born in June 2011, as an alternative to the congested, politically uncertain Suez transit corridor. EPEG links together existing fiber routes from the Iranian, Azeri, and Russian incumbents, connected to Cable and Wireless's network to approximate a Great Circle route to Frankfurt. With the aid of one final submarine hop across the Strait of Hormuz to Muscat, EPEG promised to deliver a major new low-latency, high-capacity terrestrial route to carry the Gulf states' traffic to Europe.

EPEG was scheduled to begin service in the summer of 2012, but delays on the final submarine links between Iran and Oman caused the start date to be pushed, and pushed again. When Vodafone acquired Cable and Wireless in July 2012, one of the attractions of the deal may have been the prospect of participating in EPEG — when the project reached completion, Vodafone/C&W would sit astride the first low-latency terrestrial link connecting the Gulf region to the financial and content markets of Western Europe.

We reported in February 2013 that in the wake of Russian service interruptions in the Black Sea, Iran's DCI had begun receiving Internet transit from Omantel, indicating that the missing link in the southern path was alive at last. We began watching intently for signs of end-to-end service activation on EPEG.

Last month, we saw them.

It has been a rough few weeks for the global Internet, given numerous submarine cable failures and the largest DDOS attack ever reported. While we're hard-pressed to find evidence of the purported global Internet slowdown due to the DDOS attack, the dramatic impacts of yesterday's SMW4 submarine cable cut were profound. Recent reports that the cable break was the result of sabotage make the incident even more intriguing. In this blog, we detail what happened to some of the providers in four countries along the route of the cable: Egypt, Saudi Arabia, Pakistan and India.