When ESnet engineers began to study the idea of building a new 100 Gbps network between the United States and Europe, a primary concern was ensuring that the service would be robust and built from multiple underlying links — so that if one went down, researchers could still rely on sufficient bandwidth. Based on data collected for years by Caltech physicist and networking pioneer Harvey Newman, the ESnet team understood that multiple cables are sometimes cut simultaneously.
ESnet’s transatlantic extension will use network capacity leased from the owners of four undersea cables. Each cable has several pairs of optical fiber, with one fiber in each pair used for sending data from the U.S. to Europe, and the other used for data moving in the reverse direction. Each fiber is “lit” using dense wavelength division multiplexing (DWDM), a technology that sends data from different sources using different colors of light, allowing between 40 and 80 different signals to be sent on a single fiber simultaneously.
Unfortunately, any undersea or ‘submarine’ cable faces a number of threats, from ship anchors to landslides, according to Joe Metzger, ESnet’s lead engineer for the project.
Ship anchors cause most cable cuts near the coastline. Even though undersea cables are designed with extra armor near land, and are buried up to a meter deep, the anchors dropped by supertankers riding out storms in the North Sea can still damage the fiber optics inside the cables.
Undersea earthquakes pose another hazard. According to Metzger, one such quake triggered an underwater landslide that took out a number of cables all laid through the same channel.
One of the challenges in seeking the safest routes for network links is that while each cable operator knows the locations of its own cables, there’s no accurate database covering all cable paths.
Another major issue is the time needed to repair a break. In its network connecting sites across the terrestrial U.S., ESnet can usually have an engineer at the site within four hours of a problem being reported. And specialized equipment for cable repair is usually in place within eight to 12 hours. In a worst-case scenario, it may take 36 hours to get the link back in service when that link is on land.
With an undersea break, it can take a repair ship up to a week just to find the problem. Once located, the ship uses an underwater robot to cut the cable and affix a buoy to the end, which then raises the end of the cable to the ocean surface. The process is repeated for the other end. The damaged section is removed, a new piece of cable spliced in and the repaired cable then lowered back to the seabed. The procedure can be delayed by storms and the limited number of repair ships. In some cases, it can take four weeks or longer to repair the undersea break.
The challenge for ESnet, then, was to design a transatlantic extension with four separate links to minimize the probability of a complete network outage. Each link needed to follow a separate path. After getting proposals from a number of vendors, ESnet staff wrote a computer program to analyze and rank all of the detailed cable maps to select the four connections that had the least amount of overlap. And to provide an extra measure of safety, ESnet is working with other research networks in North America and Europe to arrange mutual backup in case of catastrophic failure.
Because the scope of the ESnet project breaks new ground for the international networking community, Metzger will give a presentation on the two-year project at the 2014 Technology Exchange conference in late October 2014. The October 26 to 30 conference is the leading networking conference in the U.S. and draws attendees from around the world.
