October 17, 2013
Blazing a Trail to 40GBA Masood Shariff is an Engineer Senior Principal in the systems engineering group of CommScope and author of this article.
Many experts in the industry are seeing evidence supporting Category 8 cabling as the true trailblazer to 40G in the data center.
For years, cabling standards had to adapt at a rapid pace in order to support greater bandwidth and lower latencies, while reducing cost, bulk and power consumption. The most accepted and cost-effective networking option in both the data center and commercial environments involves Ethernet over twisted pair in conjunction with RJ-45 connectivity. However, with bandwidth requirements rapidly escalating, some in the IT industry are asking how long this technology can continue to adapt to new demands. Many see more evidence supporting Category 8 cabling as the true trailblazer to 40G in the data center.
The pressure on data center capacity has never been greater. It is now commonplace for 10 or more virtual machines to be running on the same physical server, making a high-speed uplink essential for all these applications to run without excess latency or degradation of service. As more servers are virtualized creating a data influx, the IEEE forecasts 40G uplink speeds to be a requirement for server interfaces by 2018.
While 40G transmission speeds can certainly be achieved with current technologies, such as fiber or twinaxial (Twinax) cabling, these solutions may not always be ideal. Fiber transceivers are expensive, and Twinax solutions are limited to short distances of seven meters, making them a poor choice for interconnecting devices located more than one or two racks apart. Twinax cabling is also often a proprietary technology designed to fulfill a specific function in a data center, making it inflexible in meeting new applications.
Historically, the most cost-effective solution for data center connectivity has been Ethernet over twisted pair cabling. This is the preferred way of connecting servers to switches, where ubiquity and ease of use is crucial. Thus research into 40GBASE-T was started to create a more cost-effective alternative to existing fiber or Twinax solutions. The adoption of 40GBASE-T in data centers will bring huge advantages to enterprises, opening up fresh possibilities for connectivity, supporting a structured cabling design and a host of new data-intensive applications. Lower latency connections also provide additional benefits, such as facilitating the movement of virtual machines around the data center.
Because initial 40GBASE-T applications would be limited to data centers, the traditional twisted pair Ethernet 100-meter link length is not essential. The primary use for the 40GBASE-T application is in server to access switch links in data centers. IEEE 802.3bq, TIA, and ISO have accepted a maximum channel reach of 30 meters with a maximum of two connections as shown in the Figure below.
This configuration can be used to serve an end-of-row access switch configuration as well as a top-of-rack access switch configuration. Be aware that the structured cabling paradigm that uses a modular sub-system approach is preserved and continues to offer easy moves, adds, and changes (MACs) to the network. Furthermore, this configuration is generic and can support existing as well as emerging applications.
Twisted pair cabling with the RJ-45 connector has been the technology of choice for IT professionals, based on its low cost and ease of use. IT professionals are also hugely experienced with this technology, since it has been an industry standard since the 1980s. This means that data center personnel need not be retrained to install or maintain 40GBASE-T infrastructure, significantly reducing the time and costs associated with deploying the technology.
Twisted pair is also unique in being able to seamlessly support a range of different Ethernet speeds through auto-negotiation. Unlike fiber or Twinax solutions, twisted pair cabling BASE-T solutions can automatically switch to different data rates, such as from 100MbE to 10GbE. This ability makes twisted pair extremely versatile and allows data center managers to gradually and cost-effectively evolve their networks. For example, an IT manager could install a 40GBASE-T switch at the end of a data center row to provide connectivity to a handful of servers that are highly virtualized and require high-speed connections.
Meanwhile, the remaining ports on the switch can be connected to existing 1000BASE-T or 10GBASE-T servers, which could be upgraded as needed over time. This means the transition to 40GBASE-T does not require a forklift upgrade of equipment. Instead it can take place in an evolutionary manner in response to tangible business needs, since servers and switches are not always refreshed at the same time.
Finally, deploying 40GBASE-T connections will also reduce the overall operational expenditure of a data center by alleviating the need to aggregate multiple 10GBASE-T links to achieve 40GbE data rates. By reducing the number of links in a data center, IT managers can also improve energy efficiency and thus lower the total cost of ownership.
There are others in the industry saying that Category 7A cabling offers future-proof support for 40GbE speeds. However, Category 7A is only specified to a frequency of 1GHz and PHY industry experts have shown that higher bandwidth, in the range of 1.5-2GHz, will be required to support 40GBASE-T transmission. To date, the work of standards bodies indicates that a higher bandwidth solution than Category 7A cabling is needed in support of 40GBASE-T.
Additionally, Category 7A supports three different types of connectors that are not interoperable or backwards compatible with the RJ-45 connector. Their use in data centers will require the use of hybrid patch cords and equipment cords making operations more difficult, error prone and slowing down MACs in data centers.
At the June 2013 meeting, the TIA approved the formation of the study group to study including Class II channel limits in the TIA-568-C.2-1 draft document, recognizing advances in transmission technology needed for 40G applications. TIA TR42.7 technical experts had previously rejected a proposal to develop Category 7A cabling. This decision to possibly include Class II instead of Category 7A is further confirmation of the need for higher bandwidth and technical improvements beyond Category 7A needed for 40G applications. The decision to develop Class II up to 2000MHz is a clear indication that Categor y 7A cabling specified up to 1000 MHz may not be sufficient for 40G applications.
The IEEE has made it clear that it requires a low-bulk, high-density cabling solution for 40GBASE-T, given the limited real estate in data centers. With this in mind, the many in the cabling industry agree that Category 8 is the true trailblazer to supporting 40GBASE-T.