TCP offload engine

Originally TCP was designed for unreliable low speed networks (such as early dial-up modems) but with the growth of the Internet in terms of backbone transmission speeds (using Optical Carrier, Gigabit Ethernet and 10 Gigabit Ethernet links) and faster and more reliable access mechanisms (such as DSL and cable modems) it is frequently used in data centers and desktop PC environments at speeds of over 1 Gigabit per second. The TCP software implementations on host systems require extensive computing power. In the early 2000s, full duplex gigabit TCP communication could consume more than 80% of a 2.4 GHz Pentium 4 processor (see freed-up CPU cycles), resulting in small or no processing resources left for the applications to run on the system.

TCP is a connection-oriented protocol which adds complexity and processing overhead. These aspects include:

• Connection establishment using the "3-way handshake" (SYNchronize; SYNchronize-ACKnowledge; ACKnowledge).
• Acknowledgment of packets as they are received by the far end, adding to the message flow between the endpoints and thus the protocol load.
• Checksum and sequence number calculations - again a burden on a general purpose CPU to perform.
• Sliding window calculations for packet acknowledgement and congestion control.
• Connection termination.

Moving some or all of these functions to dedicated hardware, a TCP offload engine, frees the system's main CPU for other tasks. As of 2012, very few consumer network interface cards support TOE.

Instead of replacing the TCP stack with a TOE entirely, there are alternative techniques to offload some operations in co-operation with the operating system's TCP stack. TCP checksum offload and large segment offload are supported by the majority of today's Ethernet NICs. Newer techniques like large receive offload and TCP acknowledgment offload are already implemented in some high-end Ethernet hardware, but are effective even when implemented purely in software.[2][3]

One of the first patents in this technology, for UDP offload, was issued to Auspex Systems in early 1990.[5] Auspex founder Larry Boucher and a number of Auspex engineers went on to found Alacritech in 1997 with the idea of extending the concept of network stack offload to TCP and implementing it in custom silicon. They introduced the first parallel-stack full offload network card in early 1999; the company's SLIC (Session Layer Interface Card) was the predecessor to its current TOE offerings. Alacritech holds a number of patents in the area of TCP/IP offload.[6]

By 2002, as the emergence of TCP-based storage such as iSCSI spurred interest, it was said that "At least a dozen newcomers, most founded toward the end of the dot-com bubble, are chasing the opportunity for merchant semiconductor accelerators for storage protocols and applications, vying with half a dozen entrenched vendors and in-house ASIC designs."[7]

In 2005 Microsoft licensed Alacritech's patent base and along with Alacritech created the partial TCP offload architecture that has become known as TCP chimney offload. TCP chimney offload centers on the Alacritech "Communication Block Passing Patent". At the same time, Broadcom also obtained a license to build TCP chimney offload chips.

Unlike other kernels, the Linux kernel does not include support for TOE hardware.[8] Not to be confused with other similarly named offloads, like TCP Segmentation Offload which is extensively supported.[9] While there are patches from the hardware manufacturers such as Chelsio or Qlogic that add support, the Linux kernel developers are opposed to this technology for several reasons, including[10]

• Security – because TOE is implemented in hardware, patches must be applied to the TOE firmware, instead of just software, to address any security vulnerabilities found in a particular TOE implementation. This is further compounded by the newness and vendor-specificity of this hardware, as compared to a well tested TCP/IP stack as is found in an operating system that does not use TOE.
• Limitations of hardware – because connections are buffered and processed on the TOE chip, resource starvation can more easily occur as compared to the generous CPU and memory available to the operating system.
• Complexity – TOE breaks the assumption that kernels make about having access to all resources at all times – details such as memory used by open connections are not available with TOE. TOE also requires very large changes to a networking stack in order to be supported properly, and even when that is done, features like Quality of Service and packet filtering typically do not work.
• Proprietary – TOE is implemented differently by each hardware vendor. This means more code must be rewritten to deal with the various TOE implementations, at a cost of the aforementioned complexity and, possibly, security. Furthermore, TOE firmware cannot be easily modified since it is closed-source.
• Obsolescence – Each TOE NIC has a limited lifetime of usefulness, because system hardware rapidly catches up to TOE performance levels, and eventually exceeds TOE performance levels.

Much of the current work on TOE technology is by manufacturers of 10 Gigabit Ethernet interface cards, such as Broadcom, Chelsio Communications, Emulex, Mellanox Technologies, QLogic.

• Large segment offload (LSO)
• Large receive offload (LRO)
• Scalable Networking Pack
• I/O Acceleration Technology

• Article: TCP Offload to the Rescue by Andy Currid at ACM Queue
• Patent Application 20040042487
• Mogul, Jeffrey C. (2003). "TCP offload is a dumb idea whose time has come" (PDF). Proceedings of HotOS IX: The 9th Workshop on Hot Topics in Operating Systems. USENIX Association. Retrieved 23 July 2006.
• "TCP/IP offload Engine (TOE)". 10 Gigabit Ethernet Alliance. April 2002.