TCP/IP over SCI under Linux

Abstract

In this chapter, we present how SCI devices can be integrated in the TCP/IP protocol stack of Linux for high speed communication within a cluster. We developed a packet driver which is using SCI at the physical layer to enable a data transfer with high throughput and low latency. In our solution we achieved a throughput of 31 MByte/s and a latency of 77 μs. Although the size of the protocol stack implies higher software latency than hardware latency, we can benefit from the higher bandwidth compared to traditional network technologies. This makes it conceivable to replace an existing Ethernet network by our approach in SCI clusters. Instead of replacing TCP by a user level transport protocol as done by [2] and [3] (see also Chapter 11), our solution is to implement SCIP (Scalable Coherent Interface IP) as a packet driver, sending and receiving IP packets. This is more straightforward since SCIP, like e.g. PLIP, SLIP or Ethernet, works underneath the IP layer (see also Figure 12.1). The key advantage is that we keep the socket semantics and therefore all applications work without any modifications or recompilations. In addition, we can realize this with a small amount of source code, due to the fact that we only have to transmit an IP packet through the SCI interconnect. So SCIP is a thin layer between IP and the SCI driver interface. Another important fact is that SCIP runs without any problems on multiprocessor machines since the Linux kernel provides coordinated access to the driver.