Babylon University

College of Information Technology Software Department

Multiprocessors Interconnection
Networks

By
Dr. Asaad Sabah Hadi
 Interconnection Networks Taxonomy

• An interconnection network could be either static or dynamic

• Connections in a static network are fixed links, while connections in
a dynamic network are established on the fly as needed.
• Static networks can be further classified according to their
interconnection pattern as one-dimension (1D), two-dimension
(2D), or hypercube (HC).
• Dynamic networks, on the other hand, can be classified based on
interconnection scheme as bus-based versus switch-based.
• Bus-based networks can further be classified as single bus or
multiple buses.
• Switch-based dynamic networks can be classified according to the
structure of the interconnection network as single-stage (SS),
multistage (MS), or crossbar networks

2 Dr. Asaad Sabah Hadi

A topology-based taxonomy for interconnection
networks

3 Dr. Asaad Sabah Hadi

 Bus-based Dynamic Interconnection Networks
1. Single Bus Systems
• A single bus is considered the simplest way to connect
multiprocessor systems.
• The system consists of N processors, each having its own cache,
connected by a shared bus. The use of local caches reduces the
processor–memory traffic.
• All processors communicate with a single shared memory.
• The typical size of such a system varies between 2 and 50
processors.
• The actual size is determined by the traffic per processor and the
bus bandwidth (defined as the maximum rate at which the bus can
propagate data once transmission has started).
• The single bus network complexity, measured in terms of the
number of buses used, is O(1), while the time complexity,measured
in terms of the amount of input to output delay is O(N).

4 Dr. Asaad Sabah Hadi

• Although simple and easy to expand, single bus
multiprocessors are inherently limited by the bandwidth of
the bus and the fact that only one processor can access the
bus, and in turn only one memory access can take place at any
given time.

5 Dr. Asaad Sabah Hadi

 2-Multiple Bus Systems

• The use of multiple buses to connect multiple processors is a natural

extension to the single shared bus system.
• A multiple bus multiprocessor system uses several parallel buses to
interconnect multiple processors and multiple memory modules.
• A number of connection schemes are possible in this case:
1. The Multiple Bus with Full Bus memory Connection (MBFBMC).
2. The Multiple Bus with Single Bus Memory Connection (MBSBMC)
3. The Multiple Bus with Partial Bus Memory Connection (MBPBMC)
4. The Multiple Bus with Class-based Memory Connection (MBCBMC).

6 Dr. Asaad Sabah Hadi

• In general, multiple bus multiprocessor
organization offers a number of desirable
features such as high reliability and ease of
incremental growth.
• single bus failure will leave (B - 1) distinct fault-
free paths between the processors and the
memory modules.
• On the other hand, when the number of buses is
less than the number of memory modules (or the
number of processors), bus contention is
expected to increase.

7 Dr. Asaad Sabah Hadi

8 Dr. Asaad Sabah Hadi
9 Dr. Asaad Sabah Hadi
 3- Bus Synchronization
• A bus can be classified as synchronous or asynchronous.
• The time for any transaction over a synchronous bus is known in advance. In
accepting and/or generating information over the bus, devices take the transaction
time into account.
• Asynchronous bus, on the other hand, depends on the availability of data and the
readiness of devices to initiate bus transactions.
• In a single bus multiprocessor system, bus arbitration is required in order to
resolve the bus contention that takes place when more than one processor
competes to access the bus.
• In this case, processors that want to use the bus submit their requests to bus
arbitration logic. The latter decides, using a certain priority scheme, which
processor will be granted access to the bus during a certain time interval (bus
master).
• The process of passing bus mastership from one processor to another is called
handshaking and requires the use of two control signals: bus request and bus
grant.
• The bus request indicate that a given processor is requesting mastership of the bus
• The bus grant indicate that bus mastership is granted.
• The Third Signal is the bus busy which is used to indicate whether or not the bus is
currently being used.

10 Dr. Asaad Sabah Hadi

 Characteristics of Multiple Bus Architectures

In this table,
 k represents the number of classes;
 g represents the number of buses per group
 Mj represents the number of memory modules in class j.
 N represent the number of processors.
 B represent the number of Buses.
11 Dr. Asaad Sabah Hadi
 Switch-based Interconnection Networks
• There are Three basic interconnection topologies exist for
switch-based: crossbar, single-stage, and multistage.
1. Crossbar Networks
• A crossbar network represents the other extreme to the
limited single bus network.
• While the single bus can provide only a single connection, the
crossbar can provide simultaneous connections among all its
inputs and all its outputs.
• The crossbar contains a switching element (SE) at the
intersection of any two lines extended horizontally or
vertically inside the switch.

12 Dr. Asaad Sabah Hadi

13 Dr. Asaad Sabah Hadi
• The figure illustrates the case of setting the SEs such that
simultaneous connections between Pi and for
are made. The two possible settings of an SE in the crossbar
(straight and diagonal) are also shown in the figure.
• As can be seen from the figure, the number of SEs (switching
points) required is 64 and the message delay to traverse from
the input to the output is constant, regardless of which
input/output are communicating.
• In general for an N x N crossbar, the network complexity,
measured in terms of the number of switching points, is
O(N2) while the time complexity, measured in terms of the
input to output delay, is O(1).

14 Dr. Asaad Sabah Hadi

 2. Single-Stage Networks
• In this case, a single stage of switching elements (SEs) exists between the
inputs and the outputs of the network.
• The simplest switching element that can be used is the 2 x 2 switching
element (SE).
• In the straight setting, the upper input is transferred to the upper output
and the lower input is transferred to the lower output.
• In the exchange setting the upper input is transferred to the lower output
and the lower input is transferred to the upper output.
• In the upper-broadcast setting the upper input is broadcast to both the
upper and the lower outputs.
• In the lower-broadcast the lower input is broadcast to both the upper and
the lower outputs.

15 Dr. Asaad Sabah Hadi

• To establish communication between a given input (source) to a given
output (destination), data has to be circulated a number of times around
the network.
• A well-known connection pattern for interconnecting the inputs and the
outputs of a single-stage network is the Shuffle–Exchange.
• Two operations are used. These can be defined using an m bit-wise
address pattern of the inputs, pm-1 pm-2 . . . p1 p0, as follows:
• With shuffle (S) and exchange (E) operations, data is circulated from input
to output until it reaches its destination.
• If the number of inputs, for example, processors,in a single-stage IN is N
and the number of outputs, for example, memories, is N, the number of
SEs in a stage is N/2.
• The maximum length of a path from an input to an output in the network,
measured by the number of SEs along the path, is log2 N.

16 Dr. Asaad Sabah Hadi

• Example In an 8-input single stage Shuffle–Exchange if the
source is 0 (000) and the destination is 6 (110), then the
following is the required sequence of Shuffle/Exchange
operations and circulation of data:

• The network complexity of the single-stage interconnection

network is O(N) and the time complexity is O(N).
• In addition to the shuffle and the exchange functions, there
exist a number of other interconnection patterns that are
used in forming the interconnections among stages in
interconnection networks. Among these are the Cube and the
Plus-Minus 2i(PM2I) networks. These are introduced below.

17 Dr. Asaad Sabah Hadi

The Cube Network
• The interconnection pattern used in the cube network is
defined as follows:

• Consider a 3-bit address (N = 8), then we have C2(6) = 2, C1(7)

=5 and C0(4) = 5.

18 Dr. Asaad Sabah Hadi

• The Plus–Minus 2i (PM2I) Network

• For example, consider the case N = 8, PM2+1(4) = 4 + 21 mod 8

=6. Figure below shows the PM2I for N = 8.
• It should be noted that PM2+(k-1)(P)= PM2-(k-1)(P) P, 0 <= P<N.
It should also be noted that PM2+2 = C2.

19 Dr. Asaad Sabah Hadi

The Butterfly Function
• The interconnection pattern used in the butterfly network is
defined as follows:

• Consider a 3-bit address (N= 8), the following is the butterfly

mapping:
B(000) = 000
B(001) = 100
B(010) = 010
B(011) = 110
B(100) = 001
B(101) = 101
B(110) = 011
B(111) = 111

20 Dr. Asaad Sabah Hadi

21 Dr. Asaad Sabah Hadi
 3- Multistage Network
• Multistage interconnection networks (MINs) were introduced as a means to
improve some of the limitations of the single bus system while keeping the
cost within an affordable limit.
• The most undesirable single bus limitation that MINs is set to improve is the
availability of only one single path between the processors and the memory
modules.
• Such MINs provide a number of simultaneous paths between the processors
and the memory modules.
• Figure below shows an example of an 8 8 MIN that uses the 2 x 2 SEs
described before. This network is known in the literature as the Shuffle–
Exchange network (SEN).
• The settings of the SEs in the figure illustrate how a number of paths can be
established simultaneously in the network.
• For example, the figure shows how three simultaneous paths connecting the
three pairs of input/output 000101, 101011, and 110010 can be
established. It should be noted that the interconnection pattern among stages
follows the shuffle operation.

22 Dr. Asaad Sabah Hadi

• In MINs, the routing of a message from a given source to a given
destination is based on the destination address (self-routing).
• There exist (log2 N ) stages in an N x N MIN.
• The number of bits in any destination address in the network is log2
N.
• Each bit in the destination address can be used to route the
message through one stage.
• The destination address bits are scanned from left to right and the
stages are traversed from left to right as follows :
 The first (most significant bit) is used to control the routing in the
first stage.
 The next bit is used to control the routing in the next stage, and so
on.
• The convention used in routing messages is that if the bit in the
destination address controlling the routing in a given stage is 0,
then the message is routed to the upper output of the switch. On
the other hand if the bit is 1, the message is routed to the lower
output of the switch.

23 Dr. Asaad Sabah Hadi

• Consider, for example, the routing of a message from source
input 101 to destination output 011 in the 8 x8 SEN.
• Since the first bit of the destination address is 0, therefore the
message is first routed to the upper output of the switch in
the first (leftmost) stage.
• Now, the next bit in the destination address is 1, thus the
message is routed to the lower output of the switch in the
middle stage.
• Finally, the last bit is 1, causing the message to be routed to
the lower output in the switch in the last stage.

24 Dr. Asaad Sabah Hadi

25 Dr. Asaad Sabah Hadi