Distributed Systems

Agreement Protocol
 Introduction
• In distributed systems, where sites (or processors) often compete as well as
cooperate to achieve a common goal, it is often required that sites reach
mutual agreement .
– For example, In distributed database systems, data managers at sites must agree on
whether to commit or to abort a transaction.
• The formal setting for a distributed agreement protocol is the following:
– There are M processors P=p1 ,... ,pM that are trying to reach agreement .
– A subset F of the processors are faulty, and remaining processors are non-faulty. Each
processor pi in P stores a value Vi .
• During the agreement protocol, the processors calculate an agreement value
Ai . After the protocol ends, the following two conditions should hold :
① For every pair pi and pj of non-faulty processors, Ai = Aj. This value is the
agreement value.
② The agreement valu e is a function of the initial values { Vi} of the Non-
faulty processors (P - F).
 System model
• Agreement problems have been studied under the
following system model :
– There are n processors in the system and at most m of
the processors can be faulty.
– The processors can directly communicate with other
processors by message passing.
– A receiver processor always knows the identity of the
sender processor of the message.
– The communication medium is reliable (i.e., it delivers
all messages without introducing any errors) and only
processors are prone to failure.
 Synchronous vs. Asynchronous
Computations
• Synchronous computation
– A process receives messages (1 round), performs a
computation (2 round), and send messages to
other processes (3 round).
• Asynchronous computation
– The computation at processes does not proceed in
lock steps. A process can send and receive
messages and perform computation at any time.
 Model of Processor Failures

• A processor can fail in three modes:

– Crash fault : a processor stops functioning and
never resumes operation .
– Omission fault : a processor "omits" to send
messages to some processors.
– Malicious fault (Byzantine faults): a processor
behaves randomly and arbitrarily.
 Authenticated vs. Non-Authenticated
Messages
• Authenticated message system
– A (faulty) processor cannot forge a message or change the
contents of a received message.
– A processor can verify the authenticity of a received message.
– An authenticated message is also called a signed message.
• Non-authenticated message system
– A (faulty) processor can forge a message and claim to have
received it from another processor or change the contents of a
received message before it relays the message to other processors.
– A processor have no way of verifying the authenticity of a received
message.
– A non-authenticated message is also called an oral message.
 Performance Aspects
• Performance of agreement protocols
– time: the number of rounds
– message traffic: the number of messages
exchanged to reach an agreement
– storage overhead : the amount of information that
needs to be stored at processors during the
execution of a protocol .
A Classification of Agreement Protocols
• Three well known problems
– ① Byzantine agreement problem
– ② Consensus problem
– ③ Interactive consistency problem
• Byzantine agreement problem
– A single value, which is to be agreed on, is initialized by an arbitrary
processor and all non-faulty processors have to agree on that value.
• Consensus problem
– Every processor has its own initial value and all non-faulty processor
must agree on a single common value.
• Interactive consistency problem
– Every processor has its own initial value and all non-faulty processors
must agree on a set of common values.
The Byzantine Agreement Problem
• Three generals can not reach Byzantine agreement
• An arbitrarily chosen processor, called the source processor,
broadcasts its initial value to all other processors.
• A solution to the Byzantine agreement problem should meet the
following two objectives:
– Agreement : All non-faulty processors agree on the same value.
– Validity: If the source processor is non-faulty, then the common agreed upon
value by all non-faulty processors should be initial value of the source.
• Two points should be noted :
① If the source processor is faulty, then all non-faulty processors can
agree on any common value.
② It is irrelevant what value faulty processors agree on or whether
they agree on a value at all.
The Byzantine Agreement Problem
 The Consensus Problem
• Every processor broadcasts its initial value to all other
processors.
• Initial values of the processors may be different .
• A protocol for reaching consensus should meet the following
conditions:
– Agreement : All non-faulty processors agree on the same single value.
– Validity: If the initial value of every non-faulty processor is v, then
agreed upon common value by all nonfaulty processor must be v .

• Note that if the initial values of non-faulty processors are

different, then all non-faulty processors can agree on any
common value.
 The Interactive Consistency Problem
• Every processor broadcasts its initial value to all other processors. The
initial values of the processors may be different .
• A protocol for the interactive consistency problem should meet the
following conditions:
– Agreement : All non-faulty processors agree on the same vector, (v1 , v2 , ... ,
vn ).
– Validity: If the ith processor is non-faulty and its initial value is vi, then the ith
value to be agreed on by all non-faulty processors must be vi .

• Note that if the jth processor is faulty, then all non-faulty processors can
agree on any common value for vj .

• The Byzantine agreement problem is primitive to the other two

agreement problems.
Solution to Byzantine Agreement Problem
• Also referred to as the Byzantine generals problem.

• We now show that a Byzantine agreement cannot be reached

among three processors, where one processor is faulty.
• Consider a system with three processors, p0 , p1 , and p2 . For
simplicity, we assume that there are only two values, 0 and 1, on
which processors agree and processor p0 initiates the initial value.
• Case I: p0 is not faulty.
– Since p0 is non-faulty, processor p1 must accept 1 as the agreed upon
value if condition 2 is to be satisfied.
• Case II: p0 is faulty
– p0 will agree on a value of 1 and p2 will agree on a value of 0, which will
violate condition 1 of the solution .
Solution to Byzantine Agreement Problem
 Lamport-Shostak-Pease Algorithm
• Lamport et al . ' s algorithm, referred to as the Oral Message algorithm
OM(m), m>0, solves the Byzantine agreement problem for 3m+1 or more
processors in the presence of at most m faulty processors. Let n denote
the total number of processors (clearly, n≥3m+1).
• Algorithm OM(0)
– 1. The source processor sends its value to every processor .
– 2. Each processor uses the value it receives from the source. (If it receives no value, then
it uses a default value of 0)
• Algorithm OM(m), m>0.
– 1. The source processor sends its value to every processor .
– 2. For each i, let vi be the value processor i receives from the source. (If it receives no
value, then it uses a default value of 0.). Processor i acts as the new source and initiates
Algorithm OM(m-1) wherein it sends the value vi to each of the n-2 other processors.
– 3. For each i and each j (≠ i), let vj be the value processor i received from processor j in
Step 2. using Algorithm OM(m-1). Processor i uses the value majority(v1, v2, ... , vn - 1).
• The message complexity of the algorithm is O(nm).
Lamport-Shostak-Pease Algorithm:
Example
 Lamport-Shostak-Pease Algorithm:
Example
• An execution of BG(2) on seven generals.
– Oi represents the command sent to Li, and Li : Oi is Li’s rebroadcast of its command . Lj :
Li : Oi is Lj’s rebroadcast of what Li said his order was.