Clock Synchronization

Clocks in Distributed Systems

• Needed to Order two or more events happening at same or

different nodes (Ex: Consistent ordering of updates at
different replicas, debugging from distributed logs, ordering
of multicast messages sent in a group)
• Decide if two events happened between some fixed duration
of each other (Ex: Replay of stolen messages in distributed
authentication protocols like Kerberos)
• Start events at different nodes together at the same time (Ex:
tracking in sensor networks, sleep/wakeup scheduling)
• Easy if a globally synchronized clock is available
• Perfectly synchronized clocks are impossible to achieve
• Perfect synchronization may not be needed always;
synchronization within bounds may be enough
– Degree of synchronization needed depends on
application
Synchronization
– is required for
• Fairness
• Correctness
– is complicated in DS
• Use of distributed algorithms
Properties of distributed algorithms:
– Relevant information is scattered among multiple
machines
– Decision making is based on local information
– Single point of failure should be avoided
– No common clock or precise global time source exists
Two approaches for building clocks
–Logical Clocks
–Use the notion of causality to order events
–Can what happened in one event affect what happens in
another?
• Because if not, ordering them is not important
• Useful for ordering events, but not for giving a
consistent view of “current time” across all nodes
Physical Clocks
–Each machine has its own local clock
–Clock synchronization algorithms run
periodically to keep them synchronized with
each other within some bounds
–Useful for giving a consistent view of
“current time” across all nodes within some
bounds, but cannot order events always
 Basics – Processes and Events
• A Distributed System (DS) consists of a number of processes.
– Each process has a state (values of variables).
– Each process takes actions to change its state, which may be an instruction or
a communication action (send, receive).
• An event is the occurrence of an action.
• Each process has a local clock -- events within a process can be assigned
timestamps, and thus ordered.
• But – in a DS, we also need to know the time order of events across different
processes.
• Clocks across processes are not synchronized in a DS
(unlike in a multiprocessor/parallel system, where they are)
1. Process clocks can be different
2. Need algorithms for either (a) time synchronization, or (b) for telling which
event happened before which
 Different notions of time
• use of time-stamps, e.g.
– event ordering
– life-times of authentication tickets / avoiding replay attacks

• Coordinated Universal Time (UTC)

– International time based on atomic clocks (drift rate 10-13)
– occasionally adjusted to astronomic time (leap seconds, etc.)
– broadcasted via radio stations and satellite (GPS)
– accuracy:
• land-based signals: 0.1-10ms
• GPS: 1μs
• Internal clocks
– based on counters Hi(t) with some increment frequency
– for generation of time-stamps: Ci(t)=ai*Hi(t)+bi
– ordinary quartz clocks: ~1sec drift in 11-12 days
 Logical Clocks
• Is it always necessary to give absolute time to events?
• Suppose we can assign relative time to events, in a way that does not violate
their causality
• Define a logical relation Happens-Before () among events:
On the same process: a  b, if time(a) < time(b)
If p1 sends m to p2: send(m)  receive(m)
(Transitivity) If a  b and b  c then a  c

• Predecessor events potentially causally affect successor

i.e. Past events influence future events.
• For any two events a & b in a system, a  b or b  a or a||b
First proposed by Leslie Lamport in the 70’s
Lamport Algorithm assigns logical timestamps to events:
All processes use a counter (clock) with initial value of zero
A process increments its counter when a send or an instruction happens
at it. The counter is assigned to the event as its timestamp.
A send (message) event carries its timestamp
For a receive (message) event the counter is updated by
max(local clock, message timestamp) + 1
Events Occurring at Three Processes

p1
a b m1

p2 Physical
c d time
m2

p3
e f
 Lamport Timestamps
1 2
p1
a b m1

3 4
Physical
p2
time
c d
m2

1 5
p3
e f

With Lamport Logical Timestamp

e “happens-before” f  timestamp(e) < timestamp (f),
but
timestamp(e) < timestamp (f) does not mean e “happens-
before” f
 Vector Logical Clocks
Vector Logical time is better:
All processes use a vector of counters (logical clocks), i th element is the
clock value for process i, initially all zero.
Each process i increments the ith element of its vector upon an instruction
or send event.
Vector value is timestamp of the event.
A send(message) event carries its vector timestamp (counter vector)
For a receive(message) event,
Max(Vreceiver[j] , Vmessage[j]), if j is not self
Vreceiver[j] + 1 otherwise
 Physical Clocks

• Each node has a local clock used by it to timestamp events at

the node
• Local clocks of different nodes may vary
• Need to keep them synchronized (Clock Synchronization
Problem)
• Perfect synchronization not possible because of inability to
estimate network delays exactly
Clock Synchronization
• Internal Synchronization Requires the clocks of the nodes to be
synchronized to within a pre-specified bound
• However, the clock times may not be synchronized to any external time
reference, and can vary arbitrarily from any such reference

• External Synchronization Requires the clocks to be synchronized to within

a pre-specified bound of an external reference clock
 How Computer Clocks Work

• Computer clocks are based on crystals that oscillate at a

certain frequency
• Every H oscillations, the timer chip interrupts once (clock
tick)Resolution: time between two interrupts
• The interrupt handler increments a counter that keeps track
of no. of ticks from a reference in the past (epoch)
• Knowing number of ticks per second, we can calculate year,
month, day, time of day etc.
 Why Clocks Differ: Clock Drift

• Period of crystal oscillation varies slightly due to temperature, humidity,

aging,…
• If it oscillates faster, more ticks per real second, so clock runs faster;
similar for slower clocks
• For machine p, when correct reference time is t, let machine clock show
time as C = Cp(t)
• Ideally, Cp(t) = t for all p, t
• In practice, 1 –ρ≤ dC/dt ≤ 1 + ρ
• ρ= max. clock drift rate, usually around 10-5 for cheap oscillators
• Drift results in skew between clocks (difference in clock values of two
machines)
Resynchronization
• Periodic resynchronization needed to offset skew
• If two clocks are drifting in opposite directions, max. skew after time t is
2ρt
• If application requires that clock skew < δ, then resynchronization period
r < δ/(2 ρ)
– Usually ρ and δ are known
– ρ given by crystal manufacturer
– δ specified from application requirement
• Clock Skew versus Drift
– Clock Skew = Relative Difference in clock values of two processes
– Clock Drift = Relative Difference in clock frequencies (rates) of two
processes
• A non-zero clock drift will cause skew to continuously increase.
• Maximum Drift Rate (MDR) of a clock is defined relative to Coordinated
Universal Time (UTC)
– MDR of a process depends on the environment.
• Max drift rate between two clocks with similar MDR is 2 * MDR
• Max-Synch-Interval = (MaxAcceptableSkew—CurrentSkew) / (MDR
* 2)
 Implementation of Physical Time
Service
• Obtaining accurate value when implmenting a
physical time service
• Synchronizing the concept of physical time
throughout the distributed system.
• Implementation
– Centralized
– Distributed
 Centralized Physical service
• Broadcast based
• Request driven
Problems
– Single point of failure
– Traffic around server increases
– Not scalable
Clock Synchronization Using a Time
Server

mr

mt

p Time server,S
 Broadcast based
• First approach
– Time service broadcast time periodically
– Clients evaluate their clock in comparison with
broadcast time
– Client clock ahead --- slows down its clock
– Client clock behind --- forward the clock or
gradually increase
– No fault tolerance
 II approach - Berkeley Algorithm
• Internal synchronization of group of computers
• Uses an elected master process to synchronize
among clients, without the presence of a time server
• The elected master pools or broadcasts to all
machines requesting for their time,
• Adjusts times received for RTT & latency, averages
times, and tells each machine how to adjust.
• Uses round trip time estimates to obtain slaves’ clock
values
• Elimination of out-liers (large RTTs, strong
deviations from average)
• Form average and send required adjustments to
slaves
• Upon failure of master, re-election by group
Drawbacks:
•Multiple leaders are used.
•Averaging client’s clocks may cause the entire
system to drift away from UTC over time
•Failure of the master requires some time for re-
election, so accuracy cannot be guaranteed
Request Driven - Cristian’s Algorithm

• Uses a time server to synchronize clocks

• Time server keeps the reference time
• A client asks the time server for time, the server
responds with its current time, and the client uses
the value T in the response message to set clock
Inaccuracies and other problems
– asymmetric communication links
– processing delays
– delay variations (jitter)
– drift(clock A) deviates from drift(clock B)
– Clock monotonicity
– Time server may fail/be faulty
Round-trip time introduces an error…
Let RTT = response-received-time – request-sent-time
(measurable at client)
Also, suppose we know (1) the minimum value min of the client-
server one-way transmission time
(2) that the server timestamped the message at the last possible
instant before sending it back
Then, the actual time could be between [T+min,T+RTT— min]
 Distributed Physical service
• Client broadcast its current time at predefined set intervals
• Starts timer
• Collects messages
• Calculates average values and then adjust time values.
 The Network Time Protocol (NTP)
Uses a network of time servers to synchronize all
processes on a network.
Time servers are connected by a synchronization subnet
tree. The root is adjusted directly . Each node
synchronizes its
children nodes. Primary server, direct synch.
1

Secondry servers,
synched by the 2 2 2
Strata 3,
primary server synched by the
3 3 3 3 3 3 secondary
servers