BIRLA INSTITUTE OF TECHNOLOGY & SCIENCE, PILANI

WORK INTEGRATED LEARNING PROGRAMMES

COURSE HANDOUT

Part A: Content Design

Course Title DISTRIBUTED COMPUTING

Course ID No. CC ZG526
Credit Units 5
Authors Dr. CHITTRANJAN HOTA, LUCY J GUDINO
Version No 2.1 (Updated by Anindya Neogi for MTech CC)
Date 3 July 2023

Course Objectives
No Course Objective

CO1 To provide an overview of systems architectures and communication models for distributed
computing

CO2 To provide exposure to the design aspects / attributes of distributed software including
algorithm design.

CO3 To provide an understanding of issues in and solutions for resource management issues and
global state management.

CO4 To provide an understanding of the complexities of distributed computing in the context of peer
to peer systems, cluster computing and distributed databases.

CO5 To provide an exposure to fundamental issues and approaches to distributed data.

Text Book(s)
T1 Ajay D. Kshemkalyani, and Mukesh Singhal “Distributed Computing: Principles, Algorithms,
and Systems”, Cambridge University Press, 2008 (Reprint 2013).

Reference Book(s) & other resources

R1 John F. Buford, Heather Yu, and Eng K. Lua, “P2P Networking and Applications”, Morgan
Kaufmann, 2009 Elsevier Inc.
R2 Kai Hwang, Geoffrey C. Fox, and Jack J. Dongarra, “Distributed and Cloud Computing: From
Parallel processing to the Internet of Things”, Morgan Kaufmann, 2012 Elsevier Inc.
R3 A. Tanenbaum and M. V. Steen, “Distributed Systems”, 2nd Edition, Pearson Prentice Hall
AR Additional References provided as necessary

Content Structure
Module Module Title Objectives
No
1 Introduction to This module will give an introduction to Distributed computing
Distributed Computing models including hardware and software. It will also cover
relevant communication / messaging models. Design issues and
challenges for building distributed computing solutions will also
be discussed.
2 Logical Clocks & Vector This module will explore ways of proving and managing a
Clocks global notion of time (or clock) in a distributed computing
system where there is no centralized hardware clock is available
or feasible.
3 Global state and snapshot This module will (i) highlight the complexity of recording of a
recording algorithms global state in the absence of a single physical memory and a
single physical clock in a distributed system and (ii) explore
ways or algorithms to record or collect global state of a
distributed computation.
4 Distributed algorithms – This module will introduce a setting in which distributed
Design algorithms can be designed, expressed, and analysed.
Algorithms for a few basic graph problems in a distributed
environment will also be covered.
5 Message ordering and In a distributed system, message ordering plays an important
Termination detection role on making the system consistent or stable. The order of
messages sent should match (happen prior) the order of its’
receipt in a group communication application implemented in a
distributed system. This module will discuss ways to achieve
various message ordering schemes. Also, the approaches for
detecting termination of a distributed computation will be
discussed.
6 Distributed Mutual In a distributed system when multiple entities compete for a
Exclusion & Deadlock shared resource, the access to this shared resource has to be
detection serialized (or coordinated). This module will cover different
assertion based, and tree based distributed algorithms to
implement DME. Deadlocks are very common in distributed
computing systems, and this module will discuss ways to detect
this deadlock and resolve them.
7 Distributed Fault When multiple entities cooperate with each other in solving a
Tolerance: Consensus and complex function or task in a distributed system, there are
Agreement Algorithms, instances where a majority of these entities must agree on
Failure Recovery certain decisions without which the task cannot be solved. This
module will discuss ways to achieve consensus and failure
recovery. The module will also discuss checkpointing and
logging techniques including lineage models for fault recovery.
8 Peer-to-Peer computing P2P architecture is used for build various large scale-out
and Overlay graphs systems, e.g. databases, caching, content distribution, digital
currency etc. This module covers the architectural and algorithm
design options for a P2P overlay network.
9 Cluster Computing This module introduces Cluster computing as (i) a model of
carrying out distributed computation using a collection of
homogeneous computers and (ii) as a building block for Cloud
infrastructure.
10 Distributed data Distributed systems need to provide a view of data consistency
at a global level or specific to clients connecting to the system.
This module will discuss replication, various types of
consistency and protocols. Practical examples will be discussed
through case studies of modern distributed databases.

Learning Outcomes:
No Learning Outcomes
LO1 Students will be able to understand articulate the design aspects of distributed computing
including the paradigm of “client-server” and its manifestations such as RPC and Cloud.
LO2 Students will be able to appraise/assess the complexity of fundamental issues in distributed
computing including logical time/clock, global state, and failures as well as design
solutions to deal with these features or limitations.
LO3 Students will be able to explain Mutual exclusion primitives, Agreement protocols,
Consistency and Deadlock handling scenarios in distributed systems.
LO4 Students will be able to explain the design of scalable solutions in distributed computing
environments, such as P2P, Cluster, Distributed Databases.
LO5 Students will be able to understand articulate design issues in handling distributed data and
how these issues are addressed in contemporary solutions.

Part B: Contact Session Plan

Academic Term First Semester 2024-2025

Course Title DISTRIBUTED COMPUTING

Course No CC ZG 526

Lead Instructor Anil Kumar G

Course Contents

Contact List of Topic Title Text/Ref

Sessions(#) (from content structure in Course Handout) Book/external
resource
1 M1: Introduction to Distributed Computing T1 (Chap.1)
Motivation, Multiprocessor Vs Multicomputer Systems.
Distributed Communication models: Remote Procedure Call,
Publish/Subscribe model, Message Queues etc.
Design issues and Challenges for building distributed computing
systems.

2 M2: Logical Clocks & Vector clocks T1 (Chap.3)

A framework for a system of logical clocks.
Scalar time, Vector time.
Implementation of Logical and Vector clocks, Efficient
implementation of Vector clocks.
Physical Clock synchronization: NTP

3 M3: Global state and snapshot recording algorithms T1 (Chap.4)

System model and definitions
Snapshot recording algorithms for FIFO channels
Snapshot recording algorithms for non-FIFO channels
Necessary and sufficient conditions for consistent global snapshots.
4 M4: Distributed algorithms – Design T1 (Chap.5)
Terminology and basic algorithms
Classifications and basic concepts
Elementary graph algorithms, Synchronizers.
Maximal Independent set, and Connected dominating set.

5 M5: Message ordering and Termination detection T1 (Chap.6)

Message ordering paradigms
Group Communication
Protocols for ensuring Causal order of messages

6 M5: Message ordering and Termination detection T1 (Chap.6), T1

Total order (Chap.7)
Application level multicast
[Distributed Multicast and Steiner Trees - optional]
Termination detection using distributed snapshots
Termination detection using weight throwing
A spanning-tree based termination detection algorithm

7 M6: Distributed Mutual Exclusion & Deadlock detection T1 (Chap.9)

Introduction and Preliminaries
Assertion based: Lamport’s algorithm, and Ricart-Agrawala’s
algorithm
Assertion based: Maekawa’s algorithm

8 M6: Distributed Mutual Exclusion & Deadlock detection T1 (Chap.9), T1

Token based: Suzuki-Kasami’s broadcast based algorithm (Chap.10)
Token based: Raymond’s tree based algorithm
Models of distributed deadlock
Chandy-Misra-Haas deadlock detection for AND model
Chandy-Misra-Haas deadlock detection for OR model
Deadlock resolution

9 M7: Distributed Fault Tolerance: Consensus and Agreement T1 (Chap.14)

Algorithms
Problem definition
The Byzantine agreement and other consensus problems
Overview of Results
Agreement in failure-free system (synchronous or asynchronous)
Agreement in (message-passing) synchronous systems with failures

10 M7: Distributed Fault Tolerance: Checkpointing and Rollback. T1(Chapter 13),

Checkpointing algorithms. Log-based Rollback recoveries. Lineage- AR
based models

11 M8: Peer-to-Peer computing and Overlay graphs T1 (Chap.18)

Introduction R1 (Chap. 14)
Data indexing and Overlays
Unstructured Overlays
Structured Overlays: CHORD DHT
[Following topics could move to next session]
Design issues of P2P overlays
 Security concerns from P2P networks
Mitigating security risks in P2P networks

12 M8: Peer-to-Peer computing and Overlay graphs T1 (Chap.18)

Graph structure of Complex networks
Internet Graphs
Generalized Random graph networks
Small-world and Scale-free networks

13 M9: Cluster Computing R2 (Chap.2)

Cluster development trends
Design objectives of Computer clusters
Cluster organization and resource sharing
Node architecture and MPP packaging

14 M9: Cluster Computing R2 (Chap.2)

Cluster system interconnects
Hardware, software and Middleware support
GPU Clusters for massive parallelism
Cluster job and resource management

15 M10: Consistency in Distributed Systems R3 (Chap.7), AR

CAP theorem and implications in modern distributed systems
Data centric consistency
Client centric consistency
Replica management
Consistency protocols
[Handling failures: Paxos family of algorithms - optional]

16 M10: Case studies – Distributed Databases AR

MongoDB, Cassandra, CouchDB, Dynamo architecture

Evaluation Scheme:
Legend: EC = Evaluation Component; AN = After Noon Session; FN = Fore Noon Session
No Name Type Duration Weight Day, Date, Session, Time
EC-1 Quiz - - 10% Before and after mid sem
2 Assignments - - 25% To be announced
EC-2 Mid-Semester Test Closed 2 hours 25%
Saturday, 21/09/2024 (AN)
Book
EC-3 Comprehensive Exam Open Book 2.5 hours 40% Saturday, 30/11/2024 (AN)
Note - Evaluation components can be tailored depending on the proposed model.

Important Information
Syllabus for Mid-Semester Test (Closed Book): Topics in Weeks 1-7
Syllabus for Comprehensive Exam (Open Book): All topics given in plan of study

Evaluation Guidelines:
1. EC-1 consists of either two Assignments or three Quizzes. Announcements regarding the same will
be made in a timely manner.
2. For Closed Book tests: No books or reference material of any kind will be permitted.
Laptops/Mobiles of any kind are not allowed. Exchange of any material is not allowed.
3. For Open Book exams: Use of prescribed and reference text books, in original (not photocopies) is
permitted. Class notes/slides as reference material in filed or bound form is permitted. However,
 loose sheets of paper will not be allowed. Use of calculators is permitted in all exams.
Laptops/Mobiles of any kind are not allowed. Exchange of any material is not allowed.
4. If a student is unable to appear for the Regular Test/Exam due to genuine exigencies, the student
should follow the procedure to apply for the Make-Up Test/Exam. The genuineness of the reason for
absence in the Regular Exam shall be assessed prior to giving permission to appear for the Make-up
Exam. Make-Up Test/Exam will be conducted only at selected exam centres on the dates to be
announced later.
It shall be the responsibility of the individual student to be regular in maintaining the self-study schedule as
given in the course handout, attend the lectures, and take all the prescribed evaluation components such as
Assignment/Quiz, Mid-Semester Test and Comprehensive Exam according to the evaluation scheme
provided in the handout.