Distributed Systems

(3rd Edition)

Chapter 01: Introduction

Version: March 20, 2022
Introduction: What is a distributed system?

Distributed System

Definition
A distributed system is a collection of autonomous computing elements
that appears to its users as a single coherent system.

Characteristic features
► Autonomous computing elements, also referred to as nodes, be
they hardware devices or software processes.
► Single coherent system: users or applications perceive a single
system ⇒ nodes need to collaborate.

2 / 56
Introduction: What is a distributed system? Characteristic 1: Collection of autonomous computing elements

Collection of autonomous nodes

Independent behavior
Each node is autonomous and will thus have its own notion of time:
there is no global clock. Leads to fundamental synchronization and
coordination problems.

Collection of nodes
► How to manage group membership?
► How to know that you are indeed communicating with an
authorized (non)member?

3 / 56
Introduction: What is a distributed system? Characteristic 1: Collection of autonomous computing elements

Organization

Overlay network
Each node in the collection communicates only with other nodes in the
system, its neighbors. The set of neighbors may be dynamic, or may
even be known only implicitly (i.e., requires a lookup).

Structured: each node has a well-defined set of neighbors with whom

it can communicate (tree, ring).
Unstructured: each node has references to randomly selected other
nodes from the system.

4 / 56
Introduction: What is a distributed system? Characteristic 2: Single coherent system

Coherent system

Essence
The collection of nodes as a whole operates the same, no matter
where, when, and how interaction between a user and the system
takes place.

Examples
► An end user cannot tell where a computation is taking place
► Where data is exactly stored should be irrelevant to an
application
► If or not data has been replicated is completely hidden
Keyword is distribution transparency

5 / 56
Introduction: What is a distributed system? Middleware and distributed systems

Middleware: the OS of distributed systems

Same interface everywhere

Computer 1 Computer 2 Computer 3 Computer 4

Appl. A Application B Appl. C

Distributed-system layer (middleware)

Local OS 1 Local OS 2 Local OS 3 Local OS 4

Network

What does it contain?

Commonly used components and functions that need not be
implemented by applications separately.

6 / 56
Introduction: Design goals

What do we want to achieve?

► Support sharing of resources

► Distribution transparency
► Openness
► Scalability

7 / 56
Introduction: Design goals Supporting resource sharing

Sharing resources

Canonical examples
► Cloud-based shared storage and files
► Peer-to-peer assisted multimedia streaming
► Shared mail services (think of outsourced mail systems)
► Shared Web hosting (think of content distribution networks)

Observation
“The network is the computer”
(quote from John Gage, then at Sun Microsystems)

8 / 56
Introduction: Design goals Making distribution transparent

Distribution transparency

Types
Transparency Description
Access Hide differences in data representation and how
an object is accessed
Location Hide where an object is located
Relocation Hide that an object may be moved to another
location while in use
Migration Hide that an object may move to another location
Replication Hide that an object is replicated
Concurrency Hide that an object may be shared by several
independent users
Failure Hide the failure and recovery of an object

Types of distribution transparency 9 / 56

Introduction: Design goals Making distribution transparent

Degree of transparency

Observation
Aiming at full distribution transparency may be too much:
► There are communication latencies that cannot be hidden
► Completely hiding failures of networks and nodes is (theoretically
and practically) impossible
► You cannot distinguish a slow computer from a failing one
► You can never be sure that a server actually performed an
operation before a crash
► Full transparency will cost performance, exposing distribution of
the system
► Keeping replicas exactly up-to-date with the master takes
time
► Immediately flushing write operations to disk for fault
tolerance

Degree of distribution transparency 10 / 56

Introduction: Design goals Making distribution transparent

Degree of transparency

Exposing distribution may be good

► Making use of location-based services (finding your nearby
friends)
► When dealing with users in different time zones
► When it makes it easier for a user to understand what’s going on
(when e.g., a server does not respond for a long time, report it as
failing).

Conclusion
Distribution transparency is a nice a goal, but achieving it is a different
story, and it should often not even be aimed at.

Degree of distribution transparency 11 / 56

Introduction: Design goals Being open

Openness of distributed systems

What are we talking about?

Be able to interact with services from other open systems, irrespective
of the underlying environment:
► Systems should conform to well-defined interfaces
► Systems should easily interoperate
► Systems should support portability of applications
► Systems should be easily extensible

Interoperability, composability, and extensibility 12 / 56

Introduction: Design goals Being scalable

Scale in distributed systems

Observation
Many developers of modern distributed systems easily use the
adjective “scalable” without making clear why their system actually
scales.

At least three components

► Number of users and/or
processes (size scalability)
► Maximum distance between
nodes (geographical scalability)
► Number of administrative
domains (administrative
scalability)

Observation
Most systems account only, to a certain extent, for size scalability.
Often a solution: multiple powerful servers operating independently in
Scalability dimensions 15 / 56
Introduction: Design goals Being scalable

Techniques for scaling

Hide communication latencies

► Make use of asynchronous communication
► Have separate handler for incoming response
► Problem: not every application fits this model

Scaling techniques 22 / 56
Introduction: Design goals Being scalable

Techniques for scaling

Facilitate solution by moving computations to client

Client Server

MAARTEN M
FIRST NAME A
LAST NAME VAN STEEN A
E-MAIL R
MVS@VAN-STEEN.NET T
E
N

Check form Process form

Client Server

FIRST NAME MAARTEN

MAARTEN
LAST VAN STEEN VAN STEEN
NAME E- MVS@VAN-STEEN.NET MVS@VAN-
MAIL STEEN.NET

Check form Process form

Scaling techniques 23 / 56
Introduction: Design goals Being scalable

Techniques for scaling

Partition data and computations across multiple machines

► Move computations to clients (Java applets)
► Decentralized naming services (DNS)
► Decentralized information systems (WWW)

Scaling techniques 24 / 56
Introduction: Design goals Being scalable

Techniques for scaling

Replication and caching: Make copies of data available at

different machines
► Replicated file servers and databases
► Mirrored Web sites
► Web caches (in browsers and proxies)
► File caching (at server and client)

Scaling techniques 25 / 56
Introduction: Types of distributed systems

Three types of distributed systems

► High performance distributed computing systems

► Distributed information systems
► Distributed systems for pervasive computing

28 / 56
Introduction: Types of distributed systems High performance distributed computing

Parallel computing

Observation
High-performance distributed computing started with parallel
computing

Multiprocessor and multicore versus

multicomputer
M SharedMmemoryM M M M M
Private memory

Interconnect P P P P

P P P Interconnect

Processor Memory

29 / 56
Introduction: Types of distributed systems High performance distributed computing

Cluster computing

Essentially a group of high-end systems connected

through a LAN
► Homogeneous: same OS, near-identical hardware
► Single managing node

Master node Compute node Compute node Compute node

Management Component Component Component

application of of of
parallel parallel parallel
application application application
Parallel libs

Local OS Local OS Local OS Local OS

Remote access Standard network

network
High-speed network

Cluster computing 31 / 56
Introduction: Types of distributed systems High performance distributed computing

Grid computing

The next step: lots of nodes from everywhere

► Heterogeneous
► Dispersed across several organizations
► Can easily span a wide-area network

Note
To allow for collaborations, grids generally use virtual organizations. In
essence, this is a grouping of users (or better: their IDs) that will allow
for authorization on resource allocation.

Grid computing 32 / 56
Introduction: Types of distributed systems High performance distributed computing

Architecture for grid computing

The layers
Fabric: Provides interfaces to local
resources (for querying state and
capabilities, locking, etc.)
Applications Connectivity: Communication/transaction
protocols, e.g., for moving data
Collective layer
between resources. Also various
authentication protocols.

Connectivity layer Resource layer Resource: Manages a single resource,

such as creating processes or reading
data.
Fabric layer
Collective: Handles access to multiple
resources: discovery, scheduling,
replication.
Application: Contains actual grid
applications in a single organization.

Grid computing 33 / 56
Introduction: Types of distributed systems High performance distributed computing

Cloud computing
Google docs
aa Svc
Software

Web services, multimedia, business apps Gmail

YouTube, Flickr
Application
MS Azure
Software framework (Java/Python/.Net) Google App engine
Storage (databases)
Platform
aa Svc

Platforms
Amazon S3
Computation (VM), storage (block, file) Amazon EC2
Infrastructure
Infrastructure
aa Svc

CPU, memory, disk, bandwidth Datacenters

Hardware

•Characterized by easily usable and accessible virtualized

resource pools.
•Dynamic resource configuration allows for scalability based
on user needs (public use).
•Operates on a pay-per-use model
Cloud computing 34 / 56
Introduction: Types of distributed systems High performance distributed computing

Cloud computing

Make a distinction between four layers

► Hardware: Processors, routers, power and cooling systems.
Customers normally never get to see these.
► Infrastructure: Deploys virtualization techniques. Evolves around
allocating and managing virtual storage devices and virtual
servers.
► Platform: Provides higher-level abstractions for storage
► Application: Actual applications, such as office suites (text
processors, spreadsheet applications, presentation applications).
Comparable to the suite of apps shipped with OSes.

Cloud computing 35 / 56
Introduction: Types of distributed systems Distributed information systems

Integrating applications

Situation
Organizations confronted with many networked applications, but
achieving interoperability was painful.

Basic approach
A networked application is one that runs on a server making its
services available to remote clients. Simple integration: clients
combine requests for (different) applications; send that off; collect
responses, and present a coherent result to the user.

Next step
Allow direct application-to-application communication, leading to
Enterprise Application Integration.

42 / 56
Introduction: Types of distributed systems Distributed information systems

TPM: Transaction Processing Monitor

Server
Reply
Transaction Request
Requests
Request
Client TP monitor Server
application
Reply
Reply
Request

Server
Reply

Observation
In many cases, the data involved in a transaction is distributed across
several servers. A TP Monitor is responsible for coordinating the
execution of a transaction.

Distributed transaction processing 44 / 56

 Introduction: Types of distributed systems Distributed information systems

Middleware and EAI

Client Client
application application

Communication middleware

Server-side Server-side Server-side

application application application

Middleware offers communication facilities for integration

• EAI emphasizes direct communication between applications to

enable real-time updates and parallel processing.

Enterprise application integration 45 / 56

Introduction: Types of distributed systems Pervasive systems

Distributed pervasive systems

Observation
Emerging next-generation of distributed systems in which nodes are
small, mobile, and often embedded in a larger system, characterized
by the fact that the system naturally blends into the user’s environment.

Three (overlapping) subtypes

► Ubiquitous computing systems: pervasive and continuously
present, i.e., there is a continuous interaction between system
and user.
► Mobile computing systems: pervasive, but emphasis is on the fact
that devices are inherently mobile.
► Sensor (and actuator) networks: pervasive, with emphasis on the
actual (collaborative) sensing and actuation of the environment.

47 / 56