Parallel Database

Contents

Intraquery Parallelism
Intraoperation Parallelism
Interoperation Parallelism
Design of Parallel Systems

1
 Intraquery Parallelism
• Execution of a single query in parallel on multiple
processors/disks; important for speeding up long-running
queries.
• Two complementary forms of intraquery parallelism:
– Intraoperation Parallelism – parallelize the execution of each
individual operation in the query.
– Interoperation Parallelism – execute the different operations
in a query expression in parallel.
the first form scales better with increasing parallelism
because
the number of tuples processed by each operation is
typically more than the number of operations in a query.

2
 Parallel Processing of Relational Operations

• Our discussion of parallel algorithms assumes:

– read-only queries
– shared-nothing architecture
– n processors, P0, ..., Pn-1, and n disks D0, ..., Dn-1, where disk Di is
associated with processor Pi.
• If a processor has multiple disks they can simply simulate a
single disk Di.
• Shared-nothing architectures can be efficiently simulated
on shared-memory and shared-disk systems.
– Algorithms for shared-nothing systems can thus be run on
shared-memory and shared-disk systems.
– However, some optimizations may be possible.

3
 Parallel Sort
Range-Partitioning Sort
• Choose processors P0, ..., Pm, where m  n -1 to do sorting.
• Create range-partition vector with m entries, on the sorting attributes
• Redistribute the relation using range partitioning
– all tuples that lie in the ith range are sent to processor Pi
– Pi stores the tuples it received temporarily on disk Di.
– This step requires I/O and communication overhead.
• Each processor Pi sorts its partition of the relation locally.
• Each processors executes same operation (sort) in parallel with other
processors, without any interaction with the others (data parallelism).
• Final merge operation is trivial: range-partitioning ensures that, for 1 j
m, the key values in processor Pi are all less than the key values in Pj.

4
 Parallel Sort (Cont.)
Parallel External Sort-Merge
• Assume the relation has already been partitioned among disks D0, ...,
Dn-1 (in whatever manner).
• Each processor Pi locally sorts the data on disk Di.
• The sorted runs on each processor are then merged to get the final
sorted output.
• Parallelize the merging of sorted runs as follows:
– The sorted partitions at each processor Pi are range-partitioned across the
processors P0, ..., Pm-1.
– Each processor Pi performs a merge on the streams as they are received, to get
a single sorted run.
– The sorted runs on processors P0,..., Pm-1 are concatenated to get the final
result.

5
 Parallel Join
• The join operation requires pairs of tuples to be
tested to see if they satisfy the join condition, and if
they do, the pair is added to the join output.
• Parallel join algorithms attempt to split the pairs to
be tested over several processors. Each processor
then computes part of the join locally.
• In a final step, the results from each processor can
be collected together to produce the final result.

6
 Partitioned Join
• For equi-joins and natural joins, it is possible to partition the two
input relations across the processors, and compute the join
locally at each processor.
• Let r and s be the input relations, and we want to compute r
r.A=s.B s.

• r and s each are partitioned into n partitions, denoted r0, r1, ..., rn-1
and s0, s1, ..., sn-1.
• Can use either range partitioning or hash partitioning.
• r and s must be partitioned on their join attributes r.A and s.B),
using the same range-partitioning vector or hash function.
• Partitions ri and si are sent to processor Pi,
• Each processor Pi locally computes ri ri.A=si.B si. Any of the standard
join methods can be used.

7
Partitioned Join (Cont.)

8
 Fragment-and-Replicate Join
• Partitioning not possible for some join conditions
– E.g., non-equijoin conditions, such as r.A > s.B.
• For joins were partitioning is not applicable,
parallelization can be accomplished by fragment and
replicate technique
– Depicted on next slide
• Special case – asymmetric fragment-and-replicate:
– One of the relations, say r, is partitioned; any partitioning
technique can be used.
– The other relation, s, is replicated across all the processors.
– Processor Pi then locally computes the join of ri with all of s
using any join technique.

9
Depiction of Fragment-and-Replicate Joins

10
 Fragment-and-Replicate Join (Cont.)
• General case: reduces the sizes of the relations at each
processor.
– r is partitioned into n partitions,r0, r1, ..., r n-1;s is partitioned into
m partitions, s0, s1, ..., sm-1.
– Any partitioning technique may be used.
– There must be at least m * n processors.
– Label the processors as
– P0,0, P0,1, ..., P0,m-1, P1,0, ..., Pn-1m-1.
– Pi,j computes the join of ri with sj. In order to do so, ri is replicated
to Pi,0, Pi,1, ..., Pi,m-1, while si is replicated to P0,i, P1,i, ..., Pn-1,i
– Any join technique can be used at each processor Pi,j.

11
Fragment-and-Replicate Join (Cont.)
• Both versions of fragment-and-replicate work with
any join condition, since every tuple in r can be
tested with every tuple in s.
• Usually has a higher cost than partitioning, since one
of the relations (for asymmetric fragment-and-
replicate) or both relations (for general fragment-
and-replicate) have to be replicated.
• Sometimes asymmetric fragment-and-replicate is
preferable even though partitioning could be used.
– E.g., say s is small and r is large, and already partitioned. It
may be cheaper to replicate s across all processors, rather
than repartition r and s on the join attributes.
12
 Partitioned Parallel Hash-Join
Parallelizing partitioned hash join:
• Assume s is smaller than r and therefore s is chosen as the build
relation.
• A hash function h1 takes the join attribute value of each tuple in s
and maps this tuple to one of the n processors.
• Each processor Pi reads the tuples of s that are on its disk Di, and
sends each tuple to the appropriate processor based on hash
function h1. Let si denote the tuples of relation s that are sent to
processor Pi.
• As tuples of relation s are received at the destination processors,
they are partitioned further using another hash function, h2, which
is used to compute the hash-join locally. (Cont.)

13
 Partitioned Parallel Hash-Join (Cont.)
• Once the tuples of s have been distributed, the larger relation r is
redistributed across the m processors using the hash function h1
– Let ri denote the tuples of relation r that are sent to processor Pi.

• As the r tuples are received at the destination processors, they are

repartitioned using the function h2
– (just as the probe relation is partitioned in the sequential hash-join algorithm).

• Each processor Pi executes the build and probe phases of the hash-join
algorithm on the local partitions ri and s of r and s to produce a partition of
the final result of the hash-join.
• Note: Hash-join optimizations can be applied to the parallel case
– e.g., the hybrid hash-join algorithm can be used to cache some of the incoming
tuples in memory and avoid the cost of writing them and reading them back in.

14
 Parallel Nested-Loop Join
• Assume that
– relation s is much smaller than relation r and that r is stored by partitioning.
– there is an index on a join attribute of relation r at each of the partitions of
relation r.
• Use asymmetric fragment-and-replicate, with relation s being
replicated, and using the existing partitioning of relation r.
• Each processor Pj where a partition of relation s is stored reads the
tuples of relation s stored in Dj, and replicates the tuples to every
other processor Pi.
– At the end of this phase, relation s is replicated at all sites that store tuples
of relation r.
• Each processor Pi performs an indexed nested-loop join of relation s
with the ith partition of relation r.

15
 Other Relational Operations
Selection (r)
• If  is of the form ai = v, where ai is an attribute and v a value.
– If r is partitioned on ai the selection is performed at a single
processor.
• If  is of the form l <= a <= u (i.e.,  is a range selection) and
i

the relation has been range-partitioned on a i

– Selection is performed at each processor whose partition overlaps
with the specified range of values.
• In all other cases: the selection is performed in parallel at all
the processors.

16
Other Relational Operations (Cont.)
• Duplicate elimination
– Perform by using either of the parallel sort techniques
• eliminate duplicates as soon as they are found during sorting.
– Can also partition the tuples (using either range- or hash- partitioning)
and perform duplicate elimination locally at each processor.

• Projection
– Projection without duplicate elimination can be performed as tuples
are read in from disk in parallel.
– If duplicate elimination is required, any of the above duplicate
elimination techniques can be used.

17
 Grouping/Aggregation
• Partition the relation on the grouping attributes and then compute
the aggregate values locally at each processor.
• Can reduce cost of transferring tuples during partitioning by partly
computing aggregate values before partitioning.
• Consider the sum aggregation operation:
– Perform aggregation operation at each processor Pi on those tuples stored
on disk Di
• results in tuples with partial sums at each processor.
– Result of the local aggregation is partitioned on the grouping attributes,
and the aggregation performed again at each processor Pi to get the final
result.
• Fewer tuples need to be sent to other processors during
partitioning.

18
 Cost of Parallel Evaluation of Operations
• If there is no skew in the partitioning, and there is no
overhead due to the parallel evaluation, expected
speed-up will be 1/n
• If skew and overheads are also to be taken into
account, the time taken by a parallel operation can
be estimated as
Tpart + Tasm + max (T0, T1, …, Tn-1)
– Tpart is the time for partitioning the relations
– Tasm is the time for assembling the results
– Ti is the time taken for the operation at processor Pi
• this needs to be estimated taking into account the skew, and the
time wasted in contentions.

19
 Interoperator Parallelism
• Pipelined parallelism
– Consider a join of four relations
• r1 r2 r3 r4
– Set up a pipeline that computes the three joins in parallel
• Let P1 be assigned the computation of
temp1 = r1 r2
• And P2 be assigned the computation of temp2 = temp1 r3
• And P3 be assigned the computation of temp2 r 4
– Each of these operations can execute in parallel, sending result
tuples it computes to the next operation even as it is computing
further results
• Provided a pipelineable join evaluation algorithm (e.g., indexed nested
loops join) is used

20
Factors Limiting Utility of Pipeline Parallelism

• Pipeline parallelism is useful since it avoids writing

intermediate results to disk
• Useful with small number of processors, but does
not scale up well with more processors. One reason
is that pipeline chains do not attain sufficient length.
• Cannot pipeline operators which do not produce
output until all inputs have been accessed (e.g.,
aggregate and sort) 
• Little speedup is obtained for the frequent cases of
skew in which one operator's execution cost is much
higher than the others.
21
 Independent Parallelism
• Independent parallelism
– Consider a join of four relations
r1 r 2 r 3 r 4
• Let P1 be assigned the computation of
temp1 = r1 r2
• And P2 be assigned the computation of temp2 = r3 r4
• And P3 be assigned the computation of temp1 temp2
• P1 and P2 can work independently in parallel
• P3 has to wait for input from P1 and P2
– Can pipeline output of P1 and P2 to P3, combining independent parallelism and
pipelined parallelism
– Does not provide a high degree of parallelism
• useful with a lower degree of parallelism.
• less useful in a highly parallel system.

22
 Query Optimization
• Query optimization in parallel databases is significantly more
complex than query optimization in sequential databases.
• Cost models are more complicated, since we must take into account
partitioning costs and issues such as skew and resource contention.
• When scheduling execution tree in parallel system, must decide:
– How to parallelize each operation and how many processors to use for it.
– What operations to pipeline, what operations to execute independently in
parallel, and what operations to execute sequentially, one after the other.
• Determining the amount of resources to allocate for each operation
is a problem.
– E.g., allocating more processors than optimal can result in high
communication overhead.
• Long pipelines should be avoided as the final operation may wait a
lot for inputs, while holding precious resources

23
•
Query Optimization (Cont.)
The number of parallel evaluation plans from which to choose from is much larger
than the number of sequential evaluation plans.
– Therefore heuristics are needed while optimization
• Two alternative heuristics for choosing parallel plans:
– No pipelining and inter-operation pipelining; just parallelize every operation
across all processors.
• Finding best plan is now much easier --- use standard optimization
technique, but with new cost model
• Volcano parallel database popularize the exchange-operator model
– exchange operator is introduced into query plans to partition and
distribute tuples
– each operation works independently on local data on each
processor, in parallel with other copies of the operation
– First choose most efficient sequential plan and then choose how best to
parallelize the operations in that plan.
• Can explore pipelined parallelism as an option
• Choosing a good physical organization (partitioning technique) is important to
speed up queries.

24
 Design of Parallel Systems
Some issues in the design of parallel systems:
• Parallel loading of data from external sources is needed
in order to handle large volumes of incoming data.
• Resilience to failure of some processors or disks.
– Probability of some disk or processor failing is higher in a
parallel system.
– Operation (perhaps with degraded performance) should be
possible in spite of failure.
– Redundancy achieved by storing extra copy of every data
item at another processor.

25
 Design of Parallel Systems (Cont.)
• On-line reorganization of data and schema changes
must be supported.
– For example, index construction on terabyte databases can
take hours or days even on a parallel system.
• Need to allow other processing (insertions/deletions/updates) to
be performed on relation even as index is being constructed.
– Basic idea: index construction tracks changes and “catches
up” on changes at the end.
• Also need support for on-line repartitioning and
schema changes (executed concurrently with other
processing).
26
Figure 18.01

27
Figure 18.02

28
Figure 18.03

29