Scribd
Upload
25 views31 pages

4 P2P-1

Uploaded by

spareyash
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPSX, PDF, TXT or read online on Scribd
25 views31 pages

4 P2P-1

Uploaded by

spareyash
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPSX, PDF, TXT or read online on Scribd
You are on page 1/ 31

Point-to-point

Communication

Lecture 4
Jan 17, 2023
MPI Program Execution
Memory

host1 host2 host3 host4

Intranode Internode

mpiexec –n 8 –hosts host1,host2,host3,host4 ./exe

2
MPI Program Execution
Memory

host1 host2 host3 host4


hostfile
cn023
Intranode Internode
cn024
cn025
cn026
cn023
cn024
mpiexec –n 8 –f hostfile ./exe cn025
cn026
3
MPI Reference Material
• Marc Snir, Steve W. Otto, Steven Huss-Lederman, David W. Walker and
Jack Dongarra, MPI - The Complete Reference, Second Edition,
Volume 1, The MPI Core.
• William Gropp, Ewing Lusk, Anthony Skjellum, Using MPI: portable
parallel programming with the message-passing interface, 3rd Ed.,
Cambridge MIT Press, 2014.
• https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf
Point-to-point Communication
MPI_Send Blocking send and receive
int MPI_Send (const void *buf, int count, MPI_Datatype
datatype, int dest, int tag, MPI_Comm comm)

SENDER

Tags should match


MPI_Recv

int MPI_Recv (void *buf, int count, MPI_Datatype datatype,


int source, int tag, MPI_Comm comm, MPI_Status *status)

RECEIVER
5
Simple Send/Recv Code (sendmessage.c)

No runtime or
compile-time
error
6
Runtime error
7
Message Size

Sender Receiver
message (13 bytes) Message (10 bytes)

Fatal error in MPI_Recv: Message truncated, error stack:


MPI_Recv(200)...........................: MPI_Recv(buf=0x7ffccc37c610, count=10,
MPI_CHAR, src=0, tag=99, MPI_COMM_WORLD, status=0x7ffccc37c5d0) failed
MPIDI_CH3_PktHandler_EagerShortSend(363): Message from rank 0 and tag 99
truncated; 13 bytes received but buffer size is 10
No runtime or
compile-time
error
9
Simple Send/Recv Code (sendmessage.c)
received : Hello, there

10
Output
0 7 0
Received: Welcome
1 0 7
Multiple Sends and Receives
if (myrank == 0)
MPI_Send (buf, count, MPI_INT, 1, 1, MPI_COMM_WORLD);
else
if (myrank == 1)
MPI_Recv (buf, count, MPI_INT, 0, 1, MPI_COMM_WORLD, &status);

printf ("%d %d\n", myrank, count);


$ mpirun –np 2 ./send 10

0 10
1 10
Multiple Sends and Receives
if (myrank == 0)
MPI_Send (buf, count, MPI_INT, 1, 1, MPI_COMM_WORLD),
MPI_Send (buf, count, MPI_INT, 1, 2, MPI_COMM_WORLD);
else
if (myrank == 1)
MPI_Recv (buf, count, MPI_INT, 0, 1, MPI_COMM_WORLD, &status),
MPI_Recv (buf, count, MPI_INT, 0, 2, MPI_COMM_WORLD, &status);

$ mpirun –np 2 ./send 10


printf ("%d %d\n", myrank, count);
0 10
1 10
Multiple Sends and Receives
if (myrank == 0)
MPI_Send (buf, count, MPI_INT, 1, 1, MPI_COMM_WORLD),
MPI_Send (buf, count, MPI_INT, 1, 2, MPI_COMM_WORLD);
else
if (myrank == 1)
MPI_Recv (buf, count, MPI_INT, 0, 1, MPI_COMM_WORLD, &status),
MPI_Recv (buf, count, MPI_INT, 0, 1, MPI_COMM_WORLD, &status);

$ mpirun –np 2 ./send 10


printf ("%d %d\n", myrank, count);
0 10
Multiple Sends and Receives
if (myrank == 0)
MPI_Send (buf, count, MPI_INT, 1, 1, MPI_COMM_WORLD),
MPI_Send (buf, count, MPI_INT, 1, 2, MPI_COMM_WORLD);
else
if (myrank == 1)
MPI_Recv (buf, count, MPI_INT, 0, 2, MPI_COMM_WORLD, &status),
MPI_Recv (buf, count, MPI_INT, 0, 1, MPI_COMM_WORLD, &status);

$ mpirun –np 2 ./send 10


printf ("%d %d\n", myrank, count);
0 10
1 10
MPI_Send (Blocking, Standard Mode)

• Does not return until buffer can be reused SENDER

• Message buffering can affect this


• Implementation-dependent

RECEIVER
Buffering

[Source: Cray presentation]


0 12 0

18
Multiple Sends and Receives
if (myrank == 0)
MPI_Send (buf, count, MPI_INT, 1, 1, MPI_COMM_WORLD),
MPI_Send (buf, count, MPI_INT, 1, 2, MPI_COMM_WORLD);
else
if (myrank == 1)
MPI_Recv (buf, count, MPI_INT, 0, 2, MPI_COMM_WORLD, &status),
MPI_Recv (buf, count, MPI_INT, 0, 1, MPI_COMM_WORLD, &status);

printf ("%d %d\n", myrank, count); $ mpirun –np 2 ./send 1000000


Eager vs. Rendezvous Protocol
• Eager
• Send completes without acknowledgement from destination
• MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE (check output of mpivars)
• Small messages, typically 128 KB (at least in MPICH)
• Not safe to use the send buffer if message size is small
• Rendezvous
• Requires an acknowledgement from a matching receive
• Large messages
MPI_Status
int MPI_Recv (void *buf, int count, MPI_Datatype datatype, int source,
int tag, MPI_Comm comm, MPI_Status *status)

• Source rank typedef struct _MPI_Status {


int count;
• Message tag int cancelled;
• Message length int MPI_SOURCE;
• MPI_Get_count int MPI_TAG;
int MPI_ERROR;
} MPI_Status, *PMPI_Status;

21
MPI_Get_count (status.c)

status.MPI_SOURCE
status.MPI_TAG

Output
• Rank 1 of 2 received 1000 elements 22
Timing Send/Recv (timingSend.c)

23
Timing Output

What is the
total time?
One-to-many Sends

25
Many-to-one Sends
a send
operation must
specify a unique
receiver

• MPI_Send (Blocking send)


• MPI_Recv (Blocking receive)
27
MPI_ANY_*
int MPI_Recv (void *buf, int count, MPI_Datatype datatype, int source,
int tag, MPI_Comm comm, MPI_Status *status)

• MPI_ANY_SOURCE
• Receiver may specify wildcard value for source

• MPI_ANY_TAG
• Receiver may specify wildcard value for tag

28
Receive Out-of-order

No specific
Rectify the order
receive buffer
location 29
Sum of Squares of N numbers
Serial Parallel
for i = 1 to N for i = 1 to N/P
sum += a[i] * a[i] sum += a[i] * a[i]
collate result

30
Sum of Squares
Core
Process
Memory

for i = 1 to N/P for i = 1 to N/P for i = 1 to N/P for i = 1 to N/P


sum += a[i] * a[i] sum += a[i] * a[i] sum += a[i] * a[i] sum += a[i] * a[i]

Rank = 0 Rank = 1 Rank = 2 Rank = 3


31
SIMD Parallelism

for i = N/P * rank ; i < N/P * (rank+1) ; i++


localsum += a[i] * a[i]
//Collect localsum, add up at one of the ranks
if (rank)
recv from all ranks (…)
else
send (0)

32

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy