cs110 Win2122 Lecture 8
cs110 Win2122 Lecture 8
Communication, Part 1
CS110: Principles of Computer Systems
Winter 2021-2022
Stanford University
Instructors: Nick Troccoli and Jerry Cain
Illustration courtesy of Roz Cyrus.
2
Learning About Processes
Creating
Inter-process
processes and
communication Signals Race Conditions
running other
and Pipes
programs
4
Lecture Plan
Review: our first shell
Running in the background
Introducing Pipes
What are pipes?
Pipes between processes
5
Lecture Plan
Review: our first shell
Running in the background
Introducing Pipes
What are pipes?
Pipes between processes
6
fork()
A system call that creates a new child process
The "parent" is the process that creates the other "child" process
From then on, both processes are running the code after the fork
The child process is identical to the parent, except:
it has a new Process ID (PID)
for the parent, fork() returns the PID of the child; for the child, fork() returns 0
fork() is called once, but returns twice
1 pid_t pidOrZero = fork();
2 // both parent and child run code here onwards
3 printf("This is printed by two processes.\n");
7
waitpid()
A function that a parent can call to wait for its child to exit:
pid_t waitpid(pid_t pid, int *status, int options);
pid: the PID of the child to wait on, or -1 to wait on any of our children
status: where to put info about the child's termination (or NULL)
options: optional flags to customize behavior (always 0 for now)
8
execvp()
execvp is a function that lets us run another program in the current process.
int execvp(const char *path, char *argv[]);
It runs the executable at the specified path, completely cannibalizing the current process.
If successful, execvp never returns in the calling process
If unsuccessful, execvp returns -1
execvp has many variants (execle, execlp, and so forth. Type man execvp for
more). We rely on execvp in CS110. 9
Revisiting mysystem
mysystem is our own version of the built-in function system.
It takes in a terminal command (e.g. "ls -l /usr/class/cs110"), executes it in a separate
process, and returns when that process is finished.
We can use fork to create the child process
We can use execvp in that child process to execute the terminal command
We can use waitpid in the parent process to wait for the child to terminate
10
Revisiting first-shell
1 int main(int argc, char *argv[]) {
2 char command[kMaxLineLength];
3 while (true) {
4 printf("> ");
5 fgets(command, sizeof(command), stdin);
6
7 // If the user entered Ctl-d, stop
8 if (feof(stdin)) {
9 break;
10 }
11
12 // Remove the \n that fgets puts at the end
13 command[strlen(command) - 1] = '\0';
14
15 int commandReturnCode = mysystem(command);
16 printf("return code = %d\n", commandReturnCode);
17 }
18
19 printf("\n");
20 return 0;
21 }
first-shell-soln.c
12
first-shell Takeaways
A shell is a program that repeats: read command from the user, execute that command
In order to execute a program and continue running the shell afterwards, we fork off
another process and run the program in that process
We rely on fork, execvp, and waitpid to do this!
Real shells have more advanced functionality that we will add going forward.
For your fourth assignment, you'll build on this with your own shell, stsh ("Stanford
shell") with much of the functionality of real Unix shells.
13
Lecture Plan
Review: our first shell
Running in the background
Introducing Pipes
What are pipes?
Pipes between processes
14
Supporting Background Execution
Shells usually also let you run a command in the background by adding "&" at the end:
e.g. sort myfile.txt & - create a sort process and run it in the background
only difference is specifying & with command
shell immediately re-prompts the user
process doesn't know "foreground" vs. "background"; the "&" just specifies whether or
not the shell waits
first-shell-soln-bg.c
15
Supporting Background Execution
Let's make an updated version of mysystem called executeCommand.
Takes an additional parameter bool inBackground
If false, same behavior as mysystem (spawn child, execvp, wait for child)
If true, spawn child, execvp, but don't wait for child
first-shell-soln-bg.c
16
Supporting Background Execution
1 static void executeCommand(char *command, bool inBackground) {
2 pid_t pidOrZero = fork();
3 if (pidOrZero == 0) {
4 // If we are the child, execute the shell command
5 char *arguments[] = {"/bin/sh", "-c", command, NULL};
6 execvp(arguments[0], arguments);
7 // If the child gets here, there was an error
8 exitIf(true, kExecFailed, stderr, "execvp failed to invoke this: %s.\n", command);
9 }
10
11 // If we are the parent, either wait or return immediately
12 if (inBackground) {
13 printf("%d %s\n", pidOrZero, command);
14 } else {
15 waitpid(pidOrZero, NULL, 0);
16 }
17 }
first-shell-soln-bg.c
17
Supporting Background Execution
1 static void executeCommand(char *command, bool inBackground) {
2 pid_t pidOrZero = fork();
3 if (pidOrZero == 0) {
4 // If we are the child, execute the shell command
5 char *arguments[] = {"/bin/sh", "-c", command, NULL};
6 execvp(arguments[0], arguments);
7 // If the child gets here, there was an error
8 exitIf(true, kExecFailed, stderr, "execvp failed to invoke this: %s.\n", command);
9 }
10
11 // If we are the parent, either wait or return immediately
12 if (inBackground) {
13 printf("%d %s\n", pidOrZero, command);
14 } else {
15 waitpid(pidOrZero, NULL, 0);
16 }
17 }
Line 1: Now, the caller can optionally run the command in the background.
first-shell-soln-bg.c
18
Supporting Background Execution
1 static void executeCommand(char *command, bool inBackground) {
2 pid_t pidOrZero = fork();
3 if (pidOrZero == 0) {
4 // If we are the child, execute the shell command
5 char *arguments[] = {"/bin/sh", "-c", command, NULL};
6 execvp(arguments[0], arguments);
7 // If the child gets here, there was an error
8 exitIf(true, kExecFailed, stderr, "execvp failed to invoke this: %s.\n", command);
9 }
10
11 // If we are the parent, either wait or return immediately
12 if (inBackground) {
13 printf("%d %s\n", pidOrZero, command);
14 } else {
15 waitpid(pidOrZero, NULL, 0);
16 }
17 }
Lines 11-16: The parent waits on a foreground child, but not a background child.
first-shell-soln-bg.c
19
Supporting Background Execution
In main, we add two additional things:
1 int main(int argc, char *argv[]) {
2 char command[kMaxLineLength];
3 while (true) {
4 printf("> ");
5
6
fgets(command, sizeof(command), stdin); Check for the "quit" command to exit
Allow the user to add "&" at the end of a
7 // If the user entered Ctl-d, stop
8 if (feof(stdin)) {
9 break;
10
11
} command to run that command in the
12
13
// Remove the \n that fgets puts at the end
command[strlen(command) - 1] = '\0'; background
14
15
16
if (strcmp(command, "quit") == 0) break;
Note that a background child isn't reaped!
17 bool isbg = command[strlen(command) - 1] == '&';
18
19
if (isbg) {
command[strlen(command) - 1] = '\0';
This is a problem - one we'll learn how to fix
20
21
}
soon.
22 executeCommand(command, isbg);
23 }
24
25 printf("\n");
26 return 0;
27 }
first-shell-soln-bg.c
20
Lecture Plan
Review: our first shell
Running in the background
Introducing Pipes
What are pipes?
Pipes between processes
21
Is there a way that the parent and
child processes can communicate?
22
Interprocess Communication
It's useful for a parent process to communicate with its child (and vice versa)
There are two key ways we will learn to do this: pipes and signals
Pipes let two processes send and receive arbitrary data
Signals let two processes send and receive certain "signals" that indicate
something special has happened.
23
Interprocess Communication
It's useful for a parent process to communicate with its child (and vice versa)
There are two key ways we will learn to do this: pipes and signals
Pipes let two processes send and receive arbitrary data
Signals let two processes send and receive certain "signals" that indicate
something special has happened.
24
Pipes
How can we let two processes send arbitrary data back and forth?
A core Unix principle is modeling things as files. Could we use a "file"?
Idea: a file that one process could write, and another process could read?
Problem: we don't want to clutter the filesystem with actual files every time two
processes want to communicate.
Solution: have the operating system set this up for us.
It will give us two new file descriptors - one for writing, another for reading.
If someone writes data to the write FD, it can be read from the read FD.
It's not actually a physical file on disk - we are just using files as an abstraction
25
pipe()
int pipe(int fds[]);
The pipe system call populates the 2-element array fds with two file descriptors such that
everything written to fds[1]can be read from fds[0]. Returns 0 on success, or -1 on error.
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
3 int fds[2];
4 int result = pipe(fds);
5
6 // Write message to pipe (assuming here all bytes written immediately)
7 write(fds[1], kPipeMessage, strlen(kPipeMessage) + 1);
8 close(fds[1]);
9
10 // Read message from pipe Tip: you learn to read before
11 char receivedMessage[strlen(kPipeMessage) + 1];
12 read(fds[0], receivedMessage, sizeof(receivedMessage)); you learn to write (read =
13 close(fds[0]);
14 printf("Message read: %s\n", receivedMessage); fds[0], write = fds[1]).
15
16 return 0;
17 }
1 $ ./pipe-demo
2 Message read: Hello, this message is coming through a pipe.
pipe-demo.c
26
Lecture Plan
Review: fork() and execvp()
Running in the background
Introducing Pipes
What are pipes?
Pipes between processes
27
pipe()
pipe can allow processes to communicate!
The parent's file descriptor table is
replicated in the child - both have pipe
access (increasing reference counts in open
file table)
E.g. the parent can write to the "write" end
and the child can read from the "read" end
Because they're file descriptors, there's no
global name for the pipe (another process
can't "connect" to the pipe).
Each pipe is uni-directional (one end is read,
the other write)
Illustration courtesy of Roz Cyrus. 28
Let's write a program where the parent
writes something to the pipe, and the
child reads that from the pipe.
29
Illustration courtesy of Roz Cyrus. 30
Key Idea: because the pipe file
descriptors are duplicated in the child,
we need to close the 2 pipe ends in both
the parent and the child.
31
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
Here's an example program
3
4
int fds[2];
pipe(fds);
showing how pipe works
5
6
size_t bytesSent = strlen(kPipeMessage) + 1; across processes (full
7
8
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
program link at bottom).
9 // In the child, we only read from the pipe
10 close(fds[1]);
11 char buffer[bytesSent];
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]);
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 }
17
18 // In the parent, we only write to the pipe (assume everything is written)
19 close(fds[0]);
20 write(fds[1], kPipeMessage, bytesSent);
21 close(fds[1]);
22 waitpid(pidOrZero, NULL, 0);
23 return 0;
24 }
parent-child-pipe.c
32
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
3 int fds[2];
4 pipe(fds); Make a pipe just like before.
5 size_t bytesSent = strlen(kPipeMessage) + 1;
6
7 pid_t pidOrZero = fork();
8 if (pidOrZero == 0) {
9 // In the child, we only read from the pipe
10 close(fds[1]);
11 char buffer[bytesSent];
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]);
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 }
17
18 // In the parent, we only write to the pipe (assume everything is written)
19 close(fds[0]);
20 write(fds[1], kPipeMessage, bytesSent);
21 close(fds[1]);
22 waitpid(pidOrZero, NULL, 0);
23 return 0;
24 }
parent-child-pipe.c
33
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
3 int fds[2];
4 pipe(fds);
5 size_t bytesSent = strlen(kPipeMessage) + 1;
6
7 pid_t pidOrZero = fork();
8 if (pidOrZero == 0) {
9 // In the child, we only read from the pipe
10 close(fds[1]);
11 char buffer[bytesSent];
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]);
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16
17
}
The parent must close all its
18
19
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
open FDs. It never uses the
20
21
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
Read FD so we can close it
22
23
waitpid(pidOrZero, NULL, 0);
return 0;
here.
24 }
parent-child-pipe.c
34
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
3 int fds[2];
4 pipe(fds);
5 size_t bytesSent = strlen(kPipeMessage) + 1;
6
7 pid_t pidOrZero = fork();
8 if (pidOrZero == 0) {
9 // In the child, we only read from the pipe
10 close(fds[1]);
11 char buffer[bytesSent];
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]);
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 }
17
18 // In the parent, we only write to the pipe (assume everything is written)
19
20
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
Write to the Write FD to
21 close(fds[1]); send a message to the child.
22 waitpid(pidOrZero, NULL, 0);
23 return 0;
24 }
parent-child-pipe.c
35
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
3 int fds[2];
4 pipe(fds);
5 size_t bytesSent = strlen(kPipeMessage) + 1;
6
7 pid_t pidOrZero = fork();
8 if (pidOrZero == 0) {
9 // In the child, we only read from the pipe
10 close(fds[1]);
11 char buffer[bytesSent];
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]);
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 }
17
18 // In the parent, we only write to the pipe (assume everything is written)
19
20
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
We are now done with the
21
22
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
Write FD so we can close it
23
24 }
return 0; here.
parent-child-pipe.c
36
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
3 int fds[2];
4 pipe(fds);
5 size_t bytesSent = strlen(kPipeMessage) + 1;
6
7 pid_t pidOrZero = fork();
8 if (pidOrZero == 0) {
9 // In the child, we only read from the pipe
10 close(fds[1]);
11 char buffer[bytesSent];
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]);
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 }
17
18 // In the parent, we only write to the pipe (assume everything is written)
19 close(fds[0]);
20 write(fds[1], kPipeMessage, bytesSent);
21 close(fds[1]); We wait for the child to
22 waitpid(pidOrZero, NULL, 0);
23 return 0; terminate.
24 }
parent-child-pipe.c
37
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
Key Idea: when we call fork,
3
4
int fds[2];
pipe(fds);
the child gets a copy of the
5
6
size_t bytesSent = strlen(kPipeMessage) + 1; parent's file descriptor
7
8
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
table. Any open FDs in the
9
10
// In the child, we only read from the pipe
close(fds[1]);
parent at the time fork is
11 char buffer[bytesSent]; called must be closed in both
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]); the parent and the child.
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 } This duplication means the
17
18 // In the parent, we only write to the pipe (assume everything is written) child's file descriptor table
19
20
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent); entries point to the same open
21
22
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
file table entries as the parent.
23 return 0; Thus, the open file table
24 }
entries for the two pipe FDs
both have reference counts of
parent-child-pipe.c 2. 38
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
3 int fds[2];
4 pipe(fds);
5 size_t bytesSent = strlen(kPipeMessage) + 1;
6
7
8
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
The child must close all its
9
10
// In the child, we only read from the pipe
close(fds[1]);
open FDs. It never uses the
11
12
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
Write FD so we can close it
13 close(fds[0]); here.
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 }
17
18 // In the parent, we only write to the pipe (assume everything is written)
19 close(fds[0]);
20 write(fds[1], kPipeMessage, bytesSent);
21 close(fds[1]);
22 waitpid(pidOrZero, NULL, 0);
23 return 0;
24 }
parent-child-pipe.c
39
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
Read from the Read FD to
3
4
int fds[2];
pipe(fds);
read the message from the
5
6
size_t bytesSent = strlen(kPipeMessage) + 1; parent.
7 pid_t pidOrZero = fork();
8
9
if (pidOrZero == 0) {
// In the child, we only read from the pipe Key Idea: read() blocks until
10
11
close(fds[1]);
char buffer[bytesSent]; the bytes are available or
12
13
read(fds[0], buffer, sizeof(buffer));
close(fds[0]); there is no more to read
14
15
printf("Message from parent: %s\n", buffer);
return 0;
(e.g. end of file or pipe write
16
17
}
end closed). If the parent
18
19
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
hasn't written yet, the
20
21
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
child's call to read() will
22
23
waitpid(pidOrZero, NULL, 0);
return 0;
wait.
24 }
parent-child-pipe.c
40
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
3 int fds[2];
4 pipe(fds);
5 size_t bytesSent = strlen(kPipeMessage) + 1;
6
7 pid_t pidOrZero = fork();
8 if (pidOrZero == 0) {
9 // In the child, we only read from the pipe
10 close(fds[1]);
11 char buffer[bytesSent]; We are now done with the
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]); Read FD so we can close it
14 printf("Message from parent: %s\n", buffer);
15 return 0; here. Also print the
received message.
16 }
17
18 // In the parent, we only write to the pipe (assume everything is written)
19 close(fds[0]);
20 write(fds[1], kPipeMessage, bytesSent);
21 close(fds[1]);
22 waitpid(pidOrZero, NULL, 0);
23 return 0;
24 }
parent-child-pipe.c
41
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
Key Idea: the child gets a
3
4
int fds[2];
pipe(fds);
copy of the parent's file
5
6
size_t bytesSent = strlen(kPipeMessage) + 1; descriptor table. Any open
7
8
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
FDs in the parent at the
9
10
// In the child, we only read from the pipe
close(fds[1]);
time fork is called must be
11 char buffer[bytesSent]; closed in both the parent and
12 read(fds[0], buffer, sizeof(buffer));
13 close(fds[0]); the child.
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 } Here, right before the
17
18 // In the parent, we only write to the pipe (assume everything is written) fork call, the parent has 2
open file descriptors
19 close(fds[0]);
20 write(fds[1], kPipeMessage, bytesSent);
parent-child-pipe.c
42
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
Key Idea: the child gets a
3
4
int fds[2];
pipe(fds);
copy of the parent's file
5
6
size_t bytesSent = strlen(kPipeMessage) + 1; descriptor table. Any open
7
8
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
FDs in the parent at the
9
10
// In the child, we only read from the pipe
close(fds[1]);
time fork is called must be
11
12
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
closed in both the parent and
13 close(fds[0]); the child.
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 } Therefore, when the child is
17
18 // In the parent, we only write to the pipe (assume everything is written) spawned, it also has the
same 2 open file descriptors
19 close(fds[0]);
20 write(fds[1], kPipeMessage, bytesSent);
21
22
close(fds[1]);
waitpid(pidOrZero, NULL, 0); (besides 0-2): the pipe Read
23
24 }
return 0;
FD and Write FD.
parent-child-pipe.c
43
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
Key Idea: the child gets a
3
4
int fds[2];
pipe(fds);
copy of the parent's file
5
6
size_t bytesSent = strlen(kPipeMessage) + 1; descriptor table. Any open
7
8
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
FDs in the parent at the
9
10
// In the child, we only read from the pipe
close(fds[1]);
time fork is called must be
11
12
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
closed in both the parent and
13 close(fds[0]); the child.
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 }
17
18
19
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
We should close FDs when
20
21
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
we are done with them. The
22
23
waitpid(pidOrZero, NULL, 0);
return 0;
parent closes them here.
24 }
parent-child-pipe.c
44
Parent-Child Communication
1 static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
2 int main(int argc, char *argv[]) {
Key Idea: the child gets a
3
4
int fds[2];
pipe(fds);
copy of the parent's file
5
6
size_t bytesSent = strlen(kPipeMessage) + 1; descriptor table. Any open
7
8
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
FDs in the parent at the
9
10
// In the child, we only read from the pipe
close(fds[1]);
time fork is called must be
11
12
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
closed in both the parent and
13 close(fds[0]); the child.
14 printf("Message from parent: %s\n", buffer);
15 return 0;
16 }
17
18
19
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
We should close FDs when
20
21
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
we are done with them. The
22
23
waitpid(pidOrZero, NULL, 0);
return 0;
child closes them here.
24 }
parent-child-pipe.c
45
continued...
https://cplayground.com/?p=eagle-fish-mouse
49
Pipes
This method of communication between processes relies on the fact that file descriptors are
duplicated when forking.
each process has its own copy of both file descriptors for the pipe
both processes could read or write to the pipe if they wanted.
each process must therefore close both file descriptors for the pipe when finished
This is the core idea behind how a shell can support piping between processes
(e.g. cat file.txt | uniq | sort). Let's see how this works in a shell.
50
Lecture Recap
Review: our first shell
Running in the background
Introducing Pipes
What are pipes?
Pipes between processes
51