System Programming

(ELEC462)
A Programmable Shell:
Shell Variables and the Environment

Dukyun Nam
HPC Lab@KNU
Contents
● Introduction
● Shell Programming
● smsh1 - Command-line Parsing
● Control Flow in the Shell
● Shell Variables: Local and Global
● The Environment: Personalized Settings
● Summary

2
Introduction
● Ideas and Skills
○ A Unix/Linux shell is a programming language
○ What is a shell script? How does a shell process a script?
○ How do shell control structures work? exit(0) = success
○ Shell variables: why and how
○ What is the environment? How does it work?
● System Calls and Functions
○ exit, getenv
● Commands
○ env

3
 Recap: The Shell
● A shell is a special-purpose program
○ Designed to read commands typed by a user and execute appropriate
programs in response to those commands
○ Such a program is sometimes known as a command interpreter
○ The term login shell is used to denote the process
■ that is created to run a shell when the user first logs in
○ Many different shells exist, and different users on the same computer can
simultaneously use different shells

* M. Kerrisk, The Linux Programming Interface, No Starch Press, October 2010. 4

Recap: The Shell (cont.)
● Bourne shell (sh)
○ This is the oldest of the widely used shells, and was written by Steve Bourne
○ It was the standard shell for Seventh Edition UNIX
○ The Bourne shell contains many of the features familiar in all shells: I/O redirection,
pipelines, filename generation, variables, manipulation of environment variables,
command substitution, background command execution, and functions
● C shell (csh)
○ This shell was written by Bill Joy at the University of California at Berkeley
○ The name derives from the resemblance of many of the flow-control constructs of this
shell to those of the C programming language
○ The C shell provided several useful interactive features unavailable in the Bourne shell,
including command history, command-line editing, job control, and aliases
○ The C shell was not backward compatible with the Bourne shell
5
Recap: The Shell (cont.)
● Korn shell (ksh)
○ This shell was written as the successor to the Bourne shell by David Korn at
AT&T Bell Laboratories
○ While maintaining backward compatibility with the Bourne shell, it also
incorporated interactive features similar to those provided by the C shell
● Bourne again shell (bash)
○ This shell is the GNU project’s reimplementation of the Bourne shell
○ It supplies interactive features similar to those available in the C and Korn
shells
○ The principal authors of bash are Brian Fox and Chet Ramey
○ Bash is probably the most widely used shell on Linux
6
Shell Programming
● A Linux shell
○ Runs programs AND is a programming language
■ Last time, we saw a shell runs a program
■ This time, we look at shell programming
○ Works as interpreter for a programming language
■ Interprets commands from the keyboard
■ Interprets sequences of commands stored in shell scripts

● General remarks
○ A complex task can be solved by combining several separate programs
○ The shell provides a language to control execution and communication of programs
○ The result is a programming environment
7
Shell Scripts
● Shell script: “A batch of commands”
○ A file that contains “a batch of commands”
■ “Running a script” means “executing each command in sequence” in that file
○ Can be used to perform several commands with a single request
● Example
○ The first two lines are comments
○ The shell executes the commands one by one
■ until end of file or until the shell finds an exit command

8
Shell Scripts (cont.)
● Running a shell script
○ 1) Run a shell script by passing its name as an argument to the shell

○ 2) Set the executable attribute of the file

9
Programming Features of sh:
Variables, I/O, and If..Then
● Shell scripts are REAL programs!
● Shebang = Sharp (#) + Bang (!)
○ #! interpreter [optional-arg]
■ Interpreter is generally an absolute path
to an executable program
■ The optional argument is a string
representing a single argument
○ e.g., #!/bin/sh
■ Execute the file using the Bourne shell

10
Programming Features of sh (cont.)
● Variables: e.g., BOOK, NAME
○ Not needed to be uppercase
○ $: used for retrieving the value
stored in a variable
● User input
○ read: a command telling the shell to read strings
from standard input
■ Makes scripts interactive (cf. scanf) and also gets values
from files or pipes

11
Programming Features of sh (cont.)
● Control
○ Manages program flow
■ e.g., if .. then.. else .. fi,
or while, case, and for

● Environment
○ Environment variables: allowing users
to record “personalized settings” affecting variable programs
■ HOME: contains a path to your home directory
■ PATH: contains paths that a user registers for convenience to run user-defined and
system programs
12
Command-Line Parsing: Writing smsh1
● Program Logic of smsh1

< A shell with signals, exit, and parsing >

○ The modified shell can accept the following:

find /home –name core –mtime +3 –print
■ “Find the files modified 3 days ago and print their filenames if ‘core’ is contained in the filenames”

13
[Remind] Parsing Strings
● Parsing
○ Dividing a string into tokens based on the given delimiters
● Token
○ One piece of information, a “word”
● Delimiter (구획 문자)
○ One (or more) characters used to separate tokens

● Example in Java
○ There are seven tokens: the, music, made, it, hard, to, concentrate

String phrase = "the music made it hard to

concentrate";
String delims = "[ ]+";
String[] tokens = phrase.split(delims);
14
[Remind] Parsing and Parser
● Parsing
○ The process of analyzing a string of symbols
○ Comes from Latin pars, meaning part
● Parser
○ A software component that takes input data and
builds a data structure
○ Lexical analysis (어휘 분석)
○ Syntactic analysis (구문 분석)

< Flow of data in a typical parser >

15
smsh1: Command-Line Parsing Support
● Three functions in the main function
○ next_cmd
■ Reads the next command from an input stream
■ It calls malloc to accept command lines of
any length
○ splitline
■ Splits a string into an array of words and returns that array
■ It calls malloc to accept command lines with any number of arguments
○ execute
■ Uses fork, execvp, and wait to run the command
■ This returns the termination status of the command
16
smsh1: Command-Line Parsing Support (cont.)
● $ cc -o smsh1 smsh1.c splitline.c execute.c
● smsh.h

17
smsh1: Command-Line Parsing Support (cont.)
● smsh.c

18
smsh1: Command-Line Parsing Support (cont.)
● execute.c

19
smsh1: Command-Line Parsing Support (cont.)
● splitline.c

20
smsh1: Command-Line Parsing Support (cont.)
● splitline.c (cont.)

21
smsh1: Command-Line Parsing Support (cont.)
● splitline.c (cont.)

22
smsh1: Command-Line Parsing Support (cont.)
● Execution
○ ‘ps -f’ is a child of smsh1, which is a child of bash

23
Notes on smsh1 – Need of Additional
Conveniences (Not yet supported in smsh1)
● Multiple commands on a line
○ The regular shell allows the user to separate commands with semicolons, allowing
the user to type several commands on one line
■ ls demodir; ps –f; date
● Background processing
○ The regular shell allows a user to run a process in the background (in non-blocking
mode) by ending the command with an ‘&’, as in
■ “Running in a process in the background”: you start it, the prompt returns at once, and the
process continues to run while you use the shell for other commands
■ & (ampersand): the and sign
● An exit command
○ The regular shell allows the user to type exit to quit from the shell
24
Control Flow (if..then) in the Shell
● What if does
○ The shell provides an if control structure
■ “We plan to back up our disk every Friday.”

■ The grep program calls exit(0) to indicate success if it would find “Fri”
● An exit value of 0 is signified for success
■ An else block can be added to the script. It’s like the then block.
● Check its syntax

25
Control Flow (if..then) in the Shell (cont.)

● How if works
○ (a) The shell runs the command that follows the word if
○ (b) The shell checks the exit status of the command
○ (c) An exit status of 0 means success, nonzero means failure
○ (d) The shell executes commands after the then line if success
○ (e) The shell executes commands after the else line if failure
○ (f) The keyword fi marks the end of the if block
26
The Program Logic of smsh2
● Adding a new layer, or process
● Adds the if syntax and thus changes the logic of smsh1

< Adding flow control commands to smsh >

27
What process Does and How It works
● It “manages the control flow” of a script by watching for keywords like
if, then, and fi, by calling fork and exec only when appropriate
● It views the script as a sequence of different regions
○ The want_then block
○ The then block
○ The else block
○ The neural block (outside the if structure)

< A script consists of different regions >

28
smsh2: Adding the Control Structure
● smsh2.c
○ Based on smsh1.c
has only one change

29
smsh2: Adding the Control Structure (cont.)
● process.c

30
smsh2: Adding the Control Structure (cont.)
● controlflow.c

31
smsh2: Adding the Control Structure (cont.)
● controlflow.c (cont.)

32
smsh2: Adding the Control Structure (cont.)
● Execution

33
Shell Variables: Local and Global
● Using Shell Variables

34
Shell Variables: Local and Global (cont.)
● Two types of variables
○ Local variables: works only for a user and within its current terminal
○ Environment (or global) variables
■ Their values are accessible to “all child processes” of the shell
○ e.g., set | more

35
Storage System for Variables
● How can the shell remember these variables?
○ The shell needs a “place” to store these names and values.
○ This storage system must distinguish local variables from global ones
● A possible model for the storage system

36
Storage System for Variables (cont.)
● Interface (function)
○ VLstore(char *var, char *val) adds/updates var=val
○ VLlookup(char *var) retrieves value for var
○ VLlist lists table to stdout
● Data structure for a table
○ Could be a linked list, a hash table, a tree, but an array of structs for now!

< A storage system for shell variables >

37
Adding Variable Commands: Built-ins
● We need to add the assign, list, and retrieve commands to our shell;
hopefully, we can do the following: (not yet implemented)

○ set: a “command” to our shell, not a program the shell runs

■ That is, set is different than a regular command like ‘ls’
○ To distinguish set from commands that the shell runs with exec,
■ set should be treated as built-in commands
38
Adding Variable Commands: Built-ins (cont.)
● Updated flow: smsh3

< Adding built-in commands to smsh >

39
smsh3: Adding Built-in Commands
● builtin.c

40
smsh3: Adding Built-in Commands (cont.)
● builtin.c (cont.)

41
smsh3: Adding Built-in Commands (cont.)
● varlib.h

42
smsh3: Adding Built-in Commands (cont.)
● varlib.c

43
smsh3: Adding Built-in Commands (cont.)
● varlib.c (cont.)

44
smsh3: Adding Built-in Commands (cont.)
● varlib.c (cont.)

45
smsh3: Adding Built-in Commands (cont.)
● varlib.c (cont.)

46
smsh3: Adding Built-in Commands (cont.)
● varlib.c (cont.)

47
smsh3: Adding Built-in Commands (cont.)
● process2.c

48
smsh3: Adding Built-in Commands (cont.)
● Execution
○ Not interpreted: variable substitution needed

49
Environment Variables
● Unix/Linux lets users store preferences in a set of variables called
the environment
○ Each user has a unique home directory, username, file for incoming mail,
the terminal type (pterm, xterm…), and favorite editor.
■ pterm, xterm: a type of terminal emulator for X window system
● The X Window System (X11, or simply X) is a windowing system for bitmap displays,
common on Unix-like operating systems. (source: Wiki)
● X provides the basic framework for a GUI environment: drawing and moving windows on
the display device and interacting with a mouse and keyboard. (source: Wiki)
○ Many customized settings are kept in “environment variables”.
■ The settings can be referenced when scripts utilizing the settings are run for
convenience.
50
1) Using the Environment
● Listing your environment
○ Shows all the settings in the environment env
● Updating the environment
○ var=value
○ export
■ a built-in command to add a new variable
○ These can be combined:
export var=value
● Reading the environment
○ getenv: C library function

51
1) Using the Environment (cont.)
● What is the environment? How it works?
○ The environment (environ) is an array of pointers to strings.

< The environment is an array of pointers to strings >

52
2) Showing the Environment
● showenv.c

53
2) Showing the Environment
● Execution

54
3) Changing the Environment
● changeenv.c: changes the environment and then runs env

55
exec Wipes Out All Data! – Copied Environment

< Strings in environ are copied by exec() >

56
smsh4: Adding Environment Handling to smsh
● Access to environment variables

< Copying values from the environment to vartab >

57
smsh4: Adding Environment Handling to smsh
(cont.)
● Changing the environment

< Copying values from vartab to a new environment >

58
smsh4: Adding Environment Handling to smsh
(cont.)
● Updated flow to smsh4

<Adding environment handling to smsh >

59
smsh4: Adding Environment Handling to smsh
(cont.)
● Changes to smsh
○ setup() in smsh4.c (based on smsh2.c)

60
smsh4: Adding Environment Handling to smsh
(cont.)
● Changes to smsh (cont.)
○ execute2.c based on execute.c

61
smsh4: Adding Environment Handling to smsh
(cont.)
● Execution
○ “Update time zone (TZ) to Pacific Standard Time (PST)”

62
Summary
● A shell runs programs, called shell scripts, which can run programs,
accept user input, use variables, and follow complex logic
● The if..then logic in the shell depends on the convention that a
program returns an exit value of 0 to indicate success
○ The shell uses wait to obtain the exit status from a program.
● The shell programming language includes variables
○ These variables store strings
○ These variables may be used in any command
○ Shell variables are “local” to the script
63
Summary (cont.)
● Every program inherits “a list of strings,” called environment, from
its calling process
○ The environment is used to
■ Define “global settings” for the session, and
■ Set “parameters” for specific programs
○ The shell allows users to “view and modify” the environment

64