Linux

JUNIA-ISEN - CPI2
2022-2023

1
M1 Scripting Linux JUNIA-ISEN

Contents
1 Introduction to Linux 1
1.1 The operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Folders, files and permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2.1 Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Kernel and shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1 Linux kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.2 The shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 The terminal and the first commands . . . . . . . . . . . . . . . . . . . . . . 7
1.4.1 Install packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.2 Process management . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.3 Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.4 Wilcards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.5 Directory management . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.6 File management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4.7 File content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4.8 Quick help on a command . . . . . . . . . . . . . . . . . . . . . . . . 9

2 First scripts with command lines 11

2.1 Script parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Create scripts with one command . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Create scripts with multiple commands . . . . . . . . . . . . . . . . . . . . . 12
2.4 Outputs and display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5 Text manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Advanced scripting 14
3.1 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 Command substitutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3 Conditions and loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.4 Parameters and options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2
M1 Scripting Linux JUNIA-ISEN

1 Introduction to Linux
1.1 The operating system
Linux systems can be confusing, there exists a variety of operating system labeled as Linux.
You most probably know Ubuntu, Debian or Fedora. They are in fact distributions. Many
different Linux distributions are available to meet just about any computing requirement
you could have. Most distributions are customized for a specific user group, such as business
users, multimedia enthusiasts, software developers, or average home users. Every distribu-
tions and therefore every Linux system are composed of four main parts:
• The Linux kernel
• The GNU utilities
• A graphical desktop environment
• Application software
Every distributions are different flavors of these four elements.
Each of these parts has a specific job in the Linux system.
The core of the Linux system is the kernel. The kernel controls all the hardware and soft-
ware on the computer system, allocating hardware when necessary and executing software
when required.
Besides having a kernel to control hardware devices, a computer operating system needs
utilities to perform standard functions, such as controlling files and programs. The GNU
organization (GNU stands for GNU’s Not Unix) developed a complete set of Unix utilities,
but had no kernel system to run them on.
With the popularity of Microsoft Windows, computer users expected more than the old text
interface to work with. This spurred more development in the OSS community, and the Linux
graphical desktops emerged. There exists various graphical desktop environment (DE). Few
examples are Gnome, KDE, XFCE. . . Desktop environment runs on top of a display server
and a window manager that handle how windows should appear on the screen.
Application softwares can be run on any DE, however some have also been produced to
integrate well on a particular graphical environment. They require libraries to be installed,
and sometimes those libraries are used by core applications in a specific DE. For instance
GNOME uses the GTK library.

1.2 Folders, files and permissions

Unlike some other operating systems, the Linux kernel can support different types of filesys-
tems to read and write data to and from hard drives. Besides having over a dozen filesystems
of its own, Linux can read and write to and from filesystems used by other operating sys-
tems, such as Microsoft Windows. The kernel must be compiled with support for all types
of filesystems that the system will use.

1
M1 Scripting Linux JUNIA-ISEN

Figure 1: The different filesystems supported by Linux distributions

2
M1 Scripting Linux JUNIA-ISEN

1.2.1 Permissions
User’s privileges are defined by a UID (User ID) and a GID (Group ID). Both are set in the
/etc/passwd file. Groups are defined in the /etc/group file.
The UID 0 has a special role: it is always the root account. The UIDs 1 through 99 are
traditionally reserved for special system users.
Some Linux distributions begin UIDs for non-privileged users at 100. Others, such as Red
Hat, begin them at 500, and still others, such Debian, start them at 1000. The UID 65534 is
commonly reserved for nobody, a user with no system privileges, as opposed to an ordinary.
Also, it can be convenient to reserve a block of UIDs for local users, such as 1000 through
9999, and another block for remote users (i.e., users elsewhere on the network), such as
10000 to 65534. The important thing is to decide on a scheme and adhere to it.
Unlike user 0 (the root user), group 0 does not have any special privilege at the kernel level.
Traditionally, group 0 had special privileges on many unix variants — either the right to
use su to become root (after typing the root password), or the right to become root without
typing a password. Basically, the users in group 0 were the system administrators. When
group 0 has special privileges, it is called wheel
On Linux, you need permissions to be able to read, write or execute files or directories.
The file permissions works as follows:
The umask is how you will restrict, by default, on newly created files. Think of working
the opposite way of the file permissions describe in the image 2.
Here is an example:
You start with the full permission on read, write and execute for owner, group and guest
then you substract what you want to restrict. Here we do not want guest to be able to use
the files, and we restrict write permission for group

3
M1 Scripting Linux JUNIA-ISEN

Figure 2: Linux file permission system

Figure 3: umask property to remove the permissions to a device or a file

4
M1 Scripting Linux JUNIA-ISEN

1.3 Kernel and shell

1.3.1 Linux kernel
The kernel is primarily responsible for four main functions:
• System memory management
• Software program management
• Hardware management
• Filesystem management

System memory management Not only does the kernel manage the physical memory
available on the server, but it can also create and manage virtual memory, or memory that
does not actually exist. It does this by using space on the hard disk, called the swap space.
The kernel swaps the contents of virtual memory locations back and forth from the swap
space to the actual physical memory. This allows the system to think there is more memory
available than what physically exists
The memory locations are grouped into blocks called pages. The kernel locates each page
of memory either in the physical memory or the swap space. The kernel then maintains a
table of the memory pages that indicates which pages are in physical memory and which
pages are swapped out to disk. The kernel keeps track of which memory pages are in use
and automatically copies memory pages that have not been accessed for a period of time to
the swap space area (called swapping out), even if there’s other memory available. When a
program wants to access a memory page that has been swapped out, the kernel must make
room for it in physical memory by swapping out a different memory page and swapping
in the required page from the swap space. Obviously, this process takes time and can slow
down a running process. The process of swapping out memory pages for running applications
continues for as long as the Linux system is running.

Software program management The Linux operating system calls a running program a
process. A process can run in the foreground, displaying output on a display, or it can run
in the background, behind the scenes. The kernel controls how the Linux system manages
all the processes running on the system. The kernel creates the first process, called the init
process, to start all other processes on the system. When the kernel starts, it loads the init
process into virtual memory. As the kernel starts each additional process, it gives it a unique
area in virtual memory to store the data and code that the process uses.
The Linux operating system uses an init system that utilizes run levels. A run level can be
used to direct the init process to run only certain types of processes. There are five init run
levels in the Linux operating system. At run level 1, only the basic system processes are
started, along with one console terminal process. This is called single-user mode. Single-user
mode is most often used for emergency filesystem maintenance when something is broken.
The standard init run level is 3. At this run level, most application software, such as network

5
M1 Scripting Linux JUNIA-ISEN

Figure 4: The core of the Linux system is the kernel. The kernel controls all the hardware
and software on the computer system, allocating hardware when necessary and
executing software when required.

support software, is started. Another popular run level in Linux is run level 5. This is the
run level where the system starts the graphical X Window software and allows you to log in
using a graphical desktop window.

Hardware management Any device that the Linux system must communicate with needs
driver code inserted inside the kernel code. The driver code allows the kernel to pass data
back and forth to the device, acting as a middle man between applications and the hardware.
Two methods are used for inserting device driver code in the Linux kernel:
• Drivers compiled in the kernel
• Driver modules added to the kernel
Programmers developed the concept of kernel modules to allow you to insert driver code
into a running kernel without having to recompile the kernel. Also, a kernel module could
be removed from the kernel when the device was finished being used.

Filesystem management See above (1.2) for details about filesystem management.

6
M1 Scripting Linux JUNIA-ISEN

1.3.2 The shell

The GNU/Linux shell is a special interactive utility. It provides a way for users to start
programs, manage files on the filesystem, and manage processes running on the Linux system.
The core of the shell is the command prompt. The command prompt is the interactive part
of the shell. It allows you to enter text commands, and then it interprets the commands and
executes them in the kernel.
There are several popular shells or command interpreters for the Unix / Linux systems.
sh: Bourne Shell. The oldest and installed on all Unix-based OS, but poor in functionality
compared to other shells.
bash: Bourne Again Shell. An improved shell, available by default on Linux and Mac OS X
ksh: Korn Shell. A powerful shell quite present on the proprietary Unix, but also available
in free version, compatible with bash (Korn’s shell includes Bourne’s shell)
csh: C Shell. A shell using a syntax close to the C language
tcsh: Tenex C Shell. An improved C Shell
zsh: Z Shell. Pretty new shell with the best ideas of bash, ksh and tcsh

1.4 The terminal and the first commands

Many graphical terminal emulator packages are available for the graphical environment.
Each package provides its own unique set of features and options.
To access the graphical terminal emulator, you can usually press simultaneously CTRL +
ALT + T. You can navigate to the application drawer and find Console or Terminal.

Info. Do not underestimate the use of the help command man.

1.4.1 Install packages

First we will try to install packages.
Debian and Ubuntu : apt-get
Fedora : dnf
CentOS and RedHat : yum
Arch and Manjaro : pacman

Question 1.1. Try to install a package on your system. And try to find the exe-
cutable in the folder structure.

7
M1 Scripting Linux JUNIA-ISEN

1.4.2 Process management

command : Launch a command
command & : Launch a command on background
[ctrl z] followed by bg : Put a command on background
ps : View the list of user processes
kill : Stop a process
top : View the list of all processes

Question 1.2. Run a command, put it into the background and try to put it back
into the foreground.

Question 1.3. Put a command into the background and review the list of all pro-
cesses to find it there.

1.4.3 Paths
Absolute path: /dir/sub_dir_1/sub_dir_2/file
Relative path from working directory ./dir: sub_dir_1/sub_dir_2/file (notice
there is no / at the beginning)

1.4.4 Wilcards
/ system’s root directory
. working directory
.. directory one level up
~ or $HOME login directory
~otheruser otheruser’s home directory, if otheruser permission access is granted
* all files

1.4.5 Directory management

pwd : View the name of the working directory
cd : Change working directory
ls : View the content of a directory
mkdir : Create a directory

8
M1 Scripting Linux JUNIA-ISEN

rmdir : Delete a directory

1.4.6 File management

touch : Create an empty file and change file date/time
mv : Rename or move a file
cp : Copy a file into another file
rm : Delete a file

Question 1.4. Play a little bit with the commands above to be familiar with them.

1.4.7 File content

cat : Display the content of a file or merge a list of files and display the result
more : Display the content of a file page per page
less : Display the content of a file page per page
head : Display the beginning of a file
tail : Display the end of a file
wc : Count the number of words, lines or characters
diff : Find differences between two files

Question 1.5. Create a file in a newly created folder and enter a random text. Use
the commands above to understand the differences.

1.4.8 Quick help on a command

There are too many commands to experiment with. As a matter of fact, this course will
not cover all of them. As a quick help you can use this command curl cheat.sh/command,
where you replace command, by the command you want. For instance curl cheat.sh/cat
will display this:

9
M1 Scripting Linux JUNIA-ISEN

Figure 5: Quick help using the curl command for cat.

10
M1 Scripting Linux JUNIA-ISEN

2 First scripts with command lines

Shell scripting is the set of commands to be executed by the shell. It is said to be the
combination of long and repeatable command sequences into one script so that it can be
executed as and when required. The main idea behind creating a shell script is to lessen the
load of the end-user.
A script file is just a text file with a shebang at the very beginning. A shebang is started
with #! following by the binary of the shell you want to use. Here we will use bash, then
the complete shebang is #!/bin/bash
Once your script is finished, you need to make it executable. To do so, in the terminal,
you can type chmod +x myScript.sh, if the name of your script is myScript.sh. Then to
execute your script you have to type ./myScript.sh directly in your terminal.

2.1 Script parameters

Parameters represent argument you pass to the script when calling it. For instance with 2
arguments: ./myScript.sh first second
$# : This parameter represents the total number of arguments passed to the script.
$0 : This parameter represents the script name.
$n : This parameter represents the positional arguments corresponding to a script when a
script is invoked such $1, $2. . . In the example above $1 is first, $2 is second
$* : This parameter describes the positional parameters to be distinct by space. For example,
if there are two arguments passed to the script, this parameter will describe them as $1, $2.
$$ : This parameter represents the process ID of a shell in which the execution is taking
place.
$! : This parameter represents the process number of the background that was executed
last.
$@ : This parameter is similar to the parameter $*.
$? : This parameter represents exit status of the last command that was executed. Here 0
represents success and 1 represents failure.
$_ : This parameter represents the command which is being executed previously.
$- : This parameter will print the current options flags where the set command can be used
to modify the options flags.

11
M1 Scripting Linux JUNIA-ISEN

2.2 Create scripts with one command

Question 2.1. Following the commands you used in the introduction, try to create
a script that move a file into a different folder.

Question 2.2. As a second task, create a script that takes as a parameter the PID
of a process to monitor.

2.3 Create scripts with multiple commands

Question 2.3. Complete the first script by displaying the content of file you moved.

Question 2.4. Try to think of a script that takes as a parameter the name of a
text file and a sentence to find in the text file. (Hint: use cat piped (|) into a grep to
filter text.)

2.4 Outputs and display

Before or after each command, it is possible to print messages in the terminal. These
messages are useful to see what the script is executing or they can help you debug your
script. The command is echo ’text’

Question 2.5. Complete the scripts above to print information about the script
work (for instance Moving file or Displaying file content. . . )

Question 2.6. Create a script that calculates the operation on two numbers given
as parameters. Display the value using echo

Question 2.7. Create a script that

• Display the current directory
• Print the contents of the home directory
• Create a new directory
• Move into the new directory
• Print the current working directory
• Create a new file
• Print the contents of the current directory
• Remove the directory and its contents

While it is possible to display information as you execute a script. Sometimes it is necessary

to redirect the output to a file. It is done by using ./script.sh > file.txt to create or

12
M1 Scripting Linux JUNIA-ISEN

override the content of a file. If you want to append to an already existing file, you can use
>> instead of a single >.

Question 2.8. Output the result of the calculation from the script above in a new
file. Do the same multiple times and append the new results instead.

2.5 Text manipulation

The goal of this subsection is to get familiar with text manipulation. It is useful to automate
string manipulation in a text file, or to apply filters (for instance, selecting the extension in
a filename). The sed editor is called a stream editor, as opposed to a normal interactive text
editor. In an interactive text editor, such as vim, you interactively use keyboard commands
to insert, delete, or replace text in the data. A stream editor (sed) edits a stream of data
based on a set of rules you supply ahead of time, before the editor processes the data.
Here’s the format for using the sed command: sed options script file
An example of the sed command can be the following:

echo "This is a test" | sed ’s/test/big test/’

The s is for substitute. / are delimiters. The first word is the string to change (test) and
the second is the target (big test).

Question 2.9. Try the command and understand how it is working. Review the
man page and try different option (special sequences, global, confirm. . . ).

13
M1 Scripting Linux JUNIA-ISEN

3 Advanced scripting
3.1 Variables
The Linux shells use specific environment variables by default to define the system environ-
ment. They are available for every softwares in your system. Some variables are used by the
system itself. For instance, the PATH variable declares a list of paths where software binaries
can be found. A software requiring an external application will look into the list of paths
defined in PATH to search for the specific application. If the binary is not found, the software
will return an error. An environment variable is defined by the dollar sign following by the
name of the variable. To display the content of the variable you can do echo $VARIABLE.
Try the command with your PATH. The command to create new environment variable is
export VARIABLE. The dollar sign is required only to call the variable.

Question 3.1. Create a new environment variable in your terminal and display the
content.

It is convenient to gather all user-created variable in one place. For that in your $HOME
directory, there exists multiple files to list the variables. In practice two of them are used
.bashrc and .bashenv. They are hidden files (a dot exists in front of the name). .bashrc
is used in interactive shell. Meaning that everything that is defined in the file is available
when you interact with a terminal or a terminal-based software. .bashenv is used in non-
interactive cases.
A cool feature of environment variables is that they can be used as arrays. To set multiple
values for an environment variable, just list them in parentheses, with values separated by
spaces: mytest=(one two three four five). To select an item: echo ${mytest}[2]. In
the previous example, mytest is a local variable.

Info. Any variables set but not exported by the parent shell are local variables.
Local variables are not inherited by a subshell.

3.2 Command substitutions

One of the most useful features of shell scripts is the ability to extract information from the
output of a command and assign it to a variable. After you assign the output to a variable,
you can use that value anywhere in your script. This comes in handy when processing data
in your scripts. The usual format is $().

Question 3.2. Try the command substitution by assigning the output of the date
command.

14
M1 Scripting Linux JUNIA-ISEN

Question 3.3. First, install fzf, it is a fuzzy finder that has a terminal prompt to
select one output. Create a script that find all file in a given directory, and cat the
content of the file you selected with fzf.

In principle you can as well use command substitution with any codes you might have. For
instance, a python script that outputs values in the terminal can be used using command
substitution.

Question 3.4. Try the command substitution with a python script.

3.3 Conditions and loops

Conditions are useful to control your script behaviour. It can bypass unnecessary parts if a
condition is met for instance. In bash script, you can find the traditionnal if:

if [[ conditional_statement ]]

then

command

fi

As for loops, you can use the for:

#!/bin/bash
for i in LIST
do
echo "Welcome $i times"
done

Here LIST can be integers 1 2 3 4 5, a list of files, the output of a command (using com-
mand substitution). . .

Question 3.5. By combining both condition and loops, create a script that checks
if files in a given directory have .bak files, and if not, the script creates them.

Question 3.6. Using command substitution, check the permissions of the .bak files
from the previous script and if they are not equal to 644, change them. (Hint: To
extract the permissions use the command stat -c ’%a’ FILE)

15
M1 Scripting Linux JUNIA-ISEN

Question 3.7. Be creative and find a practical script with the knowledge you have
until now!

Often, you must iterate through items stored inside a file to process data. For instance,
extract the name or values on each line and in a column. Next example, we will try to
separate lines and items stored in /etc/passwd which list users, their user and group ID. . .

#!/bin/bash
for entry in $(cat /etc/passwd); do
echo "Values in $entry"
done

Question 3.8. Try the previous script think of what is wrong with this script.
(Hint: try to cat the file in your terminal and see what’s the output)

By default, the internal field separator (the field tat separates two items) or IFS is space.
Therefore when there is a space, the script reacts as if there is a new line. Instead, we
want to separate items when there is actually a new line. It’s done via defining the variable
IFS=$’\n’.

#!/bin/bash
IFS=$’\n’
for entry in $(cat /etc/passwd); do
echo "Values in $entry"
done

Question 3.9. Add a new nested for loop and try to isolate elements of each line.

3.4 Parameters and options

The goal of this section is to have more advanced knowledge on how to handle parameters and
inputs. To iterate through input parameters, numbers are given to the position of the input
parameters. $1 is the first positional parameter, $2 is the second etc. There is another great
way to iterate through command line parameters, especially if you don’t know how many
parameters are available. You can just operate on the first parameter, shift the parameters
over, and then operate on the first parameter again. The command is called shift. Here is
an example of the shift command :

#!/bin/bash
echo
count=1
while [ -n "$1" ]
do

16
M1 Scripting Linux JUNIA-ISEN

echo "Parameter #$count = $1"

count=$[ $count + 1 ]
shift
done

Question 3.10. Try the code above and understand how it is working

Often you’ll run into situations where you’ll want to use both options and parameters for a
shell script.

#!/bin/bash
echo
while [ -n "$1" ]
do
case "$1" in
-a) echo "Found the -a option" ;;
-b) echo "Found the -b option";;
-c) echo "Found the -c option" ;;
--) shift
break ;;
*) echo "$1 is not an option";;
esac
shift
done
#echo the parameters
count=1
for param in $@
do
echo "Parameter #$count: $param"
count=$[ $count + 1 ]
done

Question 3.11. Try the code above and understand how it is working. Take good
attention to the case command as we will use it in later exercices.

Question 3.12. Modify the script above so that options require a parameter (./
script.sh -a param1 -b param2) instead of separating options and parameters with
--

17