10. Some Basics of Shell Programming

As described in chapter 4, a shell is a command language interface to the UNIX operating system. But a shell can also be used as a programming language. You might write a shell script to make a complicated sequence of commands easy to execute or even use such a script as a substitute for a program in a more conventional programming language. The Bourne shell is the one most used for shell programming and it will be described in this section. When you call a shell script from the C shell, and #!/bin/sh is the first line of the file, it is the Bourne shell that executes the script. Note carefully, then, that any shell built-in commands you use in a shell script must be those for the Bourne shell. For a description of the Bourne shell, see

     man sh 
This chapter does not try to teach you to write shell scripts. Its purpose is to give you a basic understanding of the Bourne shell's capabilities as a programming language.

Running a Shell Script

A shell script is simply a file containing shell commands that you can execute like this:
     sh filename [arg1 arg2 ... argn] 
A shell script may also be executed by name if the file containing the shell commands has read and execute permission (see chapter 5). If file ``do_it'' contains the shell commands and has such permissions, then the previous example is equivalent to:
     do_it [arg1 arg2 ... argn] 
In this case, executing a shell script works the same as executing a program. Remember that its first line should be #!/bin/sh to be sure the Bourne shell is the command interpreter that reads the script.

Simple Shell Scripts

The simplest shell script contains one or more complete commands. For example, if you wanted to know the number of files in your current directory, you could use
     ls -l | wc -l 
If you were to create a file called ``countf'' that contained this line (and with the correct read and execute permissions), you could then count the number of files simply by typing:
     countf 
Any number of commands can be included in a file to create shell scripts of any complexity. For more than simple scripts, though, it is usually necessary to use shell variables and to make use of special shell programming commands.

Shell Variables

Shell variables are used for storing and manipulating strings of characters. A shell variable name begins with a letter and can contain letters, digits, and underscores, such as
     x
     x1
     abc_xyz
Shell variables can be assigned values like this:
     x=file1
     x1=/usr/man/man1/sh.1
     abc_xyz=4759300
Notice that there are no spaces before or after the equals-sign. The value will be substituted for the shell variable name if the name is preceded by a $. For example,
     echo $x1 
would echo
     /usr/man/man1/sh.1
Several special shell variables are predefined. Some useful ones are
     $#, $*, $?, and $$.
Arguments can be passed to a shell script. These arguments can be accessed inside the script by using the shell variables $1, $2,...,$n for positional parameter 1,2,...,n. The filename of the shell script itself is $0. The number of such arguments is $#. For example, if file do_it is a shell script and it is called by giving the command
     do_it xyz
then $0 has the value do_it, $1 has the value xyz, and $# has the value 1.

$* is a variable containing all the arguments (except for $0) and is often used for passing all the arguments to another program or script.

$? is the exit status of the program most recently executed in the shell script. Its value is 0 for successful completion. This variable is useful for error handling (see section 10.6).

$$ is the process id of the executing shell and is useful for creating unique filenames. For example,

     cat $1 $2 >> tempfile.$$ 
concatenates the files passed as parameters 1 and 2, appending them to a file called tempfile.31264 (assuming the process id is 31264).

Flow Control

Most programming languages provide constructs for looping and for testing conditions to know when to stop looping. The shell provides several such flow control constructs, including if, for, and while.

if

The if command performs a conditional branch. It takes the form

     if command-list1 
     then
     command-list2
     else
     command-list3
     fi
A command-list is one or more commands. You can put more than one command on a line, but if you do so, separate them by semicolons. If the last command of command-list1 has exit status 0, then command-list2 is executed. But if the exit status is nonzero, then command-list3 is executed.

for

The for command provides a looping construct of the form

     for shell-variable in word-list
     do command-list
     done
The shell variable is set to the first word in word-list and then command-list is executed. The shell variable is then set to the next word in word-list and the process continues until word-list is exhausted. A common use of for-loops is to perform several commands on all (or a subset) of the files in your directory. For example, to print all the files in your directory, you could use
     for i in *
     do echo printing file $i
     lpr $i
     done
In this case, * would expand to a list of all filenames in your directory, i would be set to each filename in turn, and $i would then substitute the filename for i (in the echo and lpr commands).

while

The while command provides a slightly different looping construct:

     while command-list1
     do  command-list2
     done
While the exit status of the last command in command-list1 is 0, command-list2 is executed.

Test

The test command can be used to compare two integers, to test if a file exists or is readable, to determine if two strings are equal, or to test several other conditions. For example, to test whether the value of shell variable x is equal to 5, use
     test $x -eq 5 
If $x is equal to 5, test returns true.

Other useful tests include

     test -s file    (true if file exists and has a size larger than 0)
     test -w file    (true if file exists and is writable)
     test -z string  (true if the length of string is 0)
     test string1 != string2
                     (true if string1 and string2 are not identical)
The test command is often used with the flow-control constructs described above. Here is an example of test used with the if command:
     if test "$1" = ""            (or if ["$1" = ""] )
     then
     echo usage: myname  xxxx
     exit 1
     fi
This tests to see if the command line contains an argument ($1). If it does not ($1 is null), then echo prints a message.

A complete list of test operators can be found in the man page for test.

Error Handling

Each time a program is executed from within a shell script, a value is returned to indicate whether the program ran successfully or not. In most cases, a value of zero is returned on successful execution, and a nonzero number is returned if the program encountered an error. This exit status is available in the shell variable $?.

For example,

     grep $1 phonelist
     if test $? -ne 0
     then
             echo I have no phone number for $1
     fi 
will run a program (grep) and examine the exit status to determine if the program ran properly.

Traps

Some signals cause shell scripts to terminate. The most common one is the interrupt signal ^c typed while a script is running. Sometimes a shell script will need to do some cleanup, such as deleting temporary files, before exiting. The trap command can be used either to ignore signals or to catch them to perform special processing. For example, to delete all files called ``tmp.*'' before quitting when an interrupt signal is generated, use the command
     trap 'rm tmp.*; exit' 2 
The interrupt signal is signal 2, and two commands will be executed when an interrupt is received (rm tmp.* and exit). You can make a shell script continue to run after logout by having it ignore the hangup signal (signal 1). The command
     trap ' ' 1
allows shell procedures to continue after a hangup (logout) signal.

Command Substitution

A useful capability in shell programming is to assign the output from a program to a shell variable or use it as a pattern in another command. This is done by enclosing the program call between accent grave (`) characters. For example, the command
     where=`pwd` 
will assign the string describing the current working directory (the results of the pwd command) to the shell variable where. Here is a more complicated example:
     for i in `ls -t *.f`
     do f77 $i
             a.out >output
             cat $i output | lpr -P$1
             rm a.out output
     done
In this case, the shell script executes a series of commands for each file that ends with ``.f'' (all Fortran programs). The `ls -t *.f` is executed and expands into all filenames ending with ``.f'', sorted by time, most recent to oldest. Each is compiled and executed. Then the source file and output file are sent to the printer identified by the first argument ($1) passed to the shell script. Then these files are deleted.

I/O Redirection

Besides the I/O redirection already described, there are a few additional forms of redirection. Under the Bourne shell, standard input is also associated with file descriptor 0, standard output with file descriptor 1, and standard error output with file descriptor 2. So both >file and 1>file redirect standard output to file, and 2>file redirects standard error output to file.

To merge standard output (file descriptor 1) and standard error output (file descriptor 2), then redirect them to another file, use this notation:

     command >file 2>&1
Another method of redirecting input in shell scripts allows a command to read its input from the shell script itself without using temporary files. For instance, to run the editor ed to change all x's in a file to z's, you could create a temporary file of ed commands, then read it to perform those commands, and finally delete it, like this:
     echo "1,$s/x/z/g" >edtmp.$$
     echo "w"     >>edtmp.$$
     echo  "q"    >>edtmp.$$
     ed  filename  <edtmp.$$
     rm  edtmp.$$
     echo  "x's  changed  to  z's" 
The same thing can be accomplished without a temporary file by using the << symbol and a unique string, like this:
     ed filename <<%
     1,$s/x/z/g
     w
     q
     %
     echo  "x's  changed  to  z's" 
The << symbol redirects the standard input of the command to be right here in the shell script, beginning from the next line and continuing up to the line that matches the string following the << (in this case %). The terminating string must be on a line by itself. The string is arbitrary: for example, <<EOF will read up to a line that consists of the string EOF.

Debugging Shell Scripts

Two methods of tracing shell script execution are useful for debugging. The script may be called with a verbose flag (-v) or execution trace flag (-x). The -v flag causes each line of the shell script to be echoed as it is read but after all substitutions are performed. The -x flag causes each line to be echoed just before it is executed. For example:
     sh -v do_it
     sh -x do_it 
To turn on both flags, use -vx.

Go back to table of contents