UNIX Bourne Shell Scripting<\/p>\n
These are the workshop notes I used to use for teaching a course on Bourne shell (sh) scripting. I haven’t taught the class for some time now, but the material is all still current since the Bourne shell hasn’t changed. Often you’ll find that these notes are a bit short of description. This is because they were intended by be a lecture aid.<\/p>\n
What do you need to know to follow this course? These notes were originally written as a second class in UNIX. The first class taught how to use the basic UNIX commands (like sed, grep and find) and this class teaches how to combine these tools to accomplish bigger tasks.<\/p>\n
In addition to the material in this course you might be interested in the Korn shell (ksh) and the Bourne again shell (bash), both of which are excellent shells that enchance the original Bourne shell. These alternate shells are upwardly-compatible with the original Bourne shell, meaning that a script written for sh can run in ksh or bash. However, there are additional features in bash and ksh that are not available in the Bourne shell.<\/p>\n
The focus of this guide is to get you to understand and run some Bourne shell scripts. On several pages there are example scripts for you to run. On most of these pages there is a link you can click on (with the right mouse button) and download the script to your computer and run it.<\/p>\n
Shell scripting skills have many applications, including:<\/p>\n
Ability to automate tasks, such as
\nBackups
\nAdministration tasks
\nPeriodic operations on a database via cron
\nAny repetetive operations on files
\nIncrease your general knowledge of UNIX
\nUse of environment
\nUse of UNIX utilities
\nUse of features such as pipes and I\/O redirection
\nFor example, I recently wrote a script to make a backup of one of the subdirectories where I was developing a project. I quickly wrote a shell script that uses \/bin\/tar to create an archive of the entire subdirectory and then copy it to one of our backup systems here at SDSC and store it under a subdirectory named according to today’s date.<\/p>\n
As another example, I have some software that runs on UNIX that I distribute and people were having trouble unpacking the software and getting it running. I designed and wrote a shell script that automated the process of unpacking the software and configuring it. Now people can get and install the software without having to contact me for help, which is good for them and good for me, too!<\/p>\n
For shell script experts one of the things to consider is whether to use the Bourne shell (or ksh or bash), the C shell, or a richer scripting language like perl or python. I like all these tools and am not especially biased toward any one of them. The best thing is to use the right tool for each job. If all you need to do is run some UNIX commands over and over again, use a Bourne or C shell script. If you need a script that does a lot of arithmetic or string manipulation, then you will be better off with perl or python. If you have a Bourne shell script that runs too slowly then you might want to rewrite it in perl or python because they can be much faster.<\/p>\n
Historically, people have been biased toward the Bourne shell over the C shell because in the early days the C shell was buggy. These problems are fixed in many C shell implementations these days, especially the excellent ‘T’ C shell (tcsh), but many still prefer the Bourne shell.<\/p>\n
There are other good shells available. I don’t mean to neglect them but rather to talk about the tools I am familiar with.<\/p>\n
If you are interested also in learning about programming in the C shell I also have a comparison between features of the C shell and Bourne shell.<\/p>\n
Table of Contents:<\/p>\n
Review of a few Basic UNIX Topics (Page 1)
\nStoring Frequently Used Commands in Files: Shell Scripts (Page 6)
\nMore on Using UNIX Utilities (Page 9)
\nPerforming Search and Replace in Several Files (Page 11)
\nUsing Command-line Arguments for Flexibility (Page 14)
\nUsing Functions (Page 30)
\nMiscellaneous (Page 38)
\nTrapping Signals (Page 43)
\nUnderstanding Command Translation (Page 50)
\nWriting Advanced Loops (Page 59)
\nCreating Remote Shells (Page 67)
\nMore Miscellaneous (Page 73)
\nUsing Quotes (Page 75)<\/p>\n
Section 1: Review of a few Basic UNIX Topics <\/p>\n
Shell scripting involves chaining several UNIX commands together to accomplish a task. For example, you might run the ‘date’ command and then use today’s date as part of a file name. I’ll show you how to do this below.<\/p>\n
Some of the tools of the trade are variables, backquotes and pipes. First we’ll study these topics and also quickly review a few other UNIX topics.<\/p>\n
Variables<\/p>\n
Topics covered: storing strings in variables
\nUtilities covered: echo, expr
\nTo try the commands below start up a Bourne shell:
\n\/bin\/sh
\nA variable stores a string (try running these commands in a Bourne shell)
\nname=”John Doe”
\necho $name
\nThe quotes are required in the example above because the string contains a special character (the space)
\nA variable may store a number
\nnum=137
\nThe shell stores this as a string even though it appears to be a number
\nA few UNIX utilities will convert this string into a number to perform arithmetic
\nexpr $num + 3
\nTry defining num as ‘7m8’ and try the expr command again
\nWhat happens when num is not a valid number?
\nNow you may exit the Bourne shell with
\nexit<\/p>\n
Page 1<\/p>\n
I\/O Redirection<\/p>\n
Topics covered: specifying the input or capturing the output of a command in a file
\nUtilities covered: wc, sort
\nThe wc command counts the number of lines, words, and characters in a file
\nwc \/etc\/passwd
\nwc -l \/etc\/passwd
\nYou can save the output of wc (or any other command) with output redirection
\nwc \/etc\/passwd > wc.file
\nYou can specify the input with input redirection
\nwc < \/etc\/passwd\nMany UNIX commands allow you to specify the input file by name or by input redirection\nsort \/etc\/passwd\nsort < \/etc\/passwd\nYou can also append lines to the end of an existing file with output redirection\nwc -l \/etc\/passwd >> wc.file<\/p>\n
Page 2<\/p>\n
Backquotes<\/p>\n
Topics covered: capturing output of a command in a variable
\nUtilities covered: date
\nThe backquote character looks like the single quote or apostrophe, but slants the other way
\nIt is used to capture the output of a UNIX utility
\nA command in backquotes is executed and then replaced by the output of the command
\nExecute these commands
\ndate
\nsave_date=`date`
\necho The date is $save_date
\nNotice how echo prints the output of ‘date’, and gives the time when you defined the save_date variable
\nStore the following in a file named backquotes.sh and execute it (right click and save in a file)
\n#!\/bin\/sh
\n# Illustrates using backquotes
\n# Output of ‘date’ stored in a variable
\nToday=”`date`”
\necho Today is $Today
\nExecute the script with
\nsh backquotes.sh
\nThe example above shows you how you can write commands into a file and execute the file with a Bourne shell
\nBackquotes are very useful, but be aware that they slow down a script if you use them hundreds of times
\nYou can save the output of any command with backquotes, but be aware that the results will be reformated into one line. Try this:
\nLS=`ls -l`
\necho $LS<\/p>\n
Page 3<\/p>\n
Pipes<\/p>\n
Topics covered: using UNIX pipes
\nUtilities covered: sort, cat, head
\nPipes are used for post-processing data
\nOne UNIX command prints results to the standard output (usually the screen), and another command reads that data and processes it
\nsort \/etc\/passwd | head -5
\nNotice that this pipe can be simplified
\ncat \/etc\/passwd | head -5
\nYou could accomplish the same thing more efficiently with either of the two commands:
\nhead -5 \/etc\/passwd
\nhead -5 < \/etc\/passwd\nFor example, this command displays all the files in the current directory sorted by file size\nls -al | sort -n -r +4\nThe command ls -al writes the file size in the fifth column, which is why we skip the first four columns using +4.\nThe options -n and -r request a numeric sort (which is different than the normal alphabetic sort) in reverse order\n\nPage 4\n\n\n\nawk\n\nTopics covered: processing columnar data\nUtilities covered: awk\nThe awk utility is used for processing columns of data\nA simple example shows how to extract column 5 (the file size) from the output of ls -l\nls -l | awk '{print $5}'\nCut and paste this line into a Bourne shell and you should see a column of file sizes, one per file in your current directory.\nA more complicated example shows how to sum the file sizes and print the result at the end of the awk run\nls -al | awk '{sum = sum + $5} END {print sum}'\nIn this example you should see printed just one number, which is the sum of the file sizes in the current directory.\n\nPage 5\n\n\n\n\n\n\n\n\nSection 2: Storing Frequently Used Commands in Files: Shell Scripts \n\nShell Scripts\n\nTopics covered: storing commands in a file and executing the file\nUtilities covered: date, cal, last (shows who has logged in recently)\nStore the following in a file named simple.sh and execute it\n#!\/bin\/sh\n# Show some useful info at the start of the day\ndate\necho Good morning $USER\ncal\nlast | head -6\nShows current date, calendar, and a six of previous logins\nNotice that the commands themselves are not displayed, only the results\nTo display the commands verbatim as they run, execute with\nsh -v simple.sh\nAnother way to display the commands as they run is with -x\nsh -x simple.sh\nWhat is the difference between -v and -x? Notice that with -v you see '$USER' but with -x you see your login name\nRun the command 'echo $USER' at your terminal prompt and see that the variable $USER stores your login name\nWith -v or -x (or both) you can easily relate any error message that may appear to the command that generated it\nWhen an error occurs in a script, the script continues executing at the next command\nVerify this by changing 'cal' to 'caal' to cause an error, and then run the script again\nRun the 'caal' script with 'sh -v simple.sh' and with 'sh -x simple.sh' and verify the error message comes from cal\nOther standard variable names include: $HOME, $PATH, $PRINTER. Use echo to examine the values of these variables\n\nPage 6\n\n\n\nStoring File Names in Variables\n\nTopics covered: variables store strings such as file names, more on creating and using variables\nUtilities covered: echo, ls, wc\nA variable is a name that stores a string\nIt's often convenient to store a filename in a variable\nStore the following in a file named variables.sh and execute it\n#!\/bin\/sh\n# An example with variables\nfilename=\"\/etc\/passwd\"\necho \"Check the permissions on $filename\"\nls -l $filename\necho \"Find out how many accounts there are on this system\"\nwc -l $filename\nNow if we change the value of $filename, the change is automatically propagated throughout the entire script\n\nPage 7\n\n\n\nScripting With sed\n\nTopics covered: global search and replace, input and output redirection\nUtilities covered: sed\nHere's how you can use sed to modify the contents of a variable:\necho \"Hello Jim\" | sed -e 's\/Hello\/Bye\/'\nCopy the file nlanr.txt to your home directory and notice how the word 'vBNS' appears in it several times\nChange 'vBNS' to 'NETWORK' with\nsed -e 's\/vBNS\/NETWORK\/g' < nlanr.txt\nYou can save the modified text in a file with output redirection\nsed -e 's\/vBNS\/NETWORK\/g' < nlanr.txt > nlanr.new
\nSed can be used for many complex editing tasks, we have only scratched the surface here<\/p>\n
Page 8<\/p>\n
Section 3: More on Using UNIX Utilities <\/p>\n
Performing Arithmetic<\/p>\n
Topics covered: integer arithmetic, preceding ‘*’ with backslash to avoid file name wildcard expansion
\nUtilities covered: expr
\nArithmetic is done with expr
\nexpr 5 + 7
\nexpr 5 \\* 7
\nBackslash required in front of ‘*’ since it is a filename wildcard and would be translated by the shell into a list of file names
\nYou can save arithmetic result in a variable
\nStore the following in a file named arith.sh and execute it
\n#!\/bin\/sh
\n# Perform some arithmetic
\nx=24
\ny=4
\nResult=`expr $x \\* $y`
\necho “$x times $y is $Result”<\/p>\n
Page 9<\/p>\n
Translating Characters<\/p>\n
Topics covered: converting one character to another, translating and saving string stored in a variable
\nUtilities covered: tr
\nCopy the file sdsc.txt to your home directory
\nThe utility tr translates characters
\ntr ‘a’ ‘Z’ < sdsc.txt\nThis example shows how to translate the contents of a variable and display the result on the screen with tr\nStore the following in a file named tr1.sh and execute it\n#!\/bin\/sh\n# Translate the contents of a variable\nCat_name=\"Piewacket\"\necho $Cat_name | tr 'a' 'i'\nThis example shows how to change the contents of a variable\nStore the following in a file named tr2.sh and execute it\n#!\/bin\/sh\n# Illustrates how to change the contents of a variable with tr\nCat_name=\"Piewacket\"\necho \"Cat_name is $Cat_name\"\nCat_name=`echo $Cat_name | tr 'a' 'i'`\necho \"Cat_name has changed to $Cat_name\"\nYou can also specify ranges of characters.\nThis example converts upper case to lower case\ntr 'A-Z' 'a-z' < file\nNow you can change the value of the variable and your script has access to the new value\n\nPage 10\n\n\n\n\n\n\n\n\nSection 4: Performing Search and Replace in Several Files \n\nProcessing Multiple Files\n\nTopics covered: executing a sequence of commands on each of several files with for loops\nUtilities covered: no new utilities\nStore the following in a file named loop1.sh and execute it\n#!\/bin\/sh\n# Execute ls and wc on each of several files\n# File names listed explicitly\nfor filename in simple.sh variables.sh loop1.sh\ndo\n\techo \"Variable filename is set to $filename...\"\n\tls -l $filename\n\twc -l $filename\ndone\nThis executes the three commands echo, ls and wc for each of the three file names\nYou should see three lines of output for each file name\nfilename is a variable, set by \"for\" statement and referenced as $filename\nNow we know how to execute a series of commands on each of several files\n\nPage 11\n\n\n\nUsing File Name Wildcards in For Loops\n\nTopics covered: looping over files specified with wildcards\nUtilities covered: no new utilities\nStore the following in a file named loop2.sh and execute it\n#!\/bin\/sh\n# Execute ls and wc on each of several files\n# File names listed using file name wildcards\nfor filename in *.sh\ndo\n\techo \"Variable filename is set to $filename...\"\n\tls -l $filename\n\twc -l $filename\ndone\nYou should see three lines of output for each file name ending in '.sh'\nThe file name wildcard pattern *.sh gets replaced by the list of filenames that exist in the current directory\nFor another example with filename wildcards try this command\necho *.sh\n\nPage 12\n\n\n\nSearch and Replace in Multiple Files\n\nTopics covered: combining for loops with utilities for global search and replace in several files\nUtilities covered: mv\nSed performs global search and replace on a single file\nsed -e 's\/application\/APPLICATION\/g' sdsc.txt > sdsc.txt.new
\nThe original file sdsc.txt is unchanged
\nHow can we arrange to have the original file over-written by the new version?
\nStore the following in a file named s-and-r.sh and execute it
\n#!\/bin\/sh
\n# Perform a global search and replace on each of several files
\n# File names listed explicitly
\nfor text_file in sdsc.txt nlanr.txt
\ndo
\n\techo “Editing file $text_file”
\n\tsed -e ‘s\/application\/APPLICATION\/g’ $text_file > temp
\n\tmv -f temp $text_file
\ndone
\nFirst, sed saves new version in file ‘temp’
\nThen, use mv to overwrite original file with new version<\/p>\n
Page 13<\/p>\n
Section 5: Using Command-line Arguments for Flexibility <\/p>\n
What’s Lacking in the Scripts Above?<\/p>\n
Topics covered: looping over files specified with wildcards
\nUtilities covered: no new utilities
\nFile names are hard-coded inside the script
\nWhat if you want to run the script but with different file names?
\nTo execute for loops on different files, the user has to know how to edit the script
\nNot simple enough for general use by the masses
\nWouldn’t it be useful if we could easily specify different file names for each execution of a script?<\/p>\n
Page 14<\/p>\n
What are Command-line Arguments?<\/p>\n
Topics covered: specifying command-line arguments
\nUtilities covered: no new utilities
\nCommand-line arguments follow the name of a command
\nls -l .cshrc \/etc
\nThe command above has three command-line arguments
\n-l\t(an option that requests long directory listing)
\n.cshrc\t(a file name)
\n\/etc\t(a directory name)
\nAn example with file name wildcards:
\nwc *.sh
\nHow many command-line arguments were given to wc? It depends on how many files in the current directory match the pattern *.sh
\nUse ‘echo *.sh’ to see them
\nMost UNIX commands take command-line arguments. Your scripts may also have arguments<\/p>\n
Page 15<\/p>\n
Accessing Command-line Arguments<\/p>\n
Topics covered: accessing command-line arguments
\nUtilities covered: no new utilities
\nStore the following in a file named args1.sh
\n#!\/bin\/sh
\n# Illustrates using command-line arguments
\n# Execute with
\n#\tsh args1.sh On the Waterfront
\necho “First command-line argument is: $1”
\necho “Third argument is: $3”
\necho “Number of arguments is: $#”
\necho “The entire list of arguments is: $*”
\nExecute the script with
\nsh args1.sh -x On the Waterfront
\nWords after the script name are command-line arguments
\nArguments are usually options like -l or file names<\/p>\n
Page 16<\/p>\n
Looping Over the Command-line Arguments<\/p>\n
Topics covered: using command-line arguments in a for loop
\nUtilities covered: no new utilities
\nStore the following in a file named args2.sh and execute it
\n#!\/bin\/sh
\n# Loop over the command-line arguments
\n# Execute with
\n#\tsh args2.sh simple.sh variables.sh
\nfor filename in “$@”
\ndo
\n\techo “Examining file $filename”
\n\twc -l $filename
\ndone
\nThis script runs properly with any number of arguments, including zero
\nThe shorter form of the for statement shown below does exactly the same thing
\nfor filename
\ndo
\n…
\nDon’t use
\nfor filename in $*
\nFails if any arguments include spaces
\nAlso, don’t forget the double quotes around $@<\/p>\n
Page 17<\/p>\n
If Blocks<\/p>\n
Topics covered: testing conditions, executing commands conditionally
\nUtilities covered: test (used by if to evaluate conditions)
\nThis will be covered on the whiteboard
\nSee Chapter 8 of the book<\/p>\n
Page 18<\/p>\n
The read Command<\/p>\n
Topics covered: reading a line from the standard input
\nUtilities covered: no new utilities
\nstdin is the keyboard unless input redirection used
\nRead one line from stdin, store line in a variable
\nread variable_name
\nAsk the user if he wants to exit the script
\nStore the following in a file named read.sh and execute it
\n#!\/bin\/sh
\n# Shows how to read a line from stdin
\necho “Would you like to exit this script now?”
\nread answer
\nif [ “$answer” = y ]
\nthen
\n\techo “Exiting…”
\n\texit 0
\nfi<\/p>\n
Page 19<\/p>\n
Command Exit Status<\/p>\n
Topics covered: checking whether a command succeeds or not
\nUtilities covered: no new utilities
\nEvery command in UNIX should return an exit status
\nStatus is in range 0-255
\nOnly 0 means success
\nOther statuses indicate various types of failures
\nStatus does not print on screen, but is available thru variable $?
\nExample shows how to examine exit status of a command
\nStore the following in a file named exit-status.sh and execute it
\n#!\/bin\/sh
\n# Experiment with command exit status
\necho “The next command should fail and return a status greater than zero”
\nls \/nosuchdirectory
\necho “Status is $? from command: ls \/nosuchdirectory”
\necho “The next command should succeed and return a status equal to zero”
\nls \/tmp
\necho “Status is $? from command: ls \/tmp”
\nExample shows if block using exit status to force exit on failure
\nStore the following in a file named exit-status-test.sh and execute it
\n#!\/bin\/sh
\n# Use an if block to determine if a command succeeded
\necho “This mkdir command fails unless you are root:”
\nmkdir \/no_way
\nif [ “$?” -ne 0 ]
\nthen
\n\t# Complain and quit
\n\techo “Could not create directory \/no_way…quitting”
\n\texit 1 # Set script’s exit status to 1
\nfi
\necho “Created directory \/no_way”
\nExit status is $status in C shell<\/p>\n
Page 20<\/p>\n
Regular Expressions<\/p>\n
Topics covered: search patterns for editors, grep, sed
\nUtilities covered: no new utilities
\nZero or more characters: .*
\ngrep ‘provided.*access’ sdsc.txt
\nsed -e ‘s\/provided.*access\/provided access\/’ sdsc.txt
\nSearch for text at beginning of line
\ngrep ‘^the’ sdsc.txt
\nSearch for text at the end of line
\ngrep ‘of$’ sdsc.txt
\nAsterisk means zero or more the the preceeding character
\na* zero or more a’s
\naa* one or more a’s
\naaa* two or more a’s
\nDelete all spaces at the ends of lines
\nsed -e ‘s\/ *$\/\/’ sdsc.txt > sdsc.txt.new
\nTurn each line into a shell comment
\nsed -e ‘s\/^\/# \/’ sdsc.txt<\/p>\n
Page 21<\/p>\n
Greed and Eagerness<\/p>\n
Attributes of pattern matching
\nGreed: a regular expression will match the largest possible string
\nExecute this command and see how big a string gets replaced by an underscore
\necho ‘Big robot’ | sed -e ‘s\/i.*o\/_\/’
\nEagerness: a regular expression will find the first match if several are present in the line
\nExecute this command and see whether ‘big’ or ‘bag’ is matched by the regular expression
\necho ‘big bag’ | sed -e ‘s\/b.g\/___\/’
\nContrast with this command (notice the extra ‘g’)
\necho ‘big bag’ | sed -e ‘s\/b.g\/___\/g’
\nExplain what happens in the next example
\necho ‘black dog’ | sed -e ‘s\/a*\/_\/’
\nHint: a* matches zero or more a’s, and there are many places where zero a’s appear
\nTry the example above with the extra ‘g’
\necho ‘black dog’ | sed -e ‘s\/a*\/_\/g’<\/p>\n
Page 22<\/p>\n
Regular Expressions Versus Wildcards<\/p>\n
Topics covered: clarify double meaning of asterisk in patterns
\nUtilities covered: no new utilities
\nAsterisk used in regular expressions for editors, grep, sed
\nDifferent meaning in file name wildcards on command line and in find command and case statement (see below)
\nregexp wildcard meaning<\/p>\n
.* * zero or more characters, any type
\n. ? exactly one character, any type
\n[aCg] [aCg] exactly one character, from list: aCg
\nRegexps can be anchored to beginning\/ending of line with ^ and $
\nWildcards automatically anchored to both extremes
\nCan use wildcards un-anchored with asterisks
\nls *bub*<\/p>\n
Page 23<\/p>\n
Getting Clever With Regular Expressions<\/p>\n
Topics covered: manipulating text matched by a pattern
\nUtilities covered: no new utilities
\nCopy the file animals.txt to your home directory
\nTry this sed command, which changes the first line of animals.txt
\nsed -e “s\/big \\(.*\\) dog\/small \\1 cat\/” animals.txt
\nBracketing part of a pattern with \\( and \\) labels that part as \\1
\nBracketing additional parts of a pattern creates labels \\2, \\3, …
\nThis sed command reverses the order of two words describing the rabbit
\nsed -e “s\/Flopsy is a big \\(.*\\) \\(.*\\) rabbit\/A big \\2 \\1 rabbit\/” < animals.txt\n\nPage 24\n\n\n\nThe case Statement\n\nTopics covered: choosing which block of commands to execute based on value of a string\nUtilities covered: no new utilities\nThe next example shows how to use a case statement to handle several contingencies\nThe user is expected to type one of three words\nA different action is taken for each choice\nStore the following in a file named case1.sh and execute it\n#!\/bin\/sh\n# An example with the case statement\n# Reads a command from the user and processes it\necho \"Enter your command (who, list, or cal)\"\nread command\ncase \"$command\" in\n\twho)\n\t\techo \"Running who...\"\n\t\twho\n\t\t;;\n\tlist)\n\t\techo \"Running ls...\"\n\t\tls\n\t\t;;\n\tcal)\n\t\techo \"Running cal...\"\n\t\tcal\n\t\t;;\n\t*)\n\t\techo \"Bad command, your choices are: who, list, or cal\"\n\t\t;;\nesac\nexit 0\nThe last case above is the default, which corresponds to an unrecognized entry\nThe next example uses the first command-line arg instead of asking the user to type a command\nStore the following in a file named case2.sh and execute it\n#!\/bin\/sh\n# An example with the case statement\n# Reads a command from the user and processes it\n# Execute with one of\n#\tsh case2.sh who\n#\tsh case2.sh ls\n#\tsh case2.sh cal\necho \"Took command from the argument list: '$1'\"\ncase \"$1\" in\n\twho)\n\t\techo \"Running who...\"\n\t\twho\n\t\t;;\n\tlist)\n\t\techo \"Running ls...\"\n\t\tls\n\t\t;;\n\tcal)\n\t\techo \"Running cal...\"\n\t\tcal\n\t\t;;\n\t*)\n\t\techo \"Bad command, your choices are: who, list, or cal\"\n\t\t;;\nesac\nThe patterns in the case statement may use file name wildcards\n\nPage 25\n\n\n\nThe while Statement\n\nTopics covered: executing a series of commands as long as some condition is true\nUtilities covered: no new utilities\nThe example below loops over two statements as long as the variable i is less than or equal to ten\nStore the following in a file named while1.sh and execute it\n#!\/bin\/sh\n# Illustrates implementing a counter with a while loop\n# Notice how we increment the counter with expr in backquotes\ni=\"1\"\nwhile [ $i -le 10 ]\ndo\n\techo \"i is $i\"\n\ti=`expr $i + 1`\ndone\n\nPage 26\n\n\n\nExample With a while Loop\n\nTopics covered: Using a while loop to read and process a file\nUtilities covered: no new utilities\nCopy the file while2.data to your home directory\nThe example below uses a while loop to read an entire file\nThe while loop exits when the read command returns false exit status (end of file)\nStore the following in a file named while2.sh and execute it\n#!\/bin\/sh\n# Illustrates use of a while loop to read a file\ncat while2.data | \\\nwhile read line\ndo\n\techo \"Found line: $line\"\ndone\n\nThe entire while loop reads its stdin from the pipe\nEach read command reads another line from the file coming from cat\nThe entire while loop runs in a subshell because of the pipe\nVariable values set inside while loop not available after while loop\n\nPage 27\n\n\n\nInterpreting Options With getopts Command\n\nTopics covered: Understand how getopts command works\nUtilities covered: getopts\ngetopts is a standard UNIX utility used for our class in scripts getopts1.sh and getopts2.sh\nIts purpose is to help process command-line options (such as -h) inside a script\nIt handles stacked options (such as -la) and options with arguments (such as -P used as -Pprinter-name in lpr command)\nThis example will help you understand how getopts interprets options\nStore the following in a file named getopts1.sh and execute it\n#!\/bin\/sh\n\n# Execute with\n#\n#\tsh getopts1.sh -h -Pxerox file1 file2\n#\n# and notice how the information on all the options is displayed\n#\n# The string 'P:h' says that the option -P is a complex option\n# requiring an argument, and that h is a simple option not requiring\n# an argument.\n#\n\n# Experiment with getopts command\nwhile getopts 'P:h' OPT_LETTER\ndo\n\techo \"getopts has set variable OPT_LETTER to '$OPT_LETTER'\"\n\techo \"\tOPTARG is '$OPTARG'\"\ndone\n\nused_up=`expr $OPTIND - 1`\n\necho \"Shifting away the first \\$OPTIND-1 = $used_up command-line arguments\"\n\nshift $used_up\n\necho \"Remaining command-line arguments are '$*'\"\n\nLook over the script\ngetopts looks for command-line options\nFor each option found, it sets three variables: OPT_LETTER, OPTARG, OPTIND\nOPT_LETTER is the letter, such as 'h' for option -h\nOPTARG is the argument to the option, such as -Pjunky has argument 'junky'\nOPTIND is a counter that determines how many of the command-line arguments were used up by getopts (see the shift command in the script)\nExecute it several times with\nsh getopts1.sh -h -Pjunky\nsh getopts1.sh -hPjunky\nsh getopts1.sh -h -Pjunky \/etc \/tmp\nNotice how it interprets -h and gives you 'h' in variable OPT_LETTER\nNow you can easily implement some operation when -h is used\nNotice how the second execution uses stacked options\nNotice how the third execution examines the rest of the command-line after the options (these are usually file or directory names)\n\nPage 28\n\n\n\nExample With getopts\n\nTopics covered: interpreting options in a script\nUtilities covered: getopts\nThe second example shows how to use if blocks to take action for each option\nStore the following in a file named getopts2.sh and execute it\n#!\/bin\/sh\n#\n# Usage:\n#\n#\tgetopts2.sh [-P string] [-h] [file1 file2 ...]\n#\n# Example runs:\n#\n#\tgetopts2.sh -h -Pxerox file1 file2\n#\tgetopts2.sh -hPxerox file1 file2\n#\n# Will print out the options and file names given\n#\n\n# Initialize our variables so we don't inherit values\n# from the environment\nopt_P=''\nopt_h=''\n\n# Parse the command-line options\nwhile getopts 'P:h' option\ndo\n\tcase \"$option\" in\n\t\"P\")\topt_P=\"$OPTARG\"\n\t\t;;\n\t\"h\")\topt_h=\"1\"\n\t\t;;\n\t?)\techo \"getopts2.sh: Bad option specified...quitting\"\n\t\texit 1\n\t\t;;\n\tesac\ndone\n\nshift `expr $OPTIND - 1`\n\nif [ \"$opt_P\" != \"\" ]\nthen\n\techo \"Option P used with argument '$opt_P'\"\nfi\n\nif [ \"$opt_h\" != \"\" ]\nthen\n\techo \"Option h used\"\nfi\n\nif [ \"$*\" != \"\" ]\nthen\n\techo \"Remaining command-line:\"\n\tfor arg in \"$@\"\n\tdo\n\t\techo \"\t$arg\"\n\tdone\nfi\n\nExecute it several times with\nsh getopts2.sh -h -Pjunky\nsh getopts2.sh -hPjunky\nsh getopts2.sh -h -Pjunky \/etc \/tmp\nCan also implement actions inside case statement if desired\n\nPage 29\n\n\n\n\n\n\n\n\nSection 6: Using Functions \n\nFunctions\n\nSequence of statements that can be called anywhere in script\nUsed for\nGood organization\nCreate re-usable sequences of commands\n\nPage 30\n\n\n\nDefine a Function\n\nDefine a function\necho_it () {\n echo \"In function echo_it\"\n}\nUse it like any other command\necho_it\nPut these four lines in a script and execute it\n\nPage 31\n\n\n\nFunction Arguments\n\nFunctions can have command-line arguments\necho_it () {\n echo \"Argument 1 is $1\"\n echo \"Argument 2 is $2\"\n}\necho_it arg1 arg2\nWhen you execute the script above, you should see\nArgument 1 is arg1\nArgument 2 is arg2\nCreate a script 'difference.sh' with the following lines:\n#!\/bin\/sh\necho_it () {\n\techo Function argument 1 is $1\n}\necho Script argument 1 is $1\necho_it Barney\nExecute this script using\nsh difference.sh Fred\nNotice that '$1' is echoed twice with different values\nThe function has separate command-line arguments from the script's\n\nPage 32\n\n\n\nExample With Functions\n\nUse functions to organize script\nread_inputs () { ... }\ncompute_results () { ... }\nprint_results () { ... }\nMain program very readable\nread_inputs\ncompute_results\nprint_results\n\nPage 33\n\n\n\nFunctions in Pipes\n\nCan use a function in a pipe\nls_sorter () {\n sort -n +4\n}\nls -al | ls_sorter\nFunction in pipe executed in new shell\nNew variables forgotten when function exits\n\nPage 34\n\n\n\nInherited Variables\n\nVariables defined before calling script available to script\nfunc_y () {\n echo \"A is $A\"\n return 7\n}\nA='bub'\nfunc_y\nif [ $? -eq 7 ] ; then ...\nTry it: is a variable defined inside a function available to the main program?\n\nPage 35\n\n\n\nFunctions -vs- Scripts\n\nFunctions are like separate scripts\nBoth functions and scripts can:\nUse command-line arguments\necho First arg is $1\nOperate in pipes\necho \"test string\" | ls_sorter\nReturn exit status\nfunc_y arg1 arg2\nif [ $? -ne 0 ] ...\n\nPage 36\n\n\n\nLibraries of Functions\n\nCommon to store definitions of favorite functions in a file\nThen execute file with\n. file\nPeriod command executes file in current shell\nCompare to C shell's source command\n\nPage 37\n\n\n\n\n\n\n\n\nSection 7: Miscellaneous \n\nHere Files\n\nData contained within script\ncat << END\nThis script backs up the directory\nnamed as the first command-line argument,\nwhich in your case in $1.\nEND\nTerminator string must begin in column one\nVariables and backquotes translated in data\nTurn off translation with \\END\n\nPage 38\n\n\n\nExample With Here File\n\nSend e-mail to each of several users\nfor name in login1 login2 login3\ndo\n mailx -s 'hi there' $name << EOF\n Hi $name, meet me at the water\n fountain\nEOF\ndone\nUse <<- to remove initial tabs automatically\n\nPage 39\n\n\n\nSet: Shell Options\n\nCan change Bourne shell's options at runtime\nUse set command inside script\nset -v\nset +v\nset -xv\nToggle verbose mode on and off to reduce amount of debugging output\n\nPage 40\n\n\n\nSet: Split a Line\n\nCan change Bourne shell's options\nset -- word1 word2\necho $1, $2\n word1, word2\nDouble dash important!\nWord1 may begin with a dash, what if word1 is '-x'?\nDouble dash says \"even if first word begins with '-', do not treat it as an option to the shell\n\nPage 41\n\n\n\nExample With Set\n\nRead a line from keyboard\nEcho words 3 and 5\nread var\nset -- $var\necho $3 $5\nBest way to split a line into words\n\nPage 42\n\n\n\n\n\n\n\n\nSection 8: Trapping Signals \n\nWhat are Signals?\n\nSignals are small messages sent to a process\nProcess interrupted to handle signal\nPossibilities for managing signal:\nTerminate\nIgnore\nPerform a programmer-defined action\n\nPage 43\n\n\n\nCommon Signals\n\nCommon signals are\nSIGINTR\tsent to foreground process by ^C\nSIGHUP\tsent when modem line gets hung up\nSIGTERM\tsent by kill -9\nSignals have numeric equivalents\n2 SIGINTR\n9 SIGTERM\n\nPage 44\n\n\n\nSend a Signal\n\nSend a signal to a process\nkill -2 PID\nkill -INTR PID\n\nPage 45\n\n\n\nTrap Signals\n\nHandling Signals\ntrap \"echo Interrupted; exit 2\" 2\nIgnoring Signals\ntrap \"\" 2 3\nRestoring Default Handler\ntrap 2\n\nPage 46\n\n\n\nWhere to Find List of Signals\n\nSee file\n\/usr\/include\/sys\/signal.h\n\nPage 47\n\n\n\nUser Signals\n\nSIGUSR1, SIGUSR2 are for your use\nSend to a process with\nkill -USR1 PID\nDefault action is to terminate process\n\nPage 48\n\n\n\nExperiment With Signals\n\nScript that catches USR1\nEcho message upon each signal\ntrap 'echo USR1' 16\nwhile : ; do\n date\n sleep 3\ndone\nTry it: does signal interrupt sleep?\n\nPage 49\n\n\n\n\n\n\n\n\nSection 9: Understanding Command Translation \n\nCommand Translation\n\nCommon translations include\nSplitting at spaces, obey quotes\n$HOME -> \/users\/us\/freddy
\n`command` -> output of command
\nI\/O redirection
\nFile name wildcard expansion
\nCombinations of quotes and metacharacters confusing
\nResolve problems by understanding order of translations<\/p>\n
Page 50<\/p>\n
Experiment With Translation<\/p>\n
Try wildcards in echo command
\necho b*
\nb budget bzzzzz
\nb* translated by sh before echo runs
\nWhen echo runs it sees
\necho b budget bzzzzz
\nEcho command need not understand wildcards!<\/p>\n
Page 51<\/p>\n
Order of Translations<\/p>\n
Splits into words at spaces and tabs
\nDivides commands at
\n; & | && || (…) {…}
\nEchos command if -v
\nInterprets quotes
\nPerforms variable substitution<\/p>\n
Page 52<\/p>\n
Order of Translations (continued)<\/p>\n
Performs command substitution
\nImplements I\/O redirection and removes redirection characters
\nDivides command again according to IFS
\nExpands file name wildcards
\nEchos translated command if -x
\nExecutes command<\/p>\n
Page 53<\/p>\n
Exceptional Case<\/p>\n
Delayed expansion for variable assignments
\nVAR=b*
\necho $VAR
\n b b_file
\nWildcard re-expanded for each echo<\/p>\n
Page 54<\/p>\n
Examples With Translation<\/p>\n
Variables translated before execution
\nCan store command name in variable
\ncommand=”ls”
\n$command
\n file1 file2 dir1 dir2…
\nVariables translated before I\/O redirection
\ntempfile=”\/tmp\/scriptname_$$”
\nls -al > $tempfile<\/p>\n
Page 55<\/p>\n
Examples (continued)<\/p>\n
Delayed expansion of wildcards in variable assignment
\nOutput of this echo command changes when directory contents change (* is re-evaluated each time the command is run)
\nx=*
\necho $x
\nCan view values stored in variables with
\nset
\nTry it: verify that the wildcard is stored in x without expansion<\/p>\n
Page 56<\/p>\n
Examples (continued)<\/p>\n
Wildcards expanded after redirection (assuming file* matches exactly one file):
\ncat < file*\n file*: No such file or directory\nCommand in backquotes expanded fully (and before I\/O redirection)\ncat < `echo file*`\n (contents of file sent to screen)\n\nPage 57\n\n\n\nEval Command\n\nForces an extra evaluation of command\neval cat \\< file*\n (contents of matching file)\nBackslash delays translation of < until second translation\n\nPage 58\n\n\n\n\n\n\n\n\nSection 10: Writing Advanced Loops \n\nWhile loops\n\nExecute statements while a condition is true\ni=0\nwhile [ $i -lt 10 ]\ndo\n echo I is $i\n i=`expr $i + 1`\ndone\n\nPage 59\n\n\n\nUntil loops\n\nExecute statements as long as a condition is false\nuntil grep \"sort\" dbase_log > \/dev\/null
\ndo
\n sleep 10
\ndone
\necho “Database has been sorted”
\nExample executes until grep is unsuccessful<\/p>\n
Page 60<\/p>\n
Redirection of Loops<\/p>\n
Can redirect output of a loop
\nfor f in *.c
\ndo
\n wc -l $f
\ndone > loop.out
\nLoop runs in separate shell
\nNew variables forgotten after loop
\nBackgrounding OK, too<\/p>\n
Page 61<\/p>\n
Continue Command<\/p>\n
Used in for, while, and until loops
\nSkip remaining statements
\nReturn to top of loop
\nfor name in *
\ndo
\n if [ ! -f $name ] ; then
\n continue
\n fi
\n echo “Found file $name”
\ndone
\nExample loops over files, skips directories<\/p>\n
Page 62<\/p>\n
Break Command<\/p>\n
Used in for, while, and until loops
\nSkip remaining statements
\nExit loop
\nfor name in *
\ndo
\n if [ ! -r $name ] ; then
\n echo “Cannot read $name, quitting loop”
\n break
\n fi
\n echo “Found file or directory $name”
\ndone
\nExample loops over files and directories, quits if one is not readable<\/p>\n
Page 63<\/p>\n
Case Command<\/p>\n
Execute one of several blocks of commands
\ncase “string” in
\npattern1)
\n commands ;;
\npattern2)
\n commands ;;
\n*) # Default case
\n commands ;;
\nesac
\nPatterns specified with file name wildcards
\nquit) …
\nqu*) …<\/p>\n
Page 64<\/p>\n
Example With Case<\/p>\n
Read commands from keyboard and interpret
\nEnter this script ‘case.sh’
\necho Enter a command
\nwhile read cmd
\ndo
\n case “$cmd” in
\n list) ls -al ;;
\n freespace) df . ;;
\n quit|Quit) break ;;
\n *) echo “$cmd: No such command” ;;
\n esac
\ndone
\necho “All done”
\nWhen you run it, the script waits for you to type one of:
\nlist
\nfreespace
\nquit
\nQuit
\nTry it: modify the example so any command beginning with characters “free” runs df<\/p>\n
Page 65<\/p>\n
Infinite Loops<\/p>\n
Infinite loop with while
\nwhile :
\ndo
\n …
\ndone
\n: is no-op, always returns success status
\nMust use break or exit inside loop for it to terminate<\/p>\n
Page 66<\/p>\n
Section 11: Forking Remote Shells <\/p>\n
Remote Shells<\/p>\n
Rsh command
\nrsh hostname “commands”
\nRuns commands on remote system
\nMust have .rhosts set up
\nCan specify different login name
\nrsh -l name hostname “commands”<\/p>\n
Page 67<\/p>\n
Examples With rsh<\/p>\n
Check who’s logged on
\nrsh spooky “finger”
\nRun several remote commands
\nrsh spooky “uname -a; time”
\nExecutes .cshrc on remote system
\nBe sure to set path in .cshrc instead of .login<\/p>\n
Page 68<\/p>\n
Access Control with .Rhosts<\/p>\n
May get “permission denied” error from rsh
\nFix this with ~\/.rhosts on remote system
\nExample: provide for remote shell from spunky to spooky
\nspunky % rlogin spooky
\nspooky % vi ~\/.rhosts
\n\t(insert “spunky login-name”)
\nspooky % chmod 600 ~\/.rhosts
\nspooky % logout
\nspunky % rsh spooky uname -a
\n\tspooky 5.5 sparc SUNW,Ultra-1
\nMay also rlogin without password: security problem!<\/p>\n
Page 69<\/p>\n
Remote Shell I\/O<\/p>\n
Standard output sent to local host
\nrsh spooky finger > finger.spooky
\nStandard input sent to remote host
\ncat local-file | rsh spooky lpr –<\/p>\n
Page 70<\/p>\n
Return Status<\/p>\n
Get return status of rsh
\nrsh mayer “uname -a”
\necho $?
\nReturns 0 if rsh managed to connect to remote host
\nReturns 1 otherwise
\nInvalid hostname
\nPermission denied<\/p>\n
Page 71<\/p>\n
Remote Return Status<\/p>\n
What about exit status of remote command?
\nHave to determine success or failure from stdout or stderr<\/p>\n
Page 72<\/p>\n
Section 12: More Miscellaneous <\/p>\n
Temporary Files<\/p>\n
Use unique names to avoid clashes
\ntempfile=$HOME\/Weq_$$
\ncommand > $tempfile
\n$$ is PID of current shell
\nAvoids conflict with concurrent executions of script
\nDo not use \/tmp!<\/p>\n
Page 73<\/p>\n
Wait Command<\/p>\n
Wait for termination of background job
\ncommand &
\npid=$!
\n(other processing)
\nwait $pid
\nAllows overlap of two or more operations<\/p>\n
Page 74<\/p>\n
Section 13: Using Quotes <\/p>\n
Quotes<\/p>\n
Provide control of collapsing of spaces and translation of variables
\nTry it: run three examples
\nNo quotes (variables translated, spaces collapsed)
\necho Home: $HOME
\n Home: \/users\/us\/freddy
\nDouble quotes (no collapsing)
\necho “Home: $HOME”
\n Home: \/users\/us\/freddy
\nSingle quotes (no translation or collapsing)
\necho ‘Home: $HOME’
\n Home: $HOME
\nTry it: single quotes within double quotes
\necho “Home directory ‘$HOME’ is full…”<\/p>\n
Page 75<\/p>\n
Metacharacters<\/p>\n
Characters with special meaning to shell
\n” ‘ ` $ * [ ] ?
\n; > < & ( ) \\\nAvoid special meaning with quoting\necho 'You have $20'\nBackslash like single quotes\nApplies only to next character\necho You have \\$20\n\nPage 76\n\n\n\nExamples With Quotes\n\nBad command line:\ngrep dog.*cat file\nShell tries to expand dot.*cat as file name wildcard\nUse quotes to avoid translation\ngrep 'dog.*cat' file\nSingle quotes OK in this case because we don't need variable translation\n\nPage 77\n\n\n\nMore Examples With Quotes\n\nRead name and search file for name\nread name\ngrep \"$name\" dbase\nSingle quotes not OK because we need variable translation\n\nPage 78\n\n\n\nSearching for Metacharacters\n\nBad command line: search for dollar sign\ngrep \"Gimme.*$20\" file\nProblem: shell translates variable $20\nSolution: use single quotes\ngrep 'Gimme.*$20' file\n\n<\/p>\n","protected":false},"excerpt":{"rendered":"
UNIX Bourne Shell Scripting<\/p>\n
These are the workshop notes I used to use for teaching a course on Bourne shell (sh) scripting. I haven’t taught the class for some time now, but the material is all still current since the Bourne shell hasn’t changed. Often you’ll find that these notes are a bit short of […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[47],"tags":[],"_links":{"self":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3276"}],"collection":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3276"}],"version-history":[{"count":1,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3276\/revisions"}],"predecessor-version":[{"id":3277,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3276\/revisions\/3277"}],"wp:attachment":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3276"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3276"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3276"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}