The Bourne shell (/bin/sh) is present on all Unix installations and scripts written in this language are (quite) portable; man 1 sh is a good reference.
Basics
Variables and arguments
Assign with variable=value and get content with $variable
MESSAGE=”Hello World” # Assign a string
PI=3.1415 # Assign a decimal number
N=8
TWON=`expr $N * 2` # Arithmetic expression (only integers)
TWON=$(($N * 2)) # Other syntax
TWOPI=`echo “$PI * 2” | bc -l` # Use bc for floating point operations
ZERO=`echo “c($PI/4)-sqrt(2)/2” | bc -l`
The command line arguments are
$0, $1, $2, … # $0 is the command itself
$# # The number of arguments
$* # All arguments (also $@)
Special Variables
$$ # The current process ID
$? # exit status of last command
command
if [ $? != 0 ]; then
echo “command failed”
fi
mypath=`pwd`
mypath=${mypath}/file.txt
echo ${mypath##*/} # Display the filename only
echo ${mypath%%.*} # Full path without extention
foo=/tmp/my.dir/filename.tar.gz
path = ${foo%/*} # Full path without extention
var2=${var:=string} # Use var if set, otherwise use string
# assign string to var and then to var2.
size=$(stat -c%s “$file”) # get file size in bourne script
filesize=${size:=-1}
Constructs
for file in `ls`
do
echo $file
done
count=0
while [ $count -lt 5 ]; do
echo $count
sleep 1
count=$(($count + 1))
done
myfunction() {
find . -type f -name “*.$1” -print # $1 is first argument of the function
}
myfunction “txt”
Generate a file
MYHOME=/home/colin
cat > testhome.sh << _EOF
# All of this goes into the file testhome.sh
if [ -d “$MYHOME” ] ; then
echo $MYHOME exists
else
echo $MYHOME does not exist
fi
_EOF
sh testhome.sh
Bourne script example
As a small example, the script used to create a PDF booklet from this xhtml document:
#!/bin/sh
# This script creates a book in pdf format ready to print on a duplex printer
if [ $# -ne 1 ]; then # Check the argument
echo 1>&2 “Usage: $0 HtmlFile”
exit 1 # non zero exit if error
fi
file=$1 # Assign the filename
fname=${file%.*} # Get the name of the file only
fext=${file#*.} # Get the extension of the file
prince $file -o $fname.pdf # from www.princexml.com
pdftops -paper A4 -noshrink $fname.pdf $fname.ps # create postscript booklet
cat $fname.ps |psbook|psnup -Pa4 -2 |pstops -b “2:0,1U(21cm,29.7cm)” > $fname.book.ps
ps2pdf13 -sPAPERSIZE=a4 -sAutoRotatePages=None $fname.book.ps $fname.book.pdf
# use #a4 and #None on Windows!
exit 0 # exit 0 means successful
Some awk commands
Awk is useful for field stripping, like cut in a more powerful way. Search this document for other examples. See for example gnulamp.com and one-liners for awk for some nice examples.
awk ‘{ print $2, $1 }’ file # Print and inverse first two columns
awk ‘{printf(“%5d : %s\n”, NR,$0)}’ file # Add line number left aligned
awk ‘{print FNR “\t” $0}’ files # Add line number right aligned
awk NF test.txt # remove blank lines (same as grep ‘.’)
awk ‘length > 80’ # print line longer than 80 char)
Some sed commands
Here is the one liner gold minehttp://student.northpark.edu/pemente/sed/sed1line.txt. And a good introduction and tutorial to sedhttp://www.grymoire.com/Unix/Sed.html.
sed ‘s/string1/string2/g’ # Replace string1 with string2
sed -i ‘s/wroong/wrong/g’ *.txt # Replace a recurring word with g
sed ‘s/\(.*\)1/\12/g’ # Modify anystring1 to anystring2
sed ‘/
/,/<\/p>/d’ t.xhtml # Delete lines that start with
# and end with
sed ‘/ *#/d; /^ *$/d’ # Remove comments and blank lines
sed ‘s/[ \t]*$//’ # Remove trailing spaces (use tab as \t)
sed ‘s/^[ \t]*//;s/[ \t]*$//’ # Remove leading and trailing spaces
sed ‘s/[^*]/[&]/’ # Enclose first char with [] top->[t]op
sed = file | sed ‘N;s/\n/\t/’ > file.num # Number lines on a file
Regular Expressions
Some basic regular expression useful for sed too. See Basic Regex Syntaxhttp://www.regular-expressions.info/reference.html for a good primer.
[\^$.|?*+() # special characters any other will match themselves
\ # escapes special characters and treat as literal
* # repeat the previous item zero or more times
. # single character except line break characters
.* # match zero or more characters
^ # match at the start of a line/string
$ # match at the end of a line/string
.$ # match a single character at the end of line/string
^ $ # match line with a single space
[^A-Z] # match any line beginning with any char from A to Z
Some useful commands
The following commands are useful to include in a script or as one liners.
sort -t. -k1,1n -k2,2n -k3,3n -k4,4n # Sort IPv4 ip addresses
echo ‘Test’ | tr ‘[:lower:]’ ‘[:upper:]’ # Case conversion
echo foo.bar | cut -d . -f 1 # Returns foo
PID=$(ps | grep script.sh | grep bin | awk ‘{print $1}’) # PID of a running script
PID=$(ps axww | grep [p]ing | awk ‘{print $1}’) # PID of ping (w/o grep pid)
IP=$(ifconfig $INTERFACE | sed ‘/.*inet addr:/!d;s///;s/ .*//’) # Linux
IP=$(ifconfig $INTERFACE | sed ‘/.*inet /!d;s///;s/ .*//’) # FreeBSD
if [ `diff file1 file2 | wc -l` != 0 ]; then […] fi # File changed?
cat /etc/master.passwd | grep -v root | grep -v \*: | awk -F”:” \ # Create http passwd
‘{ printf(“%s:%s\n”, $1, $2) }’ > /usr/local/etc/apache2/passwd
testuser=$(cat /usr/local/etc/apache2/passwd | grep -v \ # Check user in passwd
root | grep -v \*: | awk -F”:” ‘{ printf(“%s\n”, $1) }’ | grep ^user$)
:(){ :|:& };: # bash fork bomb. Will kill your machine
tail +2 file > file2 # remove the first line from file
I use this little trick to change the file extension for many files at once. For example from .cxx to .cpp. Test it first without the | sh at the end. You can also do this with the command rename if installed. Or with bash builtins.
# ls *.cxx | awk -F. ‘{print “mv “$0” “$1″.cpp”}’ | sh
# ls *.c | sed “s/.*/cp & &.$(date “+%Y%m%d”)/” | sh # e.g. copy *.c to *.c.20080401
# rename .cxx .cpp *.cxx # Rename all .cxx to cpp
# for i in *.cxx; do mv $i ${i%%.cxx}.cpp; done # with bash builtins
Bourne Shell Reference:
This file contains short tables of commonly used items in this shell. In
most cases the information applies to both the Bourne shell (sh) and the
newer bash shell.
Tests (for ifs and loops) are done with [ ] or with the test command.
Checking files:
-r file Check if file is readable.
-w file Check if file is writable.
-x file Check if we have execute access to file.
-f file Check if file is an ordinary file (as opposed to a directory,
a device special file, etc.)
-s file Check if file has size greater than 0.
-d file Check if file is a directory.
-e file Check if file exists. Is true even if file is a directory.
Example:
if [ -s file ]
then
such and such
fi
Checking strings:
s1 = s2 Check if s1 equals s2.
s1 != s2 Check if s1 is not equal to s2.
-z s1 Check if s1 has size 0.
-n s1 Check if s2 has nonzero size.
s1 Check if s1 is not the empty string.
Example:
if [ $myvar = “hello” ]
then
echo “We have a match”
fi
Checking numbers:
Note that a shell variable could contain a string that represents a number.
If you want to check the numerical value use one of the following:
n1 -eq n2 Check to see if n1 equals n2.
n1 -ne n2 Check to see if n1 is not equal to n2.
n1 -lt n2 Check to see if n1 < n2.
n1 -le n2 Check to see if n1 <= n2.
n1 -gt n2 Check to see if n1 > n2.
n1 -ge n2 Check to see if n1 >= n2.
Example:
if [ $# -gt 1 ]
then
echo “ERROR: should have 0 or 1 command-line parameters”
fi
Boolean operators:
! not
-a and
-o or
Example:
if [ $num -lt 10 -o $num -gt 100 ]
then
echo “Number $num is out of range”
elif [ ! -w $filename ]
then
echo “Cannot write to $filename”
fi
Note that ifs can be nested. For example:
if [ $myvar = “y” ]
then
echo “Enter count of number of items”
read num
if [ $num -le 0 ]
then
echo “Invalid count of $num was given”
else
… do whatever …
fi
fi
The above example also illustrates the use of read to read a string from
the keyboard and place it into a shell variable. Also note that most UNIX
commands return a true (nonzero) or false (0) in the shell variable status
to indicate whether they succeeded or not. This return value can be checked.
At the command line echo $status. In a shell script use something like this:
if grep -q shell bshellref
then
echo “true”
else
echo “false”
fi
Note that -q is the quiet version of grep. It just checks whether it is true
that the string shell occurs in the file bshellref. It does not print the
matching lines like grep would otherwise do.
I/O Redirection:
pgm > file Output of pgm is redirected to file.
pgm < file Program pgm reads its input from file.
pgm >> file Output of pgm is appended to file.
pgm1 | pgm2 Output of pgm1 is piped into pgm2 as the input to pgm2.
n > file Output from stream with descriptor n redirected to file.
n >> file Output from stream with descriptor n appended to file.
n >& m Merge output from stream n with stream m.
n <& m Merge input from stream n with stream m.
<< tag Standard input comes from here through next tag at start of line.
Note that file descriptor 0 is normally standard input, 1 is standard output,
and 2 is standard error output.
Shell Built-in Variables:
$0 Name of this shell script itself.
$1 Value of first command line parameter (similarly $2, $3, etc)
$# In a shell script, the number of command line parameters.
$* All of the command line parameters.
$- Options given to the shell.
$? Return the exit status of the last command.
$$ Process id of script (really id of the shell running the script)
Pattern Matching:
* Matches 0 or more characters.
? Matches 1 character.
[AaBbCc] Example: matches any 1 char from the list.
[^RGB] Example: matches any 1 char not in the list.
[a-g] Example: matches any 1 char from this range.
Quoting:
\c Take character c literally.
`cmd` Run cmd and replace it in the line of code with its output.
“whatever” Take whatever literally, after first interpreting $, `…`, \
‘whatever’ Take whatever absolutely literally.
Example:
match=`ls *.bak` Puts names of .bak files into shell variable match.
echo \* Echos * to screen, not all filename as in: echo *
echo ‘$1$2hello’ Writes literally $1$2hello on screen.
echo “$1$2hello” Writes value of parameters 1 and 2 and string hello.
Grouping:
Parentheses may be used for grouping, but must be preceded by backslashes
since parentheses normally have a different meaning to the shell (namely
to run a command or commands in a subshell). For example, you might use:
if test \( -r $file1 -a -r $file2 \) -o \( -r $1 -a -r $2 \)
then
do whatever
fi
Case statement:
Here is an example that looks for a match with one of the characters a, b, c.
If $1 fails to match these, it always matches the * case. A case statement
can also use more advanced pattern matching.
case “$1” in
a) cmd1 ;;
b) cmd2 ;;
c) cmd3 ;;
*) cmd4 ;;
esac
Shell Arithmetic:
In the original Bourne shell arithmetic is done using the expr command as in:
result=`expr $1 + 2`
result2=`expr $2 + $1 / 2`
result=`expr $2 \* 5` (note the \ on the * symbol)
With bash, an expression is normally enclosed using [ ] and can use the
following operators, in order of precedence:
* / % (times, divide, remainder)
+ – (add, subtract)
< > <= >= (the obvious comparison operators)
== != (equal to, not equal to)
&& (logical and)
|| (logical or)
= (assignment)
Arithmetic is done using long integers.
Example:
result=$[$1 + 3]
In this example we take the value of the first parameter, add 3, and place
the sum into result.
Order of Interpretation:
The bash shell carries out its various types of interpretation for each line
in the following order:
brace expansion (see a reference book)
~ expansion (for login ids)
parameters (such as $1)
variables (such as $var)
command substitution (Example: match=`grep DNS *` )
arithmetic (from left to right)
word splitting
pathname expansion (using *, ?, and [abc] )
Other Shell Features:
$var Value of shell variable var.
${var}abc Example: value of shell variable var with string abc appended.
# At start of line, indicates a comment.
var=value Assign the string value to shell variable var.
cmd1 && cmd2 Run cmd1, then if cmd1 successful run cmd2, otherwise skip.
cmd1 || cmd2 Run cmd1, then if cmd1 not successful run cmd2, otherwise skip.
cmd1; cmd2 Do cmd1 and then cmd2.
cmd1 & cmd2 Do cmd1, start cmd2 without waiting for cmd1 to finish.
(cmds) Run cmds (commands) in a subshell.
See a good reference book for information on traps, signals, exporting of
variables, functions, eval, source, etc.
Recent Comments