Principle of Script<\/p>\n
Defining the Shell Type<\/p>\n
To make a ksh script (which is a ksh program) crate a new file with a starting line like:
\n#!\/usr\/bin\/ksh
\nIt is important that the path to the ksh is propper and that the line doesn not have more than 32 characters. The shell from which you are starting the script will find this line and and hand the whole script over to to ksh. Without this line the script would be interpreted by the same typ of shell as the one, from which it was started. But since the syntax is different for all shells, it is necessary to define the shell with that line.<\/p>\n
Four Types of Lines<\/p>\n
A script has four types of lines: The shell defining line at the top, empty lines, commentary lines starting with a # and command lines. See the following top of a script as an example for these types of lines:<\/p>\n
#!\/usr\/bin\/ksh<\/p>\n
# Commentary……<\/p>\n
file=\/path\/file
\nif [[ $file = $1 ]];then
\n command
\nfi
\nStart and End of Script<\/p>\n
The script starts at the first line and ends either when it encounters an “exit” or the last line. All “#” lines are ignored.<\/p>\n
Start and End of Command<\/p>\n
A command starts with the first word on a line or if it’s the second command on a line with the first word after a”;’.
\nA command ends either at the end of the line or whith a “;”. So one can put several commands onto one line:<\/p>\n
print -n “Name: “; read name; print “”<\/p>\n
One can continue commands over more than one line with a “\\” immediately followed by a newline sign which is made be the return key:<\/p>\n
grep filename | sort -u | awk ‘{print $4}’ | \\
\nuniq -c >> \/longpath\/file
\nName and Permissions of Script File<\/p>\n
The script mus not have a name which is identical to a unix command: So the script must NOT be called “test”!
\nAfter saveing the file give it the execute permissions with: chmod 700 filename.<\/p>\n
Variables<\/p>\n
Filling in<\/p>\n
When filling into a variable then one uses just it’s name: state=”US” and no blanks. There is no difference between strings and numbers: price=50.<\/p>\n
Using<\/p>\n
When using a variable one needs to put a $ sign in front of it: print $state $price.<\/p>\n
Arrays<\/p>\n
Set and use an array like:<\/p>\n
arrname[1]=4\tTo fill in
\nprint ${arraname[1]}\tTo print out
\n${arrname[*]}\tGet all elements
\n${#arrname[*]}\tGet the number of elements
\nDeclaration<\/p>\n
There are happily no declarations of variables needed in ksh. One cannot have decimals only integers.<\/p>\n
Branching<\/p>\n
if then fi<\/p>\n
if [[ $value -eq 7 ]];then
\n print “$value is 7”
\nfi
\nor:<\/p>\n
if [[ $value -eq 7 ]]
\nthen
\n print “$value is 7”
\nfi
\nor:<\/p>\n
if [[ $value -eq 7 ]];then print “$value is 7”;fi
\nif then else fi<\/p>\n
if [[ $name = “John” ]];then
\n print “Your welcome, ${name}.”
\nelse
\n print “Good bye, ${name}!”
\nfi
\nif then elif then else fi<\/p>\n
if [[ $name = “John” ]];then
\n print “Your welcome, ${name}.”
\nelif [[ $name = “Hanna” ]];then
\n print “Hello, ${name}, who are you?”
\nelse
\n print “Good bye, ${name}!”
\nfi
\ncase esac<\/p>\n
case $var in
\n john|fred) print $invitation;;
\n martin) print $declination;;
\n *) print “Wrong name…”;;
\nesac
\nLooping<\/p>\n
while do done<\/p>\n
while [[ $count -gt 0 ]];do
\n print “\\$count is $count”
\n (( count -= 1 ))
\ndone
\nuntil do done<\/p>\n
until [[ $answer = “yes” ]];do
\n print -n “Please enter \\”yes\\”: ”
\n read answer
\n print “”
\ndone
\nfor var in list do done<\/p>\n
for foo in $(ls);do
\n if [[ -d $foo ]];then
\n print “$foo is a directory”
\n else
\n print “$foo is not a directory”
\n fi
\ndone
\ncontinue…break<\/p>\n
One can skip the rest of a loop and directly go to the next iteration with: “continue”.<\/p>\n
while read line
\ndo
\n if [[ $line = *.gz ]];then
\n continue
\n else
\n print $line
\n fi
\ndone
\nOne can also prematurely leave a loop with: “break”.<\/p>\n
while read line;do
\n if [[ $line = *!(.c) ]];then
\n break
\n else
\n print $line
\n fi
\ndone
\nCommand Line Arguments<\/p>\n
(Officially they are called “positional parameters”)<\/p>\n
The number of command line arguments is stored in $# so one can check
\nfor arguments with:
\nif [[ $# -eq 0 ]];then
\n print “No Arguments”
\n exit
\nfi
\nThe single Arguments are stored in $1, ….$n and all are in $* as one string. The arguments cannot
\ndirectly be modified but one can reset the hole commandline for another part of the program.
\nIf we need a first argument $first for the rest of the program we do:
\nif [[ $1 != $first ]];then
\n set $first $*
\nfi
\nOne can iterate over the command line arguments with the help of the shift command. Shift indirectly removes the first argument.
\nuntil [[ $# -qe 0 ]];do
\n # commands ….
\n shift
\ndone
\nOne can also iterate with the for loop, the default with for is $*:<\/p>\n
for arg;do
\n print $arg
\ndone
\nThe program name is stored in $0 but it contains the path also!<\/p>\n
Comparisons<\/p>\n
To compare strings one uses “=” for equal and “!=” for not equal.
\nTo compare numbers one uses “-eq” for equal “-ne” for not equal as well as “-gt” for greater than
\nand “-lt” for less than.<\/p>\n
if [[ $name = “John” ]];then
\n # commands….
\nfi
\nif [[ $size -eq 1000 ]];then
\n # commands….
\nfi
\nWith “&&” for “AND” and “||” for “OR” one can combine statements:<\/p>\n
if [[ $price -lt 1000 || $name = “Hanna” ]];then
\n # commands….
\nfi
\nif [[ $name = “Fred” && $city = “Denver” ]];then
\n # commands….
\nfi<\/p>\n
Variable Manipulations<\/p>\n
Removing something from a variable<\/p>\n
Variables that contain a path can very easily be stripped of it: ${name##*\/} gives you just the filename.
\nOr if one wants the path: ${name%\/*}. % takes it away from the left and # from the right.
\n%% and ## take the longest possibility while % and # just take the shortest one.
\nReplacing a variable if it does not yet exits<\/p>\n
If we wanted $foo or if not set 4 then: ${foo:-4} but it still remains unset. To change that we use:
\n${foo:=4}
\nExiting and stating something if variable is not set<\/p>\n
This is very important if our program relays on a certain vaiable: ${foo:?”foo not set!”}
\nJust check for the variable<\/p>\n
${foo:+1} gives one if $foo is set, otherwise nothing.<\/p>\n
Ksh Regular Expressions<\/p>\n
Ksh has it’s own regular expressions.
\nUse an * for any string. So to get all the files ending it .c use *.c.
\nA single character is represented with a ?. So all the files starting with any sign followed bye 44.f can be fetched by: ?44.f.<\/p>\n
Especially in ksh there are quantifiers for whole patterns:<\/p>\n
?(pattern) matches zero or one times the pattern.
\n*(pattern) matches any time the pattern.
\n+(pattern) matches one or more time the pattern.
\n@(pattern) matches one time the pattern.
\n!(pattern) matches string without the pattern.<\/p>\n
So one can question a string in a variable like: if [[ $var = fo@(?4*67).c ]];then …<\/p>\n
Functions<\/p>\n
Description<\/p>\n
A function (= procedure) must be defined before it is called, because ksh is interpreted at run time.
\nIt knows all the variables from the calling shell except the commandline arguments. But has it’s
\nown command line arguments so that one can call it with different values from different places in
\nthe script. It has an exit status but cannot return a value like a c funcition can.
\nMaking a Function<\/p>\n
One can make one in either of the following two ways:
\nfunction foo {
\n # commands…
\n}<\/p>\n
foo(){
\n # commands…
\n}
\nCalling the Function<\/p>\n
To call it just put it’s name in the script: foo. To give it arguments do: foo arg1 arg2 …
\nThe arguments are there in the form of $1…$n and $* for all at once like in the main code.
\nAnd the main $1 is not influenced bye the $1 of a particular function.
\nReturn<\/p>\n
The return statement exits the function imediately with the specified return value as an exit status.<\/p>\n
Data Redirection<\/p>\n
General<\/p>\n
Data redirection is done with the follwoing signs: “> >> < <<\". Every program has at least a\n\nstandardinput, standardoutput and standarderroroutput. All of these can be redirected.\n\nCommand Output to File\n\nFor writing into a new file or for overwriting a file do: command > file<\/p>\n
For appending to a file do: command >> file<\/p>\n
Standard Error Redirection<\/p>\n
To redirect the error output of a command do: command 2> file<\/p>\n
To discard the error alltogether do: command 2>\/dev\/null<\/p>\n
To put the error to the same location as the normal output do: command 2>&1<\/p>\n
File into Command<\/p>\n
If a program needs a file for input over standard input do: command < file\n\nCombine Input and Output Redirection\n\ncommand < infile > outfile
\ncommand < infile > outfile 2>\/dev\/null<\/p>\n
Commands into Program ( Here Document )<\/p>\n
Every unix command can take it’s commands from a text like listing with:<\/p>\n
command < An awk script can have three types of blocks. One of them must be there. The BEGIN{} block is processed before the file is checked. The {} block runs for every line of input and the END{} block is processed after the final line of the input file.<\/p>\n awk ‘ Awk autosplits a line on whitespace as default. The fields are stored in $1 through $NF and the whole line is in $0. One can match or not match an individual field.<\/p>\n awk ‘ Awk can do for() loops like in c++ and has the normal if and while structures. In NR is current line number and in NF the number of fields on the current line.<\/p>\n awk ‘ Extract email addresses from incoming mail. The mail would be guided to the following script from within the ~\/.forward file. This is not an eficient method, but only an example to show serial processing of text. The next example will do the same thing within awk only and will be efficient. The mail comes in over standardinput into the script. if [[ $(grep -c $addr \/path\/antimailfile) -gt 0 ]];then<\/p>\n mail $addr < With #!\/usr\/bin\/nawk -f the whole script is interpreted intirely as an awk script and no more shell escapes are needed, but one can and has to do everything in awk itself. It’s nawk because of the getline function. This script es an example only. For really extracting email addresses several special cases would have to be considered…<\/p>\n #!\/usr\/bin\/nawk -f<\/p>\n # Lines from a mail are dropping in over stdin. Append every line to a { print >> “\/path\/mymailfile” }<\/p>\n # Find lines with with From: or Replay: at beginning.<\/p>\n \/^From:\/ || \/^Replay\/ {<\/p>\n # Find fields with @. Iterate over the fields and check for @<\/p>\n for(i=1;i<=nf;i++){\n\n if($i ~ \/@\/){\n\n # Clean the email addresses with gsub()\n\n gsub(\/[<>()”]\/,””,$i)<\/p>\n # Check whether the email address is in the antimailfile<\/p>\n while( getline antiaddr < \"\/path\/antimailfile\" ){\n\n # Compare actual address in $i with loaded address\n\n if($i == antiaddr){\n\n # Send a negative mail\n\n system(\"mail \" $i \" < \/path\/badmail\")\n\n # Now end the while loop \n \n break\n\n }else{\n\n # Send a positive mail\n\n system(\"mail \" $i \" < \/path\/goodmail\")\n }\n }\n }\n }\n}\nCalculate on columns and print formated output\n\nIf one has a formated input of number columns one can still split them on white space, but has to consider the format for output with printf()\n\n#!\/usr\/bin\/nawk -f\n\n# Reprintet lines without foo or boo\n\n! \/(foo|boo)\/ { print }\n\n# Rearange and calculate with columns but only on lines with foo or boo\n\n\/(foo|boo)\/ {\n\n # Extract fields\n\n mytype = $1\n var1 = $2\n var2 = $3\n var3 = $4\n \n # Calculate\n\n if(mytype == \"foo\"){\n\n var1 *= 10\n var2 += 20\n var3 = log(var3)\n }\n if(mytype == \"boo\"){\n\n var1 *= 4\n var2 += 10\n var3 = cos(var3)\n }\n\n # Print formated output in reverse order\n\n printf(\"%-4s%10.3f%10.3f%10.3f\\n\",mytype,var3,var2,var1)\n}\nHow to iterate over each word of a shell variable in awk\n\nIn this example there is first a shell variable filled in and then it is given to awk. Awk splits it into an array and then iterates over the array and looks for each word on the current line of a file. If it finds it, it prints the whole line.\n#!\/usr\/bin\/ksh\nvar=\"term1 term2 term3 term4 term5\"\nawk '\n BEGIN { split(myvar,myarr) }\n {\n for(val in myarr){\n if($0 ~ myarr[val]){\n print\n } \n } \n } \n' myvar=\"$var\" file\nFunctions\n\nThis example substitutes the first three occurences of \"searchterm\" with a different term in each case and from the fourth case it just prints the line as it is.\nIt should show where to place a function and how to call it.\n\n\n\n#!\/usr\/bin\/nawk -f\nBEGIN{\n mysub1 = \"first_sub\"\n mysub2 = \"second_sub\"\n mysub3 = \"third_sub\"\n mycount = 1\n find = \"searchterm\"\n}\n{\n if($0 ~ find){\n if(mycount == 1){ replace(mysub1); }\n if(mycount == 2){ replace(mysub2); }\n if(mycount == 3){ replace(mysub3); }\n if(mycount > 3){ print; } sub(find,mysub) As an example for a CGI script in awk I make one which presents the unix man pages in html.<\/p>\n man.cgi<\/p>\n String functions<\/p>\n sub(regexp,sub)\tSubstitute sub for regexp in $0 Perl can do 100 times more than awk can, but awk is present on any standard unix system, where perl first has to be installed. And for short commands awk seems to be more practical. The autosplit mode of perl splits into pieces called: $F[0] through $F[$#F] which is not so nice as $1 through $NF where awk retains the whole line in $0 at the same time. awk ‘{print $1}’ infile Principle of Script<\/p>\n Defining the Shell Type<\/p>\n To make a ksh script (which is a ksh program) crate a new file with a starting line like: #!\/usr\/bin\/ksh It is important that the path to the ksh is propper and that the line doesn not have more than 32 characters. The shell from which you are […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[47],"tags":[],"_links":{"self":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3453"}],"collection":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3453"}],"version-history":[{"count":3,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3453\/revisions"}],"predecessor-version":[{"id":3456,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3453\/revisions\/3456"}],"wp:attachment":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3453"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3453"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3453"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n‘ a=$var
\nBEGIN { }, { } and end { }<\/p>\n
\n BEGIN { myvalue = 1700 }
\n \/debt\/ { myvalue -= $4 }
\n \/want\/ { myvalue += $4 }
\n END { print myvalue }
\n‘ infile
\nMatch in a particular field<\/p>\n
\n $1 ~ \/fred\/ && $4 !~ \/ok\/ {
\n print “Fred has not yet paid $3”
\n }
\n‘ infile
\nFor, If, substr()<\/p>\n
\n BEGIN { count = 0 }
\n \/myline\/ {
\n for(i=1;i<=NF;i++){\n if(substr($i,3,2) == \"ae\"){\n bla = \"Found it on line: \"\n print bla NR \" in field: \" i\n count++\n\t\t\t }\n }\n }\n END { print \"Found \" count \" instances of it\" }\n' infile\n\nTurn around each word in a file:\n\nawk '\n { for(i=1;i<=NF;i++){\n len = length($i)\n for(j=len;j>0;j–){
\n char = substr($i,j,1)
\n tmp = tmp char
\n }
\n $i = tmp
\n tmp = “”
\n }
\n print
\n }
\n‘ infile
\nAwk scripts within a shell script<\/p>\n
\nBetween the commands there must be a pipe “|”. For continuing on the next line one needs a “\\” behind the pipe to escape the invisible newline.
\n#!\/usr\/bin\/ksh
\n{ while read line;do
\n print – “$line”
\ndone } |\\
\ntee -a \/path\/mymailfile |\\
\nawk ‘
\n \/^From\/ || \/^Replay\/ {
\n for(i=1;i<=NF;i++){\n if($i ~ \/@\/){\n print $i\n }\n }\n }\n' |\\\nsed '\n s\/[<>]\/\/g;
\n s\/[()]\/\/g;
\n s\/”\/\/g;
\n …more substitutions for really extracting the email only…
\n‘ |\\
\n{ while read addr;do<\/p>\n
\nWhile iterates until the expression becomes wrong or until a break is encountered.
\nGsub() is for string substitution.
\nGetline reads in a line each time it es called.
\nSystem() executes a unix command.
\n“>>” appends to a file.<\/p>\n
\n# file before checking anything.<\/p>\n
\n mycount++
\n }else{
\n print
\n }
\n}
\nfunction replace(mysub) {<\/p>\n
\n print
\n break
\n}
\nCGI with gawk<\/p>\n
\nsub(regexp,sub,var)\tSubstitute sub for regexp in var
\ngsub(regexp,sub)\tGlobally substitute sub for regexp in $0
\ngsub(regexp,sub,var)\tGlobally substitute sub for regexp in var
\nsplit(var,arr)\tSplit var on white space into arr
\nsplit(var,arr,sep)\tSplit var on white space into arr on sep as separator
\nindex(bigvar,smallvar)\tFind index of smallvar in bigvar
\nmatch(bigvar,expr)\tFind index for regexp in bigvar
\nlength(var)\tNumber of characters in var
\nsubstr(var,num)\tExtract chars from posistion num to end
\nsubstr(var,num1,num2)\tExtract chars from num1 through num2
\nsprintf(format,vars)\tFormat vars to a string
\nWhen to use awk, when to use perl?<\/p>\n
\nTo get the first column of any file in awk and in perl:<\/p>\n
\nperl -nae ‘print $F[0],”\\n”;’ infile <\/p>\n","protected":false},"excerpt":{"rendered":"