Shell program
View System Date, Calender
Calender
[root@cluster1 ~]# cal
April 2015
Su Mo Tu We Th Fr Sa
1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30
[root@cluster1 ~]# cal 7 2015
July 2015
Su Mo Tu We Th Fr Sa
1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31
date
Syntax to specify format
date +FORMAT
[root@cluster1 ~]# date
Mon Apr 27 23:14:35 SGT 2015
[root@cluster1 ~]# date ‘+DATE:%m-%y%nTime:%H:%M:%S’
DATE:04-15
Time:23:15:14
[root@cluster1 ~]#
[root@cluster1 ~]# date +%Y%m%d
20150427
[root@cluster1 ~]# date +%Y-%m-%d
2015-04-27
[root@cluster1 ~]# date ‘+%d/%m/%Y_%H:%M:%S
[root@cluster1 ~]# date ‘+%d/%m/%Y_%H:%M:%S’
27/04/2015_23:16:39
[root@cluster1 ~]# date ‘+%d/%m/%Y-%H:%M:%S’
27/04/2015-23:16:49
[root@cluster1 ~]# date +”%x %r %Z”
04/27/2015 11:18:01 PM SGT
[root@cluster1 ~]# right_now=$(date +”%x %r %Z”)
time_stamp=”Updated on $right_now by $USER”
[root@cluster1 ~]# echo $time_stamp
Updated on 04/27/2015 11:18:23 PM SGT by root
Task: Display date in mm-dd-yy format
Type the command as follows:
$ date +”%m-%d-%y”
Output:
02-27-07
Turn on 4 digit year display:
$ date +”%m-%d-%Y”
Just display date as mm/dd/yy format:
$ date +”%D”
Task: Display time only
Type the command as follows:
$ date +”%T”
Output:
19:55:04
Display locale’s 12-hour clock time
$ date +”%r”
Output:
07:56:05 PM
Display time in HH:MM format:
$ date +”%H-%M”
How do I save time/date format to a variable?
Simply type command as follows at a shell prompt:
$ NOW=$(date +”%m-%d-%Y”)
To display a variable use echo / printf command:
$ echo $NOW
Sample shell script:
#!/bin/bash
NOW=$(date +”%m-%d-%Y”)
FILE=”backup.$NOW.tar.gz”
# rest of script Complete list of FORMAT control characters supported by date command
FORMAT controls the output.It can be the combination of any one of the following:
%%
a literal %
%a
locale’s abbreviated weekday name (e.g., Sun)
%A
locale’s full weekday name (e.g., Sunday)
%b
locale’s abbreviated month name (e.g., Jan)
%B
locale’s full month name (e.g., January)
%c
locale’s date and time (e.g., Thu Mar 3 23:05:25 2005)
%C
century; like %Y, except omit last two digits (e.g., 21)
%d
day of month (e.g, 01)
%D
date; same as %m/%d/%y
%e
day of month, space padded; same as %_d
%F
full date; same as %Y-%m-%d
%g
last two digits of year of ISO week number (see %G)
%G
year of ISO week number (see %V); normally useful only with %V
%h
same as %b
%H
hour (00..23)
%I
hour (01..12)
%j
day of year (001..366)
%k
hour ( 0..23)
%l
hour ( 1..12)
%m
month (01..12)
%M
minute (00..59)
%n
a newline
%N
nanoseconds (000000000..999999999)
%p
locale’s equivalent of either AM or PM; blank if not known
%P
like %p, but lower case
%r
locale’s 12-hour clock time (e.g., 11:11:04 PM)
%R
24-hour hour and minute; same as %H:%M
%s
seconds since 1970-01-01 00:00:00 UTC
%S
second (00..60)
%t
a tab
%T
time; same as %H:%M:%S
%u
day of week (1..7); 1 is Monday
%U
week number of year, with Sunday as first day of week (00..53)
%V
ISO week number, with Monday as first day of week (01..53)
%w
day of week (0..6); 0 is Sunday
%W
week number of year, with Monday as first day of week (00..53)
%x
locale’s date representation (e.g., 12/31/99)
%X
locale’s time representation (e.g., 23:13:48)
%y
last two digits of year (00..99)
%Y
year
%z
+hhmm numeric timezone (e.g., -0400)
%:z
+hh:mm numeric timezone (e.g., -04:00)
%::z
+hh:mm:ss numeric time zone (e.g., -04:00:00)
%:::z
numeric time zone with : to necessary precision (e.g., -04, +05:30)
%Z
Retrieve yesterday date using bash shell
$ date ‘+%A, %Y-%m-%d’
Saturday, 2014-06-28
$ date -d “yesterday” ‘+%A, %Y-%m-%d’
Friday, 2014-06-27
[root@cluster1 ~]# echo “Today is `date +%A` `date +%e`th of `date +%B` ”
Today is Wednesday 29th of April
A specific date minus one day, formatted as we wish:
date -d “2014-10-01 -1 day” +%Y-%m-%d
2014-09-30
==============================================================================================================================
Create Files & Directories
a) touch
b) mkdir
mkdir dir1
mkdir – create directory
mkdir command can take more arguments. For example we can take two directories with only one command:
mkdir dir2 dir3
mkdir – create two directories
mkdir command can also create entire directory tree. By using -p option mkdir will also create parent directories if required:
mkdir -p dir4/dir4.1
mkdir -m 744 dir5
mkdir -p /home/mohan/test1 /home/mohan/test2 /home/mohan/test3
c) PWD
d) cat
cat > test
this test file
cat < test this test file cat test test1 > test2
cat test2
this test file
this test1 file
e) touch
g) mv
h) rm
i) rmdir
=======================================================================================================================
Copies, Links to Files & Directories
copy
cp [OPTION]… [-T] SOURCE DEST
cp [OPTION]… SOURCE… DIRECTORY
cp [OPTION]… -t DIRECTORY SOURCE…
-i, –interactive
prompt before overwrite
-p same as –preserve=mode,ownership,timestamps
-R, -r, –recursive
copy directories recursively
-v, –verbose explain what is being done
cp -r dir2/ dir1/
Copy file1 from dir1 to dir2 but prompt before overwrite:
cp -i dir1/file1 dir2/
Copy file1 from dir1 to dir2 but prompt before overwrite:
cp -i dir1/file1 dir2/
Link command
HARD LINK :It is copy of the old file to new file if the source file is deleted still we can have the use link file
ln old new
ln with soft link
SOFT LINK :It is copy of the old file to if the source file is deleted we wil lose the new link as well.
ln -s old old_soft
rm old
cat old_soft
error doesnt exist
=========================================================================================================================
File Permissions list files ls
chmod 775
umask
ls
ls -d
ls -l
=====================================================================================================================================
uname -a
#!/bin/sh
SYSTEM=`uname -s`
if [ $SYSTEM = “Linux” ] ; then
echo “Linux”
elif [ $SYSTEM = “FreeBSD” ] ; then
echo “FreeBSD”
elif [ $SYSTEM = “Solaris” ] ; then
echo “Solaris”
else
echo “What?”
fi
#system info
system_info() {
echo “**********************************************”
echo “system info:”
echo
echo ” System-release : `cat /etc/redhat-release`”
echo ” Kernel-release : `uname -a|awk ‘{print $1,$3}’`”
echo ” Server-Model : `dmidecode | grep “Product Name:”|sed -n ‘1p’|awk -F’: ‘ ‘{print $2}’`”
echo
}
fn_distro(){
arch=$(uname -m)
kernel=$(uname -r)
if [ -f /etc/lsb-release ]; then
os=$(lsb_release -s -d)
elif [ -f /etc/debian_version ]; then
os=”Debian $(cat /etc/debian_version)”
elif [ -f /etc/redhat-release ]; then
os=`cat /etc/redhat-release`
else
os=”$(uname -s) $(uname -r)”
fi
}
#!/bin/bash
fn_distro(){
arch=$(uname -m)
kernel=$(uname -r)
if [ -f /etc/lsb-release ]; then
os=$(lsb_release -s -d)
elif [ -f /etc/debian_version ]; then
os=”Debian $(cat /etc/debian_version)”
elif [ -f /etc/redhat-release ]; then
os=`cat /etc/redhat-release`
else
os=”$(uname -s) $(uname -r)”
fi
}
fn_uptime(){
uptime=$(
===================================================================================================================
Count Lines, Words & Characters Using ‘wc’
$ cat sample.txt
today is a
good day
$ wc -w sample.txt
5 sample.txt
time perl -nle ‘$word += scalar(split(” “, $_)); END{print $word}’ subst.c
real 0m0.021s
user 0m0.016s
sys 0m0.004s
wc -l : Prints the number of lines in a file.
wc -w : prints the number of words in a file.
wc -c : Displays the count of bytes in a file.
wc -m : prints the count of characters from a file.
wc -L : prints only the length of the longest line in a file.
===================================================================================================================
sort
cat > animals
owls
camels
pigs
dogs
lions
elephants
asses
ctrl D
base ball
[root@cluster1 tmp]# sort animals
asses
camels
dogs
elephants
lions
owls
pigs
[root@cluster1 tmp]# sort sports
base ball
cricket
ice hockey
soccer
$ cat test
zzz
sss
qqq
aaa
BBB
ddd
AAA
sort test
Create the following test file for this example:
$ cat test
2K
2G
1K
6T
1T
1G
2M
The following sort command sorts human readable numbers (i.e 1K = 1 Thousand, 1M = 1 Million, 1G = 1 Giga, 1T = 1 Tera) in test file and displays sorted output.
$ sort -h test
1K
2K
2M
1G
2G
1T
6T
3. Sort Months of an Year using -M option
If we want to sort in the order of months of year, then we can use -M or –month-sort option.
Create the following test file for this example:
$ cat test
sept
aug
jan
oct
apr
feb
mar11
The following sort command sorts lines in test file as per month order. Note, lines in file should contain at least 3 character name of month name at start of line (e.g. jan, feb, mar). If we will give, ja for January or au for August, then sort command would not consider it as month name.
$ sort -M test
jan
feb
mar11
apr
aug
sept
oct
4. Check if Content is Already Sorted using -c option
If we want to check data in text file is sorted or not, then we can use -c or –check, –check=diagnose-first option.
Create the following test file for this example:
$ cat test
2
5
1
6
The following sort command checks whether text file data is sorted or not. If it is not, then it shows first occurrence with line number and disordered value.
$ sort -c test
sort: test:3: disorder: 1
5. Reverse the Output and Check for Uniqueness using -r and -u options
If we want to get sorted output in reverse order, then we can use -r or –reverse option. If file contains duplicate lines, then to get unique lines in sorted output, “-u” option can be used.
Create the following test file for this example:
$ cat test
5
2
2
1
4
4
The following sort command sorts lines in test file in reverse order and displays sorted output.
$ sort -r test
5
4
4
2
2
1
The following sort command sorts lines in test file in reverse order and removes duplicate lines from sorted output.
$ sort -r -u test
5
4
2
1
6. Selectively Sort the Content, Customize delimiter, Write output to a file using -k, -t, -o options
If we want to sort on the column or word position in lines of text file, then “-k” option can be used. If we each word in each line of file is separated by delimiter except ‘space’, then we can specify delimiter using “-t” option. We can get sorted output in any specified output file (using “-o” option) instead of displaying output on standard output.
Create the following test file for this example:
$ cat test
aa aa zz
aa aa ff
aa aa tt
aa aa kk
The following sort command sorts lines in test file on the 3rd word of each line and displays sorted output.
$ sort -k3 test
aa aa ff
aa aa kk
aa aa tt
aa aa zz
$ cat test
aa|5a|zz
aa|2a|ff
aa|1a|tt
aa|3a|kk
Here, several options are used altogether. In test file, words in each line are separated by delimiter ‘|’. It sorts lines in test file on the 2nd word of each line on the basis of numeric value and stores sorted output into specified output file.
$ sort -n -t’|’ -k2 test -o outfile
The contents of output file are shown below.
$ cat outfile
aa|1a|tt
aa|2a|ff
aa|3a|kk
aa|5a|zz
sort -r filename.txt
Example6: Some times its required to sort the file and display only uniq values.
sort -u filename
Note: though the values on other field are different this will not consider by -u option.
Example7: I want to sort a file according to my requirement and save it to a different file. Use -o option to save the sorted output to a file.
sort -o temp.txt filename.txt
Example8: Do you have file content with sizes like 10K, 20G, 45M, 32T etc. You can sort accourding to human readable by using -h option. This option works RHEL5 and above versions.
sort -h filename.txt
Similar to above example we can use -m for sorting according to month of the year.
sort -M filename.txt
Example9: Check if the file is alrady in sorted format or not by using -c option. This option will show you what is the first occurence disorderd value.
sort -c filename.txt
You can now mix above options to get your sorting work done.
There are two types of sorting keys definition in Unix sort:
new (with -k option)
old (+n -n).
New-style sort keys definitions
New style definition use -k option:
-k field_start [type] [,field_end [type] ]
where:
field_start and field_end
define a key field restricted to a portion of the line.
type
is a modifier from the list of characters bdfiMnr. The b modifier behaves like the -b option, but applies only to the field_start or field_end to which it is attached and characters within a field are counted from the first non-blank character in the field. (This applies separately to first_character and last_character.) The other modifiers behave like the corresponding options, but apply only to the key field to which they are attached. They have this effect if specified with field_start, field_end or both. If any modifier is attached to a field_start or to a field_end, no option applies to either.
When there are multiple key fields, later keys are compared only after all earlier keys compare equal. Except when the -u option is specified, lines that otherwise compare equal are ordered as if none of the options -d, -f, -i, -n or -k were present (but with -r still in effect, if it was specified) and with all bytes in the lines significant to the comparison.
The notation:
-k field_start[type][,field_end[type]]
defines a key field that begins at field_start and ends at field_end inclusive, unless field_start falls beyond the end of the line or after field_end, in which case the key field is empty. A missing field_end means the last character of the line.
A field comprises a maximal sequence of non-separating characters and, in the absence of option -t, any preceding field separator.
The field_start portion of the keydef option-argument has the form:
field_number[.first_character]
Fields and characters within fields are numbered starting with 1. field_number and first_character, interpreted as positive decimal integers, specify the first character to be used as part of a sort key. If .first_character is omitted, it refers to the first character of the field.
The field_end portion of the keydef option-argument has the form:
field_number[.last_character]
The field_number is as described above for field_start. last_character, interpreted as a non-negative decimal integer, specifies the last character to be used as part of the sort key. If last_character evaluates to zero or .last_character is omitted, it refers to the last character of the field specified by field_number.
If the -b option or b type modifier is in effect, characters within a field are counted from the first non-blank character in the field. (This applies separately to first_character and last_character.)
Old-style keys definition
There is also “old-style” key definition[+pos1 [-pos2]] that are now formally obsolite, but still is extremly widely used. Provide functionality equivalent to the -kkeydef option.
pos1 and pos2 each have the form m.n optionally followed by one or more of the flags bdfiMnr. A starting position specified by +m.n is interpreted to mean the n+1st character in the m+1st field. A missing .n means .0, indicating the first character of the m+1st field. If the b flag is in effect n is counted from the first non-blank in the m+1st field; +m.0b refers to the first non-blank character in the m+1st field.
A last position specified by -m.n is interpreted to mean the nth character (including separators) after the last character of the mth field. A missing .n means .0, indicating the last character of the mth field. If the b flag is in effect n is counted from the last leading blank in the m+1st field; -m.1b refers to the first non-blank in the m+1st field.
The fully specified +pos1 -pos2 form with type modifiers T and U: +w.xT -y.zU
is equivalent to:
undefined (z==0 & U contains b & -t is present)
-k w+1.x+1T,y.0U (z==0 otherwise)
-k w+1.x+1T,y+1.zU (z > 0)
Implementations support at least nine occurrences of the sort keys (the -k option and obsolescent +pos1 and -pos2) which are significant in command line order. If no sort key is specified, a default sort key of the entire line is used.
If the ordering rule options precede the sort key options, they are globally applied to all sort keys. For example, sort -r +2 -3 infile
reverses the order based on field 3. If the ordering rule options are attached to a sort key option, they override the global ordering options for only that sort key. For example,
sort -r +2d -3d infile
sorts field 3 by dictionary comparison but sorts the rest of the line using reverse comparison.
The most common mistake is to forget to use -n option for sorting numeric fields. Also specifying delimiter (option -t) with an unquoted character after it can be a source of problems; it’s better to use single quotes around the character that you plan to use as a delimiter. for example -t ‘:’
The most common mistake is to forget to use -n option for sorting numeric fields
Here is a standard example of usage of the sort utility, sorting /etc/passwd file (user database) by UID (the third colon-separated field in the passwd file structure):
sort -t ‘:’ +2 /etc/passwd # incorrect result, the field is numeric
sort -n -t ‘:’ +2 /etc/passwd # order of the numbers is now correct
sort -t ‘:’ -k 3,3n /etc/passwd
sort -t ‘:’ +2 -3n /etc/passwd
sort -n -t ‘:’ -k 3,3 /etc/passwd
See Sorting key definitions and Examples for more details. Generally you will be surprised how often the result is not what you want due to the obscurity of the definitions
Be careful and always test your sorting keys on a small sample before sorting the whole file. You will be surprised how often the result is not what you want.
By default sort sorts the file in ascending order using the entire line as a sorting key. Please note that a lot of WEB resources interpret this sort utility behavior incorrectly (most often they state that by default sorting is performed on the first key).
The three most important options of Unix soft are -n (numeric keys sorting), +n (sorting using n-th field, counting from zero) and -r (sort in reverse). For example:
Sort the entire lines as a key: sort
Sort in numeric order: sort -n
Sort on n+1 st field (old style key definition): sort +n
Comparisons are based on one or more sort keys extracted from each line of input. Again, please remember that by default, there is one sort key, the entire input line.
Lines are ordered according to the collating sequence of the current locale. By changing locale you can change the behavior of the sort.
In Solaris there are two variants of sort: System V version and BSD version. Both have identical options:
/usr/bin/sort [ -cmu ] [ -o output ] [ -T directory ] [ -y [ kmem ]] [ -z recsz ] [ -dfiMnr ] [ – b ] [ t char ] [ -k keydef ] [ +pos1 [ -pos2 ]] [ file…]
/usr/xpg4/bin/sort [ -cmu ] [ -o output ] [ -T directory ] [ -y [ kmem ]] [ -z recsz ] [ – dfiMnr ] [ -b ] [ -t char ] [ -k keydef ] [ +pos1 [ -pos2 ]] [ file…]
The sort command can (and should) be used in pipes or have its output redirected as desired. Here are some practically important examples that illustrates using of this utility (for more examples please look into our sort examples collection page):
1. Sort the file and display the sorted file, one page at a time. (If you prefer the standard command “more”, you can use this instead of “less”. “less” is an enhanced version of “more” – for example, it allows you to move backwards and forwards in the file; to exit “less” , use “q”.)
sort file less
2. Read and sorts infile and display the result on standard output, which has been redirected to outfile:
sort infile > outfile # sorts infile and writes the results to outfile
3. Write sorted output directly to outfile.
sort -o outfile infile # same as in 1, but using an option -o
4. Read and sort infile “in place” writing to the same file
sort -o infile infile # sort file “in place”
5. Sort the file and pipe it to uniq command with the number of identical keys counter printed (-c option in uniq)
sort infile uniq -c
6. Pipe the output of the who command to the input of the sort command:
who sort
7. Classic “number of instances” analysis of log files:
cat messages awk ‘{“select the keywords”}’ sort uniq -c sort -nr
In simple cases cut can be used instead of AWK. For example the following example couts distinc visitors from HTTP logs (assuming this is the first field in the logs):
cat http.log cut -d ” ” -f 1 sort uniq -c sort -nr
8. Sort the file, then prepend line numbers to each line (not many Unix adminitrators know that cat can be used to number lines):
sort -n file cat -n
This can be useful if you want to count the number of lines in which the first entry is in a given range: simply subtract the line numbers corresponding to the beginning and end of the range.
As I mentioned about by default the sort command uses entire lines as a key. It compares the characters starting with the first, until non-matching character or the end of the shortest line. Leading blanks (spaces and tabs) are considered valid characters to compare. Thus a line beginning with a space precedes a line beginning with the letter A. If you do not want this effect you need to delete leading spaces beforehand.
Multiple sort keys may be used on the same command line. If two lines have the same value in the same field, sort uses the next set of sort keys to resolve the equal comparison. For example,
sort +4 -5 +1 -2 infile
means to sort based on field 5 (+4 -5). If two lines have the same value in field 5, sort those two lines based on field 2 (+1 -2).
Beside sorting Unix sort is useful for merging files (option -m). It can also checked whether the file is sorted or not (option -c). It can also suppress duplicates (option -u):
-c Check whether the given files are already sorted: if they are not all sorted, print an error message and exit with a status of 1.
-m Merge the given files by sorting them as a group. Each input file should already be individually sorted. It always works to sort instead of merge; merging is provided because it is faster, in the case where it works.
-u Suppress all but one occurrence of matching keys.
In case Unix sort does not produce the required results you might want to look into Perl built-in function. If it is too slow more memory can be specified on invocation.
The most important options
The following list describes the options and their arguments that may be used to control how sort functions.
– Forces sort to read from the standard input. Useful for reading from pipes and files simultaneously.
-c Verifies that the input is sorted according to the other options specified on the command line. If the input is sorted correctly then no output is provided. If the input is not sorted then sort informs you of the situation. The message resembles this.
sort: disorder: This line not in sorted order.
-m Merges the sorted input. sort assumes the input is already sorted. sort normally merges input as it sorts. This option informs sort that the input is already sorted, thus sort runs much faster.
-o output Sends the output to file output instead of the standard output. The output file may be the same name as one of the input files.
-u Suppress all but one occurrence of matching keys. Normally, the entire line is the key. If field or character keys are specified, then the suppressing is done based on the keys.
-y kmem Use kmem kilobytes of main memory to initially start the sorting. If more memory is needed, sort automatically requests it from the operating system. The amount of memory allocated for the sort impacts the speed of the sort significantly. If no kmem is specified, sort starts with the default amount of memory (usually 32K). The maximum (usually 1 Megabyte) amount of memory may be allocated if needed. If 0 is specified for kmem, the minimum (usually 16K) amount of memory is allocated.
-z recsz Specifies the record size used to store each line. Normally the recsz is set to the longest line read during the sort phase. If the -c or -m options are specified, the sort phase is not performed and thus the record size defaults to a system size. If this default size is not large enough, sort may abort during the merge phase. To alleviate this problem you can specify a recsz that will allow the merge phase to run without aborting.
Ordering Options
-d Specifies dictionary sort. Only blanks, digits, and alphabetic characters are significant in the comparison.
-f Fold lowercase letters to uppercase. Ignores the difference between upper and lowercase ASCII characters.
-i Ignore characters outside the ASCII range of 040 (octal) and 0176 (octal). Only alphabetic characters, blanks, digits, and punctuation are used for comparison (printable characters). Control characters are ignored. This is only valid for nonnumeric sorts.
-M Compare fields as months. The first three nonblank characters are folded (see -i option) to uppercase and compared. Thus January is compared as JAN. JAN precedes FEB, and fields not containing months precede JAN. The -b option is placed in effect automatically.
-n Sorts the input numerically. The comparison is based on numerical value instead of alphabetic value. The number field used for comparison can contain optional blanks, optional minus signs, optional decimal point, and zero or more digits. The -b option is placed in effect automatically. Exponential numbers are not sorted correctly.
-r Reverse the order of the output.
Old-style Sort Key Options
+pos1
Specifies the beginning position of the input line used for field comparison. If pos1 is not specified then comparison begins at the beginning of the line. The pos1 position has the notation of f.c. The f specifies the number of fields to skip. The c specifies the number of characters to skip. For example, 3.2 is interpreted as skip three fields and two characters before performing comparisons. Omitting the .c portion is equivalent to specifying .0. Field one is referred to as position 0. If f is set to 0 then character positions are used for comparison.
-pos2
Specifies the ending position of the input line used for field comparison. If pos2 is not specified then comparison is done through the end of the line. The pos2 position has the notation of f.c. The f specifies to compare through field f. The c specifies the number of characters to compare through after field f. For example, -4.3 is interpreted as compare through three characters after the end of field four. Omitting the .c portion is equivalent to specifying .0.
-b
Ignores leading blanks when using the restricted sort key positions (+pos1 and -pos2). If the -b option is placed before the first +pos1 argument, then it applies to all +pos1 arguments. The -b option can be attached to each pos string to affect only that field.
-t c
Use character c as the field separator. Multiple adjacent c’s in the records are interpreted empty fields surrounded by separators. If you need to use multiple characters as a separator you need to convert record so that they are represented by a single character using sed or AWK. You can use nonprintable characters as a separator to ensure their uniqueness.
Old News 😉
[Jul 14, 2007] My SysAd Blog — UNIX Sort Files by Their Filesizes
Here’s a convenient way of finding those space hogs in your home directory (can be any directory). For me, those large files are usually a result of mkfile event (testing purposes) and can be promptly deleted. Here’s an example of its use.
#cd /export/home/esofthub
#ls -l sort +4n awk ‘{print $5 “\t” $9}’
Find recursively (a little awkward)
#ls -lR sort +4n awk ‘{print $5 “\t” $9}’ more
ps -ef sort
This command pipeline sorts the output of the “ps -ef” command. Because no arguments are supplied to the sort command, the output is sorted in alphabetic order by the first column of the ps -ef output (i.e., the output is sorted alphabetically by username).
ls -al sort +4n
This command performs a numeric sort on the fifth column of the “ls -al” output. This results in a file listing where the files are listed in ascending order, from smallest in size to largest in size.
ls -al sort +4n more
The same command as the previous, except the output is piped into the more command. This is useful when the output will not all fit on one screen.
ls -al sort +4nr
This command reverses the order of the numeric sort, so files are listed in descending order of size, with the largest file listed first, and the smallest file listed last.
[Feb 15, 2007] UNIX Disk Usage Simplifying Analysis with sort
The output of du has been very informative, but it’s difficult to scan a listing to ascertain the four or five largest directories, particularly as more and more directories and files are included in the output. The good news is that the Unix sort utility is just the tool we need to sidestep this problem.
# du -s * sort -nr
13984 Lynx
10464 IBM
3092 Gator
412 bin
196 DEMO
84 etcpasswd
76 CBO_MAIL
48 elance
36 CraigsList
16 Exchange
4 gettermsheet.sh
4 getstocks.sh
4 getmodemdriver.sh
4 buckaroo
4 browse.sh
4 badjoke.rot13
4 badjoke
0 gif.gif
One final concept and we’re ready to move along. If you want to only see the five largest files or directories in a specific directory, all that you’d need to do is pipe the command sequence to head:
# du -s * sort -nr head -5
13984 Lynx
10464 IBM
3092 Gator
412 bin
196 DEMO
The ! command (pronounced “bang”) creates a temporary file to be used with a program that requires a filename in its command line. This is useful with shells that don’t support process substitution. For example, to diff two files after sorting them, you might do:
diff `! sort file1` `! sort file2`
This command pipeline sorts the output of the “ps -ef” command. Because no arguments are supplied to the sort command, the output is sorted in alphabetic order by the first column of the ps -ef output (i.e., the output is sorted alphabetically by username).
ls -al sort +4n
This command performs a numeric sort on the fifth column of the “ls -al” output. This results in a file listing where the files are listed in ascending order, from smallest in size to largest in size.
ls -al sort +4n more
The same command as the previous, except the output is piped into the more command. This is useful when the output will not all fit on one screen.
ls -al sort +4nr
This command reverses the order of the numeric sort, so files are listed in descending order of size, with the largest file listed first, and the smallest file listed last.
======================================================================================================================================
Cut Command
Linux command cut is used for text processing. You can use this command to extract portion of text from a file by selecting columns.
Cutting specific fileds
Comma as Delimiter
Space as Delimiter
Pipe as Delimiter
Cutting from first field
Cutting till last field
Cutting range of fields
Ignore lines without delimiter
Invert field selection
Specify output delimiter
Cut specified characters
Cut specified bytes
-c, –characters=LIST
select only these characters
-d, –delimiter=DELIM
use DELIM instead of TAB for field delimiter
-f, –fields=LIST
select only these fields; also print any line that contains no delimiter character, unless the -s option is specified
-s, –only-delimited
do not print lines not containing delimiters
The cut command, as the man page states, “removes sections from each line of a file.” The cut command can also be used on a stream and it can do more than just remove section. If a file is not specified or “-” is used the cut command takes input from standard in. The cut command can be used to extract sections from a file or stream based upon a specific criteria. An example of this would be cutting specific fields from a csv (comma separated values) file. For instance, cut can be used to extract the name and email address from a csv file with the following content:
id, date, username, first name, last name, email address, phone, fax
1,2012-01-01,franklinf, Ford, Franklin, ff@gmail.com, 7575551212, 7775551234
2,2012-02-01,levona, Allan, Levon, allanl@tllts.org, 3177771212,
3,2012-02-17,mannyt, Trish, Manny, tmanny@hpr.org,7275551212,8885551236
he syntax for cut would be:
cut -d”,” -f4,5,6 users.csv
The result would be displayed on standard out:
first name, last name, email address
Ford, Franklin, ff@gmail.ccom
Allan, Levon, allanl@tllts.org
Trish, Manny, tmanny@hpr.org
The -d option specifies the delimiter which is defaults to a TAB. In the example above the cut command will “cut” the line at each “,” instead of a TAB. The -f option indicates which fields to select, in this case fields 4, 5, and 6 which correspond to “first name,” “last name,” and “email address.”
The cut command can operate on fields, characters or bytes and must include one and only one of these options.
The field option operates on the cuts defined by the delimiter (-d), which is TAB by default. The -d option can only be used with the field option. Attempting to use the -d option with the character (-c) or bytes (-b) options will result in an error. The -f value can be a command separated list or a range separated by a “-“:
cut -d”,” -f 1,2,3,4
cut -d”,” -f 1-4
cut -f 1-4,7,9
cut -d”,” -f -7
cut -d”,” -f 7-
cut is a very frequently used command for file parsing. It is very useful in splitting columns on files with or without delimiter. In this article, we will see how to use the cut command on files having a delimiter.
Let us consider a sample file, say file1, with a comma as delimiter as shown below:
$ cat file1
Rakesh,Father,35,Manager
Niti,Mother,30,Group Lead
Shlok,Son,5,Student
The first column indicates name, second relationship, the third being the age, and the last one is their profession.
cut command has 2 main options to work on files with delimiters:
-f – To indicate which field(s) to cut.
-d – To indicate the delimiter on the basis of which the cut command will cut the fields.
Let us now try to work with this command with a few examples:
1. To get the list of names alone from the file, which is the first column:
$ cut -d, -f 1 file1
Rakesh
Niti
Shlok
The option “-d’ followed by a comma indicates to cut the file on the basis of comma. “-f” followed by 1 indicates to retrieve the field 1 from the file1, and hence we got the names alone.
2. To get the relationship alone:, i.e, 2nd field
$ cut -d, -f 2 file1
Father
Mother
Son
3. To get 2 fields, say Name and Age:
$ cut -d, -f 1,3 file1
Rakesh,35
Niti,30
Shlok,5
Giving 1,3 means to retrieve the first and third fields which happens to be name and age respectively.
4. To get the name, relationship and age, excluding the profession, i.e, 1st to 3rd fields:
$ cut -d, -f 1-3 file1
Rakesh,Father,35
Niti,Mother,30
Shlok,Son,5
The option 1-3 means from first field till third field. Whenever we need a range of fields to be retrieved, we use the ‘-‘ option.
The same result above can also be retrieved in other ways also:
$ cut -d, -f -3 file1
Rakesh,Father,35
Niti,Mother,30
Shlok,Son,5
This is the best of the 3 methods to retrieve a range of fields. The option “-3” means from the beginning i.e, the first field till the third field. And hence we get the fields 1, 2 and 3.
5. To retrieve all the fields except the name field. i.e, to retrieve from field 2 to field 4:
$ cut -d, -f 2- file1
Father,35,Manager
Mother,30,Group Lead
Son,5,Student
Similar to the last result, “2-” means from the second field till the end which is the 4th field. Whenever the beginning of the range is not specified, it defaults to 1, similarly when the end of the range is not given, it defaults to the last field. The same result could have been achieved using the option “2-4″ as well.
Let us consider the same input file with a space as the delimiter:
$ cat file1
Rakesh Father 35 Manager
Niti Mother 30 GL
Shlok Son 5 Student
The same options and commands used above hold good but for the delimiter specified. When comma is the delimiter, we can give it after the -d option. However, for the space as delimiter, we need to quote the delimiter as shown below. In fact, we can always quote the delimiter to be in the safer side.
6. To retrieve the first field from a space delimited file:
$ cut -d” ” -f 1 file1
Rakesh
Niti
Shlok
Let us consider the same file separated by tab space:
$ cat file1
Rakesh Father 35 Manager
Niti Mother 30 GL
Shlok Son 5 Student
To actually confirm the file is indeed separated by tab space, use the “-t” option with the cat command:
$ cat -t file1
Rakesh^IFather^I35^IManager
Niti^IMother^I30^IGL
Shlok^ISon^I5^IStudent
The ^I indicates a tab space.
7. To retrieve the first field from this tab separated file. How to specify the tab space with the “-d” option?
$ cut -f 1 file1
Rakesh
Niti
Shlok
he options to cut by are below.
N N’th byte, character or field, counted from 1
N- from N’th byte, character or field, to end of line
N-M from N’th to M’th (included) byte, character or field
-M from first to M’th (included) byte, character or field
The options pretty much explain themselves but I have included some simple examples below:
Cutting by characters (command on top, output below)
echo “123456789” | cut -c -5
12345
echo “123456789” | cut -c 5-
56789
echo “123456789” | cut -c 3-7
34567
echo “123456789” | cut -c 5
5
Sometimes output from a command is delimited so a cut by characters will not work. Take the example below:
echo -e “1\t2\t3\t4\t5” |cut -c 5-7
3 4
To echo a tab you have to use the -e switch to enable echo to process back slashed characters. If the desired output is 3\t4 then this would work great if the strings were always 1 character but if anywhere before field 3 a character was added the output would be completely changed as followed:
echo -e “1a\t2b\t3c\t4d\t5e” | cut -c 5-7
b 3
This is resolved by cutting by fields.
Cutting by fields
The syntax to cut by fields is the same as characters or bytes. The two examples below display different output but are both displaying the same fields (Fields 3 Through to the end of line.)
echo -e “1\t2\t3\t4\t5” | cut -f 3-
3 4 5
echo -e “1a\t2a\t3a\t4a\t5a” | cut -f 3-
3a 4a 5a
The default delimiter is a tab, if the output is delimited another way a custom delimiter can be specified with the -d option. It can be just about any printable character, just make sure that the character is escaped (back slashed) if needed. In the example below I cut the string up using the pipe as the delimiter.
echo “1|2|3|4|5” | cut -f 3- -d \|
3|4|5
One great feature of cut is that the delimiter that was used for input can be changed by the output of cut. In the example below I change the format of the string from a dash delimited output and change it to a comma.
echo -e “1a-2a-3a-4a-5a” | cut -f 3- -d – –output-delimiter=,
3a,4a,5a
Formatting with Cut Example
Sometimes certain Linux applications such as uptime do not have options to format the output. Cut can be used to pull out the information that is desired.
Normal up-time Command:
root@cluster1 :~$ uptime
19:18:40 up 1 day, 22:15, 4 users, load average: 0.45, 0.10, 0.03
Time with up-time displayed:
root@cluster1 :~$ uptime |cut -d , -f 1,2 | cut -c 2-
19:19:36 up 1 day, 22:22
For the above example I pipe the output of uptime to cut and tell it I want to split it with a comma , delimiter. I then choose fields 1 and 2. The output from that cut is piped into another cut that removes the spaces in front of the output.
Load averages extracted from uptime:
root@cluster1 :~$ uptime |cut -d , -f 4- | cut -c 3-
load average: 0.42, 0.10, 0.03
This is about the same as the previous example except the fields changed. Instead of fields 1 and 2 I told it to display fields 4 through the end. The output from that is piped to another cut which removes the three spaces that were after the comma in “4 users, ” by starting at the 3rd character.
The great thing about cutting by fields is that no matter if the field length changes the data stays the same. Take the example below. I now have 17 users logged in which would have broke the output if I had used -c (since there is an extra character due to a double digit number of users being logged in.)
root@cluster1 :~$ uptime
19:25:11 up 1 day, 22:28, 17 users, load average: 0.00, 0.06, 0.04
root@cluster1 :~$ uptime |cut -d , -f 4- | cut -c 3-
load average: 0.00, 0.06, 0.04
That just about covers everything for the cut command. Now you know about it you can use cut to chop up all types of strings. It is one of the many great tools available for string manipulation in bash. If you can remember what cut does it will make your shell scripting easier, you don’t need to memorize the syntax because all of the information on how to use cut is available here, in the man pages and all over the web.
========================================================================================================================================================================================================
Convert & Copy Files With ‘dd’
The Linux command ‘dd’ is one of the very powerful utility which can be used in a variety of ways. This tool is mainly used for copying and converting data, hence it stands for ‘data duplicator’.
This tool can be used for.
• Backing up and restoring an entire hard drive or a partition
• Copy regions of raw device files like backing up MBR(master boot record)
• Converting data formats like ASCII to EBCDIC
• Converting lowercase to uppercase and vice versa
• Creating files with fixed size
Only superuser can execute this command. You should be very careful while using this command as improper usage may cause huge data loss. So, some people consider this tool as ‘data destroyer’.
Syntax of ‘dd’ command
dd if=
[root@cluster1 tmp]# cat doc
Hi how are you and Now you know about it you can use cut to chop up all types of strings.
[root@cluster1 tmp]# dd if=doc of=out conv=ucase
0+1 records in
0+1 records out
90 bytes (90 B) copied, 0.00021344 s, 422 kB/s
1. Backing up and restoring an entire hard drive or a partition
a. Backup entire hard drive to another drive.
dd if=/dev/sda of=/dev/sdb bs=4096 conv=noerror,sync
Here, ‘if’ stands for input file , ‘of’ stands for output file and ‘bs’ stands for the block size (number of bytes to be read/write at a time). The conversion parameter ‘noerror’ allows the tool to continue to copy the data eventhough it encounter any errors. The sync option allows to use synchronized I/O.
The above command will copy all the data from the disk /dev/sda to /dev/sdb. ‘dd’ doesn’t know anything about the filesystem or partitions- it will just copy everything from /dev/sda to /dev/sdb. So, this will clone the disk with the same data on same partition.
b. Creating a disk image
dd if=/dev/sda of=/tmp/sdadisk.img
Backing up a disk to an image will be faster than copying the exact data. Also, disk image make the restoration much more easier.
c. Creating a compressed disk image
dd if=/dev/sda | gzip >/tmp/sdadisk.img.gz
d. Restoring hard disk image
dd if=/tmp/sdadisk.img of=/dev/sda
e. Restoring compressed image
gzip –dc /tmp/sdadisk.img.gz | dd of=/dev/sda
f. Clone one partition to another
dd if=/dev/sda1 of=/dev/sdb1 bs=4096 conv=noerror,sync
This will synchronize the partition /dev/sda1 to /dev/sdb1. You must verify that the size of /dev/sdb1 should be larger than /dev/sda1
2. Backing up and Restoring MBR
Master Boot record is the boot sector which houses the GRUB boot loader. If MBR got corrupted, we will not be able to boot into Linux. MBR -512 byte data- is located at the first sector of the hard disk. It consists of 446 byte bootstrap, 64 byte partition table and 2 bytes signature.
a. Backing up MBR
dd if=/dev/sda of=/tmp/mbr.img bs=512 count=1
The option “count” refers to the number of input blocks to be copied
b. Backing up the boot data of MBR excluding the partition table
dd if=/dev/sda of=/tmp/mbr.img bs=446 count=1
c. Restoring MBR from MBR image
dd if=/tmp/mbr.img of=/dev/sda
d. Display master boot record
dd if=/dev/hda of=mbr.bin bs=512 count=1
od -xa mbr.bin
3. Converting data formats
a. Convert the data format of a file from ASCII to EBCDIC
dd if=textfile.ascii of=textfile.ebcdic conv=ebcdic
b. Convert the data format of a file from EBCDIC to ASCII
dd if=textfile.ebcdic of=textfile.ascii conv=ascii
4. Converting case of a file
a. Converting a file to Uppercase
dd if=file1 of=file2 conv=ucase
b. Converting a file to lowercase
dd if=file1 of=file2 conv=lcase
5. Creating or modifying data files
a. Create a fixed size, say 10MB file
dd if=/dev/zero of=file1 bs=10485760 count=1
The block size is calculated as 10MB=10*1024*1024
b. Modify the first 512 bytes of a file with null data
dd if=/dev/zero of=file1 bs=512 count=1 conv=notrunc
The option ‘notrunc’ refers to do not truncate the file, only replace the first 512 bytes, if it exists. Otherwise, you will get a 512 byte file.
Creating an image file with dd command
First, make sure you’ve sufficient disk space to create a image file using dd:
$ df -H
To create 1MB file (1024kb), enter:
$ dd if=/dev/zero of=test.img bs=1024 count=0 seek=1024
You will get an empty files (also known as “sparse file”) of arbitrary size using above syntax. To create 10MB file , enter:
$ dd if=/dev/zero of=test.img bs=1024 count=0 seek=$[1024*10]
To create 100MB file , enter:
$ dd if=/dev/zero of=test.img bs=1024 count=0 seek=$[1024*100]
$ ls -lh test.img
To create 1GB, file:
$ dd if=/dev/zero of=1g.img bs=1 count=0 seek=1G
Sample output:
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000235686 s, 0.0 kB/s
Verify file size (note bs factor in original dd command):
$ ls -lh 1g.img
$ stat 1g.img
$ du -h 1g.im
Burn an ISO File
Using the Terminal
To burn the iso on an usb stick, enter the following command in a terminal :
dd bs=4M if=/path/to/manjaro.iso of=/dev/sd[drive letter]
Where [drive letter] is the letter of your removable device. Please note that it is the device (e.g. /dev/sdb), and not the partition number (e.g. /dev/sdb1).
To find which drive letter it might be write:
fdisk -l
============================================================================================================
Get Help, View Fancy Text & Reduce File Size
man
banner command
How do I use figlet?
Simply use it as follows:
figlet “Shell Hacks”
____ _ _ _ _ _ _
/ ___|| |__ ___| | | | | | | __ _ ___| | _____
\___ \| ‘_ \ / _ \ | | | |_| |/ _` |/ __| |/ / __|
___) | | | | __/ | | | _ | (_| | (__| <\__ \
|____/|_| |_|\___|_|_| |_| |_|\__,_|\___|_|\_\___/
To change the font, use the -f option, for example :
$ figlet -f digital "Shell Hacks"
+-+-+-+-+-+ +-+-+-+-+-+
|S|h|e|l|l| |H|a|c|k|s|
+-+-+-+-+-+ +-+-+-+-+-+
Use the -c option if you would prefer centered output :
$ figlet -c "Shell Hacks"
_ _____ __ _
(_) / ____| / _| |
_ __ ___ _| | _ __ __ _| |_| |_
| '_ \| \ \/ / | | '__/ _` | _| __|
| | | | |> <| |____| | | (_| | | | |_
|_| |_|_/_/\_\\_____|_| \__,_|_| \__|
======================================================================================================================
Programming 1
Hello World
[root@cluster1 shell]# cat test.sh
#!/bin/bash
echo "Hello World"
echo -e "
##################################
#
# Welcome to `hostname`
# This system is running `cat /etc/redhat-release`
# kernel is `uname -r`
#
# You are logged in as `whoami`
#
##################################
"
=========================================================================================================================
Shell Variables, Grab User Input Using 'read'
===========================================================================================================================================================
###########Positional Parameters###########
a=25
echo$a
25
a="Hey, what's up"
echo $a
#!/bin/bash
echo "please provide the arugument"
mv $1 $2
cat $2
#!/bin/bash
chmod 744 $1
ls -l $1
set I love india
echo $1
echo $*
Reverse Quotes Or Accent Graves
#!/bin/bash
name=$1
set `who am i `
mv $name $name.$1
[root@cluster1 shell]# cat shell2.sh
echo i am not doing good while runing this shell program=$#
[root@cluster1 shell]# ./shell2.sh Count The Number of Command Line Arguments Using
i am not doing good while runing this shell program=8
==================================================================================================================================================================================
Math On Integers Using 'expr'
Integer Math
First way to do math with integer (and only integer) is to use the command “expr — evaluate expression”.
root@cluster1 $ expr 1 + 1
2
root@cluster1 $ myvar=$(expr 1 + 1)
root@cluster1 $ echo $myvar
2
root@cluster1 $ expr $myvar + 1
3
root@cluster1 $ expr $myvar / 3
1
root@cluster1 $ expr $myvar \* 3
9
When doing a “multiply by” make sure to backslash the “asterisk” as it’s a wildcard in Bash used for expansion.
Another alternative to expr, is to use the bash builtin command let.
root@cluster1 $ echo $myvar
6
root@cluster1 $ let myvar+=1
root@cluster1 $ echo $myvar
7
root@cluster1 $ let myvar+1
root@cluster1 $ echo $myvar
7
root@cluster1 $ let myvar2=myvar+1
root@cluster1 $ echo $myvar2
8
Also, you can simply use the parentheses or square brackets :
root@cluster1 $ echo $myvar
3
root@cluster1 $ echo $((myvar+2))
5
root@cluster1 $ echo $[myvar+2]
5
root@cluster1 $ myvar=$((myvar+3))
This allow you to use C-style programming :
root@cluster1 $ echo $myvar
3
root@cluster1 $ echo $((myvar++))
3
root@cluster1 $ echo $myvar
4
root@cluster1 $ echo $((++myvar))
5
root@cluster1 $ echo $myvar
5
Floating point arithmetic
You can’t do floating point arithmetic natively in bash, you will have to use a command line tool, the most common one being “bc - An arbitrary precision calculator language”.
root@cluster1 $ bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
3*5.2+7/8
15.6
15.6+299.33*2.3/7.4
108.6
Of course you can use the STDIN to send your formula to “bc” then get the output on STDOUT.
root@cluster1 $ echo "3.4+7/8-(5.94*3.14)" | bc
-15.25
or by using the here-doc notation:
root@cluster1 $ bc <<< "3.4+7/8-(5.94*3.14)"
-15.25
I encourage you too take a look at the man pages to get more detail on how it works (man bc).
There are four special variables, scale, ibase, obase, and last. scale defines how some operations use digits after the decimal point. The default value of scale is 0. ibase and obase define the conver-
sion base for input and output numbers. The default for both input and output is base 10. last (an extension) is a variable that has the value of the last printed number.
The “scale” variable is really important for the precision of your results, especially when using integers only (Note: you can also use “bc -l” to use mathlib and see the result at max scale) .
root@cluster1 $ echo "2/3" | bc
0
root@cluster1 $ echo "scale=2; 2/3" | bc
.66
root@cluster1 $ echo "(2/3)+(7/8)" | bc
0
root@cluster1 $ echo "scale=2;(2/3)+(7/8)" | bc
1.53
root@cluster1 $ echo "scale=4;(2/3)+(7/8)" | bc
1.5416
root@cluster1 $ echo "scale=6;(2/3)+(7/8)" | bc
1.541666
root@cluster1 $ echo "(2/3)+(7/8)" | bc -l
1.54166666666666666666
Another way to do floating point arithmetic is to use AWK:
root@cluster1 $ awk "BEGIN {print 100/3}"
33.3333
You can use printf to adjust the precision of the results:
root@cluster1 $ awk "BEGIN {printf \"%.2f\n\", 100/3}"
33.33
When using negative values, make sure to leave a white space between signs.
WRONG
root@cluster1 $ awk "BEGIN {print -8.4--8}"
awk: cmd. line:1: BEGIN {print -8.4--8}
awk: cmd. line:1: ^ syntax error
GOOD
root@cluster1 $ awk "BEGIN {print -8.4 - -8}"
-0.4
#!/bin/bash
a=30 b=30
echo `expr $a + $b`
echo `expr $a - $b`
echo `expr $a \* $b`
echo `expr $a / $b`
echo `expr $a % $b`
==================================================================================================================================================================================
Operator Precedence
1st preference = /, * , %
2nd preference = +, -
#!/bin/bash
a=40 b=35 c=25 d=50
#!/bin/bash
a=40 b=35 c=25 d=50
echo `expr $a \* \( $b + $c \) / $d`
==================================================================================================================================================================================
Math On Real Numbers Floating Point Arithmetic
bc
#!/bin/bash
a=2.5 b=3.5
c=`echo $a + $b | bc`
d=`echo $a - $b | bc`
f=`echo $a \* $b | bc`
g=`echo $a / $b | bc`
echo $c, $d, $e, $f
$ result=$(echo "scale=2; 5 * 7 /3;" | bc)
$ echo $result
11.66
echo 'scale=2; (2.777 - 1.4744)/1' | bc
1.30
In contrast, note that the bash shell only performs integer arithmetic, e.g.:
$ result=$((5 * 7 /3))
$ echo $result
11
Command substitution stores the output of a command into a variable.
var3=$(echo "scale=2;$var1/$var2" | bc)
Calculate for float values
printf "%s\n" 'scale = 10; 196.304492/615.348986' | bc
bash-3.00$ x=$( printf "%s\n" 'scale = 10; 196.304492/615.348986' | bc)
bash-3.00$ echo $x
.3190132696
A bash round function:
round()
{
echo $(printf %.$2f $(echo "scale=$2;(((10^$2)*$1)+0.5)/(10^$2)" | bc))
};
Used in your code example:
#!/bin/bash
# the function "round()" was taken from
# the round function:
round()
{
echo $(printf %.$2f $(echo "scale=$2;(((10^$2)*$1)+0.5)/(10^$2)" | bc))
};
echo "Insert the price you want to calculate:"
read float
echo "This is the price without taxes:"
#echo "scale=2; $float/1.18" |bc -l
echo $(round $float/1.18 2);
read -p "Press any key to continue..."
#!/bin/bash
echo "----- Addition ------"
x=10.2
y=20.2
z=`echo $x+$y|bc`
echo $z
echo "------ Division -----"
x=3
y=25
z=`echo $y/$x|bc`
echo $z
echo "----- Substration -----"
x=20.9
y=33.67
z=`echo $y-$x|bc`
echo $z
echo "----- Multiply -----"
x=2.23
y=4.44
z=`echo $x*$y|bc`
echo $z
echo "----- Square root -----"
x=215
z=`echo "scale=4;sqrt($x)" | bc -l`
echo $z
echo "----- Powers of -----"
x=4.2
y=2
z=`echo $x^$y | bc -l`
echo $z
Output: /floating.sh
----- Addition ------
30.4
------ Division -----
8
----- Substration -----
12.77
----- Multiply -----
9.90
----- Square root -----
14.6628
----- Powers of -----
17.64
Math in Shell Scripts
One thing that often confuses new users to the Unix / Linux shell, is
how to do (even very simple) maths. In most languages, x = x + 1 (or
even x++) does exactly what you would expect. The Unix/Linux shell is
different,however. It doesn’t have any built-in mathematical operators
for variables. It can do comparisons, but maths isn’t supported, not
even simple addition.
Shell script variables are by default treated as strings,not numbers,
which adds some complexity to doing math in shell script. To keep with
script programming paradigm and allow for better math support, langua-
ges such Perl or Python would be better suited when math is desired.
However, it is possible to do math with shell script.In fact, over the
years, multiple facilities have been added to Unix to support working
with numbers.
You can do maths using any one of the following methods.
1. Using expr command
2 Using $(()) construct.
3 Using let command
4 Using bc command.
5 Using $[] construct.
expr command:
expr command performs arithmetic operations on integers. It can
perform the four basic arithmetic operations, as well as the modulus
(remainder function).
$ expr 5 + 10
15
$ a=10 b=5
$ expr $a + $b
15
$ expr $a / $b
2
$ expr $a * $b
expr: syntax error
$ expr $a \* $b
50
The operand, be it +,-,* etc., must be enclosed on either side by
whitespace. Observe that the multiplication operand (*) has to be
escaped to prevent the shell from interpreting it as the filename meta
character. Since expr can handle only integers, division yields only
the integral part.
expr is often used with command substitution to assign a variable.
For example, you can set a variable x to the sum of two numbers:
$ x=`expr $a + $b`
$ echo $x
15
Note: As you can see, for expr, you must put spaces around the
arguments: "expr 123+456" doesn’t work. "expr 123 + 456" works.
With double parentheses: $(()) and (())
In bash version 3.2 and later you can (and should) use $(()) and (())
for integer arithmetic expressions. You may have may not have spaces
around the operators, but you must not have spaces around the equal
sign, as with any bash variable assignment.
$ c=$(($a+9))
$ echo $c
19
$ c=$((a+9)) #Also correct, no need of $ sign.
$ c=$((a + 9)) #Also correct, no restriction on spaces.
$ c= $((a + b)) #Incorrect, space after assignment operator.
You may also use operations within double parentheses without
assignment.
$ ((a++))
$ echo $a
11
$ ((a+=1)) ; echo $a
12
$ ((d=a+b+9)) ; echo $d
26
$ ((a+=$b)) #Correct
$ (($a+=1)) #Incorrect
let command:
The let command carries out arithmetic operations on variables. In
many cases, it functions as a less complex version of expr command.
As you can see, it is also a little picky about spaces, but it wants
the opposite of what expr wanted. let also relaxes the normal rule of
needing a $ in front of variables to be read.
$ let a=10 # Same as 'a=11'
$ let a=a+5 # Equivalent to let "a = a + 5"
# Quotes permit the use of spaces in variable assignment. (Double
# quotes and spaces make it more readable.)
$ let a=$a + 5 # Without quotes spaces not allowed.
bash: let: +: syntax error: operand expected (error token is "+")
You need to use quotes if you want to use spaces between tokens of
the expression, for example
$ let "a = a + 5";echo $a
20
The only construct that is more convenient to use with let is incre-
ment such as
$ let a++ ; echo $a # as well as to ((i++))
16
bc: THE CALCULATOR
Bash doesn't support floating point arithmetic. The obvious candidate
for adding floating point capabilities to bash is bc. bc is not only a
command, it also a pseudo-programming language featuring arrays,funct-
ions,conditional(if) and loops(for and while). It also comes with a
library for performing scientific calculations. It can handle very,
very large numbers. If a computation results in a 900 digit number, bc
will show each and every digit. But you have to treat the variables as
strings.
Here is what happens when we try to do floating point math with the
shell:
$ let a=12.5
bash: let: a=12.5: syntax error: invalid arithmetic operator (error
token is ".5")
$ ((b=1*0.5))
bash: ((: b=1*0.5: syntax error: invalid arithmetic operator (error
token is ".5")
I can't explain everything about bc here, it needs another post.
But I will give some examples here.
Most of the bellow examples follow a simple formula:
$ echo '57+43' | bc
100
$ echo '57*43' | bc
2451
$ echo '6^6' | bc # Power
46656
$ echo '1.5*5'|bc # Allows floating point math.
7.5
$[] construct:
$ x=85
$ y=15
$ echo $[x+y]
100
$ echo $[x/y]
5
$ c=$[x*y]
$ echo $c
1275
Working of above methods shell dependent. Bash shell supports all 5
methods. Following shell script demonstrates above methods.
#!/bin/bash
# SCRIPT: basicmath.sh
# USAGE: basicmath.sh
# PURPOSE: Addition, Subtraction, Division and Multiplication of
# two numbers.
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
#####################################################################
# Variable Declaration #
#####################################################################
clear #Clears Screen
Bold="\033[1m" #Storing escape sequences in a variable.
Normal="\033[0m"
echo -e "$Bold Basic mathematics using bash script $Normal\n"
items="1. ADDITTION
2. SUBTRACTION
3. MULTIPLICATION
4. DIVISION
5. EXIT"
choice=
#####################################################################
# Define Functions Here #
#####################################################################
# If didn't understand these functions, simply remove functions and
# its entries from main script.
exit_function()
{
clear
exit
}
#Function enter is used to go back to menu and clears screen
enter()
{
unset num1 num2
ans=
echo ""
echo -e "Do you want to continue(y/n):\c"
stty -icanon min 0 time 0
# When -icanon is set then one character has been received.
# min 0 means that read should read 0 characters.
# time 0 ensures that read is terminated the moment one character
# is hit.
while [ -z "$ans" ]
do
read ans
done
#The while loop ensures that so long as at least one character is
# not received read continue to get executed
if [ "$ans" = "y" -o "$ans" = "Y" ]
then
stty sane # Restoring terminal settings
clear
else
stty sane
exit_function
fi
}
#####################################################################
# Main Starts #
#####################################################################
while true
do
echo -e "$Bold \tPROGRAM MENU $Normal\n"
echo -e "\t$items \n"
echo -n "Enter your choice : "
read choice
case $choice in
1) clear
echo "Enter two numbers for Addition : "
echo -n "Number1: "
read num1
echo -n "Number2: "
read num2
echo "$num1 + $num2 = `expr $num1 + $num2`"
enter ;;
2) clear
echo "Enter two numbers for Subtraction : "
echo -n "Number1: "
read num1
echo -n "Number2: "
read num2
echo "$num1 - $num2 = $((num1-num2))"
enter ;;
3) clear
echo "Enter two numbers for Multiplication : "
echo -n "Number1: "
read num1
echo -n "Number2: "
read num2
echo "$num1 * $num2 = `echo "$num1*$num2"|bc`"
enter ;;
4) clear
echo "Enter two numbers for Division : "
echo -n "Number1: "
read num1
echo -n "Number2: "
read num2
let div=num1/num2
echo "$num1 / $num2 = $div"
enter ;;
5) exit_function ;;
*) echo "You entered wrong option, Please enter 1,2,3,4 or 5"
echo "Press enter to continue"
read
clear
esac
done
OUTPUT:
[venu@localhost ~]$ sh basicmath.sh
Basic mathematics using bash script
PROGRAM MENU
1. ADDITTION
2. SUBTRACTION
3. MULTIPLICATION
4. DIVISION
5. EXIT
Enter your choice : 1
Enter two numbers for Addition :
Number1: 123
Number2: 456
123 + 456 = 579
Do you want to continue(y/n):y
PROGRAM MENU
1. ADDITTION
2. SUBTRACTION
3. MULTIPLICATION
4. DIVISION
5. EXIT
Enter your choice : 3
Enter two numbers for Multiplication :
Number1: 12.5
Number2: 2
12.5 * 2 = 25.0
Do you want to continue(y/n):n
rithmetic Operators:
There are following arithmetic operators supported by Bourne Shell.
Assume variable a holds 10 and variable b holds 20 then:
Show Examples
Operator Description Example
+ Addition - Adds values on either side of the operator `expr $a + $b` will give 30
- Subtraction - Subtracts right hand operand from left hand operand `expr $a - $b` will give -10
* Multiplication - Multiplies values on either side of the operator `expr $a \* $b` will give 200
/ Division - Divides left hand operand by right hand operand `expr $b / $a` will give 2
% Modulus - Divides left hand operand by right hand operand and returns remainder `expr $b % $a` will give 0
= Assignment - Assign right operand in left operand a=$b would assign value of b into a
== Equality - Compares two numbers, if both are same then returns true. [ $a == $b ] would return false.
!= Not Equality - Compares two numbers, if both are different then returns true. [ $a != $b ] would return true.
It is very important to note here that all the conditional expressions would be put inside square braces with one spaces around them, for example [ $a == $b ] is correct where as [$a==$b] is incorrect.
All the arithmetical calculations are done using long integers.
Relational Operators:
Bourne Shell supports following relational operators which are specific to numeric values. These operators would not work for string values unless their value is numeric.
For example, following operators would work to check a relation between 10 and 20 as well as in between "10" and "20" but not in between "ten" and "twenty".
Assume variable a holds 10 and variable b holds 20 then:
Show Examples
Operator Description Example
-eq Checks if the value of two operands are equal or not, if yes then condition becomes true. [ $a -eq $b ] is not true.
-ne Checks if the value of two operands are equal or not, if values are not equal then condition becomes true. [ $a -ne $b ] is true.
-gt Checks if the value of left operand is greater than the value of right operand, if yes then condition becomes true. [ $a -gt $b ] is not true.
-lt Checks if the value of left operand is less than the value of right operand, if yes then condition becomes true. [ $a -lt $b ] is true.
-ge Checks if the value of left operand is greater than or equal to the value of right operand, if yes then condition becomes true. [ $a -ge $b ] is not true.
-le Checks if the value of left operand is less than or equal to the value of right operand, if yes then condition becomes true. [ $a -le $b ] is true.
It is very important to note here that all the conditional expressions would be put inside square braces with one spaces around them,
for example [ $a <= $b ] is correct where as [$a <= $b] is incorrect.
Boolean Operators:
There are following boolean operators supported by Bourne Shell.
Assume variable a holds 10 and variable b holds 20 then:
Show Examples
Operator Description Example
! This is logical negation. This inverts a true condition into false and vice versa. [ ! false ] is true.
-o This is logical OR. If one of the operands is true then condition would be true. [ $a -lt 20 -o $b -gt 100 ] is true.
-a This is logical AND. If both the operands are true then condition would be true otherwise it would be false. [ $a -lt 20 -a $b -gt 100 ] is false.
String Operators:
There are following string operators supported by Bourne Shell.
Assume variable a holds "abc" and variable b holds "efg" then:
Show Examples
Operator Description Example
= Checks if the value of two operands are equal or not, if yes then condition becomes true. [ $a = $b ] is not true.
!= Checks if the value of two operands are equal or not, if values are not equal then condition becomes true. [ $a != $b ] is true.
-z Checks if the given string operand size is zero. If it is zero length then it returns true. [ -z $a ] is not true.
-n Checks if the given string operand size is non-zero. If it is non-zero length then it returns true. [ -z $a ] is not false.
str Check if str is not the empty string. If it is empty then it returns false. [ $a ] is not false.
File Test Operators:
There are following operators to test various properties associated with a Unix file.
Assume a variable file holds an existing file name "test" whose size is 100 bytes and has read, write and execute permission on:
Show Examples
Operator Description Example
-b file Checks if file is a block special file if yes then condition becomes true. [ -b $file ] is false.
-c file Checks if file is a character special file if yes then condition becomes true. [ -c $file ] is false.
-d file Check if file is a directory if yes then condition becomes true. [ -d $file ] is not true.
-f file Check if file is an ordinary file as opposed to a directory or special file if yes then condition becomes true. [ -f $file ] is true.
-g file Checks if file has its set group ID (SGID) bit set if yes then condition becomes true. [ -g $file ] is false.
-k file Checks if file has its sticky bit set if yes then condition becomes true. [ -k $file ] is false.
-p file Checks if file is a named pipe if yes then condition becomes true. [ -p $file ] is false.
-t file Checks if file descriptor is open and associated with a terminal if yes then condition becomes true. [ -t $file ] is false.
-u file Checks if file has its set user id (SUID) bit set if yes then condition becomes true. [ -u $file ] is false.
-r file Checks if file is readable if yes then condition becomes true. [ -r $file ] is true.
-w file Check if file is writable if yes then condition becomes true. [ -w $file ] is true.
-x file Check if file is execute if yes then condition becomes true. [ -x $file ] is true.
-s file Check if file has size greater than 0 if yes then condition becomes true. [ -s $file ] is true.
-e file Check if file exists. Is true even if file is a directory but exists. [ -e $file ] is true.
==================================================================================================================================================================================
Escape Sequences
Escape Characters
Certain characters are significant to the shell; we have seen, for example, that the use of double quotes (") characters affect how spaces and TAB characters are treated, for example:
$ echo Hello World
Hello World
$ echo "Hello World"
Hello World
So how do we display: Hello "World" ?
$ echo "Hello \"World\""
The first and last " characters wrap the whole lot into one parameter passed to echo so that the spacing between the two words is kept as is. But the code:
$ echo "Hello " World ""
would be interpreted as three parameters:
"Hello "
World
""
So the output would be
Hello World
Note that we lose the quotes entirely. This is because the first and second quotes mark off the Hello and following spaces; the second argument is an unquoted "World" and the third argument is the empty string; "".
Thanks to Patrick for pointing out that this:
$ echo "Hello "World""
is actually only one parameter (no spaces between the quoted parameters), and that you can test this by replacing the echo command with (for example) ls.
Most characters (*, ', etc) are not interpreted (ie, they are taken literally) by means of placing them in double quotes (""). They are taken as is and passed on to the command being called. An example using the asterisk (*) goes:
$ echo *
case.shtml escape.shtml first.shtml
functions.shtml hints.shtml index.shtml
ip-primer.txt raid1+0.txt
$ echo *txt
ip-primer.txt raid1+0.txt
$ echo "*"
*
$ echo "*txt"
*txt
In the first example, * is expanded to mean all files in the current directory.
In the second example, *txt means all files ending in txt.
In the third, we put the * in double quotes, and it is interpreted literally.
In the fourth example, the same applies, but we have appended txt to the string.
However, ", $, `, and \ are still interpreted by the shell, even when they're in double quotes.
The backslash (\) character is used to mark these special characters so that they are not interpreted by the shell, but passed on to the command being run (for example, echo).
So to output the string: (Assuming that the value of $X is 5):
A quote is ", backslash is \, backtick is `.
A few spaces are and dollar is $. $X is 5.
we would have to write:
$ echo "A quote is \", backslash is \\, backtick is \`."
A quote is ", backslash is \, backtick is `.
$ echo "A few spaces are ; dollar is \$. \$X is ${X}."
A few spaces are ; dollar is $. $X is 5.
We have seen why the " is special for preserving spacing. Dollar ($) is special because it marks a variable, so $X is replaced by the shell with the contents of the variable X. Backslash (\) is special because it is itself used to mark other characters off; we need the following options for a complete shell:
$ echo "This is \\ a backslash"
This is \ a backslash
$ echo "This is \" a quote and this is \\ a backslash"
This is " a quote and this is \ a backslash
Quotes and escaping
Quoting and escaping is really an important way to influence the way, Bash treats your input. There are three recognized types:
per-character escaping using a backslash: \$stuff
weak quoting with double-quotes: "stuff"
strong quoting with single-quotes: 'stuff'
All three forms have the very same purpose: They give you general control over parsing, expansion and expansion's results.
Beside these common basic variants, there are some more special quoting methods (like interpreting ANSI-C escapes in a string) you'll meet below.
❗ ATTENTION ❗ These quote characters (", double quote and ', single quote) are a syntax element that influences parsing. It is not related to eventual quote characters that are passed as text to the commandline! The syntax-quotes are removed before the command is called! Look:
### NO NO NO: this passes three strings:
### (1) "my
### (2) multiword
### (3) argument"
MYARG="\"my multiword argument\""
somecommand $MYARG
### THIS IS NOT (!!!!) THE SAME AS ###
command "my multiword argument"
### YOU NEED ###
MYARG="my multiword argument"
command "$MYARG"
Per-character escaping
Per-character escaping is useful in different places, also here, on expansions and substitutions. In general, a character that has a special meaning for Bash, like the dollar-sign ($) to introduce some expansion types, can be masked to not have that special meaning using the backslash:
echo \$HOME is set to \"$HOME\"
\$HOME won't expand because it's not variable expansion syntax anymore
The quotes are masked with the backslash to be literal - otherwise they would be interpreted by Bash
The sequence \
# escapestr_sed()
# read a stream from stdin and escape characters in text that could be interpreted as
# special characters by sed
escape_sed() {
sed \
-e ‘s/\//\\\//g’ \
-e ‘s/\&/\\\&/g’
}
The backslash can be used to mask every character that has a special meaning for bash. Exception: Inside a single-quoted string (see below).
Weak quoting
Inside a weak-quoted string there’s no special interpretion of:
spaces as word-separators (on inital commandline splitting and on word splitting!)
single-quotes to introduce strong-quoting (see below)
characters for pattern matching
pathname expansion
process substitution
Everything else, especially parameter expansion, is performed!
ls -l “*”
Will not be expanded. ls gets the literal * as argument. It will, unless you have a file named *, spit out an error.
echo “Your PATH is: $PATH”
Will work as expected. $PATH is expanded, because it’s only double- (weak-) quoted.
If a backslash in double-quotes (“weak quoting”) occurs, there are 2 ways to deal with it
if the baskslash is followed by a character that would have a special meaning even inside double-quotes, the backslash is removed and the following character looses its special meaning
if the backslash is followed by a character without special meaning, the backslash is not removed
In particuar this means that “\$” will become $, but “\x” will become \x.
Strong quoting
Strong quoting is very easy to explain:
Inside a single-quoted string nothing(!!!!) is interpreted, except the single-quote that closes the quoting.
echo ‘Your PATH is: $PATH’
That $PATH won’t be expanded, it’s interpreted as normal ordinary text, because it’s surrounded by strong quotes.
In practise that means, to produce a text like Here’s my test… as a single-quoted string, you have to leave and re-enter the single-quoting to get the character “‘” as literal text:
# WRONG
echo ‘Here’s my test…’
# RIGHT
echo ‘Here’\”s my test…’
# ALTERNATIVE: It’s also possible to mix-and-match quotes for readability:
echo “Here’s my test”
ANSI C like strings
There’s another quoting mechanism, Bash provides: Strings that are scanned for ANSI C like escape sequences. The Syntax is
$’string’
where the following escape sequences are decoded in string:
Code Meaning
\” double-quote
\’ single-quote
\\ backslash
\a terminal alert character (bell)
\b backspace
\e escape (ASCII 033)
\E escape (ASCII 033) \E is non-standard
\f form feed
\n newline
\r carriage return
\t horizontal tab
\v vertical tab
\cx a control-x character, for example $’\cZ’ to print the control sequence composed by Ctrl-Z (^Z)
\uXXXX Interprets XXXX as hexadecimal number and prints the corresponding character from the character set (4 digits) (Bash 4.2-alpha)
\UXXXXXXXX Interprets XXXX as hexadecimal number and prints the corresponding character from the character set (8 digits) (Bash 4.2-alpha)
\nnn the eight-bit character whose value is the octal value nnn (one to three digits)
\xHH the eight-bit character whose value is the hexadecimal value HH (one or two hex digits)
hell scripts commonly used ANSI escape codes for color output.
Following table shows Numbers representing colors in Escape Sequences.
Color Foreground Background
Black 30 40
Red 31 41
Green 32 42
Yellow 33 43
Blue 34 44
Magenta 35 45
Cyan 36 46
White 37 47
The numbers in the above table work for xterm terminal.Result may vary
for other terminal emulators.
Use the following template for writing colored text.
echo -e “\033[COLORm Sample text”
The “\033[” begins the escape sequence.You can also use “\e[” instead
of “\033[“. COLOR specifies a foreground color, according to the table
above.The “m” terminates escape sequence, and text begins immediately
after that.
Note: With an echo, the -e option enables the escape sequences.You can
also use printf instead of echo.
printf “\e[COLORm sample text\n”
To print Green text
echo -e “\033[32m Hello World”
or
printf “\e[32m Hello World”
The problem with above statement is that the blue color that starts
with the 32 color code is never switched back to the regular color, so
any text you type after the prompt and even prompt also is still in the
Green color.
To return to the plain, normal mode, we have yet another sequence.
echo -e “\033[0m”
Now you won’t see anything new on the screen, as this echo statement
was not passed any string to display. But it has done its job, which
was to restore the normal viewing mode. Whatever yor type now will be
avoid of any fancy effects.
Escape sequence also allow you to control the manner in which
characters are displayed on the screen.
The following table summarizes numbers representing text attributes
in Escape Sequences.
ANSI CODE Meaning
0 Normal Characters
1 Bold Characters
4 Underlined Characters
5 Blinking Characters
7 Reverse video Characters
Note: Blink attribute doesn’t work in any terminal emulator, but it
will work on the console.
Combining all these Escape Sequences, you can get more fancy effect.
Use the following template for writing colored text on a colored
background.
echo -e “\033[COLOR1;COLOR2m sample text\033[0m”
The semicolon separated numbers “COLOR1” and “COLOR2″ specify a
foreground and a background color.The order of the numbers does not
matter, since the foreground and background numbers fall in non-
overlapping ranges.”m” terminates the escape sequence, and the text
begins immediately after that.Although setting the colors separately
also work (i.e. \033[44m\033[32m).
There are some differences between colors when combining colors with
bold text attribute.
The following table summarises these differences.
Bold off color Bold on color
0;30 Balck 1;30 Dark Gray
0;31 Red 1;31 Dark Red
0;32 Green 1;32 Dark Green
0;33 Brown 1;33 Yellow
0;34 Blue 1;34 Dark Blue
0;35 Magenta 1;35 Dark Magenta
0;36 Cyan 1;30 Dark Cyan
0;37 Light Gray 1;30 White
The following shell script prints all the colors and codes on the
screen.
#!/bin/bash
# This script echoes colors and codes
echo -e “\n\033[4;31mLight Colors\033[0m \t\t\033[1;4;31mDark Colors\033[0m”
echo -e “\e[0;30;47m Black \e[0m 0;30m \t\e[1;30;40m Dark Gray \e[0m 1;30m”
echo -e “\e[0;31;47m Red \e[0m 0;31m \t\e[1;31;40m Dark Red \e[0m 1;31m”
echo -e “\e[0;32;47m Green \e[0m 0;32m \t\e[1;32;40m Dark Green \e[0m 1;32m”
echo -e “\e[0;33;47m Brown \e[0m 0;33m \t\e[1;33;40m Yellow \e[0m 1;33m”
echo -e “\e[0;34;47m Blue \e[0m 0;34m \t\e[1;34;40m Dark Blue \e[0m 1;34m”
echo -e “\e[0;35;47m Magenta \e[0m 0;35m \t\e[1;35;40m DarkMagenta\e[0m 1;35m”
echo -e “\e[0;36;47m Cyan \e[0m 0;36m \t\e[1;36;40m Dark Cyan \e[0m 1;36m”
echo -e “\e[0;37;47m LightGray\e[0m 0;37m \t\e[1;37;40m White \e[0m 1;37m”
OUTPUT:
Some examples:
Block background and white text
echo -e “\033[40;37m Hello World\033[0m”
Reverse video text attribute option interchanges fg and bg colors.
Bellow statement prints block on white
echo -e “\033[40;37;7m Hello World\033[0m”
echo -e “\033[33;44m Yellow text on blue background\033[0m”
echo -e “\033[1;33;44m Bold yellow text on blue background\033[0m”
echo -e “\033[1;4;33;44mBold yellow underlined text on blue background\033[0m”
The “tput” command:
Other than echo there is a command called tput using which we
can control the way the output is displayed on the screen.But it is
less flexible than ANSI escape sequences.
#!/bin/bash
echo “Hey world, \what’s up?”
Users who have been using Linux for awhile often learn that creating a basic script is a good way to run multiple, often-repeated commands. Adding a little color to scripts can additionally provide nice feedback. This can be done in a fairly straight-forward way by using the tput command.
A common way of doing this is to define the colors that tput can produce by putting them at the beginning of the bash script:
#!/bin/bash
# scriptname – description of script
# Text color variables
txtund=$(tput sgr 0 1) # Underline
txtbld=$(tput bold) # Bold
bldred=${txtbld}$(tput setaf 1) # red
bldblu=${txtbld}$(tput setaf 4) # blue
bldwht=${txtbld}$(tput setaf 7) # white
txtrst=$(tput sgr0) # Reset
info=${bldwht}*${txtrst} # Feedback
pass=${bldblu}*${txtrst}
warn=${bldred}*${txtrst}
ques=${bldblu}?${txtrst}
When writing new scripts using templates with these variables already defined can quicken the creation process and help keep scripts organized (my Bash Templates).
If just needing to use tput colors for specific instances this script can display the tput definitions and their corresponding possibilities:
#!/bin/bash
# tputcolors
echo
echo -e “$(tput bold) reg bld und tput-command-colors$(tput sgr0)”
for i in $(seq 1 7); do
echo ” $(tput setaf $i)Text$(tput sgr0) $(tput bold)$(tput setaf $i)Text$(tput sgr0) $(tput sgr 0 1)$(tput setaf $i)Text$(tput sgr0) \$(tput setaf $i)”
done
echo ‘ Bold $(tput bold)’
echo ‘ Underline $(tput sgr 0 1)’
echo ‘ Reset $(tput sgr0)’
echo
tput Command
The tput command is used to set terminal features. With tput you can set:
Move the cursor around the screen.
Get information about terminal.
Set colors (background and foreground).
Set bold mode.
Set reverse mode and much more.
Here is a sample code:
#!/bin/bash
# clear the screen
tput clear
# Move cursor to screen location X,Y (top left is 0,0)
tput cup 3 15
# Set a foreground colour using ANSI escape
tput setaf 3
echo “XYX Corp LTD.”
tput sgr0
tput cup 5 17
# Set reverse video mode
tput rev
echo “M A I N – M E N U”
tput sgr0
tput cup 7 15
echo “1. User Management”
tput cup 8 15
echo “2. Service Management”
tput cup 9 15
echo “3. Process Management”
tput cup 10 15
echo “4. Backup”
# Set bold mode
tput bold
tput cup 12 15
read -p “Enter your choice [1-4] ” choice
tput clear
tput sgr0
tput rc
===================================if-then=====================================================================================
if-then’ Statement in Action
check the exit status wither command executed successfully or not
[root@cluster1 shell]# mkdir test
[root@cluster1 shell]# echo $?
0
[root@cluster1 shell]# mkdir test
mkdir: cannot create directory `test’: File exists
[root@cluster1 shell]# echo $?
1
[root@cluster1 shell]#
What Are Conditions?
Conditions in the realm of computing work similarly. We can test whether a string matches another string, whether it doesn’t match another string, or even if it exists at all.
Similarly, we can test numerical arguments to see if one is great than, less than, or equal to another.
To get something to happen after the conditions of the test are met, we use “if-then” statements. Their format is pretty simple.
if CONDITION
then
command1
command2
…
commandn
fi
if test $1 -gt $2
then
echo “$1 is greater than $2?
fi
if…fi statement is the fundamental control statement that allows Shell to make decisions and execute statements conditionally.
Syntax:
if [ expression ]
then
Statement(s) to be executed if expression is true
fi
Here Shell expression is evaluated. If the resulting value is true, given statement(s) are executed. If expression is false then no statement would be not executed. Most of the times you will use comparison operators while making decisions.
Give you attention on the spaces between braces and expression. This space is mandatory otherwise you would get syntax error.
If expression is a shell command then it would be assumed true if it return 0 after its execution. If it is a boolean expression then it would be true if it returns true.
Example:
#!/bin/sh
a=10
b=20
if [ $a == $b ]
then
echo “a is equal to b”
fi
if [ $a != $b ]
then
echo “a is not equal to b”
fi
This will produce following result:
a is not equal to b
Here are some other numerical operators you may want to try out:
-eq: equal to
-ne: not equal to
-lt: less than
-le: less than or equal to
-gt: greater than
-ge: greater than or equal to
Shell scripts use fairly standard syntax for if statements. The conditional statement is executed using either the test command or the [ command. In its most basic form an if statement is:
#!/bin/bash
if [ “$#” -gt 0 ]
then
echo “There’s Beans”
fi
if [ “$1” = “cool” ]
then
echo “Cool Beans”
fi
(Notice that the fi is simply if spelled backwards). To add an else, we just use standard syntax.
#!/bin/bash
if [ “$1” = “cool” ]
then
echo “Cool Beans”
else
echo “Not Cool Beans”
fi
Adding an else-if statement structure is used with the elif command.
#!/bin/bash
if [ “$1” = “cool” ]
then
echo “Cool Beans”
elif [ “$1” = “neat” ]
then
echo “Neato cool”
else
echo “Not Cool Beans”
fi
An if statement does not require two parameters. You can use single flags as well. The following code tests to see if the first parameter is a file or not.
#!/bin/bash
if [ -f “$1” ]
then
echo “$1 is a file”
else
echo “$1 is not a file”
fi
There are many different ways that an conditional statement can be used. These are summarized here:
String Comparison Description
Str1 = Str2 Returns true if the strings are equal
Str1 != Str2 Returns true if the strings are not equal
-n Str1 Returns true if the string is not null
-z Str1 Returns true if the string is null
Numeric Comparison Description
expr1 -eq expr2 Returns true if the expressions are equal
expr1 -ne expr2 Returns true if the expressions are not equal
expr1 -gt expr2 Returns true if expr1 is greater than expr2
expr1 -ge expr2 Returns true if expr1 is greater than or equal to expr2
expr1 -lt expr2 Returns true if expr1 is less than expr2
expr1 -le expr2 Returns true if expr1 is less than or equal to expr2
! expr1 Negates the result of the expression
File Conditionals Description
-d file True if the file is a directory
-e file True if the file exists (note that this is not particularly portable, thus -f is generally used)
-f file True if the provided string is a file
-g file True if the group id is set on a file
-r file True if the file is readable
-s file True if the file has a non-zero size
-u True if the user id is set on a file
-w True if the file is writable
-x True if the file is an executable
File attributes comparisons
-a file
True if file exists.
Example
[ -a /etc/resolv.conf ] && echo “File found” || echo “Not found”
-b file
True if file exists and is a block special file.
Example
[ -b /dev/zero ] && echo “block special file found” || echo “block special file not found”
OR
[ -b /dev/sda ] && echo “block special file found” || echo “block special file not found”
-c file
True if file exists and is a character special file.
Example
[ -c /dev/tty0 ] && echo “Character special file found.” || echo “Character special file not found.”
-d dir
True if file exists and is a directory.
Example
#!/bin/bash
DEST=”/backup”
SRC=”/home”
# Make sure backup dir exits
[ ! -d “$DEST” ] && mkdir -p “$DEST”
# If source directory does not exits, die…
[ ! -d “$SRC” ] && { echo “$SRC directory not found. Cannot make backup to $DEST”; exit 1; }
# Okay, dump backup using tar
echo “Backup directory $DEST…”
echo “Source directory $SRC…”
/bin/tar zcf “$DEST/backup.tar.gz” “$SRC” 2>/dev/null
# Find out if our backup job failed or not and notify on screen
[ $? -eq 0 ] && echo “Backup done!” || echo “Backup failed”
-e file
True if file exists.
Example
[ -e /tmp/test.txt ] && echo “File found” || echo “File not found”
-f file
True if file exists and is a regular file.
Example
[ ! -f /path/to/file ] && echo “File not found!”
A sample shell script that compare various file attributes and create webalizer (application that generates web pages of analysis, from access and usage log) stats configuration file to given Internet domain name.
#!/bin/bash
# Purpose: A Shell Script To Create Webalizer Stats Configration File
# Written by: Vivek Gite
# ———————————————————————
# Set vars
# Apache vroot for each domain
HTTPDROOT=”/home/httpd”
# Path to GeoIP DB
GEOIPDBPATH=”/usr/local/share/GeoIP/GeoIP.dat”
# Get the Internet domain such as cyberciti.biz
echo “*** A Shell Script To Create Webalizer Stats Configration File ***”
read -p “Enter a domain name : ” DOMAIN
# Make sure we got the Input else die with an error on screen
[ -z $DOMAIN ] && { echo “Please enter a domain name. Try again!”; exit 1; }
# Alright, set some variable based upon $DOMAIN
OUT=”$HTTPDROOT/$DOMAIN/stats/webalizer.conf”
CONFROOT=”$HTTPDROOT/$DOMAIN/stats”
LOGFILE=”$HTTPDROOT/$DOMAIN/logs/access.log”
# Die if configuration file exits…
[ -f $OUT ] && { echo “Webalizer configuration file ‘$OUT’ exits for domain $DOMAIN.”; exit 2; }
# Make sure configuration directory exists
[ ! -d $CONFROOT ] && mkdir -p $CONFROOT
# Write a log file
>$OUT
echo “LogFile $LOGFILE” >> $OUT
echo “LogType clf” >> $OUT
echo “OutputDir $CONFROOT/out” >> $OUT
echo “HistoryName $CONFROOT/webalizer.hist” >> $OUT
echo “Incremental yes” >> $OUT
echo “IncrementalName $CONFROOT/webalizer.current” >> $OUT
echo “HostName $DOMAIN” >> $OUT
echo “Quiet yes” >> $OUT
echo “FoldSeqErr yes” >> $OUT
echo “AllSearchStr yes” >> $OUT
echo “HideSite $DOMAIN” >> $OUT
echo “HideSite localhost” >> $OUT
echo “HideReferrer $DOMAIN” >> $OUT
echo “HideURL *.gif” >> $OUT
echo “HideURL *.GIF” >> $OUT
echo “HideURL *.jpg” >> $OUT
echo “HideURL *.JPG” >> $OUT
echo “HideURL *.png” >> $OUT
echo “HideURL *.PNG” >> $OUT
echo “HideURL *.ra” >> $OUT
echo “GroupReferrer yahoo.com/ Yahoo!” >> $OUT
echo “GroupReferrer excite.com/ Excite” >> $OUT
echo “GroupReferrer infoseek.com/ InfoSeek” >> $OUT
echo “GroupReferrer webcrawler.com/ WebCrawler” >> $OUT
echo “SearchEngine .yahoo. p=” >> $OUT
echo “SearchEngine altavista.com q=” >> $OUT
echo “SearchEngine .google. q=” >> $OUT
echo “SearchEngine eureka.com q=” >> $OUT
echo “SearchEngine lycos.com query=” >> $OUT
echo “SearchEngine hotbot.com MT=” >> $OUT
echo “SearchEngine msn.com MT=” >> $OUT
echo “SearchEngine infoseek.com qt=” >> $OUT
echo “SearchEngine webcrawler searchText=” >> $OUT
echo “SearchEngine excite search=” >> $OUT
echo “SearchEngine netscape.com search=” >> $OUT
echo “SearchEngine mamma.com query=” >> $OUT
echo “SearchEngine alltheweb.com query=” >> $OUT
echo “SearchEngine northernlight.com qr=” >> $OUT
echo “CountryFlags yes” >> $OUT
echo “GeoIP yes” >> $OUT
echo “GeoIPDatabase $GEOIPDBPATH” >> $OUT
echo “GraphMonths 72” >> $OUT
echo “IndexMonths 120” >> $OUT
echo “GraphMonths 72” >> $OUT
echo “TopReferrers 20” >> $OUT
echo “TopSites 20” >> $OUT
echo “TopURLs 50” >> $OUT
echo “TopKURLs 50” >> $OUT
echo “Weblizer config wrote to $OUT”
-g file
True if file exists and is set-group-id.
-h file
True if file exists and is a symbolic link.
-k file
True if file exists and its ‘‘sticky’’ bit is set.
-p file
True if file exists and is a named pipe (FIFO).
-r file
True if file exists and is readable.
-s file
True if file exists and has a size greater than zero.
-t fd
True if file descriptor fd is open and refers to a terminal.
-u file
True if file exists and its set-user-id bit is set.
-w file
True if file exists and is writable.
-x file
True if file exists and is executable.
-O file
True if file exists and is owned by the effective user id.
-G file
True if file exists and is owned by the effective group id.
-L file
True if file exists and is a symbolic link.
-S file
True if file exists and is a socket.
-N file
True if file exists and has been modified since it was last read.
In its most basic form an if statement is:
#!/bin/bash if [ “$1” -eq “abc” ] then echo “in if block” fi
(Notice that the fi is simply if spelled backwards).
To add an else, we just use standard syntax.
#!/bin/bash if [ “$1” -eq “abc” ] then echo “in if block” else echo “in else block” fi
Adding an else-if statement structure is used with the elif command.
#!/bin/bash if [ “$1” -eq “abc” ] then echo “in if block” elif [ “$1” -eq “xyz” ] echo “in else if block” else echo “in else block” fi
An if statement does not require two parameters.
You can use single flags as well. The following code tests to see if the first parameter is a file or not.
#!/bin/bash if [ -f “$1” ] then echo “$1 is a file” else echo “$1 is not a file” fi
#!/bin/bash
#if then statement
echo “Enter source and target files names.”
read source target
if mv $source $target
then
echo “Your file has been successully renamed..”
fi
Enter source and target files names.
test test1
Your file has been successully renamed..
[root@cluster1 shell]# ls -ltr
if…else…fi statement is the next form of control statement that allows Shell to execute statements in more controlled way and making decision between two choices.
Syntax:
if [ expression ]
then
Statement(s) to be executed if expression is true
else
Statement(s) to be executed if expression is not true
fi
Here Shell expression is evaluated. If the resulting value is true, given statement(s) are executed. If expression is false then no statement would be not executed.
then
command executed successfully
execute all commands up to else statement
or to fi if there is no else statement
else
command failed so
execute all commands up to fi
fi
OR
if test var -eq val
then
command executed successfully
execute all commands up to else statement
or to fi if there is no else statement
else
if command failed then
execute all commands up to fi
fi
OR
if [ condition ]
then
if given condition true
execute all commands up to else statement
or to fi if there is no else statement
else
if given condition false
execute all commands up to fi
fi
if [ condition ]
then
< code block >
fi
For example – If we need to check if input value is equal to 10 or not. If value is equal to 10, then it will print “Value of i is 10?, but if not nothing will be printed.
#!/bin/bash
read -p “Enter value of i :” i
if [ $i -eq 10 ]
then
echo “Value of i is 10”
fi
2. Bash if else Statement
if else statements are useful where we have a two programs for execution, and need to execute only one based on results of if condition.
if [ condition ]
then
< code block >
else
< code block >
fi
For example – If input taken value is 10 then it will print “Value of i is 10?, if not program will execute else block statement and print “Value of i is not equal to 10?.
#!/bin/bash
read -p “Enter value of i :” i
if [ $i -eq 10 ]
then
echo “Value of i is 10”
else
echo “Value of i is not equal to 10”
fi
3. Bash if elif Statement
if elif and else statements are useful where we have more than two programs for execution, and need to execute only one based on results of if and elif condition.
if [ condition ]
then
< code block >
elif [ condition ]
then
< code block >
else
< code block >
fi
For example – Below example will check input value if equal to 5, if its true then program will print “Value of i is 5? otherwise program will go to elif statement where one more conditional will be checked and based of these results elif or elase block code will be executed.
#!/bin/bash
read -p “Enter value of i :” i
if [ $i -eq 5 ]
then
echo “Value of i is 5”
elif [ $i -eq 10 ]
then
echo “Value of i is 10”
else
echo “Value of i is not equal to 5 or 10”
fi
4. Bash elif Ladder Statements
This is something similar to above one where we are adding multiple elif statements together. elif (else if) ladder are useful where we have multiple programs for execution and need to execute only one based on results of if and elif condition.
if [ condition ]
then
< code block >
elif [ condition ]
then
< code block >
elif [ condition ]
then
< code block >
elif [ condition ]
then
< code block >
else
< code block >
fi
For example –
#!/bin/bash
read -p “Enter value of i :” i
if [ $i -eq 5 ]
then
echo “Value of i is 5”
elif [ $i -eq 10 ]
then
echo “Value of i is 10”
elif [ $i -eq 20 ]
then
echo “Value of i is 20”
elif [ $i -eq 30 ]
then
echo “Value of i is 30”
else
echo “Value of i is not equal to 5,10,20 or 30”
fi
5. Bash nested if Statements
Nested if are useful in the situation where one condition will be checked based on results of outer condition.
if [ condition ]
then
if [ condition ]
then
< code block >
else
< code block >
fi
else
if [ condition ]
then
< code block >
fi
fi
For example below small shell program is for finding the greatest value between 3 values taken input by user. This program will work with numeric values only. If two values are similar it will print only one value.
#!/bin/bash
read -p “Enter value of i :” i
read -p “Enter value of j :” j
read -p “Enter value of k :” k
if [ $i -gt $j ]
then
if [ $i -gt $k ]
then
echo “i is greatest”
else
echo “k is greatest”
fi
else
if [ $j -gt $k ]
then
echo “j is greatest”
else
echo “k is greatest”
fi
fi
Nested ifs
if condition
then
if condition
then
…..
..
do this
else
….
..
do this
fi
else
…
…..
do this
fi
Number Testing Script
Create a shell script called testnum.sh:
#!/bin/bash
read -p “Enter number : ” n
if test $n -ge 0
then
echo “$n is positive number.”
else
echo “$n number is negative number.”
fi
!/bin/bash
#if then statement
echo “Enter source and target files names.”
read source target
if mv $source $target
then
echo “Your file has been successully renamed..”
else
echo “Your file is not been moved…”
fi
#!/bin/bash
read -p “Enter first Number:” n1
read -p “Enter second Number:” n2
read -p “Enter third Number:” n3
read -p “Enter fourth Number:” n4
if((n1>n2)) ; then
if((n1>n3)); then
if((n1>n4)); then
echo “$n1 is a Greatest Number”
else
echo “$n4 is a Greatest Number”
fi
elif((n3>n4)); then
echo “$n3 is a Greatest Number”
else
echo “$n4 is a Greatest Number”
fi
elif((n2>n3)); then
if((n2>n4)); then
echo “$n2 is a Greatest Number”
else
echo “$n4 is a Greatest Number”
fi
elif((n3>n4)); then
echo “$n3 is a Greatest Number”
else
echo “$n4 is a Greatest Number”
fi
========================================Exit status of a command Parameters Set by the Shell==================================================================
exit status of a command Parameters Set by the Shell
Bash shell set several special parameters. For example $? (see return values section) holds the return value of the executed command.
All command line parameters or arguments can be accessed via $1, $2, $3,…, $9.
$* holds all command line parameters or arguments.
$# holds the number of positional parameters.
$- holds flags supplied to the shell.
$? holds the return value set by the previously executed command.
$$ holds the process number of the shell (current shell).
$! hold the process number of the last background command.
$@ holds all command line parameters or arguments.
Use echo command to display special shell parameters:
echo $#
You can store them to a shell variables as follows:
status=$?
[ $status -eq 0 ] && echo “Lighttpd … [Ok]” || echo “Lighttpd … [Failed]”
Assignment to special parameter is not allowed:
# okay
status=$?
# noop not allowed
$?=-1
Exit Status
Every Linux command executed by the shell script or user, has an exit status.
The exit status is an integer number.
The Linux man pages stats the exit statuses of each command.
0 exit status means the command was successful without any errors.
A non-zero (1-255 values) exit status means command was failure.
You can use special shell variable called ? to get the exit status of the previously executed command. To print ? variable use the echo command:
echo $?
date # run date command
echo $? # print exit status
foobar123 # not a valid command
echo $? # print exit status
How Do I See Exit Status Of The Command?
Type the following command:
date
To view exist status of date command, enter:
echo $?
Sample Output:
0
Try non-existence command
date1
echo $?
ls /eeteec
echo $?
Sample Output:
2
According to ls man page – exit status is 0 if OK, 1 if minor problems, 2 if serious trouble.
How Do I Store Exit Status Of The Command In a Shell Variable?
Assign $? to a shell variable:
ls -l /tmp
status=$?
echo “ls command exit stats – $status”
Exit Status Shell Script Example
A simple shell script to locate username (finduser.sh)
#!/bin/bash
# set var
PASSWD_FILE=/etc/passwd
# get user name
read -p “Enter a user name : ” username
# try to locate username in in /etc/passwd
grep “^$username” $PASSWD_FILE > /dev/null
# store exit status of grep
# if found grep will return 0 exit stauts
# if not found, grep will return a nonzero exit stauts
status=$?
if test $status -eq 0
then
echo “User ‘$username’ found in $PASSWD_FILE file.”
else
echo “User ‘$username’ not found in $PASSWD_FILE file.”
fi
Save and close the file. Run it as follows:
chmod +x finduser.sh
./finduser.sh
Sample Outputs:
Enter a user name : vivek
User ‘vivek’ found in /etc/passwd file.
Run it again:
chmod +x finduser.sh
./finduser.sh
Sample Outputs:
Enter a user name : tommy
User ‘tommy’ not found in /etc/passwd file.
You can combine the grep and if command in a single statement as follows:
if grep “^$username:” /etc/passwd >/dev/null
then
echo “User ‘$username’ found in $PASSWD_FILE file.”
else
echo “User ‘$username’ not found in $PASSWD_FILE file.”
fi
#!/bin/bash
echo “Enter a number between 10 and 20 ”
read num
if [ $num -lt 10 ]
then
echo “That was under the limit you entered”
elif [ $num -gt 20 ]
then
echo “That over the limit you entered”
else
echo “Now your are over the limit”
fi
#!/bin/bash
echo “Enter file name :\c”
read fname
if [ -f $fname ]
then
if [ -w $fname ]
then
echo “Type matter to append. To quit press ctrl+d ”
cat >> $fname
else
echo “You do not have permission to write. ”
fi
fi
#!/bin/sh
# Prompt for a user name…
echo “Please enter your name:”
read USERNAME
# Check for the file.
if [ -s ${USERNAME}_DAT ]; then
# Read the age from the file.
AGE=`cat ${USERNAME}_DAT`
echo “You are $AGE years old!”
else
# Ask the user for his/her age
echo “How old are you?”
read AGE
if [ “$AGE” -le 2 ]; then
echo “You are too young!”
else
if [ “$AGE” -ge 100 ]; then
echo “You are too old!”
else
# Write the age to a new file.
echo $AGE > ${USERNAME}_DAT
fi
fi
fi
The syntax is as follows:
exit N
The exit statement is used to exit from the shell script with a status of N.
Use the exit statement to indicate successful or unsuccessful shell script termination.
The value of N can be used by other commands or shell scripts to take their own action.
If N is omitted, the exit status is that of the last command executed.
Use the exit statement to terminate shell script upon an error.
If N is set to 0 means normal shell exit. Create a shell script called exitcmd.sh:
#!/bin/bash
echo “This is a test.”
# Terminate our shell script with success message
exit 0
Save and close the file. Run it as follows:
chmod +x exitcmd.sh
./exitcmd.sh
Sample outputs:
This is a test.
To see exit status of the script, enter (see the exit status of a command for more information about special shell variable $?) :
echo $?
Shell script example
Any non zero value indicates unsuccessful shell script termination.
Create a shell script called datatapebackup.sh:
#!/bin/bash
BAK=/data2
TAPE=/dev/st0
echo “Trying to backup ${BAK} directory to tape device ${TAPE} ..”
# See if $BAK exits or not else die
# Set unsuccessful shell script termination with exit status # 1
[ ! -d $BAK ] && { echo “Source backup directory $BAK not found.”; exit 1; }
# See if $TAPE device exits or not else die
# Set unsuccessful shell script termination with exit status # 2
[ ! -b $TAPE ] && { echo “Backup tape drive $TAPE not found or configured.”; exit 2; }
# Okay back it up
tar cvf $TAPE $BAK 2> /tmp/error.log
if [ $? -ne 0 ]
then
# die with unsuccessful shell script termination exit status # 3
echo “An error occurred while making a tape backup, see /tmp/error.log file”.
exit 3
fi
# Terminate our shell script with success message i.e. backup done!
exit 0
Save and close the file. Run it as follows:
chmod +x datatapebackup.sh
./datatapebackup.sh
echo $?
=============================================================================================
String Comparison Operators in Shell Script
OPERATOR DESCRIPTION
str1 = str2 True if str1 and str2 are identical
str1 != str2 True if str1 and str2 are not identical
-n str1 True if str1 is not null (size is greater than zero)
-z str1 True if str1 is null
$ cat > strtest.sh
#!/bin/bash
a=’mohan’ b=’mohan’ c=’server’
#null string
d=’ ‘
if [ $a = $b ] ; then
echo “a and b are identical”
fi
if [ $a != $c ] ; then
echo “a and c are not identical” fi
if [ -z $d ] ; then
echo “d is a null string”
fi
Sourcesystem=”ABC”
if [ “$Sourcesystem” = “XYZ” ]; then
echo “Sourcesystem Matched”
else
echo “Sourcesystem is NOT Matched $Sourcesystem”
fi;
if [ ‘XYZ’ == ‘ABC’ ]; then # Double equal to will work in Linux but not on HPUX boxes it should be if [ ‘XYZ’ = ‘ABC’ ] which will work on both
echo “Match”
else
echo “No Match”
fi
if [ “$var” == “” ] then
echo variable is null
fi
To determine if the value of a variable is not empty:
if [ “$var” != “” ] then
echo variable is not null
fi
To compare the contents of a variable to a fixed string:
if [ “$var” == “value” ] then
echo is the same
fi
To determine if variable’s contents are not equal to a fixed string:
if [ “$var” != “value” ] then
echo not the same
fi
Empty string in Bash
In Bash you quite often need to check to see if a variable has been set or has a value other than an empty string. This can be done using the -n or -z string comparison operators.
The -n operator checks whether the string is not null. Effectively, this will return true for every case except where the string contains no characters. ie:
VAR=”hello”
if [ -n “$VAR” ]; then
echo “VAR is not empty”
fi
Similarly, the -z operator checks whether the string is null. ie:
VAR=””
if [ -z “$VAR” ]; then
echo “VAR is empty”
fi
The following program shows the comparison of two strings in shell script
#!/bin/sh
str1=”happy”
str2=”happy”
[ $str1 = $str2 ]
echo $?
Case Insensitive comparision of strings in Shell script
str1=”MATCH”
str2=”match”
shopt -s nocasematch
case “$str1” in
$str2 ) echo “match”;;
*) echo “no match”;;
esac
var1=match
var2=MATCH
if echo $var1 | grep -i “^${var2}$” > /dev/null ; then
echo “MATCH”
fi
I want to compare 2 strings with ignore case in bash shell.
Suppose, if [ “test” = “TEst” ] … should be true and enter into if loop.
Solution:
Please try this:
if [ `echo “TeSt” | tr -s ‘[:upper:]’ ‘[:lower:]` = `echo “Test” | tr -s ‘[:upper:]’ ‘[:lower:]` ]
then
echo “true”
else
echo “false”
fi
or
Try similar, it still worked.
input=TESt
input=`echo $input | tr ‘[A-Z]’ ‘[a-z]’`
if [[ “$input” =~ “test” ]]; then
echo “equal”
else
echo “not equal”
fi
#!/bin/bash
str1=”Hey You!”
str2=”What’s up?”
str3=””
[ “$str1” = “$str2” ]
echo $?
[ “$str1” != “$str2” ]
echo $?
[ -n “$str1” ]
echo $?
[ -z “$str3” ]
echo $?
=================================================================================================
The ‘AND’ Logical Operator
Logical and (&&) is boolean operator. It can execute commands or shell functions based on the exit status of another command.
command1 && command2
OR
First_command && Second_command
Boolean AND:
&& or -a used to do boolean AND operations in bash/shell scripts.
To validate boolean AND condition there are two ways:
if [ $condition1 ] && [ $condition2 ]
You can also validate as
if [ $condition1 -a $condition2 ]
Type the following at a shell prompt:
rm /tmp/filename && echo “File deleted.”
Lookup a username in /etc/passwd file
grep “^vivek” /etc/passwd && echo “Vivek found in /etc/passwd”
Exit if a directory /tmp/foo does not exist
test ! -d /tmp/foo && { read -p “Directory /tmp/foo not found. Hit [Enter] to exit…” enter; exit 1; }
#!/bin/bash
echo “Enter a number between 50 and 100”
read num
if [ $num -le 100 -a $num -ge 50 ]
then
echo “You are within limits.”
else
echo “Your are out of limits.”
fi
You can test multiple expressions at once by using the || (or) operator or the && (and) operator. This can save you from writing extra code to nest if statements. The above code has a nested if statement where it checks if the age is greater than or equal to 100. This could be changed as well by using elif (else if). The structure of elif is the same as the structure of if, we will use it in an example below. In this example, we will check for certain age ranges. If you are less than 20 or greater than 50, you are out of the age range. If you are between 20 and 30 you are in your 20’s and so on.
#!/bin/sh
# Prompt for a user name…
echo “Please enter your age:”
read AGE
if [ “$AGE” -lt 20 ] || [ “$AGE” -ge 50 ]; then
echo “Sorry, you are out of the age range.”
elif [ “$AGE” -ge 20 ] && [ “$AGE” -lt 30 ]; then
echo “You are in your 20s”
elif [ “$AGE” -ge 30 ] && [ “$AGE” -lt 40 ]; then
echo “You are in your 30s”
elif [ “$AGE” -ge 40 ] && [ “$AGE” -lt 50 ]; then
echo “You are in your 40s”
fi
Count The Number of Characters in User’s Input in Your Script
#!/bin/bash
echo “Enter a charcter ”
read var
if [ ‘echo $var | wc -c ` -eq 2 ]
then
echo “You entered a character. ”
else
echo “Invalid Input.”
fi
===========================================================================================
Logical OR
Logical OR (||) is boolean operator. It can execute commands or shell functions based on the exit status of another command.
Syntax
command1 || command2
OR
First_command || Second_command
command2 is executed if, and only if, command1 returns a non-zero exit status. In other words, run command1 successfully or run command2.
Boolean OR:
|| or -o used to do boolean OR operations in bash/shell scripts.
To validate boolean OR there are 2 ways:
if [ $condition1 ] || [ $condition2 ]
You can also validate boolean OR as
if [ $condition1 -o $condition2 ]
Example
cat /etc/shadow 2>/dev/null || echo “Failed to open file”
The cat command will try to display /etc/shadow file and it (the cat command) sets the exit stats to non-zero value if it failed to open /etc/shadow file. Therefore, ‘Failed to open file’ will be displayed cat command failed to open the file.
Find username else display an error
grep “^vivek” /etc/passwd || echo “User vivek not found in /etc/passwd”
How Do I Combine Both Logical Operators?
Try it as follows:
cat /etc/shadow 2>/dev/null && echo “File successfully opened.” || echo “Failed to open file.”
Make sure only root can run this script:
test $(id -u) -eq 0 && echo “You are root” || echo “You are NOT root”
OR
test $(id -u) -eq 0 && echo “Root user can run this script.” || echo “Use sudo or su to become a root user.”
=============================================================================================================================================================
Logical Not !
Logical not (!) is boolean operator, which is used to test whether expression is true or not. For example, if file not exists, then display an error on screen.
Syntax
The test command syntax is as follows:
! expression
OR
[ ! expression ]
OR
if test ! condition
then
command1
command2
fi
if [ ! condition ]
then
command1
command2
fi
Where,
True if expression is false.
Examples
Try the following example:
test ! -f /etc/resolv.conf && echo “File /etc/resolv.conf not found.”
OR
test ! -f /etc/resolv.conf && echo “File /etc/resolv.conf not found.” || echo “File /etc/resolv.conf found.”
Create a directory /backup, if doesn’t exits:
[ ! -d /backup ] && mkdir /backup
Die (exit) if $HOME/.config file not found:
[ ! -f $HOME/.config ] && { echo “Error: $HOME/.config file not found.”; exit 1; }
Die (exit) if directory /usr/bin not found
[ ! -d /usr/bin ] && exit
================================================================================================================
count
#!/bin/bash
BEGIN=1 # Start counting here
END=10 # Stop counting here
END=`expr $END + 1`
while [ $END -ne $BEGIN ]; do # While END is not equal to BEGIN do …
echo This is iteration $BEGIN
BEGIN=`expr $BEGIN + 1` # Increasing the value of BEGIN by one
done
========================================================================================================
Conditional expression
The test command is used to check file types and compare values. You can also use [ as test command. It is used for:
File attributes comparisons
Perform string comparisons.
Arithmetic comparisons.
Syntax
[ condition ]
OR
[ ! condition ]
OR
[ condition ] && true-command
OR
[ condition ] || false-command
OR
[ condition ] && true-command || false-command
Examples
[ 5 == 5 ] && echo “Yes” || echo “No”
[ 5 == 15 ] && echo “Yes” || echo “No”
[ 5 != 10 ] && echo “Yes” || echo “No”
[ -f /etc/resolv.conf ] && echo “File /etc/resolv.conf found.” || echo “File /etc/resolv.conf not found.”
[ -f /etc/resolv1.conf ] && echo “File /etc/resolv.conf found.” || echo “File /etc/resolv.conf not found.”
====================================================================================================
The ‘case’ Statement
he case statement is good alternative to multilevel if-then-else-fi statement. It enable you to match several values against one variable. It is easier to read and write.
Syntax
The syntax is as follows:
case $variable-name in
pattern1)
command1
…
….
commandN
;;
pattern2)
command1
…
….
commandN
;;
patternN)
command1
…
….
commandN
;;
*)
esac
OR
case $variable-name in
pattern1|pattern2|pattern3)
command1
…
….
commandN
;;
pattern4|pattern5|pattern6)
command1
…
….
commandN
;;
pattern7|pattern8|patternN)
command1
…
….
commandN
;;
*)
esac
The case statement allows you to easily check pattern (conditions) and then process a command-line if that condition evaluates to true.
In other words the $variable-name is compared against the patterns until a match is found.
*) acts as default and it is executed if no match is found.
The pattern can include wildcards.
You must include ;; at the end of each commandN. The shell executes all the statements up to the two semicolons that are next to each other.
The esac is always required to indicate end of case statement.
Example
Create a shell script called rental.sh:
#!/bin/bash
# if no command line arg given
# set rental to Unknown
if [ -z $1 ]
then
rental=”*** Unknown vehicle ***”
elif [ -n $1 ]
then
# otherwise make first arg as a rental
rental=$1
fi
# use case statement to make decision for rental
case $rental in
“car”) echo “For $rental rental is Rs.20 per k/m.”;;
“van”) echo “For $rental rental is Rs.10 per k/m.”;;
“jeep”) echo “For $rental rental is Rs.5 per k/m.”;;
“bicycle”) echo “For $rental rental 20 paisa per k/m.”;;
“enfield”) echo “For $rental rental Rs.3 per k/m.”;;
“thunderbird”) echo “For $rental rental Rs.5 per k/m.”;;
*) echo “Sorry, I can not get a $rental rental for you!”;;
esac
Save and close the file. Run it as follows:
chmod +x rental.sh
./rental.sh
./rental.sh jeep
./rental.sh enfield
./rental.sh bike
Sample outputs:
Sorry, I can not get a *** Unknown vehicle *** rental for you!
For jeep rental is Rs.5 per k/m.
For enfield rental Rs.3 per k/m.
Sorry, I can not get a bike rental for you!
The case statement first checks $rental against each option for a match. If it matches “car”, the echo command will display rental for car. If it matches “van”, the echo command will display rental for van and so on. If it matches nothing i.e. * (default option), an appropriate warning message is printed.
Using Multiple Patterns
#!/bin/bash
NOW=$(date +”%a”)
case $NOW in
Mon)
echo “Full backup”;;
Tue|Wed|Thu|Fri)
echo “Partial backup”;;
Sat|Sun)
echo “No backup”;;
*) ;;
esac
The following shell script demonstrate the concept of command line parameters processing using the case statement (casecmdargs.sh):
#!/bin/bash
OPT=$1 # option
FILE=$2 # filename
# test -e and -E command line args matching
case $OPT in
-e|-E)
echo “Editing $2 file…”
# make sure filename is passed else an error displayed
[ -z $FILE ] && { echo “File name missing”; exit 1; } || vi $FILE
;;
-c|-C)
echo “Displaying $2 file…”
[ -z $FILE ] && { echo “File name missing”; exit 1; } || cat $FILE
;;
-d|-D)
echo “Today is $(date)”
;;
*)
echo “Bad argument!”
echo “Usage: $0 -ecd filename”
echo ” -e file : Edit file.”
echo ” -c file : Display file.”
echo ” -d : Display current date and time.”
;;
esac
Run it as follows:
chmod +x casecmdargs.sh
./casecmdargs.sh
./casecmdargs.sh -e /tmp/file
./casecmdargs.sh -E /tmp/file
./casecmdargs.sh -e
./casecmdargs.sh -D
Creating a backup script
Create a backup script called allinonebackup.sh:
#!/bin/bash
# A shell script to backup mysql, webserver and files to tape
opt=$1
case $opt in
sql)
echo “Running mysql backup using mysqldump tool…”
;;
sync)
echo “Running backup using rsync tool…”
;;
tar)
echo “Running tape backup using tar tool…”
;;
*)
echo “Backup shell script utility”
echo “Usage: $0 {sql|sync|tar}”
echo ” sql : Run mySQL backup utility.”
echo ” sync : Run web server backup utility.”
echo ” tar : Run tape backup utility.” ;;
esac
Save and close the file. Run it as follows:
chmod +x allinonebackup.sh
# run sql backup
./allinonebackup.sh sql
# Dump file system using tape device
./allinonebackup.sh tar
# however, the following will fail as patterns are case sensitive
# you must use command line argument tar and not TAR, Tar, TaR etc.
./allinonebackup.sh TAR
#!/bin/sh
echo “Please talk to me …”
while :
do
read INPUT_STRING
case $INPUT_STRING in
hello)
echo “Hello yourself!”
;;
bye)
echo “See you again!”
break
;;
*)
echo “Sorry, I don’t understand”
;;
esac
done
echo
echo “That’s all folks!”
Okay, so it’s not the best conversationalist in the world; it’s only an example!
Try running it and check how it works…
$ ./talk.sh
Please talk to me …
hello
Hello yourself!
What do you think of politics?
Sorry, I don’t understand
bye
See you again!
That’s all folks!
#!/bin/bash
MENU=”
1 Date and Time
2 Calendar for current month
3 quit
”
while true; do
clear
echo “$MENU”
echo -n “Please make your choice: ”
read INPUT # Read user input and assign it to variable INPUT
case $INPUT in
1)
date
echo press ENTER to continue
read
;;
2)
cal
echo press ENTER to continue
read
;;
3|q|Q) # If user presses 3, q or Q we terminate
exit 0
;;
*) # All other user input results in an usage message
clear
echo Please choose alternatves 1, 2 or 3
sleep 2
;;
esac
done
!/bin/sh
FRUIT=”apple”
case “$FRUIT” in
“apple”) echo “Apple pie is quite tasty.”
;;
“banana”) echo “I like banana nut bread.”
;;
“kiwi”) echo “New Zealand is famous for kiwi.”
;;
esac
#!/bin/sh
echo “Enter a character”
read var
case $var in
[a-z])
echo “You entered a lower case alphabet.”
;;
[A-Z])
echo “You entered a upper case alphabet.”
;;
[0-9])
echo ” You entered a digit.”
;;
?)
echo ” You entered a speical sysmbol.”
;;
*)
echo ” You entered more than one character.”
;;
esac
root@cluster1 shell]# ./case.sh
Enter a character
a
You entered a lower case alphabet.
[root@cluster1 shell]# ./case.sh
Enter a character
A
You entered a lower case alphabet.
[root@cluster1 shell]# ./case.sh
Enter a character
asfhadad
=============================================================================================================================
=================================================================================================================================================
Quick Reference
This is a quick reference guide to the meaning of some of the less easily guessed commands and codes.
Command Description Example
& Run the previous command in the background ls &
&& Logical AND if [ “$foo” -ge “0” ] && [ “$foo” -le “9”]
|| Logical OR if [ “$foo” -lt “0” ] || [ “$foo” -gt “9” ] (not in Bourne shell)
^ Start of line grep “^foo”
$ End of line grep “foo$”
= String equality (cf. -eq) if [ “$foo” = “bar” ]
! Logical NOT if [ “$foo” != “bar” ]
$$ PID of current shell echo “my PID = $$”
$! PID of last background command ls & echo “PID of ls = $!”
$? exit status of last command ls ; echo “ls returned code $?”
$0 Name of current command (as called) echo “I am $0”
$1 Name of current command’s first parameter echo “My first argument is $1”
$9 Name of current command’s ninth parameter echo “My ninth argument is $9”
$@ All of current command’s parameters (preserving whitespace and quoting) echo “My arguments are $@”
$* All of current command’s parameters (not preserving whitespace and quoting) echo “My arguments are $*”
-eq Numeric Equality if [ “$foo” -eq “9” ]
-ne Numeric Inquality if [ “$foo” -ne “9” ]
-lt Less Than if [ “$foo” -lt “9” ]
-le Less Than or Equal if [ “$foo” -le “9” ]
-gt Greater Than if [ “$foo” -gt “9” ]
-ge Greater Than or Equal if [ “$foo” -ge “9” ]
-z String is zero length if [ -z “$foo” ]
-n String is not zero length if [ -n “$foo” ]
-nt Newer Than if [ “$file1” -nt “$file2” ]
-d Is a Directory if [ -d /bin ]
-f Is a File if [ -f /bin/ls ]
-r Is a readable file if [ -r /bin/ls ]
-w Is a writable file if [ -w /bin/ls ]
-x Is an executable file if [ -x /bin/ls ]
parenthesis:
( … ) Function definition function myfunc() { echo hello }
Quiz: ShellCheck is aware of many common usage problems. Are you?
find . -name *.mp3
sudo echo 3 > /proc/sys/vm/drop_caches
PS1=’\e[0;32m\$\e[0m ‘
find . | grep “*.mp3”
[ $n > 7 ]
[[ $n > 7 ]]
tr ‘A-Z’ ‘a-z’
cmd 2>&1 > log
array=(1, 2, 3)
echo $10
[[ $a=$b ]]
[[ $a = $b ]]
progress=$((i/total*100))
trap “echo \”Time used: $SECONDS\”” EXIT
find dir -exec cp {} /backup && rm {} \;
[[ $keep = [yY] ]] && mv file /backup || rm file
ShellCheck gives more helpful messages for many Bash syntax errors
Bash says ShellCheck points to the exact position and says
: command not found Literal carriage return. Run script through tr -d ‘\r’
unexpected token: `fi’ Can’t have empty then clauses (use ‘true’ as a no-op)
unexpected token `(‘ Shells are space sensitive. Use ‘< <(cmd)', not '<<(cmd)'
unexpected token `(‘ ‘(‘ is invalid here. Did you forget to escape it?
echo foo: command not found This is a . Delete it and retype as space.
ShellCheck suggests style improvements
Code ShellCheck suggestion
basename "$var" Use parameter expansion instead, such as ${var##*/}
ls | grep 'mp3$' Don’t use ls | grep. Use a glob or a for loop with a condition.
expr 3 + 2 Use $((..)), ${} or [[ ]] in place of antiquated expr.
cat foo | grep bar Useless cat. Consider ‘cmd < file | ..' or 'cmd file | ..' instead.
length=$(echo "$var" | wc -c") See if you can use ${#variable} instead
ShellCheck recognizes common but wrong attempts at doing things
Code ShellCheck tip
var$n=42 For indirection, use (associative) arrays or ‘read “var$n” <<< "value"'".
(Bash says “var3=42: command not found”)
${var$n} To expand via indirection, use name=”foo$n”; echo “${!name}”
(Bash says “bad substitution”. )
echo 'It\'s time' Are you trying to escape that single quote? echo ‘You’\”re doing it wrong’
(Bash says “unexpected end of file”)
[ grep a b ] Use ‘if cmd; then ..’ to check exit code, or ‘if [[ $(cmd) == .. ]]’ to check output
(Bash says “[: a: binary operator expected”)
var=grep a b To assign the output of a command, use var=$(cmd)
(Bash says “a: command not found”)
=================================================================================================================================================
Recent Comments