Bash Shortcuts and Tips

Repeating an argument
You can repeat the last argument of the previous command in multiple ways. Have a look at this example:

$ mkdir /path/to/dir
$ cd !$

The second command might look a little strange, but it will just cd to /path/to/dir.

Some keyboard shortcuts for editing

There are some pretty useful keyboard shortcuts for editing in bash. They might appear familiar to Emacs users:

• Ctrl + a => Return to the start of the command you’re typing
• Ctrl + e => Go to the end of the command you’re typing
• Ctrl + u => Cut everything before the cursor to a special clipboard
• Ctrl + k => Cut everything after the cursor to a special clipboard
• Ctrl + y => Paste from the special clipboard that Ctrl + u and Ctrl + k save their data to
• Ctrl + t => Swap the two characters before the cursor (you can actually use this to transport a character from the left to the right, try it!)
• Ctrl + w => Delete the word / argument left of the cursor
• Ctrl + l => Clear the screen

Redirecting both Standard Output and Standard Error:

# ls -ltR 2>&1 > /tmp/temp.txt

Specify this in .bashrc

Make Bash append rather than overwrite the history on disk:

# shopt -s histappend

Whenever displaying the prompt, write the previous line to disk:

# export PROMPT_COMMAND=’history -a’

To erase duplicate entries in History.

# export HISTCONTROL=erasedups
(or)
# export HISTCONTROL=ignoreboth

To see the history with timestamps

# export HISTTIMEFORMAT=”%d/%m/%Y-%H:%M:%S ”

To set the Size of the historyHISTSIZE: The number of commands to remember in the command history. The default value is 500.

# export HISTSIZE=500

Searching the Past

• Ctrl+R searches previous lines

This will put bash in history mode, allowing you to type a part of the command you’re looking for. In the meanwhile, it will show the most recent occasion where the string you’re typing was used. If it is showing you a too recent command, you can go further back in history by pressing Ctrl + r again and again. Once you found the command you were looking for, press enter to run it.

• !tcp will execute the previous command which starts with “tcp”

rsync examples from command line.

To copy all the files in /home/lokams/* of remote host to local directory /backup/home/lokams/
# rsync -e ssh -avz –delete –exclude dir/* –delete-excluded –stats user@remotehost:/home/lokams/ /backup/home/lokams/

To copy the directory /home/lokams of remote host to local directory /backup/home. Directory “lokams” will get created in /backup/home.
# rsync -e ssh -avz –delete –exclude-from=~/.rsync/excludeFile –delete-excluded –stats user@remotehost:/home/lokams /backup/home/

To ignore the permissions errors while copying to remote windows/mac systems, use “-rlt” option.
# rsync -e ssh -avz –delete –exclude dir/* –delete-excluded –stats -rlt user@remotehost:/home/lokams/ /backup/home/lokams/

rsync behaves differently if the source directory has a trailing slash. Study and learn the difference between the following two commands before moving on. Use the -n option to rsync when testing to preview what would happen.

$ rsync -n -av /tmp .
$ rsync -n -av /tmp/ .

More rsync examples:
————————–

# rsync -crogpvltz -e ssh –exclude “bin” –exclude “ifany” –delete –delete-after –bwlimit 20 root@10.144.17.1:/apps/uae/ /apps/uae > /apps/uae/logs/sync.log

# rsync -avz -e ssh root@remotehost:/backup/reptest /backtmp/reptest/

# rsync -avz -e ssh /logstage/archive/DXBSEC/NODE0000/ db2inst1@10.20.202.236:/db2/archivelog/security/logstage/retrieve/DXBSEC/NODE0000/

# rsync -e ssh -avz –timeout=999 –delete –exclude dir-or-file-to-exclude –delete-excluded –stats -rlt user@remotehost:/home/lokams/

rsync useful options:
————————-

-a, –archive archive mode, equivalent to -rlptgoD
-n, –dry-run show what would have been transferred ( Preview mode )

-c – always checksum
-r – recursive into directories
-o – preserve owner
-g – preserve group
-p – preserve permissions
-t – preserve times
-v – verbose
-l – copy symlinks as symlinks
-z – compress file data
-P – show progress during transfer
-q – quite (decrease verbosity)
-e – specify the remote shell
-b – make backup
-R – Use relative path names
-u – skip files that are newer on the receiver

–stats give some file-transfer stats
–timeout=TIME set I/O timeout in seconds

–backup-dir – make backups into this directory
–bwlimit=KBPS – limit I/O bandwidth, KBytes per second
–delete – delete files that don’t exist on the sending side
–delete-after – receiver deletes after transferring, not before
–daemon run as an rsync daemon
–address=ADDRESS bind to the specified address
–exclude=PATTERN exclude files matching PATTERN
–exclude-from=FILE exclude patterns listed in FILE
–include=PATTERN don’t exclude files matching PATTERN
–include-from=FILE don’t exclude patterns listed in FILE
–min-size=SIZE don’t transfer any file smaller than SIZE
–max-size=SIZE don’t transfer any file larger than SIZE

# –numeric-ids:
Tells rsync to not map user and group id numbers local user and group names
# –delete:
Makes server copy an exact copy of the source by removing any files that have been removed on the remote machine

Find Command

Some examples:

Find all files with 777 permissions:
find . -type f -perm 777 -print

Find all files that are older than 14 days and ls:
find . -type f -mtime +14 -exec ls -l {} \;

Find all your writable files and list them:
find . -perm -0777 -type f -ls

Find files and change their permission to writeable:
find . -name “NAME*” -exec chmod 666 {} \;

Find files created in the past 7 days:
find . -mtime -7 -type f

Find files modified in the past 7 days:
find . -mtime -7 -type f

Find files owned by a particular user:
find . -user esofthub

Find files that were modified more than 7 days ago and tar them into an archive (with a date in the name):
find . -type f -mtime +7 | xargs tar -cvf `date ‘+%d%m%Y’_archive.tar`

Find files that were modified less than 7 days ago and tar them into an archive (with a date in the name):
find . -type f -mtime -7 | xargs tar -cvf `date ‘+%d%m%Y’_archive.tar`

Find how many files are in a path:
find . -type f -exec basename {} \; | wc -l

Find the top 50 Biggest Files:
find / -type f -not -path “/proc/*” -not -path “/dev/*” -not -path “/sys/*” -not -path “/home/rack/*” -printf “%s %h/%f\n” | sort -rn -k1 | head -n 50 | awk ‘{print $1/1048576 “MB” ” ” $2}’

xargs and find to selected copy

$ 4.2.29 8 508---> find  . -name "*gimp*"  -print0   | xargs -0 -I {}  cp  {} ../6raphics/
$ 4.2.29 9 509---> find  . -name "*gimp*"  -print0   | xargs -0 -I {}  cp  {} ../6raphics/
$ 4.2.29 10 510---> find  . -name "*blender*"  -print0   | xargs -0 -I {}  cp  {} ../6raphics/
$ 4.2.29 11 511---> find  . -name "*inkscape*"  -print0   | xargs -0 -I {}  cp  {} ../6raphics/
$ 4.2.29 12 512---> find  . -name "*Grap*"  -print0   | xargs -0 -I {}  cp  {} ../6raphics/
$ 4.2.29 13 513---> find  . -name "*grap*"  -print0   | xargs -0 -I {}  cp  {} ../6raphics/
$ 4.2.29 14 514---> 

find + grep 

find /schedapps/test/log -type f -name "*abc*log"

find /schedapps/test/log -type f -name "*log" | grep abc

find /schedapps/test/log -type f ! -name "*abc*log"

find /schedapps/test/log -type f -name "*log" | grep -v abc | grep -v def

find /schedapps/test/log -type f -name "*log" -mtime +7 | xargs rm -f

find /schedapps/etmd -type f -perm -o+w -exec ls -l {} \; | more

find /SqLogs –type f –name accesslog | xargs grep "TCP_DENIED"

find  path-name type -f -name "aliencoders*" | xargs -r ls -lRt | awk '{ print \$9 }'

Find occurrences of strings in files AND report file names

find  -name '*xml' -print | xargs grep 'hostName' /dev/null

Find files over a certain size eg 10K

find / -xdev -size +1024 -exec ls -al {} \; | sort -r -k 5

Find and tar files on the fly

find /opt/IBM/WebSphere/PortalServer/log -name 'System*log'| xargs tar -rvf System_logstar

Output wsadmin to console AND file

/opt/IBM/WebSphere/wp_profile/bin/wsadminsh -lang jython | tee wsadminlog

Find files touched in, say, the past 20 minutes

find * -mmin -20

Run a command on a loop

watch -d=10 -n 1 "netstat -a"

$ find . -name *.gif -exec ls {} \;

The -exec parameter holds the real power. When a file is found that matches the search criteria, the -exec parameter defines what to do with the file. This example tells the computer to:

 1. Search from the current directory on down, using the dot (.) just after find.
 2. Locate all files that have a name ending in .gif (graphic files).
 3. List all found files, using the ls command.

The -exec parameter requires further scrutiny. When a filename is found that matches the search criteria, the find command executes the ls {} string, substituting the filename and path for the {} text. If saturn.gif was found in the search, find would execute this command:

$ ls ./gif_files/space/solar_system/saturn.gif

An important alternative to the -exec parameter is -ok; it behaves the same as -exec, but it prompts you to see if you want to run the command on that file. Suppose you want to remove most of the .txt files in your home directory, but you wish to do it on a file-by-file basis. Delete operations like the UNIX rm command are dangerous, because it's possible to inadvertently delete files that are important when they're found by an automated process like find; you might want to scrutinize all the files the system finds before removing them.

The following command lists all the .txt files in your home directory. To delete the files, you must enter Y or y when the find command prompts you for action by listing the filename:

$ find $HOME/. -name *.txt -ok rm {} \;

Each file found is listed, and the system pauses for you to enter Y or y. If you press the Enter key, the system won't delete the file.

If too many files are involved for you to spend time with the -ok parameter, a good rule of thumb is to run the find command with -exec to list the files that would be deleted; then, after examining the list to be sure no important files will be deleted, run the command again, replacing ls with rm.

Both -exec and -ok are useful, and you must decide which works best for you in your current situation. Remember, safety first!

Examples:

• To remove all temp, swap and core files in the current directory.

$ find . \( -name '*.tmp* -o -name '*.swp' -o -name 'core' \) -exec rm {} \;

• To copy the entire contents of a directory while preserving the permissions, times, and ownership of every file and subdirectory

$ cd /path/to/source/dir
$ find . | cpio -vdump /path/to/destination/dir

• To list the first line in every text file in your home directory

$ find $HOME/. -name *.txt -exec head -n 1 -v {} \; > report.txt
$ less <>

• To maintain LOG and TMP file storage space for applications that generate a lot of these files, you can put the following commands into a cron job that runs daily:
• The first command runs all the directories (-type d) found in the $LOGDIR directory wherein a file’s data has been modified within the last 24 hours (-mtime +0) and compresses them (compress -r {}) to save disk space. The second command deletes them (rm -f {}) if they are more than a work-week old (-mtime +5), to increase the free space on the disk. In this way, the cron job automatically keeps the directories for a window of time that you specify.

$ find $LOGDIR -type d -mtime +0 -exec compress -r {} \;
$ find $LOGDIR -type d -mtime +5 -exec rm -f {} \;

• To find links that point to nothing

$ find / -type l -print | perl -nle '-e || print';

• To list zero-length files

$ find . -empty -exec ls {} \;

• To delete all *.tmp files in the home directory

$ find ~/. -name "*.tmp" | xargs rm
(or)
$ find ~/. -name "*.tmp" -exec rm {} \;

• To see what hidden files in your home directory changed in the last 5 days:

$ find ~ -mtime -5 -name \.\*

• If you know something has changed much more recently than that, say in the last 14 minutes, and want to know what it was there’s the mmin argument:

$ find ~ -mmin 14 -name \.\*

• To locate files that have been modified since some arbitrary date use this little trick:

$ touch -d "13 may 2001 17:54:19" date_marker
$ find . -newer date_marker 

• To find files created before that date, use the cnewer and negation conditions:

$ find . \! -cnewer date_marker

• To find files containing between 600 to 700 characters, inclusive.

$ find . -size +599c -and -size -701c 

Thus we can use find to list files of a certain size:

$ find /usr/bin -size 48k

• To find empty files

$ find . -size 0c

• Using the -empty argument is more efficient. To delete empty files in the current directory:

$ find . -empty -maxdepth 1 -exec rm {} \;

• To locate files belonging to a certain user:

# find /etc -type f \! -user root -exec ls -l {} \;

• To search for files by the numerical group ID use the -gid argument:

$ find -gid 100

• To find directories with ‘_of_’ in their name we’d use:

$ find . -type d -name '*_of_*'

• To redirect the error messages to /dev/null

$ find / -name foo 2>/dev/null

• To remove all files named core from your system.

# find / -name core | xargs /bin/rm -f
# find / -name core -exec /bin/rm -f '{}' \; # same thing
# find / -name core -delete # same if using Gnu find

• To find files modified less than 10 minutes ago. I use this right after using some system administration tool, to learn which files got changed by that tool:

# find / -mmin -10

• When specifying time with find options such as -mmin (minutes) or -mtime (24 hour periods, starting from now), you can specify a number n to mean exactly n, -n to mean less than n, and +n to mean more than n. 2 For example:

# find . -mtime 0 # find files modified within the past 24 hours
# find . -mtime -1 # find files modified within the past 24 hours
# find . -mtime 1 # find files modified between 24 and 48 hours ago
# find . -mtime +1 # find files modified more than 48 hours ago
# find . -mmin +5 -mmin -10 # find files modifed between 6 and 9 minutes ago

• To find all files containing “house” in the name that are newer than two days and are larger than 10K, try this:

# find . -name “*house*” -size +10240 -mtime -2

• The -xdev prevents the file “scan” from going to another disk volume (refusing to cross mount points, for example). Thus, you can look for all regular directories on the current disk from a starting point like this:

# find /var/tmp -xdev -type d -print

• To find world writables in your system:

# find / -perm 777 | xargs ls -ld | grep -v ^l | grep -v ^s
# find / -perm 666 | xargs ls -ld | grep -v ^l | grep -v ^s

# find . -perm +o=w -exec ls -ld {} \; | grep -v ^l | grep -v ^s | grep -v ^c
or
# find . -perm +o=w | xargs ls -ld | grep -v ^l | grep -v ^s | grep -v ^c

• To find the orphan files and directories in your system:

# find / -nouser -nogroup | xargs ls -ld

• To find the files changed in last 5 mins and move them to a different folder.

# find /tmp -mmin -5 -type f -exec mv {} /home/lokams \;
(or)
# mv `find . -mmin -5 -type f` /tmp/
(or)
# find . -mmin -10 -type f | xargs -t -i {} mv {} /tmp/

• To search on multiple directories

$ find /var /etc /usr -type f -user root -perm 644 -name '*ssh*'

• To find and list all regular files set SUID to root (or anyone else with UID 0 😉

# find / -type f -user 0 -perm -4000

• To find all regular files that are world-writable and removes world-writability:

# find / -type f -perm -2 -exec chmod o-w {} \;

• To find all files owned by no one in particular and give them to root:

# find / -nouser -exec chown root {} \;

• To find all files without group ownership and give them to the system group:

# find / -nogroup -exec chgrp system {} \;

• To find and gzip regular files in current directory that do not end in .gz

$ gzip `find . -type f \! -name '*.gz' -print`

• To find all empty files in my home directory and delete after being prompted:

$ find $HOME -size 0 -ok rm -f {} \;

• To find all files or symlinks in /usr not named fred:

$ find /usr \( -type f -o -type l \) \! -name fred

• If you have a file with spaces, control characters, leading hyphens or other nastiness and you want to delete it, here’s how find can help. Forget the filename, use the inumber instead. Say we have a file named “-rf .”, spaces and all. We wouldn’t dare attempt removing it the normal way for fear of issuing ‘rm -rf /’ at the command line.

$ echo jhjhg > "-rf /"
$ ls -la
total 4
-rw-r----- 1 mongoose staff 6 Nov 07 15:57 -rf /
drwxr-x--- 2 mongoose staff 512 Nov 07 15:57 .
drwxr-xr-x 3 mongoose bin 1024 Nov 07 15:53 ..

Find the I-number of the file using the -i flag in ls:

$ ls -lai .
total 4
18731 -rw-r----- 1 mongoose staff 6 Nov 07 15:57 -rf .
18730 drwxr-x--- 2 mongoose staff 512 Nov 07 15:57 .
1135 drwxr-xr-x 3 mongoose bin 1024 Nov 07 15:53 ..
^^^^^
There it is: I-node 18731. Now plug it into find and make find delete it:

$ find . -inum 18731 -ok rm {} \;
rm ./-rf . (?) y
$ ls -la
total 3
drwxr-x--- 2 mongoose staff 512 Nov 07 16:03 .
drwxr-xr-x 3 mongoose bin 1024 Nov 07 15:53