find . -type f -exec grep ‘NMX_FXNG_AND_CONTRACT_DBF’ {} \;
history
history|awk ‘{print $2}’|awk ‘BEGIN {FS=”|”} {print $1}’|sort|uniq -c|sort -r
history|tail -1000|awk ‘{print $2}’|awk ‘BEGIN {FS=”|”} {print $1}’|sort|uniq -c|sort -r
-daystart This flag starts at the beginning of the day.
-atime The time the file was last accessed — in number of days.
-ctime The time the file’s status last changed — in number of days.
-mtime The time the file was last modified — in number of days.
-amin The time the file was last accessed — in number of minutes. (It is not available on all implementations.)
-cmin The time the file’s status last changed — in number of minutes. (It is not available on all implementations.)
-mmin The time the file was last modified — in number of minutes. (It is not available on all implementations.)
-type This flag describes the type of file, such as d for directories.
-userX Files belonging to user X.
-groupX Files belonging to group X.
-newerX Files that are newer than file X.
find /home/$1/mail/*/mail/.spam/cur -type f -mtime +7 -exec rm {} \;
find /home/$1/mail/*/mail/.spam/new -type f -mtime +7 -exec rm {} \;
Here’s how to list all the files in your home directory tree that were modified exactly one hour ago:$ find ~ -mmin 60 \! -type d
Giving a negative value for a flag means to match that number or sooner. For example, here’s how to list all the files in your
home directory tree that were modified exactly one hour ago or any time since
find ~ -mmin -60 \! -type d
$ date
Mon Oct 23 09:42:42 EDT 2006
$ touch -t 10230842 temp
$ ls -l temp
-rw-r–r– 1 joe joe 0 Oct 23 08:42 temp
$ find ~ -newer temp \! -type d
find / -user `whoami` -daystart -atime -1 \! -type d
Give different values for the various time flags to change the search times. You can also combine flags. For instance,
you can list all the files in your home directory tree that were both accessed and modified between now and seven days ago:
find ~ -daystart -atime -7 -mtime -7 \! -type d
find mtime
find . -mtime 0 # find files modified between now and 1 day ago
# (i.e., within the past 24 hours)
find . -mtime -1 # find files modified less than 1 day ago
# (i.e., within the past 24 hours, as before)
find . -mtime 1 # find files modified between 24 and 48 hours ago
find . -mtime +1 # find files modified more than 48 hours ago
find . -mmin +5 -mmin -10 # find files modified between
# 6 and 9 minutes ago
* find top 10 largest files in /var:
$ find /var -type f -ls | sort -k 7 -r -n | head -10
* find all files having size more than 5 GB in /var/log/:
$ find /var/log/ -type f -size +5120M -exec ls -lh {} \;
* find all today’s files and copy them to another directory:
$ find /home/me/files -ctime 0 -print -exec cp {} /mnt/backup/{} \;
* find all temp files older than a week and delete:
$ find /temp/ -mtime +7-type f | xargs /bin/rm -f
* find and rename all mp3 files by changing their uppercase names to lowercase:
$ find /home/me/music/ -type f -name *.mp3 -exec rename ‘y/[A-Z]/[a-z]/’ ‘{}’ \;
Grep
some examples of grep command:
* Print Apache’s documentroot directory name:
$ grep -i documentroot /etc/httpd/conf/httpd.conf
* View file contents without comments and empty lines:
$ grep -Ev “^$|^#” /etc/my.cnf
* print only IP address assigned to the interface:
$ ifconfig eth0 | grep ‘inet addr:’ | cut -d’:’ -f2 | awk ‘{ print $1}’
* How many email messages sent for a particular date:
$ cat /var/log/maillog | grep “status=sent” | grep “May 25” | wc -l
* Find out a running process/daemon from process list (thanks to staranneph for recalling this):
ps -ef | grep mysql
* You can also note cpu/mem usage by using above. like in below command output, you can see that Plesk’s statistics process is utilizing more than 18% cpu alone:
[root@myserver ~]# ps aux | grep statistics
root 8183 18.4 0.0 58384 2848 ? D 04:05 3:00 /usr/l
source
$ touch -d “Aug 1 2006” file.start
$ touch -d “Sep 1 2006” file.end
$ find /usr/share -daystart -newer file.start \! -daystart -newer file.end
egrep -i “err|panic|crit|warn” /var/log/messages
User management
groupadd sshd
useradd -g sshd -d /var/empty -s /bin/false sshd
groupadd -r dovecot
useradd -r -g dovecot -s /sbin/nologin dovecot
groupadd -r vmail
useradd -r -g vmail -s /sbin/nologin vmail
Disk space issue
ls -ltrhS | grep “Apr [2-4]” | awk -F’ ‘ ‘{print $9}’ | xargs rm -f
Memory Usage Command
ps -eo pmem,pcpu,rss,vsize,args | sort -k 1 -r | more
Cpu Usage
ps -eo pcpu,pid,args | sort -k 1 -r | head -10
Sometime Apache process, keeps on execution (Seems like Hangs), so generally trying to get the exact PHP file that is running by Apache Process, So here is my Try.
I used Strace to get the opened files by the apache process. (Get PID of
Apache process that is taking time, though you can also get it From top command)
# pstree -p -n | grep http
(This will show each files that is being processed by that Apache Proc)
# strace -p <PID of Apache>
The list of files could also be get using lsof, but that could not be of full use, as you need the files continuus
To see all the memory used by a process we use pmap(process mapping ).
#pmap pid-of-that-application
pmap 1946
Example :
# pmap `pgrep apache` | grep total
Linux System Monitoring Tools
#1: top – Process Activity Command
Commonly Used Hot Keys
The top command provides several useful hot keys:
Hot Key Usage
t Displays summary information off and on.
m Displays memory information off and on.
A Sorts the display by top consumers of various system resources. Useful for quick identification of performance-hungry tasks on a system.
f Enters an interactive configuration screen for top. Helpful for setting up top for a specific task.
o Enables you to interactively select the ordering within top.
r Issues renice command.
k Issues kill command.
z Turn on or off color/mono
What are the CPU states found in “top” output?
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
# us -> User CPU time: The time the CPU has spent running users’ processes that are not niced.
# sy -> System CPU time: The time the CPU has spent running the kernel and its processes.
# ni -> Nice CPU time: The time the CPU has spent running users’ process that have been niced.
# wa -> iowait: Amount of time the CPU has been waiting for I/O to complete.
# hi -> Hardware IRQ: The amount of time the CPU has been servicing hardware interrupts.
# si -> Software Interrupts.: The amount of time the CPU has been servicing software interrupts.
#2: vmstat – System Activity, Hardware and System Information
The command vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.
vmstat 3
Display Memory Utilization Slabinfo
# vmstat -m
Tail, Vmstat and Date in Loop, Output every 10 Sec
# vmstat 1 1;for ((;;));do date; vmstat 10 2 | tail -n1;done
3: w – Find Out Who Is Logged on And What They Are Doing
w command displays information about the users currently on the machine, and their processes.
# w username
# w vivek
#4: uptime – Tell How Long The System Has Been Running
# uptime
#5: ps – Displays The Processes
Show Long Format Output
# ps -Al
Print All Process On The Server
# ps ax
# ps axu
Memmory commands cpu commands
Print Security Information
# ps -eo euser,ruser,suser,fuser,f,comm,label
Set Output In a User-Defined Format
ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
ps -eopid,tt,user,fname,tmout,f,wchan
Display sorted process taking most CPU in descending order
# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
CPU COMMAND
10 cpu usage command
ps -e -o pcpu,cpu,nice,state,cputime,args –sort pcpu | sed ‘/^ 0.0 /d’
####Watch changeable data continuously
watch -n.1 ‘cat /proc/interrupts’
cat /proc/interrupts
Check CPU Temperature
# echo `date +%b-%d-%H:%M:%S` | tr -d ‘\ 012’ ; echo -n ‘ ‘; sensors | awk ‘/CPU Temp:/{ print $3 }’
Check those commands which have been used most
# history|awk ‘{print $2}’ |awk ‘{print $1}’ | sort | uniq -c | sort -rn | head -10
Memmory commands
Free Memory on Linux at Runtime
# sync
# echo 3 > /proc/sys/vm/drop_caches
Display Only The Process IDs of Lighttpd
# ps -C lighttpd -o pid=
OR
# pgrep lighttpd
OR
# pgrep -u vivek php-cgi
Display The Name of PID 55977
# ps -p 55977 -o comm=
Total Memory Usage Calculation
Print Total sum of actual Memory Usages
ps aux | awk ‘{sum +=$4}; END {print sum}’
Check Actual Memory Consumption
free -m
Check total usage of One particular process
ps aux | awk ‘{print $4,$11}’ | sort |tail -n 23 |grep -w ‘someprocess’ |wc -l
All most consuming process
ps aux | awk ‘{print $4,$11}’ | sort |tail -n 20
Create many Files Sequentially
# seq -w 1 30 | xargs -i -t zcat in_Feb2011/in_Files-{}May2011.gz | grep -E ‘name.html?secsid=3304847|name.html?secsid=30780899’
Delete Empty Directories
# find folder/ -type d -empty | xargs -i -t rm -rf {}
or
# find folder/ -type d -empty -delete
Real Time Monitoring on Linux
# watch -n1 –difference “echo “Uptime”; uptime; echo \n ; ps -eo pcpu,pid,args | sort -k 1 -r |grep -v watch | head -10; echo “\n” ; tail /var/log/cron | grep “check_load” “
DISK SPACE COMMAND
Find files based and sorted on Size
# find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk ‘{ print $NF “: ” $5 }’ | sort -nrk 2,2
Killing processes in one Line
# kill -9 `ps -ef | grep rsync| grep -v grep| awk ‘{print $2}’`
Check Memory Fault
# dd if=/dev/urandom bs=768304 of=/tmp/memtest count=1050
# md5sum /tmp/memtest; md5sum /tmp/memtest; md5sum /tmp/memtest
Repair Mysql MYISAM File
# myisamchk –force –sort_buffer_size=64M –key_buffer_size=16M –read_buffer_size=8M –write_buffer_size=8M ../data/phplists/phplist_linktrack.MYI
Command to make Services off in Defined Level on Linux
# chkconfig –list | awk ‘{print $1}’ | cut -d: -f1 | grep -vE ‘^crond|^network|^sshd|^syslog|^iptables’ | awk ‘{print $1}’ | while read line; do chkconfig –level 3 $line off; count=`expr $count + 1`; echo $count $line;done
or Use this… One lineer
# chkconfig –list | awk ‘{print $1}’| grep -vE ‘^crond|^network|^sshd|^syslog|^iptables’ | xargs -i chkconfig –level 3 {} off
Counting Hits from Web Server Access log
# awk ‘{print $1}’ /opt/indian.com/access_log | grep -vE ‘^:|^common|^-‘ | sort | uniq -c | sort -nr > /var/www/reports/ips/indian.txt
or # awk ‘$1>10000 {print $1}’ /opt/indian.com/access_log | uniq -c | sort -nr > /var/www/reports/ips/indian.txt
PERL Search and Replace Text Pattern using Perl On linux Platform
# find . -type f -name “*.html” | xargs perl -pi~ -e ‘s/\/js\/active18\//\/read\/js\/active18\//’
BACKUP SCRIPTS
15 2 * * * root /usr/bin/mysqldump -u root -pPASSWORD –all-databases | gzip > /mnt/disk2/database_`data ‘ %m-%d-%Y’`.sql.gz
Details about SUID, SGID and Sticky bit permission on linux os
* SUID or setuid: change user ID on execution. If setuid bit is set, when the file will be executed by a user, the process will have the same rights as the owner of the file being executed.
* SGID or setgid: change group ID on execution. Same as above, but inherits rights of the group of the owner of the file on execution. For directories it also may mean that when a new file is created in the directory it will inherit the group of the directory (and not of the user who created the file).
* Sticky bit. It was used to trigger process to “stick” in memory after it is finished, now this usage is obsolete. Currently its use is system dependant and it is mostly used to suppress deletion of the files that belong to other users in the folder where you have “write” access to.
Numeric representation :
Octal digit Binary value Meaning
0 000 setuid, setgid, sticky bits are cleared
1 001 sticky bit is set
2 010 setgid bit is set
3 011 setgid and sticky bits are set
4 100 setuid bit is set
5 101 setuid and sticky bits are set
6 110 setuid and setgid bits are set
7 111 setuid, setgid, sticky bits are set
file : 2644
dir : 2755
Textual representation :
SUID If set, then replaces “x” in the owner permissions to “s”, if owner has execute permissions, or to “S” otherwise. Examples:
-rws—— both owner execute and SUID are set
-r-S—— SUID is set, but owner execute is not set
SGID If set, then replaces “x” in the group permissions to “s”, if group has execute permissions, or to “S” otherwise. Examples:
-rwxrws— both group execute and SGID are set
-rwxr-S— SGID is set, but group execute is not set
Sticky If set, then replaces “x” in the others permissions to “t”, if others have execute permissions, or to “T” otherwise. Examples:
-rwxrwxrwt both others execute and sticky bit are set
-rwxrwxr-T sticky bit is set, but others execute is not set
drwxrwxrwt – Sticky Bits – chmod 1777
drwsrwxrwx – SUID set – chmod 4777
drwxrwsrwx – SGID set – chmod 2777
Procedure to add a swap file
You need to use dd command to create swapfile. Next you need to use mkswap command to set up a Linux swap area on a device or in a file.
a) Login as the root user
b) Type following command to create 512MB swap file (1024 * 512MB = 524288 block size):
# dd if=/dev/zero of=/swapfile1 bs=1024 count=524288
c) Set up a Linux swap area:
# mkswap /swapfile1
d) Activate /swapfile1 swap space immediately:
# swapon /swapfile1
e) To activate /swapfile1 after Linux system reboot, add entry to /etc/fstab file. Open this file using text editor such as vi:
# vi /etc/fstab
Append following line:
/swapfile1 swap swap defaults 0 0
So next time Linux comes up after reboot, it enables the new swap file for you automatically.
g) How do I verify swap is activated or not?
Simply use free command:
$ free -m
strace
October 23rd, 2008 No comments
strace -q -f -c -p
strace -f verbose=all -e write=all -o /tmp/strace.log -p [pid]
strace -T -t -q -f -e trace=file,open,close,read -o /tmp/prod.strace.log -p <oms or ui pid>
-f traces all child processes as they are created byt he currently traced process as a result of the fork() system call.
-e is a qualifying expression which modifies which events to trace or how to race them
verbose=all dereferences structures for all system calls
write=all performs a full hexadecimal and ASCII dump of all the data written to all file descriptors
-o output file
-p process id to trace
Replace -p [pid] with [command] to trace a specific command.
Drop Caches
October 21st, 2008 No comments
Kernels 2.6.16 and newer provide a mechanism to have the kernel drop the page cache and/or inode and dentry caches on command, which can help free up a lot of memory. Now you can throw away that script that allocated a ton of memory just to get rid of the cache…
To use /proc/sys/vm/drop_caches, just echo a number to it.
To free pagecache:
# echo 1 > /proc/sys/vm/drop_caches
To free dentries and inodes:
# echo 2 > /proc/sys/vm/drop_caches
To free pagecache, dentries and inodes:
echo 3 > /proc/sys/vm/drop_caches
As this is a non-destructive operation and dirty objects are not freeable, the user should run “sync” first!
How to copy data in VI editor
January 20th, 2009 No comments
vi first edit source file
then move your cursor to start of selection
ma mark current position with letter a
then move your cursor to end of selection
y’a yank to buffer x from current position to mark a
:e other edit target file
move cursor to where you want the data
p put from buffer x
Copying a block of text from one file to another in Vi
December 29th, 2008 No comments
To copy a block of text between files execute the commands:
Command Explaination
1. Edit the file containing the text you want to copy.
2. Go to the top line to be copied.
3. ma Mark this line as mark “a”.
4. Go to the bottom line to be copied
5. y’a Yank (y) the text from the current cursor location to the mark “a” (‘a)
6. :split second-file Open another window containing the second file. (This the file in which the text is to be inserted.)
7. Go to the line where the insert is to occur. The text will be place after this line.
8. p Put the text after the cursor.
For Vi Editor
Do you like www.linuxnix.com ? Please consider supporting us by becoming a subscriber and get a Linux basics e-book for free.
unix_vi_editor
VI editor is the default file editor in most of the Linux/Nix machines. It is having great capabilities to edit a file with in few key strokes.
Lets start with some general information and then move on to some good things what vi editor can do for you while editing a file.
1. Vi stands for visual.
2. Vi have its variants like vim which stands for Vi IMproved, VimX11 for gui and winvi for MS windows.
3. Vi is the most popular editor and next most popular editor is gedit.
4. Do you know there is a book on VI editor from orally which is of 600+ pages.
5. Some other editors which will do the work of editing files are neno, pico, gedit, emacs, joe, nedit, ed etc.
Learning vi editor and remembering them is a very a easy task if you learn it in a systematic way.
a. Modes of VI
b. Navigational commands
c. Editing commands.
d. Search and Replace
e. Save and quiting a file.
a. Modes of VI :
Vi have two mode of operation.
1. Command mode
2. Inserting mode
Command mode :
Vi editor begins in command mode, where cursor movement(navigation in the file) and editing occur. To enter in to command mode from Inserting mode press esc button.
Inserting mode :
Used for entering text, this is similar to notepad in Windows. To enter in to inserting mode you can use any of the following.
i or I => present line
o => one line down the present line
O => one line above
Note : All comments will work in command mode only.
b. Navigational commands :
1. Character navigation k, h, l and j
h => To move one character left.
j => To move one line down.
k => To move one line up.
l => To move one character right.
How to use above commands in clever way?
Examples :
6j => to move 6 lines down from the present courser.
7k => to move 7 lines above from the present courser.
2. Word Navigation
w => word forward.
e =>word forward, but end of the word.
b => one word backward.
Examples :
32w => To move 32 words forward
6b => To move 6 words back.
3. Setting (nu) mbering to lines
:set nu
Removing of (nonu)mbering to lines
:set nonu
4. Moving paragraphs
move one paragraph up => {{
move one paragraph down => }}
5. Moving page up/down
For up => ctrl+b
For down => ctrl+f
6. Moving start/end of the file
Starting of the file(first line => [[
End of the file(last line) => ]]
7. Going to any line :
:lineno
Example :
If we want to go to 56 line then type
:56
c. Editing commands
8. Replace one letter
Replace one letter => r
Delete one letter => x
>9. Editing one word
Edit one word => cw
Delete one word => dw
10. Editing one line
Editing a line from courser to the end of that line => d$
11. Cutting
deleting(cutting) one line => dd
Examples :
2dd(deleting/cutting two lines)
12. Pasting
Pasting a line below the courser => p
Pasting a line above the courser => P
13. Coping
Copying one line => yy
Copying n lines => nyy
14. Special commands
joining lines => J
undoing things => u
repeating previous command => .
d. Search and replace
15. Search for a term /term
Example : If you want to search for suresh then press /suresh enter
/suresh
Moving to next occurrence, press “n” with out quotes moving to previous occurrence, press “N” with out quotes.
16. Searching and replacing a term(here separator is / )
:%s/searchterm/replaceterm/g
change default separator
:%s_/home/surya/grade_/home/testing/dest_g
To search and replace particular term from given line to other given line.
:%s294,304/sahana/xyz/g
e)Save and quiting a file
:w => save the file
:q => quit the file
:wq => save and quit
:w! => force save the file
:q! => force quit with out save
:wq => save and quit forcefully
==========================================================================================================================================================
Disk Space issues
mpstat -P ALL
cat /proc/interrupts
iostat -kd 1
iostat -c -t
iostat -c -t 2 10
iostat -c 3
iostat -d 5
iostat -x 1
sar -b
vmstat -S M 2 5
Find process causing high iowait
netstat -autpn | grep :80
netstat -autpn | grep :3306
netstat -na
netstat -an|grep :80|sort|more
netstat -an|grep ESTABLISHED
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -nr | more
ps ax | awk ‘$3 ~ /^D/ { print $0 }’
netstat -an | grep :80 | wc -l
DO NOT DO THIS until you are certain what is going on.
echo 100 > /proc/sys/vm/inactive_clean_percent
echo 2 10 20 > /proc/sys/vm/pagecache
==========================================================================================================================================================
To View Or List Only Directories In Linux?
Do you like www.linuxnix.com ? Please consider supporting us by becoming a subscriber and get a Linux basics e-book for free.
How to view/list only directories in Linux?
Ans : This can be achieved in two ways
1. Through ls command
2. Through find command
With ls we have to use grep to get the directory listings.
Ls –l grep ^d
Example :
[root@test surendra_a]# ls -l grep ^d
d——— 2 surendra_a surendra_a 4096 Sep 8 09:54 HTWFAIP
drwxrwxr-x 2 surendra_a root 4096 Nov 27 12:30 LinuxCBT – RHEL5
drwxrwxr-x 2 surendra_a root 4096 Oct 12 16:40 Software
[root@test surendra_a]#
With find we can have more controle on how to display only directories.
A. To display all the directories and sub-directories in present directory
#find . -type d
B. Displaying only directories in present directory
#find /root/ -type d –maxdepth 1
C. Displaying just directories in present directry and its sub-directories
#find /root/ -type d –maxdepth 2
How TO kill to Process
a) pgrep httpd | xargs kill -9
b) kill -9 `ps -ef | grep ‘httpd’ | grep -v grep | awk ‘{print $2}’`
Top to Cpu Usage commands
ps auxf | sort -nr -k 3 | head -10
Top processes consume RAM Memory
ps auxf | sort -nr -k 4 | head -10
Find and delete / remove files older 7 days / 1 week on Linux
find /root/server -type f -mtime +7 -exec rm -rf {} \;
Remove files older 90 days / 3 months / 1 quarter
find /var/ -type f -mtime +90 -exec rm -rf {} \;
Remove files older 365
find /var/ -type f -mtime +365 -exec rm -rf {} \;
check your public IP
n Linux, we have some best ways to check Public IP for that server.
1. curl ifconfig.me
2. wget -qO- ipecho.net/plain
3. wget -qO- ifconfig.me/ip
check your public IP
Recent Comments