find . -type f -exec grep ‘NMX_FXNG_AND_CONTRACT_DBF’ {} \;

history

 history|awk ‘{print $2}’|awk ‘BEGIN {FS=”|”} {print $1}’|sort|uniq -c|sort -r

 history|tail -1000|awk ‘{print $2}’|awk ‘BEGIN {FS=”|”} {print $1}’|sort|uniq -c|sort -r

-daystart    This flag starts at the beginning of the day.
-atime    The time the file was last accessed — in number of days.
-ctime    The time the file’s status last changed — in number of days.
-mtime    The time the file was last modified — in number of days.
-amin    The time the file was last accessed — in number of minutes. (It is not available on all implementations.)
-cmin    The time the file’s status last changed — in number of minutes. (It is not available on all implementations.)
-mmin    The time the file was last modified — in number of minutes. (It is not available on all implementations.)
-type    This flag describes the type of file, such as d for directories.
-userX    Files belonging to user X.
-groupX    Files belonging to group X.
-newerX    Files that are newer than file X.

find /home/$1/mail/*/mail/.spam/cur -type f -mtime +7 -exec rm {} \;
find /home/$1/mail/*/mail/.spam/new -type f -mtime +7 -exec rm {} \;
 

Here’s how to list all the files in your home directory tree that were modified exactly one hour ago:$ find ~ -mmin 60 \! -type d

Giving a negative value for a flag means to match that number or sooner. For example, here’s how to list all the files in your
home directory tree that were modified exactly one hour ago or any time since

find ~ -mmin -60 \! -type d

$ date
Mon Oct 23 09:42:42 EDT 2006
$ touch -t 10230842 temp
$ ls -l temp
-rw-r–r–    1 joe        joe               0 Oct 23 08:42 temp
$ find ~ -newer temp \! -type d

find / -user `whoami` -daystart -atime -1 \! -type d

Give different values for the various time flags to change the search times. You can also combine flags. For instance,
you can list all the files in your home directory tree that were both accessed and modified between now and seven days ago:
find ~ -daystart -atime -7 -mtime -7  \! -type d

find mtime

find . -mtime 0 # find files modified between now and 1 day ago
# (i.e., within the past 24 hours)
find . -mtime -1 # find files modified less than 1 day ago
# (i.e., within the past 24 hours, as before)
find . -mtime 1 # find files modified between 24 and 48 hours ago
find . -mtime +1 # find files modified more than 48 hours ago

find . -mmin +5 -mmin -10 # find files modified between
# 6 and 9 minutes ago

* find top 10 largest files in /var:

$ find /var -type f -ls | sort -k 7 -r -n | head -10

* find all files having size more than 5 GB in /var/log/:

$ find /var/log/ -type f -size +5120M -exec ls -lh {} \;

* find all today’s files and copy them to another directory:

$ find /home/me/files -ctime 0  -print -exec cp {} /mnt/backup/{} \;

* find all temp files older than a week and delete:

$ find /temp/ -mtime +7-type f | xargs /bin/rm -f

* find and rename all mp3 files by changing their uppercase names to lowercase:

$ find /home/me/music/ -type f -name *.mp3 -exec rename ‘y/[A-Z]/[a-z]/’ ‘{}’ \;

Grep

some examples of grep command:

* Print Apache’s documentroot directory name:

$ grep -i documentroot  /etc/httpd/conf/httpd.conf

* View file contents without comments and empty lines:

$ grep -Ev “^$|^#” /etc/my.cnf

* print only IP address assigned to the interface:

$ ifconfig eth0 | grep ‘inet addr:’ | cut -d’:’ -f2 | awk ‘{ print $1}’

* How many email messages sent for a particular date:

$ cat /var/log/maillog | grep “status=sent” | grep “May 25” | wc -l

* Find out a running process/daemon from process list (thanks to staranneph for recalling this):

ps -ef | grep mysql

* You can also note cpu/mem usage by using above. like in below command output, you can see that Plesk’s statistics process is utilizing more than 18% cpu alone:

[root@myserver ~]# ps aux | grep statistics
root      8183 18.4  0.0  58384  2848 ?        D    04:05   3:00 /usr/l

source

$ touch -d “Aug 1 2006” file.start
$ touch -d “Sep 1 2006” file.end
$ find /usr/share -daystart -newer file.start \! -daystart -newer file.end

 egrep -i “err|panic|crit|warn” /var/log/messages

User management

groupadd sshd
useradd -g sshd -d /var/empty -s /bin/false sshd
groupadd -r dovecot
useradd -r -g dovecot -s /sbin/nologin dovecot
groupadd -r vmail
useradd -r -g vmail -s /sbin/nologin vmail

Disk space issue

ls -ltrhS | grep “Apr  [2-4]” | awk -F’ ‘ ‘{print $9}’ | xargs rm -f

Memory Usage  Command

ps -eo pmem,pcpu,rss,vsize,args | sort -k 1 -r | more

 Cpu Usage

ps -eo pcpu,pid,args | sort -k 1 -r | head -10

Sometime Apache process, keeps on execution (Seems like Hangs), so generally trying to get the exact PHP file that is running by Apache Process, So here is my Try.

I used Strace to get the opened files by the apache process. (Get PID of
Apache process that is taking time, though you can also get it From top  command)

# pstree -p -n | grep http
(This will show each files that is being processed by that Apache Proc)

# strace -p <PID of Apache>
The list of files could also be get using lsof, but that could not be of full use, as you need the files continuus

To see all the memory used by a process we use pmap(process mapping ).
#pmap pid-of-that-application

pmap 1946

Example :
# pmap `pgrep apache` | grep total

Linux System Monitoring Tools

#1: top – Process Activity Command

Commonly Used Hot Keys

The top command provides several useful hot keys:
Hot Key    Usage
t    Displays summary information off and on.
m    Displays memory information off and on.
A    Sorts the display by top consumers of various system resources. Useful for quick identification of performance-hungry tasks on a system.
f    Enters an interactive configuration screen for top. Helpful for setting up top for a specific task.
o    Enables you to interactively select the ordering within top.
r    Issues renice command.
k    Issues kill command.
z    Turn on or off color/mono

What are the CPU states found in “top” output?
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

# us -> User CPU time: The time the CPU has spent running users’ processes that are not niced.
# sy -> System CPU time: The time the CPU has spent running the kernel and its processes.
# ni -> Nice CPU time: The time the CPU has spent running users’ process that have been niced.
# wa -> iowait: Amount of time the CPU has been waiting for I/O to complete.
# hi -> Hardware IRQ: The amount of time the CPU has been servicing hardware interrupts.
# si -> Software Interrupts.: The amount of time the CPU has been servicing software interrupts.

#2: vmstat – System Activity, Hardware and System Information

The command vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.

vmstat 3

Display Memory Utilization Slabinfo

# vmstat -m

Tail, Vmstat and Date  in Loop, Output every 10 Sec
# vmstat 1 1;for ((;;));do date; vmstat 10 2 | tail -n1;done

3: w – Find Out Who Is Logged on And What They Are Doing

w command displays information about the users currently on the machine, and their processes.
# w username
# w vivek

#4: uptime – Tell How Long The System Has Been Running

# uptime

#5: ps – Displays The Processes

Show Long Format Output

# ps -Al

Print All Process On The Server

# ps ax
# ps axu

Memmory commands   cpu commands

Print Security Information

# ps -eo euser,ruser,suser,fuser,f,comm,label

Set Output In a User-Defined Format

ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm

ps -eopid,tt,user,fname,tmout,f,wchan

Display sorted process taking most CPU in descending order
# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10

CPU COMMAND

10 cpu usage command

ps -e -o pcpu,cpu,nice,state,cputime,args –sort pcpu | sed ‘/^ 0.0 /d’

####Watch changeable data continuously

watch -n.1 ‘cat /proc/interrupts’

cat /proc/interrupts

Check CPU Temperature
# echo `date +%b-%d-%H:%M:%S` | tr -d ‘\ 012’ ; echo -n ‘ ‘; sensors | awk ‘/CPU Temp:/{ print $3 }’

Check those commands which have been used most
# history|awk ‘{print $2}’ |awk ‘{print $1}’ | sort | uniq -c | sort -rn | head -10

Memmory commands

Free Memory  on Linux at Runtime
# sync
# echo 3 > /proc/sys/vm/drop_caches  

Display Only The Process IDs of Lighttpd

# ps -C lighttpd -o pid=
OR
# pgrep lighttpd
OR
# pgrep -u vivek php-cgi
Display The Name of PID 55977

# ps -p 55977 -o comm=

Total Memory Usage Calculation

Print Total sum of actual Memory Usages

ps aux | awk ‘{sum +=$4}; END {print sum}’

Check Actual Memory Consumption

free -m

Check total usage of One particular process

ps aux | awk ‘{print $4,$11}’ | sort |tail -n 23 |grep -w ‘someprocess’ |wc -l

All most consuming process

ps aux | awk ‘{print $4,$11}’ | sort |tail -n 20

Create many Files Sequentially
# seq -w 1 30 | xargs -i -t zcat in_Feb2011/in_Files-{}May2011.gz | grep -E ‘name.html?secsid=3304847|name.html?secsid=30780899’

Delete Empty Directories
# find folder/ -type d -empty | xargs -i -t rm -rf {}
or
# find folder/ -type d -empty -delete

Real Time Monitoring on Linux
# watch -n1 –difference “echo “Uptime”; uptime; echo \n ; ps -eo pcpu,pid,args | sort -k 1 -r |grep -v watch | head -10; echo “\n” ; tail /var/log/cron | grep “check_load” “

DISK SPACE COMMAND

Find files based and sorted on Size
# find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk ‘{ print $NF “: ” $5 }’  | sort -nrk 2,2

Killing processes in one Line
# kill -9 `ps -ef | grep rsync| grep -v grep| awk ‘{print $2}’`

Check Memory Fault
# dd if=/dev/urandom bs=768304 of=/tmp/memtest count=1050
# md5sum /tmp/memtest; md5sum /tmp/memtest; md5sum /tmp/memtest

Repair Mysql MYISAM File
# myisamchk –force –sort_buffer_size=64M –key_buffer_size=16M –read_buffer_size=8M –write_buffer_size=8M ../data/phplists/phplist_linktrack.MYI

Command to make Services off in Defined Level on Linux
# chkconfig –list | awk ‘{print $1}’ | cut -d: -f1 | grep -vE ‘^crond|^network|^sshd|^syslog|^iptables’ | awk ‘{print $1}’ | while read line; do chkconfig  –level 3 $line off; count=`expr $count + 1`; echo $count $line;done

or Use this… One lineer  
# chkconfig –list | awk ‘{print $1}’| grep -vE ‘^crond|^network|^sshd|^syslog|^iptables’ | xargs -i chkconfig –level 3 {} off

Counting Hits from Web Server Access log
# awk ‘{print $1}’ /opt/indian.com/access_log | grep -vE ‘^:|^common|^-‘ | sort | uniq -c | sort -nr > /var/www/reports/ips/indian.txt

or  # awk ‘$1>10000 {print $1}’ /opt/indian.com/access_log | uniq -c | sort -nr  > /var/www/reports/ips/indian.txt

PERL Search and Replace Text Pattern using Perl On linux Platform
# find . -type f -name “*.html” | xargs perl -pi~ -e ‘s/\/js\/active18\//\/read\/js\/active18\//’

BACKUP SCRIPTS

15 2 * * * root /usr/bin/mysqldump -u root -pPASSWORD –all-databases | gzip > /mnt/disk2/database_`data ‘ %m-%d-%Y’`.sql.gz  

Details about SUID, SGID and Sticky bit permission on linux os
* SUID or setuid: change user ID on execution. If setuid bit is set, when the file will be executed by a user, the process will have the same rights as the owner of the file being executed.
* SGID or setgid: change group ID on execution. Same as above, but inherits rights of the group of the owner of the file on execution. For directories it also may mean that when a new file is created in the directory it will inherit the group of the directory (and not of the user who created the file).
* Sticky bit. It was used to trigger process to “stick” in memory after it is finished, now this usage is obsolete. Currently its use is system dependant and it is mostly used to suppress deletion of the files that belong to other users in the folder where you have “write” access to.

Numeric representation :

Octal digit Binary value Meaning
0 000 setuid, setgid, sticky bits are cleared
1 001 sticky bit is set
2 010 setgid bit is set
3 011 setgid and sticky bits are set
4 100 setuid bit is set
5 101 setuid and sticky bits are set
6 110 setuid and setgid bits are set
7 111 setuid, setgid, sticky bits are set

file : 2644
dir : 2755

Textual representation :

SUID If set, then replaces “x” in the owner permissions to “s”, if owner has execute permissions, or to “S” otherwise. Examples:
-rws—— both owner execute and SUID are set
-r-S—— SUID is set, but owner execute is not set

SGID If set, then replaces “x” in the group permissions to “s”, if group has execute permissions, or to “S” otherwise. Examples:
-rwxrws— both group execute and SGID are set
-rwxr-S— SGID is set, but group execute is not set

Sticky If set, then replaces “x” in the others permissions to “t”, if others have execute permissions, or to “T” otherwise. Examples:
-rwxrwxrwt both others execute and sticky bit are set
-rwxrwxr-T sticky bit is set, but others execute is not set

drwxrwxrwt – Sticky Bits – chmod 1777
drwsrwxrwx – SUID set – chmod 4777
drwxrwsrwx – SGID set – chmod 2777

Procedure to add a swap file

You need to use dd command to create swapfile. Next you need to use mkswap command to set up a Linux swap area on a device or in a file.

a) Login as the root user

b) Type following command to create 512MB swap file (1024 * 512MB = 524288 block size):

# dd if=/dev/zero of=/swapfile1 bs=1024 count=524288

c) Set up a Linux swap area:

# mkswap /swapfile1

d) Activate /swapfile1 swap space immediately:

# swapon /swapfile1

e) To activate /swapfile1 after Linux system reboot, add entry to /etc/fstab file. Open this file using text editor such as vi:

# vi /etc/fstab

Append following line:

/swapfile1 swap swap defaults 0 0

So next time Linux comes up after reboot, it enables the new swap file for you automatically.

g) How do I verify swap is activated or not?

Simply use free command:

$ free -m

strace
October 23rd, 2008 No comments

strace -q -f -c -p

strace -f verbose=all -e write=all -o /tmp/strace.log -p [pid]

strace -T -t -q -f -e trace=file,open,close,read -o /tmp/prod.strace.log -p <oms or ui pid>

-f traces all child processes as they are created byt he currently traced process as a result of the fork() system call.

-e is a qualifying expression which modifies which events to trace or how to race them

verbose=all dereferences structures for all system calls

write=all performs a full hexadecimal and ASCII dump of all the data written to all file descriptors

-o output file

-p process id to trace

Replace -p [pid] with [command] to trace a specific command.

Drop Caches
October 21st, 2008 No comments

Kernels 2.6.16 and newer provide a mechanism to have the kernel drop the page cache and/or inode and dentry caches on command, which can help free up a lot of memory. Now you can throw away that script that allocated a ton of memory just to get rid of the cache…

To use /proc/sys/vm/drop_caches, just echo a number to it.

To free pagecache:

# echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

# echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation and dirty objects are not freeable, the user should run “sync” first!

How to copy data in VI editor
January 20th, 2009 No comments

vi first  edit source file
then move your cursor to start of selection
ma       mark current position with letter a
then move your cursor to end of selection
y’a     yank to buffer x from current position to mark a
:e other  edit target file
move cursor to where you want the data
p       put from buffer x

Copying a block of text from one file to another in Vi
December 29th, 2008 No comments

To copy a block of text between files execute the commands:
Command     Explaination
1.         Edit the file containing the text you want to copy.
2.         Go to the top line to be copied.
3.     ma     Mark this line as mark “a”.
4.         Go to the bottom line to be copied
5.     y’a     Yank (y) the text from the current cursor location to the mark “a” (‘a)
6.     :split second-file     Open another window containing the second file. (This the file in which the text is to be inserted.)
7.         Go to the line where the insert is to occur. The text will be place after this line.
8.     p     Put the text after the cursor.

For Vi Editor
Do you like www.linuxnix.com ? Please consider supporting us by becoming a subscriber and get a Linux basics e-book for free.
unix_vi_editor

VI editor is the default file editor in most of the Linux/Nix machines. It is having great capabilities to edit a file with in few key strokes.

Lets start with some general information and then move on to some good things what vi editor can do for you while editing a file.
1. Vi stands for visual.
2. Vi have its variants like vim which stands for Vi IMproved, VimX11 for gui and winvi for MS windows.
3. Vi is the most popular editor and next most popular editor is gedit.
4. Do you know there is a book on VI editor from orally which is of 600+ pages.
5. Some other editors which will do the work of editing files are neno, pico, gedit, emacs, joe, nedit, ed etc.

Learning vi editor and remembering them is a very a easy task if you learn it in a systematic way.
a. Modes of VI
b. Navigational commands
c. Editing commands.
d. Search and Replace
e. Save and quiting a file.

a. Modes of VI :
Vi have two mode of operation.
1. Command mode
2. Inserting mode

Command mode :
Vi editor begins in command mode, where cursor movement(navigation in the file) and editing occur. To enter in to command mode from Inserting mode press esc button.

Inserting mode :
Used for entering text, this is similar to notepad in Windows. To enter in to inserting mode you can use any of the following.
i or I => present line
o => one line down the present line
O => one line above

Note : All comments will work in command mode only.

b. Navigational commands :
1. Character navigation k, h, l and j
h => To move one character left.
j => To move one line down.
k => To move one line up.
l => To move one character right.

How to use above commands in clever way?
Examples :
6j => to move 6 lines down from the present courser.
7k => to move 7 lines above from the present courser.

2. Word Navigation
w => word forward.
e =>word forward, but end of the word.
b => one word backward.

Examples :
32w => To move 32 words forward
6b => To move 6 words back.

3. Setting (nu) mbering to lines
:set nu
Removing of (nonu)mbering to lines
:set nonu

4. Moving paragraphs
move one paragraph up => {{
move one paragraph down => }}

5. Moving page up/down
For up => ctrl+b
For down => ctrl+f

6. Moving start/end of the file
Starting of the file(first line => [[
End of the file(last line) => ]]

7. Going to any line :
:lineno

Example :
If we want to go to 56 line then type
:56

c. Editing commands

8. Replace one letter
Replace one letter => r
Delete one letter => x

>9. Editing one word
Edit one word => cw
Delete one word => dw

10. Editing one line
Editing a line from courser to the end of that line => d$

11. Cutting
deleting(cutting) one line => dd

Examples :
2dd(deleting/cutting two lines)

12. Pasting
Pasting a line below the courser => p
Pasting a line above the courser => P

13. Coping
Copying one line => yy
Copying n lines => nyy

14. Special commands
joining lines => J
undoing things => u
repeating previous command => .

d. Search and replace

15. Search for a term /term

Example : If you want to search for suresh then press /suresh enter
/suresh
Moving to next occurrence, press “n” with out quotes moving to previous occurrence, press “N” with out quotes.

16. Searching and replacing a term(here separator is / )
:%s/searchterm/replaceterm/g
change default separator
:%s_/home/surya/grade_/home/testing/dest_g

To search and replace particular term from given line to other given line.
:%s294,304/sahana/xyz/g

e)Save and quiting a file
:w => save the file
:q => quit the file
:wq => save and quit
:w! => force save the file
:q! => force quit with out save
:wq => save and quit forcefully

==========================================================================================================================================================
Disk Space issues

mpstat -P ALL

cat /proc/interrupts

iostat -kd 1
iostat -c -t
iostat -c -t 2 10
iostat -c 3
iostat -d 5
iostat -x 1

sar -b

vmstat -S M 2 5

Find process causing high iowait

netstat -autpn | grep :80
netstat -autpn | grep :3306
netstat -na
netstat -an|grep :80|sort|more
netstat -an|grep ESTABLISHED
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -nr | more
ps ax | awk ‘$3 ~ /^D/ { print $0 }’
netstat -an | grep :80 | wc -l

DO NOT DO THIS until you are certain what is going on.

echo 100 > /proc/sys/vm/inactive_clean_percent
echo 2 10 20 > /proc/sys/vm/pagecache

==========================================================================================================================================================

To View Or List Only Directories In Linux?
Do you like www.linuxnix.com ? Please consider supporting us by becoming a subscriber and get a Linux basics e-book for free.

How to view/list only directories in Linux?
Ans : This can be achieved in two ways
1. Through ls command
2. Through find command

With ls we have to use grep to get the directory listings.
Ls –l grep ^d

Example :
[root@test surendra_a]# ls -l grep ^d
d——— 2 surendra_a surendra_a 4096 Sep 8 09:54 HTWFAIP
drwxrwxr-x 2 surendra_a root 4096 Nov 27 12:30 LinuxCBT – RHEL5
drwxrwxr-x 2 surendra_a root 4096 Oct 12 16:40 Software
[root@test surendra_a]#

With find we can have more controle on how to display only directories.

A. To display all the directories and sub-directories in present directory
#find . -type d

B. Displaying only directories in present directory
#find /root/ -type d –maxdepth 1

C. Displaying just directories in present directry and its sub-directories
#find /root/ -type d –maxdepth 2

 

How TO kill to Process

a) pgrep httpd | xargs kill -9

b) kill -9 `ps -ef | grep ‘httpd’ | grep -v grep | awk ‘{print $2}’`

Top to Cpu Usage commands

ps auxf | sort -nr -k 3 | head -10

Top processes consume RAM Memory

ps auxf | sort -nr -k 4 | head -10

Find and delete / remove files older 7 days / 1 week on Linux

find /root/server -type f -mtime +7 -exec rm -rf {} \;

Remove files older 90 days / 3 months / 1 quarter

find /var/ -type f -mtime +90 -exec rm -rf {} \;

Remove files older 365

find /var/ -type f -mtime +365 -exec rm -rf {} \;

check your public IP

n Linux, we have some best ways to check Public IP for that server.

1. curl ifconfig.me

2. wget -qO- ipecho.net/plain

3. wget -qO- ifconfig.me/ip

check your public IP