{"id":3472,"date":"2014-08-21T09:29:50","date_gmt":"2014-08-21T01:29:50","guid":{"rendered":"http:\/\/rmohan.com\/?p=3472"},"modified":"2014-08-21T10:10:26","modified_gmt":"2014-08-21T02:10:26","slug":"shell-samples","status":"publish","type":"post","link":"https:\/\/mohan.sg\/?p=3472","title":{"rendered":"shell samples"},"content":{"rendered":"

awk examples<\/h1>\n

\u00a0cat 03092608.LOG|grep XYZ|awk -F”.” ‘{print $2}’ |sort|awk ‘{tot += 1 } $1 != prev {print prev,tot;tot=0;prev=$1}END{print prev,tot;tot=0;prev=$1}’<\/p>\n

\n

adhocs<\/h1>\n

1.\u00a0find_processes_using_ipcs.ksh<\/strong><\/p>\n

 <\/p>\n

ipcs -bop|grep `whoami`|awk ‘{print $10}’|egrep -v “^0|^$”|sort -u|awk ‘{print “ps -ef|grep “, \u00a0$1}’|ksh<\/p>\n

2. iostat<\/strong><\/div>\n
\u00a0 iostat -xn 4 30<\/div>\n
<\/div>\n
3. jmap<\/strong><\/div>\n
\u00a0\u00a0jmap -heap:format=b <pid><\/div>\n
<\/div>\n
4. monitorMemLeaking<\/strong><\/div>\n
\n
if [ $# -lt 1 ]<\/div>\n
then<\/div>\n
\u00a0 echo “Usage: $0 [pid]”<\/div>\n
\u00a0 exit<\/div>\n
fi<\/div>\n
<\/div>\n
PID=$1<\/div>\n
<\/div>\n
while true<\/div>\n
do<\/div>\n
\u00a0 vsz1=`ps -eo vsz,rss,pid|grep “$PID$” |awk ‘{print $2}’`<\/div>\n
\u00a0 rss1=`ps -eo vsz,rss,pid|grep “$PID$” |awk ‘{print $3}’`<\/div>\n
\u00a0 echo “vsz1=$vsz1, rss1=$rss1… wait for 5 mins\\n”<\/div>\n
\u00a0 sleep 300<\/div>\n
\u00a0 vsz2=`ps -eo vsz,rss,pid|grep “$PID$” |awk ‘{print $2}’`<\/div>\n
\u00a0 rss2=`ps -eo vsz,rss,pid|grep “$PID$” |awk ‘{print $3}’`<\/div>\n
\u00a0 echo “vsz2=$vsz2, rss2=$rss2\\n”<\/div>\n
\u00a0 let vszLeakingRate=”(${vsz2} – ${vsz1}) \/ 5″<\/div>\n
\u00a0 let rssLeakingRate=”(${rss2} – ${rss1}) \/ 5″<\/div>\n
\u00a0 echo “PID=$PID, vszLeakingRate = $vszLeakingRate (KB)\/ per min, rssLeakingRate=$rssLeakingRate (KB)\/per min\\n\\n”<\/div>\n
done<\/div>\n<\/div>\n
<\/div>\n
5.\u00a0port_check.ksh<\/strong><\/div>\n
#!\/usr\/bin\/ksh<\/div>\n
#If lsof or nmap is available, we might be able to do this easily<\/div>\n
<\/div>\n
echo “which port?>\\c “<\/div>\n
read port<\/div>\n
<\/div>\n
#for pid in `ps -ef -o pid | tail +2`<\/div>\n
#do<\/div>\n
#foundport=`\/usr\/bin\/pfiles $pid 2>&1 | grep “sockname:” | grep “port: $port$”`<\/div>\n
#if [ “$foundport” != “” ]<\/div>\n
#then<\/div>\n
# \u00a0 \u00a0echo “proc: $pid, $foundport”<\/div>\n
#fi<\/div>\n
#done<\/div>\n
#<\/div>\n
echo “Going to find out which process is using port: $port”<\/div>\n
<\/div>\n
ps -ef|grep `whoami`|grep -v grep|awk ‘{print $2}’|while read i<\/div>\n
do<\/div>\n
# \u00a0 \u00a0check=`pfiles $i 2>&-|grep “sockname:” | grep “port:” | grep $port$”`<\/div>\n
\u00a0 \u00a0 check=`pfiles $i 2>&-| grep $port$”`<\/div>\n
\u00a0 \u00a0 echo “$check.\\c”<\/div>\n
\u00a0 \u00a0 if [ ! -z “${check}” ]<\/div>\n
\u00a0 \u00a0 then<\/div>\n
\u00a0 \u00a0 \u00a0 \u00a0 print “\\n”<\/div>\n
\u00a0 \u00a0 \u00a0 \u00a0 print “The port number $port used by process ID=$i”<\/div>\n
\u00a0 \u00a0 \u00a0 \u00a0 #break<\/div>\n
\u00a0 \u00a0 fi<\/div>\n
\u00a0 \u00a0 check=””<\/div>\n
done<\/div>\n
exit<\/div>\n
<\/div>\n
6. Difference between SIZE and RSS of a process?<\/strong><\/div>\n
The basic difference is that SIZE represents the total address space size\u00a0of a process, while RSS is the actual physical memory currently occupied\u00a0by pages belonging to that process (roughly speaking). Regions of an\u00a0address space mapped to something don’t necessarily occupy any RAM.<\/div>\n
<\/div>\n
7.\u00a0RSS_vs_real_memory.txt<\/strong><\/div>\n
\n
Using the ps command it is possible to look at the resident set size\u00a0(rss) parameter of each process… it is my understanding that\u00a0the value of rss indicates how much real memory the process is using…\u00a0\u00a0if i sum up all of the rss values for each process, i sometimes find\u00a0that the sum is greater than the available real memory installed\u00a0in the system…<\/div>\n
<\/div>\n<\/div>\n
The non-modifiable parts (the compiled code & const data) of\u00a0shared libraries & ELF binaries are only loaed into memory once and\u00a0those are shared among all the processes. \u00a0The modifiable parts\u00a0(non-const data, etc.) are copied into each processes’s address space.<\/div>\n
\/usr\/proc\/bin\/pmap -x will show you how much is shared and how much is\u00a0not.<\/div>\n
<\/div>\n
8. vmstat<\/strong><\/div>\n
\n
The “free” column tells you how much real memory is *not* being used.\u00a0That’s usually more important — if that gets low, then you need more\u00a0memory. \u00a0And if it’s always very high, you could be wasting memory.\u00a0The “top” command also shows this information.<\/div>\n
<\/div>\n
The problem with top and vmstat is that they do not account for the\u00a0real memory that the OS keeps cached. Memory that the OS caches\u00a0looks to these programs like active memory. In a sense it is active\u00a0memory since the OS is hanging on to it waiting for new requests…\u00a0but, it is deceptive since the memory is actually available for use…<\/div>\n<\/div>\n
Also, just because free memory may be reported as low on a system does\u00a0not necessarily indicate that the machine needs more memory… other\u00a0things such as heavy paging and the scanner running often are better\u00a0indicators that there is not enough real memory… this is due to the\u00a0OS caching mentioned above… the system may seem low on memory but\u00a0the OS is holding onto blocks of memory that are actually available\u00a0for use by processes<\/div>\n
<\/div>\n
9.\u00a0socket-status.txt<\/strong><\/div>\n
State \u00a0 Description<\/div>\n
LISTEN \u00a0accepting connections<\/div>\n
ESTABLISHED \u00a0 \u00a0 connection up and passing data<\/div>\n
SYN_SENT \u00a0 \u00a0 \u00a0 \u00a0TCP; session has been requested by us; waiting for reply from remote endpoint<\/div>\n
SYN_RECV \u00a0 \u00a0 \u00a0 \u00a0TCP; session has been requested by a remote endpoint for a socket on which we were listening<\/div>\n
LAST_ACK \u00a0 \u00a0 \u00a0 \u00a0TCP; our socket is closed; remote endpoint has also shut down; we are waiting for a final acknowledgement<\/div>\n
CLOSE_WAIT \u00a0 \u00a0 \u00a0TCP; remote endpoint has shut down; the kernel is waiting for the application to close the socket<\/div>\n
TIME_WAIT \u00a0 \u00a0 \u00a0 TCP; socket is waiting after closing for any packets left on the network<\/div>\n
CLOSED \u00a0socket is not being used (FIXME. What does mean?)<\/div>\n
CLOSING TCP; our socket is shut down; remote endpoint is shut down; not all data has been sent<\/div>\n
FIN_WAIT1 \u00a0 \u00a0 \u00a0 TCP; our socket has closed; we are in the process of tearing down the connection<\/div>\n
FIN_WAIT2 \u00a0 \u00a0 \u00a0 TCP; the connection has been closed; our socket is waiting for the remote endpoint to shut down<\/div>\n
<\/div>\n
10. ssh setup<\/strong><\/div>\n
\n
ssh-keygen generate 2 files which go hand-in-hand. The private key file (id_rsa) should\u00a0reside on my $HOME\/.ssh, in order to encrypt things; the putlib key file(id_rsa.pub)\u00a0should be appended in remote host at its $HOME\/.ssh\/authorized_keys file.<\/div>\n
<\/div>\n
When I ssh to remote host, remote host’s daemon (sshd) will decrypt my logon\u00a0using the public key from $HOME\/.ssh\/authorized_keys file. This will allow me<\/div>\n
to logon without prompting for password.<\/div>\n<\/div>\n
\n
This is the logic:<\/div>\n
1)ssh remoteHost<\/div>\n
2)ssh look at $HOME\/.ssh\/id_rsa and also $HOME\/.ssh\/known_hosts<\/div>\n
\u00a0 create request and send to remoteHost<\/div>\n
3)remoteHost will look at its $HOME\/.ssh\/authorized_keys and if it does see one entry<\/div>\n
\u00a0 which is able to decipher the request, connect is OK<\/div>\n
4)ssh on the originating host will put an entry to $HOME\/.ssh\/known_hosts if it’s not there<\/div>\n<\/div>\n
<\/div>\n
11.\u00a0ucbps<\/strong><\/div>\n
#Long form of ps command<\/div>\n
\/usr\/ucb\/ps -auxww<\/div>\n
<\/div>\n
12.\u00a0who_did_what<\/strong><\/div>\n
#!\/bin\/ksh<\/div>\n
#<\/div>\n
#####<\/div>\n
<\/div>\n
STRING1=$1<\/div>\n
STRING2=$2<\/div>\n
STDOUT=\/tmp\/stdout.out<\/div>\n
<\/div>\n
echo “” > $STDOUT<\/div>\n
cd $HOME<\/div>\n
#for i in `ls $HOME\/.history.[a-z]*`<\/div>\n
for i in `ls .history.[a-z]*`<\/div>\n
do<\/div>\n
\u00a0 if (( `strings $i | grep -i $STRING1 | grep -i $STRING2 | wc -l` > “0” ))<\/div>\n
\u00a0 then<\/div>\n
\u00a0 \u00a0 echo ” ## `ls $i` ## ” \u00a0# >> $STDOUT<\/div>\n
\u00a0 \u00a0 strings $i | grep -i $STRING1 | grep -i $STRING2 # >> $STDOUT<\/div>\n
\u00a0 fi<\/div>\n
done<\/div>\n
<\/div>\n

 <\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

awk examples <\/p>\n

cat 03092608.LOG|grep XYZ|awk -F”.” ‘{print $2}’ |sort|awk ‘{tot += 1 } $1 != prev {print prev,tot;tot=0;prev=$1}END{print prev,tot;tot=0;prev=$1}’<\/p>\n

adhocs <\/p>\n

1. find_processes_using_ipcs.ksh<\/p>\n

 <\/p>\n

ipcs -bop|grep `whoami`|awk ‘{print $10}’|egrep -v “^0|^$”|sort -u|awk ‘{print “ps -ef|grep “, $1}’|ksh<\/p>\n

2. iostat iostat -xn 4 30 3. jmap jmap -heap:format=b <pid> 4. monitorMemLeaking if [ $# […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[47],"tags":[],"_links":{"self":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3472"}],"collection":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3472"}],"version-history":[{"count":1,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3472\/revisions"}],"predecessor-version":[{"id":3473,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3472\/revisions\/3473"}],"wp:attachment":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3472"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3472"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3472"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}