|
|
The GNU version of the tar archiving utility (and other old version of tar) can be use through network over ssh session.
1. Tarred file transfer
Scp is very inefficient when copying many small files because it sets up a separate transfer for each file. It is possible to solve this problem by creating a single archive containing all the files and piping it over SSH:
$ tar zcf – tobearchived | \
ssh user@destination_server_ip \
‘tar zxf -‘
This will put ‘tobearchived’ in the server’s home directory. It is possible to use the -C option to put the files somewhere else. (The ‘z’ tells tar to use gzip compression. To use bzip2 compressio, replace ‘z’ with ‘j’).
Copying from the server is just like the above, but in reverse:
$ ssh user@source_server_ip \
‘tar zcf – tobearchived’ | \
tar zxf –
2. Offsite backups
This is pretty much the same as above, except we want to transfer a bunch of files and leave them as a tarball on the server rather than as a bunch of files.
$ tar zcf – tobearchived | \
ssh user@destination_server_ip \
‘cat – > tobearchived.tar.gz’
It is possible to encrypt the tarball (it GPG keyring is set up):
$ tar zcf – tobearchived | \
gpg -e | \
ssh user@destination_server_ip \
‘cat – > tobearchived.tar.gz.gpg’
It is also possible to use a symmetric cipher:
$ tar zcf – tobearchived | \
openssl enc -rc4 | \
ssh user@destination_server_ip \
‘cat – > tobearchived.tar.gz.rc4’
It is also possible to choose a different cipher:
$ ssh user@destination_server_ip \
‘cat tobearchived.tar.gz.rc4’ | \
openssl enc -rc4 -d -out tobearchived.tar.gz
3. Hard drive backup/mirror
This will copy the entire drive into a file on the remote machine:
$ dd if=/dev/sdX | \
ssh user@destination_server_ip \
‘dd of=sdX.img’
To restore a local drive from the image on the server, reverse the command:
$ ssh user@source_server_ip \
‘dd if=sdX.img’ | \
dd of=/dev/sdX
Note that to read or write block devices requires you to be root. Be very careful with dd as it can be very ‘deadly’ if used carelessly.
4. Run a local script remotely
This command will run a local file script.sh on the remote server and display any output locally:
$ ssh user@destination_server_ip \
‘bash -s’ < script.sh
Using scp
The basic syntax of scp is very simple to memorize. It looks like this
$ scp source_file_path destination_file_path
Depending on the host, the file path should include the full host address, port number, username and password along with the directory path.
So if you are “sending” file from your local machine to a remote machine (uploading) the syntax would look like this
$ scp ~/my_local_file.txt user@remote_host.com:/some/remote/directory
When copying file from remote host to local host (downloading), its looks just the reverse
$ scp user@remote_host.com:/some/remote/directory ~/my_local_file.txt
# just download the file
$ scp user@192.168.1.3:/some/path/file.txt .
That is pretty much about using scp for regular tasks. Apart from it, there are a couple of extra options and functions that scp supports. Lets take a quick overview of those.
And yes, by default scp will always overwrite files on the destination. If you need to avoid that, use a more powerful tool called rsync.
1. Verbose output
With verbose output, the scp program would output lots of information about what it does in the background. This is often useful when the program fails or is unable to complete the request. The verbose output would then indicate the exact point where the program ran into issues.
$ scp -v ~/test.txt root@192.168.1.3:/root/help2356.txt
Executing: program /usr/bin/ssh host 192.168.1.3, user root, command scp -v -t /root/help2356.txt
OpenSSH_6.2p2 Ubuntu-6ubuntu0.1, OpenSSL 1.0.1e 11 Feb 2013
debug1: Reading configuration data /home/enlightened/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.168.1.3 [192.168.1.3] port 22.
debug1: Connection established.
….. OUTPUT TRUNCATED
2. Transfer multiple files
Multiple files can be specified separated by a space like this
$ scp foo.txt bar.txt username@remotehost:/path/directory/
To copy multiple files from remote host to current local directory
$ scp username@remotehost:/path/directory/\{foo.txt,bar.txt\} .
$ scp root@192.168.1.3:~/\{abc.log,cde.txt\} .
3. Copy entire directory (recursively)
To copy an entire directory from one host to another use the r switch and specify the directory
$ scp -v -r ~/Downloads root@192.168.1.3:/root/Downloads
4. Copy files across 2 remote hosts
Scp can copy files from 1 remote host to another remote host as well.
$ scp user1@remotehost1:/some/remote/dir/foobar.txt user2@remotehost2:/some/remote/dir/
5. Speed up the transfer with compression
A super cool option to speed up the transfer to save time and bandwidth. All you need to do is use the C option to enable compression. The files are compressed on the fly and decompressed on the destination.
$ scp -vrC ~/Downloads root@192.168.1.3:/root/Downloads
In the above example we moved the entire directory with compression enabled. The speed gain would depend on how much the files could be compressed.
6. Limit the bandwidth usage
If you do not want scp to take up the entire available bandwidth, then use the l option to limit the maximum speed in Kbit/s.
$ scp -vrC -l 400 ~/Downloads root@192.168.1.3:/root/Downloads
7. Connect to a different port number on remote host
If the remote server has ssh daemon running on a different port (default is 22), then you need to tell scp to use that particular port number using the ‘-P’ option.
$ scp -vC -P 2200 ~/test.txt root@192.168.1.3:/some/path/test.txt
8. Preserve file attributes
The ‘-p’ option (smallcase), would preserve modification times, access times, and modes from the original file.
$ scp -C -p ~/test.txt root@192.168.1.3:/some/path/test.txt
9. Quiet mode
In quiet mode ( ‘-q’ option ), the scp output would get suppressed, and would disable the progress meter as well as warning and diagnostic messages.
$ scp -vCq ~/test.txt root@192.168.1.3:/some/path/test.txt
10. Specify identity file
When using key based (passwordless) authentication, you would need to specify the identity file which contains the private key. This option is directly passed to the ssh command and works the same way.
$ scp -vCq -i private_key.pem ~/test.txt root@192.168.1.3:/some/path/test.txt
11. Use a different ssh_config file
Use the ‘-F’ option to specify a different ssh_config file.
$ scp -vC -F /home/user/my_ssh_config ~/test.txt root@192.168.1.3:/some/path/test.txt
12. Use different cipher
Scp by default uses the AES cipher/encryption. Sometimes you might want to use a different cipher. Using a different cipher can speed up the transfer process. For example blowfish and arcfour are known to be faster than AES (but less secure).
$ scp -c blowfish -C ~/local_file.txt username@remotehost:/remote/path/file.txt
In the above example we use the blowfish cipher along with compression. This can give significant speed boost depending on available bandwidth.
[gview file=”http://rmohan.com/wp-content/uploads/2014/08/awk.cheat_.sheet_.pdf”]
[gview file=”http://rmohan.com/wp-content/uploads/2014/08/bash-history-cheat-sheet.pdf”] [gview file=”http://rmohan.com/wp-content/uploads/2014/08/bash-vi-editing-mode-cheat-sheet.pdf”] [gview file=”http://rmohan.com/wp-content/uploads/2014/08/perl.predefined.variables.pdf”] [gview file=”http://rmohan.com/wp-content/uploads/2014/08/sed.stream.editor.cheat_.sheet_.pdf”]
.-----------------------------------------------------------------------.
| |
| AWK Cheat Sheet |
| |
'-----------------------------------------------------------------------'
| Peteris Krumins (peter@catonmat.net), 2007.08.22 |
| http://www.catonmat.net - good coders code, great reuse |
'-----------------------------------------------------------------------'
===================== Predefined Variable Summary =====================
.-------------+-----------------------------------.---------------------.
| | | Support: |
| Variable | Description '-----.-------.-------'
| | | AWK | NAWK | GAWK |
'-------------+-----------------------------------+-----+-------+-------'
| FS | Input Field Separator, a space by | + | + | + |
| | default. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| OFS | Output Field Separator, a space | + | + | + |
| | by default. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| NF | The Number of Fields in the | + | + | + |
| | current input record. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| NR | The total Number of input Records | + | + | + |
| | seen so far. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| RS | Record Separator, a newline by | + | + | + |
| | default. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| ORS | Output Record Separator, a | + | + | + |
| | newline by default. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| FILENAME | The name of the current input | | | |
| | file. If no files are specified | | | |
| | on the command line, the value of | | | |
| | FILENAME is "-". However, | + | + | + |
| | FILENAME is undefined inside the | | | |
| | BEGIN block (unless set by | | | |
| | getline). | | | |
'-------------+-----------------------------------+-----+-------+-------'
| ARGC | The number of command line | | | |
| | arguments (does not include | | | |
| | options to gawk, or the program | - | + | + |
| | source). Dynamically changing the | | | |
| | contents of ARGV control the | - | + | + |
| | files used for data. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| ARGV | Array of command line arguments. | | | |
| | The array is indexed from 0 to | - | + | + |
| | ARGC - 1. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| ARGIND | The index in ARGV of the current | - | - | + |
| | file being processed. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| BINMODE | On non-POSIX systems, specifies | | | |
| | use of "binary" mode for all file | | | |
| | I/O.Numeric values of 1, 2, or 3, | | | |
| | specify that input files, output | | | |
| | files, or all files, respectively,| | | |
| | should use binary I/O. String | | | |
| | values of "r", or "w" specify | - | - | + |
| | that input files, or output files,| | | |
| | respectively, should use binary | | | |
| | I/O. String values of "rw" or | | | |
| | "wr" specify that all files | | | |
| | should use binary I/O. Any other | | | |
| | string value is treated as "rw", | | | |
| | but generates a warning message. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| CONVFMT | The CONVFMT variable is used to | | | |
| | specify the format when | - | - | + |
| | converting a number to a string. | | | |
| | Default: "%.6g" | | | |
'-------------+-----------------------------------+-----+-------+-------'
| ENVIRON | An array containing the values | - | - | + |
| | of the current environment. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| ERRNO | If a system error occurs either | | | |
| | doing a redirection for getline, | | | |
| | during a read for getline, or | | | |
| | during a close(), then ERRNO will | - | - | + |
| | contain a string describing the | | | |
| | error. The value is subject to | | | |
| | translation in non-English locales. | | |
'-------------+-----------------------------------+-----+-------+-------'
| FIELDWIDTHS | A white-space separated list of | | | |
| | fieldwidths. When set, gawk | | | |
| | parses the input into fields of | - | - | + |
| | fixed width, instead of using the | | | |
| | value of the FS variable as the | | | |
| | field separator. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| FNR | Contains number of lines read, | - | + | + |
| | but is reset for each file read. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| IGNORECASE | Controls the case-sensitivity of | | | |
| | all regular expression and string | | | |
| | operations. If IGNORECASE has a | | | |
| | non-zero value, then string | | | |
| | comparisons and pattern matching | | | |
| | in rules, field splitting | | | |
| | with FS, record separating | | | |
| | with RS, regular expression | | | |
| | matching with ~ and !~, and the | - | - | + |
| | gensub(), gsub(), index(), | | | |
| | match(), split(), and sub() | | | |
| | built-in functions all ignore | | | |
| | case when doing regular | | | |
| | expression operations. | | | |
| | NOTE: Array subscripting is not | | | |
| | affected. However, the asort() | | | |
| | and asorti() functions are | | | |
| | affected | | | |
'-------------+-----------------------------------+-----+-------+-------'
| LINT | Provides dynamic control of the | | | |
| | --lint option from within an AWK | - | - | + |
| | program. When true, gawk prints | | | |
| | lint warnings. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| OFMT | The default output format for | - | + | + |
| | numbers. Default: "%.6g" | | | |
'-------------+-----------------------------------+-----+-------+-------'
| PROCINFO | The elements of this array | | | |
| | provide access to information | | | |
| | about the running AWK program. | | | |
| | PROCINFO["egid"]: | | | |
| | the value of the getegid(2) | | | |
| | system call. | | | |
| | PROCINFO["euid"]: | | | |
| | the value of the geteuid(2) | | | |
| | system call. | | | |
| | PROCINFO["FS"]: | | | |
| | "FS" if field splitting with FS | | | |
| | is in effect, or "FIELDWIDTHS" | | | |
| | if field splitting with | | | |
| | FIELDWIDTHS is in effect. | | | |
| | PROCINFO["gid"]: | - | - | + |
| | the value of the getgid(2) system | | | |
| | call. | | | |
| | PROCINFO["pgrpid"]: | | | |
| | the process group ID of the | | | |
| | current process. | | | |
| | PROCINFO["pid"]: | | | |
| | the process ID of the current | | | |
| | process. | | | |
| | PROCINFO["ppid"]: | | | |
| | the parent process ID of the | | | |
| | current process. | | | |
| | PROCINFO["uid"] | | | |
| | the value of the getuid(2) system | | | |
| | call. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| RT | The record terminator. Gawk sets | | | |
| | RT to the input text that matched | - | - | + |
| | the character or regular | | | |
| | expression specified by RS. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| RSTART | The index of the first character | - | + | + |
| | matched by match(); 0 if no match.| | | |
'-------------+-----------------------------------+-----+-------+-------'
| RLENGTH | The length of the string matched | - | + | + |
| | by match(); -1 if no match. | | | |
'-------------+-----------------------------------+-----+-------+-------'
| SUBSEP | The character used to separate | | | |
| | multiple subscripts in array | | | |
| | elements.Default: "\034" | - | + | + |
| | (non-printable character, | | | |
| | dec: 28, hex: 1C) | | | |
'-------------+-----------------------------------+-----+-------+-------'
| TEXTDOMAIN | The text domain of the AWK | | | |
| | program; used to find the | - | - | + |
| | localized translations for the | | | |
| | program's strings. | | | |
'-------------'-----------------------------------'-----'-------'-------'
============================ I/O Statements ===========================
.---------------------.-------------------------------------------------.
| | |
| Statement | Description |
| | |
'---------------------+-------------------------------------------------'
| close(file [, how]) | Close file, pipe or co-process. The optional |
| | how should only be used when closing one end of |
| | a two-way pipe to a co-process. It must be a |
| | string value, either "to" or "from". |
'---------------------+-------------------------------------------------'
| getline | Set $0 from next input record; set NF, NR, FNR. |
| | Returns 0 on EOF and ?1 on an error. Upon an |
| | error, ERRNO contains a string describing the |
| | problem. |
'---------------------+-------------------------------------------------'
| getline <file | Set $0 from next record of file; set NF. |
'---------------------+-------------------------------------------------'
| getline var | Set var from next input record; set NR, FNR. |
'---------------------+-------------------------------------------------'
| getline var <file | Set var from next record of file. |
'---------------------+-------------------------------------------------'
| command | | Run command piping the output either into $0 or |
| getline [var] | var, as above. If using a pipe or co-process |
| | to getline, or from print or printf within a |
| | loop, you must use close() to create new |
| | instances |
'---------------------+-------------------------------------------------'
| command |& | Run command as a co-process piping the output |
| getline [var] | either into $0 or var, as above. Co-processes |
| | are a gawk extension. |
'---------------------+-------------------------------------------------'
| next | Stop processing the current input record. |
| | The next input record is read and processing |
| | starts over with the first pattern in the AWK |
| | program. If the end of the input data is |
| | reached, the END block(s), if any, are executed.|
'---------------------+-------------------------------------------------'
| nextfile | Stop processing the current input file. The |
| | next input record read comes from the next |
| | input file. FILENAME and ARGIND are updated, |
| | FNR is reset to 1, and processing starts over |
| | with the first pattern in the AWK program. If |
| | the end of the input data is reached, the END |
| | block(s), are executed. |
'---------------------+-------------------------------------------------'
| print | Prints the current record. The output record is |
| | terminated with the value of the ORS variable. |
'---------------------+-------------------------------------------------'
| print expr-list | Prints expressions. Each expression is |
| | separated by the value of the OFS variable. |
| | The output record is terminated with the value |
| | of the ORS variable. |
'---------------------+-------------------------------------------------'
| print expr-list | Prints expressions on file. Each expression is |
| >file | separated by the value of the OFS variable. The |
| | output record is terminated with the value of |
| | the ORS variable. |
'---------------------+-------------------------------------------------'
| printf fmt, | Format and print. |
| expr-list | |
'---------------------+-------------------------------------------------'
| printf fmt, | Format and print on file. |
| expr-list >file | |
'---------------------+-------------------------------------------------'
| system(cmd-line) | Execute the command cmd-line, and return the |
| | exit status. |
'---------------------+-------------------------------------------------'
| fflush([file]) | Flush any buffers associated with the open |
| | output file or pipe file. If file is missing, |
| | then stdout is flushed. If file is the null |
| | string, then all open output files and pipes |
| | have their buffers flushed. |
'---------------------+-------------------------------------------------'
| print ... >> file | Appends output to the file. |
'---------------------+-------------------------------------------------'
| print ... | command | Writes on a pipe. |
'---------------------+-------------------------------------------------'
| print ... |& | Sends data to a co-process. |
| command | |
'---------------------'-------------------------------------------------'
=========================== Numeric Functions =========================
.---------------------.-------------------------------------------------.
| | |
| Function | Description |
| | |
'---------------------+-------------------------------------------------'
| atan2(y, x) | Returns the arctangent of y/x in radians. |
'---------------------+-------------------------------------------------'
| cos(expr) | Returns the cosine of expr, which is in radians.|
'---------------------+-------------------------------------------------'
| exp(expr) | The exponential function. |
'---------------------+-------------------------------------------------'
| int(expr) | Truncates to integer. |
'---------------------+-------------------------------------------------'
| log(expr) | The natural logarithm function. |
'---------------------+-------------------------------------------------'
| rand() | Returns a random number N, between 0 and 1, |
| | such that 0 <= N < 1. |
'---------------------+-------------------------------------------------'
| sin(expr) | Returns the sine of expr, which is in radians. |
'---------------------+-------------------------------------------------'
| sqrt(expr) | The square root function. |
'---------------------+-------------------------------------------------'
| srand([expr]) | Uses expr as a new seed for the random number |
| | generator. If no expr is provided, the time of |
| | day is used. The return value is the previous |
| | seed for the random number generator. |
'---------------------'-------------------------------------------------'
====================== Bit Manipulation Functions =====================
.---------------------.-------------------------------------------------.
| | |
| Function | Description |
| | |
'---------------------+-------------------------------------------------'
| and(v1, v2) | Return the bitwise AND of the values provided |
| | by v1 and v2. |
'---------------------+-------------------------------------------------'
| compl(val) | Return the bitwise complement of val. |
'---------------------+-------------------------------------------------'
| lshift(val, count) | Return the value of val, shifted left by |
| | count bits. |
'---------------------+-------------------------------------------------'
| or(v1, v2) | Return the bitwise OR of the values provided by |
| | v1 and v2. |
'---------------------+-------------------------------------------------'
| rshift(val, count) | Return the value of val, shifted right by |
| | count bits. |
'---------------------+-------------------------------------------------'
| xor(v1, v2) | Return the bitwise XOR of the values provided |
| | by v1 and v2. |
'---------------------'-------------------------------------------------'
=========================== String Functions ==========================
.---------------------.-------------------------------------------------.
| | |
| Function | Description |
| | |
'---------------------+-------------------------------------------------'
| asort(s [, d]) | Returns the number of elements in the source |
| | array s. The contents of s are sorted using |
| | gawk's normal rules for comparing values, and |
| | the indexes of the sorted values of s are |
| | replaced with sequential integers starting with |
| | 1. If the optional destination array d is |
| | specified, then s is first duplicated into d, |
| | and then d is sorted, leaving the indexes of |
| | the source array s unchanged. |
'---------------------+-------------------------------------------------'
| asorti(s [, d]) | Returns the number of elements in the source |
| | array s. The behavior is the same as that of |
| | asort(), except that the array indices are |
| | used for sorting, not the array values. When |
| | done, the array is indexed numerically, and the |
| | values are those of the original indices. The |
| | original values are lost; thus provide a second |
| | array if you wish to preserve the original. |
'---------------------+-------------------------------------------------'
| gensub(r, s, | Search the target string t for matches of the |
| h [, t]) | regular expression r. If h is a string |
| | beginning with g or G, then replace all matches |
| | of r with s. Otherwise, h is a number |
| | indicating which match of r to replace. If t is |
| | not supplied, $0 is used instead. Within the |
| | replacement text s, the sequence \n, where n is |
| | a digit from 1 to 9, may be used to indicate |
| | just the text that matched the n'th |
| | parenthesized subexpression. The sequence \0 |
| | represents the entire matched text, as does the |
| | character &. Unlike sub() and gsub(), the |
| | modified string is returned as the result of |
| | the function, and the original target string |
| | is not changed. |
'---------------------+-------------------------------------------------'
| gsub(r, s [, t]) | For each substring matching the regular |
| | expression r in the string t, substitute the |
| | string s, and return the number of |
| | substitutions. If t is not supplied, use $0. |
| | An & in the replacement text is replaced with |
| | the text that was actually matched. Use \& to |
| | get a literal &. (This must be |
| | typed as "\\&") |
'---------------------+-------------------------------------------------'
| index(s, t) | Returns the index of the string t in the |
| | string s, or 0 if t is not present. (This |
| | implies that characterindices start at one.) |
'---------------------+-------------------------------------------------'
| length([s]) | Returns the length of the string s, or the |
| | length of $0 if s is not supplied. |
'---------------------+-------------------------------------------------'
| match(s, r [, a]) | Returns the position in s where the regular |
| | expression r occurs, or 0 if r is not present, |
| | and sets the values of RSTART and RLENGTH. |
| | Note that the argument order is the same as for |
| | the ~ operator: str ~ re. If array a is |
| | provided, a is cleared and then elements 1 |
| | through n are filled with the portions of s |
| | that match the corresponding parenthesized |
| | subexpression in r. The 0'th element of a |
| | contains the portion of s matched by the entire |
| | regular expression r. Subscripts a[n, "start"], |
| | and a[n, "length"] provide the starting index |
| | in the string and length respectively, of each |
| | matching substring. |
'---------------------+-------------------------------------------------'
| split(s, a [, r]) | Splits the string s into the array a on the |
| | regular expression r, and returns the number of |
| | fields. If r is omitted, FS is used instead. |
| | The array a is cleared first. Splitting behaves |
| | identically to field splitting. |
'---------------------+-------------------------------------------------'
| sprintf(fmt, | Prints expr-list according to fmt, and returns |
| expr-list) | the resulting string. |
'---------------------+-------------------------------------------------'
| strtonum(str) | Examines str, and returns its numeric value. |
| | If str begins with a leading 0, strtonum() |
| | assumes that str is an octal number. If str |
| | begins with a leading 0x or 0X, strtonum() |
| | assumes that str is a hexadecimal number. |
'---------------------+-------------------------------------------------'
| sub(r, s [, t]) | Just like gsub(), but only the first matching |
| | substring is replaced. |
'---------------------+-------------------------------------------------'
| substr(s, i [, n]) | Returns the at most n-character substring of s |
| | starting at i. If n is omitted, the rest of s |
| | is used. |
'---------------------+-------------------------------------------------'
| tolower(str) | Returns a copy of the string str, with all the |
| | upper-case characters in str translated to |
| | their corresponding lower-case counterparts. |
| | Non-alphabetic characters are left unchanged. |
'---------------------+-------------------------------------------------'
| toupper(str) | Returns a copy of the string str, with all the |
| | lower-case characters in str translated to |
| | their corresponding upper-case counterparts. |
| | Non-alphabetic characters are left unchanged. |
'---------------------'-------------------------------------------------'
============================ Time Functions ===========================
.---------------------.-------------------------------------------------.
| | |
| Function | Description |
| | |
'---------------------+-------------------------------------------------'
| mktime(datespec) | Turns datespec into a time stamp of the same |
| | form as returned by systime(). The datespec is |
| | a string of the form YYYY MM DD HH MM SS[ DST]. |
| | The contents of the string are six or seven |
| | numbers representing respectively the full year |
| | including century, the month from 1 to 12, the |
| | day of the month from 1 to 31, the hour of the |
| | day from 0 to 23, the minute from 0 to 59, and |
| | the second from 0 to 60, and an optional |
| | daylight saving flag. The values of these |
| | numbers need not be within the ranges |
| | specified; for example, an hour of -1 means 1 |
| | hour before midnight. The origin-zero Gregorian |
| | calendar is assumed, with year 0 preceding year |
| | 1 and year -1 preceding year 0. The time is |
| | assumed to be in the local timezone. If the |
| | daylight saving flag is positive, the time is |
| | assumed to be daylight saving time; if zero, |
| | the time is assumed to be standard time; and if |
| | negative (the default), mktime() attempts to |
| | determine whether daylight saving time is in |
| | effect for the specified time. If datespec does |
| | not contain enough elements or if the resulting |
| | time is out of range, mktime() returns -1. |
'---------------------+-------------------------------------------------'
| strftime([format | Formats timestamp according to the |
| [, timestamp]]) | specification in format. The timestamp should |
| | be of the same form as returned by systime(). |
| | If timestamp is missing, the current time of |
| | day is used.If format is missing, a default |
| | format equivalent to the output of date(1) is |
| | used. See the specification for the strftime() |
| | function in ANSI C for the format conversions |
| | that are guaranteed to be available. A |
| | public-domain version of strftime(3) and a man |
| | page for it come with gawk; if that version was |
| | used to build gawk, then all of the conversions |
| | described in that man page are available to |
| | gawk. |
'---------------------+-------------------------------------------------'
| systime() | Returns the current time of day as the number |
| | of seconds since the Epoch (1970-01-01 00:00:00 |
| | UTC on POSIX systems). |
'---------------------'-------------------------------------------------'
=============== Internationalization (I18N) Functions ================
.---------------------.-------------------------------------------------.
| | |
| Function | |
| | |
| Description | |
| | |
'---------------------+-------------------------------------------------'
| bindtextdomain(directory [, domain]) |
| |
| Specifies the directory where gawk looks for the .mo files. It |
| returns the directory where domain is ``bound.'' The default domain |
| is the value of TEXTDOMAIN. If directory is the null string (""), |
| then bindtextdomain() returns the current binding for the given domain|
'---------------------+-------------------------------------------------'
| dcgettext(string [, domain [, category]]) |
| |
| Returns the translation of string in text domain domain for locale |
| category category. The default value for domain is the current value |
| of TEXTDOMAIN. The default value for category is "LC_MESSAGES". If |
| you supply a value for category, it must be a string equal to one of |
| the known locale categories. You must also supply a text domain. Use |
| TEXTDOMAIN if you want to use the current domain. |
'---------------------+-------------------------------------------------'
| dcngettext(string1 , string2 , number [, domain [, category]]) |
| |
| Returns the plural form used for number of the translation of string1 |
| and string2 in text domain domain for locale category category. The |
| default value for domain is the current value of TEXTDOMAIN. The |
| default value for category is "LC_MESSAGES". If you supply a value |
| for category, it must be a string equal to one of the known locale |
| categories. You must also supply a text domain. Use TEXTDOMAIN if |
| you want to use the current domain. |
'---------------------'-------------------------------------------------'
=============== GNU AWK's Command Line Argument Summary ===============
.-------------------------.---------------------------------------------.
| | |
| Argument | Description |
| | |
'-------------------------+---------------------------------------------'
| -F fs | Use fs for the input field separator |
| --field-sepearator fs | (the value of the FS predefined variable). |
'-------------------------+---------------------------------------------'
| -v var=val | Assign the value val to the variable var, |
| --assign var=val | before execution of the program begins. |
| | Such variable values are available to the |
| | BEGIN block of an AWK program. |
'-------------------------+---------------------------------------------'
| -f program-file | Read the AWK program source from the file |
| --file program-file | program-file, instead of from the first |
| | command line argument. Multiple -f |
| | (or --file) options may be used. |
'-------------------------+---------------------------------------------'
| -mf NNN | Set various memory limits to the value NNN. |
| -mr NNN | The f flag sets the maximum number of |
| | fields, and the r flag sets the maximum |
| | record size. (Ignored by gawk, since gawk |
| | has no pre-defined limits) |
'-------------------------+---------------------------------------------'
| -W compat | Run in compatibility mode. In compatibility |
| -W traditional | mode, gawk behaves identically to UNIX awk; |
| --compat--traditional | none of the GNU-specific extensions are |
| | recognized. |
'-------------------------+---------------------------------------------'
| -W copyleft | Print the short version of the GNU copyright|
| -W copyright | information message on the standard output |
| --copyleft | and exit successfully. |
| --copyright | |
'-------------------------+---------------------------------------------'
| -W dump-variables[=file]| Print a sorted list of global variables, |
| --dump-variables[=file] | their types and final values to file. If no |
| | file is provided, gawk uses a file named |
| | awkvars.out in the current directory. |
'-------------------------+---------------------------------------------'
| -W help | Print a relatively short summary of the |
| -W usage | available options on the standard output. |
| --help | |
| --usage | |
'-------------------------+---------------------------------------------'
|-W lint[=value] | Provide warnings about constructs that |
|--lint[=value] | are dubious or non-portable to other AWK |
| | impl?s. With argument fatal, lint warnings |
| | become fatal errors. With an optional |
| | argument of invalid, only warnings about |
| | things that are actually invalid are |
| | issued. (This is not fully implemented yet.)|
'-------------------------+---------------------------------------------'
| -W lint-old--lint-old | Provide warnings about constructs that are |
| | not portable to the original version of |
| | Unix awk. |
'-------------------------+---------------------------------------------'
| -W gen-po--gen-po | Scan and parse the AWK program, and |
| | generate a GNU .po format file on standard |
| | output with entries for all localizable |
| | strings in the program. The program itself |
| | is not executed. |
'-------------------------+---------------------------------------------'
| -W non-decimal-data | Recognize octal and hexadecimal values in |
| --non-decimal-data | input data. |
'-------------------------+---------------------------------------------'
| -W posix--posix | This turns on compatibility mode, with the |
| | following additional restrictions: |
| | o \x escape sequences are not recognized. |
| | o Only space and tab act as field |
| | separators when FS is set to a single |
| | space, new-line does not. |
| | o You cannot continue lines after ? and :. |
| | o The synonym func for the keyword function|
| | is not recognized. |
| | o The operators ** and **= cannot be used |
| | in place of ^ and ^=.? The fflush() |
| | function is not available. |
'-------------------------+---------------------------------------------'
| -W profile[=prof_file] | Send profiling data to prof_file. |
| --profile[=prof_file] | The default is awkprof.out. When run with |
| | gawk, the profile is just a "pretty |
| | printed" version of the program. When run |
| | with pgawk, the profile contains execution |
| | counts of each statement in the program |
| | in the left margin and function call counts |
| | for each user-defined function. |
'-------------------------+---------------------------------------------'
| -W re-interval | Enable the use of interval expressions in |
| --re-interval | regular expression matching. Interval |
| | expressions were not traditionally |
| | available in the AWK language. |
'-------------------------+---------------------------------------------'
| -W source program-text | Use program-text as AWK program source |
| --source program-text | code. This option allows the easy |
| | intermixing of library functions (used via |
| | the -f and --file options) with source code |
| | entered on the command line. |
'-------------------------+---------------------------------------------'
| -W version | Print version information for this |
| --version | particular copy of gawk on the standard |
| | output. |
'-------------------------+---------------------------------------------'
| -- | Signal the end of options. This is useful |
| | to allow further arguments to the AWK |
| | program itself to start with a "-". This |
| | is mainly for consistency with the argument |
| | parsing convention used by most other POSIX |
| | programs. |
'-------------------------'---------------------------------------------'
=======================================================================
Useful One-Line Scripts for Perl Dec 03 2013 | version 1.10
-------------------------------- ----------- ------------
Compiled by Peteris Krumins (peter@catonmat.net, @pkrumins on Twitter)
http://www.catonmat.net -- good coders code, great reuse
Latest version of this file is always at:
http://www.catonmat.net/download/perl1line.txt
This file is also available in other languages:
Chinese: https://github.com/vinian/perl1line.txt
Please email me peter@catonmat.net if you wish to translate it.
Perl One-Liners on Github:
https://github.com/pkrumins/perl1line.txt
You can send me pull requests over GitHub! I accept bug fixes,
new one-liners, translations and everything else related.
I have also written "Perl One-Liners Explained" ebook that's based on
this file. It explains all the one-liners here. Get it at:
http://www.catonmat.net/blog/perl-book/
No Starch Press has published "Perl One-Liners" as a real book too:
http://nostarch.com/perloneliners
These one-liners work both on UNIX systems and Windows. Most likely your
UNIX system already has Perl. For Windows get the Strawberry Perl at:
http://www.strawberryperl.com/
Table of contents:
1. File Spacing
2. Line Numbering
3. Calculations
4. String Creation and Array Creation
5. Text Conversion and Substitution
6. Selective Printing and Deleting of Certain Lines
7. Handy Regular Expressions
8. Perl tricks
FILE SPACING
------------
# Double space a file
perl -pe '$\="\n"'
perl -pe 'BEGIN { $\="\n" }'
perl -pe '$_ .= "\n"'
perl -pe 's/$/\n/'
perl -nE 'say'
# Double space a file, except the blank lines
perl -pe '$_ .= "\n" unless /^$/'
perl -pe '$_ .= "\n" if /\S/'
# Triple space a file
perl -pe '$\="\n\n"'
perl -pe '$_.="\n\n"'
# N-space a file
perl -pe '$_.="\n"x7'
# Add a blank line before every line
perl -pe 's//\n/'
# Remove all blank lines
perl -ne 'print unless /^$/'
perl -lne 'print if length'
perl -ne 'print if /\S/'
# Remove all consecutive blank lines, leaving just one
perl -00 -pe ''
perl -00pe0
# Compress/expand all blank lines into N consecutive ones
perl -00 -pe '$_.="\n"x4'
# Fold a file so that every set of 10 lines becomes one tab-separated line
perl -lpe '$\ = $. % 10 ? "\t" : "\n"'
LINE NUMBERING
--------------
# Number all lines in a file
perl -pe '$_ = "$. $_"'
# Number only non-empty lines in a file
perl -pe '$_ = ++$a." $_" if /./'
# Number and print only non-empty lines in a file (drop empty lines)
perl -ne 'print ++$a." $_" if /./'
# Number all lines but print line numbers only non-empty lines
perl -pe '$_ = "$. $_" if /./'
# Number only lines that match a pattern, print others unmodified
perl -pe '$_ = ++$a." $_" if /regex/'
# Number and print only lines that match a pattern
perl -ne 'print ++$a." $_" if /regex/'
# Number all lines, but print line numbers only for lines that match a pattern
perl -pe '$_ = "$. $_" if /regex/'
# Number all lines in a file using a custom format (emulate cat -n)
perl -ne 'printf "%-5d %s", $., $_'
# Print the total number of lines in a file (emulate wc -l)
perl -lne 'END { print $. }'
perl -le 'print $n=()=<>'
perl -le 'print scalar(()=<>)'
perl -le 'print scalar(@foo=<>)'
perl -ne '}{print $.'
perl -nE '}{say $.'
# Print the number of non-empty lines in a file
perl -le 'print scalar(grep{/./}<>)'
perl -le 'print ~~grep{/./}<>'
perl -le 'print~~grep/./,<>'
perl -E 'say~~grep/./,<>'
# Print the number of empty lines in a file
perl -lne '$a++ if /^$/; END {print $a+0}'
perl -le 'print scalar(grep{/^$/}<>)'
perl -le 'print ~~grep{/^$/}<>'
perl -E 'say~~grep{/^$/}<>'
# Print the number of lines in a file that match a pattern (emulate grep -c)
perl -lne '$a++ if /regex/; END {print $a+0}'
perl -nE '$a++ if /regex/; END {say $a+0}'
CALCULATIONS
------------
# Check if a number is a prime
perl -lne '(1x$_) !~ /^1?$|^(11+?)\1+$/ && print "$_ is prime"'
# Print the sum of all the fields on a line
perl -MList::Util=sum -alne 'print sum @F'
# Print the sum of all the fields on all lines
perl -MList::Util=sum -alne 'push @S,@F; END { print sum @S }'
perl -MList::Util=sum -alne '$s += sum @F; END { print $s }'
# Shuffle all fields on a line
perl -MList::Util=shuffle -alne 'print "@{[shuffle @F]}"'
perl -MList::Util=shuffle -alne 'print join " ", shuffle @F'
# Find the minimum element on a line
perl -MList::Util=min -alne 'print min @F'
# Find the minimum element over all the lines
perl -MList::Util=min -alne '@M = (@M, @F); END { print min @M }'
perl -MList::Util=min -alne '$min = min @F; $rmin = $min unless defined $rmin && $min > $rmin; END { print $rmin }'
# Find the maximum element on a line
perl -MList::Util=max -alne 'print max @F'
# Find the maximum element over all the lines
perl -MList::Util=max -alne '@M = (@M, @F); END { print max @M }'
# Replace each field with its absolute value
perl -alne 'print "@{[map { abs } @F]}"'
# Find the total number of fields (words) on each line
perl -alne 'print scalar @F'
# Print the total number of fields (words) on each line followed by the line
perl -alne 'print scalar @F, " $_"'
# Find the total number of fields (words) on all lines
perl -alne '$t += @F; END { print $t}'
# Print the total number of fields that match a pattern
perl -alne 'map { /regex/ && $t++ } @F; END { print $t }'
perl -alne '$t += /regex/ for @F; END { print $t }'
perl -alne '$t += grep /regex/, @F; END { print $t }'
# Print the total number of lines that match a pattern
perl -lne '/regex/ && $t++; END { print $t }'
# Print the number PI to n decimal places
perl -Mbignum=bpi -le 'print bpi(n)'
# Print the number PI to 39 decimal places
perl -Mbignum=PI -le 'print PI'
# Print the number E to n decimal places
perl -Mbignum=bexp -le 'print bexp(1,n+1)'
# Print the number E to 39 decimal places
perl -Mbignum=e -le 'print e'
# Print UNIX time (seconds since Jan 1, 1970, 00:00:00 UTC)
perl -le 'print time'
# Print GMT (Greenwich Mean Time) and local computer time
perl -le 'print scalar gmtime'
perl -le 'print scalar localtime'
# Print local computer time in H:M:S format
perl -le 'print join ":", (localtime)[2,1,0]'
# Print yesterday's date
perl -MPOSIX -le '@now = localtime; $now[3] -= 1; print scalar localtime mktime @now'
# Print date 14 months, 9 days and 7 seconds ago
perl -MPOSIX -le '@now = localtime; $now[0] -= 7; $now[4] -= 14; $now[7] -= 9; print scalar localtime mktime @now'
# Prepend timestamps to stdout (GMT, localtime)
tail -f logfile | perl -ne 'print scalar gmtime," ",$_'
tail -f logfile | perl -ne 'print scalar localtime," ",$_'
# Calculate factorial of 5
perl -MMath::BigInt -le 'print Math::BigInt->new(5)->bfac()'
perl -le '$f = 1; $f *= $_ for 1..5; print $f'
# Calculate greatest common divisor (GCM)
perl -MMath::BigInt=bgcd -le 'print bgcd(@list_of_numbers)'
# Calculate GCM of numbers 20 and 35 using Euclid's algorithm
perl -le '$n = 20; $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $m'
# Calculate least common multiple (LCM) of numbers 35, 20 and 8
perl -MMath::BigInt=blcm -le 'print blcm(35,20,8)'
# Calculate LCM of 20 and 35 using Euclid's formula: n*m/gcd(n,m)
perl -le '$a = $n = 20; $b = $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $a*$b/$m'
# Generate 10 random numbers between 5 and 15 (excluding 15)
perl -le '$n=10; $min=5; $max=15; $, = " "; print map { int(rand($max-$min))+$min } 1..$n'
# Find and print all permutations of a list
perl -MAlgorithm::Permute -le '$l = [1,2,3,4,5]; $p = Algorithm::Permute->new($l); print @r while @r = $p->next'
# Generate the power set
perl -MList::PowerSet=powerset -le '@l = (1,2,3,4,5); for (@{powerset(@l)}) { print "@$_" }'
# Convert an IP address to unsigned integer
perl -le '$i=3; $u += ($_<<8*$i--) for "127.0.0.1" =~ /(\d+)/g; print $u'
perl -le '$ip="127.0.0.1"; $ip =~ s/(\d+)\.?/sprintf("%02x", $1)/ge; print hex($ip)'
perl -le 'print unpack("N", 127.0.0.1)'
perl -MSocket -le 'print unpack("N", inet_aton("127.0.0.1"))'
# Convert an unsigned integer to an IP address
perl -MSocket -le 'print inet_ntoa(pack("N", 2130706433))'
perl -le '$ip = 2130706433; print join ".", map { (($ip>>8*($_))&0xFF) } reverse 0..3'
perl -le '$ip = 2130706433; $, = "."; print map { (($ip>>8*($_))&0xFF) } reverse 0..3'
STRING CREATION AND ARRAY CREATION
----------------------------------
# Generate and print the alphabet
perl -le 'print a..z'
perl -le 'print ("a".."z")'
perl -le '$, = ","; print ("a".."z")'
perl -le 'print join ",", ("a".."z")'
# Generate and print all the strings from "a" to "zz"
perl -le 'print ("a".."zz")'
perl -le 'print "aa".."zz"'
# Create a hex lookup table
@hex = (0..9, "a".."f")
# Convert a decimal number to hex using @hex lookup table
perl -le '$num = 255; @hex = (0..9, "a".."f"); while ($num) { $s = $hex[($num%16)&15].$s; $num = int $num/16 } print $s'
perl -le '$hex = sprintf("%x", 255); print $hex'
perl -le '$num = "ff"; print hex $num'
# Generate a random 8 character password
perl -le 'print map { ("a".."z")[rand 26] } 1..8'
perl -le 'print map { ("a".."z", 0..9)[rand 36] } 1..8'
# Create a string of specific length
perl -le 'print "a"x50'
# Create a repeated list of elements
perl -le '@list = (1,2)x20; print "@list"'
# Create an array from a string
@months = split ' ', "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
@months = qw/Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec/
# Create a string from an array
@stuff = ("hello", 0..9, "world"); $string = join '-', @stuff
# Find the numeric values for characters in the string
perl -le 'print join ", ", map { ord } split //, "hello world"'
# Convert a list of numeric ASCII values into a string
perl -le '@ascii = (99, 111, 100, 105, 110, 103); print pack("C*", @ascii)'
perl -le '@ascii = (99, 111, 100, 105, 110, 103); print map { chr } @ascii'
# Generate an array with odd numbers from 1 to 100
perl -le '@odd = grep {$_ % 2 == 1} 1..100; print "@odd"'
perl -le '@odd = grep { $_ & 1 } 1..100; print "@odd"'
# Generate an array with even numbers from 1 to 100
perl -le '@even = grep {$_ % 2 == 0} 1..100; print "@even"'
# Find the length of the string
perl -le 'print length "one-liners are great"'
# Find the number of elements in an array
perl -le '@array = ("a".."z"); print scalar @array'
perl -le '@array = ("a".."z"); print $#array + 1'
TEXT CONVERSION AND SUBSTITUTION
--------------------------------
# ROT13 a string
'y/A-Za-z/N-ZA-Mn-za-m/'
# ROT 13 a file
perl -lpe 'y/A-Za-z/N-ZA-Mn-za-m/' file
# Base64 encode a string
perl -MMIME::Base64 -e 'print encode_base64("string")'
perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' file
# Base64 decode a string
perl -MMIME::Base64 -le 'print decode_base64("base64string")'
perl -MMIME::Base64 -ne 'print decode_base64($_)' file
# URL-escape a string
perl -MURI::Escape -le 'print uri_escape($string)'
# URL-unescape a string
perl -MURI::Escape -le 'print uri_unescape($string)'
# HTML-encode a string
perl -MHTML::Entities -le 'print encode_entities($string)'
# HTML-decode a string
perl -MHTML::Entities -le 'print decode_entities($string)'
# Convert all text to uppercase
perl -nle 'print uc'
perl -ple '$_=uc'
perl -nle 'print "\U$_"'
# Convert all text to lowercase
perl -nle 'print lc'
perl -ple '$_=lc'
perl -nle 'print "\L$_"'
# Uppercase only the first word of each line
perl -nle 'print ucfirst lc'
perl -nle 'print "\u\L$_"'
# Invert the letter case
perl -ple 'y/A-Za-z/a-zA-Z/'
# Camel case each line
perl -ple 's/(\w+)/\u$1/g'
perl -ple 's/(?<!['])(\w+)/\u\1/g'
# Strip leading whitespace (spaces, tabs) from the beginning of each line
perl -ple 's/^[ \t]+//'
perl -ple 's/^\s+//'
# Strip trailing whitespace (space, tabs) from the end of each line
perl -ple 's/[ \t]+$//'
# Strip whitespace from the beginning and end of each line
perl -ple 's/^[ \t]+|[ \t]+$//g'
# Convert UNIX newlines to DOS/Windows newlines
perl -pe 's|\n|\r\n|'
# Convert DOS/Windows newlines to UNIX newlines
perl -pe 's|\r\n|\n|'
# Convert UNIX newlines to Mac newlines
perl -pe 's|\n|\r|'
# Substitute (find and replace) "foo" with "bar" on each line
perl -pe 's/foo/bar/'
# Substitute (find and replace) all "foo"s with "bar" on each line
perl -pe 's/foo/bar/g'
# Substitute (find and replace) "foo" with "bar" on lines that match "baz"
perl -pe '/baz/ && s/foo/bar/'
# Binary patch a file (find and replace a given array of bytes as hex numbers)
perl -pi -e 's/\x89\xD8\x48\x8B/\x90\x90\x48\x8B/g' file
SELECTIVE PRINTING AND DELETING OF CERTAIN LINES
------------------------------------------------
# Print the first line of a file (emulate head -1)
perl -ne 'print; exit'
# Print the first 10 lines of a file (emulate head -10)
perl -ne 'print if $. <= 10'
perl -ne '$. <= 10 && print'
perl -ne 'print if 1..10'
# Print the last line of a file (emulate tail -1)
perl -ne '$last = $_; END { print $last }'
perl -ne 'print if eof'
# Print the last 10 lines of a file (emulate tail -10)
perl -ne 'push @a, $_; @a = @a[@a-10..$#a]; END { print @a }'
# Print only lines that match a regular expression
perl -ne '/regex/ && print'
# Print only lines that do not match a regular expression
perl -ne '!/regex/ && print'
# Print the line before a line that matches a regular expression
perl -ne '/regex/ && $last && print $last; $last = $_'
# Print the line after a line that matches a regular expression
perl -ne 'if ($p) { print; $p = 0 } $p++ if /regex/'
# Print lines that match regex AAA and regex BBB in any order
perl -ne '/AAA/ && /BBB/ && print'
# Print lines that don't match match regexes AAA and BBB
perl -ne '!/AAA/ && !/BBB/ && print'
# Print lines that match regex AAA followed by regex BBB followed by CCC
perl -ne '/AAA.*BBB.*CCC/ && print'
# Print lines that are 80 chars or longer
perl -ne 'print if length >= 80'
# Print lines that are less than 80 chars in length
perl -ne 'print if length < 80'
# Print only line 13
perl -ne '$. == 13 && print && exit'
# Print all lines except line 27
perl -ne '$. != 27 && print'
perl -ne 'print if $. != 27'
# Print only lines 13, 19 and 67
perl -ne 'print if $. == 13 || $. == 19 || $. == 67'
perl -ne 'print if int($.) ~~ (13, 19, 67)'
# Print all lines between two regexes (including lines that match regex)
perl -ne 'print if /regex1/../regex2/'
# Print all lines from line 17 to line 30
perl -ne 'print if $. >= 17 && $. <= 30'
perl -ne 'print if int($.) ~~ (17..30)'
perl -ne 'print if grep { $_ == $. } 17..30'
# Print the longest line
perl -ne '$l = $_ if length($_) > length($l); END { print $l }'
# Print the shortest line
perl -ne '$s = $_ if $. == 1; $s = $_ if length($_) < length($s); END { print $s }'
# Print all lines that contain a number
perl -ne 'print if /\d/'
# Find all lines that contain only a number
perl -ne 'print if /^\d+$/'
# Print all lines that contain only characters
perl -ne 'print if /^[[:alpha:]]+$/
# Print every second line
perl -ne 'print if $. % 2'
# Print every second line, starting the second line
perl -ne 'print if $. % 2 == 0'
# Print all lines that repeat
perl -ne 'print if ++$a{$_} == 2'
# Print all unique lines
perl -ne 'print unless $a{$_}++'
# Print the first field (word) of every line (emulate cut -f 1 -d ' ')
perl -alne 'print $F[0]'
HANDY REGULAR EXPRESSIONS
-------------------------
# Match something that looks like an IP address
/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/
/^(\d{1,3}\.){3}\d{1,3}$/
# Test if a number is in range 0-255
/^([0-9]|[0-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])$/
# Match an IP address
my $ip_part = qr|([0-9]|[0-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|;
if ($ip =~ /^($ip_part\.){3}$ip_part$/) {
say "valid ip";
}
# Check if the string looks like an email address
/\S+@\S+\.\S+/
# Check if the string is a decimal number
/^\d+$/
/^[+-]?\d+$/
/^[+-]?\d+\.?\d*$/
# Check if the string is a hexadecimal number
/^0x[0-9a-f]+$/i
# Check if the string is an octal number
/^0[0-7]+$/
# Check if the string is binary
/^[01]+$/
# Check if a word appears twice in the string
/(word).*\1/
# Increase all numbers by one in the string
$str =~ s/(\d+)/$1+1/ge
# Extract HTTP User-Agent string from the HTTP headers
/^User-Agent: (.+)$/
# Match printable ASCII characters
/[ -~]/
# Match unprintable ASCII characters
/[^ -~]/
# Match text between two HTML tags
m|<strong>([^<]*)</strong>|
m|<strong>(.*?)</strong>|
# Replace all <b> tags with <strong>
$html =~ s|<(/)?b>|<$1strong>|g
# Extract all matches from a regular expression
my @matches = $text =~ /regex/g;
PERL TRICKS
-----------
# Print the version of a Perl module
perl -MModule -le 'print $Module::VERSION'
perl -MLWP::UserAgent -le 'print $LWP::UserAgent::VERSION'
PERL ONE-LINERS EXPLAINED E-BOOK
--------------------------------
I have written an ebook based on the one-liners in this file. If you wish to
support my work and learn more about these one-liners, you can get a copy
of my ebook at:
http://www.catonmat.net/blog/perl-book/
The ebook is based on the 7-part article series that I wrote on my blog.
In the ebook I reviewed all the one-liners, improved explanations, added
new ones, and added two new chapters - introduction to Perl one-liners
and summary of commonly used special variables.
You can read the original article series here:
http://www.catonmat.net/blog/perl-one-liners-explained-part-one/
http://www.catonmat.net/blog/perl-one-liners-explained-part-two/
http://www.catonmat.net/blog/perl-one-liners-explained-part-three/
http://www.catonmat.net/blog/perl-one-liners-explained-part-four/
http://www.catonmat.net/blog/perl-one-liners-explained-part-five/
http://www.catonmat.net/blog/perl-one-liners-explained-part-six/
http://www.catonmat.net/blog/perl-one-liners-explained-part-seven/
CREDITS
-------
Andy Lester http://www.petdance.com
Shlomi Fish http://www.shlomifish.org
Madars Virza http://www.madars.org
caffecaldo https://github.com/caffecaldo
Kirk Kimmel https://github.com/kimmel
avar https://github.com/avar
rent0n
FOUND A BUG? HAVE ANOTHER ONE-LINER?
------------------------------------
Email bugs and new one-liners to me at peter@catonmat.net!
HAVE FUN
--------
I hope you found these one-liners useful. Have fun!
#---end of file---
Hi,
I am interested in catalina.out log file rotation, I have an application where logging to catalina.out is very huge, say 0.5 MB / sec.
So I have written one script to handle this which is shown below.
# crontab -l | grep catalina
0,30 * * * * bash /catalina_log_handler.sh >/dev/null 2>&1
# more /catalina_log_handler.sh
#!/bin/sh
CATLOC=/bin/cat
SEDLOC=/usr/bin/sed
$CATLOC /catalina.out >> /catalina_backup.out à take a backup and then handle this file rotation using logadm
$SEDLOC ” catalina.out > catalina.out à Empty this file
There are two cases here
1.) The below is normal behavior where not much logging
# ls -ltrh catalina*
-rw-r—– 1 ems ems 15K Jan 31 12:07 catalina.2012-01-31.log
-rw-r—– 1 ems ems 3.9K Feb 1 06:26 catalina.2012-02-01.log
-rw-r—– 1 ems ems 345K Feb 1 06:45 catalina.out
-rw-r–r– 1 root root 14M Feb 1 07:00 catalina_backup.out
2.) The below is heavy logging behavior
# ls -ltrh catalina*
-rw-r—– 1 ems ems 4.2K Jan 28 22:14 catalina.2012-01-28.log à why I am seeing catalina file with date, have the same content as catalina.out
-rw-r—– 1 ems ems 362 Jan 30 06:30 catalina.2012-01-30.log
-rw-r—– 1 ems ems 10M Jan 30 14:43 catalina.out.2
-rw-r–r– 1 root root 1.7M Jan 30 17:00 catalina_backup.out.9.gz
-rw-r–r– 1 root root 1.6M Jan 30 20:00 catalina_backup.out.8.gz
-rw-r–r– 1 root root 1.6M Jan 30 23:00 catalina_backup.out.7.gz
-rw-r–r– 1 root root 1.6M Jan 31 02:00 catalina_backup.out.6.gz
-rw-r—– 1 ems ems 113M Jan 31 03:30 catalina.out.3
-rw-r–r– 1 root root 1.8M Jan 31 06:00 catalina_backup.out.5.gz
-rw-r—– 1 ems ems 6.2M Jan 31 10:04 catalina.out
-rw-r—– 1 ems ems 4.5K Jan 31 10:04 catalina.2012-01-31.log
-rw-r–r– 1 root root 1.8M Jan 31 10:30 catalina_backup.out.4.gz
-rw-r–r– 1 root root 1.8M Jan 31 15:00 catalina_backup.out.3.gz
-rw-r–r– 1 root root 1.8M Jan 31 19:30 catalina_backup.out.2.gz
-rw-r–r– 1 root root 1.8M Feb 1 00:00 catalina_backup.out.1.gz
-rw-r–r– 1 root root 12M Feb 1 01:00 catalina_backup.out
-rw-r—– 1 ems ems 12M Feb 1 01:12 catalina.out.1 à why this is latest file ? Is this because of huge logging, Can I disable default catalina.out rotation so that my script will take care of moving the content from catalina.out to catalina_backup.out…
Using CMD to kick user not properly logoff in remote session.
This command to list out the user. Have 2 command :
1 – quser /SERVER : [server name]
2 – qwinsta /SERVER : [server name]
This command to kick the user :
1 – logoff /SERVER : [server name] [session id]
Example :
C:\x>quser /SERVER:xxxx-xxxSxAPP
USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
root console 0 Active . 5/21/2011 5:09 AM
anuar 1 Disc none 6/7/2011 9:45 AM
izuan rdp-tcp#42 2 Active 59 6/9/2011 10:47AM
C:\x>qwinsta /SERVER:xxxx-xxxSxAPP
USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
root console 0 Active . 5/21/2011 5:09 AM
anxxxxr 1 Disc none 6/7/2011 9:45 AM
izxxx rdp-tcp#42 2 Active 59 6/9/2011 10:47AM
C:\Users\azam>logoff /SERVER:xxxx-xxxSxAPP 1
Zimbra Creating self-signed certifiate
UCSC Zimbra
===========
Backup Existing Commertical Certs
———————————
# cd /opt/zimbra/ssl/zimbra/commercial/
# mkdir -p DigiCert_old
# cp commercial* ./DigiCert_old
# cd /opt/zimbra/ssl/zimbra/server
# mkdir -p Server_old
# cp server.* Server_old
# cd /opt/zimbra/bin
Creating a CA
————-
# ./zmcertmgr createca -new
** Creating /opt/zimbra/ssl/zimbra/ca/zmssl.cnf…done
** Creating CA private key /opt/zimbra/ssl/zimbra/ca/ca.key…done.
** Creating CA cert /opt/zimbra/ssl/zimbra/ca/ca.pem…done.
Creating Certs
————–
# ./zmcertmgr createcrt -new -days 365
Validation days: 365
** Creating /opt/zimbra/conf/zmssl.cnf…done
** Backup /opt/zimbra/ssl/zimbra to /opt/zimbra/ssl/zimbra.20110423123012
** Generating a server csr for download self -new -keysize 1024
** Creating /opt/zimbra/conf/zmssl.cnf…done
** Backup /opt/zimbra/ssl/zimbra to /opt/zimbra/ssl/zimbra.20110423123012
** Creating server cert request /opt/zimbra/ssl/zimbra/server/server.csr…done.
** Saving server config key zimbraSSLPrivateKey…failed.
** Signing cert request /opt/zimbra/ssl/zimbra/server/server.csr…done.
Deploy the certificate
———————-
#./zmcertmgr deploycrt self
** Saving server config key zimbraSSLCertificate…failed.
** Saving server config key zimbraSSLPrivateKey…failed.
** Installing mta certificate and key…done.
** Installing slapd certificate and key…done.
** Installing proxy certificate and key…done.
** Creating pkcs12 file /opt/zimbra/ssl/zimbra/jetty.pkcs12…done.
** Creating keystore file /opt/zimbra/mailboxd/etc/keystore…done.
** Installing CA to /opt/zimbra/conf/ca…done.
Deploy the CA
————-
# ./zmcertmgr deployca
** Importing CA /opt/zimbra/ssl/zimbra/ca/ca.pem into CACERTS…done.
** Saving global config key zimbraCertAuthorityCertSelfSigned…failed.
** Saving global config key zimbraCertAuthorityKeySelfSigned…failed.
** Copying CA to /opt/zimbra/conf/ca…done.
Verify the certificate was deployed to all the services
——————————————————-
# ./zmcertmgr deployca
** Importing CA /opt/zimbra/ssl/zimbra/ca/ca.pem into CACERTS…done.
** Saving global config key zimbraCertAuthorityCertSelfSigned…failed.
** Saving global config key zimbraCertAuthorityKeySelfSigned…failed.
** Copying CA to /opt/zimbra/conf/ca…done.
View the Certificate
——————–
debian-zimbra:/opt/zimbra/bin# ./zmcertmgr viewdeployedcrt
::service mta::
notBefore=Apr 23 07:00:14 2011 GMT
notAfter=Apr 22 07:00:14 2012 GMT
subject= /C=US/ST=N/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk
issuer= /C=US/ST=N/A/L=N/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk
SubjectAltName=
::service proxy::
notBefore=Apr 23 07:00:14 2011 GMT
notAfter=Apr 22 07:00:14 2012 GMT
subject= /C=US/ST=N/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk
issuer= /C=US/ST=N/A/L=N/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk
SubjectAltName=
::service mailboxd::
notBefore=Apr 23 07:00:14 2011 GMT
notAfter=Apr 22 07:00:14 2012 GMT
subject= /C=US/ST=N/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk
issuer= /C=US/ST=N/A/L=N/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk
SubjectAltName=
::service ldap::
notBefore=Apr 23 07:00:14 2011 GMT
notAfter=Apr 22 07:00:14 2012 GMT
subject= /C=US/ST=N/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk
issuer= /C=US/ST=N/A/L=N/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk
SubjectAltName=
==============================================================================================================
== – ZMCERTMGR Help – ==
==============================================================================================================
./zmcertmgr -help
./zmcertmgr createca [-new] [-keysize 1024] [-subject subject]
./zmcertmgr deployca
./zmcertmgr createcsr <self|comm> [-new] [-keysize 1024] [-subject subject] [-subjectAltNames “host1,host2”]
./zmcertmgr createcrt [-new] [-days validation days] [-keysize 1024] [-subject subject] [-subjectAltNames “host1,host2”]
./zmcertmgr deploycrt <self>
./zmcertmgr deploycrt <comm> [certfile] [ca_chain_file]
./zmcertmgr savecrt
./zmcertmgr viewcsr <self|comm> [csr_file]
./zmcertmgr viewdeployedcrt [all|ldap|mta|proxy|mailboxd]
./zmcertmgr viewstagedcrt <self|comm> [certfile]
./zmcertmgr verifycrt <self|comm> [priv_key] [certfile]
./zmcertmgr verifycrtchain <ca_file> <certfile>
./zmcertmgr checkcrtexpiration [-days 30] [service]
./zmcertmgr addcacert <certfile>
./zmcertmgr migrate
Comments:
– Default <certfile>
self-signed /opt/zimbra/ssl/zimbra/server/server.crt
commerical /opt/zimbra/ssl/zimbra/commercial/commercial.crt
– Default <priv_key>
self-signed /opt/zimbra/ssl/zimbra/server/server.key
commercial /opt/zimbra/ssl/zimbra/commercial/commercial.key
– Default <subject>
“/C=US/ST=N\/A/L=N\/A/O=Zimbra Collaboration Suite/OU=Zimbra Collaboration Suite/CN=mail.ucsc.cmb.ac.lk”
– Default RSA <keysize> is 1024.
– Default <validation_days> is 365.
– Default <csr_file> is
– deploycrt self installs the certificates using self signed csr in /opt/zimbra/ssl/zimbra/server
– deploycrt comm installs the certificates using commercially signed certificate in /opt/zimbra/ssl/zimbra/commercial
– verifycrt <self|comm> compares openssl md5 [priv_key] and [certfile].
– migrate moves certs/keys from ZCS installs prior to version 5.0.x
– addcacert appends an otherwise untrusted ssl certificate to the cacerts file.
This is primarily for allowance of untrusted ssl certificates in external data sources.
awk examples
cat 03092608.LOG|grep XYZ|awk -F”.” ‘{print $2}’ |sort|awk ‘{tot += 1 } $1 != prev {print prev,tot;tot=0;prev=$1}END{print prev,tot;tot=0;prev=$1}’
adhocs
1. find_processes_using_ipcs.ksh
ipcs -bop|grep `whoami`|awk ‘{print $10}’|egrep -v “^0|^$”|sort -u|awk ‘{print “ps -ef|grep “, $1}’|ksh
2. iostat
iostat -xn 4 30
3. jmap
jmap -heap:format=b <pid>
4. monitorMemLeaking
if [ $# -lt 1 ]
then
echo “Usage: $0 [pid]”
exit
fi
PID=$1
while true
do
vsz1=`ps -eo vsz,rss,pid|grep “$PID$” |awk ‘{print $2}’`
rss1=`ps -eo vsz,rss,pid|grep “$PID$” |awk ‘{print $3}’`
echo “vsz1=$vsz1, rss1=$rss1… wait for 5 mins\n”
sleep 300
vsz2=`ps -eo vsz,rss,pid|grep “$PID$” |awk ‘{print $2}’`
rss2=`ps -eo vsz,rss,pid|grep “$PID$” |awk ‘{print $3}’`
echo “vsz2=$vsz2, rss2=$rss2\n”
let vszLeakingRate=”(${vsz2} – ${vsz1}) / 5″
let rssLeakingRate=”(${rss2} – ${rss1}) / 5″
echo “PID=$PID, vszLeakingRate = $vszLeakingRate (KB)/ per min, rssLeakingRate=$rssLeakingRate (KB)/per min\n\n”
done
5. port_check.ksh
#!/usr/bin/ksh
#If lsof or nmap is available, we might be able to do this easily
echo “which port?>\c “
read port
#for pid in `ps -ef -o pid | tail +2`
#do
#foundport=`/usr/bin/pfiles $pid 2>&1 | grep “sockname:” | grep “port: $port$”`
#if [ “$foundport” != “” ]
#then
# echo “proc: $pid, $foundport”
#fi
#done
#
echo “Going to find out which process is using port: $port”
ps -ef|grep `whoami`|grep -v grep|awk ‘{print $2}’|while read i
do
# check=`pfiles $i 2>&-|grep “sockname:” | grep “port:” | grep $port$”`
check=`pfiles $i 2>&-| grep $port$”`
echo “$check.\c”
if [ ! -z “${check}” ]
then
print “\n”
print “The port number $port used by process ID=$i”
#break
fi
check=””
done
exit
6. Difference between SIZE and RSS of a process?
The basic difference is that SIZE represents the total address space size of a process, while RSS is the actual physical memory currently occupied by pages belonging to that process (roughly speaking). Regions of an address space mapped to something don’t necessarily occupy any RAM.
7. RSS_vs_real_memory.txt
Using the ps command it is possible to look at the resident set size (rss) parameter of each process… it is my understanding that the value of rss indicates how much real memory the process is using… if i sum up all of the rss values for each process, i sometimes find that the sum is greater than the available real memory installed in the system…
The non-modifiable parts (the compiled code & const data) of shared libraries & ELF binaries are only loaed into memory once and those are shared among all the processes. The modifiable parts (non-const data, etc.) are copied into each processes’s address space.
/usr/proc/bin/pmap -x will show you how much is shared and how much is not.
8. vmstat
The “free” column tells you how much real memory is *not* being used. That’s usually more important — if that gets low, then you need more memory. And if it’s always very high, you could be wasting memory. The “top” command also shows this information.
The problem with top and vmstat is that they do not account for the real memory that the OS keeps cached. Memory that the OS caches looks to these programs like active memory. In a sense it is active memory since the OS is hanging on to it waiting for new requests… but, it is deceptive since the memory is actually available for use…
Also, just because free memory may be reported as low on a system does not necessarily indicate that the machine needs more memory… other things such as heavy paging and the scanner running often are better indicators that there is not enough real memory… this is due to the OS caching mentioned above… the system may seem low on memory but the OS is holding onto blocks of memory that are actually available for use by processes
9. socket-status.txt
State Description
LISTEN accepting connections
ESTABLISHED connection up and passing data
SYN_SENT TCP; session has been requested by us; waiting for reply from remote endpoint
SYN_RECV TCP; session has been requested by a remote endpoint for a socket on which we were listening
LAST_ACK TCP; our socket is closed; remote endpoint has also shut down; we are waiting for a final acknowledgement
CLOSE_WAIT TCP; remote endpoint has shut down; the kernel is waiting for the application to close the socket
TIME_WAIT TCP; socket is waiting after closing for any packets left on the network
CLOSED socket is not being used (FIXME. What does mean?)
CLOSING TCP; our socket is shut down; remote endpoint is shut down; not all data has been sent
FIN_WAIT1 TCP; our socket has closed; we are in the process of tearing down the connection
FIN_WAIT2 TCP; the connection has been closed; our socket is waiting for the remote endpoint to shut down
10. ssh setup
ssh-keygen generate 2 files which go hand-in-hand. The private key file (id_rsa) should reside on my $HOME/.ssh, in order to encrypt things; the putlib key file(id_rsa.pub) should be appended in remote host at its $HOME/.ssh/authorized_keys file.
When I ssh to remote host, remote host’s daemon (sshd) will decrypt my logon using the public key from $HOME/.ssh/authorized_keys file. This will allow me
to logon without prompting for password.
This is the logic:
1)ssh remoteHost
2)ssh look at $HOME/.ssh/id_rsa and also $HOME/.ssh/known_hosts
create request and send to remoteHost
3)remoteHost will look at its $HOME/.ssh/authorized_keys and if it does see one entry
which is able to decipher the request, connect is OK
4)ssh on the originating host will put an entry to $HOME/.ssh/known_hosts if it’s not there
11. ucbps
#Long form of ps command
/usr/ucb/ps -auxww
12. who_did_what
#!/bin/ksh
#
#####
STRING1=$1
STRING2=$2
STDOUT=/tmp/stdout.out
echo “” > $STDOUT
cd $HOME
#for i in `ls $HOME/.history.[a-z]*`
for i in `ls .history.[a-z]*`
do
if (( `strings $i | grep -i $STRING1 | grep -i $STRING2 | wc -l` > “0” ))
then
echo ” ## `ls $i` ## ” # >> $STDOUT
strings $i | grep -i $STRING1 | grep -i $STRING2 # >> $STDOUT
fi
done
#!/usr/bin/perl
# Fast script to grab jstack / jinfo and jmap of processes running on server.
#
### Update paths here for java_base.
$java_base='/usr/java/bin';
$jstack='jstack';
$jinfo='jinfo';
$jmap='jmap';
$script=$0;
$runuser=`/usr/xpg4/bin/id -nu`;
chomp($runuser);
#$outdir='/tmp';
($min, $hour, $day, $month, $year) = (localtime)[1,2,3,4,5];
$year=$year+1900;
if ( $month < 10 ) { $month="0$month"; }
if ( $day < 10 ) { $day="0$day"; }
if ( $hour < 10 ) { $hour="0$hour"; }
if ( $min < 10 ) { $min="0$min"; }
$tarfile="$script-$year$month$day-$hour$min.tar";
sub options
{
use Getopt::Long;
if ($#ARGV lt 0)
{
&usage;
}
GetOptions ("debug|d" => \$debug,
"verbose|v" => \$verbose,
"quiet|q" => \$quiet,
"outdir=s" => \$outdir,
"o=s" => \$outdir,
"remove|r" => \$remove,
"help|h" => \$help);
&usage if $help;
}
# sub routine to print out the usage if the no command line arguments are passed.
sub usage
{
print "This script will generate jstack / jinfo and jmap of all java processes on the server that are currently running\n";
print "It will create a tar file of all output.\n";
print "Optionally, the script will remove the output directory\n";
print "\n";
print "-outdir or -o: Specify an output directory for the traces.\n";
print "-remove or -r: Remove the output directory after creating the tar file.\n";
print "-debug or -d: Debug Output.\n";
print "-verbose or -v: Verbose Output.\n";
print "-help or -h: Help message.\n";
print "\n";
print "Example Usage:\n";
print "\t$script -o foo -v\n";
exit 1;
}
&options;
if ( $runuser ne "root" )
{
print "You need to be root to use this tool\n";
exit 1;
}
if ( ! $outdir )
{
print "Output directory is required\n";
exit 2;
}
else
{
if ( -d $outdir )
{
print "Output directory $outdir already exists - exiting\n";
exit 3;
}
if ( ! -d $outdir )
{
system("mkdir $outdir");
}
if ( $verbose )
{
print "Created $outdir\n";
}
if ( $debug )
{
print "DEBUG: mkdir $outdir\n";
}
system("/usr/bin/logger -p local0.info \"$script : invoked by $runuser \"");
@jps=`$java_base/jps | grep -v Jps`;
foreach (@jps)
{
($pid,$name)=split(/ /, $_);
chomp ($name);
if ( $debug )
{
print "DEBUG: pid: $pid\n";
print "DEBUG: name: $name\n";
}
$is_64_bit=`$java_base/$jstack $pid 2>&1 | grep -c 'debuggee is 64 bit'`;
if ( $is_64_bit == 1 ) # we need to use 64 bit option here.
{
$proc_bits=64;
}
else
{
$proc_bits=32;
}
if ( $debug )
{
print "DEBUG: $java_base/$jstack -d$proc_bits $pid 2>&1 > $outdir/jstack.$name.wri\n";
}
$cmd=`$java_base/$jstack -d$proc_bits $pid 2>&1 > $outdir/jstack.$name.wri`;
if ( $verbose )
{
print "Creating stack trace for $name\n";
}
if ( $debug )
{
print "DEBUG: $java_base/$jinfo -d$proc_bits $pid 2>&1 > $outdir/jinfo.$name.wri\n";
}
$cmd=`$java_base/$jinfo -d$proc_bits $pid 2>&1 > $outdir/jinfo.$name.wri`;
if ( $verbose )
{
print "Creating info trace for $name\n";
}
if ( $debug )
{
print "DEBUG: $java_base/$jmap -d$proc_bits $pid 2>&1 > $outdir/jmap.$name.wri\n";
}
$cmd=`$java_base/$jmap -d$proc_bits $pid 2>&1 > $outdir/jmap.$name.wri`;
if ( $verbose )
{
print "Creating jmap trace for $name\n";
}
} # end foreach
$cmd=`tar cvf $tarfile $outdir`;
$cmd=`gzip $tarfile`;
if ( $remove )
{
$cmd=`rm -rf $outdir`;
if ( $verbose )
{
print "Removing directory $outdir\n";
}
if ( $debug )
{
print "DEBUG: rm -rf $outdir\n";
}
}
} # end else.
# put your JAVA_HOME here,
|
JAVA_HOME=/opt/javavm
|
| |
PID=$1
|
IFS=”
|
top_number=10
|
if[ $# -gt 1 ] ; then
|
top_number=$2
|
fi
|
| |
top_number=$((top_number+1))
|
| |
java_stack=`$JAVA_HOME/bin/jstack $PID`
|
| |
top=`top -s -b -H -p $PID -n 1 | grep -vE ‘^top|^Tasks|^Cpu|^Mem|^Swap|^$’ | awk ‘NR==1; NR > 1 {print $0 | “sort -nrk 9”}’ | head -$top_number`
|
echo $top
|
| |
echo $top |while read psline;do
|
if[`echo $psline|grep -c PID`–gt 0];
|
thencontinue
|
fi
|
lwp_id=`echo $psline | awk ‘{print $1}’`
|
nid=`printf ‘%x\n’ $lwp_id `
|
echo “========> Java LWP: $lwp_id – Native Thread ID=$nid”
|
echo $java_stack | sed –n “/$nid/,/^$/ p”;
|
| done |
|
|
Recent Comments