How to display the contents of a gzip\/gz file
\nBy Alvin Alexander. Last updated: Aug 6, 2011
\nProblem: You have a plain text file that has been compressed with the gzip command, and you’d like to display the file contents with the Unix\/Linux cat or more commands.<\/p>\n
Solution: Instead of using the cat or more commands, use their equivalents for working with gz files, the zcat and zmore commands.<\/p>\n
For instance, if you want to display the contents of an Apache log file (which is a plain text file) that has been compressed with gzip, just use the zcat command, like this:<\/p>\n
zcat access_log.gz
\nOf course almost any Apache log file will be large, and will scroll off the screen quickly, so you’ll probably want to use the gzip equivalent of the more command, zmore, like this:<\/p>\n
zmore access_log.gz<\/p>\n
find . -name “*.gz” | while read -r file; do zcat -f “$file” | head -n 1; done<\/p>\n
zcat `man -w manpage` | groff -mandoc -T html – > filename.html<\/p>\n
save manpage as html file<\/p>\n
zcat log.tar.gz | grep -a -i “string”<\/p>\n
grep compressed log files without extracting. Useful in system where log files are compressed for archival purposes<\/p>\n
zcat \/usr\/share\/man\/man1\/man.1.gz | nroff -man | less<\/p>\n
As odd as this may be, I know of servers where the man(1) command is not installed, and there is not enough room on \/ to install it. However, zcat(1), nroff(1) and less(1) are. This is a way to read those documents without the proper tool to do so, as sad as this may seem. \ud83d\ude42<\/p>\n
This command enables the user to append a search pattern on the command line when using less as the PAGER. This is especially convenient (as the example shows) in compressed files and when searching man pages (substituting the zcat command with man, however).<\/p>\n
zcat -f $(ls -tr access.log*) find \/var\/log\/apache2 -name ‘access.log*gz’ -exec zcat {} \\; -or -name ‘access.log*’ -exec cat {} \\; find \/var\/log\/apache2 -name ‘access.log*gz’ -exec zcat {} + -or -name ‘access.log*’ -exec cat {} + zcat access_log.*.gz | awk ‘{print $7}’ | sort | uniq -c | sort -n | tail -n 20 sudo zcat \/var\/log\/auth.log.*.gz | awk ‘\/Failed password\/&&!\/for invalid user\/{a[$9]++}\/Failed password for invalid user\/{a[“*” $11]++}END{for (i in a) printf “%6s\\t%s\\n”, a[i], i|”sort -n”}’ zcat a_big_file.gz | sed -ne “$(zcat a_big_file.gz | tr -d “[:print:]” | cat -n | grep -vP “^ *\\d+\\t$” | cut -f 1 | sed -e “s\/\\([0-9]\\+\\)\/\\1=;\\1p;\/” | xargs)” | tr -c “[:print:]\\n” “?”<\/p>\n Functions: sed tr zcat zcat \/usr\/share\/doc\/vim-common\/README.gz | vim -g +23 – zcat \/usr\/share\/man\/man1\/grep.1.gz | grep “color”<\/p>\n Search gzipped files #!\/bin\/sh How to display the contents of a gzip\/gz file By Alvin Alexander. Last updated: Aug 6, 2011 Problem: You have a plain text file that has been compressed with the gzip command, and you’d like to display the file contents with the Unix\/Linux cat or more commands.<\/p>\n Solution: Instead of using the cat or more […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[47],"tags":[],"_links":{"self":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3427"}],"collection":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3427"}],"version-history":[{"count":4,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3427\/revisions"}],"predecessor-version":[{"id":3431,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3427\/revisions\/3431"}],"wp:attachment":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3427"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3427"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3427"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n<\/strong>
\nconcatenate compressed and uncompressed logs
\nwith zcat force option it’s even simpler.<\/p>\n
\nfunctions: cat find zcat
\nconcatenate compressed and uncompressed logs
\nThis command allows you to stream your log files, including gziped files, into one stream which can be piped to awk or some other command for analysis.
\nNote: if your version of ‘find’ supports it, use:<\/p>\n
\nzcat database.sql.gz | mysql -uroot -p’passwd’ database
\nFunctions: zcat
\nRestore mysql database uncompressing on the fly.
\nThis way you keep the file compressed saving disk space.
\nOther way less optimal using named pipes:
\nmysql -uroot -p’passwd’ database <\n\n\n( zcat $FILE || gzcat $FILE || bzcat2 $FILE ) | less\nGroup OR'd commands where you expect only one to work\nSomething to stuff in an alias when you are working in multiple environments. The double-pipe OR will fall through until one of the commands succeeds, \nand the rest won't be executed. Any STDERR will fall out, but the STDOUT from the correct command will bubble out of the parenthesis to the less command, or some other command you specify.\n\n\n( last ; ls -t \/var\/log\/wtmp-2* | while read line ; do ( rm \/tmp\/wtmp-junk ; zcat $line 2>\/dev\/null || bzcat $line ) > \/tmp\/junk-wtmp ; last -f \/tmp\/junk-wtmp ; done ) | less
\nFunctions: last ls read rm zcat
\nTags: last command wtmp
\nSee a full last history by expanding logrotated wtmp files
\nWhen your wtmp files are being logrotated, here’s an easy way to unpack them all on the fly to see more than a week in the past. The rm is the primitive way to prevent symlink prediction attack.<\/p>\n
\nFunctions: awk sort tail uniq zcat
\nTags: log apache zcat analysis
\nAnalyse compressed Apache access logs for the most commonly requested pages<\/p>\n
\nFunctions: awk printf sudo zcat
\nTags: Security awk brute force
\nShow the number of failed tries of login per account. If the user does not exist it is marked with *.<\/p>\n
\nScan a gz file for non-printable characters and display each line number and line that contains them.
\nScans the file once to build a list of line numbers that contain non-printable characters
\nScans the file again, passing those line numbers to sed as two commands to print the line number and the line itself. Also passes the output through a tr to replace the characters with a ?<\/p>\n
\nFunctions: vim zcat
\nPipe a textfile to vim and move the cursor to a certain line
\nThis command is more for demonstrating piping to vim and jumping to a specific line than anything else.
\nExit vim with :q!
\n+23 jumps to line 23
\n– make vim receive the data from the pipe<\/p>\n
\nThis decompresses the file and sends the output to STDOUT so it can be grepped. A good one to put in loops for searching directories of gzipped files, such as man pages.<\/p>\n
\nSTAMP=`date ‘+%Y%m%d-%H:%M’`
\nREMOTE_MYCNF=\/var\/log\/mysoft\/mysoft.log
\nREMOTE_GZ=\/var\/log\/mysoft\/mysoft.log.1.gz
\nREMOTE_DIR=\/var\/log\/mysoft\/
\nBACKUP_DIR=\/home\/dev\/logs\/
\nNEWLOG=”foo-temp.log”
\nssh $1 “zcat $REMOTE_GZ >> $REMOTE_DIR$NEWLOG”
\nssh $1 “cat $REMOTE_MYCNF >> $REMOTE_DIR$NEWLOG”<\/p>\n","protected":false},"excerpt":{"rendered":"