How to display the contents of a gzip/gz file
By Alvin Alexander. Last updated: Aug 6, 2011
Problem: You have a plain text file that has been compressed with the gzip command, and you’d like to display the file contents with the Unix/Linux cat or more commands.
Solution: Instead of using the cat or more commands, use their equivalents for working with gz files, the zcat and zmore commands.
For instance, if you want to display the contents of an Apache log file (which is a plain text file) that has been compressed with gzip, just use the zcat command, like this:
zcat access_log.gz
Of course almost any Apache log file will be large, and will scroll off the screen quickly, so you’ll probably want to use the gzip equivalent of the more command, zmore, like this:
zmore access_log.gz
find . -name “*.gz” | while read -r file; do zcat -f “$file” | head -n 1; done
zcat `man -w manpage` | groff -mandoc -T html – > filename.html
save manpage as html file
zcat log.tar.gz | grep -a -i “string”
grep compressed log files without extracting. Useful in system where log files are compressed for archival purposes
zcat /usr/share/man/man1/man.1.gz | nroff -man | less
As odd as this may be, I know of servers where the man(1) command is not installed, and there is not enough room on / to install it. However, zcat(1), nroff(1) and less(1) are. This is a way to read those documents without the proper tool to do so, as sad as this may seem. 🙂
This command enables the user to append a search pattern on the command line when using less as the PAGER. This is especially convenient (as the example shows) in compressed files and when searching man pages (substituting the zcat command with man, however).
zcat -f $(ls -tr access.log*)
concatenate compressed and uncompressed logs
with zcat force option it’s even simpler.
find /var/log/apache2 -name ‘access.log*gz’ -exec zcat {} \; -or -name ‘access.log*’ -exec cat {} \;
functions: cat find zcat
concatenate compressed and uncompressed logs
This command allows you to stream your log files, including gziped files, into one stream which can be piped to awk or some other command for analysis.
Note: if your version of ‘find’ supports it, use:
find /var/log/apache2 -name ‘access.log*gz’ -exec zcat {} + -or -name ‘access.log*’ -exec cat {} +
zcat database.sql.gz | mysql -uroot -p’passwd’ database
Functions: zcat
Restore mysql database uncompressing on the fly.
This way you keep the file compressed saving disk space.
Other way less optimal using named pipes:
mysql -uroot -p’passwd’ database <
( zcat $FILE || gzcat $FILE || bzcat2 $FILE ) | less
Group OR'd commands where you expect only one to work
Something to stuff in an alias when you are working in multiple environments. The double-pipe OR will fall through until one of the commands succeeds,
and the rest won't be executed. Any STDERR will fall out, but the STDOUT from the correct command will bubble out of the parenthesis to the less command, or some other command you specify.
( last ; ls -t /var/log/wtmp-2* | while read line ; do ( rm /tmp/wtmp-junk ; zcat $line 2>/dev/null || bzcat $line ) > /tmp/junk-wtmp ; last -f /tmp/junk-wtmp ; done ) | less
Functions: last ls read rm zcat
Tags: last command wtmp
See a full last history by expanding logrotated wtmp files
When your wtmp files are being logrotated, here’s an easy way to unpack them all on the fly to see more than a week in the past. The rm is the primitive way to prevent symlink prediction attack.
zcat access_log.*.gz | awk ‘{print $7}’ | sort | uniq -c | sort -n | tail -n 20
Functions: awk sort tail uniq zcat
Tags: log apache zcat analysis
Analyse compressed Apache access logs for the most commonly requested pages
sudo zcat /var/log/auth.log.*.gz | awk ‘/Failed password/&&!/for invalid user/{a[$9]++}/Failed password for invalid user/{a[“*” $11]++}END{for (i in a) printf “%6s\t%s\n”, a[i], i|”sort -n”}’
Functions: awk printf sudo zcat
Tags: Security awk brute force
Show the number of failed tries of login per account. If the user does not exist it is marked with *.
zcat a_big_file.gz | sed -ne “$(zcat a_big_file.gz | tr -d “[:print:]” | cat -n | grep -vP “^ *\d+\t$” | cut -f 1 | sed -e “s/\([0-9]\+\)/\1=;\1p;/” | xargs)” | tr -c “[:print:]\n” “?”
Functions: sed tr zcat
Scan a gz file for non-printable characters and display each line number and line that contains them.
Scans the file once to build a list of line numbers that contain non-printable characters
Scans the file again, passing those line numbers to sed as two commands to print the line number and the line itself. Also passes the output through a tr to replace the characters with a ?
zcat /usr/share/doc/vim-common/README.gz | vim -g +23 –
Functions: vim zcat
Pipe a textfile to vim and move the cursor to a certain line
This command is more for demonstrating piping to vim and jumping to a specific line than anything else.
Exit vim with :q!
+23 jumps to line 23
– make vim receive the data from the pipe
zcat /usr/share/man/man1/grep.1.gz | grep “color”
Search gzipped files
This decompresses the file and sends the output to STDOUT so it can be grepped. A good one to put in loops for searching directories of gzipped files, such as man pages.
#!/bin/sh
STAMP=`date ‘+%Y%m%d-%H:%M’`
REMOTE_MYCNF=/var/log/mysoft/mysoft.log
REMOTE_GZ=/var/log/mysoft/mysoft.log.1.gz
REMOTE_DIR=/var/log/mysoft/
BACKUP_DIR=/home/dev/logs/
NEWLOG=”foo-temp.log”
ssh $1 “zcat $REMOTE_GZ >> $REMOTE_DIR$NEWLOG”
ssh $1 “cat $REMOTE_MYCNF >> $REMOTE_DIR$NEWLOG”
Recent Comments