Splitting up Apache Logs for Subdomains
1. Make sure your multiple virtual hosts log to the same log file.
example:
@ your config file
ErrorLog “|/usr/sbin/rotatelogs -l /var/log/httpd/linux/error_log.%Y%m%d 86400?
CustomLog logs/linux/access_log combinedio
CustomLog “|/usr/sbin/rotatelogs -l /var/log/httpd/linux/access_log.%Y%m%d 86400? combinedio
2. Change your Apache LogFormat
from let say:
#LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\”” combined
to:
LogFormat “%V %h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\” \”%{HTTP_X_UP_CALLING_LINE_ID}e\” \”%{HTTP_X_WSB_IDENTITY}e\ %I %O” combinedio
where: (use %V or %v based on your httpd.conf setting)
%V =the server name according to the UseCanonicalName setting
%v=the canonical ServerName of the server serving the request.
3. Split your log files into its component parts and use the perl script “split-logfile. This program can be found in the support folder of your Apache distro.
split-logfile < /logs/access_log
You can create script to automate this, e.g.
[root@dc1 log]# cat split.sh
#!/bin/bash
#define
TODAY=`date “+%Y%m%d”`
YEST=`date “+%Y%m%d” -d”1 day ago”`
LOGPATH=”/var/log/httpd/linux”
ACCESSFILE=”$LOGPATH/access_log.$YEST”
CURFOLDER=”/root/log”
#split
cd $CURFOLDER
/darwin/log/split-logfile < $ACCESSFILE
echo $ACCESSFILE
#rename
/usr/bin/rename log log.$YEST *log
-> This script basically reads the rotated access log from the log path, and split the log files according to the server name
Sample Results
@original access logs
[root@flt log]# ls -la /var/log/httpd/linux/
total 544
drwxr-xr-x 2 root root 4096 Apr 20 16:07 .
drwx—— 3 root root 4096 Apr 18 04:02 ..
-rw-r–r– 1 root root 103090 Apr 20 16:08 access_log
-rw-r–r– 1 root root 53770 Apr 13 18:11 access_log.20100420
-rw-r–r– 1 root root 48660 Apr 13 18:11 access_log.20100421
-rw-r–r– 1 root root 710 Apr 15 16:33 error_log.20100420
-rw-r–r– 1 root root 50 Apr 15 16:33 error_log.20100421
@splitted log files after executing the script
[root@dc1 log]# ls -la /root/log
total 292
drwxr-xr-x 2 root root 4096 Apr 21 01:01 .
drwxr-x— 8 root root 4096 Apr 13 15:56 ..
-rw-r–r– 1 root root 9278 Apr 15 01:01 admin.linux.com.log.20100420
-rw-r–r– 1 root root 436 Apr 15 01:01 www.linux.com.log.20100420
-rw-r–r– 1 root root 1296 Apr 20 01:01 m.linux.com.log.20100420
-rw-r–r– 1 root root 6104 Apr 14 01:01 secure.linux.com.log.20100420
Recent Comments