If you have multiple virtual hosts in your server and you specified different log files for each, it is more organized and easy to locate, but the downside is Linux has so called file descriptors or the number of files that it can be handle. Basically , it can be increased but I think it’s not really recommended just for the sake of it.
Making your apache logs more organized can be done by many ways, one way is via piped logging and use rotatelogs. And based on my tips before named Automatic unlimited subdomains via Apache mod_rewrite, one issue is the log files. But there’s one method to solve this, and this is by splitting up your log files.
1. Make sure your multiple virtual hosts log to the same log file.
example:
@ your config file
ErrorLog “|/usr/sbin/rotatelogs -l /var/log/httpd/linuxtutorials/error_log.%Y%m%d 86400”
CustomLog logs/linuxtutorials//access_log combinedio
CustomLog “|/usr/sbin/rotatelogs -l /var/log/httpd/linuxtutorials//access_log.%Y%m%d 86400” combinedio
2. Change your Apache LogFormat
from let say:
#LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\”” combined
to:
LogFormat “%V %h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\” \”%{HTTP_X_UP_CALLING_LINE_ID}e\” \”%{HTTP_X_WSB_IDENTITY}e\ %I %O” combinedio
where: (use %V or %v based on your httpd.conf setting)
%V =the server name according to the UseCanonicalName setting
%v=the canonical ServerName of the server serving the request.
3. Split your log files into its component parts and use the perl script “split-logfile. This program can be found in the support folder of your Apache distro.
split-logfile < /logs/access_log
You can create script to automate this, e.g.
[root@dc1 log]# cat split.sh
#!/bin/bash
#define
TODAY=`date “+%Y%m%d”`
YEST=`date “+%Y%m%d” -d”1 day ago”`
LOGPATH=”/var/log/httpd/linuxtutorials”
ACCESSFILE=”$LOGPATH/access_log.$YEST”
CURFOLDER=”/root/log”
#split
cd $CURFOLDER
/darwin/log/split-logfile < $ACCESSFILE
echo $ACCESSFILE
#rename
/usr/bin/rename log log.$YEST *log
-> This script basically reads the rotated access log from the log path, and split the log files according to the server name
Sample Results
@original access logs
[root@flt log]# ls -la /var/log/httpd/linuxtutorials/
total 544
drwxr-xr-x 2 root root 4096 Apr 20 16:07 .
drwx—— 3 root root 4096 Apr 18 04:02 ..
-rw-r–r– 1 root root 103090 Apr 20 16:08 access_log
-rw-r–r– 1 root root 53770 Apr 13 18:11 access_log.20100420
-rw-r–r– 1 root root 48660 Apr 13 18:11 access_log.20100421
-rw-r–r– 1 root root 710 Apr 15 16:33 error_log.20100420
-rw-r–r– 1 root root 50 Apr 15 16:33 error_log.20100421
@splitted log files after executing the script
[root@dc1 log]# ls -la /root/log
total 292
drwxr-xr-x 2 root root 4096 Apr 21 01:01 .
drwxr-x— 8 root root 4096 Apr 13 15:56 ..
-rw-r–r– 1 root root 9278 Apr 15 01:01 admin.freelinuxtutorials.com.log.20100420
-rw-r–r– 1 root root 436 Apr 15 01:01 www.freelinuxtutorials.com.log.20100420
-rw-r–r– 1 root root 1296 Apr 20 01:01 m.freelinuxtutorials.com.log.20100420
-rw-r–r– 1 root root 6104 Apr 14 01:01 secure.freelinuxtutorials.com.log.20100420