Overview
It’s always important to keep your server logs around for as long as it makes business sense. You’ll need them for auditing system access, discovering abuses, or to identify root causes to problems, among other reasons. The challenge, though, is that depending on the service being provided and the amount of traffic received, your logs are capable of growing to gargantuan sizes, consuming every last bit of disk space available.
Logrotate allows us to better manage our logs to prevent from consuming too much disk space. Depending on the schedule you decide on, your logs can be rotated every day, week, or month. Each rotation renames your existing log file, usually by appending a ‘.’ and number to the end, and then creates a new file. To preserve storage the logs that have been rotated can be compressed using Gzip.
Thankfully, most major Linux distributions, like Ubuntu, automatically rotates most logs found under /var/log.
Installation
Logrotate is installed by default on most recent distributions. As of this writing, that means Ubuntu 15.10 and CentOS 7 – 1511. However, if you find that your installation does not include it, it can easily be installed through Yum or Apt.
Create a Log Rotate Configuration
Logrotate configuration files are stored under /etc/logrotate.d. When the logrotate cron job runs, it will look execute any configuration found in that directory.
The following example is that of a default Apache2 log from a CentOS 7 server. We can use it as a template for our own application logs, separate from the default ones.
/var/log/httpd/*log {
missingok
notifempty
sharedscripts
delaycompress
postrotate
/bin/systemctl reload httpd.service > /dev/null 2>/dev/null || true
endscript
}
Here’s an explanation of the settings used above.
| /var/log/httpd/*log | The location of the log file(s). In this example, find any file that ends with ‘log’ in the /var/log/httpd directory. |
|---|---|
| missingok | Tells logrotate to ignore any errors if the log file(s) does not exist. |
| notifempty | Do not rotate log file if it is empty. |
| sharedscripts | The postrotate script will only run once after all matching log files are found. If one log file doesn’t need rotating, the script will not run. |
| delaycompress | Delay compression of a rotated log file until the next rotation. This allows us to keep the last rotation uncompressed and easily searchable while ensuring older rotations are compressed. |
| postrotate | The command executed after rotation is complete. In our example above, Apache (httpd) is restarted, which causes it to create new log files. |
Logrotate supports many more options than what’s listed above. To see a complete list, you can man logrotate or visit linuxcommand.org logrotate page. Here are a few options you may find useful.
| daily | Rotate the logs daily. |
|---|---|
| weekly | Rotate the logs weekly. |
| monthly | Rotate the logs monthly. |
| rotate x | Rotate the log x amount of times before purging older logs. Using ‘rotate 5’ means you will only keep 5 rotations of a log. |
| copy | This option copies the log file, essentially creating a snapshot of it at a particular point in time. |
| prerotate | Like postrotate, but runs a script before log rotation is completed. |
Test Your Configuration
As with anything, before you roll your configuration into production you will want to test it to ensure everything works. Logrotate includes a feature that allows us to run a configuration, in debug mode, without it doing any work. If there are errors discovered in the syntax or some other issues, you will be notified.
sudo logrotate -d /etc/logrotate.d/myapp.conf
The output will look similar to the example below. The output displays the discovered logs based on the location /var/httpd/*log, and what was done for each log. None of the logs were rotated since I ran the test on a server that was newly built. However, if the logs had more content and were aged a little more, we would see a message indicating our logs were rotated. For now, we can take this as our configuration file syntax is correct.
empty log files are not rotated, old logs are removed considering log /var/log/httpd/access_log log does not need rotating considering log /var/log/httpd/error_log log does not need rotating not running postrotate script, since no logs were rotated
