Log files are an essential part of Linux system administration and monitoring. But log files also quickly grow large and take up a lot of storage space. The good news is that a tool called Logrotate in Linux lets you manage log files effectively.
In this tutorial, you’ll learn how to configure Logrotate so you can manage and prevent log files size from getting out of hand in your machine.
Ready? Dig in to get started!
Prerequisites
This tutorial comprises step-by-step instructions. If you’d like to follow along, be sure you have the following in place:
- An Ubuntu machine – This tutorial uses Ubuntu 20.04 LTS, but other Linux distros will work.
Viewing Logrotate Configuration Files for Packages
Log rotation can be a bit confusing at first, but you’ll get the hang of it once you know what the various parameters mean. Logrotate uses configuration files to set rules for the rotation’s behavior, such as the maximum log files to keep, how often the rotation will run, and the log file’s size to compress.
Logrotate’s configuration files are located in /etc/logrotate.conf file and /etc/logrotate.d/ directory:
- The /etc/logrotate.conf configuration file – Contains default options for log rotation but does not perform any actions by itself. You can use this configuration file to specify default parameters for all of your log rotations which does not exist in the /etc/logrotate.d/ directory.
- The /etc/logrotate.d/ directory – This directory stores individual log rotation config files, such as alternatives, apt, syslog, udev, and so on. You can use these example config files as a starting point for creating your own log rotation rules.
Take a look at the Logrotate configuration file for the apt package so that you can understand the parameters in more detail.
Run the cat command below to view the contents of Logrorate’s configuration file for the apt package (/etc/logrotate.d/apt).
sudo cat /etc/logrotate.d/aptIn the screenshot below, you can see two main sections, /var/log/apt/term.log and /var/log/apt/history.log. Both sections have the same set of options that override the default Logrotate settings in the /etc/logrotate.conf file.
This tutorial can’t cover all the options you can set for your log rotation file. But you can run the
sudo man logrotatecommand to get the list of available options.
Setting up Custom Rules to Rotate Logs
One of the great things about Logrotate is that it is customizable. You can set up your own rules to rotate your logs the way you want.
There are two ways to create custom Logrotate rules:
- Add a config file to the /etc/logrotate.d directory.
- Create a script outside Logrotate’s installation directory to run Logrotate for you.
But for this example, you’ll create a custom config file in the /etc/logrotate.d directory.
Creating Logrotate rules in the /etc/logrotate.d directory rotates your logs only daily, weekly, and monthly by default.
1. Open the /etc/logrotate.d/apache2.conf file in your preferred text editor.
From this point throughout the tutorial, you need
sudoprivileges to edit a configuration file’s content.
2. Copy and paste the following code (logrotate rule) to the apache2.conf file and save the changes.
The code below rotates logs every week. Based on the specified options, the rules keep the three most recent rotated logs and compress them on a disk if they are 10 megabytes in size or less.
When the options are not defined in /etc/logrotate.d/ directory, the default settings specified in /etc/logrotate.conf will be used.
/var/log/apache2/* {
# Sets the logs to rotate every week
weekly
# Tells the system to remove old logs and only keep the three most recent rotated logs
rotate 3
# Rotated logs will be compressed and kept on disk if they are 10 MB or less.
size 10M
# compress and delaycompress: These two options are used together and indicate that
# rotated logs should be compressed (gzip) except for the most recent one.
compress
delaycompress
}3. Finally, run the logrotate command below to perform a logrotate dry run. This command turns on a dry run (-d) for the new rule you previously set (step two) in the apache2.conf file.
The dry run will simulate the rotation without actually rotating the logs. A dry run is a great way to test your new logrotate rule to ensure it’s working correctly.
sudo logrotate -d /etc/logrotate.d/apache2.conf
You might get an error, like the one below, which is normal since you are testing the rules by doing a dry run on a fictional apache web server.
But at this point, your standard Logrotate job is set and will run once a week with your new custom rules.
Setting up Custom Rules Outside Logrotate Installation Directory
You’ve just created your custom Logrotate rules in the /etc/logrotate.d/ directory. But perhaps you prefer to create the Logrotate rules in a different location. If so, for this demo, you’ll create custom rules in your home directory that rotate and compress logs files in a specific directory.
Unlike creating Logorate rules in the /etc/logrotate.d directory, custom rules set up outside the Logrotate installation directory let you set a rule to rotate the logs hourly. Hourly logs help more with debugging your system than daily logs, which may not provide enough information.
In this example, you’ll run a fictional app that generates logs stored in ~/logs/ directory.
1. Run the following command in your home directory to create a directory named logs.
mkdir ~/logs2. Next, open your text editor and create a configuration file named logrotate.conf. Populate the logrotate.conf file with the following contents.
The code below will rotate your logs every hour and compresses them, keeping 24 rotation logs. At the same time, the old logs are removed when the new logs are created in the ~/logs directory.
home/demo2/logs/*.log {
# The logs will be rotated every hour.
hourly
# If the log file is missing, go on to the next one without issuing an error message.
missingok
# keeping 24 newest logs
rotate 24
# compressing old logs
compress
# creating a new log file to replace the rotated one
create
}3. Run the following command to create a new log file named access.log in your ~/logs directory to test the Logrotate settings. This log file is the sample file to rotate and compress later (step four).
sudo touch logs/access.log4. Now run the command below to test the custom rule (~/logrotate.conf) you set up (step three). The --state flag specifies the name of the Logrotate state file (logrotate-state). While the --verbose flag prints out detailed information about the Logrorate run.
The logrotate-state file stores the last run time of the Logrotate rules. As a result, Logrotate will not run the rules if it has already run them before this new instance.
logrotate /home/demo2/logrotate.conf --state /home/demo2/logrotate-state --verboseBelow, you can see that Logrotate ran successfully but did not rotate the logs since the log file is less than one hour old.
Perhaps you want to see the current state of the logrotate-state file. If so, run the command below to view the logrotate-state file’s content.
sudo cat /home/demo2/logrotate-stateAs you can see below, the logrotate-state file contains the following information, which states the last time Logrotate ran the custom rules.
5. Run the crontab command to open the crontab file in your default text editor so you can add a cron job.
crontab -e6. In the crontab editor, add the following line to the bottom of the editor and save the changes. This line will run the logrotate command every hour using the /home/demo2/logrotate.conf and the /home/demo2/logrotate-state files.
The cronjob will run the logrotate command every hour at 14 minutes past an hour, daily.
14 * * * * /usr/sbin/logrotate /home/demo2/logrotate.conf --state /home/demo2/logrotate-state
7. Finally, run the ls command one hour after creating the access.log file to revisit the ~/logs directory.
ls logsBelow, you’ll see a new rotated and compressed access.log file (access.log.1.gz), which indicates Logrotate executed the custom rules successfully.
Conclusion
Throughout this tutorial, you’ve learned to configure Logrotate to run every hour without the need for a sudo privilege. You’ve also touched on adding your custom Logrotate rule as a cronjob to automate the Logrotate process.
At this point, you can now rotate and compress your logs files regularly, which will help you to keep your log files organized and manageable. Additionally, you can free up storage space in the /var/log directory by compressing your log files.
Now how will you build up with this newfound knowledge? Perhaps configure the Logrotate software to automatically send a daily summary of the rotated logs files to your email?
