Managing Log Retention using Logrotate

#Linux #logrotate

Mohamed El Eraki's photo
Mohamed El Eraki
·

4 min read

Managing Log Retention using Logrotate

Table of contents

Inception

Maintaining a healthy server environment involves regular monitoring and management of log files. Log files can grow significantly over time and can consume a considerable amount of disk space if not managed properly. One of the most efficient tools for managing log files in Linux-based systems is Logrotate. This guide will walk you through the basics of Logrotate, including its installation, configuration, and usage to ensure efficient log retention management.

Introduction to Logrotate

Logrotate is a system utility that manages the automatic rotation and compression of log files. If log files are not rotated, compressed, and periodically pruned, they can grow to a size that is unmanageable, not just consuming disk space but also making log analysis more difficult. Logrotate allows for the automatic rotation of log files according to various criteria such as file size, time, or upon reaching a specific size, and then compressing and optionally mailing them.

Checking if Logrotate is Installed

Before configuring Logrotate, it's a good idea to check if it's already installed on your system. You can do this by querying your package manager or by checking for the existence of the logrotate command.

For Debian-based systems, you can use dpkg:

dpkg -l | grep logrotate

For Red Hat-based systems, use rpm:

rpm -q logrotate

Alternatively, you can check for the Logrotate command directly:

which logrotate

If Logrotate is installed, the which command will return the path to the Logrotate executable. If it returns no output, you may need to install Logrotate as described in the "Installing Logrotate" section.

Installing Logrotate

Logrotate is pre-installed on most Linux distributions. However, if it's not installed on your system, you can easily install it using your distribution's package manager.

For Debian-based systems:

sudo apt-get update
sudo apt-get install logrotate

For Red Hat-based systems:

sudo yum update
sudo yum install logrotate

Configuring Logrotate

Logrotate's configuration is controlled by the /etc/logrotate.conf file and additional configuration files stored in /etc/logrotate.d/. The main configuration file includes global options and can include other configuration files.

Here is a basic example of a Logrotate configuration for a custom application log file located at /var/log/myapp/application.log:

vim /etc/logrotate.d/<fileName>

/var/log/myapp/application.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 640 root root
}

Configuration Options Explanation

  • daily: Rotate the log files each day.

  • rotate 7: Keep the last 7 rotated log files.

  • compress: Compress rotated files with gzip.

  • delaycompress: Compress the log file on the next rotation cycle. This can be useful when some processes might still be writing to the rotated log file.

  • missingok: Do not throw an error if the log file is missing.

  • notifempty: Do not rotate the log file if it is empty.

create 640 root root: Create new log files compressed with set permissions/owner/group.

Configuring Logrotate - Another Example

Logrotate configurations can manage multiple paths that shares the same configurations, Also provide a post Action (e.g. Run a script after rotated)

vim /etc/logrotate.d/<filename>

/paths/of/log/One.log
/paths/of/log/twoDir/* {
  weekly 
  rotate 1
  compress
  missingok
  create 640 <groupName> <userName>
  postrotate
    # Any post-rotate commands here
    # or for example restarting a service
  endscript
}
💡
This configurations run weekly with keep one log rotated

Configuring Logrotate - Another Example

/paths/of/log/One.log
/paths/of/log/twoDir/* {
  weekly 
  rotate 1
  compress
  missingok
  sharedscripts
  create 640 <groupName> <userName>
  postrotate
    # Any post-rotate commands here
    # or for example restarting a service
  endscript
}

sharedscripts: This directive means that the post-rotate script will only be run once after all log files have been rotated, instead of once for each log file which is the default behavior.

Testing Logrotate Configuration

To test your Logrotate configuration, you can use the -d (debug) option. This will simulate the rotation without actually changing any files:

sudo logrotate -d /etc/logrotate.conf

# if you create your own logrotate config file use:
sudo logrotate --debug /etc/logrotate.d/<fileName>

Forcing Log Rotation

If you want to force log rotation to run now, you can use the -f option:

sudo logrotate -f /etc/logrotate.conf

The compressed file should be created at the same path of the log files, Let's Check

ls -lathr /paths/of/log/

# Check the dir size
du -h /paths/of/log/
💡
Check the Dir Size Before and after to notice the difference.

There's More

Looking for more Option!? Look at here

Conclusion

Logrotate is a powerful tool for managing log files on Linux-based systems. By properly configuring Logrotate, you can ensure that your log files are rotated, compressed, and pruned regularly, keeping your system clean and ensuring that log files do not consume excessive disk space. Remember to test your Logrotate configurations before applying them to prevent any unexpected behavior.

That's it, Very straightforward, very fast🚀. Hope this article inspired you and will appreciate your feedback. Thank you.

Linuxlogging