Nagios xi - Running Out Of Disk Space On The Nagios xi VM

Overview

Running out of HDD space on the Nagios xi server could create a large number of problems - disrupted monitoring, data loss, database corruption, inability to log in the web interface, tons of notifications, etc. If you receive warning or critical alerts about "Root Partition" on localhost, you need to act fast and fix the low HDD space issue. Don't wait until your hard drive is 100% full!

Possible Causes

• Increasing the number of hosts/services has consumed more disk space
• Large number of backups
• Large log files

Possible Solutions

Always plan ahead and set up your environment in a way that you allow for some growth. You can review the general hardware requirements needed to run Nagios xi here.

It is always a good idea to backup your Nagios xi instance but if you are saving them locally and don't have large enough hard drive, you can run out of space very fast.

By default, the backups are stored in /store/backups/nagiosxi/ directory. It is a good practice to keep an eye on this folder and remove the old backups that you don't need.

Also, there are some mysql and postgres backups in /store/backups/mysql/ and /store/backups/postgresql/ directories. These are created on daily, monthly, and weekly basis by the /root/scripts/automysqlbackup and /root/scripts/autopostgresqlbackup script. Both scripts are run on a cron job. Run the following command to view the crontab:

cat /etc/cron.d/nagiosxi

Keep in mind that the /store/backups/mysql/ and /store/backups/postgresql/ directories can also grow quite large. If you need to clear some HHD space you could delete some of the old backups that are no longer needed.

You can disable these two cron jobs if you wish by commenting them out (see below):

# 0 7 * * * root   /root/scripts/automysqlbackup
# 0 8 * * * root   /root/scripts/autopostgresqlbackup

Sometimes, your log files can grow out of control, especially if you have a poorly configured checks, system issues, debugging enabled, etc. You can temporarily clear some space by deleting old logs that you might not need.

Tip: In order to view the 10 largest files under /var/log/ directory you can run:

du -a /var/log | sort -n -r | head -n 10

Another option is to add another disk and move the existing data to that location. The following KB article walks you through moving /var/log to another disk:

Documentation - Moving /var/log

If you don't care about historical data, you can also delete some old nagios log files from the /usr/local/nagios/var/archives/ directory. These files are used for some of the reporting (mainly the legacy reports and the availability report).

After clearing some space, you will need to resize the virtual machine by following the steps, outlined in our Nagios xi - Resizing the VM Disk Size document.

Final Thoughts

For any support related questions please visit the Nagios Support Forums at:

http://support.nagios.com/forum/