Nagios xi - Monitoring the Nagios xi "localhost"

Overview

This guide on Best Practices is about monitoring the Nagios xi server itself and what services you could deploy.

Monitoring the Nagios xi "localhost"

Do you know how your xi server is performing?

• Services you should ideally be monitoring

  • crond, httpd, mysqld, ndo2db, npcd, ntpd, postgresql

  • postgresql is not used on fresh installations of xi 5.x onwards. If you upgraded to xi 5 from a previous version postgres is still used.

  • snmptrapd and snmptt are not present until you follow the xi SNMP Trap procedure

  • The plugin called check_init_service that is part of the linux-nrpe-agent.tar.gz package can be used to monitor these services

    • Requires the following line in /etc/sudoers

    • nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_init_service

    • Your command definition needs to be:

    • sudo $USER1$/check_init_service $ARG1$

• File counts - check these folders to make sure the temp files are being processed

  • NPCD Perfdata spool directory

    • /usr/local/nagios/var/spool/perfdata/

  • xidpe spool directory

    • /usr/local/nagios/var/spool/xidpe/

  • Check results folder

    • /usr/local/nagios/var/spool/checkresults/

  • snmptt spool folderNOTE: These locations can vary if a RAM Disk has been implemented 

    • /var/spool/snmptt/

  • Use the Folder Watch wizard to create services to monitor these folders

• Has the nagios user account expired?

  • In some customer installations, it’s possible that the nagios user account expires. This isn’t always that obvious to troubleshoot, so checking that it hasn’t expired is a good precautionary measure.

  • check_pass_expire.plis the plugin you can use

    • https://exchange.nagios.org/directory/Plugins/Operating-Systems/Linux/check_pass_expire/details

  • File permission changes required:

    • setfacl -m u:nagios:r-- /etc/shadow

  • Plugin also needs line 23 changed

• From:

  • use lib ".";

• To:

  • use lib "/usr/local/nagios/libexec";

• root mailbox size

  • If you’re not a Linux person then you probably don’t know about the system mailbox. This is a local mail system on the linux server where messages are sometimes sent.

  • Certain components used in Nagios xi such as MRTG will send messages to this mailbox when it has a problem. An incorrect MRTG configuration can cause a message to be sent every five minutes as this is when MRTG runs. That’s about 288 messages a day. Over time the root mailbox can grow to GB in size causing issues. I wrote a plugin which can monitor this and let you know when it gets too big.

  • box293_check_mbox.plis the plugin you can use

    • https://exchange.nagios.org/directory/Plugins/Email-and-Groupware/box293_check_mbox/details

  • File permission changes required:

    • setfacl -m u:nagios:r-- /var/spool/mail/root

• MySQL / MariaDB Databases

  • If the tables are crashed and go undetected, this can have a severe impact on the system and you may not be storing important data and it may cause strange problems.

  • box293_check_mysql_table_status is the plugin you can use to check this:

    • https://exchange.nagios.org/directory/Plugins/Databases/MySQL/box293_check_mysql_table_status/details

  • Another problem can occur if the database engine runs on a different timezone to the local system.

• box293_check_mysql_date is the plugin you can use to check this:

• https://exchange.nagios.org/directory/Plugins/Databases/MySQL/box293_check_mysql_date/details

• Monitor Max_used_connections and Threads_connected

• check_mysql_query is the plugin you can use to check this, here is a guide that explains it all:

• Nagios xi - MySQL/MariaDB - Max Connections

• Overall Load

  • This service is included by default in Nagios xi

• check_load is the plugin included with Nagios xi (5) 

• Memory Free – Physical

  • Make sure your xi server doesn't run out of memory

  • check_memory.sh is the plugin you can use

    • https://exchange.nagios.org/directory/Plugins/Operating-Systems/Linux/check_memory-2Esh/details

• Swap Usage

  • If the system runs out of physical memory and starts swapping to disk, the system performance will be greatly impacted.

  • This service is included by default in Nagios xi

  • check_swap is the plugin included with Nagios xi (5)

• Disk Free

  • Disk free space is very important.

  • This service is included by default in Nagios xi however it only monitors /

  • If you have different volumes mounted then you should be monitoring each one of these.

  • check_disk is the plugin included with Nagios xi (5)

Final Thoughts

For any support related questions please visit the Nagios Support Forums at:

http://support.nagios.com/forum/