Nagios xi Performance Issues

itunixops · Post by **itunixops** » Wed Dec 30, 2020 1:37 pm

Nagios xi has been a staple of our networking group for over 3 years. We had over 10 with nagios core and have redone alot of our original plugins and so when xi came out. Our first experience with xi was version 5.2.3. Over the years we've been updating and updating the os and keeping up as best we can with updates.

But now we have some issues with performance. Its not performing as fast as it should and sometimes we get little errors. We've fixed alot of those but need to know some things.

We have about a dozen or so plugins which are NRPE based and many SNMP based. We would like to see about improving performance and understand there is a replacement for NRPE which improves on that. We also noticed over the years databases get bigger and bigger and could slow down. One idea we had was to put the database on a seperate server and see if performance will improve.

Is there a best or preferered method to improve performance on our systems. At best we have an average of 200+ servers on two systems and less than 30 on one. What could be recommened to help perofrmance? We can send profiles or so if needed.

We'd like to do this is in preparation for and RHEL8 and Nagios xi 6.0 upgrade when they become avialable. Of course we can make sure that works perfectly on our RHEL7 systems first off.

Thanks

benjaminsmith · Post by **benjaminsmith** » Wed Dec 30, 2020 4:42 pm

Hi @itunixops,

Thank you for being a long-time Nagios xi user, we really appreciate it!

The performance issues you are having, when did they start and what type of error messages are you seeing. Did they correspond with any changes to the system or a large increase in host or service checks?

I'd like to get a system profile to review the current check load (total hosts + services) along with the system stats.

To send us your system profile.
Login to the Nagios xi GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button

Generally speaking running checks with an agent like NRPE or NCPA is faster than SNMP and requires less CPU resources to execute each check.

One of the best ways to increase performance is simply to space out the check_intervals as much as possible. If you are checking every 3 minutes increasing this to every 5 minutes will really help.

Offloading the database can help but it can also cause other issues (e.g. data transfers over the network), so I would not recommend that to start.

We have a guide below with some good recommendations for increasing performance. Take a look and let me know if you have any questions.

Maximizing Performance In Nagios xi

If you haven't done so already, I highly recommend increasing the PHP defaults.

Nagios xi - Optimizing The PHP Settings File

Lastly, please post the output to the following query to check your table sizes.

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql', 'nagiosxi');" | mysql -h 127.0.0.1 -uroot -pnagiosxi --table

Best Regards,
Benjamin

itunixops · Post by **itunixops** » Thu Dec 31, 2020 10:16 am

We started seeing alot of these recently after remoivng alot of our legacy servers and adding more. Almost an equal type number being removed and installed.

I've included the profile and the output from the tables to see what we can do. We also have modified this system to meet our securit requirements as we replaced PHP 5 with 7.2.

We'll go throughe additional documentation you sent and see what we can do as well until you get a chance to review our data.

Thanks

Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.

Post by **WillemDH** » Fri Jan 01, 2021 12:14 pm

Hello, @itunixops You should probably better pm your profile etc to Bejamin Smith instead of attaching it.

ssax · Post by **ssax** » Mon Jan 04, 2021 5:05 pm

We started seeing alot of these recently after remoivng alot of our legacy servers and adding more

Please explain in more detail on what specifically you are seeing. What errors/etc are you seeing?

What is the output of these commands as root:
- Change the domain in the nslookup command to what it should be

Code: Select all

sar
ulimit -a
su -s /bin/bash -c 'ulimit -a' nagios
su -s /bin/bash -c 'ulimit -a' mysql
nslookup nagios-ia.mXXXXXXXrp.com
cat /etc/hosts

Attach this file:

Code: Select all

/etc/php.ini

I don't see anything standing out from your profile other than a single SNMP check consuming 26% CPU:

Code: Select all

top - 09:03:09 up 105 days, 20:01,  0 users,  load average: 0.17, 0.16, 0.20
Tasks: 201 total,   2 running, 199 sleeping,   0 stopped,   0 zombie
%Cpu(s):  5.3 us,  2.1 sy,  0.0 ni, 92.6 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  8010244 total,   498900 free,  2064976 used,  5446368 buff/cache
KiB Swap:  4095996 total,  4038980 free,    57016 used.  5151184 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
10236 nagios    20   0  147820  10096   2144 R  26.7  0.1   0:00.04 check_snmp+

Nagios Support Forum

Nagios xi Performance Issues

Nagios xi Performance Issues

Re: Nagios xi Performance Issues

Re: Nagios xi Performance Issues

Re: Nagios xi Performance Issues

Re: Nagios xi Performance Issues