Performance graphs stopped updating

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
FLCUISIT
Posts: 93
Joined: Mon Feb 01, 2010 12:09 pm

Performance graphs stopped updating

Post by FLCUISIT »

We have a situation where our performance graphs stopped updating over the weekend. It is happening to all of our graphs. We are on the current release of the XI R1.4B and would need help figuring out a solution.

Attached is a sample graph
performance.png
Thanks,

Kirk
You do not have the required permissions to view the files attached to this post.
tonyyarusso
Posts: 1128
Joined: Wed Mar 03, 2010 12:38 pm
Location: St. Paul, MN, USA

Re: Performance graphs stopped updating

Post by tonyyarusso »

The first thing I'd try is going under Admin and select the "Action" button for "Performance grapher", and restart that service.
Tony Yarusso
Technical Services
___
TIES
Web: http://ties.k12.mn.us/
FLCUISIT
Posts: 93
Joined: Mon Feb 01, 2010 12:09 pm

Re: Performance graphs stopped updating

Post by FLCUISIT »

That did not change anything. We are running XI under the VMWare image on an ESX box. Most of the performance graphs seem to stop on around the 4th, but we have some that were working until about yesterday afternoon.
mguthrie
Posts: 4380
Joined: Mon Jun 14, 2010 10:21 am

Re: Performance graphs stopped updating

Post by mguthrie »

Try the Admin->Reset Security Credentials and reset the credentials for the backend components.
FLCUISIT
Posts: 93
Joined: Mon Feb 01, 2010 12:09 pm

Re: Performance graphs stopped updating

Post by FLCUISIT »

That may have done something. It looks like though not current, that some of the graphs are building. Is there a process that I can see where it is reading from the database and writing to the performance graphs?
FLCUISIT
Posts: 93
Joined: Mon Feb 01, 2010 12:09 pm

Re: Performance graphs stopped updating

Post by FLCUISIT »

The performance graphs still are not up to date, while very slowly updating. System status shows load in the 15-20 percent based on the 1-min, 5-min, and 15-min, while the memory is consistently near 2GB of usage (with no swap usage of the 1GB assigned). Any thoughts? Is there some performance tweaking that can be done?
tonyyarusso
Posts: 1128
Joined: Wed Mar 03, 2010 12:38 pm
Location: St. Paul, MN, USA

Re: Performance graphs stopped updating

Post by tonyyarusso »

System status shows load in the 15-20 percent based on the 1-min, 5-min, and 15-min
I believe this is your problem. Load is not given as a percentage, but rather a value relative to the number of CPU cores available. On most systems, values of 15-20 would be rather concerning. I would suggest shutting down the VM temporarily and assigning more CPU cores to it in the VMware settings (this solved a similar issue for another customer in December).
Tony Yarusso
Technical Services
___
TIES
Web: http://ties.k12.mn.us/
FLCUISIT
Posts: 93
Joined: Mon Feb 01, 2010 12:09 pm

Re: Performance graphs stopped updating

Post by FLCUISIT »

Adding the CPU's helped for a while, but the load numbers have actually gone up since increasing the number of CPU's for the VM from 2 to 4, now sitting with values of 45-50 in the metrics, and is consuming around 8 GHz of processing on my ESX server. It still appears to stop producing occasionally, and based of other similar issues, restarting the following 3 processes (ndo2db, npcd, and nagios) helps restart the processing.

I can get you a sample of the running processes if that helps. One thing I notice is this process information occasionally:

nagios 1406 0.1 0.0 0 0 ? Z 09:46 0:00 [nagios] <defunct>

Any thoughts on what that means?
rdedon
Posts: 578
Joined: Sat Nov 20, 2010 4:51 pm

Re: Performance graphs stopped updating

Post by rdedon »

It may be service_perfdata_file_processing_command=process-service-perfdata-file-bulk in nagios.cfg


As actual name that is congigured - process-service-perfdata-file-pnp-bulk

Lets edit nagios.cfg -

Instead of:
service_perfdata_file_processing_command=process-service-perfdata-file-bulk
host_perfdata_file_processing_command=process-host-perfdata-file-bulk

and change to:

service_perfdata_file_processing_command=process-service-perfdata-file-pnp-bulk
host_perfdata_file_processing_command=process-host-perfdata-file-pnp-bulk
Rene deDon
Technical Team
___
Nagios Enterprises, LLC
Web: http://www.nagios.com
mguthrie
Posts: 4380
Joined: Mon Jun 14, 2010 10:21 am

Re: Performance graphs stopped updating

Post by mguthrie »

What kind of check load do you have on your system -> How many hosts and services are you monitoring on your server? Are you running mostly active or passive checks?

What kind of output do you get from running "top"?