Ghost unhandled problems reported

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
mrjsokol
Posts: 109
Joined: Thu Apr 16, 2020 10:27 am

Ghost unhandled problems reported

Post by mrjsokol »

Good evening,

We have reported in our widget that there are many unhandled problems, see screenshot but when going to the page non-exist.
You do not have the required permissions to view the files attached to this post.
User avatar
vtrac
Posts: 903
Joined: Tue Oct 27, 2020 1:35 pm

Re: Ghost unhandled problems reported

Post by vtrac »

Hi,
Since you opened two posts/tickets at the same time.
Is this the same (similar) issue you reported in the following post:
https://support.nagios.com/forum/viewto ... 23#p327523

Can I get more info here?

Based on the picture provide, Nagios XI dashboard show NONE hosts or services defined on the page.

From the other ticket (above), you stated that you have started to use XI again.

Were there hosts and services defined before you shut them down?

Were this XI been down a long times?

What version of your Nagios XI? ... OS?

Please upload the "profile.zip" to this post.


Regards,
Vinh
mrjsokol
Posts: 109
Joined: Thu Apr 16, 2020 10:27 am

Re: Ghost unhandled problems reported

Post by mrjsokol »

Good morning,

Approx a month and half we were shutdown due to a nagios security issue. We are running the latest version of nagios XI. The issues are separate which is the reason for two forum posts. One issue is with ghost clients giving us email messages and we believe its probably cached emailed on the nagios side we just to know how to clear out. This issue is related to widgets.

We do have an update which is odd.

We noticed if we go to the page status for the type of information we are looking at in the widget, the information will appear but if we are on the opposite type of information, we do not see any information on the page. To me it sounds like a php error related to access to the data types.

To get an example: If we are trying to see the unhandled problems of the services, we must first be on the services status page and then we can click on the widget to do the filtering. If we were on the host status page, no information would populate.

I will PM you a profile.
User avatar
vtrac
Posts: 903
Joined: Tue Oct 27, 2020 1:35 pm

Re: Ghost unhandled problems reported

Post by vtrac »

Hi,
Let first try to repair your database, please run the below command:

Code: Select all

/usr/local/nagiosxi/scripts/repair_databases.sh
Now restart httpd:

Code: Select all

systemctl restart httpd
Now, please run this command and see if there is any error in your Nagios config.
If issues found, please use CCM to fix them. Do not edit the file directly as it will be over-written by the DB.

Code: Select all

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Also, please run this and see if it fixes your ghost emails:

Code: Select all

echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -pnagiosxi nagiosxi

Regards,
Vinh
mrjsokol
Posts: 109
Joined: Thu Apr 16, 2020 10:27 am

Re: Ghost unhandled problems reported

Post by mrjsokol »

Good evening,

I have ran the first two commands. I had found the ghost alerts were as a result of referencing in the bpi config that were not removed when the host was removed from the main config for nagios xi. Is this a common issue? if so, is there a tool or method to clean bpi config entries for old entries?
mrjsokol
Posts: 109
Joined: Thu Apr 16, 2020 10:27 am

Re: Ghost unhandled problems reported

Post by mrjsokol »

Good morning,

This other existed as well but not sure if I should create a new thread because the issue is related.

Our server will only show problems in the widget properly when on the correct "status" page first

IE if you want to see the unhandled problems for host, you must first be on the host status page to properly see all the results. Is this a new feature or an issue?
User avatar
vtrac
Posts: 903
Joined: Tue Oct 27, 2020 1:35 pm

Re: Ghost unhandled problems reported

Post by vtrac »

Hi,
Please upload screenshots, if you can since I am not sure I understand the whole issue here.

What is the outputs out this command:

Code: Select all

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Also, please upload "profile.zip".

If you're unable to generate the the profile through the web interface,
please try generating it from the command line by running these commands as root:

Code: Select all

rm -rf /usr/local/nagiosxi/var/components/profile*​​

/usr/local/nagiosxi/scripts/components/getprofile.sh SUPPORT

Regards,
Vinh
mrjsokol
Posts: 109
Joined: Thu Apr 16, 2020 10:27 am

Re: Ghost unhandled problems reported

Post by mrjsokol »

Good afternoon,

I have pm'ed you the screenshots and profile as they have confidential information in them and should not be seen on this forum post.

To explain the screen shots sent. The first G1 screenshot is the default landing page in which you can see on the right hand side the two widgets reporting various problems both host and services.

When you look at a host issue from the services page (G2) that is the result.

G3 is the result of the requested command ran in terminal.

The items I had sent via pm are shown in outbox but have not been sent yet? I dont know if this forum has a restriction on amount of pm messages.
User avatar
vtrac
Posts: 903
Joined: Tue Oct 27, 2020 1:35 pm

Re: Ghost unhandled problems reported

Post by vtrac »

Hi,
Based on your screenshot, I only see just the total of 2 hosts and 3 services.

Is that all you have monitoring now?

I looked at the objects.cache and it is alot more hosts/services.

Was your Nagios XI working when you shut it down months ago?

Is your database local?

Can you connect to your database? .... is it running?

I see lots of service on your nagios.log timing out:

Code: Select all

[1616526821] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSAPP2;NetBIOS;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526821] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSWEB1;NetBIOS;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526834] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - WEB17;RDP;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526843] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - WEB16;NetBIOS;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526848] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSAPP2;RDP;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526851] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - WEB16;RDP;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526852] SERVICE ALERT: UK Rackspace Server - SYSDB Active Cluster;Disk Usage on S:/;CRITICAL;HARD;5;CRITICAL: Used disk space was 96.50 % (Used: 651.11 GiB, Free: 23.73 GiB, Total: 674.84 GiB)
[1616526852] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSAPP1;RDP;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526855] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSAPP1;NetBIOS;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526855] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSWEB3;NetBIOS;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526861] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - WEB18;NetBIOS;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526867] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - WEB18;RDP;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526869] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSWEB1;SMTP;WARNING;xi_service_notification_handler;recv() failed
[1616526877] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSAD1;NetBIOS;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526888] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSAPP3;NetBIOS;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
[1616526888] SERVICE NOTIFICATION: lee.bennett;UK Rackspace Server - SYSWEB2;RDP;CRITICAL;xi_service_notification_handler;CRITICAL - Socket timeout
Also, looks like your npcd has max load reached and no graph is generated.


Regards,
Vinh
User avatar
vtrac
Posts: 903
Joined: Tue Oct 27, 2020 1:35 pm

Re: Ghost unhandled problems reported

Post by vtrac »

Hi,
Here's are the log location of Nagios:
https://assets.nagios.com/downloads/nag ... ptions.pdf

It might be handy when searching for errors.


Regards,
Vinh