Nagios Support Forum

Good evening all,

One of the topics of my company new to nagios was can we properly stagger the checkins of the clients so that we have minimal overlap for when hitting the server since we are concerned about the load being taken on our infrastructure when reporting back.

First question: Can the reporting to server interval be massively changed in the currently xi gui IE selecting a hostgroup and editing the config to do a refresh every 80 seconds instead of current setting or does it have to be done at the host level.

Second question: If the page itself for hosts is refreshed on an interval based within the server, we found it odd that we would have a handful of various clients show offline at the same time of the page refresh but the clients were contactable via remote program, how may we investigate if this is related to, too many clients checking in at once and overloading the infrastructure versus a possible configuration issue?

I can provide our currently profile once requested via pm.

Question 1: AFAIK there is no way to signal to the passive check sender (i.e. the non-xi server being monitored) from within xi that it should send its checks on a different interval.

Passive checks are different from active checks in that they are initiated by the server being monitored, and not by Nagios xi.

For instance, if you're using NCPA to send the checks, the configuration would look like this, which would send passive check data every 5 minutes, or 300 seconds: Note: The "hostname" above is arbitrary, but it should match up to where xi needs to go for active checks.

Note 2: When changing ncpa.cfg, it may be necessary to restart the NCPA service afterward to make sure applies the correct settings.

--------------------------------------

Question 2: This makes me think your setup is using both active and passive checks to monitor hosts; an active "host" check to ping the host and passive checks to monitor the services. This is okay - it's supported.

What you'll want to do is run the check "host" check manually to attempt to diagnose issues reaching the host. Do this by going into the CCM by clicking configure (top menu) => Core Config Manager, then clicking the "[NUM] Hosts" to go to the host list, then opening up the host showing up as offline.

At this point, if the "Check Command" dropdown is blank, the host is assumed up, and we'll have to try something else.

Otherwise, click the Run Check Command button.

One caveat is that the "hostname" key in ncpa.cfg is arbitrary; it's the hostname xi assigns passive checks as coming from. If xi has passive checks from a host configured with hostname=host1.example.com but the real address of the host is foo.example.com, it'll show up in xi as coming from host1.example.com. Likewise, if you have an active check to ping that host, it'll end up pinging host1.example.com which will result in a "host down, but service up" problem like you're seeing. So it's good practice to always make sure the hostname in ncpa.cfg matches the FQDN of the host.

--------------------------------------

If you're still having issues, please PM me your system profile.

I have PM'ed you the profile. The main issue we are trying to track down is that some of our clients will randomly report as being down when in fact they are still alive and no network drop has occurred. We have other logs to confirm the network status at the time onsite. We are trying to troubleshoot why this would occur in a passive configuration.

System profile received.

The reason your hosts are showing as "down" but are actually still reachable is because the host check for a lot of them is the special check_dummy command, with the first argument 2 meaning it'll always return status "CRITICAL" for the host without doing any actual check.

I'd suggest either clearing that check, or for hosts where the "hostname" field is an actual IP address, changing it to a ping check such as "check_icmp -H $HOSTADDRESS$ -w 3000.0,80% -c 5000.0,100% -p 5" to make the up/down status more meaningful.

This won't cause any issues with your current configuration, but I'm seeing a number of hosts in your config where the "address" isn't a real network address, and that can cause problems when adding active host checks. Like I said before, the "hostname" in xi is arbitrary, but it should be a real network address in both xi and in ncpa.cfg (on the remote host) for active checks in order to avoid complications. But you only have one host with an active check, so it should work as-is.

We were advised by the nagios support team through many 1 on 1 sessions that this was a valid work around solution for passive hosts to get the down status message. I can do some testing today to remove the check dummy commands and see if host status itself is working but how would I push the custom message to nagios on a remote site without this command? Note: A vast majority of our configuration so far is passive and will continue to be passive checks. We are currently using the api to formulate a lot of checks through the ncpa agent rather than using traditional statement like below since we could not make contact with most of the clients doing so.

Please advise.

Hi mrjsokol,

I noticed in the system, the last line of the database log are as follows. I would reccomend running the repair script just to make sure everything is ok there. Regarding the use of check_dummy, this often used with passive checks to put the host or service in a non-ok state when a passive check result has not been received for some time period (otherwise the host or service will remain in the last state-reported). To use this you'll want to enable freshness check and set the appropriate threshold, see:

Service and Host Freshness Checks

Regarding the concern about too many check results overloading the server, at a current check load, hosts + services, of 1169. I don't see this as an issue, especially with the use of passive checks.

Ran DB repair and received the following results:

recovering (with sort) MyISAM-table 'nagios_acknowledgements'
Data records: 34
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_commands'
Data records: 141
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_commenthistory'
Data records: 10819
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_comments'
Data records: 17
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_configfiles'
Data records: 1
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_configfilevariables'
Data records: 131
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_conninfo'
Data records: 713
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_contact_addresses'
Data records: 1
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contactgroup_members'
Data records: 15
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contactgroups'
Data records: 14
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contact_notificationcommands'
Data records: 120
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contactnotificationmethods'
Data records: 53550
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_contactnotifications'
Data records: 53550
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4

---------

- recovering (with sort) MyISAM-table 'nagios_contacts'
Data records: 10
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contactstatus'
Data records: 10
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_customvariables'
Data records: 1169
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_customvariablestatus'
Data records: 1169
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with keycache) MyISAM-table 'nagios_dbversion'
Data records: 1

---------

- recovering (with sort) MyISAM-table 'nagios_downtimehistory'
Data records: 63
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_eventhandlers'
Data records: 6
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_externalcommands'
Data records: 41
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_flappinghistory'
Data records: 20696
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_hostchecks'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_host_contactgroups'
Data records: 92
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_host_contacts'
Data records: 116
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostdependencies'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostescalation_contactgroups'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostescalation_contacts'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostescalations'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostgroup_members'
Data records: 315
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostgroups'
Data records: 64
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_host_parenthosts'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hosts'
Data records: 118
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_hoststatus'
Data records: 118
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4
- Fixing index 5
- Fixing index 6
- Fixing index 7
- Fixing index 8
- Fixing index 9
- Fixing index 10
- Fixing index 11
- Fixing index 12
- Fixing index 13
- Fixing index 14
- Fixing index 15
- Fixing index 16
- Fixing index 17
- Fixing index 18
- Fixing index 19

---------

- recovering (with sort) MyISAM-table 'nagios_instances'
Data records: 1
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_logentries'
Data records: 1568793
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4

---------

- recovering (with sort) MyISAM-table 'nagios_notifications'
Data records: 6575585
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4

---------

- recovering (with sort) MyISAM-table 'nagios_objects'
Data records: 2526
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4
- Fixing index 5

---------

- recovering (with sort) MyISAM-table 'nagios_processevents'
Data records: 10828
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_programstatus'
Data records: 1
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_runtimevariables'
Data records: 18
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_scheduleddowntime'
Data records: 8
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_servicechecks'
Data records: 0
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4

---------

- recovering (with sort) MyISAM-table 'nagios_service_contactgroups'
Data records: 164
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_service_contacts'
Data records: 892
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_servicedependencies'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_serviceescalation_contactgroups'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_serviceescalation_contacts'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_serviceescalations'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_servicegroup_members'
Data records: 392
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_servicegroups'
Data records: 20
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_service_parentservices'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_services'
Data records: 1054
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_servicestatus'
Data records: 1054
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4
- Fixing index 5
- Fixing index 6
- Fixing index 7
- Fixing index 8
- Fixing index 9
- Fixing index 10
- Fixing index 11
- Fixing index 12
- Fixing index 13
- Fixing index 14
- Fixing index 15
- Fixing index 16
- Fixing index 17
- Fixing index 18
- Fixing index 19

---------

- recovering (with sort) MyISAM-table 'nagios_statehistory'
Data records: 924010
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_systemcommands'
Data records: 104
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_timedeventqueue'
Data records: 0
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4
- Fixing index 5
- Fixing index 6

---------

- recovering (with sort) MyISAM-table 'nagios_timedevents'
Data records: 0
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4
- Fixing index 5
- Fixing index 6

---------

- recovering (with sort) MyISAM-table 'nagios_timeperiods'
Data records: 17
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_timeperiod_timeranges'
Data records: 96
- Fixing index 1
- Fixing index 2


I will monitor the solution to see if we get any more ghost down notifications and report back tomorrow. We already have freshness set on our hosts, does the freshness of the services themselves have to be set as well or will this property be inherited by the host? Also is it necessary to to have this set since currently we are not having any issues with the freshness of the services, only the host status currently. The odd thing about the host status is visually in the nagios xi UI it will report the host as down including in the widget but there are no records in alerts or in emails of this occurring. It wouldn't be an issue but we will be having a NOC type display for our hosts so they can be visually monitored as well by our support staff.


Thank you for your help,

Joseph

Good morning all,

after the repair was ran, we are still experiencing hosts that report down in the UI but dont indicate this in alert history. The down notifications are false positive since we looked into the logs of the actual machine and dont see any network disruption. Is it possible to see in the logs provided if there is any latency issue with the communication since our hosts are in the US and our nagios server remains in the UK.

Hi @mrjsoko,'

after the repair was ran, we are still experiencing hosts that report down in the UI but dont indicate this in alert history.

Are those hosts configurated as passive checks, if so, did you set this up using NCPA? Let me know the exact name of those hosts and I will track them down in the configuration files and we'll see how they are setup.

Thanks, Benjamin

Reference
Using NCPA For Passive Check

Yes they are passive configurations and they are setup using ncpa agents. I will pm you the names of the clients.