Nagios xi no more scheduled downtimes after upgrade to 5.7.4

edwardvanhaute · Post by **edwardvanhaute** » Mon Oct 26, 2020 11:54 am

Hello,

I've tried to upgrade my Nagios xi (on a cloned vm with active checks and notifications disabled) from 5.6.12 to 5.7.4.
Upgrade is successful, but afterwards all scheduled downtimes are gone in Nagios xi interface.
When browsing to the nagios core, scheduled downtimes are ok.

Seems to be a problem with backend database, but I can't pinpoint the issue.

Obviously this upgrade cannot be performed on our prod instance before this issue is resolved.

I've tried running the repair_databases.sh script, but this hasn't resolved the issue.

Please advise,
Edward

Post by **tgriep** » Tue Oct 27, 2020 9:41 am

Please run the following as root and upload the /tmp/nagios.sql file to the post so we can check to see if the MYSQL tables are updated.

Code: Select all

mysqldump --no-data nagios -u root -pnagiosxi >/tmp/nagios.sql

Also, get the following files and upload them to the post.

Code: Select all

/usr/local/nagios/var/nagios.log
/usr/local/nagios/etc/nagios.cfg

Thanks.

edwardvanhaute · Post by **edwardvanhaute** » Fri Oct 30, 2020 4:05 am

Hi

nagios.sql and nagios.cfg are added as attachment. I will pm nagios.log

Post by **tgriep** » Fri Oct 30, 2020 9:12 am

So far what you sent in looks good so I will need to get the nagios.log file to see if there are any errors.
Before getting the log file, run this to stop the processes, repair the database and start up the processes.

Code: Select all

service npcd stop
service nagios stop
service crond stop
pkill -9 -u nagios
echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -u root -pnagiosxi nagiosxi
mysqlcheck -f -r -u root -pnagiosxi --all-databases --use-frm
service mysqld restart
rm -f /usr/local/nagios/var/rw/nagios.cmd
rm -f /usr/local/nagios/var/nagios.lock
rm -f /var/run/nagios.lock
rm -f /usr/local/nagios/var/ndo.sock
rm -f /usr/local/nagios/var/ndo2db.lock
rm -f /var/lib/mrtg/mrtg_l
rm -f /usr/local/nagiosxi/var/*.lock
pkill python
service httpd restart
service nagios start
service npcd start
service crond start

Let the system run for 5 to 10 minutes and if the downtime entries are not displayed, them PM me the nagios.log file.

edwardvanhaute · Post by **edwardvanhaute** » Fri Oct 30, 2020 10:05 am

Did the mysql repair and restarted.

Still no scheduled downtimes in the "scheduled downtime" section of xi.

I do however see that the required objects are in fact in scheduled downtime, it's just not visible in the scheduled downtime gui.

I sent you the logfile.

Post by **tgriep** » Fri Oct 30, 2020 11:53 am

Thanks for the nagios.log file. It is shared with the other techs if they need it.

I see some errors when the NDO3 broker is trying to update the commenthistory table and that is stopping the nagios process.

I would like to truncate the data in that table to see if new data can be written to it and solve the issue and while we are at it, truncate the downtime table.
To do that, run the following procedure.

Run these commands to stop the processes, clean and repair the SQL database and to restart the processes. Run them all as root.

Code: Select all

service npcd stop
service nagios stop
service ndo2db stop
service crond stop
pkill -9 -u nagios
echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -u root -pnagiosxi nagiosxi
echo "truncate table nagios_commenthistory; truncate table nagios_downtimehistory;" | mysql -u root -pnagiosxi nagios
mysqlcheck -f -r -u root -pnagiosxi --all-databases --use-frm
if grep --quiet pgsql /usr/local/nagiosxi/html/config.inc.php; then service postgresql stop; fi;
service mysqld restart
rm -f /usr/local/nagios/var/rw/nagios.cmd
rm -f /usr/local/nagios/var/nagios.lock
rm -f /var/run/nagios.lock
rm -f /usr/local/nagios/var/ndo.sock
rm -f /usr/local/nagios/var/ndo2db.lock
rm -f /var/lib/mrtg/mrtg_l
rm -f /usr/local/nagiosxi/var/*.lock
for i in `ipcs -q | grep nagios |awk '{print $2}'`; do ipcrm -q $i; done
pkill python
if grep --quiet pgsql /usr/local/nagiosxi/html/config.inc.php; then service postgresql start; fi;
service httpd restart
service ndo2db start
service nagios start
service npcd start
service crond start

Schedule downtime for a host or service and see if that entry shows up in the GUI.
Plus make sure the filter in the Scheduled Downtime menu is set to show all downtime entries.

edwardvanhaute · Post by **edwardvanhaute** » Tue Nov 03, 2020 4:36 am

Hello,

I've performed the truncates and other commands. After this the scheduled downtime is working as expected. New scheduled downtime as well as previously existing scheduled downtimes are displayed correctly.
So that seems to have solved the problem.

I'd like to test the full upgrade process again, including the steps to fix downtimes, before executing this in prod.

Do you think it will be sufficient to just upgrade and afterwards perform the commands in your last comment?

Let me know
Edward

Post by **tgriep** » Tue Nov 03, 2020 2:15 pm

Yes, if after testing the upgrade again and you see issue, the process should work but if there are any other issues, it is hard to say for sure.

Nagios Support Forum

Nagios xi no more scheduled downtimes after upgrade to 5.7.4

Nagios xi no more scheduled downtimes after upgrade to 5.7.4

Re: Nagios xi no more scheduled downtimes after upgrade to 5

Re: Nagios xi no more scheduled downtimes after upgrade to 5

Re: Nagios xi no more scheduled downtimes after upgrade to 5

Re: Nagios xi no more scheduled downtimes after upgrade to 5

Re: Nagios xi no more scheduled downtimes after upgrade to 5

Re: Nagios xi no more scheduled downtimes after upgrade to 5

Re: Nagios xi no more scheduled downtimes after upgrade to 5