Nagios xi - UI too slow after migration.

jmsanesteban.sgre · Post by **jmsanesteban.sgre** » Thu Jul 30, 2020 1:35 pm

Hi all!

I have a little problem with the Nagiosxi UI Performance.

We had to migrate our Nagiosxi environment from CMO to FMO and we've planed to do it in two parts. In the first one, we migrate the frontend to the FMO and we keep the database in CMO, and in the second one we migrate the database from CMO to FMO. We have done the first part. The migration process had some problems, because the database server hosts different db instances for other projects, so the root account is reserved to the database administrator. I had to modify the restore_xi.sh file and import the nagios.sql file manually.

After the process, evertything seemed to be OK but the Nagios xi interface was too slow.

The FMO server is an offline one. The version installed of Nagios in both servers was the same (5.6.7). We followed the process explained in: https://assets.nagios.com/downloads/nag ... ios-xi.pdf. I've tested a remote mysql connection to get all the logentries and it took 2.82 seconds, so I suppose that there is not a latency problem. The UI works more or less with the admin menu, or any other menu that doesn't show objects' information, CCM menu works, slowly, but it works. But when I try to load servers or hosts status, it take ages, and when I try to open a host or a service to force a check or to run a command, it gets unresponsiveness. I've used different browsers. I've check info using the API through the CLI, and the times are goods. I've restarted the server. I've tried to repair the database tables.

I've read some post in the forum and search on the Internet, but I couldn't find that helps me to solve the problem.

It is not a performance problem, the server has 128GB RAM, 32vCPUs... I'm a bit lost, because I suppose that it will be a delay or a bit lag, but it is impossible to work with that interface.

I will appreciate any help.Thanks in advance.

BR,
Juanma.

jmsanesteban.sgre · Post by **jmsanesteban.sgre** » Fri Jul 31, 2020 12:29 am

Adding info:

It took 2:40 to show the service status of 223 services, most of the time is used with: draw_servicestatustable_table. It is obvious but Core is working well (file method) nevertheless the display_dashlet for hosts and services status summary takes less than 10 seconds both.

Some errors in /var/log/httpd/ssl_error

Code: Select all

[Thu Jul 30 11:35:58.184264 2020] [:error] [pid 9153] [client x.x.x.x:61528] PHP Notice:  Undefined index: min_latency in /usr/local/nagiosxi/html/includes/utils-xmlsysstat.inc.php on line 119, referer: https://y.y.y.y/nagiosxi/index.php?
[Thu Jul 30 11:35:58.184309 2020] [:error] [pid 9153] [client x.x.x.x:61528] PHP Notice:  Undefined index: max_latency in /usr/local/nagiosxi/html/includes/utils-xmlsysstat.inc.php on line 120, referer: https://y.y.y.y/nagiosxi/index.php?
[Thu Jul 30 11:35:58.184318 2020] [:error] [pid 9153] [client x.x.x.x:61528] PHP Notice:  Undefined index: avg_latency in /usr/local/nagiosxi/html/includes/utils-xmlsysstat.inc.php on line 121, referer: https://y.y.y.y/nagiosxi/index.php?
[Thu Jul 30 11:35:58.184341 2020] [:error] [pid 9153] [client x.x.x.x:61528] PHP Notice:  Undefined index: min_execution_time in /usr/local/nagiosxi/html/includes/utils-xmlsysstat.inc.php on line 122, referer: https://y.y.y.y/nagiosxi/index.php?
[Thu Jul 30 11:35:58.184347 2020] [:error] [pid 9153] [client x.x.x.x:61528] PHP Notice:  Undefined index: max_execution_time in /usr/local/nagiosxi/html/includes/utils-xmlsysstat.inc.php on line 123, referer: https://y.y.y.y/nagiosxi/index.php?
[Thu Jul 30 11:35:58.184352 2020] [:error] [pid 9153] [client x.x.x.x:61528] PHP Notice:  Undefined index: avg_execution_time in /usr/local/nagiosxi/html/includes/utils-xmlsysstat.inc.php on line 124, referer: https://y.y.y.y/nagiosxi/index.php?

But I didn't find more errors in /var/log/httpd, /var/log/mariadb/, /usr/local/nagios/var/ or /usr/local/nagiosxi/var/

Thanks in advance.

BR,
Juanma.

jmsanesteban.sgre · Post by **jmsanesteban.sgre** » Fri Jul 31, 2020 4:27 am

Hi all.

The main problem is how Nagiosxi update the service and host status view, it generates a set of objects and updates them using Ajax, please, correct me if I'm wrong. The problem is that our default value for paging is set to 250 objects, so Nagiosxi has to do 250 queries to get the information of all the objects.

There is not solution for that, but we can set a smaller value for the objects in a page, so it will take less time to complete the Ajax queries.

BR,
Juanma.

Post by **mbellerue** » Fri Jul 31, 2020 11:36 am

Excellent work tracking that down! Does changing the page view count to something lower work as a work-around for this issue? What about creating dashboards using something like Minemap to view specific sections of your environment? If you got to Home -> Maps -> Minemap, you will be able to zero in on specific host and service groups, basically restricting the number of hosts you're looking at, while still being able to view the hosts in a timely fashion.

Let's also get some data on your system. If you could run this command on the server that is hosting the database, and send in the resulting .txt file, that would help.

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql', 'nagiosxi');" | mysql -uroot -pnagiosxi --table > /tmp/dbtablesizes.txt

jmsanesteban.sgre · Post by **jmsanesteban.sgre** » Mon Aug 24, 2020 4:57 am

Yes, this was exactly what we do to workaround the problem. If you reduce the amount of objects that you load in the status page you will reduce the queries. Once we migrated the database to the same CPD the performance became quite better.

In our case, working with 25 hosts limit and playing with the queries and using the API was enough to do the normal work till the migration, but you need some skills to do that, so a normal user could have problems.

The way you explain using Minemap could also work, and there are other alternatives, you can use the API or the jsonquery web, query directly the database or even use Nagios Core (remember the problem is when Nagios xi query its database), to be honest, the administrators of the application won't have problems, but the operators or the normal users will have.

In any case thanks you very much for your comment.

BR,
Juanma.

benjaminsmith · Post by **benjaminsmith** » Mon Aug 24, 2020 5:20 pm

HI Juanma,

It sounds like you got it figured out. Are you still having issues with a slow UI? Let us know if you would like to keep this thread open.

Benjamin

jmsanesteban.sgre · Post by **jmsanesteban.sgre** » Thu Sep 10, 2020 6:21 am

Hi all!

Sorry for the delay, the UI now is working properly.

BR,
Juanma.

scottwilkerson · Post by **scottwilkerson** » Thu Sep 10, 2020 6:46 am

jmsanesteban.sgre wrote:Hi all!

Sorry for the delay, the UI now is working properly.

BR,
Juanma.

Great!

Locking thread

Nagios Support Forum

Nagios xi - UI too slow after migration.

Nagios xi - UI too slow after migration.

Re: Nagios xi - UI too slow after migration.

Re: Nagios xi - UI too slow after migration.

Re: Nagios xi - UI too slow after migration.

Re: Nagios xi - UI too slow after migration.

Re: Nagios xi - UI too slow after migration.

Re: Nagios xi - UI too slow after migration.

Re: Nagios xi - UI too slow after migration.