Automating the creation of new alarms on Nagios xi

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
cmif
Posts: 12
Joined: Mon Sep 12, 2016 2:28 am

Automating the creation of new alarms on Nagios xi

Post by cmif »

So Basically this is our scenario:

We have multiple network devices (mix of Juniper and Cisco) that are sending syslog messages of to our Nagios Log Server. I have configured both inputs and filters to parse the logs properly into separate fields. I have also configured NLS to analyse Nagios xi logs and have an output configured to create Hosts and Services on the fly if any unconfigured objects are found. So far so good.

Now I would like to automatically analyse the Cisco and Juniper Logs and if it finds messages with severity of Critical and above, it will send passive alerts to Nagios xi. The Host to be used in Nagios, will be parsed from the same syslog message and if it doesn't exist, it will go to the unconfigured objects which will then be catered for as described above.

Before proceeding with the configuration, I would like to ask if this is the right way of configuring such a scenario? I believe that in order to do the above, I would need to use output scripts to analyse each message and send passive checks to Nagios xi and not using the queries/alerts pages, right?
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: Automating the creation of new alarms on Nagios xi

Post by scottwilkerson »

cmif wrote:Before proceeding with the configuration, I would like to ask if this is the right way of configuring such a scenario? I believe that in order to do the above, I would need to use output scripts to analyse each message and send passive checks to Nagios xi and not using the queries/alerts pages, right?
I believe you are correct here because the builtin Alerts functionality just get a count of the query, and does NOT pass the rows of the query to be analyzed per host.

The easiest thing I can think is to add a logstash output that looks something like this (just an example):
https://www.elastic.co/guide/en/logstas ... _nsca.html

Code: Select all

if [severity] in ["emergency", "alert", "critical"] {
    nagios_nsca {
        nagios_status => 2
        host => "nagios.example.com"
        port => 5667
        nagios_host => "%{host}"
        nagios_service => "Critical Logs"
        message_format => "%{@timestamp} %{host}: %{severity} %{message}"
    }
}
you would need to also install send_nsca on the log server


Don't forget to add your log server IP to this file on the xi server and restart xinetd
/etc/xinetd.d/nsca

As an alternate you could use this output plugin
https://www.elastic.co/guide/en/logstas ... -exec.html

with our send_nrpd client
https://support.nagios.com/kb/article/n ... t-599.html
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
cmif
Posts: 12
Joined: Mon Sep 12, 2016 2:28 am

Re: Automating the creation of new alarms on Nagios xi

Post by cmif »

Thanks.

My concern about using the passive checks to Nagios is that whenever it runs the script to adds a new host or service, it would need to restart the Nagios service. What worries me the most is that if there are several issues on the network and lots of new syslog messages are received at the same time (maybe from the same host but different message and hence difference service), it would need to create a new object in Nagios and therefore restart the Nagios service for each of these new messages.

This might impact live monitoring on Nagios xi, right? Has anyone had any issues with this?
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: Automating the creation of new alarms on Nagios xi

Post by scottwilkerson »

Another option would be to send them all to a single check, and mark the check as volatile. This will send an alert for every check received, you will just want to make sure you send the host in the message of the check so you know which hosts are affected.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart