check_snmp, performance graph and graph explorer discrepancy

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
User avatar
tgriep
Madmin
Posts: 9190
Joined: Thu Oct 30, 2014 9:02 am

Re: check_snmp, performance graph and graph explorer discrep

Post by tgriep »

Could you PM me your customer number and email address so I can check your account?
You could also PM me your profile.
Be sure to check out our Knowledgebase for helpful articles and solutions!
iptribe
Posts: 52
Joined: Sun Aug 31, 2014 11:03 am

Re: check_snmp, performance graph and graph explorer discrep

Post by iptribe »

tgriep wrote:Could you PM me your customer number and email address so I can check your account?
You could also PM me your profile.
Done.
User avatar
tgriep
Madmin
Posts: 9190
Joined: Thu Oct 30, 2014 9:02 am

Re: check_snmp, performance graph and graph explorer discrep

Post by tgriep »

It looks like the check_snmp_gauge.cfg template didn't get uploaded to your system. Follow these instructions and see if that works for you.

The graph template is called check_snmp_gauge.cfg and it needs to be uploaded to this folder in the XI system /usr/local/nagios/etc/pnp/check_commands/
Changed the permissions of the check_snmp_gauge.cfg file to the following
chown apache.nagios check_snmp_gauge.cfg
chmod 666 check_snmp_gauge.cfg

Delete the .rrd and .xml files for that service. They are in this folder under the hostname
/usr/local/nagios/share/perfdata/<HostName>/

After 15 minutes, the graph should be populated with the data.
You do not have the required permissions to view the files attached to this post.
Be sure to check out our Knowledgebase for helpful articles and solutions!
iptribe
Posts: 52
Joined: Sun Aug 31, 2014 11:03 am

Re: check_snmp, performance graph and graph explorer discrep

Post by iptribe »

tgriep wrote:It looks like the check_snmp_gauge.cfg template didn't get uploaded to your system. Follow these instructions and see if that works for you.

The graph template is called check_snmp_gauge.cfg and it needs to be uploaded to this folder in the XI system /usr/local/nagios/etc/pnp/check_commands/
Changed the permissions of the check_snmp_gauge.cfg file to the following
chown apache.nagios check_snmp_gauge.cfg
chmod 666 check_snmp_gauge.cfg

Delete the .rrd and .xml files for that service. They are in this folder under the hostname
/usr/local/nagios/share/perfdata/<HostName>/

After 15 minutes, the graph should be populated with the data.
I believe that is how I did it the last time. Anyway, I just tried it again today and got the same result.

Code: Select all

[root@NFVI-COMMON-NAGIOS-01 check_commands]# ls
check_nwstat.cfg-sample  check_snmp_gauge.cfg
[root@NFVI-COMMON-NAGIOS-01 check_commands]# rm check_snmp_gauge.cfg 
rm: remove regular file `check_snmp_gauge.cfg'? y
[root@NFVI-COMMON-NAGIOS-01 check_commands]# ls -l
total 4
-rw-rw-r-- 1 apache nagios 755 Aug 14  2014 check_nwstat.cfg-sample
[root@NFVI-COMMON-NAGIOS-01 check_commands]# cp /home/iptribe/check_snmp_gauge.cfg .
[root@NFVI-COMMON-NAGIOS-01 check_commands]# ls -l
total 8
-rw-rw-r-- 1 apache nagios 755 Aug 14  2014 check_nwstat.cfg-sample
-rw------- 1 root   nagios 752 Aug 21 11:17 check_snmp_gauge.cfg
[root@NFVI-COMMON-NAGIOS-01 check_commands]# chown apache.nagios check_snmp_gauge.cfg 
[root@NFVI-COMMON-NAGIOS-01 check_commands]# chmod 666 check_snmp_gauge.cfg 
[root@NFVI-COMMON-NAGIOS-01 check_commands]# ls -l
total 8
-rw-rw-r-- 1 apache nagios 755 Aug 14  2014 check_nwstat.cfg-sample
-rw-rw-rw- 1 apache nagios 752 Aug 21 11:17 check_snmp_gauge.cfg
[root@NFVI-COMMON-NAGIOS-01 check_commands]# cd /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/
[root@NFVI-COMMON-NAGIOS-01 RCS-PROD-SSC-01]# ls
Active_Sessions.rrd  CPU_usage_on_SBC_Nodes.rrd  _HOST_.rrd  Memory_Usage_on_SSC_Nodes.rrd  Number_of_Registered_Subscribers.rrd
Active_Sessions.xml  CPU_usage_on_SBC_Nodes.xml  _HOST_.xml  Memory_Usage_on_SSC_Nodes.xml  Number_of_Registered_Subscribers.xml
[root@NFVI-COMMON-NAGIOS-01 RCS-PROD-SSC-01]# rm Active_Sessions.*
rm: remove regular file `Active_Sessions.rrd'? y
rm: remove regular file `Active_Sessions.xml'? y
[root@NFVI-COMMON-NAGIOS-01 RCS-PROD-SSC-01]# ls -l
total 3024
-rw-rw-rw-  1 nagios nagios  384952 Aug 21 11:19 Active_Sessions.rrd
-rw-rw-rw-  1 nagios nagios    2539 Aug 21 11:19 Active_Sessions.xml
-rwxrwxr-x  1 nagios nagios  384952 Aug 21 11:19 CPU_usage_on_SBC_Nodes.rrd
-rw-rw-rw-  1 nagios nagios    2640 Aug 21 11:19 CPU_usage_on_SBC_Nodes.xml
-rwxrwxr-x. 1 nagios nagios 1534768 Sep 16  2014 _HOST_.rrd
-rwxrwxr-x  1 nagios nagios    4020 Sep 16  2014 _HOST_.xml
-rwxrwxr-x  1 nagios nagios  384952 Aug 21 11:17 Memory_Usage_on_SSC_Nodes.rrd
-rw-rw-rw-  1 nagios nagios    2606 Aug 21 11:17 Memory_Usage_on_SSC_Nodes.xml
-rwxrwxr-x  1 nagios nagios  384952 Aug 21 11:19 Number_of_Registered_Subscribers.rrd
-rw-rw-rw-  1 nagios nagios    2700 Aug 21 11:19 Number_of_Registered_Subscribers.xml
1.PNG
You do not have the required permissions to view the files attached to this post.
User avatar
lgroschen
Posts: 384
Joined: Wed Nov 27, 2013 1:17 pm

Re: check_snmp, performance graph and graph explorer discrep

Post by lgroschen »

iptribe,

Thank you for your patience as this issue will require more testing to try and resolve. We have another customer who did similar steps using tgriep's template file and it fixed the problem for that user, but we know this fix will need tweaks based on a specific mib/user/config. It is becoming clear that this is likely pnp4nagios handling the data in a certain way when a counter (c) UOM is detected and I have seen similar posts as this on their website.

Can you post the perfdata.log like you did before with the highest verbose logging set (3 i think). We need to see the lines where it accepts the command, finds the template and shows which datatype is being used.
/Luke
iptribe
Posts: 52
Joined: Sun Aug 31, 2014 11:03 am

Re: check_snmp, performance graph and graph explorer discrep

Post by iptribe »

lgroschen wrote:iptribe,

Thank you for your patience as this issue will require more testing to try and resolve. We have another customer who did similar steps using tgriep's template file and it fixed the problem for that user, but we know this fix will need tweaks based on a specific mib/user/config. It is becoming clear that this is likely pnp4nagios handling the data in a certain way when a counter (c) UOM is detected and I have seen similar posts as this on their website.

Can you post the perfdata.log like you did before with the highest verbose logging set (3 i think). We need to see the lines where it accepts the command, finds the template and shows which datatype is being used.
Hi,

Sorry for the delay. This is what I have found. Seems it is still not using the new template. And realized, it is not only the Active Session service having the issue but all the services for SSC001-003 have the "c" in their counters. Please help to check it further.
SSC-001 counters.png

Code: Select all

2015-08-31 09:50:54 [48492] [1] Found Performance Data for RCS-PROD-SSC-01 / Active_Sessions (PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minutes=1;;10000;) 
2015-08-31 09:50:54 [48496] [3]   -- servicedesc -> HTTP
2015-08-31 09:50:54 [48492] [3] DEBUG: Normal perfdata
2015-08-31 09:50:54 [48496] [3]   -- template -> check_http
2015-08-31 09:50:54 [48496] [3]   -- uom -> B
2015-08-31 09:50:54 [48492] [3] DEBUG: RAW Command -> check_xi_service_snmp!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
2015-08-31 09:50:54 [48496] [3]   -- value -> 3194
2015-08-31 09:50:54 [48492] [2] No Custom Template found for check_xi_service_snmp (/usr/local/nagios/etc/pnp/check_commands/check_xi_service_snmp.cfg) 
2015-08-31 09:50:54 [48496] [3] DEBUG: SINGLE Storage Type
2015-08-31 09:50:54 [48492] [3] RRD Datatype is GAUGE
2015-08-31 09:50:54 [48492] [2] Template is check_xi_service_snmp.php
2015-08-31 09:50:54 [48496] [3] DEBUG: TPL-> check_http
2015-08-31 09:50:54 [48496] [3] DEBUG: CRE-> DS:1:GAUGE:8460:U:U DS:2:GAUGE:8460:U:U
2015-08-31 09:50:54 [48492] [2] data2rrd called
2015-08-31 09:50:54 [48496] [3] DEBUG: UPD-> :0.001018:3194
2015-08-31 09:50:54 [48492] [3] DEBUG: XML File is 307 seconds old. UPDATE!
2015-08-31 09:50:54 [48496] [2] RRDs::update /usr/local/nagios/share/perfdata/localhost/HTTP.rrd 1440985844:0.001018:3194
2015-08-31 09:50:54 [48492] [3]  -- Job 0 
2015-08-31 09:50:54 [48492] [3]   -- auth_hostname -> RCS-PROD-SSC-01
2015-08-31 09:50:54 [48492] [3]   -- auth_servicedesc -> Active Sessions
2015-08-31 09:50:54 [48496] [2] /usr/local/nagios/share/perfdata/localhost/HTTP.rrd updated
2015-08-31 09:50:54 [48492] [3]   -- critical -> 10000
2015-08-31 09:50:54 [48492] [3]   -- disp_hostname -> RCS-PROD-SSC-01
2015-08-31 09:50:54 [48492] [3]   -- disp_servicedesc -> Active Sessions
2015-08-31 09:50:54 [48492] [3]   -- dstype -> GAUGE
2015-08-31 09:50:54 [48492] [3]   -- hostname -> RCS-PROD-SSC-01
2015-08-31 09:50:54 [48492] [3]   -- label -> PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-08-31 09:50:54 [48492] [3]   -- multi -> 0
2015-08-31 09:50:54 [48496] [1] 2 lines processed
2015-08-31 09:50:54 [48492] [3]   -- name -> PERIMETA-MIB__globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-08-31 09:50:54 [48492] [3]   -- rrd_heartbeat -> 8460
2015-08-31 09:50:54 [48492] [3]   -- rrd_max -> U
2015-08-31 09:50:54 [48496] [1] /usr/local/nagios/var/spool/perfdata//1440985849.perfdata.service-PID-48496 deleted
2015-08-31 09:50:54 [48492] [3]   -- rrd_min -> U
2015-08-31 09:50:54 [48492] [3]   -- rrd_storage_type -> SINGLE
2015-08-31 09:50:54 [48492] [3]   -- servicedesc -> Active_Sessions
2015-08-31 09:50:54 [48496] [1] PNP exiting (runtime 0.004252s) ...
2015-08-31 09:50:54 [48492] [3]   -- template -> check_xi_service_snmp
2015-08-31 09:50:54 [48492] [3]   -- uom -> 
2015-08-31 09:50:54 [48492] [3]   -- value -> 1
2015-08-31 09:50:54 [48492] [3] DEBUG: SINGLE Storage Type
2015-08-31 09:50:54 [48492] [3] DEBUG: TPL-> check_xi_service_snmp
2015-08-31 09:50:54 [48492] [3] DEBUG: CRE-> DS:1:GAUGE:8460:U:U
2015-08-31 09:50:54 [48492] [3] DEBUG: UPD-> :1
2015-08-31 09:50:54 [48492] [2] RRDs::update /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd 1440985828:1
2015-08-31 09:50:54 [48492] [2] /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd updated
2015-08-31 09:50:54 [48492] [1] 2 lines processed
2015-08-31 09:50:54 [48492] [1] /usr/local/nagios/var/spool/perfdata//1440985834.perfdata.service-PID-48492 deleted
2015-08-31 09:50:54 [48492] [1] PNP exiting (runtime 0.004987s) ...
2015-08-31 09:51:26 [48677] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2015-08-31 09:51:26 [48673] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2015-08-31 09:51:26 [48677] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2015-08-31 09:51:26 [48677] [2] searching for /usr/local/nagios/var/spool/perfdata//1440985879.perfdata.host
2015-08-31 09:51:26 [48677] [2] renaming /usr/local/nagios/var/spool/perfdata//1440985879.perfdata.host to /usr/local/nagios/var/spool/perfdata//1440985879.perfdata.host-PID-48677 for bulk update
2015-08-31 09:51:26 [48677] [2] reading /usr/local/nagios/var/spool/perfdata//1440985879.perfdata.host-PID-48677 for bulk update
2015-08-31 09:51:26 [48677] [1] 0 lines processed
2015-08-31 09:51:26 [48677] [1] /usr/local/nagios/var/spool/perfdata//1440985879.perfdata.host-PID-48677 deleted
2015-08-31 09:51:26 [48673] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2015-08-31 09:51:26 [48673] [2] searching for /usr/local/nagios/var/spool/perfdata//1440985864.perfdata.host
2015-08-31 09:51:26 [48673] [2] renaming /usr/local/nagios/var/spool/perfdata//1440985864.perfdata.host to /usr/local/nagios/var/spool/perfdata//1440985864.perfdata.host-PID-48673 for bulk update
2015-08-31 09:51:26 [48677] [3] DEBUG: Normal perfdata
2015-08-31 09:51:26 [48673] [2] reading /usr/local/nagios/var/spool/perfdata//1440985864.perfdata.host-PID-48673 for bulk update
2015-08-31 09:51:26 [48673] [2] Processing Line 1
2015-08-31 09:51:26 [48677] [3] DEBUG: RAW Command -> pnp-runtime
2015-08-31 09:51:26 [48677] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg) 
2015-08-31 09:51:26 [48673] [2] Datatype set to 'HOSTPERFDATA' 

Code: Select all

2015-08-31 09:52:17 [48897] [1] Found Performance Data for RCS-PROD-SSC-02 / Active_Sessions (PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minutes=1;;10000;) 
2015-08-31 09:52:17 [48897] [3] DEBUG: Normal perfdata
2015-08-31 09:52:17 [48896] [1] PNP exiting (runtime 0.001908s) ...
2015-08-31 09:52:17 [48897] [3] DEBUG: RAW Command -> check_xi_service_snmp!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
2015-08-31 09:52:17 [48897] [2] No Custom Template found for check_xi_service_snmp (/usr/local/nagios/etc/pnp/check_commands/check_xi_service_snmp.cfg) 
2015-08-31 09:52:17 [48897] [3] RRD Datatype is GAUGE
2015-08-31 09:52:17 [48897] [2] Template is check_xi_service_snmp.php
2015-08-31 09:52:17 [48897] [2] data2rrd called
2015-08-31 09:52:17 [48897] [3] DEBUG: XML File is 305 seconds old. UPDATE!
2015-08-31 09:52:17 [48897] [3]  -- Job 0 
2015-08-31 09:52:17 [48897] [3]   -- auth_hostname -> RCS-PROD-SSC-02
2015-08-31 09:52:17 [48897] [3]   -- auth_servicedesc -> Active Sessions
2015-08-31 09:52:17 [48897] [3]   -- critical -> 10000
2015-08-31 09:52:17 [48897] [3]   -- disp_hostname -> RCS-PROD-SSC-02
2015-08-31 09:52:17 [48897] [3]   -- disp_servicedesc -> Active Sessions
2015-08-31 09:52:17 [48897] [3]   -- dstype -> GAUGE
2015-08-31 09:52:17 [48897] [3]   -- hostname -> RCS-PROD-SSC-02
2015-08-31 09:52:17 [48897] [3]   -- label -> PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-08-31 09:52:17 [48897] [3]   -- multi -> 0
2015-08-31 09:52:17 [48897] [3]   -- name -> PERIMETA-MIB__globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-08-31 09:52:17 [48897] [3]   -- rrd_heartbeat -> 8460
2015-08-31 09:52:17 [48897] [3]   -- rrd_max -> U
2015-08-31 09:52:17 [48897] [3]   -- rrd_min -> U
2015-08-31 09:52:17 [48897] [3]   -- rrd_storage_type -> SINGLE
2015-08-31 09:52:17 [48897] [3]   -- servicedesc -> Active_Sessions
2015-08-31 09:52:17 [48897] [3]   -- template -> check_xi_service_snmp
2015-08-31 09:52:17 [48897] [3]   -- uom -> 
2015-08-31 09:52:17 [48897] [3]   -- value -> 1
2015-08-31 09:52:17 [48897] [3] DEBUG: SINGLE Storage Type
2015-08-31 09:52:17 [48897] [3] DEBUG: TPL-> check_xi_service_snmp
2015-08-31 09:52:17 [48897] [3] DEBUG: CRE-> DS:1:GAUGE:8460:U:U
2015-08-31 09:52:17 [48897] [3] DEBUG: UPD-> :1
2015-08-31 09:52:17 [48897] [2] RRDs::update /usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.rrd 1440985915:1
2015-08-31 09:52:17 [48897] [2] /usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.rrd updated

Code: Select all

2015-08-31 10:03:00 [51723] [1] Found Performance Data for RCS-PROD-SSC-03 / Active_Sessions (PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minutes=4;;10000;) 
2015-08-31 10:03:00 [51721] [2] searching for /usr/local/nagios/var/spool/perfdata//1440986569.perfdata.host
2015-08-31 10:03:00 [51723] [3] DEBUG: Normal perfdata
2015-08-31 10:03:00 [51721] [2] renaming /usr/local/nagios/var/spool/perfdata//1440986569.perfdata.host to /usr/local/nagios/var/spool/perfdata//1440986569.perfdata.host-PID-51721 for bulk update
2015-08-31 10:03:00 [51723] [3] DEBUG: RAW Command -> check_xi_service_snmp!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
2015-08-31 10:03:00 [51723] [2] No Custom Template found for check_xi_service_snmp (/usr/local/nagios/etc/pnp/check_commands/check_xi_service_snmp.cfg) 
2015-08-31 10:03:00 [51723] [3] RRD Datatype is GAUGE
2015-08-31 10:03:00 [51721] [2] reading /usr/local/nagios/var/spool/perfdata//1440986569.perfdata.host-PID-51721 for bulk update
2015-08-31 10:03:00 [51723] [2] Template is check_xi_service_snmp.php
2015-08-31 10:03:00 [51721] [2] Processing Line 1
2015-08-31 10:03:00 [51723] [2] data2rrd called
2015-08-31 10:03:00 [51723] [3] DEBUG: XML File is 305 seconds old. UPDATE!
2015-08-31 10:03:00 [51721] [2] Datatype set to 'HOSTPERFDATA' 
2015-08-31 10:03:00 [51723] [3]  -- Job 0 
2015-08-31 10:03:00 [51721] [1] Found Performance Data for RCS-PROD-METAVIEW-01 / _HOST_ (time=0.000800s;;;0.000000;10.000000) 
2015-08-31 10:03:00 [51723] [3]   -- auth_hostname -> RCS-PROD-SSC-03
2015-08-31 10:03:00 [51723] [3]   -- auth_servicedesc -> Active Sessions
2015-08-31 10:03:00 [51721] [3] DEBUG: Normal perfdata
2015-08-31 10:03:00 [51723] [3]   -- critical -> 10000
2015-08-31 10:03:00 [51723] [3]   -- disp_hostname -> RCS-PROD-SSC-03
2015-08-31 10:03:00 [51721] [3] DEBUG: RAW Command -> check_tcp!80!!!!!!!
2015-08-31 10:03:00 [51723] [3]   -- disp_servicedesc -> Active Sessions
2015-08-31 10:03:00 [51721] [2] No Custom Template found for check_tcp (/usr/local/nagios/etc/pnp/check_commands/check_tcp.cfg) 
2015-08-31 10:03:00 [51723] [3]   -- dstype -> GAUGE
2015-08-31 10:03:00 [51723] [3]   -- hostname -> RCS-PROD-SSC-03
2015-08-31 10:03:00 [51721] [3] RRD Datatype is GAUGE
2015-08-31 10:03:00 [51723] [3]   -- label -> PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-08-31 10:03:00 [51721] [2] Template is check_tcp.php
2015-08-31 10:03:00 [51723] [3]   -- multi -> 0
2015-08-31 10:03:00 [51723] [3]   -- name -> PERIMETA-MIB__globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-08-31 10:03:00 [51723] [3]   -- rrd_heartbeat -> 8460
2015-08-31 10:03:00 [51721] [2] data2rrd called
2015-08-31 10:03:00 [51723] [3]   -- rrd_max -> U
2015-08-31 10:03:00 [51723] [3]   -- rrd_min -> U
2015-08-31 10:03:00 [51721] [3] DEBUG: XML File is 305 seconds old. UPDATE!
2015-08-31 10:03:00 [51723] [3]   -- rrd_storage_type -> SINGLE
2015-08-31 10:03:00 [51723] [3]   -- servicedesc -> Active_Sessions
2015-08-31 10:03:00 [51723] [3]   -- template -> check_xi_service_snmp
2015-08-31 10:03:00 [51721] [3]  -- Job 0 
2015-08-31 10:03:00 [51723] [3]   -- uom -> 
2015-08-31 10:03:00 [51721] [3]   -- auth_hostname -> RCS-PROD-METAVIEW-01
2015-08-31 10:03:00 [51723] [3]   -- value -> 4
2015-08-31 10:03:00 [51723] [3] DEBUG: SINGLE Storage Type
2015-08-31 10:03:00 [51721] [3]   -- auth_servicedesc -> Host Perfdata
2015-08-31 10:03:00 [51721] [3]   -- disp_hostname -> RCS-PROD-METAVIEW-01
2015-08-31 10:03:00 [51723] [3] DEBUG: TPL-> check_xi_service_snmp
2015-08-31 10:03:00 [51721] [3]   -- disp_servicedesc -> Host Perfdata
2015-08-31 10:03:00 [51723] [3] DEBUG: CRE-> DS:1:GAUGE:8460:U:U
2015-08-31 10:03:00 [51721] [3]   -- dstype -> GAUGE
2015-08-31 10:03:00 [51723] [3] DEBUG: UPD-> :4
2015-08-31 10:03:00 [51721] [3]   -- hostname -> RCS-PROD-METAVIEW-01
2015-08-31 10:03:00 [51723] [2] RRDs::update /usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.rrd 1440986558:4
2015-08-31 10:03:00 [51721] [3]   -- label -> time
2015-08-31 10:03:00 [51721] [3]   -- max -> 10.000000
2015-08-31 10:03:00 [51721] [3]   -- min -> 0.000000
2015-08-31 10:03:00 [51721] [3]   -- multi -> 0
2015-08-31 10:03:00 [51723] [2] /usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.rrd updated
2015-08-31 10:03:00 [51721] [3]   -- name -> time
2015-08-31 10:03:00 [51721] [3]   -- rrd_heartbeat -> 8460
2015-08-31 10:03:00 [51721] [3]   -- rrd_max -> U
2015-08-31 10:03:00 [51721] [3]   -- rrd_min -> U
2015-08-31 10:03:00 [51721] [3]   -- rrd_storage_type -> SINGLE
2015-08-31 10:03:00 [51723] [2] Processing Line 2
2015-08-31 10:03:00 [51721] [3]   -- servicedesc -> _HOST_
2015-08-31 10:03:00 [51721] [3]   -- template -> check_tcp
2015-08-31 10:03:00 [51721] [3]   -- uom -> s
2015-08-31 10:03:00 [51723] [2] Datatype set to 'SERVICEPERFDATA' 
2015-08-31 10:03:00 [51721] [3]   -- value -> 0.000800
Attached is the full log.
Nagios-01_2015-08-31_09-47-17.zip
You do not have the required permissions to view the files attached to this post.
User avatar
tgriep
Madmin
Posts: 9190
Joined: Thu Oct 30, 2014 9:02 am

Re: check_snmp, performance graph and graph explorer discrep

Post by tgriep »

Could you run the following and post back the output?

Code: Select all

rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd
rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.rrd
rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.rrd
Could you post the cfg file for that service check from this folder /usr/local/nagios/etc/services
Also I need the commands.cfg and servicetemplates.cfg from this folder /usr/local/nagios/etc
Be sure to check out our Knowledgebase for helpful articles and solutions!
iptribe
Posts: 52
Joined: Sun Aug 31, 2014 11:03 am

Re: check_snmp, performance graph and graph explorer discrep

Post by iptribe »

tgriep wrote:Could you run the following and post back the output?

Code: Select all

rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd
rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.rrd
rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.rrd
Could you post the cfg file for that service check from this folder /usr/local/nagios/etc/services
Also I need the commands.cfg and servicetemplates.cfg from this folder /usr/local/nagios/etc
Please see below:

Code: Select all

[iptribe@NFVI-COMMON-NAGIOS-01 ~]$ rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd
filename = "/usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd"
rrd_version = "0003"
step = 60
last_update = 1441076696
ds[1].type = "GAUGE"
ds[1].minimal_heartbeat = 8460
ds[1].min = NaN
ds[1].max = NaN
ds[1].last_ds = "3"
ds[1].value = 1.6800000000e+02
ds[1].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 2880
rra[0].cur_row = 2057
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 2880
rra[1].cur_row = 1997
rra[1].pdp_per_row = 5
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 1.0506666667e+01
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 4320
rra[2].cur_row = 294
rra[2].pdp_per_row = 30
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 1.0506666667e+01
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 5840
rra[3].cur_row = 2737
rra[3].pdp_per_row = 360
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 8.3341333333e+02
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[4].cf = "MAX"
rra[4].rows = 2880
rra[4].cur_row = 1769
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[5].cf = "MAX"
rra[5].rows = 2880
rra[5].cur_row = 730
rra[5].pdp_per_row = 5
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = 2.6266666667e+00
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[6].cf = "MAX"
rra[6].rows = 4320
rra[6].cur_row = 1912
rra[6].pdp_per_row = 30
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = 2.6266666667e+00
rra[6].cdp_prep[0].unknown_datapoints = 0
rra[7].cf = "MAX"
rra[7].rows = 5840
rra[7].cur_row = 3462
rra[7].pdp_per_row = 360
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = 1.2253333333e+01
rra[7].cdp_prep[0].unknown_datapoints = 0
rra[8].cf = "MIN"
rra[8].rows = 2880
rra[8].cur_row = 816
rra[8].pdp_per_row = 1
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = NaN
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[9].cf = "MIN"
rra[9].rows = 2880
rra[9].cur_row = 2876
rra[9].pdp_per_row = 5
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = 2.6266666667e+00
rra[9].cdp_prep[0].unknown_datapoints = 0
rra[10].cf = "MIN"
rra[10].rows = 4320
rra[10].cur_row = 2041
rra[10].pdp_per_row = 30
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = 2.6266666667e+00
rra[10].cdp_prep[0].unknown_datapoints = 0
rra[11].cf = "MIN"
rra[11].rows = 5840
rra[11].cur_row = 4256
rra[11].pdp_per_row = 360
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = 1.8666666667e-01
rra[11].cdp_prep[0].unknown_datapoints = 0

Code: Select all

[iptribe@NFVI-COMMON-NAGIOS-01 ~]$ rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.rrd
filename = "/usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.rrd"
rrd_version = "0003"
step = 60
last_update = 1441076794
ds[1].type = "GAUGE"
ds[1].minimal_heartbeat = 8460
ds[1].min = NaN
ds[1].max = NaN
ds[1].last_ds = "4"
ds[1].value = 1.3600000000e+02
ds[1].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 2880
rra[0].cur_row = 568
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 2880
rra[1].cur_row = 1593
rra[1].pdp_per_row = 5
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 3.7733333333e+00
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 4320
rra[2].cur_row = 2728
rra[2].pdp_per_row = 30
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 2.0753333333e+01
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 5840
rra[3].cur_row = 4510
rra[3].pdp_per_row = 360
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 8.6435333333e+02
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[4].cf = "MAX"
rra[4].rows = 2880
rra[4].cur_row = 456
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[5].cf = "MAX"
rra[5].rows = 2880
rra[5].cur_row = 1158
rra[5].pdp_per_row = 5
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = 3.7733333333e+00
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[6].cf = "MAX"
rra[6].rows = 4320
rra[6].cur_row = 2196
rra[6].pdp_per_row = 30
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = 3.7733333333e+00
rra[6].cdp_prep[0].unknown_datapoints = 0
rra[7].cf = "MAX"
rra[7].rows = 5840
rra[7].cur_row = 1266
rra[7].pdp_per_row = 360
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = 1.2870000000e+01
rra[7].cdp_prep[0].unknown_datapoints = 0
rra[8].cf = "MIN"
rra[8].rows = 2880
rra[8].cur_row = 689
rra[8].pdp_per_row = 1
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = NaN
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[9].cf = "MIN"
rra[9].rows = 2880
rra[9].cur_row = 1130
rra[9].pdp_per_row = 5
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = 3.7733333333e+00
rra[9].cdp_prep[0].unknown_datapoints = 0
rra[10].cf = "MIN"
rra[10].rows = 4320
rra[10].cur_row = 2717
rra[10].pdp_per_row = 30
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = 1.8866666667e+00
rra[10].cdp_prep[0].unknown_datapoints = 0
rra[11].cf = "MIN"
rra[11].rows = 5840
rra[11].cur_row = 5439
rra[11].pdp_per_row = 360
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = 0.0000000000e+00
rra[11].cdp_prep[0].unknown_datapoints = 0

Code: Select all

[iptribe@NFVI-COMMON-NAGIOS-01 ~]$ rrdtool info /usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.rrd
filename = "/usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.rrd"
rrd_version = "0003"
step = 60
last_update = 1441076831
ds[1].type = "GAUGE"
ds[1].minimal_heartbeat = 8460
ds[1].min = NaN
ds[1].max = NaN
ds[1].last_ds = "5"
ds[1].value = 5.5000000000e+01
ds[1].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 2880
rra[0].cur_row = 1106
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 2880
rra[1].cur_row = 2827
rra[1].pdp_per_row = 5
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 9.8533333333e+00
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 4320
rra[2].cur_row = 526
rra[2].pdp_per_row = 30
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 3.0560000000e+01
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 5840
rra[3].cur_row = 5053
rra[3].pdp_per_row = 360
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 7.3644666667e+02
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[4].cf = "MAX"
rra[4].rows = 2880
rra[4].cur_row = 767
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[5].cf = "MAX"
rra[5].rows = 2880
rra[5].cur_row = 1209
rra[5].pdp_per_row = 5
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = 4.9266666667e+00
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[6].cf = "MAX"
rra[6].rows = 4320
rra[6].cur_row = 36
rra[6].pdp_per_row = 30
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = 4.9266666667e+00
rra[6].cdp_prep[0].unknown_datapoints = 0
rra[7].cf = "MAX"
rra[7].rows = 5840
rra[7].cur_row = 748
rra[7].pdp_per_row = 360
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = 9.7400000000e+00
rra[7].cdp_prep[0].unknown_datapoints = 0
rra[8].cf = "MIN"
rra[8].rows = 2880
rra[8].cur_row = 2087
rra[8].pdp_per_row = 1
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = NaN
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[9].cf = "MIN"
rra[9].rows = 2880
rra[9].cur_row = 1200
rra[9].pdp_per_row = 5
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = 4.9266666667e+00
rra[9].cdp_prep[0].unknown_datapoints = 0
rra[10].cf = "MIN"
rra[10].rows = 4320
rra[10].cur_row = 663
rra[10].pdp_per_row = 30
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = 2.9633333333e+00
rra[10].cdp_prep[0].unknown_datapoints = 0
rra[11].cf = "MIN"
rra[11].rows = 5840
rra[11].cur_row = 3863
rra[11].pdp_per_row = 360
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = 1.1000000000e-01
rra[11].cdp_prep[0].unknown_datapoints = 0

/usr/local/nagios/etc/services/Active Session.cfg

Code: Select all

###############################################################################
#
# Service configuration file
#
# Created by: Nagios QL Version 3.0.3
# Date:       2014-10-06 01:51:03
# Version:    Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND ---
# Nagios QL will overwite all manual settings during the next update
#
###############################################################################

define service {
        host_name                       RCS-PROD-SSC-01,RCS-PROD-SSC-02,RCS-PROD-SSC-03
        service_description             Active Sessions
        check_command                   check_xi_service_snmp!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
        initial_state                   u
        max_check_attempts              10
        check_interval                  5
        retry_interval                  10
        active_checks_enabled           1
        check_period                    24x7
        event_handler                   xi_service_event_handler
        register                        1
        }

###############################################################################
#
# Service configuration file
#
# END OF FILE
#
###############################################################################
/usr/local/nagios/etc/services/Memory Usage.cfg

Code: Select all

###############################################################################
#
# Service configuration file
#
# Created by: Nagios QL Version 3.0.3
# Date:       2014-10-15 11:07:57
# Version:    Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND ---
# Nagios QL will overwite all manual settings during the next update
#
###############################################################################

define service {
        host_name                       RCS-PROD-SSC-01,RCS-PROD-SSC-02,RCS-PROD-SSC-03
        service_description             Memory Usage on SSC Nodes
        check_command                   check_xi_service_snmp!-P 2c -o globalStatFreeMem.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -w 3276: -c 1638:!!!!!!!
        initial_state                   u
        max_check_attempts              10
        check_interval                  5
        retry_interval                  10
        active_checks_enabled           1
        check_period                    24x7
        event_handler                   xi_service_event_handler
        register                        1
        }

###############################################################################
#
# Service configuration file
#
# END OF FILE
#
###############################################################################

/usr/local/nagios/etc/services/CPU Usage.cfg

Code: Select all

###############################################################################
#
# Service configuration file
#
# Created by: Nagios QL Version 3.0.3
###############################################################################
#
# Service configuration file
#
# Created by: Nagios QL Version 3.0.3
# Date:       2014-10-15 12:08:49
# Version:    Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND ---
# Nagios QL will overwite all manual settings during the next update
#
###############################################################################

define service {
        host_name                       RCS-PROD-MSCMSRP-01,RCS-PROD-MSCRTP-01,RCS-PROD-SSC-01,RCS-PROD-SSC-02,RCS-PROD-SSC-03
        service_description             CPU usage on SBC Nodes
        check_command                   check_xi_service_snmp!-P 2c -o globalStatCPUMaxCore.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -w 700000 -c 900000!!!!!!!
        initial_state                   u
        max_check_attempts              10
        check_interval                  5
        retry_interval                  10
        active_checks_enabled           1
        check_period                    24x7
        event_handler                   xi_service_event_handler
        register                        1
        }

###############################################################################
#
# Service configuration file
#
# END OF FILE
#
###############################################################################

/usr/local/nagios/etc/services/Registered Sub.cfg

Code: Select all

###############################################################################
#
# Service configuration file
#
# Created by: Nagios QL Version 3.0.3
# Date:       2014-10-06 01:56:20
# Version:    Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND ---
# Nagios QL will overwite all manual settings during the next update
#
###############################################################################

define service {
        host_name                       RCS-PROD-SSC-01,RCS-PROD-SSC-02,RCS-PROD-SSC-03
        service_description             Number of Registered Subscribers
        check_command                   check_xi_service_snmp!-P 2c -o globalStatActiveSubscribers.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 150000!!!!!!!
        initial_state                   u
        max_check_attempts              10
        check_interval                  5
        retry_interval                  10
        active_checks_enabled           1
        check_period                    24x7
        event_handler                   xi_service_event_handler
        register                        1
        }

###############################################################################
#
# Service configuration file
#
# END OF FILE
#
###############################################################################
/usr/local/nagios/etc/commands.cfg
commands.cfg.log
/usr/local/nagios/etc/servicetemplates.cfg
servicetemplates.cfg.log
You do not have the required permissions to view the files attached to this post.
User avatar
tgriep
Madmin
Posts: 9190
Joined: Thu Oct 30, 2014 9:02 am

Re: check_snmp, performance graph and graph explorer discrep

Post by tgriep »

Create a command in XI like the following.

Code: Select all

define command {
       command_name                             check_snmp_gauge
       command_line                             $USER1$/check_snmp -H $HOSTADDRESS$ $ARG1$
}
Edit your Active Session service and change this from

Code: Select all

check_command                   check_xi_service_snmp!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
to

Code: Select all

check_command                   check_snmp_gauge!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
Delete the rrd and xml files for that service check and wait 20 minutes to see if the graphs are correct now.
Be sure to check out our Knowledgebase for helpful articles and solutions!
iptribe
Posts: 52
Joined: Sun Aug 31, 2014 11:03 am

Re: check_snmp, performance graph and graph explorer discrep

Post by iptribe »

I remember I created check_snmp_gauge in CCM > Commands long ago. But when I check in /usr/local/nagios/etc/commands.cfg it is not there. Is this normal?
check_snmp_gauge.PNG
I modified Active Session.cfg and deleted xml and RRD files but gave me same result.

Code: Select all

###############################################################################
#
# Service configuration file
#
# Created by: Nagios QL Version 3.0.3
# Date:	      2014-10-06 01:51:03
# Version:    Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND --- 
# Nagios QL will overwite all manual settings during the next update
#
###############################################################################

define service {
	host_name			RCS-PROD-SSC-01,RCS-PROD-SSC-02,RCS-PROD-SSC-03
	service_description		Active Sessions
	check_command			check_snmp_gauge!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
	initial_state			u
	max_check_attempts		10
	check_interval			5
	retry_interval			10
	active_checks_enabled		1
	check_period			24x7
	event_handler			xi_service_event_handler
	register			1
	}	

###############################################################################
#
# Service configuration file
#
# END OF FILE
#
###############################################################################
Here is the perfdata.log
Nagios-01_2015-09-03_11-46-48.zip
You do not have the required permissions to view the files attached to this post.
Last edited by iptribe on Wed Sep 02, 2015 11:04 pm, edited 2 times in total.