Updating & graphing Passive Checks

Fred Kroeger · Post by **Fred Kroeger** » Thu May 03, 2012 8:26 pm

I have setup some passive checks for servers that are not connected to my network.
I run the plugin scripts on the remote servers and save their output to a common file.
I upload this common file at different times and insert it into nagios by cat'ing the file contents to nagios.cmd
This updates the current status OK but is not consistently updating the graphs.
Looking at the graph xml file it reports an error

<TXT>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd: illegal attempt to update using time 1336093219 when last update time is 1336093219 (minimum one second step)</TXT>

It appears the update time displayed above is actually the current time and not the time of the check

The contents of the data file is as follows

Code: Select all

[1336089301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Swap Check;0;SWAP OK - 100% free (40911 MB out of 40962 MB) |swap=40911MB;4096;2048;0;40962
[1336089301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Disk Check;0;DISK OK - free space: / 42614 MB (83% inode=-); /backup 401782 MB (98% inode=-); /db2 18183 MB (88% inode=-); /db2/WOD 104082 MB (50% inode=-); /db2/WOT 204841 MB (66% inode=-); /sapmnt 5706 MB (69% inode=-); /usr/sap 18624 MB (90% inode=-); /usr/sap/trans 19895 MB (97% inode=-);| /=8591MB;46084;48644;0;51205 /backup=7807MB;368631;389110;0;409590 /db2=2297MB;18432;19456;0;20480 /db2/WOD=100716MB;184318;194558;0;204798 /db2/WOT=102353MB;276474;291834;0;307194 /sapmnt=2490MB;7376;7786;0;8196 /usr/sap=1855MB;18432;19456;0;20480 /usr/sap/trans=585MB;18432;19456;0;20480
[1336089301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Load Check;0;OK - load average: 0.37, 0.28, 0.27|load1=0.370;15.000;20.000;0; load5=0.280;10.000;15.000;0; load15=0.270;5.000;10.000;0;
[1336089301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Procs Check;0;PROCS OK: 244 processes
[1336089301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;CPU Check;0;OK - user: 0.58, nice: 0.50, sys: 0.58, iowait: 1.52, irq: 0.50, softirq: 0.50 idle: 99.30 | 'user'=0.58 'nice'=0.50 'sys'=0.58 'softirq'=0.50 'iowait'=1.52 'irq'=0.50 'idle'=99.30

The times in the data file are always consecutive (at 5min intervals), and I may have a full days worth of data to load at one time. However the graph has chunks missing when I view it

regards Fred

mguthrie · Post by **mguthrie** » Fri May 04, 2012 9:20 am

Can you post the permissions on that set of rrd's?

mguthrie · Post by **mguthrie** » Fri May 04, 2012 9:32 am

From doing a little bit more digging on this, it could either be an issue of multiple nagios instances running, or something related to the rrd heartbeat:
http://oss.oetiker.ch/rrdtool/doc/rrdcr ... d_the_STEP

We can eliminate the multiple instances possibility by running the below commands:

Code: Select all

service nagios stop
killall -9 nagios
service nagios start

Fred Kroeger · Post by **Fred Kroeger** » Sun May 06, 2012 8:38 pm

Permission on rrd files OK
Only a single instance of Nagios is running - all my ~700 Active hosts & ~5800 Active service checks are all graphing OK.

I changed the HEARTBEAT value to 260000 as I have data to load from3days ago.
As you can see from the xml file, I'm still geting the load eror because of the update time.

Basically, what I'm trying to achieve is collect performance data on a remote server and load the data into Nagios every 24hrs.
Here is the last few lines of the data file I'm tring to insert into the nagios.cmd pipe
The time does not match the time of the last entry in the xml file (which I have calculated to be the time that the insert ocurred)

Code: Select all

[1336233301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Swap Check;0;SWAP OK - 100% free (40911 MB out of 40962 MB) |swap=40911MB;4096;2048;0;40962
[1336233301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Disk Check;0;DISK OK - free space: / 42611 MB (83% inode=-); /backup 401782 MB (98% inode=-); /db2 18183 MB (88% inode=-); /db2/WOD 103486 MB (50% inode=-); /db2/WOT 204311 MB (66% inode=-); /sapmnt 5700 MB (69% inode=-); /usr/sap 18606 MB (90% inode=-); /usr/sap/trans 19894 MB (97% inode=-);| /=8594MB;46084;48644;0;51205 /backup=7807MB;368631;389110;0;409590 /db2=2297MB;18432;19456;0;20480 /db2/WOD=101311MB;184318;194558;0;204798 /db2/WOT=102882MB;276474;291834;0;307194 /sapmnt=2496MB;7376;7786;0;8196 /usr/sap=1874MB;18432;19456;0;20480 /usr/sap/trans=585MB;18432;19456;0;20480
[1336233301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Load Check;0;OK - load average: 0.37, 0.34, 0.29|load1=0.370;15.000;20.000;0; load5=0.340;10.000;15.000;0; load15=0.290;5.000;10.000;0;
[1336233301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Procs Check;0;PROCS OK: 242 processes
[1336233301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;CPU Check;0;OK - user: 0.53, nice: 0.50, sys: 0.54, iowait: 2.17, irq: 0.50, softirq: 0.50 idle: 98.73 | user=0.53 nice=0.50 sys=0.54 softirq=0.50 iowait=2.17 irq=0.50 idle=98.73

Code: Select all

-rw-rw-rw-    1 nagios nagios 5584048 May  7 09:23 CPU_Check.rrd
-rw-rw-rw-    1 nagios nagios    6058 May  7 09:23 CPU_Check.xml
-rw-rw-rw-    1 nagios nagios 6381560 May  7 09:23 Disk_Check.rrd
-rw-rw-rw-    1 nagios nagios    7504 May  7 09:23 Disk_Check.xml
-rw-rw-rw-    1 nagios nagios 2394000 May  7 09:23 Load_Check.rrd
-rw-rw-rw-    1 nagios nagios    3604 May  7 09:23 Load_Check.xml
-rw-rw-rw-    1 nagios nagios  798976 May  7 09:23 Swap_Check.rrd
-rw-rw-rw-    1 nagios nagios    2233 May  7 09:23 Swap_Check.xml

Code: Select all

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<NAGIOS>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>1</DS>
    <NAME>user</NAME>
    <LABEL>user</LABEL>
    <UNIT></UNIT>
    <ACT>0.50</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>2</DS>
    <NAME>nice</NAME>
    <LABEL>nice</LABEL>
    <UNIT></UNIT>
    <ACT>0.50</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>3</DS>
    <NAME>sys</NAME>
    <LABEL>sys</LABEL>
    <UNIT></UNIT>
    <ACT>0.53</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>4</DS>
    <NAME>softirq</NAME>
    <LABEL>softirq</LABEL>
    <UNIT></UNIT>
    <ACT>0.50</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>5</DS>
    <NAME>iowait</NAME>
    <LABEL>iowait</LABEL>
    <UNIT></UNIT>
    <ACT>0.93</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>6</DS>
    <NAME>irq</NAME>
    <LABEL>irq</LABEL>
    <UNIT></UNIT>
    <ACT>0.50</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>7</DS>
    <NAME>idle</NAME>
    <LABEL>idle</LABEL>
    <UNIT></UNIT>
    <ACT>100.03</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <RRD>
    <RC>1</RC>
    <TXT>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd: illegal attempt to update using time 1336353755 when last update time is 1336353755 (minimum one second step)</TXT>
  </RRD>
  <NAGIOS_AUTH_HOSTNAME>FMIS-Test</NAGIOS_AUTH_HOSTNAME>
  <NAGIOS_AUTH_SERVICEDESC>CPU Check</NAGIOS_AUTH_SERVICEDESC>
  <NAGIOS_CHECK_COMMAND>Passive-Check!!!!!!!!</NAGIOS_CHECK_COMMAND>
  <NAGIOS_DATATYPE>SERVICEPERFDATA</NAGIOS_DATATYPE>
  <NAGIOS_DISP_HOSTNAME>FMIS-Test</NAGIOS_DISP_HOSTNAME>
  <NAGIOS_DISP_SERVICEDESC>CPU Check</NAGIOS_DISP_SERVICEDESC>
  <NAGIOS_HOSTNAME>FMIS-Test</NAGIOS_HOSTNAME>
  <NAGIOS_HOSTSTATE>UP</NAGIOS_HOSTSTATE>
  <NAGIOS_HOSTSTATETYPE>HARD</NAGIOS_HOSTSTATETYPE>
  <NAGIOS_MULTI_PARENT></NAGIOS_MULTI_PARENT>
  <NAGIOS_PERFDATA>user=0.50 nice=0.50 sys=0.53 softirq=0.50 iowait=0.93 irq=0.50 idle=100.03 </NAGIOS_PERFDATA>
  <NAGIOS_RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</NAGIOS_RRDFILE>
  <NAGIOS_SERVICECHECKCOMMAND>Passive-Check!!!!!!!!</NAGIOS_SERVICECHECKCOMMAND>
  <NAGIOS_SERVICEDESC>CPU_Check</NAGIOS_SERVICEDESC>
  <NAGIOS_SERVICEOUTPUT>OK - user: 0.50, nice: 0.50, sys: 0.53, iowait: 0.93, irq: 0.50, softirq: 0.50 idle: 100.03</NAGIOS_SERVICEOUTPUT>
  <NAGIOS_SERVICEPERFDATA>user=0.50 nice=0.50 sys=0.53 softirq=0.50 iowait=0.93 irq=0.50 idle=100.03</NAGIOS_SERVICEPERFDATA>
  <NAGIOS_SERVICESTATE>OK</NAGIOS_SERVICESTATE>
  <NAGIOS_SERVICESTATETYPE>HARD</NAGIOS_SERVICESTATETYPE>
  <NAGIOS_TIMET>1336353755</NAGIOS_TIMET>
  <NAGIOS_XMLFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.xml</NAGIOS_XMLFILE>
  <XML>
   <VERSION>4</VERSION>
  </XML>
</NAGIOS>

scottwilkerson · Post by **scottwilkerson** » Mon May 07, 2012 9:13 am

Can you post the command you are using to "cat" the entries to nagios.cmd as well as a view of some of the file you are pushing in (showing several timestamps)

Fred Kroeger · Post by **Fred Kroeger** » Fri May 11, 2012 2:22 am

Ok - Below is

1) The results file which contains the ouput of various plugins at 5min intervals
2) Inserting the file into Nagios
3) Display of one of the Check XML files showing that it attempted to update using time 1336720290 (which was the time I inserted the data file) - not the time that is shown in the xml file

Code: Select all

# head -20 check_results.2012-05-08
[1336406401] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Load Check;0;OK - load average: 1.32, 1.03, 0.65|load1=1.320;15.000;20.000;0; load5=1.030;10.000;15.000;0; load15=0.650;5.000;10.000;0;
[1336406401] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Swap Check;0;SWAP OK - 100% free (40911 MB out of 40962 MB) |swap=40911MB;4096;2048;0;40962
[1336406401] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Disk Check;0;DISK OK - free space: / 42608 MB (83% inode=-); /backup 401782 MB (98% inode=-); /db2 18183 MB (88% inode=-); /db2/WOD 102560 MB (50% inode=-); /db2/WOT 203514 MB (66% inode=-); /sapmnt 5694 MB (69% inode=-); /usr/sap 18584 MB (90% inode=-); /usr/sap/trans 19894 MB (97% inode=-);| /=8597MB;46084;48644;0;51205 /backup=7807MB;368631;389110;0;409590 /db2=2297MB;18432;19456;0;20480 /db2/WOD=102238MB;184318;194558;0;204798 /db2/WOT=103679MB;276474;291834;0;307194 /sapmnt=2502MB;7376;7786;0;8196 /usr/sap=1895MB;18432;19456;0;20480 /usr/sap/trans=586MB;18432;19456;0;20480
[1336406401] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Procs Check;0;PROCS OK: 255 processes
[1336406401] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;CPU Check;0;OK - user: 0.96, nice: 0.50, sys: 0.63, iowait: 8.40, irq: 0.50, softirq: 0.50 idle: 92.00 | user=0.96 nice=0.50 sys=0.63 softirq=0.50 iowait=8.40 irq=0.50 idle=92.00
[1336406701] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Swap Check;0;SWAP OK - 100% free (40912 MB out of 40962 MB) |swap=40912MB;4096;2048;0;40962
[1336406701] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Load Check;0;OK - load average: 1.82, 1.46, 0.94|load1=1.820;15.000;20.000;0; load5=1.460;10.000;15.000;0; load15=0.940;5.000;10.000;0;
[1336406701] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Disk Check;0;DISK OK - free space: / 42608 MB (83% inode=-); /backup 401782 MB (98% inode=-); /db2 18183 MB (88% inode=-); /db2/WOD 102560 MB (50% inode=-); /db2/WOT 203514 MB (66% inode=-); /sapmnt 5694 MB (69% inode=-); /usr/sap 18584 MB (90% inode=-); /usr/sap/trans 19894 MB (97% inode=-);| /=8597MB;46084;48644;0;51205 /backup=7807MB;368631;389110;0;409590 /db2=2297MB;18432;19456;0;20480 /db2/WOD=102238MB;184318;194558;0;204798 /db2/WOT=103679MB;276474;291834;0;307194 /sapmnt=2502MB;7376;7786;0;8196 /usr/sap=1895MB;18432;19456;0;20480 /usr/sap/trans=586MB;18432;19456;0;20480
[1336406701] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Procs Check;0;PROCS OK: 243 processes
[1336406701] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;CPU Check;0;OK - user: 0.51, nice: 0.50, sys: 0.53, iowait: 2.17, irq: 0.50, softirq: 0.50 idle: 98.77 | user=0.51 nice=0.50 sys=0.53 softirq=0.50 iowait=2.17 irq=0.50 idle=98.77
[1336407001] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Swap Check;0;SWAP OK - 100% free (40912 MB out of 40962 MB) |swap=40912MB;4096;2048;0;40962
[1336407001] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Disk Check;0;DISK OK - free space: / 42608 MB (83% inode=-); /backup 401782 MB (98% inode=-); /db2 18183 MB (88% inode=-); /db2/WOD 102560 MB (50% inode=-); /db2/WOT 203514 MB (66% inode=-); /sapmnt 5694 MB (69% inode=-); /usr/sap 18584 MB (90% inode=-); /usr/sap/trans 19894 MB (97% inode=-);| /=8597MB;46084;48644;0;51205 /backup=7807MB;368631;389110;0;409590 /db2=2297MB;18432;19456;0;20480 /db2/WOD=102238MB;184318;194558;0;204798 /db2/WOT=103679MB;276474;291834;0;307194 /sapmnt=2502MB;7376;7786;0;8196 /usr/sap=1895MB;18432;19456;0;20480 /usr/sap/trans=586MB;18432;19456;0;20480
[1336407001] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Load Check;0;OK - load average: 0.38, 0.78, 0.78|load1=0.380;15.000;20.000;0; load5=0.780;10.000;15.000;0; load15=0.780;5.000;10.000;0;
[1336407001] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Procs Check;0;PROCS OK: 242 processes
[1336407001] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;CPU Check;0;OK - user: 0.63, nice: 0.50, sys: 0.59, iowait: 2.37, irq: 0.50, softirq: 0.50 idle: 98.39 | user=0.63 nice=0.50 sys=0.59 softirq=0.50 iowait=2.37 irq=0.50 idle=98.39
[1336407301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Swap Check;0;SWAP OK - 100% free (40912 MB out of 40962 MB) |swap=40912MB;4096;2048;0;40962
[1336407301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Disk Check;0;DISK OK - free space: / 42608 MB (83% inode=-); /backup 401782 MB (98% inode=-); /db2 18183 MB (88% inode=-); /db2/WOD 102560 MB (50% inode=-); /db2/WOT 203514 MB (66% inode=-); /sapmnt 5694 MB (69% inode=-); /usr/sap 18584 MB (90% inode=-); /usr/sap/trans 19894 MB (97% inode=-);| /=8597MB;46084;48644;0;51205 /backup=7807MB;368631;389110;0;409590 /db2=2297MB;18432;19456;0;20480 /db2/WOD=102238MB;184318;194558;0;204798 /db2/WOT=103679MB;276474;291834;0;307194 /sapmnt=2502MB;7376;7786;0;8196 /usr/sap=1895MB;18432;19456;0;20480 /usr/sap/trans=586MB;18432;19456;0;20480
[1336407301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Load Check;0;OK - load average: 0.22, 0.47, 0.64|load1=0.220;15.000;20.000;0; load5=0.470;10.000;15.000;0; load15=0.640;5.000;10.000;0;
[1336407301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;Procs Check;0;PROCS OK: 246 processes
[1336407301] PROCESS_SERVICE_CHECK_RESULT;FMIS-Test;CPU Check;0;OK - user: 0.53, nice: 0.50, sys: 0.59, iowait: 1.71, irq: 0.50, softirq: 0.50 idle: 99.15 | user=0.53 nice=0.50 sys=0.59 softirq=0.50 iowait=1.71 irq=0.50 idle=99.15

Code: Select all

# cat check_results.2012-05-08 > /usr/local/nagios/var/rw/nagios.cmd

Code: Select all

# cat CPU_Check.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<NAGIOS>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>1</DS>
    <NAME>user</NAME>
    <LABEL>user</LABEL>
    <UNIT></UNIT>
    <ACT>0.53</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>2</DS>
    <NAME>nice</NAME>
    <LABEL>nice</LABEL>
    <UNIT></UNIT>
    <ACT>0.50</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>3</DS>
    <NAME>sys</NAME>
    <LABEL>sys</LABEL>
    <UNIT></UNIT>
    <ACT>0.54</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>4</DS>
    <NAME>softirq</NAME>
    <LABEL>softirq</LABEL>
    <UNIT></UNIT>
    <ACT>0.50</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>5</DS>
    <NAME>iowait</NAME>
    <LABEL>iowait</LABEL>
    <UNIT></UNIT>
    <ACT>1.52</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>6</DS>
    <NAME>irq</NAME>
    <LABEL>irq</LABEL>
    <UNIT></UNIT>
    <ACT>0.50</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <DATASOURCE>
    <TEMPLATE>Passive-Check</TEMPLATE>
    <RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</RRDFILE>
    <RRD_STORAGE_TYPE>SINGLE</RRD_STORAGE_TYPE>
    <RRD_HEARTBEAT>260000</RRD_HEARTBEAT>
    <IS_MULTI>0</IS_MULTI>
    <DS>7</DS>
    <NAME>idle</NAME>
    <LABEL>idle</LABEL>
    <UNIT></UNIT>
    <ACT>99.38</ACT>
    <WARN></WARN>
    <WARN_MIN></WARN_MIN>
    <WARN_MAX></WARN_MAX>
    <WARN_RANGE_TYPE></WARN_RANGE_TYPE>
    <CRIT></CRIT>
    <CRIT_MIN></CRIT_MIN>
    <CRIT_MAX></CRIT_MAX>
    <CRIT_RANGE_TYPE></CRIT_RANGE_TYPE>
    <MIN></MIN>
    <MAX></MAX>
  </DATASOURCE>
  <RRD>
    <RC>1</RC>
    <TXT>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd: illegal attempt to update using time 1336720290 when last update time is 1336720290 (minimum one second step)</TXT>
  </RRD>
  <NAGIOS_AUTH_HOSTNAME>FMIS-Test</NAGIOS_AUTH_HOSTNAME>
  <NAGIOS_AUTH_SERVICEDESC>CPU Check</NAGIOS_AUTH_SERVICEDESC>
  <NAGIOS_CHECK_COMMAND>Passive-Check!!!!!!!!</NAGIOS_CHECK_COMMAND>
  <NAGIOS_DATATYPE>SERVICEPERFDATA</NAGIOS_DATATYPE>
  <NAGIOS_DISP_HOSTNAME>FMIS-Test</NAGIOS_DISP_HOSTNAME>
  <NAGIOS_DISP_SERVICEDESC>CPU Check</NAGIOS_DISP_SERVICEDESC>
  <NAGIOS_HOSTNAME>FMIS-Test</NAGIOS_HOSTNAME>
  <NAGIOS_HOSTSTATE>UP</NAGIOS_HOSTSTATE>
  <NAGIOS_HOSTSTATETYPE>HARD</NAGIOS_HOSTSTATETYPE>
  <NAGIOS_MULTI_PARENT></NAGIOS_MULTI_PARENT>
  <NAGIOS_PERFDATA>user=0.53 nice=0.50 sys=0.54 softirq=0.50 iowait=1.52 irq=0.50 idle=99.38 </NAGIOS_PERFDATA>
  <NAGIOS_RRDFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.rrd</NAGIOS_RRDFILE>
  <NAGIOS_SERVICECHECKCOMMAND>Passive-Check!!!!!!!!</NAGIOS_SERVICECHECKCOMMAND>
  <NAGIOS_SERVICEDESC>CPU_Check</NAGIOS_SERVICEDESC>
  <NAGIOS_SERVICEOUTPUT>OK - user: 0.53, nice: 0.50, sys: 0.54, iowait: 1.52, irq: 0.50, softirq: 0.50 idle: 99.38</NAGIOS_SERVICEOUTPUT>
  <NAGIOS_SERVICEPERFDATA>user=0.53 nice=0.50 sys=0.54 softirq=0.50 iowait=1.52 irq=0.50 idle=99.38</NAGIOS_SERVICEPERFDATA>
  <NAGIOS_SERVICESTATE>OK</NAGIOS_SERVICESTATE>
  <NAGIOS_SERVICESTATETYPE>HARD</NAGIOS_SERVICESTATETYPE>
  <NAGIOS_TIMET>1336720290</NAGIOS_TIMET>
  <NAGIOS_XMLFILE>/usr/local/nagios/share/perfdata/FMIS-Test/CPU_Check.xml</NAGIOS_XMLFILE>
  <XML>
   <VERSION>4</VERSION>
  </XML>
</NAGIOS>

scottwilkerson · Post by **scottwilkerson** » Fri May 11, 2012 11:11 am

I now know why this is happening, but as of yet don't have a solution.

The error you are getting if from the graphing engine, not nagios core. The performance files are being created using the current time, not the command time..

I'm going to run this by some of the developers to see if it is even possible to have the graphing RRD's update after the fact, and I will let you know.

Fred Kroeger · Post by **Fred Kroeger** » Sun May 13, 2012 7:31 pm

Thanks Scott
I know I have done this previously manually - eg: create an rrd file and then load it with old data. So it is possible.
This works as long as you haven't inserted any entry that is newer than the times of the old data.

This problem appears to indicate that an entry is being created in the rrd file for the current time before I can insert the old data.

regards Fred

scottwilkerson · Post by **scottwilkerson** » Mon May 14, 2012 9:28 am

We are going to work on what needs to be put in place to allow this, but I would guess it will be several revisions before it is complete. It is possible there may be mods that need to be made to nagios core for it to be a reality.

Fred Kroeger · Post by **Fred Kroeger** » Mon May 14, 2012 8:13 pm

Thanks for the follow-up. I will describe what I'm trying to achieve in case you have a better way of doing it.

I have a remote site that I can't connect to with my NagiosXI Server. I have installed the plugins on the remote servers and am using cron to run them on a regular basis.
I am saving the output of each plugin into a common file in a format that I can insert into the nagios.cmd pipe.
Every day, I email the file back to myself as an attachment, so that I can insert the performance stats into Nagios. All the Nagios monitors have been setup as Passive Monitors.
Later on when I am happy all is working OK, I intend to email the datafile every hour so that our monitoring & reporting is semi-current.
The insertion of the data is important for me as I need to keep the historical data for Capacity planning.

regards.... Fred

Nagios Support Forum

Updating & graphing Passive Checks

Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks

Re: Updating & graphing Passive Checks