Nagios Support Forum

Return-Path: <redacted@gmail.com>
Received: from localhost (redacted.net. [redacted])
by mx.google.com with ESMTPSA id m203sm33312930ywb.33.2015.06.30.09.16.51
for <redacted@gmail.com>
(version=TLSv1 cipher=RC4-SHA bits=128/128);
Tue, 30 Jun 2015 09:16:51 -0700 (PDT)
Message-ID: <40210.1193062379-sendEmail@localhost>
From: "redacted@gmail.com" <redacted@gmail.com>
To: "redacted@gmail.com" <redacted@gmail.com>
Subject: 0001 redacted: 30-06-15 12:16:55 R Z000 0102 SYST 001 SYSTEM |SMART HDD ALERT TEMP
Date: Tue, 30 Jun 2015 16:16:55 +0000
X-Mailer: sendEmail-1.56
MIME-Version: 1.0
Content-Type: multipart/related; boundary="----MIME delimiter for sendEmail-282640.359903784"

This is a multi-part message in MIME format. To properly display this message you need a MIME-Version 1.0 compliant Email program.

------MIME delimiter for sendEmail-282640.359903784
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

0001 redacted: 30-06-15 12:16:55 R Z000 0102 SYST 001 SYSTEM |SMART HDD ALERT TEMP


------MIME delimiter for sendEmail-282640.359903784--

{:timestamp=>"2015-06-29T21:55:27.307000-0400", :message=>"Using milestone 1 input plugin 'syslog'. This plugin should work, but would benefit from use by folks like you. Please let us know if you find bugs or have suggestions on how to improve this plugin. For more information on plugin milestones, see http://logstash.net/docs/1.4.2/plugin-milestones", :level=>:warn}

{:timestamp=>"2015-06-29T21:55:27.372000-0400", :message=>"Using milestone 2 input plugin 'tcp'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, http://logstash.net/docs/1.4.2/plugin-milestones", :level=>:warn}

{:timestamp=>"2015-06-29T21:55:27.407000-0400", :message=>"Using milestone 1 input plugin 'imap'. This plugin should work, but would benefit from use by folks like you. Please let us know if you find bugs or have suggestions on how to improve this plugin. For more information on plugin milestones, see http://logstash.net/docs/1.4.2/plugin-milestones", :level=>:warn}

{:timestamp=>"2015-06-30T01:20:34.402000-0400", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::IMAP type=>\"mail\", host=>\"imap.gmail.com\", user=>\"redacted@gmail.com\", content_type=>\"text/plain\">\n Error: Bad file descriptor - Bad file descriptor", :level=>:error}

{:timestamp=>"2015-06-30T12:25:52.620000-0400", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::IMAP type=>\"mail\", host=>\"imap.gmail.com\", user=>\"redacted@gmail.com\", content_type=>\"text/plain\">\n Error: initialize: name or service not known", :level=>:error}

{:timestamp=>"2015-06-30T12:25:52.632000-0400", :message=>"syslog udp listener died", :address=>"0.0.0.0:5544", :exception=>#<SocketError: recvfrom: name or service not known>, :backtrace=>["/usr/local/nagioslogserver/logstash/lib/logstash/inputs/syslog.rb:119:in `udp_listener'", "org/jruby/RubyKernel.javain `loop'", "/usr/local/nagioslogserver/logstash/lib/logstash/inputs/syslog.rb:118:in `udp_listener'", "/usr/local/nagioslogserver/logstash/lib/logstash/inputs/syslog.rb:76:in `run'"], :level=>:warn}

Supposing we're at a dead end, is there anyway to make a watchdog for the service that reboots it after a crash? I know this is a work around but I could run a standalone server just for the imap functionality that outputs to the NLS cluster. This way I don't lose data on the cluster from all my other sources.

cd ~
vi logstashwatchdog.sh
chmod +x logstashwatchdog.sh

#!/bin/bash
/etc/init.d/logstash status > /dev/null
EXITC=$(echo $?)

if [ $EXITC != 0 ]; then
/etc/init.d/logstash restart
fi

crontab -e

* * * * * /root/logstashwatchdog.sh