Peter, it's a bit difficult to tell what's going on without seeing the rest of the syslog-ng configuration and your script's source code. However, a couple possibilities come to mind: - Your script is only reading one line at a time. syslog-ng starts a program() output persistently and expects that it can send multiple messages into its pipe to your script's stdin. - Messages are being buffered inside of syslog-ng. Check out the flush_lines() and flush_timeout() flags to syslog-ng's program() output. Find the right page for your version, but here's v3.3.: http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guide... - Messages are being buffered in your shell or script. Maybe try some non-blocking IO with a smallish buffer to see data as it comes in before a whole line or block fills and flushes in. To Anurag's question about open source log management with a WebUI, I agree with Blake: logstash ingesting syslog and inputting it into elasticsearch makes for a great backend for Kibana. The logstash grok filter is great for pulling apart and indexing weird vendor-specific logging formats: http://logstash.net/docs/1.4.1/filters/grok Cheers, jof On Sat, May 10, 2014 at 2:24 AM, Peter Persson <webbax@webbax.se> wrote:
Hey,
I got a weird problem with my syslog-ng setup, im logging from alot of cisco machines and that works great. The problem is that when i "pass" this further to a shell program, some lines disapere.
My destination looks like this destination hosts { file("/var/log/ciscorouters/$HOST.log" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); program("/scripts/irc/syslog_wrapper_new.sh" template(t_irctempl)); }; The "/var/log/ciscorouters/$HOST.log" writes correct, but the data thats putted trough to "/scripts/irc/syslog_wrapper_new.sh" only get the first line, if it gets flooded (like 5 rows per second).
Do anyone of you have any idea of what might be the problem?
Regards, Peter