Greetings, Anyone has good recommendations for an open-sourced log parsing and analyzing application? It will be used to work with syslog-ng and other general syslog and application logs. I have been looking at swatch and logwatch, but would like to find out if there are other good choices, thanks FD
Splunk ZanOSS PHP-Syslog-NG aka logzilla LogLogic On 2/22/10 3:15 PM, "fedora fedora" <fedorafans@gmail.com> wrote:
Greetings,
Anyone has good recommendations for an open-sourced log parsing and analyzing application? It will be used to work with syslog-ng and other general syslog and application logs.
I have been looking at swatch and logwatch, but would like to find out if there are other good choices, thanks
FD
SEC (Simplet Event Correlator) is a very effective tool for this, IMHO. I am by no means an expert with it, but I know several people who are, and while it is not as well known as splunk or some other tools, I have been very impressed by the results I've seen using it. As with any event correlation tool, there is a significant level of invested effort required to make use of this. http://simple-evcorr.sourceforge.net/ Below is a presentation about SEC. http://www.occam.com/sa/CentralizedLogging2009.pdf On Mon, Feb 22, 2010 at 2:15 PM, fedora fedora <fedorafans@gmail.com> wrote:
Greetings,
Anyone has good recommendations for an open-sourced log parsing and analyzing application? It will be used to work with syslog-ng and other general syslog and application logs.
I have been looking at swatch and logwatch, but would like to find out if there are other good choices, thanks
FD
-- -- Darren Bolding -- -- darren@bolding.org --
I personally like SEC (Simple Event Correlator), check out http://simple-evcorr.sourceforge.net/ Jeff Rooney jtrooney@nexdlevel.com On Mon, Feb 22, 2010 at 4:15 PM, fedora fedora <fedorafans@gmail.com> wrote:
Greetings,
Anyone has good recommendations for an open-sourced log parsing and analyzing application? It will be used to work with syslog-ng and other general syslog and application logs.
I have been looking at swatch and logwatch, but would like to find out if there are other good choices, thanks
FD
ah, never heard of SEC before and it really looks interesting, Thanks everyone for the great input! FD On Mon, Feb 22, 2010 at 4:34 PM, Jeff Rooney <jtrooney@nexdlevel.com> wrote:
I personally like SEC (Simple Event Correlator), check out http://simple-evcorr.sourceforge.net/
Jeff Rooney jtrooney@nexdlevel.com
On Mon, Feb 22, 2010 at 4:15 PM, fedora fedora <fedorafans@gmail.com> wrote:
Greetings,
Anyone has good recommendations for an open-sourced log parsing and analyzing application? It will be used to work with syslog-ng and other general syslog and application logs.
I have been looking at swatch and logwatch, but would like to find out if there are other good choices, thanks
FD
On Feb 22, 2010, at 4:49 PM, fedora fedora wrote:
ah, never heard of SEC before and it really looks interesting,
Take a look at SLCT, also by Risto Vaarandi: http://ristov.users.sourceforge.net/slct/ SLCT can parse huge amounts of logs very fast. We use it to crunch firewall logs and also to find ports that are flapping excessively. Dale
On Mon, 2010-02-22 at 18:14 -0600, Dale W. Carder wrote:
Take a look at SLCT, also by Risto Vaarandi:
http://ristov.users.sourceforge.net/slct/
SLCT can parse huge amounts of logs very fast. We use it to crunch firewall logs and also to find ports that are flapping excessively.
+1, SLCT definitely finds the needles in haystacks of huge syslog files Gord -- best viewed in mailx
On Mon, Feb 22, 2010 at 04:15:22PM -0600, fedora fedora wrote:
Anyone has good recommendations for an open-sourced log parsing and analyzing application? It will be used to work with syslog-ng and other general syslog and application logs.
I have been looking at swatch and logwatch, but would like to find out if there are other good choices, thanks
SEC does seem to be the "gold standard" in advanced log correlation beyond that available in "grep | mail" type systems such as logwatch. However it is incredibly arcane, and despite reading a lot of documentation for it I've never really been able to wrap my head around it. A colleague has started to write a SEC-like tool with (I hope) a more approachable mental model; take a look at http://github.com/rodjek/grok. I must (embarrasedly) admit I haven't looked at it yet, but he claims that he reimplemented sshd_sentry (the fail2ban equivalent we use) in two lines of rules, which seems like a nice (basic) demonstration. - Matt
participants (7)
-
Dale W. Carder -
Darren Bolding -
fedora fedora -
gordon b slater -
Jeff Rooney -
Matthew Palmer -
Steven J. Hutchison