Parsing Syslog and Acting on it, using other input too
Hello. I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own. I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this. Regards, Sam
You should look into SPLUNK (http://www.splunk.com/), it will collect/store your syslog data and you can run customized reports and then act on them. On Thu, Aug 29, 2013 at 8:03 AM, Kasper Adel <karim.adel@gmail.com> wrote:
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this.
Regards,
Sam
-- Jason
Look at Logstash, http://logstash.net. Rsyslog can do a bit, on Windows you could look at the Solarwinds Kiwi syslog server. On Thu, Aug 29, 2013 at 9:10 AM, Jason Biel <jason@biel-tech.com> wrote:
You should look into SPLUNK (http://www.splunk.com/), it will collect/store your syslog data and you can run customized reports and then act on them.
On Thu, Aug 29, 2013 at 8:03 AM, Kasper Adel <karim.adel@gmail.com> wrote:
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this.
Regards,
Sam
-- Jason
Yes. Logstash shipper on your syslog proxy, forward to elasticsearch. Graylog2 is very cool. Tried kibana and didn't care for it. Actually setting up graylog2 right now to do AD authentication. So workflow is End device -> syslog-ng vm -> graylog2/elasticsearch vm and other destinations (it corp security cloud for stuff they want to track, observium for anything matching my network gear hostname pattern, etc). I have the middle syslog-ng box so I can have great control over where certain hosts ultimately send data. However that system can be used in any template, if I don't filter it just gets dumped to graylog. Kevin Stone <kstone@inetlabs.net> wrote:
Look at Logstash, http://logstash.net.
Rsyslog can do a bit, on Windows you could look at the Solarwinds Kiwi syslog server.
On Thu, Aug 29, 2013 at 9:10 AM, Jason Biel <jason@biel-tech.com> wrote:
You should look into SPLUNK (http://www.splunk.com/), it will collect/store your syslog data and you can run customized reports and then act on them.
On Thu, Aug 29, 2013 at 8:03 AM, Kasper Adel <karim.adel@gmail.com> wrote:
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this.
Regards,
Sam
-- Jason
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
My view on splunk, +1 if you intend to have a human act on the reports, it does an excellent job of reducing huge amounts of audit data into the valuable bits. -1 Seemed to be a pita to integrate with my scripting enviroment. I ended up kludging wget,awk and telnet together in a totally undignified way to make it reach out and act on something. +2 Customizable ingestion/parsing, I'm feeding everything from linux audit data to weird proprietary serial output from a multiplexer into it. -1 Proprietary database I would have liked to see an sql plugin for data storage, I would like the data in Mysql/Oracle but no-joy from splunk so that I can use other tools on it easily. +1 Free demo. You can download an eval version that is rate limited and cripples itself after a fixed time. -1 because The license costs are a bit high if your moving lots of data through it Sam Moats On 2013-08-29 09:10, Jason Biel wrote:
You should look into SPLUNK (http://www.splunk.com/), it will collect/store your syslog data and you can run customized reports and then act on them.
On Thu, Aug 29, 2013 at 8:03 AM, Kasper Adel <karim.adel@gmail.com> wrote:
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this.
Regards,
Sam
Since you said you are willing to entertain home grown as well. I would recommend looking at simple event correlator which is a perl script designed to do the kind of thing you are talking about. I've used it in the past to trigger bgp black holing and mail blacklists for example. On Thu, Aug 29, 2013 at 8:25 AM, Sam Moats <sam@circlenet.us> wrote:
My view on splunk, +1 if you intend to have a human act on the reports, it does an excellent job of reducing huge amounts of audit data into the valuable bits. -1 Seemed to be a pita to integrate with my scripting enviroment. I ended up kludging wget,awk and telnet together in a totally undignified way to make it reach out and act on something.
+2 Customizable ingestion/parsing, I'm feeding everything from linux audit data to weird proprietary serial output from a multiplexer into it. -1 Proprietary database I would have liked to see an sql plugin for data storage, I would like the data in Mysql/Oracle but no-joy from splunk so that I can use other tools on it easily.
+1 Free demo. You can download an eval version that is rate limited and cripples itself after a fixed time. -1 because The license costs are a bit high if your moving lots of data through it
Sam Moats
On 2013-08-29 09:10, Jason Biel wrote:
You should look into SPLUNK (http://www.splunk.com/), it will collect/store your syslog data and you can run customized reports and then act on them.
On Thu, Aug 29, 2013 at 8:03 AM, Kasper Adel <karim.adel@gmail.com> wrote:
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this.
Regards,
Sam
On Aug 29, 2013, at 8:03 PM, Kasper Adel wrote:
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ?
<http://simple-evcorr.sourceforge.net/> <http://www.splunk.com/> If network traffic is of interest, don't forget about flow telemetry like NetFlow and/or IPFIX. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Luck is the residue of opportunity and design. -- John Milton
For some straightforward things I have used Logdog (http://caspian.dotconf.net/menu/Software/LogDog/). With kind regards, Thijs Stuurman
-----Original Message----- From: Kasper Adel [mailto:karim.adel@gmail.com] Sent: donderdag 29 augustus 2013 15:03 To: NANOG list Subject: Parsing Syslog and Acting on it, using other input too
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re- use to do the above ? Please point me to known public or home-grown scripts in use to achieve this.
Regards,
Sam
On 8/29/2013 9:03 AM, Kasper Adel wrote:
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and
You might want to look at http://www.ossec.net/ ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
I wrote a script in Linux that watches for unauthorized login attempts and adds the ip address to the blocked list in my firewall. You might want to search sourceforge for a DYN Firewall and modify it from there. On Thu, Aug 29, 2013 at 10:44 AM, Mike Tancsa <mike@sentex.net> wrote:
On 8/29/2013 9:03 AM, Kasper Adel wrote:
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and
You might want to look at
---Mike
-- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
-- --------------------------------------------- Don Wilder --------------------------------------------- Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.
On Thu, Aug 29, 2013 at 10:50 AM, Don Wilder <don.wilder@gmail.com> wrote:
I wrote a script in Linux that watches for unauthorized login attempts and adds the ip address to the blocked list in my firewall. You might want to search sourceforge for a DYN Firewall and modify it from there.
because fail2ban was too hard to install? or because you just wanted to test yourself?
Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Thu, Aug 29, 2013 at 10:50 AM, Don Wilder <don.wilder@gmail.com> wrote:
I wrote a script in Linux that watches for unauthorized login attempts and adds the ip address to the blocked list in my firewall. You might want to search sourceforge for a DYN Firewall and modify it from there.
because fail2ban was too hard to install? or because you just wanted to test yourself?
Actually I did the same. I use ipset lists (generally with a timeout) and take a regex or two and black / white list from a YAML file and just take (possibly multiple inputs) from piping tail -F. I also store addresses for future reference (by the script or otherwise). This is quite maintainable as I can look at a list of people who have attacked the mail server and compare it to web attacks. Each process is a different type of service (different config file) and probably a different ipset. Due to ipset not actually doing anything until I make an iptables rule for it, I can run my script in a test mode (by default) and just see what happens (check it's logs and the ipset list it generates). I haven't found the need for this yet but I can use cymru to look up how big their net is (see geocidr for an example of how to do this in perl) and use a hash:net ipset type and cover a whole net. Basically what I'm saying in doing it this way is quite expandable and isn't very hard and I can do tons of stuff that fail2ban can't (I don't think - it's been a while since I looked).
On Fri, Aug 30, 2013 at 8:55 AM, Shawn Wilson <ag4ve.us@gmail.com> wrote:
Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Thu, Aug 29, 2013 at 10:50 AM, Don Wilder <don.wilder@gmail.com> wrote:
I wrote a script in Linux that watches for unauthorized login attempts and adds the ip address to the blocked list in my firewall. You might want to search sourceforge for a DYN Firewall and modify it from there.
because fail2ban was too hard to install? or because you just wanted to test yourself?
Actually I did the same. I use ipset lists (generally with a timeout) and take a regex or two and black / white list from a YAML file and just take (possibly multiple inputs) from piping tail -F. I also store addresses for future reference (by the script or otherwise).
This is quite maintainable as I can look at a list of people who have attacked the mail server and compare it to web attacks. Each process is a different type of service (different config file) and probably a different ipset. Due to ipset not actually doing anything until I make an iptables rule for it, I can run my script in a test mode (by default) and just see what happens (check it's logs and the ipset list it generates). I haven't found the need for this yet but I can use cymru to look up how big their net is (see geocidr for an example of how to do this in perl) and use a hash:net ipset type and cover a whole net.
Basically what I'm saying in doing it this way is quite expandable and isn't very hard and I can do tons of stuff that fail2ban can't (I don't think - it's been a while since I looked).
you seem to be describing what fail2ban does... that and some grep of syslog for fail2ban messages. If your solution works then great! :)
Ah it seems they do: https://github.com/fail2ban/fail2ban/blob/master/config/action.d/iptables-ip... IDK enough about fail2ban to know whether I can assign a per proto or per log type config (I assume I can). In which casethis does what my script does and then some. I would probably dump out a ipset save on exit and try to 'restore' on resume (which /I/ do) and I'm sure there's a way fail2ban can check a store of addresses and check what network a host belongs to (instead of just a host). So, fail2ban is probably the way to go. On Fri, Aug 30, 2013 at 10:00 AM, Christopher Morrow < morrowc.lists@gmail.com> wrote:
On Fri, Aug 30, 2013 at 8:55 AM, Shawn Wilson <ag4ve.us@gmail.com> wrote:
Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Thu, Aug 29, 2013 at 10:50 AM, Don Wilder <don.wilder@gmail.com> wrote:
I wrote a script in Linux that watches for unauthorized login attempts and adds the ip address to the blocked list in my firewall. You might want to search sourceforge for a DYN Firewall and modify it from there.
because fail2ban was too hard to install? or because you just wanted to test yourself?
Actually I did the same. I use ipset lists (generally with a timeout)
and take a regex or two and black / white list from a YAML file and just take (possibly multiple inputs) from piping tail -F. I also store addresses for future reference (by the script or otherwise).
This is quite maintainable as I can look at a list of people who have
attacked the mail server and compare it to web attacks. Each process is a different type of service (different config file) and probably a different ipset. Due to ipset not actually doing anything until I make an iptables rule for it, I can run my script in a test mode (by default) and just see what happens (check it's logs and the ipset list it generates). I haven't found the need for this yet but I can use cymru to look up how big their net is (see geocidr for an example of how to do this in perl) and use a hash:net ipset type and cover a whole net.
Basically what I'm saying in doing it this way is quite expandable and
isn't very hard and I can do tons of stuff that fail2ban can't (I don't think - it's been a while since I looked).
you seem to be describing what fail2ban does... that and some grep of syslog for fail2ban messages. If your solution works then great! :)
Check out Sagan: http://sagan.quadrantsec.com/ On 8/29/13 6:03 AM, Kasper Adel wrote:
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this.
Regards,
Sam
+1 on Splunk or if you don't mind using a SAS service check out https://papertrailapp.com/ Carlos Alcantar Race Communications / Race Team Member 1325 Howard Ave. #604, Burlingame, CA. 94010 Phone: +1 415 376 3314 / carlos@race.com / http://www.race.com -----Original Message----- From: Kasper Adel <karim.adel@gmail.com> Date: Thursday, August 29, 2013 6:03 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Parsing Syslog and Acting on it, using other input too Hello. I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own. I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this. Regards, Sam
http://www.elasticsearch.com/blog/welcome-jordan-logstash/ So now Logstash and Elasticsearch will be even more integrated than before. With Kibana on top of that, this seems like the ultimate log data "do stuff" stack. --chip On Thu, Aug 29, 2013 at 2:03 PM, Carlos Alcantar <carlos@race.com> wrote:
+1 on Splunk or if you don't mind using a SAS service check out https://papertrailapp.com/
Carlos Alcantar Race Communications / Race Team Member 1325 Howard Ave. #604, Burlingame, CA. 94010 Phone: +1 415 376 3314 / carlos@race.com / http://www.race.com
-----Original Message----- From: Kasper Adel <karim.adel@gmail.com> Date: Thursday, August 29, 2013 6:03 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Parsing Syslog and Acting on it, using other input too
Hello.
I am looking for a way to do proactive monitoring of my network, what I am specifically thinking about is receiving syslog msgs from the routers and the backend engine would correlate certain msgs with output/data that i am receiving through SSH/telnet sessions. What i am after is not exposed to SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting upon it but i wonder if there is something more flexible out there that I can just re-use to do the above ? Please point me to known public or home-grown scripts in use to achieve this.
Regards,
Sam
-- Just my $.02, your mileage may vary, batteries not included, etc....
participants (16)
-
Blake Dunlap -
Carlos Alcantar -
Charles N Wyble -
chip -
Christopher Morrow -
Dobbins, Roland -
Don Wilder -
Gino O'Donnell -
Jason Biel -
Kasper Adel -
Kevin Stone -
Mike Tancsa -
Sam Moats -
shawn wilson -
Shawn Wilson
-
Thijs Stuurman