'lo all, Is anybody out there aware of a piece of software that can take data from an arbitrary source and then present it, using a MIB or set of OIDs of your choosing, as an SNMP-interrogatable device? We have some CPE that supports SNMP, but considers it to be a mutually-exclusive "remote management" protocol such that if you use another supported method for deployment and provisioning (e.g., TR-069), you cannot have both that AND SNMP enabled simultaneously. It's one or the other. We currently monitor and graph some device stats for these CPE with Cacti, but we want to be able to provision using a TR-069 ACS. The ACS can collect some of the same data we are graphing right now, but cannot present it in a fashion that is nearly as useful as the way Cacti/RRDtool does (not to mention the staff is already used to navigating Cacti). We know what SQL database table the stats are being stored in by the ACS, though, so my thought was that there must be some way that we can have a host respond to SNMP gets and then have it turn around and collect the value to be returned from a database. Basically, an ODBC -> SNMP proxy. We'd then point Cacti at that IP instead of the individual CPEs. But I can't seem to find anything like this. Thanks, -- Nathan
It is fairly easy to extend the snmpd on a Linux host to retrieve data from non-SNMP sources... For example: http://www.adventuresinoss.com/2009/09/30/the-many-uses-of-net-snmp/ https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/htm... Anything you can retrieve with a shell script can be turned into a 1-minute-interval cron job that outputs a text file with a number in it, then the SNMP 'extend' parameter in the snmpd.conf can be used to 'cat' the file. For example something from a solar charge controller that only speaks RS485 over an RS485-to-RS232 (or USB) bridge. Or data retrieved via curl and sed. I use this to graph WSDOT traffic driving times patterns and flows and feed them into a charting system. For smaller setups you can have all the shell scripts for data acquisition on the same host that runs the snmpd. On Fri, May 20, 2016 at 4:43 PM, Nathan Anderson <nathana@fsr.com> wrote:
'lo all,
Is anybody out there aware of a piece of software that can take data from an arbitrary source and then present it, using a MIB or set of OIDs of your choosing, as an SNMP-interrogatable device?
We have some CPE that supports SNMP, but considers it to be a mutually-exclusive "remote management" protocol such that if you use another supported method for deployment and provisioning (e.g., TR-069), you cannot have both that AND SNMP enabled simultaneously. It's one or the other.
We currently monitor and graph some device stats for these CPE with Cacti, but we want to be able to provision using a TR-069 ACS. The ACS can collect some of the same data we are graphing right now, but cannot present it in a fashion that is nearly as useful as the way Cacti/RRDtool does (not to mention the staff is already used to navigating Cacti). We know what SQL database table the stats are being stored in by the ACS, though, so my thought was that there must be some way that we can have a host respond to SNMP gets and then have it turn around and collect the value to be returned from a database. Basically, an ODBC -> SNMP proxy. We'd then point Cacti at that IP instead of the individual CPEs. But I can't seem to find anything like this.
Thanks,
-- Nathan
Hey, thanks guys! I had never really looked that deeply into Net-SNMP and had only ever installed it either to use as a client (snmpget/snmpwalk) or a basic agent w/ standard MIBs for the host it's running on, so I was unaware of its extensibility. And it even looks like it ships with a Perl module. That sounds like a perfect solution; thanks for pointing me in the right direction. -- Nathan
Hi Nathan, You should probably write a cacti script to ingest the data instead of this SNMP proxy thing. Writing scripts to ingest data into cacti is simple, you just need to output the values you want in key: value format and then do some clicking in cacti. There are good docs for how to do this. -- emj On 05/21/2016 08:11 AM, Nathan Anderson wrote:
Hey, thanks guys! I had never really looked that deeply into Net-SNMP and had only ever installed it either to use as a client (snmpget/snmpwalk) or a basic agent w/ standard MIBs for the host it's running on, so I was unaware of its extensibility. And it even looks like it ships with a Perl module. That sounds like a perfect solution; thanks for pointing me in the right direction.
-- Nathan
This doesn't scale on a large cacti installation with hundreds of hosts and 60-second poller intervals. Cacti data input method scripts spawn a new php worker for each data acquisition target (they do NOT use the 'spine' SNMP poller). Exposing the data via SNMP on the host to be monitored distributes the CPU load individually onto each host (example: an 'extend' script in the snmpd.conf which runs "curl http://localhost/server-status" for apache2 status and parses the results) rather than centralizing it on the cacti host. This allows cacti or opennms or anything else to poll the hosts to be monitored via something that scales better than php script workers, using the 'spine' SNMP data acquisition method and the equivalent in other snmp polling platforms. On Sat, May 21, 2016 at 6:03 AM, Eric Lindsjö <eric@emj.se> wrote:
Hi Nathan,
You should probably write a cacti script to ingest the data instead of this SNMP proxy thing. Writing scripts to ingest data into cacti is simple, you just need to output the values you want in key: value format and then do some clicking in cacti. There are good docs for how to do this.
-- emj
On 05/21/2016 08:11 AM, Nathan Anderson wrote:
Hey, thanks guys! I had never really looked that deeply into Net-SNMP and had only ever installed it either to use as a client (snmpget/snmpwalk) or a basic agent w/ standard MIBs for the host it's running on, so I was unaware of its extensibility. And it even looks like it ships with a Perl module. That sounds like a perfect solution; thanks for pointing me in the right direction.
-- Nathan
+1 On May 23, 2016 4:53 PM, "Eric Kuhnke" <eric.kuhnke@gmail.com> wrote:
This doesn't scale on a large cacti installation with hundreds of hosts and 60-second poller intervals. Cacti data input method scripts spawn a new php worker for each data acquisition target (they do NOT use the 'spine' SNMP poller).
Exposing the data via SNMP on the host to be monitored distributes the CPU load individually onto each host (example: an 'extend' script in the snmpd.conf which runs "curl http://localhost/server-status" for apache2 status and parses the results) rather than centralizing it on the cacti host.
This allows cacti or opennms or anything else to poll the hosts to be monitored via something that scales better than php script workers, using the 'spine' SNMP data acquisition method and the equivalent in other snmp polling platforms.
On Sat, May 21, 2016 at 6:03 AM, Eric Lindsjö <eric@emj.se> wrote:
Hi Nathan,
You should probably write a cacti script to ingest the data instead of this SNMP proxy thing. Writing scripts to ingest data into cacti is simple, you just need to output the values you want in key: value format and then do some clicking in cacti. There are good docs for how to do this.
-- emj
On 05/21/2016 08:11 AM, Nathan Anderson wrote:
Hey, thanks guys! I had never really looked that deeply into Net-SNMP and had only ever installed it either to use as a client (snmpget/snmpwalk) or a basic agent w/ standard MIBs for the host it's running on, so I was unaware of its extensibility. And it even looks like it ships with a Perl module. That sounds like a perfect solution; thanks for pointing me in the right direction.
-- Nathan
On 05/23/2016 05:51 PM, Eric Kuhnke wrote:
Exposing the data via SNMP on the host to be monitored distributes the CPU load individually onto each host
So much this. Most importantly, it removes the fork/exec overhead from the monitoring server.
This allows cacti or opennms or anything else to poll the hosts to be monitored via something that scales better than php script workers, using the 'spine' SNMP data acquisition method and the equivalent in other snmp polling platforms.
Point of fact for OpenNMS users: all our collectors are in-process, so in cases where SNMP is impractical you can use the XML/JSON or JDBC or WS-Management collector and still achieve great scale with a single server. But SNMP still scales and interoperates like nothing else. -jeff
Eric Kuhnke <eric.kuhnke@gmail.com> writes:
http://www.adventuresinoss.com/2009/09/30/the-many-uses-of-net-snmp/
Ha! I've never seen that article, thanks for pointing it out. Note that the performance of Net-SNMP's extensibility mechanisms should way into the decision. The fastest backend needs to be written in C, and embedded perl is an easy second. Beyond that, pass_persist is somewhere in the middle and pass/extend/other execs are the slowest because of the need to exec a command for every incoming request which is expensive. Great for bootstrapping and testing, but in the long run look to the better coding solutions. Tutorials for most of these exist: http://www.net-snmp.org/wiki/index.php/Tutorials#Coding_Tutorials [as a point of history: Net-SNMP has always been very extensible since it was started based on my need to add extensibility to an agent way back in 1995-ish in order to monitor some special cases on a map with HP OV (as it was known back then)] -- Wes Hardaker My Pictures: http://capturedonearth.com/ My Thoughts: http://blog.capturedonearth.com/
On Fri, May 20, 2016 at 04:43:07PM -0700, Nathan Anderson wrote:
Is anybody out there aware of a piece of software that can take data from an arbitrary source and then present it, using a MIB or set of OIDs of your choosing, as an SNMP-interrogatable device?
Many, many years ago, I wrote a fairly minimal SNMP agent in Ruby (http://theshed.hezmatt.org/rubysnmpd/index.html), which was explicitly designed to be easily extensible (it didn't do anything useful by itself). If your team's into Ruby, that might be a possibility. Otherwise, net-snmp does come with some fairly decent extensibility capabilities these days. It has a variety of mechanisms by which you can feed extra data into it, and in general I'd recommend going down that route. - Matt
On 5/20/2016 7:43 PM, Nathan Anderson wrote:
'lo all,
Is anybody out there aware of a piece of software that can take data from an arbitrary source and then present it, using a MIB or set of OIDs of your choosing, as an SNMP-interrogatable device?
We have some CPE that supports SNMP, but considers it to be a mutually-exclusive "remote management" protocol such that if you use another supported method for deployment and provisioning (e.g., TR-069), you cannot have both that AND SNMP enabled simultaneously. It's one or the other.
We currently monitor and graph some device stats for these CPE with Cacti, but we want to be able to provision using a TR-069 ACS. The ACS can collect some of the same data we are graphing right now, but cannot present it in a fashion that is nearly as useful as the way Cacti/RRDtool does (not to mention the staff is already used to navigating Cacti). We know what SQL database table the stats are being stored in by the ACS, though, so my thought was that there must be some way that we can have a host respond to SNMP gets and then have it turn around and collect the value to be returned from a database. Basically, an ODBC -> SNMP proxy. We'd then point Cacti at that IP instead of the individual CPEs. But I can't seem to find anything like this. I would move away from this CPE vendor. Your solution has merit in the short term, but monitoring through the ACS is pointlessly putting more load on a server that already has it's own responsibilities. You can't scale out with this. Well, not without deploying more ACS servers.. which are a bit more heavyweight than SNMP pollers.
As mentioned already, net-snmp can do this easily enough. The biggest problem you'll face is figuring out how you want to name OIDs to match up to each CPE and the elements you're graphing. .... you might be better off pulling the data out of the database via SQL queries to a remote host and proxying the data there. Or possibly have cacti run the SQL query directly. It looks like they have many general (non SNMP) templates that you could use to base it on. http://docs.cacti.net/templates http://forums.cacti.net/viewtopic.php?f=12&t=15067
Thanks,
-- Nathan
Thanks, Robert
On Friday May 20, 2016 @ 21:45, Robert Drake <rdrake@direcpath.com> wrote:
I would move away from this CPE vendor.
I'm not thrilled with it either, but at this moment in time, this is easier said than done for many unfortunately good and unavoidable reasons. We will see how the future plays out, though.
[...] Or possibly have cacti run the SQL query directly. It looks like they have many general (non SNMP) templates that you could use to base it on.
Another interesting suggestion & possibility. Thanks. -- Nathan
participants (8)
-
Eric Kuhnke -
Eric Lindsjö -
Jeff Gehlbach -
Josh Reynolds -
Matt Palmer -
Nathan Anderson -
Robert Drake -
Wes Hardaker