Flow collection and analysis
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool? Thanks!
I’ll be the minority voice here - I have been very happy with Argus (qosient.com) but it does not have a GUI and that seems to be a factor for some Dave
On Jan 25, 2022, at 10:46 AM, David Bass <davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
Thanks!
My company uses Auvik. It's easy to setup but needs some tuning and is furiously expensive. The traffic analysis is pretty good and can be export for you to use in other tools. If netflow is all you are using it for look elsewhere regardless of what a sales person sales. Kevin On Tue, Jan 25, 2022, at 10:46 AM, David Bass wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
Thanks!
If your looking to go low-cost (free) try: 1) Carnegie/Mellon's very robust, flexible and efficient collector analyzer (command line): SiLK - https://tools.netsa.cert.org/silk 2) FlowViewer - A comprehensive web-based user interface for SiLK which provides textual, graphical analysis, long term tracking and dashboard: http:flowviewer.net or https://sourceforge.net/projects/flowviewer Best! Joe On 1/25/2022 10:46 AM, David Bass wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
Thanks!
On Tue, 25 Jan 2022 11:46:14 -0400 David Bass <davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
Two open source tools you might consider: nfdump <https://github.com/phaag/nfdump> pmacct <http://www.pmacct.net/> John
On Tuesday, January 25th, 2022 at 15:46, David Bass <davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
Thanks!
Not a suggestion, but a question .... Curious to know if anyone (apart from Cloudflare, obvs !) is using Goflow ? (https://github.com/cloudflare/goflow)
We use, depending on the situation, Intermapper, PRTG, and NTop. Intermapper has the most powerful viewing app, but it’s expensive in that you have to pay for each flow collector. It’s an actual app (Windows, Mac, and Linux), rather than a web-based interface, so they can do more tricks with the GUI, like drill down and sorting. PRTG includes its web-based flow collector and viewer for free, and there is even a free 100-sensor edition of the product that lets you use just the flow stuff fir free forever. NTop is an open source web-based flow sensor and viewer, with a combo paid license model specifically for the flow collection. It connects directly to a mirror port and sucks up the flow info, where is the other products required to have some hardware device that exports flows. But you can get dirt cheap used Cisco routers that do this, such as the 1941, which although bulky do the job for a few hundred bucks. Once you get into 10 Gb speeds though you need dedicated hardware sensors in newer gear, which is pretty pricey. But if you have 10 Gb traffic you can afford it :-) Ntop also has a commercial arm called Nbox, Which has a range of hardware based ready to go solutions. However it’s all supported out of Italy, and pretty much by one guy, so we’ve had uneven results with customers that purchased it. -mel
On Jan 25, 2022, at 8:30 AM, Laura Smith via NANOG <nanog@nanog.org> wrote:
On Tuesday, January 25th, 2022 at 15:46, David Bass <davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
Thanks!
Not a suggestion, but a question ....
Curious to know if anyone (apart from Cloudflare, obvs !) is using Goflow ? (https://github.com/cloudflare/goflow)
Hi, There is also Elastiflow https://docs.elastiflow.com/docs/ https://github.com/robcowart/elastiflow. Cordialement / Best regards Pierre Lancastre Le mar. 25 janv. 2022 à 17:45, Mel Beckman <mel@beckman.org> a écrit :
We use, depending on the situation, Intermapper, PRTG, and NTop.
Intermapper has the most powerful viewing app, but it’s expensive in that you have to pay for each flow collector. It’s an actual app (Windows, Mac, and Linux), rather than a web-based interface, so they can do more tricks with the GUI, like drill down and sorting.
PRTG includes its web-based flow collector and viewer for free, and there is even a free 100-sensor edition of the product that lets you use just the flow stuff fir free forever.
NTop is an open source web-based flow sensor and viewer, with a combo paid license model specifically for the flow collection. It connects directly to a mirror port and sucks up the flow info, where is the other products required to have some hardware device that exports flows. But you can get dirt cheap used Cisco routers that do this, such as the 1941, which although bulky do the job for a few hundred bucks. Once you get into 10 Gb speeds though you need dedicated hardware sensors in newer gear, which is pretty pricey. But if you have 10 Gb traffic you can afford it :-)
Ntop also has a commercial arm called Nbox, Which has a range of hardware based ready to go solutions. However it’s all supported out of Italy, and pretty much by one guy, so we’ve had uneven results with customers that purchased it.
-mel
On Jan 25, 2022, at 8:30 AM, Laura Smith via NANOG <nanog@nanog.org> wrote:
On Tuesday, January 25th, 2022 at 15:46, David Bass < davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
Thanks!
Not a suggestion, but a question ....
Curious to know if anyone (apart from Cloudflare, obvs !) is using Goflow ? (https://github.com/cloudflare/goflow)
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, January 25th, 2022 at 16:44, Mel Beckman <mel@beckman.org> wrote:
We use, depending on the situation, Intermapper, PRTG, and NTop.
PRTG includes its web-based flow collector and viewer for free, and there is even a free 100-sensor edition of the product that lets you use just the flow stuff fir free forever.
Once upon a time we used to use PRTG. Nothing bad to say about it as a product, apart from the fact it only runs on Windows. It is an unfortunate fact in today's world with Microsoft's desire to push everyone to Azure and make on-prem licensing increasingly unattractive.
On Tue, Jan 25, 2022 at 10:53 AM David Bass <davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
a question not asked, and answer not provided here, is: "What are you actually trying to do with the netflow?" Answers of the form: "Dos detection and mitigation planning" "Discover peering options/opportunities" "billing customers" "traffic analysis for future network planning" "abuse monitoring/management/investigations" "pretty noc graphs" are helpful.. I'm sure other answers would as well.. but: "how do you collect?" is "with a collector" and isn't super helpful if the collector can't feed into the tooling / infrastructure / long-term goal you have.
Most of these things, yes. Add: Troubleshooting/operational support Customer reporting On Tue, Jan 25, 2022 at 1:38 PM Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Tue, Jan 25, 2022 at 10:53 AM David Bass <davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
a question not asked, and answer not provided here, is: "What are you actually trying to do with the netflow?"
Answers of the form: "Dos detection and mitigation planning" "Discover peering options/opportunities" "billing customers" "traffic analysis for future network planning" "abuse monitoring/management/investigations" "pretty noc graphs"
are helpful.. I'm sure other answers would as well.. but: "how do you collect?" is "with a collector" and isn't super helpful if the collector can't feed into the tooling / infrastructure / long-term goal you have.
Elastiflow is pretty cool. https://www.elastiflow.com or the old open source version: https://github.com/robcowart/elastiflow You can pretty much do the same thing with Elastic’s filebeat (https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netfl...). Pmacct is also good for grabbing netflow http://www.pmacct.net and sending it somewhere (file, database, kafka, etc.) You can also grab BMP and streaming telemetry with it. If you’re looking for open source DDoS detection using netflow, check out https://github.com/pavel-odintsov/fastnetmon Shameless plug, check out my tool to look for spoofed UDP amplification request traffic coming into your network https://github.com/racompton/tattle-tale FYI, you can send netflow to multiple collectors with https://github.com/sleinen/samplicator -Rich From: NANOG <nanog-bounces+rich.compton=charter.com@nanog.org> on behalf of David Bass <davidbass570@gmail.com> Date: Tuesday, January 25, 2022 at 11:06 AM To: Christopher Morrow <morrowc.lists@gmail.com> Cc: NANOG list <nanog@nanog.org> Subject: [EXTERNAL] Re: Flow collection and analysis CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance. Most of these things, yes. Add: Troubleshooting/operational support Customer reporting On Tue, Jan 25, 2022 at 1:38 PM Christopher Morrow <morrowc.lists@gmail.com<mailto:morrowc.lists@gmail.com>> wrote: On Tue, Jan 25, 2022 at 10:53 AM David Bass <davidbass570@gmail.com<mailto:davidbass570@gmail.com>> wrote: Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool? a question not asked, and answer not provided here, is: "What are you actually trying to do with the netflow?" Answers of the form: "Dos detection and mitigation planning" "Discover peering options/opportunities" "billing customers" "traffic analysis for future network planning" "abuse monitoring/management/investigations" "pretty noc graphs" are helpful.. I'm sure other answers would as well.. but: "how do you collect?" is "with a collector" and isn't super helpful if the collector can't feed into the tooling / infrastructure / long-term goal you have. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.
On Tuesday, January 25th, 2022 at 23:50, Compton, Rich A <Rich.Compton@charter.com> wrote:
You can pretty much do the same thing with Elastic’s filebeat (https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netfl...).
Has Elastic decided to join the rest of the world in the 21st century yet ? Last time I looked at it (not too many years ago) they had no TLS support. Bit of a show-stopper in today's security environment.
elastiflow is extremely easy to run on an httpd listening only on localhost and proxy behind a simple nginx TLS1.2/1.3 only configuration listening on port 443. as are a number of other tools. On Tue, 25 Jan 2022 at 16:06, Laura Smith via NANOG <nanog@nanog.org> wrote:
On Tuesday, January 25th, 2022 at 23:50, Compton, Rich A < Rich.Compton@charter.com> wrote:
You can pretty much do the same thing with Elastic’s filebeat ( https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netfl...).
Has Elastic decided to join the rest of the world in the 21st century yet ?
Last time I looked at it (not too many years ago) they had no TLS support. Bit of a show-stopper in today's security environment.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, January 26th, 2022 at 11:08, Eric Kuhnke <eric.kuhnke@gmail.com> wrote:
elastiflow is extremely easy to run on an httpd listening only on localhost and proxy behind a simple nginx TLS1.2/1.3 only configuration listening on port 443.
I don't know about anyone else here, but frankly in 2022 TLS support should be a first class citizen. If I have to mess around with running something else as a proxy in front of it then that's the end of my software evaluation. Crypto is no longer "nice to have" option these days.
Why is it even necessary for such a function? ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Laura Smith via NANOG" <nanog@nanog.org> To: "nanog@nanog.org list" <nanog@nanog.org> Sent: Wednesday, January 26, 2022 7:17:09 AM Subject: Re: [EXTERNAL] Re: Flow collection and analysis ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, January 26th, 2022 at 11:08, Eric Kuhnke <eric.kuhnke@gmail.com> wrote:
elastiflow is extremely easy to run on an httpd listening only on localhost and proxy behind a simple nginx TLS1.2/1.3 only configuration listening on port 443.
I don't know about anyone else here, but frankly in 2022 TLS support should be a first class citizen. If I have to mess around with running something else as a proxy in front of it then that's the end of my software evaluation. Crypto is no longer "nice to have" option these days.
Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:
Why is it [TLS] even necessary for such a function?
confidentiality and integrity, even if you do not care about authentication. I am surprised that question is asked. The fewer things that are left unprotected, the better for everyone. those with concern about erosion of their privacy and human rights benefit from everything being protected, everywhere for everyone.
People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans DIY automobile security, which started with a screwed-on metal hasp and padlock, and then continued to a range of additional “layers”. Not “defense-in-depth”, merely unwarranted “complexity-in-depth”: https://youtu.be/CCl_KxGLgOA TLS is a standardized, fully open-source package that can be integrated into even tiny IoT devices (witness this $10 WiFi module https://www.adafruit.com/product/4201<https://www.adafruit.com/product/4201>). The argument that people who want intrinsically secure products can just bolt-on their own security are missing the point entirely. Every web-enabled product should be required to implement TLS and then let custiners decide when they want to enable it. Vendors who are so weak that they can’t should have their products go straight into /dev/null. -mel via cell On Jan 26, 2022, at 6:51 AM, heasley <heas@shrubbery.net> wrote: Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett: Why is it [TLS] even necessary for such a function? confidentiality and integrity, even if you do not care about authentication. I am surprised that question is asked. The fewer things that are left unprotected, the better for everyone. those with concern about erosion of their privacy and human rights benefit from everything being protected, everywhere for everyone.
While I agree that, yes everything SHOULD support TLS, there's a perfectly good reason for terminating TLS in something like (nginx/caddy/apache/etc): X number of things supporting TLS on their web interface means X number of ways of configuring TLS. If I terminate it on nginx, there's only a single way: the nginx config, which is then farily easily leveraged into having a single set allowed protocols and ciphers. On Wed, Jan 26, 2022, at 9:33 AM, Mel Beckman wrote:
People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans DIY automobile security, which started with a screwed-on metal hasp and padlock, and then continued to a range of additional “layers”. Not “defense-in-depth”, merely unwarranted “complexity-in-depth”:
TLS is a standardized, fully open-source package that can be integrated into even tiny IoT devices (witness this $10 WiFi module https://www.adafruit.com/product/4201). The argument that people who want intrinsically secure products can just bolt-on their own security are missing the point entirely. Every web-enabled product should be required to implement TLS and then let custiners decide when they want to enable it. Vendors who are so weak that they can’t should have their products go straight into /dev/null.
-mel via cell
On Jan 26, 2022, at 6:51 AM, heasley <heas@shrubbery.net> wrote:
Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:
Why is it [TLS] even necessary for such a function?
confidentiality and integrity, even if you do not care about authentication. I am surprised that question is asked.
The fewer things that are left unprotected, the better for everyone. those with concern about erosion of their privacy and human rights benefit from everything being protected, everywhere for everyone.
Nick, you can always choose to use nginx if you like, but there’s no reason anyone else should be forced to. -mel On Jan 26, 2022, at 7:55 AM, Nick Suan via NANOG <nanog@nanog.org> wrote: While I agree that, yes everything SHOULD support TLS, there's a perfectly good reason for terminating TLS in something like (nginx/caddy/apache/etc): X number of things supporting TLS on their web interface means X number of ways of configuring TLS. If I terminate it on nginx, there's only a single way: the nginx config, which is then farily easily leveraged into having a single set allowed protocols and ciphers. On Wed, Jan 26, 2022, at 9:33 AM, Mel Beckman wrote: People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans DIY automobile security, which started with a screwed-on metal hasp and padlock, and then continued to a range of additional “layers”. Not “defense-in-depth”, merely unwarranted “complexity-in-depth”: https://youtu.be/CCl_KxGLgOA TLS is a standardized, fully open-source package that can be integrated into even tiny IoT devices (witness this $10 WiFi module https://www.adafruit.com/product/4201<https://www.adafruit.com/product/4201>). The argument that people who want intrinsically secure products can just bolt-on their own security are missing the point entirely. Every web-enabled product should be required to implement TLS and then let custiners decide when they want to enable it. Vendors who are so weak that they can’t should have their products go straight into /dev/null. -mel via cell On Jan 26, 2022, at 6:51 AM, heasley <heas@shrubbery.net> wrote: Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett: Why is it [TLS] even necessary for such a function? confidentiality and integrity, even if you do not care about authentication. I am surprised that question is asked. The fewer things that are left unprotected, the better for everyone. those with concern about erosion of their privacy and human rights benefit from everything being protected, everywhere for everyone.
Not at all, what I'm recommending is that people who develop something that is specialized (like netflow analysis software) don't need to expend the person-hours and extensive development time to implement something that has already been better implemented by people who are httpd specialists. The amount of possible design complexities and security risks that go into shipping a 'stable' versio of apache2 or nginx are beyond the scope of any small to medium sized non-httpd-related opens source software project. Let the apache2 or nginx developers handle that. It's like saying that because a piece of software communicates with something externally by SMTP, either inbound or outbound email or both, some software developer should take the time to re-implemnt and write from scratch their own SMTP, rather than relaying mail via a postfix daemon running on the same server. Or because you have a piece of software that queries something over SNMP, don't use the perfectly good ISC SNMP packages that exist for centos or debian to issue snmpgets, but write from scratch your own snmp poller. On Wed, 26 Jan 2022 at 07:34, Mel Beckman <mel@beckman.org> wrote:
People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans DIY automobile security, which started with a screwed-on metal hasp and padlock, and then continued to a range of additional “layers”. Not “defense-in-depth”, merely unwarranted “complexity-in-depth”:
TLS is a standardized, fully open-source package that can be integrated into even tiny IoT devices (witness this $10 WiFi module https://www.adafruit.com/product/4201 <https://www.adafruit.com/product/4201>). The argument that people who want intrinsically secure products can just bolt-on their own security are missing the point entirely. Every web-enabled product should be required to implement TLS and then let custiners decide when they want to enable it. Vendors who are so weak that they can’t should have their products go straight into /dev/null.
-mel via cell
On Jan 26, 2022, at 6:51 AM, heasley <heas@shrubbery.net> wrote:
Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:
Why is it [TLS] even necessary for such a function?
confidentiality and integrity, even if you do not care about authentication. I am surprised that question is asked.
The fewer things that are left unprotected, the better for everyone. those with concern about erosion of their privacy and human rights benefit from everything being protected, everywhere for everyone.
But nobody asked for anything from scratch Eric. Open SSL is it complete ready to integrate package. Any developer worth his salt should be able to put it on any web application. In addition to OpenSSL, there are very compact commercial SSL libraries such as Mocana NanoSSL and wolfSSL, if you want to really simplify the process. Nobody need write any crypto software at all, and the extensive manhours you claim are not real. -mel On Jan 27, 2022, at 6:26 PM, Eric Kuhnke <eric.kuhnke@gmail.com> wrote: Not at all, what I'm recommending is that people who develop something that is specialized (like netflow analysis software) don't need to expend the person-hours and extensive development time to implement something that has already been better implemented by people who are httpd specialists. The amount of possible design complexities and security risks that go into shipping a 'stable' versio of apache2 or nginx are beyond the scope of any small to medium sized non-httpd-related opens source software project. Let the apache2 or nginx developers handle that. It's like saying that because a piece of software communicates with something externally by SMTP, either inbound or outbound email or both, some software developer should take the time to re-implemnt and write from scratch their own SMTP, rather than relaying mail via a postfix daemon running on the same server. Or because you have a piece of software that queries something over SNMP, don't use the perfectly good ISC SNMP packages that exist for centos or debian to issue snmpgets, but write from scratch your own snmp poller. On Wed, 26 Jan 2022 at 07:34, Mel Beckman <mel@beckman.org<mailto:mel@beckman.org>> wrote: People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans DIY automobile security, which started with a screwed-on metal hasp and padlock, and then continued to a range of additional “layers”. Not “defense-in-depth”, merely unwarranted “complexity-in-depth”: https://youtu.be/CCl_KxGLgOA TLS is a standardized, fully open-source package that can be integrated into even tiny IoT devices (witness this $10 WiFi module https://www.adafruit.com/product/4201<https://www.adafruit.com/product/4201>). The argument that people who want intrinsically secure products can just bolt-on their own security are missing the point entirely. Every web-enabled product should be required to implement TLS and then let custiners decide when they want to enable it. Vendors who are so weak that they can’t should have their products go straight into /dev/null. -mel via cell On Jan 26, 2022, at 6:51 AM, heasley <heas@shrubbery.net<mailto:heas@shrubbery.net>> wrote: Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett: Why is it [TLS] even necessary for such a function? confidentiality and integrity, even if you do not care about authentication. I am surprised that question is asked. The fewer things that are left unprotected, the better for everyone. those with concern about erosion of their privacy and human rights benefit from everything being protected, everywhere for everyone.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Friday, January 28th, 2022 at 03:55, Mel Beckman <mel@beckman.org> wrote:
But nobody asked for anything from scratch Eric. Open SSL is it complete ready to integrate package. Any developer worth his salt should be able to put it on any web application. In addition to OpenSSL, there are very compact commercial SSL libraries such as Mocana NanoSSL and wolfSSL, if you want to really simplify the process.
Yup. Every single modern programming language out there has a crypto library. The high-level languages (e.g. Go) have crypto built into the standard library. The low-level languages (e.g C or Rust) all have at least one or more well supported third party crypto libraries (e.g. for C there's OpenSSL, GnuTLS, LibreSSL, Boring SSL, Mbed TLS ... and those are the ones that I can think of off the top of my head). There's no need to do any crypto "from scratch", and indeed you SHOULD NOT.
Why DNS are still travelling in clear text? The software running the DNS services worldwide are probably written in C or any languages you mentioned below. Why don't they just strap a libressl on DNS or NanoSSL? Okay, there is DNS over https. I don't know the stats, but I doubt it's close to 100% adoption worldwide. I don't understand what is the issue about SSL, zero trust has anything to do about collecting flows. Do I need ssl to run shell commands in my terminal to read flows? Not really. Do I need to strap ssl on grep, notepad and excel? I'm not sure how could one do that. When you see the flows of your customers, you have access to how many times did they use Netflix, facebook and anything you could think of because these people are querying DNS to reach these... in clear text. They are also hitting servers that are well known. I would worry more about who is reading the flows of my business' customers than these flows being not protected by SSL. They are anyway in a highly secure environment with zero trust. So if you don't like elastiflow or any software that are not being protected by SSL, then maybe switch off your computer. Protonmail won't help you to keep your digital life secure. This email was sent by a secure infrastructure using TLS 1.2 and clear text dns. Thank you Jean -----Original Message----- From: NANOG <nanog-bounces+jean=ddostest.me@nanog.org> On Behalf Of Laura Smith via NANOG Sent: January 28, 2022 5:15 AM To: Mel Beckman <mel@beckman.org> Cc: nanog@nanog.org list <nanog@nanog.org> Subject: Re: [EXTERNAL] Re: Flow collection and analysis ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Friday, January 28th, 2022 at 03:55, Mel Beckman <mel@beckman.org> wrote:
But nobody asked for anything from scratch Eric. Open SSL is it complete ready to integrate package. Any developer worth his salt should be able to put it on any web application. In addition to OpenSSL, there are very compact commercial SSL libraries such as Mocana NanoSSL and wolfSSL, if you want to really simplify the process.
Yup. Every single modern programming language out there has a crypto library. The high-level languages (e.g. Go) have crypto built into the standard library. The low-level languages (e.g C or Rust) all have at least one or more well supported third party crypto libraries (e.g. for C there's OpenSSL, GnuTLS, LibreSSL, Boring SSL, Mbed TLS ... and those are the ones that I can think of off the top of my head). There's no need to do any crypto "from scratch", and indeed you SHOULD NOT.
My colleagues developers started adding valid let's encrypt certs everywhere. Now I have multiple NAT entry points for build-servers in my VPC because of the renewal frequency. I feel less secure with them adding valid SSL certs everywhere that runs on a PRIVATE NETWORK. It's just dumb reasoning, and the CTO agreed with them. They are all gone by now, but their legacy remains. Now I have to find all those certs and replace them with 10 year self-signed, and add --no-check-certificates flags in their http client requests. All NAT entrypoints are gone. I'm feeling safe now. On Fri, 28 Jan 2022 at 13:26, Jean St-Laurent via NANOG <nanog@nanog.org> wrote:
Why DNS are still travelling in clear text?
The software running the DNS services worldwide are probably written in C or any languages you mentioned below.
Why don't they just strap a libressl on DNS or NanoSSL?
Okay, there is DNS over https. I don't know the stats, but I doubt it's close to 100% adoption worldwide.
I don't understand what is the issue about SSL, zero trust has anything to do about collecting flows. Do I need ssl to run shell commands in my terminal to read flows? Not really. Do I need to strap ssl on grep, notepad and excel? I'm not sure how could one do that.
When you see the flows of your customers, you have access to how many times did they use Netflix, facebook and anything you could think of because these people are querying DNS to reach these... in clear text. They are also hitting servers that are well known.
I would worry more about who is reading the flows of my business' customers than these flows being not protected by SSL. They are anyway in a highly secure environment with zero trust.
So if you don't like elastiflow or any software that are not being protected by SSL, then maybe switch off your computer. Protonmail won't help you to keep your digital life secure.
This email was sent by a secure infrastructure using TLS 1.2 and clear text dns.
Thank you
Jean
-----Original Message----- From: NANOG <nanog-bounces+jean=ddostest.me@nanog.org> On Behalf Of Laura Smith via NANOG Sent: January 28, 2022 5:15 AM To: Mel Beckman <mel@beckman.org> Cc: nanog@nanog.org list <nanog@nanog.org> Subject: Re: [EXTERNAL] Re: Flow collection and analysis
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Friday, January 28th, 2022 at 03:55, Mel Beckman <mel@beckman.org> wrote:
But nobody asked for anything from scratch Eric. Open SSL is it complete ready to integrate package. Any developer worth his salt should be able to put it on any web application. In addition to OpenSSL, there are very compact commercial SSL libraries such as Mocana NanoSSL and wolfSSL, if you want to really simplify the process.
Yup. Every single modern programming language out there has a crypto library.
The high-level languages (e.g. Go) have crypto built into the standard library.
The low-level languages (e.g C or Rust) all have at least one or more well supported third party crypto libraries (e.g. for C there's OpenSSL, GnuTLS, LibreSSL, Boring SSL, Mbed TLS ... and those are the ones that I can think of off the top of my head).
There's no need to do any crypto "from scratch", and indeed you SHOULD NOT.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, January 26th, 2022 at 14:49, heasley <heas@shrubbery.net> wrote:
confidentiality and integrity, even if you do not care about authentication.
I am surprised that question is asked.
Indeed. And to add the obvious to the obvious observation above, in certain industries and/or jurisdictions its effectively mandatory to encrypt the whole stack. And that's before we start talking about the "modern" "Zero Trust" mentality (which incidentally is nothing new and has been around since at least 2004 with the Jericho Forum... but I guess "Zero Trust" sounds cooler).
Once upon a time, Laura Smith <n5d9xq3ti233xiyif2vp@protonmail.ch> said:
I don't know about anyone else here, but frankly in 2022 TLS support should be a first class citizen.
If I have to mess around with running something else as a proxy in front of it then that's the end of my software evaluation.
Crypto is no longer "nice to have" option these days.
Having every thing under the sun trying to implement the complexities of TLS leads to lots of failures and security issues... so lots of web things are designed to be simple and only implement HTTP, listen on localhost, and let a well-optimized front-end (e.g. nginx) handle the crypto side (as well as all the weird things browsers do). It also makes it easier from an system admin point of view, because handling cert updates in nginx is easy and well-known, so you don't have to figure out 27 different ways alternate software handles certs and updates. -- Chris Adams <cma@cmadams.net>
If the purpose of the software is not to be a dedicated purpose http daemon, use something that already exists with a deep feature set that can be configured as needed for the purpose, such as apache2 with openssl or nginx. It's not reasonable to expect that the developers of elastiflow reinvent the wheel and write their own httpd with TLS support, if it can be easily put "behind" apache2 or nginx. The risks of having people who aren't full time httpd specialists write their own http daemon and mitigate every possible security risk in a TLS setup are greater than using what already exists. It's a one page size configuration file in nginx. On Wed, 26 Jan 2022 at 05:17, Laura Smith via NANOG <nanog@nanog.org> wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, January 26th, 2022 at 11:08, Eric Kuhnke < eric.kuhnke@gmail.com> wrote:
elastiflow is extremely easy to run on an httpd listening only on localhost and proxy behind a simple nginx TLS1.2/1.3 only configuration listening on port 443.
I don't know about anyone else here, but frankly in 2022 TLS support should be a first class citizen.
If I have to mess around with running something else as a proxy in front of it then that's the end of my software evaluation.
Crypto is no longer "nice to have" option these days.
Samplicator is a nifty tool. --John On 1/25/22 16:50, Compton, Rich A wrote:
Elastiflow is pretty cool. https://www.elastiflow.com or the old open source version: https://github.com/robcowart/elastiflow
You can pretty much do the same thing with Elastic’s filebeat (https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netfl...).
Pmacct is also good for grabbing netflow http://www.pmacct.net and sending it somewhere (file, database, kafka, etc.) You can also grab BMP and streaming telemetry with it.
If you’re looking for open source DDoS detection using netflow, check out https://github.com/pavel-odintsov/fastnetmon
Shameless plug, check out my tool to look for spoofed UDP amplification request traffic coming into your network https://github.com/racompton/tattle-tale
FYI, you can send netflow to multiple collectors with https://github.com/sleinen/samplicator
-Rich
*From: *NANOG <nanog-bounces+rich.compton=charter.com@nanog.org> on behalf of David Bass <davidbass570@gmail.com> *Date: *Tuesday, January 25, 2022 at 11:06 AM *To: *Christopher Morrow <morrowc.lists@gmail.com> *Cc: *NANOG list <nanog@nanog.org> *Subject: *[EXTERNAL] Re: Flow collection and analysis
*CAUTION:*The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance.
Most of these things, yes.
Add:
Troubleshooting/operational support
Customer reporting
On Tue, Jan 25, 2022 at 1:38 PM Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Tue, Jan 25, 2022 at 10:53 AM David Bass <davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
a question not asked, and answer not provided here, is: "What are you actually trying to do with the netflow?"
Answers of the form: "Dos detection and mitigation planning" "Discover peering options/opportunities" "billing customers"
"traffic analysis for future network planning"
"abuse monitoring/management/investigations"
"pretty noc graphs"
are helpful.. I'm sure other answers would as well.. but: "how do you collect?" is "with a collector" and isn't super helpful if the collector can't feed into the tooling / infrastructure / long-term goal you have.
The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.
I agree with you. The tool doesn’t really matter. Windows, linux, cloud or not. It’s really important to first understand what are you trying to solve or improve? If this step is forgotten, then it will just be another tool to support to add in your long list of useless tools. My personal favorites are a mix of: * Ntop with PF_RING enabled. * Nfdump * Elasticsearch I’m sure all the other tools are also very good. Csv in excel or grep/awk could also work if you know exactly what you’re looking for. 😉 Jean From: NANOG <nanog-bounces+jean=ddostest.me@nanog.org> On Behalf Of Christopher Morrow Sent: January 25, 2022 12:38 PM To: David Bass <davidbass570@gmail.com> Cc: <nanog@nanog.org> <nanog@nanog.org> Subject: Re: Flow collection and analysis On Tue, Jan 25, 2022 at 10:53 AM David Bass <davidbass570@gmail.com <mailto:davidbass570@gmail.com> > wrote: Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool? a question not asked, and answer not provided here, is: "What are you actually trying to do with the netflow?" Answers of the form: "Dos detection and mitigation planning" "Discover peering options/opportunities" "billing customers" "traffic analysis for future network planning" "abuse monitoring/management/investigations" "pretty noc graphs" are helpful.. I'm sure other answers would as well.. but: "how do you collect?" is "with a collector" and isn't super helpful if the collector can't feed into the tooling / infrastructure / long-term goal you have.
Are you asking for commercial solutions? Free solutions? Open Source?
On Jan 25, 2022, at 10:46 AM, David Bass <davidbass570@gmail.com> wrote:
Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?
Thanks!
participants (20)
-
Chris Adams -
Christopher Morrow -
Compton, Rich A -
Dave -
David Bass -
Eric Kuhnke -
heasley -
Jean St-Laurent -
Joe Loiacono -
Joel Esler -
John Kristoff -
John Schiel -
Kevin Glass -
Laura Smith -
Marcel Mitsuto -
Mark Tinka -
Mel Beckman -
Mike Hammett -
Nick Suan -
Pierre LANCASTRE