Hello, I'm currently writing a paper for school and I talk about net neutrality which brings the subject of NetFlow/IPFIX. Should those protocols be considered as tools to perform DPI ? Thanks, Antoine.
On 05/07/14 15:11 +0200, Antoine Meillet wrote:
Hello,
I'm currently writing a paper for school and I talk about net neutrality which brings the subject of NetFlow/IPFIX.
Should those protocols be considered as tools to perform DPI ?
That question can be taken a couple of ways. Netflow is useful for calculating information useful to providers and operators through sampling of data on high bandwidth links, where performing DPI is not feasible or desired. It is not a robust solution for DPI - or analysis of higher layer packet data, which is typically performed by a mirrored interface or an inline box/firewall that can perform high speed forwarding. -- Dan White
On May 7, 2014, at 8:11 PM, Antoine Meillet <antoine.meillet@gmail.com> wrote:
Should those protocols be considered as tools to perform DPI ?
No - they're flow telemetry exported by routers and switches, and they provide layer-4 information. It's possible with Cisco Flexible NetFlow and with PSAMP exported over IPFIX to get packet contents; however, few if any collection/analysis systems utilize either extended telemetry format, to date. I've never seen either implemented in a production network. NetFlow and IPFIX are primarily used for security purposes such as DDoS detection/classification/traceback and botnet C&C identification; for traffic engineering analysis; capacity planning analysis; for troubleshooting; and for billing purposes. Flow telemetry is an essential tool that ISPs and larger enterprises utilize in order to get a view into their network traffic, because it scales for large networks - and it does so because it doesn't typically include packet payloads, just the layer-4 information. It's sort of like a near-time mobile phone bill for the network. 'DPI' generally (but not always) refers to devices which are placed inline and perform full multi-packet payload reassembly and inspection. The term has been used (and misused) so broadly as to becoming essentially meaningless. NetFlow and IPFIX are merely telemetry formats used by network engineers for the purposes noted above. This presentation talks about how NetFlow is used by network operators: <https://app.box.com/s/mnshn99c13uekrggy99b> Network neutrality is largely an issue of policy and of economics, not of technology, per se. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Luck is the residue of opportunity and design. -- John Milton
Another role for IPFIX/NetFlow in the context of DPI (on top of PSAMP that was already mentioned by Roland) is to serve as a transport mechanism to travel flow data along with their DPI classification from probes to remote collectors, for persistent storage, analysis, etc. This model is supported on the export side by Cisco with their NetFlow/NBAR integration and on the collection side by some collector. Maybe this (*) recent university report from the good Tomasz Bujlov can be of interest to you. What i described above is specifically captured in chapters 3.4.6 and 3.5.6. Cheers, Paolo (*) http://vbn.aau.dk/files/78068418/report.pdf On Wed, May 07, 2014 at 03:11:59PM +0200, Antoine Meillet wrote:
Hello,
I'm currently writing a paper for school and I talk about net neutrality which brings the subject of NetFlow/IPFIX.
Should those protocols be considered as tools to perform DPI ?
Thanks, Antoine.
On May 7, 2014, at 10:45 PM, Paolo Lucente <pl+list@pmacct.net> wrote:
This model is supported on the export side by Cisco with their NetFlow/NBAR integration and on the collection side by some collector.
As you'll note in reading that report, NBAR didn't seem to work very well for them; I haven't run across its use in any ISP network, on ISP-grade hardware (i.e., not a small software-based router like a 7200), or even in a large-scale enterprise environment. Again, I haven't yet run across anyone actually using this on a production network. It would be very interesting to hear from someone with first-hand experience with NBAR exported over Flexible NetFlow in a production environment. Also, it's important to note that this sort of thing doesn't scale across networks of any real size/speed. There's a great deal of potential utility in the security, troubleshooting, and traffic engineering arenas for on-demand triggered packet sampling of this nature, but so far, the control-plane hooks aren't really there to do this in a programmatic manner (one presumes that SDN of one flavor or another might provide these capabilities). That was my biggest gripe about Flexible NetFlow when I was at Cisco, and one which I felt ensured the technology wouldn't make it into production networks - no organic control-plane interface. So, perhaps now we can de-conflate flow telemetry and 'DPI', since the real-life export, collection, and analysis of anything other than layer-4 information via flow telemetry isn't at all commonplace (if it in fact exists at all) on production networks), at this juncture. 'DPI' generally alludes boxes positioned at points of ingress/egress symmetry (either natural or purposely engineered) within a network. Flow telemetry per se is not really within the rubric of 'DPI'. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Luck is the residue of opportunity and design. -- John Milton
Please note NBAR/NetFlow integration wanted to be an example of using NetFlow/ IPFIX as a transport for DPI classification info (where classification could be performed with any other in-line technology than NBAR). Whether NBAR works or does not as a classification technology is out of scope for me here - and seems also out of the op request. Inline: On Wed, May 07, 2014 at 04:15:44PM +0000, Dobbins, Roland wrote:
So, perhaps now we can de-conflate flow telemetry and 'DPI', since the real-life export, collection, and analysis of anything other than layer-4 information via flow telemetry isn't at all commonplace (if it in fact exists at all) on production networks), at this juncture.
I disagree if anybody conflates here. I don't. I see two disjoint pieces: classification technology and transport of classification info to a central location. IPFIX, for example, is general (and standardized) enough to transport/encapsulate other info than just flow info, this might include DPI classification or other stuff. You can also read this as: if you have to travel some info, why re invent the wheel and not leverage a general-enough, standardized transport protocol (that btw you can contribute at any point to enhance if not satisfactory enough)? And please it's nice to have different positions - no need to escalate. Cheers, Paolo
Thank you Matt (offlist), Dan, Roland and Paolo for your answers ! Antoine. On 7 mai 2014, at 18:43, Paolo Lucente <pl+list@pmacct.net> wrote:
Please note NBAR/NetFlow integration wanted to be an example of using NetFlow/ IPFIX as a transport for DPI classification info (where classification could be performed with any other in-line technology than NBAR).
Whether NBAR works or does not as a classification technology is out of scope for me here - and seems also out of the op request.
Inline:
On Wed, May 07, 2014 at 04:15:44PM +0000, Dobbins, Roland wrote:
So, perhaps now we can de-conflate flow telemetry and 'DPI', since the real-life export, collection, and analysis of anything other than layer-4 information via flow telemetry isn't at all commonplace (if it in fact exists at all) on production networks), at this juncture.
I disagree if anybody conflates here. I don't. I see two disjoint pieces: classification technology and transport of classification info to a central location. IPFIX, for example, is general (and standardized) enough to transport/encapsulate other info than just flow info, this might include DPI classification or other stuff. You can also read this as: if you have to travel some info, why re invent the wheel and not leverage a general-enough, standardized transport protocol (that btw you can contribute at any point to enhance if not satisfactory enough)?
And please it's nice to have different positions - no need to escalate.
Cheers, Paolo
participants (4)
-
Antoine Meillet -
Dan White -
Dobbins, Roland -
Paolo Lucente