sflow -> aggregated aspath visualization?
I’m looking for product recommendations: We’ve noticed that about 20% of our traffic here lately has decamped from the free (or, at least, flat-rate) connection to CANARIE (our R&E network) and its various connected content-delivery networks, and onto our commercial provider. While this is presumptively a legitimate shift, we’d like to better understand these changes when they occur, in a way that our executive can understand at a glance. We do have sFlow (et al.) going to an Arbor PeakFlow box for analysis, but it’s lacklustre at best at understanding changes like this. I want: * Top #n ASNs by traffic volume, per router/interface, stacked chart * Some way to visualize large jumps in that dataset, e.g. if Cloudflare ditched their CANARIE connection and now that traffic all goes commercial, I don’t know what sort of graphic would be useful, maybe a stacked polar chart so you could see when an AS jumped from one sector to another? Even stacked bar charts could be useful. If anyone knows of tools capable of generating easy-to-understand reports, dashboards, including historical “what changed this week”-type data, please let me know. For that matter, if you have a technique of collecting this data and using Excel to do the reporting, that would work too. (Yes, I could theoretically build this off of existing open source tools… eventually) Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.ca<mailto:athompson@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
<https://techfieldday.com/video/kentik-interconnection-and-metrics-from-kentik-for-service-provider-networks/> On Sat, Mar 14, 2020 at 12:33 PM Adam Thompson <athompson@merlin.mb.ca> wrote:
I’m looking for product recommendations:
We’ve noticed that about 20% of our traffic here lately has decamped from the free (or, at least, flat-rate) connection to CANARIE (our R&E network) and its various connected content-delivery networks, and onto our commercial provider.
While this is presumptively a legitimate shift, we’d like to better understand these changes when they occur, in a way that our executive can understand at a glance.
We do have sFlow (et al.) going to an Arbor PeakFlow box for analysis, but it’s lacklustre at best at understanding changes like this.
I want:
- Top #n ASNs by traffic volume, per router/interface, stacked chart - Some way to visualize large jumps in that dataset, e.g. if Cloudflare ditched their CANARIE connection and now that traffic all goes commercial, I don’t know what sort of graphic would be useful, maybe a stacked polar chart so you could see when an AS jumped from one sector to another? Even stacked bar charts could be useful.
I haven't used Kentik in production, but heard good things about it https://techfieldday.com/video/the-kentik-experience-an-overview-demo-with-a... https://techfieldday.com/video/kentik-interconnection-and-metrics-from-kenti... Just a reminder network devices might not export 100% samples/flows correctly (sampling rate/export rate limitation, dropped packets on ingress/egress, recirculated packet, policy routing actions, multiple routing tables/vrf). The accuracy/availability of metadata in flow itself (sFlow Extended Flow Data, sFlow input/output/source interface, IPFIX information elements that are not directly extracted from packet lookup header) might have limitations
On Fri, Mar 13, 2020 at 06:54:15PM +0000, Adam Thompson wrote:
If anyone knows of tools capable of generating easy-to-understand reports, dashboards, including historical “what changed this week”-type data, please let me know.
If you have access to a modern ELK stack, you could try out Elastiflow: https://github.com/robcowart/elastiflow My understanding is that tuning the number and settings of the Logstash instances can be challenging, but it's very pretty when it's working correctly. -- Brandon Ewing (brandon.ewing@warningg.com)
You could use Prometheus / Grafana to build the dashboards. The following example is a starting point (top ASNs / Countries by traffic volume): https://grafana.com/grafana/dashboards/11146 The example could be modified to make the make router / interface selectable, or cloned to create separate per router / interface dashboards. On Sat, Mar 14, 2020 at 12:33 PM Adam Thompson <athompson@merlin.mb.ca> wrote:
I’m looking for product recommendations:
We’ve noticed that about 20% of our traffic here lately has decamped from the free (or, at least, flat-rate) connection to CANARIE (our R&E network) and its various connected content-delivery networks, and onto our commercial provider.
While this is presumptively a legitimate shift, we’d like to better understand these changes when they occur, in a way that our executive can understand at a glance.
We do have sFlow (et al.) going to an Arbor PeakFlow box for analysis, but it’s lacklustre at best at understanding changes like this.
I want:
- Top #n ASNs by traffic volume, per router/interface, stacked chart - Some way to visualize large jumps in that dataset, e.g. if Cloudflare ditched their CANARIE connection and now that traffic all goes commercial, I don’t know what sort of graphic would be useful, maybe a stacked polar chart so you could see when an AS jumped from one sector to another? Even stacked bar charts could be useful.
If anyone knows of tools capable of generating easy-to-understand reports, dashboards, including historical “what changed this week”-type data, please let me know.
For that matter, if you have a technique of collecting this data and using Excel to do the reporting, that would work too.
(Yes, I could theoretically build this off of existing open source tools… eventually)
participants (5)
-
Adam Thompson -
Brandon Ewing -
Mark Tinka -
Peter Phaal -
Yang Yu