November 2018 - Test - lists.nanog.org

Technical Contact at Yahoo
by Brock Tice 09 Nov '18

09 Nov '18

Yahoo have done a very good job of locking down their contact info. We have an issue where they seem to be blocking one of our CGN public addresses. Does anyone know how I can get in touch with an appropriate person there?

2 1

Technical Contact at Yahoo
by Brock Tice 09 Nov '18

09 Nov '18

Yahoo have done a very good job of locking down their contact info. We have an issue where they seem to be blocking one of our CGN public addresses. Does anyone know how I can get in touch with an appropriate person there?

1 0

Re: Amazon network engineering contact? re: DDoS traffic
by John 09 Nov '18

09 Nov '18

Zach, As mentioned before, I am open to peering (where possible) but have not received a response. My goal is to connect with someone at Amazon and work with them on a technical solution, which is why I posted asking for that here. Various factors mean that we can't just upgrade our way out of this one, and manual whac-a-mole on the sources has shown to have limited use. These attacks also impact Amazon and the networks in between the sources and targets, and they take time to handle by the abuse teams, so there are good reasons to investigate them further and find better ways to mitigate or prevent them. -John On 11/8/2018 1:12 PM, Zach Puls wrote: > Makes sense, that's understandable. Do you peer with AWS? If not, maybe opening up a peering agreement will give you a better contact, and a bit more pull when attacks occur? I know someone with a peering agreement with AWS, and they have been able to get resolutions fairly quickly when issues arise. > > Other than that, I'm not sure of a solution other than more IP transit. > > Thanks, > > Zach Puls > Network Engineer | MEF-CECP > KsFiberNet > > -----Original Message----- > From: John Weekes <jw(a)nuclearfallout.net> > Sent: Thursday, November 08, 2018 15:03 > To: Zach Puls <zpuls(a)ksfiber.net> > Cc: nanog(a)nanog.org > Subject: Re: Amazon network engineering contact? re: DDoS traffic > > Zach, > > Yes, RTBH is used to distribute the null-routes that I mentioned. > > Unfortunately, even brief saturation events lasting just 5-10 seconds (a typical amount of time to detect the loss, issue the null-route, and see the traffic start to fall off as it is distributed upstream) can cause real damage to those customers who are sensitive to latency and packet loss. So while null-routes limit the duration of the impact, they can't eliminate it entirely. And, of course, the actual target of the attack > -- the now-null-routed IP address -- becomes unreachable, which was presumably the goal of the attacker. > > -John > > On 11/8/2018 12:54 PM, Zach Puls wrote: >> No idea about an Amazon abuse contact, but do you have RTBH communities enabled with your upstream provider(s)? As a general practice, when you detect a (D)DoS attack in progress, it would help to automatically advertise that prefix to your upstream(s) with the black-hole community. This would at least help mitigate the effects of the attacks when they do occur, even if they come from a different source than AWS. >> >> Thanks, >> >> Zach Puls >> Network Engineer | MEF-CECP >> KsFiberNet >> >> -----Original Message----- >> From: NANOG <nanog-bounces(a)nanog.org> On Behalf Of John Weekes >> Sent: Thursday, November 08, 2018 14:44 >> To: nanog(a)nanog.org >> Subject: Amazon network engineering contact? re: DDoS traffic >> >> We've been seeing significant attack activity from Amazon over the last two months, involving apparently compromised instances that commonly send 1-10G of traffic per source and together generate Nx10G of total traffic. Even when our overall upstream capacity exceeds an attack's overall size, the nature of load-balancing over multiple 10G upstream links means that an individual link can be saturated by multiple large flows, forcing our systems to null-route the target to limit impact. >> >> We've sent an abuse notification about every traffic source to Amazon, and specific sources seem to stop their involvement over time (suggesting that abuse teams are following up on them), but there is an endless parade of new attackers, and each source participates in many damaging attacks before it is shut down. >> >> Is there anyone at Amazon who can help with an engineering solution in terms of programmatically detecting and rate-limiting attack traffic sources, to our networks or overall? Or applying the kludge of a rate-limit for all Amazon traffic to our networks? Or working with us on some other option? >> >> At least one other large cloud provider has an automatic rate-limiting system in place that is effective in reducing the damage from repeat high-volume attacks. >> >> Emails to the Amazon NOC, peering contacts (since that would be another possible solution), and abuse department have not connected me with anyone. >> >> Thanks, >> John >> >

3 2

FCC Launches Re-Examination of Wireless Resiliency Cooperative Framework In Light of Recent Hurricanes
by Sean Donelan 08 Nov '18

08 Nov '18

The public, first responders and other service providers can also submit comments to the FCC. https://www.fcc.gov/document/fcc-seeks-industry-input-review-wireless-resil… To that end, Chief Fowlkes’ letters ask wireless companies participating in the framework to summarize how it was used for all disaster events to which the framework applied. The letters request detailed lists of mutual aid and roaming agreements carriers have established with each other, as well as any instances where such agreements were modified, impeded, or even declined outright. The agency also asks for information as to each company’s implementation of industry best practices.

1 0

Re: Amazon network engineering contact? re: DDoS traffic
by John Weekes 08 Nov '18

08 Nov '18

Zach, Yes, RTBH is used to distribute the null-routes that I mentioned. Unfortunately, even brief saturation events lasting just 5-10 seconds (a typical amount of time to detect the loss, issue the null-route, and see the traffic start to fall off as it is distributed upstream) can cause real damage to those customers who are sensitive to latency and packet loss. So while null-routes limit the duration of the impact, they can't eliminate it entirely. And, of course, the actual target of the attack -- the now-null-routed IP address -- becomes unreachable, which was presumably the goal of the attacker. -John On 11/8/2018 12:54 PM, Zach Puls wrote: > No idea about an Amazon abuse contact, but do you have RTBH communities enabled with your upstream provider(s)? As a general practice, when you detect a (D)DoS attack in progress, it would help to automatically advertise that prefix to your upstream(s) with the black-hole community. This would at least help mitigate the effects of the attacks when they do occur, even if they come from a different source than AWS. > > Thanks, > > Zach Puls > Network Engineer | MEF-CECP > KsFiberNet > > -----Original Message----- > From: NANOG <nanog-bounces(a)nanog.org> On Behalf Of John Weekes > Sent: Thursday, November 08, 2018 14:44 > To: nanog(a)nanog.org > Subject: Amazon network engineering contact? re: DDoS traffic > > We've been seeing significant attack activity from Amazon over the last two months, involving apparently compromised instances that commonly send 1-10G of traffic per source and together generate Nx10G of total traffic. Even when our overall upstream capacity exceeds an attack's overall size, the nature of load-balancing over multiple 10G upstream links means that an individual link can be saturated by multiple large flows, forcing our systems to null-route the target to limit impact. > > We've sent an abuse notification about every traffic source to Amazon, and specific sources seem to stop their involvement over time (suggesting that abuse teams are following up on them), but there is an endless parade of new attackers, and each source participates in many damaging attacks before it is shut down. > > Is there anyone at Amazon who can help with an engineering solution in terms of programmatically detecting and rate-limiting attack traffic sources, to our networks or overall? Or applying the kludge of a rate-limit for all Amazon traffic to our networks? Or working with us on some other option? > > At least one other large cloud provider has an automatic rate-limiting system in place that is effective in reducing the damage from repeat high-volume attacks. > > Emails to the Amazon NOC, peering contacts (since that would be another possible solution), and abuse department have not connected me with anyone. > > Thanks, > John >

1 0

Amazon network engineering contact? re: DDoS traffic
by John Weekes 08 Nov '18

08 Nov '18

We've been seeing significant attack activity from Amazon over the last two months, involving apparently compromised instances that commonly send 1-10G of traffic per source and together generate Nx10G of total traffic. Even when our overall upstream capacity exceeds an attack's overall size, the nature of load-balancing over multiple 10G upstream links means that an individual link can be saturated by multiple large flows, forcing our systems to null-route the target to limit impact. We've sent an abuse notification about every traffic source to Amazon, and specific sources seem to stop their involvement over time (suggesting that abuse teams are following up on them), but there is an endless parade of new attackers, and each source participates in many damaging attacks before it is shut down. Is there anyone at Amazon who can help with an engineering solution in terms of programmatically detecting and rate-limiting attack traffic sources, to our networks or overall? Or applying the kludge of a rate-limit for all Amazon traffic to our networks? Or working with us on some other option? At least one other large cloud provider has an automatic rate-limiting system in place that is effective in reducing the damage from repeat high-volume attacks. Emails to the Amazon NOC, peering contacts (since that would be another possible solution), and abuse department have not connected me with anyone. Thanks, John

1 0

Spoofer Report for NANOG for Oct 2018
by CAIDA Spoofer Project 08 Nov '18

08 Nov '18

In response to feedback from operational security communities, CAIDA's source address validation measurement project (https://spoofer.caida.org) is automatically generating monthly reports of ASes originating prefixes in BGP for systems from which we received packets with a spoofed source address. We are publishing these reports to network and security operations lists in order to ensure this information reaches operational contacts in these ASes. This report summarises tests conducted within usa, can. Inferred improvements during Oct 2018: ASN Name Fixed-By 14361 HOPONE-GLOBAL 2018-10-03 7233 YAHOO-US 2018-10-04 237 MERIT-AS-14 2018-10-05 4 ISI 2018-10-10 15176 AS-INOC 2018-10-17 3356 LEVEL3 2018-10-24 10326 WORCESTER-1 2018-10-29 Further information for the inferred remediation is available at: https://spoofer.caida.org/remedy.php Source Address Validation issues inferred during Oct 2018: ASN Name First-Spoofed Last-Spoofed 5650 FRONTIER-FRTR 2016-02-22 2018-10-04 577 BACOM 2016-03-09 2018-10-27 18978 ENZUINC-US 2016-04-15 2018-10-26 54825 PACKET 2016-04-15 2018-10-11 7922 COMCAST-7922 2016-06-04 2018-10-09 19230 NANOG 2016-06-13 2018-10-04 7029 WINDSTREAM 2016-06-21 2018-10-30 209 CENTURYLINK-US-LEGACY-QWEST 2016-08-16 2018-10-28 6128 CABLE-NET-1 2016-09-03 2018-10-31 20412 CLARITY-TELECOM 2016-09-30 2018-10-31 6181 FUSE-NET 2016-10-10 2018-10-30 15305 SYRINGANETWORKS 2016-10-21 2018-10-27 25787 ROWE-NETWORKS 2016-10-21 2018-10-26 174 COGENT-174 2016-10-21 2018-10-31 2828 XO-AS15 2016-10-25 2018-10-06 31857 PRIORITY-TERABIT 2016-10-25 2018-10-19 30341 SCTA-ASN 2016-10-31 2018-10-14 32440 LONI 2016-11-03 2018-10-29 33182 DimeNOC 2016-11-08 2018-10-25 12083 WOW-INTERNET 2016-11-09 2018-10-27 3549 LVLT-3549 2016-11-16 2018-10-03 21832 KELLINCOM-1 2017-02-03 2018-10-26 7122 MTS-ASN 2017-05-16 2018-10-25 6461 ZAYO-6461 2017-06-21 2018-10-31 63296 AMARILLO-WIRELESS 2017-09-01 2018-10-30 7233 YAHOO-US 2017-09-07 2018-10-31 33523 ROWANUNIVERSITY 2017-10-29 2018-10-31 2152 CSUNET-NW 2017-11-08 2018-10-10 546 PARSONS-PGS-1 2017-11-20 2018-10-10 12222 AKAMAI 2018-02-14 2018-10-27 29384 Qatar-Foundation 2018-03-08 2018-10-25 23148 TERRENAP 2018-03-15 2018-10-30 20009 WADSNET 2018-04-13 2018-10-19 4201 ORST 2018-04-19 2018-10-29 11827 WSU 2018-04-19 2018-10-31 393564 SPOKANE 2018-06-05 2018-10-03 35911 BNQ-1 2018-06-06 2018-10-21 225 VIRGINIA 2018-06-18 2018-10-30 53646 HARRIS-BROADBAND 2018-08-10 2018-10-02 40911 L2NC 2018-08-31 2018-10-15 2381 WISCNET1 2018-09-04 2018-10-31 54804 CSMIII-BUNKIELA 2018-09-15 2018-10-16 33452 RW 2018-09-19 2018-10-14 20448 VPNTRANET-LLC 2018-09-20 2018-10-12 11996 LOBOIS 2018-09-24 2018-10-16 10326 WORCESTER-1 2018-09-30 2018-10-05 10929 NETELLIGENT 2018-10-02 2018-10-03 36210 SFCF 2018-10-13 2018-10-13 36327 VINAKOM 2018-10-16 2018-10-16 14031 SCXY 2018-10-18 2018-10-26 19919 VSW 2018-10-23 2018-10-30 22462 NOLA-BROADBAND-INC 2018-10-30 2018-10-30 Further information for these tests where we received spoofed packets is available at: https://spoofer.caida.org/recent_tests.php?country_include=usa,can&no_block… Please send any feedback or suggestions to spoofer-info(a)caida.org

1 0

Routinator 3000 and the RPKI project
by Alex Band 07 Nov '18

07 Nov '18

Hello all! Several months ago NLnet Labs committed to building a free open source RPKI toolset to help making BGP routing more secure. This project includes a Certificate Authority, allowing you to run a Delegated CA on your own systems as a child of one or more RIRs, a Publication Server that lets you publish RPKI material or let a third party do it on your behalf and lastly Relying Party software, in order to validate RPKI data and feed it to your routers. I want to give you a little update on where we are now. Since kicking off this project, RIPE NCC and NIC.br have graciously committed to funding these efforts, ensuring we can dedicate full time resources on this in the coming years. In the mean time, we’ve released (and then fixed :cough:) the first version of our Relying Party software. It’s designed to be super lean (as in, runs fine on a Pi Zero) and implements the basic set of functionality: fetching and validating RPKI data and exposing route origin attestations both as output (CSV, JSON, RPSL) and to routers via the RPKI-RTR protocol. We’re very much looking forward to your operational feedback, to ensure this package runs well in a wide variety of environments. Going forward, we’ll be focussing on monitoring for the next release. You can find the source code and further details on Github: https://github.com/NLnetLabs/routinator Cheers, Alex Band NLnet Labs

1 0

Switch with high ACL capacity
by Mike Hammett 06 Nov '18

06 Nov '18

I am looking for recommendations as to a 10G or 40G switch that has the ability to hold a large number of entries in ACLs. Preferred if I can get them there via the BGP flow spec, but some sort of API or even just brute force on the console would be good enough. Used or even end of life is fine. -----Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISP

4 7

RE: Switch with high ACL capacity
by Mike Hammett 06 Nov '18

06 Nov '18

Other than it completes the DDoS. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: Zach Puls <zpuls(a)ksfiber.net> To: Mike Hammett <nanog(a)ics-il.net> Cc: 'nanog list' <nanog(a)nanog.org> Sent: Tue, 06 Nov 2018 13:55:22 -0600 (CST) Subject: RE: Switch with high ACL capacity Wouldn’t it be more beneficial to just have a low-cost system for detection, then trigger an RTBH community advertisement to your upstream(s)? Zach Puls Network Engineer | MEF-CECP KsFiberNet -----Original Message----- From: NANOG <nanog-bounces+zpuls=ksfiber.net(a)nanog.org> On Behalf Of Mike Hammett Sent: Tuesday, November 06, 2018 13:47 To: Lotia, Pratik M <Pratik.Lotia(a)charter.com> Cc: 'nanog list' <nanog(a)nanog.org> Subject: Re: Switch with high ACL capacity The intent is to see if I can construct a poor man's DDOS scrubber. There are low cost systems out there for the detection, but they just trigger something else to do the work. Obviously there is black hole routing, but I'm looking for something with a bit more finesse. If I need to get a switch anyway, might as well try to take advantage of it for other uses. -----Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISP ----- Original Message ----- From: Lotia, Pratik M <Pratik.Lotia(a)charter.com> To: Mike Hammett <nanog(a)ics-il.net>, 'nanog list' <nanog(a)nanog.org> Sent: Tue, 06 Nov 2018 12:29:15 -0600 (CST) Subject: Re: Switch with high ACL capacity Mike, Can you shed some light on the use case? Looks like you are confusing ACLs and BGP Flowspec. ACLs and Flowspec rules are similar in some ways but they have a different use case. ACLs cannot be configured using Flowspec announcements. Flowspec can be loosely explained as 'Routing based on L4 rules' (there's a lot more to it than just L4). I doubt if a there is a Switch which can hold a large number of Flowspec entries. ~Pratik Lotia “Improvement begins with I.” On 11/6/18, 10:39, "NANOG on behalf of Mike Hammett" <nanog-bounces(a)nanog.org on behalf of nanog(a)ics-il.net> wrote: I am looking for recommendations as to a 10G or 40G switch that has the ability to hold a large number of entries in ACLs. Preferred if I can get them there via the BGP flow spec, but some sort of API or even just brute force on the console would be good enough. Used or even end of life is fine. -----Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISP E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

1 0