BCP38 adoption "incentives"?
Does anyone know if any upstream and tiered internet providers include in their connection contracts a mandatory requirement that all directly-connected routers be in compliance with BCP38? Does anyone know if large ISPs like Comcast, Charter, or AT&T have put in place internal policies requiring retail/business-customer-aggregating routers to be in compliance with BCP38? Does any ISP, providing business Internet connectivity along with a block of IP addresses, include language in their contracts that any directly connected router must be in compliance with BCP38? I've seen a lot of moaning and groaning about how BCP38 is pretty much being ignored. Education is one way to help, but that doesn't hit anyone in the wallet. You have to motivate people to go out of their way to *learn* about BCP38; most business people are too busy with things that make them money to be concerned with "Internet esoterica" that doesn't add to the bottom line. You have to make their ignorance SUBTRACT from the bottom line. Contracts, properly enforced, can make a huge dent in the problem of BCP38 adoption. At a number of levels. Equipment manufacturers not usually involved in this sort of thing (home and SOHO market) would then have market incentive to provide equipment at the low end that would provide BCP38 support. Especially equipment manufacturers that incorporate embedded Linux in their products. They can be creative in how they implement their product; let creativity blossom. I know, I know, BCP38 was originally directed at Internet Service Providers at their edge to upstreams. I'm thinking that BCP38 needs to be in place at any point -- every point? -- where you have a significant-sized collection of systems/devices aggregated to single upstream connections. Particular systems/devices where any source address can be generated and propagated -- including compromised desktop computers, compromised light bulbs, compromised wireless routers, compromised you-name-it. (That is one nice thing about NAT -- the bad guys can't build spoofed packets. They *can* build, um, "other" packets...which is a different subject entirely.) (N.B.: Now you know why I'm trying to get the simplest possible definition of BCP38 into words. The RFCs don't contain "executive summaries".)
On Tue, 27 Sep 2016, Stephen Satchell wrote:
You have to make their ignorance SUBTRACT from the bottom line.
I'd say there is no way to actually achieve this. BCP38 non-compliance doesn't hurt the one not in compliance in any significant amount, it hurts everybody else. The only way I can imagine BCP38 ever happening widely is by means of legislation, which of course is really hard because Internet spans countries/continents. Doing anti-spoofing should be done at the edge, the further up into the core you try to do it, the bigger risk you're breaking lots of users' connectivity, causing customer complaints. In some countries I'm sure BCP38 compliance could be increased by means of legislation and fining companies that do not do BCP38 filtering. But before we do that, we need to agree that BCP38 compliance is a must. I don't think we're there. I have heard people say that if they don't allow some of their customers to spoof, they're losing business, because some customers have built complete (deployed) solutions that are built on the fact that they can spoof packets. These people will have to be convinced that they're doing it wrong and re-design their solutions. This is going to cost them dearly, so they're going to be upset. With all the IoT devices out there, do people even need to spoof anymore? -- Mikael Abrahamsson email: swmike@swm.pp.se
The implementation of BCP38 over local market strongly increased after massive DDoS attacks in 2013 affecting major part of the industry thanks to an initiative of the most important local IXP. There is a special separate last-resort "island mode" network, which is intended to be activated in case of really major attacks to keep an accessibility of (at least) local services for local users. Implementation of BCP38 is one of the (lot of) requirements. Positive motivation is definitely better than asking politicians for regulations. More: https://fe.nix.cz/en/ Regards, Zbynek Dne 27.09.16 v 14:46 Mikael Abrahamsson napsal(a):
On Tue, 27 Sep 2016, Stephen Satchell wrote:
You have to make their ignorance SUBTRACT from the bottom line.
I'd say there is no way to actually achieve this. BCP38 non-compliance doesn't hurt the one not in compliance in any significant amount, it hurts everybody else.
The only way I can imagine BCP38 ever happening widely is by means of legislation, which of course is really hard because Internet spans countries/continents.
Doing anti-spoofing should be done at the edge, the further up into the core you try to do it, the bigger risk you're breaking lots of users' connectivity, causing customer complaints. re In some countries I'm sure BCP38 compliance could be increased by means of legislation and fining companies that do not do BCP38 filtering. But before we do that, we need to agree that BCP38 compliance is a must. I don't think we're there. I have heard people say that if they don't allow some of their customers to spoof, they're losing business, because some customers have built complete (deployed) solutions that are built on the fact that they can spoof packets. These people will have to be convinced that they're doing it wrong and re-design their solutions. This is going to cost them dearly, so they're going to be upset.
With all the IoT devices out there, do people even need to spoof anymore?
On Tue, 27 Sep 2016, Zbyněk Pospíchal wrote:
The implementation of BCP38 over local market strongly increased after massive DDoS attacks in 2013 affecting major part of the industry thanks to an initiative of the most important local IXP.
Hm, so the IX operator looks at packets at the IX (sFlow perhaps), see who is sending attack packets, and if they're spoofed, this ISP is then put in "quarantine", ie their IX port is basically now useless. That's an effective way of achieving local compliance. Wonder how this would work in other markets, commonly it's bad business to deny service to paying customers... But if most agree that this should be done, it's definitely a way to achieve compliance. -- Mikael Abrahamsson email: swmike@swm.pp.se
Dne 27.09.16 v 15:17 Mikael Abrahamsson napsal(a):
Hm, so the IX operator looks at packets at the IX (sFlow perhaps), see who is sending attack packets, and if they're spoofed, this ISP is then put in "quarantine", ie their IX port is basically now useless.
Definitely not. Try to read first instead of speculations. Regards, Zbynek
On Tue, 27 Sep 2016, Zbyněk Pospíchal wrote:
Dne 27.09.16 v 15:17 Mikael Abrahamsson napsal(a):
Hm, so the IX operator looks at packets at the IX (sFlow perhaps), see who is sending attack packets, and if they're spoofed, this ISP is then put in "quarantine", ie their IX port is basically now useless.
Definitely not. Try to read first instead of speculations.
The first page was completely devoid of any real technical information until I found the PDF (which from the color choice doesn't even look like a link). (https://www.nix.cz/cs/file/NIX_RULES_FENIX) It's still not obvious what the FENIX connection is used for from that PDF. It's called "last resort connection". What does that mean? Apart from that, it looks more like https://www.routingmanifesto.org/ in that organisations that have joined are stating that they will follow some operational guidelines (which make a lot of sense), but it's not that much more technical when it comes to inter-provider traffic. -- Mikael Abrahamsson email: swmike@swm.pp.se
Dne 27.09.16 v 16:30 Mikael Abrahamsson napsal(a):
The first page was completely devoid of any real technical information until I found the PDF (which from the color choice doesn't even look like a link). (https://www.nix.cz/cs/file/NIX_RULES_FENIX)
It's still not obvious what the FENIX connection is used for from that PDF. It's called "last resort connection". What does that mean?
It means that a network suffering massive DDoS can switch off their transit and peering links and turn itself to anything like "island mode" operation, still keeping access to some of root and local TLD DNS servers, local content and/or local users. Fortunately, it has been never used except tests, but it is still an option when a network is in such kind of trouble.
Apart from that, it looks more like https://www.routingmanifesto.org/ in that organisations that have joined are stating that they will follow some operational guidelines (which make a lot of sense), but it's not that much more technical when it comes to inter-provider traffic
Routing manifesto cannot provide you anything in the oposite if you comply it, except, maybe, good feeling. Fenix (or Dutch Trusted Network Initiative) project can. Best Regards, Zbynek
What would it take to test for BCP38 for a specific AS? Joe Klein "Inveniam viam aut faciam" PGP Fingerprint: 295E 2691 F377 C87D 2841 00C1 4174 FEDF 8ECF 0CC8 On Tue, Sep 27, 2016 at 8:31 AM, Stephen Satchell <list@satchell.net> wrote:
Does anyone know if any upstream and tiered internet providers include in their connection contracts a mandatory requirement that all directly-connected routers be in compliance with BCP38?
Does anyone know if large ISPs like Comcast, Charter, or AT&T have put in place internal policies requiring retail/business-customer-aggregating routers to be in compliance with BCP38?
Does any ISP, providing business Internet connectivity along with a block of IP addresses, include language in their contracts that any directly connected router must be in compliance with BCP38?
I've seen a lot of moaning and groaning about how BCP38 is pretty much being ignored. Education is one way to help, but that doesn't hit anyone in the wallet. You have to motivate people to go out of their way to *learn* about BCP38; most business people are too busy with things that make them money to be concerned with "Internet esoterica" that doesn't add to the bottom line. You have to make their ignorance SUBTRACT from the bottom line.
Contracts, properly enforced, can make a huge dent in the problem of BCP38 adoption. At a number of levels.
Equipment manufacturers not usually involved in this sort of thing (home and SOHO market) would then have market incentive to provide equipment at the low end that would provide BCP38 support. Especially equipment manufacturers that incorporate embedded Linux in their products. They can be creative in how they implement their product; let creativity blossom.
I know, I know, BCP38 was originally directed at Internet Service Providers at their edge to upstreams. I'm thinking that BCP38 needs to be in place at any point -- every point? -- where you have a significant-sized collection of systems/devices aggregated to single upstream connections. Particular systems/devices where any source address can be generated and propagated -- including compromised desktop computers, compromised light bulbs, compromised wireless routers, compromised you-name-it.
(That is one nice thing about NAT -- the bad guys can't build spoofed packets. They *can* build, um, "other" packets...which is a different subject entirely.)
(N.B.: Now you know why I'm trying to get the simplest possible definition of BCP38 into words. The RFCs don't contain "executive summaries".)
On Tue, 27 Sep 2016, Joe Klein wrote:
What would it take to test for BCP38 for a specific AS?
Well, you can get people to run https://www.caida.org/projects/spoofer/#software I tried to get OpenWrt to include similar software, on by default, but some people are afraid that they might incur legal action on themselves by doing antispoofing-testing. https://www.ripe.net/participate/ripe/tf/anti-spoofing might be of interest. -- Mikael Abrahamsson email: swmike@swm.pp.se
On 27 September 2016 at 15:32, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
On Tue, 27 Sep 2016, Joe Klein wrote:
What would it take to test for BCP38 for a specific AS?
Well, you can get people to run https://www.caida.org/projects/spoofer/#software
I tried to get OpenWrt to include similar software, on by default, but some people are afraid that they might incur legal action on themselves by doing antispoofing-testing.
Any network operator should know if their network is blocking it or not without having to deploy active probes across their network. If a network is thinking about it enough to want to block it, they will probably do so by turning knobs on their routers rather than deploying another patch to the CPE. I don't think the CPE is the solution here. - Mike Jones
On Tue, 27 Sep 2016, Mike Jones wrote:
Any network operator should know if their network is blocking it or not without having to deploy active probes across their network.
Err... I was not referring to the operator doing this on the CPEs they provide to their customers. I was referring to enthusiast customers who are running their own CPEs to test if their ISPs are doing anti-spoofing properly or not, and report this information to someone centrally. -- Mikael Abrahamsson email: swmike@swm.pp.se
The knobs that are available to push adoption of any standard can include "Doing nothing", "Educating the community", "Incentives", "Public Shaming", "Loss of business", "Engaging the policy & legal wanks". It seems to me the first two options have not moved the ball much. Must we move the last four to fix the DDOS problem? The last one scares me, but the other three might be valid method to move the ball. Joe Klein "Inveniam viam aut faciam" PGP Fingerprint: 295E 2691 F377 C87D 2841 00C1 4174 FEDF 8ECF 0CC8 On Tue, Sep 27, 2016 at 10:32 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
On Tue, 27 Sep 2016, Joe Klein wrote:
What would it take to test for BCP38 for a specific AS?
Well, you can get people to run https://www.caida.org/projects /spoofer/#software
I tried to get OpenWrt to include similar software, on by default, but some people are afraid that they might incur legal action on themselves by doing antispoofing-testing.
https://www.ripe.net/participate/ripe/tf/anti-spoofing might be of interest.
-- Mikael Abrahamsson email: swmike@swm.pp.se
<sarcasm> Do not forget the "NRA" ways. Circular discussions every time an event arise, let it die out after a few days, and hopefully, nothing change. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 09/27/16 14:16, Joe Klein wrote:
The knobs that are available to push adoption of any standard can include "Doing nothing", "Educating the community", "Incentives", "Public Shaming", "Loss of business", "Engaging the policy & legal wanks". It seems to me the first two options have not moved the ball much.
Must we move the last four to fix the DDOS problem? The last one scares me, but the other three might be valid method to move the ball.
Joe Klein "Inveniam viam aut faciam"
PGP Fingerprint: 295E 2691 F377 C87D 2841 00C1 4174 FEDF 8ECF 0CC8
On Tue, Sep 27, 2016 at 10:32 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
On Tue, 27 Sep 2016, Joe Klein wrote:
What would it take to test for BCP38 for a specific AS? Well, you can get people to run https://www.caida.org/projects /spoofer/#software
I tried to get OpenWrt to include similar software, on by default, but some people are afraid that they might incur legal action on themselves by doing antispoofing-testing.
https://www.ripe.net/participate/ripe/tf/anti-spoofing might be of interest.
-- Mikael Abrahamsson email: swmike@swm.pp.se
In article <xs4all.CAP032Tv1QtSOOEiL5K5yLQjM87bfvutvCQeqfsyzan4QAmH4mA@mail.gmail.com> you write:
What would it take to test for BCP38 for a specific AS?
Well, if a certain browser vendor let the browser deduce the external IP address, then send out a UDP DNS PTR query for <reverse-ip>.in-addr.browser-vendor.com to say, a large DNS resolving cluster they also happen to be running, from a random source address, once every so often, they could very quickly build up a name-and-shame list of providers that don't do proper filtering. Just saying. Mike.
----- Original Message -----
From: "Joe Klein" <jsklein@gmail.com>
What would it take to test for BCP38 for a specific AS?
There's a tester tool, which I believe I put a link to on the wiki. One of the Cali tech universities; Berkeley? Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274
It would be incredibly low impact to have the residential CPE block any source address not assigned by the ISP. Done. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Stephen Satchell" <list@satchell.net> To: nanog@nanog.org Sent: Tuesday, September 27, 2016 7:31:24 AM Subject: BCP38 adoption "incentives"? Does anyone know if any upstream and tiered internet providers include in their connection contracts a mandatory requirement that all directly-connected routers be in compliance with BCP38? Does anyone know if large ISPs like Comcast, Charter, or AT&T have put in place internal policies requiring retail/business-customer-aggregating routers to be in compliance with BCP38? Does any ISP, providing business Internet connectivity along with a block of IP addresses, include language in their contracts that any directly connected router must be in compliance with BCP38? I've seen a lot of moaning and groaning about how BCP38 is pretty much being ignored. Education is one way to help, but that doesn't hit anyone in the wallet. You have to motivate people to go out of their way to *learn* about BCP38; most business people are too busy with things that make them money to be concerned with "Internet esoterica" that doesn't add to the bottom line. You have to make their ignorance SUBTRACT from the bottom line. Contracts, properly enforced, can make a huge dent in the problem of BCP38 adoption. At a number of levels. Equipment manufacturers not usually involved in this sort of thing (home and SOHO market) would then have market incentive to provide equipment at the low end that would provide BCP38 support. Especially equipment manufacturers that incorporate embedded Linux in their products. They can be creative in how they implement their product; let creativity blossom. I know, I know, BCP38 was originally directed at Internet Service Providers at their edge to upstreams. I'm thinking that BCP38 needs to be in place at any point -- every point? -- where you have a significant-sized collection of systems/devices aggregated to single upstream connections. Particular systems/devices where any source address can be generated and propagated -- including compromised desktop computers, compromised light bulbs, compromised wireless routers, compromised you-name-it. (That is one nice thing about NAT -- the bad guys can't build spoofed packets. They *can* build, um, "other" packets...which is a different subject entirely.) (N.B.: Now you know why I'm trying to get the simplest possible definition of BCP38 into words. The RFCs don't contain "executive summaries".)
Hi Mike, This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible. It would make sense for most ISPs to have egress filtering at the edge (transit and peering points) to filter out packets that should not originate from the ISP's ASN, although this does not prevent spoofing between points in the ISP's network. Andrew NB: My personal opinion and not official communiqué of Charter. Andrew White Desk: 314.394-9594 | Cell: 314-452-4386 | Jabber andrew.white2@charter.com Systems Engineer III, DAS DNS group Charter Communications 12405 Powerscourt Drive, St. Louis, MO 63131 -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Mike Hammett Sent: Tuesday, September 27, 2016 3:33 PM Cc: nanog@nanog.org Subject: Re: BCP38 adoption "incentives"? It would be incredibly low impact to have the residential CPE block any source address not assigned by the ISP. Done. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Stephen Satchell" <list@satchell.net> To: nanog@nanog.org Sent: Tuesday, September 27, 2016 7:31:24 AM Subject: BCP38 adoption "incentives"? Does anyone know if any upstream and tiered internet providers include in their connection contracts a mandatory requirement that all directly-connected routers be in compliance with BCP38? Does anyone know if large ISPs like Comcast, Charter, or AT&T have put in place internal policies requiring retail/business-customer-aggregating routers to be in compliance with BCP38? Does any ISP, providing business Internet connectivity along with a block of IP addresses, include language in their contracts that any directly connected router must be in compliance with BCP38? I've seen a lot of moaning and groaning about how BCP38 is pretty much being ignored. Education is one way to help, but that doesn't hit anyone in the wallet. You have to motivate people to go out of their way to *learn* about BCP38; most business people are too busy with things that make them money to be concerned with "Internet esoterica" that doesn't add to the bottom line. You have to make their ignorance SUBTRACT from the bottom line. Contracts, properly enforced, can make a huge dent in the problem of BCP38 adoption. At a number of levels. Equipment manufacturers not usually involved in this sort of thing (home and SOHO market) would then have market incentive to provide equipment at the low end that would provide BCP38 support. Especially equipment manufacturers that incorporate embedded Linux in their products. They can be creative in how they implement their product; let creativity blossom. I know, I know, BCP38 was originally directed at Internet Service Providers at their edge to upstreams. I'm thinking that BCP38 needs to be in place at any point -- every point? -- where you have a significant-sized collection of systems/devices aggregated to single upstream connections. Particular systems/devices where any source address can be generated and propagated -- including compromised desktop computers, compromised light bulbs, compromised wireless routers, compromised you-name-it. (That is one nice thing about NAT -- the bad guys can't build spoofed packets. They *can* build, um, "other" packets...which is a different subject entirely.) (N.B.: Now you know why I'm trying to get the simplest possible definition of BCP38 into words. The RFCs don't contain "executive summaries".)
They don't need to manage the router. The raw DSL modem, cable modem, etc. can watch the packets and see what's assigned. This would need new hardware, but it's not like this is happening quickly any other way. Yes, there are some consumer purchased DSL routers and cable routers, but doing what you can with what you can. FWIW, I believe most American ISPs *DO* manage their end-user routers. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Andrew White" <Andrew.White2@charter.com> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Tuesday, September 27, 2016 3:44:35 PM Subject: RE: BCP38 adoption "incentives"? Hi Mike, This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible. It would make sense for most ISPs to have egress filtering at the edge (transit and peering points) to filter out packets that should not originate from the ISP's ASN, although this does not prevent spoofing between points in the ISP's network. Andrew NB: My personal opinion and not official communiqué of Charter. Andrew White Desk: 314.394-9594 | Cell: 314-452-4386 | Jabber andrew.white2@charter.com Systems Engineer III, DAS DNS group Charter Communications 12405 Powerscourt Drive, St. Louis, MO 63131 -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Mike Hammett Sent: Tuesday, September 27, 2016 3:33 PM Cc: nanog@nanog.org Subject: Re: BCP38 adoption "incentives"? It would be incredibly low impact to have the residential CPE block any source address not assigned by the ISP. Done. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Stephen Satchell" <list@satchell.net> To: nanog@nanog.org Sent: Tuesday, September 27, 2016 7:31:24 AM Subject: BCP38 adoption "incentives"? Does anyone know if any upstream and tiered internet providers include in their connection contracts a mandatory requirement that all directly-connected routers be in compliance with BCP38? Does anyone know if large ISPs like Comcast, Charter, or AT&T have put in place internal policies requiring retail/business-customer-aggregating routers to be in compliance with BCP38? Does any ISP, providing business Internet connectivity along with a block of IP addresses, include language in their contracts that any directly connected router must be in compliance with BCP38? I've seen a lot of moaning and groaning about how BCP38 is pretty much being ignored. Education is one way to help, but that doesn't hit anyone in the wallet. You have to motivate people to go out of their way to *learn* about BCP38; most business people are too busy with things that make them money to be concerned with "Internet esoterica" that doesn't add to the bottom line. You have to make their ignorance SUBTRACT from the bottom line. Contracts, properly enforced, can make a huge dent in the problem of BCP38 adoption. At a number of levels. Equipment manufacturers not usually involved in this sort of thing (home and SOHO market) would then have market incentive to provide equipment at the low end that would provide BCP38 support. Especially equipment manufacturers that incorporate embedded Linux in their products. They can be creative in how they implement their product; let creativity blossom. I know, I know, BCP38 was originally directed at Internet Service Providers at their edge to upstreams. I'm thinking that BCP38 needs to be in place at any point -- every point? -- where you have a significant-sized collection of systems/devices aggregated to single upstream connections. Particular systems/devices where any source address can be generated and propagated -- including compromised desktop computers, compromised light bulbs, compromised wireless routers, compromised you-name-it. (That is one nice thing about NAT -- the bad guys can't build spoofed packets. They *can* build, um, "other" packets...which is a different subject entirely.) (N.B.: Now you know why I'm trying to get the simplest possible definition of BCP38 into words. The RFCs don't contain "executive summaries".)
At least as far as cable is concerned, there is already configuration on the CMTS (e.g. https://www.cisco.com/c/en/us/support/docs/broadband-cable/cable-security/20... <https://www.cisco.com/c/en/us/support/docs/broadband-cable/cable-security/20691-source-verify.html>) that rejects things not coming from the assigned address, and AFAIK, it's best practice to enable it for more reasons than attack prevention. However... most residential IPv4 traffic lives behind a NATing CPE. The CPE will either: a) drop anything sourced from addresses not part of the configured LAN prefix b) NAT everything regardless of its source c) NAT things from its configured LAN, but bridge/forward anything else A and C result in spoofed traffic being dropped, either at the CPE or the CMTS. Same is true if the CPE itself has been compromised and is sending spoofed traffic. B results in it no longer being spoofed traffic, meaning that it defuses reflection attacks (the source address is no longer your attack target's address) but if it's raw packet floods, the attack still works but is now traceable back to its source. The behavior of a specific CPE is largely dependent on its raw source materials. Many CPE cheap plastic routers are built from a few common reference architectures from the chipset makers (Broadcom, Intel, etc) and then modified and adapted to brand their UI with the name silk-screened on the plastic, add features to distinguish one cheap plastic router from another, etc. Reasonably recent linux-based kernels do some of A by themselves, may even do things like RPF check, TCP sequence number window check, state comparison, so unless the CPE vendor defeats it when they adapt it for their use, it mostly works. Devices built to captive standards (i.e. purpose-built for Cable, DSL providers) could have specific guidance about which behavior is the correct one, but that may or may not affect what happens to the ones that show up at your favorite big box retailer. --Wes George, who has learned a thing or two about cable, but is speaking only for himself.
On Sep 27, 2016, at 4:51 PM, Mike Hammett <nanog@ics-il.net> wrote:
They don't need to manage the router. The raw DSL modem, cable modem, etc. can watch the packets and see what's assigned. This would need new hardware, but it's not like this is happening quickly any other way. Yes, there are some consumer purchased DSL routers and cable routers, but doing what you can with what you can.
FWIW, I believe most American ISPs *DO* manage their end-user routers.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
----- Original Message -----
From: "Andrew White" <Andrew.White2@charter.com> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Tuesday, September 27, 2016 3:44:35 PM Subject: RE: BCP38 adoption "incentives"?
Hi Mike,
This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible.
It would make sense for most ISPs to have egress filtering at the edge (transit and peering points) to filter out packets that should not originate from the ISP's ASN, although this does not prevent spoofing between points in the ISP's network.
Andrew
NB: My personal opinion and not official communiqué of Charter.
Andrew White Desk: 314.394-9594 | Cell: 314-452-4386 | Jabber andrew.white2@charter.com Systems Engineer III, DAS DNS group Charter Communications 12405 Powerscourt Drive, St. Louis, MO 63131
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Mike Hammett Sent: Tuesday, September 27, 2016 3:33 PM Cc: nanog@nanog.org Subject: Re: BCP38 adoption "incentives"?
It would be incredibly low impact to have the residential CPE block any source address not assigned by the ISP. Done.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
----- Original Message -----
From: "Stephen Satchell" <list@satchell.net> To: nanog@nanog.org Sent: Tuesday, September 27, 2016 7:31:24 AM Subject: BCP38 adoption "incentives"?
Does anyone know if any upstream and tiered internet providers include in their connection contracts a mandatory requirement that all directly-connected routers be in compliance with BCP38?
Does anyone know if large ISPs like Comcast, Charter, or AT&T have put in place internal policies requiring retail/business-customer-aggregating routers to be in compliance with BCP38?
Does any ISP, providing business Internet connectivity along with a block of IP addresses, include language in their contracts that any directly connected router must be in compliance with BCP38?
I've seen a lot of moaning and groaning about how BCP38 is pretty much being ignored. Education is one way to help, but that doesn't hit anyone in the wallet. You have to motivate people to go out of their way to *learn* about BCP38; most business people are too busy with things that make them money to be concerned with "Internet esoterica" that doesn't add to the bottom line. You have to make their ignorance SUBTRACT from the bottom line.
Contracts, properly enforced, can make a huge dent in the problem of BCP38 adoption. At a number of levels.
Equipment manufacturers not usually involved in this sort of thing (home and SOHO market) would then have market incentive to provide equipment at the low end that would provide BCP38 support. Especially equipment manufacturers that incorporate embedded Linux in their products. They can be creative in how they implement their product; let creativity blossom.
I know, I know, BCP38 was originally directed at Internet Service Providers at their edge to upstreams. I'm thinking that BCP38 needs to be in place at any point -- every point? -- where you have a significant-sized collection of systems/devices aggregated to single upstream connections. Particular systems/devices where any source address can be generated and propagated -- including compromised desktop computers, compromised light bulbs, compromised wireless routers, compromised you-name-it.
(That is one nice thing about NAT -- the bad guys can't build spoofed packets. They *can* build, um, "other" packets...which is a different subject entirely.)
(N.B.: Now you know why I'm trying to get the simplest possible definition of BCP38 into words. The RFCs don't contain "executive summaries".)
IPv6? Is that common in CMTSes or just in certain ones? ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Wesley George" <wesgeorge@puck.nether.net> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Wednesday, September 28, 2016 10:08:00 AM Subject: Re: BCP38 adoption "incentives"? At least as far as cable is concerned, there is already configuration on the CMTS (e.g. https://www.cisco.com/c/en/us/support/docs/broadband-cable/cable-security/20... ) that rejects things not coming from the assigned address, and AFAIK, it's best practice to enable it for more reasons than attack prevention. However... most residential IPv4 traffic lives behind a NATing CPE. The CPE will either: a) drop anything sourced from addresses not part of the configured LAN prefix b) NAT everything regardless of its source c) NAT things from its configured LAN, but bridge/forward anything else A and C result in spoofed traffic being dropped, either at the CPE or the CMTS. Same is true if the CPE itself has been compromised and is sending spoofed traffic. B results in it no longer being spoofed traffic, meaning that it defuses reflection attacks (the source address is no longer your attack target's address) but if it's raw packet floods, the attack still works but is now traceable back to its source. The behavior of a specific CPE is largely dependent on its raw source materials. Many CPE cheap plastic routers are built from a few common reference architectures from the chipset makers (Broadcom, Intel, etc) and then modified and adapted to brand their UI with the name silk-screened on the plastic, add features to distinguish one cheap plastic router from another, etc. Reasonably recent linux-based kernels do some of A by themselves, may even do things like RPF check, TCP sequence number window check, state comparison, so unless the CPE vendor defeats it when they adapt it for their use, it mostly works. Devices built to captive standards (i.e. purpose-built for Cable, DSL providers) could have specific guidance about which behavior is the correct one, but that may or may not affect what happens to the ones that show up at your favorite big box retailer. --Wes George, who has learned a thing or two about cable, but is speaking only for himself. On Sep 27, 2016, at 4:51 PM, Mike Hammett < nanog@ics-il.net > wrote: They don't need to manage the router. The raw DSL modem, cable modem, etc. can watch the packets and see what's assigned. This would need new hardware, but it's not like this is happening quickly any other way. Yes, there are some consumer purchased DSL routers and cable routers, but doing what you can with what you can. FWIW, I believe most American ISPs *DO* manage their end-user routers. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Andrew White" < Andrew.White2@charter.com > To: "Mike Hammett" < nanog@ics-il.net > Cc: nanog@nanog.org Sent: Tuesday, September 27, 2016 3:44:35 PM Subject: RE: BCP38 adoption "incentives"? Hi Mike, This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible. It would make sense for most ISPs to have egress filtering at the edge (transit and peering points) to filter out packets that should not originate from the ISP's ASN, although this does not prevent spoofing between points in the ISP's network. Andrew NB: My personal opinion and not official communiqué of Charter. Andrew White Desk: 314.394-9594 | Cell: 314-452-4386 | Jabber andrew.white2@charter.com Systems Engineer III, DAS DNS group Charter Communications 12405 Powerscourt Drive, St. Louis, MO 63131 -----Original Message----- From: NANOG [ mailto:nanog-bounces@nanog.org ] On Behalf Of Mike Hammett Sent: Tuesday, September 27, 2016 3:33 PM Cc: nanog@nanog.org Subject: Re: BCP38 adoption "incentives"? It would be incredibly low impact to have the residential CPE block any source address not assigned by the ISP. Done. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Stephen Satchell" < list@satchell.net > To: nanog@nanog.org Sent: Tuesday, September 27, 2016 7:31:24 AM Subject: BCP38 adoption "incentives"? Does anyone know if any upstream and tiered internet providers include in their connection contracts a mandatory requirement that all directly-connected routers be in compliance with BCP38? Does anyone know if large ISPs like Comcast, Charter, or AT&T have put in place internal policies requiring retail/business-customer-aggregating routers to be in compliance with BCP38? Does any ISP, providing business Internet connectivity along with a block of IP addresses, include language in their contracts that any directly connected router must be in compliance with BCP38? I've seen a lot of moaning and groaning about how BCP38 is pretty much being ignored. Education is one way to help, but that doesn't hit anyone in the wallet. You have to motivate people to go out of their way to *learn* about BCP38; most business people are too busy with things that make them money to be concerned with "Internet esoterica" that doesn't add to the bottom line. You have to make their ignorance SUBTRACT from the bottom line. Contracts, properly enforced, can make a huge dent in the problem of BCP38 adoption. At a number of levels. Equipment manufacturers not usually involved in this sort of thing (home and SOHO market) would then have market incentive to provide equipment at the low end that would provide BCP38 support. Especially equipment manufacturers that incorporate embedded Linux in their products. They can be creative in how they implement their product; let creativity blossom. I know, I know, BCP38 was originally directed at Internet Service Providers at their edge to upstreams. I'm thinking that BCP38 needs to be in place at any point -- every point? -- where you have a significant-sized collection of systems/devices aggregated to single upstream connections. Particular systems/devices where any source address can be generated and propagated -- including compromised desktop computers, compromised light bulbs, compromised wireless routers, compromised you-name-it. (That is one nice thing about NAT -- the bad guys can't build spoofed packets. They *can* build, um, "other" packets...which is a different subject entirely.) (N.B.: Now you know why I'm trying to get the simplest possible definition of BCP38 into words. The RFCs don't contain "executive summaries".)
On Tue, 27 Sep 2016, White, Andrew wrote:
This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible.
Which is why the manufacturer should deploy a default config which does this. Whatever the WAN IP, and by default, and in 90%+ configurations, there is a single WAN IP for CPE, ACLs are automatically managed to block all outbound packets that are NOT From: the WAN IP. And when DHCP or PPPoE gives a new IP, the rules are rewritten automatically by the CPE with updated rules. This won't fix the DDOS attach from IoT devices or IP Cameras or whatnot that don't attempt to hide their IP, but it would help with spoofing at the edge for the non-network saavy.
It would make sense for most ISPs to have egress filtering at the edge (transit and peering points) to filter out packets that should not originate from the ISP's ASN, although this does not prevent spoofing between points in the ISP's network.
Multi-tiered approaches are excellent. Start with the CPE, move to your aggs, then your big iron at the edges. Automate deployments and rule generation. --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
On Tue, 27 Sep 2016 20:44:35 -0000, "White, Andrew" said:
This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible.
Hopefully, if you've been burnt by this, you remembered to add it to the requirements list for the next time you buy customer-facing gear.
In a message written on Tue, Sep 27, 2016 at 08:44:35PM +0000, White, Andrew wrote:
This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible.
Unicast RFP should be a feature every ISP requires of all edge devices for at least 15 years now. It should be on by default for virtually all connections, and disabled only by request or when there are circumstances to suggest it would break things (e.g. a request for BGP with full tables over the link). At this point there's no excuse, anyone who has gear who can't do that has been asleep at the switch. It's been a standard feature in too much gear for too long. -- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/
Well there is money to be made in DDoS protection... See our "friends" still hosting "those" pay sites. Do not expect the vendors to cut themself of that market. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 09/29/16 11:31, Leo Bicknell wrote:
This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible. Unicast RFP should be a feature every ISP requires of all edge devices for at least 15 years now. It should be on by default for virtually all connections, and disabled only by request or when
In a message written on Tue, Sep 27, 2016 at 08:44:35PM +0000, White, Andrew wrote: there are circumstances to suggest it would break things (e.g. a request for BGP with full tables over the link).
At this point there's no excuse, anyone who has gear who can't do that has been asleep at the switch. It's been a standard feature in too much gear for too long.
Even if the customers are unaware of the spoofed traffic, ISPs should be aware which leaves them open for "aiding and abetting". This doesn't require inspecting the payload of the packets. This is the IP header which they are expected to examine and for which there is a BCP saying to drop spoofed packets. Sources are used for policy routing so the source field is expected to be processed. I would expect a Judge to take into consideration the BCP in deciding whether a ISP should be aware of the issue when deciding if a ISP is aiding and abetting by allowing spoofed packets to enter their network. Mark In message <b01d17bf-c4fe-4a60-0f1e-f7c2e61c5650@pubnix.net>, Alain Hebert writes:
Well there is money to be made in DDoS protection... See our "friends" still hosting "those" pay sites.
Do not expect the vendors to cut themself of that market.
----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
On 09/29/16 11:31, Leo Bicknell wrote:
This assumes the ISP manages the customer's CPE or home router, which is often not the case. Adding such ACLs to the upstream device, operated by the ISP, is not always easy or feasible. Unicast RFP should be a feature every ISP requires of all edge devices for at least 15 years now. It should be on by default for virtually all connections, and disabled only by request or when
In a message written on Tue, Sep 27, 2016 at 08:44:35PM +0000, White, Andrew wrote: there are circumstances to suggest it would break things (e.g. a request for BGP with full tables over the link).
At this point there's no excuse, anyone who has gear who can't do that has been asleep at the switch. It's been a standard feature in too much gear for too long.
-- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
participants (16)
-
Alain Hebert -
Jay R. Ashworth -
Joe Klein -
Leo Bicknell -
Mark Andrews -
Mikael Abrahamsson -
Mike Hammett -
Mike Jones -
Miquel van Smoorenburg -
Nick Hilliard -
Peter Beckman -
Stephen Satchell -
Valdis.Kletnieks＠vt.edu -
Wesley George -
White, Andrew -
Zbyněk Pospíchal