Announcing Peering-LAN prefixes to customers
Hi all, this might be a stupid question but today I was discussing with a colleague if Peering-LAN prefixes should be re-distributed/announced to direct customers/peers. My standpoint is that in any case, Peering-LAN prefixes should be filtered and not announced to peers/customers because a Peering-LAN represents some sort of DMZ and there is simply no need for them to be reachable by third-parties not being physically connected to an IXP themselves. Also from a security point of view, a lot of new issues might occur in this situation. I’ve been seeing a few transit providers lately announcing (even reachable) Peering-LAN prefixes (for example DE-CIX Peering LAN) to their customers. I’m wondering if there is any document or RFC particularly describing this matter? Thanks Dominic
Dear Dominic, On Thu, Dec 20, 2018 at 6:49 PM Dominic Schallert <ds@schallert.com> wrote:
this might be a stupid question but today I was discussing with a colleague if Peering-LAN prefixes should be re-distributed/announced to direct customers/peers. My standpoint is that in any case, Peering-LAN prefixes should be filtered and not announced to peers/customers because a Peering-LAN represents some sort of DMZ and there is simply no need for them to be reachable by third-parties not being physically connected to an IXP themselves. Also from a security point of view, a lot of new issues might occur in this situation.
I’ve been seeing a few transit providers lately announcing (even reachable) Peering-LAN prefixes (for example DE-CIX Peering LAN) to their customers. I’m wondering if there is any document or RFC particularly describing this matter?
It is NTT's policy to reject Peering LAN prefixes (and any more-specifics) of any IXPs NTT is connected; on both our ingress EBGP and egress EBGP policies. We don't see a need for NTT to attempt to make such peering lan networks reachable for third parties. Such reachability may negatively impact operations, especially when more-specifics of Peering LAN prefixes are distributed through the default-free zone. As a consequence, for IXPs this policy suggests that it is a necessity to host their own infrastructure (IXP website, mail server, etc) outside the peering lan prefix. Kind regards, Job
Job Snijders Sent: Thursday, December 20, 2018 5:55 PM We don't see a need for NTT to attempt to make such peering lan networks reachable for third parties. Such reachability may negatively impact operations, especially when more-specifics of Peering LAN prefixes are distributed through the default-free zone.
As a consequence, for IXPs this policy suggests that it is a necessity to host their own infrastructure (IXP website, mail server, etc) outside the peering lan prefix.
Reading this thread makes me wonder if the reasoning mentioned thus far should in fact be extrapolated to any Providers' infrastructure prefixes (essentially the plumbing prefixes). Wondering what is the community's stance on this. adam
This brings to mind the following (old) blog post from CloudFlare: https://blog.cloudflare.com/the-ddos-that-almost-broke-the-internet/ Relevant excerpt here:
Beyond attacking CloudFlare's direct peers, the attackers also attacked the core IX infrastructure on the London Internet Exchange (LINX), the Amsterdam Internet Exchange (AMS-IX), the Frankfurt Internet Exchange (DE-CIX), and the Hong Kong Internet Exchange (HKIX). From our perspective, the attacks had the largest effect on LINX which caused impact over the exchange and LINX's systems that monitor the exchange, as visible through the drop in traffic recorded by their monitoring systems. (Corrected: see below for original phrasing.) The congestion impacted many of the networks on the IXs, including CloudFlare's. As problems were detected on the IX, we would route traffic around them. However, several London-based CloudFlare users reported intermittent issues over the last several days. This is the root cause of those problems. The attacks also exposed some vulnerabilities in the architecture of some IXs. We, along with many other network security experts, worked with the team at LINX to better secure themselves. In doing so, we developed a list of best practices for any IX in order to make them less vulnerable to attacks. Two specific suggestions to limit attacks like this involve making it more difficult to attack the IP addresses that members of the IX use to interchange traffic between each other. We are working with IXs to ensure that: 1) these IP addresses should not be announced as routable across the public Internet; and 2) packets destined to these IP addresses should only be permitted from other IX IP addresses. We've been very impressed with the team at LINX and how quickly they've worked to implement these changes and add additional security to their IX and are hopeful other IXs will quickly follow their lead.
On Thu, Dec 20, 2018 at 12:51 PM Dominic Schallert <ds@schallert.com> wrote:
Hi all,
this might be a stupid question but today I was discussing with a colleague if Peering-LAN prefixes should be re-distributed/announced to direct customers/peers. My standpoint is that in any case, Peering-LAN prefixes should be filtered and not announced to peers/customers because a Peering-LAN represents some sort of DMZ and there is simply no need for them to be reachable by third-parties not being physically connected to an IXP themselves. Also from a security point of view, a lot of new issues might occur in this situation.
I’ve been seeing a few transit providers lately announcing (even reachable) Peering-LAN prefixes (for example DE-CIX Peering LAN) to their customers. I’m wondering if there is any document or RFC particularly describing this matter?
Thanks Dominic
IXP LANs should not be announced via BGP (or your IGP either). See section 3.1: http://nabcop.org/index.php/BCOP-Exchange_Points_v2 On Thu, Dec 20, 2018 at 12:50 PM Dominic Schallert <ds@schallert.com> wrote:
Hi all,
this might be a stupid question but today I was discussing with a colleague if Peering-LAN prefixes should be re-distributed/announced to direct customers/peers. My standpoint is that in any case, Peering-LAN prefixes should be filtered and not announced to peers/customers because a Peering-LAN represents some sort of DMZ and there is simply no need for them to be reachable by third-parties not being physically connected to an IXP themselves. Also from a security point of view, a lot of new issues might occur in this situation.
I’ve been seeing a few transit providers lately announcing (even reachable) Peering-LAN prefixes (for example DE-CIX Peering LAN) to their customers. I’m wondering if there is any document or RFC particularly describing this matter?
Thanks Dominic
-- [stillwaxin@gmail.com ~]$ cat .signature cat: .signature: No such file or directory [stillwaxin@gmail.com ~]$
Dear Job, Michael, Ross, thank you very much for sharing your opinion, the detailed info and references. That’s pretty much what I excpected. Just wondered because I couldn’t find any IXP Conection Agreement stating this „issue“ explicitly yet. Maybe MANRS IXP actions has some recommendations regarding this, checking that now. Best wishes and happy holidays Cheers Dominic
Am 20.12.2018 um 19:06 schrieb Michael Still <stillwaxin@gmail.com>:
IXP LANs should not be announced via BGP (or your IGP either). See section 3.1: http://nabcop.org/index.php/BCOP-Exchange_Points_v2 <http://nabcop.org/index.php/BCOP-Exchange_Points_v2>
On Thu, Dec 20, 2018 at 12:50 PM Dominic Schallert <ds@schallert.com <mailto:ds@schallert.com>> wrote: Hi all,
this might be a stupid question but today I was discussing with a colleague if Peering-LAN prefixes should be re-distributed/announced to direct customers/peers. My standpoint is that in any case, Peering-LAN prefixes should be filtered and not announced to peers/customers because a Peering-LAN represents some sort of DMZ and there is simply no need for them to be reachable by third-parties not being physically connected to an IXP themselves. Also from a security point of view, a lot of new issues might occur in this situation.
I’ve been seeing a few transit providers lately announcing (even reachable) Peering-LAN prefixes (for example DE-CIX Peering LAN) to their customers. I’m wondering if there is any document or RFC particularly describing this matter?
Thanks Dominic
-- [stillwaxin@gmail.com <mailto:stillwaxin@gmail.com> ~]$ cat .signature cat: .signature: No such file or directory [stillwaxin@gmail.com <mailto:stillwaxin@gmail.com> ~]$
Hi Dominic, On Thu, 2018-12-20 at 19:15 +0100, Dominic Schallert wrote:
Dear Job, Michael, Ross, thank you very much for sharing your opinion, the detailed info and references. That’s pretty much what I excpected. Just wondered because I couldn’t find any IXP Conection Agreement stating this „issue“ explicitly yet.
Maybe MANRS IXP actions has some recommendations regarding this, checking that now.
We don't have it in our connection agreement as such, but it is in section 3.2 of our (admittedly aged) Configuration Guide: https://ams-ix.net/technical/specifications-descriptions/config-guide#3.2 3.2. Peering LAN Prefix The IPv4 prefix for the AMS-IX peering LAN (80.249.208.0/21) is part of AS1200, and is not supposed to be globally routable. This means the following: 1. Do not configure "network 80.249.208.0/21" in your router's BGP configuration (seriously, we have seen this happen!). 2. Do not redistribute the route, a supernet, or a more specific outside of your AS. We (AS1200) announce it with a no-export attribute, please honour it. In short, you can take the view that the Peering LAN is a link-local address range and you may decide to not even redistribute it internally (but in that case you may want to set a static route for management access so you can troubleshoot peering, etc.). AFAIK, pretty much all IXP operators take this view. Cheers, Steven
Best wishes and happy holidays
Cheers Dominic
Am 20.12.2018 um 19:06 schrieb Michael Still <stillwaxin@gmail.com> :
IXP LANs should not be announced via BGP (or your IGP either). See section 3.1: http://nabcop.org/index.php/BCOP-Exchange_Points_v2
On Thu, Dec 20, 2018 at 12:50 PM Dominic Schallert < ds@schallert.com> wrote:
Hi all,
this might be a stupid question but today I was discussing with a colleague if Peering-LAN prefixes should be re- distributed/announced to direct customers/peers. My standpoint is that in any case, Peering-LAN prefixes should be filtered and not announced to peers/customers because a Peering-LAN represents some sort of DMZ and there is simply no need for them to be reachable by third-parties not being physically connected to an IXP themselves. Also from a security point of view, a lot of new issues might occur in this situation.
I’ve been seeing a few transit providers lately announcing (even reachable) Peering-LAN prefixes (for example DE-CIX Peering LAN) to their customers. I’m wondering if there is any document or RFC particularly describing this matter?
Thanks Dominic
-- [stillwaxin@gmail.com ~]$ cat .signature cat: .signature: No such file or directory [stillwaxin@gmail.com ~]$
Hi, Dominic -- On 20/12/2018, 17:49, Dominic Schallert <ds@schallert.com> wrote:
this might be a stupid question but today I was discussing with a colleague if Peering-LAN prefixes should be re-distributed/announced to direct customers/peers. My standpoint is that in any case, Peering-LAN prefixes should be filtered and not announced to peers/customers because a Peering-LAN represents some sort of DMZ and there is simply no need for them to be reachable by third-parties not being physically connected to an IXP themselves.
There are no stupid questions! It is a good idea to not BGP announce and perhaps also to drop traffic toward peering LAN prefixes at customer-borders, this was already well discussed in the thread. But there wasn’t a discussion on how we got to this point. Until the Cloudflare 2013 BGP speaker attack, that sought to flood Cloudflare’s transfer networks and exchange connectivity (and with it saturating IXP inter-switch links and IXP participant ports), it was common for IXP IPv4/6 peering LANs to be internet reachable and BGP transited. This facilitated troubleshooting (e.g. traceroutes showing peering lan interfaces in traceroutes instead of ‘starring out’) and PMTUD (e.g. see recommendation in https://www.ripe.net/ripe/mail/archives/ipv6-wg/2011-July/001839.html which actually asked for IXP peering LANs to be announced). There are good reasons to announce but there are better reasons to filter. The security benefits of filtering outweigh the upsides on today’s internet, but fashions and best practice may further evolve over time. Andy -- Andy Davidson Director, Asteroid International BV www.asteroidhq.com Director, Euro-IX - The European Internet Exchange Association www.euro-ix.net
On 3/Jan/19 22:08, Andy Davidson wrote:
There are no stupid questions! It is a good idea to not BGP announce and perhaps also to drop traffic toward peering LAN prefixes at customer-borders, this was already well discussed in the thread. But there wasn’t a discussion on how we got to this point. Until the Cloudflare 2013 BGP speaker attack, that sought to flood Cloudflare’s transfer networks and exchange connectivity (and with it saturating IXP inter-switch links and IXP participant ports), it was common for IXP IPv4/6 peering LANs to be internet reachable and BGP transited.
That's interesting to learn. Running a few exchange points in Africa since 2002, the news was that the exchange point LAN should not be visible anywhere on the Internet. It would be interesting to know that this wasn't the case in other parts of the world. Mark.
On Wed, Jan 16, 2019 at 10:56 Mark Tinka <mark.tinka@seacom.mu> wrote:
On 3/Jan/19 22:08, Andy Davidson wrote:
There are no stupid questions! It is a good idea to not BGP announce and perhaps also to drop traffic toward peering LAN prefixes at customer-borders, this was already well discussed in the thread. But there wasn’t a discussion on how we got to this point. Until the Cloudflare 2013 BGP speaker attack, that sought to flood Cloudflare’s transfer networks and exchange connectivity (and with it saturating IXP inter-switch links and IXP participant ports), it was common for IXP IPv4/6 peering LANs to be internet reachable and BGP transited.
That's interesting to learn.
Running a few exchange points in Africa since 2002, the news was that the exchange point LAN should not be visible anywhere on the Internet. It would be interesting to know that this wasn't the case in other parts of the world.
Some IX’s use a globally reachable peering lan prefix as a convenience for their participants as “poor man’s out-of-band”, or can’t designate a separate /24 for the IXP’s website / public services. I can see some use cases, but in today’s internet landscape the practice just increases the attack surface, so it’s not the Best Current Practise. Kind regards, Job
On 16/01/2019 08:56, Mark Tinka wrote:
Running a few exchange points in Africa since 2002, the news was that the exchange point LAN should not be visible anywhere on the Internet.
Do you use AS0 as origin on the RPKI objects for said exchange point LAN(s) to prevent route propagation? -Christoffer
On Wed, Jan 16, 2019 at 12:39 Christoffer Hansen <christoffer@netravnen.de> wrote:
On 16/01/2019 08:56, Mark Tinka wrote:
Running a few exchange points in Africa since 2002, the news was that the exchange point LAN should not be visible anywhere on the Internet.
Do you use AS0 as origin on the RPKI objects for said exchange point LAN(s) to prevent route propagation?
Either AS 0, or the ASN of the IXP’s service network are valid options. Whatever ASN is listed in the RPKI ROA, should simply never announce the prefix. IXPs should make sure to not set MaxLength to allow anything more-l specific, it should be equal to the prefix length. Kind regards, Job
(Perhaps off-topic) KINX are using 192.145.251.0/24 as their Peering IPv4 space. However, I couldn't find any valid SWIP or IRR record created by IP owner Hiawatha Broadband Communications, Inc. Some ISP like Hurricane Electric will route this prefix to KINX but I'm not sure if it's authorized by IP owner. Regards, Siyuan On Wed, Jan 16, 2019 at 5:44 PM Job Snijders <job@ntt.net> wrote:
On Wed, Jan 16, 2019 at 12:39 Christoffer Hansen <christoffer@netravnen.de> wrote:
On 16/01/2019 08:56, Mark Tinka wrote:
Running a few exchange points in Africa since 2002, the news was that the exchange point LAN should not be visible anywhere on the Internet.
Do you use AS0 as origin on the RPKI objects for said exchange point LAN(s) to prevent route propagation?
Either AS 0, or the ASN of the IXP’s service network are valid options. Whatever ASN is listed in the RPKI ROA, should simply never announce the prefix.
IXPs should make sure to not set MaxLength to allow anything more-l specific, it should be equal to the prefix length.
Kind regards,
Job
On Wed, Jan 16, 2019 at 14:49 Mark Tinka <mark.tinka@seacom.mu> wrote:
On 16/Jan/19 11:38, Christoffer Hansen wrote:
Do you use AS0 as origin on the RPKI objects for said exchange point LAN(s) to prevent route propagation?
I don't operate any exchange points anymore, but I am not aware of any such operation of AS0 this side of the globe.
Perhaps that is because for a while you couldn’t set AS 0 in AFRINIC ROAs as Origin ASN. I think that’s fixed now. Kind regards, Job
On Wed, 16 Jan 2019, Amreesh Phokeer wrote:
On Wed, Jan 16, 2019 at 3:55 PM Job Snijders <job@ntt.net> wrote: Perhaps that is because for a while you couldn’t set AS 0 in AFRINIC ROAs as Origin ASN. I think that’s fixed now.
You absolutely can. But not sure if anyone ever created one.
not for AFRINIC, see http://rpki-browser.realmv6.org/ (select AFRINIC, filter for Resource AS0) Cheers matthias -- Matthias Waehlisch . Freie Universitaet Berlin, Computer Science .. http://www.cs.fu-berlin.de/~waehl
On Wed, Jan 16, 2019 at 15:24 Randy Bush <randy@psg.com> wrote:
Do you use AS0 as origin on the RPKI objects for said exchange point LAN(s) to prevent route propagation?
but as0 does not exactly do that as it can be overridden by a different roa for the same prefix. as0 is pretty useless.
Why would the IXP create such a second ROA? Do you mean to say “a tool can be useless when applied incorrectly”?
Running a few exchange points in Africa since 2002, the news was that the exchange point LAN should not be visible anywhere on the Internet. It would be interesting to know that this wasn't the case in other parts of the world.
slide 8 of http://archive.psg.com/970210.nanog.pdf
participants (14)
-
adamv0025＠netconsultings.com -
Amreesh Phokeer -
Andy Davidson -
Christoffer Hansen -
Dominic Schallert -
Job Snijders -
Job Snijders -
Mark Tinka -
Matthias Waehlisch -
Michael Still -
Randy Bush -
Ross Tajvar -
Siyuan Miao -
Steven Bakker