Route Server Filters at IXPs and 4-byte ASNs
Hi there, as 2-byte ASNs are close to depletion (see NRO announcement this week), we have come across a topic, that might influence the adoption of 4-byte ASNs. First of all, we have no data or experience about 4-byte ASN adoption and are therefore unable to evaluate, if we should rush for the last available numbers. So here's the thing: IXPs usually implement N:M filtering based on standard community strings. As standard BGP communities support only 4 bytes, this only works for IXPs with 2-byte ASNs and peers with 2-byte ASNs. Extended communities to the rescue? We are not sure: While RFC4260 (Extended Communities) allows for <Global Admin,2bytes>:<Local Admin, 4bytes> community strings (3.1 Two-Octet AS Specific Extended Community), this works as long as the IXP itself uses a 2-byte ASN. What happens, if the IXP uses a 4-byte ASN? RFC5668 (4-Octet AS Specific BGP Extended Community) defines <Global Admin,4bytes>:<Local Admin, 2bytes>. I have been asking some IXP operators, about their practice and their reply was "4-byte ASNs are supported by our RS". What's your experience? Did you see IXPs, that do not support them? Do you think, the IXP operators are aware of this limitation? Have you seen IXPs with 4-byte ASNs or do RIRs reserve 2-byte ASNs for future IXPs? What other operational problems did you experience while using 4-byte ASNs? A lot of questions. I am very curious about your answers. Cheers, Sebastian -- SEBASTIAN SPIES lnked.in/sspies
What happens, if the IXP uses a 4-byte ASN? RFC5668 (4-Octet AS Specific BGP Extended Community) defines <Global Admin,4bytes>:<Local Admin, 2bytes>.
I have been asking some IXP operators, about their practice and their reply was "4-byte ASNs are supported by our RS". What's your experience? Did you see IXPs, that do not support them? Do you think, the IXP operators are aware of this limitation? Have you seen IXPs with 4-byte ASNs or do RIRs reserve 2-byte ASNs for future IXPs? What other operational problems did you experience while using 4-byte ASNs?
Do you see an issue with future IXP or future IXP members ? You can see at this list of members of one IXP that there are both 2-byte and 4-byte ASNs living in harmony, with a large number of 4-byte ones: http://ptt.br/particip/sp The IXP itself is using a 2-byte public ASN, so if at some point communities are used (although there are route-servers capable of granular policy without communities), they could use standard 2-byte community format. And with a bit coordination, 2-byte private ASNs and communities could be used to overcome limitations in member routers support of 4-byte ASNs. Rubens telnet lg.sp.ptt.br Trying 200.160.1.3... Connected to lg.sp.ptt.br. Escape character is '^]'. ============================================== = PTTMetro SP = = Contact: eng@ptt.br = = +55 11 5509-3550 = = inoc-dba: 22548*100 = = = = Looking Glass Server = = All connections and keystrokes logged = ============================================== lg.sp.ptt.br> show ip bgp summary BGP router identifier 187.16.218.252, local AS number 20121 RIB entries 32868, using 3081 KiB of memory Peers 4, using 18 KiB of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 187.16.216.253 4 26162 3294906 1619885 0 0 0 05w0d23h 19247 187.16.216.254 4 26162 3468468 1617924 0 0 0 01w5d06h 19227 Total number of neighbors 2
Dear Sebastian, On Sat, Jan 25, 2014 at 02:56:16PM +0100, Sebastian Spies wrote:
So here's the thing: IXPs usually implement N:M filtering based on standard community strings. As standard BGP communities support only 4 bytes, this only works for IXPs with 2-byte ASNs and peers with 2-byte ASNs.
There are various directions in which a workable solution may for IXP Operators: - Use a mapping table: 32 bit values to 16 bit values. Every participant with a 4-byte ASN is represented with 2-bytes, there are pro's and con's to this approach. - Use an out-of-band mechanism which allows IXP participants to signal the desired routing policy to the Route Server. (VIX.at offers a web-interface where people can click and point to which ASNs they want to export or not). Another approach would be to signal the desired routing policy through RPSL. I've been working on some extensions which would allow an IXP participant to publish what the Route Server should do on their behalf, in this example 6777 is a Route Server and AS4247483647 is an IXP participant with a 4-byte ASN: import-via: AS6777 from AS4247483647 accept AS4247483647 export-via: AS6777 to AS4247483647 announce AS-ATRATO (read more here: http://tools.ietf.org/html/draft-snijders-rpsl-via ) Support for these extensions will be available in the next release of the RIPE Whois Database. After that IXP Operators can consider implementing support for RPSL-via. All of the above approaches have disadvantages: - Mapping tables add complexity - Most Out-of-band methods would differ from each other - RPSL-via is has yet to be implemented in the wild
Extended communities to the rescue?
Nope. Not as it stands today.
I have been asking some IXP operators, about their practice and their reply was "4-byte ASNs are supported by our RS". What's your experience?
Most of them are lying through their teeth. :-) Kind regards, Job
I have over 100,000 servers located in routing diverse datacenters with 4byte ASN numbers and have not had 1 problem or complaint related to the ASN for not able to communicate with the datacenter. The first 1 did make me really nervous for all of the reasons already mentioned but turned out to be a non-issue. While it would be nice to see community string support increase to 4byte, I think this is more of an educational challenge for the IXPs on how to setup your community strings to work and not really a technical problem. Bryan
On Sat, Jan 25, 2014 at 10:04:30AM -0500, Bryan Socha wrote:
I have over 100,000 servers located in routing diverse datacenters with 4byte ASN numbers and have not had 1 problem or complaint related to the ASN for not able to communicate with the datacenter. The first 1 did make me really nervous for all of the reasons already mentioned but turned out to be a non-issue.
This thread is not about reachability of prefixes announced by 4-byte ASNs. This thread is about prefix filtering on Route Servers at Internet Exchanges.
While it would be nice to see community string support increase to 4byte, I think this is more of an educational challenge for the IXPs on how to setup your community strings to work and not really a technical problem.
Can you elaborate on how you would setup 'community strings'? Kind regards, Job
Re-reading, I was thinking of someone connecting to an IXP, not a new IXP needing a 2Byte. This is an interesting situation and you are correct, my comment was off topic. *Bryan Socha* Network Engineer 646.450.0472 | *bryan@serverstack.com <bryan@serverstack.com>* *ServerStack* | Scale Big On Sat, Jan 25, 2014 at 10:18 AM, Job Snijders < job.snijders@hibernianetworks.com> wrote:
On Sat, Jan 25, 2014 at 10:04:30AM -0500, Bryan Socha wrote:
I have over 100,000 servers located in routing diverse datacenters with 4byte ASN numbers and have not had 1 problem or complaint related to the ASN for not able to communicate with the datacenter. The first 1 did make me really nervous for all of the reasons already mentioned but turned out to be a non-issue.
This thread is not about reachability of prefixes announced by 4-byte ASNs. This thread is about prefix filtering on Route Servers at Internet Exchanges.
While it would be nice to see community string support increase to 4byte, I think this is more of an educational challenge for the IXPs on how to setup your community strings to work and not really a technical problem.
Can you elaborate on how you would setup 'community strings'?
Kind regards,
Job
Am 25.01.2014 16:38, schrieb Bryan Socha:
Re-reading, I was thinking of someone connecting to an IXP, not a new IXP needing a 2Byte. This is an interesting situation and you are correct, my comment was off topic.
Sorry for not mentioning the beef: Extended Communities effectively leave 6 bytes to the user. So, it is not possible, that a 4-byte-ASN IXP encodes its own 4-byte-ASN and a 4-byte-ASN customer in one extended community string. This is needed, as the IXP RS usually interpret their own ASN and a peer ASN as: "Send this prefix out to the peer ASN". To make things worse: even if the IXPs ASN is 2-byte, I would assume, that RS implementors chose to interpret extended community strings as always being in the format 4-byte:2-byte (see RFC5668). Best regards, Sebastian
On 25/01/2014 15:48, Sebastian Spies wrote:
To make things worse: even if the IXPs ASN is 2-byte, I would assume, that RS implementors chose to interpret extended community strings as always being in the format 4-byte:2-byte (see RFC5668).
some ixp operators (e.g. me) are rather enthusiastic about the idea of a modified form of draft-raszuk-wide-bgp-communities getting more traction. This would solve this particular problem and many others. Nick
Sent from my iPad
On Jan 25, 2014, at 1:37 PM, Nick Hilliard <nick@foobar.org> wrote:
On 25/01/2014 15:48, Sebastian Spies wrote: To make things worse: even if the IXPs ASN is 2-byte, I would assume, that RS implementors chose to interpret extended community strings as always being in the format 4-byte:2-byte (see RFC5668).
some ixp operators (e.g. me) are rather enthusiastic about the idea of a modified form of draft-raszuk-wide-bgp-communities getting more traction. This would solve this particular problem and many others.
Wide communities is the wrong tool here. You want this: http://tools.ietf.org/html/draft-ietf-idr-as4octet-extcomm-generic-subtype-0... -- Jeff
Nick
Jeffrey, On Tue, 4 Feb 2014 22:53:40 -0500 Jeffrey Haas <jhaas@pfrc.org> wrote:
Sent from my iPad
On Jan 25, 2014, at 1:37 PM, Nick Hilliard <nick@foobar.org> wrote:
On 25/01/2014 15:48, Sebastian Spies wrote: To make things worse: even if the IXPs ASN is 2-byte, I would assume, that RS implementors chose to interpret extended community strings as always being in the format 4-byte:2-byte (see RFC5668).
some ixp operators (e.g. me) are rather enthusiastic about the idea of a modified form of draft-raszuk-wide-bgp-communities getting more traction. This would solve this particular problem and many others.
Wide communities is the wrong tool here. You want this: http://tools.ietf.org/html/draft-ietf-idr-as4octet-extcomm-generic-subtype-0...
This draft does not cater for the use case of describing a 32-bit ASN peering with a 32-bit route server, which would require a 4-byte Global Administrator as well as a 4-byte Local Administrator sub-field. Kind regards, Martin
Martin, On Wed, Feb 05, 2014 at 10:06:31AM +0100, Martin Pels wrote:
Wide communities is the wrong tool here. You want this: http://tools.ietf.org/html/draft-ietf-idr-as4octet-extcomm-generic-subtype-0...
This draft does not cater for the use case of describing a 32-bit ASN peering with a 32-bit route server, which would require a 4-byte Global Administrator as well as a 4-byte Local Administrator sub-field.
I think that's the first clear articulation I've read about why some people want wide comms vs. a simple replacement for existing regular communities as extended communities. Thanks. Wide comms can do that, but they're intended to be a somewhat bigger hammer. This case is probably worth chatting with the authors and others in IDR at IETF to see if we should do something simpler. -- Jeff
On Feb 5, 2014, at 8:52 AM, Jeffrey Haas <jhaas@pfrc.org> wrote:
This draft does not cater for the use case of describing a 32-bit ASN peering with a 32-bit route server, which would require a 4-byte Global Administrator as well as a 4-byte Local Administrator sub-field.
I think that's the first clear articulation I've read about why some people want wide comms vs. a simple replacement for existing regular communities as extended communities. Thanks.
I suspect the operator confusion is that’s how they’ve been using 16-bit ASNs all along, so how did the IETF end up with something different. http://www.onesc.net/communities/ is a fairly comprehensive list of how they are used today. - jared
On Wed, Feb 05, 2014 at 09:02:52AM -0500, Jared Mauch wrote:
On Feb 5, 2014, at 8:52 AM, Jeffrey Haas <jhaas@pfrc.org> wrote:
This draft does not cater for the use case of describing a 32-bit ASN peering with a 32-bit route server, which would require a 4-byte Global Administrator as well as a 4-byte Local Administrator sub-field.
I think that's the first clear articulation I've read about why some people want wide comms vs. a simple replacement for existing regular communities as extended communities. Thanks.
I suspect the operator confusion is that?s how they?ve been using 16-bit ASNs all along, so how did the IETF end up with something different.
http://www.onesc.net/communities/ is a fairly comprehensive list of how they are used today.
Thanks for the list. Browsing the first several entries somewhat supports the reason why I'm acting "surprised". While there are some cases where the right hand side of the community is an AS number, a significant amount of it was either an arbitrary value or something more structured. The generic 4-byte draft I'm part of is intended to be "low hanging fruit". We don't need to deploy a new attribute, the format specifier is already present in code. All that was needed was a code point to say "go use this like existing RFC 1997 comms". The wide comms draft (and flex comms, where some of the ideas were pulled in from) was intended to address the messier case where the meaning of a community was already structured. To pick on one of the items in the list: http://www.onesc.net/communities/as209/ Coding these using regexes is a PITA, both as an implementor of the underlying policy and as a sender who has to remember what the magic value means. Ideally the operator should end up with something simple: Tell AS209, Do not announce to AS foo. Prepend N times to PST peers. Etc. Right now, these things are magic values. The last few rounds of wide comms were attempts to get encoding formats in place that accommodated some amount of grouping logic (peer-class CUSTOMER & North America, e.g.). We did manage to work out a way to do that encoding correctly. But it turned out that the real killer was transitive manipulation - you can't meddle with such a thing without breaking logic. So, back to a simpler drawing board. An update to the wide-comm draft should be out by end of this week. We'd welcome some constructive criticism on it. -- Jeff
On Feb 5, 2014, at 9:21 AM, Jeffrey Haas <jhaas@pfrc.org> wrote:
The wide comms draft (and flex comms, where some of the ideas were pulled in from) was intended to address the messier case where the meaning of a community was already structured. To pick on one of the items in the list: http://www.onesc.net/communities/as209/
Coding these using regexes is a PITA, both as an implementor of the underlying policy and as a sender who has to remember what the magic value means. Ideally the operator should end up with something simple: Tell AS209, Do not announce to AS foo. Prepend N times to PST peers. Etc. Right now, these things are magic values.
When possible (e.g.: here at AS2914) we have used things like this: 65500:nnn do not announce to peer where the NNN is the peer ASN. Using such literal numbering is easier for the humans involved. The ability to see what route is learned from specific ASN is also helpful, as things like AS_PATH are just a bit-string that can be arbitrarily set and sent by a peer. I will try to keep my eye open for the draft. (perhaps see you in Atlanta or London). - Jared
Food for thought: - ASNs can be reused at different locations by IXPs, barring perhaps certain business or administrative reasons. Ask Equinix. - For IXPs that already have 16-bit ASNs for route servers, this saves additional allocations from RIRs and mitigates concerns for the IXP getting potentially a 32-bit ASN, thus having trouble with BGP communities as described. Having a single ASN may raise other issues but it is an option. - I believe it would help if RIRs reserved 16-bit ASNs in addition to IPv4 micro allocations for IXPs, until a formal solution can be finally found about the 32-bit ASN BGP communities issue. Aris Lambrianidis
On Sat, 25 Jan 2014 14:56:16 +0100, Sebastian Spies said:
ASNs. First of all, we have no data or experience about 4-byte ASN adoption and are therefore unable to evaluate, if we should rush for the last available numbers.
2-byte ASN depletion - the other white meat....
participants (10)
-
Aris Lambrianidis -
Bryan Socha -
Jared Mauch -
Jeffrey Haas -
Job Snijders -
Martin Pels -
Nick Hilliard -
Rubens Kuhl -
Sebastian Spies -
Valdis.Kletnieks＠vt.edu