RE:"Tactical" /24 announcements
Ytti, We have introduced the scalable as-set into the XR route policy language. as-path-set does not scale well with 1000's of ASNs. Now, you don't need to expand AS-SET into prefix-set, just enter it directly. Example: as-set test 2914, 3356, end-set ! route-policy sample if as-path originates-from test then pass endif end-policy If this does not meet your needs and you need improvements, let me know. Kind Regards, Jakob. ------------------------------------------------------------- Date: Mon, 9 Aug 2021 19:10:23 +0300 From: Saku Ytti <saku@ytti.fi> We just recently learned of a IOS-XR prefix-set limit of 300001 when a particular customer AS-SET expanded to a higher number of prefixes. -- ++ytti
Hey Jakob, Is there documentation for this somewhere? Are you saying that the IOS-XR host will connect to some (configured?) server to expand the as-set, and at what time? Commit time? Once every N? On Sun, 15 Aug 2021 at 04:50, Jakob Heitz (jheitz) via NANOG <nanog@nanog.org> wrote:
Ytti,
We have introduced the scalable as-set into the XR route policy language. as-path-set does not scale well with 1000's of ASNs. Now, you don't need to expand AS-SET into prefix-set, just enter it directly. Example: as-set test 2914, 3356, end-set ! route-policy sample if as-path originates-from test then pass endif end-policy
If this does not meet your needs and you need improvements, let me know.
Kind Regards, Jakob.
------------------------------------------------------------- Date: Mon, 9 Aug 2021 19:10:23 +0300 From: Saku Ytti <saku@ytti.fi>
We just recently learned of a IOS-XR prefix-set limit of 300001 when a particular customer AS-SET expanded to a higher number of prefixes.
-- ++ytti
-- ++ytti
On 8/15/21 08:11, Saku Ytti wrote:
Hey Jakob,
Is there documentation for this somewhere? Are you saying that the IOS-XR host will connect to some (configured?) server to expand the as-set, and at what time? Commit time? Once every N?
Yes, same question for me. We've dumped all of our IOS XR eBGP-facing routers for Junos in the past 3 years, but it would be something good to know to give us options for the future. Mar.
Saku, The feature is in 7.2.1. The documentation has not made it to the command reference. There is no expansion to prefix-set. The command checks the origin-AS in the route. You should confirm the origin-AS with the prefix using RPKI and/or another route-policy statement. This way the final route-policy configuration will be much smaller. I'm happy to answer more questions or requests for improvement on or off list. Regards, Jakob. -----Original Message----- From: Saku Ytti <saku@ytti.fi> Sent: Saturday, August 14, 2021 11:11 PM To: Jakob Heitz (jheitz) <jheitz@cisco.com> Cc: nanog@nanog.org Subject: Re: "Tactical" /24 announcements Hey Jakob, Is there documentation for this somewhere? Are you saying that the IOS-XR host will connect to some (configured?) server to expand the as-set, and at what time? Commit time? Once every N? On Sun, 15 Aug 2021 at 04:50, Jakob Heitz (jheitz) via NANOG <nanog@nanog.org> wrote:
Ytti,
We have introduced the scalable as-set into the XR route policy language. as-path-set does not scale well with 1000's of ASNs. Now, you don't need to expand AS-SET into prefix-set, just enter it directly. Example: as-set test 2914, 3356, end-set ! route-policy sample if as-path originates-from test then pass endif end-policy
If this does not meet your needs and you need improvements, let me know.
Kind Regards, Jakob.
------------------------------------------------------------- Date: Mon, 9 Aug 2021 19:10:23 +0300 From: Saku Ytti <saku@ytti.fi>
We just recently learned of a IOS-XR prefix-set limit of 300001 when a particular customer AS-SET expanded to a higher number of prefixes.
-- ++ytti
-- ++ytti
hi jakob, i am confused between
There is no expansion to prefix-set.
and your earlier
We have introduced the scalable as-set into the XR route policy language. as-path-set does not scale well with 1000's of ASNs. Now, you don't need to expand AS-SET into prefix-set, just enter it directly.
expanding AS-SET into prefix filters is exactly what we do. ``` % peval -s RIPE AS-RG-SEA ({198.180.153.0/24, 198.180.151.0/24, 147.28.8.0/24, 147.28.9.0/24, 147.28.10.0/24, 147.28.11.0/24, 147.28.12.0/24, 147.28.13.0/24, 147.28.14.0/24, 147.28.15.0/24, 147.28.4.0/24, 147.28.5.0/24, 147.28.6.0/24, 147.28.7.0/24, 147.28.2.0/24, 147.28.3.0/24, 147.28.0.0/23, 45.132.188.0/24, 45.132.189.0/24, 45.132.190.0/24, 45.132.191.0/24}) ``` i do not see how to get around this. clue bat please randy
I share your confusion Randy. It seems like perhaps Jakob answered a slightly different question and his answer is roughly. a) Use this as-set feature to ensure valid set of ASNs from given peer b) Validate prefix using RPKI (I'm assuming with rejecting unknowns and invalids) c) Don't punch in prefix-lists anywhere Which in theory works, but in practice it does not, as RPKI validity cover is incomplete. Somewhat related, when JNPR implemented RTR the architecture was planned so that the RTR implementation itself isn't tightly coupled to RPKI validity. It was planned day1 that customers could have multiple RTR setups feeding prefixes and the NOS side could use these for other purposes too. So technically JNPR is mostly missing CLI work to allow you to feed prefix-lists dynamically over RTR, instead of punching them in vendor-specific way in config. I really hope JNPR does that work, I really like the appeal of doing things off-box and using the same protocol to talk to on-box. Also, give me gRPC/protobuf route policy API, so I can write my route-policy in a real programming language once for all my NOS. On Mon, 16 Aug 2021 at 20:32, Randy Bush <randy@psg.com> wrote:
hi jakob,
i am confused between
There is no expansion to prefix-set.
and your earlier
We have introduced the scalable as-set into the XR route policy language. as-path-set does not scale well with 1000's of ASNs. Now, you don't need to expand AS-SET into prefix-set, just enter it directly.
expanding AS-SET into prefix filters is exactly what we do.
``` % peval -s RIPE AS-RG-SEA ({198.180.153.0/24, 198.180.151.0/24, 147.28.8.0/24, 147.28.9.0/24, 147.28.10.0/24, 147.28.11.0/24, 147.28.12.0/24, 147.28.13.0/24, 147.28.14.0/24, 147.28.15.0/24, 147.28.4.0/24, 147.28.5.0/24, 147.28.6.0/24, 147.28.7.0/24, 147.28.2.0/24, 147.28.3.0/24, 147.28.0.0/23, 45.132.188.0/24, 45.132.189.0/24, 45.132.190.0/24, 45.132.191.0/24}) ```
i do not see how to get around this. clue bat please
randy
-- ++ytti
I quite like this approach as well - for those that would like to do more complicated policy logic off-box, the RTR architecture very much lends itself to that. JNPR already has accessible APIs (JET-based / RPC) you can leverage to push configuration into the ephemeral database or be called on certain events (e.g. prefix learn). This, however comes with the acceptance of quite a few other risks. RTR could be used to signal other prefix options which would potentially remove the risks of dealing with the ephemeral config construct for certain use-cases, e.g. complex peer prefix filtering. - Tim
On 17 Aug 2021, at 16:24, Saku Ytti <saku@ytti.fi> wrote:
I share your confusion Randy. It seems like perhaps Jakob answered a slightly different question and his answer is roughly.
a) Use this as-set feature to ensure valid set of ASNs from given peer b) Validate prefix using RPKI (I'm assuming with rejecting unknowns and invalids) c) Don't punch in prefix-lists anywhere
Which in theory works, but in practice it does not, as RPKI validity cover is incomplete.
Somewhat related, when JNPR implemented RTR the architecture was planned so that the RTR implementation itself isn't tightly coupled to RPKI validity. It was planned day1 that customers could have multiple RTR setups feeding prefixes and the NOS side could use these for other purposes too. So technically JNPR is mostly missing CLI work to allow you to feed prefix-lists dynamically over RTR, instead of punching them in vendor-specific way in config.
I really hope JNPR does that work, I really like the appeal of doing things off-box and using the same protocol to talk to on-box. Also, give me gRPC/protobuf route policy API, so I can write my route-policy in a real programming language once for all my NOS.
On Mon, 16 Aug 2021 at 20:32, Randy Bush <randy@psg.com> wrote:
hi jakob,
i am confused between
There is no expansion to prefix-set.
and your earlier
We have introduced the scalable as-set into the XR route policy language. as-path-set does not scale well with 1000's of ASNs. Now, you don't need to expand AS-SET into prefix-set, just enter it directly.
expanding AS-SET into prefix filters is exactly what we do.
``` % peval -s RIPE AS-RG-SEA ({198.180.153.0/24, 198.180.151.0/24, 147.28.8.0/24, 147.28.9.0/24, 147.28.10.0/24, 147.28.11.0/24, 147.28.12.0/24, 147.28.13.0/24, 147.28.14.0/24, 147.28.15.0/24, 147.28.4.0/24, 147.28.5.0/24, 147.28.6.0/24, 147.28.7.0/24, 147.28.2.0/24, 147.28.3.0/24, 147.28.0.0/23, 45.132.188.0/24, 45.132.189.0/24, 45.132.190.0/24, 45.132.191.0/24}) ```
i do not see how to get around this. clue bat please
randy
-- ++ytti
Somewhat related, when JNPR implemented RTR the architecture was planned so that the RTR implementation itself isn't tightly coupled to RPKI validity. It was planned day1 that customers could have multiple RTR setups feeding prefixes and the NOS side could use these for other purposes too.
back in the day, the developing RP-rtr spec assumed multiple sources, RPKI, IRR, ... there was also an open-ended operator defined Color 0 8 16 24 31 .-------------------------------------------. | Protocol | PDU | | | Version | Type | Color | | 0 | 4 | | +-------------------------------------------+ | | | Length=20 | | | +-------------------------------------------+ | | Prefix | Max | Data | | Flags | Length | Length | Source | | | 0..32 | 0..32 | RPKI/IRR | +-------------------------------------------+ | | | IPv4 prefix | | | +-------------------------------------------+ | | | Autonomous System Number | | | `-------------------------------------------' until, at ietf maastricht (2010), the unfortunate actions of a bully who wanted more and more caused us to walk away, give up, and fall back to the simpler basic we have today. 0 8 16 24 31 .-------------------------------------------. | Protocol | PDU | | | Version | Type | reserved = zero | | 0 | 4 | | +-------------------------------------------+ | | | Length=20 | | | +-------------------------------------------+ | | Prefix | Max | | | Flags | Length | Length | zero | | | 0..32 | 0..32 | | +-------------------------------------------+ | | | IPv4 Prefix | | | +-------------------------------------------+ | | | Autonomous System Number | | | `-------------------------------------------' randy
Hi Saku, On 08/17, Saku Ytti wrote:
I share your confusion Randy. It seems like perhaps Jakob answered a slightly different question and his answer is roughly.
a) Use this as-set feature to ensure valid set of ASNs from given peer b) Validate prefix using RPKI (I'm assuming with rejecting unknowns and invalids) c) Don't punch in prefix-lists anywhere
Which in theory works, but in practice it does not, as RPKI validity cover is incomplete.
This, and (more fundamentally) RPKI-breakage gets translated into a dataplane outage.
Somewhat related, when JNPR implemented RTR the architecture was planned so that the RTR implementation itself isn't tightly coupled to RPKI validity. It was planned day1 that customers could have multiple RTR setups feeding prefixes and the NOS side could use these for other purposes too. So technically JNPR is mostly missing CLI work to allow you to feed prefix-lists dynamically over RTR, instead of punching them in vendor-specific way in config.
We already do essentially this on arista EOS using a custom agent. It runs under the EOS process supervisor and calls home to a REST-API wrapper around bgpq3. It looks for specific config lines to work out which prefix lists to build, and then fetches them on a configurable interval. This has been in production for a year or two, without major incident. It's all open source, available at https://github.com/wolcomm/eos-prefix-list-agent. Pull-requests welcomed ;-) I'm in the middle of writing the equivalent tool for junos at the moment. Assuming that it works, we'll open source that too. HTH, Ben
for junos, i build the prefix list externally and push config. sad to say, the code is so old ('90s) that it's pearl and uses `peval`. i should fix but (copious spare time) == 0. originally i tried to also build and push for cisco ios classic, but it died in the push. breathe on the router and it reset bgp sessions. i gather from heas that things are better these years. i guess i really should have a go at doing it for arcos, but ...
It's all open source, available at https://github.com/wolcomm/eos-prefix-list-agent
very cool. randy
We do something similar - build the prefix lists externally (based on PeeringDB, IRR, RPKI data) and push them with config management on regular intervals. This sort of automated policy architecture is clearly becoming more common, and the drive (see: MANRS) is ever-increasing. I'd really like some sort of dynamic, standard method to achieve this off-box.
It's all open source, available at https://github.com/wolcomm/eos-prefix-list-agent
Very neat indeed! - Tim On Wed, Aug 18, 2021 at 2:45 AM Randy Bush <randy@psg.com> wrote:
for junos, i build the prefix list externally and push config. sad to say, the code is so old ('90s) that it's pearl and uses `peval`. i should fix but (copious spare time) == 0.
originally i tried to also build and push for cisco ios classic, but it died in the push. breathe on the router and it reset bgp sessions. i gather from heas that things are better these years.
i guess i really should have a go at doing it for arcos, but ...
It's all open source, available at https://github.com/wolcomm/eos-prefix-list-agent
very cool.
randy
Hi Randy, On 08/17, Randy Bush wrote:
for junos, i build the prefix list externally and push config. sad to say, the code is so old ('90s) that it's pearl and uses `peval`. i should fix but (copious spare time) == 0.
Spare time must be > 0 if you're willing to wait for peval to finish ;-)
originally i tried to also build and push for cisco ios classic, but it died in the push. breathe on the router and it reset bgp sessions. i gather from heas that things are better these years.
Better, but not good (or even tolerable). There is a reason I didn't provide an example of doing this kind of thing on IOS classic/XE.
i guess i really should have a go at doing it for arcos, but ...
The thing that EOS and JUNOS have in common that allows this to work is a mechanism to store config state outside the main "running config". In JUNOS that's the ephemeral DBs, in EOS they call it "URL based import" for as-path and prefix lists. If ArcOS doesn't already have something similar, I'd get it on the list. Cheers, Ben
I'm also in the externally managed space...very cool tool though. I love the idea of distributing some of this functionality. Are you also exporting and managing this data outside? On Tue, Aug 17, 2021 at 12:23 PM Ben Maddison via NANOG <nanog@nanog.org> wrote:
Hi Saku,
On 08/17, Saku Ytti wrote:
I share your confusion Randy. It seems like perhaps Jakob answered a slightly different question and his answer is roughly.
a) Use this as-set feature to ensure valid set of ASNs from given peer b) Validate prefix using RPKI (I'm assuming with rejecting unknowns and invalids) c) Don't punch in prefix-lists anywhere
Which in theory works, but in practice it does not, as RPKI validity cover is incomplete.
This, and (more fundamentally) RPKI-breakage gets translated into a dataplane outage.
Somewhat related, when JNPR implemented RTR the architecture was planned so that the RTR implementation itself isn't tightly coupled to RPKI validity. It was planned day1 that customers could have multiple RTR setups feeding prefixes and the NOS side could use these for other purposes too. So technically JNPR is mostly missing CLI work to allow you to feed prefix-lists dynamically over RTR, instead of punching them in vendor-specific way in config.
We already do essentially this on arista EOS using a custom agent.
It runs under the EOS process supervisor and calls home to a REST-API wrapper around bgpq3. It looks for specific config lines to work out which prefix lists to build, and then fetches them on a configurable interval.
This has been in production for a year or two, without major incident. It's all open source, available at https://github.com/wolcomm/eos-prefix-list-agent. Pull-requests <https://github.com/wolcomm/eos-prefix-list-agent.Pull-requests> welcomed ;-)
I'm in the middle of writing the equivalent tool for junos at the moment. Assuming that it works, we'll open source that too.
HTH,
Ben
Hi David, On 08/18, David Bass wrote:
I'm also in the externally managed space...very cool tool though. I love the idea of distributing some of this functionality.
Are you also exporting and managing this data outside?
[assuming that was directed to me...] I'm not sure what you mean by "exporting and managing this data outside". Would you elaborate? Cheers, Ben
Ben, Yes, sorry. Pulling/pushing the config data to a server, and then managing it there in addition to on the box. Like, if I want to run some reports to see how many PL are defined on each box, it’s easier to do that with the data centralized and managed. David On Thu, Aug 19, 2021 at 6:35 AM Ben Maddison <benm@workonline.africa> wrote:
Hi David,
On 08/18, David Bass wrote:
I'm also in the externally managed space...very cool tool though. I love the idea of distributing some of this functionality.
Are you also exporting and managing this data outside?
[assuming that was directed to me...]
I'm not sure what you mean by "exporting and managing this data outside". Would you elaborate?
Cheers,
Ben
Hi David, On 08/19, David Bass wrote:
Ben,
Yes, sorry.
Pulling/pushing the config data to a server, and then managing it there in addition to on the box. Like, if I want to run some reports to see how many PL are defined on each box, it’s easier to do that with the data centralized and managed.
Thanks for clarifying. A bit of additional context: We build and push whole device configs, doing a full replace on every change. The configs are built from centralized, version controlled data which describes devices connectivity, customer services, etc, etc, etc. On every change, we retrieve a diff from the device (e.g. show arch config diff ... on IOS). Having the *contents* of IRR derived prefix-lists in the configs has two major downsides: - it makes the config dependent on data that we don't own (i.e. a box gets a new config even though we didn't change any of our internal data), which in turn makes the diffs large and noisy; and - the size of the generated configs is huge, which slows down deployment and makes the whole process fragile. The tool I mentioned allows us to put a single line (syntactically equivalent to an empty prefix list) in the generated config. The agent "sees" that line, and fills in the details, keeping it up to date. The contents of the list never show up in a "show run", keeping noise levels down. There are ultimately three sources of policy data that contribute to the runtime operation of a device: - config pushed from our deployment tools - rpki-rtr data - prefix-list contents, from our mirrors on the various IRR DBs If I need to know what prefix lists are on a given box, and what they contain, I can simply look at those data sources directly. The key to reliability here is to share as much logic between operational tools as possible, so that you can be confident that the "ad-hoc troubleshooting tool" gives an answer that is consistent with the "config generation tool". Hope that kinda answers the question? Cheers, Ben
Hi Jakob, but the as-set only checks the origin AS in the announcement, it doesn't lookup the prefix <-> as relation from RADB/RIPE/Whatever, as i understand it correctly! Or is there some lookup mechanism as Ytti/Mark mentioned? regards, tim On Sun, Aug 15, 2021 at 01:46:40AM +0000, Jakob Heitz (jheitz) via NANOG wrote:
Ytti,
We have introduced the scalable as-set into the XR route policy language. as-path-set does not scale well with 1000's of ASNs. Now, you don't need to expand AS-SET into prefix-set, just enter it directly. Example: as-set test 2914, 3356, end-set ! route-policy sample if as-path originates-from test then pass endif end-policy
If this does not meet your needs and you need improvements, let me know.
Kind Regards, Jakob.
------------------------------------------------------------- Date: Mon, 9 Aug 2021 19:10:23 +0300 From: Saku Ytti <saku@ytti.fi>
We just recently learned of a IOS-XR prefix-set limit of 300001 when a particular customer AS-SET expanded to a higher number of prefixes.
-- ++ytti
-- Tim Weippert http://weiti.org - weiti@weiti.org GPG Fingerprint - E704 7303 6FF0 8393 ADB1 398E 67F2 94AE 5995 7DD8
participants (8)
-
Ben Maddison -
David Bass -
Jakob Heitz (jheitz) -
Mark Tinka -
Randy Bush -
Saku Ytti -
Tim Raphael -
Tim Weippert