Large prefix lists/sets on IOS-XR
Hi, What is the best/most efficient/most convenient way to push large prefix lists or sets to an XR router for BGP prefix filtering? Pushing thousands of lines through the CLI seems foolish, I tried using the load command but it seems horribly slow. What am I missing? :) Cheers! Sander --- for every complex problem, there’s a solution that is simple, neat, and wrong
Netconf is really nice for atomic changes to network devices, though it would still take some time for the device to process such a large change. On Thu, Dec 8, 2022 at 6:05 PM Sander Steffann <sander@steffann.nl> wrote:
Hi,
What is the best/most efficient/most convenient way to push large prefix lists or sets to an XR router for BGP prefix filtering? Pushing thousands of lines through the CLI seems foolish, I tried using the load command but it seems horribly slow. What am I missing? :)
Cheers! Sander
--- for every complex problem, there’s a solution that is simple, neat, and wrong
Two options: - gRPC - Netconf You can use tools like paramiko,netmiko or napalm that are widely used to programmatically configure and manage your XR router. On Fri, Dec 9, 2022 at 2:24 AM Joshua Miller <contemno@gmail.com> wrote:
Netconf is really nice for atomic changes to network devices, though it would still take some time for the device to process such a large change.
On Thu, Dec 8, 2022 at 6:05 PM Sander Steffann <sander@steffann.nl> wrote:
Hi,
What is the best/most efficient/most convenient way to push large prefix lists or sets to an XR router for BGP prefix filtering? Pushing thousands of lines through the CLI seems foolish, I tried using the load command but it seems horribly slow. What am I missing? :)
Cheers! Sander
--- for every complex problem, there’s a solution that is simple, neat, and wrong
-- *Cheers,* *Andrian Visnevschi*
Can Andrian and Joshua explain what they specifically mean, and how they expect it to perform over what Steffann is already doing (e.g. load https://nms/cfg/router.txt)? How much faster will it be, and why? Can Steffan explain how large a file they are copying, over what protocol, how long does it take, and how long does the commit take. We used to have configurations in excess of a million lines before 'or-longer' halved them, and we've seen much longer times than 30min to get a new config pushed+commtited. We use FTP and while the FTP does take its sweet time, the commit itself is very long as well. I refrain from expressing my disillusionment with the utility of doing IRR based filtering. On Fri, 9 Dec 2022 at 15:38, Andrian Visnevschi via NANOG <nanog@nanog.org> wrote:
Two options: - gRPC - Netconf
You can use tools like paramiko,netmiko or napalm that are widely used to programmatically configure and manage your XR router.
On Fri, Dec 9, 2022 at 2:24 AM Joshua Miller <contemno@gmail.com> wrote:
Netconf is really nice for atomic changes to network devices, though it would still take some time for the device to process such a large change.
On Thu, Dec 8, 2022 at 6:05 PM Sander Steffann <sander@steffann.nl> wrote:
Hi,
What is the best/most efficient/most convenient way to push large prefix lists or sets to an XR router for BGP prefix filtering? Pushing thousands of lines through the CLI seems foolish, I tried using the load command but it seems horribly slow. What am I missing? :)
Cheers! Sander
--- for every complex problem, there’s a solution that is simple, neat, and wrong
--
Cheers,
Andrian Visnevschi
-- ++ytti
Hi Saku. I don't know that Netconf or gRPC are any faster than loading cli. Those protocols facilitate automation so that the time it takes to load any one device is not a significant factor, especially when you can roll out changes to devices in parallel. Also, it's easier to build the changes into a structured format than assemble the right syntax to interact with the CLI. On Fri, Dec 9, 2022, 09:38 Saku Ytti <saku@ytti.fi> wrote:
Can Andrian and Joshua explain what they specifically mean, and how they expect it to perform over what Steffann is already doing (e.g. load https://nms/cfg/router.txt)? How much faster will it be, and why?
Can Steffan explain how large a file they are copying, over what protocol, how long does it take, and how long does the commit take.
We used to have configurations in excess of a million lines before 'or-longer' halved them, and we've seen much longer times than 30min to get a new config pushed+commtited. We use FTP and while the FTP does take its sweet time, the commit itself is very long as well.
I refrain from expressing my disillusionment with the utility of doing IRR based filtering.
On Fri, 9 Dec 2022 at 15:38, Andrian Visnevschi via NANOG <nanog@nanog.org> wrote:
Two options: - gRPC - Netconf
You can use tools like paramiko,netmiko or napalm that are widely used
to programmatically configure and manage your XR router.
On Fri, Dec 9, 2022 at 2:24 AM Joshua Miller <contemno@gmail.com> wrote:
Netconf is really nice for atomic changes to network devices, though it
On Thu, Dec 8, 2022 at 6:05 PM Sander Steffann <sander@steffann.nl>
wrote:
Hi,
What is the best/most efficient/most convenient way to push large
would still take some time for the device to process such a large change. prefix lists or sets to an XR router for BGP prefix filtering? Pushing thousands of lines through the CLI seems foolish, I tried using the load command but it seems horribly slow. What am I missing? :)
Cheers! Sander
--- for every complex problem, there’s a solution that is simple, neat,
and wrong
--
Cheers,
Andrian Visnevschi
-- ++ytti
On Fri, 9 Dec 2022 at 17:07, Joshua Miller <contemno@gmail.com> wrote:
I don't know that Netconf or gRPC are any faster than loading cli. Those protocols facilitate automation so that the time it takes to load any one device is not a significant factor, especially when you can roll out changes to devices in parallel. Also, it's easier to build the changes into a structured format than assemble the right syntax to interact with the CLI.
As a programmer I don't really find output format to be significant cost. If I have source of data how I emit them out doesn't matter much. I accept preferences that people have, but don't think it to be important part of solution. Adrian mentioned paramiko, and if we imagine paramiko logging into IOS-XR, and doing 'load http://...' + 'commit'. We've automated the task. Depending on your platform netconf/yang/grpc can be asset or liability, I put IOS-XR strongly in the liability part, because they don't have proper infrastructure that is datafirst, they don't have proper module for even handling configurations, but configurations are owned by individual component teams (like tunnel teams owns GRE config and so forth). Contrasting with Juniper, which is datafirst, and even CLI is 2nd class citizen taking formal data from XML RPC. In IOS-XR you will find all kind of gaps, where you can't rely on netconf/yang, which you will then spend cycles to deal with vendor. Compared to people who use the first class citizen approach, CLI format, who are already done. I did not read Steffan as though he'd be punching in anything manually, he wants to make the process itself faster, without any delays introduced by humans. And I have personally nothing to offer him, except put your server closer to the router, so you can deal with the limited TCP window sizes that hurt transfer speed. -- ++ytti
Pushing thousands of lines via CLI/expect automation is def not a great idea, no. Putting everything into a file, copying that to the device, and loading from there is generally best regardless. The slowness you refer to is almost certainly just because of how XR handles config application. If I'm following correctly, that seems to be the crux of your question. On Thu, Dec 8, 2022 at 6:04 PM Sander Steffann <sander@steffann.nl> wrote:
Hi,
What is the best/most efficient/most convenient way to push large prefix lists or sets to an XR router for BGP prefix filtering? Pushing thousands of lines through the CLI seems foolish, I tried using the load command but it seems horribly slow. What am I missing? :)
Cheers! Sander
--- for every complex problem, there’s a solution that is simple, neat, and wrong
On Fri, 9 Dec 2022 at 17:30, Tom Beecher <beecher@beecher.cc> wrote:
Pushing thousands of lines via CLI/expect automation is def not a great idea, no. Putting everything into a file, copying that to the device, and loading from there is generally best regardless. The slowness you refer to is almost certainly just because of how XR handles config application. If I'm following correctly, that seems to be the crux of your question.
If you read carefully, that is what Steffann is doing. He is doing 'load location:file' + 'commit'. He is not punching anything by hand. So the answer we are looking for is how to make that go faster. In Junos answer would be 'ephemeral config', but in IOS-XR as far as I know, the only thing you can do is improve the 'load' part by moving the server closer, other than that, you get what you get. -- ++ytti
I agree, I don't think you can get around the XR config bottleneck. But that's not really the end of the story. Let's think about why the loading time matters to Sander (Sander, please chime in here): 1. They have to address an immediate issue for a customer (internal or external) and the customer is impatient. 2. They're sitting there waiting for the load to complete before they can move on to their next task. Possibly having to make the same change to multiple devices. I can't really offer anything for the first case. But in the second case, automation can help address that by making changes more unattended and parallelizable. In terms of structured vs unstructured data, sure, assembling text is not a huge lift. Though, when you're talking about layering on complex use cases, then it gets more complicated. Especially if you want to compute the inverse configuration to remove service instances that are no longer needed. In terms of vendor support, I'd hope that if you're paying that kind of money, you're getting a product that meets your requirements. Something that should be assessed during vendor selection and procurement. That's just my preference; do whatever works best for your use cases. On Fri, Dec 9, 2022 at 10:35 AM Saku Ytti <saku@ytti.fi> wrote:
On Fri, 9 Dec 2022 at 17:30, Tom Beecher <beecher@beecher.cc> wrote:
Pushing thousands of lines via CLI/expect automation is def not a great idea, no. Putting everything into a file, copying that to the device, and loading from there is generally best regardless. The slowness you refer to is almost certainly just because of how XR handles config application. If I'm following correctly, that seems to be the crux of your question.
If you read carefully, that is what Steffann is doing. He is doing 'load location:file' + 'commit'. He is not punching anything by hand.
So the answer we are looking for is how to make that go faster.
In Junos answer would be 'ephemeral config', but in IOS-XR as far as I know, the only thing you can do is improve the 'load' part by moving the server closer, other than that, you get what you get. -- ++ytti
On Fri, 9 Dec 2022 at 17:58, Joshua Miller <contemno@gmail.com> wrote:
In terms of structured vs unstructured data, sure, assembling text is not a huge lift. Though, when you're talking about layering on complex use cases, then it gets more complicated. Especially if you want to compute the inverse configuration to remove service instances that are no longer needed. In terms of vendor support, I'd hope that if you're paying that kind of money, you're getting a product that meets your requirements. Something that should be assessed during vendor selection and procurement. That's just my preference; do whatever works best for your use cases.
Deltas are _super_ hard. But you never need to do them. Always produce a complete config, and let the vendor deal with the problem. We've done this with Junos, IOSXR, EOS (compass, not arista, RIP), SROS (MDCLI) for years If you remove the need for deltas the whole problem becomes extremely trivial. Fill in all the templates with data, push it. -- ++ytti
On Friday, 9 December, 2022 16:04, "Saku Ytti" <saku@ytti.fi> said:
If you remove the need for deltas the whole problem becomes extremely trivial. Fill in all the templates with data, push it.
Or at least, you've moved the problem from "generate config" to "have complete and correct data". Which statement should probably come with some kind of trigger-warning... Cheers, Tim.
On Fri, 9 Dec 2022 at 20:19, tim@pelican.org <tim@pelican.org> wrote: Hey Tim,
Or at least, you've moved the problem from "generate config" to "have complete and correct data". Which statement should probably come with some kind of trigger-warning...
I think it's a lot easier than you think. I understand that all older networks and practical access networks have this problem, the data is in the network, it's of course not the right way to do it, but it's the way they are. But there is no reason to get discouraged. First you gotta ignore waterfall model, you can never order something ready and have utility out of it, because no data. What you can do, day1 a) copy configs as-is, as templates b) only edit the template c) push templates to network boom, now you are FAR, and that took an hour or day depending on the person. Maybe you feel like you've not accomplished much, but you have. Now you can start modelling data out of the template into the database, and keep shrinking the 'blobs'. You can do this at whatever pace is convenient, and you can trivially measure which one to do next, which one will reduce total blob bytes most. You will see constant, measurable progress. And you always know the network state is always what is in your files, as you are now always replacing the entire config with the generated config. -- ++ytti
On Saturday, 10 December, 2022 06:47, "Saku Ytti" <saku@ytti.fi> said:
What you can do, day1
a) copy configs as-is, as templates b) only edit the template c) push templates to network
That's a take on it I really hadn't considered. I'm very aware that moving from a decade or two of legacy manual config to full data model/automation in a big bang is never going to work, but I'd been looking at what individual elements could be pulled out and automated with judicious use of "replace". Never considered making the *entire* legacy config a starting point for the template, and then effectively working on automating replacing parts of that config file with data-driven versions of the same. Food for thought, thanks for that. Cheers, Tim.
On 12/12/22 12:26, tim@pelican.org wrote:
That's a take on it I really hadn't considered. I'm very aware that moving from a decade or two of legacy manual config to full data model/automation in a big bang is never going to work, but I'd been looking at what individual elements could be pulled out and automated with judicious use of "replace". Never considered making the *entire* legacy config a starting point for the template, and then effectively working on automating replacing parts of that config file with data-driven versions of the same.
Food for thought, thanks for that.
Agree. This is why when vendors come to market their new shiny NETCONF/YANG-based SDN thingie that will make my life easier, I sit back and smile :-). Real life is very different from what we think "automation" should be. And as I mentioned on some list a year or two ago, "automation" means different things to different people. So rather than trying to box yourself into the word "automation", just find ways to make your life easier. Whether someone wants to call it automation or not, is irrelevant... well, until we can walk into a shop and buy Automation off the shelf. Mark.
That's a take on it I really hadn't considered. I'm very aware that moving from a decade or two of legacy manual config to full data model/automation in a big bang is never going to work, but I'd been looking at what individual elements could be pulled out and automated with judicious use of "replace". Never considered making the *entire* legacy config a starting point for the template, and then effectively working on automating replacing parts of that config file with data-driven versions of the same.
It really makes life a LOT easier to do it that way. It can create some quirky downsides where different people create the sections to replace in different ways, and operational confusion (Is this section under automation now or not?) , but it's still light years ahead of trying to build a monolithic template from scratch. Working section by section also makes it much easier to handle deltas and exceptions for different sites and locations. On Mon, Dec 12, 2022 at 5:27 AM tim@pelican.org <tim@pelican.org> wrote:
On Saturday, 10 December, 2022 06:47, "Saku Ytti" <saku@ytti.fi> said:
What you can do, day1
a) copy configs as-is, as templates b) only edit the template c) push templates to network
That's a take on it I really hadn't considered. I'm very aware that moving from a decade or two of legacy manual config to full data model/automation in a big bang is never going to work, but I'd been looking at what individual elements could be pulled out and automated with judicious use of "replace". Never considered making the *entire* legacy config a starting point for the template, and then effectively working on automating replacing parts of that config file with data-driven versions of the same.
Food for thought, thanks for that.
Cheers, Tim.
Hi Ytti,
Pushing thousands of lines via CLI/expect automation is def not a great idea, no. Putting everything into a file, copying that to the device, and loading from there is generally best regardless. The slowness you refer to is almost certainly just because of how XR handles config application. If I'm following correctly, that seems to be the crux of your question.
If you read carefully, that is what Steffann is doing. He is doing 'load location:file' + 'commit'. He is not punching anything by hand.
So the answer we are looking for is how to make that go faster.
In Junos answer would be 'ephemeral config', but in IOS-XR as far as I know, the only thing you can do is improve the 'load' part by moving the server closer, other than that, you get what you get.
Perfect answer :) Not what I was hoping to hear, but if that’s what it is, then that’s what it is. Cheers! Sander
Fri, Dec 09, 2022 at 05:33:09PM +0200, Saku Ytti:
If you read carefully, that is what Steffann is doing. He is doing 'load location:file' + 'commit'. He is not punching anything by hand.
So the answer we are looking for is how to make that go faster.
In Junos answer would be 'ephemeral config', but in IOS-XR as far as I know, the only thing you can do is improve the 'load' part by moving the server closer, other than that, you get what you get.
set the tcp default mss higher use rcpd remove unnecessary whitespace; ios parser is slow as are the bits behind it remove comments
Adding to Heasley’s comments here. You can transfer via SSH based methods and increase speeds using this command as of IOS XR 7.1.x+ ssh server tcp-window-scale On 12/9/22, 11:46, "NANOG" <nanog-bounces+wjnicholson=gmail.com@nanog.org> wrote: Fri, Dec 09, 2022 at 05:33:09PM +0200, Saku Ytti:
If you read carefully, that is what Steffann is doing. He is doing 'load location:file' + 'commit'. He is not punching anything by hand.
So the answer we are looking for is how to make that go faster.
In Junos answer would be 'ephemeral config', but in IOS-XR as far as I know, the only thing you can do is improve the 'load' part by moving the server closer, other than that, you get what you get.
set the tcp default mss higher use rcpd remove unnecessary whitespace; ios parser is slow as are the bits behind it remove comments
participants (9)
-
Andrian Visnevschi -
heasley -
Joseph Nicholson -
Joshua Miller -
Mark Tinka -
Saku Ytti -
Sander Steffann -
tim＠pelican.org -
Tom Beecher