[eng/rtg] changing loopbacks
so i have junipers, ciscos, and a few <gasp> zebras in an ospf and ibgp mesh. they're peering via loopbacks, of course. unfortunately, i need to recover the space from which the loopbacks are taken. of course, i would like to do so with minimal disruption. i am thinking of something like the following: o add second loopbacks to all routers with new address in new block o set up ibgp peerings to new addresses from existing peerings o change the source of routing updates to new addresses o remove old peerings o remove old loopbacks what [else] am i missing? randy
Are you using any logging-source or source interface for your authentication on the boxes? I've found that most versions of IOS require a reboot to accept a change of these. Similarly, OSPF under IOS will need a bounce/reboot to change it's router-ID if you are using the loopback for that. Personally, the cleanest way I've been able to accomplish changing this in regard to OSPF, logging and authentication on Cisco's is to suck down the running config, make the changes in your editor of choice, push it back up to startup-config and schedule a reboot. iBGP is much easier to make the changes as you outline. As always, your mileage may vary. David On Sep 29, 2005, at 11:05 AM, Randy Bush wrote:
so i have junipers, ciscos, and a few <gasp> zebras in an ospf and ibgp mesh. they're peering via loopbacks, of course. unfortunately, i need to recover the space from which the loopbacks are taken. of course, i would like to do so with minimal disruption. i am thinking of something like the following:
o add second loopbacks to all routers with new address in new block o set up ibgp peerings to new addresses from existing peerings o change the source of routing updates to new addresses o remove old peerings o remove old loopbacks
what [else] am i missing?
randy
Personally, the cleanest way I've been able to accomplish changing this in regard to OSPF, logging and authentication on Cisco's is to suck down the running config, make the changes in your editor of choice, push it back up to startup-config and schedule a reboot. iBGP is much easier to make the changes as you outline.
this is my fear. which is why i asked. pushing out new configs (the canonic config is on disk, not the router [0]) and setting a reload of a bunch of routers at time t0 does not give me warm fuzzies about what the world will be like at time tn (n > 0). but i may have to take that path. i am hoping folk will give me a magic pill. after all, any group with such a deep understanding of how to deal with the world's social ills must know a bit of router magic <smirk>. randy --- [0] - i once worked at a large company who was proud of saying that "the network is the database of record." i left there rather quickly.
this is my fear. which is why i asked. pushing out new configs (the canonic config is on disk, not the router [0]) and setting a reload of a bunch of routers at time t0 does not give me warm fuzzies about what the world will be like at time tn (n > 0).
but i may have to take that path. i am hoping folk will give me a magic pill. after all, any group with such a deep understanding of how to deal with the world's social ills must know a bit of router magic <smirk>.
I think with OSPF this will be very difficult to do without rebooting (or as long an outage as rebooting). We migrated from OSPF to IS-IS and changed some loopbacks a while ago, the IS-IS change was totally transparent - no issue, but on the change of loopback caused a lot of BGP churn. It was easier to change it and reboot and do it over a period of time in small network triangles. I always thought that the billing system was the database of record ;-) Neil.
So, on vendor C boxes you might be able to get away from having to do a full reboot to change your OSPF ID by doing a "clear ip ospf process". If you don't do this, even though you change the loopback address, your router will still keep the old address as the OSPF router ID[1]. You won't actually end up with a route to the old loopback, but it will still be in the OSPF database. While this is less than optimal, it will still work (note, I don't recommend running your network like this!). It is somewhat disconcerting if you don't know that changing loopback address doesn't automatically change OSPF ID[2] and look in your OSPF database and see addresses that you shouldn't / you retired, etc, especially because most people only page through their OSPF database when they suspect something is odd... Warren Kumari [1] As with most things, I am sure that the exact behavior depends upon hardware and software version, phase of moon, flavor of doughnut, etc. [2] Sure it seem obvious when you thin about it, but most people don't seem to think. On Sep 29, 2005, at 12:20 PM, Neil J. McRae wrote:
this is my fear. which is why i asked. pushing out new configs (the canonic config is on disk, not the router [0]) and setting a reload of a bunch of routers at time t0 does not give me warm fuzzies about what the world will be like at time tn (n > 0).
but i may have to take that path. i am hoping folk will give me a magic pill. after all, any group with such a deep understanding of how to deal with the world's social ills must know a bit of router magic <smirk>.
I think with OSPF this will be very difficult to do without rebooting (or as long an outage as rebooting). We migrated from OSPF to IS-IS and changed some loopbacks a while ago, the IS-IS change was totally transparent - no issue, but on the change of loopback caused a lot of BGP churn. It was easier to change it and reboot and do it over a period of time in small network triangles.
I always thought that the billing system was the database of record ;-)
Neil.
Randy Bush wrote:
Personally, the cleanest way I've been able to accomplish changing this in regard to OSPF, logging and authentication on Cisco's is to suck down the running config, make the changes in your editor of choice, push it back up to startup-config and schedule a reboot. iBGP is much easier to make the changes as you outline.
this is my fear. which is why i asked. pushing out new configs (the canonic config is on disk, not the router [0]) and setting a reload of a bunch of routers at time t0 does not give me warm fuzzies about what the world will be like at time tn (n > 0).
but i may have to take that path. i am hoping folk will give me a magic pill. after all, any group with such a deep understanding of how to deal with the world's social ills must know a bit of router magic <smirk>.
You may need to change your BGP router-id to match if you set it explicitly, which *may* alter path selection (a long way down the tree I admit). Another nasty is if you run TE and use the old Loopback as your TE-ID, even with IS-IS. Plus of course, your zone/hosts file for managing/polling these nodes in the first place :-) -- Ian Dickinson Development Engineer PIPEX ian.dickinson@pipex.net http://www.pipex.net This e-mail is subject to: http://www.pipex.net/disclaimer.html
On Fri, Sep 30, 2005 at 12:14:46AM +0100, Ian Dickinson wrote:
Randy Bush wrote:
Personally, the cleanest way I've been able to accomplish changing this in regard to OSPF, logging and authentication on Cisco's is to suck down the running config, make the changes in your editor of choice, push it back up to startup-config and schedule a reboot. iBGP is much easier to make the changes as you outline.
this is my fear. which is why i asked. pushing out new configs (the canonic config is on disk, not the router [0]) and setting a reload of a bunch of routers at time t0 does not give me warm fuzzies about what the world will be like at time tn (n > 0).
but i may have to take that path. i am hoping folk will give me a magic pill. after all, any group with such a deep understanding of how to deal with the world's social ills must know a bit of router magic <smirk>.
You may need to change your BGP router-id to match if you set it explicitly, which *may* alter path selection (a long way down the tree I admit).
Another nasty is if you run TE and use the old Loopback as your TE-ID, even with IS-IS.
Plus of course, your zone/hosts file for managing/polling these nodes in the first place :-) -- Ian Dickinson Development Engineer PIPEX ian.dickinson@pipex.net http://www.pipex.net
This e-mail is subject to: http://www.pipex.net/disclaimer.html
Of course, you can always pre-deploy ibgp sessions and TE configs to the new address (which will remain down until the new address exists), go for your OOB access to the box, change the loopback IP address (and associated references), "clear ip bgp *" and accept the 2 minute interruption. Changing loopback IPs is difficult to do gracefully but can still be done relatively quickly. IS-IS and OSPF should be able to pick up the change as soon as the IP address is entered into the config. --- Wayne Bouchard web@typo.org Network Dude http://www.typo.org/~web/
On Thu, 29 Sep 2005, Wayne E. Bouchard wrote: <chopped ian's good infos>
Of course, you can always pre-deploy ibgp sessions and TE configs to the new address (which will remain down until the new address exists), go for your OOB access to the box, change the loopback IP address (and associated references), "clear ip bgp *" and accept the 2 minute interruption. Changing loopback IPs is difficult to do gracefully but can still be done relatively quickly.
The clearing MIGHT have a larger effect, especially depending on the number of peers you have and this device's placement in your network. I'd think that a graceful bgp switchover would be simple enough (and wouldn't require clear ip bgp *) The devil, IMHO, is in the IGP switches :( (not withstanding ian's noted bgp router-id change)
IS-IS and OSPF should be able to pick up the change as soon as the IP address is entered into the config.
ospf doesn't, for router-id on cisco's atleast, as Warren pointed out :( however! switching from ospf to 'another igp' (ISIS would work well) would avoid that, slide off ospf and onto ISIS, kill ospf when all next-hops switch, which should be 'as soon as isis converges'.
On Fri, 30 Sep 2005, Christopher L. Morrow wrote:
ospf doesn't, for router-id on cisco's atleast, as Warren pointed out :( however! switching from ospf to 'another igp' (ISIS would work well) would avoid that, slide off ospf and onto ISIS, kill ospf when all next-hops switch, which should be 'as soon as isis converges'.
It is a good reason to assign seperate router IDs, rather than rely on the OSPF implementation to decide which IP is the one it will use. Of course not all vendors support this. The major router vendors do, though. -Sean
On Sep 29, 2005, at 12:05 PM, Randy Bush wrote:
so i have junipers, ciscos, and a few <gasp> zebras in an ospf and ibgp mesh. they're peering via loopbacks, of course. unfortunately, i need to recover the space from which the loopbacks are taken. of course, i would like to do so with minimal disruption. i am thinking of something like the following:
o add second loopbacks to all routers with new address in new block o set up ibgp peerings to new addresses from existing peerings o change the source of routing updates to new addresses o remove old peerings o remove old loopbacks
what [else] am i missing?
I vaguely remember going through this exercise in a previous life... As people have already mentioned, some things that are generally tied to loopback address (router-id for one) require a reboot on some vendors equipment. Because of that, we followed a procedure something like this: 1) for all routers: Add IBGP sessions to new loopback IPs. Don't add new loopback IP to the router, or change IBGP source address. At this point, all added sessions will be down. 2) for each router: Change "startup config" to the new loopback IP and reload the router on the new config. Router still has IBGP sessions for both old and new loopback IPs of each IBGP peer. 3) for all routers: Remove all IBGP sessions to old loopback IPs. At this point all sessions to old IPs will be down and all sessions to new IPs will be up. Each IBGP sessions ends up transitioning through these stages: old <-> old reboot one of the routers on new loopback new <-> old reboot other router on new loopback new <-> new --Stacy
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Randy Bush wrote:
so i have junipers, ciscos, and a few <gasp> zebras in an ospf and ibgp mesh. they're peering via loopbacks, of course. unfortunately, i need to recover the space from which the loopbacks are taken. of course, i would like to do so with minimal disruption. i am thinking of something like the following:
o add second loopbacks to all routers with new address in new block o set up ibgp peerings to new addresses from existing peerings o change the source of routing updates to new addresses o remove old peerings o remove old loopbacks
what [else] am i missing?
In addition to what others have said, I'd ask: - - Any ACL's anywhere that filter based on the old loopbacks? - - Any VTY access controls on the router based on the old loopbacks? - - Any external systems like authentication servers, management systems, etc, etc that need the old loopbacks and can't dynamically adapt? - - Any internal routing policies that reference the old loopbacks? - - Any DNS entries that need to be migrated (CNAME->A references)? - -- ========= bep -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (MingW32) iD8DBQFDPE3ME1XcgMgrtyYRApizAKDUz+80NvW2tdMfyivgLGA8+uJ1dQCdF54a VzUeshCuqbV0dlI3D3Poqxw= =jOI/ -----END PGP SIGNATURE-----
### On Thu, 29 Sep 2005 13:25:48 -0700, Bruce Pinsky <bep@whack.org> ### casually decided to expound upon Randy Bush <randy@psg.com> the ### following thoughts about "Re: [eng/rtg] changing loopbacks": BP> > what [else] am i missing? BP> BP> In addition to what others have said, I'd ask: BP> BP> - - Any ACL's anywhere that filter based on the old loopbacks? BP> - - Any VTY access controls on the router based on the old loopbacks? BP> - - Any external systems like authentication servers, management systems, BP> etc, etc that need the old loopbacks and can't dynamically adapt? BP> - - Any internal routing policies that reference the old loopbacks? BP> - - Any DNS entries that need to be migrated (CNAME->A references)? Also want to keep in mind things like tunnel endpoints (IPv6, VOIP, multicast, VPN, etc). Barring any sort of advanced config management package, grep and diff become your friends (some would say despite). As a first pass, I'd snarf down all configs and do a grep for the loopbacks to indtify which ones need attention. Then make your changes in each config and do diffs to verify. Then I'd stage out deployment with stub and leaf nodes going last to minimise churn in OSPF. If you've got iBGP going and are using route-reflectors then do the top-most hierarchy first before the lower clusters. -- /*===================[ Jake Khuon <khuon@NEEBU.Net> ]======================+ | Packet Plumber, Network Engineers /| / [~ [~ |) | | --------------- | | for Effective Bandwidth Utilisation / |/ [_ [_ |) |_| N E T W O R K S | +=========================================================================*/
participants (11)
-
Bruce Pinsky -
Christopher L. Morrow -
David Sinn -
Ian Dickinson -
Jake Khuon -
Neil J. McRae -
Randy Bush -
Sean Figgins -
Stacy W. Smith -
Warren Kumari -
Wayne E. Bouchard