Best way to have redundancy announcing on separate routers
Hi Folks, I'm trying to achieve total redundancy on a multihomed environment: ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 Where X is my Network. In the example below, he announces separate blocks to each ISP. https://www.networkstraining.com/cisco-bgp-configuration-tutorial/ I would like to do a failover model, where if one ISP goes down the other would take over. Please share your thoughts on this. Best regards,
Sorry, this is not a general help list for basic networking skills. There are many options for appropriate training available, but this is not an appropriate channel for this query. From: NANOG <nanog-bounces+tony=wicks.co.nz@nanog.org> On Behalf Of Jean Franco Sent: Tuesday, 24 December 2024 12:33 pm To: North American Network Operators' Group <nanog@nanog.org> Subject: Best way to have redundancy announcing on separate routers Hi Folks, I'm trying to achieve total redundancy on a multihomed environment: ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 Where X is my Network. In the example below, he announces separate blocks to each ISP. https://www.networkstraining.com/cisco-bgp-configuration-tutorial/ I would like to do a failover model, where if one ISP goes down the other would take over. Please share your thoughts on this. Best regards,
Tony, "The NANOG Mailing List is a community-moderated forum, open to all. Established in 1994 to provide the open exchange of technical information, it provides the opportunity for lively discussions of specific implementation challenges that require cooperation among network service providers." - NANOG Usage Guidelines<https://nanog.org/legal/usage-guidelines/> I disagree with your remark that "this is not an appropriate channel for this query". I would believe that Jean's query falls under the "open exchange of technical information" category. Regards, Christopher Hawker ________________________________ From: NANOG <nanog-bounces+chris=thesysadmin.au@nanog.org> on behalf of Tony Wicks <tony@wicks.co.nz> Sent: Tuesday, December 24, 2024 11:33 AM To: 'Jean Franco' <jfranco@maila.inf.br> Cc: 'North American Network Operators' Group' <nanog@nanog.org> Subject: RE: Best way to have redundancy announcing on separate routers Sorry, this is not a general help list for basic networking skills. There are many options for appropriate training available, but this is not an appropriate channel for this query. From: NANOG <nanog-bounces+tony=wicks.co.nz@nanog.org> On Behalf Of Jean Franco Sent: Tuesday, 24 December 2024 12:33 pm To: North American Network Operators' Group <nanog@nanog.org> Subject: Best way to have redundancy announcing on separate routers Hi Folks, I'm trying to achieve total redundancy on a multihomed environment: ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 Where X is my Network. In the example below, he announces separate blocks to each ISP. https://www.networkstraining.com/cisco-bgp-configuration-tutorial/ I would like to do a failover model, where if one ISP goes down the other would take over. Please share your thoughts on this. Best regards,
Sorry, this is not a general help list for basic networking skills.
wow! that was helpful, polite, and considerate. good on ya the list ihas been abput helping others, including the less senior, for some decades. the nanog list has been the one of the most significant public help to networkers since dirt was invented. randy
Hi Jean, You can establish an iBGP session between the two routers that exchange either default & own routes, or they can send their own routes with fulls and use local pref to preference the directly-connected transit session before routes learnt from the iBGP session, depending on how you want engineer your traffic. If you are not receiving full tables and only getting a default from each transit provider you would need to weight the defaults so it uses the preferred default. If you're planning to add (for example) peering or PNIs to either router in the future, you will want full tables for greater traffic control. Regards, Christopher Hawker ________________________________ From: NANOG <nanog-bounces+chris=thesysadmin.au@nanog.org> on behalf of Jean Franco <jfranco@maila.inf.br> Sent: Tuesday, December 24, 2024 10:33 AM To: North American Network Operators' Group <nanog@nanog.org> Subject: Best way to have redundancy announcing on separate routers Hi Folks, I'm trying to achieve total redundancy on a multihomed environment: ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 Where X is my Network. In the example below, he announces separate blocks to each ISP. https://www.networkstraining.com/cisco-bgp-configuration-tutorial/ I would like to do a failover model, where if one ISP goes down the other would take over. Please share your thoughts on this. Best regards,
Hi Chris, Thank you for taking your time and point me in the right direction! I'm getting full routes, so it should be easy for me to achieve your concept. Best regards, On Mon, Dec 23, 2024 at 9:53 PM Christopher Hawker <chris@thesysadmin.au> wrote:
Hi Jean,
You can establish an iBGP session between the two routers that exchange either default & own routes, or they can send their own routes with fulls and use local pref to preference the directly-connected transit session before routes learnt from the iBGP session, depending on how you want engineer your traffic. If you are not receiving full tables and only getting a default from each transit provider you would need to weight the defaults so it uses the preferred default. If you're planning to add (for example) peering or PNIs to either router in the future, you will want full tables for greater traffic control.
Regards, Christopher Hawker ------------------------------ *From:* NANOG <nanog-bounces+chris=thesysadmin.au@nanog.org> on behalf of Jean Franco <jfranco@maila.inf.br> *Sent:* Tuesday, December 24, 2024 10:33 AM *To:* North American Network Operators' Group <nanog@nanog.org> *Subject:* Best way to have redundancy announcing on separate routers
Hi Folks,
I'm trying to achieve total redundancy on a multihomed environment:
ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 Where X is my Network.
In the example below, he announces separate blocks to each ISP.
https://www.networkstraining.com/cisco-bgp-configuration-tutorial/
I would like to do a failover model, where if one ISP goes down the other would take over. Please share your thoughts on this.
Best regards,
On Mon, Dec 23, 2024 at 4:53 PM Christopher Hawker <chris@thesysadmin.au> wrote:
You can establish an iBGP session between the two routers that exchange either default & own routes, or they can send their own routes with fulls and use local pref to preference the directly-connected transit session before routes learnt from the iBGP session, depending on how you want engineer your traffic.
That's the easy part. If you want the ISPs to be equal with the shortest path getting the traffic then you're done. Congratulations.
I would like to do a failover model, where if one ISP goes down the other would take over.
If you want to weigh one ISP to be "primary" and the other to be "backup," you've a long hard road ahead of you. Localprefs can make you prefer one ISP over the other for _outbound_ traffic but the levers for controlling _inbound_ traffic are more complicated. You can get part of the way there by "prepending" your AS number several times on the backup path. That makes the AS path longer from the backup ISP which tends to cause BGP selection to pick the shorter path via the primary ISP. That's basically BGP's default: shorter AS path wins. Except for all the jackals out there who use a local mechanism to pick the best path without regard to the AS path length. For those, you'll have to learn about "communities." Communities are basically tags: you tag a route and if your ISP understands the tag it does something different than normal with that route. Your ISPs publish a list of communities they understand along with what they will do differently if you tag a route with that community. Typically you'll want to find the community that tells your ISP to set their own localpref differently than the default. You may even need to find the communities that tell your ISP's ISPs to set their localprefs differently than their defaults. It gets complicated fast. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
Thanks Bill for the well explanation! I'll probably will have to go into the communities then, some of tests I've done got me nowhere! I'm using VyOS (quagga) and prepending didn't help. Best regards, On Tue, Dec 24, 2024 at 12:42 AM William Herrin <bill@herrin.us> wrote:
You can establish an iBGP session between the two routers that exchange either default & own routes, or they can send their own routes with fulls and use local pref to preference the directly-connected transit session before routes learnt from the iBGP session, depending on how you want engineer your
On Mon, Dec 23, 2024 at 4:53 PM Christopher Hawker <chris@thesysadmin.au> wrote: traffic.
That's the easy part. If you want the ISPs to be equal with the shortest path getting the traffic then you're done. Congratulations.
I would like to do a failover model, where if one ISP goes down the other would take over.
If you want to weigh one ISP to be "primary" and the other to be "backup," you've a long hard road ahead of you. Localprefs can make you prefer one ISP over the other for _outbound_ traffic but the levers for controlling _inbound_ traffic are more complicated.
You can get part of the way there by "prepending" your AS number several times on the backup path. That makes the AS path longer from the backup ISP which tends to cause BGP selection to pick the shorter path via the primary ISP. That's basically BGP's default: shorter AS path wins.
Except for all the jackals out there who use a local mechanism to pick the best path without regard to the AS path length. For those, you'll have to learn about "communities." Communities are basically tags: you tag a route and if your ISP understands the tag it does something different than normal with that route. Your ISPs publish a list of communities they understand along with what they will do differently if you tag a route with that community. Typically you'll want to find the community that tells your ISP to set their own localpref differently than the default. You may even need to find the communities that tell your ISP's ISPs to set their localprefs differently than their defaults. It gets complicated fast.
Regards, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
On 12/24/24 5:19 AM, Jean Franco wrote:
Thanks Bill for the well explanation! I'll probably will have to go into the communities then, some of tests I've done got me nowhere! I'm using VyOS (quagga) and prepending didn't help.
If you want, post your IP block(s), ASN and your upstreams' ASNs, a number of people will be able to see if you're prepending correctly. Also feel free to contact your upstreams' BGP support, you're paying them, and they may explain what they support too if it's not written down publicly. IMHO, this is exactly the thing NANOG is here for, helping others run BGP. -- Bryan Fields 727-409-1194 - Voice http://bryanfields.net
On Dec 25, 2024, at 12:09, Bryan Fields <Bryan@bryanfields.net> wrote:
If you want, post your IP block(s), ASN and your upstreams' ASNs, a number of people will be able to see if you're prepending correctly. Also feel free to contact your upstreams' BGP support, you're paying them, and they may explain what they support too if it's not written down publicly.
IMHO, this is exactly the thing NANOG is here for, helping others run BGP.
Wholeheartedly agree — I would *much* rather someone send a message to NANOG asking for help, instead of haphazardly doing BGP to the detriment of their customers and the internet as a whole. Helping someone may be a small “inconvenience” but sure as hell beats another AS7007 incident.
-- Bryan Fields
727-409-1194 - Voice http://bryanfields.net
On Dec 25, 2024, at 17:18, Randy Bush <randy@psg.com> wrote:
IMHO, this is exactly the thing NANOG is here for, helping others run BGP.
where does one go for is-is help? the mtu issie can be painful!!!
I don’t think a mailing list would be of help… perhaps a local tavern instead!
IMHO, this is exactly the thing NANOG is here for, helping others run BGP. where does one go for is-is help? the mtu issie can be painful!!! I don’t think a mailing list would be of help… perhaps a local tavern instead!
actually, the mailing list, particularly saku among others if i remember aright, helped a lot. randy
On 12/25/24 6:18 PM, Randy Bush wrote:
where does one go for is-is help? the mtu issie can be painful!!!
I think here would be good too. I recently had to do this between a Cisco 3945e and a Juniper, and from my unrevised notes: vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } } :-) -- Bryan Fields 727-409-1194 - Voice http://bryanfields.net
Hi guys, I've been on the list for as long as I cannot even remember. So just you know, I'm not new at this. This is no easy task, that's why I came here looking for help. I'm sorry if I brought anguish to the experts on the list! I thought I could bring something that someone may have experienced before. I haven't solved this yet, but at least I've received some valuable suggestions and I Thank you! About all the details of the connections, numbers of peerings, PNI's and IXP's I have left them out, since I figured this additional information could make things worse. ISP 1 <router01> ====20KM====<Router>====20KM====<router02> ISP2 The ISP connections are all 10G. I don't believe these routers are DFZ capable. All the routers are well capable and already receive the full routes. The connections between these routers are 40G. Best regards, On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields <Bryan@bryanfields.net> wrote:
On 12/25/24 6:18 PM, Randy Bush wrote:
where does one go for is-is help? the mtu issie can be painful!!!
I think here would be good too. I recently had to do this between a Cisco 3945e and a Juniper, and from my unrevised notes:
vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } }
:-)
-- Bryan Fields
727-409-1194 - Voice http://bryanfields.net
Jean- Yeah, don't worry about people complaining. Is this an accurate description of what you are trying to achieve? - Have 2 different sets of prefixes that you announce. Set A via router1/ISP1 , Set B via router2/ISP2 - If BGP to one of your ISPs goes down, start announcing those prefixes to the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B over ISP1 ) On Thu, Dec 26, 2024 at 8:16 AM Jean Franco <jfranco@maila.inf.br> wrote:
Hi guys, I've been on the list for as long as I cannot even remember. So just you know, I'm not new at this.
This is no easy task, that's why I came here looking for help. I'm sorry if I brought anguish to the experts on the list! I thought I could bring something that someone may have experienced before.
I haven't solved this yet, but at least I've received some valuable suggestions and I Thank you!
About all the details of the connections, numbers of peerings, PNI's and IXP's I have left them out, since I figured this additional information could make things worse.
ISP 1 <router01> ====20KM====<Router>====20KM====<router02> ISP2
The ISP connections are all 10G. I don't believe these routers are DFZ capable. All the routers are well capable and already receive the full routes. The connections between these routers are 40G.
Best regards,
On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields <Bryan@bryanfields.net> wrote:
On 12/25/24 6:18 PM, Randy Bush wrote:
where does one go for is-is help? the mtu issie can be painful!!!
I think here would be good too. I recently had to do this between a Cisco 3945e and a Juniper, and from my unrevised notes:
vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } }
:-)
-- Bryan Fields
727-409-1194 - Voice http://bryanfields.net
Hi Tom, This is exactly what I was planning. I'm announcing a block via ISP1 and another set of blocks via ISP2, and have iBGP running between them. Thanks a lot!! Best regards, On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher <beecher@beecher.cc> wrote:
Jean-
Yeah, don't worry about people complaining.
Is this an accurate description of what you are trying to achieve?
- Have 2 different sets of prefixes that you announce. Set A via router1/ISP1 , Set B via router2/ISP2 - If BGP to one of your ISPs goes down, start announcing those prefixes to the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B over ISP1 )
On Thu, Dec 26, 2024 at 8:16 AM Jean Franco <jfranco@maila.inf.br> wrote:
Hi guys, I've been on the list for as long as I cannot even remember. So just you know, I'm not new at this.
This is no easy task, that's why I came here looking for help. I'm sorry if I brought anguish to the experts on the list! I thought I could bring something that someone may have experienced before.
I haven't solved this yet, but at least I've received some valuable suggestions and I Thank you!
About all the details of the connections, numbers of peerings, PNI's and IXP's I have left them out, since I figured this additional information could make things worse.
ISP 1 <router01> ====20KM====<Router>====20KM====<router02> ISP2
The ISP connections are all 10G. I don't believe these routers are DFZ capable. All the routers are well capable and already receive the full routes. The connections between these routers are 40G.
Best regards,
On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields <Bryan@bryanfields.net> wrote:
On 12/25/24 6:18 PM, Randy Bush wrote:
where does one go for is-is help? the mtu issie can be painful!!!
I think here would be good too. I recently had to do this between a Cisco 3945e and a Juniper, and from my unrevised notes:
vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } }
:-)
-- Bryan Fields
727-409-1194 - Voice http://bryanfields.net
We have a similiar setup with each of our BGP routers connected to different ISPs. Each ISP sends us full tables. The simplest method we came up with for outbound connection redundancy was to have router 1 use a default route to router 2, and router 2 use a default route to router 1. We don't redistribute those static defaults or our learned BGP routes to the other routers. During normal times, that default route is never used, as there are more specific routes learned via BGP. If router 1 loses its connection to ISP 1 for any reason, it simply pushes traffic over to router 2 as it's routing table only has the default and any learned OSPF routes from our PE boxes. Outbound traffic from router 1 then goes out through my BGP router 2, then ISP2 based on its learned BGP routes. Redundancy for inbound traffic just works via BGP magic 🙂. 10+ years with this setup and no major issues that we know of. We do have more config & peers on the routers, but I've omitted that for simplicity. ISP 1(announcing full tables) <---> <my BGP router01> ============<my BGP router02> <------>ISP2 (announcing full tables) Happy New Year everyone!! Sam Roche Manager, Network Operations [Logo Description automatically generated] 196 Taylor Road, Bracebridge, ON P1L 1J9 Support: support@lakelandnetworks.com<mailto:support@lakelandnetworks.com> 705-640-0556 TF: 1-844-444-4249 Direct: sroche@lakelandnetworks.com<mailto:sroche@lakelandnetworks.com> 705-640-0086 | https://www.lakelandnetworks.com/faqs/ Lakeland Networks<http://www.lakelandnetworks.com/> ________________________________ From: NANOG <nanog-bounces+sroche=lakelandnetworks.com@nanog.org> on behalf of Jean Franco <jfranco@maila.inf.br> Sent: Friday, December 27, 2024 6:09 PM To: Tom Beecher <beecher@beecher.cc> Cc: Bryan Fields <Bryan@bryanfields.net>; nanog@nanog.org <nanog@nanog.org> Subject: Re: Best way to have redundancy announcing on separate routers Hi Tom, This is exactly what I was planning. I'm announcing a block via ISP1 and another set of blocks via ISP2, and have iBGP running between them. Thanks a lot!! Best regards, On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher <beecher@beecher.cc<mailto:beecher@beecher.cc>> wrote: Jean- Yeah, don't worry about people complaining. Is this an accurate description of what you are trying to achieve? - Have 2 different sets of prefixes that you announce. Set A via router1/ISP1 , Set B via router2/ISP2 - If BGP to one of your ISPs goes down, start announcing those prefixes to the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B over ISP1 ) On Thu, Dec 26, 2024 at 8:16 AM Jean Franco <jfranco@maila.inf.br<mailto:jfranco@maila.inf.br>> wrote: Hi guys, I've been on the list for as long as I cannot even remember. So just you know, I'm not new at this. This is no easy task, that's why I came here looking for help. I'm sorry if I brought anguish to the experts on the list! I thought I could bring something that someone may have experienced before. I haven't solved this yet, but at least I've received some valuable suggestions and I Thank you! About all the details of the connections, numbers of peerings, PNI's and IXP's I have left them out, since I figured this additional information could make things worse. ISP 1 <router01> ====20KM====<Router>====20KM====<router02> ISP2 The ISP connections are all 10G. I don't believe these routers are DFZ capable. All the routers are well capable and already receive the full routes. The connections between these routers are 40G. Best regards, On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields <Bryan@bryanfields.net<mailto:Bryan@bryanfields.net>> wrote: On 12/25/24 6:18 PM, Randy Bush wrote:
where does one go for is-is help? the mtu issie can be painful!!!
I think here would be good too. I recently had to do this between a Cisco 3945e and a Juniper, and from my unrevised notes: vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } } :-) -- Bryan Fields 727-409-1194 - Voice http://bryanfields.net
Jean- Thanks. Many BGP implementations have the ability to do conditional advertisements, where you announce (or don't) a set of prefixes based on the presents (or absence) of other routes. I don't think quagga does natively, and not sure if VyOS has added that on. Conceptually, you want to be doing "announce these prefixes from this router only if I don't see routes from the upstream on the other router". The 'safest' way is probably to just monitor default, but it depends on your environment. On Fri, Dec 27, 2024 at 6:09 PM Jean Franco <jfranco@maila.inf.br> wrote:
Hi Tom, This is exactly what I was planning. I'm announcing a block via ISP1 and another set of blocks via ISP2, and have iBGP running between them.
Thanks a lot!!
Best regards,
On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher <beecher@beecher.cc> wrote:
Jean-
Yeah, don't worry about people complaining.
Is this an accurate description of what you are trying to achieve?
- Have 2 different sets of prefixes that you announce. Set A via router1/ISP1 , Set B via router2/ISP2 - If BGP to one of your ISPs goes down, start announcing those prefixes to the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B over ISP1 )
On Thu, Dec 26, 2024 at 8:16 AM Jean Franco <jfranco@maila.inf.br> wrote:
Hi guys, I've been on the list for as long as I cannot even remember. So just you know, I'm not new at this.
This is no easy task, that's why I came here looking for help. I'm sorry if I brought anguish to the experts on the list! I thought I could bring something that someone may have experienced before.
I haven't solved this yet, but at least I've received some valuable suggestions and I Thank you!
About all the details of the connections, numbers of peerings, PNI's and IXP's I have left them out, since I figured this additional information could make things worse.
ISP 1 <router01> ====20KM====<Router>====20KM====<router02> ISP2
The ISP connections are all 10G. I don't believe these routers are DFZ capable. All the routers are well capable and already receive the full routes. The connections between these routers are 40G.
Best regards,
On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields <Bryan@bryanfields.net> wrote:
On 12/25/24 6:18 PM, Randy Bush wrote:
where does one go for is-is help? the mtu issie can be painful!!!
I think here would be good too. I recently had to do this between a Cisco 3945e and a Juniper, and from my unrevised notes:
vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } }
:-)
-- Bryan Fields
727-409-1194 - Voice http://bryanfields.net
On Thu, Jan 2, 2025 at 2:55 PM Tom Beecher <beecher@beecher.cc> wrote:
Jean-
Thanks. Many BGP implementations have the ability to do conditional advertisements, where you announce (or don't) a set of prefixes based on the presents (or absence) of other routes. I don't think quagga does natively, and not sure if VyOS has added that on.
Conceptually, you want to be doing "announce these prefixes from this router only if I don't see routes from the upstream on the other router". The 'safest' way is probably to just monitor default, but it depends on your environment.
That sort of thing seems like extra complexity, no? If the 2 internal routers have iBGP and you are fairly sure that you won't lose that path/view you should be able to just announce the same prefixes to both ISP peerings and possibly add some metric-equivalent data to distance one link vs the other, no? (common metric for this is the as-path, add your as N times, where N is <10 and > 2 probably?) how exact do you want your split here to be jfranco ? (is 'mostly everything over PRIMARY with some over SECONDARY' ok?)
On Fri, Dec 27, 2024 at 6:09 PM Jean Franco <jfranco@maila.inf.br> wrote:
Hi Tom, This is exactly what I was planning. I'm announcing a block via ISP1 and another set of blocks via ISP2, and have iBGP running between them.
Thanks a lot!!
Best regards,
On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher <beecher@beecher.cc> wrote:
Jean-
Yeah, don't worry about people complaining.
Is this an accurate description of what you are trying to achieve?
- Have 2 different sets of prefixes that you announce. Set A via router1/ISP1 , Set B via router2/ISP2 - If BGP to one of your ISPs goes down, start announcing those prefixes to the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B over ISP1 )
On Thu, Dec 26, 2024 at 8:16 AM Jean Franco <jfranco@maila.inf.br> wrote:
Hi guys, I've been on the list for as long as I cannot even remember. So just you know, I'm not new at this.
This is no easy task, that's why I came here looking for help. I'm sorry if I brought anguish to the experts on the list! I thought I could bring something that someone may have experienced before.
I haven't solved this yet, but at least I've received some valuable suggestions and I Thank you!
About all the details of the connections, numbers of peerings, PNI's and IXP's I have left them out, since I figured this additional information could make things worse.
ISP 1 <router01> ====20KM====<Router>====20KM====<router02> ISP2
The ISP connections are all 10G. I don't believe these routers are DFZ capable. All the routers are well capable and already receive the full routes. The connections between these routers are 40G.
Best regards,
On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields <Bryan@bryanfields.net> wrote:
On 12/25/24 6:18 PM, Randy Bush wrote:
where does one go for is-is help? the mtu issie can be painful!!!
I think here would be good too. I recently had to do this between a Cisco 3945e and a Juniper, and from my unrevised notes:
vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } }
:-)
-- Bryan Fields
727-409-1194 - Voice http://bryanfields.net
I'm not saying I *would* do it, or you *should* do it, I'm just answering the questions being asked. :) On Thu, Jan 2, 2025 at 3:21 PM Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Thu, Jan 2, 2025 at 2:55 PM Tom Beecher <beecher@beecher.cc> wrote:
Jean-
Thanks. Many BGP implementations have the ability to do conditional
advertisements, where you announce (or don't) a set of prefixes based on the presents (or absence) of other routes. I don't think quagga does natively, and not sure if VyOS has added that on.
Conceptually, you want to be doing "announce these prefixes from this
router only if I don't see routes from the upstream on the other router". The 'safest' way is probably to just monitor default, but it depends on your environment.
That sort of thing seems like extra complexity, no? If the 2 internal routers have iBGP and you are fairly sure that you won't lose that path/view you should be able to just announce the same prefixes to both ISP peerings and possibly add some metric-equivalent data to distance one link vs the other, no? (common metric for this is the as-path, add your as N times, where N is <10 and > 2 probably?)
how exact do you want your split here to be jfranco ? (is 'mostly everything over PRIMARY with some over SECONDARY' ok?)
On Fri, Dec 27, 2024 at 6:09 PM Jean Franco <jfranco@maila.inf.br> wrote:
Hi Tom, This is exactly what I was planning. I'm announcing a block via ISP1 and another set of blocks via ISP2, and
have iBGP running between them.
Thanks a lot!!
Best regards,
On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher <beecher@beecher.cc> wrote:
Jean-
Yeah, don't worry about people complaining.
Is this an accurate description of what you are trying to achieve?
- Have 2 different sets of prefixes that you announce. Set A via
- If BGP to one of your ISPs goes down, start announcing those
router1/ISP1 , Set B via router2/ISP2 prefixes to the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B over ISP1 )
On Thu, Dec 26, 2024 at 8:16 AM Jean Franco <jfranco@maila.inf.br>
wrote:
Hi guys, I've been on the list for as long as I cannot even remember. So just you know, I'm not new at this.
This is no easy task, that's why I came here looking for help. I'm sorry if I brought anguish to the experts on the list! I thought I could bring something that someone may have experienced
before.
I haven't solved this yet, but at least I've received some valuable
suggestions and I Thank you!
About all the details of the connections, numbers of peerings, PNI's
and IXP's I have left them out, since I figured this additional information could make things worse.
ISP 1 <router01> ====20KM====<Router>====20KM====<router02> ISP2
The ISP connections are all 10G. I don't believe these routers are DFZ capable. All the routers are well capable and already receive the full routes. The connections between these routers are 40G.
Best regards,
On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields <Bryan@bryanfields.net>
wrote:
On 12/25/24 6:18 PM, Randy Bush wrote: > where does one go for is-is help? the mtu issie can be painful!!!
I think here would be good too. I recently had to do this between a
Cisco
3945e and a Juniper, and from my unrevised notes:
vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } }
:-)
-- Bryan Fields
727-409-1194 - Voice http://bryanfields.net
On Thu, Jan 2, 2025 at 3:24 PM Tom Beecher <beecher@beecher.cc> wrote:
I'm not saying I *would* do it, or you *should* do it, I'm just answering the questions being asked. :)
oh! fair enough... "that is a ton of complexity" still applies, or at least for my view that's more complexity than: "always I announce to all, with some metrics to make me seem there but just farther away on link X"
On Thu, Jan 2, 2025 at 3:21 PM Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Thu, Jan 2, 2025 at 2:55 PM Tom Beecher <beecher@beecher.cc> wrote:
Jean-
Thanks. Many BGP implementations have the ability to do conditional advertisements, where you announce (or don't) a set of prefixes based on the presents (or absence) of other routes. I don't think quagga does natively, and not sure if VyOS has added that on.
Conceptually, you want to be doing "announce these prefixes from this router only if I don't see routes from the upstream on the other router". The 'safest' way is probably to just monitor default, but it depends on your environment.
That sort of thing seems like extra complexity, no? If the 2 internal routers have iBGP and you are fairly sure that you won't lose that path/view you should be able to just announce the same prefixes to both ISP peerings and possibly add some metric-equivalent data to distance one link vs the other, no? (common metric for this is the as-path, add your as N times, where N is <10 and > 2 probably?)
how exact do you want your split here to be jfranco ? (is 'mostly everything over PRIMARY with some over SECONDARY' ok?)
On Fri, Dec 27, 2024 at 6:09 PM Jean Franco <jfranco@maila.inf.br> wrote:
Hi Tom, This is exactly what I was planning. I'm announcing a block via ISP1 and another set of blocks via ISP2, and have iBGP running between them.
Thanks a lot!!
Best regards,
On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher <beecher@beecher.cc> wrote:
Jean-
Yeah, don't worry about people complaining.
Is this an accurate description of what you are trying to achieve?
- Have 2 different sets of prefixes that you announce. Set A via router1/ISP1 , Set B via router2/ISP2 - If BGP to one of your ISPs goes down, start announcing those prefixes to the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B over ISP1 )
On Thu, Dec 26, 2024 at 8:16 AM Jean Franco <jfranco@maila.inf.br> wrote:
Hi guys, I've been on the list for as long as I cannot even remember. So just you know, I'm not new at this.
This is no easy task, that's why I came here looking for help. I'm sorry if I brought anguish to the experts on the list! I thought I could bring something that someone may have experienced before.
I haven't solved this yet, but at least I've received some valuable suggestions and I Thank you!
About all the details of the connections, numbers of peerings, PNI's and IXP's I have left them out, since I figured this additional information could make things worse.
ISP 1 <router01> ====20KM====<Router>====20KM====<router02> ISP2
The ISP connections are all 10G. I don't believe these routers are DFZ capable. All the routers are well capable and already receive the full routes. The connections between these routers are 40G.
Best regards,
On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields <Bryan@bryanfields.net> wrote: > > On 12/25/24 6:18 PM, Randy Bush wrote: > > where does one go for is-is help? the mtu issie can be painful!!! > > I think here would be good too. I recently had to do this between a Cisco > 3945e and a Juniper, and from my unrevised notes: > > vlan { > unit 405 { > family iso { > # holy shit this is important. CISCO and Juniper will not talk unless the > MTU is set > mtu 1492; > } > } > } > > :-) > > -- > Bryan Fields > > 727-409-1194 - Voice > http://bryanfields.net
On Thu, 26 Dec 2024 at 05:55, Bryan Fields <Bryan@bryanfields.net> wrote:
vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } }
This is almost certainly the wrong solution to a real problem. Real problem being, you have MTU mismatch, and the wrong solution is you solve it for one ifa. You almost certainly want the MTU to be fixed in ifl or maybe even ifd. -- ++ytti
On 12/26/24 8:24 AM, Saku Ytti wrote:
On Thu, 26 Dec 2024 at 05:55, Bryan Fields <Bryan@bryanfields.net> wrote:
vlan { unit 405 { family iso { # holy shit this is important. CISCO and Juniper will not talk unless the MTU is set mtu 1492; } } } This is almost certainly the wrong solution to a real problem. Real problem being, you have MTU mismatch, and the wrong solution is you solve it for one ifa. You almost certainly want the MTU to be fixed in ifl or maybe even ifd.
I believe it was that one side was vlan tagged and the other wasn't or something to the effect of how the ISO MTU was calculated in the classic-IOS version vs. Junos. As this was for less than a carrier grade setup (tactical deployment during an hurricane) I didn't have time to get deep into it. A 3945e isn't exactly current anyway. I think the other juniper involved was a 4500, so kind of in the same age as the cisco. -- Bryan Fields 727-409-1194 - Voice http://bryanfields.net
On Thu, 26 Dec 2024 at 17:51, Bryan Fields <Bryan@bryanfields.net> wrote:
I believe it was that one side was vlan tagged and the other wasn't or something to the effect of how the ISO MTU was calculated in the classic-IOS version vs. Junos. As this was for less than a carrier grade setup (tactical deployment during an hurricane) I didn't have time to get deep into it. A 3945e isn't exactly current anyway. I think the other juniper involved was a 4500, so kind of in the same age as the cisco.
My money is that doing IPv4 ping with DF-bit on, you'd see loss from one side with lower MTU. Likely the Junos side had a lower MTU, as you are supposed to increase the IFD MTU by 4 if you add a VLAN encap, which you do not need to do in 3945. However, personally if you're going to have ifl's, I would set the ifd to maximum, and set ifls at 1500, so that when one ifl one day wants larger mtu, you don't have to break the world. -- ++ytti
Hi Jean, On 24/12/2024 03:33, Jean Franco wrote:
Hi Folks,
I'm trying to achieve total redundancy on a multihomed environment:
ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 Where X is my Network. [...] I would like to do a failover model, where if one ISP goes down the other would take over. Please share your thoughts on this.
I would suggest you read the materials here https://learn.nsrc.org/bgp "Multi-Homing" section. I believe you will find more extensive documentation about what you are trying to achieve. -- Willy Manga
How much capacity and how much network do you have between router 1 and router 2? Are the routers between DFZ capable? Do the links have the ability to carry the full load? How many routers between router 1 and router 2? ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Jean Franco" <jfranco@maila.inf.br> To: "North American Network Operators' Group" <nanog@nanog.org> Sent: Monday, December 23, 2024 5:33:06 PM Subject: Best way to have redundancy announcing on separate routers Hi Folks, I'm trying to achieve total redundancy on a multihomed environment: ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 Where X is my Network. In the example below, he announces separate blocks to each ISP. https://www.networkstraining.com/cisco-bgp-configuration-tutorial/ I would like to do a failover model, where if one ISP goes down the other would take over. Please share your thoughts on this. Best regards,
On 12/23/2024 5:33 PM, Jean Franco wrote:
I'm trying to achieve total redundancy on a multihomed environment:
ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 Where X is my Network.
The hardest part can be handling a failure of either of the routers and having X still be able to talk to the other in smaller networks. While VRRP, MC-LAG, and MPLS do exist, platform, vendor, and your requirements all make for a lot of fun. It's easy to accidentally make routers do things the vendor hadn't intended (What do you mean subscriber services aren't designed to work with mc-ae? Is that why dhcp sync only works with vrrp and mpls and not mc-ae with unnumbered interfaces?) I'll try not to cover what others have said, but there are a few things to consider on dealing with your ISPs. They may run RPF filtering, so even if you don't want them to route traffic for a network to you, if you might send traffic from that network out, they'll need a route, so always send the aggregates to everyone you send outbound traffic to. If you have trouble getting a network added to a peer, you may have to not send any outbound their way. Many ISPs run local prefs to prefer directly connected networks over more costly paths. This will override AS prepends. Some may let you change it with a community. Some will not. If you must force traffic, use a more specific route. Even if others filter it out, it should still get enough distance to force traffic the way you want. If your redundancy is slightly oversold and you need rough load balancing, more specific routes are the way to handle that, but try and minimize their use. We do have routing table bloat. Jack
participants (16)
-
Bryan Fields -
Christopher Hawker -
Christopher Morrow -
Dave Edelman -
Jack Bates -
Jean Franco -
Mike Hammett -
Pedro Prado -
Randy Bush -
Saku Ytti -
Sam Roche -
Tim Burke -
Tom Beecher -
Tony Wicks -
William Herrin -
Willy Manga