Newbies Question: Do I really need to sacrifice Prefix-aggregation to do BGP Load-sharing? (the case of Multi-homed + Multi-routers + Multi-upstreams)
Dear Guru(s), My apologies if these questions have already been asked; in that case, please kindly point me to the answer(s). I hope the following information sufficiently describes my current "context": - Single customer: ourselves - One big IPv4 block + one big IPv6 block - Native Dual-Stack, Non-tunneling - Non-transit (actually, a “multi-homed Stub”) - “All-green” IRR & RPKI registered (based on IRRexplorer report) - Fully-aggregated route announcement (based on CIDR report) - Two (Cisco) gateway routers on our side - Two upstreams (See the following lines), fully cross-connected to our gateways - One (pure) commercial ISP - One academic consortium ISP (who actually uses the above-mentioned commercial ISP as one of its upstreams as well) My current “situation”: - All inbounds “flock” in through the commercial ISP, overflowing the bandwidth; since (my guess) the academic ISP also uses that commercial ISP as its upstream, there is no way for its path to be shorter. Questions: 1. Do I really have to “de-aggregate” the address blocks, so I can do the “manual BGP load-sharing”? I hate to do it because it will increase the global route-table entries, plus there will be IRR & RPKI “hijack gaps” to contend with at my end. 2. If the answer to the above question is definitely “yes”, please point me to the Best-Practice in doing the “manual BGP load-sharing (on Cisco)”. Right now, all I have is: https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13... Thanks in advance for all the pointers and help given (off mailing-list is also welcome). Best Regards, Pirawat.
On Wed, Oct 19, 2022 at 1:29 AM Pirawat WATANAPONGSE via NANOG <nanog@nanog.org> wrote:
1. Do I really have to “de-aggregate” the address blocks, so I can do the “manual BGP load-sharing”?
Why not prepend toward the commercial ISP? Seems that should make the path longer and less desirable. -- Hunter Fuller (they) Router Jockey VBH M-1C +1 256 824 5331 Office of Information Technology The University of Alabama in Huntsville Network Engineering
On Tue, Oct 18, 2022 at 11:27 PM Pirawat WATANAPONGSE via NANOG <nanog@nanog.org> wrote:
- Two upstreams (See the following lines), fully cross-connected to our gateways - One (pure) commercial ISP - One academic consortium ISP (who actually uses the above-mentioned commercial ISP as one of its upstreams as well)
My current “situation”: - All inbounds “flock” in through the commercial ISP, overflowing the bandwidth; since (my guess) the academic ISP also uses that commercial ISP as its upstream, there is no way for its path to be shorter.
This is what "prepends" are for. You make the AS path sent to the commercial ISP longer by adding extra copies of your own AS to the front. This causes the AS path via the consortium to be the same length or even shorter if you prefer. By default, BGP prefers the most specific prefix first and then the shortest AS path second. Depending on the ISP you may also need to "community" to tell it not to prefer the direct path with you (which they may have done with a localpref). But start with the prepends and see what happens. Regards, Bill Herrin -- For hire. https://bill.herrin.us/resume/
The inbound traffic will be determined by how the Tier 1’s decide to route, as you are observing they will pick either you or your other upstream. Traffic engineering as the Tier 3 carrier you have described has this kind of unexpected traffic routing. As you have obviously already tried common BGP traffic engineering tool of AS Padding your left with next worst option. Best of luck! Kevin Burke 802-540-0979 Burlington Telecom 200 Church St, Burlington, VT From: NANOG <nanog-bounces+kburke=burlingtontelecom.com@nanog.org> On Behalf Of Pirawat WATANAPONGSE via NANOG Sent: Wednesday, October 19, 2022 2:28 AM To: nanog@nanog.org Subject: Newbies Question: Do I really need to sacrifice Prefix-aggregation to do BGP Load-sharing? (the case of Multi-homed + Multi-routers + Multi-upstreams) WARNING!! This message originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. Dear Guru(s), My apologies if these questions have already been asked; in that case, please kindly point me to the answer(s). I hope the following information sufficiently describes my current "context": - Single customer: ourselves - One big IPv4 block + one big IPv6 block - Native Dual-Stack, Non-tunneling - Non-transit (actually, a “multi-homed Stub”) - “All-green” IRR & RPKI registered (based on IRRexplorer report) - Fully-aggregated route announcement (based on CIDR report) - Two (Cisco) gateway routers on our side - Two upstreams (See the following lines), fully cross-connected to our gateways - One (pure) commercial ISP - One academic consortium ISP (who actually uses the above-mentioned commercial ISP as one of its upstreams as well) My current “situation”: - All inbounds “flock” in through the commercial ISP, overflowing the bandwidth; since (my guess) the academic ISP also uses that commercial ISP as its upstream, there is no way for its path to be shorter. Questions: 1. Do I really have to “de-aggregate” the address blocks, so I can do the “manual BGP load-sharing”? I hate to do it because it will increase the global route-table entries, plus there will be IRR & RPKI “hijack gaps” to contend with at my end. 2. If the answer to the above question is definitely “yes”, please point me to the Best-Practice in doing the “manual BGP load-sharing (on Cisco)”. Right now, all I have is: https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13... Thanks in advance for all the pointers and help given (off mailing-list is also welcome). Best Regards, Pirawat.
On 10/18/22 23:27, Pirawat WATANAPONGSE via NANOG wrote:
Dear Guru(s),
My apologies if these questions have already been asked; in that case, please kindly point me to the answer(s).
I hope the following information sufficiently describes my current "context": - Single customer: ourselves - One big IPv4 block + one big IPv6 block - Native Dual-Stack, Non-tunneling - Non-transit (actually, a “multi-homed Stub”) - “All-green” IRR & RPKI registered (based on IRRexplorer report) - Fully-aggregated route announcement (based on CIDR report) - Two (Cisco) gateway routers on our side - Two upstreams (See the following lines), fully cross-connected to our gateways - One (pure) commercial ISP - One academic consortium ISP (who actually uses the above-mentioned commercial ISP as one of its upstreams as well)
My current “situation”: - All inbounds “flock” in through the commercial ISP, overflowing the bandwidth; since (my guess) the academic ISP also uses that commercial ISP as its upstream, there is no way for its path to be shorter.
Many of the commercial ISPs will accept communities allowing you to fine-tune how your advertisements are propagated, some of which are quite granular. This is the best method to do so. Prepending can help but your commercial upstream will by default prefer your route as a direct customer regardless of prepending. Onestep.net used to have a comprehensive listing of many transit providers and the communities they accept but it's returning 403 forbidden now. Ask your upstream for the communities they accept. -- Jay Hennigan - jay@west.net Network Engineering - CCIE #7880 503 897-8550 - WB6RDV
I imagine it's an ISP you are talking about, where the traffic is mostly inbound. Hire transit companies that have good traffic engineering community policies. - Selective prepending or seletive no-export by: -> Type of peer. -> Geographic location of their routers. -> ASN specific. And then you can get the best out of every transit. And so the bandwidth balancing will happen not based on your network prefixes, but based on how the origins see your network. Additionally: DO NOT HIRE transit companies that arbitrarily remove all the communities you mark on the routes you advertise. By targeting communities with ASNs that are 2 or 3 hops away from the AS you can also influence how the rest of the world views your network. And most (not all) of the companies that remove the communities you tag do this to force you to use what they choose and not what you think is best for your network. Em qua., 19 de out. de 2022 03:31, Pirawat WATANAPONGSE via NANOG < nanog@nanog.org> escreveu:
Dear Guru(s),
My apologies if these questions have already been asked; in that case, please kindly point me to the answer(s).
I hope the following information sufficiently describes my current "context": - Single customer: ourselves - One big IPv4 block + one big IPv6 block - Native Dual-Stack, Non-tunneling - Non-transit (actually, a “multi-homed Stub”) - “All-green” IRR & RPKI registered (based on IRRexplorer report) - Fully-aggregated route announcement (based on CIDR report) - Two (Cisco) gateway routers on our side - Two upstreams (See the following lines), fully cross-connected to our gateways - One (pure) commercial ISP - One academic consortium ISP (who actually uses the above-mentioned commercial ISP as one of its upstreams as well)
My current “situation”: - All inbounds “flock” in through the commercial ISP, overflowing the bandwidth; since (my guess) the academic ISP also uses that commercial ISP as its upstream, there is no way for its path to be shorter.
Questions: 1. Do I really have to “de-aggregate” the address blocks, so I can do the “manual BGP load-sharing”? I hate to do it because it will increase the global route-table entries, plus there will be IRR & RPKI “hijack gaps” to contend with at my end. 2. If the answer to the above question is definitely “yes”, please point me to the Best-Practice in doing the “manual BGP load-sharing (on Cisco)”. Right now, all I have is:
https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13...
Thanks in advance for all the pointers and help given (off mailing-list is also welcome).
Best Regards,
Pirawat.
participants (6)
-
Douglas Fischer -
Hunter Fuller -
Jay Hennigan -
Kevin Burke -
Pirawat WATANAPONGSE -
William Herrin