anycast load balancing issue
Hi, I'm in the process of deploying an anycast DNS service internally. We're on a pretty provider-like network, where we run MPLS to provide several network overlays for different services. iBGP is used to distribute routing information, and ISIS is used as IGP. In one of the VRFen we would like to place name servers using a common IP address. To get speedy network updates when outages occur we'll be using OSPF on the name servers to inject the routes into the IGP. The P/E router then redistributes the route into the right VRF. (the name server OSPF process is not aware of MPLS; it just talks to a router.) So far so good. This works. Trouble is, we find that (untweaked) cost and metric are such that all nodes are equal. The last resort (peer router ID) gets invoked and all traffic goes to one single instance. Of course, when that instance falls off the net recalculation takes place and another node steps in, but I'd like true path lengths (IGP hop count) to influence more than iBGP (route-reflector-style) selection. Any clues? Oh, all-cisco, all ASR1000 series. All links GE. ~90 routers in IGP. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 ... this must be what it's like to be a COLLEGE GRADUATE!!
Subject: anycast load balancing issue Date: Wed, Jan 04, 2012 at 01:02:55PM +0100 Quoting Måns Nilsson (mansaxel@besserwisser.org):
Trouble is, we find that (untweaked) cost and metric are such that all nodes are equal.
s/all nodes/all nodes in my pathetically small test case/ Was no issue. I just was unlucky in selecting test cases. Sorry. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Do you have exactly what I want in a plaid poindexter bar bat??
On Jan 4, 2012 4:52 AM, "Måns Nilsson" <mansaxel@besserwisser.org> wrote:
Subject: anycast load balancing issue Date: Wed, Jan 04, 2012 at
01:02:55PM +0100 Quoting Måns Nilsson (mansaxel@besserwisser.org):
Trouble is, we find that (untweaked) cost and metric are such that all nodes are equal.
s/all nodes/all nodes in my pathetically small test case/
Was no issue. I just was unlucky in selecting test cases. Sorry.
-- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Do you have exactly what I want in a plaid poindexter bar bat??
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAk8ES2oACgkQ02/pMZDM1cUXpgCfQtLkFUBsbO5Z3wDPiWV1djQB SukAnA7hBBWC83iTzjjogsxPIfI5GxmK =L5pI -----END PGP SIGNATURE-----
I use: Anycast = server loop back Protocol to server = bgp / bfd This allows for ecmp horizontal scaling for n number of dns servers (where n is less than Max ecmp paths) You may need to turn the bgp ecmp multipath knob.
Hi, On 04.01.2012 13:02, Måns Nilsson wrote:
[..snipped..]
Trouble is, we find that (untweaked) cost and metric are such that all nodes are equal. The last resort (peer router ID) gets invoked and all traffic goes to one single instance. Of course, when that instance falls off the net recalculation takes place and another node steps in, but I'd like true path lengths (IGP hop count) to influence more than iBGP (route-reflector-style) selection.
Any clues?
Oh, all-cisco, all ASR1000 series. All links GE. ~90 routers in IGP.
Since you mention route-reflector route selection - are you already using per-VRF, per-PE route distinguishers for that L3VPN instance? If not, I'd recommend doing so - this will cause your RR to see all paths as unique routes, distributing all of them (instead just the best one from the RR perspective) to RR clients. As result all PEs will always have all paths for this particular prefix (and can then take the best path decision based on local IGP metric to the respective BGP next hops). Doing that can also significantly improve reconvergence times for certain failure scenarios (e.g. ingress PE failure), as PEs can start using alternative paths (already available in local BGP RIB) as soon as the IGP nexthop for the failed PE is invalidated and do not need to wait for BGP RR reconvergence. cheers, -jr
Subject: Re: anycast load balancing issue Date: Thu, Jan 05, 2012 at 04:12:33PM +0100 Quoting Johannes Resch (jr@xor.at):
Any clues?
Since you mention route-reflector route selection - are you already using per-VRF, per-PE route distinguishers for that L3VPN instance?
Problem solved - what I did not tell (shame on me) was that there are two islands of IGP (growing pains...) redistributing to each other... The metric in that redistribution was too low, resulting in artificially "cheap" paths to the wrong places. Thanks all who made me think a second round and solve this. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Hold the MAYO & pass the COSMIC AWARENESS ...
On 6 Jan 2012, at 07:33, "Måns Nilsson" <mansaxel@besserwisser.org> wrote:
Thanks all who made me think a second round and solve this.
Hence why people prefer to ask people and not GOOG et-al. -- Leigh Porter ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________
participants (4)
-
Cameron Byrne -
Johannes Resch -
Leigh Porter -
Måns Nilsson