Hey Erik, 1) This messes up the design and introduces unnecessary complexity. As your issue is not directly caused by having a RR topology, I’d avoid doing that. 2) That, IMHO, would be the optimal solution, assuming you don’t have enough internal routes to overflow the TCAM of your PEs in the near future. This would also solve some potential loops if you ever want to pass unlabelled traffic. 3) That’s a very generic question with broad potential answers. 4) You could, but you’d have to evaluate that trade-off yourself ;) Kind regards, My thoughts and words are my own. Spyros On 04/05/2018, 09:02, "NANOG on behalf of Erik Sundberg" <nanog-bounces@nanog.org on behalf of ESundberg@nitelusa.com> wrote: I have a RR Client design question...... CORE1-------------------2x10G-----------------------CORE2 | | | | | 10G Ring | | | | | PE1----------PE2----------PE3----------PE4----------PE5 -Core1 & Core2 are RR Reflectors with full IPV4 Tables (ASR9K) -MPLS LDP Enabled -IGP is ISIS -Each PE peers only with Core1 and Core2 as RR Clients with iBGP -PE's are only receiving a default route from the Core Routers due to TCAM size of 20K (ASR920's\ME3800's) -The ring does not have that much traffic on it <500m, so I do not want to use additional 10G ports on the Core's and is why I have it in a 10G U ring. -Primary link to the cores is via the PE1 --- CORE1 Like......... For this discussion the link between PE5 to CORE2 is set up as a backup link. The scenario is I have traffic between PE2 and PE3. Since the PE's are only receiving a default route from the Cores. Traffic is label switch from PE2 - PE1 - Core1 does a IP lookup at Ingress then label switches back to PE1-PE2-PE3. This ends up being 5 hops and doubling the traffic on the link to the Cores. My questions is how do I get traffic to go directly between the PE's without going to the Core Routers? 1. Can I enable iBGP between the PE's in a full mesh to allow traffic between the PE's without going to the core's. Or does this break the Route Reflector model? 2. Create a route policy on the Core's advertising routes learned from the PE's back to all the PE's on the ring. 3. Is this one of the down sides to U Rings? 4. Leave it alone and move on to bigger and better things.... Thanks Erik ________________________________ CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.

Hello, In order to accept only the default route, I assume that you want to have internet access to the ASR920 inside a vrf. ?!? If this is the case, your consideration of the default route and the TCAM size is correct. But, if there is internet traffic between the PE2-PE3 in the same vrf , then I think that its ok to leak more specific prefixes from PE2 to PE3 (by using specific Route Targets) from the CORE1 & CORE2 (RR). Unless there is something that I miss, option #2, is more favorable. Michalis Bersimis -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Erik Sundberg Sent: Friday, May 04, 2018 9:02 AM To: NANOG Subject: Route Reflector Client Design Question I have a RR Client design question...... CORE1-------------------2x10G-----------------------CORE2 | | | | | 10G Ring | | | | | | PE1----------PE2----------PE3----------PE4----------PE5 -Core1 & Core2 are RR Reflectors with full IPV4 Tables (ASR9K) -MPLS LDP Enabled -IGP is ISIS -Each PE peers only with Core1 and Core2 as RR Clients with iBGP -PE's are only receiving a default route from the Core Routers due to TCAM size of 20K (ASR920's\ME3800's) -The ring does not have that much traffic on it <500m, so I do not want to use additional 10G ports on the Core's and is why I have it in a 10G U ring. -Primary link to the cores is via the PE1 --- CORE1 Like......... For this discussion the link between PE5 to CORE2 is set up as a backup link. The scenario is I have traffic between PE2 and PE3. Since the PE's are only receiving a default route from the Cores. Traffic is label switch from PE2 - PE1 - Core1 does a IP lookup at Ingress then label switches back to PE1-PE2-PE3. This ends up being 5 hops and doubling the traffic on the link to the Cores. My questions is how do I get traffic to go directly between the PE's without going to the Core Routers? 1. Can I enable iBGP between the PE's in a full mesh to allow traffic between the PE's without going to the core's. Or does this break the Route Reflector model? 2. Create a route policy on the Core's advertising routes learned from the PE's back to all the PE's on the ring. 3. Is this one of the down sides to U Rings? 4. Leave it alone and move on to bigger and better things.... Thanks Erik ________________________________ CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.

Erik, Before I email my suggestions, can you clarify the followings; Do Core1 and Core2 also provide the function of BDRs peering with your upstream/s? Or Just acting as Core/RRs with 500Mbps of traffic traversing through them? Cheers Ahad On Fri, May 4, 2018 at 4:01 PM, Erik Sundberg <ESundberg@nitelusa.com> wrote:

I have a RR Client design question......

CORE1-------------------2x10G-----------------------CORE2 | | | | | 10G Ring | | | | | PE1----------PE2----------PE3----------PE4----------PE5

-Core1 & Core2 are RR Reflectors with full IPV4 Tables (ASR9K) -MPLS LDP Enabled -IGP is ISIS -Each PE peers only with Core1 and Core2 as RR Clients with iBGP -PE's are only receiving a default route from the Core Routers due to TCAM size of 20K (ASR920's\ME3800's) -The ring does not have that much traffic on it <500m, so I do not want to use additional 10G ports on the Core's and is why I have it in a 10G U ring. -Primary link to the cores is via the PE1 --- CORE1 Like......... For this discussion the link between PE5 to CORE2 is set up as a backup link.

The scenario is I have traffic between PE2 and PE3. Since the PE's are only receiving a default route from the Cores. Traffic is label switch from PE2 - PE1 - Core1 does a IP lookup at Ingress then label switches back to PE1-PE2-PE3. This ends up being 5 hops and doubling the traffic on the link to the Cores.

My questions is how do I get traffic to go directly between the PE's without going to the Core Routers?

1. Can I enable iBGP between the PE's in a full mesh to allow traffic between the PE's without going to the core's. Or does this break the Route Reflector model? 2. Create a route policy on the Core's advertising routes learned from the PE's back to all the PE's on the ring. 3. Is this one of the down sides to U Rings? 4. Leave it alone and move on to bigger and better things....

Thanks

Erik

________________________________

CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.

On 4/May/18 08:01, Erik Sundberg wrote:

My questions is how do I get traffic to go directly between the PE's without going to the Core Routers?

1. Can I enable iBGP between the PE's in a full mesh to allow traffic between the PE's without going to the core's. Or does this break the Route Reflector model?

You could do, but then you lose the point of the RR in the first place, as it's likely your Metro-E nodes will continue to grow, making this iBGP mesh thing, well, messy.

2. Create a route policy on the Core's advertising routes learned from the PE's back to all the PE's on the ring.

You could do, but adds unnecessary routing complexity since the role of an RR is to, well, reflect.

3. Is this one of the down sides to U Rings?

Not really a downside, just the perks of extending IP/MPLS all the way into the Access (I drink to the death of Layer 2 Metro-E networks - my liver will probably give out before I do, though...).

4. Leave it alone and move on to bigger and better things....

Now where's the fun in that :-)? So we've solved this problem by using BGP-SD (Selective Download): * For every prefix each Metro-E node handles, originate that toward both RR's with NEXT_HOP=self. * Attach a BGP community along with the routes originated toward the RR's. For maximum saving of your precious FIB in your Metro-E nodes, use a BGP community that is unique to the ring. This way, you don't need to accept routes into each Metro-E's FIB that don't require the "local" forwarding. * Ensure the RR's reflect the routes they learn from each Metro-E node to the other Metro-E nodes. * Setup BGP-SD on each Metro-E node. Match the ring-specific BGP community you added to each Metro-E node's prefix origination. Accept those routes into FIB + default. Reject everything else (from populating the FIB). That should give you local forwarding within the ring, while maintaining very sparse population of your Metro-E nodes' FIB's. Mark.