Canadian routes with BGP no-export communities
Does anyone know what's going on in Canada? In the last couple of weeks fONOROLA/iStar.net changed how they announce aggregated networks. Now fONOROLA is sending some more specific network announcements for multi-homed networks in 198.53.0.0 with a "no-export" BGP community to MCI, and ANS. As long as you only connect with MCI or ANS, and use their IGP, things look pretty typical because the more specifics are carried internally. But if you use BGP, the no-export means any peers with MCI won't see the more specific network announcement from MCI. The problem shows up when you see routes from Sprintlink. A few Canadian network connect through Sprint/Canada as well as fONOROLA. The more specific networks from 198.53 are announced by Sprintlink. Since the no-export suppressed the more specific announcement to MCI BGP peers, the traffic for those Canadian sites follows the more specific network announcement and heads for Sprint. I understand the BGP mechanics are working as specified, so it isn't "broken" per se. But I'm just wondering if Canadian sites really expect traffic to go via Stockton, California? I haven't been able to reach anyone at fONOROLA/Istar. I was just wondering if any other North American network operators had heard what was supposed to happen last month when fONOROLA made these changes. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation
In message <960606181706.9ede@SDG.DRA.COM>, Sean Donelan writes:
Does anyone know what's going on in Canada?
In the last couple of weeks fONOROLA/iStar.net changed how they announce aggregated networks. Now fONOROLA is sending some more specific network announcements for multi-homed networks in 198.53.0.0 with a "no-export" BGP community to MCI, and ANS.
As long as you only connect with MCI or ANS, and use their IGP, things look pretty typical because the more specifics are carried internally. But if you use BGP, the no-export means any peers with MCI won't see the more specific network announcement from MCI.
The problem shows up when you see routes from Sprintlink. A few Canadian network connect through Sprint/Canada as well as fONOROLA. The more specific networks from 198.53 are announced by Sprintlink. Since the no-export suppressed the more specific announcement to MCI BGP peers, the traffic for those Canadian sites follows the more specific network announcement and heads for Sprint.
I understand the BGP mechanics are working as specified, so it isn't "broken" per se. But I'm just wondering if Canadian sites really expect traffic to go via Stockton, California?
I haven't been able to reach anyone at fONOROLA/Istar. I was just wondering if any other North American network operators had heard what was supposed to happen last month when fONOROLA made these changes. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation
Sean, I can't speak for Istar, but I do know that they had tried to make this a painless aggregation by indicating to both ANS and MCI which prefixes were in their aggregates that did not need to be advertised outside of ANS and MCI. Istar saved some 750 prefixes. I think this is roughly half what they had previously been announcing and they have over 3500 prefixes registered, indicating a lot were already site aggregated. This is certainly commendable. Unfortunately, since there is no way for them to know which Istar prefixes also have connectivity through Sprint or someone else (with Sprint failing to register this in the IRR), there is no way that Istar can determine that those component have to be passed on by ANS and MCI except to aggregate and see what breaks. This is the type of "information exchange" and provider cooperation through the IRR that we have been talking about that isn't universally accepted. As this type of non-trivial aggregation becomes more common, we can expect this sort of thing to be a regular occurance unless certain other providers take the time to register things. Yes, we know it is work, but nothing near the work Istar did to get these prefixes aggregated with as little disruption as possible. The more specific prefixes blocked by ANS and MCI are listed below. We can determine which are heard from Sprint with a routing dump and unblock them. This is the "break it, see what broke, then fix it" method, but the end result will work. Cheers, Curtis echo '( AS2493 AND ANSIGPONLY )' | Peval ({204.191.254.0/24, 204.191.9.0/24, 204.191.182.0/24, 198.53.169.0/24, 198.53.159.0/24, 204.191.76.0/24, 204.191.176.0/24, 204.191.111.0/24, 198.53.239.0/24, 204.191.20.0/24, 204.191.85.0/24, 198.53.33.0/24, 204.191.105.0/24, 198.53.173.0/24, 204.191.187.0/24, 204.191.163.0/24, 198.53.183.0/24, 204.191.119.0/24, 204.191.4.0/24, 204.191.126.0/24, 198.53.158.0/24, 198.53.224.0/24, 198.53.228.0/24, 198.53.223.0/24, 198.53.211.0/24, 198.53.164.0/24, 204.191.14.0/24, 204.191.10.0/24, 198.53.19.0/24, 204.191.125.0/24, 205.206.16.0/24, 204.191.60.0/24, 198.53.144.0/24, 198.53.197.0/24, 198.53.217.0/24, 198.53.178.0/24, 204.191.98.0/24, 204.191.58.0/24, 204.191.188.0/24, 198.53.26.0/24, 204.191.174.0/24, 204.191.97.0/24, 204.191.49.0/24, 198.53.184.0/24, 204.191.211.0/24, 198.53.16.0/24, 198.53.176.0/24, 198.53.168.0/24, 204.191.28.0/24, 204.191.233.0/24, 204.191.30.0/24, 204.191.207.0/24, 204.191.224.0/24, 198.53.230.0/24, 198.53.155.0/24, 198.53.207.0/24, 204.191.177.0/24, 204.191.110.0/24, 198.53.236.0/24, 198.53.31.0/24, 204.191.103.0/24, 205.206.17.0/24, 204.191.15.0/24, 204.191.225.0/24, 198.53.167.0/24, 205.206.21.0/24, 198.53.177.0/24, 204.191.160.0/24, 204.191.189.0/24, 205.206.20.0/24, 204.191.175.0/24, 198.53.65.0/24, 204.191.206.0/24, 198.53.27.0/24, 198.53.204.0/24, 198.53.229.0/24, 198.53.80.0/24, 198.53.226.0/24, 198.53.201.0/24, 204.191.127.0/24, 204.191.210.0/24, 204.191.7.0/24, 204.191.122.0/24, 204.191.112.0/24, 204.191.249.0/24, 198.53.66.0/24, 198.53.171.0/24, 204.191.12.0/24, 204.191.179.0/24, 204.191.25.0/24, 198.53.203.0/24, 198.53.220.0/24, 204.191.185.0/24, 198.53.191.0/24, 204.191.205.0/24, 198.53.209.0/24, 204.191.232.0/24, 198.53.30.0/24, 204.191.190.0/24, 198.53.181.0/24, 204.191.102.0/24, 198.53.149.0/24, 204.191.26.0/24, 198.53.67.0/24, 204.191.223.0/24, 204.191.29.0/24, 204.191.240.0/24, 198.53.199.0/24, 198.53.196.0/24, 198.53.162.0/24, 204.191.32.0/24, 204.191.41.0/24, 198.53.25.0/24, 204.191.50.0/24, 204.191.88.0/24, 204.191.27.0/24, 204.191.208.0/24, 198.53.50.0/24, 204.191.5.0/24, 198.53.194.0/24, 204.191.89.0/24, 204.191.248.0/24, 204.191.203.0/24, 204.191.161.0/24, 198.53.150.0/24, 198.53.18.0/24, 204.191.114.0/24, 204.191.183.0/24, 198.53.200.0/24, 198.53.179.0/24, 204.191.79.0/24, 205.206.23.0/24, 198.53.153.0/24, 198.53.146.0/24, 204.191.101.0/24, 204.191.51.0/24, 198.53.234.0/24, 204.191.8.0/24, 204.191.94.0/24, 204.191.42.0/24, 204.191.33.0/24, 198.53.28.0/24, 204.191.67.0/24, 204.191.184.0/24, 198.53.254.0/24, 204.191.124.0/24, 198.53.17.0/24, 198.53.23.0/24, 198.53.68.0/24, 204.191.173.0/24, 198.53.215.0/24, 204.191.100.0/24, 204.191.186.0/24, 204.191.162.0/24, 198.53.214.0/24, 204.191.22.0/24, 204.191.221.0/24, 198.53.219.0/24, 204.191.6.0/24, 198.53.195.0/24, 198.53.198.0/24, 204.191.178.0/24, 198.53.231.0/24, 198.53.152.0/24, 204.191.52.0/24, 198.53.156.0/24, 204.191.43.0/24, 198.53.151.0/24, 204.191.34.0/24, 198.53.24.0/24, 204.191.75.0/24, 204.191.86.0/24, 204.191.117.0/24, 204.191.81.0/24, 204.191.2.0/24, 198.53.218.0/24, 204.191.72.0/24, 205.206.22.0/24, 204.191.19.0/24, 204.191.77.0/24, 204.191.228.0/24, 198.53.175.0/24, 204.191.180.0/24, 204.191.121.0/24, 198.53.49.0/24, 204.191.172.0/24, 204.191.222.0/24, 204.191.237.0/24, 198.53.208.0/24, 204.191.252.0/24, 204.191.84.0/24, 198.53.34.0/24, 204.191.69.0/24, 204.191.106.0/24, 198.53.210.0/24, 204.191.1.0/24, 198.53.188.0/24, 204.191.229.0/24, 204.191.96.0/24, 204.191.116.0/24, 198.53.233.0/24, 198.53.48.0/24, 198.53.190.0/24, 198.53.232.0/24, 198.53.237.0/24, 204.191.251.0/24, 204.191.47.0/24, 204.191.35.0/24, 204.191.38.0/24, 204.191.44.0/24, 204.191.194.0/24, 204.191.204.0/24, 198.53.212.0/24, 204.191.99.0/24, 198.53.166.0/24, 204.191.53.0/24, 204.191.56.0/24, 204.191.83.0/24, 204.191.80.0/24, 204.191.21.0/24, 204.191.48.0/24, 204.191.39.0/24, 198.53.161.0/24, 204.191.243.0/24, 198.53.180.0/24, 204.191.57.0/24, 198.53.160.0/24, 204.191.195.0/24, 198.53.165.0/24, 204.191.59.0/24, 204.191.36.0/24, 204.191.45.0/24, 204.191.54.0/24, 198.53.157.0/24, 204.191.71.0/24, 204.191.123.0/24, 204.191.74.0/24, 204.191.193.0/24, 204.191.202.0/24, 198.53.222.0/24, 204.191.11.0/24, 204.191.242.0/24, 204.191.220.0/24, 204.191.31.0/24, 204.191.40.0/24, 198.53.238.0/24, 204.191.164.0/24, 204.191.192.0/24, 198.53.225.0/24, 198.53.174.0/24, 198.53.145.0/24, 204.191.64.0/24, 198.53.170.0/24, 204.191.109.0/24, 198.53.22.0/24, 204.191.70.0/24, 198.53.32.0/24, 204.191.91.0/24, 204.191.104.0/24, 204.191.169.0/24, 198.53.213.0/24, 204.191.181.0/24, 198.53.221.0/24, 198.53.47.0/24, 204.191.209.0/24, 198.53.147.0/24, 204.191.253.0/24, 204.191.92.0/24, 204.191.165.0/24, 204.191.241.0/24, 204.191.198.0/24, 204.191.62.0/24, 204.191.65.0/24, 198.53.182.0/24, 204.191.113.0/24, 198.53.216.0/24, 198.53.35.0/24, 204.191.107.0/24, 198.53.193.0/24, 204.191.55.0/24, 204.191.3.0/24, 204.191.46.0/24, 204.191.118.0/24, 204.191.37.0/24, 204.191.18.0/24, 204.191.201.0/24, 198.53.235.0/24, 204.191.197.0/24, 204.191.108.0/24, 198.53.29.0/24, 204.191.168.0/24, 198.53.187.0/24, 204.191.13.0/24, 204.191.87.0/24, 204.191.61.0/24, 204.191.23.0/24, 198.53.227.0/24, 204.191.166.0/24, 198.53.154.0/24, 204.191.236.0/24, 204.191.78.0/24, 198.53.21.0/24, 204.191.191.0/24, 204.191.170.0/24, 198.53.20.0/24, 198.53.206.0/24, 204.191.17.0/24, 204.191.196.0/24, 198.53.148.0/24, 198.53.205.0/24, 198.53.185.0/24, 205.206.19.0/24, 198.53.172.0/24, 204.191.199.0/24, 204.191.66.0/24, 204.191.93.0/24, 198.53.189.0/24, 198.53.186.0/24, 204.191.250.0/24, 204.191.73.0/24, 204.191.167.0/24, 204.191.16.0/24, 204.191.200.0/24, 204.191.90.0/24, 204.191.115.0/24, 204.191.95.0/24, 204.191.171.0/24, 204.191.82.0/24, 204.191.68.0/24, 205.206.18.0/24, 198.53.192.0/24, 204.191.63.0/24, 198.53.163.0/24, 198.53.64.0/24, 205.206.64.0/20, 204.191.224.0/19, 205.206.48.0/20, 205.206.4.0/24, 205.206.32.0/20, 205.206.8.0/22, 204.191.226.0/24, 204.191.227.0/24, 204.191.230.0/24, 204.191.231.0/24, 204.191.234.0/24, 204.191.235.0/24, 205.206.8.0/24, 205.206.9.0/24, 205.206.10.0/24, 205.206.11.0/24, 205.206.5.0/24, 205.206.6.0/24, 205.206.12.0/24, 205.206.80.0/24, 205.206.81.0/24, 205.206.82.0/24, 205.206.96.0/24, 205.206.97.0/24, 205.206.98.0/24, 205.206.99.0/24, 205.206.89.0/24, 205.206.94.0/24, 205.206.88.0/24, 205.206.90.0/24, 205.206.91.0/24, 205.206.92.0/24, 205.206.93.0/24, 205.206.83.0/24, 205.206.84.0/24, 205.206.85.0/24, 205.206.1.0/24, 205.206.104.0/24, 205.206.105.0/24, 205.206.106.0/24, 205.206.107.0/24, 205.206.108.0/24, 205.206.109.0/24, 205.206.110.0/24, 205.206.111.0/24, 205.206.112.0/24, 205.206.113.0/24, 205.206.114.0/24, 205.206.115.0/24, 205.206.102.0/24, 205.206.103.0/24, 205.206.116.0/24, 205.206.120.0/24, 205.206.121.0/24, 205.206.117.0/24, 205.206.124.0/24, 205.206.125.0/24, 205.206.126.0/24, 205.206.127.0/24, 205.206.122.0/24, 205.206.128.0/24, 205.206.24.0/24, 205.206.132.0/24, 205.206.133.0/24, 205.206.134.0/24, 205.206.135.0/24, 205.206.129.0/24, 205.206.136.0/24, 205.206.137.0/24, 205.206.131.0/24, 205.206.146.0/24, 205.206.123.0/24, 205.206.145.0/24, 205.206.144.0/24, 205.206.143.0/24, 205.206.142.0/24, 205.206.138.0/24, 205.206.139.0/24, 205.206.140.0/24, 205.206.141.0/24, 205.206.157.0/24, 198.53.92.0/24, 198.53.93.0/24, 198.53.94.0/24, 198.53.95.0/24, 205.206.151.0/24, 205.206.163.0/24, 205.206.164.0/24, 198.53.252.0/24, 205.206.159.0/24, 205.206.160.0/24, 205.206.158.0/24, 205.206.165.0/24, 205.206.177.0/24, 205.206.173.0/24, 205.206.176.0/24, 205.206.174.0/24, 205.206.175.0/24, 205.206.167.0/24, 205.206.166.0/24, 205.206.172.0/24, 205.206.171.0/24, 205.206.170.0/24, 205.206.169.0/24, 205.206.168.0/24, 205.206.162.0/24, 205.206.147.0/24, 205.206.180.0/24, 205.206.154.0/24, 205.206.155.0/24, 205.206.152.0/24, 205.206.161.0/24, 205.206.183.0/24, 205.206.182.0/24, 205.206.181.0/24, 205.206.190.0/24, 205.206.191.0/24, 205.206.186.0/24, 205.206.149.0/24, 204.191.24.0/24, 205.206.187.0/24, 205.206.184.0/24, 205.206.95.0/24, 205.206.118.0/24, 205.206.119.0/24, 205.206.189.0/24, 205.206.199.0/24, 205.206.196.0/24, 205.206.198.0/24, 205.206.200.0/24, 205.206.197.0/24, 205.206.202.0/24, 205.206.203.0/24, 205.206.201.0/24, 205.206.188.0/24, 205.206.207.0/24, 205.206.206.0/24, 205.206.213.0/24, 205.206.214.0/24, 205.206.215.0/24, 205.206.219.0/24, 205.206.216.0/24, 205.206.217.0/24, 205.206.205.0/24, 205.206.222.0/24, 205.206.212.0/24, 205.206.228.0/24, 205.206.208.0/24, 205.206.209.0/24, 205.206.210.0/24, 205.206.211.0/24, 205.206.223.0/24, 205.206.226.0/24, 205.206.225.0/24, 205.206.227.0/24, 205.206.229.0/24, 205.206.244.0/24, 205.206.246.0/24, 205.206.247.0/24, 205.206.248.0/24, 205.206.249.0/24, 205.206.250.0/24, 205.206.251.0/24, 205.206.240.0/24, 206.116.6.0/24, 198.53.46.0/24, 198.53.82.0/24, 198.53.44.0/24, 205.206.232.0/24, 206.116.1.0/24, 206.116.2.0/24, 206.116.3.0/24, 206.116.4.0/24, 206.116.5.0/24, 205.206.241.0/24, 205.206.242.0/24, 205.206.243.0/24, 205.206.230.0/24, 205.206.231.0/24, 205.206.233.0/24, 205.206.234.0/24, 205.206.235.0/24, 205.206.236.0/24, 205.206.237.0/24, 205.206.238.0/24, 205.206.239.0/24, 206.116.11.0/24, 206.116.10.0/24, 205.206.25.0/24, 205.206.26.0/24, 205.206.27.0/24, 205.206.28.0/24, 205.206.29.0/24, 205.206.30.0/24, 205.206.31.0/24, 206.116.17.0/24, 206.116.15.0/24, 206.116.16.0/24, 206.116.19.0/24, 206.116.8.0/24, 206.116.13.0/24, 206.116.7.0/24, 206.116.20.0/24, 206.116.14.0/24, 206.116.28.0/24, 206.116.27.0/24, 206.116.25.0/24, 206.116.26.0/24, 206.116.22.0/24, 206.116.23.0/24, 206.116.29.0/24, 206.116.31.0/24, 206.116.30.0/24, 204.191.216.0/24, 204.191.217.0/24, 206.116.48.0/24, 206.116.32.0/24, 205.206.218.0/24, 206.116.60.0/24, 206.116.65.0/24, 206.116.50.0/24, 205.206.192.0/24, 206.116.69.0/24, 206.116.44.0/24, 206.116.21.0/24, 206.116.56.0/24, 206.116.86.0/24, 206.116.87.0/24, 206.116.71.0/24, 206.116.73.0/24, 205.206.220.0/24, 205.206.221.0/24, 205.206.193.0/24, 206.116.47.0/24, 206.116.46.0/24, 206.116.45.0/24, 206.116.43.0/24, 206.116.39.0/24, 206.116.36.0/24, 206.116.33.0/24, 206.116.35.0/24, 206.116.63.0/24, 204.191.212.0/24, 204.191.213.0/24, 206.116.100.0/24, 206.116.12.0/24, 206.116.79.0/24, 206.116.249.0/24, 206.116.242.0/24, 206.116.234.0/24, 206.116.221.0/24, 206.116.232.0/24, 206.116.224.0/24, 206.116.225.0/24, 206.116.226.0/24, 206.116.227.0/24, 206.116.228.0/24, 206.116.243.0/24, 206.116.239.0/24, 206.116.215.0/24, 206.116.212.0/24, 206.116.55.0/24, 198.53.251.0/24, 206.116.254.0/24, 206.116.90.0/24, 206.116.97.0/24, 198.53.63.0/24, 206.116.92.0/24, 204.191.144.0/22, 204.191.148.0/22, 204.191.136.0/21, 204.191.152.0/22, 206.116.210.0/24, 206.116.211.0/24, 206.116.112.0/24, 206.116.132.0/24, 206.116.133.0/24, 204.191.128.0/24, 204.191.132.0/24, 198.53.248.0/24, 198.53.249.0/24, 198.53.250.0/24, 198.53.60.0/24, 206.116.116.0/24, 206.116.70.0/24, 206.116.103.0/24, 206.116.113.0/24, 206.116.125.0/24, 206.116.150.0/24, 206.116.130.0/24, 206.116.9.0/24, 206.116.169.0/24, 206.116.172.0/24, 206.116.167.0/24, 206.116.168.0/24, 198.53.128.0/24, 206.116.173.0/24, 206.116.161.0/24, 206.116.162.0/24, 206.116.163.0/24, 206.116.121.0/24, 206.116.146.0/24, 206.116.244.0/24, 198.53.114.0/24, 206.116.192.0/24, 206.116.214.0/24, 205.206.7.0/24, 206.116.196.0/24, 207.6.1.0/24, 207.6.5.0/24, 207.6.24.0/24, 207.6.31.0/24, 207.6.27.0/24, 206.116.62.0/24, 207.6.112.0/24, 207.6.88.0/24, 207.6.63.0/24, 207.6.85.0/24, 207.6.121.0/24, 207.6.89.0/24, 207.6.90.0/24, 207.6.86.0/24, 207.6.149.0/24, 207.6.153.0/24, 207.6.165.0/24, 207.6.166.0/24, 207.6.178.0/24, 205.206.14.0/24, 207.6.163.0/24, 204.191.120.0/24, 206.116.42.0/24, 206.116.106.0/24, 207.81.128.0/17, 207.6.91.0/24, 207.6.202.0/24, 204.191.192.0/23, 204.191.180.0/22, 204.191.184.0/22, 198.53.120.0/21, 198.53.0.0/20, 205.206.252.0/24, 206.116.208.0/24, 206.116.107.0/24, 206.116.148.0/24, 207.6.147.0/24, 207.6.250.0/24, 207.6.251.0/24, 207.6.252.0/24, 207.6.253.0/24, 207.6.254.0/24, 207.81.128.0/19, 198.53.202.0/24, 206.116.124.0/24, 207.134.128.0/17, 207.81.160.0/20, 198.53.16.0/21, 198.53.24.0/23, 198.53.80.0/20, 198.53.172.0/22, 198.53.176.0/21, 198.53.240.0/21, 204.191.0.0/17, 204.191.0.0/23, 204.191.144.0/21, 204.191.192.0/22, 204.191.216.0/22, 205.206.160.0/22, 205.206.192.0/21, 205.206.226.0/23, 205.206.248.0/21, 206.116.12.0/23, 206.116.24.0/23, 206.116.32.0/22, 206.116.46.0/23, 206.116.50.0/23, 206.116.56.0/21, 206.116.56.0/23, 206.116.74.0/23, 206.116.90.0/23, 206.116.106.0/23, 206.116.118.0/23, 206.116.120.0/24, 206.116.126.0/23, 206.116.148.0/23, 206.116.160.0/22, 206.116.160.0/24, 206.116.188.0/22, 206.116.208.0/23, 206.116.232.0/22, 206.116.235.0/24, 206.116.237.0/24, 207.6.7.0/24, 207.6.8.0/21, 207.6.19.0/24, 207.6.30.0/23, 207.6.32.0/20, 207.6.32.0/22, 207.6.47.0/24, 207.6.59.0/24, 207.6.61.0/24, 207.6.64.0/24, 207.6.72.0/22, 207.6.88.0/22, 207.6.112.0/23, 207.6.120.0/23, 207.6.128.0/20, 207.6.146.0/23, 207.6.152.0/22, 207.6.160.0/21, 207.6.167.0/24, 207.6.171.0/24, 207.6.172.0/23, 207.6.198.0/23, 207.6.248.0/21, 207.81.192.0/18})
Curtis, Sean, Since these networks presumable are being announced as more specific by customers of fONOROLA/iStar.net, wouldn't it be best to try and get someone who has an interest in making it work right to do the work, stead of kicking the backbone provider? Since the prefix originator is the one who's routing is breaking, and if they can just register in the RADB to fix things, I don't really see a problem. Personally, I'd like to see this type of aggregation working more often. This seems to be the best way to get reasonable aggregation out of the "SWAMP" and the rest of the ~20,000 legacy addresses. This combined with prefix length filtering, to help "encourage" people to get address space from their providers, will greatly reduce the total number of prefixes in the IPv4, "end state" routing table. (I suspect, but don't have data to prove (YET), that the best we will be able to do with the ~20,000 is reduce it to ~10k to ~15k.). If your routers can hold then hold about 100,000 prefixes at ~4 paths per prefix, you should be pretty well off for the next year or so. I personally think that allowing /19s in 208-223 is a little too generous with router ram. (And there is the issue of what happens to 64k Cisco SSP's when forwarding tables exceed 64k)... Of course if you are using a different routing vendor, you will see slightly different resource limits.
In message <960606181706.9ede@SDG.DRA.COM>, Sean Donelan writes:
Does anyone know what's going on in Canada?
In the last couple of weeks fONOROLA/iStar.net changed how they announce aggregated networks. Now fONOROLA is sending some more specific network announcements for multi-homed networks in 198.53.0.0 with a "no-export" BGP community to MCI, and ANS.
As long as you only connect with MCI or ANS, and use their IGP, things look pretty typical because the more specifics are carried internally. But if you use BGP, the no-export means any peers with MCI won't see the more specific network announcement from MCI.
The problem shows up when you see routes from Sprintlink. A few Canadian network connect through Sprint/Canada as well as fONOROLA. The more specific networks from 198.53 are announced by Sprintlink. Since the no-export suppressed the more specific announcement to MCI BGP peers, the traffic for those Canadian sites follows the more specific network announcement and heads for Sprint.
I understand the BGP mechanics are working as specified, so it isn't "broken" per se. But I'm just wondering if Canadian sites really expect traffic to go via Stockton, California?
I haven't been able to reach anyone at fONOROLA/Istar. I was just wondering if any other North American network operators had heard what was supposed to happen last month when fONOROLA made these changes. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation
Sean,
I can't speak for Istar, but I do know that they had tried to make this a painless aggregation by indicating to both ANS and MCI which prefixes were in their aggregates that did not need to be advertised outside of ANS and MCI. Istar saved some 750 prefixes. I think this is roughly half what they had previously been announcing and they have over 3500 prefixes registered, indicating a lot were already site aggregated. This is certainly commendable.
Unfortunately, since there is no way for them to know which Istar prefixes also have connectivity through Sprint or someone else (with Sprint failing to register this in the IRR), there is no way that Istar can determine that those component have to be passed on by ANS and MCI except to aggregate and see what breaks. This is the type of "information exchange" and provider cooperation through the IRR that we have been talking about that isn't universally accepted.
As this type of non-trivial aggregation becomes more common, we can expect this sort of thing to be a regular occurance unless certain other providers take the time to register things. Yes, we know it is work, but nothing near the work Istar did to get these prefixes aggregated with as little disruption as possible.
The more specific prefixes blocked by ANS and MCI are listed below. We can determine which are heard from Sprint with a routing dump and unblock them. This is the "break it, see what broke, then fix it" method, but the end result will work.
Cheers,
Curtis
-- Jeremy Porter, Freeside Communications, Inc. jerry@fc.net PO BOX 80315 Austin, Tx 78708 | 1-800-968-8750 | 512-339-6094 http://www.fc.net
Sounds to me like Policy Based Routing is starting to evolve in a particularly interesting way. -alan ......... Sean Donelan is rumored to have said: ] ] Does anyone know what's going on in Canada? ] ] In the last couple of weeks fONOROLA/iStar.net changed how they ] announce aggregated networks. Now fONOROLA is sending some more ] specific network announcements for multi-homed networks in 198.53.0.0 ] with a "no-export" BGP community to MCI, and ANS. ] ] As long as you only connect with MCI or ANS, and use their IGP, ] things look pretty typical because the more specifics are carried ] internally. But if you use BGP, the no-export means any peers ] with MCI won't see the more specific network announcement from MCI. ] ] The problem shows up when you see routes from Sprintlink. A few ] Canadian network connect through Sprint/Canada as well as fONOROLA. ] The more specific networks from 198.53 are announced by Sprintlink. ] Since the no-export suppressed the more specific announcement to ] MCI BGP peers, the traffic for those Canadian sites follows the ] more specific network announcement and heads for Sprint. ] ] I understand the BGP mechanics are working as specified, so it ] isn't "broken" per se. But I'm just wondering if Canadian sites ] really expect traffic to go via Stockton, California? ] ] I haven't been able to reach anyone at fONOROLA/Istar. I was just ] wondering if any other North American network operators had heard ] what was supposed to happen last month when fONOROLA made these ] changes. ] -- ] Sean Donelan, Data Research Associates, Inc, St. Louis, MO ] Affiliation given for identification not representation ] ] ]
Sean, A long time ago, you wrote in this forum about our use of BGP no-export.
Does anyone know what's going on in Canada?
Yes, we do. The more specific routes of 198.53.0.0 are doing exactly what is intended, which is routing via Sprint. Some customers, which were sadly allocated from our 198.53.0.0 aggregate, and are, well, still working on returning the space, are also connected to Sprint. We have the aggregate, they are dual connected routes which we still have connectivity to, Sprint carries the more specifics and advertises them because the customer is advertising the more specifics to Sprint. Nothing iSTAR can do about this... We have withdrawn alot of more specifics recently (around a thousand, I think) and are now working on more aggregation. This is a good thing. But we still need to surpress the downstream specifics that we use for MEDs and getting our east/west loading correct. Thus the BGP no-export on things we aggregate. In fact, with any non-portable aggregate we have, we have longer prefixes exported no-export to ensure that the MEDs work correctly and the MED-ed longer prefixes are not passed downstream. I do not ever recall hearing about you trying to contact us on this issue. noc@istar.ca works as an inter-ISP contact point. Please write me privately if there is an issue here for you - I would be fascinated in your findings on dually announced (and unregistered) routes or longer prefixes. Its very hard these days to track who is punching holes in your aggregates, and registering them in multiple ASes. Still catching up on old email, Eric M. Carroll iSTAR internet Inc. Voice: (613) 780-2287 250 Albert St, Ste 202 Fax: (613) 780-6666 Ottawa ON K1P 6M1 Corporate Support: (888) 478 2778 Subscriber Support: (888) GO ISTAR
participants (5)
-
Alan Hannan -
Curtis Villamizar -
Eric M. Carroll -
Jeremy Porter -
Sean Donelan