I would expect Verizon to be able to contact CT and figure out why they aren't passing the correct routes just as I might expect Baidu to do the same thing, I suppose.

What defines 'correct'? ASN's routinely make traffic engineering or business decisions not to announce certain prefixes to certain other ASNs. This certainly does sometimes cause reachability issues for end users, but that's a choice someone made. That's part of why it's generally good practice to get more than 1 upstream if you can; it gives you more control to mitigate the impact of these choices.

It is completely possible that Baidu or CT are intentionally not announcing prefixes to VZ. It is also completely possible that they are and VZ is not accepting it.

On Thu, Jul 21, 2022 at 11:24 AM holow29 <holow29@gmail.com> wrote:

I would expect Verizon to be able to contact CT and figure out why they aren't passing the correct routes just as I might expect Baidu to do the same thing, I suppose. Ultimately, whose responsibility is it other than CT? That is my question. Maybe in this instance, it is common in the industry for this to be the responsibility of Baidu. That seems counterintuitive to me as it is Verizon without the proper route ultimately, but I am not an expert.
Certainly, I think it is incorrect to ask the customer to try to resolve these issues after bringing it to the attention of these services. If Verizon couldn't reach one of Google's edge servers, would it be Verizon or Google's responsibility to fix that if the issue were an intermediate peer failing to broadcase the correct route? I have the sneaking suspicion that Verizon might actually take some action in that instance...

On Thu, Jul 21, 2022 at 9:31 AM Tom Beecher <beecher@beecher.cc> wrote:
Unfortunately it seems like policy that Verizon pushes any issues that aren't internal routing issues to an external party, but surely they have a responsibility to maintain their peering and routes to external services as well.

Baidu is behind CT. If CT is not passing on routes to 701 , what exactly do you expect Verizon to do about it?

On Wed, Jul 20, 2022 at 6:40 PM holow29 <holow29@gmail.com> wrote:
To follow up on this:
I've engaged Verizon's executive office to finally try to get to a network engineer (because I don't have a contact myself). The (proxied) response from the engineer was that they aren't receiving any announcements for these routes to AS701, and I would need to take it up with Baidu. I guess I would like to understand if that seems reasonable to people since it is my presumption that Baidu would return and say something similar (that they advertise their routes to their peers correctly and to take it up with Verizon). To me, it seems like there is clearly a failing in one of Verizon's peers where they are not advertising or accepting this route correctly, but that it would be incumbent on Verizon to do the legwork to fix it since they are the ones who know their peering agreements and have these contacts. Unfortunately it seems like policy that Verizon pushes any issues that aren't internal routing issues to an external party, but surely they have a responsibility to maintain their peering and routes to external services as well. In other words, this type of buckpassing does not seem right to me (and I've heard it from them on other routing issues before), especially given that they are the ones empowered to fix it. Any thoughts?

(As it happens, pan.baidu.com now resolves to an IP range that is routable by Verizon, but it could always revert, and it seems like Verizon should have these routes regardless.)

Thanks.

On Fri, Jun 24, 2022 at 7:41 AM Matthew Huff <mhuff@ox.com> wrote:

From my limited vantage point it appears that there is some issue between Verizon & Baidu. Baidu has 182.61.0.0/16 registered, but is only advertising pieces of it globally (or at least from what I can see). In our tables,we are receiving none from Verizon of the subnets that are advertised directly from Baidu (origin AS of 38365). The few within that registered range that have a different origin AS are coming to us from Verizon. For example:

*>   182.61.0.0/19    144.121.203.141                        0 46887 3356 4134 58466 38365 i

*>   182.61.0.0/18    144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

*>   182.61.32.0/19   144.121.203.141                        0 46887 3356 4134 58466 38365 i

*>   182.61.64.0/18   204.148.121.221                        0 701 6453 55967 i

*    182.61.128.0/23 204.148.121.221                        0 701 4134 4134 4134 4134 4134 58540 ?

*>   182.61.130.0/24 144.121.203.141                        0 46887 6461 4134 23724 38365 38365 38365 i

*>   182.61.130.0/23 144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

*>   182.61.131.0/24 144.121.203.141                        0 46887 6461 4134 23724 38365 38365 38365 i

*>   182.61.132.0/23 144.121.203.141                        0 46887 3356 4134 58466 38365 i

*>   182.61.132.0/22 144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

*>   182.61.134.0/23 144.121.203.141                        0 46887 3356 4134 58466 38365 i

*>   182.61.136.0/22 144.121.203.141                        0 46887 3356 4134 58466 38365 i

*>   182.61.136.0/21 144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

*>   182.61.140.0/22 144.121.203.141                        0 46887 3356 4134 58466 38365 i

*>   182.61.144.0/21 144.121.203.141                        0 46887 3356 4134 58466 38365 i

*>   182.61.144.0/20 144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

*>   182.61.160.0/19 204.148.121.221                        0 701 6453 55967 i

*>   182.61.192.0/23 144.121.203.141                        0 46887 3356 4134 58540 i

*>   182.61.194.0/23 144.121.203.141                        0 46887 3356 4134 58540 i

*>   182.61.200.0/22 144.121.203.141                        0 46887 6461 4134 23724 38365 i

*>   182.61.200.0/21 144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

*>   182.61.216.0/21 144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

*>   182.61.223.0/24 144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

*>   182.61.224.0/19 144.121.203.141                        0 46887 6461 4134 58466 38365 38365 i

We are getting the 182.61.200.0/21 and 182.61.200.0/22 from all of our other peers:

asr-inet2#sh ip bgp 182.61.200.0/21

BGP routing table entry for 182.61.200.0/21, version 15779018

Paths: (2 available, best #2, table default)

Advertised to update-groups:

     3

  Refresh Epoch 1

54004 6128 1299 4134 58466 38365 38365, (aggregated by 38365 119.75.208.225)

    148.77.99.201 from 148.77.99.201 (24.157.4.25)

      Origin IGP, localpref 100, valid, external, atomic-aggregate

      rx pathid: 0, tx pathid: 0

      Updated on Apr 29 2022 21:02:05 EDT

Refresh Epoch 1

46887 6461 4134 58466 38365 38365, (aggregated by 38365 119.75.208.225)

    129.77.17.254 from 129.77.17.254 (129.77.40.7)

      Origin IGP, metric 0, localpref 100, valid, internal, atomic-aggregate, best

      rx pathid: 0, tx pathid: 0x0

      Updated on May 3 2022 04:02:50 EDT

From: NANOG <nanog-bounces+mhuff=ox.com@nanog.org> On Behalf Of holow29
Sent: Thursday, June 23, 2022 5:49 PM
To: nanog@nanog.org
Subject: Verizon no BGP route to some of AS38365 (182.61.200.0/24)

I've been trying (to no avail) for over a month now to get Verizon to investigate their lack of BGP routing to 182.61.200.0/24, which hosts Baidu Wangpan at pan.baidu.com (Baidu's cloud services/equivalent of Google Drive).

Easily verified through Verizon's Looking Glass.

We all know Verizon's BGP routing is a disaster, but does anyone have any ideas?