Hey folks, Looking at an mtr going out via a couple of different transit circuits, Google seems to be doing weird things. RTT pinging google.com is coming up with like 250-300ms times, but mtr's are telling me my packets are hitting google's network very quickly. Google's network then seems to send them on a rather long trip before reaching the google.com frontend servers. An example: 6 213.200.123.170 (213.200.123.170) 3.450 ms ae-1-3502.ear4.Newark1.Level3.net (4.69.211.177) 1.469 ms 1.634 ms 7 72.14.213.34 (72.14.213.34) 1.336 ms 1.372 ms 1.381 ms 8 108.170.248.52 (108.170.248.52) 2.474 ms * 2.150 ms 9 216.239.62.170 (216.239.62.170) 1.401 ms 216.239.62.150 (216.239.62.150) 1.400 ms 216.239.62.168 (216.239.62.168) 2.985 ms 10 216.239.57.136 (216.239.57.136) 20.043 ms 216.239.59.0 (216.239.59.0) 20.235 ms 216.239.57.196 (216.239.57.196) 20.382 ms 11 209.85.254.241 (209.85.254.241) 2.155 ms 108.170.235.61 (108.170.235.61) 74.295 ms 209.85.241.43 (209.85.241.43) 78.593 ms 12 72.14.239.155 (72.14.239.155) 96.254 ms 216.239.57.196 (216.239.57.196) 19.672 ms 72.14.239.155 (72.14.239.155) 96.328 ms 13 108.170.235.217 (108.170.235.217) 153.391 ms 108.170.236.119 (108.170.236.119) 153.445 ms 108.170.235.221 (108.170.235.221) 152.858 ms 14 172.253.51.111 (172.253.51.111) 220.084 ms 66.249.94.141 (66.249.94.141) 218.039 ms 72.14.239.197 (72.14.239.197) 75.008 ms 15 209.85.241.86 (209.85.241.86) 276.281 ms 72.14.235.160 (72.14.235.160) 276.104 ms 277.497 ms 16 108.170.235.105 (108.170.235.105) 217.030 ms 209.85.248.4 (209.85.248.4) 217.338 ms 66.249.94.141 (66.249.94.141) 217.573 ms 17 72.14.236.75 (72.14.236.75) 276.349 ms 276.097 ms 72.14.239.235 (72.14.239.235) 277.180 ms 18 bom07s01-in-f14.1e100.net (216.58.199.142) 276.139 ms 276.980 ms 64.233.174.27 (64.233.174.27) 279.212 ms As you can see from this traceroute output, Level3 is delivering my packets to Google (hop#7 and beyond) just fine, however all of the hops including #7 and beyond are all inside of google's network. My packets are originating from AS 394102. Anyone from google have any idea what's going on there? Thanks, Matt
I would say that it says BOM at the start of the name, perhaps they are sending you to India? Are you using a DNS service that uses ECS facing the various CDN/Cloud providers or a different one? - Jared
On May 23, 2019, at 3:47 PM, Matt Harris <matt@netfire.net> wrote:
Hey folks, Looking at an mtr going out via a couple of different transit circuits, Google seems to be doing weird things.
RTT pinging google.com is coming up with like 250-300ms times, but mtr's are telling me my packets are hitting google's network very quickly. Google's network then seems to send them on a rather long trip before reaching the google.com frontend servers.
An example:
6 213.200.123.170 (213.200.123.170) 3.450 ms ae-1-3502.ear4.Newark1.Level3.net (4.69.211.177) 1.469 ms 1.634 ms 7 72.14.213.34 (72.14.213.34) 1.336 ms 1.372 ms 1.381 ms 8 108.170.248.52 (108.170.248.52) 2.474 ms * 2.150 ms 9 216.239.62.170 (216.239.62.170) 1.401 ms 216.239.62.150 (216.239.62.150) 1.400 ms 216.239.62.168 (216.239.62.168) 2.985 ms 10 216.239.57.136 (216.239.57.136) 20.043 ms 216.239.59.0 (216.239.59.0) 20.235 ms 216.239.57.196 (216.239.57.196) 20.382 ms 11 209.85.254.241 (209.85.254.241) 2.155 ms 108.170.235.61 (108.170.235.61) 74.295 ms 209.85.241.43 (209.85.241.43) 78.593 ms 12 72.14.239.155 (72.14.239.155) 96.254 ms 216.239.57.196 (216.239.57.196) 19.672 ms 72.14.239.155 (72.14.239.155) 96.328 ms 13 108.170.235.217 (108.170.235.217) 153.391 ms 108.170.236.119 (108.170.236.119) 153.445 ms 108.170.235.221 (108.170.235.221) 152.858 ms 14 172.253.51.111 (172.253.51.111) 220.084 ms 66.249.94.141 (66.249.94.141) 218.039 ms 72.14.239.197 (72.14.239.197) 75.008 ms 15 209.85.241.86 (209.85.241.86) 276.281 ms 72.14.235.160 (72.14.235.160) 276.104 ms 277.497 ms 16 108.170.235.105 (108.170.235.105) 217.030 ms 209.85.248.4 (209.85.248.4) 217.338 ms 66.249.94.141 (66.249.94.141) 217.573 ms 17 72.14.236.75 (72.14.236.75) 276.349 ms 276.097 ms 72.14.239.235 (72.14.239.235) 277.180 ms 18 bom07s01-in-f14.1e100.net (216.58.199.142) 276.139 ms 276.980 ms 64.233.174.27 (64.233.174.27) 279.212 ms
As you can see from this traceroute output, Level3 is delivering my packets to Google (hop#7 and beyond) just fine, however all of the hops including #7 and beyond are all inside of google's network.
My packets are originating from AS 394102.
Anyone from google have any idea what's going on there?
Thanks, Matt
not sure where you are starting from (really) .. can you provide a: dig www.google.com for me? My guess is that as Jared noted you got somehow looking like you are in india to whatever does that magic :) On Thu, May 23, 2019 at 3:48 PM Matt Harris <matt@netfire.net> wrote:
Hey folks, Looking at an mtr going out via a couple of different transit circuits, Google seems to be doing weird things.
RTT pinging google.com is coming up with like 250-300ms times, but mtr's are telling me my packets are hitting google's network very quickly. Google's network then seems to send them on a rather long trip before reaching the google.com frontend servers.
An example:
6 213.200.123.170 (213.200.123.170) 3.450 ms ae-1-3502.ear4.Newark1.Level3.net (4.69.211.177) 1.469 ms 1.634 ms 7 72.14.213.34 (72.14.213.34) 1.336 ms 1.372 ms 1.381 ms 8 108.170.248.52 (108.170.248.52) 2.474 ms * 2.150 ms 9 216.239.62.170 (216.239.62.170) 1.401 ms 216.239.62.150 (216.239.62.150) 1.400 ms 216.239.62.168 (216.239.62.168) 2.985 ms 10 216.239.57.136 (216.239.57.136) 20.043 ms 216.239.59.0 (216.239.59.0) 20.235 ms 216.239.57.196 (216.239.57.196) 20.382 ms 11 209.85.254.241 (209.85.254.241) 2.155 ms 108.170.235.61 (108.170.235.61) 74.295 ms 209.85.241.43 (209.85.241.43) 78.593 ms 12 72.14.239.155 (72.14.239.155) 96.254 ms 216.239.57.196 (216.239.57.196) 19.672 ms 72.14.239.155 (72.14.239.155) 96.328 ms 13 108.170.235.217 (108.170.235.217) 153.391 ms 108.170.236.119 (108.170.236.119) 153.445 ms 108.170.235.221 (108.170.235.221) 152.858 ms 14 172.253.51.111 (172.253.51.111) 220.084 ms 66.249.94.141 (66.249.94.141) 218.039 ms 72.14.239.197 (72.14.239.197) 75.008 ms 15 209.85.241.86 (209.85.241.86) 276.281 ms 72.14.235.160 (72.14.235.160) 276.104 ms 277.497 ms 16 108.170.235.105 (108.170.235.105) 217.030 ms 209.85.248.4 (209.85.248.4) 217.338 ms 66.249.94.141 (66.249.94.141) 217.573 ms 17 72.14.236.75 (72.14.236.75) 276.349 ms 276.097 ms 72.14.239.235 (72.14.239.235) 277.180 ms 18 bom07s01-in-f14.1e100.net (216.58.199.142) 276.139 ms 276.980 ms 64.233.174.27 (64.233.174.27) 279.212 ms
As you can see from this traceroute output, Level3 is delivering my packets to Google (hop#7 and beyond) just fine, however all of the hops including #7 and beyond are all inside of google's network.
My packets are originating from AS 394102.
Anyone from google have any idea what's going on there?
Thanks, Matt
On Thu, May 23, 2019 at 2:55 PM Jared Mauch <jared@puck.nether.net> wrote:
I would say that it says BOM at the start of the name, perhaps they are sending you to India?
Are you using a DNS service that uses ECS facing the various CDN/Cloud providers or a different one?
This is my thinking, too, however my recursive DNS servers are all on the same network as the systems trying to reach google, all of which are on IP space that I own and announced exclusively by AS 394102 here in the US. I've also taken care to maintain as many geoip service entries as could be found/maintained, including maxmind's. Where they would get the idea that my packets should go to India is beyond me. On Thu, May 23, 2019 at 3:06 PM Christopher Morrow <morrowc.lists@gmail.com> wrote:
not sure where you are starting from (really) .. can you provide a: dig www.google.com
for me? My guess is that as Jared noted you got somehow looking like you are in india to whatever does that magic :)
Google's coming back with bom* addresses; no idea why though. ;; ANSWER SECTION: www.google.com. 300 IN A 172.217.26.228 Hoping someone over there can shed some light on why they are sending my packets on a world trip. :) Thanks, Matt
Google maintains their own GeoIP database. If you peer with them and have access to the peering portal, you can correct the location yourself. Otherwise they have a public form somewhere. --- Filip On 23 May 2019 10:11:30 pm GMT+02:00, Matt Harris <matt@netfire.net> wrote:
On Thu, May 23, 2019 at 2:55 PM Jared Mauch <jared@puck.nether.net> wrote:
I would say that it says BOM at the start of the name, perhaps they are sending you to India?
Are you using a DNS service that uses ECS facing the various CDN/Cloud providers or a different one?
This is my thinking, too, however my recursive DNS servers are all on the same network as the systems trying to reach google, all of which are on IP space that I own and announced exclusively by AS 394102 here in the US. I've also taken care to maintain as many geoip service entries as could be found/maintained, including maxmind's. Where they would get the idea that my packets should go to India is beyond me.
On Thu, May 23, 2019 at 3:06 PM Christopher Morrow <morrowc.lists@gmail.com> wrote:
not sure where you are starting from (really) .. can you provide a: dig www.google.com
for me? My guess is that as Jared noted you got somehow looking like you are in india to whatever does that magic :)
Google's coming back with bom* addresses; no idea why though.
;; ANSWER SECTION: www.google.com. 300 IN A 172.217.26.228
Hoping someone over there can shed some light on why they are sending my packets on a world trip. :)
Thanks, Matt
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On Thu, May 23, 2019 at 3:24 PM Filip Hruska <fhr@fhrnet.eu> wrote:
Google maintains their own GeoIP database. If you peer with them and have access to the peering portal, you can correct the location yourself. Otherwise they have a public form somewhere.
--- Filip
Googling around a bit does not yield results for that form... any chance anyone here has a link to that? Would be much appreciated! Thanks, Matt
https://support.google.com/websearch/contact/ip/ Am 23.05.2019 um 22:55 schrieb Matt Harris:
On Thu, May 23, 2019 at 3:24 PM Filip Hruska <fhr@fhrnet.eu <mailto:fhr@fhrnet.eu>> wrote:
Google maintains their own GeoIP database. If you peer with them and have access to the peering portal, you can correct the location yourself. Otherwise they have a public form somewhere.
--- Filip
Googling around a bit does not yield results for that form... any chance anyone here has a link to that? Would be much appreciated!
Thanks, Matt
On Thu, May 23, 2019 at 4:01 PM Patrick Schultz <lists-nanog@schultz.top> wrote:
Thanks! Giving that a shot. It's still loading www.google.com though if I try to hit it in a browser (not redirecting to a different language/CCTLD specific site though) so I had to put that in along with that I'm in the US, not sure that whoever sees that form will understand my issue and there's no freeform comments section to mention "but it's loading from India!"
Yeah, that's honestly a pretty crappy form. No room for an explanation, no individual contact, and an ETR of a month. I'm surprised there's not a better way to address issues like this On Thu, May 23, 2019, 5:13 PM Matt Harris <matt@netfire.net> wrote:
On Thu, May 23, 2019 at 4:01 PM Patrick Schultz <lists-nanog@schultz.top> wrote:
Thanks!
Giving that a shot. It's still loading www.google.com though if I try to hit it in a browser (not redirecting to a different language/CCTLD specific site though) so I had to put that in along with that I'm in the US, not sure that whoever sees that form will understand my issue and there's no freeform comments section to mention "but it's loading from India!"
Seems to be more end-user oriented rather than targeted at netadmins. There's no real contact to the GeoIP team besides the peering portal and that form, except maybe the NOC. (at least none I found yet) Am 23.05.2019 um 23:23 schrieb Ross Tajvar:
Yeah, that's honestly a pretty crappy form. No room for an explanation, no individual contact, and an ETR of a month. I'm surprised there's not a better way to address issues like this
On Thu, May 23, 2019, 5:13 PM Matt Harris <matt@netfire.net <mailto:matt@netfire.net>> wrote:
On Thu, May 23, 2019 at 4:01 PM Patrick Schultz <lists-nanog@schultz.top> wrote:
https://support.google.com/websearch/contact/ip/
Thanks!
Giving that a shot. It's still loading www.google.com <http://www.google.com> though if I try to hit it in a browser (not redirecting to a different language/CCTLD specific site though) so I had to put that in along with that I'm in the US, not sure that whoever sees that form will understand my issue and there's no freeform comments section to mention "but it's loading from India!"
Same thought here, yet I tried to report a wrong GeoIP subnet for my AS multiple times on that form. Never got feedback nor did they made any correction. On 5/24/19 5:27 AM, Patrick Schultz wrote:
Seems to be more end-user oriented rather than targeted at netadmins. There's no real contact to the GeoIP team besides the peering portal and that form, except maybe the NOC. (at least none I found yet)
Am 23.05.2019 um 23:23 schrieb Ross Tajvar:
Yeah, that's honestly a pretty crappy form. No room for an explanation, no individual contact, and an ETR of a month. I'm surprised there's not a better way to address issues like this
On Thu, May 23, 2019, 5:13 PM Matt Harris <matt@netfire.net <mailto:matt@netfire.net>> wrote:
On Thu, May 23, 2019 at 4:01 PM Patrick Schultz <lists-nanog@schultz.top> wrote:
https://support.google.com/websearch/contact/ip/
Thanks!
Giving that a shot. It's still loading www.google.com <http://www.google.com> though if I try to hit it in a browser (not redirecting to a different language/CCTLD specific site though) so I had to put that in along with that I'm in the US, not sure that whoever sees that form will understand my issue and there's no freeform comments section to mention "but it's loading from India!"
On May 23, 2019, at 4:11 PM, Matt Harris <matt@netfire.net> wrote:
On Thu, May 23, 2019 at 2:55 PM Jared Mauch <jared@puck.nether.net> wrote: I would say that it says BOM at the start of the name, perhaps they are sending you to India?
Are you using a DNS service that uses ECS facing the various CDN/Cloud providers or a different one?
This is my thinking, too, however my recursive DNS servers are all on the same network as the systems trying to reach google, all of which are on IP space that I own and announced exclusively by AS 394102 here in the US. I've also taken care to maintain as many geoip service entries as could be found/maintained, including maxmind's. Where they would get the idea that my packets should go to India is beyond me.
On Thu, May 23, 2019 at 3:06 PM Christopher Morrow <morrowc.lists@gmail.com> wrote: not sure where you are starting from (really) .. can you provide a: dig www.google.com
for me? My guess is that as Jared noted you got somehow looking like you are in india to whatever does that magic :)
Google's coming back with bom* addresses; no idea why though.
;; ANSWER SECTION: www.google.com. 300 IN A 172.217.26.228
Hoping someone over there can shed some light on why they are sending my packets on a world trip. :)
If you send the query to 8.8.8.8 do you get a more favorable response (just curious). You can also run this query: dig TXT whoami.ds.akahelp.net. Which may assist. - jared
On Thu, May 23, 2019 at 4:11 PM Matt Harris <matt@netfire.net> wrote:
On Thu, May 23, 2019 at 3:06 PM Christopher Morrow <morrowc.lists@gmail.com> wrote:
not sure where you are starting from (really) .. can you provide a: dig www.google.com
for me? My guess is that as Jared noted you got somehow looking like you are in india to whatever does that magic :)
Google's coming back with bom* addresses; no idea why though.
;; ANSWER SECTION: www.google.com. 300 IN A 172.217.26.228
that's an ip in india alright :) I don't see why that's happening (in quick searching).
Hoping someone over there can shed some light on why they are sending my packets on a world trip. :)
I'd be cuirous about: dig www.google.com @8.8.8.8 as well, please (jared's question as well)
On Thu, May 23, 2019 at 3:44 PM Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Thu, May 23, 2019 at 4:11 PM Matt Harris <matt@netfire.net> wrote:
On Thu, May 23, 2019 at 3:06 PM Christopher Morrow < morrowc.lists@gmail.com> wrote:
not sure where you are starting from (really) .. can you provide a: dig www.google.com
for me? My guess is that as Jared noted you got somehow looking like you are in india to whatever does that magic :)
Google's coming back with bom* addresses; no idea why though.
;; ANSWER SECTION: www.google.com. 300 IN A 172.217.26.228
that's an ip in india alright :) I don't see why that's happening (in quick searching).
Hoping someone over there can shed some light on why they are sending my
packets on a world trip. :)
I'd be cuirous about: dig www.google.com @8.8.8.8
as well, please (jared's question as well)
Interestingly... user@host # dig www.google.com @8.8.8.8 ; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> www.google.com @8.8.8.8 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2110 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.google.com. IN A ;; ANSWER SECTION: www.google.com. 299 IN A 216.58.203.164 ;; Query time: 16 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Thu May 23 16:55:04 EDT 2019 ;; MSG SIZE rcvd: 59 user@host # host 216.58.203.164 164.203.58.216.in-addr.arpa domain name pointer bom07s11-in-f4.1e100.net. Still comes back with a bom* host, so it looks like it's not based on the DNS recursion server used.
participants (7)
-
Christopher Morrow -
Filip Hruska -
Jared Mauch -
Matt Harris -
Patrick Schultz -
Quan Zhou -
Ross Tajvar