Hello, I'm seeing an oddity when doing DNS lookups for www.google.com from our London datacenter, and I'm curious if other people are seeing the same behavior. It appears that when we ask for www.google.com. we sometimes get an answer that only contains records for www-anycast.google.com., which our resolver ignores as they don't match the query. As seen with dig: ``` # dig @ns1.google.com. www.google.com. aaaa ; <<>> DiG 9.10.3-P4-Ubuntu <<>> @ns1.google.com. www.google.com. aaaa ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42641 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.google.com. IN AAAA ;; ANSWER SECTION: www-anycast.google.com. 300 IN AAAA 2001:4860:4802:34::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:38::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:36::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:32::75 ;; Query time: 7 msec ;; SERVER: 216.239.32.10#53(216.239.32.10) ;; WHEN: Fri Sep 06 19:05:32 UTC 2019 ;; MSG SIZE rcvd: 167 ``` So far I've observed this with A and AAAA queries. It's my understanding that without a CNAME record in the answer, the resolver is doing the right thing by ignoring the answer, as there's no linkage between www and www-anycast. Is this broken, or is this just some weird DNS trick I've not come across before? -- Chip Marshall <chip@2bithacker.net>
On Sep 6, 2019, at 3:11 PM, Chip Marshall via NANOG <nanog@nanog.org> wrote:
Hello, I'm seeing an oddity when doing DNS lookups for www.google.com from our London datacenter, and I'm curious if other people are seeing the same behavior.
It appears that when we ask for www.google.com. we sometimes get an answer that only contains records for www-anycast.google.com., which our resolver ignores as they don't match the query.
As seen with dig:
``` # dig @ns1.google.com. www.google.com. aaaa
; <<>> DiG 9.10.3-P4-Ubuntu <<>> @ns1.google.com. www.google.com. aaaa ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42641 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.google.com. IN AAAA
;; ANSWER SECTION: www-anycast.google.com. 300 IN AAAA 2001:4860:4802:34::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:38::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:36::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:32::75
;; Query time: 7 msec ;; SERVER: 216.239.32.10#53(216.239.32.10) ;; WHEN: Fri Sep 06 19:05:32 UTC 2019 ;; MSG SIZE rcvd: 167 ```
So far I've observed this with A and AAAA queries. It's my understanding that without a CNAME record in the answer, the resolver is doing the right thing by ignoring the answer, as there's no linkage between www and www-anycast.
Is this broken, or is this just some weird DNS trick I've not come across before?
You may want to post on dns-operations instead. Can you do a dig +trace www.google.com instead, that would be more instructive about what’s happening at each layer of the delegation. - Jared
Do you see the same behavior when you execute your dig query without the trailing dot? Thanks, Stephen
On Sep 6, 2019, at 3:11 PM, Chip Marshall via NANOG <nanog@nanog.org> wrote:
Hello, I'm seeing an oddity when doing DNS lookups for www.google.com from our London datacenter, and I'm curious if other people are seeing the same behavior.
It appears that when we ask for www.google.com. we sometimes get an answer that only contains records for www-anycast.google.com., which our resolver ignores as they don't match the query.
As seen with dig:
``` # dig @ns1.google.com. www.google.com. aaaa
; <<>> DiG 9.10.3-P4-Ubuntu <<>> @ns1.google.com. www.google.com. aaaa ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42641 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.google.com. IN AAAA
;; ANSWER SECTION: www-anycast.google.com. 300 IN AAAA 2001:4860:4802:34::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:38::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:36::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:32::75
;; Query time: 7 msec ;; SERVER: 216.239.32.10#53(216.239.32.10) ;; WHEN: Fri Sep 06 19:05:32 UTC 2019 ;; MSG SIZE rcvd: 167 ```
So far I've observed this with A and AAAA queries. It's my understanding that without a CNAME record in the answer, the resolver is doing the right thing by ignoring the answer, as there's no linkage between www and www-anycast.
Is this broken, or is this just some weird DNS trick I've not come across before?
You may want to post on dns-operations instead.
Can you do a dig +trace www.google.com instead, that would be more instructive about whatтs happening at each layer o f the delegation.
- Jared
Unable to replicate this in London: ``` ; <<>> DiG 9.11.5-P1-1ubuntu2.5-Ubuntu <<>> @ns1.google.com. www.google.com. aaaa ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61970 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.google.com. IN AAAA ;; ANSWER SECTION: www.google.com. 300 IN AAAA 2a00:1450:4009:80d::2004 ``` going by the latency, ns1.google.com (https://link.getmailspring.com/link/C27D5EBE-B680-425A-B057-218C6300A7B4@getmailspring.com/0?redirect=ns1.google.com&recipient=bmFub2dAbmFub2cub3Jn) travels to NL from our UK PoPs though: ``` Host Loss% Snt Last Avg Best Wrst StDev 1. ??? 2. ??? 3. ae26-0.ebr01.lon3.uk.globalone 0.0% 13 2.1 6.2 1.0 45.7 12.9 4. 2001:7f8:4::3b41:1 0.0% 13 0.7 0.8 0.6 1.7 0.4 5. 2001:4860:0:1101::10 0.0% 13 0.7 2.7 0.7 14.2 4.2 6. 2001:4860::c:4000:cf5b 0.0% 13 1.8 2.1 1.5 4.0 0.7 7. 2001:4860::8:4000:d325 0.0% 13 8.6 7.3 6.6 9.5 0.9 8. 2001:4860::22:4001:70b 0.0% 13 6.4 9.5 6.4 36.9 8.3 9. 2001:4860:0:1::be7 23.1% 13 7.3 7.5 7.3 7.7 0.1 10. ??? 11. ??? 12. ??? 13. ??? 14. ??? 15. ??? 16. ??? 17. ??? 18. ??? 19. ns1.google.com 0.0% 12 6.4 6.4 6.3 6.5 0.0 ``` On Sep. 6 2019, at 9:49 pm, Stephen Stuart <stuart@tech.org> wrote:
Do you see the same behavior when you execute your dig query without the trailing dot?
Thanks, Stephen
On Sep 6, 2019, at 3:11 PM, Chip Marshall via NANOG <nanog@nanog.org> wrote: Hello, I'm seeing an oddity when doing DNS lookups for www.google.com from our London datacenter, and I'm curious if other people are seeing the same behavior.
It appears that when we ask for www.google.com. we sometimes get an answer that only contains records for www-anycast.google.com., which our resolver ignores as they don't match the query.
As seen with dig: ``` # dig @ns1.google.com. www.google.com. aaaa
; <<>> DiG 9.10.3-P4-Ubuntu <<>> @ns1.google.com. www.google.com. aaaa ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42641 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.google.com. IN AAAA
;; ANSWER SECTION: www-anycast.google.com. 300 IN AAAA 2001:4860:4802:34::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:38::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:36::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:32::75
;; Query time: 7 msec ;; SERVER: 216.239.32.10#53(216.239.32.10) ;; WHEN: Fri Sep 06 19:05:32 UTC 2019 ;; MSG SIZE rcvd: 167 ```
So far I've observed this with A and AAAA queries. It's my understanding that without a CNAME record in the answer, the resolver is doing the right thing by ignoring the answer, as there's no linkage between www and www-anycast.
Is this broken, or is this just some weird DNS trick I've not come across before?
You may want to post on dns-operations instead. Can you do a dig +trace www.google.com instead, that would be more instructive about whatт€™s happening at each layer o f the delegation.
- Jared
Yes, this is no longer occurring / is resolved. Apologies, W On Mon, Sep 9, 2019 at 1:37 PM Florian Brandstetter via NANOG < nanog@nanog.org> wrote:
Unable to replicate this in London:
``` ; <<>> DiG 9.11.5-P1-1ubuntu2.5-Ubuntu <<>> @ns1.google.com. www.google.com. aaaa ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61970 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.google.com. IN AAAA ;; ANSWER SECTION: www.google.com. 300 IN AAAA 2a00:1450:4009:80d::2004 ```
going by the latency, ns1.google.com <https://link.getmailspring.com/link/C27D5EBE-B680-425A-B057-218C6300A7B4@getmailspring.com/0?redirect=ns1.google.com&recipient=bmFub2dAbmFub2cub3Jn> travels to NL from our UK PoPs though:
``` Host Loss% Snt Last Avg Best Wrst StDev 1. ??? 2. ??? 3. ae26-0.ebr01.lon3.uk.globalone 0.0% 13 2.1 6.2 1.0 45.7 12.9 4. 2001:7f8:4::3b41:1 0.0% 13 0.7 0.8 0.6 1.7 0.4 5. 2001:4860:0:1101::10 0.0% 13 0.7 2.7 0.7 14.2 4.2 6. 2001:4860::c:4000:cf5b 0.0% 13 1.8 2.1 1.5 4.0 0.7 7. 2001:4860::8:4000:d325 0.0% 13 8.6 7.3 6.6 9.5 0.9 8. 2001:4860::22:4001:70b 0.0% 13 6.4 9.5 6.4 36.9 8.3 9. 2001:4860:0:1::be7 23.1% 13 7.3 7.5 7.3 7.7 0.1 10. ??? 11. ??? 12. ??? 13. ??? 14. ??? 15. ??? 16. ??? 17. ??? 18. ??? 19. ns1.google.com 0.0% 12 6.4 6.4 6.3 6.5 0.0 ``` On Sep. 6 2019, at 9:49 pm, Stephen Stuart <stuart@tech.org> wrote:
Do you see the same behavior when you execute your dig query without the trailing dot?
Thanks, Stephen
On Sep 6, 2019, at 3:11 PM, Chip Marshall via NANOG <nanog@nanog.org> wrote:
Hello, I'm seeing an oddity when doing DNS lookups for www.google.com from our London datacenter, and I'm curious if other people are seeing the same behavior.
It appears that when we ask for www.google.com. we sometimes get an answer that only contains records for www-anycast.google.com., which our resolver ignores as they don't match the query.
As seen with dig:
``` # dig @ns1.google.com. www.google.com. aaaa
; <<>> DiG 9.10.3-P4-Ubuntu <<>> @ns1.google.com. www.google.com. aaaa ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42641 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.google.com. IN AAAA
;; ANSWER SECTION: www-anycast.google.com. 300 IN AAAA 2001:4860:4802:34::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:38::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:36::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:32::75
;; Query time: 7 msec ;; SERVER: 216.239.32.10#53(216.239.32.10) ;; WHEN: Fri Sep 06 19:05:32 UTC 2019 ;; MSG SIZE rcvd: 167 ```
So far I've observed this with A and AAAA queries. It's my understanding that without a CNAME record in the answer, the resolver is doing the right thing by ignoring the answer, as there's no linkage between www and www-anycast.
Is this broken, or is this just some weird DNS trick I've not come across before?
You may want to post on dns-operations instead.
Can you do a dig +trace www.google.com instead, that would be more instructive about whatт€™s happening at each layer o
f the delegation.
- Jared
[image: Sent from Mailspring]
-- I don't think the execution is relevant when it was obviously a bad idea in the first place. This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. ---maf
On 2019-09-06, Stephen Stuart <stuart@tech.org> sent:
Do you see the same behavior when you execute your dig query without the trailing dot?
Yes. dig adds on the trailing dot to make it an FQDN anyway, so the on-wire qname is the same either way. -- Chip Marshall <chip@2bithacker.net>
On 2019-09-06, Jared Mauch <jared@puck.nether.net> sent:
You may want to post on dns-operations instead.
Will do.
Can you do a dig +trace www.google.com instead, that would be more instructive about what’s happening at each layer of the delegation.
# dig +trace www.google.com. aaaa ; <<>> DiG 9.10.3-P4-Ubuntu <<>> +trace www.google.com. aaaa ;; global options: +cmd . 40841 IN NS b.root-servers.net. . 40841 IN NS g.root-servers.net. . 40841 IN NS k.root-servers.net. . 40841 IN NS i.root-servers.net. . 40841 IN NS m.root-servers.net. . 40841 IN NS c.root-servers.net. . 40841 IN NS l.root-servers.net. . 40841 IN NS e.root-servers.net. . 40841 IN NS d.root-servers.net. . 40841 IN NS f.root-servers.net. . 40841 IN NS h.root-servers.net. . 40841 IN NS j.root-servers.net. . 40841 IN NS a.root-servers.net. . 40841 IN RRSIG NS 8 0 518400 20190917050000 20190904040000 59944 . W93v8sQLROIXL1qvcezKKnL8XwzzxuFb6VbyV7h+SG27BIgJiOGrNE5q M6ncTYozvKd3tKJ/cQZcnIO9zi9tInPKgVctNF1Fp2FGb8TnFuTkIOMy MEVzbWEZrZErcToDRaK1WzlrxBL6gsIfegE8gjC/2XVnKQENZCJ4qgg8 V/u1CKbJGV0nmnVusCZ6pXnkVJDDdvvicaUf0IoxqEONh1h/xKghX14R 6leOUCJpAtdS0M9eyPeBL5myCm7olOVhi/A+9QjZLv60vefYAF7aREtW 5mEvg/YyNz4dUOHrhz/iRbK/wGIbtyuTpvy3Gg/F2dtrVfJBzobDnGpv sFO4xA== ;; Received 525 bytes from 8.8.8.8#53(8.8.8.8) in 1 ms com. 172800 IN NS a.gtld-servers.net. com. 172800 IN NS b.gtld-servers.net. com. 172800 IN NS c.gtld-servers.net. com. 172800 IN NS d.gtld-servers.net. com. 172800 IN NS e.gtld-servers.net. com. 172800 IN NS f.gtld-servers.net. com. 172800 IN NS g.gtld-servers.net. com. 172800 IN NS h.gtld-servers.net. com. 172800 IN NS i.gtld-servers.net. com. 172800 IN NS j.gtld-servers.net. com. 172800 IN NS k.gtld-servers.net. com. 172800 IN NS l.gtld-servers.net. com. 172800 IN NS m.gtld-servers.net. com. 86400 IN DS 30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766 com. 86400 IN RRSIG DS 8 1 86400 20190919170000 20190906160000 59944 . ep9gNcyySwR/AqNOnfjXq3OCw5IwOJnIxU4U25UdZ2ejwbJqLf8ytp68 O5DQz1N/PvrEhi1Wg8XyQHZM+fc38cYhhjG5HMVOcEN3wvifnxTWEwBs ay2GxF10TtUpg9TF4Qs2+V8k0ABWwAKIBbSAeZ+C+l5mBg18CCnTgjeg PR+466SgA7sHbzaI9PYK57suhq3uLrphcC2Ti7jmV9V41H5D52gNTiV5 eQ2BsPo+l5LyLrvusailMOzogav9v4M9bnOSGTcc85nf/wD5/Vo4R4MU OexIxio0NGBl7GeS3zoPKV29CYnfcuZBkD2VBuPKZafxp0nIo4olMznn szi9lg== ;; Received 1174 bytes from 199.7.83.42#53(l.root-servers.net) in 60 ms google.com. 172800 IN NS ns2.google.com. google.com. 172800 IN NS ns1.google.com. google.com. 172800 IN NS ns3.google.com. google.com. 172800 IN NS ns4.google.com. CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q1GIN43N1ARRC9OSM6QPQR81H5M9A NS SOA RRSIG DNSKEY NSEC3PARAM CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20190912044627 20190905033627 17708 com. kXWtAEptQhH9JpsAJzpvEwEwRtybI/FVl9Hrd1lr/GTkZ3P4clnR7YLB quX4CVf8E0+gEfwf4U2PpmphROV1eHweyycVydvTE8etaDipTpItbtyG 7Iz/uKjp1TY3RD+qNa6LZ1juEs70aKPsbmEV79rtiTW2kurdgqslP5jH Jg0= S84BDVKNH5AGDSI7F5J0O3NPRHU0G7JQ.com. 86400 IN NSEC3 1 1 0 - S84CFH3A62N0FJPC5D9IJ2VJR71OGLV5 NS DS RRSIG S84BDVKNH5AGDSI7F5J0O3NPRHU0G7JQ.com. 86400 IN RRSIG NSEC3 8 2 86400 20190913045601 20190906034601 17708 com. bJE7LV1REfTtY1jFj/9qA1CKIDBgCJOTV42tSwf92aqhTAkflM9QFH7/ 3Z5440IkZ8PoWMt9Yn7fn+Q+cTZVnbj071jVpiLNXshhMQbtDC1eJkLz AIuATIj+dqWTWQg7vut0oiy0wnJ2ktSgqTFe4JtwRD0lWO6+NgnhbgQD 2yg= ;; Received 776 bytes from 192.43.172.30#53(i.gtld-servers.net) in 74 ms www-anycast.google.com. 300 IN AAAA 2001:4860:4802:32::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:34::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:38::75 www-anycast.google.com. 300 IN AAAA 2001:4860:4802:36::75 ;; Received 167 bytes from 216.239.38.10#53(ns4.google.com) in 6 ms -- Chip Marshall <chip@2bithacker.net>
Chip Marshall via NANOG wrote on 06/09/2019 20:11:
Hello, I'm seeing an oddity when doing DNS lookups for www.google.com from our London datacenter, and I'm curious if other people are seeing the same behavior.
I saw a bunch of monitoring systems queries for www.google.com/A return back with no A records at ~20:35 BST. It's returned back to normal now, but we needed to flush a bunch of DNS caches. Nick
Nick Hilliard wrote on 06/09/2019 21:19:
Chip Marshall via NANOG wrote on 06/09/2019 20:11:
Hello, I'm seeing an oddity when doing DNS lookups for www.google.com from our London datacenter, and I'm curious if other people are seeing the same behavior.
I saw a bunch of monitoring systems queries for www.google.com/A return back with no A records at ~20:35 BST. It's returned back to normal now, but we needed to flush a bunch of DNS caches.
... aaaand there we go again at 22:07 BST. Nick
Where are you based? I can check if this can be replicated in our backbone, in case we have a PoP close. On Sep. 6 2019, at 11:17 pm, Nick Hilliard <nick@foobar.org> wrote:
Nick Hilliard wrote on 06/09/2019 21:19:
Chip Marshall via NANOG wrote on 06/09/2019 20:11:
Hello, I'm seeing an oddity when doing DNS lookups for www.google.com from our London datacenter, and I'm curious if other people are seeing the same behavior.
I saw a bunch of monitoring systems queries for www.google.com/A return back with no A records at ~20:35 BST. It's returned back to normal now, but we needed to flush a bunch of DNS caches.
... aaaand there we go again at 22:07 BST. Nick
participants (6)
-
Chip Marshall -
Florian Brandstetter -
Jared Mauch -
Nick Hilliard -
Stephen Stuart -
Warren Kumari