IRR mirrors de-sync from ARIN or irrd4 bug or ARIN streaming is broken?
Hi folks! I noticed something unusual today, and perhaps some of you know the answer. Consider ARIN rr: $ whois -h rr.arin.net 199.52.73.0/24 route: 199.52.73.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-10-19T08:37:50Z last-modified: 2023-11-27T15:10:44Z source: ARIN everything looks fine. Now RADB mirror: $ whois -h whois.radb.net 199.52.73.0/24 % No entries found for the selected source(s). nothing! I suspect the mirror is out of sync. Now NTT mirror: $ whois -h rr1.ntt.net 199.52.73.0/24 route: 199.52.73.0/24 descr: RPKI ROA for 199.52.73.0/24 / AS132055 remarks: This AS132055 route object represents routing data retrieved from the RPKI. This route object is the result of an automated RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin As you can see it returns only RPKI data, and not ARIN. So ARIN data is not in sync there as well? However for 199.52.53.0/24 it returns both $ whois -h rr1.ntt.net 199.52.53.0/24 route: 199.52.53.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-07-11T07:05:30Z last-modified: 2023-11-27T15:10:44Z source: ARIN rpki-ov-state: valid route: 199.52.53.0/24 descr: RPKI ROA for 199.52.53.0/24 / AS132055 remarks: This AS132055 route object represents routing data retrieved from the RPKI. This route object is the result of an automated RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin My question is how and what happened? I suspect whois stream was incostent. Because if one check todays ARIN DB it surely has the data $ wget ftp://ftp.arin.net/pub/rr/arin.db.gz -O - 2>/dev/null | gzip -d | grep -A10 199.52.73.0/24 route: 199.52.73.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-10-19T08:37:50Z last-modified: 2023-11-27T15:10:44Z source: ARIN Thanks!
If you look at TC, you will see that this object is part of ARIN-NONAUTH: https://bgp.net.br/whois.html?q=199.52.73.0%2F24 route: 199.52.72.0/22 descr: Ernst & Young, Gurgaon Cyberpark, India origin: AS132055 mnt-by: MNT-EYL changed: zanub-h.kalathingal@sg.ey.com 20210614 source: ARIN-NONAUTH remarks: **************************** remarks: * THIS OBJECT CONTAINS PLACEHOLDER DATA remarks: * Please note that all data that is generally regarded as personal remarks: * data has been removed from this object. remarks: * To view the original object, please query the ARIN Database at: remarks: * http://www.arin.net/whois remarks: **************************** rpki-ov-state: not_found # No ROAs found, or RPKI validation not enabled for source It's also mapped to an existing RPKI entry: route: 199.52.73.0/24 descr: RPKI ROA for 199.52.73.0/24 / AS132055 remarks: This AS132055 route object represents routing data retrieved from the RPKI. This route object is the result of an automated RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin Perhaps RADB is preferring not to mirror non-authoritative databases ? Rubens On Wed, Jul 10, 2024 at 10:05 PM Aliaksei Sheshka <sheshkaoss@gmail.com> wrote:
Hi folks!
I noticed something unusual today, and perhaps some of you know the answer.
Consider ARIN rr:
$ whois -h rr.arin.net 199.52.73.0/24 route: 199.52.73.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-10-19T08:37:50Z last-modified: 2023-11-27T15:10:44Z source: ARIN
everything looks fine.
Now RADB mirror:
$ whois -h whois.radb.net 199.52.73.0/24 % No entries found for the selected source(s).
nothing! I suspect the mirror is out of sync.
Now NTT mirror:
$ whois -h rr1.ntt.net 199.52.73.0/24 route: 199.52.73.0/24 descr: RPKI ROA for 199.52.73.0/24 / AS132055 remarks: This AS132055 route object represents routing data retrieved from the RPKI. This route object is the result of an automated RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin
As you can see it returns only RPKI data, and not ARIN. So ARIN data is not in sync there as well?
However for 199.52.53.0/24 it returns both
$ whois -h rr1.ntt.net 199.52.53.0/24 route: 199.52.53.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-07-11T07:05:30Z last-modified: 2023-11-27T15:10:44Z source: ARIN rpki-ov-state: valid
route: 199.52.53.0/24 descr: RPKI ROA for 199.52.53.0/24 / AS132055 remarks: This AS132055 route object represents routing data retrieved from the RPKI. This route object is the result of an automated RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin
My question is how and what happened? I suspect whois stream was incostent. Because if one check todays ARIN DB it surely has the data
$ wget ftp://ftp.arin.net/pub/rr/arin.db.gz -O - 2>/dev/null | gzip -d | grep -A10 199.52.73.0/24 route: 199.52.73.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-10-19T08:37:50Z last-modified: 2023-11-27T15:10:44Z source: ARIN
Thanks!
Rubens, ARIN-NONAUTH was deprecated two years ago: https://www.arin.net/vault/announcements/20220404-irr/ Aliaksei, Indeed, it appears both NTT’s and RADB’s mirror instances are desynchronized in relationship to ARIN’s IRR. Both NTT and RADB should do a database reload to rectify the issue. Desynchronisation can happen at either server side or client side, so the root cause isn’t apparent to me. Kind regards, Job On Thu, 11 Jul 2024 at 10:15, Rubens Kuhl <rubensk@gmail.com> wrote:
If you look at TC, you will see that this object is part of ARIN-NONAUTH:
https://bgp.net.br/whois.html?q=199.52.73.0%2F24
route: 199.52.72.0/22 descr: Ernst & Young, Gurgaon Cyberpark, India origin: AS132055 mnt-by: MNT-EYL changed: zanub-h.kalathingal@sg.ey.com 20210614 source: ARIN-NONAUTH remarks: **************************** remarks: * THIS OBJECT CONTAINS PLACEHOLDER DATA remarks: * Please note that all data that is generally regarded as personal remarks: * data has been removed from this object. remarks: * To view the original object, please query the ARIN Database at: remarks: * http://www.arin.net/whois remarks: **************************** rpki-ov-state: not_found # No ROAs found, or RPKI validation not enabled for source
It's also mapped to an existing RPKI entry:
route: 199.52.73.0/24 descr: RPKI ROA for 199.52.73.0/24 / AS132055 remarks: This AS132055 route object represents routing data retrieved from the RPKI. This route object is the result of an automated RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin
Perhaps RADB is preferring not to mirror non-authoritative databases ?
Rubens
On Wed, Jul 10, 2024 at 10:05 PM Aliaksei Sheshka <sheshkaoss@gmail.com> wrote:
Hi folks!
I noticed something unusual today, and perhaps some of you know the
answer.
Consider ARIN rr:
$ whois -h rr.arin.net 199.52.73.0/24 route: 199.52.73.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-10-19T08:37:50Z last-modified: 2023-11-27T15:10:44Z source: ARIN
everything looks fine.
Now RADB mirror:
$ whois -h whois.radb.net 199.52.73.0/24 % No entries found for the selected source(s).
nothing! I suspect the mirror is out of sync.
Now NTT mirror:
$ whois -h rr1.ntt.net 199.52.73.0/24 route: 199.52.73.0/24 descr: RPKI ROA for 199.52.73.0/24 / AS132055 remarks: This AS132055 route object represents routing data
retrieved
from the RPKI. This route object is the result of an
automated
RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin
As you can see it returns only RPKI data, and not ARIN. So ARIN data is
not in sync there as well?
However for 199.52.53.0/24 it returns both
$ whois -h rr1.ntt.net 199.52.53.0/24 route: 199.52.53.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-07-11T07:05:30Z last-modified: 2023-11-27T15:10:44Z source: ARIN rpki-ov-state: valid
route: 199.52.53.0/24 descr: RPKI ROA for 199.52.53.0/24 / AS132055 remarks: This AS132055 route object represents routing data
retrieved
from the RPKI. This route object is the result of an
automated
RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin
My question is how and what happened? I suspect whois stream was
incostent.
Because if one check todays ARIN DB it surely has the data
$ wget ftp://ftp.arin.net/pub/rr/arin.db.gz -O - 2>/dev/null | gzip -d | grep -A10 199.52.73.0/24 route: 199.52.73.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-10-19T08:37:50Z last-modified: 2023-11-27T15:10:44Z source: ARIN
Thanks!
On Wed, Jul 10, 2024 at 9:26 PM Job Snijders via NANOG <nanog@nanog.org> wrote:
Aliaksei,
Indeed, it appears both NTT’s and RADB’s mirror instances are desynchronized in relationship to ARIN’s IRR. Both NTT and RADB should do a database reload to rectify the issue.
Desynchronisation can happen at either server side or client side, so the root cause isn’t apparent to me.
Now I'm concerned about how frequent such silent corruptions are. Also, hard (impossible?) to tell when that happened. IRR `route` and friend should be retired, the sooner the better.
On Wed, Jul 10, 2024 at 09:37:22PM -0400, Aliaksei Sheshka wrote:
On Wed, Jul 10, 2024 at 9:26 PM Job Snijders via NANOG <nanog@nanog.org> wrote:
Indeed, it appears both NTT’s and RADB’s mirror instances are desynchronized in relationship to ARIN’s IRR. Both NTT and RADB should do a database reload to rectify the issue.
Desynchronisation can happen at either server side or client side, so the root cause isn’t apparent to me.
Now I'm concerned about how frequent such silent corruptions are. Also, hard (impossible?) to tell when that happened.
Work is underway to at least get rid of NRTM v3 https://www.ietf.org/archive/id/draft-ietf-grow-nrtm-v4-04.html
IRR `route` and friend should be retired, the sooner the better.
I am with you on that one, eventually, because 'just turning it off' doesn't seem a great solution to me. Not everyone who has access to IRR services has access to RPKI services. Not everyone agrees with ARIN's Relying Party Agreement. RPKI doesn't do all the things that IRR does (but I am not in favor of blindly porting all features & misfeatures from IRR to RPKI!). There is a fair bit of study to be done in this space. Kind regards, Job
On Wed, Jul 10, 2024 at 07:10:48PM -0400, Aliaksei Sheshka wrote:
nothing! I suspect the mirror is out of sync.
Now NTT mirror:
Seems reloading helped: $ date Thu Jul 11 03:50:22 UTC 2024 $ whois -h rr.ntt.net 199.52.73.0/24 route: 199.52.73.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-10-19T08:37:50Z last-modified: 2023-11-27T15:10:44Z source: ARIN rpki-ov-state: valid route: 199.52.73.0/24 descr: RPKI ROA for 199.52.73.0/24 / AS132055 remarks: This AS132055 route object represents routing data retrieved from the RPKI. This route object is the result of an automated RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin
From the CAIDA study on this <https://www.caida.org/catalog/papers/2022_irr_hygiene_rpki_era/irr_hygiene_rpki_era.pdf>: "In October 2021, we found matching Route Origin Authorization objects (ROAs) for around 20% of RADB IRR records, and a consistency of 38% and 60% in v4 and v6." Do think much has improved here.
On Wed, Jul 10, 2024 at 10:52 PM Job Snijders via NANOG <nanog@nanog.org> wrote:
On Wed, Jul 10, 2024 at 07:10:48PM -0400, Aliaksei Sheshka wrote:
nothing! I suspect the mirror is out of sync.
Now NTT mirror:
Seems reloading helped:
$ date Thu Jul 11 03:50:22 UTC 2024
$ whois -h rr.ntt.net 199.52.73.0/24 route: 199.52.73.0/24 origin: AS132055 descr: EY India admin-c: IAM12-ARIN tech-c: DNSAD85-ARIN tech-c: IAM12-ARIN mnt-by: MNT-EYL-Z created: 2022-10-19T08:37:50Z last-modified: 2023-11-27T15:10:44Z source: ARIN rpki-ov-state: valid
route: 199.52.73.0/24 descr: RPKI ROA for 199.52.73.0/24 / AS132055 remarks: This AS132055 route object represents routing data retrieved from the RPKI. This route object is the result of an automated RPKI-to-IRR conversion process performed by IRRd. max-length: 24 origin: AS132055 source: RPKI # Trust Anchor: arin
participants (4)
-
Aliaksei Sheshka -
Innocent Obi -
Job Snijders -
Rubens Kuhl