how would draft-ymbk-opsawg-finding-geofeeds work in noam
would folk familiar with the north american RIR and IRR registries be kind enough to suggest how this might adapt? thanks. A new version of I-D, draft-ymbk-opsawg-finding-geofeeds-02.txt has been successfully submitted by Randy Bush and posted to the IETF repository. Name: draft-ymbk-opsawg-finding-geofeeds Revision: 02 Title: Finding and Using Geofeed Data Document date: 2020-09-11 Group: Individual Submission Pages: 16 URL: https://www.ietf.org/id/draft-ymbk-opsawg-finding-geofeeds-02.txt Status: https://datatracker.ietf.org/doc/draft-ymbk-opsawg-finding-geofeeds/ Htmlized: https://datatracker.ietf.org/doc/html/draft-ymbk-opsawg-finding-geofeeds Htmlized: https://tools.ietf.org/html/draft-ymbk-opsawg-finding-geofeeds-02 Diff: https://www.ietf.org/rfcdiff?url2=draft-ymbk-opsawg-finding-geofeeds-02 Abstract: This document describes how to find and authenticate geofeed data.
On Fri, Sep 11, 2020 at 1:49 PM Randy Bush <randy@psg.com> wrote:
would folk familiar with the north american RIR and IRR registries be
kind enough to suggest how this might adapt? thanks.
A new version of I-D, draft-ymbk-opsawg-finding-geofeeds-02.txt
has been successfully submitted by Randy Bush and posted to the
IETF repository.
Name: draft-ymbk-opsawg-finding-geofeeds
Revision: 02
Title: Finding and Using Geofeed Data
Document date: 2020-09-11
Group: Individual Submission
Pages: 16
URL: https://www.ietf.org/id/draft-ymbk-opsawg-finding-geofeeds-02.txt
Status: https://datatracker.ietf.org/doc/draft-ymbk-opsawg-finding-geofeeds/
Htmlized: https://datatracker.ietf.org/doc/html/draft-ymbk-opsawg-finding-geofeeds
Htmlized: https://tools.ietf.org/html/draft-ymbk-opsawg-finding-geofeeds-02
Diff: https://www.ietf.org/rfcdiff?url2=draft-ymbk-opsawg-finding-geofeeds-02
Abstract:
This document describes how to find and authenticate geofeed data.
I suppose in a pinch i could read this in detail, it is a problem i would like to solve, but you lost my interest at RPSL and i totally tuned out by the mention of cryptographic signatures... and some blah blah about rpki... although i did enjoy the inclusion or the random text strings displaying what entropy looks like 1. How is this easier than me emailing a spreadsheet to Akamai and others with subnets and zip codes and make them figure it out? They are being paid to figure this out, not me the eyeball network. Put the burden on the org getting paid. 2. Why can’t arin just provide me a web portal to add city / state info in my already existing rpki roa form? I hate paying Merit for radb, i dont want to rely on them and their janky website for another thing. Every time i touch it i am slightly afraid some auto generated filter is going blackhole my network because i made a typo, and they only update their pull of radb every 72 hours ... Make it is easy, and it will get done ... this excludes rpsl and anything requiring me to use the openssl command line
hi camron sad to say, the days of faxing around number assigments have passed. the kiddie googlers who wrote the geofeeds rfc probably have not even seen a fax machine. they just did not like having a hundred gnomes in the basement dealing with everybody's formats. given silicon valley housing costs, gnome armies do not scale well. and, imiho, the format they developed is sufficiently boring that it might work. ten thousand providers emailing each other spreadsheets is not very pretty or reliable at global scale. the plague is teaching normal folk the concept of exponential. so where A *finds* B's geofeed data is an ugly issue. i agree that rpsl sucks. like democracy, it may suck less than the alternatives. but if you have a better suggestion, puhleeze!
2. Why can’t arin just provide me a web portal to add city / state info in my already existing rpki roa form?
and then where do those data go? you wanna go through the ietf process of putting it in the rpki, which has no 'remarks' as an intermediate hack? and 10,000 consumers of the data would have to sign arin legal papers. and arin is just one corner of a large world and becoming less and less relevant. as to using zip codes; aside from the international issues, in many locales postal codes are sufficiently specific to reveal personal identity. or at least this is why some geofeed designers told me they avoided postal codes. but this document is not about those choices anyway; geofeed file format was assumed. i hope you noted that the rpki-based signing is entirely optional. i certainly do not sign my geofeed files, and am not aware of any other deployment of this tech which does. randy
On Sat, Sep 12, 2020 at 1:13 PM Randy Bush <randy@psg.com> wrote:
hi camron
sad to say, the days of faxing around number assigments have passed. the kiddie googlers who wrote the geofeeds rfc probably have not even seen a fax machine. they just did not like having a hundred gnomes in the basement dealing with everybody's formats. given silicon valley housing costs, gnome armies do not scale well. and, imiho, the format they developed is sufficiently boring that it might work.
ten thousand providers emailing each other spreadsheets is not very pretty or reliable at global scale. the plague is teaching normal folk the concept of exponential.
so where A *finds* B's geofeed data is an ugly issue. i agree that rpsl sucks. like democracy, it may suck less than the alternatives. but if you have a better suggestion, puhleeze!
2. Why can’t arin just provide me a web portal to add city / state info in my already existing rpki roa form?
and then where do those data go? you wanna go through the ietf process of putting it in the rpki, which has no 'remarks' as an intermediate hack?
and 10,000 consumers of the data would have to sign arin legal papers. and arin is just one corner of a large world and becoming less and less relevant.
as to using zip codes; aside from the international issues, in many locales postal codes are sufficiently specific to reveal personal identity. or at least this is why some geofeed designers told me they avoided postal codes. but this document is not about those choices anyway; geofeed file format was assumed.
i hope you noted that the rpki-based signing is entirely optional. i certainly do not sign my geofeed files, and am not aware of any other deployment of this tech which does.
randy
Hi Randy, Just business in this email. I gave your I-D a try in the real world, and it does not work with a major player. I thought you might know this, and could have saved me a few hours of tinkering... But to help others along, inetnum does not work at RADB. Web form, email, nada.
hi ca
I gave your I-D a try in the real world, and it does not work with a major player.
i.e. arin and radb, your region, which does not do rpsl as others do. we know this, which is why the OP $subjest was "how would draft-ymbk-opsawg-finding-geofeeds work in noam?"
I thought you might know this, and could have saved me a few hours of tinkering...
yikes! did not mean to send you down a rabbit hole. apologies. i had naïvely assumed the $subject would have warned you.
But to help others along, inetnum does not work at RADB. Web form, email, nada.
my personal guess is that radb might choose to adapt, similarly to irrd's thoughts. but arin? ha ha. so my guess is an open source simple shim to adapt the the noam snowflakiness. i.e. be able to consume arin nethandle and radb comments. but i am hoping that others might have brighter ideas. randy
On Sat, Sep 12, 2020 at 4:43 PM Randy Bush <randy@psg.com> wrote:
hi ca
I gave your I-D a try in the real world, and it does not work with a
major player.
i.e. arin and radb, your region, which does not do rpsl as others do.
we know this, which is why the OP $subjest was
"how would draft-ymbk-opsawg-finding-geofeeds work in noam?"
I thought you might know this, and could have saved me a few hours of
tinkering...
yikes! did not mean to send you down a rabbit hole. apologies. i had
naïvely assumed the $subject would have warned you.
Ok, then i can reply to the subject. It does not work There are likely 100 ways to do this that do work (dns, peeringdb, other rpsl comment fields ...), and you choose the one that does not. Try harder. CB
But to help others along, inetnum does not work at RADB. Web form,
email, nada.
my personal guess is that radb might choose to adapt, similarly to
irrd's thoughts. but arin? ha ha.
so my guess is an open source simple shim to adapt the the noam
snowflakiness. i.e. be able to consume arin nethandle and radb
comments.
but i am hoping that others might have brighter ideas.
randy
On Fri, Sep 11, 2020 at 1:48 PM Randy Bush <randy@psg.com> wrote:
would folk familiar with the north american RIR and IRR registries be kind enough to suggest how this might adapt? thanks.
Hi Randy, Why not publish RFC8805 Geofeed directly in inetnum remarks section? Then there is no more need to host HTTP server. 1 HTTP request per inetnum/inetnum6 object seems a bit tedious. How to handle other stuff that might exist in remarks field? Or the draft would explicitly require Geofeed to be in its own remarks field? Specific to the north american RIR, NetHandle is in registry (not in irr) abd bulk access requires application (https://www.arin.net/reference/research/bulkwhois/#accessing-bulk-whois-data), download requires cookies and takes ~10 mins. imo this could be made easier like https://ftp.ripe.net/ripe/dbase/split/ripe.db.inetnum.gz NetHandle is not exactly inetnum (e.g. comments instead of remarks field, different prefix format). Would the RIR provide converted inetnum objects or the users would be expected to handle this? Yang
yang,
Why not publish RFC8805 Geofeed directly in inetnum remarks section?
for some flat fan out last kilometer providers that could be the inetnum: object from hell. there are global providers which segment large prefixes over diverse areas. etc. i doubt the rpsl providers would like multi-megabyte inetnum:s. rpsl providers already throttle in defense.
Then there is no more need to host HTTP server. 1 HTTP request per inetnum/inetnum6 object seems a bit tedious.
we are not expecting these lookups to be done frequently. i agree that would hammer servers, both rpsl and geofeed. do you have stronger words to suggest than 5. Operational Considerations ... An entity fetching geofeed data through these mechanisms MUST NOT do frequent real-time look-ups to prevent load on RPSL servers. And do not fetch at midnight, because everyone else may. i agree that we do not want the DDoS that is currently happening to RPKI publication servers. perhaps explicit time limits, e.g daily?
How to handle other stuff that might exist in remarks field? Or the draft would explicitly require Geofeed to be in its own remarks field?
the document tries to be explicit that it is the latter. that is the intent of 3. inetnum: Class ... the syntax of a Geofeed remarks: attribute which contains a URL of a geofeed file. The format MUST be as in this example, "remarks: Geofeed " followed by a URL which will vary. inetnum: 192.0.2.0/24 # example remarks: Geofeed https://example.com/geofeed.csv Any particular inetnum: object MAY have, at most, one geofeed reference. is there more specific wording that would help clarify? note that use of the remarks: attribute is for rpsl services which do not implement the specific geofeed: attribute. [ e.g. see the PR for IIRDv4 https://github.com/irrdnet/irrd/issues/396 ]
Specific to the north american RIR, NetHandle is in registry (not in irr) abd bulk access requires application (https://www.arin.net/reference/research/bulkwhois/#accessing-bulk-whois-data), download requires cookies and takes ~10 mins. imo this could be made easier like https://ftp.ripe.net/ripe/dbase/split/ripe.db.inetnum.gz
there are a lot of things arin could make life more easy for operators. and cash could fall from the sky.
NetHandle is not exactly inetnum (e.g. comments instead of remarks field, different prefix format). Would the RIR provide converted inetnum objects or the users would be expected to handle this?
i currently fear a custom stub to do just this for consumers of north american data. randy
Why not publish RFC8805 Geofeed directly in inetnum remarks section?
for some flat fan out last kilometer providers that could be the inetnum: object from hell. there are global providers which segment large prefixes over diverse areas. etc.
i doubt the rpsl providers would like multi-megabyte inetnum:s. rpsl providers already throttle in defense.
I was thinking about geofeed on customer assignment objects for networks that manage their own objects (https://www.afrinic.net/press/214-creating-customer-assignments), only 1 line of geofeed remark needed on each object (more objects should be created if used in different locations). But not all RIR have customer assignment objects (can't create sub-assignment objects on ARIN direct assignment resources). HTTP feed does make sense when customer assignment object is not an option.
we are not expecting these lookups to be done frequently. i agree that would hammer servers, both rpsl and geofeed. do you have stronger words to suggest than
5. Operational Considerations ... An entity fetching geofeed data through these mechanisms MUST NOT do frequent real-time look-ups to prevent load on RPSL servers. And do not fetch at midnight, because everyone else may.
i agree that we do not want the DDoS that is currently happening to RPKI publication servers. perhaps explicit time limits, e.g daily? I am more concerned about clients having to make large amount of HTTP requests if this field gets widely used, maybe some clients can just read input like RPKI VRP (https://rpki.cloudflare.com/rpki.json) Client can try to keep track of geofeed URLs and only download the file during iteration, despite the same file referenced by multiple objects.
How to handle other stuff that might exist in remarks field? Or the draft would explicitly require Geofeed to be in its own remarks field?
the document tries to be explicit that it is the latter. that is the intent of
3. inetnum: Class ... the syntax of a Geofeed remarks: attribute which contains a URL of a geofeed file. The format MUST be as in this example, "remarks: Geofeed " followed by a URL which will vary. ---> probably add clarification here that Geofeed MUST be the only value in this particular remarks field, nothing before/after it
inetnum: 192.0.2.0/24 # example remarks: Geofeed https://example.com/geofeed.csv
Any particular inetnum: object MAY have, at most, one geofeed reference.
is there more specific wording that would help clarify?
see above Yang
we are not expecting these lookups to be done frequently. i agree that would hammer servers, both rpsl and geofeed. do you have stronger words to suggest than
5. Operational Considerations ... An entity fetching geofeed data through these mechanisms MUST NOT do frequent real-time look-ups to prevent load on RPSL servers. And do not fetch at midnight, because everyone else may.
i agree that we do not want the DDoS that is currently happening to RPKI publication servers. perhaps explicit time limits, e.g daily?
I am more concerned about clients having to make large amount of HTTP requests if this field gets widely used
i have a, possibly mistaken, model that the fetchers are few, content and similar providers such as netflix, news and sports media, ... and there are not many, maybe O(1,000). and they make a pass once a week or less frequently, as the data do not change much and their pushing data out to their infrastructure is not frequent. given O(10^5) inetnum:s with geofeed refs, and a sweep rate of once a week or less, that seems pretty trivial. the rpsl server damping could dominate.
3. inetnum: Class ... the syntax of a Geofeed remarks: attribute which contains a URL of a geofeed file. The format MUST be as in this example, "remarks: Geofeed " followed by a URL which will vary. ---> probably add clarification here that Geofeed MUST be the only value in this particular remarks field, nothing before/after it
between the MUST and the example, i do not see the wiggle room you fear. randy
the edit buffer, yet to be published, has Currently, the registry data published by ARIN is not RPSL; therefore, when fetching from ARIN whois, the "NetRange" attribute/ key must be treated as "inetnum" and the "Comment" attribute must be treated as "remarks". comments appreciated randy
On Mon, Sep 14, 2020 at 3:02 PM Randy Bush <randy@psg.com> wrote:
the edit buffer, yet to be published, has
Currently, the registry data published by ARIN is not RPSL;
therefore, when fetching from ARIN whois, the "NetRange" attribute/
key must be treated as "inetnum" and the "Comment" attribute must be
treated as "remarks".
comments appreciated
Is it possible to have one method work in all regions instead of a one off for arin ?
randy
the edit buffer, yet to be published, has
Currently, the registry data published by ARIN is not RPSL; therefore, when fetching from ARIN whois, the "NetRange" attribute/ key must be treated as "inetnum" and the "Comment" attribute must be treated as "remarks".
comments appreciated
Is it possible to have one method work in all regions instead of a one off for arin ?
rirs have a compatibility problem and arin is the worst. rirs' "stats" files, the data of record, are pretty incompatible, and we could not add data to them anyway. reverse dns delegations might be interesting except for the showstopper issues i recently posted in another message. that leaves whois and the rpsl-like data. unless i am missing something. < imagine snark here > randy
On Mon, Sep 14, 2020 at 5:53 PM Randy Bush <randy@psg.com> wrote:
the edit buffer, yet to be published, has
Currently, the registry data published by ARIN is not RPSL;
therefore, when fetching from ARIN whois, the "NetRange" attribute/
key must be treated as "inetnum" and the "Comment" attribute must be
treated as "remarks".
comments appreciated
Is it possible to have one method work in all regions instead of a one
off for arin ?
rirs have a compatibility problem and arin is the worst.
rirs' "stats" files, the data of record, are pretty incompatible, and we
could not add data to them anyway.
reverse dns delegations might be interesting except for the showstopper
issues i recently posted in another message.
that leaves whois and the rpsl-like data. unless i am missing
something.
< imagine snark here >
Sorry, let me be more specific. Would your arin approach of netrange work in all regions?
randy
On Mon, Sep 14, 2020 at 6:00 PM Randy Bush <randy@psg.com> wrote:
Would your arin approach of netrange work in all regions?
no. to the best of my knowledge, other regional registries and
independent irr registries use rpsl; i.e. inetnum: and remarks:.
Radb only supports route -route6 -aut-num -as-set -route-set
randy
Radb only supports
again, north america is a special snowflake. radb is a routing registry, not an allocation registry. inetnum: and NetRange: are allocation objects. if you get your address space from arin, you have to use their webby stuff to add the Comments: to the allocation object. ryuu.rg.net:/Users/randy> whois -h whois.arin.net 192.169.0.0 NetRange: 192.169.0.0 - 192.169.1.255 CIDR: 192.169.0.0/23 NetName: PSG169 NetHandle: NET-192-169-0-0-1 Parent: NET192 (NET-192-0-0-0-0) NetType: Direct Assignment OriginAS: Organization: RGnet, LLC (RGNETI-1) RegDate: 2005-04-12 Updated: 2020-09-14 Comment: Geofeed https://rg.net/geofeed Ref: https://rdap.arin.net/registry/ip/192.169.0.0 randy
participants (3)
-
Ca By -
Randy Bush -
Yang Yu