Hi Job,

Thank you for this good analysis and for sharing your findings.
The issue has since been fixed and the team will publish a post-mortem accordingly once we are done with making sure the issue will not reappear.
Your recommendation is well noted and I cc my colleague so that they can take that into consideration in our improvement roadmap.
Best regards,

==============================
Cedrick Adrien MBEYET
Ebene Cybercity, Mauritius
+230 5851 7674

+++ Never give up, Keep moving forward +++


On Sun, Nov 20, 2022 at 3:49 PM Job Snijders via NANOG <nanog@nanog.org> wrote:
Hi all,

It appears PacketVis correctly identified an issue.

AFRINIC's self-signed root AfriNIC.cer [1] points via its SIA to
'afrinic-ca.cer' [2] which in turn references a RPKI Manifest named
'K1eJenypZMPIt_e92qek2jSpj4A.mft'.

The K1eJenypZMPIt_e92qek2jSpj4A Manifest lists 499 Certificate
Authorities. This Manifest represents the demarcation point between
"Afrinic as root CA operator" and "Afrinic hosting rpki on behalf of its
members". In other words; this is an important top-level Manifest in the
critical path towards the ROAs of the Afrinic members.

There was a ~ 7 hour gap in the validity window of this Manifest and its
companion CRL (from 20221120T000311Z until 20221120T071514Z). The
serials 1E19 and 1E1A (respectively 12B2 and 12B3) are successive.

rpki.afrinic.net/repository/afrinic/K1eJenypZMPIt_e92qek2jSpj4A.crl
    CRL Serial Number:        1E19
    CRL valid since:          Nov 18 00:03:11 2022 GMT
    CRL valid until:          Nov 20 00:03:11 2022 GMT

    CRL Serial Number:        1E1A
    CRL valid since:          Nov 20 07:15:12 2022 GMT
    CRL valid until:          Nov 22 07:15:12 2022 GMT

rpki.afrinic.net/repository/afrinic/K1eJenypZMPIt_e92qek2jSpj4A.mft
    Manifest Number:          12B2
    Manifest valid since:     Nov 18 00:03:13 2022 GMT
    Manifest valid until:     Nov 20 00:03:13 2022 GMT

    Manifest Number:          12B3
    Manifest valid since:     Nov 20 07:15:14 2022 GMT
    Manifest valid until:     Nov 22 07:15:14 2022 GMT

(The above can be reconstructed using archives from http://www.rpkiviews.org)

The rcynic validator hosted at Afrinic also noticed a gap in objects:
https://validator.afrinic.net/rpki/rcynic/rpki.afrinic.net_week_svg.html

A possible recommendation might be to increase the validity window of
these two objects from a sliding 48-hour window to a 1 or 2 week window.
This way any stalling in the issuance process wouldn't case operational
issues on the weekend.

Kind regards,

Job

[1]: SKI EB:68:0F:38:F5:D6:C7:1B:B4:B1:06:B8:BD:06:58:50:12:DA:31:B6
[2]: SKI 2B:57:89:7A:7C:A9:64:C3:C8:B7:F7:BD:DA:A7:A4:DA:34:A9:8F:80



On Sat, Nov 19, 2022 at 08:36:23PM -0800, Randy Bush wrote:
> From: PacketVis <notifications@packetvis.com>
> Date: Sun, 20 Nov 2022 04:30:44 +0000
>
> Possible TA malfunction or incomplete VRP file: 73.95% of the ROAs disappeared from afrinic
>
> See more details about the event:
> https://packetvis.com/#/bgp/event/905ec8b7d37e89a2d7b547bca99fd57e-372b0bf3-9056-407e-9e8d-e986567155fc/4f309cb51ba9314fafa64da53d007e342faca613