Hi Job, Thank you for this good analysis and for sharing your findings. The issue has since been fixed and the team will publish a post-mortem accordingly once we are done with making sure the issue will not reappear. Your recommendation is well noted and I cc my colleague so that they can take that into consideration in our improvement roadmap. Best regards, ============================== Cedrick Adrien MBEYET Ebene Cybercity, Mauritius +230 5851 7674 +++ Never give up, Keep moving forward +++ On Sun, Nov 20, 2022 at 3:49 PM Job Snijders via NANOG <nanog@nanog.org> wrote:
Hi all,
It appears PacketVis correctly identified an issue.
AFRINIC's self-signed root AfriNIC.cer [1] points via its SIA to 'afrinic-ca.cer' [2] which in turn references a RPKI Manifest named 'K1eJenypZMPIt_e92qek2jSpj4A.mft'.
The K1eJenypZMPIt_e92qek2jSpj4A Manifest lists 499 Certificate Authorities. This Manifest represents the demarcation point between "Afrinic as root CA operator" and "Afrinic hosting rpki on behalf of its members". In other words; this is an important top-level Manifest in the critical path towards the ROAs of the Afrinic members.
There was a ~ 7 hour gap in the validity window of this Manifest and its companion CRL (from 20221120T000311Z until 20221120T071514Z). The serials 1E19 and 1E1A (respectively 12B2 and 12B3) are successive.
rpki.afrinic.net/repository/afrinic/K1eJenypZMPIt_e92qek2jSpj4A.crl CRL Serial Number: 1E19 CRL valid since: Nov 18 00:03:11 2022 GMT CRL valid until: Nov 20 00:03:11 2022 GMT
CRL Serial Number: 1E1A CRL valid since: Nov 20 07:15:12 2022 GMT CRL valid until: Nov 22 07:15:12 2022 GMT
rpki.afrinic.net/repository/afrinic/K1eJenypZMPIt_e92qek2jSpj4A.mft Manifest Number: 12B2 Manifest valid since: Nov 18 00:03:13 2022 GMT Manifest valid until: Nov 20 00:03:13 2022 GMT
Manifest Number: 12B3 Manifest valid since: Nov 20 07:15:14 2022 GMT Manifest valid until: Nov 22 07:15:14 2022 GMT
(The above can be reconstructed using archives from http://www.rpkiviews.org)
The rcynic validator hosted at Afrinic also noticed a gap in objects: https://validator.afrinic.net/rpki/rcynic/rpki.afrinic.net_week_svg.html
A possible recommendation might be to increase the validity window of these two objects from a sliding 48-hour window to a 1 or 2 week window. This way any stalling in the issuance process wouldn't case operational issues on the weekend.
Kind regards,
Job
[1]: SKI EB:68:0F:38:F5:D6:C7:1B:B4:B1:06:B8:BD:06:58:50:12:DA:31:B6 [2]: SKI 2B:57:89:7A:7C:A9:64:C3:C8:B7:F7:BD:DA:A7:A4:DA:34:A9:8F:80
On Sat, Nov 19, 2022 at 08:36:23PM -0800, Randy Bush wrote:
From: PacketVis <notifications@packetvis.com> Date: Sun, 20 Nov 2022 04:30:44 +0000
Possible TA malfunction or incomplete VRP file: 73.95% of the ROAs disappeared from afrinic
See more details about the event:
https://packetvis.com/#/bgp/event/905ec8b7d37e89a2d7b547bca99fd57e-372b0bf3-...