Hi there, I’m contacting you because after spending 2 days troubleshooting I can’t seem to find a solution to the following. We (AS45021) bought/transffered the 86.104.228.0/24 prefix a few months back because we couldn’t wait longer on the RIPE waiting list. Before you ask, yes, AS45021 is currently single homed, this will change in a week (it requires travelling a few hundred miles and I couldn’t do it before). Since we started announcing this prefix, things have been spotty, at best. While it seems visible in all the looking glasses I tried, it spends sometimes hours, sometimes days, being unreachable (you can try for ex. 86.104.228.1 or 86.104.228.26). I have full access (up to packet capture) on the AS and its upstream. When I ping one of the IPs from various ISPs, I see the ICMP Echo Request and Reply on the wire, going where it’s supposed to go, but it doesn’t reach the pinging host. Pinging any IP of the upstream (AS42275 / 85.208.69.0/24 in this location) works. ROAs and RPKI seem fine to me. I’m starting to suspect that maybe the previous user of the prefix is still announcing it somewhere and “shouting louder” than me. It seems when I clear sessions, it immediately works for a while, then stops. Do you all have any idea what I should check / try next? BR, Michel
One more thing: it seems that no matter what, the prefix is always reachable from AS3257 which makes the whole thing even weirder.
On 25 Mar 2023, at 09:54, ic <lists@benappy.com> wrote:
Hi there,
I’m contacting you because after spending 2 days troubleshooting I can’t seem to find a solution to the following.
We (AS45021) bought/transffered the 86.104.228.0/24 prefix a few months back because we couldn’t wait longer on the RIPE waiting list.
Before you ask, yes, AS45021 is currently single homed, this will change in a week (it requires travelling a few hundred miles and I couldn’t do it before).
Since we started announcing this prefix, things have been spotty, at best. While it seems visible in all the looking glasses I tried, it spends sometimes hours, sometimes days, being unreachable (you can try for ex. 86.104.228.1 or 86.104.228.26).
I have full access (up to packet capture) on the AS and its upstream. When I ping one of the IPs from various ISPs, I see the ICMP Echo Request and Reply on the wire, going where it’s supposed to go, but it doesn’t reach the pinging host. Pinging any IP of the upstream (AS42275 / 85.208.69.0/24 in this location) works.
ROAs and RPKI seem fine to me.
I’m starting to suspect that maybe the previous user of the prefix is still announcing it somewhere and “shouting louder” than me. It seems when I clear sessions, it immediately works for a while, then stops.
Do you all have any idea what I should check / try next?
BR, Michel
On Sat, Mar 25, 2023 at 1:54 AM ic <lists@benappy.com> wrote:
Do you all have any idea what I should check / try next?
A good tool for diagnosing BGP problems is: https://www.routeviews.org/routeviews/ While the problem is occurring, pick some of the collector hosts from https://www.routeviews.org/routeviews/index.php/collectors/ and telnet to them. This will drop you into a Cisco-like CLI where you can "show ip bgp 86.104.228.0" and find out what the BGP path to your network is from a bunch of points around the world. This should help you identify the fault if the echo-request from 86.104.228.1 reaches the remote host but the echo reply from the remote host doesn't make it back to 86.104.228.1.
When I ping one of the IPs from various ISPs, I see the ICMP Echo Request and Reply on the wire, going where it’s supposed to go, but it doesn’t reach the pinging host.
The echo-request reaches your host at 86.104.228.1 but the echo-reply doesn't reach the pinging host? That sounds more like a packet filtering problem than a BGP problem. Try doing a traceroute to the remote pinging host from two sources: 86.104.228.1 and one of your ISP's IP addresses (get them to assign you one if you don't have one). The difference between the two may give you an idea where the filtering error is. Regards, Bill Herrin -- For hire. https://bill.herrin.us/resume/
yeah i see what you mean by, it doesn't work, then it starts working... i traced to it, and it wasn't responding at first, then later it worked C:\>tracert -w 1 86.104.228.1 Tracing route to 86.104.228.1 over a maximum of 30 hops ... 9 118 ms * 119 ms prs-bb1-link.ip.twelve99.net [62.115.112.243] 10 125 ms 124 ms 126 ms ffm-bb1-link.ip.twelve99.net [62.115.123.12] 11 * * * Request timed out. 12 * * * Request timed out. 13 133 ms 133 ms 133 ms ipmax-ic340750-zch-b2.ip.twelve99-cust.net [62.115.168.201] 14 130 ms * 130 ms po5.er01.zrh56.ch.ip-max.net [46.20.254.13] 15 128 ms 129 ms 129 ms three-fourteen.cust.zrh56.ch.ip-max.net [46.20.240.71] 16 * * * Request timed out. 17 * * * Request timed out. 18 * * * Request timed out. 19 * * * Request timed out. 20 * * * Request timed out. 21 * * * Request timed out. 22 * * * Request timed out. 23 * * * Request timed out. 24 * * * Request timed out. 25 * * * Request timed out. 26 * * * Request timed out. 27 * * * Request timed out. 28 * * * Request timed out. 29 * * * Request timed out. 30 * * * Request timed out. Trace complete. C:\>tracert -w 1 86.104.228.1 Tracing route to 86.104.228.1 over a maximum of 30 hops ... 9 119 ms 118 ms 118 ms prs-bb1-link.ip.twelve99.net [62.115.112.243] 10 * 125 ms 124 ms ffm-bb1-link.ip.twelve99.net [62.115.123.12] 11 * * * Request timed out. 12 * * * Request timed out. 13 132 ms 132 ms 133 ms ipmax-ic340750-zch-b2.ip.twelve99-cust.net [62.115.168.201] 14 129 ms * 129 ms po5.er01.zrh56.ch.ip-max.net [46.20.254.13] 15 129 ms 129 ms 129 ms three-fourteen.cust.zrh56.ch.ip-max.net [46.20.240.71] 16 129 ms * 129 ms 86.104.228.1 Trace complete. C:\> On 3/25/2023 3:54 AM, ic wrote:
Hi there,
I’m contacting you because after spending 2 days troubleshooting I can’t seem to find a solution to the following.
We (AS45021) bought/transffered the 86.104.228.0/24 prefix a few months back because we couldn’t wait longer on the RIPE waiting list.
Before you ask, yes, AS45021 is currently single homed, this will change in a week (it requires travelling a few hundred miles and I couldn’t do it before).
Since we started announcing this prefix, things have been spotty, at best. While it seems visible in all the looking glasses I tried, it spends sometimes hours, sometimes days, being unreachable (you can try for ex. 86.104.228.1 or 86.104.228.26).
I have full access (up to packet capture) on the AS and its upstream. When I ping one of the IPs from various ISPs, I see the ICMP Echo Request and Reply on the wire, going where it’s supposed to go, but it doesn’t reach the pinging host. Pinging any IP of the upstream (AS42275 / 85.208.69.0/24 in this location) works.
ROAs and RPKI seem fine to me.
I’m starting to suspect that maybe the previous user of the prefix is still announcing it somewhere and “shouting louder” than me. It seems when I clear sessions, it immediately works for a while, then stops.
Do you all have any idea what I should check / try next?
BR, Michel
-- -Aaron
IRR Explorer is showing RPKI-Invalid. Maybe RPKI is causing the issue or there is an issue with IRR Explorer? https://irrexplorer.nlnog.net/prefix/86.104.228.0/24 I do see RIPE and Cloudflare are showing RPKI as valid. https://rpki-validator.ripe.net/ui/86.104.228.0%2F24/45021?include=related_a... https://rpki.cloudflare.com/?view=validator&validateRoute=45021_86.104.228.0%2F24 Curious why IRR Explorer is showing invalid. Thank you, Kevin McCormick -----Original Message----- From: NANOG <nanog-bounces+kmccormick=mdtc.net@nanog.org> On Behalf Of ic Sent: Saturday, March 25, 2023 3:55 AM To: nanog@nanog.org Subject: Issues with prefix / help needed CAUTION: This email originated from outside your organization. Exercise caution when opening attachments or clicking links, especially from unknown senders. Hi there, I’m contacting you because after spending 2 days troubleshooting I can’t seem to find a solution to the following. We (AS45021) bought/transffered the 86.104.228.0/24 prefix a few months back because we couldn’t wait longer on the RIPE waiting list. Before you ask, yes, AS45021 is currently single homed, this will change in a week (it requires travelling a few hundred miles and I couldn’t do it before). Since we started announcing this prefix, things have been spotty, at best. While it seems visible in all the looking glasses I tried, it spends sometimes hours, sometimes days, being unreachable (you can try for ex. 86.104.228.1 or 86.104.228.26). I have full access (up to packet capture) on the AS and its upstream. When I ping one of the IPs from various ISPs, I see the ICMP Echo Request and Reply on the wire, going where it’s supposed to go, but it doesn’t reach the pinging host. Pinging any IP of the upstream (AS42275 / 85.208.69.0/24 in this location) works. ROAs and RPKI seem fine to me. I’m starting to suspect that maybe the previous user of the prefix is still announcing it somewhere and “shouting louder” than me. It seems when I clear sessions, it immediately works for a while, then stops. Do you all have any idea what I should check / try next? BR, Michel
On Mon, Mar 27, 2023 at 9:05 AM Kevin McCormick <kmccormick@mdtc.net> wrote:
IRR Explorer is showing RPKI-Invalid. Maybe RPKI is causing the issue or there is an issue with IRR Explorer?
https://irrexplorer.nlnog.net/prefix/86.104.228.0/24
I do see RIPE and Cloudflare are showing RPKI as valid.
https://rpki-validator.ripe.net/ui/86.104.228.0%2F24/45021?include=related_a...
https://rpki.cloudflare.com/?view=validator&validateRoute=45021_86.104.228.0%2F24
Curious why IRR Explorer is showing invalid.
Thank you,
Kevin McCormick
That seems to just be indicating there are route-objects in RADB that don't match RPKI, and not related to anything in BGP.
* charles.lists@camonson.com (Charles Monson) [Mon 27 Mar 2023, 16:31 CEST]:
On Mon, Mar 27, 2023 at 9:05 AM Kevin McCormick <kmccormick@mdtc.net> wrote:
IRR Explorer is showing RPKI-Invalid. Maybe RPKI is causing the issue or there is an issue with IRR Explorer? https://irrexplorer.nlnog.net/prefix/86.104.228.0/24
I do see RIPE and Cloudflare are showing RPKI as valid. https://rpki-validator.ripe.net/ui/86.104.228.0%2F24/45021?include=related_a... https://rpki.cloudflare.com/?view=validator&validateRoute=45021_86.104.228.0%2F24 Curious why IRR Explorer is showing invalid.
That seems to just be indicating there are route-objects in RADB that don't match RPKI, and not related to anything in BGP.
It shows all green right now, perhaps RADb removed an object? (or IRR Explorer updated its own mirror between your mail and mine) -- Niels.
hi,
On 27 Mar 2023, at 21:49, Niels Bakker <niels=nanog@bakker.net> wrote:
It shows all green right now, perhaps RADb removed an object? (or IRR Explorer updated its own mirror between your mail and mine)
Not sure my last reply reached the list, it ended up being an ingress filter on one of our upstreams interface (NOT BGP filter, but packet filter). That’s why we observed packets disappearing in this particular link without explanation. Filter has been updated, all good now! Regards, Michel
participants (6)
-
Aaron Gould -
Charles Monson -
ic -
Kevin McCormick -
Niels Bakker -
William Herrin