Sprint (1239) blackhole ? Or bogus /32 route ?
Hi, I am trying to figure out if either sprint (as1239) has blackholed a single IP address in my network or something strange is up. If anyone has transit connectivity to AS1239, can you tell me if Sprint is sending 199.212.134.9/32 as a prefix ?? e.g. from as1239's website looking glass http://oxide.sprintlink.net/cgi-bin/glass.pl (only a traceroute interface) sl-bb20-ana>trace 199.212.134.9 Type escape sequence to abort. Tracing the route to smtp2.sentex.ca (199.212.134.9) 1 * * * Yet, on that same subnet all else is fine sl-bb20-ana>trace 199.212.134.1 Type escape sequence to abort. Tracing the route to ns.sentex.ca (199.212.134.1) 1 sl-bb22-ana-14-0.sprintlink.net (144.232.1.177) 4 msec sl-bb23-fw-10-2.sprintlink.net (144.232.18.241) 24 msec sl-bb22-ana-14-0.sprintlink.net (144.232.1.177) 0 msec 2 sl-bb25-chi-6-0.sprintlink.net (144.232.9.25) 56 msec sl-bb22-fw-10-1.sprintlink.net (144.232.9.250) 24 msec sl-bb25-chi-6-0.sprintlink.net (144.232.9.25) 52 msec 3 sl-bb22-chi-11-0.sprintlink.net (144.232.18.121) 48 msec sl-bb25-chi-15-0.sprintlink.net (144.232.26.82) 52 msec sl-bb22-chi-11-0.sprintlink.net (144.232.18.121) 44 msec 4 sl-gw33-chi-10-0.sprintlink.net (144.232.26.42) 52 msec sl-gw33-chi-9-0.sprintlink.net (144.232.26.22) 60 msec sl-gw33-chi-10-0.sprintlink.net (144.232.26.42) 48 msec 5 sl-splk-telus-1-0.sprintlink.net (144.223.35.30) 48 msec 52 msec 48 msec 6 chcnil23gr01.bb.telus.com (154.11.11.90) [AS 852] 48 msec chcnil23gr01.bb.telus.com (154.11.11.94) [AS 852] 48 msec chcnil23gr01.bb.telus.com (154.11.11.90) [AS 852] 48 msec 7 toroonxnbr00.bb.telus.com (154.11.11.5) [AS 852] 56 msec 64 msec 56 msec 8 toroonzddr00.bb.telus.com (154.11.6.67) [AS 852] 64 msec 56 msec 64 msec 9 peer.toroonzddr00.bb.telus.com (209.115.141.5) [AS 852] 60 msec 64 msec 64 msec 10 iolite.sentex.ca (209.112.4.3) [AS 15290] 64 msec 60 msec 64 msec 11 ns.sentex.ca (199.212.134.1) [AS 11647] 64 msec 64 msec 60 msec sl-bb20-ana> I am guessing a blackhole, but I dont see where they told me or what list that IP address is on... www.openrbl.org shows clean and all the box does is outbound smtp... Anyone else see strange things like this ? ---Mike -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet since 1994 www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike
Here's what I see: BGP routing table entry for 199.212.134.0/24, version 5658446 Paths: (3 available, best #2, table Default-IP-Routing-Table) Advertised to peer-groups: tn-core 18984 3561 852 11647 216.182.0.33 (metric 2965760) from 216.182.0.33 (216.182.0.33) Origin IGP, localpref 100, valid, internal Community: 233373696 1244135434 1239 852 11647 144.228.242.224 from 144.228.242.224 (144.228.242.224) Origin IGP, localpref 100, valid, external, best 1239 852 11647, (received-only) 144.228.242.224 from 144.228.242.224 (144.228.242.224) Origin IGP, metric 49, localpref 100, valid, external core1-nwtnj#trace 199.212.134.9 Type escape sequence to abort. Tracing the route to smtp2.sentex.ca (199.212.134.9) 1 sl-gw32-pen-6-0-0-TS21.sprintlink.net (144.223.38.121) [AS 1239] 4 msec sl-gw32-pen-1-0-0-TS18.sprintlink.net (144.223.15.121) [AS 1239] 4 msec sl-gw32-pen-1-0-0-TS21.sprintlink.net (144.223.15.125) [AS 1239] 20 msec 2 sl-bb20-pen-0-0.sprintlink.net (144.232.16.241) [AS 1239] !H * !H Looks like something isn't right... I see the announcement from Sprint with an AS path of 1239 852 11647, but it never gets past one of the routers on Sprint's network. I have no problem going through Cable and Wireless: Type escape sequence to abort. Tracing the route to smtp2.sentex.ca (199.212.134.9) 1 63-121-101-106.focaldata.net (63.121.101.106) [AS 18984] 0 msec 0 msec 0 msec 2 acr2-so-3-3-0.newyork.cw.net (206.24.193.153) [AS 3561] 0 msec 4 msec 0 msec 3 agr4-loopback.newyork.cw.net (206.24.194.104) [AS 3561] 4 msec 0 msec agr3-loopback.newyork.cw.net (206.24.194.103) [AS 3561] 4 msec 4 dcr1-so-7-2-0.newyork.cw.net (206.24.207.73) [AS 3561] 4 msec dcr1-so-6-2-0.newyork.cw.net (206.24.207.57) [AS 3561] 0 msec dcr1-so-7-3-0.newyork.cw.net (206.24.207.77) [AS 3561] 4 msec 5 telus-services-inc.newyork.cw.net (206.24.207.90) [AS 3561] 24 msec 24 msec 20 msec 6 toroonnlbr00.bb.telus.com (154.11.11.130) [AS 852] 20 msec 24 msec 20 msec 7 toroonzddr00.bb.telus.com (154.11.6.67) [AS 852] 24 msec 24 msec 20 msec 8 peer.toroonzddr00.bb.telus.com (209.115.141.5) [AS 852] 28 msec 28 msec 32 msec 9 iolite.sentex.ca (209.112.4.3) [AS 15290] 24 msec 24 msec 24 msec 10 smtp2.sentex.ca (199.212.134.9) [AS 11647] 28 msec 24 msec 32 msec I would contact Sprint. Good luck! At 01:12 PM 9/26/2002 -0400, Mike Tancsa wrote:
Hi, I am trying to figure out if either sprint (as1239) has blackholed a single IP address in my network or something strange is up. If anyone has transit connectivity to AS1239, can you tell me if Sprint is sending 199.212.134.9/32 as a prefix ??
e.g. from as1239's website looking glass http://oxide.sprintlink.net/cgi-bin/glass.pl (only a traceroute interface)
sl-bb20-ana>trace 199.212.134.9
Type escape sequence to abort. Tracing the route to smtp2.sentex.ca (199.212.134.9)
1 * * *
Yet, on that same subnet all else is fine
sl-bb20-ana>trace 199.212.134.1
Type escape sequence to abort. Tracing the route to ns.sentex.ca (199.212.134.1)
1 sl-bb22-ana-14-0.sprintlink.net (144.232.1.177) 4 msec sl-bb23-fw-10-2.sprintlink.net (144.232.18.241) 24 msec sl-bb22-ana-14-0.sprintlink.net (144.232.1.177) 0 msec 2 sl-bb25-chi-6-0.sprintlink.net (144.232.9.25) 56 msec sl-bb22-fw-10-1.sprintlink.net (144.232.9.250) 24 msec sl-bb25-chi-6-0.sprintlink.net (144.232.9.25) 52 msec 3 sl-bb22-chi-11-0.sprintlink.net (144.232.18.121) 48 msec sl-bb25-chi-15-0.sprintlink.net (144.232.26.82) 52 msec sl-bb22-chi-11-0.sprintlink.net (144.232.18.121) 44 msec 4 sl-gw33-chi-10-0.sprintlink.net (144.232.26.42) 52 msec sl-gw33-chi-9-0.sprintlink.net (144.232.26.22) 60 msec sl-gw33-chi-10-0.sprintlink.net (144.232.26.42) 48 msec 5 sl-splk-telus-1-0.sprintlink.net (144.223.35.30) 48 msec 52 msec 48 msec 6 chcnil23gr01.bb.telus.com (154.11.11.90) [AS 852] 48 msec chcnil23gr01.bb.telus.com (154.11.11.94) [AS 852] 48 msec chcnil23gr01.bb.telus.com (154.11.11.90) [AS 852] 48 msec 7 toroonxnbr00.bb.telus.com (154.11.11.5) [AS 852] 56 msec 64 msec 56 msec 8 toroonzddr00.bb.telus.com (154.11.6.67) [AS 852] 64 msec 56 msec 64 msec 9 peer.toroonzddr00.bb.telus.com (209.115.141.5) [AS 852] 60 msec 64 msec 64 msec 10 iolite.sentex.ca (209.112.4.3) [AS 15290] 64 msec 60 msec 64 msec 11 ns.sentex.ca (199.212.134.1) [AS 11647] 64 msec 64 msec 60 msec sl-bb20-ana>
I am guessing a blackhole, but I dont see where they told me or what list that IP address is on... www.openrbl.org shows clean and all the box does is outbound smtp...
Anyone else see strange things like this ?
---Mike -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet since 1994 www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike
Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct) PGP Key Fingerprint: 3BC5 9A48 FC78 03D3 82E0 E935 5325 FBCB 0100 977A Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
At 01:31 PM 26/09/2002 -0400, Vinny Abello wrote:
Looks like something isn't right... I see the announcement from Sprint with an AS path of 1239 852 11647, but it never gets past one of the routers on Sprint's network. I have no problem going through Cable and Wireless:
Yes, and the strange thing is that is just one IP address :-( 199.212.134.9... If you try 199.212.134.1 I bet you can get to it via sprint.
Type escape sequence to abort. Tracing the route to smtp2.sentex.ca (199.212.134.9)
1 63-121-101-106.focaldata.net (63.121.101.106) [AS 18984] 0 msec 0 msec 0 msec 2 acr2-so-3-3-0.newyork.cw.net (206.24.193.153) [AS 3561] 0 msec 4 msec 0 msec 3 agr4-loopback.newyork.cw.net (206.24.194.104) [AS 3561] 4 msec 0 msec agr3-loopback.newyork.cw.net (206.24.194.103) [AS 3561] 4 msec 4 dcr1-so-7-2-0.newyork.cw.net (206.24.207.73) [AS 3561] 4 msec dcr1-so-6-2-0.newyork.cw.net (206.24.207.57) [AS 3561] 0 msec dcr1-so-7-3-0.newyork.cw.net (206.24.207.77) [AS 3561] 4 msec 5 telus-services-inc.newyork.cw.net (206.24.207.90) [AS 3561] 24 msec 24 msec 20 msec 6 toroonnlbr00.bb.telus.com (154.11.11.130) [AS 852] 20 msec 24 msec 20 msec 7 toroonzddr00.bb.telus.com (154.11.6.67) [AS 852] 24 msec 24 msec 20 msec 8 peer.toroonzddr00.bb.telus.com (209.115.141.5) [AS 852] 28 msec 28 msec 32 msec 9 iolite.sentex.ca (209.112.4.3) [AS 15290] 24 msec 24 msec 24 msec 10 smtp2.sentex.ca (199.212.134.9) [AS 11647] 28 msec 24 msec 32 msec
I would contact Sprint. Good luck!
Thanks, I did. Responder robot said they would try to get back to me in 72hrs :-( ---Mike
Thanks to all who have responding with information from their network vantage point. It does indeed seem to be an IGP or blackholing issue inside of Sprint. In the interim I made an advertising change to hopefully minimize the impact until I hear from someone at Sprint as to what the issue is. ---Mike At 01:35 PM 26/09/2002 -0400, Mike Tancsa wrote:
At 01:31 PM 26/09/2002 -0400, Vinny Abello wrote:
Looks like something isn't right... I see the announcement from Sprint with an AS path of 1239 852 11647, but it never gets past one of the routers on Sprint's network. I have no problem going through Cable and Wireless:
Yes, and the strange thing is that is just one IP address :-( 199.212.134.9... If you try 199.212.134.1 I bet you can get to it via sprint.
Type escape sequence to abort. Tracing the route to smtp2.sentex.ca (199.212.134.9)
1 63-121-101-106.focaldata.net (63.121.101.106) [AS 18984] 0 msec 0 msec 0 msec 2 acr2-so-3-3-0.newyork.cw.net (206.24.193.153) [AS 3561] 0 msec 4 msec 0 msec 3 agr4-loopback.newyork.cw.net (206.24.194.104) [AS 3561] 4 msec 0 msec agr3-loopback.newyork.cw.net (206.24.194.103) [AS 3561] 4 msec 4 dcr1-so-7-2-0.newyork.cw.net (206.24.207.73) [AS 3561] 4 msec dcr1-so-6-2-0.newyork.cw.net (206.24.207.57) [AS 3561] 0 msec dcr1-so-7-3-0.newyork.cw.net (206.24.207.77) [AS 3561] 4 msec 5 telus-services-inc.newyork.cw.net (206.24.207.90) [AS 3561] 24 msec 24 msec 20 msec 6 toroonnlbr00.bb.telus.com (154.11.11.130) [AS 852] 20 msec 24 msec 20 msec 7 toroonzddr00.bb.telus.com (154.11.6.67) [AS 852] 24 msec 24 msec 20 msec 8 peer.toroonzddr00.bb.telus.com (209.115.141.5) [AS 852] 28 msec 28 msec 32 msec 9 iolite.sentex.ca (209.112.4.3) [AS 15290] 24 msec 24 msec 24 msec 10 smtp2.sentex.ca (199.212.134.9) [AS 11647] 28 msec 24 msec 32 msec
I would contact Sprint. Good luck!
Thanks, I did. Responder robot said they would try to get back to me in 72hrs :-(
---Mike
Yep, you're right. Looks like they might blackholing the /32 with a null route on their network somewhere. At 01:35 PM 9/26/2002 -0400, Mike Tancsa wrote:
At 01:31 PM 26/09/2002 -0400, Vinny Abello wrote:
Looks like something isn't right... I see the announcement from Sprint with an AS path of 1239 852 11647, but it never gets past one of the routers on Sprint's network. I have no problem going through Cable and Wireless:
Yes, and the strange thing is that is just one IP address :-( 199.212.134.9... If you try 199.212.134.1 I bet you can get to it via sprint.
Type escape sequence to abort. Tracing the route to smtp2.sentex.ca (199.212.134.9)
1 63-121-101-106.focaldata.net (63.121.101.106) [AS 18984] 0 msec 0 msec 0 msec 2 acr2-so-3-3-0.newyork.cw.net (206.24.193.153) [AS 3561] 0 msec 4 msec 0 msec 3 agr4-loopback.newyork.cw.net (206.24.194.104) [AS 3561] 4 msec 0 msec agr3-loopback.newyork.cw.net (206.24.194.103) [AS 3561] 4 msec 4 dcr1-so-7-2-0.newyork.cw.net (206.24.207.73) [AS 3561] 4 msec dcr1-so-6-2-0.newyork.cw.net (206.24.207.57) [AS 3561] 0 msec dcr1-so-7-3-0.newyork.cw.net (206.24.207.77) [AS 3561] 4 msec 5 telus-services-inc.newyork.cw.net (206.24.207.90) [AS 3561] 24 msec 24 msec 20 msec 6 toroonnlbr00.bb.telus.com (154.11.11.130) [AS 852] 20 msec 24 msec 20 msec 7 toroonzddr00.bb.telus.com (154.11.6.67) [AS 852] 24 msec 24 msec 20 msec 8 peer.toroonzddr00.bb.telus.com (209.115.141.5) [AS 852] 28 msec 28 msec 32 msec 9 iolite.sentex.ca (209.112.4.3) [AS 15290] 24 msec 24 msec 24 msec 10 smtp2.sentex.ca (199.212.134.9) [AS 11647] 28 msec 24 msec 32 msec
I would contact Sprint. Good luck!
Thanks, I did. Responder robot said they would try to get back to me in 72hrs :-(
---Mike
Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct) PGP Key Fingerprint: 3BC5 9A48 FC78 03D3 82E0 E935 5325 FBCB 0100 977A Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
At 02:27 PM 26/09/2002 -0400, Vinny Abello wrote:
Yep, you're right. Looks like they might blackholing the /32 with a null route on their network somewhere.
To mitigate the impact, I am sending 199.212.134.0/24 as a more specific route through my other transit provider (15290) who does not transit with 1239. I am trying to limit the damage to just inside 1239 and those single homed off 1239. I am sending 199.212.134.0/23 through Telus (852) who also has transit with AS1239. Someone else told me off list that Sprint usually blackholes with ACLs and not NULL0 routing. So perhaps an IGP issue ? If so I would have thought others would be seeing strange things as well. ---Mike
participants (2)
-
Mike Tancsa -
Vinny Abello