- NANOG - lists.nanog.org

Looking for GBLX contact + check your ACL for 66.249.96.0/20 please :)
by Swen Wulf 16 Feb '09

16 Feb '09

Hello, We have been assigned 66.249.96.0/20 in April of 2008. Little did we know that this network was used by a spammer in 2004 (Lightwave Transit) and with this is deeply embedded in a lot of ACL's that seem to be unmaintained. If a GLBX engineer could take a look, it seems traffic originating from 66.249.96.0/20 can't reach some website when passing thru GBLX. Other prefixes we announce from our AS (22442) such as 66.60.224.0/20 don't have this problem. Customer with source of 66.249.106.112/27 can't get to the website of www.ups.com (72.246.89.243 (akamai)). core1#traceroute 72.246.89.243 source 66.249.106.113 Type escape sequence to abort. Tracing the route to a72-246-89-243.deploy.akamaitechnologies.com (72.246.89.243) 1 ge-1-11.r03.hstntx01.us.bb.gin.ntt.net (128.241.5.1) 0 msec 0 msec 0 msec 2 xe-0-1-0.r20.hstntx01.us.bb.gin.ntt.net (129.250.2.228) [AS 2914] 0 msec 0 msec 0 msec 3 p64-1-3-0.r20.dllstx09.us.bb.gin.ntt.net (129.250.3.129) [AS 2914] 8 msec 8 msec 8 msec 4 po-2.r03.dllstx09.us.bb.gin.ntt.net (129.250.4.38) [AS 2914] 4 msec 4 msec 8 msec 5 xe-0.globalcrossing.dllstx09.us.bb.gin.ntt.net (129.250.8.190) [AS 2914] 4 msec 8 msec 4 msec 6 te7-1-10G.ar2.DAL2.gblx.net (67.16.129.117) [AS 3549] 8 msec 12 msec 4 msec 7 * * * 8 * * And from a different source IP from the same router core1#traceroute 72.246.89.243 source 66.60.233.1 Type escape sequence to abort. Tracing the route to a72-246-89-243.deploy.akamaitechnologies.com (72.246.89.243) 1 ge-1-11.r03.hstntx01.us.bb.gin.ntt.net (128.241.5.1) 4 msec 0 msec 0 msec 2 xe-0-1-0.r20.hstntx01.us.bb.gin.ntt.net (129.250.2.228) [AS 2914] 0 msec 4 msec 0 msec 3 p64-1-3-0.r20.dllstx09.us.bb.gin.ntt.net (129.250.3.129) [AS 2914] 8 msec 8 msec 8 msec 4 po-2.r03.dllstx09.us.bb.gin.ntt.net (129.250.4.38) [AS 2914] 4 msec 8 msec * 5 xe-0.globalcrossing.dllstx09.us.bb.gin.ntt.net (129.250.8.190) [AS 2914] 4 msec 8 msec 8 msec 6 te7-1-10G.ar2.DAL2.gblx.net (67.16.129.117) [AS 3549] 4 msec 8 msec 8 msec 7 PCCW-Global-Chicago-Dallas.te3-2.ar2.DAL2.gblx.net (64.211.211.186) [AS 3549] 200 msec 24 msec 212 msec 8 * * * 9 I tried an email to their puck noc email address, but without being a customer I can't open a ticket. Thanks, Swen Wulf Network Security Manager Email: swulf(a)phonoscope.com Phone: 832-615-7743 (direct) Fax: 832-213-0110 --- * Network Operations Center (NOC): 713-272-4600 (24x7x365) --- Phonoscope - moving at the speed of light

1 0

3/11 (invalid or corrupt AS path)
by Ozar 16 Feb '09

16 Feb '09

I am starting to see random BGP neighbor messages from multiple neighbors on different boxes. %BGP-3-NOTIFICATION: received from neighbor X.X.X.X 3/11 (invalid or corrupt AS path) 516 bytes I dont see much documentation on this, and we are in the process of opening a TAC case, just curious if anyone else has seen these and may be able to shed some light. Thanks

3 2

Littering the global BGP - AS 47868
by Ran Liebermann 16 Feb '09

16 Feb '09

Hi all, We're seeing some garbage being thrown into the global BGP table by AS 47868. %BGP-6-ASPATH: Long AS path 3549 3257 29113 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 received from 208.49.128.49: Has more than 255 AS This is hitting some routers with older software versions. Is there someone here from AS 29113 (Sloane Park Property Trust in the Czech Repulic) that can filter this from the source? Cheers, -- Ran Liebermann VP Engineering, PurePeak ranl(a)purepeak.com http://purepeak.com

1 0

Re: Linux Router: TCP slow, UDP fast
by Chris 15 Feb '09

15 Feb '09

Thanks, Karl, Allen and Nickola. I failed-over to another router last night and briefly had full expected throughput but this morning despite dropping providers and moving between routers again for trial and error I still see _outbound_ TCP at about the same 300 - 600kbps per session. I eliminated conntract modules firstly, then iptables as a whole. I've eliminated TSO and checksumming (which caused very sticky connections) on the e1000 NIC. The failover router has a slightly older kernel and was working before Christmas so it's not most likely not kernel versions. I've also tried removing FIB_TRIE as a stab in the dark with no success. And the failover router connects using FE not GE so I've eliminated NICs and connection speeds to a front-facing switch. The only constant is the front-facing switch (it's negotiating perfectly at FD though) so all I can think of is removing that from the equation. It's definitely only _outbound_ TCP getting buffered though ! I've pushed 92Mbps on a FE link with UDP and uploaded at 16Mbps on a 16Mbps link. Any last ideas appreciated before causing headaches removing switches would be appreciated. Thanks, Chris

2 1

Re: Linux Router: TCP slow, UDP fast
by Chris 15 Feb '09

15 Feb '09

Thanks for the suggestions everyone. I've got to the bottom of the problem now (I'm sure there will be a collective sigh of relief from the list because of the noise this thread generated :-)). I installed two brand new, low spec, 3Com switches one at the 'front' of the network and one 'behind' the routers. They are the same model, same latest firmware, same config (saved to and then copied off disk) and only their IPs were different. The front switch was the problem. As two final tests before removing it we switched off unicast/multicast broadcast control and flow control (and simultaneously on the same port on the switch behind the routers) because there were pause frames showing but not a massive amount in terms of percentage. The switch behind the routers however is serving the same bandwidth equally well ! We've put an ancient switch in place of the front switch and its been working perfectly so far. My lesson learned is too change as little as possible at once ! That said, recent network changes were spread about a month apart and this very odd issue was far easier to dismiss than believe due to its bizarre nature. Especially when providers have changes in their network conditions as testing is done. I really appreciate all the input and have learnt loads, possibly just not in the way I would have liked to :-) Doubles all round, Chris

1 0

Re: Linux Router: TCP slow, UDP fast
by Mathias Wolkert 15 Feb '09

15 Feb '09

I would turn off ethernet flow control. Maybe you already have. It can be really mean on tcp's own flow control if the switch has an issue of some kind (load). /Tias 15 feb 2009 kl. 10.24 skrev Chris <chris(a)ghostbusters.co.uk>: > Thanks, Karl, Allen and Nickola. > I failed-over to another router last night and briefly had full > expected > throughput but this morning despite dropping providers and moving > between > routers again for trial and error I still see _outbound_ TCP at > about the > same 300 - 600kbps per session. > > I eliminated conntract modules firstly, then iptables as a whole. I've > eliminated TSO and checksumming (which caused very sticky > connections) on > the e1000 NIC. > > The failover router has a slightly older kernel and was working before > Christmas so it's not most likely not kernel versions. I've also tried > removing FIB_TRIE as a stab in the dark with no success. And the > failover > router connects using FE not GE so I've eliminated NICs and connection > speeds to a front-facing switch. > > The only constant is the front-facing switch (it's negotiating > perfectly at > FD though) so all I can think of is removing that from the equation. > > It's definitely only _outbound_ TCP getting buffered though ! I've > pushed > 92Mbps on a FE link with UDP and uploaded at 16Mbps on a 16Mbps link. > > Any last ideas appreciated before causing headaches removing > switches would > be appreciated. > > Thanks, > > Chris

1 0

Re: Global Blackhole Service
by Nuno Vieira - nfsi telecom 15 Feb '09

15 Feb '09

Hi Suresh, But in the meanwhile, a decade later, it does not longer exist. At least, i can't reach that host, and i was unable to find working documentation on google of how about this project works, today. In fact, the first link that google gave out, says that this project is dead at least 2 years ago. http://www.dnsbl.com/2007/02/status-of-rblmapsvixcom-invalid-domain.html I think that we all have a real opportunity here for make something that can be useful to all. And, we are not talk of spam here, but, to mitigate time, money and patience consuming DDoS attacks, which often are easier to mitigate only at the Source and at the Destination, while at Destinatation, sink is the only viable solution that is out there today. regards, --- Nuno Vieira nfsi telecom, lda. nuno.vieira(a)nfsi.pt Tel. (+351) 21 949 2300 - Fax (+351) 21 949 2301 http://www.nfsi.pt/ ----- "Suresh Ramasubramanian" <ops.lists(a)gmail.com> wrote: > On Fri, Feb 13, 2009 at 8:27 PM, Jens Ott - PlusServer AG > <j.ott(a)plusserver.de> wrote: > > - - What do you think about such service? > > - - Would you/your ASN participate in such a service? > > - - Do you see some kind of usefull feature in such a service? > > - - Do you have any comments? > > Ah. rbl.maps.vix.com from about a decade back when it was available > as > a bgp feed. But only for ddos sources. > > srs

10 14

Re: Linux Router: TCP slow, UDP fast
by Chris 14 Feb '09

14 Feb '09

Thanks for all the answers. I'm currently going down the path of looking at IPtables' conntrack slowing the forwarding rate. If I can't find any more docs then I'll boot the router with a kernel without any IPtables built-in and see if that's it. As Lee said "rollback" ! That's the last change to the box. If I can rule out the logging of traffic from conntrack is slowing down the forwarding then I can look into hardware further ;-) Chris

2 2

Linux Router: TCP slow, UDP fast
by Chris 14 Feb '09

14 Feb '09

Hi All, I'm losing the will to live with this networking headache ! Please feel free to point me at a Linux list if NANOG isn't suitable. I'm at a loss where else to ask. I've diagnosed some traffic oddities and after lots of head-scratching, reading and trial and error I can say with certainty that: With and without shaping and over different bandwidth providers using the e1000 driver for an Intel PRO/1000 MT Dual Port Gbps NIC (82546EB) I can replicate full, expected throughput with UDP but consistently only get 300kbps - 600kbps throughput _per connection_ for outbound TCP (I couldn't find a tool I trusted to replicate ICMP traffic). Multiple connections are cumulative and increase incrementally at roughly 300kbps - 600kbps. Inbound seems slightly erratic in holding a consistent speed but manages 15Mbps as expected, a far cry from 300kbps to 600kbps. The router is Quad Core sitting at no load and there's very little traffic being forwarded back and forth. The NIC's kernel parameters are set at default as 'built-in'. NAPI is not enabled though (enabling it requires a reboot which is a problem as this box is in production). The only other change to the box is that over Christmas IPtables (ip_conntrack and its associated modules mainly) was loaded into the kernel as 'built-in'. There's no sign of packet loss on any tests and I upped the conntrack max_connections size suitably for the amount of RAM. Has anyone come across IPtables without any rules loaded causing throughput issues ? I've also changed the following kernel parameters with no luck: net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 net.ipv4.tcp_no_metrics_save = 1 net.core.netdev_max_backlog = 2500 echo 0 > /proc/sys/net/ipv4/tcp_window_scaling It feels to me like a buffer limit is being reached 'per connection'. The throughput spikes at around 1.54Mbps and TCP backs off to about 300kbps - 600kbps or so. What am I missing ? Is NAPI that essential for such low traffic ? A very similar build moved far higher throughput on cheap NICs. MTU is at 1500, txqueuelen is 1000. Any help would be massively appreciated ! Chris

5 13

Re: One /22 Two ISP no BGP
by Charles Regan 14 Feb '09

14 Feb '09

The problem we have now is that we got our /22 from arin to do multihoming. If we dump tlb, no more multihoming? No /22. Is that correct? We also have a contract with tlb. $$$ 1.5yrs left... 2009/2/13, Seth Mattinen <sethm(a)rollernet.us>: > Charles Regan wrote: >> Isp2 is vtl not bell >> >> 2009/2/13, Seth Mattinen <sethm(a)rollernet.us>: >>> Charles Regan wrote: >>>> Just got final confirmation from ISP1 that they will not do BGP with us. >>>> >>>> ISP1 is Telebec. >>>> http://www.iptools.com/dnstools.php?tool=ipwhois&user_data=142.217.0.0&subm… >>>> >>>> My subnet >>>> http://www.iptools.com/dnstools.php?tool=ipwhois&user_data=204.144.60.0&sub… >>>> >>>> What can we do now ? Any suggestions ? >>>> >>> Do you know who is upstream of ISP2? We've established that Telebec is >>> only connected to Bell Canada. If ISP2 also has a connection to Bell >>> then you don't gain anything with Telebec except this huge mess and >>> horrible hacks to work around their lack of BGP. >>> >>> ~Seth >>> >> > > > Also, VTL peers with Sprint and SAVVIS. Based on this information I'd > just drop Telebec completely. They only have one upstream. You won't get > any redundancy with them since they're just giving you a connection to > Bell, which VTL already gives you. Here's the view from my SAVVIS router > with Sprint as the preferred path: > > routy-border0>show ip bgp 216.113.0.0/17 > BGP routing table entry for 216.113.0.0/17, version 78286019 > Paths: (3 available, best #1, table Default-IP-Routing-Table) > Not advertised to any peer > 1239 5769, (received & used) > 208.79.242.129 (metric 3) from 208.79.242.129 (208.79.242.129) > Origin IGP, metric 439, localpref 100, valid, internal, best > Community: 11170:1239 > 3561 5769 > 216.88.158.93 from 216.88.158.93 (206.24.210.102) > Origin IGP, localpref 90, valid, external > Community: 3561:11840 11170:3561 > 3561 5769, (received-only) > 216.88.158.93 from 216.88.158.93 (206.24.210.102) > Origin IGP, localpref 90, valid, external > Community: 3561:11840 > > > -- > Seth Mattinen sethm(a)rollernet.us > Roller Network LLC > -- Envoyé avec mon mobile

4 3