- Test - lists.nanog.org

selective blackholing: implementation, usage & effectiveness
by Job Snijders 10 Feb '14

10 Feb '14

Dear fellow networkers, Through this tutorial-styled email I'd like to introduce the concept, usage and implementation of "selective blackholing" through the BGP protocol to the community. This email contains some python code, example router configurations references to RIPE Atlas data to demonstrate effectiveness. Selective Blackholing is a DDoS damage control mechanism which is (compared to scrubbing or conventional blackholing) cheaper and more effective from the perspective of both the Service Provider and the End-User. In today's internet DDoS attacks are common-place, and the balance is not particularly fair: very expensive on the receiver-side and very cheap for the evil-doer (think DNS-, NTP- and SNMP-amplification attacks). I propose a method which is effective for businesses with a scoped radius of interaction with their customers. To elaborate on what 'scoped' (read: selective) means: a Dutch herring-shop owner will see most legitimate visitors come from within the Netherlands; a Dallas, TX based Automotive Bulletin Board will see most conversions from visitor to part-buyer come from visitors within the United States. If we look at today's DDoS mitigation solutions (scrubbers: expensive hardware or subscription services, or BGP blackhole: all or nothing reachability)... it seems intuitive that a lot of business-owners would care more about traffic from visitors in the surrounding 1000 kilometers (or from the same country), than say (for the duration of the attack) traffic sourced from other continents. In this document we sequentially number any line of router config or python code, to allow for easy references. Let's focus on the implementation following four features: * discard traffic sourced outside 'this' country (5580:664) * discard traffic sourced outside 'this' continent (5580:660) * discard traffic sourced outside a 1000 km radius from 'here' (5580:663) * discard traffic sourced outside a 2500 km radius from 'here' (5580:662) The 'this' and 'here' designators refer to the point of interconnection with any given customer: 'this' and 'here' for a customer connecting to AS5580 in Amsterdam, would respectable be "Netherlands, Europe, 'most of West Europe', 'most of West + Central Europe'". It is important to understand that all proposed mechanisms are designed to perform uniformly in terms of intended usage regardless of location of interconnection, so for a Dutch customer 5580:664 means 'discard any traffic for my prefix received on PEs outside the Netherlands' but for an Dallas, TX based customer BGP community '5580:663' would mean 'discard traffic destined to my prefix on routers located outside a 1000 km radius around Dallas'. The 1000 km and 2500 km distances referenced throughout this document mean the distance between AS 5580 network devices as calculated with a haversine formula with the GPS coordinates of devices as input. The haversine formula is an equation giving great-circle distances between two points on a sphere from their longitudes and latitudes. Actual length of datapath or optical paths is not taken into consideration, nor is a peering partner's home country a factor. To create the above described features we need a simplistic CMDB database, some python code, static inbound iBGP route-map and semi-dynamic customer facing inbound route-map. For illustration purposes we assume a fictional ISP with a POP in the following cities: Tokyo, San Jose, Dallas, New York, London, Amsterdam and Stockholm. +-----------+-----------+----------+-------+---------------------+ | router | continent | country | metro | latitude, longitude | | name | id | ISO31661 | id | | +-----------+-----------+----------+-------+---------------------+ | r1.tky.jp | 3 | 392 | 46 | 35.65671, 139.80342 | | r1.sjo.us | 1 | 840 | 29 | 37.44569,-122.16111 | | r1.dal.us | 1 | 840 | 33 | 32.80096, -96.81962 | | r1.nyc.us | 1 | 840 | 26 | 40.71780, -74.00885 | | r1.lon.uk | 2 | 276 | 23 | 51.51173, -0.00197 | | r1.ams.nl | 2 | 528 | 20 | 52.35600, 4.95068 | | r1.sto.se | 2 | 752 | 22 | 59.36264, 17.95560 | +-----------+-----------+----------+-------+---------------------+ Table 1. Simplistic CMDB Based on the above CMDB table we can construct the following device specific configuration snippets. Within these snippets we adhere to the concept that edge devices are smart and core devices dumb. At the edge of the network we accept BGP Communities from customers, which we translate through a process of testing & branching from 'something the customer understands' to 'something the network understands'. r1.tky.jp: 001. ip community-list THIS:METRO seq 5 permit 65123:30046 002. ip community-list THIS:COUNTRY seq 5 permit 65123:392 003. ip community-list THIS:CONTINENT seq 5 permit 65123:3000 r1.sjo.us: 004. ip community-list THIS:METRO seq 5 permit 65123:10029 005. ip community-list THIS:COUNTRY seq 5 permit 65123:840 006. ip community-list THIS:CONTINENT seq 5 permit 65123:1000 r1.dal.us: 007. ip community-list THIS:METRO seq 5 permit 65123:10033 008. ip community-list THIS:COUNTRY seq 5 permit 65123:840 009. ip community-list THIS:CONTINENT seq 5 permit 65123:1000 r1.nyc.us: 010. ip community-list THIS:METRO seq 5 permit 65123:10026 011. ip community-list THIS:COUNTRY seq 5 permit 65123:840 012. ip community-list THIS:CONTINENT seq 5 permit 65123:1000 r1.lon.uk: 013. ip community-list THIS:METRO seq 5 permit 65123:20023 014. ip community-list THIS:COUNTRY seq 5 permit 65123:276 015. ip community-list THIS:CONTINENT seq 5 permit 65123:2000 r1.ams.nl: 016. ip community-list THIS:METRO seq 5 permit 65123:20020 017. ip community-list THIS:COUNTRY seq 5 permit 65123:528 018. ip community-list THIS:CONTINENT seq 5 permit 65123:2000 r1.sto.us: 019. ip community-list THIS:METRO seq 5 permit 65123:20022 020. ip community-list THIS:COUNTRY seq 5 permit 65123:725 021. ip community-list THIS:CONTINENT seq 5 permit 65123:2000 all devices (common references for more humane route-map definitions): 022. ! communities which are exposed in public documentation for customers 023. ip community-list OUTSIDE:THIS:CONTINENT:DISCARD seq 5 permit 5580:660 024. ip community-list OUTSIDE:2500KM:RADIUS:DISCARD seq 5 permit 5580:662 025. ip community-list OUTSIDE:1000KM:RADIUS:DISCARD seq 5 permit 5580:663 026. ip community-list OUTSIDE:THIS:COUNTRY:DISCARD seq 5 permit 5580:664 027. ip community-list GLOBAL:BLACKHOLE seq 5 permit 5580:666 028. ip community-list SCOPED:ACTION seq 5 permit 5580:660 029. ip community-list SCOPED:ACTION seq 5 permit 5580:662 030. ip community-list SCOPED:ACTION seq 5 permit 5580:663 031. ip community-list SCOPED:ACTION seq 5 permit 5580:664 032. ip route 10.0.0.1/32 null0 all devices (static inbound iBGP route-map): 033. route-map RX:iBGP permit 100 034. match community GLOBAL:BLACKHOLE 035. continue 1100 036. route-map RX:iBGP permit 100 037. match community OUTSIDE:THIS:COUNTRY:DISCARD OUTSIDE:THIS:CONTINENT:DISCARD 038. continue 1100 039. route-map RX:iBGP permit 110 040. match community OUTSIDE:1000KM:RADIUS:DISCARD OUTSIDE:2500KM:RADIUS:DISCARD 041. continue 1100 042. route-map RX:iBGP permit 1000 043 route-map RX:iBGP permit 1100 044. match community THIS:METRO THIS:COUNTRY THIS:CONTINENT 045. route-map RX:iBGP permit 1101 046. set community no-export additive 047. set ip next-hop 10.0.0.1 048. set local-preference 1000000 The above route-map allows BGP speakers in our routing domain to apply different actions depending on the (geographically significant) communities. Let's use three example input and run them through the route-map to illustrate what happens: evaluation takes place on: router r1.lon.uk input: communities: none result: prefix accepted without modifications why: The first match is 'permit 1000' at line 042, because there is no match statement all prefixes will be accepted and the route-map is not evaulated any further. evaluation takes place on: router r1.lon.uk input: communities: "5580:662 65123:10033 65123:10029" (for example when a customer attached to r1.sjo.us sets "5580:662 - discard outside 2500km") result: traffic for the prefix will be discarded why: The prefix matches with the clause 110 (line 039) because 5580:662 matches OUTSIDE:2500KM:RADIUS:DISCARD (line 040). Because a match is found on line 041 we jump to line 043 for further matching. Because the community string cannot match any of the communities on line 044 (in the case of r1.lon.uk 65123:20023 OR 65123:276 OR 65123:2000), the prefix is further evaluated at line 045. There is no match statement in the 'permit 1101' clause so for any prefix that arrives at this particular clause in the route-map the next-hop will be 10.0.0.1 (discard). Because the prefix matches with clause 1101 the route-map will not be evaluated any further. evaluation takes place on: router r1.dal.us input: communities: "5580:662 65123:10033 65123:10029" (for example when a customer attached to r1.sjo.us sets "5580:662 - discard outside 2500km") result: prefix will be accepted with no further modifications why: The prefix matches with the clause 110 (line 039) because 5580:662 matches OUTSIDE:2500KM:RADIUS:DISCARD (line 040). Because a match is found, the continue statement (line 041) is followed and we jump to line 043 for further matching. Because the community string matches one of the communities on line 044 (in the case of r1.dal.us 65123:10033 OR 65123:840 OR 65123:1000), further evaluation of the route-map is halted, and the clause which actually makes it a blackhole is never reached. The attentive reader will now probably think "How come you are inputting "5580:662 65123:10033 65123:10029", and not just 5580:662?! You promised simple end-user configuration, no customer will want to invest time to figure out magic values such as 65123:10033 65123:10029! Now we get down the the secret sauce: extensive community rewriting.... Based on our CMDB database, we can calculate sets of magic values that will cause the network to behave according to the desired feature. I've made a sample python program available that will do these calculations. This is the output: 049. Eleanor:~ job$ wget -q http://noc.as5580.net/~job/example_community_calculator.py 050. Eleanor:~ job$ python example_community_calculator.py 051. r1.lon.uk - rewrite targets: 052. 1000 km: 65123:20020 65123:276 053. 2500 km: 65123:2000 054. 055. r1.dal.us - rewrite targets: 056. 1000 km: 65123:10033 057. 2500 km: 65123:840 058. 059. r1.sjo.us - rewrite targets: 060. 1000 km: 65123:10029 061. 2500 km: 65123:10033 65123:10029 062. 063. r1.nyc.us - rewrite targets: 064. 1000 km: 65123:10026 065. 2500 km: 65123:10026 65123:10033 066. 067. r1.tky.jp - rewrite targets: 068. 1000 km: 65123:30046 069. 2500 km: 65123:30046 070. 071. r1.sto.se - rewrite targets: 072. 1000 km: 65123:20022 073. 2500 km: 65123:2000 074. 075. r1.ams.nl - rewrite targets: 076. 1000 km: 65123:20020 65123:276 077. 2500 km: 65123:2000 078. Eleanor:~ job$ Now we can use the above to create the route-map for a customer connected to r1.sjo.us (San Jose, US): 079. ip prefix-list 15562:PREFIX seq 5 permit 194.33.96.0/24 080. ip prefix-list 15562:PREFIX:BLACKHOLE seq 5 permit 194.33.96.0/24 le 32 081. route-map IMPORT:FROM:15562:CUSTOMER:IPv4 permit 200 082. match ip address prefix-list 15562:PREFIX:BLACKHOLE 083. match community GLOBAL:BLACKHOLE 084. set community 5580:26220 no-export additive 085. set ip next-hop 10.0.0.1 086. set local-preference 1000000 087. route-map IMPORT:15562:CUSTOMER:IPv4 permit 300 088. match ip address prefix-list 15562:PREFIX:BLACKHOLE 089. match community SCOPED:ACTION 090. set community 5580:26220 no-export additive 091. set local-preference 650 092. continue 600 093. route-map IMPORT:15562:CUSTOMER:IPv4 permit 400 094. match ip address prefix-list 15562:PREFIX 095. set community 5580:26220 additive 096. set local-preference 650 097. route-map IMPORT:15562:CUSTOMER:IPv4 deny 500 098. route-map IMPORT:15562:CUSTOMER:IPv4 permit 600 099. match community OUTSIDE:1000KM:RADIUS:DISCARD 100. set community 65123:10029 additive 101. continue 1200 102. route-map IMPORT:15562:CUSTOMER:IPv4 permit 700 103. match community OUTSIDE:2500KM:RADIUS:DISCARD 104. set community 65123:10033 65123:10029 additive 105. continue 1200 106. route-map IMPORT:15562:CUSTOMER:IPv4 permit 900 107. match community OUTSIDE:THIS:COUNTRY:DISCARD 108. set community 65123:840 additive 109. continue 1200 110. route-map IMPORT:15562:CUSTOMER:IPv4 permit 1100 111. match community OUTSIDE:THIS:CONTINENT:DISCARD 112. set community 65123:1000 additive 113. route-map IMPORT:15562:CUSTOMER:IPv4 permit 1200 Various rewrites need to happen to offer the four scoped features we promised at the beginning of the document, the rewrites come either directly from the CMDB (line 108 and 112) or from the python program (line 100 and 104). Any network that wants to implement this should run software such as the python program every time a new POP is added, and update all IMPORT route-maps on all devices accordingly. Lines 081 to 113 bring us back to the second two examples where I illustrated what happens with various community strings if they are parsed through the RX:iBGP route-map. The customer in San Jose just sets 5580:662 (matched at line 102), but the IMPORT route-map will rewrite that to "5580:662 65123:10033 65123:10029" (line 104). Et voila, we have accomplished selective blackholing based on distance. :-) I ran some measurements with the RIPE Atlas system to test which probes can reach a given prefixes when a selective blackhole community is attached, the results are available here: http://noc.as5580.net/~job/selective_blackholing/ Disclaimer: Seeming route-map inefficiencies might be more related to the particular vendor on which I implemented this rather than my logic capabilities. If this document contains any errors, they most likely originated in the process of converting real world configuration to something of educational value. :-) I'd like to thank Saku Ytti Torsten Blum and Peter van Dijk for their gracious inspiration. If not for free and unhindered dialogue between organisations, we would never be able to create mechanisms like these. If you have any questions, please approach me through email or in person at NANOG60 in Atlanta! Kind regards, Job

1 0

cert's routeviews mirror
by John Kemp 10 Feb '14

10 Feb '14

Someone asked me for the link... http://routeviews-mirror.cert.org/ John Kemp help(a)routeviews.org

1 0

GEO location issue with google
by Praveen Unnikrishnan 10 Feb '14

10 Feb '14

Hi, We are an ISP based in UK. We have got an ip block from RIPE which is 5.250.176.0/20. All the main search engines like yahoo shows we are based in UK. But Google thinks we are from Saudi Arabia and we redirected to www.google.com.sa<http://www.google.com.sa> instead of googlw.co.uk. I have sent lot of emails to google but no luck. All the information from google are in Arabic and youtube shows some weird videos as well. Could anyone please help me to sort this out? Would be much appreciated for your time. Praveen Unnikrishnan Network Engineer PMGC Technology Group Ltd T: 020 3542 6401 M: 07827921390 F: 087 1813 1467 E: piu(a)pmgroupuk.com<mailto:piu@pmgroupuk.com> [cid:image001.png@01CF2418.27F29CA0] [cid:image002.jpg@01CE1663.96B300D0] www.pmgroupuk.com<http://www.pmgroupuk.com/> | www.pmgchosting.com <http://www.pmgchosting.com/> How am I doing? Contact my manager, click here<mailto:shazad@pmgroupuk.com?subject=How's%20Praveen%20doing?>. ________________________________ [cid:image003.jpg@01CE1663.96B300D0] PMGC Managed Hosting is now live! After a successful 2012, PMGC continues to innovate and develop by offering tailored IT solutions designed to empower you through intelligent use of technologies. www.pmgchosting.com<http://www.pmgchosting.com/>. PMGC Technology Group Limited is a company registered in England and Wales. Registered number: 7974624 (3/F Sutherland House, 5-6 Argyll Street, London. W1F 7TE). This message contains confidential (and potentially legally privileged) information solely for its intended recipients. Others may not distribute copy or use it. If you have received this communication in error please contact the sender as soon as possible and delete the email and any attachments without keeping copies. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company or its associated companies unless otherwise specifically stated. All incoming and outgoing e-mails may be monitored in line with current legislation. It is the responsibility of the recipient to ensure that emails are virus free before opening. PMGC(r) is a registered trademark of PMGC Technology Group Ltd.

5 5

MENOG 14 in Dubai
by Philip Smith 10 Feb '14

10 Feb '14

Hi everyone, In the spirit of keeping each of the NOG communities in touch with activities going on in each other's regions, the MENOG Program Committee is hoping that some of you would be interesting in joining the Middle East operations community for their 14th meeting in Dubai at the end of March. If you are interested in presenting at the conference or participating in the Peering Forum, or are planning to pass through Dubai enroute to other events, please consider submitting a presentation proposal: http://papers.menog.org/user/login.php?event=7 The programme committee is looking forward to your contributions; the submission deadline is just 2 weeks away now. MENOG 14 is being held at the Grosvenor House Hotel in Dubai Marina - the conference takes place on the 30th March, the MENOG Peering Forum on the 31st March, and tutorials are on 1st April. Workshops take place from 23rd to 27th March. More info at http://www.menog.org/meetings/menog-14. Thanks and hopefully see you in Dubai! philip (on behalf of the MENOG Coordination Team) --

1 0

Re: Need trusted NTP Sources
by Larry Sheldon 09 Feb '14

09 Feb '14

On 2/9/2014 7:04 PM, Larry Sheldon wrote: > In the 1990s I found myself administering a campus network for a > University--the only people less prepared than I as everybody else. In the 1990s I found myself administering a campus network for a University--the only people less prepared than I Was everybody else. -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker)

1 0

Re: Need trusted NTP Sources
by Larry Sheldon 09 Feb '14

09 Feb '14

On 2/9/2014 6:42 PM, James R Cutler wrote: > On Feb 9, 2014, at 3:50 PM, Larry Sheldon <LarrySheldon(a)cox.net> > wrote: > >> On 2/9/2014 2:45 PM, Jay Ashworth wrote: >> >>> Or do I understand NTP less well than I think? >> >> I am of the private opinion that if your name is not "David Mill" >> (and MAYBE if it IS) the answer is either "42" or "yes". — ... > > From > http://www.eecis.udel.edu/~mills/database/brief/overview/overview.pdf >> > Intersection and clustering algorithms pick best true chimers and discard false tickers. > You should look at this presentation and see why Larry Sheldon’s > private opinion is spot on. > > I won’t begin to try explaining in technical detail how this works. > The bottom line is that, within a peer group of NTP servers looking > at a reasonably large set of NTP source servers, all kinds of > variations in input data are reduced to a coherent local time truth. In the 1990s I found myself administering a campus network for a University--the only people less prepared than I as everybody else. A need arose to have a uniform notion of time across the campus (my recollection had to do with resolving who did it first squabbles as well as trying to solve some problems having to do with the date and time in emails regarding assignments due. I stumbled across NTP somewhere and decided that was the answer, I didn't know about "42" then. Nobody I was in contact with knew any more about it that I did, so I spent a lot of time on eecis learning how to make it play, and how not to be a rude participant. > > My template for NTP service deployment for any organization is very > simple: > > 1. Select four or more local systems and configure them as peer NTP > servers. In many instances one can leverage local DNS server > machines running almost any OS — the NTP daemon runs on at least > Windows, OS X, UNIX, Linux. Don’t forget appropriate restrict > commands. I don't remember now how many boxes I had in my NTP backbone but it was lots--every cisco router I knew the password for (there were a lot of them, supporting frame-relay links to off-campus points), every HP9000 box I had root on, maybe the two Wellfleets -- I don't remember. They all were peers and I connected to a couple of off-network public stratum 1s and 2s not as peers (I had no budget for a stratum 0). > 2. Configure ntpd on the local servers to also select as servers a > list of 8-10 open access servers like pool.ntp.org, usno.navy.mil, > nist-????-ustiming.org. If you can arrange authenticated access to > other servers, that is possibly better. I tried, using "ping", to pick sturdy-sounding servers that were "close" to Omaha. > 3. As desired, configure ntpd on selected local servers for local > clocks or GPS clocks. This has little effect on accuracy, but may > enhance reliability. In many cases, it also requires building > penetrations for antennas. (Not easy for network guys.) > > 4. Configure all local time consumers to select from the list of > local NTP servers. Authenticate or not as you see fit. You can even > use DHCP to inform end systems of NTP server addresses. The router > folks will have to include NTP server addresses as part of each > configuration package. Did that. Told machines and people to use their default gateway address as their NTP (or SNTP) server. > Over the years I have successfully applied this template for NTP > service deployments to several large networks. It just works. It does. It does. -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker)

1 0

Re: Need trusted NTP Sources
by Larry Sheldon 09 Feb '14

09 Feb '14

On 2/9/2014 2:45 PM, Jay Ashworth wrote: > Or do I understand NTP less well than I think? I am of the private opinion that if your name is not "David Mill" (and MAYBE if it IS) the answer is either "42" or "yes". -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker)

2 1

Re: Need trusted NTP Sources
by Notify Me 09 Feb '14

09 Feb '14

According to the auditors, "trusted" means 1. Universities or Research facilities (nuclear/atomic facilities, space research (such as NASA) etc.) 2. Main country internet/telecom providers 3. Government departments 4. Satellites (using GPS module) Which is a bit of a tall order over here. On Thu, Feb 6, 2014 at 11:16 AM, Marc Storck <mstorck(a)voipgate.com> wrote: > You may start by checking who is providing NTP services in Africa via the NTP pool. In Africa there are 27 public servers (http://www.pool.ntp.org/zone/africa) > > But then all depends on your definition of "trusted". > > Regards, > > Marc > ________________________________________ > From: Notify Me [notify.sina(a)gmail.com] > Sent: Thursday, February 06, 2014 11:03 > To: nanog(a)nanog.org list; afnog(a)afnog.org > Subject: Need trusted NTP Sources > > Hi ! > > I'm trying to help a company I work for to pass an audit, and we've > been told we need trusted NTP sources (RedHat doesn't cut it). Being > located in Nigeria, Africa, I'm not very knowledgeable about trusted > sources therein. > > Please can anyone help with sources that wouldn't mind letting us sync > from them? > > Thanks a lot! >

16 26

TWC (AS11351) blocking all NTP?
by Jonathan Towne 08 Feb '14

08 Feb '14

This evening all of my servers lost NTP sync, stating that our on-site NTP servers hadn't synced in too long. Reference time noted by the local NTP servers: Fri, Jan 31 2014 19:11:29.725 Apparently since then, NTP has been unable to traverse the circuit. Our other provider is shuffling NTP packets just fine, and after finding an NTP peer that return routed in that direction, I was able to get NTP back in shape. Spot checking various NTP peers configured on my end with various looking glasses close to the far-end confirm that anytime the return route is through AS11351, we never get the responses. Outbound routes almost always take the shorter route through our other provider. Is anyone else seeing this, or am I lucky enough to have it localized to my region (Northern NY)? I've created a ticket with the provider, although with it being the weekend, I have doubts it'll be a quick resolution. I'm sure its a strange knee-jerk response to the monlist garbage. Still, stopping time without warning is Uncool, Man. -- Jonathan Towne

42 109

Odd Cogentco routing?
by David Hill 07 Feb '14

07 Feb '14

Hello - While doing some traceroutes, I have found a few destinations that I found a little odd. For example: 5.|-- bbr01aldlmi-bue-2.aldl.mi.charter.com 0.0% 60 152.1 47.2 8.3 367.6 66.0 6.|-- bbr01sgnwmi-bue-5.sgnw.mi.charter.com 0.0% 60 102.3 53.4 15.6 317.9 66.3 7.|-- be4041.nr21.b016069-0.dtw04.atlas.cogentco.com 0.0% 60 78.9 58.3 19.6 267.9 62.1 8.|-- te0-5-0-5.rcr21.dtw04.atlas.cogentco.com 0.0% 60 32.6 58.2 20.3 218.3 54.6 9.|-- te4-2.ccr01.tol01.atlas.cogentco.com 0.0% 60 359.6 95.2 21.4 359.6 84.5 10.|-- te0-5-0-1.ccr21.cle04.atlas.cogentco.com 0.0% 60 24.3 70.0 24.3 219.3 52.5 11.|-- te7-2.ccr01.buf02.atlas.cogentco.com 0.0% 60 29.3 89.9 28.8 245.4 66.0 12.|-- te7-2.ccr01.yhm01.atlas.cogentco.com 0.0% 60 117.6 105.0 30.6 467.8 87.7 13.|-- te0-7-0-34.ccr21.yyz02.atlas.cogentco.com 0.0% 60 167.5 65.8 32.0 316.7 51.7 14.|-- level3.yyz02.atlas.cogentco.com 15.0% 60 132.0 91.2 47.6 326.1 57.9 15.|-- ae-13-13.bar1.Toronto1.Level3.net 61.7% 60 62.4 107.9 49.5 282.7 73.0 16.|-- ae-9-9.ebr1.Chicago1.Level3.net 0.0% 60 92.4 116.8 69.6 315.0 59.2 17.|-- ae-1-100.ebr2.Chicago1.Level3.net 1.7% 60 92.3 127.7 71.2 370.1 62.1 18.|-- ae-3-3.ebr2.Atlanta2.Level3.net 0.0% 60 74.6 125.5 70.7 261.0 50.3 19.|-- ae-2-52.edge4.Atlanta2.Level3.net 0.0% 60 75.0 121.0 71.0 252.4 45.6 Detroit -> Toledo -> Cleveland -> Buffalo -> Hamilton, ON -> Toronto -> Chicago.. Is it odd that Cogentco is routing USA traffic to Canada and handing it off there to Level3 just to bring it back in to the USA? There is also some packetloss at the exchange... Thanks, David

1 0