Anyone happen to have more information on the problems that have been happening with the peering between Cogent and Level3. Cogent gives the standard answer when you call support, but some more specific information would be great. Thanks Dale
Possibly a result of this: http://www.webhostingtalk.com/showthread.php?s=&threadid=96985 Kevin
Anyone happen to have more information on the problems that have been happening with the peering between Cogent and Level3. Cogent gives the standard answer when you call support, but some more specific information would be great.
Thanks Dale
Might have to do with http://isp-lists.isp-planet.com/isp-bandwidth/0212/msg00978.html (AOL vs Cogent Peering issue) ---Mike At 09:51 AM 18/12/2002 -0500, Dale Levesque wrote:
Anyone happen to have more information on the problems that have been happening with the peering between Cogent and Level3. Cogent gives the standard answer when you call support, but some more specific information would be great.
Thanks Dale
AOL (AS1668) stopped peering with cogent yesterday for reasons they did not disclose publicly. Cogent sends same letter to all customers who asked for what is going on and in the letter they say that two weeks ago, peering to AOL was upgraded to OC48 from OC12 and now for some reason AOL stopped peering and if somebody has questions they should call AOL to complain .. (with phone# to their NOC provided in the letter - not very nice thing to do it like this in my opinion). Afterwards direct peering to AOL was lost, cogent started sending out all their data though Level3 (AOL upstrem), making them pay for that traffic and because of amount of data the peering links got congested easily - PAIX cogent->level3 peering in bay area for example had > 1 sec latency yesterday afternoon. After I spent some time yesterday investigating all that I found that actually cogent is only using former netrail (AS4006) peering links to send data out to that is going to AS1668 and only some of those links. On the other hand most level3 destined data from cogent is sent through PSI links (i.e. 16631 174 3356).. So for me currently I put up filters to route all AS1668 and Level3 data through savvis for routes that look like (16631 4006 3356 ... ) and it works out fine for now. Today I'm going to deal with incoming traffic but the problem I had is that I could not find AOL (AS1668) looking glass, traceroute or some other tool like that. I checked traceroute.org and was very surprised that large network like theirs does not have publicly available looking glass listed. So those of you who peer with them - how do you deal with network troubleshooting when you need to check soemthing from their network side? If you know where their looking glass is - please send me a link! P.S. If somebody is seeing level3->cogent route that is congested I'd also like to know as this is something I have to deal with today as well. Send this traceroute to me privately. On Wed, 18 Dec 2002, Dale Levesque wrote:
Anyone happen to have more information on the problems that have been happening with the peering between Cogent and Level3. Cogent gives the standard answer when you call support, but some more specific information would be great.
Thanks Dale
On Wed, Dec 18, 2002 at 08:44:55AM -0800, william@elan.net wrote:
AOL (AS1668) stopped peering with cogent yesterday for reasons they did not disclose publicly. Cogent sends same letter to all customers who asked for what is going on and in the letter they say that two weeks ago, peering to AOL was upgraded to OC48 from OC12 and now for some reason AOL stopped peering and if somebody has questions they should call AOL to complain .. (with phone# to their NOC provided in the letter - not very nice thing to do it like this in my opinion).
If nothing else, Cogent could be using their idle 6461 transit, instead of grandstanding by overloading their Level 3 capacity so they can blame AOL. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
I pointed that out on another list too but somebody else responded that abovenet to aol connection is congested as it is and more then likely would not have been able to take all the extra traffic. I'm not a customer of MFN/abovenet so I really do not know but I did not like it how cogent deal with it all either - if somebody blames too much somebody else, more then likely they are at fault to considerable degree... On Wed, 18 Dec 2002, Richard A Steenbergen wrote:
On Wed, Dec 18, 2002 at 08:44:55AM -0800, william@elan.net wrote:
AOL (AS1668) stopped peering with cogent yesterday for reasons they did not disclose publicly. Cogent sends same letter to all customers who asked for what is going on and in the letter they say that two weeks ago, peering to AOL was upgraded to OC48 from OC12 and now for some reason AOL stopped peering and if somebody has questions they should call AOL to complain .. (with phone# to their NOC provided in the letter - not very nice thing to do it like this in my opinion).
If nothing else, Cogent could be using their idle 6461 transit, instead of grandstanding by overloading their Level 3 capacity so they can blame AOL.
In a message written on Wed, Dec 18, 2002 at 10:02:13AM -0800, william@elan.net wrote:
I pointed that out on another list too but somebody else responded that abovenet to aol connection is congested as it is and more then likely would not have been able to take all the extra traffic. I'm not a customer
There are no congestion issues between 6461 and 1668 at this time. If someone believes there is a problem please have them contact me off list so I can look into it. I can make no comment on the volume of traffic, having no idea what volume is involved. If you consider the paths that are available for a moment, I think most people can deduce why they are choosing the Level 3 path, and it has nothing to do with performance. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org
On Wed, Dec 18, 2002 at 01:12:02PM -0500, Richard A Steenbergen wrote:
On Wed, Dec 18, 2002 at 08:44:55AM -0800, william@elan.net wrote:
AOL (AS1668) stopped peering with cogent yesterday for reasons they did not disclose publicly. Cogent sends same letter to all customers who asked for what is going on and in the letter they say that two weeks ago, peering to AOL was upgraded to OC48 from OC12 and now for some reason AOL stopped peering and if somebody has questions they should call AOL to complain .. (with phone# to their NOC provided in the letter - not very nice thing to do it like this in my opinion).
If nothing else, Cogent could be using their idle 6461 transit, instead of grandstanding by overloading their Level 3 capacity so they can blame AOL.
is it really idle ? Why wouldn't Cogent create a community string to provide its multihomed customers with prepend 16631 (or customer asn) to Level3 peering sessions to control inbounds better? -Basil
On Wed, Dec 18, 2002 at 12:24:31PM -0600, Basil Kruglov wrote:
Why wouldn't Cogent create a community string to provide its multihomed customers with prepend 16631 (or customer asn) to Level3 peering sessions to control inbounds better?
Me thinks Cogent doesn't have a problem with congestion on the inbound direction. Fix your reverse path. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
On Wed, Dec 18, 2002 at 02:36:04PM -0500, Richard A Steenbergen wrote:
Why wouldn't Cogent create a community string to provide its multihomed customers with prepend 16631 (or customer asn) to Level3 peering sessions to control inbounds better?
Me thinks Cogent doesn't have a problem with congestion on the inbound direction. Fix your reverse path.
Customers of Cogent should be/are more concerned about congestion on the inbounds at Level3 <-> Cogent; outbound is way too easy to control. If the site is multihomed to any decent Tier1 provider +Cogent; (701 or 2914 or 1239 or 3561 or 1, etc +16631) with or without prepend 16631 or site ASN, a lot of, if not all of the inbound traffic will still be going through one of those better carriers, fortunately or unfortunately. All I really want is to be able to control my inbound from Level3 ;) -Basil
Me thinks Cogent doesn't have a problem with congestion on the inbound direction. Fix your reverse path.
Customers of Cogent should be/are more concerned about congestion on the inbounds at Level3 <-> Cogent; outbound is way too easy to control.
Cogent has a pile of available inbound - websites tend to send traffic out, not take traffic in. Alex
Thing is if your connection is completely full one way, it'll effect traffic the other way too. It should not be happening with syncronyous connections, but practical observation is that it does! I suspect router hardware is to blame (possibly packet cache is way full) and I'v seen it happen only if you try to send 100% more traffic then link can handle (just 100% traffic does not efect it - have to really push it), this happened on 100Mb and even on Gb interface. On Wed, 18 Dec 2002 alex@yuriev.com wrote:
Me thinks Cogent doesn't have a problem with congestion on the inbound direction. Fix your reverse path.
Customers of Cogent should be/are more concerned about congestion on the inbounds at Level3 <-> Cogent; outbound is way too easy to control.
Cogent has a pile of available inbound - websites tend to send traffic out, not take traffic in.
Alex
william@elan.net wrote:
Thing is if your connection is completely full one way, it'll effect traffic the other way too. It should not be happening with syncronyous connections, but practical observation is that it does! I suspect router hardware is to blame (possibly packet cache is way full) and I'v seen it happen only if you try to send 100% more traffic then link can handle (just 100% traffic does not efect it - have to really push it), this happened on 100Mb and even on Gb interface.
Or could it be that your ACK packets just simply get delayed enough for the traffic to the other direction to suffer somewhat? This is quite common phenomenan in asymmetric links but also exists for symmetric ones. Pete
On Wed, 18 Dec 2002 alex@yuriev.com wrote:
Me thinks Cogent doesn't have a problem with congestion on the inbound direction. Fix your reverse path.
Customers of Cogent should be/are more concerned about congestion on the inbounds at Level3 <-> Cogent; outbound is way too easy to control.
Cogent has a pile of available inbound - websites tend to send traffic out, not take traffic in.
Alex
Thing is if your connection is completely full one way, it'll effect traffic the other way too. It should not be happening with syncronyous connections, but practical observation is that it does! I suspect router hardware is to blame (possibly packet cache is way full) and I'v seen it happen only if you try to send 100% more traffic then link can handle (just 100% traffic does not efect it - have to really push it), this happened on 100Mb and even on Gb interface.
If your circuit is full one way, it really makes no sense to be bothered dealing with reverse path *before* fixing your forward path. Fix the outbound (saturated) first, *then* look at the rest. Alex
Of course, your right about what needs to be fixed! But situation with cogent is such that I do not have that option. Their peering link with level3 is congested because of all the traffic going to AOL and some of traffic destined to me is going through same link the other way and getting jammed as a sideeffect as well. I route aol-destined traffic from my net to provider other then cogent - but how do I tell AOL and L3 (and only them) that best path to me is through somebody else? Basil is right - the best way to deal with that would be for cogent to provide special community that would allow me to direct cogent to prepend several of their ASN to level3 advertisements.
If your circuit is full one way, it really makes no sense to be bothered dealing with reverse path *before* fixing your forward path. Fix the outbound (saturated) first, *then* look at the rest.
Alex
Of course, your right about what needs to be fixed! But situation with cogent is such that I do not have that option. Their peering link with level3 is congested because of all the traffic going to AOL and some of traffic destined to me is going through same link the other way and getting jammed as a sideeffect as well. I route aol-destined traffic from my net to provider other then cogent - but how do I tell AOL and L3 (and only them) that best path to me is through somebody else?
route-map cogent-ick permit 10 match community-list <aggregate-only> set as-path prepend <iamuglypath> <iamuglypath> <iamuglypath> route-map cogent-ick deny 20 match community-list <deaggregated-routes> route-map happy-with-backup permit 10 match community-list <deaggregated-routes> route-map happy-with-backup deny 20 match community-list <aggregate-only>
Basil is right - the best way to deal with that would be for cogent to provide special community that would allow me to direct cogent to prepend several of their ASN to level3 advertisements.
Cogent doesnot do anything custom. Alex
In the middle of all of this talk re: Cogent and such, they apparantly spent some $$$ somewhere and made our test connection a whole lot better. In a nutshell, we did this a month ago and it stank. Bursty, high and variable latencies.. etc.. Apparently they fixed something. Yesterday in our first test on the new stuff, in 5 FTP sessions to various FTP sites we generated a smooth 20-30mbps So, both iperf and the web based speed test are both back online: speedy.higherbandwidth.net Please abuse and test it for a day or two.. It is supposedly an OC3 to Cogent.. It's 100mbps fdx on the test machine.. Nail that mother to the wall.. PLEASE. All test results will be made public. Iperf and Web, complete with the MTR traceroutes - just in case anyone wants to peek. --Mike--
On Wed, Dec 18, 2002 at 03:23:04PM -0500, alex@yuriev.com wrote:
Me thinks Cogent doesn't have a problem with congestion on the inbound direction. Fix your reverse path.
Customers of Cogent should be/are more concerned about congestion on the inbounds at Level3 <-> Cogent; outbound is way too easy to control.
Cogent has a pile of available inbound - websites tend to send traffic out, not take traffic in.
Somewhat true. Yet still if the inbound from one of the major players is really saturated wouldn't that hurt Cogent customers. -Basil
Me thinks Cogent doesn't have a problem with congestion on the inbound direction. Fix your reverse path.
Customers of Cogent should be/are more concerned about congestion on the inbounds at Level3 <-> Cogent; outbound is way too easy to control.
Cogent has a pile of available inbound - websites tend to send traffic out, not take traffic in.
Somewhat true. Yet still if the inbound from one of the major players is really saturated wouldn't that hurt Cogent customers.
When you have symmetric links [same up/down] (which basically all links are) the probability of the inbound link from others being saturated for a company that provides mostly transit to webhosters is nill to nothing. Alex
On Wed, Dec 18, 2002 at 08:44:55AM -0800, william@elan.net wrote:
AOL (AS1668) stopped peering with cogent yesterday for reasons they did not disclose publicly. Cogent sends same letter to all customers who asked for what is going on and in the letter they say that two weeks ago, peering to AOL was upgraded to OC48 from OC12 and now for some reason AOL stopped peering and if somebody has questions they should call AOL to complain .. (with phone# to their NOC provided in the letter - not very nice thing to do it like this in my opinion).
If nothing else, Cogent could be using their idle 6461 transit, instead of grandstanding by overloading their Level 3 capacity so they can blame AOL.
is it really idle ?
Why wouldn't Cogent create a community string to provide its multihomed customers with prepend 16631 (or customer asn) to Level3 peering sessions to control inbounds better?
Because they do not do custom anything. Alex
participants (10)
-
alex＠yuriev.com -
Basil Kruglov -
Dale Levesque -
Leo Bicknell -
Mike (meuon) Harrison -
Mike Tancsa -
Petri Helenius -
Richard A Steenbergen -
sigma＠smx.pair.com -
william＠elan.net