"David R. Dick" wrote:

...

Today, I was approached by *unnamed-ethernet-extension-company*. They extend ethernets between several US and UK peering exchanges.

While speaking with them today, thier engineer and I got into a little bit of a disagreement as to why people peer with each other at public exchange points. My belief is that generally speaking, networks meet at public exchange points (such as MAE-*, LINX, AMSIX, AADS, etc) is to exchange traffic with each other more economically (read: save money).

His belief is that people will pay a premium to get to an exchange point, because it's worth paying a premium to have 'less hops' between two networks.

The problem with this idea is that public exchange points need to be *avoided* when they get too congested. People may start out trying to minimize number of hops, but I think they eventually try to minimize total latency.

but what if the *unnamed-ethernet-extension-company* wasn't providing access to public exchange points, but was rather enabling uncongested private peering over its network? That way latency and hop count are both mimimised. BTW, public IXen in Europe don't tend to be congested. Whether this is the result of better management, or of lower traffic volumes, than IXen in the US I'm not sure... Giles

...

Essentially, he said that paying more for peering that for transit is typical, and to be expected, and most people accept this.

Whats the common opinion on this?

-- ================================================================= Giles Heron Principal Network Architect PacketExchange Ltd. ph: +44 7880 506185 "if you build it they will yawn" =================================================================

but what if the *unnamed-ethernet-extension-company* wasn't providing access to public exchange points, but was rather enabling uncongested private peering over its network? That way latency and hop count are both mimimised.

But, *unnamed-ethernet-extension-company*, to me, only marketed access to other exchange points. Also, what does hop count have to do with latency and loss? Especially, when *unnamed-ethernet-extension-company* is using MPLS/IP (presumably) to do this?

BTW, public IXen in Europe don't tend to be congested. Whether this is the result of better management, or of lower traffic volumes, than IXen in the US I'm not sure...

Probably better management, is my guess.

================================================================= Giles Heron Principal Network Architect PacketExchange Ltd. ph: +44 7880 506185 "if you build it they will yawn" =================================================================

batz wrote:

On Fri, 30 Nov 2001, David R. Dick wrote:

:The problem with this idea is that public exchange points need :to be *avoided* when they get too congested. People may start :out trying to minimize number of hops, but I think they eventually :try to minimize total latency.

Indeed. The notion of "hopcount" is objectively meaningless. You can count hops by physical wires, data segments, network hops, AS's or combinations of each (MPLS, GRE, PPP etc).

He may have been trying to suggest the value of having a low latency path to a tier-1 provider, who may have low latency across their backbone and through their cores.

But this is what you get with any good ISP, so I don't understand what makes IX's more special than being connected to UUNet, Exodus, or any other tier-1. In fact, having a connection to a tier-1, who may be at many IX's, would leave you with the benefits of diverse paths to other networks, without the premium of an IX. Especially considering the tier-1 would be doing traffic shaping to minimize latency across the backbone.

Also, I wonder if you can measure latency reliably over time for more than 3 or 4 'hops' out, and relative to a small number of destinations, before your results cease to be relative to each other? I'm sure there is a principle like this acting upon the Internet somehow.

Without latency measurements for the media you are counting hops in, you might as well count sheep. I think the engineer mentioned at the beginning of this thread was trying to hoodwink and bamboozle you.

Folks, I'm stood to one side up to now, but now this thread is drifting over to personal abuse. I was the engineer in question, and I was most certainly not trying to hoodwink or bamboozle. Neither am I an ignorant sales-droid, as someone else has said. Those of you who know me, as I think that a fair number of people on this list do, will vouch for my honesty, and my pedigree in the industry. I think Alex misunderstood what I was trying to say, and since we were shouting at each other down a very bad phone line with a loudspeaking phone at either end, he's got a certain amount of excuse. Firstly, I've been designing IP networks since the late 80s, and ISPs since the early 90s. I'm not "fresh out of cable school" to quote one correspondant. I've set peering policy for at least 2 large ISPs (BTnet and Level3 (Europe)). Peering is my field of expertise. I'm on the council of the LINX, which is the biggest peering point in Europe. I've also got experience of selling internet services in Europe, and to be frank, the customers over here have different requirements than over in the US. The first thing you get asked is "how good is your peering". They then ask you how much private peering you have, despite the fact that IXPs in Europe tend to be well run and uncongested. What I was offering him was a LAN extension service. One of the things you can use this for is to peer with other customers. It looks like a direct peering (so you keep customers happy) and it gets the bits from A to C without passing through (possibly congested and oversubscribed) B. Its not an oversubscribed service, so you get effectively private line performance. And the costing is distance independent, which makes it more like a peering point, in charging terms. Hope this sets the record straight. Note that I'm posting this from my home email address, and I'm speaking on my own behalf, not that of my company. All the best Nigel

Neil J. McRae wrote (on Nov 30):

yeah don't talk to me about some of the bozo companies I've had to talk to about "congestion at the peering points" and how they can wave their magic wand and make all those problems go away.

A current pitch from a certain newish IXP entrant in London is that they can save me money off my Transit bill. When vendors get their facts so blatantly wrong about your company, I seriously consider whether they're worth doing business with - true it's just the sales droid, but IXen don't need that many sales droids, I thought...

Sounds like something we looked at a while ago - layer 2 interconnectivity over IP or MPLS for ISP's - we couldn't make the numbers add up though.

It's something I have considered for backup connectivity, but it never went anywhere. I'm waiting for Nigel to pitch to me, in case his version makes sense. :) Chris.

On Fri Nov 30, 2001 at 10:39:57PM -0500, Alex Rubenstein wrote:

My question was simply a curiosity ping of _why_ people peer with each other; in my mind, it had always, and never not, been a way to reduce cost of traffic sent/rec'd. I was curious as to whether or not others had a similar view to mine.

For us, I'd say there's a two-fold win. 1) Cost. We try to aim for less than $100/mbps on peering connections. This isn't always the case when first connecting to a NAP, because PtP circuits and NAP ports aren't charged per mbps, so the cost per mbps is much higher for the first 10 or 20 meg, and comes down after that. It also serves to expand our potential network capacity. This can work right to extremes - in the UK (our primary geographical target region), we can reach probably 99% of users without using our transit with Level3 and others in the USA. 2) More direct relationships with end-user ISPs. Okay, there's no formal SLAs or anything like that, but there *is* a relationship between us and a peer. When a user complains about connectivity issues, it's more likely that there's just two parties involved - us and the user's ISP. We're not transitting through 2 or 3 other ISPs in the middle - all of which could be the cause of problems. Simon -- Simon Lockhart | Tel: +44 (0)1737 839676 Internet Engineering Manager | Fax: +44 (0)1737 839516 BBC Internet Services | Email: Simon.Lockhart@bbc.co.uk Kingswood Warren,Tadworth,Surrey,UK | URL: http://support.bbc.co.uk/

On Fri, 30 Nov 2001, batz wrote:

Indeed. The notion of "hopcount" is objectively meaningless. You can count hops by physical wires, data segments, network hops, AS's or combinations of each (MPLS, GRE, PPP etc).

Actually, not. The hops which "count" are places where traffic flows are stochastically multiplexed. Since every such place has to have buffers of size comparable to end-to-end RTT * bandwidth (to give enough time for TCP flow control to react), the maximal packet delay is Nhops*RTT. Note this does not say what the median variable latency (fixed latency is the signal propagation time) is going to be because it depends on probability of multiple congestions along the path. If congestion events are independent, the result isn't that bad (with long-lived congestion probability of 50% the median variable latency is 2*RTT). Unfortunately, this assumption is not likely to be realistic. I.e. in the "aggregation tree" networks (like the most modern backbones which aggregate outgoing exterior traffic at multiple points, where every aggregation point reduces the total egress bandwith), the congestion is likely to occure simultaneously at many aggregation points along the path. The way to combat variable latency is to reduce the number of hops. This effectively means either creation of meshes of fixed bit-rate circuits (which is way way way ugly because of routing scalability issues), or reducing the number of aggregators along the path by reducing the number of routers. The simplest way to do that is to replace router clusters with large parallel routers featuring non-blocking switching fabrics. Note that non-CBR virtual-circuit techniques (tunnels, MPLS, non-CBR ATM, FR, X.25 etc) don't do anything to reduce the _real_ hopcount; they simply obscure it. --vadim

Daniel Golding wrote:

.....

Hmm. Congested public exchange points were a reality at some point in the past. They are now a recurrent myth, thanks to the demise of FDDI EPs and the rise of GigE EPs. There is little congestion, at this point. Of course, this could be because most Internet traffic is exchanged over private peering.

I think that most clued folks are largely uninterested in hop-count. Latency is much more important.

that depends on your primary application, latency is not important for streaming or downloads. ak

heh! tell someone on the end of duplex satellite that! Streaming no, downloads very much yes.

Ahem. Latency doesn't affect unidirection streaming in the slightest.