Had in interesting conversation with a transit AS on behalf of a customer where I found out they are using communities to raise the local preference of routes that do not originate locally by default before sending to a other larger transit AS's. Obviously this isn't something that was asked of them and it took a few days to find since the customer is not a large company and neither them nor my company has a link or business relationship with the AS in question. This seemed strange to me for obvious reasons, but I was curious if anyone else was doing this and why. You obviously cannot use prepend to affect transit traffic again for obvious reasons. MED is a weak metric but it at least only affects traffic that was already going to transit your AS. The larger transit AS was favoring a lower bandwidth link for the customer and causing them to drop packets mysteriously. Just wondering if this practice seemed as strange to others as it does to me.
For this very reason I have advocated using longest prefix BGP routing for some years now, and checking periodically for the expected path, as it became obvious from investigating traceroutes that traffic was not being routed as intended using AS prepends. -----Original Message----- From: Keegan Holley [mailto:keegan.holley@sungard.com] Sent: Wednesday, December 14, 2011 10:08 PM To: NANOG Subject: local_preference for transit traffic? Had in interesting conversation with a transit AS on behalf of a customer where I found out they are using communities to raise the local preference of routes that do not originate locally by default before sending to a other larger transit AS's. Obviously this isn't something that was asked of them and it took a few days to find since the customer is not a large company and neither them nor my company has a link or business relationship with the AS in question. This seemed strange to me for obvious reasons, but I was curious if anyone else was doing this and why. You obviously cannot use prepend to affect transit traffic again for obvious reasons. MED is a weak metric but it at least only affects traffic that was already going to transit your AS. The larger transit AS was favoring a lower bandwidth link for the customer and causing them to drop packets mysteriously. Just wondering if this practice seemed as strange to others as it does to me. This communication, together with any attachments or embedded links, is for the sole use of the intended recipient(s) and may contain information that is confidential or legally protected. If you are not the intended recipient, you are hereby notified that any review, disclosure, copying, dissemination, distribution or use of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by return e-mail message and delete the original and all copies of the communication, along with any attachments or embedded links, from your system.
I suppose so because prepend is so easily defeated, but sometimes you don't own a prefix shorter than the one you need to advertise. Assuming I understand your suggestion correctly. 2011/12/15 Holmes,David A <dholmes@mwdh2o.com>
For this very reason I have advocated using longest prefix BGP routing for some years now, and checking periodically for the expected path, as it became obvious from investigating traceroutes that traffic was not being routed as intended using AS prepends.
-----Original Message----- From: Keegan Holley [mailto:keegan.holley@sungard.com] Sent: Wednesday, December 14, 2011 10:08 PM To: NANOG Subject: local_preference for transit traffic?
Had in interesting conversation with a transit AS on behalf of a customer where I found out they are using communities to raise the local preference of routes that do not originate locally by default before sending to a other larger transit AS's. Obviously this isn't something that was asked of them and it took a few days to find since the customer is not a large company and neither them nor my company has a link or business relationship with the AS in question. This seemed strange to me for obvious reasons, but I was curious if anyone else was doing this and why. You obviously cannot use prepend to affect transit traffic again for obvious reasons. MED is a weak metric but it at least only affects traffic that was already going to transit your AS. The larger transit AS was favoring a lower bandwidth link for the customer and causing them to drop packets mysteriously. Just wondering if this practice seemed as strange to others as it does to me.
This communication, together with any attachments or embedded links, is for the sole use of the intended recipient(s) and may contain information that is confidential or legally protected. If you are not the intended recipient, you are hereby notified that any review, disclosure, copying, dissemination, distribution or use of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by return e-mail message and delete the original and all copies of the communication, along with any attachments or embedded links, from your system.
On Thu, Dec 15, 2011 at 1:07 AM, Keegan Holley <keegan.holley@sungard.com> wrote:
Had in interesting conversation with a transit AS on behalf of a customer where I found out they are using communities to raise the local preference
That sounds like a disreputable practice. While not quite as obvious, some large transit ASes, like Level3, reset the origin to I (best) sometime between when they learn it and when they announce it to their customers and peers. This similarly causes them to suck in a bit more traffic than they might otherwise. -- Jeff S Wheeler <jsw@inconcepts.biz> Sr Network Operator / Innovative Network Concepts
I always assumed that taking in more traffic was a bad thing. I've heard about one sided peering agreements where one side is sending more traffic than the other needs them to transport. Am I missing something? Would this cause a shift in their favor allowing them to offload more customer traffic to their peers without complaint? 2011/12/15 Jeff Wheeler <jsw@inconcepts.biz>
Had in interesting conversation with a transit AS on behalf of a customer where I found out they are using communities to raise the local
On Thu, Dec 15, 2011 at 1:07 AM, Keegan Holley <keegan.holley@sungard.com> wrote: preference
That sounds like a disreputable practice.
While not quite as obvious, some large transit ASes, like Level3, reset the origin to I (best) sometime between when they learn it and when they announce it to their customers and peers. This similarly causes them to suck in a bit more traffic than they might otherwise.
-- Jeff S Wheeler <jsw@inconcepts.biz> Sr Network Operator / Innovative Network Concepts
On Thu, Dec 15, 2011 at 2:24 AM, Keegan Holley <keegan.holley@sungard.com> wrote:
I always assumed that taking in more traffic was a bad thing. I've heard about one sided peering agreements where one side is sending more traffic than the other needs them to transport. Am I missing something? Would this cause a shift in their favor allowing them to offload more customer traffic to their peers without complaint?
Well, if Level3 wanted less ingress traffic, they would probably stop this practice. I would imagine they thought about it carefully. -- Jeff S Wheeler <jsw@inconcepts.biz> Sr Network Operator / Innovative Network Concepts
In a message written on Thu, Dec 15, 2011 at 02:24:13AM -0500, Keegan Holley wrote:
I always assumed that taking in more traffic was a bad thing. I've heard about one sided peering agreements where one side is sending more traffic than the other needs them to transport. Am I missing something? Would this cause a shift in their favor allowing them to offload more customer traffic to their peers without complaint?
It's one of many techniques used by peers to "balance" the ratio. However, there may be a simpler explanation. If you bill by the bit as a transit provider it's in your best interest to make sure your customer gets as many bits through you as possible. Plus if you can fill their pipe, they need to buy an upgrade to you. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
On Thursday, December 15, 2011 10:42:37 PM Leo Bicknell wrote:
However, there may be a simpler explanation. If you bill by the bit as a transit provider it's in your best interest to make sure your customer gets as many bits through you as possible. Plus if you can fill their pipe, they need to buy an upgrade to you.
Indeed. Mark.
On Thursday, December 15, 2011 10:42:37 PM Leo Bicknell wrote:
However, there may be a simpler explanation. If you bill by the bit as a transit provider it's in your best interest to make sure your customer gets as many bits through you as possible. Plus if you can fill their pipe, they need to buy an upgrade to you.
Indeed.
Forgive my ignorance, but are connections between ISP's normally billed by
2011/12/15 Mark Tinka <mtinka@globaltransit.net> the bit? I'm a transit AS but not an ISP in the traditional sense, so I just have the normal monthly billing.
On Friday, December 16, 2011 12:27:48 AM Keegan Holley wrote:
Forgive my ignorance, but are connections between ISP's normally billed by the bit? I'm a transit AS but not an ISP in the traditional sense, so I just have the normal monthly billing.
Per-bit billing, for us, is not a pre-requisite for us to encourage traffic toward our customers to use the transit link they purchase from us. Mark.
Jeff Wheeler writes:
On Thu, Dec 15, 2011 at 1:07 AM, Keegan Holley <keegan.holley@sungard.com> wrote:
Had in interesting conversation with a transit AS on behalf of a customer where I found out they are using communities to raise the local preference
That sounds like a disreputable practice.
While not quite as obvious, some large transit ASes, like Level3, reset the origin to I (best) sometime between when they learn it and when they announce it to their customers and peers. This similarly causes them to suck in a bit more traffic than they might otherwise.
Once upon a time, UUNET did the opposite by setting origin to unknown for peer routes, in an attempt to prefer customer routes over peer routes. We moved to local preference shortly thereafter as it became clear this was "changing" the routes in some meaningful way; if a customer was multihomed to us and another provider, this might affect path selection. (The original thought was that local pref might be too heavyweight, but of course later it became the standard.) Joe
On Friday, December 16, 2011 05:02:33 AM Joe Malcolm wrote:
Once upon a time, UUNET did the opposite by setting origin to unknown for peer routes, in an attempt to prefer customer routes over peer routes. We moved to local preference shortly thereafter as it became clear this was "changing" the routes in some meaningful way; if a customer was multihomed to us and another provider, this might affect path selection.
This raises an interesting question we've dealt with many a time in our network - outside of situations mandated by governments or some such, are ISP's happy to peer with their customers (where "peer" = settlement-free exchanging of routes/traffic across public interconnects while "customers" = servicing a commercial IP Transit contract)? Mark.
On Sat, Dec 17, 2011 at 12:14 AM, Mark Tinka <mtinka@globaltransit.net> wrote:
On Friday, December 16, 2011 05:02:33 AM Joe Malcolm wrote:
Once upon a time, UUNET did the opposite by setting origin to unknown for peer routes, in an attempt to prefer customer routes over peer routes. We moved to local preference shortly thereafter as it became clear this was "changing" the routes in some meaningful way; if a customer was multihomed to us and another provider, this might affect path selection.
This raises an interesting question we've dealt with many a time in our network - outside of situations mandated by governments or some such, are ISP's happy to peer with their customers (where "peer" = settlement-free exchanging of routes/traffic across public interconnects while "customers" = servicing a commercial IP Transit contract)?
Mark.
I've been able to negotiate peering+transit relationships with providers, but only by threat of total revenue loss; ie "we currently pay you $x million/year; we want your on-net routes as settlement-free routes, and will continue to pay for off-net transit traffic. Otherwise, we will be transferring all that revenue to your competitor, X" This tends to be effective only for content providers, though, where the outbound traffic dominates, and you don't care if the inbound bits are coming over the "pay for" pipe vs the "settlement free" pipe. If you're an inbound-heavy shop, though, this won't really buy you much benefit. (And, if the revenue point isn't in the $x millions/year for the transit provider, they're more likely to just shrug and say "too much hassle...please, go be a headache for our competitor" rather than configuring a dual relationship like that--so it really only works for higher-volume relationships.) Matt
I've had similar experiences to Mr. Petach. Depending on order of operations, you can look at this from a different prospective as well -- why go with a soulless entity for your transit (or transport, collocation, ...) requirements, when you can "keep it in the family" and engage a peer who already understands your service model and is committed to maintaining mutual benefit? Indeed, the old adage of "once a customer, never a peer" could never be wronger. -a
On Sunday, December 18, 2011 04:49:46 AM Adam Rothschild wrote:
Indeed, the old adage of "once a customer, never a peer" could never be wronger.
Socially, "once a customer, then a peer, then a customer again" is even more interesting yet. The second instance of "customer" could rise during M&A situations, where an ISP Z peers with ISP A, but buys transit from ISP B. Then ISP A and ISP B decide to merge. But I suppose this is certainly going way off-topic now :-). Mark.
On Sunday, December 18, 2011 12:32:03 AM Matthew Petach wrote:
I've been able to negotiate peering+transit relationships with providers, but only by threat of total revenue loss; ie "we currently pay you $x million/year; we want your on-net routes as settlement-free routes, and will continue to pay for off-net transit traffic. Otherwise, we will be transferring all that revenue to your competitor, X"
If the customer is taking these on-net routes via an exchange point or private peering arrangement, this should be fairly easy to do. If they choose to take it over the same link as their off- net service, not only does the provider need to support a visible way in which these services can be separated over the same wire, but it may also not make much sense for the customer as there is potential for on-net traffic to hog the link, making the case to upgrade the link for traffic that may not necessarily incentivise them to do so. But it's hard to judge this one, especially if the ISP is large with tons of other on-net customers "talking" to the customer negotiating such an arrangement. I can see ISP's accepting to do this if the ratio of on- net:off-net traffic is disproportionate, in favor of more off-net traffic.
This tends to be effective only for content providers, though, where the outbound traffic dominates, and you don't care if the inbound bits are coming over the "pay for" pipe vs the "settlement free" pipe.
It's also mostly useful where the ISP is sufficiently large in a meaningful way for their on-net routes to make any sense to the downstream customer negotiating such an agreement.
If you're an inbound-heavy shop, though, this won't really buy you much benefit. (And, if the revenue point isn't in the $x millions/year for the transit provider, they're more likely to just shrug and say "too much hassle...please, go be a headache for our competitor" rather than configuring a dual relationship like that--so it really only works for higher-volume relationships.)
Maybe what you meant to say is "if the revenue point isn't high enough" :-). Relatively, different ISP's may be kings in their part of town, but still be small enough to accept fewer dollars for such a deal. On the whole, I can envisage cases where trying to fix this "peering with customers" issue can end up causing inadvertent competition with exchange points. Mark.
On 12/17/11 00:14 , Mark Tinka wrote:
On Friday, December 16, 2011 05:02:33 AM Joe Malcolm wrote:
Once upon a time, UUNET did the opposite by setting origin to unknown for peer routes, in an attempt to prefer customer routes over peer routes. We moved to local preference shortly thereafter as it became clear this was "changing" the routes in some meaningful way; if a customer was multihomed to us and another provider, this might affect path selection.
This raises an interesting question we've dealt with many a time in our network - outside of situations mandated by governments or some such, are ISP's happy to peer with their customers (where "peer" = settlement-free exchanging of routes/traffic across public interconnects while "customers" = servicing a commercial IP Transit contract)?
In the circumstances where I've seen this are rare... We have had transit providers that we used who also peered with us on exchange fabrics for v6 that's about it.
Mark.
On Sunday, December 18, 2011 02:35:37 AM Joel jaeggli wrote:
In the circumstances where I've seen this are rare... We have had transit providers that we used who also peered with us on exchange fabrics for v6 that's about it.
Funny, we have something similar :-). But yes, we've seen this in situations where public exchange points are supported by governments, and license holders are "urged" to peer with all members. Mark.
participants (9)
-
Adam Rothschild -
Holmes,David A -
Jeff Wheeler -
Joe Malcolm -
Joel jaeggli -
Keegan Holley -
Leo Bicknell -
Mark Tinka -
Matthew Petach