I'm doing some research on multiple origin AS problems of IXPs. As I know, generally there are two types of IXPs type 1: use exchange routers, which works in layer 3 type 2: use switches and Ethernet topology, which works in layer 2. So I have a couple of qustions: 1. For type 1, the exchange routers may use several IP prefixes for routing, how often does the IP prefixes have their own AS? 2. For type 2, all peers connected to the IXP must work in the same subnet required by Ethernet rules. Is possible that the subnet IP prefixes belong to some private IP address space, such as 192.168.x.x? How often does this happen? If the subnet only contains public IP addresses, how are the addresses announced? Thanks, Yaoqing
-----Original Message----- From: Yaoqing(Joey) Liu [mailto:joey.liuyq@gmail.com] Sent: Thursday, February 17, 2011 6:03 PM To: nanog@nanog.org Subject: Internet Exchange Point(IXP) questions
I'm doing some research on multiple origin AS problems of IXPs. As I know, generally there are two types of IXPs type 1: use exchange routers, which works in layer 3 type 2: use switches and Ethernet topology, which works in layer 2. So I have a couple of qustions: 1. For type 1, the exchange routers may use several IP prefixes for routing, how often does the IP prefixes have their own AS? 2. For type 2, all peers connected to the IXP must work in the same subnet required by Ethernet rules. Is possible that the subnet IP prefixes belong to some private IP address space, such as 192.168.x.x? How often does this happen? If the subnet only contains public IP addresses, how are the addresses announced?
Thanks, Yaoqing
Hello: On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although we sometimes have to chase down people that announce it. Those addresses aren't the destination for any traffic, they are merely part of the transport to a destination, so there is no need for them to be in the DFZ. Regards, Mike -- Michael K. Smith - CISSP, GSEC, GISP Chief Technical Officer - Adhost Internet LLC mksmith@adhost.com w: +1 (206) 404-9500 f: +1 (206) 404-9050 PGP: B49A DDF5 8611 27F3 08B9 84BB E61E 38C0 (Key ID: 0x9A96777D)
In a message written on Fri, Feb 18, 2011 at 02:17:48AM +0000, Michael K. Smith - Adhost wrote:
On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although we sometimes have to chase down people that announce it. Those addresses aren't the destination for any traffic, they are merely part of the transport to a destination, so there is no need for them to be in the DFZ.
I've had to deal with exchanges like this in the past, and frankly they have always been a pain for the support organization. You see, customers use tools like mtr or Visual Traceroute that do a traceroute and then continuously ping each hop. Many of these customers don't have a default route, or default to their _other_ provider. These tools end up showing 100% loss at the exchange, as they get the traceroute response and then can't ping it. They then open a ticket, and your support organization has to explain to them how all of this works and why it isn't the real cause of their problem. My preference is that the exchange get an ASN, peer with everyone (e.g. from the route server) and announce the exchange prefix. That way it's consistently announced. For exchange that don't do this, I've always put the prefix into BGP in such a way that I will announce it but only to my customers to work around this problem. Please get your own ASN and announce the route, for the sake of all of your members. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although we sometimes have to chase down people that announce it.
I've had to deal with exchanges like this in the past, and frankly they have always been a pain for the support organization.
You see, customers use tools like mtr or Visual Traceroute that do a traceroute and then continuously ping each hop. Many of these customers don't have a default route, or default to their _other_ provider. These tools end up showing 100% loss at the exchange, as they get the traceroute response and then can't ping it.
They then open a ticket, and your support organization has to explain to them how all of this works and why it isn't the real cause of their problem.
<aol>
My preference is that the exchange get an ASN, peer with everyone (e.g. from the route server) and announce the exchange prefix.
i do not like route servers or peering with strange things. treat the exchange as an internal route and announce it within your net and to your customer cone. randy
On Thu, Feb 17, 2011 at 8:17 PM, Michael K. Smith - Adhost < mksmith@adhost.com> wrote:
-----Original Message----- From: Yaoqing(Joey) Liu [mailto:joey.liuyq@gmail.com] Sent: Thursday, February 17, 2011 6:03 PM To: nanog@nanog.org Subject: Internet Exchange Point(IXP) questions
I'm doing some research on multiple origin AS problems of IXPs. As I know, generally there are two types of IXPs type 1: use exchange routers, which works in layer 3 type 2: use switches and Ethernet topology, which works in layer 2. So I have a couple of qustions: 1. For type 1, the exchange routers may use several IP prefixes for routing, how often does the IP prefixes have their own AS? 2. For type 2, all peers connected to the IXP must work in the same subnet required by Ethernet rules. Is possible that the subnet IP prefixes belong to some private IP address space, such as 192.168.x.x? How often does this happen? If the subnet only contains public IP addresses, how are the addresses announced?
Thanks, Yaoqing
Hello:
On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although we sometimes have to chase down people that announce it. Those addresses aren't the destination for any traffic, they are merely part of the transport to a destination, so there is no need for them to be in the DFZ.
But I just checked the IXP prefix list, and found SIX owns prefix 206.81.80.0/23. And it has been announced by three ASNs, AS11537(Internet 2), AS3130(RGnet, LLC) and AS25973(Mzima Networks, Inc). I'm not sure if my info is correct. Does SIX own its own ASN other than the three above? Yaoqing
Regards,
Mike -- Michael K. Smith - CISSP, GSEC, GISP Chief Technical Officer - Adhost Internet LLC mksmith@adhost.com w: +1 (206) 404-9500 f: +1 (206) 404-9050 PGP: B49A DDF5 8611 27F3 08B9 84BB E61E 38C0 (Key ID: 0x9A96777D)
From: Yaoqing(Joey) Liu [mailto:joey.liuyq@gmail.com] Sent: Thursday, February 17, 2011 7:04 PM To: Michael K. Smith - Adhost Cc: nanog@nanog.org Subject: Re: Internet Exchange Point(IXP) questions On Thu, Feb 17, 2011 at 8:17 PM, Michael K. Smith - Adhost <mksmith@adhost.com<mailto:mksmith@adhost.com>> wrote:
-----Original Message----- From: Yaoqing(Joey) Liu [mailto:joey.liuyq@gmail.com<mailto:joey.liuyq@gmail.com>] Sent: Thursday, February 17, 2011 6:03 PM To: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Internet Exchange Point(IXP) questions
I'm doing some research on multiple origin AS problems of IXPs. As I know, generally there are two types of IXPs type 1: use exchange routers, which works in layer 3 type 2: use switches and Ethernet topology, which works in layer 2. So I have a couple of qustions: 1. For type 1, the exchange routers may use several IP prefixes for routing, how often does the IP prefixes have their own AS? 2. For type 2, all peers connected to the IXP must work in the same subnet required by Ethernet rules. Is possible that the subnet IP prefixes belong to some private IP address space, such as 192.168.x.x? How often does this happen? If the subnet only contains public IP addresses, how are the addresses announced?
Thanks, Yaoqing Hello:
On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although we sometimes have to chase down people that announce it. Those addresses aren't the destination for any traffic, they are merely part of the transport to a destination, so there is no need for them to be in the DFZ. But I just checked the IXP prefix list, and found SIX owns prefix 206.81.80.0/23<http://206.81.80.0/23>. And it has been announced by three ASNs, AS11537(Internet 2), AS3130(RGnet, LLC) and AS25973(Mzima Networks, Inc). I'm not sure if my info is correct. Does SIX own its own ASN other than the three above? Yaoqing
-- Michael K. Smith - CISSP, GSEC, GISP Chief Technical Officer - Adhost Internet LLC mksmith@adhost.com w: +1 (206) 404-9500 f: +1 (206) 404-9050 PGP: B49A DDF5 8611 27F3 08B9 84BB E61E 38C0 (Key ID: 0x9A96777D) From: Yaoqing(Joey) Liu [mailto:joey.liuyq@gmail.com] Sent: Thursday, February 17, 2011 7:04 PM To: Michael K. Smith - Adhost Cc: nanog@nanog.org Subject: Re: Internet Exchange Point(IXP) questions On Thu, Feb 17, 2011 at 8:17 PM, Michael K. Smith - Adhost <mksmith@adhost.com> wrote:
-----Original Message----- From: Yaoqing(Joey) Liu [mailto:joey.liuyq@gmail.com] Sent: Thursday, February 17, 2011 6:03 PM To: nanog@nanog.org Subject: Internet Exchange Point(IXP) questions
I'm doing some research on multiple origin AS problems of IXPs. As I know, generally there are two types of IXPs type 1: use exchange routers, which works in layer 3 type 2: use switches and Ethernet topology, which works in layer 2. So I have a couple of qustions: 1. For type 1, the exchange routers may use several IP prefixes for routing, how often does the IP prefixes have their own AS? 2. For type 2, all peers connected to the IXP must work in the same subnet required by Ethernet rules. Is possible that the subnet IP prefixes belong to some private IP address space, such as 192.168.x.x? How often does this happen? If the subnet only contains public IP addresses, how are the addresses announced?
Thanks, Yaoqing Hello:
On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although >we sometimes have to chase down people that announce it. Those addresses aren't the destination for any traffic, they are merely part of the transport to a destination, so there is no need for >them to be in the DFZ.
But I just checked the IXP prefix list, and found SIX owns prefix 206.81.80.0/23. And it has been announced by three ASNs, AS11537(Internet 2), AS3130(RGnet, LLC) and AS25973(Mzima >Networks, Inc). I'm not sure if my info is correct. Does SIX own its own ASN other than the three above?
Sorry for the misfire on my last email. The 206.81.80.0/23 network is assigned to the SIX from ARIN. In general, we don't want people to announce that space to the DFZ, so the three providers listed above are not filtering their announcements properly. It is, as others have said, a good idea to announce the exchange block to your customers, but not out to the DFZ. Regards, Mike
On Fri, Feb 18, 2011 at 1:43 PM, Michael K. Smith - Adhost <mksmith@adhost.com> wrote:
Sorry for the misfire on my last email. The 206.81.80.0/23 network is assigned to the SIX from ARIN. In general, we don't want people to announce that space to the DFZ, so the three providers listed above are not filtering their announcements properly. It is, as others have said, a good idea to announce the exchange block to your customers, but not out to the DFZ.
why is it a good idea to send this to your customers? the next-hop info is surely only useful to your local network? done right it's even only relevant to the IX connected router, right? it seems wholely unusful to your customers. (to me at least)
-----Original Message----- From: christopher.morrow@gmail.com [mailto:christopher.morrow@gmail.com] On Behalf Of Christopher Morrow Sent: Friday, February 18, 2011 11:34 AM To: Michael K. Smith - Adhost Cc: Yaoqing(Joey) Liu; nanog@nanog.org Subject: Re: Internet Exchange Point(IXP) questions
On Fri, Feb 18, 2011 at 1:43 PM, Michael K. Smith - Adhost <mksmith@adhost.com> wrote:
Sorry for the misfire on my last email. The 206.81.80.0/23 network is assigned to the SIX from ARIN. In general, we don't want people to announce that space to the DFZ, so the three providers listed above are not filtering their announcements properly. It is, as others have said, a good idea to announce the exchange block to your customers, but not out to the DFZ.
why is it a good idea to send this to your customers? the next-hop info is surely only useful to your local network? done right it's even only relevant to the IX connected router, right? it seems wholely unusful to your customers. (to me at least)
I was thinking about what Leo said about tools that test each hop through a path. At least my downstream customers will be able to test through the SIX connection if I announce the /23 to them. Mike
On Fri, Feb 18, 2011 at 4:24 PM, Michael K. Smith - Adhost <mksmith@adhost.com> wrote:
-----Original Message----- From: christopher.morrow@gmail.com why is it a good idea to send this to your customers? the next-hop info is surely only useful to your local network? done right it's even only relevant to the IX connected router, right? it seems wholely unusful to your customers. (to me at least)
I was thinking about what Leo said about tools that test each hop through a path. At least my downstream customers will be able to test through the SIX connection if I announce the /23 to them.
hopefully the path to the IXP prefix is the same as to the item they are testing failure of? :)
In a message written on Fri, Feb 18, 2011 at 04:37:05PM -0500, Christopher Morrow wrote:
On Fri, Feb 18, 2011 at 4:24 PM, Michael K. Smith - Adhost
I was thinking about what Leo said about tools that test each hop through a path. At least my downstream customers will be able to test through the SIX connection if I announce the /23 to them.
hopefully the path to the IXP prefix is the same as to the item they are testing failure of? :)
Of course it isn't. Perhaps you missed my implication in the original mail I wrote. :) The customers cloging up your help desk with this sort of stuff are idiots. Unfortunately that's where the majority of your helpdesk time goes... -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
On Fri, Feb 18, 2011 at 4:46 PM, Leo Bicknell <bicknell@ufp.org> wrote:
The customers cloging up your help desk with this sort of stuff are idiots. Unfortunately that's where the majority of your helpdesk time goes...
i admit to missing it :( but yes, now with the explanation, I get your point :)
On 2011-02-18, at 14:34, Christopher Morrow wrote:
On Fri, Feb 18, 2011 at 1:43 PM, Michael K. Smith - Adhost <mksmith@adhost.com> wrote:
Sorry for the misfire on my last email. The 206.81.80.0/23 network is assigned to the SIX from ARIN. In general, we don't want people to announce that space to the DFZ, so the three providers listed above are not filtering their announcements properly. It is, as others have said, a good idea to announce the exchange block to your customers, but not out to the DFZ.
why is it a good idea to send this to your customers? the next-hop info is surely only useful to your local network? done right it's even only relevant to the IX connected router, right? it seems wholely unusful to your customers. (to me at least)
Well, except for the reason that Leo mentioned. The NEXT_HOP in the exchange point subnet will not make it to the customer router. It's not a transitive attribute. The customer will see a NEXT_HOP corresponding to the provider router (or whatever they decide to re-write it as). See RFC 4271 section 5.1.3. Joe
In a message written on Fri, Feb 18, 2011 at 02:34:21PM -0500, Christopher Morrow wrote:
why is it a good idea to send this to your customers? the next-hop info is surely only useful to your local network? done right it's even only relevant to the IX connected router, right? it seems wholely unusful to your customers. (to me at least)
If by "done right" you mean perhaps a feature like returning ICMP's from a loopback IP rather than the interface IP, there are two issues with that: The far end ISP controls this feature. If they don't enable it you must work around by announcing the prefix to your customer. One person doing it wrong at the exchange is enough that you have to work around it. I at least find it useful when traceroute shows the interface. I believe it saves time for your NOC, and burning IP's for interfaces makes a lot of sense in terms of speeding troubleshooting. Even if all of my gear allowed me to send ICMP's from the loopback it's quite likely I would not use that feature. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
On Fri, Feb 18, 2011 at 4:29 PM, Leo Bicknell <bicknell@ufp.org> wrote:
In a message written on Fri, Feb 18, 2011 at 02:34:21PM -0500, Christopher Morrow wrote:
why is it a good idea to send this to your customers? the next-hop info is surely only useful to your local network? done right it's even only relevant to the IX connected router, right? it seems wholely unusful to your customers. (to me at least)
If by "done right" you mean perhaps a feature like returning ICMP's from a loopback IP rather than the interface IP, there are two issues with
sorry, I was only talking|thinking about routing bits, I missed your point about people being able to ping an IX interface... I'd submit that in many networks the path to the nexthop may be a vastly different one than the path to 'the broken thing' through the isp/ixp/isp set of routers. I meant: "Is the nexthop in your (the ixp connected isp) network the IXP interface IP, or the loopback of your IXP connected router?" 'Done right' (I agree that this is an individual perspective) here meant, to me, that the IXP prefix wasn't necessary in the IXP connected ISP's network, reset to loopback in ibgp policy and never send the IXP prefix (connected route) off the IXP connected router. leaking the IX prefix to customers, to me, seems like a recipe for much wider/unintended leakage :(
In a message written on Fri, Feb 18, 2011 at 04:36:28PM -0500, Christopher Morrow wrote:
leaking the IX prefix to customers, to me, seems like a recipe for much wider/unintended leakage :(
Oh, it is. I remember when MAE-EAST was injected by at least 50 people into the DFZ because back then people weren't careful enough to just send such things to customers. AMS-IX (and others) have the better solution. They have AS1200, announce the exchange LAN from AS1200 (195.69.144.0/22). They will peer with you if you are at the exchange, see http://www.ams-ix.net/as1200-peering/. I believe, but can't find a reference really quick that they get transit for it from a couple of providers so those that don't peer still have the route. I mean really, you have a block. If your IXP matters it's already taking up space in all of the largest ISP's tables anyway, so there's no "saving a route argument". Get an ASN, which since your multi-homed is trivial, announce the block from there and peer with your exchange participants. Everyone is happy, the route is consistent, and life is good. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
Hi, On Fri, 18 Feb 2011 13:44:56 -0800 Leo Bicknell <bicknell@ufp.org> wrote:
In a message written on Fri, Feb 18, 2011 at 04:36:28PM -0500, Christopher Morrow wrote:
leaking the IX prefix to customers, to me, seems like a recipe for much wider/unintended leakage :(
Oh, it is. I remember when MAE-EAST was injected by at least 50 people into the DFZ because back then people weren't careful enough to just send such things to customers.
AMS-IX (and others) have the better solution. They have AS1200, announce the exchange LAN from AS1200 (195.69.144.0/22). They will peer with you if you are at the exchange, see http://www.ams-ix.net/as1200-peering/. I believe, but can't find a reference really quick that they get transit for it from a couple of providers so those that don't peer still have the route.
We advertise 195.69.144.0/22 with no-export. Kind regards, Martin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 17, 2011, at 6:03 PM, Yaoqing(Joey) Liu wrote:
As I know, generally there are two types of IXPs
This is incorrect.
type 1: use exchange routers, which works in layer 3
This is not an IXP. This is a router. That router would be owned by someone, who would have some sort of policy in the router, which would make it an Internet service provider, not an Internet exchange point.
type 2: use switches and Ethernet topology, which works in layer 2.
This is an IXP. Routers belonging to Internet service providers, communicating with each other across a switch fabric, which is an Internet exchange point.
1. For type 1, the exchange routers may use several IP prefixes for routing, how often does the IP prefixes have their own AS?
Since this is not an IXP, I think the question is irrelevant to your research. If an ISP wants to participate in BGP routing, and originate an IP prefix, that ISP must have an AS.
2. For type 2, all peers connected to the IXP must work in the same subnet required by Ethernet rules.
Generally, yes, though some IXPs are not that prescriptive, and would allow a subset of the ISPs to peer on a different subnet if they wished.
Is possible that the subnet IP prefixes belong to some private IP address space, such as 192.168.x.x?
It is possible, but it does not follow best-practices, because it breaks traceroute and other diagnostic tools.
How often does this happen?
Very very rarely. Only two IXPs out of more than three hundred are using FRC1918 space at this point: Maputo and Santiago de Compostela. This used to be a more common mistake, but as communications with the operators of new IXPs has improved over time, it's become very rare.
If the subnet only contains public IP addresses, how are the addresses announced?
They are generally not announced. Occasionally they're announced by one or more participating ISPs at the IXP. Sometimes that's purposeful, other times it's accidental. Some IXPs have rules prohibiting the announcement of the exchange subnet, others actively seek out sources of transit for the exchange subnet. -Bill Woodcock Research Director Packet Clearing House -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) iEYEARECAAYFAk1d5I8ACgkQGvQy4xTRsBFXOwCfbsutsSyYHHwQu5W06BgasXQm QNgAoMScxNcjOLQNdJC5mz4enD1/839f =6iFI -----END PGP SIGNATURE-----
type 1: use exchange routers, which works in layer 3 This is not an IXP. This is a router. That router would be owned by someone, who would have some sort of policy in the router, which would make it an Internet service provider, not an Internet exchange point.
this from the guy who pushed "layer three exchange points" for years? rofl!
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 17, 2011, at 7:24 PM, Randy Bush wrote:
this from the guy who pushed "layer three exchange points" for years? rofl!
I was one of the people who built one in 1994, and used it quite happily for a few years, until it had outlasted its need. Do you have something else in mind? Or are you just trying to keep your blood pressure up? -Bill -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) iEYEARECAAYFAk1eAPEACgkQGvQy4xTRsBFWSwCfcmER1ApNJDCYxUh34tTTBd/e e8sAoLxQ4Q3U1//nOuBF6KLSsQS2K0MD =Rgi7 -----END PGP SIGNATURE-----
participants (8)
-
Bill Woodcock -
Christopher Morrow -
Joe Abley -
Leo Bicknell -
Martin Pels -
Michael K. Smith - Adhost -
Randy Bush -
Yaoqing(Joey) Liu