I am starting up a small IX. The thought process was a /24 for every IX location (there will be multiple of them geographically disparate), even though we never expected anywhere near that many on a given fabric. Then okay, how do we do v6? We got a /48, so the thought was a /64 for each. That one seems more cut and dry. A NANOG BCOP says to subnet no smaller than /64, so that makes sense to have one for each location. However, that brings me back to v4. Should I be cutting that /24 down into say /25s or /26s? We don't expect to have more than a /27 worth of networks at any one location, so a /26 should provide enough risk avoidance in not re-numbering an IX. That said... maybe best practice is to just leave it as /24. That's what I've seen at the other small IXes. Yes, I looked at NANOG's BCOPs and an article put out by Euro-IX. Didn't see much there. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
On Sat, 04 Apr 2015 16:06:02 -0500, Mike Hammett said:
I am starting up a small IX. The thought process was a /24 for every IX location (there will be multiple of them geographically disparate), even though we nqever expected anywhere near that many on a given fabric. Then okay, how do < we d o v6? We got a /48, so the thought was a /64 for each.
You probably want a /56 for each so you can hand a /64 to each customner. That way, customer isolation becomes easy because it's a routing problem. If customers share a subnet, it gets a little harder....
That makes sense. I do recall now reading about having that 8 bit separation between tiers of networks. However, in an IX everyone is supposed to be able to talk to everyone else. Traditionally (AFAIK), it's all been on the same subnet. At least the ones I've been involved with have been single subnets, but that's v4 too. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com ----- Original Message ----- From: "Valdis Kletnieks" <Valdis.Kletnieks@vt.edu> To: "Mike Hammett" <nanog@ics-il.net> Cc: "NANOG" <nanog@nanog.org> Sent: Saturday, April 4, 2015 5:49:37 PM Subject: Re: Small IX IP Blocks On Sat, 04 Apr 2015 16:06:02 -0500, Mike Hammett said:
I am starting up a small IX. The thought process was a /24 for every IX location (there will be multiple of them geographically disparate), even though we nqever expected anywhere near that many on a given fabric. Then okay, how do < we d o v6? We got a /48, so the thought was a /64 for each.
You probably want a /56 for each so you can hand a /64 to each customner. That way, customer isolation becomes easy because it's a routing problem. If customers share a subnet, it gets a little harder....
IPv4 and IPv6 subnets are different. While a single IPv4 is taken to be a single device, an IPv6 /64 is designed to be treated as an end user subnet. https://tools.ietf.org/html/rfc3177 section 3. On 05/04/2015 9:05 am, "Mike Hammett" <nanog@ics-il.net> wrote:
That makes sense. I do recall now reading about having that 8 bit separation between tiers of networks. However, in an IX everyone is supposed to be able to talk to everyone else. Traditionally (AFAIK), it's all been on the same subnet. At least the ones I've been involved with have been single subnets, but that's v4 too.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
----- Original Message -----
From: "Valdis Kletnieks" <Valdis.Kletnieks@vt.edu> To: "Mike Hammett" <nanog@ics-il.net> Cc: "NANOG" <nanog@nanog.org> Sent: Saturday, April 4, 2015 5:49:37 PM Subject: Re: Small IX IP Blocks
On Sat, 04 Apr 2015 16:06:02 -0500, Mike Hammett said:
I am starting up a small IX. The thought process was a /24 for every IX location (there will be multiple of them geographically disparate), even though we nqever expected anywhere near that many on a given fabric. Then okay, how do < we d o v6? We got a /48, so the thought was a /64 for each.
You probably want a /56 for each so you can hand a /64 to each customner.
That way, customer isolation becomes easy because it's a routing problem. If customers share a subnet, it gets a little harder....
Okay, so I decided to look at what current IXes are doing. It looks like AMS-IX, Equinix and Coresite as well as some of the smaller IXes are all using /64s for their IX fabrics. Seems to be a slam dunk then as how to handle the IPv6. We've got a /48, so a /64 per IX. For all of those advocating otherwise, do you have much experience with IXes? Multiple people talked about routing. There is no routing within an IX. I may grow, but an IX in a tier-2 American city will never scale larger than AMS-IX. If it's good enough for them, it's good enough for me. Back to v4, I went through a few pages of PeeringDB and most everyone used a /24 or larger. INEX appears to use a /25 for each of their segments. IX Australia uses mainly /24s, but two locations split a /24 into /25s. A couple of the smaller single location US IXes used /25s and /26s. It seems there's precedent for people using smaller than /24s, but it's not overly common. Cash and address space preservation. What does the community think about IXes on smaller than /24s? ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com ----- Original Message ----- From: "Brendan Halley" <brendan@halley.net.au> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Saturday, April 4, 2015 6:10:34 PM Subject: Re: Small IX IP Blocks IPv4 and IPv6 subnets are different. While a single IPv4 is taken to be a single device, an IPv6 /64 is designed to be treated as an end user subnet. https://tools.ietf.org/html/rfc3177 section 3. On 05/04/2015 9:05 am, "Mike Hammett" < nanog@ics-il.net > wrote: That makes sense. I do recall now reading about having that 8 bit separation between tiers of networks. However, in an IX everyone is supposed to be able to talk to everyone else. Traditionally (AFAIK), it's all been on the same subnet. At least the ones I've been involved with have been single subnets, but that's v4 too. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com ----- Original Message ----- From: "Valdis Kletnieks" < Valdis.Kletnieks@vt.edu > To: "Mike Hammett" < nanog@ics-il.net > Cc: "NANOG" < nanog@nanog.org > Sent: Saturday, April 4, 2015 5:49:37 PM Subject: Re: Small IX IP Blocks On Sat, 04 Apr 2015 16:06:02 -0500, Mike Hammett said:
I am starting up a small IX. The thought process was a /24 for every IX location (there will be multiple of them geographically disparate), even though we nqever expected anywhere near that many on a given fabric. Then okay, how do < we d o v6? We got a /48, so the thought was a /64 for each.
You probably want a /56 for each so you can hand a /64 to each customner. That way, customer isolation becomes easy because it's a routing problem. If customers share a subnet, it gets a little harder....
Mike, I think it's fine to cut it up smaller than /24, and might actually help in keeping people from routing the IX prefix globally. -Laszlo On Apr 5, 2015, at 12:35 AM, Mike Hammett <nanog@ics-il.net> wrote:
Okay, so I decided to look at what current IXes are doing.
It looks like AMS-IX, Equinix and Coresite as well as some of the smaller IXes are all using /64s for their IX fabrics. Seems to be a slam dunk then as how to handle the IPv6. We've got a /48, so a /64 per IX. For all of those advocating otherwise, do you have much experience with IXes? Multiple people talked about routing. There is no routing within an IX. I may grow, but an IX in a tier-2 American city will never scale larger than AMS-IX. If it's good enough for them, it's good enough for me.
Back to v4, I went through a few pages of PeeringDB and most everyone used a /24 or larger. INEX appears to use a /25 for each of their segments. IX Australia uses mainly /24s, but two locations split a /24 into /25s. A couple of the smaller single location US IXes used /25s and /26s. It seems there's precedent for people using smaller than /24s, but it's not overly common. Cash and address space preservation. What does the community think about IXes on smaller than /24s?
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
----- Original Message -----
From: "Brendan Halley" <brendan@halley.net.au> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Saturday, April 4, 2015 6:10:34 PM Subject: Re: Small IX IP Blocks
IPv4 and IPv6 subnets are different. While a single IPv4 is taken to be a single device, an IPv6 /64 is designed to be treated as an end user subnet. https://tools.ietf.org/html/rfc3177 section 3. On 05/04/2015 9:05 am, "Mike Hammett" < nanog@ics-il.net > wrote:
That makes sense. I do recall now reading about having that 8 bit separation between tiers of networks. However, in an IX everyone is supposed to be able to talk to everyone else. Traditionally (AFAIK), it's all been on the same subnet. At least the ones I've been involved with have been single subnets, but that's v4 too.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
----- Original Message -----
From: "Valdis Kletnieks" < Valdis.Kletnieks@vt.edu > To: "Mike Hammett" < nanog@ics-il.net > Cc: "NANOG" < nanog@nanog.org > Sent: Saturday, April 4, 2015 5:49:37 PM Subject: Re: Small IX IP Blocks
On Sat, 04 Apr 2015 16:06:02 -0500, Mike Hammett said:
I am starting up a small IX. The thought process was a /24 for every IX location (there will be multiple of them geographically disparate), even though we nqever expected anywhere near that many on a given fabric. Then okay, how do < we d o v6? We got a /48, so the thought was a /64 for each.
You probably want a /56 for each so you can hand a /64 to each customner.
That way, customer isolation becomes easy because it's a routing problem. If customers share a subnet, it gets a little harder....
I've been involved in IX renumbering efforts because exchange(s) decided to use /25's instead of /24's. It's painful because troubleshooting can be a little difficult as differing subnetmasks are in play. If you have the address space, use a /24. ARIN has IPv4 address space specifically reserved for the use by IXPs. charles On Sat, Apr 4, 2015 at 8:35 PM, Mike Hammett <nanog@ics-il.net> wrote:
Okay, so I decided to look at what current IXes are doing.
It looks like AMS-IX, Equinix and Coresite as well as some of the smaller IXes are all using /64s for their IX fabrics. Seems to be a slam dunk then as how to handle the IPv6. We've got a /48, so a /64 per IX. For all of those advocating otherwise, do you have much experience with IXes? Multiple people talked about routing. There is no routing within an IX. I may grow, but an IX in a tier-2 American city will never scale larger than AMS-IX. If it's good enough for them, it's good enough for me.
Back to v4, I went through a few pages of PeeringDB and most everyone used a /24 or larger. INEX appears to use a /25 for each of their segments. IX Australia uses mainly /24s, but two locations split a /24 into /25s. A couple of the smaller single location US IXes used /25s and /26s. It seems there's precedent for people using smaller than /24s, but it's not overly common. Cash and address space preservation. What does the community think about IXes on smaller than /24s?
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
----- Original Message -----
From: "Brendan Halley" <brendan@halley.net.au> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Saturday, April 4, 2015 6:10:34 PM Subject: Re: Small IX IP Blocks
IPv4 and IPv6 subnets are different. While a single IPv4 is taken to be a single device, an IPv6 /64 is designed to be treated as an end user subnet. https://tools.ietf.org/html/rfc3177 section 3. On 05/04/2015 9:05 am, "Mike Hammett" < nanog@ics-il.net > wrote:
That makes sense. I do recall now reading about having that 8 bit separation between tiers of networks. However, in an IX everyone is supposed to be able to talk to everyone else. Traditionally (AFAIK), it's all been on the same subnet. At least the ones I've been involved with have been single subnets, but that's v4 too.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
----- Original Message -----
From: "Valdis Kletnieks" < Valdis.Kletnieks@vt.edu > To: "Mike Hammett" < nanog@ics-il.net > Cc: "NANOG" < nanog@nanog.org > Sent: Saturday, April 4, 2015 5:49:37 PM Subject: Re: Small IX IP Blocks
On Sat, 04 Apr 2015 16:06:02 -0500, Mike Hammett said:
I am starting up a small IX. The thought process was a /24 for every IX location (there will be multiple of them geographically disparate), even though we nqever expected anywhere near that many on a given fabric. Then okay, how do < we d o v6? We got a /48, so the thought was a /64 for each.
You probably want a /56 for each so you can hand a /64 to each customner.
That way, customer isolation becomes easy because it's a routing problem. If customers share a subnet, it gets a little harder....
On Apr 4, 2015, at 7:28 PM, Charles Gucker <cgucker@onesc.net> wrote:
I've been involved in IX renumbering efforts because exchange(s) decided to use /25's instead of /24's. It's painful because troubleshooting can be a little difficult as differing subnetmasks are in play. If you have the address space, use a /24. ARIN has IPv4 address space specifically reserved for the use by IXPs.
Yes. Listen to Charlie. We did a bunch of analysis on size of IXP subnets, and how it changes over time, relative to the age of the IXP. To summarize drastically, the first /24 typically lasts about 15-18 years. Only a tiny handful of exchanges (less than 2%) are actually supporting more than 254 participants yet at this point. That number will continue to grow over time. At the same time, it’s not worth the trouble of renumbering more than once in that time period, so don’t be penny-wise and pound-foolish by trying to use a /25, particularly when ARIN hands out /24s to IXPs specifically to keep them from running into that trap. -Bill
+1 I worked for a provider until recently that happened to get an IP assignment at an IXP that was transitioning from /25 to /24. It was painful chasing down peers to get them to change their netmask just so we could connect. This went on for several months dealing with the peering/network contacts of whom many of them didn't know the mask had changed in the first place. Paul -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Bill Woodcock Sent: Saturday, April 4, 2015 10:36 PM To: Mike Hammett Cc: NANOG list Subject: Re: Small IX IP Blocks
On Apr 4, 2015, at 7:28 PM, Charles Gucker <cgucker@onesc.net> wrote:
I've been involved in IX renumbering efforts because exchange(s) decided to use /25's instead of /24's. It's painful because troubleshooting can be a little difficult as differing subnetmasks are in play. If you have the address space, use a /24. ARIN has IPv4 address space specifically reserved for the use by IXPs.
Yes. Listen to Charlie. We did a bunch of analysis on size of IXP subnets, and how it changes over time, relative to the age of the IXP. To summarize drastically, the first /24 typically lasts about 15-18 years. Only a tiny handful of exchanges (less than 2%) are actually supporting more than 254 participants yet at this point. That number will continue to grow over time. At the same time, it's not worth the trouble of renumbering more than once in that time period, so don't be penny-wise and pound-foolish by trying to use a /25, particularly when ARIN hands out /24s to IXPs specifically to keep them from running into that trap. -Bill
On 5 Apr 2015, at 04:29, Paul Stewart <paul@paulstewart.org> wrote:
I worked for a provider until recently that happened to get an IP assignment at an IXP that was transitioning from /25 to /24. It was painful chasing down peers to get them to change their netmask just so we could connect. This went on for several months dealing with the peering/network contacts of whom many of them didn't know the mask had changed in the first place.
If you had problems peering because other participants have the wrong netmask, the IXP is not being operated correctly. It’s such a very bad thing to have the incorrect netmask on interfaces (think, more-specifics, route leaks, etc) that the IXP should manage the netmask change process itself - in fact to the point of disconnecting networks who do not configure it correctly. When we renumbered LONAP from /24 to /22, we had to change netblocks too. I can’t recall if we had any netmask problems too but it seems perfectly possible if lazy people just went %s/193.203.5/5.57.80/g. So we did check for that - it’s quite a simple task. From an IXP user point of view, the change was easier for J users, but we built a config validator/renumbererer for C IOS users to help them out. (‘paste your config in this webform’ ‘examine the output’ sort of thing) Will
On 5/Apr/15 02:35, Mike Hammett wrote:
Okay, so I decided to look at what current IXes are doing.
It looks like AMS-IX, Equinix and Coresite as well as some of the smaller IXes are all using /64s for their IX fabrics. Seems to be a slam dunk then as how to handle the IPv6. We've got a /48, so a /64 per IX. For all of those advocating otherwise, do you have much experience with IXes? Multiple people talked about routing. There is no routing within an IX. I may grow, but an IX in a tier-2 American city will never scale larger than AMS-IX. If it's good enough for them, it's good enough for me.
Back to v4, I went through a few pages of PeeringDB and most everyone used a /24 or larger. INEX appears to use a /25 for each of their segments. IX Australia uses mainly /24s, but two locations split a /24 into /25s. A couple of the smaller single location US IXes used /25s and /26s. It seems there's precedent for people using smaller than /24s, but it's not overly common. Cash and address space preservation. What does the community think about IXes on smaller than /24s?
Your main issue with a smaller IPv4 subnet is when you grow, you'll end up having to renumber. This has hit some large exchange points in the recent past. Of course, it's easy to say that you won't grow beyond X members now, but there's no knowing how that will go if you're working hard at your project. Mark.
On Sat, 2015-04-04 at 18:02 -0500, Mike Hammett wrote:
That makes sense. I do recall now reading about having that 8 bit separation between tiers of networks. However, in an IX everyone is supposed to be able to talk to everyone else. Traditionally (AFAIK), it's all been on the same subnet. At least the ones I've been involved with have been single subnets, but that's v4 too.
Think flexible, think big, think future. Limiting yourself to tiny subnets and assuming your circumstances and requirements will not change is a recipe for difficult times ahead. Go as large as you can now, and route between participants. They might not always be friends with each other, or indeed with you, and the ability to isolate, redirect, offload, recombine and filter is critical to the flexibility of your (future) product offering. Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@biplane.com.au) http://www.biplane.com.au/kauer http://twitter.com/kauer389 GPG fingerprint: 3C41 82BE A9E7 99A1 B931 5AE7 7638 0147 2C3C 2AC4 Old fingerprint: EC67 61E2 C2F6 EB55 884B E129 072B 0AF0 72AA 9882
participants (10)
-
Bill Woodcock -
Brendan Halley -
Charles Gucker -
Karl Auer -
Laszlo Hanyecz -
Mark Tinka -
Mike Hammett -
Paul Stewart -
Valdis.Kletnieks＠vt.edu -
Will Hargrave