Starting a greenfield carrier backbone network that can scale to national and international service. What would you do?
Hello everyone, It's been some time since I've been subscribed/replied/posted here (or on WISPA for that matter). I've been pretty busy running a non profit startup (protip: don't do that. It's really really terrible) :) I'm cofounder and CTO of the Free Networking Foundation. Our goal is to bring broadband (5 mbps symmetric to start) bandwidth to the 2/3 of Americans who currently can't get it (rural, urban core, undeserved, "$ILEC stops on otherside of street" etc). Efforts so far primarily have consisted of WiFI last (square) mile delivery using Ubiquiti hardware and the qmp.cat firmware (also meraki access points that were donated, for some reason this seems to happen quite a bit). We've helped numerous networks get started, grow and (soon we hope) become self sustaining in Austin, Kansas City, Los Angeles, Detroit, New York and a few other places throughout the US. The networks are in various stages of maturity of course, but a number of them are fully operational and passing real traffic. Especially the one in Kansas City (it spans both states). These are (point to point, routed) access/distribution networks which connect into colocation providers blended networks. So that's the background and current state of affairs. Not really NANOG material. The next step is to secure our v6 space and AS number. Now that's not horribly difficult or really worthy of NANOG (though I do greatly appreciate folks on the list who helped me through the theory/practice of that process sometime ago). It appears to be fairly straightforward if you are not an LIR. Simply go through the paperwork (LOA, submit to ARIN, get out the credit card, textbook BGP config and done). And if FNF was operating the networks (we don't, we just help with organizing/consulting/software guidance/hardware spend optimization/logistics etc) and if there was just one POP (and associated administrative body), then again it wouldn't be that interesting or worth cluttering up NANOG. FNF goal is to serve as an LIR, SWIPing out /48 chunks to neighborhood level operators. They would then peer with whatever upstream ISPs are regionally close and announce out the space. This of course would be associated with a training program, registration in an IPAM tool etc. Regarding the above? What do the operators on this list wish they could of been trained in starting out? I mean obviously they should have good mastery and working experience of CCNA level material, along with exposure to higher level concepts of WAN networking. What are the tricks, the gotchas, the "man that would of saved my company a million bucks in transit costs". Yes I realize these sort of things are usually closely held. I also am striving to create an entirely new breed of operators running BGP enabled sites with ipv6. The more I can do to help ease those folks integration into the internet, the better. In short, the often debated issue on this list of v6 endpoint explosion is going to be very very very real. What IPAM tools out there can scale to a multi hundred million node, distributed, "eventual consistency" national level? (I've been working closely with guifi.net, and we are attempting to relaunch that as a very slick Apple like experience with a libremap (couchdb based) system. I'd love to hear from folks across the spectrum of experience and network size. From folks who have been dual homed for <~1 year at a single site, to tier1 operators who were there when it all started. So what would you like to see done in a greenfield, open source, open governance carrier backbone network? What would a dream TIER1 (and I use that in the default free zone sense of the word) look like to you? Also how the heck would one get this bootstrapped at a sustainable pace? Would one create numerous tier2 regional carriers, and they would feed into an over arching tier1? I'm thinking something like a 501c8 type structure ( http://www.irs.gov/Charities-&-Non-Profits/Other-Non-Profits/Fraternal-Societies[1] ) As far as I know, this is the first time that an intentional community type approach is taken and a tier1 is the end goal. Not evolving into one, buying ones way into it, but a manifest destiny type approach to building a backbone. Please feel free to reach out to me directly (charles@thefnf.org[2] ) if you wish to have a one on one discussion. In particular I'm interested in legal expertise in these sort of areas (law/compliance/contracting/negotiations for right of way etc etc etc). Thanks for reading. I look forward to the discussion! PS: Yes, I'm young and idealistic. I'm also grounded/practical/focused. I'm currently working on making the access portion of the network as smooth and turnkey as possible. (That basically means packaging up zeroshell/observium/powerdns/libremap/trigger and other bits/bobs into a nice livecd/ova/openvz package). I also like to think about the next wave of issues while working on the current one. It will take another year or so before we need to really be building out the backbone (if nothing else, to link up the rapidly growing regional networks). This is about physical, layer 1 infrastructure. This isn't yet another overlay network (CJDNS/GNu FreeNet etc). Yes it's messy, yes it's all about non technical end users, yes it's about taking a rather complex stack (auth/network awareness/routing platform) and making it accessible to power users/"IT professionals". It's also a whole lot of fun! Please feel free to visit us at https://www.thefnf.org for more information.
Let's start with your basic assumption here. Why would you build a backbone at all if your goal is to solve last mile problems? It seems to me that the expense and distraction of building a large backbone network doesn't contribute to your goals at all, given that there are many high quality, nationwide backbone networks in North America today available at reasonable cost. On Thu, 3 Apr 2014, charles@thefnf.org wrote:
Hello everyone,
It's been some time since I've been subscribed/replied/posted here (or on WISPA for that matter). I've been pretty busy running a non profit startup (protip: don't do that. It's really really terrible) :) I'm cofounder and CTO of the Free Networking Foundation. Our goal is to bring broadband (5 mbps symmetric to start) bandwidth to the 2/3 of Americans who currently can't get it (rural, urban core, undeserved, "$ILEC stops on otherside of street" etc).
Efforts so far primarily have consisted of WiFI last (square) mile delivery using Ubiquiti hardware and the qmp.cat firmware (also meraki access points that were donated, for some reason this seems to happen quite a bit). We've helped numerous networks get started, grow and (soon we hope) become self sustaining in Austin, Kansas City, Los Angeles, Detroit, New York and a few other places throughout the US. The networks are in various stages of maturity of course, but a number of them are fully operational and passing real traffic. Especially the one in Kansas City (it spans both states).
These are (point to point, routed) access/distribution networks which connect into colocation providers blended networks.
So that's the background and current state of affairs. Not really NANOG material.
The next step is to secure our v6 space and AS number. Now that's not horribly difficult or really worthy of NANOG (though I do greatly appreciate folks on the list who helped me through the theory/practice of that process sometime ago). It appears to be fairly straightforward if you are not an LIR. Simply go through the paperwork (LOA, submit to ARIN, get out the credit card, textbook BGP config and done). And if FNF was operating the networks (we don't, we just help with organizing/consulting/software guidance/hardware spend optimization/logistics etc) and if there was just one POP (and associated administrative body), then again it wouldn't be that interesting or worth cluttering up NANOG.
FNF goal is to serve as an LIR, SWIPing out /48 chunks to neighborhood level operators. They would then peer with whatever upstream ISPs are regionally close and announce out the space. This of course would be associated with a training program, registration in an IPAM tool etc.
Regarding the above?
What do the operators on this list wish they could of been trained in starting out? I mean obviously they should have good mastery and working experience of CCNA level material, along with exposure to higher level concepts of WAN networking. What are the tricks, the gotchas, the "man that would of saved my company a million bucks in transit costs". Yes I realize these sort of things are usually closely held. I also am striving to create an entirely new breed of operators running BGP enabled sites with ipv6. The more I can do to help ease those folks integration into the internet, the better. In short, the often debated issue on this list of v6 endpoint explosion is going to be very very very real.
What IPAM tools out there can scale to a multi hundred million node, distributed, "eventual consistency" national level? (I've been working closely with guifi.net, and we are attempting to relaunch that as a very slick Apple like experience with a libremap (couchdb based) system.
I'd love to hear from folks across the spectrum of experience and network size. From folks who have been dual homed for <~1 year at a single site, to tier1 operators who were there when it all started.
So what would you like to see done in a greenfield, open source, open governance carrier backbone network? What would a dream TIER1 (and I use that in the default free zone sense of the word) look like to you?
Also how the heck would one get this bootstrapped at a sustainable pace? Would one create numerous tier2 regional carriers, and they would feed into an over arching tier1? I'm thinking something like a 501c8 type structure ( http://www.irs.gov/Charities-&-Non-Profits/Other-Non-Profits/Fraternal-Societies[1] )
As far as I know, this is the first time that an intentional community type approach is taken and a tier1 is the end goal. Not evolving into one, buying ones way into it, but a manifest destiny type approach to building a backbone.
Please feel free to reach out to me directly (charles@thefnf.org[2] ) if you wish to have a one on one discussion. In particular I'm interested in legal expertise in these sort of areas (law/compliance/contracting/negotiations for right of way etc etc etc).
Thanks for reading. I look forward to the discussion!
PS: Yes, I'm young and idealistic. I'm also grounded/practical/focused. I'm currently working on making the access portion of the network as smooth and turnkey as possible. (That basically means packaging up zeroshell/observium/powerdns/libremap/trigger and other bits/bobs into a nice livecd/ova/openvz package). I also like to think about the next wave of issues while working on the current one. It will take another year or so before we need to really be building out the backbone (if nothing else, to link up the rapidly growing regional networks).
This is about physical, layer 1 infrastructure. This isn't yet another overlay network (CJDNS/GNu FreeNet etc). Yes it's messy, yes it's all about non technical end users, yes it's about taking a rather complex stack (auth/network awareness/routing platform) and making it accessible to power users/"IT professionals". It's also a whole lot of fun!
Please feel free to visit us at https://www.thefnf.org for more information.
-- Brandon Ross Yahoo & AIM: BrandonNRoss +1-404-635-6667 ICQ: 2269442 Skype: brandonross Schedule a meeting: http://www.doodle.com/bross
I recently saw an interesting talk about this at 30c3, this is the way some French ISPs are solving this: http://media.ccc.de/browse/congress/2013/30C3_-_5391_-_en_-_saal_6_-_2013122... D. Oplerno is built upon empowering faculty and students -- Daniël W. Crompton <daniel.crompton@gmail.com> <http://specialbrands.net/> <http://specialbrands.net/> http://specialbrands.net/ <http://twitter.com/webhat> <http://www.facebook.com/webhat><http://plancast.com/webhat><http://www.linkedin.com/in/redhat> On 4 April 2014 03:50, Brandon Ross <bross@pobox.com> wrote:
Let's start with your basic assumption here. Why would you build a backbone at all if your goal is to solve last mile problems?
It seems to me that the expense and distraction of building a large backbone network doesn't contribute to your goals at all, given that there are many high quality, nationwide backbone networks in North America today available at reasonable cost.
On Thu, 3 Apr 2014, charles@thefnf.org wrote:
Hello everyone,
It's been some time since I've been subscribed/replied/posted here (or on WISPA for that matter). I've been pretty busy running a non profit startup (protip: don't do that. It's really really terrible) :) I'm cofounder and CTO of the Free Networking Foundation. Our goal is to bring broadband (5 mbps symmetric to start) bandwidth to the 2/3 of Americans who currently can't get it (rural, urban core, undeserved, "$ILEC stops on otherside of street" etc).
Efforts so far primarily have consisted of WiFI last (square) mile delivery using Ubiquiti hardware and the qmp.cat firmware (also meraki access points that were donated, for some reason this seems to happen quite a bit). We've helped numerous networks get started, grow and (soon we hope) become self sustaining in Austin, Kansas City, Los Angeles, Detroit, New York and a few other places throughout the US. The networks are in various stages of maturity of course, but a number of them are fully operational and passing real traffic. Especially the one in Kansas City (it spans both states).
These are (point to point, routed) access/distribution networks which connect into colocation providers blended networks.
So that's the background and current state of affairs. Not really NANOG material.
The next step is to secure our v6 space and AS number. Now that's not horribly difficult or really worthy of NANOG (though I do greatly appreciate folks on the list who helped me through the theory/practice of that process sometime ago). It appears to be fairly straightforward if you are not an LIR. Simply go through the paperwork (LOA, submit to ARIN, get out the credit card, textbook BGP config and done). And if FNF was operating the networks (we don't, we just help with organizing/consulting/software guidance/hardware spend optimization/logistics etc) and if there was just one POP (and associated administrative body), then again it wouldn't be that interesting or worth cluttering up NANOG.
FNF goal is to serve as an LIR, SWIPing out /48 chunks to neighborhood level operators. They would then peer with whatever upstream ISPs are regionally close and announce out the space. This of course would be associated with a training program, registration in an IPAM tool etc.
Regarding the above?
What do the operators on this list wish they could of been trained in starting out? I mean obviously they should have good mastery and working experience of CCNA level material, along with exposure to higher level concepts of WAN networking. What are the tricks, the gotchas, the "man that would of saved my company a million bucks in transit costs". Yes I realize these sort of things are usually closely held. I also am striving to create an entirely new breed of operators running BGP enabled sites with ipv6. The more I can do to help ease those folks integration into the internet, the better. In short, the often debated issue on this list of v6 endpoint explosion is going to be very very very real.
What IPAM tools out there can scale to a multi hundred million node, distributed, "eventual consistency" national level? (I've been working closely with guifi.net, and we are attempting to relaunch that as a very slick Apple like experience with a libremap (couchdb based) system.
I'd love to hear from folks across the spectrum of experience and network size. From folks who have been dual homed for <~1 year at a single site, to tier1 operators who were there when it all started.
So what would you like to see done in a greenfield, open source, open governance carrier backbone network? What would a dream TIER1 (and I use that in the default free zone sense of the word) look like to you?
Also how the heck would one get this bootstrapped at a sustainable pace? Would one create numerous tier2 regional carriers, and they would feed into an over arching tier1? I'm thinking something like a 501c8 type structure ( http://www.irs.gov/Charities-&-Non-Profits/Other-Non- Profits/Fraternal-Societies[1] )
As far as I know, this is the first time that an intentional community type approach is taken and a tier1 is the end goal. Not evolving into one, buying ones way into it, but a manifest destiny type approach to building a backbone.
Please feel free to reach out to me directly (charles@thefnf.org[2] ) if you wish to have a one on one discussion. In particular I'm interested in legal expertise in these sort of areas (law/compliance/contracting/negotiations for right of way etc etc etc).
Thanks for reading. I look forward to the discussion!
PS: Yes, I'm young and idealistic. I'm also grounded/practical/focused. I'm currently working on making the access portion of the network as smooth and turnkey as possible. (That basically means packaging up zeroshell/observium/powerdns/libremap/trigger and other bits/bobs into a nice livecd/ova/openvz package). I also like to think about the next wave of issues while working on the current one. It will take another year or so before we need to really be building out the backbone (if nothing else, to link up the rapidly growing regional networks).
This is about physical, layer 1 infrastructure. This isn't yet another overlay network (CJDNS/GNu FreeNet etc). Yes it's messy, yes it's all about non technical end users, yes it's about taking a rather complex stack (auth/network awareness/routing platform) and making it accessible to power users/"IT professionals". It's also a whole lot of fun!
Please feel free to visit us at https://www.thefnf.org for more information.
-- Brandon Ross Yahoo & AIM: BrandonNRoss +1-404-635-6667 ICQ: 2269442 Skype: brandonross Schedule a meeting: http://www.doodle.com/bross
On 4/3/14, 4:52 PM, charles@thefnf.org wrote:
Hello everyone,
It's been some time since I've been subscribed/replied/posted here (or on WISPA for that matter). I've been pretty busy running a non profit startup (protip: don't do that. It's really really terrible) :) I'm cofounder and CTO of the Free Networking Foundation. Our goal is to bring broadband (5 mbps symmetric to start) bandwidth to the 2/3 of Americans who currently can't get it (rural, urban core, undeserved, "$ILEC stops on otherside of street" etc).
Please feel free to visit us at https://www.thefnf.org for more information.
I'm equally confused. Last mile is much more of a problem than backbone. I run a (for a WISP) mid size end user network. Raw bandwidth cost is <8% of our expenses. Last mile delivery and transport around our own network is the expensive part. Nearly all of the action in new last mile networks is wireless or small provider FTTx deployments. I would look at what WISPA (www.wispa.org) is doing, as well at the FTTH council (www.ftthcouncil.com) to see what is being done in last mile. The FCC and Agriculture departments is also heavily involved in rural and last mile deployments and is (depending on your view) either funding these deployments, distorting the markets by discouraging private investment, or wasting lots of money. Mark -- Mark Radabaugh Amplex mark@amplex.net 419.837.5015 x 1021
On 2014-04-04 09:08, Mark Radabaugh wrote:
On 4/3/14, 4:52 PM, charles@thefnf.org wrote:
Hello everyone,
It's been some time since I've been subscribed/replied/posted here (or on WISPA for that matter). I've been pretty busy running a non profit startup (protip: don't do that. It's really really terrible) :) I'm cofounder and CTO of the Free Networking Foundation. Our goal is to bring broadband (5 mbps symmetric to start) bandwidth to the 2/3 of Americans who currently can't get it (rural, urban core, undeserved, "$ILEC stops on otherside of street" etc).
Please feel free to visit us at https://www.thefnf.org for more information.
I'm equally confused. Last mile is much more of a problem than backbone.
Quite true. This is why we've started there, and it's been our primary focus. We have more work to do of course. However efforts are promising and ongoing. I run a (for a WISP) mid size end user network. Raw
bandwidth cost is <8% of our expenses.
Nice. That's not horrible. You have an AS/ip space? Or buying blended? Last mile delivery and
transport around our own network is the expensive part.
Yes. It certainly is. Gear, end user support, truck rolls etc.
Nearly all of the action in new last mile networks is wireless or small provider FTTx deployments. I would look at what WISPA (www.wispa.org) is doing,
Yes. I'm quite connected with the WISPA folks, especially the principals. as well at the FTTH council
(www.ftthcouncil.com)
Wasn't familiar with them. Thanks! to see what is being done in last mile. The
FCC and Agriculture departments is also heavily involved in rural and last mile deployments and is (depending on your view) either funding these deployments, distorting the markets by discouraging private investment, or wasting lots of money.
Yeah. I've been keeping an eye on that. We've helped several network builds happen via grants. Usually from local economic development councils.
participants (4)
-
Brandon Ross -
charles＠thefnf.org -
Daniël W. Crompton -
Mark Radabaugh