Last mile multihoming
So isnt the most likely interruption to service due to a last mile physical media issue? Or say a regional fiber cut that takes out the towers you can reach and the upstream connection from your cable and telco providers? Imo at the edge, BGP mostly protects you from layer 8 fail (if youve done some basic best practice configuration). In theory, issues below that (at least in the dist/core at l1 to 3) are handled by other redundancy protections hidden from you (hsrp, fiber ring with protected path etc). As for dfz explosion, would mpls/private as/ vrf be a workable approach for bgp at the edge? So I live in Austin. I have available to me two hfc providers (grande and twc) and att. I also have sprint/clear vzw/tmo. I havent done an analysis of wisp offerings (if any are on list, please email me at charles@thefnf.org as im looking for a non ilec path for redunancy). So lets break this down: I only know of one att co in town. (Im sure if there is more, you will let me know). So the chances of that failing are decently high. Also my experience with att dsl have been mixed, unless im homed direct to the co. Vz dsl otoh has always been rock solid. Also att is retiring dsl/copper. I refuse to use uverse as they dont offer a unbundled modem/router or a way to do bridge mode. Oh and no ipv6. (If you can put a modem in bridge mode and still have working tv, please let me know. Ive not been able to find a solution). The chances of someone driving into the dslam serving my complex or the pedastal down the street is high (100% as it has happend a couple times). So this means I need a wireless backhaul. All of the providers I can reach colocate on exactly one tower. Surrounded by a chain link fence, across from a walmart. (Im in north austin near cameron and 183 for anyone who lives in town). The chances of the fiber serving that tower being cut is unknown, but not outside the realm of possibility. Or say the walmart big rig over correcting due to a driver coming around the blind curve near there and plowing into thr tower. Etc. So my best bet for uninterrupted connectivity seems to be running two openvpn tunels on my home edge pfsense router, each to a endpoint in a colo. I already have a full rack of gear in joesdatacenter in kc, and its fully redundant. I also run all of my web/mail/software dev from there, so its not soley for bgp purposes. Most folks I imagine may have their stuff in a colo as well and not want to run that at home. (I started a thread on that once upon a time). It so happens, that I have various things which I cant run there (rf equipment which I need to frequently reflash and move around). So running bgp on my colo gear and announcing a /48 that ive assigned to my house seems like a good idea. And I can easily cross connect to kcix and have lots of bgp fun. The latency would be a bit high, but it already is and I dont have any redundant connectivitym Ok. So thats great. Now who is my secondary? Is a vps at say linode sufficient for a secondary bgp announcer? Will they sell me bgp enabled transit? Will other vps providers? Do I need a box in a rack at a local nap? Is there an ix in austin, or should I rack a box in Dallas? Once i have two providerdls, then i can easily use pfsense multi wan failover and if a circuit goes down, life goes on as I rely on bgp to detect the link failure and handle it. Yes? No? Maybe? So to me, this seems like a solved problem. Run multilple diverse (carrier, media type) circuits to your edge, put a pfsense (asa, whatever is your poison but i like pfsense the best for multi wan failover), openvpn (i cant stand ipsec) to colo, cross connect to ... oh I dunno he.net :) bgp for free. Done. For about... hmmm.. 500.00 a month? (Many colos might not do bgp with you for less then a quarter rack, and I presume anyone serious enough about uninterrupted service on a reasonable budget can do 500.00 a month). Thie discussion on soho multihoming has been fascinating to me, and I wanted to go through a thought exercise for what I imagine is a common scenario (main gear in a bgp enabled sp, office gear needing to be reachable by remote personnel in a non bgp enabled sp). Would love to hear what you folks think. -- Charles Wyble charles@thefnf.org / 818 280 7059 CTO Free Network Foundation (www.thefnf.org)
SOHO failover would be significantly easier if you had a VPN server in a datacenter, and setup something like pfSense to connect to the VPN over one or many ISP connections. You really could just buy 2-3 local ISP connections, and let the VPN tunnel reestablish in the event of an outage (under a second, usually, states and connections preserved). I am unsure of bonding all those VPN connections at the same time, but I imagine there is a method to do that. On Mon, Mar 25, 2013 at 12:56 AM, Charles Wyble < charles-lists@knownelement.com> wrote:
So isnt the most likely interruption to service due to a last mile physical media issue? Or say a regional fiber cut that takes out the towers you can reach and the upstream connection from your cable and telco providers? Imo at the edge, BGP mostly protects you from layer 8 fail (if youve done some basic best practice configuration). In theory, issues below that (at least in the dist/core at l1 to 3) are handled by other redundancy protections hidden from you (hsrp, fiber ring with protected path etc).
As for dfz explosion, would mpls/private as/ vrf be a workable approach for bgp at the edge?
So I live in Austin. I have available to me two hfc providers (grande and twc) and att. I also have sprint/clear vzw/tmo. I havent done an analysis of wisp offerings (if any are on list, please email me at charles@thefnf.org as im looking for a non ilec path for redunancy).
So lets break this down:
I only know of one att co in town. (Im sure if there is more, you will let me know). So the chances of that failing are decently high. Also my experience with att dsl have been mixed, unless im homed direct to the co. Vz dsl otoh has always been rock solid. Also att is retiring dsl/copper. I refuse to use uverse as they dont offer a unbundled modem/router or a way to do bridge mode. Oh and no ipv6. (If you can put a modem in bridge mode and still have working tv, please let me know. Ive not been able to find a solution).
The chances of someone driving into the dslam serving my complex or the pedastal down the street is high (100% as it has happend a couple times).
So this means I need a wireless backhaul. All of the providers I can reach colocate on exactly one tower. Surrounded by a chain link fence, across from a walmart. (Im in north austin near cameron and 183 for anyone who lives in town). The chances of the fiber serving that tower being cut is unknown, but not outside the realm of possibility. Or say the walmart big rig over correcting due to a driver coming around the blind curve near there and plowing into thr tower. Etc.
So my best bet for uninterrupted connectivity seems to be running two openvpn tunels on my home edge pfsense router, each to a endpoint in a colo.
I already have a full rack of gear in joesdatacenter in kc, and its fully redundant. I also run all of my web/mail/software dev from there, so its not soley for bgp purposes. Most folks I imagine may have their stuff in a colo as well and not want to run that at home. (I started a thread on that once upon a time). It so happens, that I have various things which I cant run there (rf equipment which I need to frequently reflash and move around). So running bgp on my colo gear and announcing a /48 that ive assigned to my house seems like a good idea. And I can easily cross connect to kcix and have lots of bgp fun. The latency would be a bit high, but it already is and I dont have any redundant connectivitym
Ok. So thats great. Now who is my secondary? Is a vps at say linode sufficient for a secondary bgp announcer? Will they sell me bgp enabled transit? Will other vps providers? Do I need a box in a rack at a local nap? Is there an ix in austin, or should I rack a box in Dallas?
Once i have two providerdls, then i can easily use pfsense multi wan failover and if a circuit goes down, life goes on as I rely on bgp to detect the link failure and handle it. Yes? No? Maybe?
So to me, this seems like a solved problem. Run multilple diverse (carrier, media type) circuits to your edge, put a pfsense (asa, whatever is your poison but i like pfsense the best for multi wan failover), openvpn (i cant stand ipsec) to colo, cross connect to ... oh I dunno he.net :) bgp for free. Done.
For about... hmmm.. 500.00 a month? (Many colos might not do bgp with you for less then a quarter rack, and I presume anyone serious enough about uninterrupted service on a reasonable budget can do 500.00 a month).
Thie discussion on soho multihoming has been fascinating to me, and I wanted to go through a thought exercise for what I imagine is a common scenario (main gear in a bgp enabled sp, office gear needing to be reachable by remote personnel in a non bgp enabled sp).
Would love to hear what you folks think.
-- Charles Wyble charles@thefnf.org / 818 280 7059 CTO Free Network Foundation (www.thefnf.org)
On Mar 26, 2013, at 8:14 AM, Nick Zurku <Nick@nzurku.com> wrote:
SOHO failover would be significantly easier if you had a VPN server in a datacenter, and setup something like pfSense to connect to the VPN over one or many ISP connections.
I'm essentially doing this now. It does not reduce the DFZ impact. I advertise my routes out of the two data centers where my VPNs terminate (well, GRE tunnels, really) and run iBGP across the tunnels. It works just fine, but it's not simpler than it would be if I could just peer with my direct upstreams.
You really could just buy 2-3 local ISP connections, and let the VPN tunnel reestablish in the event of an outage (under a second, usually, states and connections preserved). I am unsure of bonding all those VPN connections at the same time, but I imagine there is a method to do that.
Yes, that's what happens today. How is managing a mesh of VPNs, equipment in two additional data centers, a bunch of tunnels, and an extra 4+ BGP sessions simpler? Owen
On Mon, Mar 25, 2013 at 12:56 AM, Charles Wyble < charles-lists@knownelement.com> wrote:
So isnt the most likely interruption to service due to a last mile physical media issue? Or say a regional fiber cut that takes out the towers you can reach and the upstream connection from your cable and telco providers? Imo at the edge, BGP mostly protects you from layer 8 fail (if youve done some basic best practice configuration). In theory, issues below that (at least in the dist/core at l1 to 3) are handled by other redundancy protections hidden from you (hsrp, fiber ring with protected path etc).
As for dfz explosion, would mpls/private as/ vrf be a workable approach for bgp at the edge?
So I live in Austin. I have available to me two hfc providers (grande and twc) and att. I also have sprint/clear vzw/tmo. I havent done an analysis of wisp offerings (if any are on list, please email me at charles@thefnf.org as im looking for a non ilec path for redunancy).
So lets break this down:
I only know of one att co in town. (Im sure if there is more, you will let me know). So the chances of that failing are decently high. Also my experience with att dsl have been mixed, unless im homed direct to the co. Vz dsl otoh has always been rock solid. Also att is retiring dsl/copper. I refuse to use uverse as they dont offer a unbundled modem/router or a way to do bridge mode. Oh and no ipv6. (If you can put a modem in bridge mode and still have working tv, please let me know. Ive not been able to find a solution).
The chances of someone driving into the dslam serving my complex or the pedastal down the street is high (100% as it has happend a couple times).
So this means I need a wireless backhaul. All of the providers I can reach colocate on exactly one tower. Surrounded by a chain link fence, across from a walmart. (Im in north austin near cameron and 183 for anyone who lives in town). The chances of the fiber serving that tower being cut is unknown, but not outside the realm of possibility. Or say the walmart big rig over correcting due to a driver coming around the blind curve near there and plowing into thr tower. Etc.
So my best bet for uninterrupted connectivity seems to be running two openvpn tunels on my home edge pfsense router, each to a endpoint in a colo.
I already have a full rack of gear in joesdatacenter in kc, and its fully redundant. I also run all of my web/mail/software dev from there, so its not soley for bgp purposes. Most folks I imagine may have their stuff in a colo as well and not want to run that at home. (I started a thread on that once upon a time). It so happens, that I have various things which I cant run there (rf equipment which I need to frequently reflash and move around). So running bgp on my colo gear and announcing a /48 that ive assigned to my house seems like a good idea. And I can easily cross connect to kcix and have lots of bgp fun. The latency would be a bit high, but it already is and I dont have any redundant connectivitym
Ok. So thats great. Now who is my secondary? Is a vps at say linode sufficient for a secondary bgp announcer? Will they sell me bgp enabled transit? Will other vps providers? Do I need a box in a rack at a local nap? Is there an ix in austin, or should I rack a box in Dallas?
Once i have two providerdls, then i can easily use pfsense multi wan failover and if a circuit goes down, life goes on as I rely on bgp to detect the link failure and handle it. Yes? No? Maybe?
So to me, this seems like a solved problem. Run multilple diverse (carrier, media type) circuits to your edge, put a pfsense (asa, whatever is your poison but i like pfsense the best for multi wan failover), openvpn (i cant stand ipsec) to colo, cross connect to ... oh I dunno he.net :) bgp for free. Done.
For about... hmmm.. 500.00 a month? (Many colos might not do bgp with you for less then a quarter rack, and I presume anyone serious enough about uninterrupted service on a reasonable budget can do 500.00 a month).
Thie discussion on soho multihoming has been fascinating to me, and I wanted to go through a thought exercise for what I imagine is a common scenario (main gear in a bgp enabled sp, office gear needing to be reachable by remote personnel in a non bgp enabled sp).
Would love to hear what you folks think.
-- Charles Wyble charles@thefnf.org / 818 280 7059 CTO Free Network Foundation (www.thefnf.org)
participants (3)
-
Charles Wyble -
Nick Zurku -
Owen DeLong