ISP inbound failover without BGP
This may sound like dumb question, but... I'm used to asking those. Here's the scenario Another ISP, say AT&T, is the primary ISP for a customer. Customer has publicly accessible servers in their office, using the AT&T address space. I am the customer's secondary ISP. Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet. What about the publicly accessible servers that have AT&T addresses, though? One thought I had was having them use Dynamic DNS service. Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block? It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.
This may sound like dumb question, but... I'm used to asking those.=0A=0AHe= re's the scenario=0A=0AAnother ISP, say AT&T, is the primary ISP for a cust= omer.=0A=0ACustomer has publicly accessible servers in their office, using = the AT&T address space.=0A=0AI am the customer's secondary ISP.=0A=0ANow, i= f AT&T link fails, I can provide the customer outbound Internet access fair= ly easily.=A0 So they can surf and get to the Internet.=0A=0AWhat about the= publicly accessible servers that have AT&T addresses, though?=0A=0AOne tho= ught I had was having them use Dynamic DNS service.=A0 =0A=0AAre there any = other solutions, short of using BGP multihoming and having them try to get = their own ASN and IPv4 /24 block?=0A=0A=0AIt looks like a few router manufa= cturers have devices that might work, but it looks like a short DNS TTL (or= Dynamic DNS) needs to be set so when the primary ISP fails, the secondary = ISP address is advertised.
The usual solution is to get the public servers stuck in a colo that's multihomed. Most of the other solutions tend to be a bit dodgy. If your gear is sufficiently competent, you can hack up a solution with multiple addresses for each of the servers (one on each ISP) and then use a short TTL to fail over, but this has more of "fail" than "fail over" about it, because there are a bunch of issues that typically result. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Depends on the application, SIP, VPN, SMTP, etc just setup both IPs and let the end-user application figure it out (SIP-UA register to both IPs for example) HTTP/HTTPS setup a proxy server in a colo that is multi-homed to frontend the requests. Then it can load balance traffic over both IPs. DNS TTL ‘tricks’ are just that, they work ‘kinda’ Fatpipe? Crazy expensive IMHO but I hear they work ok. -Matt -- Matthew S. Crocker President Crocker Communications, Inc. PO BOX 710 Greenfield, MA 01302-0710 E: matthew@crocker.com P: (413) 746-2760 F: (413) 746-3704 W: http://www.crocker.com On Mar 3, 2014, at 8:11 PM, Eric A Louie <elouie@yahoo.com> wrote:
This may sound like dumb question, but... I'm used to asking those.
Here's the scenario
Another ISP, say AT&T, is the primary ISP for a customer.
Customer has publicly accessible servers in their office, using the AT&T address space.
I am the customer's secondary ISP.
Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet.
What about the publicly accessible servers that have AT&T addresses, though?
One thought I had was having them use Dynamic DNS service.
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.
Depending on their business, using dynamic DNS providers could be a really bad idea. If they deal only with home users who won't even know, it'll probably work. If their customers are security-aware businesses, they probably block all sites hosted with dynamic DNS systems. Ray
Subject: Re: ISP inbound failover without BGP From: matthew@corp.crocker.com Date: Mon, 3 Mar 2014 20:50:26 -0500 To: elouie@yahoo.com CC: nanog@nanog.org
Depends on the application,
SIP, VPN, SMTP, etc just setup both IPs and let the end-user application figure it out (SIP-UA register to both IPs for example)
HTTP/HTTPS setup a proxy server in a colo that is multi-homed to frontend the requests. Then it can load balance traffic over both IPs.
DNS TTL ‘tricks’ are just that, they work ‘kinda’
Fatpipe? Crazy expensive IMHO but I hear they work ok.
-Matt
-- Matthew S. Crocker President Crocker Communications, Inc. PO BOX 710 Greenfield, MA 01302-0710
E: matthew@crocker.com P: (413) 746-2760 F: (413) 746-3704 W: http://www.crocker.com
On Mar 3, 2014, at 8:11 PM, Eric A Louie <elouie@yahoo.com> wrote:
This may sound like dumb question, but... I'm used to asking those.
Here's the scenario
Another ISP, say AT&T, is the primary ISP for a customer.
Customer has publicly accessible servers in their office, using the AT&T address space.
I am the customer's secondary ISP.
Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet.
What about the publicly accessible servers that have AT&T addresses, though?
One thought I had was having them use Dynamic DNS service.
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.
Depending on their business=2C using dynamic DNS providers could be a reall= y bad idea. If they deal only with home users who won't even know=2C it'll = probably work. If their customers are security-aware businesses=2C they pro= bably block all sites hosted with dynamic DNS systems.
Where do dynamic DNS /providers/ enter the picture? You update *your own* DNS records. Those could conceivably be hosted with a dynamic DNS provider, but why not just use the API for whatever system you currently use for DNS? ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
That's a good point Ray - thank you.
________________________________ From: Ray <sixsigma44@hotmail.com> To: Matthew Crocker <matthew@corp.crocker.com>; Eric A Louie <elouie@yahoo.com> Cc: NANOG <nanog@nanog.org> Sent: Monday, March 3, 2014 6:31 PM Subject: RE: ISP inbound failover without BGP
Depending on their business, using dynamic DNS providers could be a really bad idea. If they deal only with home users who won't even know, it'll probably work. If their customers are security-aware businesses, they probably block all sites hosted with dynamic DNS systems.
Ray
Subject: Re: ISP inbound failover without BGP From: matthew@corp.crocker.com Date: Mon, 3 Mar 2014 20:50:26 -0500 To: elouie@yahoo.com CC: nanog@nanog.org
Depends on the application,
SIP, VPN, SMTP, etc just setup both IPs and let the end-user application figure it out (SIP-UA register to both IPs for example)
HTTP/HTTPS setup a proxy server in a colo that is multi-homed to frontend the requests. Then it can load balance traffic over both IPs.
DNS TTL ‘tricks’ are just that, they work ‘kinda’
Fatpipe? Crazy expensive IMHO but I hear they work ok.
-Matt
-- Matthew S. Crocker President Crocker Communications, Inc. PO BOX 710 Greenfield, MA 01302-0710
E: matthew@crocker.com P: (413) 746-2760 F: (413) 746-3704 W: http://www.crocker.com
On Mar 3, 2014, at 8:11 PM, Eric A Louie <elouie@yahoo.com> wrote:
This may sound like dumb question, but... I'm used to asking those.
Here's the scenario
Another ISP, say AT&T, is the primary ISP for a customer.
Customer has publicly accessible servers in their office, using the AT&T address space.
I am the customer's secondary ISP.
Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet.
What about the publicly accessible servers that have AT&T addresses, though?
One thought I had was having them use Dynamic DNS service.
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.
On Mon, 3 Mar 2014, Eric A Louie wrote:
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
For what it sounds like the customer wants to do, this really is the right solution. Most everything else has some level of 'ugly hack' attached to it, that can cause reliability/reachability problems at inopportune times (as opposed to problems that happen at opportune times). If the customer is just looking for failover capabilities, they can take default via BGP from both providers, prefer one over the other, and use some other bits to prefer one provider over for inbound connectivity. They would not need very beefy routers for this. That will get you basic service redundancy. Add a second router, and they can have some protection in the event of a router failure. It all really boils down to what the customer wants and is willing to pay for. If they have services that need to be reliably reachable, then using a time-tested design is a prudent decision. jms
Is there some technical reason that BGP is not an option? You could allow them to announce their AT&T space via you as a secondary. -Randy ----- Original Message -----
This may sound like dumb question, but... I'm used to asking those.
Here's the scenario
Another ISP, say AT&T, is the primary ISP for a customer.
Customer has publicly accessible servers in their office, using the AT&T address space.
I am the customer's secondary ISP.
Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet.
What about the publicly accessible servers that have AT&T addresses, though?
One thought I had was having them use Dynamic DNS service.
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.
On Mon, Mar 3, 2014 at 7:20 PM, Randy Carpenter <rcarpen@network1.net>wrote:
Is there some technical reason that BGP is not an option? You could allow them to announce their AT&T space via you as a secondary.
unless it is a /26, /25 or something shorter. Even with a /24 things may get messy. IPv4 is coming to an end, but it is not possible for the customer to get their own space and use BGP? Regards, as
Honestly? Because the end-customers are not technically competent enough to run dual-homed BGP, and we don't want to be their managed service providers on the IT side. And announcing the AT&T space is fine until something goes wrong, and I have to troubleshoot the problem (Customer - "How come AT&T is down, and we're not getting inbound traffic to our servers?", and I discover L3 or CenturyLink isn't accepting my advertisement for some weird reason, but they won't fess up to it for a few frustrating hours)
________________________________ From: Randy Carpenter <rcarpen@network1.net> To: Eric A Louie <elouie@yahoo.com> Cc: NANOG <nanog@nanog.org> Sent: Monday, March 3, 2014 7:20 PM Subject: Re: ISP inbound failover without BGP
Is there some technical reason that BGP is not an option? You could allow them to announce their AT&T space via you as a secondary.
-Randy
----- Original Message -----
This may sound like dumb question, but... I'm used to asking those.
Here's the scenario
Another ISP, say AT&T, is the primary ISP for a customer.
Customer has publicly accessible servers in their office, using the AT&T address space.
I am the customer's secondary ISP.
Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet.
What about the publicly accessible servers that have AT&T addresses, though?
One thought I had was having them use Dynamic DNS service.
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.
On Mon, 3 Mar 2014, Eric A Louie wrote:
Honestly? Because the end-customers are not technically competent enough to run dual-homed BGP, and we don't want to be their managed service providers on the IT side. And announcing the AT&T space is fine until something goes wrong, and I have to troubleshoot the problem (Customer - "How come AT&T is down, and we're not getting inbound traffic to our servers?", and I discover L3 or CenturyLink isn't accepting my advertisement for some weird reason, but they won't fess up to it for a few frustrating hours)
If they're not technically competent enough to handle BGP, they won't be technically competent enough to deal with solutions that play the short DNS TTL game. As someone else mentioned in this thread - would colocating the servers be a workable solution for them? Put the servers some place where the redundancy exists already. jms
On Mon, 3 Mar 2014, Justin M. Streiner wrote:
If they're not technically competent enough to handle BGP, they won't be technically competent enough to deal with solutions that play the short DNS TTL game.
As someone else mentioned in this thread - would colocating the servers be a workable solution for them? Put the servers some place where the redundancy exists already.
My vote goes to the traditional BGP multihomed solution. It's the right way to solve the problem and the easiest to manage. If getting AT&T to do BGP and buying a BGP capable router (they don't even need full routes...so so anything that'll speak BGP, take a pair of default routes, and handle whatever their traffic level is will do) is too costly[1], another possible option I've not seen mentioned is VPN. They could put one machine/router somewhere with decent redundancy and setup a VPN gateway at their office that connects to the colo'd device. You might even offer this as a service. Spammers have been doing this for years. It makes moving their operations easier as their public facing servers get cancelled. All they do is move the VPN server(s) and their systems that do all the "work" remain online and hidden. [1] If only I had a dollar for every time a client said redundancy was too expensive to have, but when their non-redundant stuff went offline, they claimed to be losing millions of $ per small unit of time. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
There are other elaborate solutions to accomplish this, however all of them would require a competent IT/Network Person to manage the network. If we were the ISP, we would look at such a case an an opportunity, and become the managed service provider, for a fee (typically a premium), and provide the service. As service providers, we all complain about the end-customer being a pain, but we often forget that it the the PITA end-customers that give us the ability to earn our daily bread!.... I think too many of us are overworked and providing highly under-paid services for peanuts, where we often overlook at opportunities to get premium value as a PITA, and not worth it... :) Just my personal two cents,..... Faisal Imtiaz Snappy Internet & Telecom ----- Original Message -----
From: "Eric A Louie" <elouie@yahoo.com> To: "Randy Carpenter" <rcarpen@network1.net> Cc: "NANOG" <nanog@nanog.org> Sent: Monday, March 3, 2014 11:49:21 PM Subject: Re: ISP inbound failover without BGP
Honestly? Because the end-customers are not technically competent enough to run dual-homed BGP, and we don't want to be their managed service providers on the IT side. And announcing the AT&T space is fine until something goes wrong, and I have to troubleshoot the problem (Customer - "How come AT&T is down, and we're not getting inbound traffic to our servers?", and I discover L3 or CenturyLink isn't accepting my advertisement for some weird reason, but they won't fess up to it for a few frustrating hours)
________________________________ From: Randy Carpenter <rcarpen@network1.net> To: Eric A Louie <elouie@yahoo.com> Cc: NANOG <nanog@nanog.org> Sent: Monday, March 3, 2014 7:20 PM Subject: Re: ISP inbound failover without BGP
Is there some technical reason that BGP is not an option? You could allow them to announce their AT&T space via you as a secondary.
-Randy
----- Original Message -----
This may sound like dumb question, but... I'm used to asking those.
Here's the scenario
Another ISP, say AT&T, is the primary ISP for a customer.
Customer has publicly accessible servers in their office, using the AT&T address space.
I am the customer's secondary ISP.
Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet.
What about the publicly accessible servers that have AT&T addresses, though?
One thought I had was having them use Dynamic DNS service.
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.
On 3/3/14, 7:20 PM, Randy Carpenter wrote:
Is there some technical reason that BGP is not an option? You could allow them to announce their AT&T space via you as a secondary.
With the risk of starting holy war on how BGP works on dialup and that providers should permit such, the OP has not specified what transport methods is in play with AT&T which could limit such an option. ~Seth
I've been doing the suggestion below for many years using the IP addresses that Cogent gives us. All I needed to do is get LOA from them and submit it to my backup ISP. I've never had an issue with my Cogent IP's *not* being advertised by my other ISP and I really don't think there is very much management overhead for the customer once this is setup. I have an SNMP based alerting system (Cacti) set up so I can be alerted if too much traffic ever "shifts" to the backup link. The client getting their own ASN is the better way to go but you should be able to do the above until that comes through. On 3/3/2014 10:20 PM, Randy Carpenter wrote:
Is there some technical reason that BGP is not an option? You could allow them to announce their AT&T space via you as a secondary.
-Randy
----- Original Message -----
This may sound like dumb question, but... I'm used to asking those.
Here's the scenario
Another ISP, say AT&T, is the primary ISP for a customer.
Customer has publicly accessible servers in their office, using the AT&T address space.
I am the customer's secondary ISP.
Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet.
What about the publicly accessible servers that have AT&T addresses, though?
One thought I had was having them use Dynamic DNS service.
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.
-- Vlad
On Mon, Mar 3, 2014 at 8:11 PM, Eric A Louie <elouie@yahoo.com> wrote:
One thought I had was having them use Dynamic DNS service.
Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?
Hi Eric, I went through this a couple years ago with continuity of operations planning. The bottom line is: with the notable exception of low-activity electronic mail, switching the address record in the DNS entry will generally not work as expected. For folks serious about reliable access to their servers, BGP isn't just the best way, it's the only way. Reasons why dynamic DNS fails to perform as expected include: * Web browser DNS pinning can result in a customer's web browser holding the old IP address indefinitely. * Host-level caching of looked up names which discards the TTL. Remember: your desktop or laptop performs lookups against multiple name services, e.g. DNS, /etc/hosts, lmhosts, NIS+. DNS TTL is no longer in scope once the name to address map enters the generic host lookup mechanism. Most OSes have a fixed timeout of one sort or another, some old ones as long as 24 hours. * Custom applications with either IP addresses hardcoded into the configuration or with getaddrinfo() called only once and the resulting IP address held for the lifetime of the application. * Anti-spam systems block IP addresses when receiving large quantities of email from formerly-quiescent IP addresses. This is a problem if your mail server sends a lot of email and suddenly switches to a new sending IP address. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004
Am 04.03.2014 05:19, schrieb William Herrin:
Reasons why dynamic DNS fails to perform as expected include:
* Web browser DNS pinning can result in a customer's web browser holding the old IP address indefinitely.
* Host-level caching of looked up names which discards the TTL. Remember: your desktop or laptop performs lookups against multiple name services, e.g. DNS, /etc/hosts, lmhosts, NIS+. DNS TTL is no longer in scope once the name to address map enters the generic host lookup mechanism. Most OSes have a fixed timeout of one sort or another, some old ones as long as 24 hours.
* Eyeball ISPs' DNS resolvers might tamper with TTL values. -- SEBASTIAN SPIES lnked.in/sspies vastly.de
participants (13)
-
Arturo Servin -
Eric A Louie -
Faisal Imtiaz -
Joe Greco -
Jon Lewis -
Justin M. Streiner -
Matthew Crocker -
Randy Carpenter -
Ray -
Sebastian Spies -
Seth Mattinen -
Vlade Ristevski -
William Herrin