On the subject of multihoming
I'm working on a small experiment which utilizes multiple outbound links (in the experiments case multiple consumer 3G connections [to 2 Sprint/2 Verizon/1 AT&T], Time Warner Cable Modem and an SBC Global DSL connection. What is the best way to do outbound traffic engineering? I would like to be able to determine the best path possible and send traffic out the appropriate link. Could this be done with a copy of the BGP tables? Obviously as they are consumer connections, I wouldn't get a BGP feed so would need to download a copy, which has the risk of stale data. Perhaps some sort of multihop BGP setup? I have done some research and found a lot of references to small site multihoming without BGP for link redundancy but not for traffic engineering. Thanks. Charles
On 2008/11/04 10:32 PM Charles Wyble wrote:
Obviously as they are consumer connections, I wouldn't get a BGP feed so would need to download a copy, which has the risk of stale data. Perhaps some sort of multihop BGP setup?
I have done some research and found a lot of references to small site multihoming without BGP for link redundancy but not for traffic engineering.
I've played with that before. Essentially just EBGP Multi-hop with next-hop rewrites on various community prefixes. Of course I had access to a donor feed, that is probably the largest hurdle. There is good use in general for a public no-distribute feed but I have yet to find such a thing. Is there a reason for that, or could I bribe my datacenter to give me a feed and then create my own public server with some el-cheapo Quagga and a bag of rainbows for hope?
Colin Alston wrote:
On 2008/11/04 10:32 PM Charles Wyble wrote:
Obviously as they are consumer connections, I wouldn't get a BGP feed so would need to download a copy, which has the risk of stale data. Perhaps some sort of multihop BGP setup?
I have done some research and found a lot of references to small site multihoming without BGP for link redundancy but not for traffic engineering.
I've played with that before. Essentially just EBGP Multi-hop with next-hop rewrites on various community prefixes. Of course I had access to a donor feed, that is probably the largest hurdle.
My first job was at a place with a direct ARIN allocation and BGP to Sprint and AT&T. I'm still friends with the remaining ops person and can probably setup a peering session with him. I also have another buddy with the ability to do BGP via Cogent.
There is good use in general for a public no-distribute feed but I have yet to find such a thing. Is there a reason for that, or could I bribe my datacenter to give me a feed and then create my own public server with some el-cheapo Quagga and a bag of rainbows for hope?
Good question. Perhaps I could peer with my above mentioned sources and http://www.quagga.net/route-server.php or http://www.routeviews.org/ (config instructions at http://www.routeviews.org/config.html) That would be a fairly diverse set of views and hopefully sufficient for my needs. By the way I have a wiki page up with the details (more or less what I outlined already) at http://www.socalwifi.net/index.php/Mesh_Experiment I will write everything up there as well as post back results here.
Charles Wyble wrote:
Colin Alston wrote:
On 2008/11/04 10:32 PM Charles Wyble wrote:
Obviously as they are consumer connections, I wouldn't get a BGP feed so would need to download a copy, which has the risk of stale data. Perhaps some sort of multihop BGP setup?
I have done some research and found a lot of references to small site multihoming without BGP for link redundancy but not for traffic engineering.
I've played with that before. Essentially just EBGP Multi-hop with next-hop rewrites on various community prefixes. Of course I had access to a donor feed, that is probably the largest hurdle.
My first job was at a place with a direct ARIN allocation and BGP to Sprint and AT&T. I'm still friends with the remaining ops person and can probably setup a peering session with him. I also have another buddy with the ability to do BGP via Cogent.
There is good use in general for a public no-distribute feed but I have yet to find such a thing. Is there a reason for that, or could I bribe my datacenter to give me a feed and then create my own public server with some el-cheapo Quagga and a bag of rainbows for hope?
Good question. Perhaps I could peer with my above mentioned sources and http://www.quagga.net/route-server.php or http://www.routeviews.org/ (config instructions at http://www.routeviews.org/config.html)
That would be a fairly diverse set of views and hopefully sufficient for my needs.
By the way I have a wiki page up with the details (more or less what I outlined already) at http://www.socalwifi.net/index.php/Mesh_Experiment
I will write everything up there as well as post back results here.
So changing my search terms a bit to utilizing bgp feeds outbound traffic engineering, returns http://www.caida.org/workshops/isma/0210/ISMAagenda.xml which seems to be near what I want. It certainly provides some interesting reading and ways to measure / analyze the necessary data.
On 11/4/08, Colin Alston <karnaugh@karnaugh.za.net> wrote:
There is good use in general for a public no-distribute feed but I have yet to find such a thing. Is there a reason for that
answering that question is easy. Yes, there are a number of reasons that nobody provides one: 1) most people don't want the details of their internal routing policies known to outside entities. Perhaps I pay Level3 for routes, but don't want the rest of the world to know it; if I provide a BGP feed, unless I very carefully sanitize it before it goes out, it will become clear to people outside that there are prefixes seen across the path that would not be seen by an ordinary peer. 2) It requires careful filtering to ensure that nobody recieving the feed accidentally or intentionally uses the routing information in their forwarding table without altering next-hops as appropriate; nobody wants to end up carrying unexpected traffic. 3) Requires ongoing support personnel to maintain and keep it up and running; while not a huge cost, it nonetheless presents a continuous, residual drain on resources that few companies are willing to underwrite 4) It would be of limited value, unless it covers a relatively large swath of the Internet with relatively good splay; having a BGP feed that simply shows a default route from your upstream's upstream's upstream wouldn't really give much useful data to work with. 5) Much of the data is already available with a bit of delay from RIS/RIPE and routeviews; unless you really need the real-time aspect, you might as well just get the data from there.
or could I bribe my datacenter to give me a feed and then create my own public server with some el-cheapo Quagga and a bag of rainbows for hope?
I'd recommend OpenBGPd -- it's been scaling much better for me, and converges about 8 times faster than Quagga on the internal route collectors I run. Matt
-----Original Message----- From: Charles Wyble [mailto:charles@thewybles.com] Sent: Tuesday, November 04, 2008 12:32 PM To: NANOG list Subject: On the subject of multihoming
I'm working on a small experiment which utilizes multiple outbound
This sort of thing is usually done with some sort of multi-port outbound NAT device that chooses the source interface to NAT from based on some "quality" metric it generates for the destination, and a state table it keeps for all the outside IPs. Products that do this include FatPipe, Radware Linkproof, and Mushroom networks. links
(in the experiments case multiple consumer 3G connections [to 2 Sprint/2 Verizon/1 AT&T], Time Warner Cable Modem and an SBC Global DSL connection.
What is the best way to do outbound traffic engineering? I would like to be able to determine the best path possible and send traffic out the appropriate link.
Could this be done with a copy of the BGP tables?
Obviously as they are consumer connections, I wouldn't get a BGP feed so would need to download a copy, which has the risk of stale data. Perhaps some sort of multihop BGP setup?
I have done some research and found a lot of references to small site multihoming without BGP for link redundancy but not for traffic engineering.
Thanks.
Charles
Yes bgp multihop is a GREAT* way to figure out if a cablemodem** is even /really/ online. Alas, I've not see much on the traffic engineering side either. * Read "the only way i've found to do this with cisco's ios" ** or any other pipe for that matter. On Tue, 4 Nov 2008, Charles Wyble wrote:
Date: Tue, 04 Nov 2008 12:32:11 -0800 From: Charles Wyble <charles@thewybles.com> To: NANOG list <nanog@nanog.org> Subject: On the subject of multihoming
I'm working on a small experiment which utilizes multiple outbound links (in the experiments case multiple consumer 3G connections [to 2 Sprint/2 Verizon/1 AT&T], Time Warner Cable Modem and an SBC Global DSL connection.
What is the best way to do outbound traffic engineering? I would like to be able to determine the best path possible and send traffic out the appropriate link.
Could this be done with a copy of the BGP tables?
Obviously as they are consumer connections, I wouldn't get a BGP feed so would need to download a copy, which has the risk of stale data. Perhaps some sort of multihop BGP setup?
I have done some research and found a lot of references to small site multihoming without BGP for link redundancy but not for traffic engineering.
Thanks.
Charles
[ resent to list, was sent from the wrong address -sd ] Charles Wyble wrote:
I'm working on a small experiment which utilizes multiple outbound links (in the experiments case multiple consumer 3G connections [to 2 Sprint/2 Verizon/1 AT&T], Time Warner Cable Modem and an SBC Global DSL connection.
What is the best way to do outbound traffic engineering? I would like to be able to determine the best path possible and send traffic out the appropriate link.
Not sure if this is useful, but I thought I'd contribute a point on the curve... from NANOG 9: http://www.academ.com/nanog/feb1997/multihoming.html Obquote: from Paul Vixie's presentation, from Stan Barber's notes, here is the "meat of the matter": _ _ _ _ _ _ _ Per-interface Default Route * BSD TCP binds outbound route to PCB on SYN-ACK * Our trick: remember the inbound interface identity from the SYN * Each interface has its own "default route" * For outbound TCP and all UDP, a normal default is also needed. _ _ _ _ _ _ _ Hope that helps... -Scott
Right now wee are also looking into the same question with the help of Overlay Routing. As far as Multihoming is concerned, there is a good work by jenifer rexford http://www.cs.princeton.edu/~jrex/papers/multipath06.pdf<http://www.cs.princeton.edu/%7Ejrex/papers/multipath06.pdf>. In fact IETF guys were thinking to include it in BGP implementation. Hope it would be helpful On Wed, Nov 12, 2008 at 10:18 PM, Scott Doty <scott@sonic.net> wrote:
[ resent to list, was sent from the wrong address -sd ]
Charles Wyble wrote:
I'm working on a small experiment which utilizes multiple outbound links (in the experiments case multiple consumer 3G connections [to 2 Sprint/2 Verizon/1 AT&T], Time Warner Cable Modem and an SBC Global DSL connection.
What is the best way to do outbound traffic engineering? I would like to be able to determine the best path possible and send traffic out the appropriate link.
Not sure if this is useful, but I thought I'd contribute a point on the curve...
from NANOG 9:
http://www.academ.com/nanog/feb1997/multihoming.html
Obquote: from Paul Vixie's presentation, from Stan Barber's notes, here is the "meat of the matter":
_ _ _ _ _ _ _ Per-interface Default Route
* BSD TCP binds outbound route to PCB on SYN-ACK * Our trick: remember the inbound interface identity from the SYN * Each interface has its own "default route" * For outbound TCP and all UDP, a normal default is also needed.
_ _ _ _ _ _ _
Hope that helps...
-Scott
-- Ghulam Murtaza Lahore University of Management Sciences
participants (7)
-
Charles Wyble
-
Colin Alston
-
kb3ien+nanog@databit7.com
-
Matthew Petach
-
Murtaza
-
Scott Doty
-
Tomas L. Byrnes