there's actually a not insignificant part of the 'network device' world which is in fact just really a container and "quagga" (or similar). James, do you care about being close to a 'cisco like' config world? (quagga) more programmatic? (exa-bgp, gobgp .. a few others) something else? On Thu, Jun 14, 2018 at 3:05 PM Dovid Bender <dovid@telecurve.com> wrote:

I know of a telco that has been doing this it helps them be able to move around containers and not have constantly configure IP's on servers.

On Thu, Jun 14, 2018 at 3:00 PM, Brielle Bruns <bruns@2mbit.com> wrote:

...

On 6/14/2018 12:56 PM, james jones wrote:

...

I am working on an personal experiment and was wondering what is the best option for running BGP in a docker base container. I have seen a lot blogs and docs referencing Quagga. I just want to make sure I am not over looking any other options before I dive in. Any thoughts or suggestions?

-James

*twitches*

Please don't let this be an actual thing with something as critical as BGP.

-- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org

This is generally in the context of routing-on-the-host setups. We're using BIRD for that in a kubernetes deployment. -- Hugo Slabbert | email, xmpp/jabber: hugo@slabnet.com pgp key: B178313E | also on Signal On Thu 2018-Jun-14 13:05:58 -0600, Michael Crapse <michael@wi-fiber.io> wrote:

I agree, i hope that this is for testing/testbench purposes only, or only running iBGP, as no one in the world would like for you to be running a public BGP through a docker instance.

On 14 June 2018 at 13:00, Brielle Bruns <bruns@2mbit.com> wrote:

...

On 6/14/2018 12:56 PM, james jones wrote:

...

I am working on an personal experiment and was wondering what is the best option for running BGP in a docker base container. I have seen a lot blogs and docs referencing Quagga. I just want to make sure I am not over looking any other options before I dive in. Any thoughts or suggestions?

-James

*twitches*

Please don't let this be an actual thing with something as critical as BGP.

-- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org

re: Exa: Our use case was both on exporting service IPs as well as receiving routes from ToRs. Exa is more geared towards the former than the latter. Rather then working on getting imports and route installation through Exa, we found it simpler with BIRD exporting the service IP from it bound to a loopback to run local healthchecks on the nodes and then have them yank the service IP from the loopback on failing healthchecks in order to stop exporting. But, YMMV etc. -- Hugo Slabbert | email, xmpp/jabber: hugo@slabnet.com pgp key: B178313E | also on Signal On Thu 2018-Jun-14 15:07:35 -0400, james jones <james.voip@gmail.com> wrote:

Yes, that's it.

On Thu, Jun 14, 2018 at 3:05 PM Michel 'ic' Luczak <lists@benappy.com> wrote:

...

...

On 14 Jun 2018, at 20:56, james jones <james.voip@gmail.com> wrote:

I am working on an personal experiment and was wondering what is the best option for running BGP in a docker base container. I have seen a lot blogs and docs referencing Quagga. I just want to make sure I am not over looking any other options before I dive in. Any thoughts or suggestions?

I guess / hope what you’re trying to achieve is to announce services from the containers using BGP. If this is the case, what you’re looking for is called exabgp.

ic

On 06/14/2018 09:22 PM, Michael Thomas wrote:

So I have to ask, why is it advantageous to put this in a container rather than just run it directly on the container's host?

Most any host now-a-days has quite a bit of horse power to run services. All those services could be run natively all in one namespace on the same host, or ... I tend to gravitate towards running services individually in LXC containers. This creates a bit more overhead than running chroot style environments, but less than running full fledged kvm style virtualization for each service. I typically automate the provisioning and the spool up of the container and its service. This makes it easy to up-keep/rebuild/update/upgrade/load-balance services individually and enmasse across hosts. By running BGP within each container, as someone else mentioned, BGP can be used to advertise the loopback address of the service. I go one step further: for certain services I will anycast some addresses into bgp. This provides an easy way to load balance and provide resiliency of like service instances across hosts, Therefore, by running BGP within the container, and on the host, routes can be distributed across a network with all the policies available within the bgp protocol. I use Free Range Routing, which is a fork of Quagga, to do this. I use the eBGP variant for the hosts and containers, which allows for the elimination of OSPF or similar internal gateway protocol. Stepping away a bit, this means that BGP is used in tiered scenario. There is the regular eBGP with the public ASN for handling DFZ style public traffic. For internal traffic, private eBGP ASNs are used for routing traffic between and within hosts and containers. With recent improvements to Free Range Routing and the Linux Kernel, various combinations of MPLS, VxLAN, EVPN, and VRF configurations can be used to further segment and compartmentalize traffic within a host, and between containers. It is now very easy to run vlan-less between hosts through various easy to configure encapsulation mechanisms. To be explicit, this relies on a resilient layer 3 network between hosts, and eliminates the bothersome layer 2 redundancy headaches. That was a very long winded way to say: keep a very basic host configuration running a minimal set of functional services, and re-factor the functionality and split it across multiple containers to provide easy access to and maintenance of individual services like dns, smtp, database, dashboards, public routing, private routing, firewalling, monitoring, management, ... There is a higher up-front configuration cost, but over the longer term, if configured via automation tools like Salt or similar, maintenance and security is improved. It does require a different level of sophistication with operational staff.

Mike

On 06/14/2018 05:03 PM, Richard Hicks wrote:

...

I'm happy with GoBGP in a docker container for my BGP Dashboard/LookingGlass project. https://github.com/rhicks/bgp-dashboard

Its just piping RIB updates, as JSON, to script to feed into MongoDB container.

At work we also looked at GoBGP as a route-server for a small IXP type of setup, but ran into few issues that we didn't have the time to fully debug.  So we switched to BIRD for that project. We are happy with both.

On Thu, Jun 14, 2018 at 11:56 AM, james jones <james.voip@gmail.com> wrote:

...

I am working on an personal experiment and was wondering what is the best option for running BGP in a docker base container. I have seen a lot blogs and docs referencing Quagga. I just want to make sure I am not over looking any other options before I dive in. Any thoughts or suggestions?

-James

-- Raymond Burkholder ray@oneunified.net https://blog.raymond.burkholder.net -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

On Thu, Jun 14, 2018 at 7:22 PM, Michael Thomas <mike@mtcc.com> wrote:

So I have to ask, why is it advantageous to put this in a container rather than just run it directly > on the container's host?

There is no real reason not to run it in a container, and all the advantages of running ALL applications in standardized containers (whether the choice be the likes of vSphere,XEN,KVM,Virtuozzo, LXC, or Docker). Assuming the host runs containers: running one app. outside the container (BGP) would put the other applications at risk, since there could be a security vulnerability in the BGP implementation allowing runaway resource usage or remote code exploitation, or in theory, the install process for that app could "spoil" the host or introduce incompatibilities or divergence from expected host configuration. One of the major purposes of containers is to mitigate such problems. For example the BGP application could be exploited but the container boundary prevents access to sensitive data of other apps. sharing the hardware; the application installer running in a container cannot introduce conflicts or impact operating settings of the host platform. Also, the common model of virtualizing the compute resource calls for treating hosts as a shared compute farm --- no one host is special: any container can run equally on other hosts in the same pod, and you hardly ever even check which host a particular container has been scheduled to run on. Users of the resource are presented an interface for running their application: containers. No other option is offered... there is no such thing as "run my program (or run this container) directly on host X" option. ---- no host runs directly any programs or services which have configurations different from any other host, and also every host config is about identical other than hostname & ip address; Simply put: being able to run a program outside a container would violate the service model for datacenter compute services that is most commonly used these days. Running the BGP application in a container on a shared storage system managed by a host cluster would also make it easier to start the service up on a different host when the first host fails or requires maintenance. On the other hand, running directly on a host, suggests that individual hosts need to be backed up again, and some sort of manual restore of local files from the lost host will be required to copy the non-containerized application to a new host.

Mike -- -JH

Years back I ran ExaBGP inside a Docker container (when it wasn't "production ready") to anycast a contained service both within a datacenter and across them. To make routing work correctly I had to also run another BGP daemon on the Docker host machine; I can't remember if I used bird for this, but it seems like what I'd use since I didn't need programmatic control of prefixes. Would I do it that way today? Not a chance. How would I do it? That would really depend on two things: what I'm trying to accomplish with BGP and what the service is. If you just want portability of a service (not redundancy/balancing via anycast) is BGP really the best option? I'd make a strong case for OSPF due to it needing far less config. The same need for a routing instance on the Docker host would apply, but you wouldn't need to manage configuration for neighbors as containers come up and go down (since the IP will likely change). Sure, you could just add neighbor config for every IP Docker might use, however-- ouch. Jeff Walter On Mon, Jun 18, 2018 at 8:45 AM, Hugo Slabbert <hugo@slabnet.com> wrote:

On Sat 2018-Jun-16 00:51:15 -0500, Jimmy Hess <mysidia@gmail.com> wrote:

...

Running the BGP application in a container on a shared storage system managed by a host cluster would also make it easier to start the service up on a different host when the first host fails or requires maintenance.

On the other hand, running directly on a host, suggests that individual hosts need to be backed up again, and some sort of manual restore of local files from the lost host will be required to copy the non-containerized application to a new host.

Even if the BGP speaker is running right on the host, the shared storage or backups thing doesn't click for me. What about your BGP speaker will need persistent storage? At least in our environment, everything unique about the BGP speaker is config injected at startup or can be derived at startup. This might be based on differences in how we're using them (BGP daemon per container host in our case, rather than "I need X number of BGP speakers; schedule them somewhere"), I guess.

-- Hugo Slabbert | email, xmpp/jabber: hugo@slabnet.com pgp key: B178313E | also on Signal

On Thu, Jun 14, 2018 at 8:46 PM Mike Hammett <nanog@ics-il.net> wrote:

I wonder which part of the proposal people find offensive.

I have no idea. All - You know no one is trying to make *you* run BGP inside of a container, right?

There's no reason why it shouldn't work well. It's just a minor paradigm shift that requires some solid testing and knowhow on the ops team. On Thu, Jun 14, 2018, 22:26 Eric Tykwinski, <eric-list@truenet.com> wrote:

The funny part is I don’t like containers but love VMs, so kvm, vmware, citrix, hvm, et al. Not much difference but I tend to like the separation of OS knowledge, with all the bugs lately though I wonder if it’s worth it.

Sincerely,

Eric Tykwinski TrueNet, Inc. P: 610-429-8300

...

On Jun 14, 2018, at 10:14 PM, Hunter Fuller <hf0002+nanog@uah.edu> wrote:

On Thu, Jun 14, 2018 at 8:46 PM Mike Hammett <nanog@ics-il.net> wrote:

...

I wonder which part of the proposal people find offensive.

I have no idea. All - You know no one is trying to make *you* run BGP inside of a container, right?

FRR, the modern fork of quagga, has a pre built docker container. https://hub.docker.com/r/cumulusnetworks/frrouting/ On Thu, Jun 14, 2018, 10:41 PM Oliver O'Boyle <oliver.oboyle@gmail.com> wrote:

There's no reason why it shouldn't work well. It's just a minor paradigm shift that requires some solid testing and knowhow on the ops team.

On Thu, Jun 14, 2018, 22:26 Eric Tykwinski, <eric-list@truenet.com> wrote:

...

The funny part is I don’t like containers but love VMs, so kvm, vmware, citrix, hvm, et al. Not much difference but I tend to like the separation of OS knowledge, with all the bugs lately though I wonder if it’s worth it.

Sincerely,

Eric Tykwinski TrueNet, Inc. P: 610-429-8300

...

On Jun 14, 2018, at 10:14 PM, Hunter Fuller <hf0002+nanog@uah.edu> wrote:

On Thu, Jun 14, 2018 at 8:46 PM Mike Hammett <nanog@ics-il.net> wrote:

...

I wonder which part of the proposal people find offensive.

I have no idea. All - You know no one is trying to make *you* run BGP inside of a container, right?