Comparison of freeware open source switch software?
I have seen numerous comparisons and RIPE presentations on performance issues of BIRD vs Quagga vs FRR. I am looking for the same thing for freeware switch software. Has anyone done a feature comparison between: http://openvswitch.org/ https://www.openswitch.net/ https://cumulusnetworks.com/products/cumulus-linux/ ...any other I am missing... I am familiar with: http://packetpushers.net/open-networking-cheat-sheet/ https://www.networkworld.com/article/2919599/cisco-subnet/clearing-the-fog-a... so to clarify I am interested only in bare-metal or whitebox swicthes and freeware, open source software. And even better - has anyone done a benchmark to see which performs best? Thanks, Hank
Here's one you missed: http://www.projectfloodlight.org/indigo/ If you're only interested in stuff that goes on iron, openvswitch is out - it's pure software meant to run on hypervisors -Ed -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Hank Nussbacher Sent: Tuesday, January 9, 2018 2:18 AM To: nanog@nanog.org Subject: Comparison of freeware open source switch software? I have seen numerous comparisons and RIPE presentations on performance issues of BIRD vs Quagga vs FRR. I am looking for the same thing for freeware switch software. Has anyone done a feature comparison between: http://openvswitch.org/ https://www.openswitch.net/ https://cumulusnetworks.com/products/cumulus-linux/ ...any other I am missing... I am familiar with: http://packetpushers.net/open-networking-cheat-sheet/ https://www.networkworld.com/article/2919599/cisco-subnet/clearing-the-fog-a... so to clarify I am interested only in bare-metal or whitebox swicthes and freeware, open source software. And even better - has anyone done a benchmark to see which performs best? Thanks, Hank
Just a note that Cumulus is disaggregated and built on FOSS, but it is not free (costs dollars). Some of the below lifted from http://packetpushers.net/virtual-toolbox/list-network-operating-systems/ Other somewhat free options, some user assembly required at times: https://snaproute.com/ https://azure.github.io/SONiC/ http://opennetlinux.org/ https://github.com/facebook/fboss Others that play in this space while not necessarily free: http://www.pica8.com/products/picos https://www.ipinfusion.com/products/ocnos/ -- Hugo Slabbert | email, xmpp/jabber: hugo@slabnet.com pgp key: B178313E | also on Signal On Tue 2018-Jan-09 13:14:10 +0000, Edwin Pers <EPers@ansencorp.com> wrote:
Here's one you missed: http://www.projectfloodlight.org/indigo/
If you're only interested in stuff that goes on iron, openvswitch is out - it's pure software meant to run on hypervisors
-Ed
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Hank Nussbacher Sent: Tuesday, January 9, 2018 2:18 AM To: nanog@nanog.org Subject: Comparison of freeware open source switch software?
I have seen numerous comparisons and RIPE presentations on performance issues of BIRD vs Quagga vs FRR.
I am looking for the same thing for freeware switch software. Has anyone done a feature comparison between: http://openvswitch.org/ https://www.openswitch.net/ https://cumulusnetworks.com/products/cumulus-linux/ ...any other I am missing...
I am familiar with: http://packetpushers.net/open-networking-cheat-sheet/ https://www.networkworld.com/article/2919599/cisco-subnet/clearing-the-fog-a... so to clarify I am interested only in bare-metal or whitebox swicthes and freeware, open source software.
And even better - has anyone done a benchmark to see which performs best?
Thanks, Hank
If you're only interested in stuff that goes on iron, openvswitch is out - it's pure software meant to run on hypervisors
Not necessarily true anymore. Look for SwitchDev, which is incorporated into the Linux kernel , is undergoing continuous improvement, and allows the kernel to offload forwarding rules to the hardware. This allows open source software like Open vSwitch and Free Range Routing to work natively/directly with hardware. http://www.mellanox.com/page/press_release_item?id=1983 https://www.kernel.org/doc/Documentation/networking/switchdev.txt https://blog.raymond.burkholder.net/ Raymond Burkholder. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
If you're only interested in stuff that goes on iron, openvswitch is out - it's pure software meant to run on hypervisors
Not necessarily true anymore. Look for SwitchDev, which is incorporated into the Linux kernel , is undergoing continuous improvement, and allows the kernel to offload forwarding rules to the hardware.
The overall architecture of openswitch, however, seems (to me) to be focused on software implementations, rather than hardware. You probably want to look at SONiC as well, as this is what LinkedIn is using; there are a lot of improvements currently going into code. For FR Routing, the performance is changing rapidly, as there are a lot of commits going in just about every week, and the community is quite active. :-) /r http://rule11.tech
Not necessarily true anymore. Look for SwitchDev, which is incorporated into the Linux kernel , is undergoing continuous improvement, and allows the kernel to offload forwarding rules to the hardware.
The overall architecture of openswitch, however, seems (to me) to be focused on software implementations, rather than hardware.
I suppose that could be a fair statement, to a certain degree. But when you strip away all the abstracted things it does, and get into the core of the product, you find an OpenFlow engine, which at the heart, is a mechanism for defining traffic rules. Many of those rules translate easily into the kernel's forwarding information base. And the kernel's FIB can easily be two-way sync'd with hardware. So, to take this to the logical conclusion, other than for the interesting rules which can be created in OVS or FRR, it is possible to do away with OVS/FRR, and use only Linux itself with iproute2 and bridge commands to drive the hardware directly with no further abstraction or vendor black boxes. And, rather than using a J or a C syntax, you use an /etc/network/interfaces syntax for permanence. https://blog.raymond.burkholder.net/ Raymond Burkholder -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
SwitchDev, which is incorporated into the Linux kernel Neat! I'll have to keep my eyes on this in the future, it'd be cool if we could have VyOS handling routing on the hardware and the vm hosts, would save me a bit of brainpower
-Ed -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Raymond Burkholder Sent: Tuesday, January 9, 2018 11:12 AM To: nanog@nanog.org Subject: RE: Comparison of freeware open source switch software? <snip>
On Tue, 09 Jan 2018 02:17:59 -0500, Hank Nussbacher <hank@efes.iucc.ac.il> wrote:
so to clarify I am interested only in bare-metal or whitebox swicthes and freeware, open source software.
It's my understanding that there simply is no such thing. Because none of the HARDWARE has open source code. Sure, anyone can write software to spirit packets between NICs (linux and *BSD has had that capability for decades.) But doing that "at scale" with the various manufacturers SoCs requires vendor specific code to setup and control the chip. The broadcom "NDK" is just a shim on top of a pre-compiled proprietary SDK blob. --Ricky
https://www.opennetworking.org/ Hardware works quite well. I have a number of whitebox units deployed based off their designs and will be ordering more. On Tue, Jan 9, 2018 at 6:09 PM, Ricky Beam <jfbeam@gmail.com> wrote:
On Tue, 09 Jan 2018 02:17:59 -0500, Hank Nussbacher <hank@efes.iucc.ac.il> wrote:
so to clarify I am interested only in bare-metal or whitebox swicthes and freeware, open source software.
It's my understanding that there simply is no such thing. Because none of the HARDWARE has open source code. Sure, anyone can write software to spirit packets between NICs (linux and *BSD has had that capability for decades.) But doing that "at scale" with the various manufacturers SoCs requires vendor specific code to setup and control the chip. The broadcom "NDK" is just a shim on top of a pre-compiled proprietary SDK blob.
--Ricky
-- :o@>
My understanding is the same as Ricky's. At least in the Broadcom word, you have to license the SDK from Broadcom in order to develop against it and, more importantly, have documentation of which register does what. I don't know if you need to license it to program the ASIC (assuming you can do it without SDK in a sensible fashion). My understanding was that when you buy software such as Cumulus Linux, what you are actually paying for is the Broadcom license. You can actually go and download Cumulus Linux and it's all open source except, you guessed it, switchd, which is what takes the info from the linux kernel and programs it into the hardware. My understanding was that the rest the "open source" OSs operate the same way. Please, correct me if it is at all possible to buy a whitebox switch and then load a "no cost OS" on it and it starts switching packets through hardware. -Andrey --Andrey On Tue, Jan 9, 2018 at 6:23 PM, Oliver O'Boyle <oliver.oboyle@gmail.com> wrote:
https://www.opennetworking.org/
Hardware works quite well. I have a number of whitebox units deployed based off their designs and will be ordering more.
On Tue, Jan 9, 2018 at 6:09 PM, Ricky Beam <jfbeam@gmail.com> wrote:
On Tue, 09 Jan 2018 02:17:59 -0500, Hank Nussbacher < hank@efes.iucc.ac.il> wrote:
so to clarify I am interested only in bare-metal or whitebox swicthes and freeware, open source software.
It's my understanding that there simply is no such thing. Because none of the HARDWARE has open source code. Sure, anyone can write software to spirit packets between NICs (linux and *BSD has had that capability for decades.) But doing that "at scale" with the various manufacturers SoCs requires vendor specific code to setup and control the chip. The broadcom "NDK" is just a shim on top of a pre-compiled proprietary SDK blob.
--Ricky
-- :o@>
On 01/09/2018 10:46 PM, Andrey Khomyakov wrote:
My understanding was that when you buy software such as Cumulus Linux, what you are actually paying for is the Broadcom license. You can actually go and download Cumulus Linux and it's all open source except, you guessed it, switchd, which is what takes the info from the linux kernel and programs it into the hardware.
I believe that is Cumulus' business model where they ride the requirement for the Broadcom license. So part of the license fee is for Broadcom and part is for Cumulus. Cumulus has used their licensing fees to develop and maintain tooling for their ecosystem. In that process, they have released many of their tools to the opensource world. Things like ifupdown2 and Free Range Routing are a result of that model.
My understanding was that the rest the "open source" OSs operate the same way.
Pica8 does something similar.
Please, correct me if it is at all possible to buy a whitebox switch and then load a "no cost OS" on it and it starts switching packets through hardware.
The only case in which I know that you can purchase a bare metal switch and put a 'no cost OS' such as Linux on is Mellanox. They are the ones who have developed the switchdev module I mentioned in an earlier post. There is _no_ requirement for a black box module to interface between the software and the hardware. That whole path down to but not including the hardware is opensource. There is a large carrier who is making use of switchdev: https://lwn.net/Articles/675826/
so to clarify I am interested only in bare-metal or whitebox switches and freeware, open source software.
There is something else called SAI, which may do something similar to switchdev: http://packetpushers.net/sai-and-switchdev-need-to-succeed/
It's my understanding that there simply is no such thing. Because none of the HARDWARE has open source code. Sure, anyone can write software to spirit packets between NICs (linux and *BSD has had that capability for decades.) But doing that "at scale" with the various manufacturers SoCs requires vendor specific code to setup and control the chip. The broadcom "NDK" is just a shim on top of a pre-compiled proprietary SDK blob.
As mentioned above, and in another message, Mellanox is the only one, that I know of, for providing a mechanism to drive the hardware (both switch and network cards) from the OS, without a pre-compiled proprietary SDK blob (black box). http://www.mellanox.com/related-docs/prod_switch_software/PB_Spectrum_Linux_... Disclaimer: I not affiliated with Mellanox, only a customer. -- Raymond Burkholder ray@oneunified.net https://blog.raymond.burkholder.net -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
My understanding is the same as Ricky's. At least in the Broadcom word, you have to license the SDK from Broadcom in order to develop against it and, more importantly, have documentation of which register does what. I don't know if you need to license it to program the ASIC (assuming you can do it without SDK in a sensible fashion).
If you can live within SAI's capabilities, SAI is freely available for Broadcom. This is what SONiC uses to communicate to the Broadcom chipset, and it works well. If you really want a rundown, please ping me off list; I have a good bit of information I could share from prior presentations... You can also look up my presentations at nanog/lacnog on the topic, but I'm glad to fill in what I know beyond what's there if it's helpful. 😊 Russ
participants (8)
-
7riw77＠gmail.com -
Andrey Khomyakov -
Edwin Pers -
Hank Nussbacher -
Hugo Slabbert -
Oliver O'Boyle -
Raymond Burkholder -
Ricky Beam