The basic idea of RAMBOOT is typical in Embedded Linux development. Linux makes use of multi-stage boot process. One of the stages involves using an initial ramdisk (initrd) to provide a base root filesystem which can be used to locate and mount the system root, then continue the boot process from there. For an in-memory OS, instead of the boot process mounting a pre-loaded storage device with the expected root filesystem (e.g. your installed HDD), you modify it to: 1) Create and format a ramdisk. 2) Create the expected directory structure and system files for the root filesystem on that ramdisk. The root filesystem includes the /dev directory and appropriate device nodes, the basic Linux filesystem and your init program. The easy way to do that is just to have a TAR archive that you extract to the ramdisk on boot, better yet use compression on it (e.g. tar.gz) so that the archive can be read from storage (e.g. USB flash) more quickly. Today, the initramfs in Linux handles a lot more than simply mounting the storage device. It performs hardware discovery and loads appropriate modules. As such the Debian project has a dynamic build system for initramfs that is run to build the initrd when a new kernel package is installed, it's called "initramfs-tools". You can manually build your own initramfs using the examples on the RAMBOOT website, but the point of RAMBOOT is to make building an in-memory OS quick and simple. RAMBOOT instead adds configuration to initramfs-tools so that each time a new initrd is generated, it includes the code needed for RAMBOOT. The RAMBOOT setup adds handling of a new boot target called "ramboot" to the kernel arguments. This allows the same kernel to be used for a normal installation and remain unaffected, but when you add the argument "boot=ramboot" as a kernel option to the bootloader, it triggers the RAMBOOT process described above. Having a common kernel between your development environment and embedded environment makes it much easier to test and verify functionality. The other part of RAMBOOT is that it makes use of "Ubuntu Core". Ubuntu Core is a stripped down minimal (and they really do mean minimal) root filesystem for Embedded Linux development. It includes apt-get, though, so you can install all the packages you need from Ubuntu on the running system. RAMBOOT then has a development script to make a new root filesystem archive with the packages you've installed as a baseline. This allows for you to boot a RAMBOOT system, install your desired packages and change system configuration files as desired, then build a persistent image of that install that will be used for future boots. I also have the start of a script to remove unused kernel modules, and other files (internationalization for example) which add to the OS footprint. You could build the root filesystem on your own (and compile all the necessary packages) but using Ubuntu Core provides a solid base and allows for the rapid addition of packages from the giant Ubuntu repository. Lastly, I make use of SYSLINUX as a bootloader because my goal was to use a USB stick as the bootflash on an Atom box. Unfortunately, the Atom BIOS will only boot a USB device if it has a DOS boot partition, so GRUB was a no-go. The upside is that since the USB uses SYSLINUX and is DOS formatted, it's easily mounted in Windows or Mac OS X, allowing you to copy new images or configuration to it easily. For the boot device I make use of the on-board vertical USB socket on the system board (typical for most system boards these days) and a low-profile USB stick. I find the Verbatim "Store 'n' Go" 8GB USB stick ideally suited for this as it's less than a quarter-inch high after the USB adapter. RAMBOOT as a project is in the very early stages, so you should be comfortable with Linux before you build a system on it. And I really feel it's more of an example than anything at this point. There are several advantages though: The most common point of failure on a Linux system is the storage device (either HDD or SSD). The biggest bottleneck in system performance is storage IO. Using a ramdisk eliminates both these concerns (in fact, even an Atom system has surprisingly great performance when run using a ramdisk). The result is that you get a very reliable, high-performance system. The other benefit to RAMBOOT is that the root filesystem is NOT persistent. This means that like a Cisco device, every boot of the system brings you to a known working state OS-wise. There are hundreds of system files in a Linux system; any one of which being modified could cause problems. For both security and availability concerns a lot of effort is invested in detecting changes to system files and avoiding them. With RAMBOOT the problem is easily avoided. A minimal system can fit within a 512MB ramdisk. But with RAM being so cheap these days, I think even reserving up to 2GB of RAM for a ramdisk would be fine (e.g. for a 4-8GB system). Here is the hardware configuration I originally started RAMBOOT for in 2011 (wanted to avoid the cost of an HDD): $326.36 (shipping included): 1U rack-mount case Supermicro CSE-502L-200B Intel Atom D510 system board with dual Gigabit (Intel 82574L) Supermicro MBD-X7SPA-H-O 2GB RAM 8GB low-profile USB flash drive (which will connect to the internal USB port and be low enough to fit in the case) Verbatim "Store n Stay". No HD; the system will boot off the 8GB flash into RAM and run the OS on a ramdisk. Using the RAMBOOT release that's currently up, I can build a custom Linux in-memory OS in half a day. I can easily update packages for security updates from the Ubuntu project and re-generate a new, updated, image in less time than that. So the initial goal of being able to build something useful quickly was satisfied, at least. My attention has now moved on to building a configuration management system, similar to Vyatta or VyOS and building a real distribution. I was going to call it "Carrier-grade Linux" (cglinux.org), but given the momentum VyOS has I might try to help the VyOS community instead of doing something new on my own. For what it's worth, I'm actually working with the VyOS project to try and incorporate some of the RAMBOOT ideas into VyOS as an install option for in-memory only. If you make use of RAMBOOT I would love to hear about it. :-) On Thu, Dec 26, 2013 at 4:22 PM, Nick Cameo <symack@gmail.com> wrote:
Inline response exist,
You can build using commodity hardware and get pretty good results.
I've had really good luck with Supermicro whitebox hardware, and Intel-based network cards. The "Hot Lava Systems" cards have a nice selection for a decent price if you're looking for SFP and SFP+ cards
On 12/26/13, Ray Soucy <rps@maine.edu> wrote: that
use Intel chipsets.
I like the supermicro as well however we have a couple of IBM x3250 with 2 pcie v3 x8 that are begging for a intel network card.
There might be some benefits in going with something like FreeBSD, but I find that Linux has a lot more eyeballs on it making it much easier to develop for, troubleshoot, and support. There are a few options if you want to go the Linux route.
This is very important to consider. I would be speculating, or even worse, expecting the same type of community support from the BSD verse that I have been getting from the linux community.
Option 1: Roll your own OS. This takes quite a bit of effort, but if you have the tallant to do it you can generally get exactly what you want.
If Free/OpenBSD is ruled out, I could crack open the LFS project. You only have to do it once right? Or maybe just reach out to the gentoo community for a stripped version, and build outwards.
The biggest point of failure I've experienced with Linux-based routers on whitebox hardware has been HDD failure. Other than that, the 100+ units I've had deployed over the past 3+ years have been pretty much flawless.
SSD
Thankfully, they currently run an in-memory OS, so a disk failure only affects logging. If you want to build your own OS, I'll shamelessly plug a side project of mine: RAMBOOT
RAMBOOT makes use of the Ubuntu Core rootfs, and a modified boot process (added into initramfs tools, so kernel updates generate the right kernel automatically). Essentially, I use a kernel ramdisk instead of an HDD for the root filesystem and "/" is mounted on "/dev/ram1".
The bootflash can be removed while the system is running as it's only mounted to save system configuration or update the OS.
I haven't polished it up much, but there is enough there to get going pretty quickly.
Ummm, if it's ok with the community, can you kindly elaborate :). I am not too fond of Debian since my horrible experience with Squeeze Desktop. I would maybe like to try this using the combination of SSD, in memory, and Gentoo?
You'll also want to pay attention to the settings you use for the kernel. Linux is tuned as a desktop or server, not a router, so there are some basics you should take care of (like disabling ICMP redirects, increasing the ARP table size, etc).
Totally strip it as much as possible. If anyone has a Gentoo stripped kernel config that they would like to share, please do :).
I have some examples in: http://soucy.org/xorp/xorp-1.7-pre/TUNING or http://soucy.org/tmp/netfilter.txt (more recent, but includes
firewall
examples).
Will definitely look into all your sites.
Also a note of caution. I would stick with a longterm release of Linux. I've had good experience with 2.6.32, and 3.10. I'm eager to use some
of
the post-3.10 features, though, so I'm anxious for the next longterm branch to be locked in.
We are comfy with 3.4 right now...
One of the biggest advantages is the low cost of hardware allows you to maintain spare systems, reducing the time to service restoration in the event of failure. Dependability-wise, I feel that whitebox Linux systems are pretty much at Cisco levels these days, especially if running in-memory.
Really interested with the "in-memory", however, I would love to implement it using gentoo as mentioned above.
Kind Regards,
N.
-- Ray Patrick Soucy Network Engineer University of Maine System T: 207-561-3526 F: 207-561-3531 MaineREN, Maine's Research and Education Network www.maineren.net