Michael Dillon writes:

You can also try building a machine with a boot device like the 2.88 megabyte floppies. Using the same techniques FreeBSD uses for their boot disks, you can decompress the boot floppy into a large RAMDISK and run that way. Or simply use a ZIP drive for the boot device but run from RAM as before. It's not as good as 100% solid state but it comes pretty close.

This isn't clear to me. Why do you assume a ZIP is likely to be more reliable that a hard disk? ZIPs haven't been around long enough to be sure of this, and HDs are pretty reliable these days. Of course I'm not saying that I *Want* to use an HD in this situation; flash is clearly a big win. But I don't see how using a floppy or ZIP improves wins. FWIW, I suspect that building a 1.4MB fs that can boot and then nfs-mount (or ftp to a memory fs) needed binaries would be not a lot harder for FreeBSD or BSDi than it was for NetBSD. /a

Michael Dillon writes:

I'm not suggesting that the ZIP be used while running the router, just to boot it up and create a RAM drive to run in. The advantage of the ZIP over the floppy in this scenario is that you don't have to compress and squeeze everything in order to modify the boot diskette. If you make a change to gated.conf it can just be copied to the ZIP drive and be ready to go in the event that the router needs to be rebooted.

I've managed it on one floppy without compressing the kernel. Although I won't deny that compression would give a lot of room to play around in.

...

FWIW, I suspect that building a 1.4MB fs that can boot and then nfs-mount (or ftp to a memory fs) needed binaries would be not a lot harder for FreeBSD or BSDi than it was for NetBSD.

My floppy/ZIP boot scenario assumes that you would not be NFS mounting anything but that the router would be self contained.

My scenario doesn't run off the NFS mount, it just uses it (or ftp) to pull files over. Then it runs from RAM. If the router is totally isolated (all paths out are down) then it loses, but in that case, what difference does it make? /a

And besides, as Paul Traina said a couple of years back on the topic of BSDI boxes vs. Ciscos: "Hey, you can't play DOOM on those boring old Cisco routers!"

Maybe someone is, and that is why so many routers are flapping. :) -Deepak.

From: Michael Dillon <michael@memra.com> On Sat, 19 Oct 1996, Robert E. Seastrom wrote:

With cold-convenient-swappable IDE drawers that let even a kindergartener swap out an IDE hard drive and high quality 100mb hard drives available for like $50

Where on earth do you find new drives this small??? http://www.corpsys.com But apparently with their current stock of new drives you'll have to settle for IBM 3.5" 170mb drives for $99 each ($89 qty 10). Now, if you don't mind refurbs, there are $65/$55 80mb drives and $69/$59 131mb drives. I've dealt with these folks before and they're righteous. Oh well, the $50 was a little optimistic, or maybe not if you're buying a bunch of 'em... ---Rob

On Sat, 19 Oct 1996 20:00:23 -0400 (EDT) Deepak Jain <deepak@jain.com> alleged:

Our vendors have told us there were having a hard time finding any kind of quantities of IDE drives < 2.5GB.

Its a good question.

I'd be changing my vendor in such a case :-) Neil -- Neil J. McRae. Alive and Kicking. E A S Y N E T G R O U P P L C neil@EASYNET.NET NetBSD/sparc: 100% SpF (Solaris protection Factor) Free the daemon in your <A HREF="http://www.NetBSD.ORG/">computer!</A> [N.B.: I have no affilation with Cyberia or the CyberWorkers Unite Party]

Robert E. Seastrom writes:

From: Alexis Rosen <alexis@panix.com> [...] This isn't clear to me. Why do you assume a ZIP is likely to be more reliable that a hard disk? ZIPs haven't been around long enough to be sure of this, and HDs are pretty reliable these days.

I think his plan was to boot ramdisk unix from it and then to spin the unit down. Spun-down units are fairly reliable, and besides I think the main thrust here was to replace the drive with something that could be swapped easily for upgrades.

That's more easily done using Vixie's scheme. *My* main thrust is a system with no moving parts (assuming that that's a good way to achieve higher reliability). Of course, both goals are desireable.

Of course I'm not saying that I *Want* to use an HD in this situation; flash is clearly a big win. But I don't see how using a floppy or ZIP improves wins.

With cold-convenient-swappable IDE drawers that let even a kindergartener swap out an IDE hard drive and high quality 100mb hard drives available for like $50 (at this point you're probably paying more for the snazzy mounting kit than you are for the drive), I daresay the Zip and flash solutions are far too expensive for what they buy you. Take a look at the MTBF on your hard drives and then look at the MTBF on power supplies and floppies, and gee... Alexis is dead on here. Pay more, get less...

I think that realistically, finding a stable supply of anything smaller than 1GB is unlikely, unless you're buying large quantities. More to the point, disks for less than $100 are unlikely. But it's not the money that's key here. One service call to a lights-out facility costs lots more than that. Another solution occurs to me. Use dual-anything (floppies, HDs, flash gizmos). Pay someone to modify the BIOS in this small but important way: have it alternate which device it boots from. Let the machine write check a state file kept somewhere else each time it boots. If the same device has booted twice in a row, there's a problem and you notify the NOC that a device has failed. Otherwise rewrite the state file. There's another better solution. It would take a little bit more work but it would be infinitely more useful: Build an ISA card that looks like an MDA adapter, but which sends output to a VT100. (A simple algorithm will produce good results here even when the cursor jumps all over the place.) You also need a gizmo (one exists alread) that makes serial input look like PC keyboard scan codes. And lastly, you need to special-case a long-break so that it causes the ISA card to reset the machine. All of a sudden you've got a working remote console device. This is an extraordinarily useful gizmo, and I'll bet you could sell a million of them. (Everyone who uses Linux, FreeBSD, or NetBSD as a server would buy it, as would the millions of poor souls who run Novell, SCO, etc.) I'd buy fifty of them at $100 apiece, and at that price you'd probably see a gross margin of 300%, even in small quantities, and including the keyboard gizmo. (Now all we need is a better word than "gizmpo" and we've got a real product. :-) Anyone interested in some hardware design work? /a

as long as we're doing hardware design on the nanog list again, i'd like to mention that there's a little device that sits on an ISA bus and has an onboard PCIC (PCMCIA bus controller). to this one attaches a pair of 50-pin ribbon cables, and to these one attaches a device that fits physically where a 3.5-inch floppy drive would fit (which means you generally need a 5.25-inch expansion mount, cost:$3.00). the whole thing costs about $150. if your UNIX-like system that runs on an ISA bus also would run on a laptop and knows how to support PCMCIA devices, it will see these slots as completely normal. and if you put an NCR WaveLAN into one, you have yourself an ether/wireless router. and if you put a 20MB ATA Flash card into it, it looks like a (slow) file system. and if your ROM knows how to boot from ATA (as it might if it can boot from an ATA CDROM), presto: BSD router, no moving parts. naturally you need to edit the hell out of /etc/rc to make it build a big ramdisk, populate it from the Flash's binary tarball, and chroot to it before starting gated. the chroot'd /etc/gated.conf should be a symlink to a small config file system on a second ATA Flash. "/", even though a RAM disk, is mounted read-only. system upgrades are done by powering off the unit, replacing "drive 0" with an updated 20MB Flash card, and powering it back up. caveats: 20MB isn't very large for a BSD system, even with shlibs -- you have to be highly selective about what you take; also, for /var/log it is probably a good idea to include a rotating magnetic media, unless you're going to use syslog's "remote log server" mode, which since it's UDP is not reliable enough for some forms of auditing that i've needed to design for. hope this helps.

On Sun, 20 Oct 1996, Paul A Vixie wrote:

as long as we're doing hardware design on the nanog list again, i'd like to mention that there's a little device that sits on an ISA bus and has an onboard PCIC (PCMCIA bus controller). to this one attaches a pair of 50-pin ribbon cables, and to these one attaches a device that fits physically where a 3.5-inch floppy drive would fit (which means you generally need a 5.25-inch expansion mount, cost:$3.00). the whole thing costs about $150.

if your UNIX-like system that runs on an ISA bus also would run on a laptop and knows how to support PCMCIA devices, it will see these slots as completely normal. and if you put an NCR WaveLAN into one, you have yourself an ether/wireless router.

Yes, the prob is that it uses the CPU to do the BGP stuff and to route packets. This is a bad way to do it, you dont want to use your CPU to move the packets. Nathan Stratton CEO, NetRail, Inc. Tracking the future today! --------------------------------------------------------------------------- Phone (703)524-4800 NetRail, Inc. Fax (703)534-5033 2007 N. 15 St. Suite 5 Email sales@netrail.net Arlington, Va. 22201 WWW http://www.netrail.net/ Access: (703) 524-4802 guest --------------------------------------------------------------------------- "Therefore do not worry about tomorrow, for tomorrow will worry about itself. Each day has enough trouble of its own." Matthew 6:34

I said how it could be done, not that it ought to be done. I have found a P5-150 with BSD/OS, GateD, ScreenD, and DEC FDDI or Ether (PCI DMA either way) to be a perfectly useful gateway/firewall. It won't do full FDDI but my root name server can't tell the difference so I must not be facing that load. I've also run four T1's, or 64 28.8K modems, through one of these boxes. But the bit and packet loads in these cases are "trivial" compared to a core router inside any nationwide/worldwide network, either Inter or Intra. When only a Cisco or Netstar will do, my boxes are toys. But the world has an ongoing need for more toys -- not every router is doing 300K packets per second with multiple OC12 links. Really, I do not like PC-based routers, through this kind of routers have some advantages:

(1) when PC-based router became out of memory, I have to add some more memory - I pay about 200$ for extra 16Mb of ram, and that's all; (2) when PC-based router became out of CPU, it can be upgraded to the faster CPU easy. Intel's power increases draqmatically every month, and I have'not pay extra 100,000$ for the new super/giga/huge-ROUTER (as 7513) - I pay new 1,500$ and get new PC with Pentium/200, for example. And I know there would be available better processor in next 6 month - and I would'not have to pay next 100,000$ (or I there have to pay new 20,000$ for the new CS4700, for example - why can't I change CPU in CS4500, or why can't I add extra 32Mb of the RAM into my CS4500, and WHY have I to pay 3,500$ for the 32Mb ram if this RAM costs 600$ on the free market???). This is the advantages of PC. Hope you know disadvantages too -:)

To the argument that Cisco IOS is inherently easier or harder to configure ^^^^^^^^^^^^^^^^^^^^^^^^^^ -:) This is a joke... there is nothing more complex and orderless than IOS's config -:)

than GateD, I say: "Feh." If you can get an IOS geek with 7+ years worth of IOS-shaped tire tracks down their backside, then IOS will seem a lot more powerful. If all you can get is me, IOS will seem slippery and awkward and confusing and gated.conf will seem like deliverance. Anybody who cuts and pastes config examples to demonstrate why one is "obviously clearer" is just blowing smoke. The rare element here is human expertise, not documentation clarity or parser simplicity or any of the things geeks like to argue about.

In an overlooked comment of a few days ago, someone here mentioned that it was generally easier to get someone with nonzero expertise to come help run your network if you configured it via Cisco IOS rather than gated.conf. And this is true. For now. If someone else gets market share (which is usually done via other means than technical merit, btw) then the other guy's config syntax will start to get known by more folks. Given that it is *definitely* better to build a network that new hires can help you run, if that network is expected to grow at all, Cisco IOS has a real edge right now. I don't consider Cisco terribly vulnerable since if they wanted to drop their prices by half they'd still make a pile of money. Not someone to compete against; they can beat you coming or going. That's why I so admire the folks who *are* trying to beat Cisco in this game. What chuzpah! <clink>.

--- Aleksei Roudnev, Network Operations Center, Relcom, Moscow (+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 239-10-10, N 13729 (pager) (+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)

alex@relcom.eu.net writes:

...

and WHY have I to pay 3,500$ for the 32Mb ram if this RAM costs 600$ on the free market???).

Dunno for sure about the 4xxx, but the 7xxx use standard SIMMs, and Cisco has even qualified some manufacturers' parts. Oddly enough, they have also *disqualified* some parts that you'd think were exactly the same. I don't know the details there. The sales types obviously don't like to talk about it, but you really *can* get $12k of 7500 memory for $800. And they'll even agree that it works, if you talk to the right cisco folks.

/a

$400 now... Avi

Nathan, I have to follow up on this comment you made. I work for an ISP in Indianapolis called IHETS. We have 7 cisco 7500 series routers on our backbone. Wre just spent a chunck of money to upgrade the RAM on them to 64Meg. If yoiu are saying that regular SIMMS work in Cicso routers, then I'll shoot myself in the foot. I'd like to know if you have used this method and how well it works out in the field. Does latency increase with *unqualified* Cisco memory, or is the *unqualified* *qualified* Cisco ram stuff just a big ruse? Thanks for your input.

On Mon, 21 Oct 1996, Alexis Rosen wrote:

...

alex@relcom.eu.net writes:

...

and WHY have I to pay 3,500$ for the 32Mb ram if this RAM costs 600$ on the free market???).

Dunno for sure about the 4xxx, but the 7xxx use standard SIMMs, and Cisco has even qualified some manufacturers' parts. Oddly enough, they have also *disqualified* some parts that you'd think were exactly the same. I don't know the details there. The sales types obviously don't like to talk about it, but you really *can* get $12k of 7500 memory for $800. And they'll even agree that it works, if you talk to the right cisco folks.

Yep, it uses standard SIMMs, I have upgraded many of them.

On Mon, 21 Oct 1996, Nathan Stratton wrote: ==>> how well it works out in the field. Does latency increase with ==>*unqualified* > Cisco memory, or is the *unqualified* *qualified* Cisco ==>ram stuff just a big > ruse? Thanks for your input. ==> ==>Na, ram is ram. It is just like FDDI cables. I spent $250 on FDDI cables, ==>and then found out that I can get the exact same cable for $50 each. There is a list of third-party RAM that Cisco approves, and this list is available on CCO *somewhere*. I don't remember where, though; Paul Ferguson is really good at remembering where this stuff is, though. The reason you should at least follow Cisco's recommendation for RAM is to keep your router at least running sanely. I have seen WAY TOO MANY machines (including routers) do really weird stuff because the operator(s) used cheap re-labeled RAM from Fry's or similar. Usually, it's in the form of weird segfaults in UNIX, or software-forced-reloads in the case of routers. /cah

If we are going to argue, then let's talk about functionalities of various implementations and then take it to respective developers of your favorite implementation so that all our tools are better. We are using _both_, and have some comparation:

(1) IOS have more features in comparation with gated - no doubt, because IOS was developed in the longer time. Through gated.conf is well-defined language, IOS's config is a great heap of very strange commands and nodoby (almost) can config IOS coprrectly withouth great practice. (2) There is some tasks we can't solve in IOS at all, or can solve in very hacker's vays. That's because IOS and gated have quite different schemes of redistridutions (in gated, I can control PROTO->PROTO redistribution on the per-neighbour basis; in IOS I can control redistribution into the BGP protocol totally); (3) There is very interesting idea in gated - it recalculates 'localpref' into the 'preference'; it allows to provide some back-up schemas we can't do in IOS at all. Of cource gated have not (yet?) some IOS's functionality - bgp reflection, bgp community (IMPORTANT!), have quite other (in comparation with IOS) 'localpref' control. But when we are generating our gated's and IOS's configs from the data base - this task is much easy for _gated_ than for _IOS_. And I hope if Ascend have made commercial revision of gated for their routers - they does have a chance to compete with the IOS. --- Aleksei Roudnev, Network Operations Center, Relcom, Moscow (+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 239-10-10, N 13729 (pager) (+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)

I'd really like to see this move to gated-people@gated.org or something since I think it's gotten kind of off-topic, but there were a few comments recently that seemed to call for a reply... At 4:11 PM +0400 10/21/96, alex@relcom.eu.net wrote:

Of cource gated have not (yet?) some IOS's functionality - bgp reflection, bgp community (IMPORTANT!)

At 9:13 AM -0400 10/21/96, Avi Freedman wrote:

Does it have AS-Path padding? I know some (hi Peter) think it's evil, but many use it...

Note that we have added all of this stuff to Ascend's (Netstar's) GateD. As Sue Hares pointed out in an earlier message, these features will eventually be folded back into the public GateD release. Regards, --John -- John Scudder email: jgs@ieng.com Internet Engineering Group, LLC phone: (313) 669-8800 122 S. Main, Suite 280 fax: (313) 669-8661 Ann Arbor, MI 41804 www: http://www.ieng.com

...

But when we are generating our gated's and IOS's configs from the data base - this task is much easy for _gated_ than for _IOS_.

It would be useful if at least some of the folks doing config generations from databases could get together and do some collaboration.

Its amazing to me the number of ISPs that don't have these tools. --asp@partan.com (Andrew Partan)

Perhaps a brief review of at least one of the types of tools at NANOG would be in order. I'm on the hook to discuss the RtConfig and ROE tools and have the hooks (with Dave Meyers scripts) to build an IOS configuration with some ease. -- --bill

==> From: alex@relcom.eu.net ==> Tue, 22 Oct 96 22:14:39 +0400

There exist RIPE and other data bases - yes. They does have route objects. But this objects are (usially) oriented to the As path filtering, not more. Who really builds filter lists for the networks via data bases? I know about MCI (and their DB); our provider EUnet build this list by hands (exactly Peer Bilse's hands -:)); no one of scientific networks there does more than AS-path filtering.

Small correction. The current EuropaNET/INCS service is run by BT and they definitely create network filters based on the RIPE DB, on top of AS path filters. Cheers, Steven

We there are strictly controlling our customers via filters and access lists because we know exactly ip networks he have, and all channels he can use to anounce this networks, and so on - (except some peers).

I think that I agree with your approach.

There exist RIPE and other data bases - yes. They does have route objects.

There are other parts of the config than just customer routes - like interfaces, IGP/BGP issues, LAN/WAN stuff. It would be interesting to work on something that tried to cover more than just a few parts of the config, and that could handle both cisco & gated configs (I'll take others as a bonus). --asp@partan.com (Andrew Partan)

Avi Freedman writes:

...

This is the advantages of PC. Hope you know disadvantages too -:)

...

To the argument that Cisco IOS is inherently easier or harder to configure ^^^^^^^^^^^^^^^^^^^^^^^^^^ -:) This is a joke... there is nothing more complex and orderless than IOS's config -:)

This is my last comment on IOS vs. Gated, but: [etc.]

Everyone on this list should try mucking about with, say, a Motorola Vanguard. Then you'd quit whining. :-) IOS and Gated are both *miles* ahead of some of the junk out there. Hm. In order to settle this once and for all, I suggest the following unassailably scientific test: Get a pair of twins who are both admins who know nothing about IOS or GateD. Have experts in GateD teach one and experts in IOS teach the other. The first admin who can configure a box that successfully drops in as a replacement for any defaultless core router has proven that his software is better. If either one commits suicide before one succeeds, get another set of twins. /a

ok so livingstons are easy to configure.. but are they easy to REconfigure?? NOPE!! Is it easy to take a look at the entire configuration at one glance??? NOPE!! If your livingston were to go down and you needed a new livingston, start praying to god. Miss one detail and you are still dead. Pritish On Mon, 21 Oct 1996 edd@acm.org wrote:

...

Bay is not. Proteon is not. Gated (for me, so far) is not. Microrouters are not. Morningstars are/(?were?). Livingstons are not.

Why Livingstons are not??? They're terribly easy to configure, IMHO...

...

...

Aleksei Roudnev, Network Operations Center, Relcom, Moscow

I can only speak for myself, of course.

Me too,

Ed

...

Bay is not. Proteon is not. Gated (for me, so far) is not. Microrouters are not. Morningstars are/(?were?). Livingstons are not.

Why Livingstons are not??? They're terribly easy to configure, IMHO...

You *must* be joking. Or maybe you haven't worked with a Livingston frame relay port that has multiple PVCs on it. It is *in no way* "super-elegant and thus easy to predict and use", not even close. -- Shields, CrossLink.

Paul A Vixie writes:

[...] naturally you need to edit the hell out of /etc/rc to make it build a big ramdisk, populate it from the Flash's binary tarball, and chroot to it before starting gated. the chroot'd /etc/gated.conf should be a symlink to a small config file system on a second ATA Flash. "/", even though a RAM disk, is mounted read-only. system upgrades are done by powering off the unit, replacing "drive 0" with an updated 20MB Flash card, and powering it back up.

caveats: 20MB isn't very large for a BSD system, even with shlibs -- you

It's not at all clear that you need to chroot at all. And it's true that 20MB isn't much, but how much (outside of /) do you really need? Give yourself a few MB of buffer cache and the speed of flash won't matter much anyway. You can be damn sure you're not going to be swapping, and that's as it should be, given what this device is doing. (If it's not a router it's probably a nameserver, and neither wants to swap.) Now, since it's a real unix, it would be nice if you could use all your favorite utilities while doing administration. But that's not hard- temporary NFS mounts aren't all that nasty, and if you don't want to jam up your backbone with NFS, copy stuff via ftp into a tempfs.

have to be highly selective about what you take; also, for /var/log it is probably a good idea to include a rotating magnetic media, unless you're going to use syslog's "remote log server" mode, which since it's UDP is not reliable enough for some forms of auditing that i've needed to design for.

Hm. Interesting. I wonder how hard it would be to make syslog use TCP? Not very, I think. The only remaining fly in the ointment would be short-term situations where you can't move bytes of the net fast enough. Switching log devices to an MFS/tempfs file temporarily might solve that too. /a