My comment was incorrect on T1/E1. We were trying to do full DES encryption of a T1 stream and couldn't saturate the link using a modified BSDI kernel and a 486/66. Full T1/E1 is possible using existing cards if you don't try to encrypt.

Additionally, some early versions of BSDI had problems with doing the processing of the protocol queues via netisr in a timely manner and this limited the maximum packet rate. I haven't tested more recent versions of BSDI for this problem.

Wrt ethernet - I was under the impression that the fastest EISA card was the 3C579 and the fastest ISA the TN-1500.

I've never benchmarked a 3c579 (or seen one for that matter), but if it's anything like other recent boards from 3COM...I would be surprised if it actually worked very well. perhaps the important point that I should make here is that bus master DMA does not necessarily mean fast or low overhead.

I'd be surprised to hear that a WD/SMC EISA card can saturate 2-3 ethernets at small packet sizes (typical traffic seen by a router). This is more a reflection of my opinion of PC card technology, not FreeBSD or BSDI.

Actually, I think it could saturate 2 ethers with an SMC ISA (not EISA) board. Maybe I should actually try this. :-) The driver I wrote for FreeBSD is actually quite well done (I'm patting myself on the back :-)) and uses a multi buffering technique I invented that has to my knowledge never been previously implemented. A typical "ttcp" run looks like this: (transmitter) [corbin:davidg]> ttcp -t implode ttcp-t: buflen=8192, nbuf=2048, align=16384/+0, port=5001 tcp -> implode ttcp-t: socket ttcp-t: connect ttcp-t: 16777216 bytes in 14.95 real seconds = 1096.08 KB/sec +++ ttcp-t: 2048 I/O calls, msec/call = 7.47, calls/sec = 137.01 ttcp-t: 0.0user 3.6sys 0:14real 24% 32i+188d 110maxrss 0+2pf 5602+9csw (receiver) [implode:davidg]> ttcp -r ttcp-r: buflen=8192, nbuf=2048, align=16384/+0, port=5001 tcp ttcp-r: socket ttcp-r: accept from 198.145.90.2 ttcp-r: 16777216 bytes in 14.96 real seconds = 1095.00 KB/sec +++ ttcp-r: 11372 I/O calls, msec/call = 1.35, calls/sec = 760.03 ttcp-r: 0.0user 2.4sys 0:14real 16% 32i+188d 152maxrss 0+2pf 11265+33csw I think the most important thing above is the (low) percent CPU time. The actual consumed CPU time is somewhat higher (not all of the system time is reflected in the percentage) - kernel profiling shows it to be about 30%.

If anyone has measured troughput under small packet load, I'd be interested in hearing the result.

Do you have a suggestion on how I might do such a test? It's a problem because TCP will coalesce socket writes until a full frame is gathered unless the TCP_NODELAY socket option is used. I believe setting this option will have other bad side effects (like setting the PUSH bit, and thus might solicit an immediate acknowledge). I'd be tempted to do a UDP test, but due to bugs in the UDP layer of all Net/2 derived code I've seen (including BSDI, NetBSD, and FreeBSD), the UDP performance is rather low - and additionally is difficult to test because of the lack of connection flow control. In fact, BSDI and NetBSD have a bug that causes a pathological condition whereby partial UDP datagrams will be fragmented/sent and resent repeatedly should a client application attempt to write packets too quickly to the socket.

Wrt DS3 - I didn't think a HSSI card for EISA existed. Does one exist?

I don't know; I did say that "this assertion is unproven". I based the opinion on extensive kernel profiling analysis of the socket and TCP performance.

Also, keep in mind that routers see an average packet size on the order of 200 bytes. Do you really think an EISA bus can handle the small packet throughput? Big packets are easy, too bad real traffic isn't all big packets.

All the analysis I've done of bottlenecks in FreeBSD has shown that the actual overhead of packet handling was fairly low compared to the time to get/put data out to ISA shared memory. Unlike BSDI, FreeBSD's interrupt overhead is extremely low and would not be significantly affected by the increased interrupt rate. This isn't the only factor of course, and the large number of variables makes it difficult to guess how well it might work without actually testing specific hardware.

BTW - I've corrected my own untruths by way of Cc. Thanks for the note and the good work that you are doing on FreeBSD.

Thanks. I hope this rather lengthy reply isn't totally boring other folks on this mailing list. BTW, is this a list I should be subscribed to? -DG