And while I certainly like open source solutions, there are plenty of commercial products that do things to optimize this. Depending on the type of traffic the products do different things. Many of the serial-byte caching variety (e.g. Riverbed/F5) now also do connection/flow optimization and proxying, while many of the network optimizers now are doing serial-byte caching. I also for a while was looking for multicast based file transfer tools, but couldn't find any that were stable. I'd be interested in seeing the names of some of the projects Robert is talking about- perhaps I missed a few when I looked. One thing that is a simple solution? Split the file and then send all the parts at the same time. This helps a fair bit, and is easy to implement. Few things drive home the issues with TCP window scaling better than moving a file via ftp and then via ttcp. Sure, you don't always get all the file, but it does get there fast! --D On Thu, Jun 12, 2008 at 5:02 PM, Randy Bush <randy@psg.com> wrote:
The idea is to use tuned proxies that are close to the source and destination and are optimized for the delay. Local systems can move data through them without dealing with the need to tune for the delay-bandwidth product. Note that this "man in the middle" may not play well with many security controls which deliberately try to prevent it, so you still may need some adjustments.
and for those of us who are addicted to simple rsync, or whatever over ssh, you should be aware of the really bad openssh windowing issue.
randy
-- -- Darren Bolding -- -- darren@bolding.org --