- Test - lists.nanog.org

Web Cache Question
by sonick＠mediaone.net 29 Jun '98

29 Jun '98

This is purely an effort to discover the cause of a recent problem that has surfaced inside our organization. About the time the whole web cacheing/digex issue broke here, the organization I work for began experiencing unstability in IIS4 web servers that provide real time market data via the internet to customers all over the world. These servers show an incredible increased rate in FIN_WAIT connections, compared against the amount they used to show on average (literally one week ago.) It is a problem because the inetinfo process simply stops responding. My question is this: is it possible there are web cacheing servers still in-line with major transit providers that we don't know about? If these servers exist: do they all operate on the premise of spoofing the source address as coming from the client, or would I actually see the cacheing servers source address? At this point, the only evident proxies in a server netstat are from AOL, but this is nothing new. The real increase in unexplained stale sockets are from random source addresses (spoofed?). Again, this mail is solely for the purpose of troubleshooting an exasperating, unexplained problem, not a blame the cache mail, so no return spam please. Any thoughts or recent similar experience sharing would be greatly appreciated. Jayson Anderson An Organization

2 1

on the issue of proxy-caching and new technologies
by alan＠mindvision.com 29 Jun '98

29 Jun '98

I've just read through the 20 or so email messages in the thread regarding the use of proxy-cache traffic on http traffic. I believe DIGEX should be congratulated on their forward exploration of technology. Their creative solutions to problems like this are the spirit that will sustain the Internet. DWDM and huge fiber builds will get us part of the way to handling internet growth, but we need to find ways to utilize upper-layer tweaks to do more with less. -alan

3 2

Re: Digex transparent proxying
by Paul Vixie 29 Jun '98

29 Jun '98

> >(performance for the FIRST fetch through a proxy is SLOWER - it HAS TO BE, > >since the proxy must first get the data before it can pass it on). > > It doesn't have to -- think cut-thru switching. I've no idea if > any/enough of the proxies do this. Yes, some do. And I wanted to give one thing Sean said more air time so folks won't just gloss over it: > An intercepting proxy which runs a modern TCP stack and which > avoids the "herds of mice" problem by aggregating multiple > parallel connections into single ones, and which is well-located > to avoid frequent fifo tail-drop at the last hop, has a benefit > to the ISP that outweighs the cache hit:miss ratio. > > That is, a cache which imposes decent long-haul TCP behaviour > reduces the number of packets which are delivered all the way > from the web server to the terminal server but tail-dropped there > rather than being delivered to the end user. This is rather important, both because the stacks used in last-mile devices (such as the Microsoft Program Loader) are not very modern, and because HTTP persistent connections end up not being very helpful until they are aggregated across multiple last-mile devices. And in these senses (cut through with segment reblocking; reasonable/modern TCP handling; more use of persistent HTTP), someone with tcpdump actually would be able to tell that my particular transparent caching box was in use. I'll leave the challenge to Karl open, though, since I meant specifically "be able to tell from the client or server end" not "be able to tell using tcpdump on a LAN wire upstream of the caching box". -- Paul Vixie La Honda, CA "Many NANOG members have been around <paul(a)vix.com> longer than most." --Jim Fleming pacbell!vixie!paul (An H.323 GateKeeper for the IPv8 Network)

2 1

Re: Digex transparent proxying
by Sean M. Doran 29 Jun '98

29 Jun '98

| some of the content providers in the US would turn over in their graves if | they knew what people who pay heaps more $$$ for traffic are doing to | their web sites :). Let them turn. The easy fix for them is to duplicate their content elsewhere in the world, so they can control it themselves, and reduce the need for caching by operators like you. Sean.

1 0

Greener grass on the other side of the fence
by Sean Donelan 29 Jun '98

29 Jun '98

>As an aside, it always amazes me that the Internet even works (particularly >for an infrastructure that the unwashed masses seems to think will replace >the existing telephone network), but I guess the same could be said of any >chaotic (in the non-linear sense) system... I'm not sure the telephone network is really that robust. Think about the SS7 network and a world with 50,000 CLEC's. Like most things, you don't want to really know what's in the sausage. Some systems are designed with a high degree of control, so everything works correctly. But they often have the property of catastrophic failure when any part fails to work perfectly, such as the cascading failures of the western power grid last year. Central, hierarchial control is alluring. But there are still trade-offs. It used to take a week (or more) to add/change/update network routes in the prdb (and its predecessors). Perhaps acceptable when the central control was working, but it also took that long to fix things when the central database was screwed up. If you were really lucky, you might be able to convince someone to do an 'emergency' change in a day or two. We (myself included) often express amazament how the network manages to continue functioning. However, the system might not be as out-of-control as some people think. It is definitely chaotic. On the other hand, it has shown itself remarkably resiliant to the many evil things people and nature have done. I'll note I'm writing this note while watching white scan lines roll over my highly reliable TCI cable service, NOT! -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

3 2

Digex transparent proxying
by Jon Lewis 29 Jun '98

29 Jun '98

I went searching through my email (since Digex support claimed customers had been notified of Digex's intent to start transparent proxying) and found that I did get "a message" that I'd missed while out of town for Linux Expo. Here's the message: --- Date: Fri, 29 May 1998 19:37:03 -0400 From: DIGEX Customer Support <support(a)digex.net> To: DIGEX Customer Support <support(a)digex.net> Subject: DIGEX Service Enhancement Notification: 06/02/1998 0400 - 0800 EDT Service Enhancement: 06/02/1998 0400-0800 EDT Location: DCA1 POP, Washington, DC During a configuration window between 4am and 8am on 06/02/1998, DIGEX will be upgrading the dca1-cpe router to which your leased line is connected. These upgrades will increase the speed at which you retrieve web pages, as well as improve your web surfing experience. The downtime will be minimal, 15 minutes at the most. We apologize for any inconvenience this scheduled maintenance will cause, and trust the changes to be made will improve your Internet connectivity. Please feel free to contact the DIGEX Customer Service Team with any questions concerning this maintenance, referencing trouble ticket TT-LL0025428. --- That's pretty darn cryptic. "We'll be upgrading the router you connect to...to increase the spead at which you retrieve pages and surf". What does that mean? More memory for the router? A faster CPU module? Why didn't they just come out and say "We'll be reconfiguring the router to which you connect to do policy based routing so we can hijack all your web traffic and run it through a web cache...we'd rather blow our $ on marketing than continue to invest in the infrastructure required to support our leased line sales." Digex is seriously missing the point that not all customers are an office of drones surfing the web all day. We (FDT) use Digex for transit. Several ISPs use FDT for their internet connectivity. By forcing all our traffic through a web cache, they've seriously impacted the service we provide to our customers and the service our customers provide to their customers. Many hours have been wasted at FDT and by FDT's customers debugging problems that I'd originally written off as "user BS" as the description of the problem was just too off the wall to believe. I think we have at least one customer interested in the possibility of legal action against Digex. ------------------------------------------------------------------ Jon Lewis <jlewis(a)fdt.net> | Spammers will be winnuked or Network Administrator | drawn and quartered...whichever Florida Digital Turnpike | is more convenient. ______http://inorganic5.fdt.net/~jlewis/pgp for PGP public key____

4 3

backbone transparent proxy / connection hijacking
by Jon Lewis 29 Jun '98

29 Jun '98

Has anyone else noticed Digex playing with transparent proxying on their backbone? We have one of our T1's through them, and found that all web traffic going out our Digex connection goes through a proxy. We've got customers with web sites that are broken now because they can't communicate with things like Cybercash, because their outgoing http requests are hijacked and sent through a Digex web cache. Digex wants us to register each web server out on the rest of the internet that hosts from our network need to talk directly to. This looks like the beginning of a big PITA. I wouldn't have a problem with Digex setting up some web caches and encouraging customers to setup their own caches and have them talk to the Digex ones via ICP...but caching everything without our knowledge/consent stinks. ------------------------------------------------------------------ Jon Lewis <jlewis(a)fdt.net> | Spammers will be winnuked or Network Administrator | drawn and quartered...whichever Florida Digital Turnpike | is more convenient. ______http://inorganic5.fdt.net/~jlewis/pgp for PGP public key____

15 17

Re: Digex transparent proxying
by Sean M. Doran 29 Jun '98

29 Jun '98

Patrick Gilmore writes: | Which is why I've asked | them to give me some input. Lots of people here have thought about this a | lot more than I have, perhaps they would care to share their thoughts about | this issue? It's nice that you want to learn, but you have the habit of arguing every little point vigorously more often than asking questions like this one, and frankly, that tends to lead people to be disinclined to try to help you come to your own conclusions, much as was the case with Tim Bass, who behaved similarly. Sean.

2 1

Re: Digex transparent proxying
by Sean M. Doran 29 Jun '98

29 Jun '98

Patrick Gilmore wrote: > I find it repugnant that one of my peers has done so. On an operational note, it seems that Priori has just indicated that it wishes to terminate its peering with Digex. I don't suppose anyone knows which way the packets between those two networks will flow once this principled demonstration of revulsion has been implemented? Sean. (prepared to be told by Randy Bush to contact the DIGEX NOC since Ebone is a customer of Digex's in cooperation with ACONET; not that it matters, since the ASes behind Priori are probably not very crucial)

2 1

Re: Digex transparent proxying
by Sean M. Doran 29 Jun '98

29 Jun '98

Cool, Karl. Then you would be denying your customer access to a market -- a potentially large market -- without telling them. Even only indirectly countable "views" are good for brand managers and marketers, hence the popularity of such things as outdoor advertising. Ironic. Where can they sue? Sean.

8 11