- Test - lists.nanog.org

running summary on caching
by Jeremy Porter 30 Jun '98

30 Jun '98

Here is my running summary of this debate, with only mild editorizing: >From: alan(a)mindvision.com (Alan Hannan) I believe DIGEX should be congratulated on their forward exploration of technology. Their creative solutions to problems like this are the spirit that will sustain the Internet. DWDM and huge fiber builds will get us part of the way to handling internet growth, but we need to find ways to utilize upper-layer tweaks to do more with less. --- >From: andrew khoo <andrewk(a)aussie.net> more aggressive caching techniques would necessitate that the tags be ignored anyway, and the "dynamic" content would still be cached. expires etc are only useful if the caching box decides to honour them. some of the content providers in the US would turn over in their graves if they knew what people who pay heaps more $$$ for traffic are doing to their web sites :). we have done some funky stuff with content (read: caching of dynamic pages, even .asps, "tokenized" HTML, dropping realaudio UDP etc etc etc). --- (I'm stilling waiting for the first successful copyright lawsuit in Australia because the cache isn't honoring expire headers or modifying the data) >From: "Sean M. Doran" <smd(a)clock.org> Let them turn. The easy fix for them is to duplicate their content elsewhere in the world, so they can control it themselves, and reduce the need for caching by operators like you. Sean. >From: Paul Vixie <vixie(a)vix.com> And I wanted to give one thing Sean said more air time so folks won't just gloss over it: ... > > That is, a cache which imposes decent long-haul TCP behaviour > reduces the number of packets which are delivered all the way > from the web server to the terminal server but tail-dropped there > rather than being delivered to the end user. This is rather important, both because the stacks used in last-mile devices (such as the Microsoft Program Loader) are not very modern, and because HTTP persistent connections end up not being very helpful until they are aggregated across multiple last-mile devices. --- >From: Patrick McManus <mcmanus(a)AppliedTheory.com> In a previous episode Michael Dillon said... :: [...] Caching is a temporary hack that provides some limited :: benefit at the moment but the need for it is fading except in areas that :: are a couple of years behind in bandwidth deployment. I can't believe you said that. Heirarchical caching reduces latency and increases availability _in addition to_ conserving bandwidth. Those (particularly the second) will remain critically important features no matter if you're traversing a 512k line or OC48... --- >From: Pushpendra Mohta <pushp(a)CERF.NET> No amount of capacity upgrades will obviate the need for an intelligent architecture for content distribution on the net. Utilization will explode to consume all bandwidth available and in the absence of an elegant architecture will do so rather wastefully. Most sites on the web are currently misdeployed/incorrectly-located on the network. Techniques that prevents unwarranted replication of data and bring the content to the user rather than the user to the content will bring about more efficient use of bandwidth while increasing performance for all users. These need to be complemented with techniques to address the concerns of other constituents, like marketeers. --pushpendra --- >From: Jon Lewis <jlewis(a)inorganic5.fdt.net> Actually, I thought I was the loud one complaining. I'm not a Digex competitor either...I'm the Digex customer that started this thread. --- (Your comments did not compare in volume or tone to some made by Digex's competitors, it would seem a good number of people felt you have a reasonable complaint about notfication but were to polite to rummage through Digex's dirty laundry in this forum.) >From: John Fraizer <John.Fraizer(a)EnterZone.Net> Horse-hockey. Purchasing the proper amount of bandwidth from your upstream and your upstream implementing the facilities to deliver that bandwidth is what is necessary to scale the internet into the future. Over-allocation is bad enough on the dialup-ISP side of the equation without backbone providers doing the same thing. If your backbone provider doesn't have the capacity to deliver what they have sole to you, it's time to contact two entities: Your lawyer and another backbone provider. --- (Question: what off the shelf technology goes beyond DWDM OC192? Second Question: When will that bandwidth be needed?) >From: Pete Farmer <pfarmer(a)strategies-u.com> As a customer, I don't give a hoot for belief systems and opinions on where things 'ought' to be. Don't talk to me about 'unethical,' talk to me about better ways to solve my business problem. --- >From: "Sean M. Doran" <smd(a)clock.org> Yes, and I am sure they fully realize that the loudest noises have been made by DIGEX's *competitors* complaining about DIGEX's customer-notification or service change, and consume a large block of salt. Sean. --- >From: "Patrick W. Gilmore" <patrick(a)priori.net> And before anyone goes off once again about how great caching is, I will once again publicly state that I am not at all opposed to caching - even forced caching of customers. It's your network, do as you please. ---(comments relating the the business pratices of Digex and its relations with its customers are not Operational issues and have been deleted. They might explain why everyone is bashing Patrick) >From: Karl Denninger <karl(a)mcs.net> I have no problem with a proxy where the user has agreed to its use. I have a major problem with a provider hijacking a stream of data and reprocessing it, then delivering what the client *thinks* is a direct communication session. In fact, I'm not even sure that such an act, absent consent, is legal. --- (This is my favorite, Karl makes all assumptions here about jurisdiction, contracts between two parties, etc. Karl is a competitor of Digex also. I still have this funny image of the Chicago policy showing up in Copenhagen, Denmark, to arrest a Canadian.) >From: Mark Milhollan <mlm(a)ftel.net> Karl Denninger writes: >(performance for the FIRST fetch through a proxy >is SLOWER - it HAS TO BE, since the proxy must first get the data before it >can pass it on). It doesn't have to -- think cut-thru switching. I've no idea if any/enough of the proxies do this. --- (they do, and Karl should have known better) >From: Hank Nussbacher <hank(a)ibm.net.il> For the past 4 years, AS378 has blocked port 80 (all academic network of 8 universities and many colleges) and forced all users thru multiple proxy servers (most recently Squid). Over the years we had to build up a list of sites to bypass. But in addition, one has to provide the ability to bypass based on source IP address (some users just don't want it - even if it speeds up their page downloads and toasts their bread at the same time.) --- (In the US people don't pay extra for un filtered access because it is so cheap. Probably don't want caching because they don't want people to know what type of porn they are downloading. (And don't understand that tcpdump could still monitor them.) So in New Zeeland, Australia, or other places where they pay extra for not going through the cache, its ok to do this, but in the US where the cost is carried by the provider its WRONG. Oh ok, end users have a common law right to dictate the technology used to deliver services to them. I'll remember that when I buy my next car without an emissions control system, because I have the fundemental right to pollute.)) >From: Karl Denninger <karl(a)mcs.net> See, that's the problem. Proxies are fine WHERE CUSTOMERS HAVE AGREED TO THEIR USE. STEALING someone's packet flow to force it through a proxy is NOT fine. --- (This same arguement could be apply to Ip deliver as specified in RFC 1149 "Standard for the transmission of IP datagrams on avian carriers." Data is data if the transparency is working correctly, the data delivered is identical and delivered faster. The streams aren't being taken anywhere. This is like arguing that echo cancelers can't be used on satilite communications because it breaks some types of transmissions, but helps the majory of customers, and it does in fact modify the STREAM.) So it seems: Against transparent proxying: Karl D. Against Digex transparent proxing: Patrick G. Upset about the problems with transparency not working: Jon L. Against transparency (for operational reasons even!) Mike O'Dell Thinks that bandwidth grows on trees: John F Pro hiarcheial caching, even transparency: Pushpendra M Hank N Patrick M Paul V Sean Doran andrew k Alan H Paul G Rich S Lamenting the lack of a Digex NOC to report transparency problems to: Randy B --- All well. I'd guess that content providers had better start working with cache vendors to make things work right, as I would guess a good bit of the content is going to go through caches, if you like it or not and everyone is going to be better off with it. There should be a real good flame war when people role out CoS/Voice over IP/ RSVP to exist next to best effort delivery. (My contract with provider X clearly implies that my traffic should be treated with equal priority to all other traffic on the network, therefore you are in breach of contract for offering higher priorities than best effort, oh and BTW you are doing unlawful wiretapping because you are classifying traffic by type which means looking past the IP address in the header.) --- Jeremy Porter, Freeside Communications, Inc. jerry(a)fc.net PO BOX 80315 Austin, Tx 78708 | 512-458-9810 http://www.fc.net

4 3

Host for NANOG 14 & modified NANOG Business Model
by William B. Norton 30 Jun '98

30 Jun '98

Hi all - As the subject line indicates, we are looking for a local host for the next NANOG. The tradional role of the local host has been to take care of all things local (facilities, hotel, catering, Audio/Video, Internet Connectivity, etc.), allowing the NANOG staff (pam, srh and myself) to concentrate on the meeting logistics, agenda, speakers, etc. The group has grown to the size where it is no longer reasonable for the local host to take care of the local logistics and bear the weight of the financial burden of feeding everyone. As I mentioned during the opening of NANOG 13, the catering expenses for 500+ is many tens of thousands of dollars. After many conversations with folks at the NANOG 13 meeting, we have come up with a modified funding model for NANOG that will increase the NANOG registration fee to cover more of the hosting expenses. We expect the new registration fee will increase from $150 (which covers the percentages of the 3 NANOG staff) to between $250 and $300 which will cover all catering and A/V costs. This will reduce the local host's financial burden substantially and return the local host role to that of providing local logistics support, the terminal room, and Internet connectivity. We have also been fortunate enough to have companies loan us technology for the event. For example, Bay has loaned routers and hubs and Cabletron has loaned wireless equipment that folks liked so well. Many others have contributed too. The point is, we expect this type of support to continue, so the load on the local host will similarly be reduced by these generous community contributions. We hope that this will encourage more types of organizations to volunteer to host NANOG 14. We'll be modifying the "Tips for hosting a NANOG" web page accordingly. So, if you are interested in hosting NANOG 14, don't be discouraged by the cost figures from past NANOGs. The local hosting costs will be substantially reduced. Please send e-mail to wbn(a)merit.edu. What I will need is an indication of proposed dates in or around October, a hotel or indication of facilities that you have looked into, and how you would plan to provide Internet access to the facilities. Multiple organizations working together is fine. Thanks - Bill -------------------------------------------------------------- William B. Norton <wbn(a)merit.edu> (734) 764-9430 (New Area Code)

1 0

Re: Digex transparent proxying
by Paul Vixie 30 Jun '98

30 Jun '98

> Or detect a proxy and refuse service (which, if I was doing this, is exactly > what I would do). "Go ahead, make my day." If you can detect the proxy box I used to sell via MII, and refuse service to it, I will post a retraction right here. > Not to mention those to derive advertising revenue from "views", and have no > way to measure them in a cached environment. "No way"? How about: rfc2227.txt -- Simple Hit-Metering and Usage-Limiting for HTTP. J. Mogul, P. Leach. October 1997. (Format: TXT=85127 bytes) (Status: PROPOSED STANDARD) > If I was doing that, I would also deny service to proxy servers and > display a nice message telling the user to remove the proxy or bitch about > its forced use. Playboy.COM did something like that to @Home last year for a similar reason. -- Paul Vixie La Honda, CA "Many NANOG members have been around <paul(a)vix.com> longer than most." --Jim Fleming pacbell!vixie!paul (An H.323 GateKeeper for the IPv8 Network)

2 1

Re: backbone transparent proxy / connection hijacking
by Eric Wieling 30 Jun '98

30 Jun '98

This is just my take on the issue. @Home caches. AOL caches. Browsers cache. (does the Direct/PC service cache?), many, many ISP's cache. Many companies cache. Caching is a fact of life. From my point of view any advertiser that does not take caching into account is screwing themselves and their customers. We cache. I try to configure our cache to err on the side of fresh content, rather than err on the side of hit rates. Every once in a while we get a customer complaint about a specific site and we address the issue on a case by case basis. So far complaints are fairly uncommon. We are not a backbone provider. Our cache is only available to customers that subscribe to our "security" services. (Just a note most of our customers have less then 300 users and although our firewall services are not air tight, I think we do a decent job of protecting our customers. If these customers did not have this service from us, most of them would have no protection at all. I'm just amazed at how much smaller business just don't seem to care about network security.) Making your site cache friendly is not rocket science. Putting reasonable expires information on objects is not tough. Telling your web server to set the expires as a percentage of the age of the document is supported in some servers. Using client side forms validation is not really that much tougher than using server side validation. Many SSI web pages are not really dynamic. We use SSI extensively on our web sites to create a consistent look and feel. Of course, some SSI pages are dynamic and should not be cached. I do think that it's utterly unacceptable for a backbone provider to force their customers to use their cache. I do, however, wish that more backbone providers would provide caching services to those people what want the service. --Eric Wieling >>> John Fraizer <John.Fraizer(a)EnterZone.Net> 6/29/98 6:20:31 PM >>> <snip> So, If someone is using site exec, etc in their code, and their provider/webmaster/mother didn't set up Progma: nocache, they're effectively screwed...erm...cached, right? <snip> Fantastic. So, lets say I'm Joe Banner Advertizer. Company X has paid me present their banner. They wanted to limit the amount of money they spent so, they had me code my servers to only display their banner X times per day since I bill them on impressions. Backbone provider Z installs one of your boxes. By default, no matter how many connections on the limited....erm...client side of the box are initiated to retrieve a "fresh" banner from our banner-farm, you send them Company X until the cache times out.

1 0

Re: small vent
by Tim Salo 30 Jun '98

30 Jun '98

> Date: Sat, 27 Jun 1998 23:55:20 +0800 > To: brian(a)meganet.net > From: "David R. Conrad" <davidc(a)apnic.net> > Subject: Re: small vent > > One reason might be that clueless folk in the US that complain to APNIC or > RIPE (or NANOG) directly instead of looking stuff up in the appropriate > whois database ("ARIN has all the IP information, right?") and complaining > to the people who can actually do something about the problem. > [...] Having to specify the proper database host on the whois command always struck me a bit like requiring every user to specify the appropriate server for, e.g. ".com", every time the user makes a DNS request. Why can't the registry folks get their collective act together and provide users with a system that allows a simple request like "whois name" to find the appropriate [piece] of the database. -tjs

8 10

info
by Stew L. Piggien 30 Jun '98

30 Jun '98

In case anyone needs it, here's rolex's real info: Name Raymond Torricelli Address 926 Webster Ave New Rochelle, NY 10804-3533 Phone num (914) 235-5245 Alt: (914) 576-7859 ______________________________________________________ Get Your Private, Free Email at http://www.hotmail.com

1 0

RE: backbone transparent proxy / connection hijacking
by Pete Farmer 30 Jun '98

30 Jun '98

On Monday, June 29, 1998 John Fraizer [mailto:John.Fraizer@EnterZone.Net] wrote: >Backbone providers are here to provide unhindered >connection to the internet for their clients. Is that necessarily what the customer is after, or is that a polemic? <customer_view> I think my backbone provider is there to provide me with access to Internet content, delivering the mix of throughput, latency, and price that's right for me. And I can see where caching *might* allow a service provider to better meet my needs. </customer_view> >Hijacking your clients, their clients, etc connections >and shoving them through a proxy cache to save money is >unethical. > >Transparent proxys have one place. On the CLIENT side of the connection. As a customer, I don't give a hoot for belief systems and opinions on where things 'ought' to be. Don't talk to me about 'unethical,' talk to me about better ways to solve my business problem. Pete ______________________________________________________________________ Peter J. Farmer E-mail: pfarmer(a)strategies-u.com Strategies Unlimited Voice: +1 650 941 3438 201 San Antonio Circle, Suite 205 Fax: +1 650 941 5120 Mountain View, CA 94040 WWW: http://www.strategies-u.com

1 0

Re: backbone transparent proxy / connection hijacking
by John Fraizer 29 Jun '98

29 Jun '98

At 04:31 PM 6/26/98 -0700, you wrote: >performance for non-HTTP protocols. Many people believe that >large-scale caching is necessary and inevitable in order to scale >the Internet into the future. Horse-hockey. Purchasing the proper amount of bandwidth from your upstream and your upstream implementing the facilities to deliver that bandwidth is what is necessary to scale the internet into the future. Over-allocation is bad enough on the dialup-ISP side of the equation without backbone providers doing the same thing. If your backbone provider doesn't have the capacity to deliver what they have sole to you, it's time to contact two entities: Your lawyer and another backbone provider. >The Cybercash server performed client authentication based on >the IP address of the TCP connection. Placing a proxy (transparent >or otherwise) in between clients and that server will break >that authentication model. The fix was to simply configure Traffic >Server to pass Cybercash traffic onwards without any attempt to >proxy or cache the content. With web commerce companies popping up all over, who wants to be the one to track these companies and their netblocks and reconfigure the proxy every time one comes online or changes providers? This would be a nightmare for the dialup provider, let alone their provider or their providers provider. If we purchase transit from provider X because provider Y doesn't peer with provider Z from whom Commerce Provider Alpha gets their connection, who do I yell at when my customers yell at me? Sounds like a huge finger-pointing match to me. Like we don't already encounter too many of those? > >The second example was of a broken keepalive implementation in >an extremely early Netscape proxy cache. The Netscape proxy >falsely propagated some proxy-keepalive protocol pieces, even >though it was not able to support it. The fix was to configure >Traffic Server to not support keepalive connections from that >client. Afterwards, there were no further problems. See above. Backbone providers are here to provide unhindered connection to the internet for their clients. If someone wants to set up a proxy, it's their business. Publish the address of the proxy and have at it. Hijacking your clients, their clients, etc connections and shoving them through a proxy cache to save money is unethical. Transparent proxys have one place. On the CLIENT side of the connection. >These two problems are examples of legacy issues. IP-based >authentication is widely known to be a weak security measure. IP based authentication, alone, may be somewhat weaker than we would like. It is however much more secure than having one central point on Backbone Z that all web based traffic goes through. IP based authentication is also very nice augmentation for challenge/response and even simple username/password schemes. It limits the usefulness of authentication tokens that fall into the hands of those who do not possess the skill-sets necessary to do blind attacks. >The Netscape server in question was years old. As time goes >on, there will be a diminishing list of such anomalies to deal >with. This is like saying, "Internal combustion automobiles are outmoded with the advent of plentiful gerbils to power the vehicle on treadmills." Haven't we all learned the backward compatibility lesson? >The second issue being discussed is the correctness of cached >content. Posters have suggested mass boycotting of caching by >content providers concerned with the freshness of their content. >Most content providers have no such concerns, frankly. The problem >of dealing with cached content is well understood by publishers >since caching has been in heavy use for years. Every web browser >has a cache in it. AOL has been caching the lion's share of And we all know that AOL is the #1 choice of the uninitiated, a proving ground until they graduate to a SERVICE PROVIDER vs. a Content Provider. >US home web surfers for years. For more information on the ways >in which publishers benefit from caching see our white paper >on the subject of caching dynamic and advertising content: >http://www.inktomi.com/products/traffic/tech/ads.html "Dynamic content heuristics In the event a publisher has not incorporated cache-control headers into their site, Traffic Server uses a number of heuristics to recognize dynamic Web pages (for example, pages generated using cgi-bin or form submissions). These pages are automatically refreshed on every access." So, If someone is using site exec, etc in their code, and their provider/webmaster/mother didn't set up Progma: nocache, they're effectively screwed...erm...cached, right? "Refresh algorithms For pages not utilizing cache-control headers, Traffic Server ships with a default refresh cycle of one hour for all pages. In addition, the cache uses sophisticated algorithms to selectively update rapidly changing pages at a faster rate. These updates are accomplished using an "If-Modified-Since" request, which saves bandwidth if the page has not changed. " Fantastic. So, lets say I'm Joe Banner Advertizer. Company X has paid me present their banner. They wanted to limit the amount of money they spent so, they had me code my servers to only display their banner X times per day since I bill them on impressions. Backbone provider Z installs one of your boxes. By default, no matter how many connections on the limited....erm...client side of the box are initiated to retrieve a "fresh" banner from our banner-farm, you send them Company X until the cache times out. "Users see improved performance because static graphics and applets are delivered from nearby caches, insulating them from congestion in the Internet cloud." Let me rebut this quote with yet another quote from you: "Every web browser has a cache in it." I'll bet on the speed of the slowest LOCAL cache against one at the provider any day. >And finally, there has been confusion concerning the >confidentiality and legal issues of transparent caching. >Transparent caching does not present any new threat to the >confidentiality of data or usage patterns. All of these issues >are already present in abundance in the absence of caching. Oh really? Lets say that I'm Joe Internet Terrorist. (Hey, they thought that the mailman was harmless too!) I work at backbone provider Z. I want to create panic on a global scale. I insert a machine on the business end of your proxy/cache that claims to be some corporate giant who has 80% share of the desktop market. Now, any requests that your cache makes are served up by my "local" box and not the REAL webserver. Just to make things interesting, I tell the webserver to serve up ONE page no matter what the request is for. On that page, we put something to the effect of "We have declared bankruptcy. We will no longer support any product we have sold you. You're screwed." You're saying "It would be just as easy and probably more likely that someone would put that box at Company X instead of at the proxy/cache." Not if they also wanted to spoof the webservers of the FBI, CIA, etc. Now, we have the entire internet community served by carrier Z convinced that they're screwed. "I couldn't resist Your Honor. It was just too easy. It was better than broadcasting an alien invasion on the radio!" ------- John Fraizer (root) | __ _ | The System Administrator | / / (_)__ __ ____ __ | The choice mailto:root@EnterZone.Net | / /__/ / _ \/ // /\ \/ / | of a GNU http://www.EnterZone.Net/ | /____/_/_//_/\_,_/ /_/\_\ | Generation A 486 is a terrible thing to waste...

1 0

Re: Digex transparent proxying
by Sean M. Doran 29 Jun '98

29 Jun '98

Patrick Gilmore writes: | As for the average NANOG reader, I would have to agree with you that they | are capable of thinking for themselves. Yes, and I am sure they fully realize that the loudest noises have been made by DIGEX's *competitors* complaining about DIGEX's customer-notification or service change, and consume a large block of salt. Sean.

2 1

Re: Filtering Unregistered Blocks (WAS: small vent)
by dlr＠bungi.com 29 Jun '98

29 Jun '98

[In the message entitled "Filtering Unregistered Blocks (WAS: small vent)" on Jun 29, 11:46, "Patrick W. Gilmore" writes:] > At 12:45 PM 6/28/98 PDT, Dave Rand wrote: > > >This has, in fact, happened before. One of the reasons that the unallocated > >spaces are listed on the RBL. > > This topic comes up every now and then. I've searched the ARIN site and > found some very outdated lists (some as old as 1996, but none newer than > Feb, 1998). I've searched the archives, but no one seems to have an answer > - except you. How do you tell what is and what is not allocated? Unallocated is (once again) a state of mind. By unallocated, the RBL looks upon the IANA allocation of space, not the ARIN view. So, for example, 2.0.0.0/8 is on the RBL (as is 0.0.0.0/8). We watch for IANA allocation of new blocks, and when they are allocated, remove them from the RBL. Of course, this takes more work, and requires that we watch closely. As an example, AS3259 is currently advertising a bogon. *>i213.102.1.0 207.240.113.165 10 100 0 3847 3259 3259 i > > Perhaps you could export just the non-assigned blocks in the RBL? Maybe > under a second AS or something? I have no problem with the RBL myself, but > some of my customers want a "full table", and won't take an RBL filtered > one. That's not how the RBL works in BGP mode. You can (easily) accept a feed from the RBL, and filter to accept only /8's from the RBL, which will give you the desired information. You are not permitted to redistribute AS7777 routes to your customers (or anyone, for that matter) without express approval and a disclaimer in place, so AS7777 is typically marked as a "no-export" community. The RBL, in BGP mode, is used by route-mapping the addresses listed on the RBL to a specific address. You can, for example, route all traffic to RBL listed hosts to go through a 9600 bps dialup port. Or you can route them to a T1. Or you can route them to the loopback port, which is what most people do. The RBL doesn't filter the BGP table, at all. > I do, however, filter blocks which should not be routed - e.g RFC > 1918 - and would like to include the non-assigned blocks. I just can't > figure out a way to automate such a process. Especially since the whois > servers limit the number of queries. (I'm not complaining, I understand > the reasons, I'm just stating a fact.) You can't automate it, easily. But by using the RBL, you can certainly get the real-time aspect of it handled well. -- Dave Rand dlr(a)bungi.com http://www.bungi.com

2 1