- NANOG - lists.nanog.org

Re: backbone transparent proxy / connection hijacking
by Joe Provo - Network Architect 01 Jul '98

01 Jul '98

[snip] > I would like to "form" a developer group which forms htcp > into squid allowing "more people" to use a new scheme for that, instead > of using icp for the neighbor queries. I started to look at the code > and will be ready to start in about 1-2 weeks. I would hope you're looking at the 1.2 beta code, with htcp hooks, etc in place (just nothing operational). Anyone else interested should get involved in the apropos squid mailing lists. (squid-dev@nlanr, squid-users@ircache, etc). Joe -- Joe Provo, Network Architect 508.229.8400 x3006 Commercial Internet Services Group Fax 508.229.2375 UltraNet Communications, Inc., an RCN Company <jprovo(a)ultra.net>

1 0

Host for NANOG 14 & modified NANOG Business Model (fwd)
by jamie＠dilbert.ais.net 01 Jul '98

01 Jul '98

Speaking for myself, nobody else: >From: "William B. Norton" <wbn(a)merit.edu> >We expect the new registration fee will increase from $150 (which covers >the percentages of the 3 NANOG staff) to between $250 and $300 which will >cover all catering and A/V costs. This will reduce the local host's >financial burden substantially and return the local host role to that of >providing local logistics support, the terminal room, and Internet >connectivity. Uhm. My aunt runs a catering business. *Good* weddings are $40 a plate. Two days, $80. Average three snickers bars ($1 each) and 6 cans of pop ($2) and you have a per-person food cost of $85. Shirt $15, youre at $100. This would mean, at 500 participants @ $250, the A/V cost is roughly $125,000.00 for two days? Is ARIN running the A/V dept now? *hides* (Kim we love ya) >We have also been fortunate enough to have companies loan us technology for >the event. For example, Bay has loaned routers and hubs and Cabletron has >loaned wireless equipment that folks liked so well. Many others have >contributed too. The point is, we expect this type of support to continue, >so the load on the local host will similarly be reduced by these generous >community contributions. Registration costs just went from $100 to $150, now they're going to $250? With a 250% increase in costs over less than a year, I'd like to see where this money goes.. or even better, costs of previous NANOGs. Not that I don't believe it, I just wonder. -- jamie rishaw (dal/efnet:gavroche) American Information Systems, Inc. Tel:312.425.7140, FAX:312.425.7240 "Could you only justify a Pepsi, but took a Coke anyway?" --> -cjw(a)corp.home.net to kimh(a)arin.net, after kimh took the last Coke at NANOG 13

4 6

[ops] ASN10856
by Rob M VanHooren 01 Jul '98

01 Jul '98

Well, got the /19 for ASN10856 today. I feel special. <groan/grin> -R. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-.._..-*~ Rob M. VanHooren - Chief Plumber, +1 519 679-1155 x33 Packet Pusher, and Resident Mad Scientist(tm) Alarms to 646-4724 Network Engineering Services 171 Queens Avenue, Suite 320 Linkdata Communications Inc. London, CANADA

1 0

Re: backbone transparent proxy / connection hijacking
by Paul Vixie 30 Jun '98

30 Jun '98

> I do think that it's utterly unacceptable for a backbone provider to > force their customers to use their cache. I do, however, wish that > more backbone providers would provide caching services to those people > what want the service. Most backbone providers run Squid in their datacenters and/or POPs and offer to do ICP with any customer who wants it. I don't like ICP -- see http://www.vix.com/ietf/htcp.txt for the protocol I proposed to replace it. But the model is sound, and I would like to see more backbone providers doing this. > So, If someone is using site exec, etc in their code, and their > provider/webmaster/mother didn't set up Progma: nocache, they're > effectively screwed...erm...cached, right? No. > Fantastic. So, lets say I'm Joe Banner Advertizer. Company X has paid me > present their banner. They wanted to limit the amount of money they spent > so, they had me code my servers to only display their banner X times per > day since I bill them on impressions. Backbone provider Z installs one of > your boxes. By default, no matter how many connections on the limited.. > ..erm.. client side of the box are initiated to retrieve a "fresh" > banner from our banner-farm, you send them Company X until the cache > times out. No. For now, use freshness lifetimes (including pre-expiry for banners) and correctly behaving caches will at worst do a GET/If-Modified-Since whenever they are considering reusing the element -- so your origin server can count the hits and can control when the object can no longer be reused. The HTTP standard already supports this. It costs a TCP transaction per reuse, but it avoids the actual transmission of the banner ad whenever reuse is correct. In the near future, we'll see a different reuse model, based on RFC 2227: rfc2227.txt -- Simple Hit-Metering and Usage-Limiting for HTTP. J. Mogul, P. Leach. October 1997. (Format: TXT=85127 bytes) (Status: PROPOSED STANDARD) And again, the advertisers will be in full control, the standards will get followed, and the backbone will have more bits free for Internet Telephony. -- Paul Vixie La Honda, CA "Many NANOG members have been around <paul(a)vix.com> longer than most." --Jim Fleming pacbell!vixie!paul (An H.323 GateKeeper for the IPv8 Network)

2 1

Re: Digex transparent proxying
by Chris A. Icide 30 Jun '98

30 Jun '98

[NANOG WARNING: This document contains neither directions on contacting the operational support group for any entity, nor the configuration code for any network hardware] Against my better judgement, I've decided to add my tiny bit of content to this thread. If there is anyone to thank for getting me into it, Mr. Porter has done so with his excellent review of the thread. Without it, I might not have seen the humor in it. However, there is something that I just can't understand. It's amazing to me that with the number of examples in history, people are able to make grand statements like many have in this thread. Please allow me to point out a few historical statements that I feel are very similar to some of the grand statements made in this thread: 1. The world is flat and rests on the back of a giant turtle. 2. Man will never be able to fly faster than the speed of sound. 3. The Dow Jones Industrial Average will never break 4000 Grandiose statements can be generally risky, and almost always the more grand the statement, the more likely it will be wrong to some degree. So if you think some technology is just a short term hack, and is doomed to be forgotten, sure, tell us, and then tell us why you believe it. Just the grand statement alone really is worthless. On to the topic: As far as I can tell there are three major subjects to this thread. The ethical nature of using a transparent cache. The technical and operational problems and solutions associated with the implementation of a transparent cache. And finally, the customer support and management process involved with the implementation of the transparent cache. I'll address these in reverse order. Customer Support and process management seems to be the Achilles' heel of the Internet industry. This industry has seemed to build up a terrible reputation for poor customer support. Coming from the Nuclear industry, I am truly jaded when it comes to process management, but the Internet industry seems to have taken a track as far as possible on the other side. It amazes me the lack of process management in many (not all) of the organizations involved in this industry. Procedures seem to be a dirty word, and tend to result in the exodus of technical talent when imposed upon an organization. In many cases, you can find people in the industry jokingly referring to their facilities and really neat toys. Compared to the multiple interactions in the Internet, a nuclear power plant is simple. And yet, there are very few people if any who could singly determine what one particular action would have upon the operation of the plant as a whole. Obviously, the consequences of something bad happening is far worse, but in many cases, bad means the plant is shut down for some time period, and the company loses cash at an astonishing rate. However, the use of the Internet as a tool for many companies revenue stream continues to grow, and the resulting problems generated by poor process management is beginning to have much wider financial effect. It's apparent that we on the operational side are truly more at fault than anyone else. It's our duty to provide this service to our customers as best we can. If there is any area in which the customer could claim negligence or seek damages, this is one place that they may find a good chance. I'm not pointing fingers here, I've been in organizations that were party to this myself. I'm merely making a general observation. Many organizations have made good headway here. Events such as the infamous 7007 event, the widespread connectivity problems concurrent with filter implementations, down to local effects of such as those resulting in this thread will continue to occur as long as we continue to take a lax view as to the results of our actions. There apparently have been several organizations developed recently that claim to be attempting to address this problem. However, I have not seen an industry-wide affect by the operations of these organizations. As far as the technical and operational problems associated with the implementation of current transparent caching technology, it appears to me that the people on this list who have the most background in cache technology seem to have come to an agreement that technological answers to the problems experienced in this event exist. I can't, nor won't feign any significant technical knowledge in the caching field. I have a general knowledge level at best. >From a general level, this industries technological growth rate is phenomenal and the resulting technological implementation problems can be significant. However, there are organizations that are going to continue to develop and implement these new technologies. The successful ones will be financially rewarded in our capitalistic society, and the unsuccessful ones will fare less well. Sooner or later those implementing the new technology will have a significant advantage over those who don't. To avoid the challenge of new technology can be extremely damaging to the viability of a company in the Internet industry. Yes, I admit that the possibility that any current implementation of a new technology such as caching may actually end up being a temporary fix. However, I would be amazed to find that the effort put into development and operational testing of such a technology did not result in some advancement in that field. From caching, we may find interesting advancements in distributed technology for example. Finally, the ethical question comes around. Is it, or is it not ethical for someone to use a transparent cache? IMHO, the arguments on this portion of the thread have been nothing but sensationalist. It reeks of paranoia. Perhaps the X-Files can (or have they already) done a show on this. THEY are stealing your packets without your knowledge! THEY are monitoring your every transmission and receipt! THEY know your innermost secrets! Bah! Does transparent (or any caching technology, for that matter) give someone additional tools to pry into the data flow? Yes. Does the technology specifically do this and provide this information? No. It's a tool that could be mismanaged and used for something other than its original purpose, and perhaps even intrusive actions. However, it requires someone who wants to do such a thing, and the tools exist already to do similar things if someone so chooses. Attempting to limit the growth of a technology because it could result in the technology being used wrongly is pure folly. In the case of transparent caching, it's my belief that as long as an end user receives content as provided by the content provider in the format specified by the content provider (yes, this includes dynamic information), both the content provider and the end user could care less what physical, electromagnetic, or optical transformations take place from end to end. If the end to end transit is truly transparent, then the goal is accomplished. However, the expectation that this goal could be accomplished while content providers, transit providers, and client/server software providers work independently of each other, is ludicrous. These entities must work together to achieve this. The pointing of fingers and blaming of the other parties does nothing more than delay the maturation process of this industry. This is a complex and extremely interactive system, and must be dealt with as a whole for an optimal solution to arise. Perhaps it's the place here for a Grand Statement concerning the likelihood of this occurring. :) -Against my better judgement, Chris

1 0

running summary on caching
by Jeremy Porter 30 Jun '98

30 Jun '98

Here is my running summary of this debate, with only mild editorizing: >From: alan(a)mindvision.com (Alan Hannan) I believe DIGEX should be congratulated on their forward exploration of technology. Their creative solutions to problems like this are the spirit that will sustain the Internet. DWDM and huge fiber builds will get us part of the way to handling internet growth, but we need to find ways to utilize upper-layer tweaks to do more with less. --- >From: andrew khoo <andrewk(a)aussie.net> more aggressive caching techniques would necessitate that the tags be ignored anyway, and the "dynamic" content would still be cached. expires etc are only useful if the caching box decides to honour them. some of the content providers in the US would turn over in their graves if they knew what people who pay heaps more $$$ for traffic are doing to their web sites :). we have done some funky stuff with content (read: caching of dynamic pages, even .asps, "tokenized" HTML, dropping realaudio UDP etc etc etc). --- (I'm stilling waiting for the first successful copyright lawsuit in Australia because the cache isn't honoring expire headers or modifying the data) >From: "Sean M. Doran" <smd(a)clock.org> Let them turn. The easy fix for them is to duplicate their content elsewhere in the world, so they can control it themselves, and reduce the need for caching by operators like you. Sean. >From: Paul Vixie <vixie(a)vix.com> And I wanted to give one thing Sean said more air time so folks won't just gloss over it: ... > > That is, a cache which imposes decent long-haul TCP behaviour > reduces the number of packets which are delivered all the way > from the web server to the terminal server but tail-dropped there > rather than being delivered to the end user. This is rather important, both because the stacks used in last-mile devices (such as the Microsoft Program Loader) are not very modern, and because HTTP persistent connections end up not being very helpful until they are aggregated across multiple last-mile devices. --- >From: Patrick McManus <mcmanus(a)AppliedTheory.com> In a previous episode Michael Dillon said... :: [...] Caching is a temporary hack that provides some limited :: benefit at the moment but the need for it is fading except in areas that :: are a couple of years behind in bandwidth deployment. I can't believe you said that. Heirarchical caching reduces latency and increases availability _in addition to_ conserving bandwidth. Those (particularly the second) will remain critically important features no matter if you're traversing a 512k line or OC48... --- >From: Pushpendra Mohta <pushp(a)CERF.NET> No amount of capacity upgrades will obviate the need for an intelligent architecture for content distribution on the net. Utilization will explode to consume all bandwidth available and in the absence of an elegant architecture will do so rather wastefully. Most sites on the web are currently misdeployed/incorrectly-located on the network. Techniques that prevents unwarranted replication of data and bring the content to the user rather than the user to the content will bring about more efficient use of bandwidth while increasing performance for all users. These need to be complemented with techniques to address the concerns of other constituents, like marketeers. --pushpendra --- >From: Jon Lewis <jlewis(a)inorganic5.fdt.net> Actually, I thought I was the loud one complaining. I'm not a Digex competitor either...I'm the Digex customer that started this thread. --- (Your comments did not compare in volume or tone to some made by Digex's competitors, it would seem a good number of people felt you have a reasonable complaint about notfication but were to polite to rummage through Digex's dirty laundry in this forum.) >From: John Fraizer <John.Fraizer(a)EnterZone.Net> Horse-hockey. Purchasing the proper amount of bandwidth from your upstream and your upstream implementing the facilities to deliver that bandwidth is what is necessary to scale the internet into the future. Over-allocation is bad enough on the dialup-ISP side of the equation without backbone providers doing the same thing. If your backbone provider doesn't have the capacity to deliver what they have sole to you, it's time to contact two entities: Your lawyer and another backbone provider. --- (Question: what off the shelf technology goes beyond DWDM OC192? Second Question: When will that bandwidth be needed?) >From: Pete Farmer <pfarmer(a)strategies-u.com> As a customer, I don't give a hoot for belief systems and opinions on where things 'ought' to be. Don't talk to me about 'unethical,' talk to me about better ways to solve my business problem. --- >From: "Sean M. Doran" <smd(a)clock.org> Yes, and I am sure they fully realize that the loudest noises have been made by DIGEX's *competitors* complaining about DIGEX's customer-notification or service change, and consume a large block of salt. Sean. --- >From: "Patrick W. Gilmore" <patrick(a)priori.net> And before anyone goes off once again about how great caching is, I will once again publicly state that I am not at all opposed to caching - even forced caching of customers. It's your network, do as you please. ---(comments relating the the business pratices of Digex and its relations with its customers are not Operational issues and have been deleted. They might explain why everyone is bashing Patrick) >From: Karl Denninger <karl(a)mcs.net> I have no problem with a proxy where the user has agreed to its use. I have a major problem with a provider hijacking a stream of data and reprocessing it, then delivering what the client *thinks* is a direct communication session. In fact, I'm not even sure that such an act, absent consent, is legal. --- (This is my favorite, Karl makes all assumptions here about jurisdiction, contracts between two parties, etc. Karl is a competitor of Digex also. I still have this funny image of the Chicago policy showing up in Copenhagen, Denmark, to arrest a Canadian.) >From: Mark Milhollan <mlm(a)ftel.net> Karl Denninger writes: >(performance for the FIRST fetch through a proxy >is SLOWER - it HAS TO BE, since the proxy must first get the data before it >can pass it on). It doesn't have to -- think cut-thru switching. I've no idea if any/enough of the proxies do this. --- (they do, and Karl should have known better) >From: Hank Nussbacher <hank(a)ibm.net.il> For the past 4 years, AS378 has blocked port 80 (all academic network of 8 universities and many colleges) and forced all users thru multiple proxy servers (most recently Squid). Over the years we had to build up a list of sites to bypass. But in addition, one has to provide the ability to bypass based on source IP address (some users just don't want it - even if it speeds up their page downloads and toasts their bread at the same time.) --- (In the US people don't pay extra for un filtered access because it is so cheap. Probably don't want caching because they don't want people to know what type of porn they are downloading. (And don't understand that tcpdump could still monitor them.) So in New Zeeland, Australia, or other places where they pay extra for not going through the cache, its ok to do this, but in the US where the cost is carried by the provider its WRONG. Oh ok, end users have a common law right to dictate the technology used to deliver services to them. I'll remember that when I buy my next car without an emissions control system, because I have the fundemental right to pollute.)) >From: Karl Denninger <karl(a)mcs.net> See, that's the problem. Proxies are fine WHERE CUSTOMERS HAVE AGREED TO THEIR USE. STEALING someone's packet flow to force it through a proxy is NOT fine. --- (This same arguement could be apply to Ip deliver as specified in RFC 1149 "Standard for the transmission of IP datagrams on avian carriers." Data is data if the transparency is working correctly, the data delivered is identical and delivered faster. The streams aren't being taken anywhere. This is like arguing that echo cancelers can't be used on satilite communications because it breaks some types of transmissions, but helps the majory of customers, and it does in fact modify the STREAM.) So it seems: Against transparent proxying: Karl D. Against Digex transparent proxing: Patrick G. Upset about the problems with transparency not working: Jon L. Against transparency (for operational reasons even!) Mike O'Dell Thinks that bandwidth grows on trees: John F Pro hiarcheial caching, even transparency: Pushpendra M Hank N Patrick M Paul V Sean Doran andrew k Alan H Paul G Rich S Lamenting the lack of a Digex NOC to report transparency problems to: Randy B --- All well. I'd guess that content providers had better start working with cache vendors to make things work right, as I would guess a good bit of the content is going to go through caches, if you like it or not and everyone is going to be better off with it. There should be a real good flame war when people role out CoS/Voice over IP/ RSVP to exist next to best effort delivery. (My contract with provider X clearly implies that my traffic should be treated with equal priority to all other traffic on the network, therefore you are in breach of contract for offering higher priorities than best effort, oh and BTW you are doing unlawful wiretapping because you are classifying traffic by type which means looking past the IP address in the header.) --- Jeremy Porter, Freeside Communications, Inc. jerry(a)fc.net PO BOX 80315 Austin, Tx 78708 | 512-458-9810 http://www.fc.net

4 3

Host for NANOG 14 & modified NANOG Business Model
by William B. Norton 30 Jun '98

30 Jun '98

Hi all - As the subject line indicates, we are looking for a local host for the next NANOG. The tradional role of the local host has been to take care of all things local (facilities, hotel, catering, Audio/Video, Internet Connectivity, etc.), allowing the NANOG staff (pam, srh and myself) to concentrate on the meeting logistics, agenda, speakers, etc. The group has grown to the size where it is no longer reasonable for the local host to take care of the local logistics and bear the weight of the financial burden of feeding everyone. As I mentioned during the opening of NANOG 13, the catering expenses for 500+ is many tens of thousands of dollars. After many conversations with folks at the NANOG 13 meeting, we have come up with a modified funding model for NANOG that will increase the NANOG registration fee to cover more of the hosting expenses. We expect the new registration fee will increase from $150 (which covers the percentages of the 3 NANOG staff) to between $250 and $300 which will cover all catering and A/V costs. This will reduce the local host's financial burden substantially and return the local host role to that of providing local logistics support, the terminal room, and Internet connectivity. We have also been fortunate enough to have companies loan us technology for the event. For example, Bay has loaned routers and hubs and Cabletron has loaned wireless equipment that folks liked so well. Many others have contributed too. The point is, we expect this type of support to continue, so the load on the local host will similarly be reduced by these generous community contributions. We hope that this will encourage more types of organizations to volunteer to host NANOG 14. We'll be modifying the "Tips for hosting a NANOG" web page accordingly. So, if you are interested in hosting NANOG 14, don't be discouraged by the cost figures from past NANOGs. The local hosting costs will be substantially reduced. Please send e-mail to wbn(a)merit.edu. What I will need is an indication of proposed dates in or around October, a hotel or indication of facilities that you have looked into, and how you would plan to provide Internet access to the facilities. Multiple organizations working together is fine. Thanks - Bill -------------------------------------------------------------- William B. Norton <wbn(a)merit.edu> (734) 764-9430 (New Area Code)

1 0

Re: Digex transparent proxying
by Paul Vixie 30 Jun '98

30 Jun '98

> Or detect a proxy and refuse service (which, if I was doing this, is exactly > what I would do). "Go ahead, make my day." If you can detect the proxy box I used to sell via MII, and refuse service to it, I will post a retraction right here. > Not to mention those to derive advertising revenue from "views", and have no > way to measure them in a cached environment. "No way"? How about: rfc2227.txt -- Simple Hit-Metering and Usage-Limiting for HTTP. J. Mogul, P. Leach. October 1997. (Format: TXT=85127 bytes) (Status: PROPOSED STANDARD) > If I was doing that, I would also deny service to proxy servers and > display a nice message telling the user to remove the proxy or bitch about > its forced use. Playboy.COM did something like that to @Home last year for a similar reason. -- Paul Vixie La Honda, CA "Many NANOG members have been around <paul(a)vix.com> longer than most." --Jim Fleming pacbell!vixie!paul (An H.323 GateKeeper for the IPv8 Network)

2 1

Re: backbone transparent proxy / connection hijacking
by Eric Wieling 30 Jun '98

30 Jun '98

This is just my take on the issue. @Home caches. AOL caches. Browsers cache. (does the Direct/PC service cache?), many, many ISP's cache. Many companies cache. Caching is a fact of life. From my point of view any advertiser that does not take caching into account is screwing themselves and their customers. We cache. I try to configure our cache to err on the side of fresh content, rather than err on the side of hit rates. Every once in a while we get a customer complaint about a specific site and we address the issue on a case by case basis. So far complaints are fairly uncommon. We are not a backbone provider. Our cache is only available to customers that subscribe to our "security" services. (Just a note most of our customers have less then 300 users and although our firewall services are not air tight, I think we do a decent job of protecting our customers. If these customers did not have this service from us, most of them would have no protection at all. I'm just amazed at how much smaller business just don't seem to care about network security.) Making your site cache friendly is not rocket science. Putting reasonable expires information on objects is not tough. Telling your web server to set the expires as a percentage of the age of the document is supported in some servers. Using client side forms validation is not really that much tougher than using server side validation. Many SSI web pages are not really dynamic. We use SSI extensively on our web sites to create a consistent look and feel. Of course, some SSI pages are dynamic and should not be cached. I do think that it's utterly unacceptable for a backbone provider to force their customers to use their cache. I do, however, wish that more backbone providers would provide caching services to those people what want the service. --Eric Wieling >>> John Fraizer <John.Fraizer(a)EnterZone.Net> 6/29/98 6:20:31 PM >>> <snip> So, If someone is using site exec, etc in their code, and their provider/webmaster/mother didn't set up Progma: nocache, they're effectively screwed...erm...cached, right? <snip> Fantastic. So, lets say I'm Joe Banner Advertizer. Company X has paid me present their banner. They wanted to limit the amount of money they spent so, they had me code my servers to only display their banner X times per day since I bill them on impressions. Backbone provider Z installs one of your boxes. By default, no matter how many connections on the limited....erm...client side of the box are initiated to retrieve a "fresh" banner from our banner-farm, you send them Company X until the cache times out.

1 0

Re: small vent
by Tim Salo 30 Jun '98

30 Jun '98

> Date: Sat, 27 Jun 1998 23:55:20 +0800 > To: brian(a)meganet.net > From: "David R. Conrad" <davidc(a)apnic.net> > Subject: Re: small vent > > One reason might be that clueless folk in the US that complain to APNIC or > RIPE (or NANOG) directly instead of looking stuff up in the appropriate > whois database ("ARIN has all the IP information, right?") and complaining > to the people who can actually do something about the problem. > [...] Having to specify the proper database host on the whois command always struck me a bit like requiring every user to specify the appropriate server for, e.g. ".com", every time the user makes a DNS request. Why can't the registry folks get their collective act together and provide users with a system that allows a simple request like "whois name" to find the appropriate [piece] of the database. -tjs

8 10