In article <35935600.FF54EBFE@infowest.com>, "Aaron D. Gifford" <agifford@infowest.com> wrote:
It might also be time for content providers of time-sensitive data on the web to redirect requests coming from Digex' proxy harvest machines to a web page that says something along the lines of "Digex has intercepted your web request and directed it through their web caching system. This impacts the time-sensitive data at this site. Hence you cannot access this site in this manner.
Any content provider savvy enough to put that up could just as easily put in the proper headers to tag their content as cacheable or uncacheable or cacheable until time X, and that would be better for *everyone* involved. -- Shields, CrossLink.
On 27 Jun 1998, Michael Shields wrote:
In article <35935600.FF54EBFE@infowest.com>, "Aaron D. Gifford" <agifford@infowest.com> wrote:
It might also be time for content providers of time-sensitive data on the web to redirect requests coming from Digex' proxy harvest machines to a web page that says something along the lines of "Digex has intercepted your web request and directed it through their web caching system. This impacts the time-sensitive data at this site. Hence you cannot access this site in this manner.
Any content provider savvy enough to put that up could just as easily put in the proper headers to tag their content as cacheable or uncacheable or cacheable until time X, and that would be better for *everyone* involved.
Except when listening to the "proper" headers has been turned off by the cache (which some of these caching products support or will support). bye, ken emery
On Fri, 26 Jun 1998, ken emery wrote:
Except when listening to the "proper" headers has been turned off by the cache (which some of these caching products support or will support).
In thoise instances though the implimenter of the cache is the fool - not the manufacturer - like anything the operator/implimentor of a caching product should have some understanding of WTF they are doing - sadly you are right - Ken - thre undoubtably are ppl out there that will say "Hey this will let me cache it longer - kewl beans!" - and will suffer the consequences of their decision - but that doesn't really mean that the product should not afford the operater the options of fine tuning their performance. -- I am nothing if not net-Q! - ras@poppa.clubrich.tiac.net
On Sat, Jun 27, 1998 at 03:39:48AM +0000, Michael Shields wrote:
In article <35935600.FF54EBFE@infowest.com>, "Aaron D. Gifford" <agifford@infowest.com> wrote:
It might also be time for content providers of time-sensitive data on the web to redirect requests coming from Digex' proxy harvest machines to a web page that says something along the lines of "Digex has intercepted your web request and directed it through their web caching system. This impacts the time-sensitive data at this site. Hence you cannot access this site in this manner.
Any content provider savvy enough to put that up could just as easily put in the proper headers to tag their content as cacheable or uncacheable or cacheable until time X, and that would be better for *everyone* involved. -- Shields, CrossLink.
Or detect a proxy and refuse service (which, if I was doing this, is exactly what I would do). Not to mention those to derive advertising revenue from "views", and have no way to measure them in a cached environment. If I was doing that, I would also deny service to proxy servers and display a nice message telling the user to remove the proxy or bitch about its forced use. -- -- Karl Denninger (karl@MCS.Net)| MCSNet - Serving Chicagoland and Wisconsin http://www.mcs.net/ | T1's from $600 monthly / All Lines K56Flex/DOV | NEW! Corporate ISDN Prices dropped by up to 50%! Voice: [+1 312 803-MCS1 x219]| EXCLUSIVE NEW FEATURE ON ALL PERSONAL ACCOUNTS Fax: [+1 312 803-4929] | *SPAMBLOCK* Technology now included at no cost
On Sat, 27 Jun 1998, Karl Denninger wrote:
Not to mention those to derive advertising revenue from "views", and have no way to measure them in a cached environment. If I was doing that, I would also deny service to proxy servers and display a nice message telling the user to remove the proxy or bitch about its forced use.
Karl that is silly - in the a lot less time and effort that you would put in setting up your redirect - you could have just used a correct Expires header and has your content (or at least just the suff that you want to remain dynamic) avoid the cache. It serves you as the designer as much as it serves the enduser (speedier loads). If your page comes up quickly chances are the person will venture around a bit londer and click more links in the 20 minutes that they have before their date picks them up etc... It's common sense. -- I am nothing if not net-Q! - ras@poppa.clubrich.tiac.net
On Sat, Jun 27, 1998 at 05:49:34AM -0400, Rich Sena wrote:
On Sat, 27 Jun 1998, Karl Denninger wrote:
Not to mention those to derive advertising revenue from "views", and have no way to measure them in a cached environment. If I was doing that, I would also deny service to proxy servers and display a nice message telling the user to remove the proxy or bitch about its forced use.
Karl that is silly - in the a lot less time and effort that you would put in setting up your redirect - you could have just used a correct Expires header and has your content (or at least just the suff that you want to remain dynamic) avoid the cache. It serves you as the designer as much as it serves the enduser (speedier loads). If your page comes up quickly chances are the person will venture around a bit londer and click more links in the 20 minutes that they have before their date picks them up etc... It's common sense.
As soon as that happens widely the cache is useless (or worse) and therefore people who use them have a reason to ignore the Expires headers. -- -- Karl Denninger (karl@MCS.Net)| MCSNet - Serving Chicagoland and Wisconsin http://www.mcs.net/ | T1's from $600 monthly / All Lines K56Flex/DOV | NEW! Corporate ISDN Prices dropped by up to 50%! Voice: [+1 312 803-MCS1 x219]| EXCLUSIVE NEW FEATURE ON ALL PERSONAL ACCOUNTS Fax: [+1 312 803-4929] | *SPAMBLOCK* Technology now included at no cost
On Sat, 27 Jun 1998, Karl Denninger wrote:
As soon as that happens widely the cache is useless (or worse) and therefore people who use them have a reason to ignore the Expires headers.
Sure if "everything" on the site is tagged as dynamic or is pre-expired then what is the point of caching at all - but "everything" doesn't need to be for caching and hit-stats to coexist - just count your hits on one item, just leave those items that ABSOLUTELY must be dynamic be dynamic - and items that should be updated daily - should have the correct expires date set - so that IMS can work to their advantage. -- I am nothing if not net-Q! - ras@poppa.clubrich.tiac.net
On Sat, Jun 27, 1998 at 06:22:07AM -0400, Rich Sena wrote:
On Sat, 27 Jun 1998, Karl Denninger wrote:
As soon as that happens widely the cache is useless (or worse) and therefore people who use them have a reason to ignore the Expires headers.
Sure if "everything" on the site is tagged as dynamic or is pre-expired then what is the point of caching at all - but "everything" doesn't need to be for caching and hit-stats to coexist - just count your hits on one item, just leave those items that ABSOLUTELY must be dynamic be dynamic - and items that should be updated daily - should have the correct expires date set - so that IMS can work to their advantage.
That doesn't work, since the meta-tag is on the PAGE, not the element (which technically doesn't exist and is a figment of the server's imagination) Therefore, any page which has a non-cacheable element (ie: an ad or time-sensitive data) must be marked non-cachable. Congratulations - you just specified that any shtml or asp page must be marked non-cachable, along with any time-sensitive or advertiser-sponsored page. If that actually happens, then the proxy server operators will start shutting off recognition of the headers, and now we're right back where we started, along with the performance problems that this causes (forcing the traffic through a proxy server for EACH access actually HURTS performance, not helps it). -- -- Karl Denninger (karl@MCS.Net)| MCSNet - Serving Chicagoland and Wisconsin http://www.mcs.net/ | T1's from $600 monthly / All Lines K56Flex/DOV | NEW! Corporate ISDN Prices dropped by up to 50%! Voice: [+1 312 803-MCS1 x219]| EXCLUSIVE NEW FEATURE ON ALL PERSONAL ACCOUNTS Fax: [+1 312 803-4929] | *SPAMBLOCK* Technology now included at no cost
On Sat, 27 Jun 1998, Karl Denninger wrote:
That doesn't work, since the meta-tag is on the PAGE, not the element (which technically doesn't exist and is a figment of the server's imagination)
That is why it must first be parsed by the web server.
Therefore, any page which has a non-cacheable element (ie: an ad or time-sensitive data) must be marked non-cachable.
No sireee Karl - no the entire page but the specific element that is time sensitive.
Congratulations - you just specified that any shtml or asp page must be marked non-cachable, along with any time-sensitive or advertiser-sponsored page.
Yikes - Karl there are many differnt items on the page - they do not all have to have the same attributes - a cgi, an asp a banner-add, or maybe some other item on the page can be marked dynamic while the rest of the elements can be cached.
If that actually happens, then the proxy server operators will start shutting off recognition of the headers, and now we're right back where we started, along with the performance problems that this causes (forcing the traffic through a proxy server for EACH access actually HURTS performance, not helps it).
Yeah assuming that everyone who owns/operates a proxy/cache is a total and utter moron. -- I am nothing if not net-Q! - ras@poppa.clubrich.tiac.net
On Sat, 27 Jun 1998, Rich Sena wrote:
Sure if "everything" on the site is tagged as dynamic or is pre-expired then what is the point of caching at all - but "everything" doesn't need to be for caching and hit-stats to coexist - just count your hits on one item, just leave those items that ABSOLUTELY must be dynamic be dynamic - and items that should be updated daily - should have the correct expires date set - so that IMS can work to their advantage.
more aggressive caching techniques would necessitate that the tags be ignored anyway, and the "dynamic" content would still be cached. expires etc are only useful if the caching box decides to honour them. some of the content providers in the US would turn over in their graves if they knew what people who pay heaps more $$$ for traffic are doing to their web sites :). we have done some funky stuff with content (read: caching of dynamic pages, even .asps, "tokenized" HTML, dropping realaudio UDP etc etc etc). | andrew khoo <andrew@aussie.net> | aus: +61-2-9310-3315 | | network architect for hire | gsm: +61-416-161-234 | | SkyBytes - coming to .au in Aug | usa: +1-415-902-8820 |
On Sun, 28 Jun 1998, andrew khoo wrote:
more aggressive caching techniques would necessitate that the tags be ignored anyway, and the "dynamic" content would still be cached. expires etc are only useful if the caching box decides to honour them.
The tags are usually ignored by default - the html tags I mean - unless itis parsed into http most caches will ignore it and you do have a problem. The majority of caches I've seen have a built exclusion for the usual "always dynamic" items - cgi, "?" in path, asp... .
some of the content providers in the US would turn over in their graves if they knew what people who pay heaps more $$$ for traffic are doing to their web sites :). we have done some funky stuff with content (read: caching of dynamic pages, even .asps, "tokenized" HTML, dropping realaudio UDP etc etc etc).
Yup - you folk in Australiaview caching as the norm - not the enemy. Go figure... -- I am nothing if not net-Q! - ras@poppa.clubrich.tiac.net
This seems like more of a marketing issue than an operational issue. I see transparent proxying as another product offering in which you would have two options. Option 1 -- Utilize transparent proxying by applying policy based routing. This causes the backbone providers to use less bandwidth, so customers cost is reduced (i.e., highest cost is the backbone, so apply a discount for less bandwidth use). If contracts prior to the placement of the proxy are in place, the customer would have to provide explicit approval (and realize a reduced cost) before this policy would be applied. Option 2 -- Apply no policy to the end users connection. This is typically what "in place" contracts had in mind. If I agree to pay you $xxxx.xx/month for a connection before the cache was put in place, I should still receive that connection unaltered. BTW, I am in favor of transparent proxying...but not if I don't control it. ;) D. Derek Elder US Web / Gray Peak Technologies Network Engineering 212-548-7468 Pager - 888-232-5028 delder@graypeak.com http://www.usweb.com A Strategic Partner for the Information Age.
participants (6)
-
andrew khoo -
Derek Elder -
Karl Denninger -
ken emery -
Rich Sena -
shields＠crosslink.net