Dean Anderson wrote:

I'd rather expect this sort of behavior with anycasted servers...

I would not expect this kind of behavior from an anycasted address. You'd need a LOT of routing churn to see different caches every few seconds. It's much more likely some kind of load balancer in front of a DNS server farm.

With a cache, the behavior is confusing, but also harms DNS TCP support, just like that described for authoritative servers.

I verified it wasn't anycast by trying to exploit this very issue. I did a query that fell back to TCP while doing multiple small queries. I ran a network capture to pick out the short quries that occurred while the TCP query was going on. Short quries during the TCP connection came back with verying TTLs indicating that I was talking to different caches, i.e. different servers. Yet the TCP query continued without any hiccups. This indicates there is some type of per-session load balancing going on, not anycast routing.

Further there isn't a good reason to have anycasted caches. Indeed, with DHCP-learned nameservers, there is negative reasons to have anycast. Anycast balancing will never be as good as random selection from the appropriate set given by DHCP.

Frequently, [judging by the questions asked on DNSOP about setting up anycast caches, anyway], the goal of such gyration is high availability. However, its [been] fairly easilly shown that this goal isn't achieved.

Since this isn't anycast routing, can we save the religious diatribes for another thread?

On Tue, 19 Apr 2005, Crist Clark wrote:

...

FWIW, I did some 'dig'ing on my Comcast home service. The DHCP is handing out 204.127.198.4 and 63.240.76.4 for DNS at the moment.

I ran a query for a name in a zone I control that has a five minute TTL on 204.127.198.4. The first query came up with 5 minutes. I quickly made a change to the zone. hirty seconds after the initial query, I try again... err... and come up with the change. Hmm... Not caching at all? Another 30 seconds and I get the change, with 5m TTL. Thirty seconds later, I get the original response with appropriately decremented TTL. Another thirty seconds, I get the change, with 4m TTL.

My findings: Comcast is now using some kind of load balancing that messes with this kind of testing. 204.127.198.4 is not a single resolver. However, as far as I could tell, they were obeying the TTL. After 5 minutes, all of the responses were coming back with the change. The TTL values in the responses were decrementing as expected.

-- Crist J. Clark crist.clark@globalstar.com Globalstar Communications (408) 933-4387 The information contained in this e-mail message is confidential, intended only for the use of the individual or entity named above. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any review, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this e-mail in error, please contact postmaster@globalstar.com

On Apr 20, 2005, at 2:13 PM, Dean Anderson wrote:

No, you are thinking of the (wrong) claims originally made by ISC about how anycast would affect TCP to an anycast authoritative server. ISC wrongly asserted that since BGP routes don't churn very fast compared with DNS TCP connection lifetimes, that there should be no problem with anycast and TCP. This view has been shown to be wrong in the face of Per Packet Load Balancing (PPLB) which has been demonstrated to work on BGP links by haesu@towardex.com. Further, I showed that if you have PPLB on interior (eg OSPF) links leading to different BGP peers, the problem also happens. Packets are sent on a per packet basis to different places.

And I can show that if you give a pig wings.... Look, it breaks in certain situations. But anycast implementations of TCP apps have worked "well" for a decade now. Deal with the fact that not only do people use it, but users don't notice it. Or don't. No one here cares if you do. Reality trumps lab tests.

But caching servers are usually setup to load balance. Usually, the servers with the same IP address share an ethernet along with multiple routers. So the packets are switched on essentially a per-packet basis. Or possibly a per-arp basis that alters the MAC-based-forwarding behavior of a switch. This is fairly fine grained load balancing.

This is complete news to me. Of course, I do not run most of the caching name servers on the Internet, so what do I know. Do you? Would anyone who runs an anycast recursive name server care to supply data points to support or refute Mr. Anderson's assertion? Mr. Anderson, do you have any data points to support your assertion? -- TTFn, patrick

On Apr 20, 2005, at 3:29 PM, Dean Anderson wrote:

...

Or don't. No one here cares if you do. Reality trumps lab tests.

"Reality" for the last ten years has been that no one did either PPLB or TCP DNS. That reality is changing. It'll probably start to change faster, sooner. Then, users will start to notice the problems.

People have been using TCP applications on anycast for at least a decade, as I mentioned before. Since DNS responses tend to be very short lived TCP session, it seems to me that if it works for other applications (e.g. HTTP), it should work for DNS. Either way, reality still trumps lab tests, or mailing lists posts. Since it has worked, and continues to work, in _THE REAL WORLD_ for TCP applications much longer lived than DNS, I suggest that your assertion that "users will start to notice the problems" is incorrect. Of course, time will tell which of us is correct. Maybe I'm insane. Or maybe you are. Although I think time has already told which of us is.... -- TTFN, patrick

On Fri, 22 Apr 2005, Dean Anderson wrote:

On Wed, 20 Apr 2005, Patrick W. Gilmore wrote:

...

On Apr 20, 2005, at 3:29 PM, Dean Anderson wrote:

...

...

Or don't. No one here cares if you do. Reality trumps lab tests.

"Reality" for the last ten years has been that no one did either PPLB or TCP DNS. That reality is changing. It'll probably start to change faster, sooner. Then, users will start to notice the problems.

People have been using TCP applications on anycast for at least a decade, as I mentioned before. Since DNS responses tend to be very short lived TCP session, it seems to me that if it works for other applications (e.g. HTTP), it should work for DNS.

Its funny how I give you TWO conditions, and you ignore one of them. I'll try to use little tiny baby words:

TCP Anycast does NOT work with PPLB (Per - packet - load - balancing) Say it slowly several times.

oh well, I tried to stay quiet :) Probably the PPLB problem isn't quite as simple as: "you have pplb you can't do anycast". I'd imagine that you have to have some substantial difference in the paths that the PPLB follows, yes? like links to differing ISP's or perhaps extremely diverse links inside the same ISP. Correct?

Per Packet Load Balancing is not TCP friendly. (this discussion is orthogonal to DNS) PPLB leads to packet reordering. Quite a few empirical and theoretical papers have been published (in peer reviewed fora and elsewhere) that discuss the negative consequences of packet reordering. A Google search finds many references. On the downside for those attempting to maximize use of their circuits: packet reordering can lead to unnecessary retransmissions (squandering capacity). On the downside for users: packet reordering can lead to lower performance. Macroscopically: There is some movement (finally) towards providing the consumer with higher speed access to the Internet. (e.g. FIOS 30Mbps and other FTTH and vDSL services). Consumer adoption of such services would result in an upsurge of traffic: the need for larger backbones, enhanced server farms, more acolytes to service all of it. Adoption (and consequent resurgence of the Internet industry) will fail if consumers do not actually obtain improved performance from their new higher speed connections. PPLB only benefits those who are milking the last available profits out of a decaying industry. It is not a forward looking approach. At 01:51 AM 4/23/2005, Steve Gibbard wrote:

On Sat, 23 Apr 2005, Christopher L. Morrow wrote:

...

oh well, I tried to stay quiet :) Probably the PPLB problem isn't quite as simple as: "you have pplb you can't do anycast". I'd imagine that you have to have some substantial difference in the paths that the PPLB follows, yes? like links to differing ISP's or perhaps extremely diverse links inside the same ISP. Correct?

For anybody who's confused by this thread, this is a quick explanation, after which I'm really hoping the thread will die:

The "PPLB" Dean mentions is "per packet load balancing" in which you have two or more circuits, and packets to the same destination alternate which circuit they go down. In every case in which I've seen this used, it's been to combine multiple circuits taking the same path between the same pair of routers, to in effect create a bigger circuit. In theory, PPLB could also be used to split traffic between circuits going to different routers, perhaps even in different places. I've never seen anybody actually use the latter setup, and it seems to be universally regarded as something that would break things. I suppose it's possible that somebody's using it somewhere, probably with "interesting" results. It's the latter, theoretically possible, setup that Dean is talking about.

Anycast is a technique in which two or more servers, generally in different locations, announce the same address space. Those sending traffic into a network via one POP or exchange point will have their traffic go to the server close to that entry point, while those sending traffic into a network via another POP or exchange point will have their traffic go to the server close to that point. To an outside network, it looks the same as regular peering -- you see the same route at each peering point and can hand off traffic. The only difference is that the packets may not have to travel as far once they enter the other network.

So, just as a fun theoretical exercise, let's examine what happens in the PPLB to multiple locations scenario that Dean imagines:

Let's say somebody is in the Midwest, and has T1s to Network A and network B. And let's say that their network administrator read on the NANOG list that per packet load balancing was the trendy thing to do, so they turn on per packet load balancing between the two T1s. Now they want to send some packets to a unicast host on network C, somewhere in California.

They start with UDP DNS queries, each consisting of a single packet. Half go via network A, which peers with Network C in California. Responses come back with a 40 ms RTT. The other half go through network B, which has its closest peering point with Network C in Virginia. The packets go to Virginia and then to California, and the replies come back 80 ms later. Everything works fine.

Then they try to set up a more persistent connection, and again half their packets are taking the 40 ms path while the others are taking the 80 ms path. Now things get interesting, because the packets are arriving out of order. Some applications may do ok with this, since they'll take the sequence numbers and reorder the packets, with some buffering and processing delay. But remember, the latency amounts here are numbers I just made up, and there's no reason why it couldn't be 40 ms vs. 1 second in some parts of the world. In either case, I suppose it's possible that you'd get an HTTP connection to sort of work, and an ssh session might just seem mildly painful. But good luck getting a VOIP call or anything of the sort to function over such a connection.

Dean is correct that this setup would fall apart even further when anycast is thrown into the mix. In the anycast example, Network A hands off the packets to Network C in California, where they get sunk into a local server. Network B hands off the packets to Network C in Virginia, where they get sunk into a local server. Each server only sees half the packets, and half the retransmits, and is probably never going to get enough of the connection to put it all back together in a way that works.

So, there are a couple of different conclusions that could be drawn from this. The conclusion I come to is that there are enough problems doing per packet load balancing on non-identical paths that nobody would actually do it. I'm made more comfortable in this conclusion by having been through this discussion several times without finding anybody who claims to actually do that sort of per packet load balancing. I, therefore, declare the PPLB thing to be a non-issue.

It may also be valid to declare that PPLB over non-identical paths is important to allow people to use every last bit of bandwidth they're paying for, and that we shouldn't make their already painful predicament worse. But that's an argument I continue to be skeptical of.

On Wed, 20 Apr 2005, Patrick W. Gilmore wrote:

On Apr 20, 2005, at 3:29 PM, Dean Anderson wrote:

...

...

Or don't. No one here cares if you do. Reality trumps lab tests.

"Reality" for the last ten years has been that no one did either PPLB or TCP DNS. That reality is changing. It'll probably start to change faster, sooner. Then, users will start to notice the problems.

People have been using TCP applications on anycast for at least a decade, as I mentioned before. Since DNS responses tend to be very short lived TCP session, it seems to me that if it works for other applications (e.g. HTTP), it should work for DNS.

I don't know of any HTTP servers that do anycast. But their failure to take account of PPLB doesn't change anything. IF they are anycasting under false assumptions, they'll have problems, too. Perhaps you should read RFC 1546, which prescribes how to do TCP anycast. Then note that TCP anycast requires OS support which is not implemented in any unix-like system (or any system) that I know of. So, instead of following this prescription, the (DNS) anycast promoters have relied on an assumption of unique and slowly changing paths to eliminate the possibility that "two successive TCP segments sent to the anycast peer might be delivered to completely different hosts." But PPLB makes that "paths change slowly" assumption false, because it can use different paths on sequential packets.

From RFC 1546 (page 5)

How UDP and TCP Use Anycasting It is important to remember that anycasting is a stateless service. An internetwork has no obligation to deliver two successive packets sent to the same anycast address to the same host. Because UDP is stateless and anycasting is a stateless service, UDP can treat anycast addresses like regular IP addresses. A UDP datagram sent to an anycast address is just like a unicast UDP datagram from the perspective of UDP and its application. A UDP datagram from an anycast address is like a datagram from a unicast address. Furthermore, a datagram from an anycast address to an anycast address can be treated by UDP as just like a unicast datagram (although the application semantics of such a datagram are a bit unclear). TCP's use of anycasting is less straightforward because TCP is stateful. It is hard to envision how one would maintain TCP state with an anycast peer when two successive TCP segments sent to the anycast peer might be delivered to completely different hosts. The solution to this problem is to only permit anycast addresses as the remote address of a TCP SYN segment (without the ACK bit set). A TCP can then initiate a connection to an anycast address. When the SYN-ACK is sent back by the host that received the anycast segment, the initiating TCP should replace the anycast address of its peer, with the address of the host returning the SYN-ACK. (The initiating TCP can recognize the connection for which the SYN-ACK is destined by treating the anycast address as a wildcard address, which matches any incoming SYN-ACK segment with the correct destination port and address and source port, provided the SYN-ACK's full address, including source address, does not match another connection and the sequence numbers in the SYN-ACK are correct.) This approach ensures that a TCP, after receiving the SYN-ACK is always communicating with only one host. -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

On Sat, 23 Apr 2005, Patrick W. Gilmore wrote:

Been happening for many years. How do you think the original Boardwatch / Keynote speed tests were gamed? If you have any real experience on the Internet, you are well acquainted with anycast web servers.

Gaming speed tests sounds pretty rare. It doesn't appear that Akamai does this, but maybe I'm wrong. But it would depend on having unique paths. And it violates RFC 1546, as previously explained.

Okie, I give up. You clearly have no idea what you are actually talking about, so talk away, no one is listening.

Yes, _You_ clearly aren't listening. Which is the problem. Your cannard of "Its been done (in an archaic environment)" has no bearing on anything. The whole point is that environment is changing, and so hacks that used to be done, hacks that even RFC 1546 anticipated and warned against, won't continue to work in the future. I'm reminded of the arguments in the late 80's about threading: People (like you) said there are no multithreading operating systems, and multiprocessor systems existed only in labs. So designing threadsafe libraries or writing multithreading capable languages was a total waste of time. And they showed as evidence all the programs written from 1975 to 1985.

One last comment (although I doubt you will understand): Reality trumps... well, you.

Reality trumps alright. But you won't understand that. "Past performance is no guarantee of future performane" Let me guess: You're one of those people who won't be concerned about global warming until they need waders to walk around Manhatten at high tide. Then you'll go "Gee, where'd all this water come from? Why didn't someone say that the ice caps were melting?" Well, PPLB isn't the end of the world. But PPLB is coming, and the smart people will be prepared for it. They dumb people, well, they're dumb. What can be expected from dumb people? -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

* haesu@towardex.com (James) [Sat 23 Apr 2005, 23:10 CEST]:

With proliferation of high speed circuits and continuing trend of lower cost of bandwidth, PPLB is becoming more obsolete for many people that implemented it in the past. What is becoming more common is non-PPLB based setup, such as flow or destination based load balancing, et al to group high capacity circuits (i.e. when gig-e's aren't enough and 10GbE is too much of a capex for an immediate upgrade).

Exactly. Apparently it's little bother for router vendors to reuse the algorithms they wrote to properly support 802.3ad (link aggregation; the spec demands that `conversations' be kept on the same wire) for load balancing over multiple IP paths. Regards, -- Niels.

...

Well, PPLB isn't the end of the world. But PPLB is coming, and the smart people will be prepared for it. They dumb people, well, they're dumb. What can be expected from dumb people?

There are a variety of things that don't like PPLB, notably IPSEC. One problem is that if packet lengths aren't constant, you can get out-of-order delivery, and some protocols don't deal with that very well, as well as the load not really getting as perfectly balanced as proponents like to think. (There's also the problem that some popular small routers implement PPLB by burning too many of the CPU cycles that they don't have enough of, so you've got to consider the tradeoffs between buying more network connections vs. a bigger router.) -- ---- Thanks; Bill Note that this isn't my regular email account - It's still experimental so far. And Google probably logs and indexes everything you send it.

On Sun, Apr 24, 2005 at 02:00:48AM -0400, Valdis.Kletnieks@vt.edu wrote:

...

Well, PPLB isn't the end of the world. But PPLB is coming, and the smart people will be prepared for it. They dumb people, well, they're dumb. What can be expected from dumb people?

What you seem to be missing is that the *really* smart people will be prepared for it when it actually gets here - and will take advantage of it's lack of arrival in the meantime.

And, the even more important extension of your last comment, they'll be prepared to move on to something else, when it comes, and they can no longer take advantage of it's absence. Cheers, -- jra -- Jay R. Ashworth jra@baylink.com Designer Baylink RFC 2100 Ashworth & Associates The Things I Think '87 e24 St Petersburg FL USA http://baylink.pitas.com +1 727 647 1274 If you can read this... thank a system administrator. Or two. --me

On Sun, Apr 24, 2005 at 04:00:40PM -0400, Robert M. Enger wrote:

In your note below you speak of 'moving on to something else' when PPLB comes.

No, I actually don't.

PPLB destabilizes TCP. It elicits erroneous retransmissions, squanders capacity and lowers performance.

You are suggesting that we replace TCP in all the computers in the world?

Nope. I was observing generically that if people who take advantage of a technology window to supply a supportive technology (video capture cards for PC's) are smart, the *really* smart people are those who are prepared to move along to something else when the mainstream catches up (DV/Firewire) and their product is no longer necessary. Cheers, -- jra -- Jay R. Ashworth jra@baylink.com Designer Baylink RFC 2100 Ashworth & Associates The Things I Think '87 e24 St Petersburg FL USA http://baylink.pitas.com +1 727 647 1274 If you can read this... thank a system administrator. Or two. --me

DA> Date: Sat, 23 Apr 2005 16:13:22 -0400 (EDT) DA> From: Dean Anderson DA> And it violates RFC 1546, as previously explained. Who cares? You've railed against SMTP+AUTH because it's not a "standard". Why do you give a rat's rump about 1546? DA> Well, PPLB isn't the end of the world. But PPLB is coming, and the smart DA> people will be prepared for it. They dumb people, well, they're dumb. Perhaps PPLB becomes more common. Time for SACK, lest traditional TCP do bad things. As for anycast, there's a fair chance people building anycast clusters will work around PPLB. Maybe they'll build topologies to avoid problems. Maybe they'll have behind-the-scenes unicast intelligence to deal with TCP session transfer. I'll leave it at that. This thread is getting old, and 1xRTT latency makes SSH uncomfortable. DA> What can be expected from dumb people? Frequent NANOG posting. Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.

DA> Date: Sat, 30 Apr 2005 00:57:46 -0400 (EDT) DA> From: Dean Anderson DA> But for the record, you misrepresent my SMTP AUTH claims: Someone needs to put down the crackpipe. At least do a Google search or three to find out what I really say before putting words in my mouth. e.g., I specifically cited laws and cases that appear to apply to blacklists... now you claim I stated DNSBLs are exempt? Someone needs to put down the crackpipe. See threads like "ORBS (Re: Scanning)" on NANOG, or "Port 25 Email blocked by ISP" on cobalt-users. You might find major discrepencies between what you claim I say and what I really say. Someone needs to put down the crackpipe. You object to SMTP+AUTH because it isn't standard: http://www.merit.edu/mail/archives/nanog/199-11/msg00263.html http://www.merit.edu/mail/archives/nanog/199-11/msg00289.html You complain that SMTP+AUTH "doesn't scale"... yet viewing open relay logfiles for abusers scales?! Someone needs to put down the crackpipe. Yet you cite RFC1546 as the One True Anycast. Is RFC1546 a standard? What does its first paragraph say, again? DA> You really haven't been paying attention: There's no chance of that at DA> all: It isn't possible to build "vixie-cast" clusters that work around DA> PPLB. There are no topologies which include diverse paths that avoid DA> problems. http://www.merit.edu/mail/archives/nanog/msg07220.html Read what I said. Did I say "vixie-cast" clusters? Did I specify a particular topology, or suggest choosing topologies that work? Even when the thread is is plain sight for all to reference, you fail to cite correctly. Someone needs to put down the crackpipe. You claim PPLB over widely diverging paths will become increasingly common. If that actually happened, guess what would happen to unicast TCP? Guess what would happen to many UDP-based protocols over unicast? If you believe that PPLB problems are "vixiecast"-specific, I have a challenge for you: Connect two routers in series with multiple links. Run PPLB between them, using different latency/jitter/packetloss over each link. Do this for your production traffic. DA> > DA> What can be expected from dumb people? DA> > DA> > Frequent NANOG posting. DA> DA> There are other symptoms. Like being wrong alot, or being completely DA> unable to correctly state someone else's position. You've done a fantastic job of demonstrating both. As much as I'd love to have another protracted flamefest with you, NANOG is hardly the place, and I'm putting more priority on real work. Maybe one day my income will be proportional to how many characters I stuff in NANOG readers' mailboxes, but right now it's based on providing services. If a sane discussion of anycast is on-topic[1], I'll join. Barring that, I'm done posting to this thread. [1] Moderators? Is that operational enough, or too far in the "research" realm? Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.

On Sun, 1 May 2005, Dean Anderson wrote:

On Sun, 1 May 2005, Edward B. Dreger wrote:

...

e.g., I specifically cited laws and cases that appear to apply to blacklists... now you claim I stated DNSBLs are exempt? Someone needs to put down the crackpipe.

You agreed with me on something? I must have missed that at the time. I'm *sure* I would have made a note of that.

I would say there are laws that apply to blacklists too; just not the same laws that you cite. But IANAL, and neither are you. -- JustThe.net - Apple Valley, CA - http://JustThe.net/ - 888.480.4NET (4638) Steven J. Sobol, Geek In Charge / sjsobol@JustThe.net / PGP: 0xE3AE35ED "The wisdom of a fool won't set you free" --New Order, "Bizarre Love Triangle"

On Sun, 1 May 2005, Edward B. Dreger wrote:

You object to SMTP+AUTH because it isn't standard:

http://www.merit.edu/mail/archives/nanog/199-11/msg00263.html http://www.merit.edu/mail/archives/nanog/199-11/msg00289.html

Neither of these links actually work. But it is "Draft Standard". That is different from saying it is "Standard". It is not "Standard", and is still not "Standard", some 7 years after it went to Draft. All of my criticisms (made presumably in 1999), were correct. In 2005, SMTP AUTH is basically dead. There hasn't been a new mail client supporting SMTP AUTH in many years. What would be the point?

You complain that SMTP+AUTH "doesn't scale"... yet viewing open relay logfiles for abusers scales?! Someone needs to put down the crackpipe.

Actually, it does scale, and very nicely. Its almost trivially easy to automatically spot and block abusers. Blocking scanning prevents open relays from being found. And if they aren't found by open-relay blacklists, they aren't abused and there are no problems whatsoever. This requires almost no attention at all--Just ordinary daily and weekly usage reports, and automated detection and blocking of scans. This is helped along by blocking the blacklists nameservers, so even if they get a message in, it won't get delivered back to them, preventing them from discovering and abusing the relay. By contrast, its much, *much* harder to manage thousands of user accounts and passwords. This requires much customer service attention, and that's expensive. And when you offer backup SMTP service to a corporation for a relative pittance, you don't want to have to take on the headache of keeping all of their usernames and passwords. Many providers run open relays. They don't advertise this fact, due to, well, abusers like Matthew Sullivan. *You* need to put down the crackpipe.

Yet you cite RFC1546 as the One True Anycast. Is RFC1546 a standard? What does its first paragraph say, again?

RFC 1546 is Category: Informational However, its "informational" category doesn't mean one can mislead people into believing that "Vixie-cast" complies with RFC 1546. Calling it "TCP Anycast" and then referring to RFC 1546, misleads people. And criticism is well-deserved. If someone wants to create their own version of Anycast, let them. But they shouldn't mislead people that their version is the same as RFC 1546's version---when in fact they aren't the same. And "Vixie-cast" isn't *anything* like RFC 1546 TCP Anycast. And the fact that they have misled people is probably operational news. Especially since the DNS root server operators also assumed the Vixie was serving as their liason to the IETF, and that the IETF had "blessed" Vixie-cast. Its one thing to setup some random servers with some hairbrained scheme. Its quite another to installed some untested, non-standard, and unapproved configuration on critical network infrastructure like the DNS root nameservers.

DA> You really haven't been paying attention: There's no chance of that at DA> all: It isn't possible to build "vixie-cast" clusters that work around DA> PPLB. There are no topologies which include diverse paths that avoid DA> problems.

http://www.merit.edu/mail/archives/nanog/msg07220.html

Read what I said. Did I say "vixie-cast" clusters? Did I specify a particular topology, or suggest choosing topologies that work?

You said: "As for anycast, there's a fair chance people building anycast clusters will work around PPLB. Maybe they'll build topologies to avoid problems. Maybe they'll have behind-the-scenes unicast intelligence to deal with TCP session transfer." That would be "Vixie-cast" clusters. As I already said, UDP anycast has no problem. And I noted that RFC 1546 TCP Anycast has no problem with PPLB on diverse paths. That leaves only "vixie-cast" clusters. And you also indicated that there might exist a topology involving PPLB on diverse paths that might work. But there isn't one. There isn't any "behind-the-scenes unicast intelligence to deal with TCP session transfer" other than that described by RFC 1546, which involves changes to the TCP stack on each machine, including the client machine. Someone needs to put out their pipe and actually read most of the thread.

Even when the thread is is plain sight for all to reference, you fail to cite correctly. Someone needs to put down the crackpipe.

I cited it correctly.

You claim PPLB over widely diverging paths will become increasingly common. If that actually happened, guess what would happen to unicast TCP?

Nothing much. At worst, a performance problem for older TCP stacks. In stacks that can handle insertions, nothing at all.

Guess what would happen to many UDP-based protocols over unicast?

Nothing whatsoever would happen. A protocol implementation that doesn't behave correctly with out-of-order packets might have a problem. But this is an implementation problem. There is no obligation for an internetwork to deliver packets in order, either. --Dean -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

This seems like a new thread, so I changed the title. inline On Sun, 1 May 2005 Valdis.Kletnieks@vt.edu wrote:

On Sun, 01 May 2005 21:09:50 EDT, Dean Anderson said:

...

criticisms (made presumably in 1999), were correct. In 2005, SMTP AUTH is basically dead. There hasn't been a new mail client supporting SMTP AUTH in many years. What would be the point?

There's almost no new car companies supporting seat belts too.

_All_ car companies support seat belts, by law. So there are no new ones. But only 16 email clients (counting Netscape, Mozilla, and Firefox separately), support SMTP AUTH. But there are more than 1000 different email client programs. If you go to Microcenter, you can buy several email client programs for windows, but only one (Outlook) supports SMTP AUTH. Your comparison is precious. Its a classic sort of statistical deception, at the extreme. With seat belts, there is mandated 100% compliance. With SMTP AUTH, there is presently approximately 0.16% compliance. Lets round that up: Say 0.2% support. SMTP AUTH got 0.2% of the mail clients. The draft is dated 1998, and it got practically all of those 16 clients in 1999. And then the excitement petered out. People learned that SMTP AUTH didn't stop spam. Technically, the draft's authors knew it wouldn't stop spam, and never actually said it would. But its cheerleaders said it would (like they said POP-before-SMTP would). It was the *promise* that it would stop spam that generated the interest. And when that promise evaporated, so did the interest. But why would anyone deploy SMTP AUTH now? What would be the point of that? What's the cost/benefit analysis? There are only costs, and no benefits with SMTP AUTH. Indeed, I'm sending this from a system using SMTP AUTH. We've been trying to sell it for a couple years. Its our least used server (which why I use it). Basically, as a business proposition, its a bust. Has anyone made any money on SMTP AUTH? I don't think anyone has.

It's hard to get new ones when it's near 100% support. (Who *doesn't* support SMTP AUTH? Are there any major or even minor players that don't? Heck, even the now-venerable 'nmh' descendant of MH will do SASL auth for you....)

Cisco's mail client doesn't do SMTP AUTH. I'd say they are a major player. nmh is old, all right. Unless you want to exclude all but 16 or so mail clients (out of more than 1000), you can't really require SMTP AUTH. Some ISPs (residential) specify the mail client programs (or like AOL, provide custom software). They already have per-user accounts, and can therefore implement SMTP AUTH more easily. But then, *some* ISPs assume all their users run Windows, too. Not everyone is in that boat.

But as long as we're playing: Evolution and Mozilla's mail client are both fairly newcomers, and both do SMTP AUTH....

Err, no. Netscape was one of the original promoters and authors of the SMTP AUTH draft. They were probably the first to support SMTP AUTH in Netscape 4.0. Mozilla has the Netscape mail client. It isn't "new". Evolution isn't actually very new either, although its getting more traction after it was included in some linux distros. (I think it was installed by default in RH9.0 workstation installs) --Dean -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

On Sun, 01 May 2005 23:37:53 EDT, Valdis.Kletnieks@vt.edu said:

these either have 'Antique' tags on them or are painted National School Bus Chrome (National Bureau of Standards Color #1305)...)

Figures. Another reference says it's Standard 595a, Color 13432.. ;)

On Mon, 2 May 2005, Edward B. Dreger wrote:

DA> Date: Sun, 1 May 2005 21:09:50 -0400 (EDT) DA> From: Dean Anderson

DA> > http://www.merit.edu/mail/archives/nanog/199-11/msg00263.html DA> > http://www.merit.edu/mail/archives/nanog/199-11/msg00289.html DA> DA> Neither of these links actually work. But it is "Draft Standard". That is

s,199,1999,

you need more than that: http://www.merit.edu/mail.archives/nanog/1999-11/msg00289.html I said: "The SMTP AUTH RFC 2554 is standards track, but not standard. " I was correct. What's more interesting is what the other guy said:

Incorrect. It's a customer training issue, and a little development time on your part. If you can't use SMTP AUTH, don't. Use POP-before-SMTP. Whip up a custom finger daemon to accept a username/password pair in the same manner. Create a webpage for your customers to enter a username and password on to authenticate themselves. Use a VPN. Use magic headers or subject lines that your MTA catches and uses as identity verification. Provide a web-based interface for your customer's email. Use UUCP.

Oh sure, its a customer training issue. Who's going to pay for that? Yeah. Lets just "Whip up a custom finger daemon". What would be the benefit? Back then, it was to reduce spam, but this was a fallacy that I recognized right away. Sure, lets just make everyone use a VPN. Who's going to pay for that? And what's the benefit? Magic headers? UUCP? What kind of drugs were they on? And what's even more interesting, looking back at 1999, is that open relays were not being abused by commercial bulk emailers, but by anti-spammers. We tested this out in the late 1990s by submitting non-production relays to blacklists and monitoring connections. After scanning, they began getting abuse. I posted this back than, but it was ignored. Then, in the fall of 2003, when the major open relay blacklists shutdown, open relay abuse JUST DROPPED OFF TO NOTHING. And when SORBS started scanning, abuse picked back up again. Well, lamely. In the old days we were usually hit by 200-300 IPs, and sometimes as many as 2400+. The March abuse was only a little more than a dozen IPs. It was the same old abuse pattern: targeted at mainly 2 Korean doamins: daum.net and sayclub.com. Probably the same old extortion scam as before. They send a lot of abuse, and then get daum.net and sayclub.com to use their blacklist, eventually contributing money, of course. But this time it was all "from: webmaster@av8.com". Previously, that was kind of rare (one virus used "from: dean@av8.com", and abused our relays, but this wasn't much). In the old days, most of the open-relay zealots didn't consider domain restricted relay to be open. Though, ironically, I did. This was a minority view, though. And we caught Matthew Sullivan THREATENING MAILBOMBING---that is, threatening to spam people. As his defense, he said he didn't know that mailbombing was against the AUP(!?!) And MAPS employees were caught **working for spammers**, and that very SAME spammer was on the FTC anti-spam panel, which was stuffed with MAPS-associated people. And we caught (several times) blacklists being used for personal vendettas. There's more. The list is long and dishonorable. --Dean -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

On Mon, 2 May 2005, Matthew Sullivan wrote:

Off topic again Dean...? Can't you keep on topic and keep the personal attacks out of the list...?

Funny how its only off topic when its about your abuse.

Dean Anderson wrote:

...

ignored. Then, in the fall of 2003, when the major open relay blacklists shutdown, open relay abuse JUST DROPPED OFF TO NOTHING. And when SORBS started scanning, abuse picked back up again. Well, lamely.

FYI, until mid-late 2004 SORBS did no automatic open relay scanning.

From Day one SORBS has gone after open-proxy servers.

Ah, yes. Thats why its called the Spam and Open Relay Blacklist System. But I will grant them that they didn't start scanning until March, 2005.

...

And we caught Matthew Sullivan THREATENING MAILBOMBING---that is, threatening to spam people. As his defense, he said he didn't know that mailbombing was against the AUP(!?!) And MAPS employees were caught **working for spammers**, and that very SAME spammer was on the FTC anti-spam panel, which was stuffed with MAPS-associated people. And we caught (several times) blacklists being used for personal vendettas. There's more. The list is long and dishonorable.

And some people use mailing lists to sprout complete and utter twaddle to the world in the hope that the more they say it the more it will be believed as true... Now can we get back ontopic please?

Well, since you are the "twaddle spewer", claiming all of our netblocks hijacked in a bald lie, I guess we know what your hope is. But I suppose you mean to deny having ever threatened mailbombs, above. It is interesting that you don't actually deny it, you just imply I'm not telling the truth. Well, your messages to XO abuse are at http://www.iadl.org under the SORBS listing. People can see for themselves what sort of abuser you are. And since you've been booted for abuse, anyone calling you an abuser is not a personal attack, any more than calling Alan Brown a court-proven liar is a personal attack. That's a fact that people ought to be aware of. --Dean -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

On Wed, 4 May 2005, Matthew Sullivan wrote:

No it's because you're off topic. Whether justified or not SORBS complaints and SORBS bashing are not on-topic for NANOG.

This is not particularly about SORBS bashing. Its about the need for SMTP AUTH, whether SMTP AUTH stops spam, and who abuses open relays and how they do that. Since some people operate open relays, its good to explain how to protect their relays from you. -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

On Sun, 1 May 2005, Joe Maimon wrote:

Dean Anderson wrote:

...

And if they aren't found by open-relay blacklists, they aren't abused and there are no problems whatsoever.

How much credibility are you trying to lose?

I have 9 years of operational experience running open relays. How many years of open relay operations experience do you have? --Dean -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

On Sun, 01 May 2005 23:29:54 EDT, Dean Anderson said:

On Sun, 1 May 2005, Joe Maimon wrote:

...

How much credibility are you trying to lose? I have 9 years of operational experience running open relays.

OK.. I stand corrected. Unlike seat belts and phone service, if you've lost 99% of your credibility, it *is* possible to lose another 2%....

Speaking on Deep Background, the Press Secretary whispered:

Better yet, try to name 16 mail clients people _actually use_ which DON'T, other than MUA-only programs like mailx and mutt with no SMTP support at all. When I worked at a mediumish sized hosting company with probably well over 100k mail users, I can't _ever_ recall hearing about a complaint of a customer using a mail client that didn't support SMTP auth.

Well, if someone knows a program for the Treo 6xx that does SMTP-Auth w/APOP....I'm all ears... (& over SSL would be a big plus...) It's no longer sufficient to look merely at *nix, Mac and oh yea, M$ platforms when counting beans.. -- A host is a host from coast to coast.................wb8foz@nrk.com & no one will talk to a host that's close........[v].(301) 56-LINUX Unless the host (that isn't close).........................pob 1433 is busy, hung or dead....................................20915-1433

Speaking on Deep Background, the Press Secretary whispered:

http://palmsource.palmgear.com/index.cfm?fuseaction=software.showsoftware&prodid=44646

Look at that, a PalmOS app that does exactly what you're looking for. Doesn't seem to have been too difficult to find.

I looked at SnapperMail. It's too early for me to recall why it didn't fit the bill [caffine, STAT!] but it didn't.. It may have been the APOP issue... -- A host is a host from coast to coast.................wb8foz@nrk.com & no one will talk to a host that's close........[v].(301) 56-LINUX Unless the host (that isn't close).........................pob 1433 is busy, hung or dead....................................20915-1433

On Mon, 2 May 2005, David Lesher wrote:

Speaking on Deep Background, the Press Secretary whispered:

...

Better yet, try to name 16 mail clients people _actually use_ which DON'T, other than MUA-only programs like mailx and mutt with no SMTP support at all. When I worked at a mediumish sized hosting company with probably well over 100k mail users, I can't _ever_ recall hearing about a complaint of a customer using a mail client that didn't support SMTP auth.

Where are all these ISPs requiring SMTP AUTH. I don't see them, anywhere. This same story was given about Pop-before-SMTP in 1997. Supposedly, "everyone did it", but only a few small companies actually did it. --Dean -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

in this interminable thread from hell, someone finally said the magic words:

Thankfully, there's always procmail.

and helpfully gave a specific recipe:

:0 * ^From:.*<dean@av8\.com /dev/null

now, speaking as someone who went around the loop a few times with dv8 on the topic of PPLB, i can assure all of you that his mind (or whatever) is pretty much made up. all arguing's going to do at this point is (a) fulfill his fantasies of adequacy/relevance by making him seem worthy of refutation, (b) cause lurkers and onlookers to scratch their heads and wonder if "there's gotta be something to this, or why would the volume be so sustainably high on the thread?", and (c) annoy the hell out of everybody else on the list. it does no good for me to filter out the crackpots if the rest of you are just going to keep on replying to same. so, as RAH had LL say: "never try to teach a pig to sing, it wastes your time and annoys the pig." here's what my thread reader says this thread looks like. as you can see, most of the traffic is in response to a single crackpot. [ 30: Valdis.Kletnieks@vt.] Re: Slashdot: Providers Ignoring DNS TTL? Y - [ 75: Dean Anderson ] Re: SMTP AUTH < 25: "Patrick W. Gilmore"> Re: Slashdot: Providers Ignoring DNS TTL? < 26: Valdis.Kletnieks@vt.> < 14: Matthew Sullivan > < 84: "Edward B. Dreger" > Y - [ 19: Dean Anderson ] [ 22: "Steven J. Sobol" ] Y - [ 32: Dean Anderson ] Y - [ 116: Dean Anderson ] [ 27: "Edward B. Dreger" ] Y - [ 82: Dean Anderson ] Re: SMTP AUTH [ 8: Joe Maimon ] Re: Slashdot: Providers Ignoring... Y - [ 25: Dean Anderson ] [ 107: Will Yardley ] Re: SMTP AUTH [ 26: David Lesher ] [ 54: Valdis.Kletnieks@vt.] Re: SMTP AUTH < 24: Valdis.Kletnieks@vt.> Y - < 28: Dean Anderson > [ 19: Richard A Steenberge] [ 32: James ] [ 38: Joe Maimon ] < 9: Randy Bush > < 33: Matthew Sullivan > < 23: Tim Wilde > [ 21: David Lesher ] i'm not one of the annointed moderators, but i heard that we were supposed to practice "peer moderation" and so i'm asking you all to please show a little discipline before you hit the "FlameCrackpot" key. -- Paul Vixie

Vixie makes almost nothing but a personal attack. The one relevant mis-statement is that Vixie seems to suggest that I'm the only one to have "been around the loop" on PPLB. There were numerous others who agreed with me on DNSOP. In fact, there wasn't anyone not part of ISC who agreed with Vixie. It is always the same folks who make the personal attacks. Its almost always a proxy for Vixie. On the discussion on DNSOP about DNS Anycast, there were an interesting personal attacks from John Brown, contrary to Vixie's description of events: (same sort of thing happened with Dan Bernstein in 2002) ---------- Forwarded message ---------- Date: Thu, 30 Sep 2004 15:01:31 -0600 From: John Brown CT <john@chagres.net> To: Dean Anderson <dean@av8.com> Cc: Joe Shen <jshen@christmas.9966.org>, ietf@ietf.org, dnsop@lists.uoregon.edu Subject: Re: [dnsop] Re: Root Anycast (fwd) [...] I realize that many on this list don't reply to the troll food that Dean leaves around, but I felt it important to reply as someone thats NOT in any shape fashion or form, ISC or its staff. I am somone that has done the engineering work to make a different letter work better via Anycast. Which letter, well that doesn't matter. [...] --------------------------------------- It turned out the John Brown founded Chagres.net with Suzzanne Woolf, who was the ISC program manager promoting DNS anycast. So much for his being "someone thats NOT in any shape fashion or form, ISC or its staff." And how familiar is this message: http://www1.ietf.org/mail-archive/web/ietf/current/msg31537.html BTW, John Brown also made a frivolous DMCA complaint about this message. Apparently, he was under the wrong impression that copyright and non-disclosure are the same. Anyway, he really misunderstood fair-use, and the part of the DMCA that imposes penalties on those who make frivolous and false claims. --Dean On 2 May 2005, Paul Vixie wrote:

in this interminable thread from hell, someone finally said the magic words:

...

Thankfully, there's always procmail.

and helpfully gave a specific recipe:

...

:0 * ^From:.*<dean@av8\.com /dev/null

now, speaking as someone who went around the loop a few times with dv8 on the topic of PPLB, i can assure all of you that his mind (or whatever) is pretty much made up. all arguing's going to do at this point is (a) fulfill his fantasies of adequacy/relevance by making him seem worthy of refutation, (b) cause lurkers and onlookers to scratch their heads and wonder if "there's gotta be something to this, or why would the volume be so sustainably high on the thread?", and (c) annoy the hell out of everybody else on the list.

it does no good for me to filter out the crackpots if the rest of you are just going to keep on replying to same. so, as RAH had LL say: "never try to teach a pig to sing, it wastes your time and annoys the pig." here's what my thread reader says this thread looks like. as you can see, most of the traffic is in response to a single crackpot.

[ 30: Valdis.Kletnieks@vt.] Re: Slashdot: Providers Ignoring DNS TTL? Y - [ 75: Dean Anderson ] Re: SMTP AUTH < 25: "Patrick W. Gilmore"> Re: Slashdot: Providers Ignoring DNS TTL? < 26: Valdis.Kletnieks@vt.> < 14: Matthew Sullivan > < 84: "Edward B. Dreger" > Y - [ 19: Dean Anderson ] [ 22: "Steven J. Sobol" ] Y - [ 32: Dean Anderson ] Y - [ 116: Dean Anderson ] [ 27: "Edward B. Dreger" ] Y - [ 82: Dean Anderson ] Re: SMTP AUTH [ 8: Joe Maimon ] Re: Slashdot: Providers Ignoring... Y - [ 25: Dean Anderson ] [ 107: Will Yardley ] Re: SMTP AUTH [ 26: David Lesher ] [ 54: Valdis.Kletnieks@vt.] Re: SMTP AUTH < 24: Valdis.Kletnieks@vt.> Y - < 28: Dean Anderson > [ 19: Richard A Steenberge] [ 32: James ] [ 38: Joe Maimon ] < 9: Randy Bush > < 33: Matthew Sullivan > < 23: Tim Wilde > [ 21: David Lesher ]

i'm not one of the annointed moderators, but i heard that we were supposed to practice "peer moderation" and so i'm asking you all to please show a little discipline before you hit the "FlameCrackpot" key.

-- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

On Sun, 1 May 2005, Will Yardley wrote:

Is it time to break out the "Please do not feed the trolls" sign?

Feeding 'em anyway... but *plonk* for Mr. Anderson. For those who are masochists, read on.

On Sun, May 01, 2005 at 10:50:29PM -0400, Dean Anderson wrote:

...

But only 16 email clients (counting Netscape, Mozilla, and Firefox separately), support SMTP AUTH. But there are more than 1000 different email client programs.

Firefox isn't an email client... maybe you're thinking of Thunderbird?

Thats it.

There may be lots of programs, but most / all of the ones that people actually USE support SMTP auth. Most of the less popular ones I've heard of / seen support SMTP auth as well - Becky!, The Bat, Mulberry, OS X's Mail.app. I could probably name more than 16 off the top of my head[2].

Better yet, try to name 16 mail clients people _actually use_ which DON'T, other than MUA-only programs like mailx and mutt with no SMTP support at all. When I worked at a mediumish sized hosting company with probably well over 100k mail users, I can't _ever_ recall hearing about a complaint of a customer using a mail client that didn't support SMTP auth.

This off-list message from Jay is probably relevant: Jay had some better points to make: ============================================================ On Mon, 2 May 2005, Jay R. Ashworth wrote:

On Sun, May 01, 2005 at 10:50:29PM -0400, Dean Anderson wrote:

...

Your comparison is precious. Its a classic sort of statistical deception, at the extreme. With seat belts, there is mandated 100% compliance. With SMTP AUTH, there is presently approximately 0.16% compliance. Lets round that up: Say 0.2% support. SMTP AUTH got 0.2% of the mail clients.

.2% by *brand name*.

That shows they didn't get vendor participation.

What percentage by *deployment*?

Deployment changes over time.

*Who* is gaming statistics, Dean?

Not I. If customers don't want it, and other vendors don't pick it up, why should the current vendors continue to support it? Just more bug potential. The people who say "90% of the world is Outlook", speak very little information about the market. Its like saying the US is the only superpower. Today, perhaps. Tomorrow may be a different story. If .2% of the countries signed the nuclear non-proliferation treaty, it would be a failure. Even if those .2% included China, which includes 1/4 the worlds population, it would still be a failure. And Communism can't be wrong, either, 2 billion chinese say so. ============================================================

...

With seat belts, there is mandated 100% compliance. With SMTP AUTH, there is presently approximately 0.16% compliance.

Bullshit. The percentage as measured by number of actual USERS is high, since 99.99% [1] of all users are using an MUA which supports SMTP auth.

Speaking of bullshit, Almost none of those mailservers are using SMTP AUTH. Being capable of configuring SMTP AUTH is completely different from saying it is in use. Next you'll be saying DECnet is still highly deployed because every Cisco can run LAT. -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000

Been happening for many years. How do you think the original Boardwatch / Keynote speed tests were gamed? If you have any real experience on the Internet, you are well acquainted with anycast web servers.

Let me, let me, let me! It involved an err... locating linux boxes with the identical IP addresses given to Boardwatch/Keynote in locations very close to location of the probes. Finally, for an extra "speed up" it involved modifying kernels to spew packets back as fast as possible single the only thing that those boxes did was tricking keynote. Alex