Ray Soucy <rps@maine.edu> writes:
You can certainly do anycast with TCP, and for small stateless services it can be effective. You can't do anycast for a stateful application without taking the split-brain problem into account.
In my experience, the thing that makes anycast work *well* is having the concept of a Plan B baked into some-layer-above-4. That creates the ability to recovery gracefully in the corner case when a routing change causes your session to blow up. Choice of layer 4 protocol doesn't really enter into it, nor does the length of time that the layer 4 session exists (in the case of UDP, generally 2 packets; in the case of TCP, somewhat longer). Shorter sessions have a lower likelihood of losing, due to shorter exposure time, but even for a single-packet-each-way UDP transaction the time (and the risk) is not 0. People of course use anycast for DNS. Personal experience shows that it also seems to work great for HLS video streaming. I'd imagine it would work fine for email too, since the whole concept of multi-level MX is a "plan-B-at-higher-level" thing.
The entire CDN model was developed with anycast in mind,
Not really; practical application of anycast was nascent when US 6,108,703 (the "Akamai patent", which centered around DNS) was filed. A brief history of anycast is at https://tools.ietf.org/html/draft-mcpherson-anycast-arch-implications-00 section 3.
Taking a normal application, like mail or a dynamic website, and just using anycast for load balancing without designing the service with the anycast model in mind is probably not a good idea. You need to expect that the same user could access different systems, and design for that.
For anything at scale, wherein one has multiple back end devices, one must already design for that. Designing consistency-synchronized systems that work over continental or global scale latency is left as an exercise to the implementer.
The real point here is the problem OP is describing should be easily handled by having proper MX records, and getting into anycast for mail is likely not the right choice (unless maybe your goal is to be really efficient at SPAM).
Probably originating outbound connections to arbitrary locations from an anycast locator is a step away from goodness. -r