This is Brett Glass; I have been alerted to some of the responses to my message (which was cross-posted by a third party) and have temporarily joined the list to chime in. The following is my response to his message, edited slightly to include some new information. Dave Temkin wrote:
First and foremost, we built our CDN, Open Connect,
"Open Connect" is not, in fact, a CDN. Nor is it "peering." It is merely a set of policies for direct connection to ISPs, and for placing servers in ISPs' facilities, that is as favorable as possible in every way to Netflix. It costs Netflix as little as possible and the ISP as much as possible.
with the intention to deploy it as widely as possible in order to save ISPs who are delivering our traffic money
It does not, in fact, appear to save ISPs money. Note that Comcast asked for, and was given, additional payments even after it did all of the things that are part of Netflix' "Open Connect" program. Netflix, exercising inappropriate market power, has not offered smaller ISPs such as my own the same amount per customer. In fact, it has offered us no money at all -- even though our costs per Netflix customer are higher. Netflix thus discriminates against and threatens smaller ISPs, and by doing so, harms broadband competition.
and improve our mutual customer experience. This goes for ISPs large and small, domestic and international, big endian and little endian. We've never demanded payment from an ISP nor have we ever charged for an Open Connect Appliance.
The power and bandwidth consumed by an "Open Connect Appliance" (which is really just a hosted Netflix server) are a substantial expense for any ISP. Especially because the server is not a cache; it is stocked with content whether it is used or not and therefore wastes bandwidth on content and formats that will never be used even once.
When we first launched almost three years ago, we set a lower boundary for receiving a Netflix Open Connect Appliance (which are always free) at 5Gbps. Since then we've softened that limit to 3.5Gbps due to efficiencies of how we pre-load our appliances (more on that below).
Most small ISPs (the average in the US, in fact) have 1,000 to 2,000 accounts. If every one of those streams at 1 Mbps at the same time, which is highly unlikely, this still does not reach 3.5 Gbps. Therefore, most ISPs are excluded simply by this requirement if not by others (such as the requirement that the ISP alone pay for a dedicated connection to one of Netflix' relatively few "peering points").
We explicitly call our "cache" an Appliance because it's not a demand driven transparent or flow-through cache like the Akamai or Google caches. We do this because we know what's going to be popular the next day or even week and push a manifest to the Appliance to tell it what to download (usually in the middle of the night, but this is configurable by the ISP).
What Netflix does not say here is that (a) it can only somewhat predict what will be in demand or go "viral;" (b) it wastes bandwidth by sending multiple copies of each video to its server in different formats, rather than transcoding locally or saving bandwidth on lesser used formats via caching; and (c) its server consumes large amounts of energy and bandwidth. A cache can be much more efficient and can be owned and managed by the ISP.
The benefit of this architecture is that a single Appliance can get 70+% offload on a network, and three appliances clustered together can get 90+% offload, while consuming approximately 500 watts of power, using 4U of rack space, and serving 14Gbps per appliance.
To put this in perspective: LARIAT builds its own caches which consume as little as 20 watts and can saturate a 10 Gbps Ethernet port. The Netflix servers are large, bloated power hogs compared to a well designed cache.
The downside of this architecture is that it requires significant bandwidth to fill; in some ISPs cases significantly more than they consume at peak viewing time. This is why our solution may not work well for some small ISPs and we instead suggest peering, which has 100% offload.
Because it requires expensive bandwidth that's dedicated solely to Netflix, "peering" (as Netflix calls it; it's really just a dedicated link) has 0%, not 100%, offload. The ISP is paying for all of the bandwidth, and it cannot be used for anything else.
We've put a lot of effort into localizing our peering infrastructure worldwide. As you can see from this map (sorry for the image), we're in 49 locations around the world with the significant bulk of them in the US (blue pins = 1 location, red pins = >1 location in a metro) - more detailed version at <http://goo.gl/eDHpHU>http://goo.gl/eDHpHU and in our PeeringDB record ( <http://as2906.peeringdb.com>http://as2906.peeringdb.com) :
Our ISP connects to the Internet in Cheyenne, Wyoming (a major Internet "crossroads;" it's where I-80 meets I-25) and Denver, Colorado (which is, if anyplace can make the claim, the "center" of the entire Internet). Netflix' small and relatively sparse network of "peering" points does not include either of those locations. (I've noted, just today, that they have only recently listed a presence at a private data center outside of Denver in Englewood, Colorado -- not one of the major Denver exchanges, such as 1850 Pearl, to which we're already connected at great expense.) To get to it, we would have to spend several thousand dollars per month on an expensive connection, with no reimbursement of this expense from Netflix. If Netflix were a good citizen, it would (a) let ISPs cache content; (b) pay them equitably for direct connections (smaller and more remote ISPs have higher costs per customer and should get MORE per account than Comcast, rather than receiving nothing); and (c) work with ISPs to develop updated technology that makes streaming more efficient. Bandwidth is expensive, and unicast streaming without caching is by far the most inefficient conceivable way of delivering "fat" content to the consumer. --Brett Glass