In response to Randy's three criticisms of our recent SIGCOMM'11/NANOG'52 paper, which is available here: http://www.cs.bu.edu/~goldbe/papers/SBGPtrans_full.pdf http://www.cs.toronto.edu/~phillipa/sbgpTrans.html Point 1: "The ISP economic and incentive model is overly naive to the point of being misleading" To clarify, our paper focuses on the following question: Given that we want as many ASes as possible to deploy path validation (S*BGP), what sort of incremental deployment strategy should we use? To answer this question, one first needs to understand why an AS might have incentive to deploy S*BGP in the first place. There are many possible reasons (e.g., "the benefits of security" that Randy mentions, pressure from regulators, governments, or other ASes, PR opportunities, etc), in this paper we focused on one very specific incentive: An ISP might deploy S*BGP in order to increase the volume of traffic that it transits for its customers. We use this incentive as an "economic lever" that can be used to drive global S*BGP deployment. The paper shows that, even disregarding other economic levers (like security concerns, regulations, PR, etc), this incentive is enough to cause the majority of the Internet to deploy S*BGP, even if (a) security plays a very small role in the BGP decision process (i.e. security considerations influence routing decisions only _after_ Local-Pref and AS-PATH considerations), and even if (b) only a very small number (about 10) of ASes are "early adopters" that initially deploy S*BGP. Other economic levers (e.g. "the benefits of security") are complementary, and can only aid in driving S*BGP deployment. Our model assumes that ISPs have incentives to increase the volume of customer traffic that they transit because "the dominant form of pricing" in the Internet is based on traffic volumes sent, that is 95/5 percentile pricing: http://drpeering.net/AskDrPeering/blog/articles/Ask_DrPeering/Entries/2011/4... Thus, the more traffic (at the 95 percentile) that an ISP transits for its customer, the more they can charge that customer, and thus the more revenue they earn. Of course, this is not the case for *every* ISP: some ISPs may not use 95/5 percentile pricing at all, some ISPs may actually be losing money by providing Internet transit, and are instead earning all their revenue from other sources (e.g. IPTV, VPN, advertising, etc.), and moreover, content providers and residential ISPs are connecting directly more often, thus circumventing the charges of provider ISPs. However, major ISPs are still needed to reach most destinations, and smaller ISPs have a choice between multiple providers: http://www.peeringdb.com/ http://valas.gtnoise.net/lib/exe/fetch.php?media=comm083-valancius.pdf The fact that transit service prices are plummeting is, amongst other things, evidence of the fierce competition between ISPs over customer traffic. The key point of our incremental deployment strategy is to give ISPs one more dimension along which they can compete; namely, the ability to provide secure routes to their customers. This point is still valid as long as _most_ ISPs earn _some_ of their revenue from transiting customer traffic. The existence of services like Guavus, suggest that for many ISPs, this is indeed the case: http://www.guavus.com/solutions/tiered-pricing Point 2: "The security threat model is unrealistic and misguided" Our paper does not present a security threat model at all. We do not present a new security solution. We do not deal with the question of whether or not S*BGP should be deployed at all, which specific protocol (e.g. SBGP,soBGP, etc) should be deployed, or which security guarantees should be provided. This is the subject of many previous works. From Section 2.1: "Because our study is indifferent to attacks and adversaries, it applies equally to each of these protocols [i.e. SBGP, soBGP]." As explained above, we focus only on the question "Given that we want as many ASes as possible to adopt S*BGP, what sort of incremental deployment strategy should we use?" Thus, we are simply trying to maximize the number of ASes that deploy S*BGP. Point 3: "The simulations are questionable."
From Section 8: "The wide range of parameters involved in modeling S*BGP deployment means that our model cannot be predictive of S*BGP deployment in practice. Instead, our model was designed to (a) capture a few of the most crucial issues that might drive S*BGP deployment, while (b) taking the approach that simplicity is preferable to complexity."
Because ASes are unwilling to divulge information about routing policies, peering agreements, etc, every study of interdomain routing must contend with a dearth of ground truth with respect to AS-level topology, routing policies, and traffic matrices. We preformed extensive simulations to deal with this lack of ground truth. Please see Section 8 of our paper for detailed discussion about these issues. Here I'll address Randy's specific criticisms with direct quotes from our paper: Randy: "The paper also completely ignores the rise of the content providers as described so well in SIGCOMM 2010 by Labovitz et alia[2] It is not clear how to ‘fix’ the economic model, especially as[3] says you can not do so with rigor. Once one starts, e.g. the paper may lack Tier-N peering richness which is believed to be at the edges, we have bought into the game for which there is no clear end." Section 6.8.1: "Published AS-level topologies are known to have poor visibility into peering links at the edge of the AS-level topology [31]. This is particularly problematic for CPs, because they peer with many other ASes to cut down costs of delivering content [14] .. Thus, for sensitivity analysis, we created an augmented AS graph with ... additional peering edges from the five Content Providers." For more details on this graph, see Appendix D "AS graph Sensitivity analysis". Also, based on Labovitz's paper, we ran simulations where the content providers were assumed to source a vast majority (up to 50%) of total Internet traffic (as discussed in Section 3.1 and 6.8.1). Please see Section 6.8.2 to see how these assumptions affected our results. Randy: "The paper's simulations really should be shown not to rely on the popular but highly problematic Gao-Rexford model of inter-provider relationships, that providers prefer customers over peers (in fact, a number of global Tier-1 providers have preferred peers for decades), and that relationships are valley free, which also has significant exceptions. Yet these invalid assumptions may underpin the simulation results." Section 8.3: "In practice,... the local routing policies used by each AS, ... are arbitrary and not publicly known. Thus, we use a standard model of routing policies (Appendix A) based on business relationship and path length [16, 6]." Here we'll interject to say that while there are definitely examples that lie outside this model (e.g. ASes the prefer peer routes over provider routes), it currently remains the only general model we have, to date, of interdomain routing. As such, we note in Section 8.3: "Routing policies are likely to impact our results by determining (a) AS path lengths (longer AS paths mean it is harder to secure routes), and (b) tiebreak set size (Section 6.6). For example, we speculate that considering shortest path routing policy would lead to overly optimistic results; shortest-path routing certainly leads to shorter AS paths, and possibly also to larger tiebreak sets." Thus, while we cannot hope to accurately model every aspect of interdomain routing, nor predict how S*BGP deployment will proceed in practice, we believe that ISP competition over customer traffic is a significant economic lever for driving global S*BGP deployment. Sincerely, Sharon Goldberg and Michael Schapira -- Sharon Goldberg Assistant Professor, Computer Science, Boston University http://www.cs.bu.edu/~goldbe On Sun, Sep 4, 2011 at 6:02 AM, Randy Bush <randy@psg.com> wrote:
[ http://archive.psg.com/110904.broadside.html ]
Do Not Complicate Routing Security with Voodoo Economics a broadside
A recent NANOG presentation and SIGCOMM paper by Gill, Schapira, and Goldberg[1] drew a lot of 'discussion' from the floor. But that discussion missed significant problems with this work. I raise this because of fear that uncritical acceptance of this work will be used as the basis for others' work, or worse, misguided public policy. o The ISP economic and incentive model is overly naive to the point of being misleading, o The security threat model is unrealistic and misguided, and o The simulations are questionable.
Basic ISP economics are quite different from those described by the authors. Above the tail links to paying customers, the expenses of inter-provider traffic are often higher than the income, thanks to the telcos' race to the bottom. In this counter-intuitive world, transit can often be cheaper than peering. I.e. history shows that in the rare cases where providers have been inclined to such games, they usually shed traffic not stole it, the opposite of what the paper presumes. The paper also completely ignores the rise of the content providers as described so well in SIGCOMM 2010 by Labovitz et alia[2]
It is not clear how to ‘fix’ the economic model, especially as[3] says you can not do so with rigor. Once one starts, e.g. the paper may lack Tier-N peering richness which is believed to be at the edges, we have bought into the game for which there is no clear end.
But this is irrelevant, what will motivate deployment of BGP security is not provider traffic-shifting. BGP security is, as its name indicates, about security, preventing data stealing (think banking transactions[4]), keeping miscreants from originating address space of others (think YouTube incident) or as attack/spam sources, etc.
The largest obstacle to deployment of BGP security is that the technology being deployed, RPKI-based origin validation and later BGPsec, are based on an X.509 certificate hierarchy, the RPKI. This radically changes the current inter-ISP web of trust model to one having ISPs' routing at the mercy of the Regional Internet Registries (RIRs). Will the benefits of security - no more YouTube incidents, etc. - be perceived as worth having one's routing at the whim of an non-operational administrative monopoly? Perhaps this is the real economic game here, and will cause a change in the relationship between the operators and the RIR cartel.
The paper's simulations really should be shown not to rely on the popular but highly problematic3 Gao-Rexford model of inter-provider relationships, that providers prefer customers over peers (in fact, a number of global Tier-1 providers have preferred peers for decades), and that relationships are valley free, which also has significant exceptions. Yet these invalid assumptions may underpin the simulation results.
---
Randy Bush <randy@psg.com> Dubrovnik, 2011.9.4
[1] P. Gill, M. Schapira, and S. Goldberg, Let the Market Drive Deployment: A Strategy for Transitioning to BGP Security, SIGCOMM 2011, August 2011. http://conferences.sigcomm.org/sigcomm/2011/papers/sigcomm/p14.pdf
[2] [1] C. Labovitz, S. Iekel-Johnson, D. McPherson, J. Oberheide, and F. Jahanian, “Internet inter-domain traffic,” in SIGCOMM '10: Proceedings of the ACM SIGCOMM 2010 conference on SIGCOMM, 2010.
[3] M. Roughan, W. Willinger, O. Maennel, D. Perouli, and R. Bush, 10 Lessons from 10 Years of Measuring and Modeling the Internet's Autonomous Systems, IEEE Journal on Selected Areas in Communications, Vol. 29, No. 9, pp. 1-12, Oct. 2011. https://archive.psg.com/111000.TenLessons.pdf
[4] A. Pilosov, T. Kapela. Stealing The Internet An Internet-Scale Man In The Middle Attack, Defcon 16, August, 2008. http://www.defcon.org/images/defcon-16/dc16-presentations/defcon-16-pilosov-...