Here's a couple of other things we thought about. FDDI NAP - good short term prospects, tried and true technology. Capacity even more of a problem than SMDS, won't even support a 155 Mb/s vBNS connection. Switched FDDI over ATM helps the bandwidth problem a bit, but I don't really understand how that will extend to a large-scale network. The LAN-emulation folks in the Forum have lots of work yet to do with large LAN emulation networks. The equipment for LAN emulation over ATM is in the same stage of "immaturity" as the IP/ATM equipment. Multimedia NAP - My best reply to this is Peter Ford's statement, "NAPs Don't Route". They're a layer two service. Some layer 2 interworking may be possible in the future, and we're looking at FR<->ATM interworking boxes right now. Cheap, low speed connections to the NAP - We got squeezed by product availability here. We know that there is a bigger market in selling T1 speed connections to the NAPs than T3 and above. However, we had a requirement to support multiple T3's initially (remember that the minimum purpose of the NAPs is to support the existing NSFNET backbone for transition purposes, the NSPs who are providing access to RNP's and who have a requirement to connect to all of the priority NAPs, and the vBNS). FDDI and SMDS weren't going to survive for long under this type of load, but ATM doesn't yet have T1 capability (although it's coming). So, we proposed ATM, starting at 45 and 155 Mb/s initially, and extending to 1.5 Mb/s and 622 Mb/s as switching equipment becomes available. Believe me, we'll be offering T1 speed connections to the SF and Chicago NAPs just as soon as we can make it work. ATM immaturity - ATM itself isn't immature, it's the router and ADSU technology that's immature. We've been running ATM in our lab since (believe it or not) 1986, and switch vendors have been shipping equipment for about 3 years now. We're beginning to see second generation products on the market right now. Since the Toronto IETF, we've been running tests on the SF NAP configuration, and the problems we've found (and fixed) have all been with the ADSU's or routers. We do have a working configuration for the 1490/AAL5 protocol stack, and are running load tests on it now. Bi-lateral/multi-lateral NAP traffic agreements - There's no rules here at all. The agreements are completely up to the NSPs who connect to the NAP. A CIX-like arrangement where all parties agree to exchange traffic w/o settlements is just as possible as N**2 bi-lateral agreements with traffic sensitive settlements. The NAP managers and NSF have nothing to say here. As far as taking a long-range view on the Internet, I'll plead guilty. We're already seeing congestion problems on the Internet today, and that's even before we turn the video and audio applications completely loose. My concern is that if we don't get enough capacity into the Internet, and really soon, the growth rate is going to drop badly. I've lived on badly congested Internets (remember the 56 Kb/s NSFNET?), and it's not an experience I wish to repeat. Dave
ATM immaturity - ATM itself isn't immature, it's the router and ADSU technology that's immature. We've been running ATM in our lab since (believe it or not) 1986, and switch vendors have been shipping equipment for about 3 years now. We're beginning to see second generation products on the market right now. Since the Toronto IETF, we've been running tests on the SF NAP configuration, and the problems we've found (and fixed) have all been with the ADSU's or routers. We do have a working configuration for the 1490/AAL5 protocol stack, and are running load tests on it now.
The first generation switches had a few hundred cells of buffering. This was 2-3 FDDI packets worth. For an IP WAN at any speed, that was a joke. So ATM switches at all usable for IP on a WAN (with long delay paths) and high speed (even 45 Mb/s) have just emerged in the last year as product. Even now, individual cells are tail dropped rather than entire AAL frames and there is no adequate support in current product to cooperate with TCP end-to-end congestion control by dropping complete AAL frames and by implementing a more intellegent drop strategy or congestion indication strategy than tail drop. Sure - the ATMF is discussing backpreasure and credits and other proposals have been made (or are pending). But this does not constitute mature product. In your testing, have you tried setting up two high speed TCP flows (ie: using RFC1323 and a big window) coming in the NAP at two DS3s and going out a third DS3 with a delay equivalent to a cross country path (about 70 msec)? What was the link utilization like?
Bi-lateral/multi-lateral NAP traffic agreements - There's no rules here at all. The agreements are completely up to the NSPs who connect to the NAP. A CIX-like arrangement where all parties agree to exchange traffic w/o settlements is just as possible as N**2 bi-lateral agreements with traffic sensitive settlements. The NAP managers and NSF have nothing to say here.
IMHO - this is not my employers opinion - some guidelines might be in order. I suggest the following (excuse the style here - just trying to be as unambiguous as possible): No National Service provider (NSP) can refuse to accept traffic if the traffic is destined to a network for which that provider is providing primary connectivity and if that traffic is delivered to the NAP nearest the destination, except as noted below. Similarly, no NSP can refuse to forward traffic through their own infrastructure to the NAP closest to the destination if they are providing primary connectivity for the source, except as noted bleow. Primary connectivity for this purpose is the primary connectivity registered with the RA. There are only two exception to these rules. If policy of the destination itslef (not the provider) would exclude delivery of the traffic, the traffic may be dropped. The destination itself (either the final destination or an intermediate provider) may negociate with a high quality provider to have their destination address(es) advertised at equal metric at all NAPs (presumably for a fee) to certain other NSPs or all other NSPs. The purpose of the second case is to allow a client to choose a higher quality provider and then insure that traffic is usually symetric and will generally take the path between NAPs through the higher quality provider. NSF has made it clear that it does not want to be in the regulatory business. But if larger providers attempt unfair settlements something like this might be neccesary. Perhaps some other government entity might end up stepping in. We can only hope that things just work out or that NSF is at least asked for advise before any regulation is attempted. Repeat disclaimer - my employer has not endorsed these comments. It is purely my personal opinion.
As far as taking a long-range view on the Internet, I'll plead guilty. We're already seeing congestion problems on the Internet today, and that's even before we turn the video and audio applications completely loose. My concern is that if we don't get enough capacity into the Internet, and really soon, the growth rate is going to drop badly. I've lived on badly congested Internets (remember the 56 Kb/s NSFNET?), and it's not an experience I wish to repeat. Dave
If you haven't done performance testing considering long delay paths and what might occur when a bottleneck forms, you are not taking a long term view at all. You may not even be adequately considering the present. Throwing backplane or switching fabric bandwidth at the problem doesn't address problems of a bottleneck at an egress. Curtis
IMHO - this is not my employers opinion - some guidelines might be in order. I suggest the following (excuse the style here - just trying to be as unambiguous as possible): . . . . . . NSF has made it clear that it does not want to be in the regulatory business. But if larger providers attempt unfair settlements something like this might be neccesary. Perhaps some other government entity might end up stepping in. We can only hope that things just work out or that NSF is at least asked for advise before any regulation is attempted.
NSF is not contemplating guidelines. We hope the industry will evolve something better than N**2. If predatory practice becomes evident, we'll be happy to arbitrate. If anti-competitive behavior emerges, we'll be happy to remind those disadvantaged of their rights under existing law. -s
whoah!!!!! you've turned this over to industry but if there are predator practices you'll arbitrate (or intervene) if you have left then the FTC, FCC or Justice will "arbitrate" if the government is involved at all best for an industry association to arbitrate problems before they become predatory practice, a predatory practice is a government issue marty
IMHO - this is not my employers opinion - some guidelines might be in order. I suggest the following (excuse the style here - just trying to be as unambiguous as possible): . . . . . . NSF has made it clear that it does not want to be in the regulatory business. But if larger providers attempt unfair settlements something like this might be neccesary. Perhaps some other government entity might end up stepping in. We can only hope that things just work out or that NSF is at least asked for advise before any regulation is attempted.
NSF is not contemplating guidelines. We hope the industry will evolve something better than N**2. If predatory practice becomes evident, we'll be happy to arbitrate. If anti-competitive behavior emerges, we'll be happy to remind those disadvantaged of their rights under existing law.
-s
you've turned this over to industry
but if there are predator practices you'll arbitrate (or intervene)
Sorry, you're right. I meant *** "If asked" *** - and then just on behalf of the U.S. R&E community and their collaborators.
if you have left
then the FTC, FCC or Justice will "arbitrate" if the government is involved at all
best for an industry association to arbitrate problems before they become predatory practice,
Agreed. -s
Dave, I think you still are not understanding my point. NAP's can't do everything for everyone. You have to pick what's important and optimize for that. You can't do everything well. Here's a couple of other things we thought about. FDDI NAP - good short term prospects, tried and true technology. Capa city even more of a problem than SMDS, won't even support a 155 Mb/s vBNS connection. Switched FDDI over ATM helps the bandwidth problem a bit, but I don't really understand how that will extend to a large-scale ne twork. 2 things. FDDI is available on almost everyone's router hardware. It's mature and stable. You can deploy a concentrator based solution, then if capacity gets tight, you install a DEC gigaswitch. This is also a mature product, and has good congested state behavior (degrades gracefully). This sort of switched FDDI solution should provide reasonable capacity at the outset. Next, the IMPORTANT thing to realize is that this transition has to go well. You are talking about the entire operational core of the Internet being changed out. It's better to make things easier up front, get moved to a new architecture while protecting existing service. That means you have to make absolutely sure that reliability is the #1 criteria for an interconnect. The LAN-emulation folks in the Forum have lots of work yet to do with large LAN emulation networks. The equipment for LAN emulation over ATM is i n the same stage of "immaturity" as the IP/ATM equipment. Bletch. Multimedia bridging isn't the answer. The network management issues here are atrocious to deal with. Multimedia NAP - My best reply to this is Peter Ford's statement, "NAP s Don't Route". They're a layer two service. Some layer 2 interworking may be possible in the future, and we're looking at FR<->ATM interworking boxes right now. Agreed. You shouldn't route. And I think multimedia bridging is a very bad idea, at least for this use. I shudder to think about debugging problems because of different MTU settings in different media types. Cheap, low speed connections to the NAP - We got squeezed by product availability here. We know that there is a bigger market in selling T 1 speed connections to the NAPs than T3 and above. However, we had a requirem ent to support multiple T3's initially (remember that the minimum purpose of the NAPs is to support the existing NSFNET backbone for transition purpose s, the NSPs who are providing access to RNP's and who have a requirement to c onnect to all of the priority NAPs, and the vBNS). FDDI and SMDS weren't going to survive for long under this type of load, but ATM doesn't yet have T1 capabili ty (although it's coming). So, we proposed ATM, starting at 45 and 155 M b/s initially, and extending to 1.5 Mb/s and 622 Mb/s as switching equipme nt becomes available. Believe me, we'll be offering T1 speed connections to the SF and Chicago NAPs just as soon as we can make it work. Why offer T1 connections? The primary goal of the NAP's in the first phase of the transition is to support the replacement of 1 core NSP with a few NSP's with minimum disruption. THIS is what should be optimized for. Are you confident that all the management procedures and technologies we are deploying will support a gob of T1 NAP attached AS's? This is highly questionable in my mind, esp. with PVC's that have to be configured by hand. ATM immaturity - ATM itself isn't immature, it's the router and ADSU t echnology that's immature. We've been running ATM in our lab since (believe it or not) 1986, and switch vendors have been shipping equipment for about 3 year s now. I can't believe you said this!! The routers and ADSU's are what your customers care about. You are picking technology that's optimized for your own world view, not that of what your customers can best deal with. Also, most of the ATM switch vendors think they can get away with not learning from the lessons of the router people and build an ATM switch like a voice switch, with only enough buffering to support CBR type applications. When you push these switches even into light congestion, they start dropping cells like mad because there isn't enough buffering in them to allow the remote TCP's enough time to see the congestion building up and flow control their sending rate. This has to deal with some reasonable delay since most of these high speed streams won't be coming from geographically close to the NAP. And when you do drop cells, you won't drop all the cells out of a frame, you'll deliver 99% of them, and all that data will be tossed by the endpoint when it tries to reassemble the frame. Alan Romanow and Sally Floyd did a nice paper on this problem, and proposed a workaround, which no ATM switch vendor has implemented. So whatever problems you have from the previous problem, it's exacerbated by this issue. It's really good you have all that bandwidth to deliver these dead cells. Oh, that's right, your customers are paying for this capacity right? I'm sure your cost scheme gives people credits for incompletely delivered frames, right? We're beginning to see second generation products on the market right now. Since the Toronto IETF, we've been running tests on the SF NAP configu ration, and the problems we've found (and fixed) have all been with the ADSU's or routers. We do have a working configuration for the 1490/AAL5 protoco l stack, and are running load tests on it now. I'd really like to know the exact hardware configurations that were tested and what loads you ran through them. Most of our experiences show bad things happen when you put things under stress. Bi-lateral/multi-lateral NAP traffic agreements - There's no rules her e at all. The agreements are completely up to the NSPs who connect to the NAP. A CIX-like arrangement where all parties agree to exchange traffic w/o settlements is just as possible as N**2 bi-lateral agreements with tra ffic sensitive settlements. The NAP managers and NSF have nothing to say h ere. The issue isn't the policy, the issue is whether or not this is expected to work well with manual configuration of all the VC's, and gobs of NAP attached AS's that you've sold all those T1 connections to. If there are 40 attachees, then do you expect that people are going to establish bilateral relationships with all 39 others? And that the RS will never send them a next hop address that they don't have a VC open to right? Seems to me this all gets more complicated with the larger number of attachees. As far as taking a long-range view on the Internet, I'll plead guilty. We're already seeing congestion problems on the Internet today, and that's e ven before we turn the video and audio applications completely loose. My concern is that if we don't get enough capacity into the Internet, and really soon, the growth rate is going to drop badly. I've lived on badly congested Internets (remember the 56 Kb/s NSFNET?), and it's not an experience I wish to repeat. And what happens if the system doen't work right on day 1? I wonder how long you guys will be given a chance to fix it. You guys are STILL not optimizing for the right things. That is the issue. Now, perhaps it might be useful to describe a little bit about some of these issues are dealt with by the FIXes. Maybe some of this experience is useful in this discussion. First off, the FIXes aren't designed to handle anyone attaching. Too many peers would be a problem for the peering approach used there. However, use of the RS could allieviate many of these problems, but you still have increased network management load from each party that attaches. Next, you start off with a colocated facility for the interconnect. You pick a standard mature media for the interconnect, and all the attachees routers have to use that interface. When you have to evolve to a new media, because of capacity, or some other issue, since the routers are colocated, attachees simply add a new interface to their routers, and then move to peering over the new media, while retaining the older media as backup. Historically, the FIXes started as ethernet based. All the routers there were plugged into a multiport transciever. As traffic grew and AS's added more trunk capacity to their FIX routers, the ethernet got congested, and so more capacity was required. So an FDDI concentrator was attached, and the folks who had the biggest load issues ADDED FDDI interfaces to their routers, and offloaded this traffic onto the ring. Noone deactivated their ethernet interfaces, which allowed the ethernet to be used for backup. Some FIX attachees are more than adequately served by the ethernet, and as long as some AS's are ethernet only connected, EVERYONE has to maintain their ethernet interfaces. Eventually, traffic will grow and the next step will likely be a GIGASWITCH. You could move everyone off the concentrator into the switch, or if everyone has FDDI interfaces available, then maybe you throw away the ethernet, and then have people add a new FDDI interface for the GIGASWITCH, and use the concentrator FDDI for backup. As this gets congested, you then might move to ATM (if reliable and multiple interfaces are available), and use the GIGASWITCH as backup. Etc... Key points: Colocation buys you a lot. It means that folks can support whatever long haul media and speed they like to connect to their router at the FIX. This means that some can connect at T1, or DS-3, or SMDS, or ATM, or whatever. It's THEIR choice, and this can change and evolve over time. Since this trunk is part of the attachees network, they use their own normal network management procedures that they would use to interconnect to a customer site. The interconnect isn't over a long haul link. This greatly reduces the network management requirements for the interconnect, now that the long haul link is connected at both ends by a single AS's routers. Next, since the routers are colocated, evolution can occur gradually. New attachment media can be supported with NO additional long haul trunking required. The ability to support multiple interfaces in the router provides for gradual transitions and evolution, as long as a base interconnect technology is used by everyone. Note that customers who need the extra capacity have the incentive to do the work to add interfaces, and the older technology used as a backup. And as people move off the older technology, capacity is reclaimed. And when everyone has moved to a new technology, then you can toss the old interconnect. This above point is very important. You shouldn't attempt to force people to use the end all technology solution on day one. This is what you are doing, and it's a broken approach, esp. given the importance of these attachements. What you need to do is to build in evolvability, because whatever answer people think is right today will be wrong. Things change too quickly for this. And that evolvability needs to occur with gradual transitions, transitions that can be accomodated by multiple interfaces. You are trying to optimize for everything and in the process, will end up with a design that doesn't do anything well. Toss out some of your goals, but NOT those of reliability, operational manageability (do you guys still not have any SNMP access to the switch's port statistics available to NAP attachees?), and use of mature technology which will be absolutely required to make this transition work well. Then reevaluate your design, and I think you and your customers will all be better off. Does this style of interconnect solve all problems? No. Can you reasonably expect such an colocated facility to house hundreds of attachees? No, but then it's not clear to me that the routing technology that we have will scale to this sort of scale either. So it's not clear to me why this should be a hard requirement. Are any of these advantages derived from the fact that it's a Fed interconnect? No, they apply to any interconnect that uses a similiar architecture, with an RS or not. Also, I should point out that the FIXes aren't in the profit making business. That means that noone is out there trying to get people to attach directly to FIXes when they could adequately be served by connecting to one of the attached NSP's. This naturally results in fewer connections rather than more. I have nothing against profitmaking of course, (I'm a Republican after all), but I think engineering and operational concerns need to take priority over getting the maximum number of people to attach to a given interconnect point. And in the end, if things don't work because you've oversold the system, people will find somewhere else to pass their traffic. This is too important an issue to not get done right. thanks, Milo PS Sorry for the long note.
I totally agree with Milo. The only problem that must be solved on day 1 is *STABLE* migration away from the single NSP model. I have two nightmares: The majority of either the NSPs or NAPs or both fail to pass acceptance because they are unstable at high load. This will prevent or delay migration away from the single NSP model. Worse: we do not discover this until Nov 1st. --------------------- PSC will be vBNS attached early next year. We (and our sister supercomputing centers) have high performance users all over the Internet (plus zero local users). We depend on our NSP, our user's NSP, and the interconnects for our day to day operations. Furthermore we can move a lot of data: e.g. the interface to our file archiver is TCP/IP based and routinely sustains transfers at disk rates (160 Mbit/s as we are currently configured). This is RFC1323 TCP, and in principle, any user (without privileges mind you) can turn the necessary knobs to max out the bottleneck in any path in the Internet, today, tomorrow, and next year. (Modulo various real-world limitations) I predict that stability in the presence of congestion will prove to be far more important than actual pipe size. --MM-- P.S. If you did not see my paper "Windowed Ping: An IP Layer performance Diagnostic" at INET'94/JENC5, get it from the ISOC www server or ftp.psc.edu,pub/net_tools/WPing.ps . This describes our preferred tool for testing connectivity to our customers. I'd prefer that it be used in October, rather than November. --MM-- P.P.S. I never have understood the point of real time... What would you do with a 5 day weather forcast that took 5 days to produce? ;-)
I totally agree with Milo. The only problem that must be solved on day 1 is *STABLE* migration away from the single NSP model.
A worthy goal. [ ... nightmares about transition problems and PSC relies heavily on their network connectivity ... ]
I predict that stability in the presence of congestion will prove to be far more important than actual pipe size.
--MM--
We have been testing RFC1323 TCP under single and multiple flow fully congested conditions. In that respect we can push the test setup beyond what it needs to initially support. It would be a good idea to repeat these tests with the proposed NAP equipment before it gets deployed so we know the limits before we deploy. We can also inject full Internet routes from multiple places (without creating a forwarding path to the Internet) and see if the normal (high) level of route flap affects the proposed NAP routers when forwarding traffic. In the past this has had surprising effects. I don't know how we could generate extremely large aggregations of flows and the diversity of active destinations. This is a known problem for cache architectures that was recently addressed by Cisco, but currently requires that hardware assist be disabled. Although we can't replicate this easily on the testnet we know we need to deploy this (due to operational experience of at least two providers). The testing should therefore be done with this fix in place (which will require disabling some hardware assist in the forwarding path). We do want to make these tests as realistic as possible.
P.S. If you did not see my paper "Windowed Ping: An IP Layer performance Diagnostic" at INET'94/JENC5, get it from the ISOC www server or ftp.psc.edu,pub/net_tools/WPing.ps . This describes our preferred tool for testing connectivity to our customers. I'd prefer that it be used in October, rather than November.
--MM--
Running mping tests would be a good idea. A word of caution though. On the testnet, we've found you need cross traffic of similar average pacvket size and level to get similar results to the real network with mping (with no background you get wrong results, but they look great). While ANS doesn't have any magic, we do have a fair amount of operational experience with a large IP backbone and are as familiar as anyone with the requirements it places on the routers. I do hope Bellcore chooses to take advantage of our offer to help test. Curtis
This is a known problem for cache architectures that was recently addressed by Cisco, but currently requires that hardware assist be disabled. Write again next week. ;-) Tony
Milo S. Medin wrote ...
Cheap, low speed connections to the NAP - We got squeezed by product availability here. We know that there is a bigger market in selling T 1 speed connections to the NAPs than T3 and above. However, we had a requirem ent to support multiple T3's initially (remember that the minimum purpose of the NAPs is to support the existing NSFNET backbone for transition purpose s, the NSPs who are providing access to RNP's and who have a requirement to c onnect to all of the priority NAPs, and the vBNS). FDDI and SMDS weren't going to survive for long under this type of load, but ATM doesn't yet have T1 capabili ty (although it's coming). So, we proposed ATM, starting at 45 and 155 M b/s initially, and extending to 1.5 Mb/s and 622 Mb/s as switching equipme nt becomes available. Believe me, we'll be offering T1 speed connections to the SF and Chicago NAPs just as soon as we can make it work.
Why offer T1 connections? The primary goal of the NAP's in the first phase of the transition is to support the replacement of 1 core NSP with a few NSP's with minimum disruption. THIS is what should be optimized for. Are you confident that all the management procedures and technologies we are deploying will support a gob of T1 NAP attached AS's? This is highly questionable in my mind, esp. with PVC's that have to be configured by hand.
I knew the truth about NAPS would come out. The idea is to squeeze out the small US players and force them to pay the DS3 capable telcos for their access. Is this anti-competative or what! I'm glad I don't operate in the US -----------------------MIME Messages Accepted---------------------- Peter Dawe, Director, Unipalm plc, Email: peter@unipalm.co.uk Managing Director, PIPEX Ltd Voice: +44(0)1223 250100 216 The Science Park, Milton Road Fax: +44(0)1223 250101 Cambridge, England, CB4 4WA WWW: http://www.pipex.net ------------------------------------------------------------------- Government Regulation? The minimum needed to maintain an open and free market for all! -------------------------------------------------------------------
Get a grip Peter. Dave and Milo are arguing from different world views. Both have some valid points and somewhere in the middle there will be something that works. Raising the anti-competative flag is just a bit premature IMHO. The appropriate time is when you are told where you can and can't connect.
I knew the truth about NAPS would come out.
The idea is to squeeze out the small US players and force them to pay the DS3 capable telcos for their access.
Is this anti-competative or what!
I'm glad I don't operate in the US -----------------------MIME Messages Accepted---------------------- Peter Dawe, Director, Unipalm plc, Email: peter@unipalm.co.uk -------------------------------------------------------------------
--bill
Milo S. Medin wrote ... ... I knew the truth about NAPS would come out. I think you are overreacting here. I don't think anyone has been hiding anything intentionally. We're only recently getting enough of the details of how the NAP's are to be implemented to have cogent discussions. The idea is to squeeze out the small US players and force them to pay the DS3 capable telcos for their access. No, the idea here is to transition from a currently operating core network backbone that is central to the global Internet to a system where this functionality is provided by a set of NSPs', and RA, and do this without breaking anything. The issue that I have been trying to address is really that a NAP can't be used as a dessert topping and a floor wax at the same time. You can't optimize for everything. This is a limit of the technology and human factors that we have to deal with today. In the future, this may change, but today, if you try and be all things to all people, you will be poor in almost every aspect of whatever you try and do. Is this anti-competative or what! No, it's not. It's called partitioning the problem space so that you can actually make progress. Look, I'm sure you would be unhappy if you could get a NAP attachement and the performance and reliability was so bad that your user's switched providers because of the poor service they were getting. Besides, why do you think that you will be better off attaching to a NAP rather than attaching directly to an NSP? Do you think that some NSP is obligated to carry your traffic to the other NAP's or NSP's? The general purpose transit network that NSF is providing today is going away. Connecting directly to a NAP isn't going to change that. Remember all those bilateral agreements that people are talking about? Nothing says that people have to accept your traffic for free or even peer with you. This is true whether you are NAP connected or not (except for the issue of NSP's who are contracted with by regionals with NSF money, and then only for R&E traffic). I'm glad I don't operate in the US Well, I think the food is better in the UK, but there are more country music stations here in the US (and cowgirls too! :-)). Thanks, Milo
Milo S. Medin wrote ...
Milo S. Medin wrote ...
...
I knew the truth about NAPS would come out.
I think you are overreacting here.
I've been bringing these points to meetings for close to a year now and I've always got one of three answers, It is a pure US internal problem - IT ISNT We don't know what is going to happen in the end - I can believe it! You'll have to live with it, negotiate with the NSPs - True but I dont want the deck stacked against me The level of uncertainty here, combined with the lack of acknowledgement that there is a REAL problem here makes for what some might call over-reaction!
I don't think anyone has been hiding anything intentionally. We're only recently getting enough of the details of how the NAP's are to be implemented to have cogent discussions.
The idea is to squeeze out the small US players and force them to pay the DS3 capable telcos for their access.
No, the idea here is to transition from a currently operating core network backbone that is central to the global Internet to a system where this functionality is provided by a set of NSPs', and RA, and do this without breaking anything.
The issue that I have been trying to address is really that a NAP can't be used as a dessert topping and a floor wax at the same time. You can't optimize for everything.
You have got to optimise for the diverse character of your customers if you are not to be anti-competative, The Telecos used the - we can only cope with one teleco per a country for a long time!
This is a limit of the technology and human factors that we have to deal with today. In the future, this may change, but today, if you try and be all things to all people, you will be poor in almost every aspect of whatever you try and do.
Is this anti-competative or what!
No, it's not. It's called partitioning the problem space so that you can actually make progress. Look, I'm sure you would be unhappy if you could get a NAP attachement and the performance and reliability was so bad that your user's switched providers because of the poor service they were getting.
One (new) point here is the set-up removes any choice, to have govt-$ you MUST connect to ALL NAPs, to get connectivity to all govt-$ nets you must connect to all NAPs. How about making the NAPs a marketplace on day one! and make govt-$ only subject to connectivity to a (high) percentage of hosts on the Internet. Let the service providers choose their mechanisms, NAPs, GIX, CIX, .... My gut tells me you would end up with maybe two national, one international and many local ( state/city ) interconnects.
Besides, why do you think that you will be better off attaching to a NAP rather than attaching directly to an NSP?
If you make connection at DS3 obligatory, then many do not have a choice! definition of anti-competative, removal of choice
Do you think that some NSP is obligated to carry your traffic to the other NAP's or NSP's? The general purpose transit network that NSF is providing today is going away. Connecting directly to a NAP isn't going to change that. Remember all those bilateral agreements that people are talking about? Nothing says that people have to accept your traffic for free or even peer with you. This is true whether you are NAP connected or not (except for the issue of NSP's who are contracted with by regionals with NSF money, and then only for R&E traffic).
As an International service provider, I expect each nation to pay for their own network. If non-US ISPs have to pay NSPs for NAP connectivity then the US Internet will be subsidised by the rest of the world! Great for the US I guess, but not so good for global networking. Thanks,
Milo
...end Peter Internet:- The right to say what you want to the world, and for them not to listen
Re the NAPs and connectivity. Are there varying degrees of problems? What happens if Unipalm comes into MAE East and peers with everyone there? Including Sprintlink, MCInet, and ANS? Are we hearing that these three would refuse transit to Unipalm's traffic that would need to go to NAPs in order to get to the academic regionals? If such an arrangement would not guarantee Unipalm adequate connectivity, why not? Where would the failure points be and for what reasons? Are we hearing that to get carriage across Sprint, MCI and ANS that Unipalm would have to connect to all 3 NAPs??? Why? I thought connecting to all 3 was necessary only where a net wanted to be eligible for regional connectivity monies. UUNet and PSI will get traffic to all the regionals through MAE East or CIX?? Steve Goldstein's July 19th message to foreign nets (that I understand drew an outstanding flame from Rick Adams) cautioned foreign nets not to take their connectivity for granted. Well let me ask for example what a net like Demos that comes from Moscow into Alternet will have to do to be assured complete connectivity?? Which I guess is saying what will Alternet have to do to guarantee Demos connectivity? Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual
Gordon Cook wrote ...
Re the NAPs and connectivity. Are there varying degrees of problems? What happens if Unipalm comes into MAE East and peers with everyone there? Including Sprintlink, MCInet, and ANS? Are we hearing that these three would refuse transit to Unipalm's traffic that would need to go to NAPs in order to get to the academic regionals? If such an arrangement would not guarantee Unipalm adequate connectivity, why not? Where would the failure points be and for what reasons?
So long as ALL NSPs agree to peer at MAE-East then things should be OK, as no transit is needed because all the regionals are connected by their individual service provider. However there is no REQUIREMENT to peer. I am looking for NSF to make this a requirement (at least for a year or so) to ensure continued ubiquitous connectivity. Alternately, the customers should require their NSPs to agree peering policies constant with this. Unfortunately, there is a danger that some NSP customers are niave and will assume connectivity without contractual requirements.
Are we hearing that to get carriage across Sprint, MCI and ANS that Unipalm would have to connect to all 3 NAPs???
I thought it was 5 NAPS!
Why? I thought connecting to all 3 was necessary only where a net wanted to be eligible for regional connectivity monies.
UUNet and PSI will get traffic to all the regionals through MAE East or CIX??
Steve Goldstein's July 19th message to foreign nets (that I understand drew an outstanding flame from Rick Adams) cautioned foreign nets not to take their connectivity for granted. Well let me ask for example what a net like Demos that comes from Moscow into Alternet will have to do to be assured complete connectivity?? Which I guess is saying what will Alternet have to do to guarantee Demos connectivity?
Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual
Over in Europe we have already had examples of networks having difficulty in maintaining connectivity with large networks asking for 'contribution' towards their network from smaller networks. This is why I keep argueing. I also worry about how few networks are taking part in this discussion! Peter Internet:- The right to say what you want to the world, and for them not to listen
So long as ALL NSPs agree to peer at MAE-East then things should be OK, as no transit is needed because all the regionals are connected by their individual service provider. However there is no REQUIREMENT to peer. I am looking for NSF to make this a requirement (at least for a year or so) to ensure continued ubiquitous connectivity.
It **IS** a requirement of NSF - if you connect to any NAP, but not if you connect to MAE-East, the CIX, the GIX, the SWAB or the Dundalk Consolidated International Three Hundred Baud Interconnect (local reference). The NSF-sponsored NAPs are the only places where we have the right to shape policy. -s
Stephen Wolff wrote ...
So long as ALL NSPs agree to peer at MAE-East then things should be OK, as no transit is needed because all the regionals are connected by their individual service provider. However there is no REQUIREMENT to peer. I am looking for NSF to make this a requirement (at least for a year or so) to ensure continued ubiquitous connectivity.
It **IS** a requirement of NSF - if you connect to any NAP, but not if you connect to MAE-East, the CIX, the GIX, the SWAB or the Dundalk Consolidated International Three Hundred Baud Interconnect (local reference). The NSF-sponsored NAPs are the only places where we have the right to shape policy.
-s
...end Sorry, I can't let you get away with that statement! You can shape peering policy WITHOUT specifying the location of the interconnect. The customer wants connectivity, not NAPs! Why doesn't NSF specify connectivity rather than means? Does NFS want to ensure IT controls the Internet by controling some of the major interconnect? Hell I keep getting more and more paranoid Peter Internet:- The right to say what you want to the world, and for them not to listen
Sorry, I can't let you get away with that statement! You can shape peering policy WITHOUT specifying the location of the interconnect.
No apology needed :-)
The customer wants connectivity, not NAPs! Why doesn't NSF specify connectivity rather than means? Does NFS want to ensure IT controls the Internet by controling some of the major interconnect?
We did a lot of community consulting before settling on the current architecture. It was clear the FTS2000-like solution of another NSFNET Backbone with two or more suppliers was felt to be *too* structured, and the solution of "give the money to the end-user and get out of the way" was too loose for comfort. The NAP/RA/RNP solution had FIX/CIX/MAE-East precedent and, it seemed, just enough structure. NSF hasn't the slightest desire to "control the Internet." If the NAPs aren't useful they won't be used. I should be delighted were the technical community to arrive at a demonstrably better architecture that would be affordable by, and adequately serve, the NSF community. -s
Would you clarify this please Steve. I thought peering had a $ price tag attached to it. But NSF surely can't say that MCI must peer with COOKnet because MCI might say they want $2000 a month to do so and I might say I can nott afford such an "outrageous" price. Are you saying that MCI or XYZ must peer with everyone where peer is defined as accepting traffic for R&E destinations at no cost? To say that peering MUST take place and then leaving the cost of such peering to be negotiated seems of dubious workability.Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual On Wed, 7 Sep 1994, Stephen Wolff wrote:
So long as ALL NSPs agree to peer at MAE-East then things should be OK, as no transit is needed because all the regionals are connected by their individual service provider. However there is no REQUIREMENT to peer. I am looking for NSF to make this a requirement (at least for a year or so) to ensure continued ubiquitous connectivity.
It **IS** a requirement of NSF - if you connect to any NAP, but not if you connect to MAE-East, the CIX, the GIX, the SWAB or the Dundalk Consolidated International Three Hundred Baud Interconnect (local reference). The NSF-sponsored NAPs are the only places where we have the right to shape policy.
-s
Gordon, The regionals are responsible making sure that they provide access and transit to/from the NSFNET NAPs to all of the regional's US R&E customers as part of their cooperative agreements with NSF for RNP support. If they have an agreement with an NSP then presumably their NSP will be encumbered by the regional to meet this condition. Thus, if NSP N is provisioning regional R, then N must touch down at all NAPs and advertise connectivity to all of R's R&E customers and accept traffic to R's R&E customers at the NAPs. If a provider P pays for a connection to the NAP, then it should be able to peer with N for the purposes of getting traffic to/from R's R&E customers. Traffic between N and P for traffic other than R's R&E traffic is not part of the deal. Presumably this "other" traffic would be part of a broader inter-provider agreement between N and P. The goal for the above is to provide a reasonable level of non-discriminatory access to the US R&E community that is being supported by the NSF awards to the RNPs. peter
That is an interesting response. Provider P could be PIPEX right? And that makes it seem to me that I undserstood correctly what Steve Wolff meant when he implied that a foreign net might get better connectivity from a NAP. Yet public an private response to this question have NOT uniformly shared this interpretation. Why? Because I guess folk out their don't clearly understand. Wouldn't it be helpful for Steve Wolff to come out with an official policy statement on this matter, or at least to endore what you just wrote as official policy?? Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual On Wed, 7 Sep 1994, Peter S. Ford wrote:
Gordon,
The regionals are responsible making sure that they provide access and transit to/from the NSFNET NAPs to all of the regional's US R&E customers as part of their cooperative agreements with NSF for RNP support. If they have an agreement with an NSP then presumably their NSP will be encumbered by the regional to meet this condition.
Thus, if NSP N is provisioning regional R, then N must touch down at all NAPs and advertise connectivity to all of R's R&E customers and accept traffic to R's R&E customers at the NAPs. If a provider P pays for a connection to the NAP, then it should be able to peer with N for the purposes of getting traffic to/from R's R&E customers. Traffic between N and P for traffic other than R's R&E traffic is not part of the deal. Presumably this "other" traffic would be part of a broader inter-provider agreement between N and P.
The goal for the above is to provide a reasonable level of non-discriminatory access to the US R&E community that is being supported by the NSF awards to the RNPs.
peter
You are missing the point. NSF has arranged things such that the best way to connect to the U.S. R&E community is to connect to a NAP. Period. Nobody, including NSF, has global responsibility to assure interoperation of the rest of the Internet. Since the NAPs have mechanisms to recover costs from the non-R&E community there is a fighting chance that they will also advance this goal, but it is by no means assured. Many people are working very hard..... This means that it will be easy to get from the U.S. R&E community to anywhere in the rest of the Internet, but interconnection between different parts of the rest of the Internet depend on the non-US and non-R&E providers comming to a consensus regarding sane interconnection, compensation for transit traffic and related pricing and policy issues. This is not on NSF's plate, and they have no reason to comment on it. (It's not on my plate either, but PSC has customers out there.) This situation is a direct consequence of the vast pressure brought on NSF to "get out of the commodity backbone business". People complained when somebody was in charge, so now there is nobody in charge. Take care, --MM-- Assorted follow up: My earlier suggestion about connecting to an NSP could be translated to: "Write a contract with an NSP which provides at least the same services as they are providing to their NSF R&E customers, including transit announcements to all priority NAPs" And the following message came to me from someone who wanted to refine my message, without contradicting it: " Your comment about international providers also seems to apply to geographically compact commercial domestic providers. Thus, connecting to one NAP for a state-wide commercial network would fail, just as it fails for an international provider." Yes, because such a provider could not support traffic to anther provider also connected to only one NAP, without transit agreements from NSPs. --MM--
There have been several questions about MAE-East and the DC NAP. In particular it is asked why the NAP, and the MAE/GIX are separate facilities. I don't see them as separate facilities, instead I see them as 2 virtual networks on top of the very same MFS facilities. Both virtual networks use MFS's ability to offer bridged 802 framed/addressed/switched networks in a distributed manner. In fact, there is no really good reason why they have to be separate virtual networks, since they have the same basic technical requirements and have similar implementation policies. There has been some debate that the problem is that the NSF wants control over the NAP, but there does not appear to be any evidence to this line of argument. NSF would like to see the following policies to be in place for the NAPs: non-discriminatory access to the NAP for Internet Service Providers NSPs serving RNPs that have NSF RNP awards must advertise routes to and accept traffic to/from the RNP's US R&E customers. Bandwidth and service to scale over time It does not appear that this runs counter to the policies of the MAE/GIX. An obvious result of deliberation on the issues at hand would be to simply relabel the things as MAE/GIX/NAP and be done with it. Any such arrangement would of course be contingent on an agreement between the entity that has contracted with MFS to provide MAE-East, and the entity (NSF) who have a cooperative agreement with MFS to furnish the DC NAP service. NSF would welcome the opportunity to open talks that might lead to such an agreement. This line of discussion naturally leads into the issues of "... what about the CIX and California NAP, they are both on PacBell facilities, etc. ...". Here the issues are not so clear as in the MFS case since the PacBell NAP is based on ATM technology and the CIX has elected to use SMDS as their service. This does not preclude interconnecting the CIX with the PacBell NAP. One could imagine a CIX router getting an ATM interface and connecting to the PacBell ATM service that the PacBell NAP is built on. The NSPs that are connected to the ATM service could then *ALSO* use ATM to access each other, and *ALSO* use the ATM service to access other CIX members. Over time it is likely that SMDS will simply be another type of frame that is carried over ATM and that you will see: Router Router SMDS SMDS ATM Interface V.35/HSSI | | | SMDS DSU/CSU | | | | ATM Switch ======== SMDS Switch ATM Link This would result in the same kind of underlying facility merger you see today for the bridged 802 based services that MFS offers. I fail to see the point of maintaining separation of the CIX and PacBell NAP that appears to be the focus of many in these discussions. Interconnecting the two services (NAP and CIX) could be very useful in simplifying the issues of Internet interconnection while maintaining the CIX and NAP policy objectives. cheers, peter
There have been several questions about MAE-East and the DC NAP. In particular it is asked why the NAP, and the MAE/GIX are separate facilities. I don't see them as separate facilities, instead I see them as 2 virtual networks on top of the very same MFS facilities.
Yeah, well, that's nice, but they aren't. -a
Perhaps I'm being a little defensive..
An obvious result of deliberation on the issues at hand would be to simply relabel the things as MAE/GIX/NAP and be done with it.
Any such arrangement would of course be contingent on an agreement between the entity that has contracted with MFS to provide MAE-East, and the entity (NSF) who have a cooperative agreement with MFS to furnish the DC NAP service. NSF would welcome the opportunity to open talks that might lead to such an agreement.
"Hi, I'm from the government, and I'm here to help you." So, other than confusing the issue, exactly how do I, as an existing MAE-East participant, win from this arrangement? Mind you, I don't speak for "MAE-East", but MAE-East connects ISPs, and the NSF isn't an ISP, right? We could talk about letting an RA connect to MAE-East, but they're not an ISP, either. I'm sure MFS would be just as happy to take your money to provide MAE-East service as they would NAP service. I'm sure their interest is in keeping their customers happy, and not trying to reduce the number of products that they offer. On some other points, the technology used to deliver MAE-East+ is significantly different than what MFS proposed for the NAP, and the service is structured differently. The MAE-East+ technology is somewhat more conservative in its approach. It also encourages private provider/provider connectivity since the high-bandwidth participants will be colocated in adjacent racks. That's not to say that the technology used to deliver a NAP won't work; Alternet has actually tested the hardware because we may end up using it for other applications. Mostly its the KISS principal at work. Other than that, both the NAP and MAE-East do move level-2 frames around. Are there providers that wouldn't connect to MAE-East because it's missing the blessed "NAP" moniker? If so, the political forces at work inside that provider are eclipsing the technical issues, and I'd think twice about connecting my net to them. Their job is to move the bits around, first and foremost. A more interesting question is to find out which carriers are connecting to all 3 mandatory NAPs, and to find out if they're already connected to MAE-East. If they are, then what's left to do? Other than slather on "NAP" baggage that we don't want or need? Since these interconnect points already exist, and a so similar to the NAPs, why are the NAPs required at all since private industry has already gone off and solved this problem without any help. "Stop trying to help me. It's not broken, and doesn't need to be fixed." If there is some clear gain in somehow combining these two things, please let me know. I don't think that an RA qualifies as a "gain" since we have an existance proof that we can get along without one of those already. A more interesting prospect to consider is hooking together MAE-East and, say, the FIX. That *might* actually help. Louis A. Mamakos louie@alter.net Backbone Architecture & Engineering Guy uunet!louie AlterNet / UUNET Technologies, Inc. 3110 Fairview Park Drive., Suite 570 Voice: +1 703 204 8023 Falls Church, Va 22042 Fax: +1 703 204 8001
I read your comments Louis and I must say that I agree. Being the new kid on the block and still headed towards the building (Oct 1) planned date. I decided to have NET99 connect to MAE-E because I wanted away from the NAP's. I have zero interest in NSF meeting me there. Louis makes mention of the MAE & Fix connecting, I support that all the way. Watch out , I just agreed with Alternet ! Joseph Stroup
"Stop trying to help me. It's not broken, and doesn't need to be fixed."
If there is some clear gain in somehow combining these two things, please let me know. I don't think that an RA qualifies as a "gain" since we have an existance proof that we can get along without one of those already. A more interesting prospect to consider is hooking together MAE-East and, say, the FIX. That *might* actually help.
Ah comeon Louie, you really, REALLY want this vanilla ice-cream cone that I have for you! Actually, I think the RA (in the context of coordinated routing registries and route servers) does provide a gain. Virtually all the RIPE papers (ripe81 and derivitives) indicate that a common route registry will become a critical component in the internet. (RIPE has had a functioning RS @ mae-east for a number of years now) The use of such services has several advantages, including reducing the number of peers that you MUST peer with. None of this precludes your ability to set up as many bilateral arraingments as you as an ISP, feel you need. I would expect that ISPs would begin to realize that they function as a route registration service today, even if they don't run a database behind it. When ISPs have a conflict, they can negotiate the answer between them or they can appeal to a neutral, third party. Kind of like giveing your customers a choice... "I have your best interests at heart" or "Here, go ask a neutral party and see if I'm right". I guess that the "existance proof" argument can also be used for why we don't need CIDR or IPv6. We have things that work now, why do something new? -- --bill
Bill, Perhaps I left the wrong impression in my comments. The only reason that I mentioned the RA is that it's the only component of the NAP technology which actually might help; the question is if it connects to MAE-East or not. Honestly, I wasn't thinking of the RIPE route server on MAE-East - funny that function is duplicated to some extent as well. I didn't mean to throw out the baby with the bathwater.. louie
Actually, I think the RA (in the context of coordinated routing registries and route servers) does provide a gain. Virtually all the RIPE papers (ripe81 and derivitives) indicate that a common route registry will become a critical component in the internet. (RIPE has had a functioning RS @ mae-east for a number of years now) The use of such services has several advantages, including reducing the number of peers that you MUST peer with. None of this precludes your ability to set up as many bilateral arraingments as you as an ISP, feel you need.
Please be careful about mixing route registries and route servers. I think each serves different purposes & different arguements can be made for the usefulness of each. I presonally am not yet sure of the usefulness of route servers. However I do think that route registries are very useful. --asp@uunet.uu.net (Andrew Partan)
Actually, I think the RA (in the context of coordinated routing registries and route servers) does provide a gain. Virtually all the RIPE papers (ripe81 and derivitives) indicate that a common route registry will become a critical component in the internet. (RIPE has had a functioning RS @ mae-east for a number of years now) The use of such services has several advantages, including reducing the number of peers that you MUST peer with. None of this precludes your ability to set up as many bilateral arraingments as you as an ISP, feel you need.
Please be careful about mixing route registries and route servers. I think each serves different purposes & different arguements can be made for the usefulness of each.
I presonally am not yet sure of the usefulness of route servers. However I do think that route registries are very useful. --asp@uunet.uu.net (Andrew Partan)
I tried to keep it clear, although it can be very confusing. As a point of clarification, is the RIPE engine at mae-e a registry or server? (feel free to jump right in here martin/tony) -- --bill
I tried to keep it clear, although it can be very confusing. As a point of clarification, is the RIPE engine at mae-e a registry or server? (feel free to jump right in here martin/tony)
Which RIPE engine? I think that there may be two - a whois server (which runs off of the RIPE database) and a route server. Both of these are actually applications of the registry - the whois server is a way of querying the registry; the route server is a more specialized application. [I think that the RIPE folks are off at a meeting this week...] --asp@uunet.uu.net (Andrew Partan)
I had not remembered that there were two RIPE machines @ mae-e. If one is a server, do you know if anyone is utilizing it? --bill
asp@uunet.uu.net (Andrew Partan) writes: * > I tried to keep it clear, although it can be very confusing. As a point * > of clarification, is the RIPE engine at mae-e a registry or server? * > (feel free to jump right in here martin/tony) * The machine on mae-east is a route server. It provides next_HOP for all registered European networks. The only use operatioanlly is for backup for NORDUNET with respect to providing next_hop to ENSS 136 right now. Of course any one is free to make use of it. It is fully operational and currently peers with all Mae providers (except Netcomm - only reason being didn't get done). It is basically the "simple" RS. If you need more info on this lets take it offline and send me mail direct rather than burden lists galore. * Which RIPE engine? I think that there may be two - a whois server * (which runs off of the RIPE database) and a route server. * THe Registry box we have in the US is not quite on the MAE (almost though). It is on the visitor net at Alternet - one hop away I guess. * Both of these are actually applications of the registry - the whois * server is a way of querying the registry; the route server is a more * specialized application. * Right ,-) * [I think that the RIPE folks are off at a meeting this week...] Yep at the RIPE meetiong in Portugal ;-) * --asp@uunet.uu.net (Andrew Partan) --Tony
asp@uunet.uu.net (Andrew Partan) writes:
Please be careful about mixing route registries and route servers. I think each serves different purposes & different arguments can be made for the usefulness of each.
I personally am not yet sure of the usefulness of route servers. However I do think that route registries are very useful.
Indeed there appears to be (too) much confusion about those two. Definitely the Routing Registry is the more important and much more general concept. Without the RR it would be difficult to impossible to configure an operational Route Server. Maybe a few informal (my personal view) definitions help: Routing Registry ---------------- Definition: A neutral repository where ISPs can register their external routing policies for retrieval by everyone needing this information. Problem addressed: The cross product of all external routing policies represents the global Internet routing policy. Without knowing a big part of this, configuration and operation of an Internet with arbitrary topology is currently impossible. Implementation: A database with a suitable schema and an Internet retrieval method. The minimal schema must represent autonomous systems, the routes being originated by those ASes and the routing information exchanged between ASes. Uses: Very many and general: Helps ISPs to configure their external routing and to diagnose unexpected routing behavior. Actually it is the *only* way to tell what the *expected* routing behavior is in very many cases. Route Server ------------ Definition: An external routing peer located at an Internet exchange point that combines dynamic routing announcements with routing policy and provides the result to its users. It does not forward any packets. There can be multiple route servers using different policies. Single route servers using multiple policies have been proposed. Use of route servers is optional. They can be used in addition to direct peerings. Problem addressed: n**2 peering problem at exchange points. ISPs can peer with just the route server and do not have to peer with any of the ISPs that the route server combines information for. This solves router capacity problems as the route server keeps many more paths, peers and isolates non significant routing changes from its users. It also reduces routing configuration maintenance. Implementation: A workstation with routing software. Most commonly (euphemism!) a Unix box with gated. Configuration is derived (partly) from Routing Registry. Uses: specialised, see above. Routing Arbiter --------------- Definition: Awardee of a NSF cooperative agreement operating a Routing Registry and Route servers at the NAPs. Problem Addressed: see above Implementation: see above Uses: server those connected at the NAPs
Louie, As stated in my earlier note, NSF's goal is to obtain NAP functionality. This functionality is technology independent. The whole purpose of the note was to point out that the desired functionality can be met by taking advantage of an existing facility. Thus, an ISP who wanted to check off that they were meeting the NAP functionality that NSF was requesting could do so by saying they were doing so in part by being connected to MAE-east. This is the clear gain that you were asking for: simplification for some of the ISPs. Simply using existing existing interconnection facilities rather than "building" is advantageous to NSF. This seems to be consistent with what you are saying. Steve Wolff has said that the NSF should be using existing industry built facilities instead of growing them in numerous forums. I (speaking for myself) agree with you that there is no reason to build something if another facility meets the requirements. However, at the time the NSF solicitation was run there was not a facility for interconnecting ISPs at the data rates needed. And there was a lot of rhetoric that DS-3 rates and above did not make sense. With time circumstances should, and do, change. Now that there is a facility for ISP interconnection at DS-3 rates, it seems prudent for NSF to consider MAE-east inter-connectivity as meeting NAP requirements. Since it appears the act of putting a NAP label on MAE-east does not seem to have an impact on the functioning of MAE-east, is there any reason not to do so? thanks for your note, peter
Peter Ford writes that a network service provider could show that they are partially meeting their NAP responsibilities by connecting to MAE East and presumably treating MAE east as the Washington DC NAP. I don't understand why this is an accurate statement. For the service frovides are responsible for connecting to the new Jersey, california, and chicago NAPs.....NOT to washington which is NOT a priority NAP. Peter then adds: Now that there is a facility for ISP interconnection at DS-3 rates, it seems prudent for NSF to consider MAE-east inter-connectivity as meeting NAP requirements. Wow! Is that an interesting statement! With much fanfare NSF has announced that Sprint, MFS Ameritech and PAC Bell will build NAPs which would be fully operation on October 31. What about these FOUR facilities that have nothing to do with MAE East?? Why not use them, Peter? *UNLESS* they are unusable before the second half of next year?? Is THAT the problem? From reading Milo's posts to Dave Sincoskie and the Nap-info list it sure looks like this could well be the problem. So the NSF to save face wants to call MAE-East an NSF NAP? And Steve wolff is making statements that the feds shouldn't build facilities that private industry can do better!? Your note reads like a policy trial balloon on behalf of Steve. Whew! how the world can change! Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual
Peter Ford writes that a network service provider could show that they are partially meeting their NAP responsibilities by connecting to MAE East and presumably treating MAE east as the Washington DC NAP.
Gordon, The exact quote is: "Thus, an ISP who wanted to check off that they were meeting the NAP functionality that NSF was requesting could do so by saying they were doing so in part by being connected to MAE-east." Please note the "in part ..."
I don't understand why this is an accurate statement. For the service frovides are responsible for connecting to the new Jersey, california, and chicago NAPs.....NOT to washington which is NOT a priority NAP.
Peter then adds: Now that there is a facility for ISP interconnection at DS-3 rates, it seems prudent for NSF to consider MAE-east inter-connectivity as meeting NAP requirements.
Wow! Is that an interesting statement! With much fanfare NSF has announced that Sprint, MFS Ameritech and PAC Bell will build NAPs which would be fully operation on October 31.
Gordon, I am sure you have written more about the NSF NAP awards than NSF has. :-)
What about these FOUR facilities that have nothing to do with MAE East?? Why not use them, Peter? *UNLESS* they are unusable before the second half of next year?? Is THAT the problem? From reading Milo's posts to Dave Sincoskie and the Nap-info list it sure looks like this could well be the problem.
Gordon, I (speaking for myself) am saying that there is no reason for there to be a separation in the facitilies used for interconnection and that it is simple/trivial to make them the same in the case of DC and California. The net effect on the California and DC NAP awardees appears to be minimal and the ISPs that are to connect to the NAPs save themselves interconnection costs. This in turn should reduce the cost to the people to whom NSF awards funds for Internet connectivity. Do you find fault with this line of reasoning? It may be easier to believe in a conspiracy, but you might also want to look for economic drivers.
So the NSF to save face wants to call MAE-East an NSF NAP?
Gordon, The NSF panel recommended the award of a DC based NAP to MFS with the belief that they would co-evolve to being the same thing. I can make no claim of original thought for the implementation of a NAP being the same thing that other NSPs use for interconnection.
And Steve wolff is making statements that the feds shouldn't build facilities that private industry can do better!?
Gordon, another misquote. Is there any way we can get you to use a mail system that would allow you to cut and paste? peter
Peter reminds me that he wrote Thus, an ISP who wanted to check off that they were meeting the NAP functionality that NSF was requesting could do so by saying they were doing so in part by being connected to MAE- east saying Please note the "in part ..." Peter I did what you ask -- saying you said that a network service provider could show that they are partially meeting their NAP responsibilities by connecting to MAE East and presumably treating MAE east as the Washington DC NAP. Please note the phrase "are **partially** meeting" You wrote: Steve Wolff has said that the NSF should be using existing industry built facilities instead of growing them in numerous forums. I paraphased from memory: "wolff is making statements that the feds shouldn't build facilities that private industry can do better!?" You seem say this is another misquote....sorry ....again. I don't see it that way at all and with the two side by side readers may judge for themselves. you say: the net effect on the California and DC NAP awardees appears to be minimal and the ISPs that are to connect to the NAPs save themselves interconnection costs. This in turn should reduce the cost to the people to whom NSF awards funds for Internet connectivity. Do you find fault with this line of reasoning? It may be easier to believe in a conspiracy, but you might also want to look for economic drivers. Let me say that I endorse Louie's answer as to why this doesn't work as you suggest...... and that your post still sounds to me like a trial balloon for an NSF policy change on the NAPs....having said this I be glad to return to lurking. Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual
Let me say that I endorse Louie's answer as to why this doesn't work as you suggest...... and that your post still sounds to me like a trial balloon for an NSF policy change on the NAPs....having said this I be glad to return to lurking.
Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual
No, Gordon; I'm glad you've recovered from thinking everything is a conspiracy, but this one's not even a trial balloon. -s
As stated in my earlier note, NSF's goal is to obtain NAP functionality. This functionality is technology independent. The whole purpose of the note was to point out that the desired functionality can be met by taking advantage of an existing facility.
But it really is differnt than a NAP. There is policy stuff stuck to a "NAP". MAE-East has no requirement for traffic statistics reporting to the NSF on a periodic basis. Do we want them to? I don't know; personally, I don't think its their business. I don't want to have MFS have to do this sort of stuff.
Thus, an ISP who wanted to check off that they were meeting the NAP functionality that NSF was requesting could do so by saying they were doing so in part by being connected to MAE-east. This is the clear gain that you were asking for: simplification for some of the ISPs.
Sorry, I still don't get it. How is this simpler for *me*. I don't have a compelling need to "check-off" anything. I don't see how this simpler for any of the existing MAE-East participants, either.
Since it appears the act of putting a NAP label on MAE-east does not seem to have an impact on the functioning of MAE-east, is there any reason not to do so?
Shall we just get down to it: it's as much an emotinal issue as anything. MAE-East was built almost in spite of the the existing ANS/NSFNET NSF-sponsored network. Any now they want to come along to a facility which "we" built already, which has been a popular success and model of inter-ISP cooperation and burden it with this government label which none of us seeks. And then hold it up as a successful implementation of the network architecture proposed by the NSF; it would be a farce. NSF threw a party in Washington DC called the NAP, and nobody came. Please let us be. The reason not label it a NAP is because some of us just don't WANT you to. It's our party. If other MAE-East party-goers, er, particpants have a different opinion, I'd be happy to hear it. Peter, at this point you probably should post a polite note to the mae-east mailing list to see what other think about this harmless idea of yours; I don't know how many of them are on this list. Louis A. Mamakos louie@alter.net Backbone Architecture & Engineering Guy uunet!louie AlterNet / UUNET Technologies, Inc. 3110 Fairview Park Drive., Suite 570 Voice: +1 703 204 8023 Falls Church, Va 22042 Fax: +1 703 204 8001
to get connectivity to all govt-$ nets you must connect to all NAPs.
Peter, This is not true. The way the NSF awards to the RNPs is written, for PIPEX, or any other network, to get to an US R&E customer of the regional, it is sufficient for you to connect to only one NSFNET NAP. cheers, peter
Would someone please explain why connecting to MAE East would NOT be as good as connecting to a NAP in the case that Peter ford mentions below? Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual On Mon, 5 Sep 1994, Peter S. Ford wrote:
to get connectivity to all govt-$ nets you must connect to all NAPs.
Peter,
This is not true.
The way the NSF awards to the RNPs is written, for PIPEX, or any other network, to get to an US R&E customer of the regional, it is sufficient for you to connect to only one NSFNET NAP.
cheers, peter
Would someone please explain why connecting to MAE East would NOT be as good as connecting to a NAP in the case that Peter ford mentions below?
Gordon: No reason, I think. Providers will make individual decisions, based on their own judgement, guided by technical, financial, and political considerations. I happen to think that the NAPs are a fine thing, be it for primary interconnect, or as a contingency. That should not prevent people to find alternate means of interconnections, if they feel they are more sound for whatever reson they may have. If you were a hard-nosed non-opinionated regional/non-US/whatever service provider, just out to make money with your clients and needing interconnectivity, what would your priorities be? Stability? Cost? Need to reach above the 95th percentile of the community? Reputation? What would be your model and priorities of a good interconnect service provider in such a situation? If you have that model, you could create a matrix and make a solid decision, I would think. Hans-Werner
Thanks for an interesting reply. I have gotten some *private* replies that state the preference for a NAP connect as coming from the pledge of the NSPs getting interregional money to peer with and accept traffic from ALL networks having traffic for the regionals.... NOT just those getting interregional money. Is this REALLY correct? The conditions IMPLY that the peering will take place WITHOUT the foreign networks having to pay the NSPs to do so. If so where do you draw the line? Can't anyone say that they have SOME R&E traffic therefore you must peer with me for no charge. OR does the current state of routing PERMIT the other NSPs to accept from me ONLY packets bound for regionals? Finally, if I come into MAE EAST and peer with the NSPs there, is there ANY reason to believe they would NOT send my traffic on to the regionals?? If not I hope someone can help me understand WHY some have been saying that a NAP connect would guarantee better connectivity for foreign nets? Gordon Cook, Editor Publisher: COOK Report on Internet -> NREN 431 Greenway Ave, Ewing, NJ 08618 USA NEW E-mail: cook@mcs.com Subscriptions: $500 corporate site license; $175 edu.,non-profit & small corp. $85 Individual On Mon, 5 Sep 1994, Hans-Werner Braun wrote:
Would someone please explain why connecting to MAE East would NOT be as good as connecting to a NAP in the case that Peter ford mentions below?
Gordon:
No reason, I think. Providers will make individual decisions, based on their own judgement, guided by technical, financial, and political considerations. I happen to think that the NAPs are a fine thing, be it for primary interconnect, or as a contingency. That should not prevent people to find alternate means of interconnections, if they feel they are more sound for whatever reson they may have. If you were a hard-nosed non-opinionated regional/non-US/whatever service provider, just out to make money with your clients and needing interconnectivity, what would your priorities be? Stability? Cost? Need to reach above the 95th percentile of the community? Reputation? What would be your model and priorities of a good interconnect service provider in such a situation? If you have that model, you could create a matrix and make a solid decision, I would think.
Hans-Werner
I think people are missing the point: If you are connected to a NAP, MAE-east, or many other interconnects, you are golden as far as domestic US connectivity. (Assuming your choice of interconnect doesn't melt in the process) The problem, for non-US providers, is that trans-US connectivity is not assured. In fact, quite the contrary. A link from Europe to an east coast interconnect could easily support traffic between Europe and all customers of all US NSPs. The problem is that unless the European provider has an explicit agreement with some US NSP, none of the NSP's will announce the European networks to the west coast interconnect necessary for the routes to reach the Pacific rim. In fact any NSP that "leaks" such routes without a contract will be providing an un-funded service to the European provider. Since the "explicit agreement" means money, all forign providers are unhappy. --MM-- P.S. Not supporting T1 NAP connections is not anti-competitive, because a T1 sized provider is far below economic scale anyhow. These days, many medium sized colleges have outgrown T1. Small regionals (1 state) are likely to be pushing Ethernet aggregate rates. You can not possibly support enough customers on a T1 to finance the coast-to-coast span. P.P.S. If I were a forign provider I would strongly consider connecting directly to an NSP, as though I were a domestic customer. The NSP can provide whatever B.W. you want, announce you to all interconnects, and deal with debugging this mess..., er I mean assuring proper operation and quality of service. I should say that I think that all of this is good, and in the end, the Internet will be better for it. Take care, --MM--
to get connectivity to all govt-$ nets you must connect to all NAPs.
Peter,
This is not true.
The way the NSF awards to the RNPs is written, for PIPEX, or any other network, to get to an US R&E customer of the regional, it is sufficient for you to connect to only one NSFNET NAP.
cheers, peter
To extend on Peter's cheers, the new awards are for a transitionary phase where the Internet is becoming more and more of a commercial service offering. Reliability and integrity and scale of the Internet is so ways different from where it was at the time the NSFNET started, but there is also a significant dependence on the network by the NSFNET clientele, specifically the R&E community. As NSF is phasing out direct subsidy for a national interconnect network, and for the time being and at declining contribution over the next few years gives the next level of networks (the regional/mid-levels) the ability to acquire interconnect services on their own, NSF perceives an obligation for the network not to get compartmentalized over the forseeable future. As such, as Peter said, the way the NSF awards to the RNPs are written, their interconnect service providers have to all connect to multiple specific neutral interconnect points, to ensure the weave is being held together, until the network matures even further. The NAP service providers, as long as they conform to their agreement with the NSF, are free to expand the NAPs and attract more clients. Those clients can connect to any or all NAPs, and there is no reason some buddies and their dog cannot go off and build just another NAP, by whatever name they choose to give it. It is just a network interconnect point, after all, they had names like FEBA, FIX, CIX, GIX, MAE, and so on before, and will have new names in the future. Things are evolving, and that is just fine, as long as we can keep some measure of integrity of the network. As important as the NSFNET may have been, pioneers having shown the way that a commercial service is viable, such as PSI and Alternet, should be cheered, as I believe that it dramatically improved the long term survival chance of the network, and a path for a graceful transition away from federal subsidy, while retaining stability. Hans-Werner
One correction: they STILL have names like CIX and MAE-EAST. those two are not going away any time soon, NAPs not withstanding. -mo
One (new) point here is the set-up removes any choice, to have govt-$ you MUST connect to ALL NAPs, to get connectivity to all govt-$ nets you must connect to all NAPs. How about making the NAPs a marketplace on day one! and make govt-$ only subject to connectivity to a (high) percentage of hosts on the Internet. Let the service providers choose their mechanisms, NAPs, GIX, CIX, ....
To get connectivity to the U.S. academic community you DO NOT have to connect to all NAPs. Probably not even one, if you prefer. The change is that you cannot blindly fling all your R&E traffic at a U.S. Government-supported entity. THE U.S. R&E community is transitioning to a state in which it purchases its commodity-level data networking in the same way it acquires other commodities - from the marketplace; they're going to start buying their groceries at the supermarket just like everyone else since we're closing the government commissary.
As an International service provider, I expect each nation to pay for their own network. If non-US ISPs have to pay NSPs for NAP connectivity then the US Internet will be subsidised by the rest of the world! Great for the US I guess, but not so good for global networking.
You don't pay an NSP for NAP connectivity; you pay the NAP for NAP connectivity. If an NSP proposes to charge you for carrying your traffic to their customers, you have the option of making the reciprocal offer. -s
I knew the truth about NAPS would come out.
The idea is to squeeze out the small US players and force them to pay the DS3 capable telcos for their access.
Is this anti-competative or what!
I'm glad I don't operate in the US
Peter, It was never the intent for the NAPs to cover all inter-connectivity requirements. When the NAPs were first being considered it was assumed from day one that the CIX and other interconnects would continue to exist. As such, there is not a requirement for "small US players" to directly connect to a NAP. Many have pointed out that there are alternate means to get the connectivity they need, and alternatively it appears that many of the NAPs offer lower than DS-3 speed connectivity. Given your premises are not quite sound, it would appear that "Is this anti-competative or what!" can be answered to the negative. cheers, peter
Dave, I haven't seen a followup on my reply back to you, but I've been doing some more thinking along these lines, and talking with several other people, and there are a few other points for the colocation method that I think also should be brought up. One other advantage is that if the NAP is located in a telco CO that has IXC tenants, it should be possible to have IXC lines terminated in routers managed by the NSP's without any LEC loop at all. This will reduce the cost of attachments, in some cases such as DS-3 level attaches, by a significant amount, since just a cross connect would have to be run. This would improve the network management aspects by elimination of an extra organization in the span, esp in cases where an IXC fast packet service is being used for inter-LATA transport. For SONET interconnects at OC-3c, this would also simplify timing issues, since the NAP connection wouldn't require synchronization between IXC and LEC timing sources (though this isn't an issue for DS-3 and T1 links). Also, because no LEC fast packet service would be required, the time to implement the NAP should be shorter, since no new technology and network management services would have to be provided by the LEC. This would allow decoupling of LEC fast-packet services from NAP service, and allow each to move on their appropriate timeframes. This would allow you to accelerate NAP implementation and testing schedules. Obviously, some NSP's might want to take advantage of such services, but they could start off with simple leased lines and might to LEC fast packet services in a gradual fashion, based on cost/reliability tradeoffs. One more advantage, which I understand may not be a good thing from your viewpoint, is that CAP's and other bypass carriers would also have a shot at providing access to the Chicago and SF NAP's, which essentially require the use of LEC loop and fast-packet services now. This would encourage competition and assure lower prices to NAP users, and potentially provide access to advanced services on a faster timeframe than the normal PUC tarriff process allows. Obviously, this is something that you may view differently than your customers, but I still think it's a valid point. This is all consistent with the idea of Keeping It Simple Stupid (KISS), and allow tighter focus on the primary goal of transitioning away from the central backbone provisioning of connectivity between the regionals to the provision of this service through a distributed set of NSP's in a timely and very reliable manner. Again, I feel any aspect of NAP design and provision needs to be examined against this concise goal. Thanks, Milo
participants (19)
-
asp＠uunet.uu.net -
bmanning＠ISI.EDU -
curtis＠wawa.ans.net -
Daniel Karrenberg -
Gordon Cook -
hwb＠upeksa.sdsc.edu -
Joseph W. Stroup -
Louis A. Mamakos -
Martin Lee Schoffstall -
Matt Mathis -
Mike O'Dell -
Milo S. Medin -
nabil＠i.net -
Peter Dawe -
Peter S. Ford -
sincos＠gigabit.bellcore.com -
Stephen Wolff -
Tony Bates -
Tony Li