September 1997 - Test - lists.nanog.org

Re: not rewriting next-hop, pointing default, ...
by Sean M. Doran 11 Sep '97

11 Sep '97

rja(a)corp.home.net (Ran Atkinson) writes: > Letting LSR in two hops lets outsiders trying to debug (e.g. some > routing problem) at least perform first-level fault-isolation > so they know whose NOC to call for further debugging assistance. > So I view the 2-hop notion as an attempt at "reasonable > compromise". Cool, and I now view the 2-hop notion as the first reasonable argument for encouraging people to totally flatten their network into a full mesh. Security policy should not under any circumstances prevent the Internet as a whole from functioning reasonably well, scaling decently, or make discovering and diagnosing problems any harder than it already is. Your opinion may vary with mine, but I am solidly in line with Randy's suggestion that enabling LSRR on backbone routers should be a requirement for peering. (This is not surprising as I used to require it of a couple of peers in a previous life, because in practice it is unfortunately an irreplaceable diagnostic tool). Sean.

1 0

Re: too many routes
by Sean M. Doran 11 Sep '97

11 Sep '97

"Joseph T. Klein" <jtk(a)titania.net> writes: > Having hopelessly screwed up my facts ... I was trying to make a point here. > So the router was worse than I thought. Retaing policies that exclude > new players because of AGS+'s inability to handle large routing flaps > just does not cut it. > > Sprint imposed this at a time when 7000s with 64M of memory where available. > Will /19 remain policy when the majors are running with Cisco 12000 and GRFs? I am not sure what point you are trying to make here. If you feel like grinding this axe again, I am more than willing to play in my fleeting spare moments. An AGS+ with a CSC/4 has exactly the same CPU as a 7000 with an RP. In fact, AGS+ performance is slightly higher in some cases because of some interesting design features of the 7000 and the other AGS+ downgrade path routers (notably the 7500 series). I did not put in a filter on /18s (yes it was /18s initially, and got changed to /19s after much discussion with the registries, especially Daniel Karrenberg at RIPE, in an attempt to harmonize Sprint's filters with slow-start allocation policies) because of the AGS+ difficulties; all the routers that were carrying full routing at the time had 64Mbytes of RAM, and the two remaining AGS+es were there to implement historical things done principally for ICM (like a STUN connection and the PANAMSAT router). What triggered the filter was the observation that in the blocks freshly allocated by all three registries were very poorly aggregated. More annoyingly, those allocated to Sprint's principal peers (most notably Internet MCI) demonstrated the worst aggregation; in one case a /14 was announced almost exclusively as prefixes no shorter than 19 bits. After spending some time trying to chase this down -- with some success, as in the case of PSI's then newest blocks, but not in the case of Internet MCI, who did nothing -- I decided to issue a warning that once the /8 that the InterNIC was using had filled up (and after some discussion, once RIPE and APNIC proceeded to allocate from new /8s), I would begin filtering all new unicast addresses to ignore Sprintward announcements of any prefix longer than 18 bits. Moreover, I also announced that I would filter out any subnets of historically classful As and Bs. The warning was several months old when people started noticing that they couldn't reach things behind Sprintlink, and alot of time was spent explaining to people that this shouldn't have surprised them at all. Some changes happened, notably I dropped down to 19 bits, the registries began to explain to people that anything longer than that almost certainly would not be routable, and that allocation != routing. This measurably flattened the growth curve of the number of prefixes seen by default-free routers, changing it from a nearly exponential function to a linear one, with the slope below that of Moore's law. In other words, it probably as much as the initial introduction of supernetting as a concept acted to keep the Internet scalable while it continued to use the current set of routing protocols. > If aggregation is the goal then mechanisms should be developed > for exchanging CIDR blocks so the address space can be > re-packed. It is time for everyone to learn a term that unfortunately I did not invent: IPv4ever. NAT and other clever gatewaying effectively provides a mechanism to extend the address lifetime expectancy not only of the IPv4 unicast address space in general, but of any given host in particular. That is, there are now mechanisms which can hide address changes from hosts that deal with address changes badly, while at the same time there is increasingly good software to assist with renumbering hosts. There are mechanisms evolving which ultimately should lead to nearly any given unicast subnet of 0/0 to be perceived by everything else as having a different number than things within that subnet believe. Moreover, there are also mechanisms evolving which will cause nearly any given unicast subnet of 0/0 to renumber itself so that all the numbered entities under that subnet renumber into a different unicast subnet of 0/0. This alone should give rise to maximal aggregation, and if combined with schemes which overload some addresses or which simply compress sparsely populated large subnets into densely populated smaller ones, should eliminate a large percentage of address waste. In other words, the mechanism(s) you allude to are being worked on. I would like to see them applied to the swamp within the next year or two. The "IP addresses never change within the lifetime of a session" and "IP addresses are end-to-end" crowds who have misthought a number of protocols will probably fight tooth and nail to see this never happen. Mind you, they are mostly the same people who fought tooth and nail against the idea of renumbering in the first place, so one can expect roughly the same type of "discussions". > The /19 policy is archaic. It creates an obstacles and > only partly resolves the problem. Fixing holes in CIDR > blocks, exchanging fragmented blocks for contiguous > blocks, and cleaning up "The Swamp" can do more for the > stability and size of the routing table. If you have an implementation of something Sprint and its competitors who now do precisely the same filtering can buy that can cause the swamp to be aggregated into a small handful of prefixes from their point of view, then I can point you at people who would be happy to sign a cheque. The only real problem I saw in the implementation of the /19 filter was the bad press generated by people who refused to listen to registries' warnings that long prefixes probably would not be globally routable, and possibly the lack of a tariff which would have allowed people with money to purchase exceptions in the Sprint filters. > BTW - If you use a route server to do the dampening and calculation of peer > routes you can even make a wimpy CPUed 7000 handle backbone traffic. The wimpy 7000 still has to receive at least one copy of the NLRI, and process changes into the forwarding table(s). As the number of prefixes increase, even if the level of "background" noise (the rate at which a large set of prefixes demonstrate instability that is considerably less than that which would be prevented even by very aggressive route dampening) were to remain constant, you require more CPU even in the simple case of receiving and installing modified forwarding tables. In the absence of any feedback mechanism that holds down the total number of globally visible prefixes, the increase in CPU requirements could easily outstrip Moore's law and overwhelm even state-of-the-art processors in a matter of time. Note that the liklihood of keeping up with the economics of dealing with things which are CPU bound and ill suited to parallel processing and which grow along the same slope or on a slightly greater slope than Moore's law is small. This describes the amount of BGP processing required prior to the installation of the first prefix-length filters at Sprint's border routers. I was always open to suggestions that would accomplish the same result, and helped push Cisco to develop two of them (a large cleanup of some of their BGP implementation's processing and an implementation of something very close to Curtis Villamizar's route flap dampening algorithm), however I have yet to see suggested something that eliminates the need for such filters that is readily deployable and which will keep the slope of processing requirements below that of processing capability. I still am, I belive my successors at Sprint and like-minded people at other ISPs who implement prefix-length filtering are too. Until such a thing emerges, however, I continue to believe that inbound prefix-length filtering is a good policy that should be implemented universally. Sean.

2 1

new CAIDA tool: Mapnet (warning: of possible operational utility)
by k claffy 11 Sep '97

11 Sep '97

what: new tool from CAIDA group why: (original motviation) visualization of Internet traffic and topologies is an important next step in the global evolution of the net. as a result, the cooperative association of internet data analysis (caida) is working on a new tool -- mapnet where: www.caida.org/Tools/Mapnet/Backbones how: mapnet will permit visualization of the infrastructure of multiple backbone providers simultaneously. it also allows individual isps to update and correct information about their infrastructure, either via a GUI or ascii mail in template form. (see mapnet update at www.caida.org/Tools/Mapnet/Update) the current prototype version relies on third party data (boardwatch, with some updates and clarifications) each of the commercial backbones and government provided data about federally funded r&e and mission-oriented backbones. caida staff have translated topology information provided in the last boardwatch release into lat/long specific nodes on a global map. the data are useful in a general sense, any data from a 3rd party obviously requires authentication. we recognize that much backbone-specific info is outdated or wrong and invite each provider to designate an authorized individual to periodically update their topology data. note that we include link, ISP, bandwidth, lat/long of cities, and peering information if available, as well as contact folk for the network in question. the availability of a tool like mapnet will assist with macro-level understanding of the internet, with standardizing public depiction of isps' infrastructure, and with communicating complicated peering information. (for those isp's willing to share in such an endeavor) we would appreciate any feedback: comments, constructive criticism, suggestions. please contact me or send email to brad huffaker (lead developer, mapnet) at mapnet(a)nlanr.net. tnx k caida kc(a)caida.org nlanr kc(a)nlanr.net

1 0

Re: too many routes
by Sean M. Doran 11 Sep '97

11 Sep '97

Vadim Antonov <avg(a)pluris.com> writes: > Sprint never used cisco 4000s in the backbone. Just FYI. True. The only one I ever remember was the DANTE access router which was dropped in place before AS-path prepending in software came about, and that wasn't a backbone router by any stretch of the imagination. There may have been one deployed to handle NACSIS or some other odd case at some point, however I don't remember it concretely. > Historically, memory limitation was because CSC/4 board in AGS/+ > routers had memory soldered in. The box was absolute top of the line > when it started to fall over. Sprint, in fact, was in an OK position since it was possible to eliminate full routing on most of the AGS+es in relatively short order once the memory problem became more apparent. I believe you and I got to the point where none of our AGS+es had full routing before people like JANET started having regular problems and had to start dropping routes from places like PSI entirely in order to have any connectivity at all. The AGS+ -> 7000 migration was a conceptual disaster, since the RP+SP combination simply didn't perform well under load. The SSP made things happier, particularly when things like selective packet drop were implemented, and this in essence led to Peter's frequent (and pretty accurate) complaint that the initial introduction of the 7500 compared to a 7000 + RP + SSP was a step backwards. (Several people who used to be at cisco, and some who still are, also have been heard making that complaint from time to time...) > Can you spell "economies of scale"? Or "using fiber at cost means > owning the fiber"? If you want to play the backbone game you've > got to own long-haul transmission facilities. And if you own such transmission facilities you have to hope that your competitiveness police don't force you to separate your transmissions facilities from your value-added services, and require the transmissions people to offer consistent pricing to all parties including the value-added services people. Of course, if bandwidth becomes much cheaper than you had bargained for initially, it could be a disadvantage to be trapped into using your own company's transmissions facilities with a requirement of a decent ROI, so it kinda cuts both ways. Sean.

2 1

CHOCOLATE HEAVEN!!!
by eman＠ghostly.net 11 Sep '97

11 Sep '97

Death By Chocolate Cookies Be careful when you eat these cookies... Because they melt in your hands, not in your mouth! For the recipe: Send $2.00 and a self addressed stamped envelope (or email address) to: Innovative Concepts Attn: Nicole PO Box 2471 Carmichael, CA 95609-2471

1 0

Re: not rewriting next-hop, pointing default, ...
by Sean M. Doran 11 Sep '97

11 Sep '97

Scott Huddle <huddle(a)mci.net> writes: > This is yet another problem/opportunity, in that providers want/need > to implement policies (such as no default, or only accept traffic > from the following ASs, or only accept traffic for the following ASs, > or consistent announcement) that are are difficult to enforce and/or > automatically measure/detect with today's technology. > > Router/switch vendors and potential interconnect > providers take note! In fact, OFRV has taken note. :-) Remember that rate limiting exists and is workable. You should be able to rate limit now based on MAC address. Yakov Rekhter and others correctly observed some time ago that it is possible to forge MAC addresses to get around this. Enter the wonderful world of mixing RPF with rate limiting. Gee, that IP address shouldn't have come from you, so I'll toss it through the "not in profile" rule. Then you either play fascist and drop not in profile traffic or you play Dave Clark and set off alarm bells and drop any such traffic if there is congestion. I actually like the latter approach somewhat more (go figure). My implementation idea was to give any particular peer at a public exchange point a particular amount of "free" bandwidth, perhaps 1.5Mbps or so across a FDDI, and declare anything more than that "out of profile", and pass the traffic through a queue against which one would apply aggressive wred killing off anything that didn't look like well-behaved TCP. Then if there was still lots of traffic, one could begin negotiating terms for increasing the "floor" bandwidth. This has several advantages. Firstly, you can avoid completely or at least minimize the effects of overbooking your connection to any given public interconnect, no matter what the implementation is. (This is effectively like using an ATM fabric and carefully laying out VCs, only it probably works better :) ). Secondly, a tariff for bandwidth settlements is much easier when you can actually limit the bandwidth. I belive more than one provider is doing this now. (We are.) Moreover the rate limiting code is rather clever in that you can apply different dampening for different traffic profiles. You essentially want to be able to do something like this: -- if destination is me, use "peer" profile -- if destination is a peer of mine, use "transit" profile where the peer and transit profiles might be: -- drop or -- pass only N bits/second, M packets/second with varying values for N and M, and likely higher prices for higher values of N and M in the transit profile and possibly in the peer profile. Answering the questions, "is this for me" vs "is this for a peer" is apparently about to become much easier. :-) A long time ago I thought this was pretty clever, now I think this kind of rate limiting is absolutely essential. > > a senior engineer at a well-known provider just pointed out to us that > > a weenie provider at mae-east was > > o not rewriting next-hop > > o sending our routes to others > > o sending others' routes to us > > o likely pointing default at us Gee this sounds awfully familiar :( So, the "others" you don't peer with who are sending you traffic fall into the "drop" profile based on MAC address or RPF or some other mechanism (offline talk if you like). This was the original impetus for suggesting rate limiting to Fred Baker (to whom I am eternally indebted for having gotten it implemented pretty quickly), since this sort of things was happening to Sprint quite alot... > > when the larger providers decline to peer with the smaller, there is a sad > > reason. traceroute -g is your friend. That too. It is particularly enlightening to traceroute -g towards a network that is not in anyone's routing tables, to see if you are really being defaulted towards of if someone is simply providing transit and exploiting the next hop is on same LIS mechanism. Sean.

1 0

Re: not rewriting next-hop, pointing default, ...
by rja＠corp.home.net 11 Sep '97

11 Sep '97

On Sep 11 15:23, Randy Bush wrote: } Subject: Re: not rewriting next-hop, pointing default, ... % I also think it may be time we refuse to peer with anyone % who inhibits LSR, as it seems that validation is now mandatory. % I think we should be sending out a "LSR is mandatory" notice % to our peers. Comments? LSR is actually a significant security issue. So, while I do understand and am sympathetic to the operational debugging issues that LSR addresses, I think that requiring a peer to enable LSR more than 2 hops inside their network from the outside world is unreasonable. In a world where SSH were available in cisco routers and/or IPsec were more widely deployed, I might have different views. However, we are where we are. Regards, Ran rja(a)home.net

3 3

not rewriting next-hop, pointing default, ...
by Randy Bush 11 Sep '97

11 Sep '97

a senior engineer at a well-known provider just pointed out to us that a weenie provider at mae-east was o not rewriting next-hop o sending our routes to others o sending others' routes to us o likely pointing default at us their noc phone was answered by a modem. we suspended peering with them and wrote to their noc. we got back a snotty message. we have ceased peering with them. we installed packet filters. our traffic on the east fddi dropped noticeably. when the larger providers decline to peer with the smaller, there is a sad reason. traceroute -g is your friend. randy

2 1

NATs and addressing
by Sean M. Doran 11 Sep '97

11 Sep '97

Ok so this turns NANOG into Big-Internet... mutter... but it's more interesting than television, and people here really ought to be thinking about things like this anyway. Oh yeah, I just realized I can add an ob. ops. postscript. So there. Nyah. Michael Dillon <michael(a)priori.net> writes: > If you use NAT and renumbering technologies then you > don't give a darn what your IP address is or who gave it > to you as long as it is globally routable. No, all you care about is that two endpoints can talk to each other. You emphatically must NOT care what your IP address is, as you say, but this also means that you must not care about the scope of the address itself, nor about the scope of its routability. In any piece of the evolving Internet, a datagram may undergo transformation from one address to another. In some cases it may undergo transformation into a prefix which is expressly not globally routable (e.g., an RFC 1918 address), yet which is routable within the scope of the address's visibilty. That is, between any pair or cluster of NATs (for instance, at the edges of a network) you require routability of all addresses valid within the scope delineated by those NATs. Outside that scope, the addresses (and indeed the protocol(s) under the end-to-end protocol) may be undefined. > Routing problems involve how to design, manage and operate this internal > traffic distribution hierarchy and are essentially engineering problems, > not policy problems or social problems. I think that the desire for > portable address space is not a routing problem. PI space is already being remapped by NATs around the world; the next steps in the evolution of the Internet seek to push that remapping closer to the edge of the PI space, so that the PI addresses ultimately are visible only within the administrative scope of the user of the PI addresses. This done, the whole PI vs PA address argument will simply cease to be relevant, and the principal flaw of the post-CIDR IPv4 addressing scheme will be no more. Note that scoping address meaning within a subset of the global topology also increases the address lifetime of IPv4 itself substantially, even ignoring such things as overloading single addresses and the like. The one piece that is unfortunately missing is more an API issue than anything else, and that is that getservbyname(3) should take an FQDN as a third argument and should find its answer in the DNS rather than in /etc/services (hi Paul). Sean. P.S.: The neat thing about this is that suddenly no operator really would need to worry much about ARIN and addressing politics, since addresses would only be valid within paritcular scopes, which could be the individual provider or a collection of providers, with or without downstream customers. (I hope there is no operator who thinks that killing off silly addressing politics and the endless arguments about filtering and so forth would be a Bad Thing.)

1 0

Re: not rewriting next-hop, pointing default, ...
by Scott Huddle 11 Sep '97

11 Sep '97

This is yet another problem/opportunity, in that providers want/need to implement policies (such as no default, or only accept traffic from the following ASs, or only accept traffic for the following ASs, or consistent announcement) that are are difficult to enforce and/or automatically measure/detect with today's technology. Router/switch vendors and potential interconnect providers take note! -scott Randy Bush <randy(a)psg.com> wrote > > a senior engineer at a well-known provider just pointed out to us that > a weenie provider at mae-east was > o not rewriting next-hop > o sending our routes to others > o sending others' routes to us > o likely pointing default at us > > their noc phone was answered by a modem. we suspended peering with them and > wrote to their noc. we got back a snotty message. we have ceased peering > with them. > > we installed packet filters. our traffic on the east fddi dropped > noticeably. > > when the larger providers decline to peer with the smaller, there is a sad > reason. traceroute -g is your friend. > > randy >

2 1