March 2008 - Test - lists.nanog.org

Re: cooling door
by Brian Dickson 29 Aug '08

29 Aug '08

Paul Vixie wrote: > if you find me 300Ksqft along the caltrain fiber corridor in the peninsula > where i can get 10mW of power and have enough land around it for 10mW worth > of genset, and the price per sqft is low enough that i can charge by the > watt and floor space be damned and still come out even or ahead, then please > do send me the address. Well, there are alternatives to the 10mW power and 10mW genset... 10-20mW (or more) of nuclear power/gensets. Get off the grid, sell back some power to the grid, reduce your costs for power, especially peak rate power, and be able to brag about having your own nuke plant. This may give you more flexibility in finding the space along the fibre corridor... BTW, I did some quick googling and found a few interesting links related to smaller nuclear generator systems: http://peswiki.com/index.php/Directory:Toshiba's_Micro_Nuclear_Reactor http://www.eia.doe.gov/cneaf/nuclear/page/analysis/nucenviss2.html http://www.eoearth.org/article/Small_nuclear_power_reactors http://www.uic.com.au/nip60.htm And if you have access to a deep harbor: http://atomic.msco.ru/cgi-bin/common.cgi?lang=eng&skin=menu1&fn=projects :-) Brian

6 9

Creating demand for IPv6, and saving the planet
by Daniel Senie 24 Jun '08

24 Jun '08

A number of people have bemoaned the lack of any IPv6-only killer-content that would drive a demand for IPv6. I've thought about this, and about the government's push to make IPv6 a reality. What occurred to me is there is a satellite sitting in storage that would provide such content: http://en.wikipedia.org/wiki/Triana_(satellite) Al Gore pushed for this satellite, Triana, to provide those on earth with a view of the planet among its scientific goals. The Republicans referred to it as an "overpriced screen saver," though the effect even of just the camera component on people's lives and how they treat the planet could be considerable. By combining the launch of Triana with feeding the still images and video from servers only connected to native IPv6 bandwidth, the government would provide both a strong incentive for end users to want to move to IPv6, and a way to get the people of this planet to stop from time to time and ponder the future of the earth. Of course getting this done any time soon would require getting the present administration to reverse its bias against Triana and global warming. But it seemed to me an interesting way to advance two goals in synergy. Dan -- ----------------------------------------------------------------- Daniel Senie dts(a)senie.com Amaranth Networks Inc. http://www.amaranth.com "Our lives begin to end the day we become silent about things that matter." -- Martin Luther King, Jr.

26 54

10GE router resource
by user user 19 May '08

19 May '08

Hi everybody! I find myself in the market for some 10GE routers. As I don't buy these everyday, I was wondering if any of you guys had any good resources for evaluating different vendors and models. I'm mainly thinking about non-vendor resources as the vendorspeak sites are not that hard to find. Also I'd love to hear recommendatios for "budget" 10GE routers. The "budget" router would be used to hook up client networks through one 10GE interface and connect to different transit providers through two 10GE interfaces. - Zed ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs

34 55

RE: cooling door
by Frank Coluccio 05 Apr '08

05 Apr '08

Michael Dillon is spot on when he states the following (quotation below), although he could have gone another step in suggesting how the distance insensitivity of fiber could be further leveraged: >The high speed fibre in Metro Area Networks will tie it all together >with the result that for many applications, it won't matter where >the servers are. In fact, those same servers, and a host of other storage and network elements, can be returned to the LAN rooms and closets of most commercial buildings from whence they originally came prior to the large-scale data center consolidations of the current millennium, once organizations decide to free themselves of the 100-meter constraint imposed by UTP-based LAN hardware and replace those LANs with collapsed fiber backbone designs that attach to remote switches (which could be either in-building or remote), instead of the minimum two switches on every floor that has become customary today. We often discuss the empowerment afforded by optical technology, but we've barely scratched the surface of its ability to effect meaningful architectural changes. The earlier prospects of creating consolidated data centers were once near-universally considered timely and efficient, and they still are in many respects. However, now that the problems associated with a/c and power have entered into the calculus, some data center design strategies are beginning to look more like anachronisms that have been caught in a whip-lash of rapidly shifting conditions, and in a league with the constraints that are imposed by the now-seemingly-obligatory 100-meter UTP design. Frank A. Coluccio DTI Consulting Inc. 212-587-8150 Office 347-526-6788 Mobile On Sat Mar 29 13:57 , sent: > >> Can someone please, pretty please with sugar on top, explain >> the point behind high power density? > >It allows you to market your operation as a "data center". If >you spread it out to reduce power density, then the logical >conclusion is to use multiple physical locations. At that point >you are no longer centralized. > >In any case, a lot of people are now questioning the traditional >data center model from various angles. The time is ripe for a >paradigm change. My theory is that the new paradigm will be centrally >managed, because there is only so much expertise to go around. But >the racks will be physically distributed, in virtually every office >building, because some things need to be close to local users. The >high speed fibre in Metro Area Networks will tie it all together >with the result that for many applications, it won't matter where >the servers are. Note that the Google MapReduce, Amazon EC2, Haddoop >trend will make it much easier to place an application without >worrying about the exact locations of the physical servers. > >Back in the old days, small ISPs set up PoPs by finding a closet >in the back room of a local store to set up modem banks. In the 21st >century folks will be looking for corporate data centers with room >for a rack or two of multicore CPUs running XEN, and Opensolaris >SANs running ZFS/raidz providing iSCSI targets to the XEN VMs. > >--Michael Dillon

19 24

rack power question
by Patrick Giagnocavo 05 Apr '08

05 Apr '08

Hopefully this classifies as on-topic... I am discussing with some investors the possible setup of new datacenter space. Obviously we want to be able to fill a rack, and in order to do so, we need to provide enough power to each rack. Right now we are in spreadsheet mode evaluating different scenarios. Are there cases where more than 6000W per rack would be needed? (We are not worried about cooling due to the special circumstances of the space.) Would someone pay extra for > 7KW in a rack? What would be the maximum you could ever see yourself needing in order to power all 42U ? Cordially Patrick Giagnocavo patrick(a)zill.net

46 74

L3VPN VPNv4 NLRI - Route Reflector Scaling
by Mark Tinka 04 Apr '08

04 Apr '08

Hello all. (posted to C-NSP too; please excuse the length of the message) Considering the scaling techniques currently available for VPNv4/L3VPN deployments as regards MP-BGP route reflectors, what do folk think is currently the most elegant way to deploy this that provides an even compromise on manageability, cost and scale (see RFC 1925, section 2, part 7, :-))? While ORF and BGP RR groups are a logical way to go, they require significant maintenance and might introduce scaling issues as regards management on PE routers. To mitigate this, however, outbound filters on the route reflectors, to each PE router, may be used to propagate VPNv4 NLRI to the PE routers that actually need them, but introduces problems if this information has to traverse regional or international PoP's where route reflectors talk to other route reflectors (numerous RR clusters, perhaps) before the end-PE router is reached. If filtering outbound on the route reflectors were the network-wide policy, having to co-ordinate filters on 10's of route reflectors would be an administrative issue. One other option we theorize would be to have dedicated VPNv4 route reflectors (route reflectors that do not reflect other address families, e.g., IPv4, IPv6, e.t.c.). However, if PE routers are also used as general edge routers (in the global routing table), the amount of peering could become significant depending on the scale of the network, and trying to co-ordinate the numerous route reflectors serving the various address families could become an issue when considered from a multi-PoP point of view. Needless to say, adding route reflectors dedicated to VPNv4 (or a single address family, for that matter) would increase one's cost some. On the otherhand, PE-to-PE MP-iBGP peering means outbound filtering can easily be done toward a particular end-PE, but the scaling issues of a full-mesh iBGP solution come into play. We currently do not see a feasible way to selectively filter outbound VPNv4 NLRI from the ingress PE routers without causing reachability issues unless the end-PE router is also the route reflector. This means route reflectors would still have to carry network-wide VPNv4 NLRI, and would need to be the network elements that have to figure out to which PE routers what VPNv4 NLRI should be announced. Using partitioned RR's is also a consideration; but the administrative burden of co-ordinating which L3VPN has their VPNv4 NLRI on which route reflector in what part of the network could get problematic as more customers are signed on. One may also reduce the number of route reflectors to solve this problem, but the risk of more remote PoP's depending on centralized, far far away route reflectors is too great - and then there's the CPU/memory resource thing... My ideal scenario would be to somehow, scalably have the ingress PE router inform the route reflector on which end-PE routers actually need to receive the VPNv4 NLRI so that the decision of the route reflector to do this is somehow influenced by the ingress PE router, through some kind of capability. ORF achieves this to some extent, but requires the egress PE routers (and/or route reflectors) be configured with the appropriate filters. Obviously, this is probably not a thoroughly well-thought out implementation :-). How are folk handling these issues today? Cheers, Mark.

2 5

Re: cooling door
by Robert Boyle 01 Apr '08

01 Apr '08

At 02:11 PM 3/29/2008, Alex Pilosov wrote: >Can someone please, pretty please with sugar on top, explain the point >behind high power density? More equipment in your existing space means more revenue and more profit. >Raw real estate is cheap (basically, nearly free). Increasing power >density per sqft will *not* decrease cost, beyond 100W/sqft, the real >estate costs are a tiny portion of total cost. Moving enough air to cool >400 (or, in your case, 2000) watts per square foot is *hard*. It depends on where you are located, but I understand what you are saying. However, the space is the cheap part. Installing the electrical power, switchgear, ATS gear, Gensets, UPS units, power distribution, cable/fiber distribution, connectivity to the datacenter, core and distribution routers/switches are all basically stepped incremental costs. If you can leverage the existing floor infrastructure then you maximize the return on your investment. >I've started to recently price things as "cost per square amp". (That is, >1A power, conditioned, delivered to the customer rack and cooled). Space >is really irrelevant - to me, as colo provider, whether I have 100A going >into a single rack or 5 racks, is irrelevant. In fact, my *costs* >(including real estate) are likely to be lower when the load is spread >over 5 racks. Similarly, to a customer, all they care about is getting >their gear online, and can care less whether it needs to be in 1 rack or >in 5 racks. I don't disagree with what you have written above, but if you can get 100A into all 5 racks (and cool it!), then you have five times the revenue with the same fixed infrastructure costs (with the exception of a bit more power, GenSet, UPS and cooling, but the rest of my costs stay the same.) >To rephrase vijay, "what is the problem being solved"? For us in our datacenters, the problem being solved is getting as much return out of our investment as possible. -Robert Tellurian Networks - Global Hosting Solutions Since 1995 http://www.tellurian.com | 888-TELLURIAN | 973-300-9211 "Well done is better than well said." - Benjamin Franklin

3 2

OT: vendor spam
by Bill Nash 31 Mar '08

31 Mar '08

Anyone seen spam from Uplogix.com? As it came to this email address, which I don't give to vendors for exactly this reason, I'm suspecting it's been harvested from this list or maybe c-nsp, the only other list I'm active on, for sporadic amounts of active. Has anyone else seen this? I'd like to verify the source before I add nails to the clue-by-four. - billn

1 0

cooling door
by Paul Vixie 31 Mar '08

31 Mar '08

page 10 and 11 of <http://www.panduit.com/products/brochures/105309.pdf> says there's a way to move 20kW of heat away from a rack if your normal CRAC is moving 10kW (it depends on that basic air flow), permitting six blade servers in a rack. panduit licensed this tech from IBM a couple of years ago. i am intrigued by the possible drop in total energy cost per delivered kW, though in practice most datacenters can't get enough utility and backup power to run at this density. if "cooling doors" were to take off, we'd see data centers partitioned off and converted to cubicles. -- Paul Vixie

10 11

Re: cooling door
by Frank Coluccio 31 Mar '08

31 Mar '08

re: "Are you aware of any such cases?" Before I run too far afield of my main point, it's the LAN closets right now that are superficial and require the most attention and overhauling. Also on tap are local in-building data centers, remotes, colos, etc. And not necessarily in that order. >>every other piece of hardware that I have been responsible for in even a 35-person, simple LAN setup ... has had to have me lay hands on it at odd, unplanned times to keep working.<< You can't lay your hands on equipment that's no longer in existence because it's been eliminated, for starters. This is probably the most important point to grasp. There is no LAN equipment on site anymore once you've backhauled your desktops directly to a central site over optical channels. And to do this, fiber to the desktop (and a generous smattering of WLANs never hurts) must be considered first, no longer brought back to the local closet, but rather to one or two central locations that could either be in-building or to a nearby data center or colo. Or as far away as applications will permit. That last sentence should serve as evidence that I've learned my lesson ;) Some enterprises (including several large federal government agencies) have done parts of what I'm referring to, but none that I am aware of has done it all. By the same token, nothing that I've mentioned has NOT been tried somewhere by some enterprise before, which is to say that every aspect of fiber extension that I've mentioned thus far is currently working somewhere in an enterprise or shared data center today. In the large enterprise space we've already seen, time and again now, how leased optical facilities and dark fiber networks alike support server farms in remote data centers and public colos bringing data back to desktops in business offices. VPNs also come to mind. Think: FiOS. Why would supporting the desktops of a 1,000 or 10,000 employee company, which desktops would be scattered over a half-dozen or a hundred urban and exurban sites, be any different than supporting the same number of residential subscribers who are being supported by fiber to the home? Indeed Verizon has already deployed the equivalent of FiOS to some federal government agencies, while holding back in the enterprise. I suppose they are holding back in the enterprise for fear of cannibalizing its own lucrative leased line business, although there could be other reasons having to do with the awkward and time consuming process they undergo while formally "productizing" their services. Multiple cable system pperator Bright House Networks is using passive-optical networks (PONs) today akin to Verizon's FiOS, sans the video part, to do this very thing in support of businesses in central Florida as I type. http://www.allbusiness.com/services/business-services/4516302-1.html The method of supporting end users being used by Bright House Networks is the same as I outlined earlier in respect to the LAN admin operating from Mumbai. In general, the LAN part, or more to the point, fiber to the desktop is a relative slam dunk compared to the intricacies of tuning applications within data centers, colos and IXes, I grant you that. As they say, however, you've gotta start somewhere. Frank A. Coluccio DTI Consulting Inc. 212-587-8150 Office 347-526-6788 Mobile On Sat Mar 29 21:14 , Patrick Giagnocavo sent: > >Frank Coluccio wrote: >> > >I'd like to see a case where someone actually did what you proposed and >were successful at it, in a for-profit company. > >Are you aware of any such cases? > >Because outside of Solaris and Linux servers, every other piece of >hardware that I have been responsible for in even a 35-person, simple >LAN setup (the smallest and simplest config anyone on this list could >possibly have), has had to have me lay hands on it at odd, unplanned >times to keep working. > >Cordially > >Patrick Giagnocavo >patrick(a)zill.net

3 2