January 2001 - Test - lists.nanog.org

California -- STAGE 3 POWER EMERGENCY... AGAIN
by Matt Clauson 18 Jan '01

18 Jan '01

from the California Independent System Operator website, at: http://www.caiso.com/awe/systemstatus.html Current Active Notice(s) STAGE 3 EMERGENCY NOTICE [200100652] Effective 01/16/2001 at 07:20 the California Independent System Operator has implemented Stage 3 of the Electrical Emergency Plan. The Plan has been implemented for the following reason(s): Insufficient reserves and energy. Stage 3 is expected to be in effect from HE 8 through HE 24. Participating transmission owners are to notify the Utility Distribution Companies within their operational areas. Stage 3: Operating reserves are expected to fall below 1-1/2%. The UDC will implement their electrical emergency plan and/or other load dropping to effect 0 MW of involuntary firm demand reductions (i.e., in addition to ALL Interruptible Service reductions) no later than hours. This message is from Market Operations at the California ISO. Notice issued at: 01/16/2001 07:20 [fin] Have you checked your generator today? --mec ----- End forwarded message -----

3 2

Re: Forcasts, why won't anyone believe them?
by abender＠taqua.com 18 Jan '01

18 Jan '01

From: Sean Donelan <sean(a)donelan.com> > They may hate to give out this information, but any piece of > equipment with a NEBS level 3 rating already has this information. > I'm not asking for vendors to do the full, burn the box, NEBS > testing. But having the power drain data sheet would be very useful. > > This is really a customer demand issue. If customers started > demanding vendors supply a full NEBS-like space and power planning > data sheet, we would get the information. But customers don't, so > we don't. I suspect the availability of data is not the only/key issue here. If pressed (asked correctly, etc.), most vendors can provide some approximation of worst case system or per-subsystem current requirements, whether they have done/can spell NEBS or not. Even though it is non-trivial to measure instantaneous peak drain empirically, there are other ways to get reasonable bounds on it, and doing so is a routine practice of hardware engineers. A reason that the most careful power capacity forecasts (in particular) may *seem* to be perpetually optimistic, might be the needlessly high peak to average drain ratio of (most?) equipment. If customers can convince vendors that they have more concern about the ancillary costs of operating equipment, vendors might be induced (pun!) to make some nominal sacrifices in cost and MTBF to include features like inrush current limiting, and sequencing. Get your RFPs ready. Regards, Andrew Bender taqua.com

1 0

time zones
by matthew＠tycho.net 18 Jan '01

18 Jan '01

> > Blocks 7 and 8 went out around 11 PM local (that's almost an hour ago now) > What I meant to say here (as several have pointed out) is that blocks 7 and 8 went out at 11 *AM* I've been converting between Pacific time and Eastern time all day, and keep getting the sign bit mixed up. Meanwhile, more accurate (but not guaranteed to be correct) data: Block 5 (the part that wasn't hit yesterday), Block 6, and Block 7 were scheduled out at 10 AM and scheduled back by 11:40 AM Block 8 and Block 9 were scheduled out at 11 AM and scheduled back by 12:40 PM Further outages were suspended at noon, until 5 PM, when it is expected that, barring a miracle, the last of Block 9 and all of Block 10 will be shut down. -matthew kaufman Tycho Networks/DSL.net matthew(a)tycho.net

1 0

RE: Second day of rolling blackouts starts
by matthew＠tycho.net 18 Jan '01

18 Jan '01

Blocks 7 and 8 went out around 11 PM local (that's almost an hour ago now) More to come, certainly. -matthew kaufman Tycho Networks/DSL.net matthew(a)tycho.net Original message <Pine.GSO.4.31.0101181122130.11525-100000(a)darkwing.uoregon.edu> From: "Lucy E. Lynch" <llynch(a)darkwing.uoregon.edu> Date: Jan 18, 11:25 Subject: RE: Second day of rolling blackouts starts > > > started today with block 5 (who got only a short outage yesterday) > - as well as 6 & 7 per http://www.sfgate.com/ >

1 0

Montreal power failure (north america)
by Sean Donelan 18 Jan '01

18 Jan '01

200,000 customers in Montreal lost power when a Hydro-Quebec power station failed.

2 1

Sprint Pennsauken..
by Matt Levine 18 Jan '01

18 Jan '01

Anyone know what's up with Sprint Pennsauken right now? Nameserver (ns3-auth.sprintlink.net) has been down last 3 hours or so.. Matt -- Matt Levine, CTO <mlevine(a)efront.com> eFront Media, Inc. - http://www.efront.com Phone: +1 714 428 8500 ext. 504 Fax : +1 949 203 2156 ICQ : 17080004

2 2

RE: California regulators ordered rolling blackouts
by Roeland Meyer 18 Jan '01

18 Jan '01

If you've every worked for a telco, you'd know all about "affiliate transaction" and how PG&E can't touch that debt with anything other than cash, from the other bucket. The CA PUC would squash them like a bug. The real problem is CalISO management. > -----Original Message----- > From: hardie(a)equinix.com [mailto:hardie@equinix.com] > Sent: Wednesday, January 17, 2001 3:12 PM > To: akyol(a)akyol.org > Cc: nanog(a)merit.edu; rusty(a)hodge.com > Subject: Re: California regulators ordered rolling blackouts > > > > http://www.sfgate.com/cgi-bin/article.cgi?file=/chronicle/arch > ive/2001/01/16/MN184364.DTL > > describes one of the more interesting side notes in all of > this, how PG & E's > corporate restructuring allows them to retain profits. Among > the many comical > notes is that about half of the debt PG & E owes is to itself. > > regards, > Ted Hardie > > > > > > > > Even third world countries aren't as idiotic as California. > There when > > electricity goes off, it usually is because there is none > to be had. In our > > situation, it is manipulation and poor planning to the extreme. > > > > My power was out for at least one hour, it is back now. > > > > Bora > > > > ----- Original Message ----- > > From: "Rusty H. Hodge" <rusty(a)hodge.com> > > To: <nanog(a)merit.edu> > > Sent: Wednesday, January 17, 2001 1:37 PM > > Subject: California regulators ordered rolling blackouts > > > > > > > > > > It has begun. Welcome to the 3rd world. > > > > > > > > > http://www.sfgate.com/cgi-bin/article.cgi?file=/news/archive/2 > 001/01/17/stat > > e1503EST0161.DTL > > > > > > Blackouts hit California as utility financial woes deepen > > > > > > Associated Press, SF Gate Wednesday, January 17, 2001 > > > Breaking News Sections > > > > > > > -------------------------------------------------------------- > ---------- > > > > > > California regulators ordered rolling blackouts Wednesday for the > > > first time in the state's months-long electricity crisis, blaming > > > utility credit problems and a tight national power supply for the > > > scattered outages. > > > > > > The rotating blackouts, expected to affect about 500,000 customers > > > for an hour to 90 minutes, were restricted to Pacific Gas and > > > Electric Co. territory in northern and part of central California, > > > said Stephanie McCorkle, a spokeswoman for the Independent System > > > operator, keeper of the state's power grid. > > > > > > A PG&E spokesman said that about 250,000 people were > already without > > > power from the blackouts that began at 11:41 a.m. There > are reports > > > of outages in San Francisco's Lower Haight, Oakland's > Rockridge area, > > > the Oakland Hills, Orinda, the Peninsula from South San > Francisco to > > > San Mateo, parts of Napa and Sonoma County, downtown San Jose, > > > Cupertino's De Anza College, San Ramon, Santa Cruz, > Benicia and other > > > areas, according to radio and TV reports. > > > > > > There was a report of two students being trapped in an elevator at > > > Hastings School of Law in San Francisco. Some ATMs were > reported out > > > of service in downtown San Francisco. > > > > > > Power was cut to the Cow Palace in Daly City, which is > holding a boat > > > show, according to an administrative worker at the > exhibition hall. > > > The main building remains well-lit from emergency lighting and > > > skylights, she said. > > > > > > KICU-TV, channel 36, was knocked off the air. > > > > > > The blackouts would first affect customers in scattered > areas known > > > as blocks 3 and 4. For security reasons, the precise locations are > > > not released. Consumers can find their block numbers at > the bottom of > > > their power bills. > > > > > > PG&E and state officials urged conservation and said it > was not known > > > yet if blackouts would be ordered for blocks 5, 6 and 7. > Blocks 1 and > > > 2 had power outages last June. > > > > > > BART, fire departments, police stations and hospitals are not > > > affected. Motorists who come to non-functioning traffic signals, > > > including El Camino Real in the San Bruno/San Mateo area > and Lawrence > > > Expressway in San Jose, should treat them as four-way stops. > > > > > > Utilities try to avoid cutting power to blocks with essential > > > services such as hospitals. > > > > > > Terry Winter, president of Cal-ISO, said that a large > power plant on > > > California's Central Coast went down at about 11 a.m., > necessitating > > > the outages. > > > > > > Worry that the state's two largest utilities were on the verge of > > > bankruptcy led some suppliers to withhold power from California, > > > despite an emergency federal order requiring them to sell excess > > > electricity to the state, said Jim Detmers, the ISO's managing > > > director of operations. > > > > > > But Winter said later that he did not believe generators were > > > withholding power. Instead, he said, the main problem is > broken power > > > generating facilities, many of them older plants that > have been run > > > heavily since June. > > > > > > Compounding the problem is a general scarcity of electricity > > > nationally, and a lack of snow and rain in the > > > hydroelectric-dependent Pacific Northwest, Detmers said. > > > > > > ``If you are out in the community and get into an > intersection that > > > is in the blackout, use caution,'' he said. > > > > > > The day began with the third Stage 3 power alert within a week, > > > meaning reserves were close to just 1.5 percent. The > warning marked > > > at least the third time California neared blackouts since > its power > > > woes began last summer. > > > > > > The ISO fended off outages before by temporarily turning off huge > > > state pumps that move water from Northern California to the south, > > > sucking enough power for 600,000 homes, but that wasn't enough > > > Thursday. > > > > > > Suppliers were ``reluctant to provide power to California > because of > > > the financial situation of the utilities,'' Detmers said. > > > > > > He said the ISO wasn't probing whether suppliers were flat-out > > > ignoring Energy Secretary Bill Richardson's emergency > order insisting > > > that any spare power go to California, however. > > > > > > ``We're just trying to get the power delivered,'' he said. > > > > > > Adding to the problems, several power plants that were expected to > > > return to full operation after repairs did not, Detmers said. > > > > > > On Tuesday, Southern California Edison declared itself > unable to pay > > > hundreds of millions in wholesale electricity bills, and > it and PG&E, > > > the state's largest utilities, took another hit on Wall Street. > > > > > > SoCal Edison, which serves 11 million people, said it > cannot pay $596 > > > million in bills for wholesale energy and debt service, including > > > $215 million to the California Power Exchange. > > > > > > The Power Exchange was considering whether to make the utility buy > > > its power elsewhere and an electricity supplier > threatened to force > > > SoCal Edison into bankruptcy if it failed to pay its bills. > > > > > > The default prompted Standard & Poor's and Moody's to > downgrade the > > > credit ratings of SoCal Edison and PG&E to junk-bond status. > > > > > > The credit agency said SoCal Edison's delinquency also > tainted PG&E. > > > With just $500 million in cash left as of Jan. 10, PG&E faces due > > > dates on bills totaling $1 billion during the first two weeks of > > > February. > > > > > > Between them, PG&E and SoCal Edison have lost at least > $10 billion in > > > wholesale energy costs. A rate freeze imposed as the > state phases in > > > deregulation has blocked them from passing on higher > wholesale costs > > > to their customers. > > > > > > Wholesale power prices have risen dramatically since > June, in part of > > > because of a hot summer and a cold winter. In 1999, they averaged > > > perhaps 3.5 cents a kilowatt. Now, they are running about > 30 cents, > > > and sometimes far higher. > > > > > > Demand has remained high, supplies are strapped because > no new power > > > plants have been built in the state in recent years and > imports are > > > tight because other states are fighting over the power. > > > > > > In addition, spiraling prices for natural gas are forcing power > > > plants to raise their prices. Most power plants are fired > by natural > > > gas. > > > > > > On Tuesday, unusually high demand for natural gas, due in part to > > > cold weather, led San Diego Gas and Electric to cut > supplies to two > > > power plants, contributing to the state's Stage 3 alert. > > > > > > The utility said there was plenty of natural gas, but not enough > > > space in the pipeline to meet its customers' needs. To > maintain the > > > supply for its home and small-business users, the utility cut the > > > flow to the two power plants and six large industrial customers. > > > > > > The state avoided rolling blackouts after huge state > pumps that move > > > water from Northern California to the south were turned off > > > temporarily, conserving enough electricity to power 600,000 homes, > > > said Kellan Fluckiger, the ISO's chief operating officer. > > > > > > Joel Nelsen, president of California Citrus Mutual, spent > Tuesday on > > > the phone with Central Valley lawmakers and the governor's office > > > trying to ensure that orange growers wouldn't face outages as they > > > tried to protect crops from a cold snap. > > > > > > ``We're terribly exposed,'' said Nelsen, who heads a trade > > > association of 800 growers. ``The loss of power for a short time > > > could wreak untold damage on our crop.'' > > > > > > > > >

3 2

Re: Favorites (Re: UUNET peering policy)
by Richard A. Steenbergen 17 Jan '01

17 Jan '01

On Wed, Jan 17, 2001 at 05:39:41PM -0700, Danny McPherson wrote: > > And often enough, I've seen MEDs result in worst-exit routing. > That is, routing that was intended to be better than closest-exit > (aka. hot-potato), but turns out to be far worse than best-exit > (I believe some refer to this as cold-potato *8^/) because an > adjacent AS in unable to correctly convey the optimal entry point > into their AS via MEDs, be it because of GOOD aggregation, or > uncluefully derived MED values. I believe Avi Freedman would call that "mashed-potato" routing. -- Richard A Steenbergen <ras(a)e-gerbil.net> http://www.e-gerbil.net/humble PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)

1 0

Re: California regulators ordered rolling blackouts
by matthew＠tycho.net 17 Jan '01

17 Jan '01

Oh, we equate power failure and rain here in Santa Cruz, California as well :) It is strangely comforting to know that we're all in the same boat, albeit for different reasons. -matthew kaufman Tycho Networks/DSL.net matthew(a)tycho.net Original message <01b401c08120$36173100$9eb431c6@oemcomputer> From: "Alexei Roudnev" <alex(a)relcom.net> Date: Jan 17, 23:28 Subject: Re: California regulators ordered rolling blackouts > > There is big difference. In Russia, all data centers etc have a reliablr power > and, if located in such region as Vladivostok, have independent power supply; > power failure is common event (such as rain) for them and rollout blackouts does > not interrupt their work; in California people used to believe to the good news > and any failure became disaster.

1 0

Re: Favorites (Re: UUNET peering policy)
by Danny McPherson 17 Jan '01

17 Jan '01

> Honoring MEDs is not Black and White. I dont know one large network that > honors all MEDs from every peer. For the most part, I would say that most > providers do 'warm' routing. it just depends on what degree of warm you > can live with. Of course, the amount of brokenness involved with honoring MEDs is questionable anyways. For example, you can't compare MEDs from different ASs and attain any reasonable result. For that matter, the result may very well be persistent route oscillation. And often enough, I've seen MEDs result in worst-exit routing. That is, routing that was intended to be better than closest-exit (aka. hot-potato), but turns out to be far worse than best-exit (I believe some refer to this as cold-potato *8^/) because an adjacent AS in unable to correctly convey the optimal entry point into their AS via MEDs, be it because of GOOD aggregation, or uncluefully derived MED values. Then again, I've seen MEDs work just as intended, allowing a peer network to easily manipulate route/forwarding decsions in a neighboring network in order to alleviate congestion or other issues within their own network. Of course, there are clever ways to make MEDs of benefit, though IMO, they do more harm than good overall. -danny

1 0