August 29, 2006

Austin American-Statesman
Power Loss Also Stings at Samsung

By KIRK LADENDORF, AMERICAN-STATESMAN STAFF

Within the past three months, Samsung and Freescale Semiconductor Inc. - two of Austin Energy's biggest customers - have faced electrical outages that shut down plants and cost them millions of dollars in lost production. Freescale's troubles became public early last week. Now Samsung is disclosing a June outage that lasted only about 10 minutes but shut down the plant for a week because equipment had to be cleaned, tested and recalibrated before production could resume.

"This is not a small problem for us," Samsung spokesman Bill Cryer said. "You are talking lots of money and loss of product and equipment that has to be repaired and cleaned and so forth."

Austin Energy and top city officials have met with both companies and promised to take steps to avoid repeats. But the impact of the episodes extends beyond two of the city's biggest high-tech employers.

The outages, along with similar problems at smaller area companies, could undermine the city's efforts to recruit chip plants, data centers and other high-tech operations that need a reliable and affordable supply of electricity.

For the chip companies, the outages were hugely expensive, requiring them to trash silicon wafers being processed; disassemble, inspect and repair expensive equipment; and send highly paid workers home.

Samsung didn't provide an estimate for the cost. The outage was triggered when an insulator failed at the Decker power plant.

But Freescale said recently that four outages over the past four years have cost between $15 million and $20 million. In the most recent incident, Austin Energy cut power to Freescale's Oak Hill plant in mid-July because of a fire in a transformer at a substation next to the plant.

Austin Energy says the outages were unfortunate coincidences with different causes, not evidence of a larger problem.

"Our long-term trends say that we are continuing to improve the system," said Cheryl Mele, senior vice president for electric service delivery.

"We are committed to the quality issue and doing what we can do to make sure this doesn't happen again," she said.

Information on how Austin Energy stacks up to other utilities in terms of reliability was not immediately available.

Austin has pitched reliable and affordable electricity in its efforts to recruit new employers. But if the service deteriorates, it could hurt the city's ability to attract more data centers and tech manufacturing operations.

"Over the years, Austin Energy has had a great reputation for customer service and reliability," said economic development consultant Angelos Angelou, who has helped bring chip makers to Austin and helps tech companies search for locations nationwide. "If the system has problems, it can be harmful to the businesses that we have in Austin. It has to be addressed so this does not become an issue for economic development."

The Samsung outage happened just months after Central Texas offered its most aggressive incentive package ever, more than $233.4 million, to persuade the company to build a second factory that will be several times larger than the current plant. As part of the deal, the city agreed to build a much larger electrical substation to serve the new factory.

That factory won't go into production until late 2007, but if an outage were to shut down production, the financial impact on Samsung would be far more serious.

After the June outage, Cryer said, top city officials, including City Manager Toby Futrell, had an extended meeting with top Samsung officials.

"This is a major problem for us," Cryer said. "The city is more than well aware of that. We made them aware. And they were quick to respond. And it is ongoing. We are still working on it. It is encouraging."

Futrell said the city has outlined short- and long-term remedies for Samsung and Freescale. The long-term plans include connecting each plant to two substations so power can remain on if one substation has problems.

"At the end of the day, it is the actions that will speak the loudest," Futrell said. "Any mechanical system can have failures. All in all, we do a very good job (delivering reliable power), but that is small comfort to the company that has had a major outage."

Small techs affected

Council Member Brewster McCracken said the outages revealed a weakness in the redundancy power systems that supplied the plants. But he doesn't see it as a widespread problem, and he points to the recent upgrade of the utility's bond rating as an indication of its national reputation for reliability.

But problems aren't limited to the biggest users.

Dave Gino, chief operating officer at Molecular Imprints Inc., a nanotechnology startup in North Austin, said his company was hurt by two power outages in the past few months that damaged the high-tech equipment and caused it to scrap some products being processed.

When he talked to Austin Energy about the problem, Gino said, he was told the company should buy a custom backup power system that would include batteries and an emergency generator. But such equipment can be expensive, costing hundreds of thousands of dollars.

"For a small company, that can be challenging," Gino said.

Chad Kissinger, president of Onramp Access Inc., which runs an Internet data center in Southeast Austin, said his company has had about three power outages in the past six months, including one that lasted 20 minutes. His company has a backup power system, but he said turning it on is a jolting experience.

"The alarms go off, and there are beeps. It sounds like a submarine diving," Kissinger said. "The emergency generator goes on, and that sends off an alarm. It literally scares you. Your heart rate goes up. You would prefer not to use the backup systems ever."

Seeking solutions

Some big chip plants use such massive amounts of power that having a backup power system is impossible. Major outages can shut down all or part of Samsung's operations, and even large "voltage sags," in which the voltage briefly declines 15 percent or more, can cause problems with the manufacturing equipment.

Such machines are a chip maker's biggest investment in a factory.

Austin Energy said it works closely with major customers to investigate outages and sizable voltage sags.

But in a large utility system, there are always uncontrollables, Mele said, such as unexpected equipment failures and even animal interference with power equipment. Austin Energy will spend $8 million over the next several months to upgrade the substation nearest to Freescale and to bring in a backup power connection from another nearby substation.

In the Samsung outage, the insulator problem caused a series of failures that put the plant in the dark. Austin Energy said the failed part still is undergoing testing.

The utility inspects all the equipment used to deliver power to its largest customers, but sometimes that equipment wears out prematurely and suddenly fails, Mele said.

"We try as much as possible to require quality product in our system," Mele said. "But in dealing with equipment . . . sometimes things fail."