Reliability pays off at Alcoa power plant

Paul V. Arnold, Noria Corporation

The old saw goes that there's no such thing as bad publicity. Any publicity, even the unflattering kind, is better than none at all.

The folks at Alcoa's Warrick Power Plant in Newburgh, Ind., would disagree with that. In 2001, the 732-megawatt, coal-burning plant was mentioned several times in the global corporation's 72-page annual report.

"Our reliability was noted as deficient three times," says plant manager Peter DeQuattro. "Reliability problems here at the power plant cost Alcoa $45 million, pre-tax, against 2001 earnings. That is not good when, out of 400 Alcoa locations, you get that type of recognition - three times - in the annual report."

The Warrick references served as an explanation/apology to investors for a massive power plant failure that severely impacted capacity at the company's neighboring aluminum smelting operations. The failure occurred on December 4, 2001. It took nearly four months for the power plant and smelter to return to normal function.

Improved reliability helped Alcoa's power plant in Newburgh, Ind., secure $440 million in capital project funding.

"Our primary purpose is to provide low-cost power," says power generation manager DeWayne Todd. "Low-cost power means being on and available. The smelting operation cannot be shut down. Our reliability is essential to their profitability and their ability to stay in business."

In this symbiotic relationship, when the smelter is cut, the power plant also bleeds.

"If we aren't supplying them, we are out of work," says Brian Hall, an operations employee who serves as the health and safety representative and head shop steward for International Brotherhood of Electrical Workers union Local 702.

December 4, though, was an event, not a symptom of the plant's past reliability issues.

"To be honest, back then, this place ran like any other utility," says Rick Fox, who was hired as the site's maintenance manager in January 2005. "When you do that, from a maintenance standpoint, you are more apt to let things go without performing planned and predictive work. You are a firefighter."

Steve Strahl, a mechanical maintenance team leader and an IBEW member, has pained memories of those days.

"We came to work and it was as if the place was burning down," he says. "A stack of emergencies was waiting for us. Operations handed off a list of things that were broke. The phone rang off the hook."

With 45 maintenance workers responsible for 7,000 plant assets (80 percent of which were installed in the 1950s and '60s), this was an overload situation.

Daily overtime and six- to seven-day workweeks were common, as were middle-of-the-night call-ins. Vacations and off-days were cut short - sometimes abruptly. A mechanic was once dispatched to Kentucky to find senior mechanical engineer Rick Mayer, who was on a hunting trip. The mechanic located Mayer in a tree stand, helped him close his camp and brought him back to the plant to solve a boiler feed pump problem.

Maintenance manager Rick Fox believes his department's success is due, in no small part, to the support of upper management. "(Alcoa Primary Metals location manager) Royce Haws and (plant manager) Peter DeQuattro understand that with greater reliability, maintenance is not a cost. It is an investment," he states.

Photos by Glenn Tang, Photics LLC

'DOING THINGS RIGHT'
Shift from that time over to early 2005. The subject of conversation (wouldn't you know it?) is publicity.

"I sat in front of the maintenance team and said that one of my goals was to get this plant on the cover of a magazine," says Fox. "Being on the cover means you are being held up for doing things right. I told them that if we do the right things as a group, we will be on the cover of a magazine."

By reading this article, you know that this goal was met. The Warrick Power Plant graces the cover of the November/ December 2007 issue of Reliable Plant and serves as the role model plant in the spotlight cover story.

What happened to turn this site from annual report bad boy to national publication hot shot? From one resigned to firefighting to one focused on planned, proactive solutions? From an unreliable plant to a reliable plant that boasts 90 percent planned work, 93 percent availability, 90 percent schedule compliance, 97 percent PM compliance, etc.? From a facility that feared for its survival to one that secured $440 million in capital investments from the parent company?

warrick-alcoa-058.jpg

Mechanical maintenance team leader Steve Strahl (left) and predictive maintenance technician Gary Seaton go over a task checklist before examining a row of generators at Alcoa's power plant in Newburgh, Ind.

Photos by Glenn Tang, Photics LLC

The Alcoa site altered its view on maintenance and reliability.

"Maintenance isn't about the maintenance folks. It's not about fixing things. It's about maintaining production capacity and production availability," says DeQuattro. "Reliability is the basic execution strategy to allow you to achieve profitability and, therefore, long-term viability. It's the basis of asset management. Your fundamental assumption is that the equipment is going to be available to produce the quantity and quality of product in the time you need to produce it. Maintenance and reliability is everyone's job and it is everyone's responsibility."

The plant made the most of a difficult situation.

It embraced change.

It worked hard to bust down silos that hindered growth.

The transformation was rapid, and the payoff spectacular.

warrick-alcoa-062.jpg

Operator Bob Braun, a 42-year Alcoa employee, monitors a control room with recent hire Adam Thorne.

SIBLING RIVALRY
Initial progress came after the power plant and smelter began to approach business/survival from a total enterprise perspective.

"The power plant used to be 'the necessary evil,'" says maintenance department controller Kim Woods.

"There were two silos, with fences around the silos and razor wire on the fences," adds Fox.

But Alcoa Primary Metals location manager Royce Haws backed a vision where a closer working relationship between these site siblings would create greater efficiencies, reduced total costs and, ultimately, long-term survival in southern Indiana.

"A big thing was where we stood on the company's cost curve," says staff mechanical engineer Bill Bloodworth. "We need to be low on that curve. This site as a whole has to produce cheap metal. The only way to produce cheap metal is to give them cheap power. Most of the other costs associated with producing metal are somewhat fixed. The only real delta that you can work at is power cost. That's what made us the key."

"We are working under a common vision now," says DeQuattro. "We are competing in a global commodities business. So as part of a team, we all want to do our share."

Reliability became the silver bullet in the push to reduce the enterprise-wide total cost to produce and distribute power.

warrick-alcoa-014.jpg

Safety is serious business for Brian Hall, the head shop steward for IBEW Local 702.

It made sense. If the equipment is more reliable, there is less downtime, fewer unplanned outages and fewer emergencies. Greater reliability generally means reduced maintenance costs and lower total costs for the power plant. And, if it can provide low-cost power as well as reliability (no off-line situations), the smelter does not have to procure power from the outside market, where the costs are generally two to three times higher.

"Royce Haws and Peter DeQuattro understand that with greater reliability, maintenance is not a cost. It is an investment," says Fox.

PROACTIVE STEPS

The only question was how to provide that type of reliability.

The smelter provided some inspiration. In early 2003, its maintenance leaders teamed with consultants from Life Cycle Engineering (LCE) and the Ron Moore Group to create an approach called the Reliability Excellence Process. It was seeing very good results.

The power plant observed aspects of the deployment and decided to bring Reliability Excellence (or Rx) to its operations. LCE visited in November 2004 and performed a formal assessment in January 2005. The initial assessment score wasn't bad - .528, which placed the site in the "emerging" category (LCE defines it as between .400 and .549), a step beyond "reactive" (.000 to .399) and on the doorstep of the "proactive" (.550 to .749) category. (Scores for the top level, "excellence", are .750 to 1.000). But the .528 number didn't tell the real story.

"We were a reactive organization," says Fox. "We had little things going on that were neat, but they weren't connected to anything. There was a wheel but not many spokes. The bottom line was that we needed to change our philosophy of how we do business. We needed to get out of the reactive mode and into the proactive mode."

In spring 2005, an Rx lead team began developing an initial master plan that laid out the chores and milestones that needed to be accomplished over the next three years. Three sub-teams were spawned to oversee work controls processes, storeroom supply and predictive maintenance. "They took their pieces of the master plan and started working through them," says Fox.

The "fly by the seat of your pants" environment was put under the microscope and dissected. Structure in the form of standardized processes was infused. This is the soil that allows the seeds of planning and scheduling to take root and grow.

Teams also performed a risk-based criticality analysis of all site assets to determine necessary levels of attention and day-to-day work priorities. Equipment that has the potential or ability to cause the most business-related risk was put at the top of the list. Equipment that is inherently less risky (as the result of redundancy, infrequent use, cost or other factors) slid down the list.

"We (internally or in discussions with operations) don't need to argue anymore about what must be worked on first, second, third, etc.," says Fox. "The criticality list determines the hierarchy of work."

warrick-alcoa-027.jpg

Maintenance technician Tom Anderson uses a Mikron camera equipped with a SpyGlass fish-eye lens to take infrared readings on an electrical cabinet.

BRAINS AND BUY-IN
By late summer 2005, the site was ready for roll-out and training. Fox and Todd provided a presentation and workshop to all plant workers. These sessions explained the Rx process and foundational work, how the master plan would be implemented, the roles that everyone would play in the transformation efforts, and how success would be ultimately measured. The culture flip began.

"One of the keys during the early stages was the union's involvement and help," says controls engineer Chad Estes. "The union leaders came out and supported it on the shop floor and operations floor."

"The lead team kept the union leadership well informed," says Hall. "They shared information and all of the numbers. We attended lead team meetings. As a result, the union leadership had the ability to say, 'We will support this. We will encourage our people to participate. We see the benefits.'"

As a carrot, Fox explained that converting to a proactive organization would allow mechanics and electricians the opportunity to better use their skills and knowledge.

"I don't want my guys leaving their brain at the gate," he says. "This job is about more than their brawn and backbone. I want their ideas, creativity and intellect."

There was also cash to be had.

Reaching various markers throughout the year entitles workers to earn performance bonuses. "That is money worth going after," says Fox.

An important element of Rx is the belief that a formalized, well-defined partnership is imperative between maintenance and operations. In this partnership, operations owns the equipment and has a primary stake in reliability. Maintenance is an equal partner dedicated to provide timely and effective methods, skills, expertise and support.

"It all starts with the operators," says Todd. "They are the eyes and ears of the plant on a daily basis."

Based on intuitive experience, condition readings and process data, they can inform maintenance when equipment is beginning to perform outside of optimal parameters. They also assist in asset health by performing a wide array of preventive maintenance (PM) tasks.

FOLLOWING THE PLAN
Full process implementation began in late 2005. With processes, training, union buy-in, management support and a solid foundation in place, the power plant experienced quick results.

Areas of emphasis and growth included:

Planning and scheduling: Three years ago, the maintenance organization would begin a shift, meet with operators and find out which assets were broke. Work was then doled out by a single "planner". The barb was that the plan was a blank sheet of paper. Today, regular work is mapped out two weeks in advance by a team of three planners and scheduled by maintenance and operations leaders.

"Everyone knows what they are going to do on a daily basis," says Fox. "They know what they have to accomplish. They know how many man-hours it takes and what it takes to accomplish it correctly."

The plan isn't a suggestion. It's an expectation that is measured and monitored at the highest level.

"Adhering to schedules is big," says DeQuattro. "We track ourselves every day and every week on how we have done vs. schedule. We have reasonable expectations (REs) for every person for every day. Someone should be able to get X amount of work done per day. We capture that and capture problems that keep them from reaching their REs. We Pareto those administrative- or productivity-prohibiting issues. We focus on the biggest bottlenecks and what costs us the most time and money."

Planning is also done months in advance for large-scale outages. There are 165 outage days slated for 2008. Each will be orchestrated down not only to the day, but in some cases the hour, in order to eventually track and ensure progress to schedule.

Maximize planned downtime: Careful planning also allows maintenance to perform and complete more ancillary work when an asset is scheduled to be down.

"We are getting more quality work done in a shorter period of time, which allows for better reliability while the unit is in operation," says Fox. "What used to be a three-week outage is now 12 to 14 days. That all comes from planning and executing the plan, and being prepared to execute that plan as soon as the unit comes off line.

"When you get more work done on the unit when it is down, it runs longer, it doesn't come down for every nickel-and-dime thing. We take care of everything that we can in that timeframe."

Asset history: Three years ago, the maintenance organization had no written equipment history.

"It was all tribal knowledge," says electrical planner Mark Payne. "Nothing was documented."

Similarly, blueprints, drawings and technical manuals were out-of-date and/or missing.

"We had to say, OK, the history starts now," says Fox.

Completed work orders and condition reports are now tied to individual assets and housed in the plant's Oracle software system (which went live in October 2003 and more effectively used beginning in 2005).

Clean-up activities have organized areas that house materials. Workers have examined prints, drawings and manuals, and flagged those in need of an update. Correct and current manuals are digitized and saved as PDF files on maintenance computers.

Today, a work order packet includes all of the information that the skilled worker needs to complete the job. It also includes an end-of-task feedback form that works to build history on that piece of equipment. The form is filled out by the technician and entered in Oracle by an administrative assistant.

Standardized work: Formal, written standardization of job procedures and processes was another major step. These instructions are drafted, reviewed and approved by mechanics, electricians and operators. They spell out, step by step, how a task should be done to best ensure safety, quality, reliability and timeliness. This information is housed in the work order packet.

"It has led to a change in mentality," says electrical maintenance group leader Dan DeCastra. "The entire workforce here realizes that reliability creates long-term gains. Take it out and do it the right way. You will be rewarded in the long run."

Focus on root cause: The plant has sought to eschew patchwork solutions and pursue remedies that address the root cause of a problem.

"It's not 'fix it and forget it.' It's 'let's go find the real reason so it doesn't happen again,'" says planner Tom Schmitt.

That is a profound change from the firefighting days.

"We used to be very adept at having a failure, working through it as quickly as possible and getting the unit back on line," says Fox. "Sometimes we had a failure with no known cause. We couldn't find what caused the failure, but we'd put the unit back on line. Now, a unit can't be put back on until we know what happened. That comes directly from the plant manager, and that is regardless of the cost. That, to me, shows the support from the main man on down."

To aid in problem solving, maintenance workers receive training in root cause failure analysis. They also learn how to use lean methods such as five-why critical thinking and fishbone diagrams.

Predictive maintenance: As referenced earlier in this article, Alcoa's Warrick Power Plant runs mostly on 40- and 50-year-old equipment. That provides challenges that are addressed with predictive technologies.

The plant's PdM arsenal consists of powerful tools - infrared thermography, vibration analysis, ultrasound, resistance temperature devices, reticulating borescopes, and (coming soon) fully integrated lubrication/oil standardization and condition monitoring processes.

"Even though this plant, in many cases, is about 50 years old, we have deployed some very high-tech equipment and trained the guys how to use it," says Fox. "If these guys ask me for equipment that is going to make us more efficient and more reliable, I'll get it. As a maintenance manager, you need to stay abreast of all of the new technologies. You have to separate what is good and usable for your facility and what is not."

The department has seen the biggest impact from infrared. A recent application allowed it to identify a hot spot in a transformer. A transformer failure would have cost the plant millions of dollars in repair cost and lost capacity. By catching the issue early and making the required repairs on a planned basis, the total cost was $55,000.

IR is also saving the department time and effort.

Alcoa works under electrical high-voltage safety standard 32.60, which is even more stringent than NFPA standard 70E. Arc flash hazard is a serious issue. As a result, to open an electrical compartment to do a standard infrared survey, two electricians need to don 100-calorie, Level 5 outfits known as "the astronaut suit." The suit is necessary but cumbersome. It normally takes two workers using this method four hours to scan the 18 breakers housed in one unit.

Relief came when infrared supplier Mikron offered a SpyGlass fish-eye lens for its infrared cameras. Alcoa installed small ports on the back of the electrical cabinets during outages. Now to perform an IR scan, one electrician takes the port cover off, puts the camera with SpyGlass lens to the port and shoots the back without opening any of the compartments. A 100-calorie suit is not required. Level 2 safety gear, which a power plant worker normally wears on the floor, is sufficient. The lens solution cut IR survey time per unit from four hours down to 20 or 30 minutes. And, one person handles the task.

RISING UP THE CHART
By adding spokes to the wheel, the Alcoa plant did indeed shift into a proactive mode. A second Reliability Excellence assessment, done by LCE in early 2006, pegged the site at .652, placing it in the heart of "proactive" territory on the consultancy's growth chart.

LCE says the 124-point increase is the biggest first-to-second-assessment jump in its history of deploying the Rx process. The smelter had one of the previous top improvements, rising 106 points (from .444 in 2003 to .550 in 2004).

DeQuattro downplayed any competitiveness between the two plants, but did note, "every now and then, you don't mind shooting a better score than your big brother."

A third assessment, done in January 2007, put the power plant 62 points higher, at .714 - 36 points shy of breaking into the "excellence" category. (That goal should be met with the fourth assessment, planned for June 2008.)

On top of that, the 36-month master plan was completed in 30 months.

All in all, it has been a herculean effort.

"We didn't have the time to wander our way through the desert," says Fox. "We couldn't make this a 10-year endeavor. We had a plan and we followed it. All of this wouldn't have happened without everyone in the plant participating."

PAYOFFS AND PAYOUTS
Besides a solid Rx score, there have been a host of additional benefits.

Most glaring and consequential is the $440 million capital investment bestowed upon the plant by Alcoa in 2005. Large-scale construction projects currently under way are changing the facility's footprint and skyline.

"We earned significant sustaining capital investments when many other places did not," says DeQuattro. "We have been rewarded because of our reliability-enhancing processes, the thorough analysis that we performed and the results that we have been able to deliver."

The investments cemented the plant's future, and the future of the reliability initiative.

"That eliminated any skepticism that still existed about the importance of reliability," says Hall.

Reliability has also paid off monetarily for Alcoa corporate. Alcoa is a market participant of the Midwest Independent System Operators, an entity responsible for reliable operation of the Midwest bulk power system. If the Indiana power plant needs to supplement power or replace power that it cannot deliver, it must buy it from MISO at the going market rate. Uptime and optimum performance has given the plant the opportunity to sell excess power or create additional power for a fee to meet high market demand.

"If we have one excess megawatt, we can sell it at the market value," says Todd. "That has had a very big impact on our business. It has created many opportunities to alter the way that we operate the smelting operation and the power system. All of that is made possible by the fact that we have significantly improved the reliability of the plant."

Increased reliability has significantly cut the cost of emergency work. Last year, the plant spent $2 million in repair costs for such activities. The plant is on pace to lower that figure to $650,000 in 2007.

Four years ago, power plant issues triggered nearly 90 potline interruptions at the smelter. This year, there have been seven.

"With increased reliability, the cost of the power and services we provide to the smelter is much lower," says Woods. "The costs for the entire enterprise are lower."

Alcoa workers are also cashing in. Fox says plant employees are on target to achieve a healthy performance bonus.

Other personal benefits?

  • A little more sanity. "Instead of running around to put out fire after fire, we have time now to sit down and discuss things that are coming up today, tomorrow and next week," says Strahl.

  • More family time. Eighty-hour workweeks are a thing of the past. A few years back, maintenance employees were forced to work additional weekend shifts 60 percent of the time. Today, the figure is practically zero. Any weekend time is designed specifically to reduce the backlog of work orders (which has gone from 26 weeks in 2005 to the current figure of nine weeks). "I think that has helped to improve the morale and quality of life for the employees," says Fox.

  • A better night's sleep. "Reliability, on a personal level, means I don't get calls in the middle of the night," says Todd. "When we were getting 90 interruptions, I got many of those late-night reliability calls. I live seven miles from the plant on the highway. I have come from a dead sleep to here at the plant in 11 minutes during a crisis. It happened multiple times per year. It hasn't happened this year at all. That, to me, is huge."

THE END? THERE IS NO END
Indiana and Kentucky newspapers have chronicled the new construction taking place in Newburgh.

Alcoa's recent annual reports have only had positive snippets about the Warrick Power Plant.

The maintenance department is featured on the cover of this magazine.

That type of publicity is golden.

"It's satisfying to see all of these guys who have worked so hard get some notice," says Fox.

Leaders admit that hurdles do remain.

DeQuattro says, "We need to continue to offset wage increases with productivity improvements in order to drive costs in the right direction."

Fox says, "We are currently at 97 percent compliance on our preventive maintenance work orders. This is one of the areas where I have had to jump up and yell a little bit. The only acceptable number is 100 percent."

Bloodworth says, "Much of the time, the equipment goes down, not because of the equipment, but because of the operator or maintenance guy. It's human error. We are undergoing performance training to help us eliminate most of those human errors."

But there's no need to apologize. Proactively going after dollars and cents, percentage points and the remaining elements that stand in the way of reliability makes sense. If there's no end to the work, there's no end to the potential payoff.

Subscribe to Machinery Lubrication

About the Author