February 2007 Archives

It was one year ago today that we launched the Emerson Process Experts blog. This welcome message began the journey into the blogosphere for us here at Emerson. From rereading the initial post, I think we managed to stay pretty true to the mission:

...to use a blog to create a shorter path between process manufacturing professionals seeking solutions and the experts around Emerson Process Management who have valuable industry, application, and technology expertise.

As I did in the welcome message, I continue to thank Deb, Ed, Maryann, and Lisa for their help and innovations along the way with this blog.

Surprises after a year? The biggest by far is the sheer number of folks I've met along the way, from fellow automation and process industry bloggers to even some of the A-list bloggers as defined by the Technorati Top 100 (I've got a ways to go at 25,600!) A week does not go by where I don't get an email, phone call, or Skype message from someone around the globe from plant managers to engineers and plant operators, looking around and happening upon one of the posts.

It's really the best part--hooking folks up with folks here at Emerson who can help them. And even as long as I've been here at Emerson, meeting all the folks featured in these posts has been great.

The worst part? Starting a week with no prospects or leads for blog posts and trying to scratch and claw to find something to post. I now have a much greater appreciation for my friends in the publishing business!

Thanks for this brief indulgence, and now on to year two!

In an earlier post I discussed common transactions between enterprise, manufacturing execution, and control systems. At the heart of this exchange of information is Extensible Markup Language (XML) to pass types of data between systems in a standard, text-based way.

For instance, if you subscribe to this blog's RSS feed, it means you have an RSS reader which translates the XML data in the RSS feed and displays it in a readable format.

Another example is our Google search appliance that crawls the Emerson Process Management website to help you find things faster. The search results are in XML with eXtensible Stylesheet Language (XSL) to make the returned search results readable. Also, the latest version of Microsoft's Office (Office 2007) switched from saving data in a binary format to an XML format.

The uses for XML extend far beyond these examples and include the work being done with OPC Unified Architecture standard and many more.

I caught up with Dave Marschall who is an integration consultant in our Life Sciences industry center. He shared with me how he and his team were using the XSL/XSLT stylesheets in the process of creating custom electronic batch reports which contain information from the enterprise planning systems, manufacturing execution systems and control systems. XML is commonly used to store this batch report data from these various systems.

The XSL/XSLT stylesheets allowed the team to create different renditions or views of the same XML data. A production view might include process operations events and alarms, operator comments, equipment usage statistics etc. A quality assurance/quality control view might contain material usage, lot history, expirations, environmental data, laboratory data requests/results, etc.

Dave described a recent project where the addition of Quality Specifications data allowed the customer to add intelligence to these views of information. Instead of just displaying the data in a tabular format, the XSL/XLST stylesheets could perform comparisons between actual results versus the specifications, and change the color or highlighting of any discrepancies.

This change of colors was used to help the process manufacturer quickly scan dozens of pages of report data and identify areas of concern like out-of-spec conditions. The logic also triggered additional batch details where these abnormal conditions occurred to assist in the review process. The net effect of embedding this intelligence into the batch end report was quicker reviews of the batch which meant quicker release of the final product.

By using a text-based standard XML and XSL/XSLT approach, the solution can be well documented and more easily changed over time to meet the changing needs of the process manufacturer.

Update: Welcome readers of Gary Mintchell's Feed forward blog! Also, it was interesting timing to get a ControlGlobal.com email this morning discussing The ABCs of XML, Parts 1, 2 & 3.

Recently the Digital Bond cyber-security blog pointed to a vulnerability note on the Trend Micro anti-virus package and noted:

Software designed to protect ends up putting virtually every system on your network at risk.

This had to cause pause among anyone reading this blog. In spite of best efforts to ward off the ravages of viruses, even those packages responsible for protection can be compromised.

When I saw this article appear in my RSS feeds, I ran it by our resident cyber-security expert and DeltaV product manager, Bob Huba, who you may recall from earlier cyber-security posts. Bob noted that the unfortunate reality of today's highly interconnected world is that new vulnerabilities come up all the time.

Constant vigilance must be a critical part of your cyber-security efforts. Patch, patch, patch is your best practice and this process requires a close partnership between you and your automation suppliers to perform these practices. Suppliers have to do their part with prompt patch certification and anti-virus support to make sure these patches and additions don't break existing software functionality. This certification must get clearly communicated to the automation system/cyber-security administrators around the globe to quickly plug these vulnerabilities.

One thought Bob shared is that perhaps some installed automation systems are "over connected" with the enterprise. Each connection is a source of vulnerability and its business case should be carefully considered. The connections should be designed more with cyber-security than low cost in mind. It's likely best to have a single external connection from these automation systems that "you can guard like heck."

An analogy is the bank vault with layer upon layer of security which serves to slow down potential breaches so that they can be discovered and thwarted in time.

It sounds like the choice is either to heavily devote attention to these connections or lock down and disconnect from the network--something not too practical as process manufacturers try to optimize their business processes and manufacturing systems.

The folks at Mynah Technologies with their Mimic simulation software, virtual I/O network gateways, PLC I/O interfaces and host of drivers for the DeltaV system continue to build conversations through their forums and experts blogs within the forum. Recently Dr. Aleksandr Muravyev a simulation consultant created a Mimic Distillation Modeling package to simplify software acceptance testing and operator training around distillation columns.

Dr. Muravyev joins the automation blogging fray with this post about this distillation modeling package. If you have experiences with this product (or even if you don't) feel free to join in the conversation. It's a forum so you'll need to register first.

I did ask the Mynah folks if the forum software they are using supports RSS feeds so I could get these updates coming to me instead of going out and seeking it. It does and they will soon be adding this functionality.

That's a great thing as some RSS readers like Google Reader support both regular and mobile viewing. It means I can keep up with the automation and technology bloggers whenever a have some spare moments. From playing with the various RSS readers, I seem to be gravitating to using Outlook 2007's embedded RSS reader for my Emerson intranet-based RSS feeds and Google Reader for external RSS feeds. How about you?

Manufacturers increasingly look to optimize their businesses by integrating their business processes with their manufacturing processes. ANSI/ISA-95 (S95) is an international standard for developing an automated interface between enterprise and control systems.

Over the last several years our Life Sciences Industry Center has been working with pharmaceutical and biotech manufacturers on this optimization process in pursuit of operational excellence. I caught up with integration consultant, Dick Seemann, who discussed some commonalities he sees in integration transactions between enterprise planning systems like SAP business software, manufacturing execution system (MES) software and control systems like the DeltaV system.

A key transaction is the process order download which comes from the planning and scheduling software to the manufacturing system in the form of a request for a campaign of process orders or a single process order. The transaction contains a process order number, material components, equipment requirements, and specific parameters that are exchanged through standard web services. The team typically uses Compliance Suite as the MES software between the SAP software and the DeltaV system.

The MES software combines this information from the production, material management, and quality management software and then manages the order execution, performs weigh and dispense, and executes the manual work instructions in conjunction with the automated control system tasks. A complete electronic batch record is maintained at MES level, since it combines recorded manual processes and procedures with electronic information at the control system and enterprise planning system levels. Alarms, events, operation actions, and batch history are passed as transactions from the control system to the MES.

The process orders as the campaigns are being executed provide status transactions back to the planning systems on status changes during execution and upon completion of the process order. Materials consumed during the production and recorded by the manufacturing execution system are passed back as transactions to the material management module to accurately reflect what is available for planning future production.

Start and end times for each of the steps in the batch are also passed back as transactions to the planning systems to provide and accurate picture of equipment utilization and how long production steps take compared to standard times.

Dick notes that all process manufacturers have different business processes and that there are many more transactions that can occur between the three levels. As these paper-based processes are moved into an integrated, transactional world, end-to-end cycle times are reduced resulting in greater manufacturing efficiency.

Transcription errors are also reduced when all communications is electronic-based versus the paper batch sheets. Also, troubleshooting problems becomes easier when a history of electronic batch records is available to review to analyze where inefficiencies have begun to occur.

I recently came across a lubricant analysis story in Plant Services magazine about an electric power provider in Kansas in the U.S. It described how this producer wanted to reduce costs and improve machine reliability. The article described the importance of comprehensive oil analysis and the decision to do analysis in house with Emerson's CSI 5200 minilab versus sending the samples offsite.

I caught up with Ray Garvey, an engineer in the Machinery Health Management organization about this project. Ray said that Mark Mayworm and the team at Westar's Jeffrey Energy have configured a solution that is ideally suited for this collection of six power plants. Westar is one example of the truth in Drew Troyer's words:

Every successful oil analysis program I have observed has passionate technicians performing the work. And almost without exception, each includes some degree of onsite oil analysis.

Ray is convinced that a combination of these two things: passionate technicians and some degree of onsite oil analysis have produced a successful lubrication program for Westar.

Ray mentioned other documented cases, who like Mark Mayworm, have followed this lube program success formula and it has paid off:

Jaime Viramontes and others supporting El Paso Electric's PdM program achieved cost avoidance of $8.8 million over a period of three years. Jaime reports, "The oil analysis program has been a successful and integral part of EPEC's PdM/RBM program."

Ed Bohn documented 738% Return on investment with 2 month payback period for investment in minilab and training by General Motors.

Dennis Roinick and the entire PdM Team at Duke McGuire nuclear station won "Best Overall Predictive Maintenance Program" in Uptime's Program of the Year competition in recognition for their fully integrated program which includes an on-site minilab.

Herb Springer gives this reason for the reliability success resulting from on-site oil analysis at each of more than a dozen Southern Company plants, "The results I get from doing onsite oil analysis are more representative of the health of the machine at that moment."

Richard Kus of American Axle and Manufacturing found savings of $75,000 to $100,000 using on-site oil analysis.

Mike Lenz and others from P&H Minepro Services transport their minilabs to the mine sites as one part of their predictive diagnostic services in North and South America.

These folks and many others are Ray's heroes. Each in their own way is a champion for the cause of better lubrication practices in their diverse plant situations. Ray confided with me that he was one of a dozen developers who set out in 1991 to design cost-effective on-site oil analysis solutions and then to build credibility for those solutions.

For an engineer and inventor like Ray, there is a huge personal reward every time one of these "passionate technicians" calls in to say, "Hey, this is great! Let me tell you what I was able to find using your onsite minilab..."

I came across the following nine control fundamentals according to Mark Coughran, a consultant on Emerson's Advanced Applied Technology team. These are based up his years of experience working with process manufacturers to optimize their performance. You may recall Mark from earlier posts on planning plant turnarounds and turbomachinery pressure control.

His fundamentals include:

1. Make the process as linear as possible
2. Minimize dead time
3. Choose the PID controller to compensate for the process
4. Avoid resonance or amplification of disturbances
5. Use process capacity to absorb variability
6. Decouple the interactions by tuning if possible
7. Help the PID feedback controller with control strategy
8. Cascade, ratio, feedforward; a.k.a. advanced regulatory controls
9. Use Fuzzy, Neural, MPC if the above are insufficient

Although there is a lot behind each one, it's a way to think through the process of solving control performance issues.

Mark cited an example of a Good Automated Manufacturing Practice (GAMP) facility with eight reactors. All of the controls cycled strongly. As a result, steam was being wasted on the up cycle, and cooling water was being wasted on the down cycle. Thinking through the control fundamentals above, Mark recommended changes in the master controller parameters including tuning, jacket controller tuning, split range strategy, and control valve calibrations.

By implementing these changes on three of the eight reactors steam usage for the facility was reduced 10% reducing the plant's energy bills.

You have to admire the way a team of engineers when presented with a challenge, come up with a better, less costly approach. Such is the case with a local control panel for a safety valve that Emerson Fisher division's Riyaz Ali showed me. You may recall Riyaz from earlier posts on the topic of safety.

The challenge is that safety shutdown valves with conventional local control panels have typically required ten input/output connections between the safety system's logic solver, local control panel, solenoid and digital valve controller as the picture indicates. These panels get hard wired signals from the safety instrumented system's logic solver for light indication of valve Open, Close, and Ready to Reset. Also, if the logic solver needs to open the valve after "Ready to Reset" light indicator, "Valve Open" signal needs to be sent to local controller for field technician to open the valve on separate pair of wire. It will also require an additional I/O for shutting the valve from local controller in case of an emergency.

Now, many plants keep metrics on what it costs to install each I/O point, but a ballpark figure of $2,000 USD per I/O point is typical.

The approach Riyaz describes is based on the Fisher LCP100 local control panel which requires 5 I/O. This means roughly $10,000 savings per installed smart local control panel. If your facility is a refinery, petrochemical, or chemical plant, this could add up, based on your number of safety valves with local control panels. This panel digitally communicates directly with Emerson's Fisher DVC6000 digital valve controller to eliminate the need for separate wiring for Valve Open and Close indication, Ready to Reset indication, and pushbuttons for manual Valve Open and Close. These digital communications also provide diagnostics to reduce the ongoing costs of maintenance typical with hard-wired solutions.

Riyaz also points out the digital valve controller can provide on-line diagnostics and partial-stroke testing to assist the process manufacturer in checking the safety instrumented function which includes these shutdown valves.

As with most digital communications, the long term benefits in diagnostic coverage with this integrated approach are usually greater than the initial benefits in installation cost savings.

My colleague, Deb Franke, who was instrumental in getting this Emerson Process Experts blog going nearly a year ago, pointed me to an excellent article in Biopharm International magazine.

The article, ON-LINE PROCESS CONTROL: Automating the Control of Process-Scale Purification Columns Using On-Line Liquid Chromatography describes how Lilly used liquid chromatography to measure product purity in near real-time. This is an example of how pharmaceutical and biotech manufacturers are using the guidance from the FDA's Process Analytical Technologies (PAT) initiative to improve their operations. I'll warn you that this article is pretty technical and suited best for those Chemical Engineers among us.

I ran this article by our Life Sciences industry experts to see if they were involved in this or any similar efforts. Lead engineer Brian Crandall, whom you may recall from an earlier post, sent me a note back how his team had recently completed a project for a biotech manufacturer. They wanted an integrated way for their operators to interact with the chromatography columns with the goal of achieving process repeatability and more quantifiable product quality.

Brian and his team developed a solution based upon the ISA S88 modular standard. The major pieces of equipment used on the chromatography skid including dilution control, feed pumps, columns and valve arrays were grouped into processing categories to provide the operators a way to efficiently operate this skid. They can start new operations, hold ones that are currently running, and respond to abnormal processing conditions to take corrective action before quality is impacted.

Also critical was collecting the data from the chromatographs to do analysis around product optical density trends and equipment performance metrics. The DeltaV Batch automation system was also integrated with manufacturing execution system (MES) software to provide material genealogy with the manual and automated process information. This genealogy record included the materials transferred into and out of the chromatography skid. The solution for this information integration was based upon the ISA S95 information standard between enterprise and control systems.

To tie this information back to the business objectives of repeatability and improved quality, Brian and the team created a final batch report that displays the complete product history through the chromatography process. As stated on the PAT site:

The goal of PAT is to understand and control the manufacturing process, which is consistent with our current drug quality system: quality cannot be tested into products; it should be built-in or should be by design.

This design, integration and implementation work by Brian and team sounds exactly like what this goal is meant to achieve.

Update: Welcome Wall Street Journal readers!

I'm checking back in with Emerson Education Center's John Egnew on the great work he continues to do with his Loop Tips.

Since I last checked in, John has added two more tips: The Tricky Transmitter, and The Process that Changes its Mind.

The tricky transmitter tip deals with the case where you might perform transmitter zero and span calibration (or re-ranging.) The operators may notice a change in the loop performance which changes the gain in the transmitter. John shows the math and the corrective action to take (changing the controller gain for those of you who don't follow hyperlinks J. )

In the process that changes its mind, John discusses the case where loops behave differently at different times, going between stable and unstable operation. Most processes have changing process gains. Adjustments to controller gain can be done manually (for very slow changing processes) or through adaptive gain control strategies.

As software continues to get more sophisticated, it can help monitor control performance in each of the loops. By identifying and diagnosing problem loops, recommending tuning and maintenance improvements and continuously adapting to these changing process conditions the effects of nonlinear gain can be mitigated. DeltaV InSight is an example of this type of control performance software.

I need to work with the team to figure out how we can get an RSS feed of these great tips so that anyone interested in keeping up with these tips can subscribe to John's excellent work. There I go again harping about RSS!