How can we Measure the Effectiveness of an FMEA

[AndyN]

IMHO it's futile to try to evaluate an FMEA on it's own. As a tool used within the overall product/process development process, it's going to be difficult to separate out. If the previous work/tools are flawed, then the FMEA will be similarly flawed. For example, the (product design) FMEA is normally developed from the customer, regulatroy and organization's requirements. If the processes which develope those are flawed the resulting FMEA will not necessarily be any better. Similarly with the process FMEA; if the process flow is inaccurate the FMEA will be too, and subsequently so will the Control Plans (for both phases).

I posed the question as to why would anyone want to know this answer - to justify why they should be done? The justification is in looking at the ($$$) results of not doing them! Every organization can look at their failures and have someone say - "Didn't you know that was going to happen? It happened before............"

If I had a $1 for everytime I've heard that.......

 

[Stijloor]

Friends,

I have been following the discussion for a while. The tendency is to unnecessarily complicate and overanalyze the FMEA process.

I stand by my definition. See post #9.

Stijloor.

 

[tymer5]

Hi Tymer5

Nicely done.

Is this a theoretical construct or is this actually happening where you consult or work?

Are you describing "as it should be" or "as it is"

Please give me hope, I'd like to hear someone is actually doing FMEA for all the right reasons, not just to fill up a tab in a PPAP binder.

Caster,

The advice given is not just theoretical. We do much of this work. Of course not everyone does it as it is supposed to be done, and that is why it is necessary to have three key aspects to making FMEAs stick in an organization. They are consistent training programs that target key audiences, resources (experts) to coach and mentor, and metrics at every level (the individual FMEA, the project, the business unit, and the corporation).

An FMEA is not just a document one completes; it is a process a team goes through. Like any process it is available for process improvement. It must be mapped, analyzed and controlled. First we need to understand the value a comprehensive FMEA program brings to a product or process. The answer lies in the designs or the processes we implement. We must understand that when we create a design we not only design in the requirements and features, but we also design in all of the failure modes. The failure modes are there whether we know of them or not. A comprehensive FMEA program provides an organization with a consistent, systematic method of exposing those failure modes, and prioritizing them for corrective actions. FMEAs do not solve the problems, they identify them. The corrective actions, the preventative controls, the design verification plans and the process control plans are the ACTIONS that address the problems identified.

WOW! Now that we understand the value of the FMEAs we can begin to identify what we need to measure, and what we need to control within the FMEA process. It is apparent that FMEAs can not add the value by themselves, so we must measure more than just the FMEA. When we think of just filling out a tab in a PPAP binder, we are not measuring the value; we are measuring the ability to create a required document. There is value in the activity of proving internal and external suppliers are capable of proving the work has been done. However this should not be confused with the value a consistent, systematic method of exposing and prioritizing built-in failure modes provides.

Understand that people hold a strategic place in this process. That is why two of the three key aspects to making FMEAs stick in an organization are oriented toward people (training and mentors), and the third (metrics) measure people in many cases. If you map and analyze this process you will find most controls needed are on people.

 

[AndyN]

Caster,

The advice given is not just theoretical. We do much of this work. Of course not everyone does it as it is supposed to be done, and that is why it is necessary to have three key aspects to making FMEAs stick in an organization. They are consistent training programs that target key audiences, resources (experts) to coach and mentor, and metrics at every level (the individual FMEA, the project, the business unit, and the corporation).

An FMEA is not just a document one completes; it is a process a team goes through. Like any process it is available for process improvement. It must be mapped, analyzed and controlled. First we need to understand the value a comprehensive FMEA program brings to a product or process. The answer lies in the designs or the processes we implement. We must understand that when we create a design we not only design in the requirements and features, but we also design in all of the failure modes. The failure modes are there whether we know of them or not. A comprehensive FMEA program provides an organization with a consistent, systematic method of exposing those failure modes, and prioritizing them for corrective actions. FMEAs do not solve the problems, they identify them. The corrective actions, the preventative controls, the design verification plans and the process control plans are the ACTIONS that address the problems identified.

WOW! Now that we understand the value of the FMEAs we can begin to identify what we need to measure, and what we need to control within the FMEA process. It is apparent that FMEAs can not add the value by themselves, so we must measure more than just the FMEA. When we think of just filling out a tab in a PPAP binder, we are not measuring the value; we are measuring the ability to create a required document. There is value in the activity of proving internal and external suppliers are capable of proving the work has been done. However this should not be confused with the value a consistent, systematic method of exposing and prioritizing built-in failure modes provides.

Understand that people hold a strategic place in this process. That is why two of the three key aspects to making FMEAs stick in an organization are oriented toward people (training and mentors), and the third (metrics) measure people in many cases. If you map and analyze this process you will find most controls needed are on people.

I like this perspective. One of the (many) benefits of doing an effective PFMEA is the awareness and cultural change that can happen between the participants - if done correctly.

I also believe that it's not the PFMEA per se that should to be shown to be effective, its the PFMEA plus the actions that are identified and taken that are more important. In fact, it's really the effectiveness of the development process of which the PFMEA is a part, not the individual tool, that's most worthy of measurement.

 

[tymer5]

Exactly Andy! You can't measure how much money you saved the company by avoiding problems. However you can measure the organizations commitment to the process. This is evident in the training, and the dedication of resources (people) to the process. You can also look for the characteristics of good FMEAs within the FMEAs, and adjust your training to address any deficiencies.

One of the greatest problems I see in implementing a good FMEA culture is that people who do good FMEAs get promoted or move on (especially in high mobility companies). So there is always a void that needs to be filled. Again this is why it is important to have a consistent and systematic training program. Over time the company culture changes because more and more FMEA practitioners are promoted into managerial and decision making positions within the company.

I am a process guy so I like to think of it this way. In a heat treating process you can't measure the material properties easily, so you must focus on the process parameters that have the greatest effect on creating the material properties you are looking for. In the case of FMEAs the most critical parameters are the people doing the work. If they are well trained and given the proper environment (managerial attitude toward FMEA, time etc.) they will produce a good FMEA, with actions they have or are taking to reduce discovered risk.