How can we Measure the Effectiveness of an FMEA

#21
IMHO it's futile to try to evaluate an FMEA on it's own. As a tool used within the overall product/process development process, it's going to be difficult to separate out. If the previous work/tools are flawed, then the FMEA will be similarly flawed. For example, the (product design) FMEA is normally developed from the customer, regulatroy and organization's requirements. If the processes which develope those are flawed the resulting FMEA will not necessarily be any better. Similarly with the process FMEA; if the process flow is inaccurate the FMEA will be too, and subsequently so will the Control Plans (for both phases).

I posed the question as to why would anyone want to know this answer - to justify why they should be done? The justification is in looking at the ($$$) results of not doing them! Every organization can look at their failures and have someone say - "Didn't you know that was going to happen? It happened before............"

If I had a $1 for everytime I've heard that.......:notme:
 
Elsmar Forum Sponsor

Stijloor

Staff member
Super Moderator
#22
Friends,

I have been following the discussion for a while. The tendency is to unnecessarily complicate and overanalyze the FMEA process.

I stand by my definition. See post #9.

Stijloor.
 
T

tymer5

#23
Hi Tymer5

Nicely done.

Is this a theoretical construct or is this actually happening where you consult or work?

Are you describing "as it should be" or "as it is"

Please give me hope, I'd like to hear someone is actually doing FMEA for all the right reasons, not just to fill up a tab in a PPAP binder.
Caster,

The advice given is not just theoretical. We do much of this work. Of course not everyone does it as it is supposed to be done, and that is why it is necessary to have three key aspects to making FMEAs stick in an organization. They are consistent training programs that target key audiences, resources (experts) to coach and mentor, and metrics at every level (the individual FMEA, the project, the business unit, and the corporation).

An FMEA is not just a document one completes; it is a process a team goes through. Like any process it is available for process improvement. It must be mapped, analyzed and controlled. First we need to understand the value a comprehensive FMEA program brings to a product or process. The answer lies in the designs or the processes we implement. We must understand that when we create a design we not only design in the requirements and features, but we also design in all of the failure modes. The failure modes are there whether we know of them or not. A comprehensive FMEA program provides an organization with a consistent, systematic method of exposing those failure modes, and prioritizing them for corrective actions. FMEAs do not solve the problems, they identify them. The corrective actions, the preventative controls, the design verification plans and the process control plans are the ACTIONS that address the problems identified.

WOW! Now that we understand the value of the FMEAs we can begin to identify what we need to measure, and what we need to control within the FMEA process. It is apparent that FMEAs can not add the value by themselves, so we must measure more than just the FMEA. When we think of just filling out a tab in a PPAP binder, we are not measuring the value; we are measuring the ability to create a required document. There is value in the activity of proving internal and external suppliers are capable of proving the work has been done. However this should not be confused with the value a consistent, systematic method of exposing and prioritizing built-in failure modes provides.

Understand that people hold a strategic place in this process. That is why two of the three key aspects to making FMEAs stick in an organization are oriented toward people (training and mentors), and the third (metrics) measure people in many cases. If you map and analyze this process you will find most controls needed are on people.
 
#24
Caster,

The advice given is not just theoretical. We do much of this work. Of course not everyone does it as it is supposed to be done, and that is why it is necessary to have three key aspects to making FMEAs stick in an organization. They are consistent training programs that target key audiences, resources (experts) to coach and mentor, and metrics at every level (the individual FMEA, the project, the business unit, and the corporation).

An FMEA is not just a document one completes; it is a process a team goes through. Like any process it is available for process improvement. It must be mapped, analyzed and controlled. First we need to understand the value a comprehensive FMEA program brings to a product or process. The answer lies in the designs or the processes we implement. We must understand that when we create a design we not only design in the requirements and features, but we also design in all of the failure modes. The failure modes are there whether we know of them or not. A comprehensive FMEA program provides an organization with a consistent, systematic method of exposing those failure modes, and prioritizing them for corrective actions. FMEAs do not solve the problems, they identify them. The corrective actions, the preventative controls, the design verification plans and the process control plans are the ACTIONS that address the problems identified.

WOW! Now that we understand the value of the FMEAs we can begin to identify what we need to measure, and what we need to control within the FMEA process. It is apparent that FMEAs can not add the value by themselves, so we must measure more than just the FMEA. When we think of just filling out a tab in a PPAP binder, we are not measuring the value; we are measuring the ability to create a required document. There is value in the activity of proving internal and external suppliers are capable of proving the work has been done. However this should not be confused with the value a consistent, systematic method of exposing and prioritizing built-in failure modes provides.

Understand that people hold a strategic place in this process. That is why two of the three key aspects to making FMEAs stick in an organization are oriented toward people (training and mentors), and the third (metrics) measure people in many cases. If you map and analyze this process you will find most controls needed are on people.
I like this perspective. One of the (many) benefits of doing an effective PFMEA is the awareness and cultural change that can happen between the participants - if done correctly.

I also believe that it's not the PFMEA per se that should to be shown to be effective, its the PFMEA plus the actions that are identified and taken that are more important. In fact, it's really the effectiveness of the development process of which the PFMEA is a part, not the individual tool, that's most worthy of measurement.:2cents:
 
T

tymer5

#25
Exactly Andy! You can't measure how much money you saved the company by avoiding problems. However you can measure the organizations commitment to the process. This is evident in the training, and the dedication of resources (people) to the process. You can also look for the characteristics of good FMEAs within the FMEAs, and adjust your training to address any deficiencies.

One of the greatest problems I see in implementing a good FMEA culture is that people who do good FMEAs get promoted or move on (especially in high mobility companies). So there is always a void that needs to be filled. Again this is why it is important to have a consistent and systematic training program. Over time the company culture changes because more and more FMEA practitioners are promoted into managerial and decision making positions within the company.

I am a process guy so I like to think of it this way. In a heat treating process you can't measure the material properties easily, so you must focus on the process parameters that have the greatest effect on creating the material properties you are looking for. In the case of FMEAs the most critical parameters are the people doing the work. If they are well trained and given the proper environment (managerial attitude toward FMEA, time etc.) they will produce a good FMEA, with actions they have or are taking to reduce discovered risk.
 
Thread starter Similar threads Forum Replies Date
qualprod ISO 9001 5.1.1 - How to measure System effectiveness ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
qualprod Best criteria to measure Corrective Action effectiveness - Poor Maintenance ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
M How to measure effectiveness and efficiency of the established QMS as Auditors ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
J How to measure effectiveness of the Metrology Process General Measurement Device and Calibration Topics 5
Chennaiite How to Measure Effectiveness of the Design Validation Process? Design and Development of Products and Processes 7
V How to Measure Effectiveness of Risk Assessment Processes FMEA and Control Plans 7
M How To Measure Training Effectiveness Quality Tools, Improvement and Analysis 8
Q How to measure Effectiveness in Tube Repairments? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
C How do you measure the Effectiveness of a QMS (Quality Management System) Quality Manager and Management Related Issues 34
D How to Measure QSR Training Effectiveness? Training - Internal, External, Online and Distance Learning 6
E How to Measure Internal Audit Effectiveness General Auditing Discussions 12
J Effectiveness of training? How to Measure Training Effectiveness Training - Internal, External, Online and Distance Learning 51
T How to measure the Effectiveness of a Quality Management System? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
T Need to design a test to measure inspection effectiveness Design and Development of Products and Processes 9
L Product Preservation - Measure of effectiveness IATF 16949 - Automotive Quality Systems Standard 12
Raffy How do I measure Measuring Training Effectiveness? Performance evaluation? Other ISO and International Standards and European Regulations 26
S Measuring Training Effectiveness - How do you measure Effectiveness? Training - Internal, External, Online and Distance Learning 50
C First Article Inspections - 'how to measure' General Measurement Device and Calibration Topics 11
lanley liao Question regarding the calibration of monitoring and measure equipment. Oil and Gas Industry Standards and Regulations 0
J Should a Class 1 medical device with an option to measure body weight be considered Class 1m? EU Medical Device Regulations 0
M Gage R&R and right way to measure Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 16
P How to measure DPPM in Chemical batch manufacturing Quality Tools, Improvement and Analysis 1
silentmonkey How to measure severity if my product is designed for emergency use and failure would result in death? ISO 14971 - Medical Device Risk Management 9
A Looking for a third party to calibrate our measure equipment - South Carolina General Measurement Device and Calibration Topics 6
J Validity / outcomes measure for custom made medical device ISO 13485:2016 - Medical Device Quality Management Systems 2
D Performance specification as a Risk Control Measure, EN 14971 ISO 14971 - Medical Device Risk Management 7
J KPIs or Metrics to Measure a New Complaint Handling Process 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 9
D Torque fractional unit of measure; cN-m or N-cm? General Measurement Device and Calibration Topics 2
J How to measure a Critical Characteristic that is a Basic Dimension Misc. Quality Assurance and Business Systems Related Topics 6
I Can we use pin gauges to measure an accurate Cpk ? Capability, Accuracy and Stability - Processes, Machines, etc. 9
1 How to measure straightness on a HEX Long Bar Manufacturing and Related Processes 7
Ron Rompen Measurement Method Challenge - Measure feature #91 Inspection, Prints (Drawings), Testing, Sampling and Related Topics 9
G Measure 3 times, for calibration report? Customer Owned Reference Standard General Measurement Device and Calibration Topics 4
M Informational The ANSM decides, as a precautionary measure, to withdraw from the market macrotextured breast implants and breast implants with polyurethane-coated s Medical Device and FDA Regulations and Standards News 0
M Informational Health Canada advises Allergan of its intent to suspend its licences for Biocell breast implants as a precautionary measure Medical Device and FDA Regulations and Standards News 0
U SI 13 - Not Every Process Requires an Efficiency Measure IATF 16949 - Automotive Quality Systems Standard 1
P NPR Metric - What if Customer has relaxed measure? TL 9000 Telecommunications Standard and QuEST 1
T Tape Measure, the right device for 2 or 3 decimals accuracy on prints General Measurement Device and Calibration Topics 8
M How do we document and measure a service we do not provide? IATF 16949 - Automotive Quality Systems Standard 1
M For a Type 1 Gage R&R I want to use 1 unit and measure it 30 times Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
M How to Measure Paper considering Variation and Humidity General Measurement Device and Calibration Topics 10
rob73 Advice/recommendations for equipment to measure a soft pvc tube - Optical Measuring? General Measurement Device and Calibration Topics 4
R Height gauge to measure short hard to access heights General Measurement Device and Calibration Topics 3
D Proper Tool to Measure Wall Thickness (ISO 3611 - Micrometers) Manufacturing and Related Processes 7
S Quality Measure for Engineering Services Misc. Quality Assurance and Business Systems Related Topics 1
A What are methods and tools to measure Workload analysis ? Human Factors and Ergonomics in Engineering 1
M How do I measure variation in my process ? Capability, Accuracy and Stability - Processes, Machines, etc. 3
J How to measure process quality? Is DPPM the correct metric? Capability, Accuracy and Stability - Processes, Machines, etc. 5
M Micrometer Anvil Placement to measure the diameter of Steel Wires Measurement Uncertainty (MU) 6
S Where to Get SPC Training to Measure our Processes? Training - Internal, External, Online and Distance Learning 12

Similar threads

Top Bottom