How to classify several systems for Quality

Quality27

Involved In Discussions
#1
Hello,

I work for a large startup and we are currently in the design phase of a large system. We have identified several levels of systems that are currently in development. As one of the initial steps in Quality, I am trying to classify these systems as 1,2,3,and 4 to help with the level of Quality Assurance activities we need to undertake based on the criticality of the system. Please note that this is a high level classification of systems to provide initial guidance to design, procurement and manufacturing. We will be conducting FMEAs, etc later when we have better clarity on designed components.

How do I go about this criticality classification? Function failure impact, Loss of function during operation, System complexity are some I have on mind but nothing gives me a clear logic that I am looking for.

For example, at the main level, I have 20 systems that make up our final system. Each of these main level systems have 3-5 sub-systems and these sub systems have 5-10 super sub-systems each. If I start with my analysis and a particular main level system has a very higher impact of failure, I classify it as Level 1 within my 1,2,3,4 classification system. Then all the sub-systems and super-subsystems of this main level system get the same classification as any sub-system or super-subsystem failure would cause the main system to fail. Then this becomes a futile exercise for me as the intent of my classification is to put different levels of control based on the criticality of the system. Ideally, my goal within each system is to identify which ones belong to a critical, semi-critical, etc system so that I can build redundancy in design, slect higher end suppliers, verify conformance through inspection and testing so that I have more confidence in the overall functionality of the system we are building.

If anyone has been in similar scenarios or have thoughts to share regarding this predicament I have, I appreciate very much. Suggestions on any different approach are also welcome.

Thanks,

Tony
 
Elsmar Forum Sponsor

Steve Prevette

Deming Disciple
Leader
Super Moderator
#2
From my experience with nuclear systems, and also explosives (torpedo maintenance) you generally end up starting from the safety perspective. What is the worst thing that can happen from a safety perspective if something goes wrong. Conventional Explosion? Nuclear Criticality? Injury? Spread of hazardous materials? Spread of radioactive contamination? This starts to dictate the safety and quality controls we put in place, such as reader-worker with signoffs for procedures, versus do the steps in any order you want. And we do put in layers of controls so we don't get to some of the worst case scenarios.

For systems I've dealt with, once we get past the safety issues, then we look at - will the effectiveness / capability of the product be compromised. Issue - the USA went to war in WWII with faulty submarine torpedoes - they did not go "boom" when they hit the enemy ship. Beyond effectiveness, then comes cost effectiveness, preventing budget overruns. Then we tailor the controls as needed.

The US DOE established the Integrated Safety Management System that worked through much of this. See DOE G 450.3-3 TAILORING FOR INTEGRATED SAFETY MANAGEMENT APPLICATIONS

quoting
The Department of Energy (DOE) Integrated Safety Management (ISM) system must support many different kinds of work, from the operation of nuclear and non-nuclear facilities to laboratory experimentation to environmental restoration activities. To accomplish the work safely, and to protect workers, the public, and the environment, the system must function to identify and control all types of hazards, from commonly encountered workplace hazards to rare or one-of-a kind process hazards,
 

Steve Prevette

Deming Disciple
Leader
Super Moderator
#3
I should also point out from a strictly quality perspective, the classification of minor, major, and critical defects which represents another good risk logic.
  • A Minor defect is a discrepancy from the standards, but one that is not likely to affect the usability of an object.
  • A Major defect is one that is likely to create failure of the unit for its intended purpose.
  • A Critical defect is one that is deemed to be hazardous or unsafe.
From svw5304.tmp (elsmar.com) (MIL STD 105E, here on the COVE).
 

Quality27

Involved In Discussions
#4
s, and also explosives (torpedo maintenance) you generally end up starting from the safety perspective. What is the worst thing that can happen from a safety perspective if something
From my experience with nuclear systems, and also explosives (torpedo maintenance) you generally end up starting from the safety perspective. What is the worst thing that can happen from a safety perspective if something goes wrong. Conventional Explosion? Nuclear Criticality? Injury? Spread of hazardous materials? Spread of radioactive contamination? This starts to dictate the safety and quality controls we put in place, such as reader-worker with signoffs for procedures, versus do the steps in any order you want. And we do put in layers of controls so we don't get to some of the worst case scenarios.

For systems I've dealt with, once we get past the safety issues, then we look at - will the effectiveness / capability of the product be compromised. Issue - the USA went to war in WWII with faulty submarine torpedoes - they did not go "boom" when they hit the enemy ship. Beyond effectiveness, then comes cost effectiveness, preventing budget overruns. Then we tailor the controls as needed.

The US DOE established the Integrated Safety Management System that worked through much of this. See DOE G 450.3-3 TAILORING FOR INTEGRATED SAFETY MANAGEMENT APPLICATIONS

quoting
The Department of Energy (DOE) Integrated Safety Management (ISM) system must support many different kinds of work, from the operation of nuclear and non-nuclear facilities to laboratory experimentation to environmental restoration activities. To accomplish the work safely, and to protect workers, the public, and the environment, the system must function to identify and control all types of hazards, from commonly encountered workplace hazards to rare or one-of-a kind process hazards,
Hi Steve,

Thank you very much for the response. I am in a similar situation and with the same line of thinking. But my logic failure is happening when I analyze the functional failure of a main level system, all lower level systems get the same level as they are connected in series to the main level function. This defeats my purpose of having a multi-level QA controls depending upon the importance of a system or sub-system. How do I break this logic? Any thought?
 

Steve Prevette

Deming Disciple
Leader
Super Moderator
#5
Hi Steve,

Thank you very much for the response. I am in a similar situation and with the same line of thinking. But my logic failure is happening when I analyze the functional failure of a main level system, all lower level systems get the same level as they are connected in series to the main level function. This defeats my purpose of having a multi-level QA controls depending upon the importance of a system or sub-system. How do I break this logic? Any thought?
There is the issue of "Systems Thinking" and looking at how the whole system operates versus dissecting the system. Dr. Russ Ackoff was a great thinker here - he said "You write. Your hand does not write, if you want to prove that chop off your hand and see if it will write". So sometimes, if we dissect the system to the subsystem level - we lose what IS the system and really need to look at the whole.

On the other hand (LOL) certain subsystems may play critical roles. In my example of the bad WWII torpedoes, the exploder pin would shear off if the torpedo hit the target ship at an angle. Before the war, they did little or no testing of the exploder mechanism, and if they did any, it was at a 90 degree angle.

So I would not try to break the logic per se - but analyze "what is the whole system supposed to accomplish?" Of these things/tasks, what subsystems are in play. The motor driving the torpedo was not in play when it came to "did the torpedo explode?". The motor and guidance got the torpedo to the target, but the exploder failed to explode upon contacting the target. Now this required redesign of the exploder, and modern torpedoes actually explode at some depth BELOW the target ship's keel in order to break the keel of the ship. So we really need to know how the components interact in the system in order to assess what risk level they present to safety and mission accomplishment, and mulitple sub systems may contribute. The point is - we really need to understand how the sub systems / components work together to make the system work, and what are the failure modes in the various sub systems and components that contribute to safety / mission failures. Understanding how the WHOLE system works (rather than eating the elephant one bite at a time) gets us to understanding the controls needed.
 

Quality27

Involved In Discussions
#6
There is the issue of "Systems Thinking" and looking at how the whole system operates versus dissecting the system. Dr. Russ Ackoff was a great thinker here - he said "You write. Your hand does not write, if you want to prove that chop off your hand and see if it will write". So sometimes, if we dissect the system to the subsystem level - we lose what IS the system and really need to look at the whole.

On the other hand (LOL) certain subsystems may play critical roles. In my example of the bad WWII torpedoes, the exploder pin would shear off if the torpedo hit the target ship at an angle. Before the war, they did little or no testing of the exploder mechanism, and if they did any, it was at a 90 degree angle.

So I would not try to break the logic per se - but analyze "what is the whole system supposed to accomplish?" Of these things/tasks, what subsystems are in play. The motor driving the torpedo was not in play when it came to "did the torpedo explode?". The motor and guidance got the torpedo to the target, but the exploder failed to explode upon contacting the target. Now this required redesign of the exploder, and modern torpedoes actually explode at some depth BELOW the target ship's keel in order to break the keel of the ship. So we really need to know how the components interact in the system in order to assess what risk level they present to safety and mission accomplishment, and mulitple sub systems may contribute. The point is - we really need to understand how the sub systems / componen
Hi Steve,

Thanks again for your patience and helping with this stalemate. I kind of reached the same conclusion as the torpedo pin example and the only stage where we might find it could be while doing a DFMEA with a proper structure and functional analysis. Other than that, I am thinking about the following logic:

Score each identified system in New, Unique, Different, Difficult and Failure Impact (lets say 1-3 score on each of these). If I do this, even though a main level system is higher level and at a sub-system level there is a compressor, it gets a lower score and I may not have to build redundancy or put too many controls on a product that is readily available in the market.

Do you believe this logic could be implemented? Do you see any logical gaps?
 

Steve Prevette

Deming Disciple
Leader
Super Moderator
#7
Do you believe this logic could be implemented? Do you see any logical gaps?

Seems reasonable to me. A tried and true design should be more reliable and known than something new and innovative. As long as knowledge is applied - for example we don't want to stifle innovation, just be smart about it. And there is looking at the interactions between components - for example there also was a problem with what depth the original WWII torpedoes ran at - the depth control didn't work - and that had some interaction with the torpedo hitting the ship rather than being set off by the magnetic detonator.

Something to consider is the testing program for verification of these risks and assumptions once initial prototypes or even sub assemblies are complete.
 

BradM

Leader
Admin
#8
Hello there!

You mentioned these classifications are to assess what approach is to be undertaken within the quality system. Correct?

So I would recommend identifying the 1-4 classifications system based on quality failure.

So say it's a... restaurant, with 1 being minor and 4 being major...

1. Will this failure lead to a potential service quality near miss? Will this failure lead to management intervention?
2. Will this failure lead to a service failure? Will this lead to rejected finish product and a customer dissatisfaction?
3. Will this failure lead to a broader systemic failure? Will multiple orders be rejected and lead to a potential drop in sales?
4. Will this failure lead to a large loss of raw goods? Will this failure potentially lead to customers receiving poor quality product?

I just kind of threw that together, so you can probably see holes fairly quickly. :)

You could also tie it to (since you mentioned design) the cost of quality phases:
1- Prevention costs
2-Inspection costs
3- Internal quality failure costs
4- external quality failure costs.
 

Quality27

Involved In Discussions
#9
Do you believe this logic could be implemented? Do you see any logical gaps?

Seems reasonable to me. A tried and true design should be more reliable and known than something new and innovative. As long as knowledge is applied - for example we don't want to stifle innovation, just be smart about it. And there is looking at the interactions between components - for example there also was a problem with what depth the original WWII torpedoes ran at - the depth control didn't work - and that had some interaction with the torpedo hitting the ship rather than being set off by the magnetic detonator.

Something to consider is the testing program for verification of these risks and assumptions once initial prototypes or even sub assemblies are complete.

Thanks Steve, great to share thoughts with you. I believe this can be implemented with extensive verification/validation testing for very high risk sub systems.

I will update you on how this thinking develops.
 

Quality27

Involved In Discussions
#10
Hello there!

You mentioned these classifications are to assess what approach is to be undertaken within the quality system. Correct?

So I would recommend identifying the 1-4 classifications system based on quality failure.

So say it's a... restaurant, with 1 being minor and 4 being major...

1. Will this failure lead to a potential service quality near miss? Will this failure lead to management intervention?
2. Will this failure lead to a service failure? Will this lead to rejected finish product and a customer dissatisfaction?
3. Will this failure lead to a broader systemic failure? Will multiple orders be rejected and lead to a potential drop in sales?
4. Will this failure lead to a large loss of raw goods? Will this failure potentially lead to customers receiving poor quality product?

I just kind of threw that together, so you can probably see holes fairly quickly. :)

You could also tie it to (since you mentioned design) the cost of quality phases:
1- Prevention costs
2-Inspection costs
3- Internal quality failure costs
4- external quality failure costs.
Thank you for your thoughts. I am trying to classify systems, not defects. I will definitely assign these ratings to the nonconforming scenarios.
 
Thread starter Similar threads Forum Replies Date
D Supplier Quality - How to classify a supplier level Medical Device and FDA Regulations and Standards News 10
Q Do you classify applied parts? IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
B How to classify a medical device based on MDR? EU Medical Device Regulations 4
I How to classify a medical device based on FDA? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
A Device Classification - India CDCSO - How to classify the device? Other Medical Device Related Standards 1
D Example for Accessories of "Annex XVI non-medical devices" and how to classify it now with respect ot EU MDR Other Medical Device Regulations World-Wide 1
O Which part of ISO 14644 describe how can I classify cleanrooms between them? Other ISO and International Standards and European Regulations 0
L How to classify this computer-alike IT device EU Medical Device Regulations 2
C How to Classify a Medical Device Other Medical Device and Orthopedic Related Topics 6
W Let's play... Classify My Medical Device!!! EU Medical Device Regulations 15
A How does Japan PMDA classify Medical Devices - Classification Guidelines Japan Medical Device Regulations 6
M Is it necessary to classify nonconformance as minor, major, observations in an IQA? Internal Auditing 18
E How do you classify Contact Lenses ? Non-invasive, Invasive or Surgically Invasive? Other US Medical Device Regulations 1
I How do you classify/name your CAR (Corrective Action) types? Nonconformance and Corrective Action 6
G How to classify maintenance level of equipments? Manufacturing and Related Processes 3
K Would you classify this as a concession? Document Control Systems, Procedures, Forms and Templates 5
A How many ways do you classify your scrap? Quality Tools, Improvement and Analysis 15
C Can I apply First Time Yield (FTY) for several products or processes of different nature? Manufacturing and Related Processes 5
A Several sections of traditional 510(K) have the same or similar requirement of content. US Food and Drug Administration (FDA) 4
M Informational Several US FDA draft guidances, including some specific device guidances for the Safety and Performance Based Pathway Medical Device and FDA Regulations and Standards News 0
MDD_QNA How to deal with FDA after not reporting anything for several years Other US Medical Device Regulations 14
M Informational Several revised or updates USFDA guidance - 9/9/2019 Medical Device and FDA Regulations and Standards News 12
L What to do if a Corrective Action requires several extensions Nonconformance and Corrective Action 7
qualprod Write down same nonconformity in several processes? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
M Informational TGA – Several proposed changes to classification to better align with the EU MDR Medical Device and FDA Regulations and Standards News 0
J Referencing Medical Device in FDA records - Package contains several other components Other Medical Device Regulations World-Wide 1
Q Several Nonconformances with same Root Cause with just one CA ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 15
MDD_QNA Several CE Marks on a Repaired Class IIa Medical Device EU Medical Device Regulations 9
M UDI (Unique Device Identification) for Convenience Kits with several options Other US Medical Device Regulations 3
F Can be several measurement in a repl considered as nested factor in Minitab analysis Using Minitab Software 14
M Need several examples of procedures Document Control Systems, Procedures, Forms and Templates 5
T ISO 9001 for a Holding Company which owns several Food Companies ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
R Auditing to several standards simultaneously Internal Auditing 3
Q One Procedure Containing Several Processes Document Control Systems, Procedures, Forms and Templates 10
D Distribution of a Kit composed of Piece Parts from several Manufacturers EU Medical Device Regulations 3
Q Several Processes in One Procedure? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
bio_subbu EU publishes several new & Revised MEDDEV guidance documents (FEB 2012) EU Medical Device Regulations 0
R Involvement of several Departments in Contract Review Process Contract Review Process 12
M I came across this site with several analyzing tools (Excel) Statistical Analysis Tools, Techniques and SPC 4
hogheavenfarm Need better Job Tracking Method - Fabricated in several departments at once Manufacturing and Related Processes 5
A Auditing Checklist for MDD 93/42/EEU that will cross reference several regulations EU Medical Device Regulations 9
C Equivalence Studies - Validation Activities for several Identical Pieces of Equipment Manufacturing and Related Processes 1
C Equivalence Studies - Validation Activities for several Identical Pieces of Equipment Qualification and Validation (including 21 CFR Part 11) 2
L How to Avoid Missed Welds - Job shop and weld several small quantity jobs Manufacturing and Related Processes 11
J0anne Link to find 502 (f)(1) which is quoted several times in 21 CFR 820 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 8
Q One procedure or several? For "Developing Engineering" ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
B Control of Documents in several languages and Document Linking ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
smryan Several questions about TS 16949 including associated ISO 9001 certificate IATF 16949 - Automotive Quality Systems Standard 15
V Corporate TS 16949 system or not? Several production locations world wide IATF 16949 - Automotive Quality Systems Standard 4
V PPAP for one material but for several customers APQP and PPAP 1

Similar threads

Top Bottom