Should volume of sales be factored into risk probability assessments?

Dr. IJ Arora

Involved In Discussions
#21
Wondering aloud: would it make sense to factor in an estimated volume of sales and/or number of users and/or number of procedures when considering harm probabilities?

We've never done this (and I'm curious if anyone does), but does seems reasonable in some way.
That is, the probability occurrence of a harmful event is proportional to the device's use.
All else being equal, a device that is used by millions presumably has a much higher probability of issues that a device that only has a few hundred users.

Curious what others think...
Mark I read through this interesting chain of replies. Very educating and useful. However, my thought is to go to the basic and see the effects of increase in volume of trade to risk appreciation. I am not for my argument considering the probability of occurrence. It is relevant, but my thought is to basics. Is the increased volume not driving the augmentation of capabilities withing the organization, creation of more elaborate infrastructure, requirements for more manpower and their competency? I guess the answer is a Yes. May be the augmentation could be delayed for sometime based on the management decisions on ROI, but it is a factor. So if you went to the the context of the organization (in terms of clause 4.1 of ISO 9001), the issues have changed. So your context has changed. Further Clause 4.2 for interested parties, last paragraph, would require you to review and re define the context and therefore the risks. And I have not even gone to the relevant industry specific standard or the P. In my opinion you should consider the risk again if the volume has gone up considerably. Clause 5.1.1 c (by a stretch, but applicable) requires integration of business and quality. The business has gone up so have then the risks based on resources and so on. 5..1.2 on customer focus, specifically 5.1.2 b would require you to look at risk again.
 
Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
#22
Forgive my obstinance, but as an admitted layman I'm still not understanding the misinterpretation of "probability". If you could kindly clarify which of these statement is wrong, or where the term "probability" is being misused:
  • Probability of heads showing up in a given flip = 1/2.
  • Probability of no heads in 10 flips = 1/2^10 = 1/1024.
  • Probability of at least 1 heads in 10 flips = 1 - 1/1024 = 1023/1024
  • The probability 1023/1024 is greater than the probability 1/2. In other words, the probability of heads showing up in 10 flips is greater than the probability of heads showing up in 1 flip.
Mark – I understand that this conditional probability can make it seem like the probability is changing but it’s really describing what happens when you diverge from the expected ‘average’ of 50%. The Law of large numbers: Of course, when you only flip a coin once the experienced rate of occurrence of a head is either 100% or 0%. It is important to understand that probability applies to many multiple trials or events. So even tho you got 1 head or no head in a single flip, if you were to flip the coin many times the rate of occurrence would be asymptotically approach 50%. So yes you can almost guarantee that you will get at least 1 head in 10 flips. That is exactly what is meant by a probability of 50%.



Now let’s add to this regression to the mean. You know how we talk about ‘lucky streaks’ and ‘unlucky streaks’. Conditional probability also applies here. Actual results do not play out uniformly about the probability – which is actually the average occurrence rate or proportion. (Jim alluded to this when he said that 1 in a million occurrence could happen on the first part or the 500,000th part or the 2 millionth part). So as you go on a streak of many tails, probability math will tell you the probability of keeping that streak up, before you start flipping more heads and begin to revert towards the average or ‘base’ probability. That is what your formulas are doing. It can appear that in the short run the probability is changing – but it isn’t.



But the bigger issue is the answer to (what I think is) your real original question. The standard you refer to is asking us to look at the effect of sales volume; to take into account what happens when the numbers get high and when production equipment and capacity might get strained (wear does change the probability of adverse events occurring). The standard is doing this because there are valid reasons to look at the number and not just the probability. For reasons that have been expressed above.
 

Ed Panek

QA RA Small Med Dev Company
Trusted Information Resource
#23
Hmm
Per ISO 14971, risk involves:
  • The probability of occurrence of harm;
  • The consequences of that harm
If you assume a defective product has only a single customer and the worst case result is death that has a high risk. What if the sales increase to 1000 customers? 1000 deaths seem much more harmful than a single death, right?

Because sales volume is out of your control I think we must treat the single death as equivalent to 1000 deaths in this case and require similar controls.
 

mmasiddiqui

Involved In Discussions
#24
Increase in volume will have an impact on the following
1) On time delivery to the customer suffers due to
1) Increase in re-work and scrap - Defect, depending on the industry might surely require contingency plan​
2) Increased pressure in supporting lines​
3) Bottle neck in your process​
4) inability of suppliers to support your production volumes​
Two other things to keep in mind is Probability of risk maturing and its impact to the business will help you decide on the need for a contingency plan.​
Say if your probability of producing scrap is between 80%-100%, Impact to business (if the safety of the user rated by 90%) is high, then the over all risk is Red. That means a contingency plan is required and necessary to implement Poka-yoke​
Say if your probability of producing scrap is between 50%-80%, Impact to business (if the safety of the user rated by 70%) is Medium-High, then the over all risk is Orange. That means a contingency plan is recommended and preventive controls are required.​
Say if your probability of producing scrap is between 30%-50%, Impact to business is Medium, then the over all risk is Yellow. That means a good detection is required​
Say if your probability of producing scrap is<30%, Impact to business low, then the over all risk is Green. That means a occasional inspection is required to control the process.​
Let me know, if I can be of anymore help.​

 

indubioush

Quite Involved in Discussions
#25
Speaking philosophically for a moment, if I were a public policy maker or regulator, would it not be reasonable to expect a higher bar for safety if a device was expected to be in millions of households versus only a few hundred? My intuition tells me yes, but then I'm struggling to map this on to the way we currently approach safety in medical devices.
If you were a policy maker, then yes, you would care more if the device was in millions of households. But medical device companies are not policy makers. When we think about risk management, we are not assessing the risk to the public whether or not they use the device, we are assessing the risk to individual patients/users. We think about it from the patient's perspective. Would a person use a toothbrush with a 1/1000th chance of death? NO! You seem to be wanting to see it from a general public point of view, which is incorrect. You have to consider the individual patient's needs and determine the risks for an individual patient. Then you need to consider whether the benefits outweigh the risks for that hypothetical individual patient, not the public.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#26
Two other things to keep in mind is Probability of risk maturing and its impact to the business will help you decide on the need​
Say if your probability of producing scrap is between 80%-100%, Impact to business (if the safety of the user rated by 90%) is high, then the over all risk is Red. That means a contingency plan is required and necessary to implement Poka-yoke​
Say if your probability of producing scrap is between 50%-80%, Impact to business (if the safety of the user rated by 70%) is Medium-High, then the over all risk is Orange. That means a contingency plan is recommended and preventive controls are required.​
Say if your probability of producing scrap is between 30%-50%, Impact to business is Medium, then the over all risk is Yellow. That means a good detection is required​
Say if your probability of producing scrap is<30%, Impact to business low, then the over all risk is Green. That means a occasional inspection is required to control the process.​

Here’s a question (and maybe this needs to be broken out into a different thread as it veers from the original question a bit):
What do we mean when we say “Probability” in the context of risk assessment and FMEA and Hazard Analysis? Is it the ‘liklihood’ Tha tsomethign bad will happen or is it the occurence rate should something bad happen? These are really two different things although we casually use probability in both ways.

For example, in the quote above it seems that mmasiddiqui isnt’ really referring to probability but to occurence rate. But what if they are really referring to probability? So there may be a 30% probability of experiencing some scrap without specifying how much scrap. Or are they saying that for a 30% scrap rate of all product made? Which is how Mark Meer seemed to use the term probabability in his earlier posts. (Frankly scrap rates above 30% of all product made would bring my organization to it’s knees.).

Which is it? Which Is more insightful/useful?
 

indubioush

Quite Involved in Discussions
#27
What do we mean when we say “Probability” in the context of risk assessment and FMEA and Hazard Analysis? Is it the ‘liklihood’ That something bad will happen or is it the occurence rate should something bad happen? These are really two different things although we casually use probability in both ways.
It is the "probability of occurrence." You start off with an educated estimate on probability during device design, and then as you get device feedback from clinical trials and complaints, you use the actual ocurrence rate to update and correct your previous estimate. Over time, it is not an estimate, but an actual "probability of occurrence of harm" based on objective evidence.
 

mmasiddiqui

Involved In Discussions
#28

Here’s a question (and maybe this needs to be broken out into a different thread as it veers from the original question a bit):
What do we mean when we say “Probability” in the context of risk assessment and FMEA and Hazard Analysis? Is it the ‘liklihood’ Tha tsomethign bad will happen or is it the occurence rate should something bad happen? These are really two different things although we casually use probability in both ways.

For example, in the quote above it seems that mmasiddiqui isnt’ really referring to probability but to occurence rate. But what if they are really referring to probability? So there may be a 30% probability of experiencing some scrap without specifying how much scrap. Or are they saying that for a 30% scrap rate of all product made? Which is how Mark Meer seemed to use the term probabability in his earlier posts. (Frankly scrap rates above 30% of all product made would bring my organization to it’s knees.).

Which is it? Which Is more insightful/useful?
I am not sure, if I have the patience to explain more than this :)

Risk.jpg

This is how I would choose my risk that needs contingency plan. I will identify the KPIs as shown above.
Then compare the risk of increasing volume against each of your KPIs and rate them.
Finally, calculate the total impact score by multiplying the score of all KPIs. The Table is the team decision on what these score should be on the total impact scale.
NO, I am not referring to occurrence rating.
 

indubioush

Quite Involved in Discussions
#29
Just to add clarification for all, it looks like you are talking about risk in the context of ISO 9001. However, the original poster was asking the question in the context of ISO 14971, risk management for medical devices.
 

Tagin

Trusted Information Resource
#30
Wondering aloud: would it make sense to factor in an estimated volume of sales and/or number of users and/or number of procedures when considering harm probabilities?

We've never done this (and I'm curious if anyone does), but does seems reasonable in some way.
That is, the probability occurrence of a harmful event is proportional to the device's use.
All else being equal, a device that is used by millions presumably has a much higher probability of issues that a device that only has a few hundred users.

Curious what others think...
See ISO 14971 Annex D.3.4.2 Semi-quantitative analysis:
There are several significant factors and statistics that are important for analysing the probability of occurrence. These statistics include, but are not limited to, the following.
How often is a particular medical device used?
What is the lifetime of the medical device?
⎯ Who makes up the user and patient populations?
What is the number of users/patients?
⎯ How long and under what circumstances is the user/patient exposed?
So, it seems entirely acceptable to include forecasted sales volume (i.e., as a proxy of user/usage counts), along with anticipated usage repetition and device lifetime, to inform your risk considerations. Note, as Dr. Arora said:

...So if you went to the the context of the organization (in terms of clause 4.1 of ISO 9001), the issues have changed. So your context has changed. Further Clause 4.2 for interested parties, last paragraph, would require you to review and re define the context and therefore the risks. And I have not even gone to the relevant industry specific standard or the P. In my opinion you should consider the risk again if the volume has gone up considerably. Clause 5.1.1 c (by a stretch, but applicable) requires integration of business and quality. The business has gone up so have then the risks based on resources and so on. 5..1.2 on customer focus, specifically 5.1.2 b would require you to look at risk again.
Ah, yes - 4.2! Interested parties should inform the manner in which your risk management is designed, and so it may well be that you choose to design your risk assessment using the approach described in 14971 D.3.4.2.
 
Thread starter Similar threads Forum Replies Date
D Should low volume supplier PPM comparisons consider level of opportunity for error? Quality Tools, Improvement and Analysis 2
W Should we monitor all Significant part Characteristics with SPC? Low Volume CNC Statistical Analysis Tools, Techniques and SPC 15
Sidney Vianna Interesting Discussion Should ISO 9004 be changed from a guidance standard to a requirements standard? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
A Should I take an online course for a career in Occupational Health and Safety? Career and Occupation Discussions 2
J Should a Class 1 medical device with an option to measure body weight be considered Class 1m? EU Medical Device Regulations 0
K Should APQP/PPAP has its own section in a QM? Quality Management System (QMS) Manuals 1
S What should i choose for "testing procedure" characteristics? (N95) General Information Resources 0
P Should eIFU link per ISO 15223-1:2016 be added to labels out of scope of Reg 207/2012? EU Medical Device Regulations 1
S Which Sampling Plan(s) Should I Use? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
A Document release vs its related training. Which should come first? ISO 13485:2016 - Medical Device Quality Management Systems 18
S Which department should prepare the control plan? could you show me a standard regarding to this matter. FMEA and Control Plans 17
J Help settle a disagreement: Should external providers of preventive maintenance be on your ASL? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
N Master Samples - What should we be keeping? IATF 16949 - Automotive Quality Systems Standard 9
G Supplier delivered recent PPAP, should he deliver yearly layout inspection? IATF 16949 - Automotive Quality Systems Standard 4
John Broomfield Vote - Should ISO9004 Become a Requirements Standard? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
A Capability Study - in the beginning of your career what should you have known about the tool Quality Tools, Improvement and Analysis 11
J Should Loading and Unloading be Included in Cycle Times? Lean in Manufacturing and Service Industries 14
E Manufacturers should develop a testing device for covid19 Service Industry Specific Topics 0
T 510(k) submission - Which name should I use in the submission? Other US Medical Device Regulations 3
N ISO 19011:2018 - 5.4.2 "...audit program should engage in appropriate continual development..." Training - Internal, External, Online and Distance Learning 4
G Should I perform Gage R&R only at the beginning of a new project? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 6
DuncanGibbons Should the requirements FAA/EASA Part 21 be addressed within the QMS and AS9100D quality manual? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
M Should 510(k) Predicates be Actively Listed Devices? Other US Medical Device Regulations 12
B Why the Greek god Hephaestus should have done a design FMEA (DFMEA) on his giant robot APQP and PPAP 1
J On PFMEA for danger labels - Label always should be assigned severity 10 ? FMEA and Control Plans 3
H Who should be listed as the manufacturer/distributor on the box? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 15
M MDR, RED and LVD - Should our device comply with them? EU Medical Device Regulations 3
BeaBea How Many Processes should be created for each Department? Process Maps, Process Mapping and Turtle Diagrams 5
MrTetris Should potential bugs be considered in software risk analysis? ISO 14971 - Medical Device Risk Management 5
S Should safety checks be included in the Control Plan? IATF 16949 - Automotive Quality Systems Standard 5
M Which incubation condition should be selected to recover both bacteria and fungus effectively Miscellaneous Environmental Standards and EMS Related Discussions 3
D Is there a specific location for PPE such as safety glass holders and glove dispensers should be mounted Occupational Health & Safety Management Standards 10
Robert Stanley Which Registrar Should I Choose for ISO 9001:2015 registration? Registrars and Notified Bodies 10
M Who should receive the bills from suppliers and vendors, account payable or procurement? Consultants and Consulting 4
V IATF 16949 8.4.1 Control of externally provided processes, products and services - Should the CB be on our Approved Supplier List? IATF 16949 - Automotive Quality Systems Standard 10
A We are ISO 13485:2016 should we be audited to ISO 14971 ISO 13485:2016 - Medical Device Quality Management Systems 16
E Received a Major finding during IATF Surveillance audit for loss of BIQS Level 3 (more than 6 SPPS in 6 months)...how should we address SYSTEMIC CA? IATF 16949 - Automotive Quality Systems Standard 11
J Organization merger. Should we keep two separate ISO 13485 certificates? ISO 13485:2016 - Medical Device Quality Management Systems 6
S Companies that maintain your machine should be in ASL? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
S Use of "Shall" versus "Should" in Procedures ISO 13485:2016 - Medical Device Quality Management Systems 26
D Class II medical device - When should a complaint be closed? Customer Complaints 6
Sidney Vianna IATF 16949 News Presentations from the latest IATF Stakeholder Event - Expectation that IATF 16949 certification should equate with product quality. Misguided? IATF 16949 - Automotive Quality Systems Standard 7
L Clause 0.4 of ISO 9001 and EHS - Where should I stop the inclusion of EHS in my QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
Ed Panek Part 11 Self Certify Memo - What else should it cover? Qualification and Validation (including 21 CFR Part 11) 5
H Should I mention machine/Equipment password In SOP? Qualification and Validation (including 21 CFR Part 11) 4
D How long should we keep the spare parts available for our medical device, after we have stopped the production? ISO 13485:2016 - Medical Device Quality Management Systems 0
H Statistical Techniques Procedure - What should be included Document Control Systems, Procedures, Forms and Templates 4
Q How should I analyze measurement correlation between me and customer? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
Sidney Vianna Interesting Discussion ISO 9001:2024 - What should be changed in the next Edition of ISO 9001? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 113
N Should it even be on the hazard analysis (software)? FMEA and Control Plans 2

Similar threads

Top Bottom