How to create the Policy for determining criteria for Risk Acceptability

Marcelo

Inactive Registered Visitor
A lot of people ask me this, and, although it's too dependent on each manufacturer, I think some guidelines might help (this is really not easy, so one of the questions being answered in the new guidance to ISO 14971 (draft ISO 24971) is about how to create this policy)

Anyway, what the standard wants is that you as the manufacturer takes into consideration all the relevant information, state of the art, known risk control measures and user needs and create a policy based on that.

Please note that this policy is the MAIN driver for all safety related issues in the risk management framework of ISO 14971. It's what you help you defined if your device is safe or not. This is in fact a very important BUSINESS decision, because it could impact a lot of things, from the regulatory aspect to the perception of your product bythe stakeholders aspect.


As a generic (exxagerated) example I cite in courses I give, I say that, when X-rays where discovered, it was ok that a medical x-ray equipment killed 50 % of it's users, because there were no clear risk controls and it would stil be better than death.

On the other hand, now that we have developed better risk controls (see standards and regulations) and there's also some other treatment options, the user perception (user including all stakeholders) has changed and do not accept this kind of mortality anymore.

So your policy has to reflect all this.

Please note that possible occurences per use or anything like that are only one aspect of the policy, there's a lot more.

Also remember that the policy is for you to determine the risk acceptability criteria for each device you have. Some people confuses this and thinks that it's THE risk acceptability criteria. It's not, it's only a policy, and as a policy, it's objective is to guide you when defining the risk acceptability criteria of each of your devices.

A generic example of part of a policy related to the rationale I cited above is:

- Our devices cannot have occurrencies leading to death (or serious adverse event, or something like that) in more than 0,01 % of it's uses (this requires that, for each device, when you is defining the risk acceptability criteria, you take into consideration that anything beyong this number is unnaceptable)

- Our devices will follow the risk control measures detailed on all applicable standards and, when risk control options are not defined in standards, we will implement risk control options which reflect the current practice and current perceptions of all the involved stakeholders, including any which comes from regulatory expectations ( this in practice means that, when dfining the risk acceptability criteria for each device, you will need to consult stakeholders to define what they expect as state-of-the-art risk control measures)

- Our devices will always have a safety level comparable, and if possible better to, other diagnostic/treament solutions in the market (this means that your risk benefit/analysis always has to be based on the other treatment safety levels, not the ones you internally defines)

And it goes on.

Maybe I will try and creat some model example for this in the future, but as you might have noticed, even a simple example is not that simple.
 
Last edited:

Weeder

Involved In Discussions
After much investigation, talking to some consultants and a few colleagues, I have these comments to make regarding the risk management policy.

The standard does not do a very good job of defining what they are looking in term of policy nor do they give any examples. There is not much material on the web that may be of help

It also seems like it is a purely academic activity with no real world usefulness. When a company has already estblished its criteria for risk acceptability, why do they have to phrase the same thing as a policy.

It is easy to say, use state-of-the art or best practices but in reality such practices are not very well known from company to company and its leaves smaller companies searching for answers that cannot be found.

Most people I talked to have a very hard time understanding the requirement or the purpose of it.

Comments
 

Wes Bucey

Prophet of Profit
From my experience, and from a "liberal" interpretation of Marcelo's post, the concept of assigning a "target" for Risk Acceptability is a constantly moving target, subject to continuing evaluation based on experience and information gleaned from research. Nothing is static in the Medical Device world, otherwise, every shoe store would still have an x-ray machine for fitting shoes!

My advice: create a system to continually evaluate the parameters for Risk Acceptability and be prepared to modify them (with proper notice to affected parties) when facts seem to warrant.

At each stage of evaluation (from day 1 to day n), the number [ratio?] will be the best estimate given the facts available.
 

Attachments

  • shoe x-ray.jpg
    shoe x-ray.jpg
    16.1 KB · Views: 288

Ronen E

Problem Solver
Moderator
The standard does not do a very good job of defining what they are looking in term of policy nor do they give any examples.

Standards, in my opinion, are neither, nor meant to be, a substitute for an intellectual effort. The standard requires that there's a policy in place; the purpose is that decisions are not made "out of the blue", and that there'd be some consistency over time and projects. The standard, in my opinion, should not say how the policy should be established, nor what its shape should be in general terms. That is up to you, mate! An organisation that has lost its ability to think its way through, or draft policies that set the ship's course, is in a major problem.

It also seems like it is a purely academic activity with no real world usefulness. When a company has already estblished its criteria for risk acceptability, why do they have to phrase the same thing as a policy.

Please don't put the cart in front of the horses. Policy comes first, and acceptability criteria should be derived from it, for each and every risk management session. If it's done the other way around, it is in fact quite useless and pointless.

It is easy to say, use state-of-the art or best practices but in reality such practices are not very well known from company to company and its leaves smaller companies searching for answers that cannot be found.

The answer is research, research and some more research. State of the art, relating to acceptability of risks, basically means - what sort of risks are current stakeholders willing to take, to rip the benefits the specific technology is currently offering?; much like the example Marcelo provided on X-ray. It's not that hard to get a good idea about that - you'd see risks deemed acceptable by manufacturers as warnings, cautions, contra-indications, use limitations etc. on IFUs and other labelling. Relevant information can also come from publicly available sources such as the 510(k) summary database etc. It may not be quick and easy but it's doable in most cases.

Most people I talked to have a very hard time understanding the requirement or the purpose of it.

Yeah, many people don't understand what policies are all about, and it's often our QA/regulatory specialists job to explain that. As I stated above, to me the main purpose is consistency, transparency and in general - a rational, justifiable process.

Cheers,
Ronen.
 

c.mitch

Quite Involved in Discussions
1. I think that every company has a risk policy, made of its people and corporate culture. Writing a policy is a good way to unveil this and make it formal.

2. This kind policy may be difficult to implement. with your example: how do I measure the safety level of my competitors, versus mine?

3. Should this policy include practical items (they may be put in procedures), or is it more conceptual.
Examples of practical items:
-risk mitigation should be seeked in design changes, not warnings in the IFU
-risks linked to sterilization shall be reviewed by external consultants.
-management reviews the policy every year

Regards.
 

Marcelo

Inactive Registered Visitor
2. This kind policy may be difficult to implement. with your example: how do I measure the safety level of my competitors, versus mine?

3. Should this policy include practical items (they may be put in procedures), or is it more conceptual.
Examples of practical items:
-risk mitigation should be seeked in design changes, not warnings in the IFU
-risks linked to sterilization shall be reviewed by external consultants.
-management reviews the policy every year

The idea of a policy is to give you directions when taking decisions. That's why policies are usually high-level directions, which need to be suplemmented by more defined actions itens or objectives.

In 2 - yes, you would have to better define how to measure. In fact the you need to defined what is a safety level for you. And yes, it might not be that simple. But it's a high level policy as written and forces you to think about it and act on it.

In 3 - I would say that your examples are more defined examples coming from a high-level policy, those could be defined as objectives of your product life-cycle process or the like.
 
Last edited:
M

MIREGMGR

Discussions of high level policy are a good place to consider high level considerations that have become more apparent in recent years.

This is in fact a very important BUSINESS decision, because it could impact a lot of things, from the regulatory aspect to the perception of your product by the stakeholders aspect.

As key-market nations and regions focus increasingly on overall social costs of healthcare to their economies, such individual-company business decisions also have national/regional-social-policy consequences.

Individual device costs are driven by design and thus risk; individual device costs add up to overall social healthcare costs; and overall social healthcare costs determine how much healthcare that society will deliver. Because of this, it's a reasonable guesstimation that at some point in the future, nations and regions may want to have input on such decisions so as to exert some control over those consequences.

As an example: a maker of generally low risk devices that are used by hospitals in significant volume has two feasible ways to make a particular device. One way has somewhat better clinical performance, reflected in its risk analysis, but not so much so that a specific number of lives saved per year can be quantified compared to the other design-approach. The other way costs half as much, allowing a much lower selling price, but requires design-acceptance and non-mitigation of greater risk. The particular device type isn't individually cost-reimbursible to users; instead it's included in their lump-sum reimbursement for each procedure in which it's usually used.

Most hospitals will prefer the lower cost version, even though it has greater risk/worse clinical performance, because they're highly focused on cost.

Historically, medical device risk analyses in particular and device manufacturing/marketing directives/rules in general have been focused mostly on the safety and effectiveness of devices as used on individuals, and have considered secondarily if at all the impact of individual device cost on how much total healthcare can be delivered to overall society for a given total cost.

Such a total cost analysis approach long ago became a standard part of budgetary management in national defense, where it became obvious during the 1950s that if the designers were unconstrained and the end users were allowed to choose based only on performance, eventually the capabilities of an individual tank, ship or airplane would reach such an advanced state that the associated cost of a single unit would eat up the entire budget. That of course would be irrational, because in practical warfighting, quantity can be as important as individual performance.

The same analytical perspective is becoming more obviously applicable in healthcare. For an individual device maker, high level risk policy is one place where increasing social expectations for such risk-vs-cost balancing need to be incorporated.
 

Peter Selvey

Leader
Super Moderator
Recently I have been playing with the concept of a risk minimum, which captures some of the issues MIREGMGR raised.

The theory goes that the use of resources (in both the analysis phase and for risk controls) increases device cost, delays placement on the market, takes resources and incentive away from new development (new ideas, improvements of existing devices). This reduces the device availability, which means a reduction in clinical benefit. Thus, increased resources means increased risk.

Applying resources can also reduce risk. But there will be diminishing returns. So, at some point there will be a risk minimum, a point beyond which application of more resources only increases the risk. It is this risk minimum that should be our general target.

The interesting point, relevant to this thread, is that the risk minimum will be different for every hazardous situation. So developing broad criteria for acceptable risk for a particular medical device, or policy for developing this criteria, makes no sense.

Other interesting conclusions include:
- the ability to limit the scope of risk management to selected hazardous situations only (with a greater depth of analysis for those situations)
- methods to objectively verify the suitability of most risk controls without relying on estimates of risk
- replacing the risk / benefit argument (which is open to abuse) with a risk minimum argument (but still limited to special situations only)
- the risk minimum is not static, and will change as technology, market experience grow

Since we are stuck with ISO 14971 for now, maybe a good "policy" would be one that requires reasonable resources be applied to minimize risk, taking into account not only state of the art, published standards, national and regional regulations, but also the potential negative effects from the use of excessive resources.
 

Weeder

Involved In Discussions
I think now this discussion is making a lot more sense. I like Peter's approach. MIREGMGR has pointed out some of the issues that should be considered with very good examples. Thanks everyone.
 
Top Bottom