Folks,

I've come across something that may be old news to many of you - but I've just noticed it and "it" is creating somewhat of a "discussion" in my organization...

21CFR, Part 820, para 820.30(g) defines Design Validation as follows: "Design Validation....Design validation shall ensure that devices conform to defined used needs and intended uses..."

The FDA Medical Device Quality Systems Manual:
A Small Entity Compliance Guide
First Edition, HHS Publication FDA 97-4179, December 1996, Chapter 3 defines Design Validation as follows: "Design validation means establishing by objective evidence that device specifications conform with user needs and intended use(s)."

Conspicuously absent from this 2nd definition is the word "defined". If I may go on just a bit...

Using the 1st definition, an organization is responsible for defining and interpreting "user needs". Following this, we develop products which we hope will meet these user needs - again as defined and interpreted by the organization. Once developed, the design validation consists of assuring that the product meets the user needs as defined by the organization. IMHO, this approach does not assure that user needs were truly met. It only confirms that we designed and built what we think the customer wanted. Consequently the argument being posed here is that this definition (of design validation) does not require that the product be taken "to the field" in either a real-world, or simulated environment to confirm that user needs were truly met. It only requires that we confirm we designed and built what we thought the customer needed.

The 2nd definition, lacking the term "defined" would seem to obligate the organization to truly confirm that user needs were met - not just what the organization (Marketing, Engineering, etc) thought the customer required. Consequently, this definition of design validation would seem to obligate the manufacturer to perform a real world, or simulated environment confirmation.

Again, a subtle but potentially significant difference & one which I am posing to the "best & brightest" here at the Cove! Thank you for your patience!

Folks,

I've come across something that may be old news to many of you - but I've just noticed it and "it" is creating somewhat of a "discussion" in my organization...

21CFR, Part 820, para 820.30(g) defines Design Validation as follows: "Design Validation....Design validation shall ensure that devices conform to defined used needs and intended uses..."

The FDA Medical Device Quality Systems Manual:
A Small Entity Compliance Guide
First Edition, HHS Publication FDA 97-4179, December 1996, Chapter 3 defines Design Validation as follows: "Design validation means establishing by objective evidence that device specifications conform with user needs and intended use(s)."

Conspicuously absent from this 2nd definition is the word "defined". If I may go on just a bit...

Using the 1st definition, an organization is responsible for defining and interpreting "user needs". Following this, we develop products which we hope will meet these user needs - again as defined and interpreted by the organization. Once developed, the design validation consists of assuring that the product meets the user needs as defined by the organization. IMHO, this approach does not assure that user needs were truly met. It only confirms that we designed and built what we think the customer wanted. Consequently the argument being posed here is that this definition (of design validation) does not require that the product be taken "to the field" in either a real-world, or simulated environment to confirm that user needs were truly met. It only requires that we confirm we designed and built what we thought the customer needed.

The 2nd definition, lacking the term "defined" would seem to obligate the organization to truly confirm that user needs were met - not just what the organization (Marketing, Engineering, etc) thought the customer required. Consequently, this definition of design validation would seem to obligate the manufacturer to perform a real world, or simulated environment confirmation.

Again, a subtle but potentially significant difference & one which I am posing to the "best & brightest" here at the Cove! Thank you for your patience!

I'm not an expert in this area, but using definition #2, how could "..device specifications conform with user needs..." if user needs haven't been defined? In other words, isn't the "defined" requirement of definition #1 at least implicit in definition #2?

I'm not an expert in this area, but using definition #2, how could "..device specifications conform with user needs..." if user needs haven't been defined? In other words, isn't the "defined" requirement of definition #1 at least implicit in definition #2?

Jim,To your point, they could not & yes, to that degree "defined" would be implicit in Definition #2. That aside, the real question remains:

Is the "intended purpose" of Design Validation (in the eyes of FDA) to:

a) Confirm that we made what Marketing thought the customer wanted

- or -

b) Confirm that we made what the customer needed?

Folks,

I've come across something that may be old news to many of you - but I've just noticed it and "it" is creating somewhat of a "discussion" in my organization...

21CFR, Part 820, para 820.30(g) defines Design Validation as follows: "Design Validation....Design validation shall ensure that devices conform to defined used needs and intended uses..."

The FDA Medical Device Quality Systems Manual:
A Small Entity Compliance Guide
First Edition, HHS Publication FDA 97-4179, December 1996, Chapter 3 defines Design Validation as follows: "Design validation means establishing by objective evidence that device specifications conform with user needs and intended use(s)."

Conspicuously absent from this 2nd definition is the word "defined". If I may go on just a bit...

Using the 1st definition, an organization is responsible for defining and interpreting "user needs". Following this, we develop products which we hope will meet these user needs - again as defined and interpreted by the organization. Once developed, the design validation consists of assuring that the product meets the user needs as defined by the organization. IMHO, this approach does not assure that user needs were truly met. It only confirms that we designed and built what we think the customer wanted. Consequently the argument being posed here is that this definition (of design validation) does not require that the product be taken "to the field" in either a real-world, or simulated environment to confirm that user needs were truly met. It only requires that we confirm we designed and built what we thought the customer needed.

The 2nd definition, lacking the term "defined" would seem to obligate the organization to truly confirm that user needs were met - not just what the organization (Marketing, Engineering, etc) thought the customer required. Consequently, this definition of design validation would seem to obligate the manufacturer to perform a real world, or simulated environment confirmation.

Again, a subtle but potentially significant difference & one which I am posing to the "best & brightest" here at the Cove! Thank you for your patience!

Suggestion: Obtain a copy of "The Quality system Compendium", this is available from the Association for the Advancement of Medical Instrumentation (AAMI). The Compendium lists each of the paragraphs form 21 CFR Part 820. This list has two discussions, one from the FDA view, and one from the industry view. I think that this book will answer your questions.

Phil

Jim,To your point, they could not & yes, to that degree "defined" would be implicit in Definition #2. That aside, the real question remains:

Is the "intended purpose" of Design Validation (in the eyes of FDA) to:

a) Confirm that we made what Marketing thought the customer wanted

- or -

b) Confirm that we made what the customer needed?My suggestion is not to overthink this. If we give a medical patient (or his caregivers) what he needs, it would be an "instant cure" for whatever ails the patient, from a scraped knee to cancer. Universal instant cures are not yet realistic. Therefore, the producer of the product or drug or service says, "I am aiming the use of this (product or service) to answer this specific need of a patient ________ ."

If you think about it, almost every item or product has a defined need or niche proposed by the manufacturer. In the case of drugs, for example, some doctors go BEYOND that defined "need" and prescribe drugs "off label" for some conditions. Until recently, one of the most often prescribed "off label" uses for quinine tablets was for treatment of nighttime leg cramps, although the label strictly defines them for use as an antimalarial symptom drug. In the past few years, FDA has cracked down on the off label use of quinine and is considering several other drugs for similar policy enforcement. There is a brief treatise on a law firm page here http://www.oshmanlaw.com/pharmaceutical_litigation/quinine.asp
but you can google for other sources on the situation.

A potential solution:
One of the most used phrases of trite aphorisms in my arsenal is
"When in doubt, ask the source or authority with whom you will have to deal." (Often used in the context of Contract Review.)

I might even start with FDA's ombudsman to get directed to the proper source to answer my query:
FDA Office of the Ombudsman
5600 Fishers Lane, Rm. 13B-07
Rockville, MD 20857
Phone: 301-827-3390
Fax: 301-480-8039
Email: Ombuds@oc.fda.gov

Wes, I take it you are in the drug industry? (Just curious).

That aside, your wisdom of referring to the source we ultimately need to deal with speaks volumes. In this case (and I would suspect most cases) it is the customer - who happens also to be the user.

Erring to the conservative, I cannot imagine any circumstance where FDA, during a facility audit or 510(k) submission review, would NOT expect that design validation circle back to user needs. Clealry, the intended purpose of Design Validation is to determine whether an organization has successfully assessed, interpreted and implemented user needs.

Chapter 3 (Design Controls) of the FDA CDRH's Quality System Manual,
which can be found at www.fda.gov/cdrh/qsr/03design.html, may
provide you with some answers.

Wes, I take it you are in the drug industry? (Just curious).

That aside, your wisdom of referring to the source we ultimately need to deal with speaks volumes. In this case (and I would suspect most cases) it is the customer - who happens also to be the user.

Erring to the conservative, I cannot imagine any circumstance where FDA, during a facility audit or 510(k) submission review, would NOT expect that design validation circle back to user needs. Clealry, the intended purpose of Design Validation is to determine whether an organization has successfully assessed, interpreted and implemented user needs.Wes, I take it you are in the drug industry? (Just curious).

Not really. One of the ASQ Sections I belong to has dozens and dozens of members from Abbott and Baxter - two big houses under FDA regulation. Abbott paid a humongous penalty recently (a few years ago) for not paying attention to EXACTLY what FDA required and paid millions in lost business and consultant fees to bring a manufacturing line shuttered by FDA up to a level to get it reopened.

Some organizations have fought FDA rulings and won, but even then, they didn't win money settlements against FDA for the cost of the battle.

Much better in my opinion to defuse a potential adverse ruling after the fact by getting direct guidance before the act. When I was in the investment banking business, we used to routinely get advance rulings (not just guidance) from the IRS for schemes that were close to the edge for saving taxes on mergers and acquisitions and on issuance of tax-exempt revenue bonds. The IRS has pretty much stopped the practice of issuing advance rulings and the result has been some reckless lawyers and accounting firms have led individuals and organizations into big trouble with erroneous advice (witness actor Wesley Snipes and his upcoming sentence of 3 years!)

This discussion has been on going at my company for a while, management is of the opinion that testing of the device to performance standards is design validation. When I contacted the FDA for clarification I got the standard reply " it is up to the manufacturer to determine how they meet the design validation requirement" It would seem to me ideally simulated or actual use by end the end user would be nice, but real world this could have significant cost impacts.

Watchwait
You say that you company HOPES to meet the user needs, and that you THINK that you built what the customer wanted.
How does your company base its design criteria? Does it use customer feedback on existing product, do you use BETA products for the customer to use?
How do you know that you are not developing a product that is used opposite of what other similar products on the market are?
How does your company INTERPRET the customer needs?


Phil

reminded of an old adage reading this:

Verification - Did we make the thing right?

Validation - Did we make the right thing?

Ultimately I keep thinking about design inputs, we need to remember that these are not solely customer requirements.

Would an inspector not be concerned with Design Inputs as just an entity, you as a company may well perform QFD, Kano and a myriad of other tools to determine customer needs but at the end of the day, you have a set of design inputs that need to be need to be converted to a realised and manufactured device following 21CFR820.

Whether or not you invested time to determine the market requirements is a business decision, as presumably you would want to sell the device.

The FDA will be concerned that the design effort shows that the inputs and output match, and of course that Design Controls are in place.

If the device meets FDA requirements it will get clearance, if however it does not meet a market need, clearance or not it will not sell.

:2cents:

Watchwait
You say that you company HOPES to meet the user needs, and that you THINK that you built what the customer wanted.
How does your company base its design criteria? Does it use customer feedback on existing product, do you use BETA products for the customer to use?
How do you know that you are not developing a product that is used opposite of what other similar products on the market are?
How does your company INTERPRET the customer needs?


Phil

Great questions! Yes, we use customer feedback on existing designs and from controlled "beta" releases. We also take into consideration customers' comments on other, competitive products. The "interpretation" of customer needs is essentially the Marketing group speaking to the Engineering group about what they feel are true customer needs. As this IS one essential function of Marketing, this seems altogether appropriate. Also, as in any organization , this interpretation process is as subjective as it is objective.

Consequently, the need to confirm that Marketing has indeed correctly interpretated user (customer) requirements by nature necessitates re-visiting the customer and asking the question: "Here is what I think you want. Is this correct?" The process of asking this question & responding to the answer is, in essence, Design Validation in its purest form. It seems the only way to truely answer the pertinent question: Did we build the right product.

This discussion has been on going at my company for a while, management is of the opinion that testing of the device to performance standards is design validation. When I contacted the FDA for clarification I got the standard reply " it is up to the manufacturer to determine how they meet the design validation requirement" It would seem to me ideally simulated or actual use by end the end user would be nice, but real world this could have significant cost impacts.

I feel your pain. What you described is, by definition Design Verification. In terms of the FDA response you received, it is altogether correct as well. Too, the clarity of the response speaks volumes about the real issue: you have to perform Design Validation. It is NOT an optional excercise and Design Verification is not, by itself adequate to meet FDA requirements. Many a company has found this to be the case. In my experience, I would much prefer to have a "discussion" with the agency regarding the adequacy of my validation efforts as opposed to the absence of those efforts altogether.

It may also be helpful to recall that Design Validation can be performed in real-world, or "simulated" environments. I have seen many firms who consistently use in-house staff to perform design validation. You just need to select the right folks to do it. For example, if you are manufacturing a headset you don't want the design engineers performing the design validation! Give it to a bunch of Administrative Assistants, let them try it out & document their comments. This approach is more than adequate and will typically be the least burdensome approach.

Conversely, if your customer/user group is a "technical professional", e.g. a physician needing to assess the design adequacy of a new piece of diagnostic software, you likely need to do that with a physician as opposed to someone in-house who *thinks* they know how a physician would react. These are the types of judgement calls your company must make regarding Design Validation & they must be supported with a written rationale that can be defended. Personally, I always imagine myself stitting across from an FDA inspector having these types of discussions. That imagery always lends clarity to my thinking!

Watchwait,
21CFR, Part 820, para 820.30(g) defines Design Validation as follows: "Design Validation....Design validation shall ensure that devices conform to defined used needs and intended uses..."
para 820.30(c) says: "Each manufacturer shall establish and maintain procedures to ensure that the design requirements relating to a device are appropriate and address the intended use of the device, including the needs of the user and patient."

Design input defines the users needs. Part of validation is ensuring that the design inputs fulfill the users needs. This is often done by having knowledgable folks review and approve the design input and requirements documents. Then the validation testing is confirming that those design inputs have been satisfied.

As per the internal aurgument, just remember that 21CFR is the regulation and must be followed. If guidance is different than the regulation, the regulation wins.

Following this logic, design validation would then consist of confirming that design output met "defined user needs". In order to be "defined" they need to first be assessed, interpreted and documented - typically the job of Marketing. So...using the 21 CFR requirement, this process would only assess what was produced versus what marketing "defined" as the customer need.

Conversely, the guidance doc excludes the word "defined" and specifically states "user needs". The only way to assess this is to go back to the real (or simulated) user and compare what they (the user) wanted vs. what Marketing (et.al) said (e.g. defined) they needed;

There is a HUGE difference in these two approaches: Only the later accomplishes the true FDA objective of validation. IMHO, too many organizations try to "argue" that the first approach is adequate - typically out of cost cost/resource concerns. Big mistake.

Watchwait,
I will agree that user input and user testing is critical to a quality product, but that input/testing does not have to be part of my Validation that is done on production or simulated production product.
The user should be involved throughout the development process "validating" that what is being built is the correct product. But in some cases, the user will have to be represented by someone in-house, such as marketing (user may not want to be involved in development processes:(, user community is very large and diverse, etc.)

This quote is taken from "The Quality system Compendium, GMP requirements & Industry Practices", which is a discussion point from the FDA (820.20).

“The validation test also should address use by intended users, as the actual use of the device does not directly coincide with that envisioned by the designers. Users may attempt to use the device in ways that were not anticipated during the development process. Stress testing by those unfamiliar with the design is a valuable technique of identifying potential error conditions.”

Phil

I believe the key here is to have the wisdom and knowledge to determine when a "real world" vs. a "simulated user" validation is appropriate. In essence, the true purpose of validation is to determine if the company has correctly assessed, interpreted and implemented user needs. It then follows that asking "the Company" if they correctly interpreted user needs would be like asking Marketing if they did their job correctly. In terms of a user not wanting to be involved in product development: true enough. However I know of no user who would who would take offense at being asked: "Is this the product you wanted? I think it is, but I need you to confirm this".