V&V?

Bev D

Heretical Statistician
Leader
Super Moderator
#1
In a recent thread a regular here stated that they didn’t like the ambiguous phrase “V&V”. (Verification and Validation). I have struggled over the last 40 (!?) years with these terms myself. Having been in many industries with various regulatory agencies and quality standards I have found no universal meaning or application. Design Verify seems to be particularly confusing to me especially when using the terms design inputs and design outputs.

I’m interested in exploring what “V&V” means to Cove participants. If possible can you give specific examples and define your terms? Remember that we have many different industries represented here. For example I worked in the veterinary diagnostic industry and the USDA used the term Verification for what I would have called validation…
 
Elsmar Forum Sponsor

Miner

Forum Moderator
Leader
Admin
#2
The definitions that I use for design are:
  • Verification - focus is on whether the design meets drawing specifications
  • Validation - focus is on the application (intended use), i.e., does it work as intended
Confusion typically enters into the equation when you get specifications that simulate the application, so it is possible to have specifications that are both verification AND validation.

I use the following example to illustrate:

General Motors Quad 4 engine
Verification:
  • Tests of the engine alone (Brake HP, Torque curves, Dimensional, etc.)
Validation:
  • Tests of the engine in the Sunfire and Grand AM vehicles (acceleration, mileage, etc.)
  • The Sunfire is a lighter vehicle with fast, sporty acceleration
  • The Grand Am is heavier with mediocre acceleration
 

Mike S.

Happy to be Alive
Trusted Information Resource
#3
I was taught, VERIFY it meets spec, validate that it will do what you need it to do.

If building a bridge, veify the span, height, thickness of concrete, etc. Validate it works by seeing if it will hold a truck carrying max load driving over it.
 

Ed Panek

QA RA Small Med Dev Company
Leader
Super Moderator
#4
We have this issue in our QMS as well. For our ERP it was the verification of entering data in specific fields. Validation was the output matched against a handmade output of the same data (what we called the gold standard).

For me verification is very close to the subject whereas validation is more holistic. You can verify your IFU has all the right symbols but you validate that humans can read and understand it.
 

Tidge

Trusted Information Resource
#5
For me, I go by the (old) GHTF definitions:

Verification is confirmation by examination and provision of objective evidence that the specified requirements have been fulfilled.

_____ Validation is establishing by objective evidence that ______ consistently produces a result or product meeting its predetermined requirements.

EDIT: Much of my professional career has been in the field of medical device manufacturing.
 
Last edited:

Bev D

Heretical Statistician
Leader
Super Moderator
#6
For me, I go by the (old) GHTF definitions:

Verification is confirmation by examination and provision of objective evidence that the specified requirements have been fulfilled.

_____ Validation is establishing by objective evidence that ______ consistently produces a result or product meeting its predetermined requirements.

EDIT: Much of my professional career has been in the field of medical device manufacturing.
I am familiar with the GHTF statements but waht does it mean? how are verification and validation different in this case? Can you give an example?
 

Tidge

Trusted Information Resource
#7
Oh, thanks for the reminder, I wanted to point out some examples of when I find the joined term "V&V" to be misleading, but let me answer this first:
I am familiar with the GHTF statements but waht does it mean? how are verification and validation different in this case? Can you give an example?
For me, the most important bit of both is the "objective evidence". I don't feel the need to say much about this, but I do want to make it clear that I consider OE to be some sort of record that stands by itself.

Verification is having OE that a <thing> is satisfied. Validation is having a degree-of-belief that the <thing> is satisfied, without having to generate specific OE that every instance of the <thing> is satisfied. In the case of Validation, the OE is about the degree-of-belief, not the <thing>.

I was introduced to the concept of V&V through manufacturing. It is possible to verify that every part made meets specification by checking all characteristics of every manufactured part. However, it is more efficient to validate a manufacturing process so that it becomes unnecessary (at some established level of confidence) to inspect all the characteristics that would otherwise have to be verified. In the other (recent) thread, this is why @Ronen E bluntly stated something to the effect of "processes are validated so that it becomes unnecessary to verify outputs." There is a common (appropriate, true) chestnut in medical device manufacturing: If the important characteristic CANNOT be verified, then the process of making the characteristic MUST BE validated. The ur-example in medical devices is sterilization, as it is not possible to verify that every device is sterilized.

A follow-up post will have my thoughts on times when the term "V&V" is either abused or misused. (my opinion, of course)
 

Tidge

Trusted Information Resource
#8
Some examples (from the world of medical device design, development, manufacturing) of where I see "V&V" being used a little too casually for my taste... for some values of "casual" that could lead to "misuse".

"Design V&V". Strictly speaking, I think the best (most defensible) explanation of "V&V" is something like this: Verification focuses on the the specific design outputs (created by the design team) to satisfy specific inputs. For example: An input is something like "device shall operate for at least 1 hour on battery". There are likely to be many output specifications that derive from such an input; each output is verified to satisfy the input requirement. The battery life requirement derived from some (even) higher level user need, maybe "patient needs to get from one building to another with device applied".... the validation would be more like having an appropriate sample of patients doing pretty much what the high-level user need is.

The potential misuse of the terms V&V is when design teams confuse lab-bench verification activities and patient/user-based activities. The tell-tale sign that this is confused is when an engineer (could be a "test engineer" or a "design engineer") says something like "this is how a user uses the device." The engineer may have an idea, but they are NOT the end user. The engineer's testimony is not OE how the devices get used. Human Factors Engineering (HFE) / Usability is all about Validation, not Verification.

"(Product) Software V&V" This is an odd duck. Strictly speaking: software designs (ought to be) entirely prescriptive... that is: the software should behave the same every time given the same initial conditions. (let's ignore philosophical discussions on this point... can we agree that software solutions are certainly supposed to be more prescriptive than physical representations of batteries, belts, gears?) So in terms of software, it is generally recognized that software (by itself) is essentially only verified. The system software belongs to can of course be subject to (user) validation. I prefer to be very clinical and only refer to (medical device) software verification (in medical devices, the unit/integration/system level testing per 62304) but once the term "system level testing" is used, some people start to want to think of this as validation... again, the testing required per 62304 is by the engineers, so I stand firm that this is NOT validation. Others are much more casual.

"Non-product Software V&V" I REALLY hate bringing the terms V&V into non-regulated discussions about tools. Most people involved in my field (again, medical devices) simply can't keep their focus on what aspects of their business are subject to (which) regulations. Simplistically: There is rarely a need to have a complete set of OE that a specific software tool is doing precise things (i.e. verification), there only needs to be a (known, estimated) degree-of-belief that the software tool is doing what you need it to do. Occasionally, some software tools have quite important roles to play in guaranteeing patient safety, and so in those cases, the degree-of-belief (in the tool's performance) needs to be well-specified and have appropriate OE... this is validation. Because of the common term with "design validation", the FDA is desperate to have (medical device) manufactures use a different set of terms for NON-PRODUCT software.

"Test Method Validation" I spent a number of years working with diverse groups responsible for the development of valid test methods. Is it ok if I simply write the following? I have a healthy respect for the science and amount of round-robin testing and peer-review that goes into validating test methods even before the question of "statistical analysis" (of the method) comes into question. Folks with far less appreciation (or understanding) will occasionally equate using a calibrated piece of equipment with a using "validated test method". Sometimes they are correct (to a point), often they are not. I stopped caring deeply about trying to educate people on this point when I witnessed a company try to self-develop a validated method of measuring lengths using calibrated calipers... for items that were within the scale of the calipers. I was (minorly) disciplined for rolling my eyes because I didn't "appreciate that in this industry we are obligated to validate all of our test methods".
 

Ronen E

Problem Solver
Moderator
#9
In a recent thread a regular here stated that they didn’t like the ambiguous phrase “V&V”.
I believe that I am that "regular"... If indeed that's the case, I want to clarify that indeed I don't like the term "V&V", however I don't think it's ambiguous, and that that's not the reason for my dislike. I simply think that bundling these two Vs together misses their whole point in the context in which I'm familiar with them - medical devices Design Control.

I've written a little about what DV means to me in the referenced thread, and also in an article I published in the Resources section, here in Elsmar. That article focuses on Verification, and for a while I've contemplated writing another one about Design Validation. It's definitely a challenging topic and there are some interesting questions around it.
 
Top Bottom