How to separate Product Design Review, Verification, and Validation Activities

[sagai]

My apology for the rebellious thoughts, but validation for me far not equivalent with a kind of end user testing at the end of the development activities.
Is it part of it? Yes, definitely, but the criteria set for validation can not be covered with any kind of, any depth what so ever testing activities.
For me validation is a set of activities or even more, the organizational attitude (I would not go that far at this time ... ) in the reality, but let me go back, so a set of activities performed to ensure the intended use of the product fulfills it in the intended environment.
Some examples in order to go down to the basement of every day reality.

Configuration management activities, defect management activities, acceptance activities of supplied materials or interim products during the R&D phase also targeting the fact that we want to make sure the product at the end of R&D fulfills its intended use. Are these all of the activities we carry out to do effective validation? Far not. Verification is also a kind of activities we are doing to ensure the end product is the one and only according to our best understanding at that time.
Is verification equivalent with testing? Very much not. Review, discussion, investigation of scientific literature, etc. also part of it.

Is there any predefined sequential order in the standard?
Faaar not. Read it, it is about principles, not about the sequences.
Each company has to find its own way and awareness how to manage and organize validation.

I guess ...

Cheers!

[Peters]

What do you think - Is review of the design and development focused more on the design process or more on the design documentation/specification?

[sagai]

The review of the Design and Development is a bit vague terminology for me.
Eventually, review itself is a vague terminology, to have an ultimate rule what we are considering as part of the review.
One of the "discovery" I have recently is that mostly all terminology should be discussed between the parties to preserve mix ups.

But you do not go too far with this introduction, sorry for the length of it.

The way I would ensure that the design process is reviewed on a regular bases is the internal audit.
Simple is that.
If there is a need because of whatever reason a more frequent one, than we can schedule it accordingly, however, it is more like a symptom of a different problem.
Or if the project itself considers more frequent reviews and adjustment may need process wise for the R&D project, well, as long as the company QMS allows that and it is in accordance with the regulation, do it and get the work done.

So that's for me a kind of review for the processes.
Actually, it is more like a part of the continuous improvement of the QMS and as such has not too much thing to do with the particular product validation, but ... anyway ... some set of review and the result of the reviews can end up validation related improvement, others not at all.

The other reviews you were mentioned for specifications are definitely for me part of the verification activities (and if we can live with the concept that verification is part of the validation, than also sub-subset of the validation).

That's my opinion would be.

Regards

[Mikishots]

In Reply to Parent Post by sagai

My apology for the rebellious thoughts... <snip>

Not rebellious - misunderstood. At no point did I say that validation is equivalent to the end user testing at the end. Re-read my post.

I had a hard time following the rest, as I did not place any limitations on validation as you have suggested.

[CATERAF]

The separation is very distinct in certain industries but can be confusing in others. Take the example of designs for structures or infrastructures, they are usually major and take a lot of time and so you can see the distinct steps from input -> design -> output -> verification and then back to input again. The cycle repeats for a few round till the parties involved are satisfied before the design is validated, finalized and then ready for release. Sometimes, the cycle is repeated again if some parties are not happy during the validation stage. Also note that review is ongoing and at every of the steps.

I like this definition, and I think you have to firm understanding of it before you can identify when each part occurs/what to include.

Review - ongoing, happening regularly which basically says 'are we on track?' and then adjusts accordingly.

Ask yourself: Is there any reoccuring point in your process where you review what's going on and the progress made? I.e., a meeting where you stop and check what you've done.
If so, that's when you 'review'. Then ask - how do we record that?
E.g., in software development - we have a weekly meeting that reviews how people are going, what progress they've made, where their tests have failed, etc. We record that in the minutes of the meeting. Thus, the review is the software meeting and the records are the minutes of the meeting.

Verification - start to design something, then check you're on track so far, then adjust if you need to, and repeat. It's like saying you have one big design but to get there you have to do lots of smaller designs which you put together. The verification is verifying each smaller design (before you put it together into the big design).

Ask yourself: Are we doing checks as we go along to make sure we're on track? Where do we record those checks?
Document those checks as 'verification'.
E.g., for us, we have tickets to complete. As we do each ticket, we verify it does what it's supposed to and meets customer requirements. We record that using a program that tracks software changes.

Validation - you've done your verification cycle many times (you've verified each small task) and now you're pretty happy that it's all in working order so you put them together and check the whole lot. (As noted by harry, this is often done as a last step before you release). Note: Validation is basically checking that lots of small working parts are all working together as a bigger unit - it doesn't have to be the 'final' step only.

Ask yourself: When do we do our final check? When do we look at everything? How do you record the results? Where do you record if it doesn't work like it should?
e.g., we do software validation after we've done lots of small 'builds' (verification) - we then go and check everything works together.

Hope that helps!

[sagai]

I had no intent to offend or insult what so ever anyone in this forum.

Dear Miki, I had read enough times the post prior replying that, so thank you for mentioning, I did that before several times.

This

Design validation is where you'll determine if the output actually works.

is what it is.

This is, what triggered my reply and that still stands.

Moreover, PLEASE respect others way of thinking and YES, CHALLENGE those, with a sensible manner, but do that with reasoning.
And this is why I would appreciate this forum, the fact that it allows changing thoughts based on our experience and reasonings.

Many thanks, Cheers

[CBAL08]

In Reply to Parent Post by CATERAF

I like this definition, and I think you have to firm understanding of it before you can identify when each part occurs/what to include... <snip>

Thank you for the explanation. Makes it much clearer than reading the standards.

One thing I wanted to ask you since you seem to have experience in SW development. As a Medical device SW we have to follow the regulations and therefore we have delima that once the SW requirements / Design Specs/ Test plans have been approved and then these cannot change in the test phase? Is that true? Or could the test plans change as the testers test the product?

The argument for this is the testers are better than the developers and develop their tetsing skills as they learn more about the product.

Please help

[CATERAF]

As a Medical device SW we have to follow the regulations and therefore we have delima that once the SW requirements / Design Specs/ Test plans have been approved and then these cannot change in the test phase? Is that true? Or could the test plans change as the testers test the product?

I had a think about this and I've also had a chat with our software development manager to ask what his opinion is.. we had the same opinion.

We don't think your test plans should change once you're up to the testing phase.
In your design plans you should have defined what the requirements of the software (and software testing) were. That is to say, we want to make sure it can do A, B, and C. Then, when you get to testing, you want to make sure that A, B, and C are achieved. In that sense there is little point changing the test plans because you designed your software for specific requirements - changing the test plans essentially changes the criteria which your software was not designed to do in the first place. If you find something that you think you need to be testing for that was missed then definitely add it in to test for it, but it should be in the specifications first as criteria before it is tested. Hope that makes sense!

Just as a side note, our software developers ensure that the testers know the product. They aren't necessarily an expert in it, but they have to know the product to know how to test it to make sure it does what it is supposed to do. It is the developer who should be the expert of the software.

Hope that helps somewhat..