How to separate Product Design Review, Verification, and Validation Activities

[sagai]

The review of the Design and Development is a bit vague terminology for me.
Eventually, review itself is a vague terminology, to have an ultimate rule what we are considering as part of the review.
One of the "discovery" I have recently is that mostly all terminology should be discussed between the parties to preserve mix ups.

But you do not go too far with this introduction, sorry for the length of it.

The way I would ensure that the design process is reviewed on a regular bases is the internal audit.
Simple is that.
If there is a need because of whatever reason a more frequent one, than we can schedule it accordingly, however, it is more like a symptom of a different problem.
Or if the project itself considers more frequent reviews and adjustment may need process wise for the R&D project, well, as long as the company QMS allows that and it is in accordance with the regulation, do it and get the work done.

So that's for me a kind of review for the processes.
Actually, it is more like a part of the continuous improvement of the QMS and as such has not too much thing to do with the particular product validation, but ... anyway ... some set of review and the result of the reviews can end up validation related improvement, others not at all.

The other reviews you were mentioned for specifications are definitely for me part of the verification activities (and if we can live with the concept that verification is part of the validation, than also sub-subset of the validation).

That's my opinion would be.

Regards

 

[Mikishots]

My apology for the rebellious thoughts... <snip>

Not rebellious - misunderstood. At no point did I say that validation is equivalent to the end user testing at the end. Re-read my post.

I had a hard time following the rest, as I did not place any limitations on validation as you have suggested.

 

[CATERAF]

The separation is very distinct in certain industries but can be confusing in others. Take the example of designs for structures or infrastructures, they are usually major and take a lot of time and so you can see the distinct steps from input -> design -> output -> verification and then back to input again. The cycle repeats for a few round till the parties involved are satisfied before the design is validated, finalized and then ready for release. Sometimes, the cycle is repeated again if some parties are not happy during the validation stage. Also note that review is ongoing and at every of the steps.

I like this definition, and I think you have to firm understanding of it before you can identify when each part occurs/what to include.

Review - ongoing, happening regularly which basically says 'are we on track?' and then adjusts accordingly.

Ask yourself: Is there any reoccuring point in your process where you review what's going on and the progress made? I.e., a meeting where you stop and check what you've done.
If so, that's when you 'review'. Then ask - how do we record that?
E.g., in software development - we have a weekly meeting that reviews how people are going, what progress they've made, where their tests have failed, etc. We record that in the minutes of the meeting. Thus, the review is the software meeting and the records are the minutes of the meeting.

Verification - start to design something, then check you're on track so far, then adjust if you need to, and repeat. It's like saying you have one big design but to get there you have to do lots of smaller designs which you put together. The verification is verifying each smaller design (before you put it together into the big design).

Ask yourself: Are we doing checks as we go along to make sure we're on track? Where do we record those checks?
Document those checks as 'verification'.
E.g., for us, we have tickets to complete. As we do each ticket, we verify it does what it's supposed to and meets customer requirements. We record that using a program that tracks software changes.

Validation - you've done your verification cycle many times (you've verified each small task) and now you're pretty happy that it's all in working order so you put them together and check the whole lot. (As noted by harry, this is often done as a last step before you release). Note: Validation is basically checking that lots of small working parts are all working together as a bigger unit - it doesn't have to be the 'final' step only.

Ask yourself: When do we do our final check? When do we look at everything? How do you record the results? Where do you record if it doesn't work like it should?
e.g., we do software validation after we've done lots of small 'builds' (verification) - we then go and check everything works together.

Hope that helps!

 

[sagai]

I had no intent to offend or insult what so ever anyone in this forum.

Dear Miki, I had read enough times the post prior replying that, so thank you for mentioning, I did that before several times.

This

Design validation is where you'll determine if the output actually works.

is what it is.

This is, what triggered my reply and that still stands.

Moreover, PLEASE respect others way of thinking and YES, CHALLENGE those, with a sensible manner, but do that with reasoning.
And this is why I would appreciate this forum, the fact that it allows changing thoughts based on our experience and reasonings.

Many thanks, Cheers

 

[CBAL08]

I like this definition, and I think you have to firm understanding of it before you can identify when each part occurs/what to include... <snip>

Thank you for the explanation. Makes it much clearer than reading the standards.

One thing I wanted to ask you since you seem to have experience in SW development. As a Medical device SW we have to follow the regulations and therefore we have delima that once the SW requirements / Design Specs/ Test plans have been approved and then these cannot change in the test phase? Is that true? Or could the test plans change as the testers test the product?

The argument for this is the testers are better than the developers and develop their tetsing skills as they learn more about the product.

Please help

 

[CATERAF]

As a Medical device SW we have to follow the regulations and therefore we have delima that once the SW requirements / Design Specs/ Test plans have been approved and then these cannot change in the test phase? Is that true? Or could the test plans change as the testers test the product?

I had a think about this and I've also had a chat with our software development manager to ask what his opinion is.. we had the same opinion.

We don't think your test plans should change once you're up to the testing phase.
In your design plans you should have defined what the requirements of the software (and software testing) were. That is to say, we want to make sure it can do A, B, and C. Then, when you get to testing, you want to make sure that A, B, and C are achieved. In that sense there is little point changing the test plans because you designed your software for specific requirements - changing the test plans essentially changes the criteria which your software was not designed to do in the first place. If you find something that you think you need to be testing for that was missed then definitely add it in to test for it, but it should be in the specifications first as criteria before it is tested. Hope that makes sense!

Just as a side note, our software developers ensure that the testers know the product. They aren't necessarily an expert in it, but they have to know the product to know how to test it to make sure it does what it is supposed to do. It is the developer who should be the expert of the software.

Hope that helps somewhat..

 

[CBAL08]

Thank you for the answer. It does help.

That menas the test plans should be very undetailed as you ahve said does the SW do A, B and C and not as how do we get A, B and C. The problem we are facing is that several tetsters say that they have new ways of testing and that is not highlighted in the test plans.

What they argue is to have a live test plan which could be changed as the testers change the way they test the product. They also argue that the testers become more experienced with the product as it eveolves. Any thoughts on this ? Would be a great help.

Will give you a short example We have product A which was released in say Oct 2012 ( Product A version 4.0.0)..after it is released there are several maintenance phases ( bug fixes) and several tests are carried out. While tetsing some modules they use the test plans......after a year it is decided that Version 4.1.0 or say 5.0.0 will be released say in Oct 2013...here the test plans will be written down again......I was wondering what would happen if we changed the test plans in maintenance phases as well so that many additional tests are included in the next version testing as well....this was the testings develope and becomes better....I hope you understand what I mean to say?

 

[yeshpal]

I really do not have to divide the review, verification and validation, but ISO 9001 requires records of review, verification and validation, so I'm looking for the answer - what to record in the review, what to record in the verification and what to record in the validation? Recorded information should be consistent with the activity - hence my question about the activity.

the terminology involve definite amount of interaction. But, these are three different steps in process of establishing a product / a service to defined standards.

Review' it is more of first formal review (e.g. peer review) and may be a documented internal procedure.

Verification' is more of certifying formal review procedure with definite document to verify the procedure (e.g. verified by designated personnel).

Validation' is more of regulatory requirement for products/services under regulatory control and it is definite for establishing (Defined, documented and implemented) a product or a service (Internal (inhouse) / external / third party validation) to defined standards.

My perception,
Yeshpal Mathangi

 

[Helmut Jilling]

I think the comments are becoming too complicated. The intent is pretty clear, but the words are not so clear. In practice, you may consider this:

Design Review - periodic, multi-function team review of the progress of the Design Project. The Design Plan is required to define where these review meetings should occur.

Design Verification - Design Inputs and Outputs must be defined and documented. The Design Verification activities compare the outputs as they develop to the inputs, to make sure all the requirements are being address, and that the design as it evolves will be able to meet the requirements. If a part must be able to have a certain strength, this step verifies that those requirements appear to be met. The intent is this is usually done during the electronic stage, with DATA, CAD, specs, etc, before the expense of prototype parts are made.

Design Validation comes later, after these previous steps have been completed. At this point, the math data has been verified, and prototypes or early pre-launch parts have been developed which are verified/validated by actual physical testing. It begins to produce data as to whether all the assumptions put into the design plan are actually going to work on real parts.

NOTE: this is still before PPAP approval and launch. Of course, even after launch, continued testing and evaluation can and should be performed to monitor the product's performance.

 

[CBAL08]

I think the comments are becoming too complicated. The intent is pretty clear, but the words are not so clear. In practice, you may consider this:

Design Review - periodic, multi-function team review of the progress of the Design Project. The Design Plan is required to define where these review meetings should occur.

Design Verification - Design Inputs and Outputs must be defined and documented. The Design Verification activities compare the outputs as they develop to the inputs, to make sure all the requirements are being address, and that the design as it evolves will be able to meet the requirements. If a part must be able to have a certain strength, this step verifies that those requirements appear to be met. The intent is this is usually done during the electronic stage, with DATA, CAD, specs, etc, before the expense of prototype parts are made.

Design Validation comes later, after these previous steps have been completed. At this point, the math data has been verified, and prototypes or early pre-launch parts have been developed which are verified/validated by actual physical testing. It begins to produce data as to whether all the assumptions put into the design plan are actually going to work on real parts.

NOTE: this is still before PPAP approval and launch. Of course, even after launch, continued testing and evaluation can and should be performed to monitor the product's performance.

I agree this is possible in Hardware but what confuses me is the same processof defining Requirements, verifying them and validating them incase of Software. Would be great if how these can be conducted and recorded.