The organization shall audit products at appropriate stages of production and delivery to verify conformity to all specified requirements, such as product dimensions, functionality, packaging, labeling, at a defined frequencyThere are many approaches to analyse quality management system, product quality and process performance. In the context of the internal audit clause, internal audit for the organization should be independent of those having direct responsibility for the work performed. Personnel should not audit their own work.When I do a product audit, I start with the product in the box on a pallet, ready to ship, and work my way backwards. I make sure that the units are packaged per customer requirements, labeled correctly, open a box an assure that the product inside is complete and matches the box label (including serial number). I then track back and check for test sheets properly filled out, revisions on the work instructions & drawings, signed off work order releasing the product, proper calibration and tools, etc. I finish off with the "front office" work like production scheduling, order entry, order review, EDI, etc.

Here's the question: Should I pull the product from the box and re-test it to confirm that the work that the technician has already done was done correctly and that all the test equipment is functioning properly?

I don't want to inject any more because I may tend to color the responses; I'm in the middle of a TS Periodic Audit so I might not check back in until later.
Thanks, Icy

Icy:
For a product audit, I feel you're already going too far! By tracing back to the test results, calibration, order etc., you've done, in effect, a reverse process audit!

What you described, initially, is a great product audit, and you could, periodically, take one out and have it tested/laid out. But stop at compiling all the other stuff. It's not a product audit, then, but a process and way too much to do.......especially since you already have to do something similar, for TS!
Hope the audit goes well.....

What you described, initially, is a great product audit, and you could, periodically, take one out and have it tested/laid out. But stop at compiling all the other stuff. It's not a product audit, then, but a process and way too much to do.......especially since you already have to do something similar, for TS!Well, yes, I am doing a full process audit from end to beginning; the product audit is a subset of that. I see no sense in doing them separately. Now, the question is: Is it a TS requirement to re-test that exact product as part of the product audit? What if I audit the testing being performed on a different unit coming down the line?

My understanding is that when performing product audits, functionality should be performed (periodically). An audit of the testing/inspection function is just that, not a product audit. Also, (I don't have the standard to hand) but I believe that unless the inspection/testing verified all aspects of specification, it wouldn't count, either.

I still think you're doing too much......the drivers of product/process/system audits are not going to be identical and I feel like you're putting too much into each audit to do this each time.

Just trying to make your day easier.........

My understanding is that when performing product audits, functionality should be performed (periodically). An audit of the testing/inspection function is just that, not a product audit. Also, (I don't have the standard to hand) but I believe that unless the inspection/testing verified all aspects of specification, it wouldn't count, either.We're getting closer to the point here. Our customer's print is our print on their title block. There are a (very) few dimensions but this is not a fit & finish part. The real specification is the list of input/output voltages and power ratings. As part of the product audit, we measure the dimensions but they are not "Key Control Characteristics". However, all of the electrical specifications (and more) on the print are 100% tested and recorded on each and every unit.

I don't see the utility of re-testing the specific unit during a product audit if the test sheet is complete and correct and I am also auditing the test function, during the testing of another unit, to assure its efficacy.

We have recent returns of 12 month old units for a defect whose root cause was a poor component specification that led to an intermittent condition inside the unit. It has been suggested that our product audit frequency be increased and products re-tested during the audit. However, the component specification has been corrected and no amount of functional testing or auditing would have ever uncovered this problem.

Actually, the only conceivable way to have discovered this defect would be to run an large sample (10? 15?) units through the original reliability testing suite. Before you say, "That's a great idea, Icy, do it!": We have shipped 300 units in 18 months at a $3k cost and virtually zero margin. I'd be better off terminating production than dropping $30k to $45k worth of units into the grinder.

Here's the question: Should I pull the product from the box and re-test it to confirm that the work that the technician has already done was done correctly and that all the test equipment is functioning properly?


I wouldn't go that far unless I had a specific reason for doing so, such as reason to believe that the standard testing process wasn't working as planned. As you backtrack, you go through the testing documentation, and I think it's best to assume that if the documentation indicates that requirements have been met, then the purpose of the audit has been satisfied.

Your regular internal audits should be covering all of the processes that result in the product being in the box on the shipping dock, so the product/process audit should serve as a confirmation that they all work together as planned (since individual audits might not always cover all of the intersections). Although I think that the backtracking method is good, I think that you should also occassionally do it the other way around, when the support processes are actually in action for the product in question, so that you can actually see the testing being done. This means that under some circumstances, a product/process audit might take several days to complete, an hour here and and hour there, until you've actually followed product all the way through the system.

IMHO, ISO/Ts 16949 goes overboard on audits!:mg: I agree with Jim about the internal audit programme covering the regular processes of getting the product to the dock, however, the requirements are the requirements and I believe it says (don't have the std to hand) that you have to verify the thing to spec., including functionality. That's under the product audit requirements, so I don't see how you can verify the paperwork only. I know it doesn't make sense, but, they are what they are. I guess if you can persuade your external auditor, it might fly........

IMHO, ISO/Ts 16949 goes overboard on audits!:mg: I agree with Jim about the internal audit programme covering the regular processes of getting the product to the dock, however, the requirements are the requirements and I believe it says (don't have the std to hand) that you have to verify the thing to spec., including functionality. That's under the product audit requirements, so I don't see how you can verify the paperwork only. I know it doesn't make sense, but, they are what they are. I guess if you can persuade your external auditor, it might fly........

I understand your concern, and I agree that product functionality must be verified at some point. My point was that if you've already verified the efficacy of the testing/inspection process (or processes), and you have some history that indicates that those processes are efficacious, it shouldn't be necessary to verify product performance every time a product audit is done. In my way of thinking (which is admittedly wandering around somewhere outside the box) if I have ample historical records of process efficacy, the burden of proof is on the auditor if he's working under the assumption that processes might not be efficacious.

Although I think that the backtracking method is good, I think that you should also occassionally do it the other way around, when the support processes are actually in action for the product in question, so that you can actually see the testing being done. This means that under some circumstances, a product/process audit might take several days to complete, an hour here and and hour there, until you've actually followed product all the way through the system.There's an idea! Even though the audit might take longer to complete, the actual time spent would be the same. Following the processes from beginning to end would result in auditing the functionality of the product while documenting that the product meets the specifications by actually observing the product testing during the product audit!:bonk:
I understand your concern, and I agree that product functionality must be verified at some point. My point was that if you've already verified the efficacy of the testing/inspection process (or processes), and you have some history that indicates that those processes are efficacious, it shouldn't be necessary to verify product performance every time a product audit is done. In my way of thinking (which is admittedly wandering around somewhere outside the box) if I have ample historical records of process efficacy, the burden of proof is on the auditor if he's working under the assumption that processes might not be efficacious.Yet another cogent observation, Mr. Wynne. I have one failure at delivery and installation and the test process at the time would not have detected the defect. The improved test method was validated when it was added to the process (I wish I had generated a validation record:mad:). The rest of my defects turn up after the unit has been bounced around in a vehicle for 6 months or a year. The only process improvements that have been instituted have been validated, audited and proven to be efficacious (love that word). Design shortcomings, that would never be discovered in a product audit anyway, that's another story.

I understand your concern, and I agree that product functionality must be verified at some point. My point was that if you've already verified the efficacy of the testing/inspection process (or processes), and you have some history that indicates that those processes are efficacious, it shouldn't be necessary to verify product performance every time a product audit is done. In my way of thinking (which is admittedly wandering around somewhere outside the box) if I have ample historical records of process efficacy, the burden of proof is on the auditor if he's working under the assumption that processes might not be efficacious.

D&mn, Jim we agree!

However, I seriously believe that what you described, although a great basis for any internal audit, doesn't meet the more inspection oriented requirements of the TS std. Despite of all the principles of prevention defined in the ISO 9001 requirements, I feel that TS has a number of more 'after the fact' tools to catch problems - ostensibly to 'protect' the customer - and product audits are one of those.

D&mn, Jim we agree!The unholy trinity, run for your lives!!!
I agree again about some TS additions being "quality control" oriented. I am just trying to meet the standard while also accomplishing something that will actually improve process and product quality. Increasing the frequency of re-testing product doesn't seem to fit that bill.

Smoke signal to Chief Icy Mountain -

Always a good idea to ask your customer authorized representative . :magic:

When the canoe has washed ashore , the big lake has spoken.

When bad part sent to customer by mistake , forgot to measure twice.

" he who makes coffee nervous "

The unholy trinity, run for your lives!!!


:lmao::lmao::lmao::lmao::lmao:

What could be the frequency in conducting product audit for TS16949? Is there a requirement in TS16949 how many times to conduct product audit? And whose going to perform the audit? Is it the internal auditor? A QA Engineer? Please advise.
Thanks in advance.
Best regards,
Raffy:cool:

Raffy,
The frequency for product audits is defined by your organization based on a confidence level you establish, unless specified by your customer. TS16949 does not specify how or frequency. No one else knows your processes, capabilities, and risks. Anyone who is competent can do product audits, better yet if they can also analyze the results.

Hi ralphsulser
Thank you very much for the immediate reply.:thanx:
To date, we don't encounter any problem with regard to the product itself, since we don't have any better yet set it at least once a year.
Thank you very much.
Best regards,
Raffy:cool:

The frequency for product audits is defined by your organization based on a confidence level you establish, unless specified by your customer. TS16949 does not specify how or frequency. No one else knows your processes, capabilities, and risks.This is why this did not rise to a the level of a finding. My customer does not specify a frequency; they say, "That's your business." We functional test 100% of our units and record the results. Not a single one of our warranty returns would have been detected by quarterly or even monthly product audits that included a retest of the unit.

This is where the economic justification of quality activities is required. Unfortunately, the subject gets ignored by some quality professionals. The original forecast for this product was over 2000 units per year. We have shipped about 300 since November of 2005. We have only shipped about 25 in 2007. The margin at this rate is 0%. I cannot justify any activities that do not add value. Pulling an additional 3 or 5 or 11 units from boxes and completely retesting them isn't going to improve the long-term reliability of this product. I don't have an out-of-the box failure issue.

Our customer does not specify a frequency of conducting product audit, but during our Pre-assessment audit, it was noted that Internal Audit must be performed, e.g. System Audit, Manufacturing Audit and Product Audit. That’s why in our Internal Audit, we revise our procedure and provide a frequency of once a year, but still I did not make any specific action on how am I going to do this. I conduct a product audit and so far I don’t see any problem with regard to the product itself. In our Production Area, they are conducting 100% Visual and Functional Tests and to date, they are not experiencing any Customer returns. In QA, we conduct sampling and there were no QA Buy-off reject.
My concern is Can we omit the process of Product Audit with the data that we have both in Production and QA? Can we say that Product Audit was thoroughly performed by our QA & Production Personnel. And that per se, can replaced the product audit? Please advise.:confused:
Best regards,
Raffy :cool:

Raffy,
You cannot omit the requirement for product audit if you make and ship a product. However, from how you described your system, I think that your QA product audits would satisfy the product audit as required. You need to have records of evidence and credible documentation that shows the audit criteria and results.

Hi ralphsulser
QA Buy-off is performed in all stages of the Production Line. Including Final Packing, that is checking the labels, the shipment pre-alert document and each buy-off were documented in a QA logsheet. It also has a QA Stamp and the signature of the inspector who conduct the product audit.
In view of this, I would rather update my Internal Audit Procedure that Product Audit was performed by QA Inspector, because if I would be the one auditing it once a year, it would be a redundant process on my end.
Thank you very much for enlightening me.:thanx:
Best regards,
Raffy:cool:

Briefly - during process audit I am focused on operations (I do not care if product is OK or NOK;)), but if everything is done according to our QMS procedures and work instructions. During product audit I am interested in status "product versus specifications" (I do not care how it was done).
Schedules for both these audit types is based on a control plan, where are defined requirements related to process and product. E.g. during process audit of curing operation, I audit process parameters against work instruction and during product audit for cured part I audit product characteristics against inspection checksheets.
I simply pick up few samples of in-process parts and I ask inspector for testing them, resulting in damage of parts. In case of final product I check them only visually or measure using gauge for final inspection, because I am not able to refill missing quantity in warehouse.

Gee the last time I did a product audit was when I was a QA inspector 14yrs ago. The 16949 standard doesn't state that internal auditors have to do product audits, they just have to be done by someone that is not part of the same group makin the darn thing. QA has always and still does product audits in our company throughout the line at an operator level. They also do an out of box sample, all which will always meet the intent of the standard just fine. Your process appears to be fine Raffy...I agree with Ralph.

We have our Desk audit yesterday, we clarify with our auditor that product audit performed by Internal Audit is different from the product audit performed by our Inspector because he said that Product Audit performed by our Inspector is already part of the process. However Product Audit performed by Internal Auditor is an add value to the system. Is it really right? I’m confused. :frust:
Thanks in advance.
Please advise.
Best regards,
Raffy

I merged process and product together audits together.
Start from Goods Inwards or despatch doesnt really matter depends on your prediliction.
Follow the process through and grab a series of parts during the manufacturing process and watch the manufacturers inspect record the results.

Our Certification auditor was killing us with his 'heavy demands' i decided that walking the areas twice and generating two lots of paperwork was ludicrous.
So i merged the 2 and use the manufacturers (with my studious supervision - to ensure that it is being inspected right) was the best approach.
As for the certification auditor i stood my ground and proved requirements were being met. Now we focus on the audit not the auditors interperetations you can see the difference.

As for the product verification he accepts this as part of the audit.

Smoke signal to Chief Icy Mountain -
Always a good idea to ask your customer authorized representative . :magic:Customer representative says that the product needs to be retested by "Quality", pointing out that "the purpose of Product Audits is an independent examination of the product to determine whether or not it meets the requirements of the established product specification. This would include actually testing the product." This was coupled with a direct quote from the (non-auditable) guidance on Sections 8.2.2.1 through 8.2.2.3: "There are many approaches to analyze quality management system, product quality and process performance. In the context of the internal audit clause, internal audit for the organization should be independent of those having direct responsibility for the work performed. Personnel should not audit their own work."

Therefore, as before, I will combine the product and process audits but I am going to time the production portion of the process audit so that I am able to re-test a unit myself, right after it has been production tested. This may seem like a waste of time. However, coupled with the process audit, it allows me to get a hands-on audit of the test equipment condition, process instruction accuracy, agreement between my measurements and the technician's, etc. I can also make my other audit measurements at this time, thus saving me from actually removing the unit from the box in shipping (only checking contents and labeling).:truce:

I merged process and product together audits together.
Start from Goods Inwards or despatch doesnt really matter depends on your prediliction.
Follow the process through and grab a series of parts during the manufacturing process and watch the manufacturers inspect record the results.

Our Certification auditor was killing us with his 'heavy demands' i decided that walking the areas twice and generating two lots of paperwork was ludicrous.
So i merged the 2 and use the manufacturers (with my studious supervision - to ensure that it is being inspected right) was the best approach.
As for the certification auditor i stood my ground and proved requirements were being met. Now we focus on the audit not the auditors interperetations you can see the difference.

As for the product verification he accepts this as part of the audit.

Good for you Quality Ian, I did the same thing. The company I am at go so overboard on everything it's ridiculous. We have Process Audits, Product Audits & Doc Audits. Out inspectors perform product and doc audits everyday but they are never documented as any kind of audit at all. It took me a week to convince my supervisor that the combination of the product & doc audits into the inspectors daily duties will satisfy the standard and then some. We are doing more that we really need to but since the inspectors are already out on the floor may as well have them document what they do.

In my opinion, checking inspection records is not a product audit per TS 16949. Checking inspection records is a process audit of the inspection process.

A product audit should directly verify if the product conforms to all requirements- an independent verification from the normal process.

In my company we manufacture metal stamped & assembled products.

Our operators (level 1) and Quality inspectors (level 2) check visual & dimensional specifications on a regular basis following the control plan.

Now, some others dimensions are not covered at these levels of inspection. Neither are raw material, heat treatment & functional specifications (annual complete functional tests are sometimes paid by the customers eg. : fatigue tests).

So our product audits include those other specifications (which don't need to be checked as often) and are carried out by our internal laboratory staff (level 3).
They also include a dock audit to check the packaging & label condition.
This way we make sure that 100% of the specifications are checked on the whole.

By the way, German manufacturers impose their own product audit standard (VDA 6.5) so there is less room for interpetation when we follow it.



Note : for our type of products, the 8.2.2.3 and 8.2.4.1 requirements are almost the same.

8.2.4.1 Layout inspection and functional testing
A layout inspection and a functional verification to applicable customer engineering material and performance
standards shall be performed for each product as specified in the control plans. Results shall be available for
customer review.

Note : Layout inspection is the complete measurement of all product dimensions shown on the design records

Hope this helps.

Francois

In my opinion, checking inspection records is not a product audit per TS 16949. Checking inspection records is a process audit of the inspection process.

A product audit should directly verify if the product conforms to all requirements- an independent verification from the normal process.

I don't believe I stated anywhere that my inspectors were checking just inspection records. Along with verifying all the paperwork at a work station my inspectors inspect the product. It is an independent audit over and above the normal process. The inspectors perform this inspection every couple of hours or however long it takes them to make a round of the production floor. Once the product in inspected the inspectors then perform the Doc Audit.

Hello Hoeyster:

I think you are assuming that my comment is directed at you since my posting follows yours. Someone on the thread stated that their Product Audit consisted of a review of the inspection records. Of course, now I can't find that posting.

in addition to my previous post, I attach a feed-back from the SMMT body on Product Audits.

Francois

Thank you so much for that attachment, it really gave me some insite to what a Product Audit really is. What is SMMT?

Thank you so much for that attachment, it really gave me some insite to what a Product Audit really is. What is SMMT?

Hello Hoeyster,

The Society of Motor Manufacturers and Traders (SMMT) exists to support and promote the interests of the motor industry in the UK and abroad.

SMMT (http://www.smmt.co.uk/home.cfm)

Stijloor.

In my opinion, checking inspection records is not a product audit per TS 16949. Checking inspection records is a process audit of the inspection process.

A product audit should directly verify if the product conforms to all requirements- an independent verification from the normal process.Your opinion matches that of my customer. I am doing product audits next week. I will be personally re-testing a product that was already tested and verifying that the readings taken are correct (product and process audit simultaneously). While I still believe that this is mostly a waste of time, it will give additional depth to the process audit as I will be auditing fixtures, test equipment calibration, adherence to the process instruction, etc. This product will then go back into the normal production stream and I'll catch it again on the dock to audit product and process from test to shipment ready. I steadfastly refuse to pull a product out of a box, transport it all the way back to the lab, perform testing that has already been performed, and then take the unit all the way back to shipping.I don't believe I stated anywhere that my inspectors were checking just inspection records.I said that.I think you are assuming that my comment is directed at you since my posting follows yours. Someone on the thread stated that their Product Audit consisted of a review of the inspection records. Of course, now I can't find that posting.That would have been me. I still don't see the efficacy of randomly retesting product that is 100% tested. If I have done a process audit and validated that the Technician is following the test procedure, the fixturing and equipment are all correct, and the test results produced show the product to be within specification, what exactly am I accomplishing by testing it again?

"I still don't see the efficacy of randomly retesting product that is 100% tested. If I have done a process audit and validated that the Technician is following the test procedure, the fixturing and equipment are all correct, and the test results produced show the product to be within specification, what exactly am I accomplishing by testing it again?"

I don't see the value of product audits either but what does that have to do with conformity to TS requirements? I didn't know we can exclude requirements becaue we don't see the value.

I don't see the value of product audits either but what does that have to do with conformity to TS requirements? I didn't know we can exclude requirements becaue we don't see the value.Of course we can't. It just really burns me to do something just to satisfy the standard without adding some value to the company. I am going to use this as an opportunity to cross-train technicians on product testing.