Identification of Test Sample in Test Reports?

[Watchcat]

I have noticed that engineering test reports often do not identify the test sample(s).

By "identify," I mean by something like a batch/lot number that would allow traceability to the sample's manufacturing methods and materials, rather than by a brand/model name and number (e.g. Craftsman 1/4-in slotted screwdriver, Model#123) or a physical description (e.g., 1/4-inch slotted screwdriver 12 inches long with acetate handle, etc).

I would be interested to know other people's experience and perspective on this, especially:

If you frequently read/review test reports, what is your experience with test sample identification? Do the reports you read always/usually/sometimes/rarely identify the test sample(s)?

If you routinely prepare test reports, do you typically identify the test sample(s)? Why or why not?

Are there circumstances in which you think test sample identification is critical? Circumstances in which you think it isn't necessary?

Do you know of any authoritative sources for the format and content of an engineering test report?

 

[Jim Wynne]

In general, the requirements for test reports will vary with the circumstances. An in-house report might have different requirements from one that's being provided to a customer, or a report from a contracted independent lab. There are no "authoritative" sources for content that I know of; requirements become authoritative when prescribed or agreed to between parties.

Again in general, reports should identify the specimen(s) tested to the extent that identification is germane to the party that ordered the testing. It's important to define what the output should include when testing is ordered.

 

[Watchcat]

In other words, the report should include the amount and type of information that best addresses its purpose?

 

[Ronen E]

Do the reports you read always/usually/sometimes/rarely identify the test sample(s)?

Usually.

If you routinely prepare test reports, do you typically identify the test sample(s)? Why or why not?

I always identify the test samples to the extent I can. Sometimes the information available to me for the preparation of the report is partial.
Why? To "allow traceability to the sample's manufacturing methods and materials", as well as unambiguous specificity of the design that was evaluated.

Are there circumstances in which you think test sample identification is critical? Circumstances in which you think it isn't necessary?

Define Critical please.
There are no circumstances in which I think it isn't necessary.

Do you know of any authoritative sources for the format and content of an engineering test report?

Applicable regulations (I'm answering from the narrow perspective of medical devices).

Having said all that, it's important to note that "identification of the test specimen" is typically not binomial. Even just stating the model is some level of identification (higher than just "A screwdriver", or naming the manufacturer and providing some descriptors). Some products are manufactured in lots and are also assigned individual serial numbers, so in such instances calling out a lot number is a more precise identification than just the model number, but it's not as precise as specifying individual serial numbers of the evaluated units. What level is necessary is context-dependent.

 

[Jim Wynne]

In other words, the report should include the amount and type of information that best addresses its purpose?

Yes, but as this post illustrates, even providing the necessary information can get you in trouble if a person who reads the report doesn't know what he's reading.

 

[Watchcat]

"Against stupidity, the gods themselves contend in vain." - Frederich Schiller

 

[Ronen E]

Yes, but as this post illustrates, even providing the necessary information can get you in trouble if a person who reads the report doesn't know what he's reading.

In that case the amount and type of information didn't best address the report's purpose (in that specific context), which would have been easily appeasing even a mediocre auditor lacking the necessary background, without having to waste 45 minutes in educating them

 

[Jim Wynne]

In that case the amount and type of information didn't best address the report's purpose (in that specific context), which would have been easily appeasing even a mediocre auditor lacking the necessary background, without having to waste 45 minutes in educating them

It absolutely suited the purpose, unless you think that the purpose was to dumb down the report. Each of the specifications were cited in the report, including the one that said that one of the criteria for passing was cohesion failure of the vinyl. After the initial explanation, I should not have had to explain further. The auditor was an idiot, and I for one won't ever aim that low.

 

[Watchcat]

If I may be so bold, to me this illustrates the difference between seeking regulatory clearance for a medical product and certification for other kinds of products, as well as the role of regulatory (as opposed to compliance) in the former.

In many industries, the extra time and effort that would be required to lay out the rationale for the various tests and carefully explain the link between the results and the performance of the product under conditions of actual use (e.g., why it was a good thing that foam would come apart before the glue would fail) simply isn't worth it from a business perspective. The reasons why not vary with the business and the product, but if you have to do a lot of testing and invest this effort for every test, the value of the product itself, in terms of ROI on that investment in the form of profit, the cost will make the product unattractive as a business venture. On the whole, products with medical application offer a higher ROI than many other products, and therefore are more likely to be worth this investment.

There is also a time pressure, especially for pharma/device startups, which frequently report that they failed because "they ran out of money," a high-level reason that covers a multitude of sins. Nonetheless, it is true that many medical startups run at a crazy burn rate, and, by the time they submit their product to the regulators, they may be looking at the edge of a cliff. If their market clearance gets bogged down, they can run out of the money needed to see it through, and investors, who expect not only a certain return, but within a certain period of time, will decide to write off sunk costs and direct future investments elsewhere. In contrast, a "going concern" with a catalog of products has money coming in on an ongoing basis, and one product getting hung up in certification is unlikely to put them under.

Finally, in most cases, the relationship between test results and conditions of use is much more straightforward with non-medical products (even I got it about the glue and the foam) than with medical use/indication. I don't think it is unreasonable to expect most reviewers to be able to grasp the former relationship with reasonable ease most of the time, where I think it is unrealistic to expect any regulatory agency to have enough reviewers with the range of knowledge, coupled with experience, needed to figure out these links for all the testing done on all the many medical products put before them. And certainly not quickly and easily.

When it comes to putting together a regulatory submission, I've always described my role as writing "the glue that holds the submission together," including, in particular, carefully explaining the links between intended use/indication and test results. I work from the assumption that the reviewers will not see these links unless I explain them to them, and, even if they do, it will take them longer to figure out at least some of the links on their own than it will take them to read through a (hopefully) succinct and cogent explanation. My job is to do whatever I can do to make the review go like butter.

What is not different is that you will come across reviewers in both environments who, having missed the point the first time, will stubbornly dig in their heels, sigh. The ensuing efforts to resolve this kind of impasse can easily lead to delays that a medical startup can literally not survive.

 

[Ed Panek]

Depending on the test I keep samples. For shelf life extension we store the actual samples used in our red tag storage area on a special shelf. If a regulator could question the results ata later date I usually try to ID and retain the samples used.

Also...