Process to Validate the Accuracy of Test Reports (AS9100)

B

BoardGuy

As Ronald Reagan once said; “here they go again”. Because the RBT is not based on sound risk management principals we sometimes have other parties trying say that everything must be RBT. Ha no, Top Management get to decide that one along with what is a process and what is not a process within the organization.

While everyone is throwing the validation of test report accuracy under the RBT umbrella (6.1) you missed something. 8.4.2, says “When a customer or organization has identified raw material as a significant operational risk…” I believe that Clause 8.1.1 address management of “operational risk” and within Note 1 of this same clause it is clearly stated that “While clause 6.1 addresses the risks and opportunities when planning for the QMS of the organization, the scope of this clause (8.1.1) is limited to risk associated to operational processes needed for the provision of products and services”. OOOOPs, those darn other parties got it wrong again per the very Standard they say requires RBT for test reports.
 
Last edited:

howste

Thaumaturge
Trusted Information Resource
As Ronald Reagan once said; “here they go again”. Because the RBT is not based on sound risk management principals we sometimes have other parties trying say that everything must be RBT. Ha no, Top Management get to decide that one along with what is a process and what is not a process within the organization.

While everyone is throwing the validation of test report accuracy under the RBT umbrella (6.1) you missed something. 8.4.2, says “When a customer or organization has identified raw material as a significant operational risk…” I believe that Clause 8.1.1 address management of “operational risk” and within Note 1 of this same clause it is clearly stated that “While clause 6.1 addresses the risks and opportunities when planning for the QMS of the organization, the scope of this clause (8.1.1) is limited to risk associated to operational processes needed for the provision of products and services”. OOOOPs, those darn other parties got it wrong again per the very Standard they say requires RBT for test reports.

I don't see 6.1 and 8.1.1 as mutually exclusive. 6.1 applies to the entire QMS, while 8.1.1 is a subset that applies primarily to clause 8.

Validation of test reports is done when determined as an operational risk per 8.1.1. This activity is planned as part of the QMS, so 6.1 also applies. In other words, 6.1 and 8.1.1 can both apply to test reports. Clause 8.1.1 does not create a bubble of RBT nonapplicability around clause 8 activities.
 

Sidney Vianna

Post Responsibly
Leader
Admin
Because the RBT is not based on sound risk management principals we sometimes have other parties trying say that everything must be RBT.
the word is principle, by the way.

If you don't agree that accepting critical raw materials based solely on a supplier's CoC/test report has an inherent degree of risk for product conformity, there is nothing I can say to convince you of the contrary. Since AS9100D embeds ISO 9001:2015, what does RBT mean, according to the authors of the standard?

What is risk-based thinking?

One of the key changes in the 2015 revision of ISO 9001 is to establish a systematic approach to considering risk, rather than treating “prevention” as a separate component of a quality management system.
Risk is inherent in all aspects of a quality management system. There are risks in all systems, processes and functions. Risk-based thinking ensures these risks are identified, considered and controlled throughout the design and use of the quality management system.
In previous editions of ISO 9001, a clause on preventive action was separated from the whole. By using risk-based thinking the consideration of risk is integral. It becomes proactive rather than reactive in preventing or reducing undesired effects through early identification and action. Preventive action is built-in when a management system is risk-based.

The ASD industry does believe the risk is present and serious. So much so that they reverted the deletion of the requirement for independent checks in the AS9100D Standard.
The following slide, from an IAQG presentation on AS9100D clearly mentions the risk aspect of this activity:

Process to Validate the Accuracy of Test Reports (AS9100)

As I wrote earlier in this thread, Boeing Commercial had identified the deletion of the requirement in AS9100C as a product conformity risk and embedded it in their X31764 Supplier Quality requirements document. So, for those Industry interested parties that believe the risk exists, the question then becomes: how to mitigate the risk? Risks will exist, irrespective if a misguided supplier top management decide to accept it or not.

Let's also remember that AS9100D introduced requirements for the prevention of counterfeit products. Anyone with minimum knowledge of the ASD Industry is aware of cases of counterfeit materiel being supplied by unscrupulous vendors. Robust verification of the accuracy of supplier test reports/CoC's would obviously minimize the chances of bogus raw materiel being allowed in the supply chain.
 
Last edited:

Jim G

Involved In Discussions
If you don't agree that accepting critical raw materials based solely on a supplier's CoC/test report has an inherent degree of risk for product conformity, there is nothing I can say to convince you of the contrary.

Sidney
What would be classed as "Critical" raw materials, and who decides what is critical and what isn't?

As a small company specialising in Aircraft interiors, almost all the materials (and there are many), we use are Flammability tested by our suppliers at an independent laboratory. During our recent re-certification audit, the Auditor stated for transition next year he will be looking closely at this validation process since he regards the materials as critical.

To revalidate suppliers CofC's for all our materials would become an extremely expensive business cost and may result in us declining to quote for some work.
 

Mike S.

Happy to be Alive
Trusted Information Resource
Sidney
What would be classed as "Critical" raw materials, and who decides what is critical and what isn't?

AS9100D says, "When a customer or organization has identified raw material as a significant operational risk (e.g., critical items), the organization shall implement a process to validate the accuracy of test reports."

So you or your customer decides. It is NOT for the auditor to decide! If he/she thinks they should decide they are a pinhead and need to be corrected or, if they will not relent, be reported, IMO.

"When" is a key word in that paragraph, IMO.

Maybe you use a different method to “confirm that the product meets requirements”. If you use steel and its critical characteristic is tensile strength, maybe you do your own strength test. If flammability is a critical characteristic, maybe you do your own flammability test.
 

Sidney Vianna

Post Responsibly
Leader
Admin
Sidney
What would be classed as "Critical" raw materials, and who decides what is critical and what isn't?
AS9100D defines special requirements, critical items, key characteristics, counterfeit parts and product safety.

If the raw material has characteristics that make it a critical item, then it would be a material which has a significant operational risk. As far as I can tell, flammability requirements are a safety critical item.
 

Mike S.

Happy to be Alive
Trusted Information Resource
AS9100D 3.2, Critical Items: Those items (e.g., functions, parts, software, characteristics, processes) having significant effect on the provision and use of the products and services; including safety, performance, form, fit, function, producibility, service life,etc.; that require specific actions to ensure they are adequately managed.

IMO, with all due respect to Sidney, it is not his or any 3rd party auditor's place to determine if Jim's materials are critical or pose "a significant operational risk" and therefore would require a process to validate the accuracy of the supplier's test reports, and if validation is required, what an adequate validation process would entail.

Jim is trying to stay in business -- so this is apparently not a trivial matter for him. One should assume that between the experts at his customer and his company they have the knowledge necessary to determine if it is critical or not, and if it is, how to validate it. What 3rd party auditor has the knowledge necessary to gainsay them?
 

Sidney Vianna

Post Responsibly
Leader
Admin
IMO, with all due respect to Sidney, it is not his or any 3rd party auditor's place to determine if Jim's materials are critical or pose "a significant operational risk" and therefore would require a process to validate the accuracy of the supplier's test reports, and if validation is required, what an adequate validation process would entail.
Don't worry. With so many choices of CB's, one can always find a blind gopher auditor who is inept, incompetent and/or coward enough never to question anything is put in front of them. They simply roll over and rubber stamp whatever decision their "customers" make.

But then, years later, when people die of smoke inhalation due to a cabin fire onboard a plane, there will be some pundits who will want auditor heads on a plate for failing to keep the certified systems accountable to the regulations and requirements of the industry.

Obviously the determination of raw materials that have a significant operational risk is the responsibility of the supplier and/or their customers as the standard stipulates. For those organizations seeking ICOP-endorsed accredited certification to AS9100D, it is the auditor's job and responsibility to ensure the determination was done in a sound manner, taking into accounts the risks associated with the decision.

Aerospace is an industry with higher costs, comparatively to others. one of the reasons being the obvious fact it comes with risks that have to be managed and mitigated.
 

Mike S.

Happy to be Alive
Trusted Information Resource
For those organizations seeking ICOP-endorsed accredited certification to AS9100D, it is the auditor's job and responsibility to ensure the determination was done in a sound manner, taking into accounts the risks associated with the decision.

The question was "who decides what is critical and what isn't?"

The standard (and I) say it is the customer and supplier.

How are YOU gonna determine if it was "done in a sound manner" if you're the auditor?

Where does this "sound manner" requirement come from? It ain't in 9100 or 9101.

You review the customer's spec's and see no specific requirement for validation.

Maybe the auditee, himself an expert in the field, says it was considered and it is not an issue. Is that good enough for you?

Maybe he says he discussed it with the customer over lunch one day and they are okay with how things are being done. Are you gonna contact the customer?

How many of us have experienced auditors of all types who pretend to be more knowledgeable about technical aspects like this than the supplier and customer and end-up creating new requirements out of thin air?

If the auditor is truly qualified it's one thing, but IMO that is rarely the case when they try to insert themselves into technical decision making.
 

Sidney Vianna

Post Responsibly
Leader
Admin
How are YOU gonna determine if it was "done in a sound manner" if you're the auditor?
I can tell you that things such "financial hardship" is no justification to avoid validation; justifications like
Jim is trying to stay in business
would not be acceptable, as that is a slippery slope. If the supplier can't afford to send samples for independent validations, then, they start cutting corners in other aspects of the QMS.

So, it looks like you have not been exposed to knowledgeable, competent, professional auditors. I have, I know many that are much more knowledgeable about regulatory and system requirements than the typical registrants. Your perspective seems to be: the auditor has to accept whatever is being presented to them. So, what is the point of auditing? I strongly disagree with that perspective. Auditors MUST ensure the requirements and the intent of the requirements are being complied with. That's the ONLY WAY certification adds value to the users of the certificates.
 
Last edited:
Top Bottom