Root Cause Failure Analysis for NADCAP AMS2750D Audit Finding help wanted

zac2944

Involved In Discussions
I'm very new to quality and NADCAP, and just went through my first NADCAP heat treat/hardness testing/metalography audit. We did pretty well, but got dinged with 3 findings. Two were minor calibration cert typos, and one was an issue with how we were performing Temperature Uniformity Surveys (TUS).

I've been able to close out two of the three findings, but I'm stuck on one and I only have one more chance at it. Apparently you only get three trys to get your root cause right then they fail you. Well, the first two were not accepted.

Here's the finding:

Requirement - AMS2750D

3.5.13.3.2 Once data collection begins, temperature data shall be recorded from all TUS sensors at a frequency of at least one set of all readings every two minutes for the duration of the survey. Data from furnace sensors required by the applicable instrumentation type (see 3.3) shall be recorded as follows: (Sensors whose only function is over temperature protection do not need to be recorded.)​


Our furnace type (2B) requires a load thermocouple, and we were not recording this with a dummy load during an unloaded TUS.

We have had several prior NADCAP audits miss this and had in several NADCAP consultants that never picked up on this. We just didn't know we were doing TUS incorrectly.​

My initial root cause response was that we misinterpruted AMS2750D and the corective action would be to include NADCAP's "Pyrometry Guide" as part of our audit and review process.​

I found out after the finding that NADCAP made a clarification to this AMS2750D requirement in their Guide last March, but we didn't pick up on it at the time. I figured if they felt a clarification was required that they would understand how we missed it.​

Not the case. NADCAP responded with "Readdress Root Cause response, requirement is stated in AMS2750D, why was it not identified during review process? How will review/flow down of Specification requirements be improved, now and in the future?".​

When I submitted a second time saying the root cause was a misinterpretation of the spec I got back "Readdress Root Cause response, human error not an acceptable reply.".​

I'm at a loss for how to address this finding and if I don't get it right I might be looking for a new job. We had a system problem, but the human who created the system made a mistake. We were following our internal procedures, but the procedures were incorrect.​

Anyone got advice for a quality rookie?​
 
D

DrM2u

I happen to agree with NADCAP on this one. You have to look at the cause(s) of failure both from a prevention as well as a detection perspective. Some organizations even look at it from a systemic perspective. This being said, I would suggest:

- Investigate why this was not identified when the requirement was clarified: how do you identify (hear about), manage, communicate within and implement changes in regulations
- Investigate why the failure was not identified after the clarification: what detection methods are employed; how effective is your self-assessment process or detection method; is it a gap in knowledge that require auditor training or the audit sampling process missed this requirement; is it an equipment failure; is it ... you get my drift.
- Look at the cause(s) from a systemic perspective, not a human (error) perspective; your system is supposed to eliminate or reduce the potential for human errors; human error is usually not an acceptable root cause because it is traceable back to a planning or management failure

As one method for root cause analysis I would suggest reading a book called "Thinking for a Change" by Lisa Scheinkopf. It is my preferred method for root cause analysis because of its logical approach to cause and effect analysis.

Good luck with 3rd response.
 

zac2944

Involved In Discussions
Thanks for your help DrM2u.

If I may, I'd like to work through your 5-why type questions.

- Investigate why this was not identified when the requirement was clarified: how do you identify (hear about), manage, communicate within and implement changes in regulations

I did include this in my root cause, stating that review of NADCAP's advisories and guides was not a part of our review process. I was sticking to AMS2750D and failed to include the NADCAP advisories and guides.

Why did this happen?
I didn't know it existed. I was never trained specifically on how to prepare for a NADCAP audit. My auditor training simply states "review all applicable specification and requirements", and knowing what is required can be a challange. I know my audit focused on AMS2750D, so I focused on that. Maybe improved auditor training could be part of a root cause here?


- Investigate why the failure was not identified after the clarification: what detection methods are employed; how effective is your self-assessment process or detection method; is it a gap in knowledge that require auditor training or the audit sampling process missed this requirement; is it an equipment failure; is it ... you get my drift.

We do have a recall system (excel spreadsheet) to stay up to date and review certain customer and regulatory specs (Notice to Suppliers, Notice of Change, etc), but this NADCAP material wasn't on that recall.

Why did we fail to have this material on our recall?
We didn't have a proactive system to locate all required specs, requirements, advisories, etc. Maybe a review/improvement of this system is in order?


- Look at the cause(s) from a systemic perspective, not a human (error) perspective; your system is supposed to eliminate or reduce the potential for human errors; human error is usually not an acceptable root cause because it is traceable back to a planning or management failure

I agree with you on this. I think I need to work on my investigative skill to become better at finding system flaws. And thanks for the book recommendation. I'll be adding it to my library shortly.
 
D

DrM2u

Following your logic, you might want to take a second look at your document control process, in particular as it applies to documents of external origin (i.e. making sure that you have the latest and greatest revision on hand). A list of external regulations that is reviewed on a quarterly basis might just do the trick. I don't think that any auditor would ding you for following your procedures and not having a new requirement implemented a week after it was issued. Some regulatory bodies like FDA have sites where users can subscribe to be notified of any regulatory changes, therefore eliminating the need to review documents quarterly as I said in my example.

Regarding the detection method, auditor training probably would not make any difference if you don't train and audit to the latest requirements. It might be part of your (corrective) actions as new requirements are released, though. There is nothing wrong with tracking a failure in detection back to a failure in prevention or planning. Often times I found that poor up-front planning caused both prevention and detection failures down the road.

One might ask now why was there a failure in planning?!? Was it resources, knowledge, culture, management, etc?!? Some great mind (and someone else might be able to verify if it was Deming or not) said a long time ago that most failures are ultimately traceable to the executive management.
 
Top Bottom