Internal Audit Question - 8.5.1 Continual Improvement

[BubbaDog56]

During a recent internal audit, the audit team saw a chart in the stockroom that was being used to track ongoing accuracy of kit pulls. Aside from the dismal numbers reporting an average of 70% accuracy, they questioned the stockroom manager if there were targets established or any documented corrective actions. When he reported that neither existed, they documented it as a finding under 8.5.1:

8.5.1 Continual Improvement
The organization shall continually improve the effectiveness of the quality management system through the use of the quality policy, quality objectives, audit results, analysis of data, corrective and preventive actions and management review.

As the quality manager, I have final review and approval responsibility for audit reports prior to submittal to management. In discussing with the audit team, I feel that this is an observation rather than a finding. The poor numbers do need addressing, and it will be reported regardless of classification. However, this metric is being tracked only within the confines of the stockroom group for their use. No reporting is being done outside of the chart noted during the audit, and no target has been established.

Your inputs are appreciated....

 

[silentrunning]

Re: Audit Question - 8.5.1 Continual Improvement

I?m afraid I would have to come down on the side of the audit team. First, if a metric is important enough to record it is important enough to flow up to management for evaluation. Also, to be effective a metric must have a target. Section 8.4 pretty well spells out the importance of data analysis. I wonder if the 70% success rate played into their findings. This is a failure rate in almost anyone?s book.

 

[PaulJSmith]

Re: Audit Question - 8.5.1 Continual Improvement

Yes, it is indeed a dismal number. However, I agree with the OP that this is not necessarily a CI failure. If it's a widely known issue and still not addressed, then it may very well be. But it sounds to me like the stockroom manager is in the process of finding out just how bad it is. Just because they're tracking something doesn't mean it should automatically have a CI target goal and improvement process simultaneously in place. You first have to find the problem (and it's scope) before you can assign a potential solution.

It's good that the auditors saw this and questioned it. I just don't agree with conclusion to which they jumped.

 

[Jim Wynne]

Re: Audit Question - 8.5.1 Continual Improvement

I?m afraid I would have to come down on the side of the audit team. First, if a metric is important enough to record it is important enough to flow up to management for evaluation. Also, to be effective a metric must have a target. Section 8.4 pretty well spells out the importance of data analysis. I wonder if the 70% success rate played into their findings. This is a failure rate in almost anyone?s book.

It would be good to know the purpose of the tracking. We're not necessarily seeing a "metric" per se; it might be that information is being gathered in order to evaluate the current state of things. If that's the case, the next step would be to determine whether or not the current state is stable and acceptable and what it would take to change it. At that point a real objective could be set, if necessary.

I also question the need to cite the standard in an internal audit. That's often a sign of a deficiency in the QMS. In the end it's a question of whether or not there's actually a problem.

 

[AndyN]

Re: Audit Question - 8.5.1 Continual Improvement

It seems to me that the issue lies mainly with the internal audit process. Firstly, making it an "NC" vs "Obs" is going to cause you all kinds of problems in the future and I'd recommend that you stop doing it. Secondly, citing the standard isn't really the right thing for internal auditors - hence the described problem.

Your internal auditors should have done some research of the stockroom, its processes, documentation and goals/objectives. It's far easier, when discovering this type of clear performance issue, to make a case for action. There's clearly further investigation needed here, if they are tracking something, know it's off where it really should be (do you have such a metric established?) and that someone hasn't even attempted to document any actions.

Rather than waste time debating the grading and which element of the standard is applicable, use the opportunity to go back and see what's going on here!

 

[John Broomfield]

Re: Audit Question - 8.5.1 Continual Improvement

During a recent internal audit, the audit team saw a chart in the stockroom that was being used to track ongoing accuracy of kit pulls. Aside from the dismal numbers reporting an average of 70% accuracy, they questioned the stockroom manager if there were targets established or any documented corrective actions. When he reported that neither existed, they documented it as a finding under 8.5.1:

8.5.1 Continual Improvement
The organization shall continually improve the effectiveness of the quality management system through the use of the quality policy, quality objectives, audit results, analysis of data, corrective and preventive actions and management review.

As the quality manager, I have final review and approval responsibility for audit reports prior to submittal to management. In discussing with the audit team, I feel that this is an observation rather than a finding. The poor numbers do need addressing, and it will be reported regardless of classification. However, this metric is being tracked only within the confines of the stockroom group for their use. No reporting is being done outside of the chart noted during the audit, and no target has been established.

Your inputs are appreciated....

BubbaDog,

Failure to establish a target for kit pulls? Or failure to establish criteria for improving performance?

If you take a look at clause 4.1c you will see that ISO 9001 requires criteria for effective operation of the management system's processes including its process for managing continual improvement.

It looks to me that you have. 4.1c nonconformity.

John

 

[AndyN]

I feel your answer is too simplistic, John! You are correct that there appears to have been a lack of an established set of criteria for effective process control etc.

However, that said, the auditors appear to have "stumbled" on this chart being used and the tracking, etc! Why? If management haven't established correct criteria, it wouldn't have taken an internal audit team to discover this. They shouldn't even have spent their time starting out...

 

[John Broomfield]

I feel your answer is too simplistic, John! You are correct that there appears to have been a lack of an established set of criteria for effective process control etc.

However, that said, the auditors appear to have "stumbled" on this chart being used and the tracking, etc! Why? If management haven't established correct criteria, it wouldn't have taken an internal audit team to discover this. They shouldn't even have spent their time starting out...

Andy,

Yes, we are missing a vital bit of evidence.

Does the manager want to improve the accuracy of kit pulling or not?

John

 

[Golfman25]

So the stock room manager has some personal initiative and develops a "metric" he wants to look at. Then some QA auditor comes in, sees a chart on the wall, and kicks him in the butt because he doesn't have a "target" or corrective actions. And then we wonder why there is no buy in from the organization?


The "target" is obvious -- 100%. So what is the trend? Maybe he was at 50% a few months ago. If the trend is going up, he may be doing a lot of little things to make improvements. Why is it necessary to have "documented" corrective actions?

At best, it is an opportunity, not a finding. If you make it a "finding" your "punishing" him because he is not doing in "right," according to you (which is arguable). Now he is on the defensive. How different would it be if someone from QA or Management (probably with a little more knowledge about metrics) came to the stock room manager and said "looks like you have an interesting metric, do you need any help with it."

 

[BubbaDog56]

Thanks for the inputs, valuable and appreciated. I know our audit, classification, and reporting process needs improvement, which is an exercise in itself.

Due to a customer requirement, we were forced to put the cart before the horse and start internal audits without being able to properly lay the groundwork first. The current structure is oriented towards negative findings, observations and OFIs only, and citing the standard for findings to allow understanding by the auditee and management. It's oriented towards finding faults primarily, with continuous improvement secondary. Unfortunately, we have no shortage of faults to find and address just to get our house in order from the nuts and bolts functional side of the standard.

But back to the matter at hand. I'm doing further research to determine exactly the origin and purpose of the chart, along with how long data has been collected and if improvements have been noted.

We are trying to not become the 'quality cops', and have done a pretty good job so far in getting buy-in from the audited groups as this being a mechanism for improvement rather than 'I'm telling Dad on you and you're in trouble'. This again is some of my reasoning for seeing this as an observation/OFI rather than a finding.

Thanks again, keep the comments coming.