Management Review Analysis of Internal Audit Findings

During Management Review Meetings, as the MR of my firm, I generally present an analysis of the last internal audit findings (in ppt).

I have done this clause-wise (no. of findings against each clause of the ISO 9001 standard), against the organization procedures (number of findings against each procedure), repeated NCs in key areas (across multiple internal audits), Pareto analysis of findings against key areas, etc. How do you do this to make it more effective?

It would be great to get inputs from this forum on any other ways you have done and presented audit findings in your organization, which you found more effective to make managers act.

By the way I am working in an IT product and services company.

Thanks in advance for your valuable inputs.

Re: Analysis of audit findings

Aside from audit findings/reults, you may include the following input agenda for the management review:
- customer feedback
- process performance and product conformity
- follow up actions from previous reviews
- status of corrective and preventive actions
- changes that could affect quality management system
- recommendations for improvement

Re: Analysis of audit findings

Hi,

We do that of course since we cover all the standard's requirements in the MRM. Maybe I didn't phrase it clearly, but my question was related to showing the analysis of internal audit findings during the MRM in a way that drives the point home better. The numbers generally are more than could be discussed individually, so we discuss only major findings individually. For the rest I try to show trends at the least.

Regards,
Deepu

Re: Analysis of audit findings

Deepu said:

Hi,

We do that of course since we cover all the standard's requirements in the MRM. Maybe I didn't phrase it clearly, but my question was related to showing the analysis of internal audit findings during the MRM in a way that drives the point home better. The numbers generally are more than could be discussed individually, so we discuss only major findings individually. For the rest I try to show trends at the least.

Regards,
Deepu

Click to expand...

Deepu,

You are right, presenting data to top management is not a good idea.

It is for you to analyze the data before the meeting and then to present your summary, conclusions and recommendations.

Your recommendations may include:

1. Your next period's internal audit schedule
2. The type of person who should be selected for auditor training
3. The need for leaders to encourage improvement before the audits
4. The need for better process monitoring by operators, supervisors and managers (instead of waiting for the auditor)
5. System improvement projects requiring investment
6. New system and process objectives

...and so on.

Make sure you can back up your recommendations with facts from your Management System Performance report.

John

Re: Analysis of audit findings

It's worth revisiting the reasons for doing internal audits, before you overwhelm management with even those technical things John is talking about!

Get beyond the 'ISO says so' aspects and answer - Why do we do audits? What should we do with the results? Think of it this way:

Each Process owner should - IMHO - be discussing their particular quality objectives and the performance of their process in achieving those objectives. What's missing from this is the Management having any clue if their people actually followed the documented management system. The internal audit results should validate this 'yes we saw the people following (or using) the documented system' (or not, as the case may be)...

When management have those two facts, they can decide to take action...

It does mean that you may have to stop doing all of the review presentations and get the process owners to do some work instead of sleeping through Management Review...

Re: Analysis of audit findings

AndyN said:

It's worth revisiting the reasons for doing internal audits, before you overwhelm management with even those technical things John is talking about!

Get beyond the 'ISO says so' aspects and answer - Why do we do audits? What should we do with the results? Think of it this way:

Each Process owner should - IMHO - be discussing their particular quality objectives and the performance of their process in achieving those objectives. What's missing from this is the Management having any clue if their people actually followed the documented management system. The internal audit results should validate this 'yes we saw the people following (or using) the documented system' (or not, as the case may be)...

When management have those two facts, they can decide to take action...

It does mean that you may have to stop doing all of the review presentations and get the process owners to do some work instead of sleeping through Management Review...

Click to expand...

Andy,

Involving the Process Owners in Management Review is a wonderful idea.

They tend to be subject matter experts irrespective of their rank.

Management Review could be part of the process called "Investing in Improvement" so it addresses:

1. Mission and Strategy
2. Renewing objectives
3. Process performance
4. System performance
5. Overdue actions
6. Recommended investments

This agenda and well-prepared attendees will reduce the time and inclination to sleep!

BTW, we could do with a debate on the power of Departmental (Functional) Managers versus Process Owners/Managers.

John

Re: Analysis of audit findings

John Broomfield said:

Andy,

Involving the Process Owners in Management Review is a wonderful idea.

They tend to be subject matter experts irrespective of their rank.

Management Review could be part of the process called "Investing in Improvement" so it addresses:

1. Mission and Strategy
2. Renewing objectives
3. Process performance
4. System performance
5. Overdue actions
6. Recommended investments

This agenda and well-prepared attendees will reduce the time and inclination to sleep!

BTW, we could do with a debate on the power of Departmental (Functional) Managers versus Process Owners/Managers.

John

Click to expand...

Thanks for your gracious comments, John, but I don't see it as a wonderful idea, it's what Management Review is all about! For too long, too many Management Reps have used the ISO requirements as their agenda, without knowing what the thing should "look like". They think it's 'all about them', producing ISO-speak details to drone on about - because they feel like they have to do that. Noting could be further from the truth. In a previous life, clients were encouraged to do it the way I described and found it to be the most productive review they'd ever done...

Deepu said:

During Management Review Meetings, as the MR of my firm, I generally present an analysis of the last internal audit findings (in ppt).

I have done this clause-wise (no. of findings against each clause of the ISO 9001 standard), against the organization procedures (number of findings against each procedure), repeated NCs in key areas (across multiple internal audits), Pareto analysis of findings against key areas, etc. How do you do this to make it more effective?

It would be great to get inputs from this forum on any other ways you have done and presented audit findings in your organization, which you found more effective to make managers act.

By the way I am working in an IT product and services company.

Thanks in advance for your valuable inputs.

Click to expand...

Do you send out the agenda or the presentation ahead of time? You could have slides with "Manager ABC will present his/her RCA and C/M's". Of course, you would want them to be prepared for this so they aren't put on the spot. It may help with overall involvement in the review as well.

I'm a little jealous...my last Management Review (which was also my first as the MR) had a lot of discussion about the shallowness and lateness of internal audits. If we could just get the auditors to consistently do thorough audits, on time...

Deepu said:

During Management Review Meetings, as the MR of my firm, I generally present an analysis of the last internal audit findings (in ppt).

I have done this clause-wise (no. of findings against each clause of the ISO 9001 standard), against the organization procedures (number of findings against each procedure), repeated NCs in key areas (across multiple internal audits), Pareto analysis of findings against key areas, etc. How do you do this to make it more effective?

It would be great to get inputs from this forum on any other ways you have done and presented audit findings in your organization, which you found more effective to make managers act.

By the way I am working in an IT product and services company.

Thanks in advance for your valuable inputs.

Click to expand...

Although I have no objection to doing what you do, it is way beyond the requirements of the standard.

I would ask if there is sufficient benefit for the level of effort.

The requirement is simply to discuss the results of audits. I might add, that it is all audits, not just internal audits. If you are not discussing CB and customer audit results you are not meeting the requirement.

I may have a slightly different perspective about management review, and I'm sure that perspective drives my approach. I think that management review is comprised of two main components. Determining the state of the quality management system and then determining where you want to go with that insight.

Determining the state of the system is accomplished by discussing the items listed in element 5.6.2 a-f.

results of audits
customer feedback
process performance and product conformity
status of preventive and corrective actions
follow-up actions from previous management reviews
changes that could affect the quality management system

More can be discussed if appropriate, after all, 5.6.1 says that you need to discuss the quality policy and quality objectives to confirm they are still relevant. Three out of four topics from analysis of data (8.4) are included in the above list (customer feedback, process performance and product conformity), so discussing the results of all the KPI are appropriate.

After good discussion, the awareness level of all participants is raised concerning the state of the system.

At that point you are ready for 5.6.2 g recommendations for improvement.

These improvements are expressed as shown in 5.6.3 a-b

improvement of the effectiveness of the quality system
improvement of product related to customer requirements

These improvements can be expressed as action items -- action items to be discussed at the next management review (5.6.2 e).

There also needs to be consideration for resources needed to accomplish the improvements. (5.6.3 c).

I see more benefit to this approach than I do to an overly detailed report on the internal audit. Others may have different opinions.