Any comments on this folks?

----------snippo----------

From: ISO Standards Discussion
Date: Thu, 3 Feb 2000 07:37:10 -0600
Subject: Re: Management Review Record /El-Homsi/Lambert/Meron

From: Emanuel Meron

I do not think that an auditor, either from a second party (the customer) or third party (the registrar), may ask to see the actual minutes of a management review. If the review was worth the time, it probably dealt with sensitive items and generated information that, if leaked out, may give competitors an advantage. This is the kind of information that every company likes to guard closely and you, as an auditor, should honor this. I do however agree that evidence should be provided showing that reviews actually took place.

The FDA recognizes this situation and only asks for a management declaration stating that reviews were held in accordance with the requirements of the quality system regulation (QSR). Their auditors are not allowed to inspect the actual minutes.

If you ever find yourself on the receiving end, I suggest you refuse to divulge the actual review items, data, decisions, etc., Just give the auditor the dates of the reviews and the names or functions of the participants.

By the way - this is what started the thread....

-------------snippo----------

From: ISO Standards Discussion
Date: Thu, 3 Feb 2000 07:12:52 -0600
Subject: Re: Management Review Record /El-Homsi/Kozenko

> Heather writes:
>
> I then asked what had been shown to the auditor(s) during
> the audit, and he said the same things that had been shown
> to me. (In addition to the agenda, there are some active teams
> pursuing corrective action on key measures such as Corrective
> Actions, Non-Conforming Mat'l, etc...) which meet weekly or
> monthly.

Were there records available for these meetings?

> He replied that they had been with the registrar for some time
> and they trusted him when he said that all of the required
> people had met and that they had decided current activities
> were adequate for their quality system. The last formal meeting
> minutes were from 1995. After that, they just had the agendas.


This ought to put the cat amongst the pigeons <g>

The Registrar deserves to know that its Lead Auditor(s) performed in this manner since 1995, and your company's concerns with the Registrar's findings (or lack thereof) with respect to "Records" compliance (especially records of 4.1 Management Review records compliance). Let the Registrar go on record after investigating the apparent validity of your firm's concerns.

While what you described may be limited to a Records nonconformity and not necessarily a systemic collapse, the fact remains that aggravated Quality Managers are such bad conversationalists -- your long discussion with one does not count under 4.18 Training, you know? You can save a lot of time with the words, "Objective evidence not found."

Finally, what other suppliers does your firm have, who certify under this Registrar's banner? Maybe revise your firm's surveillance audits, consummate with risk, etc.

David Kozenko

[This message has been edited by Marc Smith (edited 15 February 2000).]

In reading this over, one thought I have is: If you can sign an affadavit swearing you did the management review, why couldn't you do the same with Internal Audits? Undoubtedly there's plenty of sensitive, proprietary info in audit findings...

Marc:

I agree with you regarding the risk of extreme positions saying something like " I am the Director, I swear on the Holy Bible that all my Quality System is OK, then I don't need to show you any evidence"

The auditors need to see evidence that a QS is in place, but I think that in such cases like 4.1.4 (Bussiness Plan) and 4.1.3 Management Review when they have to audit the existance of documents of these activities, that demonstrate that they are "live" in place, would be enough a "bird sight" quick review "from a distance" of papers. I think is not proper to allow the auditors read details of plans, costs or new technologies, sensitive and strict property of company.

Qualiman

The whole process is intrusive by its very nature. My initial objection to QS-9000 was the business plan requirement. It's pretty standard for a company not to show the auditor the business plan contents but I have yet to see a company keep an auditor from seeing actual management review records. The question becomes where is that 'thin gray line' of what can and cannot be seen by an auditor.

Audits I have witnessed were always precluded with a long speil by the lead auditor about confidentiality and such - supposedly they've got 'Top Secret' ;) clearances.

From: ISO Standards Discussion
Date: Thu, 10 Feb 2000 07:48:32 -0600
Subject: Re: Management Review Record /El-Homsi/Humphries

Heather,

> As part of the audit, we asked to see the Meeting Minutes from
> their last Management Review Meeting, which was held in the
> Spring of 1999. We were shown an agenda which included a
> summary of audits and CAR's for the last few years in table form,
> and some questions for discussion around the effectiveness of the
> quality system. The Quality Manager stated that he handed this out
> before the meeting to the attendees. When I pressed further for
> documentation that showed who had attended the meeting, and what
> the group consensus was to the questions, he was unable to present
> anything. I then asked what had been shown to the auditor(s) during
> the audit, and he said the same things that had been shown to me. (In
> addition to the agenda, there are some active teams pursuing corrective
> action on key measures such as Corrective Actions, Non-Conforming Mat'l,
> etc...) which meet weekly or monthly. He replied that they had been with
> the registrar for some time and they trusted him when he said that all of
> the required people had met and that they had decided current activities
> were adequate for their quality system. The last formal meeting minutes
> were from 1995. After that, they just had the agendas.
>
> As he was agitated with my line of questions, I reviewed ISO section
> 4.1.3 Management Review with him, and showed him the requirement
> for "records of such reviews shall be maintained". We had a long
> discussion around records vs. forms and objective evidence. We also
> discussed that the intent of Periodic Management reviews was not to
> deal with the day to day firefighting, but to take a step back and look at
> the "Big Picture" to see if the quality system in it's entirety was
> effective.

In principle you're correct: I generally require that Management Review
meetings be minuted, so that agreed actions can be followed up. As to
attendance, it is again normal to specify who should attend the meetings.

However, in practical terms, neither of these solutions is itself
specified: they are merely effective and common ways of meeting the
requirements.

A company could quite easily specify that any action decided through
management review will be monitored through the corrective action system,
and a standard agenda will be used. Under such conditions, the evidence of
the meeting (the records) will be the corrective actions that are
initiated as a consequence of the meeting, presumably on the day of the
meeting. There also won't be an individual agenda for the meeting, merely
a proforma.

It is also not stated what should go in the records, so attendance is
optional.

That having been said, in your place I would be quite suspicious that no
effective management review is taking place. Perhaps you should look for
an incident that SHOULD have triggered management review: major expansion,
restructuring, new products, etc, and ask for what records there are of
management review at those times.

Best Regards
Edwin Humphries

From: ISO Standards Discussion
Date: Thu, 17 Feb 2000 07:06:48 -0600
Subject: Re: Management Review Record/../Blair/Arter

From: Dennis Arter

>I'd have to disagree with this statement. 4.1.3
>specifically calls for Quality Records (4.16)... "Records
>of such reviews shall be maintained".

True. The standard also requires the using organization (and NOT the auditor) to define a) what their records are, b) where they will be kept, and c) how long they will be kept. Perhaps you are confusing a document with a record. I consider an agenda to be a document; although, certainly NOT a "controlled" document.

>In plainer language:
>Any company having a Management meeting without preparing
>a written agenda is wasting everyone's time. Any company

Opinion

>having a management review without attendance by top
>management is wasting everyone's time. Any company having

Opinion

>a management review with no action items and/or comments on
>the status and readiness of the quality system is wasting
>everyone's time. Records provide this proof, or, as the

Opinion

>auditor would say... "If you didn't write it down, it never
>happened"

I believe your auditor may have forgotten about the other forms of objective evidence. Written records are important, but they are not the only bits of evidence.

Dennis R. Arter

From: ISO Standards Discussion
Date: Thu, 17 Feb 2000 07:12:09 -0600
Subject: Re: Management Review Record /../Blair/Humphries/Blair

> I agree with everything you say, except for the "in plainer language".
> What you go on to say is correct in every sense, but is an interpretation
> of the Standard, not the meaning of it's requirement

O.K., lets use the actual words in the standard (and yes, I know I'm violating
Copyright Laws)

"...management with executive responsibility shall review the quality system"
Where is the objective evidence of this without an attendance list?
"at defined intervals sufficient to ensure its continued suitability and
effectiveness.."

Where is the objective evidence of this without proof that the required
elements are being addressed (4.14, 4.17) as an agenda coupled with either
action items to address relevant issues and/or a statement summarizing the
effectiveness of the current system?

If all you're saying is that the record doesn't have to be a paper record,
then I'm o.k. with that....I would pass a video-taped management review
conducted as a teleconference in a heartbeat once I had viewed the tape and
determined its completeness and effectiveness.

What I'm getting so far is that it is theoretically possible to register
someone if:

1. I have a quality policy that says I make Crap and I have clearly communicated this to everyone.
2. My management review consists of a discussion while passing in the hall, followed by one or two memos.
3, I select suppliers based on whoever has the lowest price.
4. Hiring/Training consists of holding a mirror in front of an employee's face and looking for fog.
......etc, etc.

Yes, I suppose it is possible theoretically, but please, please, say it ain't so, Shoeless Joe....

Grant Blair

From: ISO Standards Discussion
Date: Thu, 24 Feb 2000 07:45:54 -0600
Subject: Re: Management Review Record /../Meron/Pb/Hankwitz

> Subject: Re: Management Review Record /../Lambert/Meron/Pb
>
> > Emanuel stated:
> <snip>
> > I do not think that an auditor, either from a second party (the
> > customer) or third party (the registrar), may ask to see the
> > actual minutes of a management review. If the review was worth
> > the time, it probably dealt with sensitive items and generated
> > information that, if leaked out, may give competitors an advantage.
> > This is the kind of information that every company likes to guard
> > closely and you, as an auditor, should honor this. I do however
> > agree that evidence should be provided showing that reviews
> > actually took place.
> <snip>
>
> Hereabouts (India), it is the practice for auditors to pore through
> every line of the management review minutes to find omissions. I
> am looking forward to avoiding the disclosure of the minutes!
>
> Dhanish

I must be missing something in this dialog. What is everyone putting in their meeting minutes? Why is "sensitive" information being included? I don't see any requirement mandating this type of information, so why are you doing it?

Our company is (very) privately owned, so I only include information relating to the status and effective implementation of the quality system in the minutes, as stated as the primary purpose of the meeting. I then proudly provide our third party auditor with a copy of the latest minutes at each audit opening meeting. If the auditor isn't given insight into our strengths and weaknesses, how can he be expected to efficiently help us locate potential improvement areas? Unless, of course, you're not really interested in having him find them. So, then why bother having an ISO registered quality system in the first place?

I don't know how our competition knowing that our corrective action on-time closure rate increased from 80% to 95% over the past year, or that our customer dissatisfaction rate improved by 98.7% since our initial registration could possibly be a competitive advantage. Besides, if you can't trust your registrar/auditor to keep this information confidential, you need to find someone you CAN trust.

Perhaps someone could provide me with some further insight on this by providing some (bogus) examples of required "sensitive" information they wouldn't want to disclose in the minutes.

John Hankwitz

From: ISO Standards Discussion
Date: Fri, 25 Feb 2000 16:04:30 -0600
Subject: Re: Management Review Record /../Pb/Hankwitz/El-Homsi

Having started this thread, I want to thank everyone for their input. Initially, I just wanted to see if my concerns over the lack of objective evidence (records - which may be in the form of meeting minutes, action item registers, or other method that works for the company) that a management review had taken place. I believe that the group has overall supported that that is a requirement of ISO.

Now, to respond to John's request that "Perhaps someone could provide me with some further insight on this by providing some (bogus) examples of required "sensitive" information they wouldn't want to disclose in the minutes."

I often see companies that have their quality system management review as part of a larger "business" meeting. In those cases they may include details around 1 - 5 year plans, finacial info, Marketing strategies, Research directions, HSE concerns, unions, etc, etc, that they wouldn't want competitors to see.

The example I used to start the thread involved a Supplier Quality Assessment of an ISO certified company. During a supplier audit a company may (rightly so) refuse to share confidential info if there is not a non-disclosure agreement in place. Especially if they are being pitted against other suppliers for the same commodities.

I agree with John that if a company is certified in the true spirit of ISO, there needs to be trust between the company and its registrar, and if there isn't, find someone else!

Another thread of this discussion pulled in FDA requirements. I'm out of date on how FDA audits are performed currently, but a company I used to work for we made it a practice to be very open with the FDA auditors, and would willing share (NOT volunteer :-) info. We believed that this would demonstrate the company's true commitment to doing the right things, and we were proud of the improvements we had made, and how we were addressing customer complaints. This approach was very successful for us.

I was looking through this old discussion and started wondering. Has anyone here run into anything like this lately?

I was looking through this old discussion and started wondering. Has anyone here run into anything like this lately?Nope.

On the recieving end, we show what we have, trusting the auditor to shut up about any sensitive information.

When auditing someone else, they invariably show us the minutes, trusting us to shut up about any sensitive information.

Not a problem...

/Claes

Not a problem at this end, either. There is that wonderful concept about confidentiality regarding the audit's results. If sensitive information, such as the succession plan, is to be shown during the External Audit, I simply leave the room. During Internal Audits, I have made it the rule that performance appraisals can not be viewed by us - it is up to HR to demonstrate that an effective process has been established.

This is my take as an auditor. I have signed a Confidentiality Agreement prior to the audit. With that being said, I also ask them prior to the Management Review if there is any sensitive information they may not want me to see. If they say yes, I will review their procedure for Mgmt Review (99.9% of companies do have one) and see what it states as far as records, attendance sheets, action/follow up actions, agenda’s etc.. that I can review to support the fact that Mgmt review are taking place. I have not been told yet that I can’t review the minutes to the meeting.

Now, this is what we do where I work. We will show the Mgmt Review Records to our Registrar and Customers but not to the FDA. For our Registrar and Customers if there is any sensitive material we do not want them to see we will sanitize (black it out) a copy of the minutes prior to handing them over and explain why we did it, it has yet to cause any problems. The only thing we show the FDA is the agenda for the meeting to show what issues/systems were discussed and the sign in sheet to show who was in attendance.

Just a thought look at it from the auditors prospective, if you refuse to show any supporting documentation the review is being completed (agendas, sign in sheets, minutes (sanitized)), it gives the impression you are trying to hide something or that you are not doing what you say you are doing.

I have run into this from time to time with internal auditors. It goes along the same lines as having an internal auditor looking over the business plan. Some organizations do not want their employees to know everything. The old mushroom treatment. :nope:

During our audits, we show all the meeting minutes that are for general viewing by all managers. When minutes contain confidential information, a special password is attached in order to open the document. Only individuals having the password may view the document (Generally the president and one or two managers). Those types of minutes are very few in-between. I have never had a problem with this. The general minutes not password required to open cover more than enough action by management to verify the activity. We have also electronically sent copies of minutes to customers (upon request) to audit our systems. Generally, any confidential information is removed. Well, we all make mistakes and adding the password to open the document helps to ensure they cannot view it if it was sent accidentally. Again, the minutes of confidential nature are password protected to prevent viewing by anyone without the password. I should add that our system is on a company wide intra-net where everyone is encouraged to view the documents. I should also note that even confidential information is limited to what is documented. Specific secrets may be discussed by never documented in such a form as to give the secret away.

In addition, of course, our minutes include the attendees, as it should.

Even our internal auditors must view management review meeting minutes to verify conformance.

Our minutes are just minutes and give an overview or brief descriptions of information or activities, not exacting detail.

I would question our or any auditor for that matter, if they did not want to view meeting minutes and a business plan to verify activity.

The same goes for the Business Plan. We may not post our business plan for everyone in the company to view, but we do communicate information from the business plan that everyone within the company needs to know. Our auditor has viewed our business plan and our methods of communicating it. He views what he needs to and has not viewed exacting details but rather views data such as the executive summary of the business plan. That executive summary is verified by the communication and participation throughout the company.

Just MHO or 2 cents worth.

Cindy

If our internal or Regitrar's auditors ask to see our management review and/or minutes, we show them. They have signed a confidentiality agreement.
I do not show these things to customers unless it is a very quick flash of "here are the reports, here are the minutes, there may be things in these files that are sensitive for our customers, and I am sure you can appreciate the fact that we do not show your competitors information about your company." I've never had anybody push on this subject. The same goes for business plans. They see our goals, that is enough.

To take it a step further, any customer can have a copy of our quality manual, but I do not give copies of our procedures, work instructions or forms to our customers either. They can view them while they are here, but the documents never leave our premises.

During APQP activities, our customers regularly ask for copies of our procedures. How do you get around this?

During APQP activities, our customers regularly ask for copies of our procedures. How do you get around this?I'm not sure what your objection is.
Do you fear giving away trade secrets on how you perform a particular operation?
If so, compartmentalize those in a work instruction which is NOT available for outsiders to view.

Rarely have I ever seen an outsider (registrar or customer) want to delve into work instructions, other than to verify they refer to the latest revision of a product or process.

Cindy,

I just tell anyone who asks for copies of procedures or work instructions that they are welcome to see them while they are on site, but our policy is that copies are not sent off-site. It helps that we have our document control program set up to view all documents, but to print none. So, instead of using the web browser to show customers the procedures, we show them using the document control database.

To take it a step further, any customer can have a copy of our quality manual, but I do not give copies of our procedures, work instructions or forms to our customers either. They can view them while they are here, but the documents never leave our premises.

Just out of curiosity why do you not let customers have copies of your procedures? Unless the procedures are specific to another client or have some trade secrets??

We will give our customers a copy of any procedure they want as long as the above issues do not come in to play.

Just out of curiosity why do you not let customers have copies of your procedures? Unless the procedures are specific to another client or have some trade secrets??

We will give our customers a copy of any procedure they want as long as the above issues do not come in to play.This is pretty much my point of view.
I consciously structured my documents to segregate trade secrets and customer-confidential data from general procedures and operations.

In fact, we frequently use our Quality Manual as a Marketing tool with prospective customers to convince them we have a viable system.

Thus said, I realize there are many ignorant functionaries working for customers who want to amass piles of documents to demonstrate to their own bosses how hard they are working. In this quest for quantity and volume, they frequently ask for stuff to which they have no right or reason to read or possess. In such cases, you have to have your boss (or top boss) deal with someone at a higher level than the obtrusive functionary to shut down the intrusion.

Just as many companies keep pay, employee health, and other confidential information from general circulation, so, too, should they segregate any other sensitive material on a "need-to-know" basis. Customers understand this (usually.) If they don't, try asking them for copies of their pay stubs. :tg:

I would question our or any auditor for that matter, if they did not want to view meeting minutes and a business plan to verify activity.

Cindy

Reminds me of a story an auditor once told me. Was conducting a QS audit and asked to see the business plan. Auditee held it up and showed him the cover and table of contents, but would not let him touch it. Auditor said he did not want to see detail, but just had to verify it contained required topics information. After a lot of arguing that "you can't audit the business plan", the auditee eventually had to back down and show it to him - turned out to be a cover, table of contents and dozens of blank pieces of paper. The audit was terminated immediately. Don't know who the company was, or even if the story is true, but I can see companies trying it.

Tom

Reminds me of a story an auditor once told me. Was conducting a QS audit and asked to see the business plan. Auditee held it up and showed him the cover and table of contents, but would not let him touch it. Auditor said he did not want to see detail, but just had to verify it contained required topics information. After a lot of arguing that "you can't audit the business plan", the auditee eventually had to back down and show it to him - turned out to be a cover, table of contents and dozens of blank pieces of paper. The audit was terminated immediately. Don't know who the company was, or even if the story is true, but I can see companies trying it.

TomSadly, whether true or apocryphal, the anecdote typifies what happens when a supplier is extorted by an existing or prospective customer to achieve registration to a Standard. Often, the response is, "What's the MINIMUM we can get away with?"

Just out of curiosity why do you not let customers have copies of your procedures? Unless the procedures are specific to another client or have some trade secrets??
They can see them here, we don't stop them from looking at them...but in our industry there are customers that will take that information and share it with our competitors. We don't really feel that we need to share what we are doing with everyone. If they issue a nonconformance in a customer audit, or a complaint, we will share procedures (or revisions we make), possibly taking out any "sensitive" information. I've just always made it a policy not to make copies to go off-site. If you give this stuff to one customer, then pretty soon you are spending all your time making copies for all your customers...as stated in a previous post, some people just collect paper to take back to their boss. Our sales team can't even seem to be able to print out a .pdf file with our certificate on it for the customer, I can't imagine how much time I'd spend sending out copies of our procedures. lol

BTW...we even share information with some of our competitors (best marking trips) but we don't ask for their procedures and don't expect that they ask for ours.