The Value of Turtle Diagrams in Process Auditing

qusys

Trusted Information Resource
#21
Seeing how I started this thread seeking a bit of help and seeing how its got a bit hijacked by some guninely interesting but off topic stuff

I'll repost something thats important to me but got lost in the to and fro.

So; The audit I'm going to do in the next day or say is a compliance audit for the safety department (compliance to our organsaitons requirements, compliance to the clients compliance and legislative compliance)

This is a small contract ($50k) It required a scientist to visit a remote location and observe turtle (no pun intended) egg laying and hatching during a the construction of a small O&G facility on a remote beach.

I have put the following questions together as my first foray into process auditing, and, in an attempt to see how:
  • the project identified the compliance requirements in the three areas
  • what training the project manager has had in complaince requirements
  • the sources the project manager used to identify the compliance requirements from
  • how did they apply the compliance requirements to the project
  • who reviewed the complaince requirements and:
  • what happen if complaince requirements are not met;
Please review and comment on adequacy of these questions:
  • Tell me what you do (explain what this project is about)
  • How did you go about identifying what the compliance requirements for this contact?
  • Is there a document that tells you how to identify what the compliance requirements for a project are?
  • How do you know what legislation applies and what does not apply to the project?
  • how did you make sure that all applicable safety legislation was complied with?
  • How did you identify the clients requirements?
  • Do you have a list of them?
  • How do you identify what our organsation's requirements are for this contract?
  • Is there a document that tells you what our requirements are for a contract?
  • How do you go about implementing the our requirements?
  • What evidence have you got that these requirements have been implemented?
  • what evidence did you collect to ensure all requirements were met?
  • how did you establish wether or not all compliance requirements were met?
  • what did you do when a requirement was not met?
  • Are you required to report compliance issues as part of the project reporting process?
thanks
__________________
The Veteran Novice
At first glance:
Nothing about KPI?
What do they do in case of an issue? Is there any reaction mechanism?
What are the competences and skill required? Do you have some training for your personnel or what else?
Do you have records? How do you manage them?
 
Elsmar Forum Sponsor

RoxaneB

Super Moderator
Super Moderator
#22
So; The audit I'm going to do in the next day or say is a compliance audit for the safety department (compliance to our organsaitons requirements, compliance to the clients compliance and legislative compliance)
Without meaning to sound to nitpicky...oh, who am I trying to kid???...your questions are more about conformance to requirements than compliance. Your asking them about what they do to identify requirements...and adhere to there identified requirements. Are you going in to ascertain whether or not they have actually identified all requirements and are following all applicable legislation, etc.?

Example:

Conformance - My father looks over at my speedometer to verify that I am not speeding.

Compliance - Police officer gives me a ticket if I don't follow the rules of the road.
 

Randy

Super Moderator
#23
Well, I'll be nice

I have found that Turtles start being a primary concern and the process of actually looking at/for evidence of whatever becomes secondary

Fill out the legs, head, body and fanny but then forget the reason as to why your doing so to begin with and never get to your objective...Pretty picture, but no substance

A Turtle ain't nothing but a tool and like any tool it can be useful or harmful.
 
K

kgott

#24
Without meaning to sound to nitpicky...oh, who am I trying to kid???...your questions are more about conformance to requirements than compliance.

Example:

Conformance - My father looks over at my speedometer to verify that I am not speeding.

Compliance - Police officer gives me a ticket if I don't follow the rules of the road.
Thanks RC, so could I please ask, what questions I perhaps should ask? - k
 
#25
Kgott: I've often thought about audit questions as things which need to be answered, not questions you'd ask...

What I mean by that is, you don't ask the (blunt) question "Can you tell me what you do?" or something similar, you get them to respond to "Explain to me what your role in this project is"... It's not a question, it's getting them to speak about their roles and responsibilities. Let's not forget the maxim about auditors (like good prosecuting lawyers) know the answers to their questions - and you should have done your home work beforehand to know what the QMS says about the topic.

Novice auditors have a habit of getting caught up in 'asking the right questions' like there's some magic to it. If you know the 'story' before hand, your job is to get the auditee to describe the subject in their 'comfort zone', while you listen (remember, audit comes from the Latin, to listen) and connect the dots between what you know and what you're told.
 
J

JaneB

#26
I've often thought about audit questions as things which need to be answered, not questions you'd ask...

What I mean by that is, you don't ask the (blunt) question "Can you tell me what you do?" or something similar, you get them to respond to "Explain to me what your role in this project is"... It's not a question, it's getting them to speak about their roles and responsibilities. Let's not forget the maxim about auditors (like good prosecuting lawyers) know the answers to their questions - and you should have done your home work beforehand to know what the QMS says about the topic.

Novice auditors have a habit of getting caught up in 'asking the right questions' like there's some magic to it. If you know the 'story' before hand, your job is to get the auditee to describe the subject in their 'comfort zone', while you listen (remember, audit comes from the Latin, to listen) and connect the dots between what you know and what you're told.
Excellent answer, Andy.

I fully agree with all you say.

The 'magic' isn't in coming up with the perfect list of questions, at all. The skill is in eliciting the information and the available evidence (or lack of same).

I watched one excellent auditor once as he just wandered around the premises in a very low key fashion: chatted here and there with various people, always putting them at their ease. No one felt stressed or uneasy or 'under the pump' at all. Indeed, if you hadn't been told, you wouldn't have known he was an auditor, doing an audit.

We left those particular premises and returned to the room we were meeting in; at which point he turned to me and said whew, they've got more than a few problems, haven't they? And reeled off an accurate list of all the things he'd observed that they didn't know, couldn't answer, didn't have, didn't understand and weren't complying with. Most impressive.

Most of all, was the big difference between the questions he asked them (low key, nonthreatening, information-eliciting) and the questions he wanted to find answers to.

As Andy said, questions to be answered, but not necessarily questions to be asked directly. And another reason why auditor competence is important, and there is such a need for auditors to learn 'on the job' from other competent auditors.

Andy: I do like the reminder that auditing comes from the word 'to listen'. Every good auditor has taken that one thoroughly on board. (And the corollary alas is true.)
 
Last edited by a moderator:
K

kgott

#27
I watched one excellent auditor once as he just wandered around the premises in a very low key fashion: chatted here and there with various people, always putting them at their ease. No one felt stressed or uneasy or 'under the pump' at all. Indeed, if you hadn't been told, you wouldn't have known he was an auditor, doing an audit.

We left those particular premises and returned to the room we were meeting in; at which point he turned to me and said whew, they've got more than a few problems, haven't they? And reeled off an accurate list of all the things he'd observed that they didn't know, couldn't answer, didn't have, didn't understand and weren't complying with. Most impressive.
Glad you said that JaneB; it reminds me of when I worked in OH&S and I'd go round look at whats happening and how it was happing so perhaps I have some auditing skills after all.

I often ran into a problem though. When discussing issues I had found on the site at a daily or weekly progress meeting, I would often be asked about the identies of those I spoke to by the supervisors etc responsible for the areas concerned. I have always been reluctant to reveal the identity of those I spoke to or the identify of those who approached me with information as I felt I would loose their trust if I revealed their identity. (this is on big resource projects mind you.)

This reluctance contributed to my downfall on one occasion as I was accused of inventing the ''issues" and was thereafter labeled 'a problem.'

With regard to a couple of audit issues I have also run into this problem here.

Have you faced a similar situation when you do audits and if so how do you deal with the matter??

thanks k
 
J

JaneB

#28
Revealing/non-revealing of identities is a whole topic in and of itself. And somewhat different between internal and external audits, also.

What it does do (for me) is highlight the need to stay objective during audit and focus on finding and reporting facts that are supported by evidence.

So, if someone says 'oh, we never bother with doing any JSA's [Job Safety Analysis] on our projects' (for example), I would very likely look for evidence of whether that was so or not. IF I find the evidence, I report it. IF I don't, then I don't! I may write a note for myself/discuss with another auditor to follow it up, but I won't report anything that I can't back up with facts. Eg, " For these 4 operations on Project X, there was a written JSA for one of them, and that was incomplete."

As you've found, if you're reporting unsubstantiated and unattributed comments, this can make things very difficult. I share your issue about not wanting to dob others in (= get people into trouble for you non-Oz speakers) but I also have to fulfil the mandate of my audit. Hence: facts, evidence, objectivity. Anything else is or at least can be danger territory.
 
Last edited by a moderator:
Thread starter Similar threads Forum Replies Date
M Determining a tolerance value for Measuring devices in-house inspection General Measurement Device and Calibration Topics 12
Proud Liberal Improving low p-value Statistical Analysis Tools, Techniques and SPC 13
BeaBea Registrars with VAR (Value Added Reseller) experience Registrars and Notified Bodies 8
qualprod Best approach to get a real value as average? Statistical Analysis Tools, Techniques and SPC 6
F Missing value of Uncertainty B Measurement Uncertainty (MU) 10
F Calibration certificate content - Average Value and True Value ISO 17025 related Discussions 6
S In BAC5300 we have machining mismatch allowed is 3tau, what is the value of tau? Document Control Systems, Procedures, Forms and Templates 1
S Change value of target during a batch Capability, Accuracy and Stability - Processes, Machines, etc. 0
A Acceptance p-value for linearity and bias analysis in minitab results Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 0
A Torque Value for Different types of Fasteners (Socket Head, Button Head, CSK) Manufacturing and Related Processes 2
M IATF 16949 - 8.5.6.1.1 Providing a list of process controls: Does this requirement add value to QMS? IATF 16949 - Automotive Quality Systems Standard 4
qualprod Cycle times, value stream mapping, inventory, average values? Lean in Manufacturing and Service Industries 1
Ashok sunder What are the criteria for setting a target KPI value against a quality objective? Benchmarking 7
S Adding Value prior to signing EASA Form 1 EASA and JAA Aviation Standards and Requirements 5
M Creating a Plant Level Value Stream Map Process Maps, Process Mapping and Turtle Diagrams 1
A IMDS - Changing the Locked Weight Value RoHS, REACH, ELV, IMDS and Restricted Substances 1
T Value Chain Map for Food Services Industry Service Industry Specific Topics 1
O P-Value less than 0.05. but everything else in control Statistical Analysis Tools, Techniques and SPC 19
qualprod P x I = Value interpretation for residual risk? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
O CPK with a P value less than 0.005 Manufacturing and Related Processes 8
G What degrees of freedom means in uncertainty budget and how do I set the value Measurement Uncertainty (MU) 2
Q Really do they add value (Vision, Mission, Values)? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
D What is the exact Temperature Coefficient Value to be set on a Conductivity Meter? General Measurement Device and Calibration Topics 2
M Value Stream Mapping Calculations Process Maps, Process Mapping and Turtle Diagrams 5
L How to interpret the average R bar Value shown in the R Chart Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
G Non Value Added Corrective Actions IATF 16949 - Automotive Quality Systems Standard 15
R How To Calculate & Print CDF and Lower Bound CDF value for some t(time) with Minitab? Using Minitab Software 1
P Purpose of calculating Uncertainty value in calibration study Measurement Uncertainty (MU) 5
J Agenda of Value Stream Mapping workshop required Quality Tools, Improvement and Analysis 1
M What is the value of documenting in-process rework for easily-detectable issues? Quality Manager and Management Related Issues 4
R VSM - Value Stream Mapping (catheter manufacturing unit) Customer Complaints 10
W Quality for value adding or certification? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 23
T Cpk for 0 as Ideal Value Statistical Analysis Tools, Techniques and SPC 1
dubrizo Internal Audit Value - What is the point of conducting internal audits to a checklist Internal Auditing 40
L 5 minutes only for this Survey on the Value of PDCA and Your EMS Miscellaneous Environmental Standards and EMS Related Discussions 1
M How calculate P value for Linearity and Bias using Excel Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
J F and P-value in L9 Taguchi Design Using Minitab Software 22
S Value of Ppk or Cpk when targeting thinner material Statistical Analysis Tools, Techniques and SPC 4
B How to validate Microsoft Excel Spreadsheet Calculations for the Cpk value Capability, Accuracy and Stability - Processes, Machines, etc. 9
V Analysis of 'Value Added' System & Process Steps - Inspection Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 2
P Which ISO 13485 clause(es) are applicable for Value Proposition Creation ISO 13485:2016 - Medical Device Quality Management Systems 1
R The Value and Effectiveness of In Process Inspection Quality Manager and Management Related Issues 17
D Value-Added Analysis - Staff (Employee) Satisfaction Quality Tools, Improvement and Analysis 11
T ISO Auditor Training - What online training is actually value added? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
M Specific Capability case - Target Value is same as the Lower Limit Capability, Accuracy and Stability - Processes, Machines, etc. 9
S GR&R for an "R" value instrument installed on my 10K Horizontal Tester Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
S Value Stream Mapping for a Product Family Lean in Manufacturing and Service Industries 5
WCHorn Are Inspection Operations Value Added? Lean in Manufacturing and Service Industries 50
J How does Layout Inspection determine Reference Value? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
S Labeling Medical/Health Value Claims with No Sources to Back Up EU Medical Device Regulations 1

Similar threads

Top Bottom