Calibration requirements for ISO 9001:2000 - We do not conduct GRR studies

[sbickley]

Hi Everyone,

We currently use many measuring instruments in our First Article and repeat inspection processes. This equipment is currently calibrated by an outside part once per year and we do not conduct GRR studies. We also do not conduct daily tool checks when they are checked out, etc.

Is this acceptable or should we be doing more to ensure conformance with the standard?

Please give me some ammunition if we need to do more so I can sell this to management.

Thanks,
Scott

 

[Sidney Vianna]

Let's roll the dice

Scott, the (ISO 9001:2000) Standard does not require R&R studies, nor it determines the frequency of calibration/verification of the measuring/test equipment. If you believe that the calibration frequencies should be increased, or you should have more strict controls over how you manage the instruments, my suggestion is for you to collect data to make a business case.

For example, if you can determine that you had a significant number of products returned and you can trace the problem for erroneous reading of the instruments, you might be able to quantify the cost of non-quality associated with this lack of control and compare such costs with the ones derived from enhanced calibration practices, including more frequent calibrations.

I am not very familiar with the Gaming business, but I would suspect that your sector must do some significant risk management. After all the stakes are high (pun intended).
Imagine one of your clients having to pay millions of dollars to a gamer, due to equipment malfunction, and the root cause being out-of-spec equipment accepted due to IM&T equipment used to test was out of spec! ! This could have some serious repercussions for your organization, as a supplier. I can only imagine the legalese in the contracts between casinos and their equipment suppliers.

Good luck!

 

[sbickley]

Thanks Sidney!

You reaffirmed my original belief, though I didn't want to express that view so that I could get an unbiased response. Our games actually determine the "win" by a Random Number Generator and firmware programs - the meters ARE, however, used to visually indicate a jackpot. Here is where it gets tricky --- you can have the symbols line up, but have a computer malfunction and NOT GET PAID - of course that is bad. Furthermore, I haven't yet found a way to calibrate a meter motor -- it seems to work or not work. We have our own "homemade" testers that test our firmware programs - but there doesn't seem to be anything to calibrate there. Very challenging!

Any suggestions there?

 

[Govind]

When there is, software involved with the equipment the calibration definitely become challenging as to what to calibrate.
Unlike hardware, Software does not wear out and hence there is no bias. However, Software can have malfunction and cause error outputs. Firmware is an embedded software.
Changes to the operating environments (system), corruption due to high electrostatic discharge, etc can cause malfunction of the software program.
Verification and validation of software based on functional specification will be one of the ways to ensure that software is working as required. I can see relevance to Software reliability also in this context.
Iam not sure what the "homemade" tester is testing. An educated guess would be to execute the program several times and check the randomness of the output?
Regards,
Govind.

 

[sbickley]

When there is, software involved with the equipment the calibration definitely become challenging as to what to calibrate.
Unlike hardware, Software does not wear out and hence there is no bias. However, Software can have malfunction and cause error outputs. Firmware is an embedded software.
Changes to the operating environments (system), corruption due to high electrostatic discharge, etc can cause malfunction of the software program.
Verification and validation of software based on functional specification will be one of the ways to ensure that software is working as required. I can see relevance to Software reliability also in this context.
Iam not sure what the "homemade" tester is testing. An educated guess would be to execute the program several times and check the randomness of the output?
Regards,
Govind.

Govind,

You are correct. We perform extensive testing with regard to Hi-Pot testing, ESD protections, electrical interference -- in addition, we also perform a 1 million cycle game run during the latter stages of development to work out any anomolies in the firmware. The games then go to a Wide Area Market (WAM) field test which gauges game functional performance and customer acceptance (casino) and end customer acceptance (player).

 

[Govind]

sbickley,
In my opinion,you are doing all the right things wrt validating the functional performance of the software giving adequate considerations to operation environment. If this software is driving any mechanical unit, you may want to monitor for the wear that can make the output lose its randomness.
Did you see my posting on Software reliability. Since you are performing 1 million cycle, you should have very rich data for software reliability. Did you ever try fitting in a distribution (eg: Raleigh) and predict?
Govind.

 

[Graeme]

ISO 10012:2003 Measurement management systems

Hi Everyone,

We currently use many measuring instruments in our First Article and repeat inspection processes. This equipment is currently calibrated by an outside part once per year and we do not conduct GRR studies. We also do not conduct daily tool checks when they are checked out, etc.

Is this acceptable or should we be doing more to ensure conformance with the standard?

Please give me some ammunition if we need to do more so I can sell this to management.

Thanks,
Scott

Scott,

(A short answer is that if it meets your needs then it's OK. But it may not really be fully suitable, which is probably your concern.)

I can't directly answer your question, but I can point you to a standard that may help. In ISO 9001:2000, at the end of clause 7.6 Control of measuring and monitoring devices there is a note that specifies ISO 10012-1 and 10012-2 as guidance. Those standards have since been superseded by ISO 10012:2003 Measurement management systems - requirements for measurement processes and measuring equipment. This can be purchased from ASQ or (broken link removed).

ISO 10012 provides guidance for management of an organization's complete measurement system. (For some reason, many people believe it is only for the calibration/inspection/test areas, but that is not correct.) As stated in the scope, the standard provides guidance for

• management of measurement processes, and
• metrological confirmation of of measuring equipment used to support and demonstrate compliance with metrological requirements.

The "measurement processes" are not restricted by location or function. If a measurement is made that affects product quality, in a core or support process, then is can be managed using this guidance.

Metrology is the science of measurement, so "metrological confirmation" is calibration and "metrological requirements" are the measurement requirements of a product or process.

According to the standard, the measurement requirements are based on the product, customer requirements, legal and regulatory requirements, and so on. All measurement process in production or evaluation have to be managed. All measurements relating to conformance to requirements must be made with calibrated measuring instruments. ISO 10012 is the guidance standard for applying these and other principles to ISO 9001:2000.

As for testing the random number routines, I think NIST has something on that - a search in the Information Technology Laboratory area for "random number" turned up almost 150 links including this one. I think they have routines for testing the randomness according to the expected distribution, algorithms for generating specific distributions, and lots of stuff that is way above my understanding level.

 

[Hershal]

Scott,

Graeme is right on target.

Hershal

 

[sbickley]

Honestly, I have to defer to the probability experts when auditing that area. I, of course, look at the paytable test results to ensure that what we manufacture matches the test, but we have our own software to generate the high end math and RNG stuff. Sometimes, you just gotta know your limits - mine are on the sideline there.

 

[Leonardo Lopez]

Scott, the (ISO 9001:2000) Standard does not require R&R studies, nor it determines the frequency of calibration/verification of the measuring/test equipment. If you believe that the

Does this means that the studies have to be done only on that measurement equipment used for a PPAP product?