SBS - The Best Value in QMS software

Transforming or not Transforming - Dealing with Non-Normal Data

R

rafael_josem

#1
Hello Guys,

Lately, I have been doing some research about what to do when you have non-normal data for control charting purposes. I have found out that some say that it doesnt matter, others say that you should transform your data.

Can anyone help me to clarify this? I just go confused :confused:

Thanks!
 
Elsmar Forum Sponsor

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#2
Lately, I have been doing some research about what to do when you have non-normal data for control charting purposes. I have found out that some say that it doesnt matter, others say that you should transform your data.
Depends on what is causing your data to be non-normal, especially what kind of process it is. Transforming is used more in capability than control charting.
 
R

rafael_josem

#3
Let's say that the process is non normal. This is very common, at least in my experience. I haven't seen any normal distribution yet.

So, for these cases, do you have to transform or is it ok to use the data as it is for control charting?

Also, I would appreciate any good references that you can provide me.
 

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#4
You can find some basic information on non-normal SPC in Don Wheeler's book "Normality and the Process Behavior Chart". It is effective if your process variation is from one source, and independent.

If it was from a consistent variation, such as tool wear, the resource on how you handle that is my book "CorrectSPC - When 'normal' is not typical; A guide for precision machining statistical process control".

Neither reference uses transformation.

You may also have multi-modal distribution, as determined by the total variance equation. In this case you may need to spend time reducing some of the participating variances prior to SPC.

I agree, outside of "natural" variation, variation caused by excessive human intervention (running to the mean), or variation masked by significant measurement or gage error, the normal distribution is not common.
 
D

Darius

#5
Don Wheeler in his book "Advanced topics on SPC" said that (sort of) the problem with transforming is that is that most of the times this technique is very difficult to understand from the point of view of the operators, so if you are losing "readability" of your charts is not recommended.

He also pointed out that no matter which distribution your data follows, the SPC will work OK (on the same book), and IMHO, this can be true, but beware of detection rules, they really are defined for Gaussian distribution ("Normal").

As some of this thread said, there are no prefect "Normal" distribution, in may experience..., if the value is affected by time most of the times will not be "Normal", check for the rules alarms, too many will make your chart a nice Christmas tree, but what is the use of a chart if don't tell you something to improve your process.
 

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#6
Don Wheeler in his book "Advanced topics on SPC" said that (sort of) the problem with transforming is that is that most of the times this technique is very difficult to understand from the point of view of the operators, so if you are losing "readability" of your charts is not recommended.
This is a point that both I and Shewhart agree with him on in our books. "Rule No.1
Original data should be presented in a way that will preserve the evidence of the original data for all the predictions assumed to be useful."
-Dr. Walter A. Shewhart; Statistical Method from the Viewpoint of Quality Control

Transformation generally maskes the original evidence.

He also pointed out that no matter which distribution your data follows, the SPC will work OK (on the same book), and IMHO, this can be true, but beware of detection rules, they really are defined for Gaussian distribution ("Normal").
I agree, his point is particularly directed to the "signals" that traditional SPC will provide exist no matter what the distribution, but he does not test the validity of the WEC rules, most of which are designed to ensure the process is varying about the mean in a random manner.

As some of this thread said, there are no prefect "Normal" distribution, in my experience..., if the value is affected by time most of the times will not be "Normal", check for the rules alarms, too many will make your chart a nice Christmas tree, but what is the use of a chart if don't tell you something to improve your process.
Yes, that is the issue I deal with in my book. There are some natural variations. One example I use is loaves of bread coming out of an automated bakery oven. Most are a particular height - some a little higher, some a little lower. You would likely see a normal distribution there....unless it is Wonder Bread. Then they are exactly the same - and no one is sure how they do that! There are some heat treaters that would love to know the secret, though!
 
S

SPC_Newbie

#7
I agree, his point is particularly directed to the "signals" that traditional SPC will provide exist no matter what the distribution, but he does not test the validity of the WEC rules, most of which are designed to ensure the process is varying about the mean in a random manner.
Do we have a list of detection rules for non-normal data?
Thx!
 

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#8
Do we have a list of detection rules for non-normal data?
The correct answer to any question: it depends.

We know, for example, with tool wear on an OD an upward trend is totally acceptable. Number of points above and below the mean is irrelevant (because the correct curve is not random about the mean).

But, points in a row the wrong direction (e.g. downward for an OD) need attention.

So, yes, you need a different set of detection rules.
 
S

SPC_Newbie

#9
Thanks - Right, I agree we need a different set and the one one 'rule' you gave, looking for points in a row in the wrong direction, is a great one. Have you/anyone else already compiled other/additional rules they'd like to share?

Maybe a different set exists for for each of the 'typical' histograms we see (skewed towards zero for parallelism, skewed towards an upper constraint for?, etc).

My questions are coming from the precision machining perspective -

THANKS!
 
Last edited:

Bev D

Heretical Statistician
Staff member
Super Moderator
#10
All rules come from the general probabilities of 'non-random patterns': shifts, trends and cycles.
Most of the time (across all industries; some industries will have a different distribution of process behavior than others - such as tool wear) the Western Electric Rules are sufficient. Remember the probabilities of a signal are not precise, nor were they intended to be. If a specific rule doesn't make sense for you don't use it.

Bob provides a good example of trending and tool wear.

Another example (that probably doesn't apply to your circumstance) I have is that we track customer complaints as a percent of our instrument install base. However, while this is a useful surrogate for the area of opportunity for a 'complaint', it isn't perfect. We would actually need to know how many times the instrument were used and in the case of some complaints wether or not the particualr test was run. Occasionally some disease states are seasonal and so teh area of opportunity can vary by season. This makes the rule of 4 out of 5 points beyond 1sd of the mean on the same side of the mean a false alarm too often to be useful. Many of our product compaints have a 'slow seasonal roll' so we turn this rule off.

I find that that the 'shift' rules are almost always applicable (1 point out, 2 out of 3 near the same limit and 8 in a row on the same side of the average) as long as the measurement system has sufficient resolution AND the appropriate chart is selected (for example 1 point above the upper limit for a c chart that an average count of less than .1 is silly becuase teh c chart is meant for such a low defect rate (the normal approximation to the Poisson doesn't hold at such low rates. Of course, the signal is correct - it is telling you you need a different chart, not a change to the process)

to not use certain rules or to invoke custom rules requires in depth knowledge of the process at hand, an understanding of the logic of SPC and a suspension of our natural tendancy to want to rationalize variation that we dont' want to work on.

You are looking non random patterns that would tell you that something has most likely changed. It is that simple and that complicated.
 
Thread starter Similar threads Forum Replies Date
H Minitab DOE Analysis of Data and Transforming Using Minitab Software 4
V Transforming the Attitude and Culture towards Documentation and Process Controls US Food and Drug Administration (FDA) 2
M Transforming a Decision Tree into Excel Excel .xls Spreadsheet Templates and Tools 7
M Transforming your Data - Analyzing a DOE Six Sigma 5
C Non-Normally Distributed Data - Nonparametrics vs. Transforming data Using Minitab Software 6
J Non-Normal Data - Transforming the Data to Normal Statistical Analysis Tools, Techniques and SPC 12
G Dealing with non conformity caused by Supplier Components detected in the production line IATF 16949 - Automotive Quality Systems Standard 14
K Tips for dealing with third party auditors General Auditing Discussions 11
Q Dealing with a forgetful Special Processor AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 13
C Specification + MU - Dealing with reported MU Values Measurement Uncertainty (MU) 6
F Dealing with, and Analysis of, Data Statistical Analysis Tools, Techniques and SPC 6
R Dealing with Device/Design changes by a "Letter to File" vs. 60601 Retesting IEC 60601 - Medical Electrical Equipment Safety Standards Series 10
Sidney Vianna Cleaning up the Global Compact: Dealing with Corporate Free Riders Sustainability, Green Initiatives and Ecology 2
J Integrated Management System - Anyone with experience dealing with ISOQAR? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
G Dealing with Excessive Within-Part Variation when doing MSA and Cpk studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
S Auditing Career: Dealing with Mentally Unstable Managers Career and Occupation Discussions 7
K Has anyone had experience dealing with SEDEX (Supplier Ethical Data Exchange)? Customer and Company Specific Requirements 4
P Hard Milling Process - Dealing with Within Part Variation for SPC Statistical Analysis Tools, Techniques and SPC 18
optomist1 Real World SPC Dealing with Outliers Statistical Analysis Tools, Techniques and SPC 22
P Advice on dealing with difficult Colleagues Career and Occupation Discussions 24
R Effects of Dealing with Out of Tolerance Monitoring Equipment General Measurement Device and Calibration Topics 4
Z Dealing with a defiant Operations Manager Quality Manager and Management Related Issues 41
ScottK Are any other USA manufacturers dealing a lot with ARRA requests? Various Other Specifications, Standards, and related Requirements 6
N Nissim Shaked is dealing with these topics at his desk Imported Legacy Blogs 14
D Dealing With a Chinese Supplier to follow up on quality problems Quality Manager and Management Related Issues 26
D Dealing with pesky telemarketers Coffee Break and Water Cooler Discussions 4
B Dealing with Outsourced Processes in a DMR ISO 13485:2016 - Medical Device Quality Management Systems 7
Marc Government of Mozambique - Dealing with Lions Funny Stuff - Jokes and Humour 14
P Dealing with Second Party Audits Quality Manager and Management Related Issues 6
R Service Company dealing with FAI requirements - Small stocklist broker AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
A Is zero defects possible? We are dealing with 25 to 30 parameters Quality Tools, Improvement and Analysis 21
Marc Advisory Notices - Who is responsible for dealing with Advisory Notices? ISO 13485:2016 - Medical Device Quality Management Systems 7
S Process of dealing with the Customer - PRRs, GQTS Nonconformance and Corrective Action 5
samer Oracle computer records - Special procedure for dealing with non-paper Records? Records and Data - Quality, Legal and Other Evidence 2
A Capability Analysis - Dealing with non-normal data in Minitab Using Minitab Software 8
Douglas E. Purdy Dealing with Procurement Documentation and an Industry Conglomerate General Auditing Discussions 4
P Dealing with recruiters: What if a job is listed after.... Career and Occupation Discussions 17
P Recently Certified!! One minor finding dealing with Dock Audits IATF 16949 - Automotive Quality Systems Standard 11
T Industrial Belt Sander Systems - Dealing with chatter marks Manufacturing and Related Processes 5
B Dealing with dishonest customers - One customer who is a supplier - Bad Material Supplier Quality Assurance and other Supplier Issues 16
A Dealing with Possible Alcohol Use by Employees in the Work Place Misc. Quality Assurance and Business Systems Related Topics 23
B Published standards dealing with calibration of pressure and vacuum gauges? General Measurement Device and Calibration Topics 5

Similar threads

Top Bottom