Wrong standard deviation calculation in AIAG SPC manual?

[Bromus]

Hello,

According to IATF 16949 I wanted to validate my worksheet for Cp/Cpk, Pp/Ppk calculations. I have used data set from page 186 of AIAG SPC edition 2 manual. Everything seems to be fine except of total sigma. My calculation is 0,188657 but in the manual there is 0,189037. I copied the data by hand from the manual, so I searched for a bug for a long time, but didn't find it. Additionally, total mean and sigma for within variation (Rbar/d2) is correct in my calculations based on copied data.

Does this mean there is a mistake in the manual? I realize that this is not a very significant difference, but an auditor who wants to compare my calculations with the manual may question their correctness. Can you confirm my calculations ?

 

[AMIT BALLAL]

@Bromus , which formula did you use for calculating the Total standard deviation?

 

[Bromus]

Good one e.i. stdev.s.
BTW: Minitab returns the same result as Excel using the same AIAG dataset.

 

[John Predmore]

One possible difference you could check out is the Standard Deviation of a population versus Standard Deviation from sampling. Khan Academy explains. You could check this theory by doing the calculations step-by-step with N and then with N-1. Or take the larger result and multiply by SqRoot(N)/SqRoot(N-1) and see whether you get the smaller result.

 

[Miner]

Good one e.i. stdev.s.
BTW: Minitab returns the same result as Excel using the same AIAG dataset.

I get the same results as you in both Minitab and Excel using the sample standard deviation (n-1). I also manually calculated it using their formula and got the same results as Minitab and Excel.

 

[Bromus]

John Predmore: Once again, no I have used correct formula for sample (division by n-1). Of course I have tried to calulacte stdev step-by-step using square root from sum of squares divided by n -1. Still the same.

I have to low "level" to attach the file with source data and calculation so I have asked admin to do it. So you can check it by yourself.

 

[Bromus]

I get the same results as you in both Minitab and Excel using the sample standard deviation (n-1).

Thanks Miner, so we can officialy confirm unknown bug in the manual

 

[Miner]

I attached my calculations.

 

[Matt Savage]

Thanks Miner, so we can officialy confirm unknown bug in the manual

I was expecting to see an errata sheet here: Publication Errata Documents that lists the error, but nothing is listed. Is this only page that lists the standard deviation with the supporting data?

 

[Bromus]

I was expecting to see an errata sheet here: that lists the error, but nothing is listed. Is this only page that lists the standard deviation with the supporting data?

If you mean the manual - it is Appendix F with full case study, starting with dataset followed by step-by-step calulations leading to capability indices.
As far as I know there is no errata for SPC manual.