Thanks for the response Welshwizard.
Reading this again, has left me a little confused. You said the measurement process can detect down to 0.020 but then mention it detects increments between 0.004 and 0.04. Could you explain where you are getting these values from and the difference between the 0.020 and 0.04.
And yes I agree. Taking a look at operators 3 measurement technique compared to the others is planned. Also thanks for linking that paper reference. Very informative.
Hello Breadandwater, you are very welcome, it sounds like you are on the right track.
Miner has quoted an excellent resource to help with the use of the Probable Error (PE) and the general EMP approach.
To answer your question generally:
You will see and realise when you read into this some more that the PE places an upper bound on the precision of a measurement, there is therefore no point in attempting to interpret any value more precisely than +/- 1 probable error.
Bearing the above in mind we can see that the PE is the demonstrable resolution of the measurement process. We can use this property to check on whether the recorded increments for the measurements are usefull or just pure noise.
The effective increment is defined by the PE when it exceeds the smallest increment, the increment dominates when its larger than the PE. We can go on to prove that the increments being recorded are justified when in the range of 0.2 to 2 PE. So, when the increment is less than 0.2 PE we can drop a decimal place, when its more than 2 PE we woud be justified in adding a decimal place.
For your data the PE was 0.020 so your recorded increment should be in the range of 0.004 (0.2 PE) and 0.04 (2 PE). As your increments were 0.001 mm the last digit is less than 0.004 therefore its pure noise and it can justifiably be dropped therefore making your increments 0.01 and in the range of 0.2 to 2 PE.
Some people find justification very usefull, I used to work for a National Standards Institute and saw one application were the manufacturer of a very expensive device claimed a resolution of 7 decimal places but it was clear than upon testing the process could only "see" 4. Very often a manufacturer will state the resolution based upon how many digits the display has the ability to record, you now have a tool to validate this, there are other benefits for using PE which are beyond this offering.
The attached is a good explanation of PE to be added to Miners Blog and EMP III. A lot of reading material is on Dr Wheelers website
www.spcpress.com.
Thanks for your interest and good luck.