Ah! One of my favorite topics.
Before I begin, I want to ask two questions:
1) On a CMM, why do we use DIFFERENT probe tips? What is the purpose of or reason for choosing a different tip?
2) On a CMM, when we are unhappy with a diameter, or plane, or whatever, do we do things like a) Add more probe hits and b) change the tip acceleration?
Please stop and think about these two questions before going further.
Your answer to BOTH questions is most likely something to do with measurement accuracy.
Consider Question 1: We all know a short, stiff, probe tip with a big ruby works "best." The only reason we deviate from this is we can't reach some feature with "the stubby." (And hopefully, we don't just do everything with a long, slender tip because it is easier.)
Consider Question 2: We all know more hits is better. The reason we don't take 1,000 hits is basically time. You have to get parts through the CMM.
The point of this is that things like probe selection and programming VERY MUCH affect how well a CMM measures. If my two questions have not convinced you of this, you may as well stop reading now.
On to your CMM calibration ... When that dude or dudette shows up, they are NOT concerned with how well your CMM is measuring your parts. They are concerned with if the CMM is WORKING. That's a different thing. What they do is measure a VERY CONTROLLED object in a VERY CONTROLLED way at different locations in the CMMs active volume. And they want to see that they get the same results in different locations and orientations. And they will tell you the devices repeatability in XYZ and/or the Volumetric Accuracy.
You can not just take this number and say "This is my uncertainty at measuring a part" because it isn't this. In fact, you may do BETTER at measuring a part because you (most likely) will load the part in the same location and the same orientation in the CMMs active volume.
If you want to know the uncertainty of measuring a particular feature, the only proper way to do this is to do a Gage R&R and get the uncertainty from that. Yeek. Suddenly it sounds like a lot of work. If you aren't convinced this is the correct way, think about this: Let's say you write a program and you aren't careful and you are shank hitting a diameter it is hard to see. You know this will give you bad results. You CLEARLY cannot take the "uncertainty" from your calibration results and use this... But were you to do a Gage R&R, you would see the uncertainty represented correctly (large and bad).
Do you have to do this on every feature? Is there a strategy to minimize this? I have two answers ....
1) The one you won't like, but it's true.
Look, to get a Gage R&R on ANY feature, you're going to run 12 parts 3 times at least. The CMM output is IN A COMPUTER at that point. KEEP it in a computer. You can make a big spreadsheet and put the features in rows and the results in columns. And then just copy down the formulas and calculate the uncertainties. The only real hard part is the cycling of the machine to take the measurements. Everything else is just an excuse.
2) If you aren't comfortable with this, or if a big Gage R&R table is beyond the scope of your customer and they want the "pretty" sheet, well, that sucks to do all the copy pasting of a lot of dimensions. So pick some key ones. A flatness. A short distance and a long distance. A diameter. A true position. You want to look at things you know will be problems. So you pick the tightest tolerances. True positions that are based on incomplete circles are also good candidates. Test your worst cases and call it good. Eventually, you will get a "feel" for things. And you will know that tip configuration X with Y probe hits will give you an uncertainty of Z on machined holes.
That's how it is done. The only way.
