I'm not really a TE guru, but I assume you are saying you have never done a supplier evaluation. Is that correct?
For a calibration supplier I'd start with asking about any accreditations or certifications they have. Then look at their quality system.
Grandfathering is where, for example, someone does not need a certain type of training while new people to the job do. Typically the justification is that the person has been doing that job for a number of years and was doing it before the training requirement was implemented.
You can grandfather certain suppliers from an evaluation (at least for a while - typically for a year or so) - when you can cite data showing they provide services/product to your requirements. For example, if you have had a nonconformance system of some type up and running and you can show the supplier consistently shipped qood product - this is one aspect. If you track on-time delivery and the supplier is been on-time (as far as your requirement for on-time deliveries goes) - that's another 'reason' you could cite for grandfathering.
Have I confused the issue or does this make sense to you?