This is a good question, to which I'll add some further questions and partial respnse, as I am dealing with some related issues.
I think ISO17025 is a little vague on qualifications. This may perhaps be because "calibration" (my world) and testing covers such a very wide range of specialty disciplines. I think my answer would be that you need to have adequate skills to audit. That seems to answer your question without an answer. But what I mean is that when you audit, you have to have the understanding to verify that the person or process meets what ever requirement is imposed. Half of this, in my view is the auditor (quality systems) training and/or experience. The other half is (in my view) adequate understanding of the technical context.
The second half of this has been an annoyance to me for many years. I have ISO9001 Lead Auditor Training among other types of audit training, as well as military calibration training and 30+ years experience in calibration. It has long annoyed me when an auditor does not fully understand the technical side (as is too often the case).
I believe it is explicit in ISO17025 that anyone making decisions effecting the quality of the calibrations (including audits) must have adequate background that their audit decisions ensure the calibration results are valid (in this case, the results of the audit).
I would be interested in some expert opinions about this, including the quality manager as imposed in paragraph 4.1.5 and other places. What level of technical calibration expertise is required of the quality manager? or auditors? Are there other paragraphs, or other standards referenced directly or indirectly by ISO17025 that specify the technical competence of quality personnel (quality manager, auditor, etc.) in an ISO17025 accredited calibration laboratory? It seems to me that this is one of those weak spots that companies use to save money or shortcut; even though these are key roles that impact the labs quality every bit as much as the technicians performing the calibrations.