Then the auditor has no business auditing government contractors if he doesn't understand the relationship between contractors, the organization being served, the contract itself and the government contracting offices & regulations. Actually pretty typical for an auditor to be as lost as a blind folded duck in a duststorm.
How to mitigate the risk? Perhaps document it in the manual or procedure and help an "relevant interested party" (an auditor that doesn't have experience with government contracts) understand the boundaries of responsibility.