Thank you for your quick feedback and insight.
I will sit together with my colleague on Tuesday after Labor Day and proceed for the appeal process even if there's a time constraints. You are right, we have a right to appeal as an auditee. While many auditors we've seen in the past were more polite and willing to learn what we do (and many times it have been value-added experience), some wanted to push their opinions instead of going with the standard and not willing to listen (maybe temperament issue?). We had a very bad auditor one time and not only complained to the CB, but also appealed the NCs, yet the CB did not react quickly, so I had to go to IAOB to complain. This and the bad schedule management at the CB (high turnover at client services), we left that CB and switched to the current one in 2018. For the most part, we were very satisfied with the current CB until they dropped this bomb.
I guess whether the CB change is needed or not would be dependent on how this CB would react to our appeal and complaint. But meanwhile, we will go with the appeal process...we shouldn't have to go with what one auditor insists if it is not reasonably justified.