I think the definitive aspect of your operations is sales of the device. As the initial importer and seller, you are the "initial distributor" in 21 CFR 820.3(o), and therefore subject to most requirements applicable to the Manufacturer.
Given that you are subject to requirements applicable to the Manufacturer, the service you provide then is subject to 21 CFR 820.200. This, I think, is the basis for the auditor's comment.
You might want to call DSMICA to verify.

Yes, initial distributors are considered a type of manufacturer (I was only commenting on the servicing aspect).
In that regard - the FDA's QSR manual's introduction, under "GMP applications and exemptions", reads:
Initial Distributors of Imported Devices
The initial distributor is the foreign manufacturer's official correspondent with the FDA. With regards to the GMP, this initial distributor is responsible for maintaining complaint files and general record keeping requirements. A procedure shall be established and maintained for receiving, reviewing, and evaluating complaints. All complaints, including oral complaints, are to be processed in a uniform and timely manner. These complaints shall be evaluated to determine whether or not they require reporting to FDA under 21 CFR part 804 or 803, Medical Device Reporting. The initial distributor is also required to evaluate all complaints to determine whether an investigation is necessary, as well as complying with all other requirements in 820.198, Complaint Files. See Chapter 15 in this manual for more complete guidance on handling complaints.
So, apart from the issue of complaint handling, which seems to be well covered, the only other reference I see here is "With regards to the GMP, this initial distributor is responsible for... general record keeping requirements", which is a bit vague. The above-mentioned auditor's comments were more specific (at least some of them), and hence my question re. regulatory basis.
With regards to section 820.200 and the servicing issue - section 820.200 relates to "Manufacturers" so presumably that would cover all types of manufacturers, including initial distributors (importers); however, my understanding was that this part's requirements relate to the original manufacturer (in this case, the foreign manufacturer) - for instance, I wouldn't think that the FDA had expected every importer (including those that only import and have zero technical involvement in the servicing) to "establish instructions and procedures for servicing" (refer 820.200(a)), yet by assuming that 820.200 is equally applicable to all types of "Manufacturers" that would be the case. If the OP's company provides the servicing on behalf of the original (foreign) manufacturer, then 820.200's requirements should be addressed via the foreign manufacturer's QMS and as such, in my opinion, should be audited only when this QMS is audited (nevertheless, the servicing provider may be bound by contractual documenting requirements to facilitate the original manufacturer's compliance).
My comment re. "reconditioning" (servicing a multi-use device already in use) related to servicing that is provided by a 3rd party, not on behalf of the original manufacturer. I guess its applicability depends on the actual relationship between the OP's company and the original manufacturer.
I think that for clarity purposes this issue should be strictly separated into two: the company's obligations as an initial distributor (importer); and its obligations as a servicer (either an independent one or on behalf of the original manufacturer). The regulatory requirements should be determined separately for each of these aspects, and once it's done they could be integrated back into a single QMS, for efficiency.
I also agree that a formal approach to the FDA is in place in any case of doubt; however, in my opinion it's best to do so after some thorough homework, with proper analysis and justifications in place.
Cheers,
Ronen.
PS to the question in the title - I think 820.180(b) applies, because of the "general record keeping requirements" clause, applicable to initial distributors.