ISO 19011:2008 4d) states "Auditors are independant of the activity being audited and are free from bias and conflict of interest. Auditors maintain an objective state of mind throughout the audit process to ensure that the audit findings and conclusions will be based only on the audit evidence."
The supplemental guidance, S4.3 Use by small organizations, states "These principles apply equally to small organizations as stated. There is no less emphasis on independence, professional care, and evidence-based approach for a single-person operation where audit results directly impact business than for a very large organization. In fact, the smallest organizations are often more sensitive to the need for independence and a factual based approach. For such organizations, it may not be possible for internal auditors to be wholly independent of the activity being audited."
I was taught that being objective and evidence based is a higher priority than being independent of the function being audited. I work for a mid-sized company with several internal auditors, and I still end up completing most of the audits because no one has enough time. Hope this helps.