The only thing I'd add to the many good pieces of advice you've been given is: focus attention on areas that audit is going to be valuable for. By which I mean, if the scope of what you do has shrunk (say) and you now no longer do (or do almost none of) a process or set of activities that was big for you before, then don't spend valuable time/resources on auditing it now 'just for the sake of auditing it'.
After all, if it was audited and the report said 'hey, some problems in this area', then management wouldn't find much value in that, as they'd probably respond: 'so what, we're not doing that any more'. And they'd be right.
Auditing in an organisation with only a few people is a challenge, to be sure. But as Randy says, do what's best for you and your company. Don't get misled by people jumping up & down and insisting you have to be completely independent of everything (works in larger companies, with bigger resources but smaller ones have to be a bit more flexible in approach, as do external auditors of same)