Not sure that I got it all figured out
but the address of the legal manufactured should be included in your ISO 13485 certification.
Practically speaking, you can apply for one certificate with multi-sties. In your case site A and B together. Site C and D are your contract manufacturers; they don't have to have ISO 13485 although it would be nice. It's important that the quality manual per site explains which activity is being performed in that site, and to have good quality agreements with sites C and D.