The definition of "Contract Manufacturing" for European regulations is less than perfectly clear, because there are differences between German and British guidances in that regard. For US FDA, though, "Contract Manufacturing" means by definition that you build completed devices, which you have to List, and for which you have some degree of secondary regulatory responsibility in conjunction with the Specification Provider.
If in fact your intent is only to build components and assemblies, it sounds to me as if you instead will be a 21CFR 807.65(a) "Component Maker".