The sample size has to be such that it reduces the risk as far as possible. This is necessary to comply with the MDD. Unfortunately, this means you must approach 100% destructive testing to comply with the MDD.
Sorry, being a jerk, but it is an unworkable regulation. In reality, it depends upon what you the purpose is of your sterility testing. The sterilization standards typically stipulate quantities for sterility testing. Each method has it's own standard. Sterility testing cannot replace sterilization validation. If you have not validated your sterilization process, you are in very high risk for regulatory action from regulatory agencies all over the world (US, EU, everywhere).