There could certainly be a risk to patients - maybe not directly, but if they screw up the data, you could have invalid clinical data for safety or efficacy.
I treat them as pretty high risk. I want a Quality Agreement with them. (I had one client who didn't execute a quality agreement with them and didn't stipulate data ownership and, as a result, could not get the data collected, only summary reports!) I want to see any software used validated and I want to be sure they have the proper competencies in the type of clinical study to be conducted (as well as basic competencies in data integrity). And, as you note, data security is a huge concern. A breach there could cost you dearly so I want to see what those controls are (logical and physical - and for both the "live" data as well as backups).
I think there's a tendency to strictly adhere to risk-based buckets without flexibility to manage as the needs arise. Hopefully you have the flexibility to escalate the level of control irrespective of what bucket the supplier may fall in based on your criteria.