Cloud-based SaMD Validation

RedDevil_AK

Starting to get Involved
#1
Hi Cove-dwellers,

I (no software development experience) am working at a startup with a Class II (moderate LoC) SaMD designed to run on a cloud-based service hosted in AWS. The platform code provides an SDK package for device software to enable the use of an API to launch docker containers with algorithms with the use of AWS resources within AWS, track the progress of their execution and store information about such jobs in the database in order to allow monitoring and reporting.

I was hoping to get clarity on a couple of questions regarding V&V for this platform -

a. For AWS validation, what else would be required outside of their validation documentation on Part 11 compliance and data confidentiality and integrity? The device uses the platform to make API calls to AWS, so would validation consist of making dummy API calls and making sure that input/output is repeatable?

b. For the platform itself, would unit tests comprising of design and peer review, testing written into the code (Unit tests check that all the code paths work as expected (e.g. perform specific activities depending on the input and intermediate results, fail quietly with or without the log message if it is acceptable or raise exception and propagate it up the stack to alarm about the failure, etc.).) and functional testing (covering integration + system tests) be enough?

Thanks! Appreciate all responses.
 
Elsmar Forum Sponsor

yodon

Leader
Super Moderator
#2
From the description, I'm having a hard time seeing how this is a medical device, but let's assume so.

a. For AWS validation, what else would be required outside of their validation documentation on Part 11 compliance and data confidentiality and integrity? The device uses the platform to make API calls to AWS, so would validation consist of making dummy API calls and making sure that input/output is repeatable?
Validation shouldn't be just about testing. How will you know that the platform is stable and changes are rolled out in a controlled manner (allowing you time to ensure your software still works on it)? Not sure how Part 11 fits in here. Certainly you need to ensure data confidentiality and integrity (and retention). This pulls in backup and recovery as well.

b. For the platform itself, would unit tests comprising of design and peer review, testing written into the code (Unit tests check that all the code paths work as expected (e.g. perform specific activities depending on the input and intermediate results, fail quietly with or without the log message if it is acceptable or raise exception and propagate it up the stack to alarm about the failure, etc.).) and functional testing (covering integration + system tests) be enough?
62304 uses the concept "unit verification and acceptance." One of the means to do unit verification and acceptance is unit testing (and there's a little baggage that goes along with that). You define what the means for unit verification and acceptance. We do rely heavily on static analysis and code inspections for our unit verification and acceptance. We generally only use unit testing if there are some safety considerations in the unit. Anyway, your SW Dev Plan needs to define what you WILL do for unit verification and acceptance. What you describe should be acceptable. (Side note: in the last few years, FDA has taken a keen interest in static analysis - would definitely recommend that.)

Most likely, you can cover "Software System Testing" (62304) / Software Requirements Verification through your functional testing. There may be additional considerations regarding overall system testing (may not be anything else). Risk analysis may drive a need for additional efforts to demonstrate effectiveness. If you're complying with IEC 62366, you may have some additional usability testing.

And certainly don't forget cybersecurity. That's pretty much front-and-center these days when it comes to reviews.
 

RedDevil_AK

Starting to get Involved
#3
Thanks yodon!

From the description, I'm having a hard time seeing how this is a medical device, but let's assume so.
should have been more detailed - the platform is used to host the software that does the medical device part (image segmentation)

How will you know that the platform is stable and changes are rolled out in a controlled manner (allowing you time to ensure your software still works on it)?
while I understand what the intent is, I am unable to understand how it pertains to AWS as the third-party tool. I'm sorry, could you please rephrase this?
 

yodon

Leader
Super Moderator
#4
while I understand what the intent is, I am unable to understand how it pertains to AWS as the third-party tool. I'm sorry, could you please rephrase this?
I'll try. :) You are relying on AWS for certain things. They may change those things. They may deprecate or change an interface. Will these changes pull the rug out from under you or will there be a transition period to allow you to adapt?
 
#5
I was just about to post a related question, so perhaps I will just add it here.

Regarding IEC 62304 8.1 Configuration Identification. What types of Configuration Items must one record for software using a cloud like AWS?
 

yodon

Leader
Super Moderator
#6
I don't expect there would be any CIs for you. You don't have any control over what they do. I presume there's an API spec that you can cite but, again, not a CI for you.
 
#7
Thanks, Yodon, perhaps I can re-phrase. We are building a cloud-based SaMD, which uses AWS. Won't my S3 or other infrastructure I use be considered CIs?
 

ECHO

Involved In Discussions
#8
Let me try to clear a few things up a bit.

The intention of IEC 62304 8.1 is to track the changes in the software, including libraries and the environment the software runs in. Therefore, when your software quality engineer says, "Engineering Build V2.1.113 passed verification", as a team, you know exactly how to recreate a deterministic system.

The statement above implies that the system will use a fixed version of libraries and environment. Now, if you are using AWS, you don't really have control over what Amazon does to their backend.

The changes AWS would/could make doesn't really impact you if you are just using S3 to simply store an image. But if you are using EC2 to do very time sensitive calculations, how do you know that the change AWS made isn't going to add a microsecond to your calculation and throw off the entire functionality?

Below is an example of what you can do.
Automate your testing so you can catch potential problems.
When you run these automated tests, make sure to record your SW version number (or just git hash for section 8.1.1), spit out the dependent libraries (title, manufacturer and version for section 8.1.2) and similar info for the environment. Save this log (for section 8.1.3).

The frequency of the tests and what you do when you find an issue will depend how you implemented the rest of IEC 62304.
 

yodon

Leader
Super Moderator
#9
Won't my S3 or other infrastructure I use be considered CIs?
I had to chat with our software / cloud expert on this. According to him, the answer is "sort of." What we apparently do is to capture all this in build / setup instructions. So in that respect, yes, that would be a CI.
 
Thread starter Similar threads Forum Replies Date
J Cloud Based System Qualification and Validation (including 21 CFR Part 11) 7
H Existing cloud based medical device - questions regarding improving the processes IEC 62304 - Medical Device Software Life Cycle Processes 6
S DHF/DMR/MDF for a software-only, cloud-based, single-instance device Medical Information Technology, Medical Software and Health Informatics 2
P Testing cloud-based backups IT (Information Technology) Service Management 7
was named killer CLOUD BASED QUALITY DOCUMENTATION vs. SERVER BASED Document Control Systems, Procedures, Forms and Templates 5
S Validation of eQMS - Cloud based out of the box solution Other Medical Device Related Standards 18
S Moving from client-server to cloud-based, is that a new submission? Medical Information Technology, Medical Software and Health Informatics 3
Z Security for Approvals - Cloud based Complaint, NC, and CAPA systems Qualification and Validation (including 21 CFR Part 11) 8
T FDA proposed labeling standalone software cloud based US Food and Drug Administration (FDA) 4
R Online / Cloud Based Software as Medical Device EU Medical Device Regulations 8
S Cloud-Based Stand Alone Software - Software Medical Device (Class II) US Food and Drug Administration (FDA) 2
D Anyone using a cloud based QMS software? Document Control Systems, Procedures, Forms and Templates 12
DanBOS Cloud Connected Medical Device 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
Y Exporting data to the cloud is a "Significant Change"? EU Medical Device Regulations 5
Tagin Hosting in the cloud still requires DR (OVH fire) Business Continuity & Resiliency Planning (BCRP) 1
B Oracle Cloud ERP Validation during Quarterly Patch ISO 13485:2016 - Medical Device Quality Management Systems 1
shimonv Classification of a cloud- base viewer for the output from a medical device US Food and Drug Administration (FDA) 7
O ZenQMS cloud solution? Quality Assurance and Compliance Software Tools and Solutions 0
Q Storing and developing SAMD (Software as a Medical Device) in the Cloud IEC 62304 - Medical Device Software Life Cycle Processes 3
Ed Panek Do Cloud services require 21 CFR Part 11 compliance? Qualification and Validation (including 21 CFR Part 11) 7
T QMS - Documentation Cloud Storage EU Medical Device Regulations 0
R Validation of mobile app and cloud servers for data security IEC 62304 - Medical Device Software Life Cycle Processes 4
S Saving QMS documents in cloud drive - Compliance with ISO 13485 ISO 13485:2016 - Medical Device Quality Management Systems 3
C Validation of Applications in a Cloud, CFR 21 part 11 (Environmental Monitoring) Other US Medical Device Regulations 3
E Cloud Services for Medical Devices with CE Mark EU Medical Device Regulations 5
Q File Management system in Cloud for Medical Mobile Apps IEC 62304 - Medical Device Software Life Cycle Processes 2
D Can Cloud Data Management resources be qualified? Should they be? Quality Manager and Management Related Issues 3
R Cloud Computing Requirements for Design History Files for Software Medical Devices 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
Marc Anyone here affected by the volcano ash cloud? April 2010 Travel - Hotels, Motels, Planes and Trains 33
Jen Kirley What's procedurally required for "cloud computing"? TS16949 Clause 4.2.4. Records and Data - Quality, Legal and Other Evidence 8
K Definition Point-Cloud Data - Understanding of the term "Point-Cloud Data" Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 3
Marc Biggest Wi-Fi Cloud Is in Rural Oregon After Work and Weekend Discussion Topics 4
R Point Cloud Technology Validation - Point Cloud to CAD model comparisons for FAIR General Measurement Device and Calibration Topics 2
C Need help in determining applicable clause for an audit finding (based on AS9120B) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
A Information on Process-based Internal Auditing Needed Internal Auditing 6
M ADME data- substances based MD EU Medical Device Regulations 0
E Which regulatory framework for an app-based study for research purposes? EU Medical Device Regulations 1
R Does anyone use iQMS for their ISO based document control? Manufacturing and Related Processes 1
C What Theory is MIL-STD-105E Sample Size Code Letters Based on? Quality Tools, Improvement and Analysis 3
N QMS standard for a research based Organisation Quality Manager and Management Related Issues 1
S How many tester quantity we need on the line based on the cycle time and peak volume Manufacturing and Related Processes 3
S Alcohol based cleaner for Food Contact Surface? Food Safety - ISO 22000, HACCP (21 CFR 120) 0
I Excel based Gage R&R VS Minitab calculation Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
P Scenario based risk assessment IEC 27001 - Information Security Management Systems (ISMS) 1
C CBD based products registration in EU and UK EU Medical Device Regulations 4
L Economic Operator based in UK EU Medical Device Regulations 10
C Biologic Evaluation based on ISO 10993-1 EU Medical Device Regulations 2
R Select the 1 Supplier based on the Parts Durability from 6 Supplier Samples using Minitab Using Minitab Software 11
A % of defects on the whole batch based on result from inspection under AQL Level II Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
Ed Panek Does this FDA Requirement Apply to international (not USA) distributors for USA based manufacturing companies? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0

Similar threads

Top Bottom