There is a disconnect between best practice for final testing, such as field testing or beta testing, and what can be done with medical device software. For instance, one official definition of beta-testing is that it is done in the field with pre-release software with a limited set of customers/users. How does this fit with a typical design control scenario, where the software is released as a product after extensive verification/validation but because of its complexity requires clinical use to learn about potential issues/problems? Is a small clinical trial a valid method? Does anyone have any ideas or experience with FDA regarding this type of situation?
Beta testing, as described for most software, isn?t done the same way for medical device software. What one would call beta testing for a non-medical device software, is called PQ for medical device software (it?s been called other things also, such as UAT or FAT testing, depending on what industry you?re in). When developing medical device software, after OQ testing the user testing is done during PQ testing, which is testing done by users, or potential users of the system. Once PQ is completed per its? protocol and is determined it is fit for use, the system can be cut over for a process validation in the ?live? environment. At that time a clinical trial can be done.
Clinical trials usually aren?t done to learn about potential issues/problems. They?re interventional and observational studies done based on research plans or protocols which add to medical knowledge and generate safety and efficacy data. In the medical device world, a clinical trial wouldn?t be done unless the software has first been verified and validated. You?re asking people to use a system that hasn?t been fully tested on humans who could be hurt or worse if the system fails; the liability issues would be tremendous. I?ve dealt with this in the past and had to deal with it on the current project I?m on. The company developing the device, their first medical device ever and who have already stated they won?t be doing any clinical trials, wanted to put two units in the setting they will be sold in, before the final verification and validation were done, and run them to see how the users liked them (they have also had extensive interaction with potential users, focus groups, etc. already in order to define their requirements and get feedback on the system). They can?t do this, as 1) it?s a clinical trial which they said they weren?t going to do and don?t have an IDE for; 2) they?ve already had extensive feedback from their focus groups, which are made up of potential customers for the device; and 3) if someone were to get hurt during these ?clinical trials? using a system whose software has not been fully verified and validated, that someone could end up owning this company.
That said, the FDA does allow for feasibility studies, under an IDE, and what they call ?just in time? testing. The state ?And in these cases early clinical use of the device in a limited number of subjects is needed to provide initial insights into clinical safety and device function, inform subsequent clinical and non-clinical testing and/or improve device performance through iteration before finalizing the design. A guiding principle of the early feasibility study is a term we call just-in-time testing and this is a term that we borrowed and modified from industrial production. Just-in-time testing applies to the type and timing of non-clinical testing needed to justify study initiation.? So the FDA allows feasibility studies to be done, the results of which may end up altering the design of the device, but they still require some testing to be done prior to that, and an IDE. FDA states ?An investigational device exemption (IDE) allows the investigational device to be used in a clinical study in order to collect safety and effectiveness data. Clinical studies are most often conducted to support a PMA. Only a small percentage of 510(k)'s require clinical data to support the application. Investigational use also includes clinical evaluation of certain modifications or new intended uses of legally marketed devices. All clinical evaluations of investigational devices, unless exempt, must have an approved IDE before the study is initiated.?
Here?s some links that you might find useful; the FDA comments above were taken from these links.
http://clinicaltrials.gov/ct2/about-studies/learn#WhatIs
http://www.fda.gov/Training/CDRHLearn/ucm372150.htm
http://www.fda.gov/medicaldevices/d...investigationaldeviceexemptionide/default.htm