What is your favorite Usability Risk Analysis tool?

#1
Good Afternoon,

I was wondering what everyone's favorite risk tool, or what risk tool your company uses, for assessing usability failures. Every company I have worked for pushes the uFMEA, which does not technically cover normal use failure modes.

It becomes particularly muddy, in regards to the IVD industry, since tests typically have a method associate with them, which is technically a process that a user executes, so do you perform process FMEA or use FMEA or even design FMEA since the method is technically part of the test design?

In my past, I have used uFMEAs formatted with columns to provide input to the usability process. I have also performed process FMEAs with "use error" as a cause. My workaround for normal use vs single fault conditions is to put "normal use" as a failure cause. All of these were bandaid fixes to fit into the quality systems I was hired into. I now have the opportunity to start fresh and would like to hear what other people have used for usability in their past and present or even what you believe could be the next future state of usability risk assessments.
 
Elsmar Forum Sponsor

yodon

Leader
Super Moderator
#2
We use UFMEAs. I'm having trouble understanding what a "normal use failure mode" would be. Can you provide some examples?
 

Tidge

Trusted Information Resource
#3
Usability informs two fundamental aspects of my risk analysis.

1. The Hazard Analysis (top level RA document) is first divided into Hazards (separate sections), but each line of the HA is tied to one (or more) Use Cases.​

Notice that this is explicitly tying the product I intend to make with the circumstances of what it will be prescribed for. The same hazards may present different risk ratings depending on use scenarios. Failure modes are not considered except in the context of how they might inform the (pre-control) ratings (and individual lines of analysis).

2. I use a UFMEA to explicitly to analyze each use case. This is akin to dividing a PFMEA into process steps, or a DFMEA into system elements.​
Severity ratings for individual UFMEA lines of analysis derive from (and are linked to) identified lines in the Hazard Analysis. This is very often a many-to-many relationship, as the same hazards can present themselves in a varity of use scenarios. Within the UFMEA is where we bluntly consider failure modes (informed by mental models, user physiological factors, skill level, etc.) and consider the implementation of risk controls.

There are some subtleties with this approach. Allow me to try to describe one...

One is that in a "Verification and Validation" development approach, the risk controls for failure modes are initially subjected to implementation (identified by verification of implementation, VoI) and then assessed for effectiveness (identified by verification of effectiveness, VoE). It is common that within a DFMEA that the VoE is simply some single test (possibly challenging boundaries, but often not) during what is called design verification. Within a PFMEA, it is more likely that (with the help of a good process engineer) that there could be an actual process validation to serve as VoE for risk controls.

When it comes to Usability, just about every professionally worth their salt will jump to "User Validation" and start thinking about efforts in terms of "62366" (or "HE75" or some such) and effectively move themselves into a headspace where they may forget that the UFMEA is a subordinate document to the Hazard Analysis and so it needs attention prior to user validation (which is supposed to have summative testing on "finished designs"). This pre-validation attention ought to be motivating the final user validation... and I think an argument can be made that by referencing appropriate formative testing of a design (and documenting such in a UFMEA) it is possible that the UFMEA risk ratings could be low enough that certain possible failure modes wouldn't have to be included in user validation.

It is (in my experience) most appropriate to close the loop on Usability Risks by appropriately referencing the summative testing as VoE for final UFMEA risk ratings. My own preference is that the top level risk document (Hazard Analysis) NOT refer directly to analysis in UFMEA, except via that initial link to the use cases. I definitely think the link is there, it's just my feeling that it is silly to pretend that the design team has any measure of control over the users. A personal, bold statement follows: The design team is obligated to understand how the device is used, summative testing is pretty much just a confirmatory analysis that there are not elements of risk that were unaccounted (or improperly accounted) for.
 
#4
Normal use hazards would be a better way to word it since these is no fault occurring. It is the hazards that arrive from using your device as intended, so no fault conditions. An FMEA assesses single fault failures.

It is one of the gaps between an FMEA and 14971 risk analysis.
 

Ahang

Registered
#5
I need a sample or example of real usability engineering file please. I've read the related standards and now I think it is better to study a real case. I know that these would be confidential documents of each organization, but some withdrawn ones may be available.
 

Enghabashy

Involved In Discussions
#6
-PFMEA , DFMEA, poka yoke/error proof , MSA , fault tree & fishbone ,--etc, -- all could be tools of technical risk assessment & preventive measures
-The hazards could be approach occupational risk assessment issues rather than technical assessment of the processes & products ;
- The other business risk assessment should be considered also ; i.e.: COVID 19 & political issues as Ukraine could be issue risks on supply chain , orders , financial issues ; IT risks , lack of shipments & markets risks ,--etc.
 
Thread starter Similar threads Forum Replies Date
M SemaTech SSQA Standardized Supplier Quality Assessment - my favorite tool ever Manufacturing and Related Processes 1
Ninja Favorite (oft used) sayings... Coffee Break and Water Cooler Discussions 21
Marc Twilight Zone - What is your favorite Episode? Coffee Break and Water Cooler Discussions 10
F What is your favorite software for ISO 9001:2015 Internal Audits? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
bobdoering A Statistician?s Favorite Answer: ?It Depends,? Part 2 Statistical Analysis Tools, Techniques and SPC 2
bobdoering A Statistician?s Favorite Answer: ?It depends,? Part 1 Statistical Analysis Tools, Techniques and SPC 1
G What is your favorite flavor of GUM? General Measurement Device and Calibration Topics 8
QMMike My favorite Dilbert strip Funny Stuff - Jokes and Humour 7
Hershal Favorite Recipes..... Share your favorite recipes Coffee Break and Water Cooler Discussions 1
Stijloor No Traffic Jams Anymore - My Favorite Bank Card Ad. Funny Stuff - Jokes and Humour 0
Kales Veggie Favorite Podcasts Coffee Break and Water Cooler Discussions 1
B Favorite Funny Images.... Funny Stuff - Jokes and Humour 16
GStough What is your favorite source for buying more RAM? After Work and Weekend Discussion Topics 33
D Favorite Classical Music work? After Work and Weekend Discussion Topics 28
ScottK Where's your favorite place for Lunch? Coffee Break and Water Cooler Discussions 36
ScottK What companies have been your favorite customers? (a pleasure to work with) Coffee Break and Water Cooler Discussions 2
N Receiving inspection: Part and Vendor history - your favorite format/application? Records and Data - Quality, Legal and Other Evidence 1
I AFIs Top 100 Film Quotes - What are your favorite movie lines? Coffee Break and Water Cooler Discussions 52
Hershal Favorite haunts - Our favorite feeding toughs and/or watering holes Coffee Break and Water Cooler Discussions 14
C Your Favorite Movie Business Theme Coffee Break and Water Cooler Discussions 12
T Document numbering - What is your favorite identification scheme? Document Control Systems, Procedures, Forms and Templates 31
R Some of my favorite Oxymorons (for Claes) Funny Stuff - Jokes and Humour 21
T What is Your Favorite Format for Systems and Procedures? Document Control Systems, Procedures, Forms and Templates 11
A Keeping the Usability File up to date IEC 62366 - Medical Device Usability Engineering 2
H Regarding Usability Engineering File according to IEC 62366-1 CE Marking (Conformité Européene) / CB Scheme 2
MaHoDie Summative Usability test Human Factors and Ergonomics in Engineering 2
E DESIGN VALIDATION, USABILITY AND CLINICAL EVALUATION request Medical Device and FDA Regulations and Standards News 0
I "Usability" for technical documentation EU Medical Device Regulations 1
Q Summative Usability Evaluation Testing: prior or during Clinical Investigation? Human Factors and Ergonomics in Engineering 10
K Usability for custom made medical devices IEC 62366 - Medical Device Usability Engineering 5
P Human Factors / Usability validation in the time of COVID Human Factors and Ergonomics in Engineering 17
N Usability testing required for FDA IDE (investigational device exemption)? Human Factors and Ergonomics in Engineering 9
V Applicability of IEC 62366-1 usability to resorbable bone substitutes IEC 62366 - Medical Device Usability Engineering 9
pashah Looking for Usability File Template acc. IEC 62366-1 and IEC 60601-1-6 IEC 62366 - Medical Device Usability Engineering 4
C Usability IEC 62366-1:2015 and MDR 2017/745 - Risk based approach IEC 62366 - Medical Device Usability Engineering 1
T MDR article 22 - System of 2 CE products and validation / usability? EU Medical Device Regulations 10
M Informational Understanding Costs And Risks For HFE Usability Studies — Part 1: Testing In-House Medical Device and FDA Regulations and Standards News 0
P Usability/human factors engineering requirement (standard) for IVD medical device Other Medical Device Related Standards 2
P Global medical device human factors/usability requirement IEC 62366 - Medical Device Usability Engineering 3
E Usability Process - Looking for a template (IEC 62366) IEC 62366 - Medical Device Usability Engineering 1
S Usability procedure - looking for a template IEC 62366 - Medical Device Usability Engineering 3
K Usability of ME Equipment - IEC-62366-1:2015 IEC 62366 - Medical Device Usability Engineering 3
M Usability Standard - The difference between IEC 60601-1-6 to IEC 62336 Human Factors and Ergonomics in Engineering 1
V 510(k) vs. CE Marking Summative Usability Testing Expectations IEC 62366 - Medical Device Usability Engineering 4
M IEC 62366-1:2015 Usability Verification / Validation Plan example wanted IEC 62366 - Medical Device Usability Engineering 20
Q EU ROHS and GUI Usability Trial Requirements EU Medical Device Regulations 10
M New MHRA guidance on Human Factors - Usability Engineering IEC 62366 - Medical Device Usability Engineering 1
S IEC 62366 - Usability Test Protocol format Imported Legacy Blogs 0
M Does Medical Device Usability Study = Clinical Trial? Other Medical Device and Orthopedic Related Topics 26
L BSI White Paper on Human Factors/Usability Engineering IEC 62366 - Medical Device Usability Engineering 12

Similar threads

Top Bottom