KPIs or Metrics to Measure a New Complaint Handling Process

#1
We are working on developing and implementing a new medical device complaint handling process to assure that complaints are collected at the manufacturing sites, evaluated by them for the potential need to submit an MDR, and then forwarding the information to a centralized team to make the final MDR decision and to submit the MDR if needed.

I'm trying to define some KPIs or metrics that would help assure that the manufacturing sites continue to process complaints as required, and in a timely fashion. For example, if a staff member left who was responsible for reviewing and forwarding the complaint to the central office, is that still happening or is there a sudden drop-off in submissions.

Some thoughts I've had follow below. Can you suggest others that would help monitor the process?
  • Total number of Open complaints at month-end. Monitor average - maybe trigger points if we fall below a "normal" lower limit.
  • Break down by state of complaint record. (Received, Waiting Return, Under Investigation, etc.)
  • Total number of Closed complaints at month-end. Again, based on average volume is level of closed records "normal".
  • Average days to closure for records closed in the month.
  • Number of days from Aware Date to date the complaint was opened.
  • Number of days from Date Opened to 1st, 2nd, and 3rd attempts to have product returned for evaluation.
  • Number of days from date product received to date the investigation was completed. Are investigators, R&D, Engineering being timely with their evaluations?
  • Number of complaints closed without investigation – were rationales to do so valid?
  • Number of complaints miss-categorized upon initial intake – meaning how many were thought to be a minor issue (no harm) but further investigation revealed more serious situation and potential need for MDR.
 
Elsmar Forum Sponsor

John Predmore

Quite Involved in Discussions
#2
One approach which might generate fresh insights - consider a "value stream map" of how a complaint moves through your process from generation to resolution. Decide what aspects of the sequence you are interested in and how you might measure and monitor those in meaningful ways. Lean practitioners think about systems using two types of measures - speed by which items move through the sequence, and buffers (inventory) where items get held up. With complaints, do you always want FIFO or LIFO, or how is that best handled? Are there reject/rework loops for complaints? - you brought up mis-classification of majors as minors, for example.

Depending on how far upstream you want to go, you might also track aspects of the sequence like time between sale and problem noticed, or from date product is put into service to when complaint is received. Do your results vary significantly by the source of complaint, call center versus website?

I often find thinking in analogies is helpful for creative thinking, and a 10,000-foot view might lead you to consider aspects of the system which are important but less obvious.
 

RoxaneB

Super Moderator
Super Moderator
#3
My concern with metrics that are "number of..." is that they give little to no context. What if the volume of product being shipped increases? Odds are the "number of complaints" will increase. To be truly meaningful, I like to show both the number and the %...that's the story behind the process metrics.

Your list is rather long, in my opinion, and many of the items are more like control-items, not KPIs. For example, the "# of days..." - this speaks more to process flow and responsibility and maybe even resources. But are they truly KEY performance indicators.

Ask yourself...rather, ask the team...what are you trying to achieve with this new complaint process? Why is it new? Why was the old way considered obsolete? From that discussion, you may discover some true KPIs, while everything else is a driver towards those KEY metrics.
 

Ron Rompen

Trusted Information Resource
#4
I agree with Roxane, particularly about showing metrics as a percentage of ........, rather than just a number. Makes it much more relevant to what is actually happening.
I also agree that your list is far too long - pick out the 3 most important (to YOU and your company) metrics, and monitor those. After 6 months, revisit the list, and see if it needs to change. Your metrics don't need to be static - they should reflect what is really happening, and what you identify as important.
 
#5
Thanks for the input - great suggestions! I, too, was worried about the number of measurements (paralysis by analysis) and will work to pare them down from the team's first brainstorming session.
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#6
What matters to your customer? What would effectiveness and efficiency look like in customer complaint handling? What does failure look like?

Customer satisfaction can be maddeningly hard to measure, possibly more so while even more importantly so during complaint handling because the customer is already dissatisfied. This is your chance to keep them or else. Focus on what is needed to do that.

Then have a follow up contact with the customer to learn if it worked and how they feel about the interaction and problem solving process. Use that feedback to understand opportunities for improvement and make them.
 
Thread starter Similar threads Forum Replies Date
R Monitor production quality - Internal KPIs Manufacturing and Related Processes 5
C Calculating your KPIs ISO 13485:2016 - Medical Device Quality Management Systems 7
qualprod Proposed Kpis ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
P Received a minor for not having good measureables/goals. Need help with KPIs. IATF 16949 - Automotive Quality Systems Standard 52
K IATF16949 inputs/outputs & KPIs Internal Auditing 3
Q Corrective Action - Assigning KPIs to Non-Guilty Processes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
Q Suggested KPIs for a Manufacturing Production Business ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
Q How to align a Business Strategy to Operative KPIs ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
Q KPIs - Lead and Lag Indicators - ISO 9001:2015 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
A If 4.1 is a process, how to assign KPIs for this process? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
Buckyb Are KPIs a requirement to determine Process Effectiveness? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 14
J Lean Appropriate KPIs for a Distribution Centre Lean in Manufacturing and Service Industries 1
P New to AS9100 Rev. C - Setting KPIs and FAI Reports AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 14
S Process Map and defining KPIs Misc. Quality Assurance and Business Systems Related Topics 5
K Can you Suggest KPIs for a Business Process Improvement Team? Pls. Benchmarking 4
R Document Control Person - Setting KPIs, Development Plan, and Career Path Quality Manager and Management Related Issues 6
R Key Process Indicators (KPIs) for ISO 9001:2000-certified Service Organization ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
R KPIs (Key Performance Indicators) for Telecom Operators TL 9000 Telecommunications Standard and QuEST 1
B Is there a necessity of two sets of KPIs for Delivery Process Quality Manager and Management Related Issues 15
M First Draft of Policy, Objectives and KPIs - Your feedback appreciated ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
I Are KPIs (Key Performance Indicators) considered as Quality Objectives? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
A Aircraft Maintenance Quality & Safety KPIs - Industry Trend or Database EASA and JAA Aviation Standards and Requirements 3
A Aircraft Maintenance Quality & Safety KPIs - Industry Trend or Database Quality Manager and Management Related Issues 2
R Quality for Telco Companies - Quality in Customer Experiences & KPIs Quality Tools, Improvement and Analysis 5
0 Problem with setting up KPIs via Excel Pivot Table Misc. Quality Assurance and Business Systems Related Topics 6
Q Training KPIs (Key Process Indicators) Training - Internal, External, Online and Distance Learning 8
A Department KPIs - Plant, Machineries and vehicles (PMV) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
Jim Wynne "Business Coach" Touts Basing Pay on KPIs Philosophy, Gurus, Innovation and Evolution 43
S Differences Between KPIs and Objectives/Targets ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
S Do Key Performance Indicators (KPIs) really make a difference in the QMS? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
D Key Performance Indicators (KPIs) in the Construction Industry ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
G KPIs required by TOP Management for strategic planning Statistical Analysis Tools, Techniques and SPC 2
D Key Performance Indicators (KPIs) book - Reviews and suggestions wanted Statistical Analysis Tools, Techniques and SPC 9
S KPIs (Key Performance Indicators) and Risk Management Process (ISO14971:2007) ISO 14971 - Medical Device Risk Management 5
B Defining Physical Asset Maintenance KPIs (Key Performance Indicators) Benchmarking 6
Ajit Basrur Customer Satisfaction Survey Content - Which KPIs employed? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
J Key Performance Indicators (KPIs) for Management Review process Management Review Meetings and related Processes 37
D Engineering Change Order (ECO) metrics to track ISO 13485:2016 - Medical Device Quality Management Systems 6
M Quality Metrics Questions Lean in Manufacturing and Service Industries 2
CPhelan Best metrics for monitoring low volume (<1000) parts per vendors Supply Chain Security Management Systems 1
G Defining performance metrics for DFMA implementation Design and Development of Products and Processes 2
W Core Process Metrics AS9100D for Product Planning? Quotes and sales? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
R Visually impacting graphics for Nonconformance status metrics for Management Report Nonconformance and Corrective Action 5
Ed Panek User Feedback both negative and positive and acting upon those metrics 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
andika_untoro What are a good Critical to Quality (CTQ) metrics for a biotech manufacturing production? Lean in Manufacturing and Service Industries 0
D Need Resource for Industry Data - standard metrics for machine shops Lean in Manufacturing and Service Industries 3
V Information Technology Dashboard - IT Department Performance Metrics IEC 27001 - Information Security Management Systems (ISMS) 2
C What types of Calibration Metrics are Collected General Measurement Device and Calibration Topics 2
A Reporting Corrective Action System Performance Metrics Quality Tools, Improvement and Analysis 1
M PPAP Metrics that visually displays the progress through charts and/or graphs APQP and PPAP 7

Similar threads

Top Bottom