The Elsmar Cove Wiki More Free Files The Elsmar Cove Forums Discussion Thread Index Post Attachments Listing Failure Modes Services and Solutions to Problems Elsmar cove Forums Main Page Elsmar Cove Home Page
Google
  Web Elsmar.com
*Please be aware that SOME RECENT forum threads may not yet be indexed by Google.

View Full Version : Company and Department Key Performance Index (KPI)


r29728
21st August 2008, 04:13 AM
we are a engine manufacturing company, we are now draft the company and department KPI for 2009, and we have following department:
1. Sales n Marketing
2. R&D
3. Purchasing
4. Quality
5. Manufacturing (Machining n Assembly)
6. HR
7. Finance
8. ICT

Is there anybody could give me some comments or example on this? thanks in advance! :agree::agree:

prototyper
21st August 2008, 04:50 AM
KPI's are a measure of your business performance and will be specific to your business. You should be measuring things which will drive improvement and therefore give value added to the business.

Look at each department and decide what can be measured and monitored by that department. Only apply KPI's where you feel they are appropriate.

Some examples are:- customer concerns, right first time, overall equipment efficiency, raw efficiency, planned preventative maintenance (complete on time), machine down time, scrap, rework, absenteeism, training, on time delivery, consumables costs, etc.

Service industries will have totally different measures to manufacturing and a small job shop will have different measures to a large factory with dedicated production lines.

You decide, what can be measured and will it benefit the business!

Geoff Cotton
21st August 2008, 06:01 AM
All the areas suggested by prototyper are good area's to go looking.

Make sure you agree the KPI's and the measurement method with the process owner, also ensure there is a set date each month for the data to be "delivered" to the one compiling the overall picture.

Keep the KPI's to a minimum and concise....... distinguish between whats a KPI and what is a "support measure", this can sometimes be difficult so try using a SIPOC document.

I have found its best to display the KPI's graphicaly all on one sheet if you can, make the kpi's green that are showing improving trends, yellow if they are not moving, and red if they have adverse trends. Make sure the process owner has action plans for the adverse trends.

Hope this helps.

Ajit Basrur
21st August 2008, 07:12 AM
we are a engine manufacturing company, we are now draft the company and department KPI for 2009, and we have following department:
1. Sales n Marketing
2. R&D
3. Purchasing
4. Quality
5. Manufacturing (Machining n Assembly)
6. HR
7. Finance
8. ICT

Is there anybody could give me some comments or example on this? thanks in advance! :agree::agree:

I am totally against picking up KPIs and make them applicable within your organization. Pl remember that KPI (Key Performance Indicator) helps to let you know if your company is succeeding / failing with respect to the "Mission".

KPI also direct and prioritise behavior towards achievement of the Company vision, goals and objectives. So when you prepare your KPIs, have these things in mind.

For eg.

Quality could have KPIs targetted towards Product Quality (perecent rejects, Customer Returns, Internal audit observations, percent deviations etc.)

HR could have KPIs towards Training hours, Employee turnover rate

Manufacturing could have KPIs towards Machine downtime, percent yields at different stages, production actual vs planned.

Sales and Marketing could have KPIs like number of repeat customers, percent successful calls, delivery schedules etc.

and so on .........

r29728
21st August 2008, 09:47 PM
Thanks for all of your inputs, is there anybody could provide me an example for my reference? our company has heavy diesel engine manufacturing n sales function, the volume is above 100K per year......thanks for your help...

Ajit Basrur
21st August 2008, 10:20 PM
I thought I gave some examples in my earlier post :notme:

harry
21st August 2008, 10:39 PM
KPIs (Key Process Indicators) are those things that are measured in order to confirm that what you expect the process to do is actually happening. If you have trouble determining what to measure, it means that you don't understand the process, or what it's supposed to accomplish, which in turn means that you shouldn't be trying to measure anything. What is your management review process supposed to do? Other than it being a requirement of the standard, why do you do it? Answer those questions and you'll know what to measure.

This was Jim's answer to a post on KPI's for which I am in full agreement.

This subject is often misunderstood. You have to understand your own operations, what you measure or monitor, what you or your management wants or attempting to achieve. We don't know and therefore can't tell you and Ajit had given some examples in his post.

If you desire to understand this subject further, this article may help (http://www.ibisassoc.co.uk/key-performance-indicators.htm).

vaniphilip
26th August 2008, 03:46 AM
Hi Good Day.


HR could have KPIs towards Training hours, Employee turnover rate

This is the common KPI for HR.May be you can add on EXternal Training for your workers .

Cheers.:cool:

Patricia Ravanello
26th August 2008, 02:21 PM
Thanks for all of your inputs, is there anybody could provide me an example for my reference? our company has heavy diesel engine manufacturing n sales function, the volume is above 100K per year......thanks for your help...

I don't remember where this list is from, so I can't provide appropriate acknowledgement, but it's a veritable smorgasbord of metrics grouped by Department/functional area...Still, each process owner has to decide which are appropriate or meaningful in ensuring the effectiveness and efficiency of their process(es).

In reading through these you'll find that many are metrics which might be used to help identify the root cause of specific problems, and once the solution is identified, the need to monitor and measure is removed, or resources are directed elsewhere. Others are metrics which require continuous monitoring and measurement...


Accounting/Financial Measures

1. Percent of late reports
2. Percent of errors in reports
3. Errors in input to Information Services
4. Errors reported by outside auditors
5. Percent of input errors detected
6. Number of complaints by users
7. Number of hours per week correcting or changing documents
8. Number of complaints about inefficiencies or excessive paper
9. Amount of time spent appraising / correcting input errors
10. Payroll processing time
11. Percent of errors in payroll
12. Length of time to prepare and send a bill
13. Length of time billed and not received
14. Number of final accounting Jobs rerun
15. Number of equipment sales miscoded
16. Amount of intracompany accounting bill back activity
17. Time spent correcting erroneous inputs
18. Number of open items
19. Percent of deviations from cash plan
20. Percent discrepancy in Material Review Board (MRB) and line scrap reports
21. Travel expense accounts processed in three days
22. Percent of advances outstanding
23. Percent data entry errors in accounts payable and general ledger
24. Credit turnaround time
25. Machine billing turnaround time
26. Percent of shipments requiring more than one attempt to invoice
27. Number of untimely supplier invoices processed
28. Average number of days from receipt to processing
29. Percent error in budget predictions
30. Computer rerun time due to input errors
31. Computer program change cost
32. Percent of financial reports delivered on schedule
33. Number of record errors per employee
34. Percent of error free vouchers
35. Percent of bills paid so company gets price break
36. Percent of errors in checks
37. Entry errors per week
38. Number of errors found by outside auditors
39. Number of errors in financial reports
40. Percent of errors in travel advancement records
41. Percent of errors in expense accounts detected by auditors


Facilities Measurements

1. Percent of facilities on schedule
2. Percent of manufacturing time lost due to bad layouts
3. Percent of error in time estimates
4. Percent of error in purchase requests
5. Hours lost due to equipment downtown
6. Scrap and rework due to calibration errors
7. Repeat call hours for the same problem
8. Change to layout
9. Percent deviation from budget
10. Maintenance cost to equipment cost
11. Percent variation to cost estimates
12. Number of unscheduled maintenance calls
13. Number of hours used on unscheduled maintenance
14. Number of hours used on scheduled maintenance
15. Percent of equipment maintained on schedule
16. Percent of equipment overdue for calibration
17. Accuracy of assets report
18. Percent of total floor space devoted to storage
19. Number of industrial design completions past due
20. Number of mechanical / functional errors in Industrial design artwork
21. Number of errors found after construction had been accepted by the company
22. Percent of engineering action requests accepted

Information Systems

1. Keypunch errors per day
2. Input correction on CRT
3. Reruns caused by operator error
4. Percent of reports delivered on schedule
5. Errors per thousand lines of code
6. Number of changes after the program is coded
7. Percent of time required to debug programs
8. Rework costs resulting from computer program
9. Number of cost estimates revised
10. Percent error in forecast
11. Percent error in lines of code required
12. Number of coding errors found during formal testing
13. Number of test case errors
14. Number of test case runs before success
15. Number of revisions to plan
16. Number of documentation errors
17. Number of revisions to program objectives
18. Number of errors found after formal test
19. Number of error free programs delivered to customer
20. Number of process step errors before a correct package is ready
21. Number of revisions to checkpoint plan
22. Number of changes to customer requirements
23. Percent of programs not flow diagrammed
24. Percent of customer problems not corrected per schedule
25. Percent of problems uncovered before design release
26. Percent change in customer satisfaction survey
27. Percent of defect free artwork
28. System availability
29. Terminal response time
30. Mean time between system initial program loadings (IPL)
31. Mean time between system repairs
32. Time before help calls are answered

Management Measurements

1. Security violations per year
2. Percent variation from budget
3. Percent of target dates missed
4. Percent of personnel turnover rate
5. Percent increase in output per employee
6. Percent absenteeism
7. Percent error in planning estimates
8. Percent of output delivered on schedule
9. Percent of employees promoted to better jobs
10. Department morale index
11. Percent of meetings that start on schedule
12. Percent of employee time spent on first time output
13. Number of job improvement ideas per employee
14. Dollars saved per employee due to new ideas and/or methods
15. Ratio of direct to indirect employees
16. Increased percent of market
17. Return on investment
18. Percent of appraisals done on schedule
19. Percent of changes to project equipment required
20. Normal appraisal distribution
21. Percent of employee output that is measured
22. Number of grievances per month
23. Number of open doors per month
24. Percent of professional employees active in professional societies
25. Percent of managers active in community activities
26. Number of security violations per month
27. Percent of time program plans are met
28. Improvement in opinion surveys
29. Percent of employees who can detect and repair their own errors
30. Percent of delinquent suggestions
31. Percent of documents that require two management signatures
32. Percent error in personnel records
33. Percent of time cards signed by managers that have errors on them
34. Percent of employees taking higher education
35. Number of damaged equipment and property reports
36. Warranty costs
37. Scrap and rework costs
38. Cost of poor quality
39. Number of employees dropping out of classes
40. Number of decisions made by higher level management than required by procedures
41. Improvement in customer satisfaction survey
42. Volumes actual versus plan
43. Revenue actual versus plan
44. Number of formal reviews before plans are approved
45. Number of procedures with fewer than three acronyms and abbreviations
46. Percent of procedures less than 10 pages
47. Percent of employees active in improvement teams
48. Number of hours per year of career and skill development training per employee
49. Number of user complaints per month
50. Number of variances in capital spending
51. Percent revenue / expense ratio below plan
52. Percent of executive interviews with employees
53. Percent of departments with disaster recovery plans
54. Percent of appraisals with quality as a line Item that makes up more than 30 percent of the evaluation
55. Percent of employees with development plans
56. Revenue generated over strategic period
57. Number of iterations of strategic plan
58. Number of employees participating in cost effectiveness
59. Data integrity
60. Result of peer reviews
61. Number of tasks for which actual time exceeded estimated time


Shipping Measurements

1. Complaints on shipping damage
2. Percent of parts not packed to required specifications
3. Percent of output that meets customer orders and engineering specifications
4. Scrap and rework cost
5. Suggestions per employee
6. Percent of jobs that meet cost
7. Percent of jobs that meet schedule
8. Percent of product defect free at measurement operations
9. Percent of employees trained to do the job they are working on
10. Accidents per month
11. Performance against standards
12. Percent of utilities left improperly running at end of shift
13. Percent unplanned overtime
14. Number of security violations per month
15. Percent of time log book filled out correctly
16. Time and/or claiming errors per week
17. Time between errors at each operation
18. Errors per 100,000 solder connections
19. Labor utilization Index
20. Percent of operators certified to do their job
21. Percent of shipping errors
22. Defects during warranty period
23. Replacement parts defect rates
24. Percent of products defective at final test
25. Percent of control charts maintained correctly
26. Percent of invalid test data
27. Percent of shipments below plan
28. Percent of daily reports in by 7:00 a.m.
29. Percent of late shipments
30. Percent of products error free at final test



Personnel Measurements

1. Percent of employees who leave during the first year
2. Number of days to answer suggestions
3. Number of suggestions resubmitted and approved
4. Personnel cost per employee
5. Cost per new employee
6. Turnover rate due to poor performance
7. Number of grievances per month
8. Percent of employment requests filled on schedule
9. Number of days to fill an employment request
10. Management evaluation of management education courses
11. Time to process an applicant
12. Average time a visitor spends in lobby
13. Time to get security clearance
14. Time to process insurance claims
15. Percent of employees participating in company activities
16. Opinion survey ratings
17. Percent of complaints about salary
18. Percent of personnel problems handled by employees' managers
19. Percent of employees participating in voluntary health screening
20. Percent of offers accepted
21. Percent of retirees contacted yearly by phone
22. Percent of training classes evaluated excellent
23. Percent deviation to resource plan
24. Wait time in medical department
25. Number of days to respond to applicant
26. Percent of promotions and management changes publicized
27. Percent of error free newsletters

Purchasing and Materials Management

1. Percent of discount orders by consolidating
2. Errors per purchase order
3. Number of orders received with no purchase order
4. Routing and rate errors per shipment
5. Percent of supplies delivered on schedule
6. Percent decrease in parts costs
7. Expediters per direct employees
8. Number of items on the hot list
9. Percent of suppliers with 100 percent lot acceptance for one year
10. Stock costs
11. Labor hours per $10,000 purchases
12. Purchase order cycle time
13. Number of times per year line stopped due to lack of supplier parts
14. Supplier parts scrapped due to engineering changes
15. Percent of parts with two or more suppliers
16. Average time to fill emergency orders
17. Average time to replace rejected lots with good parts
18. Parts cost per total costs
19. Percent of lots received on line late
20. Actual purchased materials cost per budgeted cost
21. Time to answer customer complaints
22. Percent of phone calls dialed correctly
23. Percent of purchase orders resumed due to errors or incomplete description
24. Percent of defect free supplier model parts
25. Percent projected cost reductions missed
26. Time required to process equipment purchase orders
27. Cost of rush shipments
28. Number of items billed but not received

Health and Safety

1. Percent of clearance errors
2. Time to get clearance
3. Percent of security violations
4. Percent of documents classified incorrectly
5. Security violations per audit
6. Percent of audits conducted on schedule
7. Percent of safety equipment checked per schedule
8. Number of safety problems identified by management versus total safety problems identified
9. Safety accidents per 100,000 hours worked
10. Safety violations by department
11. Number of safety suggestions
12. Percent of sensitive parts located

Engineering

1. Percent of drafting errors per print
2. Percent of prints released on schedule
3. Percent of errors in cost estimates
4. Number of times a print is changed
5. Number of off specs approved
6. Simulation accuracy
7. Accuracy of advance materials list
8. Cost of input errors to the computer
9. How well product meets customer expectations
10. Field Performance of product
11. Percent of error free designs
12. Percent of errors found during design review
13. Percent of repeat problems corrected
14. Time to correct a problem
15. Time required to make an engineering change
16. Cost of engineering changes per month
17. Percent of reports with errors in them
18. Data recording errors per month
19. Percent of evaluations that meet engineering objectives
20. Percent of special quotations that are successful
21. Percentage of test plans that are changed
22. Percent of meetings starting on schedule
23. Spare parts cost after warranty
24. Number of meetings held per quarter where quality and defect prevention were the main subject
25. Person months per released print
26. Percent of total problems found by diagnostics as released
27. Customer cost per life of output delivered
28. Number of problems that were also encountered in previous products
29. Cycle time to correct a customer problem
30. Number of errors in publications reported from the plant and field
31. Number of products that pass independent evaluation error free
32. Number of missed shipments of prototypes
33. Number of unsuccessful pre analyses
34. Number of off specs accepted
35. Percent of requests for engineering action open for more than two weeks
36. Number of days late to pre analysis
37. Number of restarts of evaluations and tests
38. Effectiveness of regression tests
39. Number of days for the release cycle
40. Percent of corrective action schedules missed
41. Percent of bills of material that are released in error

Process/Industrial Engineering

1. Percent of facilities on schedule
2. Percent of manufacturing time lost due to bad layouts
3. Percent of error in time estimates
4. Percent of error in purchase requests
5. Hours lost due to equipment downtime
6. Scrap and rework due to calibration errors
7. Repeat call hours for the same problems
8. Changes to layout
9. Percent deviation from budget
10. Maintenance cost to equipment cost
11. Percent variation to cost estimates
12. Number of unscheduled maintenance calls
13. Number of hours used on unscheduled maintenance
14. Number of hours used on scheduled maintenance
15. Percent of equipment maintained on schedule
16. Percent of equipment overdue for calibration
17. Accuracy of assets report
18. Percent of total floor space devoted to storage
19. Number of industrial design completions past due
20. Number of mechanical / functional errors in industrial design artwork
21. Number of errors found after construction had been accepted by the company
22. Percent of engineering action requests accepted

Forecasting Measurements

1. Number of upward pricing revisions per year
2. Number of project plans that meet schedule, price, and quality
3. Percent error in sales forecasts
4. Number of forecasting assumption errors
5. Number of changes in product schedules

Legal

1. Response time on request for legal opinion
2. Time to prepare patent claims
3. Percent of cases lost

Manufacturing and Test Engineering

1. Percent of process operation where sigma limit is within engineering specification
2. Percent of tools that fail certification
3. Percent of tools that are reworked due to design errors
4. Number of process changes per operation due to errors
5. In process yields
6. Percent error in manufacturing costs
7. Time required to solve a problem
8. Number of delays because process instructions are wrong or not available
9. Labor utilization index
10. Percent error in test equipment and tooling budget
11. Number of errors in operator training documentation
12. Percent of errors that escape the operator's detection
13. Percent of testers that fail certification
14. Percent error in yield projections
15. Percent error in output product quality
16. Asset utilization
17. Percent of designed experiments needing revision
18. Percent of changes to process specifications during process design review
19. Percent of equipment ready for production on schedule
20. Percent of meetings starting on schedule
21. Percent of drafting errors found by checkers
22. Percent of manufacturing used to screen products
23. Number of problems that the test equipment cannot detect during manufacturing cycle
24. Percent correlation between testers
25. Number of waivers to manufacturing procedures
26. Percent of tools and test equipment delivered on schedule
27. Percent of tools and test equipment on change 1 level control
28. Percent functional test coverage of products
29. Percent projected cost reductions missed
30. Percent of action plan schedules missed
31. Equipment utilization

Marketing Measurements

1. Percent of proposals submitted ahead of schedule
2. Cost of sales per total costs
3. Percent error in market forecasts
4. Percent of proposals accepted
5. Percent of quota attained
6. Response time to customer inquiries
7. Inquiries per $10,000 of advertisement
8. Number of new customers
9. Percent of repeat orders
10. Percent of time customer expectations are identified
11. Sales made per call
12. Errors in orders
13. Ratio of marketing expenses to sales
14. Number of new business opportunities identified
15. Errors per contract
16. Percent of time customer expectation changes are identified before impact
17. Man hours per $10,000 sales
18. Percent reduction in residual inventory
19. Percent of customers called as promised
20. Percent of meetings starting on schedule
21. Percent of changed orders
22. Number of complimentary letters
23. Percent of phone numbers correctly dialed
24. Time required to turn in travel expense accounts
25. Number of revisions to market requirement statements per month
26. Percent of bids returned on schedule
27. Percent of customer letters answered in two weeks
28. Number of complaint reports received
29. Percent of complaint reports answered in three days

Production Control

1. Percent of late deliveries
2. Percent of errors in stocking
3. Number of items exceeding shelf life
4. Percent of manufacturing completed on schedule
5. Time required to incorporate engineering changes
6. Percent of errors in purchase requisitions
7. Percent of products that meet customer orders
8. Inventory turnover rate
9. Time that line is down due to assembly shortage
10. Percent of time parts are not in stock when ordered from common parts crib
11. Time of product in shipment
12. Cost of rush shipments
13. Spare parts availability in crib
14. Percent of errors in work in progress records versus audit data
15. Cost of inventory spoilage
16. Number of bill of lading errors not caught in shipping

Quality Assurance

1. Percent error in reliability projections
2. Percent of product that meets customer expectations
3. Time to answer customer complaints
4. Number of customer complaints
5. Number of errors detected during design and process reviews
6. Percent employees active in professional societies
7. Number of audits performed on schedule
8. Percent of quality assurance personnel to total personnel
9. Percent of quality inspectors to manufacturing directs
10. Percent of quality engineers to product and manufacturing engineers
11. Number of engineering changes after design review
12. Number of process changes after process qualification
13. Errors in reports
14. Time to correct a problem
15. Cost of scrap and rework that was not created at the rejected operation
16. Percent of suppliers at 100 percent lot acceptance for one year
17. Percent of lots going directly to stock
18. Percent of problems identified in the field
19. Variations between inspectors doing the same job
20. Percent of reports published on schedule
21. Number of complaints from manufacturing management
22. Percent of field returns correctly analyzed
23. Time to identify and solve problems
24. Percent of laboratory services not completed on schedule
25. Percent of improvement in early detection of major design errors
26. Percent of errors in defect records
27. Number of reject orders not disposition in five days
28. Number of customer calls to report errors
29. Level of customer surveys
30. Number of committed supplier plans in place
31. Percent of correlated test results with suppliers
32. Receiving inspection cycle time
33. Number of requests for corrective action being processed
34. Time required to process a request for corrective action
35. Number of off specs approved
36. Percent of part numbers going directly to stock
37. Number of manufacturing interruptions caused by supplier parts
38. Percent error in predicting customer performance
39. Percent product cost related to appraisal, scrap, and rework
40. Percent skip lot inspection
41. Percent of qualified suppliers
42. Number of problems identified in process


Good Luck!
Patricia Ravanello

QAMMAN
26th August 2008, 03:07 PM
Hi Patricia,

Your list is most timely and valuable to me. I logged on today to do a search on how others approach KPI's and wouldn't you know it, its at the top of the lsit.

I think this must be a spot others are having problems with as well. I think there are some pretty good points made in this thread.

The problem with blaming things on not understanding your processes is that just because a quality manager understands the processes and the need for key indicators doesn't mean upper management understands them.

Sometimes upper management would rather avoid the truth then have it reported (recorded).

thanks again--Rich G.

r29728
27th August 2008, 12:35 AM
I don't remember where this list is from, so I can't provide appropriate acknowledgement, but it's a veritable smorgasbord of metrics grouped by Department/functional area...


Thanks a lot for your inputs....

selsensoy
27th August 2008, 05:00 AM
Hi everyone,

I think Q1 Metrics will be useful for you, you can customize these metrics,
if you have need further information please let me know.:agree1:

rjmsteel
27th August 2008, 02:24 PM
Patricia,

That is quite an impressive metrics list with a good amount of inputs which could be useful. However I must say that in a smaller size metals processing facility, a fair amount of the metric samples for measure would Scare half of our employees into some sense of paranoia or related emotions.

On a more cheerful note: From my perspective your Management Operating/ Metric Capture System is still - by far - THE BEST :agree1: most logical; user friendly system, on the Cove, including:

The clear improvements to the Access database Issues System.
The really cool Document Tree example; etc.

Thanks very much,

:bigwave:
Rich M.
Metals Processing

Andrej
27th August 2008, 03:23 PM
Hi Patricia,

The list you have published is from the book:

EXCELLENCE THE IBM WAY, by H. James Harrington, published by ASQC Quality Press, 1988.

Andrej

Patricia Ravanello
27th August 2008, 04:13 PM
Hi Patricia,

The list you have published is from the book:

EXCELLENCE THE IBM WAY, by H. James Harrington, published by ASQC Quality Press, 1988.

Andrej

:thanks: so much Angrej
...I had no idea where it came from, or if it wasn't one of those "open source" documents that people just keep adding to.

I'm grateful that we are able to acknowledge the source.

:thanx: Patricia

ngohrvinet
19th June 2009, 11:37 PM
:thanks: so much Angrej
...I had no idea where it came from, or if it wasn't one of those "open source" documents that people just keep adding to.

I'm grateful that we are able to acknowledge the source.

:thanx: Patricia

Hi Pachicia

Your KPIs is very good.

we can use BSC and KPIs together?????

This link of marketing kpi (http://www.humanresources.hrvinet.com/marketing-kpi)that i think also useful.
rgs

WisdomseekerSC
23rd June 2009, 11:31 AM
This thread is wonderful....what is the difference between KPI and quality objectives? Does anyone have specific examples for an R&D department that also serves as specialty contract manufacturing site for medical components?

Jim Wynne
23rd June 2009, 12:25 PM
This thread is wonderful....what is the difference between KPI and quality objectives? Does anyone have specific examples for an R&D department that also serves as specialty contract manufacturing site for medical components?

"KPI" stands for Key Performance Indicator, not "Index." At any rate, quality objectives are goals (targets) and KPIs are the way the progress towards the target is measured. Objectives and KPIs are derived from the expectations for processes. If you know what's expected, you can form an objective and if the objective is properly identified, you will know what to measure. If you're unsure about what the objective(s) should be, or what to measure, it means that the expectations have not been properly identified.