|
Welcome
to the Elsmar Cove!
ISO
9000 - QS-9000 (now TS 16949) Information Exchange
John Seddon, April 1996
John Seddon is an occupational psychologist, researcher, author and consultant. He is the author of the widely acclaimed "I want you to cheat: the unreasonable guide to service and quality in organisations" and is presently publishing a cd-rom for managers entitled: "Change management thinking". The cd-rom is designed to help managers understand the problems associated with traditional, 'command and control' management thinking and how to take practical action for improvement by treating the organisation as a system. For information telephone (44) -(0)1280 - 822255, fax -822266, http://www.vanguardconsult.co.uk, e-mail: [email protected]
Introduction
In 1993, Vanguard Consulting published the first major study of the impact of ISO 9000 (then BS 5750) on organisation performance (executive summary: appendix 1). The results cast doubt on the claims made for ISO 9000 registration. More recently we have seen the publication of two studies sponsored by organisations with a pecuniary interest in ISO 9000. (see appendices 2 and 3).
A weakness common to all three studies is that they were (in the main) studies of opinion. The reliability and validity of such data will be influenced by respondents' knowledge, perception and emotional investment. The Manchester Business School report (appendix 3) makes claims which go beyond what can be interpreted from the available data.
Following the publication of our 1993 research, we visited five of the companies in the small group which had claimed ISO 9000 registration to be positive in all respects. In every case we found activity which we would argue to be damaging economic performance, which was present specifically because of registration to the Standard (see appendix 4).
To date, there has been no research which teaches us about cause and effect - what are the likely consequences to an organisation if it goes about registration to ISO 9000? Seventeen years have passed since the ISO Standard first appeared (as BS 5750) and we have no knowledge of its impact on organisation performance and competitive position. These three case studies were undertaken to learn more about cause and effect.
In each case, the companies had argued that ISO 9000 registration had been beneficial. Perhaps all registered companies would feel obliged to make such a claim as they have invested the time and effort and have the 'award' to prove their worthiness. The purpose of each case study was to understand what registration had meant in practice and how it affected performance.
While each of the cases is different, they show similarities in their approach to ISO 9000 registration. It is argued that ISO 9000, because of its implicit theory of quality, will lead to common problems in implementation; problems which damage economic performance and which may inhibit managers from ever learning about quality's potential role in improving productivity and competitive position.
For readers who would like to know more about the origins of the Standard, a brief history is provided in appendix 5. Many advocates of the Standard cite its history as though it bestows legitimacy. In this appendix we explore how the Standard's history illuminates where we went wrong.
Case 1
Background
'Tech. Co.' is a sales and distribution organisation based in the UK. All products sold by the company are manufactured by its parent company overseas. The products are high-technology products, they are sold to business and government customers.
Tech. Co. decided to register to ISO 9000 three years ago. The primary reason for seeking registration was that Government customers were insisting on it. It was also felt that the company might achieve some benefits from clarity of working procedures.
Tech. Co. took advice from a consultant. The work to first registration took three years and registration was successful.
Originally the company had seven divisions (organised by product) and each division had its own sales order function. It had recently centralised the sales order functions - in a structural sense - but the people who worked within product divisions still processed orders for the same products. The primary change with centralisation was standardisation of working procedures.
Registration to ISO 9000 meant the creation of nine manuals of procedures. One for each type of product, one called quality and one for the warehouse. The manuals documented what should be done at each stage of an order, nothing had been left out.
The manager of sales administration took the view that sales administration was now controlled, the standard procedures meant that people wouldn't get things wrong. He also viewed salespeople as needing to be controlled - "They won't be able to get away with giving administrators inadequate information and what they provide has to be written down". He also saw advantages in controlling customers: "Because we have controlled procedures we can prove to them what they have ordered if there is any dispute".
Has ISO 9000 contributed to performance?
The first operational problems were felt by sales. The attitude presented to them by administration was 'nothing proceeds without the right paperwork'. Furthermore, because of a new procedure introduced to 'comply' with ISO 9000 (see later), the warehouse would demand that sales 'sign-off' returned goods before they could be made available as inventory.
There had been a cultural tradition of sales being 'heroes', they had treated 'back office' staff work as drudgery, they were always seeking last-minute changes and rarely were accurate with their form-filling. But now administration had been instructed to work exactly to procedures. Some took the view that working to procedures would teach salespeople to pay more attention to putting the right information on their paperwork. Sales behaviour did not change, the same conflicts existed between sales and administration, they were sometimes more acute because administration had a specific stick with which to beat them.
The causes of these conflicts were many, but the one cause which appeared to be having the most influence was the complexity of pricing. Salespeople faced a variety of different circumstances when contracting with customers; the result was a need for flexible administrative practices. The complexity of current practices had been documented in the manuals. It would be unlikely that the administrator would be familiar with all situations and, furthermore, the manual would be unlikely to cover all situations (and would have to be continually up-dated). The consequences were an increasing sense of 'drudgery' amongst administrators and none of the original problems solved.
The sense of drudgery in administration was caused by the way the new procedures had been introduced (it was said that they were 'dumped' on administrative staff with the directive: you are to work to these procedures). The attitude in administration soon became 'work to procedures' (rather than serve customers). Furthermore, it was a perspective which valued working to procedures above finding and removing errors or the causes of errors.
When asked whether the new procedures had improved performance, no-one could say. There had been no collection of data on the performance of the administrative practices prior to the new procedures and hence it was not possible to determine whether things were better or worse.
The administration manager's attitude to customers ("The procedures will enable us to prove what the customer ordered if there is a dispute") was positively dangerous. What matters when the customer disputes, or for any reason wants a change, to an order is that the need is dealt with promptly and courteously. The last thing the customer wants to hear is someone seeking to prove the customer is wrong.
The thinking behind the new 'goods returned' procedure is also revealing. Goods returned stayed at the same level as prior to registration (suggesting no change to the performance of the system). What did change was the way returned goods were handled. ISO 9000 has a requirement to control product which is returned by customers ('control of non-conforming product'). All goods returned by customers were now held in 'quarantine', pending authorisation for release into the warehouse (for re-use).
There had been considerable debate about what to do with returned product. Tech. Co.'s products are sensitive to excesses of temperature and light. If they had been subjected to adverse conditions while on the customers' premises, they would be ruined. If product was tested on return it would be destroyed - the nature of the product precluded testing (or sampling) -any test would result in destruction. The choice was simple: return product to the warehouse or destroy it.
In the event, a compromise was developed. It was argued that product returned within fifteen days would be likely not to have been subjected to adverse conditions. The procedure for returned product required the warehouse supervisor to check date of receipt against date of despatch and, where it fitted the new rule, to send a list of returned product to a director for signature. Signatories were hard to find. When they were available, they were not inclined to think of signing forms from the warehouse as their top priority.
But no-one asked: Why do we have product being returned? - with a view to eradicating it. And, more interestingly, nobody had asked the question: Have we ever had a problem with product being returned and then sent out as another order? The answer, when asked, was no. There had been no need to create this 'quarantine' procedure. It would have been perfectly OK to place all returned goods in inventory. All of the available evidence showed there was not a problem. Tech. Co. had set up a procedure for an uncommon occurrence assuming it to be a common occurrence, in that way problems were now created. The result was losses to the system: at the time of this study, product to the value of half a million pounds was not available for sales.
If they had viewed their organisation as a system, they would have known that returned product had not shown signs of fault and would have taken the view that customers' receiving faulty goods would have been an unlikely occurrence. In the unlikely event of a customer receiving faulty goods, the organisation could respond by engaging in excellent 'repair'. A good way to show customers what matters to you.
There is little that can be said in support of the view that ISO 9000 contributed to organisational performance in this case. It seems the implicit purpose of registration was to create a 'quality system', rather than help the business. The emphasis on written work procedures was entirely unnecessary - a simple flow-chart would have been sufficient (and may have been more use in clarifying purpose). The manuals produced for administration should not have been necessary. If they had a purpose at all they should have been used for job training. Used in the way they were being used only exacerbated the problems between sales and administration.
This was a classic 'bomb factory' approach (see appendix 5) to ISO 9000 implementation. The purpose of this organisation is to sell. How? By creating value for customers, developing loyal customers, attending to customers' needs. Nothing was being done to understand and improve the core processes of achieving the organisation's purpose. ISO 9000 was making achievement of the purpose more difficult and more expensive.
Case 2
Background
'Laserco' is a high-tech manufacturer, employing 150 people. Five years ago, a senior director (now MD) wanted greater 'discipline' in the engineering function. Quality was chosen as a means.
The quality manager wanted to introduce continuous improvement but needed a start and felt that ISO 9000 would provide the foundations.
The organisation has three main functions/divisions: manufacturing, engineering and sales. Engineering had a pattern of high staff turnover and it was felt that the 'wheel was continually being reinvented' - the organisation did not retain 'how we did it last time'. Another influence was that bigger customers were starting to 'make noises' about the need for a quality system..
They took advice from a consultant. They found his advice 'dogmatic'. It was also found to be misleading - the consultant told them that traceability was a requirement of the Standard. They agonised over this for six months. Eventually, re-reading the Standard, they discovered that traceability was only important if the customer required it; theirs did not. They soon learned to work on their own interpretation of the Standard and not rely on the consultant.
They knew, as they went into the exercise, that they wanted to avoid too much bureaucracy.
Implementation
Manufacturing found implementation to be very easy. The personnel in manufacturing were all used to working in a structured way; they used work tickets, time sheets and had a structured management information system (MRP). The focus in manufacturing was on writing down what they did. As this was happening the quality manager had to stop people in manufacturing from improving things - as they documented their procedures, people found obvious things to change. He was concerned that people should actually write down what they did (initially without instant modification). Having got the more complete picture, he then encouraged review and improvements which were seized fairly eagerly.
In Engineering, implementation was more difficult. The prime purpose of introducing ISO 9000 into Engineering was to establish a unified design process across the various groups. There was, prior to this, no predefined method. Furthermore the personnel in these groups had previously worked in fairly ill-defined environments. To a large extent this meant starting the implementation from scratch. There was, for example, an engineering change process and ISO 9000 implementation forced a review if it. People learned just why the engineering change process had to be so involved - there were so many consequences stemming from engineering changes. With effort, implementation produced 'remarkable' changes in engineering, mostly as a consequence of introducing procedures which would minimise the 'reinventing of wheels'.
Consistent with their desire to minimise bureaucracy, the implementation team dumped their procedures manuals in favour of flow-charts. It kept the processes in focus in a simple and meaningful way.
Results
The whole effort took two and a half years. Laserco claims reductions in time, reductions in errors and reductions in waste. These were not quantified during our discussion.
Their view now, five years on, is that ISO 9000 is the foundation for other things to happen. They felt that without it, they would have been unable to get a company-wide focus for the quality effort.
It was through the endeavours of two managers that this progress had been achieved. The two of them had to 'make it happen'. They felt it took a lot of 'getting through to people', the MD's interest waned at times (especially at end of quarter/year).
At the end of our discussion, the quality manager expressed the view that the whole process had one essential driver - the need to improve efficiency in an environment of a recession economy and headcount reductions.
Has ISO 9000 contributed to performance?
At the outset of this discussion, the efforts of the two people who behaved as 'organisational entrepreneurs' - working across organisational barriers to make things happen, should be recognised. It requires commitment and effort to make change in our organisations, usually because you are fighting the current culture or modus operandi.
The managers argued that ISO 9000 registration produced benefits. Processes were more clearly defined and that resulted in less waste or more efficiency. They had recognised and sought to avoid unnecessary bureaucracy, and even went a further step away from the norm by getting rid of procedures, using flow-charts to provide clarity of focus.
Without doubt, this is a case where much of the usual damage from ISO 9000 implementation has been avoided (for example excessive bureaucratisation and control through procedures). Should that lead us to declare it a success? ISO 9000, if it is to make a contribution, should improve economic performance, both internally in how the company works and externally in respect to the company's competitive position.
The extent to which ISO 9000 can or will improve economic performance will depend on features of the Standard (and their interpretation) and features of the company's current culture. For this case, it will be helpful to first consider the impact of the current culture on performance.
This culture has various components, some of which we have learned directly from the case study and some that we can guess at from the clues we are given. Let's look at two:
Management by attention to output
That the top management's attention wavers from quality at the end of financial periods is a sure sign that the company is run on output data. It will follow that departments have budgets, targets, variance reporting (this month versus last etc.) and it then follows that time will be given over to discussion, defending, promoting and surviving the 'budget issues'.
Management works this way when it has no understanding of the concept of variation. Enormous quantities of time are given over to irrelevant and stressful activity. The company does not benefit, it loses.
The managers leading this implementation had continuous improvement as their long-term goal. Continuous improvement starts from an understanding of variation - it is not an unending series of little 'fixes' - and there is no measurement of variation anywhere in this system.
Work methods
Manufacturing found ISO 9000 straightforward because the methods were similar to the methods in use; it is normal in manufacturing to have work instructions, job sheets and so on. By contrast, what is normal in high performance or World Class systems? One example works to four principles: single piece flow, pull not push, control in the hands of operators (including the right to stop the line) and 'automatic stops' on the line (if defective performance is automatically detected).
While this company had cut time for internal processes, the total order - fulfilment time had not changed and looked inefficient when 'activity time' was compared to 'total time'.
The first fundamental of World Class thinking is to understand what matters to your customers. This company had no such data. ISO 9000 concentrates on 'contract review' ('can we do what the customer wants'), customers don't - they make their judgement of organisations from the total sum of transactions they have. This company has no idea what matters, other than internal opinion - and this is always found to be unreliable.
Another fundamental principle of World Class thinking is process management. In Laserco the work methods are functionally designed and controlled and, by implication, there is no true process management.
Has ISO 9000 helped?
The ISO 9000 advocates would say that it has and in this case we might have been persuaded to agree. After all, they have gained clarity, improved efficiency (a little) and avoided some common pitfalls. The avoidance of pitfalls is not a credit to ISO 9000, but to the two managers who implemented it.
Gaining clarity and improving efficiency are no more than the kind of gains sought by the founders of this movement in the Second World War (see appendix 5). The result, then as now, is the control of output. But quality is not concerned with the control of output, it is concerned with the improvement of output.
What is worrying is that the managers in this system might tell you they've 'done quality', after all they went through the steps as directed and achieved registration. How would this community respond if they were told there is an enormous amount still to do and it requires a fundamental re-think of current operating assumptions? This is a system which does not know what matters to its customers, is therefore unable to turn such data into useful measures and shows no understanding of the importance and measurement of variation. This is not a quality system, it is a 'command and control' system. It is bound to be sub-optimised.
The chances are they would rationalise the current position - after all they have 'got ISO 9000' - and hence any real opportunity for performance improvement would be lost. And that is, perhaps, one tragedy associated with ISO 9000, people think they have 'done quality' and they haven't even started.
Quality methods are diametrically opposed to methods used in functional, 'command and control' systems. This is a company which has avoided the worst of ISO 9000, and yet it has been unable to avoid the fact that ISO 9000 registration has led the management community to a place from which it will be difficult to influence more fundamental change. ISO 9000 has reinforced some aspects of 'business as usual', it has not been the start of a quality journey.
Even more disturbing is that this process took two years. A company of only 150 people could make a change to improve economic performance in months, not years. ISO 9000 has decreased the probability that change will be possible in this case. If this seemed a good case of ISO 9000 registration, we can only hope their competitors are as complacent.
Case3
Background
'Systems Co'. is an engineering services organisation which manufactures handling systems and installs the same on customer sites. The customers use the systems in their manufacturing and packaging operations. Systems Co. has been registered to ISO 9000 for a number of years.
To achieve initial registration, Systems Co. sought the help of an accredited assessment body. Having interviewed a number, they chose one which they felt demonstrated the least 'bureaucratic' attitude and showed the most desire to be helpful and constructive in the relationship.
In this case, we followed a routine surveillance audit being conducted by the firm of outside assessors. Our purpose was to learn about what the auditor looked for and why. The assumption is that this would tell us about how the auditor (and client) think of the Standard and how their thinking relates to quality issues in the organisation.
Prior to meeting the auditor, the client informed us that the auditor was 'one of the better ones'. We took this to mean that he would take a view of the organisation as a business system rather than be pedantic about (his) interpretation of the Standard. We had been told that the auditor's organisation had 'recognised that their market has moved to wanting focus on things which had benefit to the business'. The client saw the audit process as one of 'support and assistance' to the business.
In this routine surveillance (described as a 'health check' by the auditor), the auditor focused on several 'mandatory' issues and then two selected areas of the business: engineering and customer service. The purpose of the audit was to raise any items of non-conformance which, if found, would be given an agreed date for resolution.
The mandatory issues were:
Changes to personnel. We probed why. We were told that change to a significant position (for example the chief executive) would require investigation as to whether this could adversely impact the quality system. The criteria for choosing whether to investigate such changes was specified in a manual produced by the assessing organisation's quality system.
To summarise the discussion which ensued: If the chief executive were to change how could an auditor determine whether to record what he sees as a non-conformance? It seems the auditor would seek evidence of the new leader's commitment to the quality system through words and deeds. But what chief executive would not assert that he or she was 'up for quality'? Would chairing quality reviews be sufficient to be evidence that he or she was doing things in support of the quality system?
It seems to us that this is a method which relies on anecdote and interpretation. A judgement must be made. Any decisions are based on the auditor's view of the world.
Changes to equipment. This follows the same logic as above to the extent that the auditor takes a view on what equipment changes should be subject to review.
Changes to the documented management system. Any changes should be reviewed to ensure they are consistent with the requirements of the Standard. Once again, decisions are based on the auditor's view of his own quality system, the client's quality system and above all his view of the application of the Standard.
Results and corrective actions from internal audits. It is relevant to note that at the time of first registration the client had established a group of internal auditors (according to advice from the original consultants). Now the department has been disbanded. When the current chief executive was appointed he charged the quality manager with changing the way quality was working away from 'quality control' and towards prevention. He took the view that control should be in the hands of people doing the work.
It is also relevant to note that the chief executive has subsequently charged the quality manager with reducing the documented quality system from two manuals to fifteen pages.
Feedback from customers. Complaints, returns and any corrective actions. Again, this relies on the auditor's interpretation (of, for example, whether problems should be treated as 'special-cause' or 'common cause').
Management review. To stand back and determine the effectiveness of the quality system. Again, subject to interpretation by the auditor. What might be the auditor's mental model of the purpose and mechanics of the management review?
Use of the assessor's logo. Apparently, the assessing organisation was implementing a directive from the UK Accreditation Service which dictated requirements for the proper use of 'quality badges' on stationery and the like. In this case the issue was irrelevant, the chief executive had already decided that he saw no value in 'advertising for the assessor'.
The two areas of the business the auditor looked at in detail were engineering and customer service.
Engineering
In engineering, he took a detailed look at an enquiry file. The file records customer information and he was interested in how the system made sure the customer got what was expected: how drawings got given numbers, how you'd know you were working on the right drawing, that the right information went to the right places and how, if changes were made, they were accommodated.
The auditor followed the progress of a file through to manufacturing. He was interested in how the jobs 'came in', how the system tracked changes and how work was planned through the manufacturing resources
This is what we would call 'bomb factory' quality (see appendix 5) - adherence to procedures. It is a means of control. It really has little to do with quality. In bomb factories it dealt with the immediate problem, it controlled output. It is plausible, indeed reasonable, to presume that the control of drawings, changes and relevant customer information will help ensure that the customer gets what the customer wants. It clearly could affect quality if these things went wrong and were not repaired before something got to the customer, but if we are controlling these things it should be because we have demonstrated their influence over the quality of output. We should question whether the right things are being controlled.
Quality is concerned with improvement. Improvement requires action on the system. The only action on the system encouraged by the introduction of ISO 9000 was auditing for non-conformance and the management review. The system was assumed to be in control if people behaved according to procedures. These procedures may not relate to what matters to customers and may or may not tell us more about the performance of a process.
Why did the organisation have this 'control of activity' approach to quality? Because they had learned it from their consultants and auditors. Even though they are cutting back on it, it is still the principle means of quality control - and all it can do is control output, it should be termed management control not quality control.
It is fundamental to quality thinking that measures (and controls) should be related to purpose. What is the purpose of this system? If it is to be a quality organisation, the measures in use should tell us how well it works and tell us whether things are improving. To take two examples: how often do we deliver assembled product on-time as originally committed? There were no data readily available (and in use); everybody estimated 75%. How often can the engineers complete their work on-site without recourse to manufacturing for something which should have been foreseen? There were no data; estimates ranged from hardly ever to nearly always.
Manufacturing
In manufacturing the manager used measures of labour hours. The purpose was to charge hours to jobs. It is not an unusual measure, many organisations treat their labour as though it is to be 'bought and sold'. The manager's task is to ensure that jobs keep to budget. This is the classic problem of 'budget management': the starting number is a reasonable guess from contracting, and any variation between the starting number and the final number could be due to a variety of causes. Minor variations are ignored, major variations are reported (although there is no evidence of whether this is common-cause or special-cause) and then acted on. Acting in such a way could distort the system, as could 'being seen to come in on budget' and other consequences of managing with a departmental measure (for example, shifting labour around to suit budgets not customers). Naturally the managers told me that "No distortion of data or other dishonest actions occurred here" and that may be so. The point is it's more likely that they will. More importantly, these are the types of data which could be dangerous if used to drive a 'get costs down' understanding of improvement.
As noted earlier, the data which might help learning and improvement (thus get costs down and improve service) were not in use: how predictable is our delivery versus first commitment; how often are engineers able to fit the customers' equipment without returning for something which should have been catered for; what are the type and frequency of customer and engineer demands on the organisation?
In the manufacturing side of the organisation, the prevailing attitudes were more in tune with the implicit attitude of the Standard. People placed a high value on having the correct procedures, the right documentation (doing things right rather than doing the right thing). When we discussed some of the features of the system which, if understood, might lead to improvement, the general response reflected an assumption that these things were normal. There were no good data, only opinions.
As this organisation has stepped back from the bureaucratisation of the management system (even though they always thought it minimal), it has begun to take more of an 'improvement' approach. We begin looking at these areas by ending our journey with the auditor - customer service was his final item to audit and it is fair to say that he said something which surprised us.
The auditor was interested in the client's approach to customer service. What this meant was:
It was the last item which surprised us. Having seen so many organisations (including this one) dwelling on control of the customer's order, because of the emphasis placed on this by the 'contract review' clause of the Standard, it was a nice surprise to hear an auditor emphasise that 'what mattered to the customer' mattered.
The auditor was primarily concerned to relate customer survey data to the management system. This line of thought usually results in 'corrective' actions and their reporting. The chief executive preferred to see the data as something which would drive action in the relevant part of the business, without a formal management decision-structure over it (he subscribes to the view that action is of more value than reports). This approach was characteristic of the changes the chief executive had made to the way the organisation worked.
It would have been of greater value to the client if the auditor had focused on the methods employed to gather customer data and the development and use of related measures to improve performance. For example, customer data showed that customers wanted quotations at the time they wanted them (not within a period). The contracting department controlled this with the use of a white-board and improved performance immediately; they had acted on what mattered to customers. Note that the use of a white-board would make this procedure difficult to audit - this auditor did not comment but many others might have. As an aside, with all documented procedures (which by definition are to be audited), it is worth asking whether the documentation is only in place so that the auditor can do his job. That is to say, the procedures provide no operational value to the organisation and may even detract value. When you discover such procedures and find that they are in place at the insistence of the auditor, you start to wonder just how widespread these problems are.
To return to the contracting department: it had also been noticed that 75% of all quotations work resulted in no order. It was a healthy perspective to take, the sort of view a systems-thinker would take. The department had started doing work on understanding the causes but this work had, so far, not included data from customers. But it was typical of the way things were working in the more progressive parts of the organisation - people had the right (systems) perspective and only needed help (if any) with method.
When we asked how people were measured in the contracting department, we were told there were no individual performance measures. We regarded this as a good response. Typically in such departments one finds 'activity measures' used in such a way as to undermine performance. The measures in use in contracting related to purpose (quotation on-time as committed, revenue), the use of these will be more likely to contribute to a climate of improvement.
In other parts of the organisation, more traditional attitudes prevailed. It is not to suggest that people differed in their commitment and potential contribution. Our experience has taught us just how much behaviour is conditioned by features of the organisation (we call these things 'system conditions').
Had ISO 9000 been beneficial?
The quality manager would claim so. He would argue (as do many others) that ISO 9000 was a good starting-place for the quality journey.
There is no doubt that clarity (of procedures) is an aid to good performance. But if with clarity comes unnecessary documentation and inappropriate controls, has it been beneficial? This organisation has spent the last three years drawing back from and eradicating dysfunctional aspects of registration to ISO 9000. Should this be regarded as a necessary cost? Surely not.
It was argued that ISO 9000 gave the organisation the impetus. Wouldn't it have been more beneficial to have customers be the impetus? If so, wouldn't the organisation have moved more quickly along the road of discovering the right methods and measures to improve performance?
Systems Co. is an organisation which is learning to behave as a 'customer-driven system'; was this learning facilitated or hindered by registration to ISO 9000? It could only have been hindered. ISO 9000 registration had focused on documentation of, and adherence to, procedures. There was no evidence (other than anecdote) as to the consequences on performance.
Conclusions
In each of the three cases we find evidence of ISO 9000 registration damaging organisation performance. In the first case the damage was palpable. In the second and third it is pertinent to note how they were 'moving away' from the worst aspects of the Standard. By implication, these activities had caused sub-optimisation. And worse, the work associated with the Standard blinded the managers to the real opportunities to improve quality and productivity.
In every case it was observed that third parties (assessors or consultants) were responsible for leading the organisations to act in ways which undermined performance. Furthermore, in every case registration to ISO 9000 involved considerable time and resources.
If the best that can be said for ISO 9000 is that it has features which organisations learn to move away from, it would serve organisations well to move away from it quickly.
To satisfy ourselves of any causal relationship we need an underlying theory that explains (and thus predicts) what will happen. The ethos of ISO 9000 is one of control. Control of work through adherence to procedures; the establishment of procedures which are assumed to affect quality of output; and control through the use of internal and third-party audits. It is an ethos which was established in the first Standards in response to a crisis. It is an ethos which may have solved the problem of its time but which now has become a problem itself.
The quarantine procedure discussed in case 1 illustrates how the control ethos leads to 'just in case' controls. The same thread is found in the other cases, it is assumed, a priori, that documentation and control of people's behaviour in procedures will 'prevent' problems. Yet there is no evidence to support this view. However, such a focus leads to people behaving in ways which value procedures over purpose (whether they like it or not). It is easy to see how registration can lead to demoralisation and nil learning.
We would argue that ISO 9000 is a step in the wrong direction, not a first step on the road to becoming a quality organisation (as is so often argued). Any of these three cases could have made great strides in improving quality and improved performance in a much shorter time. ISO 9000 registration blinded them to their opportunities and, what's worse it led people to assume they had 'done quality' when nothing could be further from the truth.
Are these cases unusual or unrepresentative? Not in our experience. As we stated at the outset, every organisation we have worked in which is registered to ISO 9000 shows evidence of activity which is damaging economic performance and is present because of the perceived requirements of the Standard. We have yet to find an organisation which is registered to ISO 9000 which does not show such evidence. Given the nature of the thinking (about control), one would predict that any organisation registered to ISO 9000 would have evidence of sub-optimisation, through the establishment of inappropriate activity ('doing things right', according to the Standard) and/or through lost opportunity (not doing the 'right thing', according to the needs of customers and the business).
To those who would claim that these cases are unrepresentative, we would ask: How many cases do we need to study to confirm or deny the proposition that ISO 9000 is damaging economic performance? In other words, how many more cases like the three reported here would be necessary to establish that we have a serious problem? We are confident that these cases are representative of the experience of registering to ISO 9000 (indeed we have discussed these observations with hundreds of managers and have yet to find one who is a keen supporter of the Standard - the support comes from those who make a living from it). We would argue that ISO 9000 registration predictably leads to the types of consequences reported here, because of its inherent theory of quality.
If UK plc were a quality organisation and its product the quality Standard, on the basis of what we know about the impact of this Standard, we would 'stop production' now.
Appendix 1 Executive summary, Vanguard research 1993The growth in registrations to BS5750 has been paralleled by growth in the level of debate about the Standard's fitness for purpose. The Director General of the BSI makes claims for the benefits of registration, but there has been little research into the impact of BS5750 on business performance.
This research gathers perceptions on BS5750 implementation from 647 registered organisations.
The results can be summarised as follows:
Only 15% of the organisations surveyed believe they achieved all of the benefits claimed by the Director General of BSI. 69% report an improvement in procedural efficiency, but the results for improvements in measurable aspects of efficiency (costs, waste etc) are only half as good, suggesting that procedural 'clarity' does not necessarily also result in improved efficiency. Of particular interest, is the finding that only 25% attributed an improvement in overall costs and 15% of the organisations perceived costs to have worsened.
The results attributed to the Standard raise questions as to whether the problems lie with the Standard, the implementors (managers) or both.
General perceptions of the Standard are that it is, on the whole, a good thing, but it suffers from problems of flexibility and interpretation. Governmental 'customers' are more inclined to insist on supplier registration to the Standard than private sector organisations and the results show that standards are set more from an 'internal' perspective than being driven by customer requirements. People report that a change in culture is required to achieve success with BS5750 but there is a diversity of views as to whether the Standard encourages the practice of listening to customers and whether it is important to introduce the concept of continuous improvement prior to implementation.
People feel that the Standard has spawned many rogue consultants and many also feel that British management does not really understand it. The problems of understanding, interpretation, flexibility and so on suggest educational needs amongst British management.
Comparisons of organisations on the dimensions of 'value added' and 'impact on costs' show that those who have succeeded with the Standard undertook it for broader purposes than those who undertook it for reasons of obligation and opportunism. The narrower perspective seems to have resulted in a failure to achieve much real benefit.
Smaller organisations have more concerns about costs and have greater expectations of improved market share as a result of registration. They also have more doubts about the Standard's relevance to their business.
The results show that the Standard is achieving less than it should. One reason is the way it has been tackled. Organisations reporting positive results took a different perspective from those which achieved less. But the results also suggest problems with the Standard. Aside from issues of comprehension and flexibility, the focus of the instrument can be argued to be more internal than external.
By emphasising control of procedures, people are likely to lose focus on the purposes of their work (to serve customers by the most efficient means). The purpose of inspection against the Standard is to police deviation from procedures. Whereas, its proper function should surely be to assess whether procedures are 'in control' and generating improvement.
The conclusions offer suggestions for simplifying and improving our approach to quality standards.
Appendix 2
A comparison of Surrey University research (conducted for L.R.Q.A.) and Vanguard research into opinions of ISO 9000 registration.
Introduction
The reader should bear in mind that the following comparison is subject to differences in method which make direct comparison difficult in some circumstances. We have worked from the Surrey report and have not had access to their raw data. The Surrey report is more limited in its scope (ie the range of questions asked) and uses different scales.
Vanguard included data on the following areas not covered by Surrey:
Costs involved
Timing
Expectations
What needed to change
Interpretation of the Standard
Measurement / implementation
Value to the organisation
Opinion / perception data
The Surrey results were reported as percentages 'yes' and 'no'; Vanguard used a scaled response (1 to 7).
Reasons for registration
The Surrey results reflect the same feeling of 'obligation' that Vanguard found (although their commentary does not highlight this) - 81% of companies say they decided to obtain registration to stay in business / to be considered for tenders.
Benefits of registration
It appears that the Surrey data are directly comparable with Vanguard's regarding benefits of registration (although the Vanguard research looks at both improvement and worsening as a result of ISO 9000 registration, the Surrey research reports improvements only).
Surrey report the following benefits:
49% improved market share
69% able to stay in business / not excluded from tenders
63% public relations / advertising / marketing benefits
Vanguard report the following (comparable) changes:
31.1% say market share has improved
33.4% say number of new customers has improved
Although the figures are illustrated in the report, Surrey does not draw attention to these lower rated items:
12% report improved efficiency / less wastage.
9% report increased customer satisfaction.
14% report greater discipline / order.
4% report more quality aware / improved quality.
The benefits section in the Surrey report does not include (reduction in) costs. Costs appear in the disappointments section. Items measured differ for disappointments and benefits. (Vanguard measured all items for improvement versus worsening). The Surrey report includes one short paragraph towards the end entitled 'disappointments with certification'.
Benefits of installing a Quality Management System
Five of the eight benefits reported in the Surrey research are directly comparable to the Vanguard data:
Improved efficiency / productivity
Surrey - 69%, Vanguard - 69%
Reduced waste
Surrey - 53%, Vanguard - 37%
Improved customer service
Surrey - 73%, Vanguard - 49%
Reduced costs
Surrey - 40%, Vanguard - 25%
Improved staff motivation
Surrey - 50%, Vanguard - 31%
The Vanguard and Surrey results are broadly similar. When one takes into account the problems of reliability and validity of opinion research amongst those who have an emotional investment in ISO 9000 registration, these results ought to cause alarm. They are equivocal about the Standard's contribution to say the least.
Appendix 3
A discussion of the Manchester Business School research (conducted for SGS Yarsley).
The Manchester report is entitled: "ISO 9000 - Does it work?" The answer provided in the foreword by SGS is "Ö.a qualified but confident 'yes'". The following 'key findings' have led to this conclusion:
69% felt that their expectations of the Standard had been met or exceeded; only 5% claimed to be very dissatisfied.
We can learn nothing from the report about what people's expectations were and the data on 'expectations met' is not presented. However, data concerning level of satisfaction with impact of ISO 9000 is presented and it shows that 53% were satisfied, 26% were neither satisfied nor dissatisfied and 21% were dissatisfied. These results show a similar 'equivocal' attitude to ISO 9000 as found in the Surrey and Vanguard research.
The three most important benefits of ISO 9000 were seen as better management control, greater awareness of procedural problems and promotional value.
Given that the approach to control encouraged by the Standard can itself cause sub-optimisation (see conclusions), this finding ought to raise concern. The report presents the 'top ten' benefits as follows:
Better management control
Improving awareness of procedural problems
Using the Standard as a promotional tool
Improving customer service
Facilitating elimination of procedural problems
Improving efficiency
Keeping existing customers
Increasing customer satisfaction
Aiding induction of new staff
Improving market share
None of the results is quantified and we must remind ourselves of the problems of validity with such data. Even so, it is not an impressive list. Improved quality and productivity should perhaps be top of such a list, and improved customer satisfaction should go beyond satisfying 'obligations' - the report also tells us that most organisations focused on 'external reasons' when asked why they sought the Standard in the first place, supporting the previous research showing that 'market-place obligation' is driving registration (78% say our future customers will demand it, 58% report customer pressure).
Those most satisfied with the impact of ISO 9000 on their organisations had sought the Standard for better management control and improved customer service.
Given the nature of control encouraged by the Standard this could be telling us that those who are most satisfied have done themselves the most damage. We also can't help feeling that 'customer service' in this sentence means satisfying market-place obligation as the Standard takes a strong 'conformance' approach to customer service which in many organisations gets translated into practices which result in poor service (you are expected to be happy that your problem/request/need is in the right procedure).
More respondents believed the Standard to be cost-effective than not.
Behind this statement lies more equivocation. The data reported show 468 cases claiming cost-effectiveness (over different periods), 399 cases claiming it was not cost-effective and 215 cases saying it is too soon to say. The headline could have said: Data shows doubts about the cost-effectiveness of ISO 9000.
This is an interesting paradox. What we know is that quality leads to increased productivity, reduced costs, improved market share and customer satisfaction. If the results from ISO 9000 registration are equivocal it lends support to the need to question whether ISO 9000 has anything to do with quality.
Companies with ISO 9000 certification showed a significantly higher rate of sales growth than the national average and were four times more likely to have survived the recent recession.
This was the finding that received the most attention from the pro-ISO PR fraternity. 'You make more money if you are registered to ISO 9000' is the claim. The first claim is made on the basis of comparing sales growth with GDP. Any undergraduate researcher would note that this tells us nothing about cause and effect. One possible reason for sales growth in registered companies is that only registered companies are able to tender for contracts.
The second claim is made from analysing the progress of 185 companies which had ISO 9000 in 1991. Of the 185, 130 were still registered, 53 were still in business but not registered to ISO 9000 any longer and 2 were no longer in business. These data are used to argue that the failure rate of ISO companies is 1.1% which compares favourably with the national average for failures (5.2%). What a terrible abuse of statistics. One could equally have reported 'stopping registration to ISO 9000 won't harm your business'. It might have been valuable to explore why the 53 decided they no longer needed it.
Chief concerns with ISO 9000 included time commitments and volume of paperwork, as well as the cost of implementing and maintaining the Standard.
What is the value of this time spent on the quality bureaucracy? Does it achieve a purpose which fits with the organisation's purpose or is it a necessary evil? These problems give support to the view that ISO 9000 has nothing to do with quality. It is controlling output in a manner which could add to costs and become a barrier to quality improvement.
Company certificationÖhas grown by 10,000 certificates a year since 1992Öcertification in Europe has doubled every nine months since 1993 and there are now 95,000 certificates world-wide.
If the results from the case studies are generalised to the world stage we are guilty of causing massive economic damage to our organisations. The good news is that of some three million businesses in the UK, less than fifty thousand are registered and the rate of growth in registrations has slowed.
Appendix 4
Vanguard visited five of the companies which had participated in our 1993 research. All of these companies had regarded ISO 9000 registration to have been a success in every respect. In every case we found evidence which would suggest the contrary:
1 Design changes
In a manufacturing organisation, the time to effect design changes grew from three days to three months with the introduction of ISO 9000.
A requirement of the standard is to control design changes. The organisation chose to control changes through the introduction of committees which met quarterly. It used to be resolved by the operator and his boss. It could have stayed the same - why did these people get over-elaborate with controls?
2 Quotation time
A printer used to supply a quote over the 'phone within two hours of the customer making a request. He became registered to ISO 9000 and now takes days with forms going back and forth by fax. It could have stayed the same. Why did these people get over-elaborate with procedures?
3 Waste maintained
A manufacturer with highly automated processes chose to implement the Standard without introducing measures of process control (section 20 of the Standard). The company elected to ignore this requirement on the grounds that it did not directly affect customer requirements. 2% of its turnover is wasted in scrap. The measure is stable over time - it is predictable. It could have been improved.
4 Waste created
A building repair organisation implementing ISO 9000 chose to control surveyors' reports by registering them and counting them by volume and value. They ignored the quality of surveyors' reports. Waste created in administration (re-working errors on surveyors' reports) amounted to approximately 15% of working-time. No-one knew.
5 Reinforcing functional thinking
A small manufacturing organisation implemented ISO 9000 by having a consultant and the quality manager draw up a manual of procedures. It was given to staff and they were told 'work to these procedures'. The writers had specified functional relationships (how you record/control your inputs and outputs). It reinforced a functional perspective and led to greater allocations of blame, not greater solving of problems.
Appendix 5
A brief history of BS 5750
In order to understand organisations, it is often helpful to look into their history. We will do the same with the Standard to understand more about the tensions being experienced today. What was relevant and important at the time of the Standard's genesis has changed as markets and organisations have evolved; the Standard has become an anachronism, bestowed with legitimacy by the institutions which have grown around it, but of questionable relevance to the everyday problems of performance management and improvement in modern times.
One has to go back further than 1979 (the year BS 5750 was introduced) to understand the Standard. Its earliest predecessor was a defence industry standard, in use during the Second World War, which in time was adopted by NATO and became known as the AQAP series (Allied Quality Assurance Publications). The Standard was introduced to solve a problem of the time: munitions were exploding in factories. It solved the problem by ensuring that munitions were made strictly according to procedures (the procedures being documented and independently checked).
At that time, thinking about the organisation of work was strongly dominated by notions of work specialisation and standardisation. Specialisation of work had its roots in the work of Frederick Winslow Taylor and Adam Smith, they showed how performance could be improved by structuring work in specialist functions. Standardisation was the secret of success for Henry Ford. It enabled significant reductions in cost. These tenets of management thinking still remain in most of our organisations. They have been challenged in recent years by quality theorists and have been shown to be impediments to performance improvement , but new and different ideas have made little headway against the status quo.
When organisational thinking is governed by ideas of specialisation and standardisation, the managerial role is to specify the standards and procedures to which people will work. Documenting procedures formalises this thinking and provides a means of control (have people done what the procedures say they should do?). It is a means of control which controls output (bombs don't go off in the factory), quality thinking, by contrast, would lead to improved output. It is ironic that while many munitions factories in the UK were using such methods to control output, Deming was working with munitions factories in the United States to improve output through reducing variation.
Following the war, the idea that performance could be improved through 'prevention of defects' took hold in industry. The view was that prevention required planning and planning implied predetermining procedures. Verification (do they do what they say they do?) was conducted by the customer (usually the government department making the purchase). To save taxpayers' money, the government decided that surveillance should be conducted by the private sector.
Industry's response was to pass responsibility to their suppliers and conduct verification using their own inspectors. Naturally, there were differing views as to the necessary contents of a quality system and the establishment and assessment of differing systems consumed increasingly greater resources.
In 1972, the first British Standard, BS 4891, was published in an attempt to provide common guidance to industry. BS4891 contained clauses which were peculiar to defence requirements and was replaced by BS 5179 in 1974. Many non-defence contractors, however, still had difficulties with this document.
In 1977, Sir Frederick Warner reported to the government on the use of quality standards in British industry. Warner recommended a common standard for quality assurance, independent assessment and a register of those companies assessed as meeting the requirements. Hence the ISO 9000 (originally BS 5750) industry was born.
The birth of this industry coincided with large-scale redundancies of government inspectors. A Department of Trade and Industry scheme provided financial assistance to some firms which sought the advice and assistance of this new army of quality consultants. The consultants subscribe to the view on which ISO 9000 was originally based, that prevention requires planning and the evidence of planning should be documented procedures which can be independently verified. It is a method which, if successfully applied, will control output.
Deming, by contrast, taught the Japanese to manage their organisations as systems and to continuously improve performance by managing and reducing variation. It is a body of knowledge beyond the experience of the army of what Tom Peters called 'the procedures merchants'.
ADVERTISEMENT
View the Elsmar Cove Privacy Policy and Terms and Conditions
This page last reviewed or edited: Tuesday, August 28, 2012 6:53 PM (Coordinated Universal Time [ZULU] -5 hours)