‘Systems Co’. is an engineering services organisation which manufactures handling systems and installs the same on customer sites. The customers use the systems in their manufacturing and packaging operations. Systems Co. has been registered to ISO 9000 for a number of years.
To achieve initial registration, Systems Co. sought the help of an accredited assessment body. Having interviewed a number, they chose one which they felt demonstrated the least ‘bureaucratic’ attitude and showed the most desire to be helpful and constructive in the relationship.
In this case, we followed a routine surveillance audit being conducted by the firm of outside assessors. Our purpose was to learn about what the auditor looked for and why. The assumption is that this would tell us about how the auditor (and client) think of the Standard and how their thinking relates to quality issues in the organisation.
Prior to meeting the auditor, the client informed us that the auditor was ‘one of the better ones’. We took this to mean that he would take a view of the organisation as a business system rather than be pedantic about (his) interpretation of the Standard. We had been told that the auditor’s organisation had ‘recognised that their market has moved to wanting focus on things which had benefit to the business’. The client saw the audit process as one of ‘support and assistance’ to the business.
In this routine surveillance (described as a ‘health check’ by the auditor), the auditor focused on several ‘mandatory’ issues and then two selected areas of the business: engineering and customer service. The purpose of the audit was to raise any items of non-conformance which, if found, would be given an agreed date for resolution.
The mandatory issues were:
Changes to personnel. We probed why. We were told that change to a significant position (for example the chief executive) would require investigation as to whether this could adversely impact the quality system. The criteria for choosing whether to investigate such changes was specified in a manual produced by the assessing organisation’s quality system.
To summarise the discussion which ensued: If the chief executive were to change how could an auditor determine whether to record what he sees as a non-conformance? It seems the auditor would seek evidence of the new leader’s commitment to the quality system through words and deeds. But what chief executive would not assert that he or she was ‘up for quality’? Would chairing quality reviews be sufficient to be evidence that he or she was doing things in support of the quality system?
It seems to us that this is a method which relies on anecdote and interpretation. A judgement must be made. Any decisions are based on the auditor’s view of the world.
Changes to equipment. This follows the same logic as above to the extent that the auditor takes a view on what equipment changes should be subject to review.
Changes to the documented management system. Any changes should be reviewed to ensure they are consistent with the requirements of the Standard. Once again, decisions are based on the auditor’s view of his own quality system, the client’s quality system and above all his view of the application of the Standard.
Results and corrective actions from internal audits. It is relevant to note that at the time of first registration the client had established a group of internal auditors (according to advice from the original consultants). Now the department has been disbanded. When the current chief executive was appointed he charged the quality manager with changing the way quality was working away from ‘quality control’ and towards prevention. He took the view that control should be in the hands of people doing the work.
It is also relevant to note that the chief executive has subsequently charged the quality manager with reducing the documented quality system from two manuals to fifteen pages.
Feedback from customers. Complaints, returns and any corrective actions. Again, this relies on the auditor’s interpretation (of, for example, whether problems should be treated as ‘special-cause’ or ‘common cause’).
Management review. To stand back and determine the effectiveness of the quality system. Again, subject to interpretation by the auditor. What might be the auditor’s mental model of the purpose and mechanics of the management review?
Use of the assessor’s logo. Apparently, the assessing organisation was implementing a directive from the UK Accreditation Service which dictated requirements for the proper use of ‘quality badges’ on stationery and the like. In this case the issue was irrelevant, the chief executive had already decided that he saw no value in ‘advertising for the assessor’.
The two areas of the business the auditor looked at in detail were engineering and customer service.
Engineering
In engineering, he took a detailed look at an enquiry file. The file records customer information and he was interested in how the system made sure the customer got what was expected: how drawings got given numbers, how you’d know you were working on the right drawing, that the right information went to the right places and how, if changes were made, they were accommodated.
The auditor followed the progress of a file through to manufacturing. He was interested in how the jobs ‘came in’, how the system tracked changes and how work was planned through the manufacturing resources
This is what we would call ‘bomb factory’ quality (see Appendix 3) – adherence to procedures. It is a means of control. It really has little to do with quality. In bomb factories it dealt with the immediate problem, it controlled output. It is plausible, indeed reasonable, to presume that the control of drawings, changes and relevant customer information will help ensure that the customer gets what the customer wants. It clearly could affect quality if these things went wrong and were not repaired before something got to the customer, but if we are controlling these things it should be because we have demonstrated their influence over the quality of output. We should question whether the right things are being controlled.
Quality is concerned with improvement. Improvement requires action on the system. The only action on the system encouraged by the introduction of ISO 9000 was auditing for non-conformance and the management review. The system was assumed to be in control if people behaved according to procedures. These procedures may not relate to what matters to customers and may or may not tell us more about the performance of a process.
Why did the organisation have this ‘control of activity’ approach to quality? Because they had learned it from their consultants and auditors. Even though they are cutting back on it, it is still the principle means of quality control – and all it can do is control output, it should be termed management control not quality control.
It is fundamental to quality thinking that measures (and controls) should be related to purpose. What is the purpose of this system? If it is to be a quality organisation, the measures in use should tell us how well it works and tell us whether things are improving. To take two examples: how often do we deliver assembled product on-time as originally committed? There were no data readily available (and in use); everybody estimated 75%. How often can the engineers complete their work on-site without recourse to manufacturing for something which should have been foreseen? There were no data; estimates ranged from hardly ever to nearly always.
Manufacturing
In manufacturing the manager used measures of labour hours. The purpose was to charge hours to jobs. It is not an unusual measure, many organisations treat their labour as though it is to be ‘bought and sold’. The manager’s task is to ensure that jobs keep to budget. This is the classic problem of ‘budget management’: the starting number is a reasonable guess from contracting, and any variation between the starting number and the final number could be due to a variety of causes. Minor variations are ignored, major variations are reported (although there is no evidence of whether this is common-cause or special-cause) and then acted on. Acting in such a way could distort the system, as could ‘being seen to come in on budget’ and other consequences of managing with a departmental measure (for example, shifting labour around to suit budgets not customers). Naturally the managers told me that “No distortion of data or other dishonest actions occurred here” and that may be so. The point is it’s more likely that they will. More importantly, these are the types of data which could be dangerous if used to drive a ‘get costs down’ understanding of improvement.
As noted earlier, the data which might help learning and improvement (thus get costs down and improve service) were not in use: how predictable is our delivery versus first commitment; how often are engineers able to fit the customers’ equipment without returning for something which should have been catered for; what are the type and frequency of customer and engineer demands on the organisation?
In the manufacturing side of the organisation, the prevailing attitudes were more in tune with the implicit attitude of the Standard. People placed a high value on having the correct procedures, the right documentation (doing things right rather than doing the right thing). When we discussed some of the features of the system which, if understood, might lead to improvement, the general response reflected an assumption that these things were normal. There were no good data, only opinions.
As this organisation has stepped back from the bureaucratisation of the management system (even though they always thought it minimal), it has begun to take more of an ‘improvement’ approach. We begin looking at these areas by ending our journey with the auditor – customer service was his final item to audit and it is fair to say that he said something which surprised us.
The auditor was interested in the client’s approach to customer service. What this meant was:
– whether the client had determined how often to conduct customer surveys
– how the results of such surveys were being built back into the management system
– how the organisation knew what mattered to its customers
It was the last item which surprised us. Having seen so many organisations (including this one) dwelling on control of the customer’s order, because of the emphasis placed on this by the ‘contract review’ clause of the Standard, it was a nice surprise to hear an auditor emphasise that ‘what mattered to the customer’ mattered.
The auditor was primarily concerned to relate customer survey data to the management system. This line of thought usually results in ‘corrective’ actions and their reporting. The chief executive preferred to see the data as something which would drive action in the relevant part of the business, without a formal management decision-structure over it (he subscribes to the view that action is of more value than reports). This approach was characteristic of the changes the chief executive had made to the way the organisation worked.
It would have been of greater value to the client if the auditor had focused on the methods employed to gather customer data and the development and use of related measures to improve performance. For example, customer data showed that customers wanted quotations at the time they wanted them (not within a period). The contracting department controlled this with the use of a white-board and improved performance immediately; they had acted on what mattered to customers. Note that the use of a white-board would make this procedure difficult to audit – this auditor did not comment but many others might have. As an aside, with all documented procedures (which by definition are to be audited), it is worth asking whether the documentation is only in place so that the auditor can do his job. That is to say, the procedures provide no operational value to the organisation and may even detract value. When you discover such procedures and find that they are in place at the insistence of the auditor, you start to wonder just how widespread these problems are.
To return to the contracting department: it had also been noticed that 75% of all quotations work resulted in no order. It was a healthy perspective to take, the sort of view a systems-thinker would take. The department had started doing work on understanding the causes but this work had, so far, not included data from customers. But it was typical of the way things were working in the more progressive parts of the organisation – people had the right (systems) perspective and only needed help (if any) with method.
When we asked how people were measured in the contracting department, we were told there were no individual performance measures. We regarded this as a good response. Typically in such departments one finds ‘activity measures’ used in such a way as to undermine performance. The measures in use in contracting related to purpose (quotation on-time as committed, revenue), the use of these will be more likely to contribute to a climate of improvement.
In other parts of the organisation, more traditional attitudes prevailed. It is not to suggest that people differed in their commitment and potential contribution. Our experience has taught us just how much behaviour is conditioned by features of the organisation (we call these things ‘system conditions’).
Had ISO 9000 been beneficial?
The quality manager would claim so. He would argue (as do many others) that ISO 9000 was a good starting-place for the quality journey.
There is no doubt that clarity (of procedures) is an aid to good performance. But if with clarity comes unnecessary documentation and inappropriate controls, has it been beneficial? This organisation has spent the last three years drawing back from and eradicating dysfunctional aspects of registration to ISO 9000. Should this be regarded as a necessary cost? Surely not.
It was argued that ISO 9000 gave the organisation the impetus. Wouldn’t it have been more beneficial to have customers be the impetus? If so, wouldn’t the organisation have moved more quickly along the road of discovering the right methods and measures to improve performance?
Systems Co. is an organisation which is learning to behave as a ‘customer-driven system’; was this learning facilitated or hindered by registration to ISO 9000? It could only have been hindered. ISO 9000 registration had focused on documentation of, and adherence to, procedures. There was no evidence (other than anecdote) as to the consequences on performance.