Modern tools & techniques such as Lean and Six Sigma are well known to bring significant improvement in operations and to reduce costs, and there are many notable success stories across most industry sectors. However, frequently they fail to address serious quality problems caused by human error. Overcoming human frailty present within various operational processes continues to be a challenge and one that requires special consideration.

This is particularly acute where products are largely hand-built or where there is a substantive manual input. The approach discussed is equally applicable in medical or safety critical environments such as aviation or oil & gas industries.

Much has been discussed and debated on improving human behaviour, leadership and related topics to improve the way work is done, but what is often forgotten is the huge benefit that more traditional approaches to quality improvement and defect reduction can still bring. This is particularly relevant when used alongside Lean to harness additional gains and synergies to great effect.

Strategies
Good product design, technology, engineering excellence and Lean processes all reduces the likelihood of quality defects/errors. But where human error still prevails, this leads to high cost of non-quality. In these cases, often quite radical strategies need to be deployed to both identify and contain them at the point they are created.

The principle strategies deployed are;
• Firewall
• Defect Containment Metric
• Measurement Accuracy Improvement

Firewall
A step back in time –Juran:                                                                                                                                 Simply attempting to inspect-out defects / errors has never been an option. Dr Juran, over 50 years ago said that, collectively detecting errors result in a performance of about 80% accuracy at best.

A range of statistical based tools can be used to help front-line operators and professional staff. But reality says defects will escape both from the process step where they were created and that errors remain undetected creating a poor customer experience.

Firewall Process:
A Firewall is a much talked about word these days by politicians, economists and bankers in attempting to address the world debt crisis.  What is less well known is its application in industrial and operational environments.

We all know of the benefits of ‘a second pair of eyes’ and it must come as no surprise that in some processes where human error is a concern, then a further check needs to be made at some stage.  This is called a Firewall and needs to be completely independent. 

The objective of the Firewall is to protect the customer [internal or external] and to ensure that all quality issues are quickly identified.
The idea of a Firewall would seem to contradict Lean in tackling the effect rather than the root cause. Clearly, quality is the responsibility of the process owner no matter what tools or ideology is used; traditional or modern. But in circumstances such as this, where product quality partly relies on visually checking for defects or correcting errors created from an imperfect process, then it is required.

In these cases, the benefits of detection early in the process where defects / errors are created far outweigh the additional costs, providing it is proportional.

Firewall Removal:
That said, a Firewall must be seen as a short term solution. What it does do is give headroom to develop medium to long term strategies through better process design and execution. 

The eventual removal of a Firewall is based on achieving a break-through level of performance as determined by a Defect Containment Metric, discussed below. This quantifies and reports on the number of defects [or errors] created, identified and missed at each process stage.

In effect, it provides the essential information on process performance to set challenging, but not unreasonable targets.

Defect containment metric
The defect containment [DC] metric is used to assess how well the process performs. It records defects and errors at the stage they are created and importantly those escaping to the customer.

The metric is based on the premise that there are three opportunities to correct non-conforming output:

Mistakes: these are a natural part of learning and human intervention that occur during the process itself and caught by the Process Owner – first opportunity.
Errors: these are mistakes left in completed work at each operation prior to being handed over as good output and these are caught by a First-off check – second opportunity

Defects: these are errors that escape and are either caught by a Firewall at that stage (defects detected), or at a later stage, and attributed to the stage they were created (defects missed) – third opportunity.

The DC metric provides managers with good data, not only on how well people perform and deliver excellent quality, but also to establish priorities for process design and execution improvements. The DC metric comprises four separate key performance indicators [KPIs] summarised in figure 1:

• First-off Effectiveness [leading
• Firewall Effectiveness [leading]
• Total Effectiveness [lagging) 
• Escaping Defects Index [lagging]

To explain these measures, an actual example of the calculation based on data collected from a client is tabulated in Fig I: DC metric - example data, and described below:

First-off effectiveness: This uses Juran's definition of First-off accuracy and is the proportion of errors caught compared to total created at that stage [incl. missed].
At process stage – Second Operation is [320/350] X 100/1 = 91%

Firewall effectiveness: This is the proportion of defects caught compared to total created at that stage [incl. missed]:
At process stage – First Operation [200/ (200+140)] X 100/1 = 59%

Total effectiveness: This is the combined effect of the ‘second pair of eyes’ calculated by [(1-First-off Effectiveness) X Firewall Effectiveness] + First-off Effectiveness].

For the whole process is [(1-0.771) X 64.6% + 77.1%] = 90.6%   
This measure says that First-off Inspection allowed 22.9% (1-0.771) of errors to escape the process and these escaped errors were caught by Firewall at the rate of 64.6%, giving a combined ‘second pair of eyes’ Total Effectiveness of 90.6%. Put another way, Firewall improves performance of First-off from 77.1% to 90.6% overall.

The Juran Delta in the table is the difference between the Total Effectiveness and a combined performance using Jurans’ 80% First-off plus 80% Firewall equivalent to a standard overall effectiveness of 96%.

For the whole process is 90.6% - [(1-0.80) X 80% + 80%] = - 5.4%   

Escaping defects index: This measures the potential defects that escaped to the customer and is based on the Total Effectiveness multiplied by units of output in period:

For the whole process is [(1-0.906) x 80 (output)] = 7 potential defects.
DC metric benefits

The DC metric identifies the process stages where most defects are missed or errors created and enables appropriate focused action.  This highlights the need for improvement to address the questions; “Why were these errors made in the first place, and why were they not found at the time they were created?”

The Escaping Defects Index gives a measure of customer experienced quality. Of course, actual experienced quality may well differ from this index due to the fact that not all will impact on performance or be identified by the customer.

Finally, setting break-through targets enables managers to focus on a stretch goal. There is no hard and fast rule for these goals, other than by agreement with both internal and external customers. In our example this is set at a 99% quality level typical of many World-class companies. This equates to about 4 Sigma and as a Lean Sigma statistic or circa 100 defects / errors per 10,000 opportunities. Of course, this would be unacceptable in medical or safety critical processes where a greater Sigma value is required.

Measurement accuracy improvement
Types of Errors:
A myriad of factors can influence behaviours that impact on the ability to accurately and precisely identify non-conformities.

We have collected a considerable amount of raw data on the causes of defects / errors in typical manufacturing / operational environments. These are summarized in the fishbone diagram in figure II  - Causes of defect / errors

The basic problem
Of course, poor detection can mean accepting output with defects as well as rejecting that which is good. On the one hand this allows defects / errors to escape the process or critically, or alternatively incurring unnecessary rework correcting otherwise good output.

This is the basic problem; the detection process is imperfect and creates excessive cost of non-quality.

The first step is to install a good measurement system using a DC metric strategy that enables operations to identify the main areas of failure in the process.
The next question is; “What can be done about the problem now that it has been clearly identified and quantified?” 

In many organisations these days, a Quality Management System [QMS] is used to provide the foundation for good disciplines and a quality assured approach to carrying out operations.

However, adherence to the QMS, no matter how good it is and how well it is followed is worthless if people simply do not know what a defect looks like.
If we can’t measure something, we may not know what we need to know about it. Of course, if we can measure it, we may still not know much if the variation in the measurement due to the measurement system is large compared to the actual variation of the items under scrutiny.

The solution 1:
Focused statistical studies
Production Study: Firstly, it is essential to know what exactly happens during the course of the normal day. A structured statistical study is set up to compare those identified defects /errors against an ‘experts’ considered view.
MSA Workshop Study: A second study is done to assess inspection accuracy and precision using actual defects and real life-size photographs. Measurement System Analysis [MSA] techniques are used.

These two separate sessions are organised to assess both the test equipment to measure variable data and a person’s ability to identify defects from attribute data.

Findings: The results of these two studies will show where and how often:

• defects / errors were missed when they clearly exist
• incorrectly identified defects / errors when they do not exist, creating unnecessary rework
• incorrectly identified types of defect /errors.

The studies will also show the number of inconsistencies between people [reproducibility] in what was considered to be a defect, and the ability to repeat the decision [repeatability] for the same defect.
Finally, the company will also able to determine:

• type of defects /errors causing the most concern
• variation between the people conducting the study in the way they worked
• amount of variation due to the equipment itself and its resolution in being able to discriminate between different defects / errors.

The solution 2:
Continual assessment and improvement planning
The studies provide a wealth of reliable statistical data on which to help employees develop their skill and competence, such as:

• individual development plans based on a good understanding of actual performance on the job 
• training & coaching materials tailored to individual needs.
• On-going assessment using MSA studies to validate training & coaching, i.e. has the person improved and by how much, and if not why not.

This is a fundamental shift in thinking, using MSA studies to validate the training & coaching programmes received. This ensures applied learning is being translated into effective results.

Key outputs and benefits
Reducing defects / errors in these types of situations requires a fundamental rethink and often adopting non- Lean practices to overcome human frailty. Taken as a whole, the three strategies have been proven to work well to improve quality, reduce cost and develop people.

At the front line, the ability to identify a defect/ error both accurately and precisely is the key.  This directly impacts on productivity and in exceeding customer expectations.

New ways of working will enable managers to benchmark people detection skills and competencies and provide the basis for transferring best practice.
In the medium to longer term, reliable statistical data enables design and manufacturing processes to be improved, Firewall to be eradicated and the operation to become far more stable and predictable.l

Graham Cartwright is a senior partner at Innovation Consultancy Partnership Limited.
email: graham@icpartnership.com.
 http://icpartnership.com

MSA
Much of today's current thinking around MSA has been adapted from the reference manual prepared by the Automotive Industry Action Group [AIAG] based in Southfield Michigan USA. This is a not-for-profit organisation founded in 1982 by a group of visionary managers from Chrysler, Ford Motor Company, and General Motors.