Do overlapping test strategies enhance reliability, or just add cost?

Ed.: This is the second half of a column began last month.

A popular trend in EMT is to look at the overlapping test coverage areas – and remove them. Ideally, the intention is to permit only one test strategy to cover one range of defects, while using another test strategy to cover a different range of defects, so that the strategies operate on a mutually exclusive basis (Figures 5 and 6). The objective is straightforward. By reducing overlap, the user hopes to save test resources and, therefore, time and money. Undoubtedly, this is an admirable aim, but lower cost doesn’t necessarily mean better test.

As we know, the EMT model is basically based on test coverage. Once the Swiss Cheese Model is applied and the “latent condition” layers revealed, several things come to light. Per the Swiss Cheese Model, there are three considerations for latent conditions: organizational influences, unsafe supervision and preconditions for failure. These considerations, in one way or another, are tied to the human element, which almost always injects certain extraneous effect into any process (Figure 7).

Let’s look at some examples: Organizational influences in the manufacturing world context could be unreasonable pressure on the people to meet shipment numbers with the limited resources that are available at that time. Unsafe supervision could mean having inexperienced workers maintaining the test machines in a new plant during the night shift. Preconditions for failure could mean overworked employees, leading to unbalanced individuals with poor judgment and error-prone situations. How can we factor these into the overall possible failure scenario? Who should actually take on the ownership of looking into this?

The most obvious answer is those who have the most to lose if something does go wrong.

With fewer test overlaps comes a corresponding increase in the likelihood that those extraneous elements from the latent conditions will create a trajectory for failures. The only time we would know about this would be when the product reaches the end-users, in the form of complaints – leading to warranty claims or, worse still, product recalls. Before that happens, a considerable period of time would have lapsed, during which more of the same product could have followed through the same failure trajectory.

However, it is fair to note that insofar as warranty claims go, bright spots can positively influence it. For example: better product designs that require fewer repairs, cheaper spare parts, or even allowing customers to do simple self-repair.

Let’s look at some warranty numbers across different industries to get a sense of the level of opportunity for improvement (Table 1). Warranty accrual and warranty claim numbers presented are measured as a percentage of revenue. Although warranty claims are the actual claims made against the manufacturer, it is important to note the warranty accruals as well, as this is money set aside in anticipation of future claims. The figures are diverse, with some coming in higher than others, pointing to further room for improvement. Will cutting test overlaps on the manufacturing floor help to improve this? This is an open question that the EMT industry will have to weigh in their test strategies.

In summary, the quest for better and cheaper products is undeniably perpetual by nature. However, it is easy to develop tunnel vision in the pursuit, or sometimes, it’s just plain convenient to do so. We have to consider other implications as well, as they may become latent conditions for future failures.

Going back to my account last month on the Bashkirian air disaster, improvements have since been made. One of them, the Traffic Collision Avoidance System, is now improved with version 7.01 TCAS II, such that it is able to reverse the original resolution advisory for one of the aircrafts should it detect that the crew of the other one is not following its original TCAS resolution advisory. No doubt more advances will come, but hopefully not at the expense of more lives.

Andrew Tek is product manager, In-circuit Test Group, Agilent Technologies (agilent.com); This e-mail address is being protected from spambots. You need JavaScript enabled to view it .