How to Check Test Repeatability Before Trusting Pass/Fail Results

When the same device passes one run and fails the next, keep the actual measurement values. A pass/fail label tells you which side of a limit each reading landed on. The values help you investigate how much the result moved and under what conditions.

Repeatability is one part of that investigation. It asks how closely repeated measurements agree when you hold the measurement conditions steady. It does not establish whether the device meets every requirement or whether the measurement is accurate.

What repeatability tells you

The international vocabulary of metrology defines repeatability around repeated measurements over a short period with the same procedure, operators, measuring system, operating conditions, and location. That makes it a check of precision under specified conditions. JCGM: repeatability conditions, measurement repeatability.

Consistent readings can still share an offset. Precision describes agreement among repeated readings. Accuracy concerns agreement with the quantity's true value. Checking one does not establish the other. JCGM: precision, accuracy.

Define the check before collecting data

For a semiconductor test setup, turn those definitions into a written measurement plan:

1. Name the parameter. Record the measurement, units, test conditions, and applicable specification limits. Investigate the measured value as well as its pass/fail result.
2. Identify the test item. Record the device or reference identity. Establish whether the item can remain stable under the planned repeated testing. NIST's check-standard method depends on a stable item that represents the measurement workload. A unit called “golden” needs that evidence too. NIST: check standards.
3. Define what stays fixed. Include the tester, program revision, test site, interface hardware, operator, temperature, and measurement settings. Record the contact sequence too: repeated readings without unloading and repeated unload/reload cycles are different experiments. Describe which one you ran.
4. Agree the analysis and acceptance criteria. Choose the repetition count and criteria with the responsible test and quality engineers before the run. This guide does not prescribe a universal count or scatter limit. Keep the raw readings and their order so the analysis can distinguish spread from a trend.

These are proposed planning steps for applying the references to an ATE setup, not a standard-mandated qualification procedure.

Look at spread and changes over time

Plot the values in measurement order. Review their centre, spread, and any time-related movement. Standard deviation is a common numerical description of measurement imprecision. Keep the conditions and sample count beside the number so someone else can interpret it. JCGM: precision, Note 1.

A short repeatability run does not cover every shift, operator, instrument, or environmental condition. NIST distinguishes short-term variation from variation across days and longer periods. NIST: variability.

If results change after you alter the contact sequence or another setup factor, treat that as a lead for investigation. The difference alone does not identify a failed socket, board, tester, or device. Keep each condition identifiable rather than pooling unlike runs into one summary.

Know when you need a broader study

If the question is whether results remain dependable across operators, equipment, or time, design a study that includes those factors. NIST's gauge R&R guidance covers repeatability, reproducibility, stability, and bias. A repeated run under one fixed setup addresses only part of that picture. NIST: gauge R&R studies.

For ongoing monitoring, NIST describes measuring a stable check standard over time against an established baseline. That helps detect changes in the measurement process beyond a single short run. NIST: controlling bias and variability.

What to keep with the results

Keep the item identity, parameter and units, tester and site, program revision, interface identification, operator, timestamps, environmental conditions, contact sequence, raw readings, specification limits, and agreed acceptance criteria together. This is a proposed ATE investigation checklist. NIST provides the underlying measurement-record approach. NIST: data collection.

For a new test cell, agree how this check fits into correlation and production acceptance before equipment arrives. See test-cell buy-off.

Discuss a test setup with NXTREV: send your tester model, device/package, affected parameter, and a non-confidential description of when the result changes to sales@nxtrev.org.