Overview: Design and Analysis of Key Comparisons
|
|
Introduction
|
With the recent signing of the Mutual Recognition Arrangement (MRA), National
Metrology Institutes (NMI's) and Regional Metrology Organizations (RMO's)
around the world have committed themselves to establishing the equivalence of
their measurement standards. Currently, however, there is not wide agreement
on the best statistical procedures for analysis of the international
interlaboratory experiments called Key Comparisons which are used to
establish such equivalence. To help build consensus on the best statistical
procedures for this work, we propose to develop a unified statistical
framework and detailed guidance for the Key Comparisons process in
collaboration with our colleagues at NIST and other NMI's.
|
|
Technical Strategy
|
Key comparison testing is at its core a statistical process. Data is collected,
statistically analyzed and the degrees of equivalence between the participating
laboratories are estimated. For maximal effectiveness and efficiency,
however, we believe that the data collection phase needs a statistically sound
experimental design. This includes decisions as to the number of the traveling
standards and the pattern of the comparison (i.e. how often the transfer
standards travel back to the coordinating laboratory for monitoring). It further
includes determination of the number of repetitions for each measurement at each
lab. In complex experiments, it should also include the sequence of the
experimental runs and the randomization of the experimental units at each lab.
The analysis phase of the key comparison process deals with issues of summarization
of the NMI measurements and assessment of the corresponding uncertainty. The basic
principles for this are well known and internationally accepted. However, there
are various procedural alternatives and it will be valuable to study these and
set out general guidelines to their use. The final phase of the key comparison
process is the determination and reporting of the level of equivalence among
the participating labs.
|
|
|
In order to address the design problems, we propose to draw on the expertise of
scientists across NIST to study the issues of the experimental design phase,
ultimately identifying a core set of principles according to which a proposed
experimental design can be judged as to its adequacy to handle the proposed
task. In the later stages of our work we will develop a set of efficient
experimental designs and ultimately an interactive tool which the leading lab
will be able to use to produce an efficient test protocol. We envision that
some of this work will require original research in optimal experimental design
and expertise in computing. In order to solidify the statistical foundations of
the analysis of Key Comparison participants' data and the establishment of
equivalence among the participating labs, we propose to study the alternatives,
possibly devise new methods, and make statistically best practices available to
the international metrological community.
|
|
Expectations
|
Meeting the research challenges described above will provide a more detailed
understanding of the issues surrounding Key Comparisons, new methods for
designing efficient Key Comparison experiments, and appropriate methods for the
analysis of the subsequently obtained data. As a consequence, Key Comparison
results will improve. Because scientists across NIST will be directly involved
in this project, our results will be able to be rapidly integrated into the
institutional culture.
|
|
|
In addition to completion of the statistical research needed for the optimal
design and analysis of Key Comparisons, the main deliverable envisioned from
this project is newly developed software that implements these statistical
methods. We also plan to produce multimedia- or web-based tutorials for NIST
staff and others on a variety of topics integral to Key Comparisons such as
data reporting and uncertainty computation.
|
|
Context of Project
|
International standards projects like this one typically have fundamental, but
difficult to quantify, impacts. In the big picture, with the lowering of
trade barriers around the world, this project will facilitate international
trade. In a local context, this project directly supports NIST's efforts to
establish equivalence with other NMI's and RMO's under the MRA.
|
