3.1.11 Statistical Analysis of Network Analyzer Measurements

Jack C.M. Wang

Statistical Engineering Division, CAML

John R. Juroshek

Electromagnetic Fields Division, EEEL

George P. McCabe

Department of Statistics, Purdue University

Modern vector network analyzers can make hundreds of measurements in minutes. The accuracy of these measurements is dependent not only on the accuracy of the network analyzer's electronic and microwave hardware, but also on other external factors. Assessing the accuracy of a network analyzer is a difficult, multidimensional problem.

For the past six years, the Automatic RF Techniques Group (ARFTG) has conducted a measurement comparison program for vector network analyzers. The purpose of this program is to help participants assess the accuracy of their measurements by comparing their measurements to those of other laboratories. Five traveling measurement kits are currently in circulation for GPC-7, Type-N, 3.5 mm, 2.92 mm, and 2.4 mm connector types. The participants measures the devices in these kits and then sends the results to NIST for analysis. NIST serves as the pilot laboratory and is responsible for maintaining the data base and for analyzing the data. The participants normally receives a summary report within a week or two after sending the data in for analysis.

The analysis is complicated by the fact that each participant produces a large amount of data. The uncertainties in the measurements are typically frequency sensitive and increase with increasing frequency. Analyzing the data at few specific frequencies is normally unsatisfactory. The variability of the data is also substantial. Generally, the measurements of most participants agree to within 1% or better, while a few participants differ by 10% or more. The statistical measurements of average and standard deviation are substantially distorted by these few participants. Removing bad data manually from the analysis is impractical as the number of measurements grows, the identification of outliers becomes more subjective.

We use robust analysis techniques to assess and compare the measurement capability of the participants. These techniques are based on calculating the deviation from the median, and are not unduly influenced by outliers or bad data. The performance of each participant is summarized by three numbers: the mean deviation, and the 10th and 90th percentile deviations.

Graphics are used extensively to summarize the comparisons. For example, the accompanying figure plots the 10th percentile ($\diamond$), average ($\oplus$), and 90th percentile ($\times$) deviations in |S₁₁| (the magnitude of the S₁₁ parameter) for all participants and all five connector types and the 20 dB attenuator. The medians of the 10th percentile, average, and 90th percentile deviations are shown by dotted, solid, and dashed lines respectively.

Figure 11: Deviation in |S₁₁| versus connector type for all 20 dB attenuators.

       

Date created: 7/20/2001
Last updated: 7/20/2001
Please email comments on this WWW page to sedwww@nist.gov.