2. Measurement Process Characterization
2.3. Calibration
2.3.6. Instrument calibration over a regime
2.3.6.7. Uncertainties of calibrated values

2.3.6.7.3.

Comparison of check standard analysis and propagation of error

Propagation of error for the linear calibration

The analysis of uncertainty for calibrated values from a linear calibration line can be addressed using propagation of error. On the previous page, the uncertainty was estimated from check standard values.

Estimates from calibration data

The calibration data consist of 40 measurements with an optical imaging system on 10 line width artifacts. A linear fit to the data using the software package Omnitab (Omnitab 80 ) gives a calibration curve with the following estimates for the intercept, a, and the slope, b:

   a       .23723513
   b       .98839599
-------------------------------------------------------
 RESIDUAL STANDARD DEVIATION =            .038654864
   BASED ON DEGREES OF FREEDOM        40 -   2 =  38

with the following variances and covariances:

   a      2.2929900-04
   b     -2.9703502-05   4.5966426-06

Propagation of error using Mathematica

The propagation of error is accomplished with the following instructions using the software package Mathematica (Wolfram):

f=(y -a)/b
dfdy=D[f, {y,1}]
dfda=D[f, {a,1}]
dfdb=D[f,{b,1}]
u2 =dfdy^2 sy^2 + dfda^2 sa2 + dfdb^2 sb2 + 2 dfda dfdb sab2
% /. a-> .23723513
% /. b-> .98839599
% /. sa2 -> 2.2929900 10^-04
% /. sb2 -> 4.5966426 10^-06
% /. sab2 -> -2.9703502 10^-05
% /. sy -> .038654864
u2 = Simplify[%]
u = u2^.5
Plot[u, {y, 0, 12}]

Standard deviation of calibrated value X'

The output from Mathematica gives the standard deviation of a calibrated value, X', as a function of instrument response:

                                         -6  2 0.5
(0.00177907 - 0.0000638092 y + 4.81634 10   y )

Graph showing standard deviation of calibrated value X' plotted as a function of instrument response Y' for a linear calibration

Comparison of check standard analysis and propagation of error

Comparison of the analysis of check standard data, which gives a standard deviation of 0.062 µm, and propagation of error, which gives a maximum standard deviation of 0.042 µm, suggests that the propagation of error may underestimate the type A uncertainty. The check standard measurements are undoubtedly sampling some sources of variability that do not appear in the formal propagation of error formula.