5.
Process Improvement
5.6. Case Studies 5.6.1. Eddy Current Probe Sensitivity Case Study


Model Validation 
In the Important Factors and
Parsimonious Prediction section, we selected the following model
The next step is to validate the model. The primary method of model validation is graphical residual analysis; that is, through an assortment of plots of the differences between the observed data Y and the predicted value \( \scriptsize \hat{Y} \) from the model. For example, the design point (1, 1, 1) has an observed data point (from the Background and data section) of Y = 1.70, while the predicted value from the above fitted model for this design point is


Table of Residuals 
If the model fits well, \( \scriptsize \hat{Y} \)
should be near Y for all eight design points. Hence the eight
residuals should all be near zero. The eight predicted values and
residuals for the model with these data are:
X1 X2 X3 Observed Predicted Residual  1 1 1 1.70 1.54125 0.15875 +1 1 1 4.57 4.64375 0.07375 1 +1 1 0.55 0.67375 0.12375 +1 +1 1 3.39 3.77625 0.38625 1 1 +1 1.51 1.54125 0.03125 +1 1 +1 4.59 4.64375 0.05375 1 +1 +1 0.67 0.67375 0.00375 +1 +1 +1 4.29 3.77625 0.51375 

Residual Standard Deviation 
What is the magnitude of the typical residual? There are several
ways to compute this, but the statistically optimal measure is
the residual standard deviation:


How Should Residuals Behave? 
If the prediction equation is adequate, the residuals from that equation
should behave like random
drawings (typically from an approximately normal distribution), and
should, since presumably random, have no structural relationship with
any factor. This includes any and all potential terms (X1,
X2, X3,
X1*X2,
X1*X3,
X2*X3,
X1*X2*X3).
Further, if the model is adequate and complete, the residuals should have no structural relationship with any other variables that may have been recorded. In particular, this includes the run sequence (time), which is really serving as a surrogate for any physical or environmental variable correlated with time. Ideally, all such residual scatter plots should appear structureless. Any scatter plot that exhibits structure suggests that the factor should have been formally included as part of the prediction equation. Validating the prediction equation thus means that we do a final check as to whether any other variables may have been inadvertently left out of the prediction equation, including variables drifting with time. The graphical residual analysis thus consists of scatter plots of the residuals versus all three factors and four interactions (all such plots should be structureless), a scatter plot of the residuals versus run sequence (which also should be structureless), and a normal probability plot of the residuals (which should be near linear). We present such plots below. 

Residual Plots 
The first plot is a normal probability plot of the residuals. The second plot is a run sequence plot of the residuals. The remaining plots show the residuals plotted against each of the factors and each of the interaction terms. 

Conclusions 
We make the following conclusions based on the above plots.
