4.
Process Modeling
4.5.
Use and Interpretation of Process Models
4.5.1.
What types of predictions can I make using the model?
4.5.1.2.

How can I predict the value and and estimate the uncertainty of a single response?


A Different Type of Prediction

In addition to estimating the average value of the response variable for a given combination of
preditor values, as discussed on the previous page, it is also possible
to make predictions of the values of new measurements or observations from a process. Unlike the
true average response, a new measurement is often actually observable in the future. However,
there are a variety of different situations in which a prediction of a measurement value may be
more desirable than actually making an observation from the process.

Example

For example, suppose that a concrete supplier needs to supply concrete of a specified measured
strength for a particular contract, but knows that strength varies systematically with the
ambient temperature when the concrete is poured. In order to be sure that the concrete will meet
the specification, prior to pouring, samples from the batch of raw materials can be mixed, poured,
and measured in advance, and the relationship between temperature and strength can be modeled.
Then predictions of the strength across the range of possible field temperatures can be used to
ensure the product is likely to meet the specification. Later, after the concrete is poured (and
the temperature is recorded), the accuracy of the prediction can be verified.

\(\hat{y}=f(\vec{x},\hat{\vec{\beta}})\)

The mechanics of predicting a new measurement value associated with a combination of predictor
variable values are similar to the steps used in the estimation of the average response value. In
fact, the actual estimate of the new measured value is obtained by evaluating the estimated
regression function at the relevant predictor variable values, exactly as is done for the average
response. The estimates are the same for these two quantities because, assuming the model fits
the data, the only difference between the average response and a particular measured response is
a random error. Because the error is random, and has a mean of zero, there is no additional
information in the model that can be used to predict the particular response beyond the information
that is available when predicting the average response.

Uncertainties Do Differ

As when estimating the average response, a probabilistic interval is used when predicting a new
measurement to provide the information needed to make engineering or scientific conclusions.
However, even though the estimates of the average response and particular response values are
the same, the uncertainties of the two estimates do differ. This is because the uncertainty
of the measured response must include both the uncertainty of the estimated average response
and the uncertainty of the new measurement that could conceptually be observed. This
uncertainty must be included if the interval that will be used to summarize the prediction result
is to contain the new measurement with the specified confidence. To help distinguish the two types
of predictions, the probabilistic intervals for estimation of a new measurement value are called
prediction intervals rather than confidence intervals.

Standard Deviation of Prediction

The estimate of the standard deviation of the predicted value, \(\hat{\sigma}_f\),
is obtained as described
earlier. Because the residual standard deviation describes the
random variation in each individual measurement or observation from the process, \(\hat{\sigma}\),
the estimate of the residual standard deviation obtained when
fitting the model to the data, is used to account for the extra uncertainty needed to predict a
measurement value. Since the new observation is independent of the data used to fit the model,
the estimates of the two standard deviations are then combined by "rootsumofsquares" or
"in quadrature", according to standard formulas for computing variances, to obtain the standard
deviation of the prediction of the new measurement, \(\hat{\sigma}_p\).
The formula for \(\hat{\sigma}_p\)
is
$$ \hat{\sigma}_p = \sqrt{\hat{\sigma}^2 + \hat{\sigma}^2_f} \, . $$

Coverage Factor and Prediction Interval Formula

Because both \(\hat{\sigma}_f\)
and \(\hat{\sigma}_p\)
are mathematically nothing more than different scalings of \(\hat{\sigma}\),
and coverage factors from the t distribution
only depend on the amount of data available for estimating \(\hat{\sigma}\),
the coverage factors are the same for confidence
and prediction intervals. Combining the coverage factor and the standard deviation of the
prediction, the formula for constructing prediction intervals is given by
$$ \hat{y} \pm t_{1\alpha/2,\nu} \cdot \hat{\sigma}_p $$
As with the computation of confidence intervals, some software may
provide the total uncertainty for the prediction interval given the equation above, or may
provide the lower and upper prediction bounds. As suggested before, however, it is a good idea
to test the software on an example for which prediction limits are already available to make
sure that the software is computing the expected type of intervals.

Prediction Intervals for the Example Applications

Computing prediction intervals for the measured pressure in the
Pressure/Temperature example, at temperatures of 25,
45, and 65, and for the measured torque on specimens from the
polymer relaxation example at different
times and temperatures, gives the results listed in the tables below. Note: the number of
significant digits shown is larger than would normally be reported. However, as many significant
digits as possible should be carried throughout all calculations and results should only be rounded
for final reporting. If reported numbers may be used in further calculations, then they should
not be rounded even when finally reported. A useful rule for rounding final results that will not
be used for further computation is to round all of the reported values to one or two significant
digits in the total uncertainty, \(t_{1\alpha/2,\nu} \, \hat{\sigma}_p\).
This is the convention for rounding that has been used in the tables below.

Pressure / Temperature Example

\(x\)

\(\hat{y}\)

\(\hat{\sigma}\)

\(\hat{\sigma}_f\)

\(\hat{\sigma}_p\)

\(t_{1\alpha/2,\nu}\)

\(t_{1\alpha/2,\nu} \, \hat{\sigma}_p\)

Lower 95% Prediction Bound

Upper 95% Prediction Bound


25 
106.0025 
4.299099 
1.1976162 
4.462795 
2.024394 
9.034455 
97.0 
115.0 
45 
184.6053 
4.299099 
0.6803245 
4.352596 
2.024394 
8.811369 
175.8 
193.5 
65 
263.2081 
4.299099 
1.2441620 
4.475510 
2.024394 
9.060197 
254.1 
272.3 

Polymer Relaxation Example

\(x_1\)

\(x_2\)

\(\hat{y}\)

\(\hat{\sigma}\)

\(\hat{\sigma}_f\)

\(\hat{\sigma}_p\)

\(t_{1\alpha/2,\nu}\)

\(t_{1\alpha/2,\nu} \, \hat{\sigma}_p\)

Lower 95% Prediction Bound

Upper 95% Prediction Bound


20 
25 
5.586307 
0.04341221 
0.02840153 
0.05187742 
2.000298 
0.10377030 
5.48 
5.69 
80 
25 
4.998012 
0.04341221 
0.01217109 
0.04508609 
2.000298 
0.09018560 
4.91 
5.09 
20 
50 
6.960607 
0.04341221 
0.01371149 
0.04552609 
2.000298 
0.09106573 
6.87 
7.05 
80 
50 
5.342600 
0.04341221 
0.01007761 
0.04456656 
2.000298 
0.08914639 
5.25 
5.43 
20 
75 
7.521252 
0.04341221 
0.01205401 
0.04505462 
2.000298 
0.09012266 
7.43 
7.61 
80 
75 
6.220895 
0.04341221 
0.01330727 
0.04540598 
2.000298 
0.09082549 
6.13 
6.31 

Interpretation of Prediction Intervals

Simulation of many sets of data from a process model provides a good way to obtain a detailed
understanding of the probabilistic nature of the prediction intervals. The main advantage of
using simulation is that it allows direct comparison of how prediction intervals constructed
from a limited amount of data relate to the measured values that are being estimated.


The plot below shows 95 % prediction intervals computed from 50 independently generated data
sets that follow the same model as the data in the Pressure/Temperature example. Random errors
from the normal distribution with a mean of zero and a known standard deviation are added to
each set of true temperatures and true pressures that lie on a perfect straight line to produce
the simulated data. Then each data set is used to compute a prediction interval for a newly
observed pressure at a temperature of 65. The newly observed measurements, observed
after making the prediction, are noted with an "X" for each data set.

Prediction Intervals Computed from 50 Sets of Simulated Data


Confidence Level Specifies LongRun Interval Coverage

From the plot it is easy to see that not all of the intervals contain the pressure values observed
after the prediction was made. Data set 4 produced an interval that did not capture the newly
observed pressure measurement at a temperature of 65. However, for 49 out of 50, or not much over
95 % of the data sets, the prediction intervals did capture the measured pressure. When the number
of data sets was increased to 5000, prediction intervals computed for 4734, or 94.68 %, of the
data sets covered the new measured values. Finally, when the number of data sets was increased
to 10000, 94.92 % of the confidence intervals computed covered the true average pressure. Thus,
the simulation shows that although any particular prediction interval might not cover its
associated new measurement, in repeated experiments this method produces intervals that contain
the new measurements at the rate specified by the user as the confidence level.

Comparison with Confidence Intervals

It is also interesting to compare these results to the analogous results
for confidence intervals. Clearly the most striking difference between the two plots is in
the sizes of the uncertainties. The uncertainties for the prediction intervals are much larger
because they must include the standard deviation of a single new measurement, as well as the
standard deviation of the estimated average response value. The standard deviation of the
estimated average
response value is lower because a lot of the random error that is in each measurement cancels out
when the data are used to estimate the unknown parameters in the model. In fact, if as the sample
size increases, the limit on the width of a confidence interval approaches zero while the limit on
the width of the prediction interval as the sample size increases approaches
\(z_{1\alpha/2} \, \hat{\sigma}\).
Understanding the different types of intervals and
the bounds on interval width can be important when planning an experiment that requires a result
to have no more than a specified level of uncertainty to have engineering value.

Interpretation Summary

To summarize the interpretation of the probabilistic nature of confidence intervals in words:
in independent, repeated experiments, \(100(1\alpha) \, \%\)
of the intervals will be expected cover their true values, given that the assumptions needed
for the construction of the intervals hold.
