8. Assessing Product Reliability
8.2. Assumptions/Prerequisites
8.2.3. How can you test reliability model assumptions?

8.2.3.1.

Visual tests

A visual test of a model is a simple plot that tells us at a glance whether the model is consistent with the data

We have already seen many examples of visual tests of models. These were: Probability Plots, Cum hazard Plots, Duane Plots and Trend Plots. In all but the Trend Plots, the model was "tested" by how well the data points followed a straight line. In the case of the Trend Plots, we looked for curvature away from a straight line (cum repair plots) or increasing or decreasing size trends (inter arrival times and reciprocal inter-arrival times). 

These simple plots are a powerful diagnostic tool since the human eye can often detect patterns or anomalies in the data by studying graphs. That kind of invaluable information would be lost if the analyst only used quantitative statistical tests to check model fit. Every analysis should include as many visual tests as are applicable. 

Advantages of Visual Tests

1. Easy to understand and explain.
2. Can occasionally reveal patterns or anomalies in the data.
3. When a model "passes" a visual test, it is somewhat unlikely any quantitative statistical test will "reject" it (the human eye is less forgiving and more likely to detect spurious trends)

Combine visual tests with formal quantitative tests for the "best of both worlds" approach

Disadvantages of Visual Tests

1. Visual tests are subjective.
2. They do not quantify how well or how poorly a model fits the data.
3. They are of little help in choosing between two or more competing models that both appear to fit the data.
4. Simulation studies have shown that correct models may often appear to not fit well by sheer chance - it is hard to know when visual evidence is strong enough to reject what was previously believed to be a correct model.

You can retain the advantages of visual tests and remove their disadvantages by combining data plots with formal statistical tests of goodness of fit or trend.