7. Product and Process Comparisons
7.4. Comparisons based on data from more than two processes
7.4.6. How can we make multiple comparisons?

7.4.6.1.

Tukey's method

Tukey's method considers all possible pairwise differences of means at the same time
 
 

The studentized range q
 
 
 
 
 
 
 
 

The distribution of q is tabulated in many textbooks and can be calculated using Dataplot
 

The Tukey method applies simultaneously to the set of all pairwise comparisons 

{m_i - m_j}

The confidence coefficient for the set, when all sample sizes are equal, is exactly 1-a.  For unequal sample sizes, the confidence coefficient is greater than 1-a. In other words, the Tukey method is conservative when there are unequal sample sizes.

Studentized Range Distribution 

The Tukey method uses the studentized range distribution. 
Suppose we have r independent observations y₁, ..., y_r  from a normal distribution with mean µ and variance  s². Let w be the range for this set , i.e.,  the maximum -  the minimum. Now suppose that we have an estimate s² of the variance s² which is based on n degrees of freedom and is independent of the y_i . 
The studentized range is defined as 

q_{r, n} = w/s

The distribution of q has been tabulated and appears in many textbooks on statistics. In addition, Dataplot has a CDF function (SRACDF) and a percentile function (SRAPPF) for q.

As an example, let r = 5 and n = 10. The 95th percentile is q_.05;5,10 = 4.65. This means: 

So, if we have five observations from a normal distribution, the probability is .95 that their range is not more than 4.65 times as great as an independent sample standard deviation estimate that has 10 degrees of freedom. 
 

Tukey's Method 

The Tukey confidence limits for all pairwise comparisons with confidence coefficient of at least 1-a are: 

Notice that the point estimator and the estimated variance are the same as those for a single pairwise comparison that was illustrated previously. The only difference between the confidence limits for simultaneous comparisons and those for a single comparison is the multiple of the estimated standard deviation. 
 

Example 

We use the data from a previous example. 

The set of all pairwise comparisons consists of: 

              µ₂-µ₁,  µ₃-µ₁,  µ₁-µ_4,  µ₂-µ₃,  µ₂-µ₄,  µ₃-µ₄ 

Assume we want a confidence coefficient of 95 percent, or .95. Since r = 4 and n_t = 20, the required percentile of the studentized range distribution is q_{.05; 4,16}. Using the Tukey method formula for each of the six comparisons yields: 

The simultaneous pairwise comparisons indicate that the differences µ₁ -µ₄ and µ₂ - µ₃ are not significant different from 0 (their confidence intervals include 0), and all the other pairs are  significantly different from one another. 

It is possible to work with unequal sample sizes. In this case, one has to calculate the estimated standard deviation for each pairwise comparison. The Tukey procedure for unequal sample sizes is sometimes referred to as the Tukey-Kramer Method.