DIFFERENCE OF PORPORTION CONFIDENCE LIMITS

DIFFERENCE OF PROPORTION CONFIDENCE LIMITS

Analysis Command

Generates a confidence interval for the difference between two proportions.

Given a set of N1 observations in a variable X1 and a set of N2 observations in a variable X2, we can compute the proportion of successes in each sample as p1 and p2. We can then compute the difference of the proportions as P1 - P2. In addition, the DIFFERENCE OF PROPORTION CONFIDENCE LIMITS command computes a confidence interval for the difference between the two proportions of successes.

For most applications, you typically define success as a "1" and failure as a "0". However, in Dataplot you can also specify the range of values that constitute success (all other values denote failure) by entering the command

ANOP LIMITS <lower limit> <upper limit>

The default limits are 0.5 and 1.5 so that if your data contains 0's and 1's this command can be omitted.

Several methods have been proposed for the confidence limits for the difference between binomial proportions. The following methods are currently supported in Dataplot (other methods are available)

• WALD (or NORMAL APPROXIMATION)

  The Wald two-sided confidence interval is

  \( (\hat{p}_1 - \hat{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\hat{p}_1 (1 - \hat{p}_1)}{n_1} + \frac{\hat{p}_2 (1 - \hat{p}_2)}{n_2}} \)

  where

  \( n_1 \) is the number of observations in sample one
  \( n_2 \) is the number of observations in sample two
  \( \hat{p}_1 \) is the proportion of successses in sample one
  \( \hat{p}_2 \) is the proportion of successses in sample two
  \( z_{\alpha} \) is the percent point function of the normal distribution
  

  This method is commonly used. However, Agresti and Caffo pointed out that this method does not always perform well in the sense that the actual coverage probabilities can be less than (and often substantially less than) the nominal coverage probabilities.

• ADJUSTED WALD (or AGRESTI-CAFFO)

  Agresti and Caffo recommend the following confidence limit

  \( (\tilde{p}_1 - \tilde{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\tilde{p}_1 (1 - \tilde{p}_1)}{n_1+2} + \frac{\tilde{p}_2 (1 - \tilde{p}_2)}{n_2+2}} \)

  where

  \( \tilde{p}_1 = \frac{X_1 + 1}{n_1 + 2} \) where \( X_1 \) is the number of successes in sample one
  \( \tilde{p}_2 = \frac{X_2 + 1}{n_2 + 2} \) where \( X_2 \) is the number of successes in sample two
  

  The justification and derivation of this interval is given in the Agresti Caffo paper. This interval improves the coverage probabilities of the standard Wald interval.

• CARLIN AND LOUIS BAYESIAN

  Carlin and Louis propsed the following interval based on a Bayesian derivation

  \( (\tilde{p}_1 - \tilde{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\tilde{p}_1 (1 - \tilde{p}_1)}{n_1+3} + \frac{\tilde{p}_2 (1 - \tilde{p}_2)}{n_2+3}} \)

  where \( \tilde{p}_1 \) and \( \tilde{p}_2 \) are defined as they are for the Agresti-Caffo interval.

  This interval also improves upon the Wald interval. It is slightly less conservative with sometimes slightly narrower intervals than the Agresti-Caffo interval. However, it can also at times have poorer coverage probabilities than the Agresti-Caffo interval.

To specify the method to use, enter the command

SET DIFFERENCE OF BINOMIAL METHOD <WALD/ADJUSTED WALD/BAYESIAN>

The default is the adjusted Wald (Agresti-Caffo) interval.

Dataplot computes this inverval for a number of different probability levels.

DIFFERENCE OF PROPORTION CONFIDENCE LIMITS <y1> <y2>
                        <SUBSET/EXCEPT/FOR qualification>
where <y1> is the first response variable;
            <y2> is the second response variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

ANOP LIMITS 0.50 1.5
DIFFERENCE OF PROPORTION CONFIDENCE LIMITS Y1 Y2


SET DIFFERENCE OF BINOMIAL METHOD ADJUSTED WALD DIFFERENCE OF PROPORTION CONFIDENCE LIMITS Y1 Y2 ...
            SUBSET TAG > 2

A table of confidence intervals is printed for alpha levels of 50.0, 75.0, 90.0, 95.0, 99.0, 99.9, 99.99, and 99.999. The sample size, sample number of successes, and sample proportion of successes are also printed.

The following statistics are also supported
LET ALPHA = <value>
LET A = DIFF OF BINOMIAL PROPORTIONS LOWER CONFIDENCE ...
                LIMIT Y1 Y2
LET A = DIFF OF BINOMIAL PROPORTIONS UPPER CONFIDENCE ...
                LIMIT Y1 Y2


In addition to the above LET commands, built-in statistics are supported for about 20+ different commands (enter HELP STATISTICS for details).

The adjusted Wald (Agresti-Caffo) interval is the default

None

ANOP LIMITS	=	Specify success region for proportions.
PROPORTION CONFIDENCE LIMITS	=	Compute a proportions confidence interval.
ANOP PLOT	=	Generate an analysis of proportions plot.
CONFIDENCE LIMITS	=	Generate the confidence limits for the mean.

Snedecor and Cochran (1989), "Statistical Methods", Eigth Edition, Iowa State University Press, pp. 125-128.

Agresti and Caffo (2000), "Simple and Effective Confidence Intervals for Proportions and Differences of Proportions Result From Adding Two Successes and Two Failures", The American Statistician, Vol. 54, No. 4, pp. 280-288.

Carlin and Louis (1996), "Bayes and Empirical Bayes Methods for Data Analysis", Chapman and Hall.

Categorical Data Analysis

1999/05
2017/11: Added the adjusted Wald (Agresti-Caffo) and Bayesian intervals
2017/11: Made the adjusted Wald (Agresti-Caffo) interval the default

 
let n1 = 40
let nsuc1 = 8
let y1 = 0 for i = 1 1 n1
let y1 = 1 for i = 1 1 nsuc1
.
let n2 = 30
let nsuc2 = 11
let y2 = 0 for i = 1 1 n2
let y2 = 1 for i = 1 1 nsuc2
.
.          Now do proportions confidence interval
.
set write decimals 5
let alpha = 0.95
.
set difference of binomial method wald
difference of proportion confidence interval y1 y2
set difference of binomial method adjusted wald
difference of proportion confidence interval y1 y2
set difference of binomial method bayesian
difference of proportion confidence interval y1 y2
    

This command generates the following output.

 
             Difference of Proportions Two-Sided Confidence Limits
                                 (Wald Method)
  
 First Response Variable:  Y1
 Second Response Variable: Y2
  
 Sample 1:
 Number of Observations:                  40
 Number of Successes:                     8
 Proportion of Successes:                 0.20000
  
 Sample 2:
 Number of Observations:                  30
 Number of Successes:                     11
 Proportion of Successes:                 0.36667
  
 Difference Between Proportions:          -0.16667
  
  
  
 ------------------------------------------
   Confidence          Lower          Upper
    Value (%)          Limit          Limit
 ------------------------------------------
       50.000       -0.23975       -0.09358
       75.000       -0.29131       -0.04202
       90.000       -0.34489        0.01156
       95.000       -0.37904        0.04570
       99.000       -0.44577        0.11244
       99.900       -0.52321        0.18988
       99.990       -0.58823        0.25490
       99.999       -0.64529        0.31195
  
  
             Difference of Proportions Two-Sided Confidence Limits
                    (Agresti-Caffo (Adjusted Wald) Method)
  
 First Response Variable:  Y1
 Second Response Variable: Y2
  
 Sample 1:
 Number of Observations:                  40
 Number of Successes:                     8
 Proportion of Successes:                 0.20000
  
 Sample 2:
 Number of Observations:                  30
 Number of Successes:                     11
 Proportion of Successes:                 0.36667
  
 Difference Between Proportions:          -0.16667
  
  
  
 ------------------------------------------
   Confidence          Lower          Upper
    Value (%)          Limit          Limit
 ------------------------------------------
       50.000       -0.23252       -0.08891
       75.000       -0.28318       -0.03825
       90.000       -0.33582        0.01439
       95.000       -0.36936        0.04794
       99.000       -0.43493        0.11350
       99.900       -0.51101        0.18958
       99.990       -0.57489        0.25346
       99.999       -0.63095        0.30952
  
  
             Difference of Proportions Two-Sided Confidence Limits
                      (Carlin and Louis Bayesian Method)
  
 First Response Variable:  Y1
 Second Response Variable: Y2
  
 Sample 1:
 Number of Observations:                  40
 Number of Successes:                     8
 Proportion of Successes:                 0.20000
  
 Sample 2:
 Number of Observations:                  30
 Number of Successes:                     11
 Proportion of Successes:                 0.36667
  
 Difference Between Proportions:          -0.16667
  
  
  
 ------------------------------------------
   Confidence          Lower          Upper
    Value (%)          Limit          Limit
 ------------------------------------------
       50.000       -0.23151       -0.08992
       75.000       -0.28146       -0.03997
       90.000       -0.33337        0.01194
       95.000       -0.36644        0.04501
       99.000       -0.43109        0.10966
       99.900       -0.50611        0.18468
       99.990       -0.56909        0.24766
       99.999       -0.62437        0.30294