SED navigation bar go to SED home page go to Dataplot home page go to NIST home page SED Home Page SED Staff SED Projects SED Products and Publications Search SED Pages
Dataplot Vol 1 Vol 2

MCNEMAR TEST

Name:
    MCNEMAR TEST
Type:
    Analysis Command
Purpose:
    Perform a McNemar test for independence in a 2x2 table.
Description:
    Given two paired variables where each variable has exactly two possible outcomes (coded as 0 and 1), the McNemar test can be used to test if there is a statistically significant difference between the probability of a (0,1) pair and the probability of a (1,0) pair. For example, this test is often used for the situation where we are testing for the prescence (= 1) or absence (= 0) of something and variable 1 is the state before an experiment and variable 2 is the state after the experiment (i.e., did the experiment have an effect?).

    We can summarize the data in the following table. We call variable 1 X and variable 2 Y.

      Yi = 0 Yi = 1

      Xi = 0 a b
      Xi = 1 c d

    The McNemar test has the the following assumptions:

    1. The pairs (Xi,Yi) are mutually independent.

    2. Each Xi and Yi can be assigned to one of two possible categories.

    3. The difference

        P(Xi = 0, Yi = 1) - P(Xi = 1, Yi = 0)

      is negative for all i or zero for all i or positive for all i.

    If we let P1 = P(Xi = 0, Yi = 1) and P2 = P(Xi = 1, Yi = 0), then the McNemar test can be formulated as follows.

      H0: P1 = P2       for all i
      (this is equivalent to "new equal to old" or "before equal to after")
      Ha: P1 ≠ P2       for all i
      (this is equivalent to "new not equal to old" or "before not equal to after")
      Test Statistic: If b + c > 20,

        T1 = (b - c)2/(b + c)

      If b + c ≤ 20,

        T2 = b

      There is also a continuity corrected version of the T1:

        T1' = (|b - c| - 1)2/(b + c)
      Significance Level: \( \alpha \)
      Critical Region: T1 > \( \chi_{1-\alpha,1}^{2} \)

      T2 ≤ BINPPF(\( \alpha/2 \),0.5,b+c)
      T2 ≥ BINPPF(\( \alpha/2 \),0.5,b+c)

      where \( \chi_{\alpha,\nu}^{2} \) and BINPPF denote the percent point functions of the chi-square and binomial distributions, respectively

      Conclusion: Reject the null hypothesis if the test statistic is in the critical region
Syntax 1:
    MCNEMAR TEST <y1> <y2>             <SUBSET/EXCEPT/FOR qualification>
    where <y1> is the first response variable;
                <y2> is the second response variable;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.

    This syntax is used for the raw data case (i.e., <y1> and <y2> are variable containing 0's and 1's).

Syntax 2:
    MCNEMAR TEST <m>             <SUBSET/EXCEPT/FOR qualification>
    where <m> is a matrix containing the two-way table;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.

    This syntax is used for the case where we the data have already been cross-tabulated into a two-way table.

Syntax 3:
    MCNEMAR TEST <n11> <n12> <n21> <n22>
    where <n11> is a parameter containing the value for row 1, column 1 of a 2x2 table (i.e, a);
    <n12> is a parameter containing the value for row 1, column 2 of a 2x2 table (i.e., b);
    <n21> is a parameter containing the value for row 2, column 1 of a 2x2 table (i.e., c);
    <n22> is a parameter containing the value for row 2, column 2 of a 2x2 table (i.e., d).

    This syntax is used for the special case where you have a 2x2 table. In this case, you can enter the 4 values directly, although you do need to be careful that the parameters are entered in the order expected above.

Examples:
    MCNEMAR TEST Y1 Y2
    MCNEMAR TEST M
    MCNEMAR TEST N11 N12 N21 N22
Note:
    The McNemar test is essentially a sign test. Conover discusses how to transform a McNemar test to an explicit sign test.

    The McNemar test is also a special case of the Cochran test with c = 2 (enter HELP COCHRAN TEST for details).

Note:
    Dataplot saves the following internal parameters:

      STATVAL = the value of the McNemar test statistic
      STATCDF = the cdf for the McNemar test statistic
Default:
    None
Synonyms:
    None
Related Commands: Reference:
    Conover (1999), Practical Nonparametric Statistics, Third Edition, Wiley, pp. 166-169.

    Fleiss, Levin, and Paik (2003), Statistical Methods for Rates and Proportions, Third Edition, p. 375.

Applications:
    Categorical Data Analysis
Implementation Date:
    2007/3
Program:
     
    let n11 = 63
    let n21 = 4
    let n12 = 21
    let n22 = 12
    .
    read matrix m
    2 4
    4 6
    end of data
    .
    mcnear test n11 n21 n12 n22
    mcnemar test m
        
    The following output is generated.
               MCNEMAR TEST FOR NEW BETTER THAN OLD (2X2 TABLE)
      
     NULL HYPOTHESIS: NEW AND OLD ARE EQUAL
     ALTERNATIVE HYPOTHESIS: NEW AND OLD ARE NOT EQUAL
      
                   |       VARIABLE TWO       |      ROW
     VARIABLE ONE  |  SUCCESSES     FAILURES  |      TOTAL
     =====================================================
     SUCCESSES     |         63           21  |         84
                   |   0.940298     0.636364  |
     FAILURES      |          4           12  |         16
                   |   0.059701     0.363636  |
     =====================================================
     COLUMN TOTAL  |         67           33  |        100
      
     LARGE SAMPLE CASE (N >= 20)
     (CRITICAL VALUES BASED ON CHI-SQUARE WITH ONE DEGREE OF FREEDOM)
      
     WITHOUT CONTINUITY CORRECTION
     VALUE OF TEST STATISTIC                  =    11.56000
     CDF OF TEST STATISTIC                    =   0.9993260
      
      
     WITH CONTINUITY CORRECTION
     VALUE OF TEST STATISTIC                  =    10.24000
     CDF OF TEST STATISTIC                    =   0.9986256
      
     WITHOUT CONTINUITY CORRECTION
                                           NULL HYPOTHESIS   NULL
     NULL          CONFIDENCE    CRITICAL  ACCEPTANCE        HYPOTHESIS
     HYPOTHESIS    LEVEL         VALUE     INTERVAL          CONCLUSION
     ===================================================================
     NEW/OLD EQUAL    50.0%        0.45     (0,0.500)        REJECT
     NEW/OLD EQUAL    80.0%        1.64     (0,0.800)        REJECT
     NEW/OLD EQUAL    90.0%        2.71     (0,0.900)        REJECT
     NEW/OLD EQUAL    95.0%        3.84     (0,0.950)        REJECT
     NEW/OLD EQUAL    99.0%        6.63     (0,0.990)        REJECT
      
     WITH CONTINUITY CORRECTION
                                           NULL HYPOTHESIS   NULL
     NULL          CONFIDENCE    CRITICAL  ACCEPTANCE        HYPOTHESIS
     HYPOTHESIS    LEVEL         VALUE     INTERVAL          CONCLUSION
     ===================================================================
     NEW/OLD EQUAL    50.0%        0.45     (0,0.500)        REJECT
     NEW/OLD EQUAL    80.0%        1.64     (0,0.800)        REJECT
     NEW/OLD EQUAL    90.0%        2.71     (0,0.900)        REJECT
     NEW/OLD EQUAL    95.0%        3.84     (0,0.950)        REJECT
     NEW/OLD EQUAL    99.0%        6.63     (0,0.990)        REJECT
    
    
    
    
    
               MCNEMAR TEST FOR NEW BETTER THAN OLD (2X2 TABLE)
      
     NULL HYPOTHESIS: NEW AND OLD ARE EQUAL
     ALTERNATIVE HYPOTHESIS: NEW AND OLD ARE NOT EQUAL
      
                   |       VARIABLE TWO       |      ROW
     VARIABLE ONE  |  SUCCESSES     FAILURES  |      TOTAL
     =====================================================
     SUCCESSES     |          2            4  |          6
                   |   0.333333     0.400000  |
     FAILURES      |          4            6  |         10
                   |   0.666667     0.600000  |
     =====================================================
     COLUMN TOTAL  |          6           10  |         16
      
     SMALL SAMPLE CASE (N < 20)
     (CRITICAL VALUES BASED ON BINOMIAL WITH
     P = 0.5 AND N =  8)
      
     VALUE OF TEST STATISTIC                  =    4.000000
     CDF OF TEST STATISTIC                    =   0.6367188
      
                                 LOWER     UPPER     NULL HYPOTHESIS         NULL
     NULL          CONFIDENCE    CRITICAL  CRITICAL  ACCEPTANCE        HYPOTHESIS
     HYPOTHESIS    LEVEL         VALUE     VALUE     INTERVAL          CONCLUSION
     =============================================================================
     NEW/OLD EQUAL    50.0%        3.00      5.00     (0.250,0.750)        ACCEPT
     NEW/OLD EQUAL    80.0%        2.00      6.00     (0.100,0.900)        ACCEPT
     NEW/OLD EQUAL    90.0%        2.00      6.00     (0.050,0.950)        ACCEPT
     NEW/OLD EQUAL    95.0%        1.00      7.00     (0.025,0.975)        ACCEPT
     NEW/OLD EQUAL    98.0%        1.00      7.00     (0.010,0.990)        ACCEPT
     NEW/OLD EQUAL    99.0%        1.00      7.00     (0.005,0.995)        ACCEPT
        
Date created: 01/15/2008
Last updated: 12/11/2023

Please email comments on this WWW page to alan.heckert@nist.gov.