SED navigation bar go to SED home page go to Dataplot home page go to NIST home page SED Home Page SED Staff SED Projects SED Products and Publications Search SED Pages
Dataplot Vol 1 Vol 2

CONSENSUS MEAN PLOT

Name:
    CONSENSUS MEAN PLOT
Type:
    Graphics Command
Purpose:
    Compute an estimate of a consensus mean, and the associated uncertainty, based on the data from multiple laboratories or multiple methods and present the results in a graphical format.
Description:
    The problem of determining a consensus mean based on the data from two or more laboratories (or from using two or more methods from the same laboratory) is a common one at NIST and other measurement laboratories. There are a number of approaches to this problem. The Dataplot CONSENSUS MEANS command implements a number of the more commonly used approaches. Note that Dataplot computes estimates for a variety of methods and does not specify which is the most appropriate method for a given data set. Consult with a statistician for guidance on which method is most appropriate for your data.

    The CONSENSUS MEAN PLOT command performs a consensus means analysis and in addition to the textual output it presents the results in a graphical format. The details of the consensus means analysis are discussed in the documentation for the CONSENSUS MEANS command (enter HELP CONSENSUS MEANS for details).

    The graph consists of two parts.

    1. The left hand side of the plot displays the consensus means estimates with the associated uncertainty obtained by the various methods. You can specify the desired uncertainty to plot by entering one of the following command

        SET CONSENSUS MEAN PLOT ERROR ...
                    ONE STANDARD ERRORS
        SET CONSENSUS MEAN PLOT ERROR ...
                    TWO STANDARD ERRORS
        SET CONSENSUS MEAN PLOT ERROR ...
                    CONFIDENCE INTERVAL

      The default is to use TWO STANDARD ERRORS. If CONFIDENCE INTERVAL is used, then 95% confidence intervals will be plotted.

    2. The right hand side of the plot displays the lab means with their associated uncertainty. The SET CONSENSUS MEAN PLOT ERROR will also specify the uncertainty to use for the lab data.

    This plot allows you to compare the consensus values and associated uncertainties obtained by the different methods. It also allows you to compare the consensus values against the lab data.

    The methods are plotted in the following order:

    1. Mandel-Paule
    2. Modified Mandel-Paule
    3. Vangel-Rukhin
    4. DerSimonian-Laird (original variance)
    5. DerSimonian-Laird (HHD variance)
    6. DerSimonian-Laird (minmax variance)
    7. DerSimonian-Laird (bootstrap variance)
    8. Graybill-Deal
    9. Fairweather (Fairweather variance)
    10. Fairweather (Cox variance)
    11. Fairweather (minmax variance)
    12. Generalized Confidence Intervals
    13. Grand Mean
    14. Mean of Means
    15. Bounds on bias (BOB)
    16. Schiller-Eberhardt
    17. Bayesian Consensus Procedure
    18. Median of Means
    19. Huber Mean of Means

    In most cases, you will probably want only a subset of these methods. You can use the following commands to specify which methods to include in the consensus means analysis. This is demonstrated in the program samples below.

      SET MANDEL PAULE <ON/OFF> - default is ON
      SET MODIFIED MANDEL PAULE <ON/OFF> - default is ON
      SET VANGEL RUHKIN <ON/OFF> - default is ON
      SET DERSIMONIAN LAIRD <ON/OFF> - default is ON
      SET DERSIMONIAN LAIRD HHD <ON/OFF> - default is ON
      SET DERSIMONIAN LAIRD MINMAX <ON/OFF> - default is OFF
      SET DERSIMONIAN LAIRD BOOTSTRAP <ON/OFF> - default is OFF
      SET GRAYBILL DEAL <ON/OFF> - default is ON
      SET GENERALIZED CONFIDENCE INTERVAL <ON/OFF> - default is ON
      SET FAIRWEATHER <ON/OFF> - default is OFF
      SET MEAN OF MEANS <ON/OFF> - default is ON
      SET GRAND MEAN <ON/OFF> - default is ON
      SET BOB <ON/OFF> - default is ON
      SET SCHILLER EBERHARDT <ON/OFF> - default is OFF
      SET BAYESIAN CONSENSUS PROCEDURE <ON/OFF> - default is OFF
      SET MEDIAN OF MEANS <ON/OFF> - default is OFF
      SET HUBER MEAN OF MEANS <ON/OFF> - default is OFF
Syntax 1:
    CONSENSUS MEAN PLOT <y> <tag> <SUBSET/EXCEPT/FOR qualification>
    where <y> is the response variable;
                <tag> is the group-id variable;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.

    This is used for the raw data case.

Syntax 2:
    CONSENSUS MEAN PLOT <ymean> <ysd> <ni>
                            <SUBSET/EXCEPT/FOR qualification>
    where <ymean> is the variable containing the lab means;
                <ysd> is the variable containing the lab standard deviations;
                <ni> is the variable containing the lab sample sizes;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.

    This is used for the summary data case.

Syntax 3:
    CONSENSUS MEAN PLOT <ymean> <ysd> <ni> <labid>
                            <SUBSET/EXCEPT/FOR qualification>
    where <ymean> is the variable containing the lab means;
                <ysd> is the variable containing the lab standard deviations;
                <ni> is the variable containing the lab sample sizes;
                <labid> is the variable containing the lab-id's (numeric);
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.

    This is used for the summary data case. The is used in the labeling of the textual output. It is not used in the computation of the consensus means or in the plot.

Examples:
    CONSENSUS MEAN PLOT Y X
    CONSENSUS MEAN PLOT Y TAG
    CONSENSUS MEAN PLOT Y TAG SUBSET TAG >= 2
    CONSENSUS MEAN PLOT Y TAG SUBSET TAG = 1 TO 6
    CONSENSUS MEAN PLOT YMEAN YSD NI
Note:
    You can omit the lab data portion of the plot by entering the command

      SET CONSENSUS MEAN PLOT DATA OFF

    To reset the default of showing the lab data on the plot, enter

      SET CONSENSUS MEAN PLOT DATA ON

    This is demonstrated in the Program 2 example below.

Note:
    You can optionally sort the methods by the width of the uncertainty interval (from smallest to largest) on the plot by entering the command

      SET CONSENSUS MEAN PLOT SORTED ON

    To reset the default of not sorting the methods, enter

      SET CONSENSUS MEAN PLOT SORTED OFF

    This is demonstrated in the Program 3 example below. The program example shows how to read DPST3F.DAT to retrieve the sorted order of the methods.

Note:
    In some cases, an outlying laboratory may distort the plot (i.e., you cannot distinguish the data from the other non-outlying labs). In these cases, you may want to omit the lab from the plot while including it in the analysis (you can use the SUBSET clause to omit the lab both from the plot and the analysis).

    In order to specify one or more labs to omit from the plot (but not the analysis), enter the command

      SET CONSENSUS MEAN PLOT OMIT LABS <lab1> ... <labk>

    where <lab1> ... <labk> is a list of 1 to 10 labs to be omitted from the plot.

    To reset the default of all labs being plotted, enter

      SET CONSENSUS MEAN PLOT OMIT LABS

    This option was added 12/2016.

    Similarly, you can omit methods from the plot (but not the analysis) with the commands

      SET CONSENSUS MEAN PLOT OMIT METHOD ONE <method>
      SET CONSENSUS MEAN PLOT OMIT METHOD TWO <method>
      SET CONSENSUS MEAN PLOT OMIT METHOD THREE <method>

    When extreme outliers are included in the analysis, the results for some methods may distort the plot.

    This option was added 04/2017.

Note:
    You can use the CHARACTER and LINE commands to control the appearance of the plot. Specifically,

    1. Trace 1 is a line connecting the estimated means for each of the methods.
    2. Traces 2, 3, and 4 are the estimated means, lower interval, and upper interval, respectively, within a given method.
    3. Trace 5 is line connecting the lower and upper interval within a single method.
    This is demonstrated in the example program below.
Default:
    None
Synonyms:
    There are no synonyms for this command.
Related Commands: Reference:
    DerSimonian and Laird (1986), "Meta-analysis in Clinical Trials", Controlled Clinical Trials, 7, pp. 177-188.

    Graybill and Deal (1959), "Combining Unbiased Estimators", Biometrics, 15, pp. 543-550.

    M. S. Levenson, D. L. Banks, K. R. Eberhardt, L. M. Gill, W. F. Guthrie, H. K. Liu, M. G. Vangel, J. H. Yen, and N. F. Zhang (2000), "An ISO GUM Approach to Combining Results from Multiple Methods", Journal of Research of the National Institute of Standards and Technology, Volume 105, Number 4.

    John Mandel and Robert Paule (1970), "Interlaboratory Evaluation of a Material with Unequal Number of Replicates", Analytical Chemistry, 42, pp. 1194-1197.

    Robert Paule and John Mandel (1982), "Consensus Values and Weighting Factors", Journal of Research of the National Bureau of Standards, 87, pp. 377-385.

    Andrew Rukhin (2009), "Weighted Means Statistics in Interlaboratory Studies", Metrologia, Vol. 46, pp. 323-331.

    Andrew Ruhkin (2003), "Two Procedures of Meta-analysis in Clinical Trials and Interlaboratory Studies", Tatra Mountains Mathematical Publications, 26, pp. 155-168.

    Andrew Ruhkin and Mark Vangel (1998), "Estimation of a Common Mean and Weighted Means Statistics", Journal of the American Statistical Association, Vol. 93, No. 441.

    Andrew Ruhkin, B. Biggerstaff, and Mark Vangel (2000), "Restricted Maximum Likelihood Estimation of a Common Mean and Mandel-Paule Algorithm", Journal of Statistical Planning and Inference, 83, pp. 319-330.

    Mark Vangel and Andrew Ruhkin (1999), "Maximum Likelihood Analysis for Heteroscedastic One-Way Random Effects ANOVA in Interlaboratory Studies", Biometrics 55, 129-136.

    Susannah Schiller and Keith Eberhardt (1991), "Combining Data from Independent Analysis Methods", Spectrochimica, ACTA 46 (12).

    Susannah Schiller (1996), "Standard Reference Materials: Statistical Aspects of the Certification of Chemical SRMs", NIST SP 260-125, NIST, Gaithersburg, MD.

    Bimal Kumar Sinha (1985), "Unbiased Estimation of the Variance of the Graybill-Deal Estimator of the Common Mean of Several Normal Populations", The Canadian Journal of Statistics, Vol. 13, No. 3, pp. 243-247.

    Nien-Fan Zhang (2006), "The Uncertainty Associated with The Weighted Mean of Measurement Data", Metrologia, 43, PP. 195-204.

    Hagwood and Guthrie (2006), "Combining Data in Small Multiple-Methods Studies", Technometrics, Vol. 48, No. 2.

    Iyer, Wang, and Matthew (2004), "Models and Confidence Intervals for True Values in Interlaboratory Trials", Journal of the American Statistical Association, Vol. 99, No. 468, pp. 1060-1071.

    Fairweather (1972), "A Method for Obtaining an Exact Confidence Interval for the Common Mean of Several Normal Populations", Applied Statistics, Vol. 21, pp. 229-233.

    Cox (2002), "The Evaluation of Key Comparison Data", Metrologia, Vol. 39, pp. 589-595.

Applications:
    Standard Reference Materials, Interlaboratory Studies
Implementation Date:
    2001/08
    2011/10: Redesigned plot to incorporate the lab data
    2016/12: Support for SET CONSENSUS MEAN PLOT OMIT LABS
    2017/03: Support for SET CONSENSUS MEAN PLOT OMIT METHOD
    2017/07: Changed the default for Schiller-Eberhardt, Fairweather, and Bayesian consensus procedure to OFF
Program 1:
     
    . Step 1: Read the data
    .
    SKIP 25
    READ STUTZ86.DAT ALITE JUNK2 JUNK3 JUNK4 JUNK5 LABID
    .
    . Step 2: Specify methods to include on the plot and define
    .         strings containing method id's.  Define settings
    .         for DerSimonian-Laird bootstrap.
    .
    set write decimals 5
    set modified mandel paule off
    set schiller eberhardt off
    set mean of means off
    set grand mean off
    set graybill deal off
    set generalized confidence interval off
    set fairweather off
    set bayesian consensus procedure off
    set dersimonian laird minmax off
    set dersimonian laird bootstrap on
    .
    let string sx1 = MP
    let string sx2 = VR
    let string sx3 = DSLcr()Original
    let string sx4 = DSLcr()HHD
    let string sx5 = DSLcr()Bootstrap
    let string sx6 = BOB
    let nmeth = 6
    .
    seed 21307
    set random number generator fibonacci congruential
    bootstrap samples 100000
    .
    . Step 3: Set plot control features
    .
    .         Settings for plot characters/lines
    .
    line solid all
    character blank all
    character fill on
    character hw 1 0.75
    character circle
    line      blank
    .
    .         Settings for labels
    .
    case asis
    title asis
    title offset 2
    tic mark label case asis
    title Consensus Means Plot
    y1label Response
    .
    let nlab = unique labid
    loop for k = 1 1 nlab
         let icnt = k + nmeth
         let string sx^icnt = ^k
    end of loop
    .
    let ntot = nmeth + nlab
    xlimits 1 ntot
    major xtic mark number ntot
    minor xtic mark number 0
    tic mark offset units data
    xtic mark offset 0.5   0.5
    x1tic mark label format group label
    let igx = group label sx1 to sx^ntot
    x1tic mark label content igx
    .
    . Step 4: Generate the plot
    .
    set consensus mean plot error two standard errors
    feedback off
    consensus mean plot alite labid
    .
    . Step 5: Post plot labelling
    .
    line dotted
    drawdsds  6.5 20  6.5 90
    justification center
    moveds 3.5 5
    text Consensus Method: Two Standard Errors
    moveds 9 5
    text Laboratorycr()meansp()+/-2*sd/sqrt(n)
        
    plot generated by sample program
Program 2:
     
    . This version suppresses the lab data portion of the plot
    .
    . Step 1: Read the data
    .
    SKIP 25
    READ STUTZ86.DAT ALITE JUNK2 JUNK3 JUNK4 JUNK5 LABID
    .
    . Step 2: Specify methods to include on the plot and define
    .         strings containing method id's.  Define settings
    .         for DerSimonian-Laird bootstrap.
    .
    set write decimals 5
    set modified mandel paule off
    set schiller eberhardt off
    set mean of means off
    set grand mean off
    set graybill deal off
    set generalized confidence interval off
    set fairweather off
    set bayesian consensus procedure off
    set dersimonian laird minmax off
    set dersimonian laird bootstrap on
    .
    let string sx1 = MP
    let string sx2 = VR
    let string sx3 = DSLcr()Original
    let string sx4 = DSLcr()HHD
    let string sx5 = DSLcr()Bootstrap
    let string sx6 = BOB
    let nmeth = 6
    .
    seed 21307
    set random number generator fibonacci congruential
    bootstrap samples 100000
    .
    . Step 3: Set plot control features
    .
    .         Settings for plot characters/lines
    .
    line solid all
    character blank all
    character fill on
    character hw 1 0.75
    character circle
    line      blank
    .
    .         Settings for labels
    .
    case asis
    title asis
    title offset 2
    tic mark label case asis
    title Consensus Means Plot
    y1label Response
    .
    xlimits 1 nmeth
    major xtic mark number nmeth
    minor xtic mark number 0
    tic mark offset units data
    xtic mark offset 0.5   0.5
    x1tic mark label format group label
    let igx = group label sx1 to sx^nmeth
    x1tic mark label content igx
    x1label Consensus Method: Two Standard Errors
    x1label displacement 15
    .
    title Consensus Means Plot - Lab Data Omitted
    set consensus mean plot data off
    feedback off
    consensus mean plot alite labid
        
    plot generated by sample program
Program 3:
     
    . Step 1: Read the data
    .
    SKIP 25
    READ STUTZ86.DAT ALITE JUNK2 JUNK3 JUNK4 JUNK5 LABID
    .
    . Step 2: Specify methods to include on the plot and define
    .         strings containing method id's.  Define settings
    .         for DerSimonian-Laird bootstrap.
    .
    set write decimals 5
    set modified mandel paule off
    set schiller eberhardt off
    set mean of means off
    set grand mean off
    set graybill deal off
    set generalized confidence interval off
    set fairweather off
    set bayesian consensus procedure off
    set dersimonian laird minmax off
    set dersimonian laird bootstrap on
    .
    let string sx1 = MP
    let string sx2 = VR
    let string sx3 = DSLcr()Original
    let string sx4 = DSLcr()HHD
    let string sx5 = DSLcr()Boot
    let string sx6 = BOB
    let nmeth = 6
    .
    seed 21307
    set random number generator fibonacci congruential
    bootstrap samples 100000
    .
    . Step 3: Set plot control features
    .
    .         Settings for plot characters/lines
    .
    line solid all
    character blank all
    character fill on
    character hw 1 0.75
    character circle
    line      blank
    .
    .         Settings for labels
    .
    case asis
    title asis
    title offset 2
    tic mark label case asis
    y1label Response
    .
    let nlab = unique labid
    loop for k = 1 1 nlab
         let icnt = k + nmeth
         let string sx^icnt = ^k
    end of loop
    .
    let ntot = nmeth + nlab
    xlimits 1 ntot
    major xtic mark number ntot
    minor xtic mark number 0
    tic mark offset units data
    xtic mark offset 0.5   0.5
    x1tic mark label format group label
    let igx = group label sx1 to sx^ntot
    x1tic mark label content igx
    .
    . Step 4: Generate the plot
    .
    loop for k = 1 1 ntot
        let string sxnew^k = ^sx^k
    end of loop
    .
    .         Generate dummy plot, read dpst3f.dat to obtain
    .         sorted labels
    .
    set consensus mean plot sorted on
    set consensus mean plot error two standard errors
    .
    device 1 off
    device 2 off
    print off
    consensus mean plot alite labid
    skip 0
    read dpst3f.dat indx
    loop for k = 1 1 nmeth
        let icnt = indx(k)
        let string sxnew^k = ^sx^icnt
    end of loop
    let igx = group label sxnew1 to sxnew^ntot
    x1tic mark label content igx
    device 1 on
    device 2 on
    printing on
    .
    .         Now generate plot with sorted labels
    .
    title Consensus Means Plot - Method Sorted by Width of Expanded Uncertainty
    feedback off
    consensus mean plot alite labid
    .
    . Step 5: Post plot labelling
    .
    line dotted
    drawdsds  6.5 20  6.5 90
    justification center
    moveds 3.5 5
    text Consensus Method: Two Standard Errors
    moveds 9 5
    text Laboratorycr()meansp()+/-2*sd/sqrt(n)
        
    plot generated by sample program

Privacy Policy/Security Notice
Disclaimer | FOIA

NIST is an agency of the U.S. Commerce Department.

Date created: 08/17/2001
Last updated: 12/08/2016

Please email comments on this WWW page to alan.heckert@nist.gov.