SED navigation bar go to SED home page go to Dataplot home page go to NIST home page SED Home Page SED Contacts SED Projects SED Products and Publications Search SED Pages
Dataplot Vol 1 Auxiliary Chapter

MEDIAN CONFIDENCE LIMITS

Name:
    MEDIAN CONFIDENCE LIMITS
Type:
    Analysis Command
Purpose:
    Generates a median based confidence interval for the location of a variable.
Description:
    Mosteller and Tukey (see Reference section below) define two types of robustness:

    1. resistance means that changing a small part, even by a large amount, of the data does not cause a large change in the estimate

    2. robustness of efficiency means that the statistic has high efficiency in a variety of situations rather than in any one situation. Efficiency means that the estimate is close to optimal estimate given that we distribution that the data comes from. A useful measure of efficiency is:

        Efficiency = (lowest variance feasible)/ (actual variance)

    Standard confidence intervals are based on the mean and variance. These are the optimal estimators if the data are in fact from a Gaussian population. However, the mean lacks both resistance and robustness of efficiency. The median is less affected by outliers (i.e., resistance) than the mean. However, the median is not particularly robust with regards to efficiency.

    Dataplot generates confidence intervals for the median using the following two methods:

    1. Method 1 is the Hettmansperger-Sheather interpolation method. The steps in this method are:
      • Suppose W is a binomial random variable with n trials and a probability of success p = 0.5. For any integer, k, between 0 and [n/2], let gammak = P(k <= W <= n - k).

        A 95% confidence interval for the median is:

          (Xk, Xn-k+1)

        with X denoting the sorted observations.

      • Determine k such that gammak+1 < 1 - alpha < gammak.

      • Compute

          I = (gamma(k) -1 - alpha)/(gamma(k) - gamma(k+1))

        and

          lambda = (n-k)*I/(k + (n-2*k)*I)

      • An approximate (1-alpha) confidence interval is

          LCL = lambda*X(k+1) + (1-lambda)*X(k)
          UCL = lambda*X(n-k) + (1-lambda)*X(n-k+1)

    2. Method 2, based on a method given by Wilcox (see Reference below) on page 87, is based on the Maritz-Jarrett estimate of the standard error for a quantile. Specifically,

        xhat(q) +/- NORPPPF(1-alpha/2)*sigmahat(mj)

      where

        q = the desired quantile (q = 0.5 for the median)

        xhatq = the estimated sample quantile

        NORPPF = the percent point function of the standard normal distribution

        alpha = the significance level

        sigmahatmj = the quantile standard error based on the Maritz-Jarrett method

      Note that this method can be applied to quantiles other than the median. However, the accuracy of this method has not been studied for quantiles other than 0.5.

Syntax 1:
    MEDIAN CONFIDENCE LIMITS <y>       <SUBSET/EXCEPT/FOR qualification>
    where <y> is the response variable,
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Syntax 2:
    QUANTILE CONFIDENCE LIMITS <y>       <SUBSET/EXCEPT/FOR qualification>
    where <y> is the response variable, and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Examples:
    MEDIAN CONFIDENCE LIMITS Y1

    LET P100 = 0.25
    QUANTILE CONFIDENCE LIMITS Y1 SUBSET TAG = 2

Note:
    For quantiles other than the median, the desired quantile is specified with the LET command. Specifically, define the parameter P100. For example,

      LET P100 = 0.25

    Only the method based on the Maritz-Jarrett standard error is supported for quantiles other than the median.

Note:
    A table of confidence intervals is printed for alpha levels of 50.0, 75.0, 90.0, 95.0, 99.0, 99.9, 99.99, and 99.999.
Note:
    Alternative methods for generating confidence intervals for medians or quantiles are available.

    1. You can use the BOOTSTRAP MEDIAN PLOT or the BOOTSTRAP QUANTILE PLOT command. For example, 
        LET Y = CAUCHY RANDOM NUMBERS FOR I = 1 1 100
BOOTSTRAP MEDIAN PLOT Y
LET LCL = B025
LET UCL = B975
.
LET XQ = 0.75
BOOTSTRAP QUANTILE PLOT Y
LET LCL = B025
LET UCL = B975
    2. Wilcox suggests the following method for quantiles (the median is a special case with the quantile = 0.5) on page 86.

        thetahat(q) +/- chat*sigmahat(hd)

      where

        thetahatq = the Herrell-Davis quantile estimate

        sigmahathd = the bootstrap estimate of the Herrell-Davis quantile standard error

        chat = 0.5064*N**(-0.25) + 1.96 for N >= 11 and 0.3 <= q <= 0.7
              for N > 21 and 0.2 <= q <= 0.8
              for N > 41 0.1 <= q <= 0.9
           = -6.23*(1/N) + 5.01 for 11 <= N <= 21, q = 0.2, 0.8
           = 36.2*(1/N) + 1.31 for N > 41, q = 0.1, 0.9

      This can be coded in the following Dataplot macro: 

        SET QUANTILE METHOD HERRELL DAVIS
LET P100 = 0.5
LET THETAHAT = QUANTILE Y
BOOTSTRAP QUANTILE STANDARD ERROR PLOT Y
LET SIGMAHAT = B50
LET N = SIZE Y
IF N < 11
  QUIT
END OF IF
LET C = 0.5064*N**(-0.25) + 1.96
LET IQFLAG = 1
IF P100 <= 0.19
  IF N > 41
    LET C = 36.2*(1/N) + 1.31
  END OF IF
ELSEIF P100 <= 0.29
  IF N <= 21
    LET C = -6.23*(1/N) + 5.01
  END OF IF
ELSE IF P100 >= 0.81
  IF N > 41
    LET C = 36.2*(1/N) + 1.31
  END OF IF
ELSE IF P100 >= 0.71
  IF N <= 21
    LET C = -6.23*(1/N) + 5.01
  END OF IF
ENDIF
LET LOWLIMIT = THETAHAT - C*B50
LET UPPLIMIT = THETAHAT + C*B50
Default:
    None
Synonyms:
    MEDIAN CONFIDENCE INTERVAL
Related Commands:
    MEDIAN = Compute the median.
    MEDIAN PLOT = Generate a trimmed mean (versus subset plot).
    BOOTSTRAP PLOT = Generate a bootstrap plot.
    CONFIDENCE LIMITS = Compute a Gaussian based confidence limit.
    BIWEIGHT CONF LIMITS = Compute a biweight location based confidence limit.
    TRIMMED MEAN CONF LIMITS = Compute a trimmed mean based confidence limit.
    T-TEST = Perform a t-test.
Reference:
    "Introduction to Robust Estimation and Hypothesis Testing", Rand R. Wilcox, Academic Press, 1997.

    "Confidence Interval Based on Interpolated Order Statistics", T. P. Hettmansperger and S. J. Sheather, Statistical Probability Letters 4, 1986, 75-79.

Applications:
    Robust Data Analysis
Implementation Date:
    2003/2
Program: 
     
LET Y1 = NORMAL RANDOM NUMBERS FOR I = 1 1 100
LET Y2 = LOGISTIC RANDOM NUMBERS FOR I = 1 1 100
LET Y3 = CAUCHY RANDOM NUMBERS FOR I = 1 1 100
LET Y4 = DOUBLE EXPONENTIAL RANDOM NUMBERS FOR I = 1 1 100
MEDIAN CONFIDENCE LIMITS Y1
MEDIAN CONFIDENCE LIMITS Y2
MEDIAN CONFIDENCE LIMITS Y3
MEDIAN CONFIDENCE LIMITS Y4
    Dataplot generates the following output: 
      
       ***********************************
       **  MEDIAN CONFIDENCE LIMITS Y1  **
       ***********************************
  
  
                    CONFIDENCE LIMITS FOR MEDIAN
                    (BASED ON MARITZ-JARRETT STANDARD ERROR FOR QUANTILES)
  
           NUMBER OF OBSERVATIONS     =      100
           ESTIMATE OF MEDIAN             =   0.1541439E-02
           QUANTILE     STANDARD ERROR    =   0.8534409
  
    CONFIDENCE   Z     Z X STDERR       LOWER         UPPER
    VALUE (%)  VALUE                    LIMIT         LIMIT
 ---------------------------------------------------------------
      50.000   0.674  0.575637      -.574096      0.577178
      75.000   1.150  0.981755      -.980214      0.983297
      90.000   1.645   1.40379      -1.40224       1.40533
      95.000   1.960   1.67271      -1.67117       1.67426
      99.000   2.576   2.19832      -2.19678       2.19986
      99.900   3.291   2.80829      -2.80675       2.80983
      99.990   3.891   3.32036      -3.31881       3.32190
      99.999   4.417   3.76955      -3.76800       3.77109
  
  
  
  
                    CONFIDENCE LIMITS FOR MEDIAN
                    (BASED ON HETTMANSPERGER-SHEATHER  INTERPOLATION)
  
           NUMBER OF OBSERVATIONS     =       100
           ESTIMATE OF MEDIAN             =   0.1541439E-02
  
    CONFIDENCE   LOWER         UPPER
    VALUE (%)    LIMIT         LIMIT
 ---------------------------------------------------------------
       50.000  -.642535E-01  0.410208E-01
       75.000  -.102452      0.105255
       90.000  -.131545      0.231773
       95.000  -.186693      0.238178
       99.000  -.352005      0.268148
       99.900  -.382770      0.378904
       99.990  -.446305      0.406448
       99.999  -.466646      0.425056
  
  
       ***********************************
       **  MEDIAN CONFIDENCE LIMITS Y2  **
       ***********************************
  
  
                    CONFIDENCE LIMITS FOR MEDIAN
                    (BASED ON MARITZ-JARRETT STANDARD ERROR FOR QUANTILES)
  
           NUMBER OF OBSERVATIONS     =      100
           ESTIMATE OF MEDIAN             =   0.1506642
           QUANTILE     STANDARD ERROR    =    1.463245
  
    CONFIDENCE   Z     Z X STDERR       LOWER         UPPER
    VALUE (%)  VALUE                    LIMIT         LIMIT
 ---------------------------------------------------------------
      50.000   0.674  0.986944      -.836280       1.13761
      75.000   1.150   1.68324      -1.53258       1.83391
      90.000   1.645   2.40682      -2.25616       2.55749
      95.000   1.960   2.86791      -2.71724       3.01857
      99.000   2.576   3.76907      -3.61841       3.91973
      99.900   3.291   4.81488      -4.66421       4.96554
      99.990   3.891   5.69283      -5.54217       5.84350
      99.999   4.417   6.46298      -6.31231       6.61364
  
  
  
  
                    CONFIDENCE LIMITS FOR MEDIAN
                    (BASED ON HETTMANSPERGER-SHEATHER  INTERPOLATION)
  
           NUMBER OF OBSERVATIONS     =       100
           ESTIMATE OF MEDIAN             =   0.1506642
  
    CONFIDENCE   LOWER         UPPER
    VALUE (%)    LIMIT         LIMIT
 ---------------------------------------------------------------
       50.000  -.322523E-01  0.354998
       75.000  -.484279E-01  0.508575
       90.000  -.694156E-01  0.526755
       95.000  -.104975      0.545388
       99.000  -.139049      0.562220
       99.900  -.543264      0.718403
       99.990  -.778990      0.808447
       99.999  -1.02075      0.984866
  
  
       ***********************************
       **  MEDIAN CONFIDENCE LIMITS Y3  **
       ***********************************
  
  
                    CONFIDENCE LIMITS FOR MEDIAN
                    (BASED ON MARITZ-JARRETT STANDARD ERROR FOR QUANTILES)
  
           NUMBER OF OBSERVATIONS     =      100
           ESTIMATE OF MEDIAN             =   0.2121047E-01
           QUANTILE     STANDARD ERROR    =    2.131877
  
    CONFIDENCE   Z     Z X STDERR       LOWER         UPPER
    VALUE (%)  VALUE                    LIMIT         LIMIT
 ---------------------------------------------------------------
      50.000   0.674   1.43793      -1.41672       1.45914
      75.000   1.150   2.45240      -2.43119       2.47361
      90.000   1.645   3.50663      -3.48541       3.52784
      95.000   1.960   4.17840      -4.15719       4.19961
      99.000   2.576   5.49135      -5.47014       5.51256
      99.900   3.291   7.01504      -6.99383       7.03625
      99.990   3.891   8.29418      -8.27297       8.31539
      99.999   4.417   9.41624      -9.39503       9.43746
  
  
  
  
                    CONFIDENCE LIMITS FOR MEDIAN
                    (BASED ON HETTMANSPERGER-SHEATHER  INTERPOLATION)
  
           NUMBER OF OBSERVATIONS     =       100
           ESTIMATE OF MEDIAN             =   0.2121047E-01
  
    CONFIDENCE   LOWER         UPPER
    VALUE (%)    LIMIT         LIMIT
 ---------------------------------------------------------------
       50.000  -.859306E-01  0.157986
       75.000  -.157865      0.289813
       90.000  -.224773      0.379127
       95.000  -.332729      0.438898
       99.000  -.379803      0.451503
       99.900  -.411627      0.621219
       99.990  -.484651      0.895623
       99.999  -.682689      0.948429
  
  
       ***********************************
       **  MEDIAN CONFIDENCE LIMITS Y4  **
       ***********************************
  
  
                    CONFIDENCE LIMITS FOR MEDIAN
                    (BASED ON MARITZ-JARRETT STANDARD ERROR FOR QUANTILES)
  
           NUMBER OF OBSERVATIONS     =      100
           ESTIMATE OF MEDIAN             =   0.2327367E-01
           QUANTILE     STANDARD ERROR    =    2.771725
  
    CONFIDENCE   Z     Z X STDERR       LOWER         UPPER
    VALUE (%)  VALUE                    LIMIT         LIMIT
 ---------------------------------------------------------------
      50.000   0.674   1.86950      -1.84623       1.89277
      75.000   1.150   3.18845      -3.16518       3.21173
      90.000   1.645   4.55908      -4.53581       4.58236
      95.000   1.960   5.43248      -5.40921       5.45576
      99.000   2.576   7.13949      -7.11622       7.16277
      99.900   3.291   9.12049      -9.09722       9.14377
      99.990   3.891   10.7835      -10.7603       10.8068
      99.999   4.417   12.2424      -12.2191       12.2657
  
  
  
  
                    CONFIDENCE LIMITS FOR MEDIAN
                    (BASED ON HETTMANSPERGER-SHEATHER  INTERPOLATION)
  
           NUMBER OF OBSERVATIONS     =       100
           ESTIMATE OF MEDIAN             =   0.2327367E-01
  
    CONFIDENCE   LOWER         UPPER
    VALUE (%)    LIMIT         LIMIT
 ---------------------------------------------------------------
       50.000  -.321698E-01  0.478474E-01
       75.000  -.628340E-01  0.691212E-01
       90.000  -.832817E-01  0.100314
       95.000  -.136664      0.155923
       99.000  -.213284      0.218869
       99.900  -.311872      0.304638
       99.990  -.388700      0.427280
       99.999  -.479437      0.455274

Date created: 2/26/2003
Last updated: 4/4/2003
Please email comments on this WWW page to alan.heckert@nist.gov.