SED navigation bar go to SED home page go to Dataplot home page go to NIST home page SED Home Page SED Staff SED Projects SED Products and Publications Search SED Pages
Dataplot Vol 1 Vol 2

MEDIAN CONFIDENCE LIMITS

Name:
    MEDIAN CONFIDENCE LIMITS
Type:
    Analysis Command
Purpose:
    Generates a median based confidence interval for the location of a variable.
Description:
    Mosteller and Tukey (see Reference section below) define two types of robustness:

    1. resistance means that changing a small part, even by a large amount, of the data does not cause a large change in the estimate

    2. robustness of efficiency means that the statistic has high efficiency in a variety of situations rather than in any one situation. Efficiency means that the estimate is close to optimal estimate given that we distribution that the data comes from. A useful measure of efficiency is:

        Efficiency = (lowest variance feasible)/ (actual variance)

    Standard confidence intervals are based on the mean and variance. These are the optimal estimators if the data are in fact from a Gaussian population. However, the mean lacks both resistance and robustness of efficiency. The median is less affected by outliers (i.e., resistance) than the mean. However, the median is not particularly robust with regards to efficiency.

    Dataplot generates confidence intervals for the median using the following two methods:

    1. Method 1 is the Hettmansperger-Sheather interpolation method. The steps in this method are:
      • Suppose W is a binomial random variable with n trials and a probability of success p = 0.5. For any integer, k, between 0 and [n/2], let \( \gamma_{k} = P(k \le W \le n - k) \).

        A 95% confidence interval for the median is:

          (Xk, Xn-k+1)

        with X denoting the sorted observations.

      • Determine k such that \( \gamma_{k+1} < 1 - \alpha < \gamma_{k} \).

      • Compute

          \( I = \frac{\gamma_k -1 - \alpha}{\gamma_k - \gamma_{k+1}} \)

        and

          \( \lambda = \frac{(n-k)I}{k + (n - 2k)I} \)

      • An approximate (1-\( \alpha \)) confidence interval is

          \( LCL = \lambda X_{k+1} + (1 - \lambda) X_k \)
          \( UCL = \lambda X_{n-k} + (1 - \lambda) X_{n-k+1} \)

    2. Method 2, based on a method given by Wilcox (see Reference below) on page 87, is based on the Maritz-Jarrett estimate of the standard error for a quantile. Specifically,

        \( \hat{x}_{q} \pm \Phi^{-1}(1-\alpha/2)\hat{\sigma}_{mj} \)

      where

        q = the desired quantile (q = 0.5 for the median)

        \( \hat{x} \) = the estimated sample quantile

        \( \Phi^{-1} \) = the percent point function of the standard normal distribution

        \( \alpha \) = the significance level

        \( \hat{\sigma}_{mj} \) = the quantile standard error based on the Maritz-Jarrett method

      Note that this method can be applied to quantiles other than the median. However, the accuracy of this method has not been studied for quantiles other than 0.5.

Syntax 1:
    MEDIAN CONFIDENCE LIMITS <y>       <SUBSET/EXCEPT/FOR qualification>
    where <y> is the response variable,
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Syntax 2:
    QUANTILE CONFIDENCE LIMITS <y>
                            <SUBSET/EXCEPT/FOR qualification>
    where <y> is the response variable, and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Examples:
    MEDIAN CONFIDENCE LIMITS Y1

    LET P100 = 0.25
    QUANTILE CONFIDENCE LIMITS Y1 SUBSET TAG = 2

Note:
    For quantiles other than the median, the desired quantile is specified with the LET command. Specifically, define the parameter P100. For example,

      LET P100 = 0.25

    Only the method based on the Maritz-Jarrett standard error is supported for quantiles other than the median.

Note:
    A table of confidence intervals is printed for alpha levels of 50.0, 75.0, 90.0, 95.0, 99.0, 99.9, 99.99, and 99.999.
Note:
    Alternative methods for generating confidence intervals for medians or quantiles are available.

    1. You can use the BOOTSTRAP MEDIAN PLOT or the BOOTSTRAP QUANTILE PLOT command. For example,
        LET Y = CAUCHY RANDOM NUMBERS FOR I = 1 1 100
        BOOTSTRAP MEDIAN PLOT Y
        LET LCL = B025
        LET UCL = B975
        .
        LET XQ = 0.75
        BOOTSTRAP QUANTILE PLOT Y
        LET LCL = B025
        LET UCL = B975
                   
    2. Wilcox suggests the following method for quantiles (the median is a special case with the quantile = 0.5) on page 86.

        \( \hat{\theta} = \pm \; \hat{c} \; \hat{\sigma}_{hd} \)

      where

        \( \hat{\theta} \) = the Herrell-Davis quantile estimate

        \( \hat{\sigma}_{hd} \) = the bootstrap estimate of the Herrell-Davis quantile standard error

        \( \hat{c} \) = 0.5064*N**(-0.25) + 1.96 for N >= 11 and 0.3 <= q <= 0.7
              for N > 21 and 0.2 <= q <= 0.8
              for N > 41 0.1 ≤ q ≤ 0.9
           = -6.23*(1/N) + 5.01 for 11 ≤ N ≤ 21, q = 0.2, 0.8
           = 36.2*(1/N) + 1.31 for N > 41, q = 0.1, 0.9

      This can be coded in the following Dataplot macro:

        SET QUANTILE METHOD HERRELL DAVIS
        LET P100 = 0.5
        LET THETAHAT = QUANTILE Y
        BOOTSTRAP QUANTILE STANDARD ERROR PLOT Y
        LET SIGMAHAT = B50
        LET N = SIZE Y
        IF N < 11
          QUIT
        END OF IF
        LET C = 0.5064*N**(-0.25) + 1.96
        LET IQFLAG = 1
        IF P100 <= 0.19
          IF N > 41
            LET C = 36.2*(1/N) + 1.31
          END OF IF
        ELSEIF P100 <= 0.29
          IF N <= 21
            LET C = -6.23*(1/N) + 5.01
          END OF IF
        ELSE IF P100 >= 0.81
          IF N > 41
            LET C = 36.2*(1/N) + 1.31
          END OF IF
        ELSE IF P100 >= 0.71
          IF N <= 21
            LET C = -6.23*(1/N) + 5.01
          END OF IF
        ENDIF
        LET LOWLIMIT = THETAHAT - C*B50
        LET UPPLIMIT = THETAHAT + C*B50
                   
Default:
    None
Synonyms:
    MEDIAN CONFIDENCE INTERVAL
Related Commands: Reference:
    Rand Wilcox (1997), "Introduction to Robust Estimation and Hypothesis Testing," Academic Press, p. 87.

    T. P. Hettmansperger and S. J. Sheather (1986), "Confidence Interval Based on Interpolated Order Statistics," Statistical Probability Letters 4, pp. 75-79.

Applications:
    Robust Data Analysis
Implementation Date:
    2003/2
Program:
     
    LET Y1 = NORMAL RANDOM NUMBERS FOR I = 1 1 100
    LET Y2 = LOGISTIC RANDOM NUMBERS FOR I = 1 1 100
    LET Y3 = CAUCHY RANDOM NUMBERS FOR I = 1 1 100
    LET Y4 = DOUBLE EXPONENTIAL RANDOM NUMBERS FOR I = 1 1 100
    SET WRITE DECIMALS 4
    MEDIAN CONFIDENCE LIMITS Y1 TO Y4
        
    Dataplot generates the following output:
                 Confidence Limits for the Median
                 (Based on Maritz-Jarrett Standard Error for Quantiles)
      
     Response Variable: Y1
      
     Summary Statistics:
     Number of Observations:                  100
     Sample Minimum:                          -3.4580
     Sample Maximum:                          2.0059
     Sample Median:                           0.0015
     Sample Quantile Standard Error:          0.1119
      
      
      
     -----------------------------------------------------------------
       Confidence       Z      Z-Value X          Lower          Upper
        Value (%)   Value         StdErr          Limit          Limit
     -----------------------------------------------------------------
           50.000   0.674         0.0755        -0.0739         0.0770
           75.000   1.150         0.1287        -0.1272         0.1302
           90.000   1.645         0.1840        -0.1825         0.1856
           95.000   1.960         0.2193        -0.2177         0.2208
           99.000   2.576         0.2882        -0.2866         0.2897
           99.900   3.291         0.3682        -0.3666         0.3697
           99.990   3.891         0.4353        -0.4337         0.4368
           99.999   4.417         0.4942        -0.4927         0.4957
      
      
                 Hettmansperger-Sheater Median Confidence Limits
      
     -----------------------------------
       Confidence   Lower          Upper
        Value (%)   Limit          Limit
     -----------------------------------
           50.000  -0.064         0.0410
           75.000  -0.102         0.1053
           90.000  -0.132         0.2318
           95.000  -0.187         0.2382
           99.000  -0.352         0.2681
           99.900  -0.383         0.3789
           99.990  -0.446         0.4064
           99.999  -0.467         0.4250
      
      
                 Confidence Limits for the Median
                 (Based on Maritz-Jarrett Standard Error for Quantiles)
      
     Response Variable: Y2
      
     Summary Statistics:
     Number of Observations:                  100
     Sample Minimum:                          -5.0249
     Sample Maximum:                          5.3818
     Sample Median:                           0.1507
     Sample Quantile Standard Error:          0.2162
      
      
      
     -----------------------------------------------------------------
       Confidence       Z      Z-Value X          Lower          Upper
        Value (%)   Value         StdErr          Limit          Limit
     -----------------------------------------------------------------
           50.000   0.674         0.1458         0.0048         0.2965
           75.000   1.150         0.2487        -0.0981         0.3994
           90.000   1.645         0.3556        -0.2050         0.5063
           95.000   1.960         0.4238        -0.2731         0.5744
           99.000   2.576         0.5569        -0.4063         0.7076
           99.900   3.291         0.7115        -0.5608         0.8621
           99.990   3.891         0.8412        -0.6905         0.9919
           99.999   4.417         0.9551        -0.8044         1.1057
      
      
                 Hettmansperger-Sheater Median Confidence Limits
      
     -----------------------------------
       Confidence   Lower          Upper
        Value (%)   Limit          Limit
     -----------------------------------
           50.000  -0.032         0.3550
           75.000  -0.048         0.5086
           90.000  -0.069         0.5268
           95.000  -0.105         0.5454
           99.000  -0.139         0.5622
           99.900  -0.543         0.7184
           99.990  -0.779         0.8084
           99.999  -1.021         0.9848
      
      
                 Confidence Limits for the Median
                 (Based on Maritz-Jarrett Standard Error for Quantiles)
      
     Response Variable: Y3
      
     Summary Statistics:
     Number of Observations:                  100
     Sample Minimum:                          -27.0517
     Sample Maximum:                          8.6177
     Sample Median:                           0.0212
     Sample Quantile Standard Error:          0.1866
      
      
      
     -----------------------------------------------------------------
       Confidence       Z      Z-Value X          Lower          Upper
        Value (%)   Value         StdErr          Limit          Limit
     -----------------------------------------------------------------
           50.000   0.674         0.1258        -0.1046         0.1470
           75.000   1.150         0.2146        -0.1934         0.2358
           90.000   1.645         0.3069        -0.2856         0.3281
           95.000   1.960         0.3656        -0.3444         0.3869
           99.000   2.576         0.4805        -0.4593         0.5018
           99.900   3.291         0.6139        -0.5927         0.6351
           99.990   3.891         0.7258        -0.7046         0.7470
           99.999   4.417         0.8241        -0.8028         0.8453
      
      
                 Hettmansperger-Sheater Median Confidence Limits
      
     -----------------------------------
       Confidence   Lower          Upper
        Value (%)   Limit          Limit
     -----------------------------------
           50.000  -0.086         0.1580
           75.000  -0.158         0.2898
           90.000  -0.225         0.3791
           95.000  -0.333         0.4389
           99.000  -0.380         0.4515
           99.900  -0.412         0.6212
           99.990  -0.485         0.8956
           99.999  -0.683         0.9482
      
      
                 Confidence Limits for the Median
                 (Based on Maritz-Jarrett Standard Error for Quantiles)
      
     Response Variable: Y4
      
     Summary Statistics:
     Number of Observations:                  100
     Sample Minimum:                          -9.6504
     Sample Maximum:                          3.0304
     Sample Median:                           0.0233
     Sample Quantile Standard Error:          0.0698
      
      
      
     -----------------------------------------------------------------
       Confidence       Z      Z-Value X          Lower          Upper
        Value (%)   Value         StdErr          Limit          Limit
     -----------------------------------------------------------------
           50.000   0.674         0.0471        -0.0238         0.0703
           75.000   1.150         0.0803        -0.0570         0.1036
           90.000   1.645         0.1148        -0.0915         0.1381
           95.000   1.960         0.1368        -0.1135         0.1601
           99.000   2.576         0.1798        -0.1565         0.2030
           99.900   3.291         0.2296        -0.2064         0.2529
           99.990   3.891         0.2715        -0.2482         0.2948
           99.999   4.417         0.3083        -0.2850         0.3315
      
      
                 Hettmansperger-Sheater Median Confidence Limits
      
     -----------------------------------
       Confidence   Lower          Upper
        Value (%)   Limit          Limit
     -----------------------------------
           50.000  -0.032         0.0478
           75.000  -0.063         0.0691
           90.000  -0.083         0.1003
           95.000  -0.137         0.1559
           99.000  -0.213         0.2189
           99.900  -0.312         0.3046
           99.990  -0.389         0.4273
           99.999  -0.479         0.4552
        
Date created: 02/26/2003
Last updated: 12/11/2023

Please email comments on this WWW page to alan.heckert@nist.gov.