SED navigation bar go to SED home page go to Dataplot home page go to NIST home page SED Home Page SED Staff SED Projects SED Products and Publications Search SED Pages
Dataplot Vol 2 Vol 1

PRECISION

Name:
    PRECISION (LET)
Type:
    Let Subcommand
Purpose:
    Compute the precision of a variable.
Description:
    The sample precision is defined as the reciprocal of the sample variance. That is

      \( \mbox{Precision} = \frac{n-1} {\sum_{i=1}^{n}{(X_{i} - \bar{x})^2}} \)

    where n and \( \bar{x} \) denote the sample size and sample mean, respectively.

    Note that high variance implies low precision while low variance implies high precision.

    Some sources define the precision as the reciprocal of the standard deviation.

Syntax 1:
    LET <par> = PRECISION <y>             <SUBSET/EXCEPT/FOR qualification>
    where <y> is a response variable;
                <par> is a parameter where the precision value is saved;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Syntax 2:
    LET <par> = DIFFERENCE OF PRECISION <y1> <y2>             <SUBSET/EXCEPT/FOR qualification>
    where <y1> is the first response variable;
                <y2> is the second response variable;
                <par> is a parameter where the difference of the precision values is saved;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Examples:
    LET PREC = PRECISION Y1
    LET PREC = PRECISION Y1 SUBSET TAG > 2

    LET PREC = DIFFERENCE OF PRECISION Y1 Y2

Note:
    Dataplot statistics can be used in a number of commands. For details, enter

Default:
    None
Synonyms:
    None
Related Commands:
    VARIANCE = Compute the variance of a variable.
    MEAN = Compute the mean of a variable.
    STANDARD DEVIATION = Compute the standard deviation of a variable.
Applications:
    Data Analysis
Implementation Date:
    2017/01
    2017/03: Added DIFFERENCE OF PRECISION
Program 1:
     
    SKIP 25
    READ ZARR13.DAT Y
    LET P = PRECISION Y
        
Program 2:
     
    . Step 1:   Create the data
    .
    skip 25
    read gear.dat y x
    skip 0
    .
    char X
    line blank
    y1label Precision
    x1label Group
    x1tic mark offset 0.5 0.5
    label case asis
    title case asis
    title Precision of GEAR.DAT
    title offset 2
    .
    set statistic plot reference line average
    precision plot y x
    .
    set write decimals 5
    tabulate precision y x
        
    plot generated by sample program

     
                Cross Tabulate PRECISION
     
    (Response Variables: Y        )
    ---------------------------------------------
           X          |         PRECISION
    ---------------------------------------------
            1.00000   |       52941.17647
            2.00000   |       36749.69375
            3.00000   |       63202.24719
            4.00000   |       67365.26946
            5.00000   |       17411.49158
            6.00000   |       10231.92360
            7.00000   |       16114.59266
            8.00000   |       76013.51351
            9.00000   |       58403.63400
           10.00000   |       35211.26761
        
Program 3:
     
    SKIP 25
    READ IRIS.DAT Y1 TO Y4 X
    .
    LET A = DIFFERENCE OF PRECISION Y1 Y2
    SET WRITE DECIMALS 4
    TABULATE DIFFERENCE OF PRECISION Y1 Y2 X
    
    
     
                Cross Tabulate DIFFERENCE OF PRECISION
     
    (Response Variables: Y1       Y2      )
    ---------------------------------------------
           X          |   DIFFERENCE OF P
    ---------------------------------------------
             1.0000   |            1.0889
             2.0000   |           -6.4021
             3.0000   |           -7.1419
    
    . XTIC OFFSET 0.2 0.2 X1LABEL GROUP ID Y1LABEL DIFFERENCE OF PRECISION CHAR X LINE BLANK DIFFERENCE OF PRECISION PLOT Y1 Y2 X

    plot generated by sample program

    CHAR X ALL LINE BLANK ALL BOOTSTRAP DIFFERENCE OF PRECISION PLOT Y1 Y2 X

    plot generated by sample program

Privacy Policy/Security Notice
Disclaimer | FOIA

NIST is an agency of the U.S. Commerce Department.

Date created: 01/24/2017
Last updated: 06/30/2017

Please email comments on this WWW page to alan.heckert@nist.gov.