SED navigation bar go to SED home page go to SED publications page go to NIST home page SED Home Page SED Contacts SED Projects SED Products and Publications Search SED Pages

contents     previous     next

5.3 Statistical Reference Datasets

M. Carroll Croarkin

James J. Filliben

Lisa M. Gill

William F. Guthrie

Eric S. Lagergren

Hung-Kung Liu

Mark G. Vangel

Nien-Fan Zhang

Statistical Engineering Division, ITL

Janet E. Rogers

Bert W. Rust

Mathematical & Computational Sciences Division, ITL

Phoebe Fagan

Standard Reference Data Program, TS

With the widespread use and availability of statistical software, concerns about the numerical accuracy of such software are now greater than ever. Inevitably, numerical accuracy problems can exist with some of this software despite extensive testing. Indeed, this has been a continuing cause of concern for statisticians, see e.g. Francis, Heiberger, and Velleman (American Statistician, 1975) and Eddy and Cox (Chance, 1991). Many have cited the need for an easily-accessible repository of reference datasets. To date no such collection has been available. In response to concerns of both the statistical community and industrial users, the Statistical Engineering Division in collaboration with the Mathematical & Computational Sciences Division and Standard Reference Data Program has developed a Web-based service that provides reference datasets with certified values for a variety of statistical methods. This service is called Statistical Reference Datasets (StRD).

Currently 62 datasets with certified values are provided for assessing the accuracy of software for univariate statistics, analysis of variance, linear regression, and nonlinear regression. The collection includes both generated and ``real-world" data of varying levels of difficulty. Generated datasets are designed to challenge specific computations. These include the classic Wampler datasets for testing linear regression algorithms and the Simon & Lesage datasets for testing analysis of variance algorithms. Real-world data include challenging datasets such as the Longley data for linear regression, and more benign datasets such as the Daniel & Wood data for nonlinear regression.

Certified results for linear procedures were obtained using extended precision software to code simple algorithms for each type of computation. Carrying 500 digits through all of the computations allowed calculation of output unaffected by floating point representation errors. Certified values for nonlinear regression are the ``best-available" solutions, obtained using 64-bit precision and confirmed by at least two different algorithms and software packages using analytic derivatives.

In the coming year, the team will be publicizing the StRD web service. A special contributed paper session, ``Statistical Reference Datasets (StRD) for Assessing the Numerical Accuracy of Statistical Software," will be presented at the 1997 Joint Statistical Meetings in Anaheim, CA.

contents     previous     next

Date created: 7/20/2001
Last updated: 7/20/2001
Please email comments on this WWW page to