Eric S. Lagergren, M. Carroll Croarkin, James J. Filliben, Lisa M. Gill, William F. Guthrie, Hung-kung Liu, Mark G. Vangel, Nien Fan Zhang
Statistical Engineering Division, ITL
Janet E. Rogers, Bert W. Rust
Mathematical & Computational Sciences Division, ITL
Standard Reference Data Program, TS
With the widespread use and availability of statistical software, concerns about the numerical accuracy of such software are now greater than ever. Inevitably, numerical accuracy problems can exist with some of this software despite extensive testing. Indeed, this has been a continuing cause of concern for statisticians. Many have cited the need for an easily-accessible repository of reference datasets. To date no such collection has been available. In response to concerns of both the statistical community and industrial users, the Statistical Engineering Division in collaboration with the Mathematical & Computational Sciences Division and Standard Reference Data Program have developed a Web-based service that provides reference datasets with certified values for a variety of statistical methods. This service is called Statistical Reference Datasets (StRD).
Currently 58 datasets with certified values are provided for assessing the accuracy of software for univariate statistics, analysis of variance, linear regression, and nonlinear regression. The collection includes both generated and "real-world" data of varying levels of difficulty. Generated datasets are designed to challenge specific computations. These include the classic Wampler datasets for testing linear regression algorithms and the Simon & Lesage datasets for testing analysis of variance algorithms. Real-world data include challenging datasets such as the Longley data for linear regression, and more benign datasets such as the Daniel & Wood data for nonlinear regression.
Certified results for linear procedures were obtained using extended precision software to code simple algorithms for each type of computation. Carrying 500 digits through all of the computations allowed calculation of output unaffected by floating point representation errors. Certified values for nonlinear regression are the "best-available" solutions, obtained using 64-bit precision and confirmed by at least two different algorithms and software packages using analytic derivatives.
The team officially released the StRD web service in August 1997 and spent the latter part of the year publicizing the web service. A special contributed paper session was presented at the 1997 Joint Statistical Meetings in August. Talks were also given at NIST Gaithersburg and Boulder. The StRD home page has been hit approximately 900 times each month. In the coming year, we plan to publish a NIST Technical Report documenting the development of the StRD web service and collect feedback from users as to how to improve the web service.
Figure 33: The StRD home page.
Date created: 7/20/2001