ITL
PUBLICATIONS
CONTENTS
ITL PUBLICATIONS |
CONTENTS |
ITL is one of NIST's seven laboratories covering technical activities in areas of electronics and electrical engineering, manufacturing engineering, chemical science and technology, physics, material science and engineering, building and fire research, as well as information technology. NIST laboratories develop measurements, standards, reference materials, and data that help U.S. companies achieve higher quality products, more reliable and more flexible processes, fewer rejected parts, speedier product development, and improved interoperability of equipment.
This report describes some of the areas in which ITL is working with industry to develop a needed technical infrastructure for IT. Our goals are to:
This IT infrastructure is not complete because we have not yet mastered the use of measurements for information technology. Measurement science, or metrology, has been an important factor in the advancement of commerce and the development of useful, standard products in almost all technical areas. We know how to measure physical and chemical properties and to apply those measurements to improve products. But IT is a relatively new and rapidly changing technology, and many tests, test methods, and other technical references for measurement are missing.
In 1997, NIST staff members studied this problem, and issued a report, "Metrology for Information Technology," which establishes a scope and a conceptual basis for IT metrology. As the study explains, there is a measurement chain of related events that starts with the concept of a property to be measured and concludes with the actual measurement. Once the attribute or quantity to be measured has been defined, the units for expressing the measurement, scales, and a system of standards for calibrating, testing, and measuring must be developed.
We have had some success in measuring the performance of IT hardware and telecommunications, in some cases using traditional physical and chemical measurement science. But we are challenged to test and measure the quality of software. The many combinations and possibilities of software lead to unforeseen outcomes and uncertain results. Measurement science has the potential to help us improve software development and testing, measure and control complex network reactions, estimate the accuracy of simulations, and validate the trustworthiness of information exchanged in networks. International electronic commerce is just one area that will depend upon a measurement infrastructure to enable participants to have confidence that their transactions are protected and conducted as intended.
With industry partners, ITL has started to address some of these critical challenges. We know that we cannot do the job by working alone. We expect to strengthen and intensify our efforts with our partners and to develop many new partners. We have started discussions with organizations such as the Institute of Electrical and Electronics Engineers, Educom, the National Science Foundation, the Department of Defense, industry organizations and consortia, international organizations, and universities.
We invite all who are interested in collaborating with us under Cooperative Research and Development Agreements (CRADAs) to contact us. We welcome your thoughts and comments on the work currently being done and on future needs for measurements for information technology.
Shukri A. Wakid, Director
Information Technology Laboratory
E-mail: itlab@nist.gov
| Director: | Shukri A. Wakid |
| Deputy Director (Acting): | Kathleen M. Roberts |
| Special Advisor for Computing Security Operations: |
Robert Raybold |
| Senior Scientist: | Andre Deprit |
| Associate Director for Computing: | Fred Johnson |
| Assistant Director for Boulder: | Paul Domich |
| Associate Director for Program Implementation: |
R.J. (Jerry) Linn |
| Associate Director for Federal and Industrial Relations: |
David Kahaner |
| Executive Office: | Judith Lyons |
Tests and Test Methods for IT
The Information Technology Laboratory focuses on research and development of tests, quality assurance techniques, tools, models, and reference data that define a measurement science for IT. We collaborate with many people and organizations - industry, research, government and standards organizations - to solve generic measurement problems, and to make the results broadly available. Tests and measurements build a common language for technology advancement, and provide objective criteria that enable developers and users to evaluate product quality and performance and to develop IT standards.
ITL's activities cover a broad range of technologies in mathematical and computational sciences, advanced networks, computer security, information access and user interfaces, high performance systems and services, distributed computing and information services, software diagnostics and conformance testing, and statistical engineering. Our products include tests and test methods, reference data sets and evaluation software, advanced software tools, automated software testing techniques, conformance tests and test methods, statistical model-based testing, and specialized databases. We also provide high-quality services to the NIST staff, including hardware and software support, Internet and intranet services, high performance computing services, and mathematical, computational, and statistical consulting services.
Interaction with Industry
Working in partnership with industry, ITL helps to promote the U.S. economy and expand the Nation's technology base. Industry customers include IT providers and users, research communities, industry standards organizations, software and Web browser developers, security technology developers and vendors, and specialized constituencies such as the mathematical software and high-speed network and wireless technology industries. These product developers and IT providers share common problems with interoperability, scalability, usability, and security of IT products. Our commitment to federal agencies remains strong; we develop standards and guidance for our federal customers and provide technical assistance, in fulfilling our responsibility under the Computer Security Act of 1987.
To pursue common goals, we work with industry organizations and federal agencies by means of the Cooperative Research and Development Agreement (CRADA). In 1997, we collaborated with 37 government, industry, and academic institutions through CRADAs. ITL contributed to the activities of industry groups such as the ATM Forum, the Digital Audio Visual Council (DAVIC), and the Internet Engineering Task Force (IETF) to support interoperability and forward-looking standards. We continued to support the North American Integrated Services Digital Network (ISDN) Users' Forum (NIUF). Many other informal interactions with government and industry partners involved the sharing of equipment or expertise. These cooperative arrangements benefit all participants through a better understanding of the advantages and barriers to the development and use of information technology.
G7 Information Society Pilot Projects and Related Activities
ITL serves as the U.S. contact for the G7 Information Society Global Inventory Pilot Project (GIP) and the Electronic Commerce component of the Global Marketplace for Small and Medium Enterprises (SMEs) Pilot Project.
These pilot projects resulted from the Naples Economic Summit in July 1994, when the G7 leaders decided to "encourage and promote innovation and the spread of new technologies including, in particular, the development of an open, competitive, and integrated worldwide information infrastructure." Eleven information society pilot project themes were identified which demonstrate the potential of the information society and stimulate its deployment. The key objectives are to support international consensus on common principles governing the need of access to networks and applications and their interoperability and to help create markets for new products and services. ITL serves on the Steering committee for the GIP and SME Projects. Further, ITL is responsible for developing and maintaining Web access to the National Inventory of projects related to the eleven G7 theme areas and developing and maintaining the registration process and Web site for the Electronic Commerce testbed projects. The G7 material is found on the NII Virtual Library Home Page at http://nii.nist.gov.
ITL also participated as the NIST representative in the following electronic commerce-related activities:
ITL's work is accomplished in eight technical divisions:
Funding for ITL programs in FY 1997 consisted of $44.5 million from the NIST Congressional appropriation (STRS), including $12.3 million for the Consolidated Scientific Computing System (Super Computer) and $0.6 million for Technical Competence; $2.2 million from the Advanced Technology Program; and $11.9 million in reimbursable funds, mostly from other federal agencies for direct technical assistance. See the Interactions and Accomplishments section of this report for a complete list of our collaborative interactions.
Sharing Information with our Customers
Through a broad range of publications and online resources, we share information and technology with industry, government, academia, and the public. We publish Federal Information Processing Standards (FIPS) and guidelines; special publications series focusing on information technology, computer security, and federal electronic data interchange (EDI) implementation conventions; technical interagency reports on research and tests; a quarterly "ITL" newsletter; and a ITL bulletin series published about eight times a year on topics of interest to the information systems community. See the Interactions and Accomplishments section for a list of publications currently available for sale through the Government Printing Office (GPO) or the National Technical Information Service (NTIS). ITL also sponsors, co-sponsors, and hosts a variety of conferences and workshops throughout the year, and our staff members address many industry and government organizations.
We welcome your interest in the Information Technology Laboratory and invite you to visit our Web site at: http://www.itl.nist.gov
To Top
TO CONTENTS PAGE
| Chief (Acting): | Shukri A. Wakid |
| Group Managers: | James L. Blue, Mathematical Modeling |
| Ronald F. Boisvert, Mathematical Software | |
| Shukri A. Wakid (Acting), Optimization and Computational Geometry |
|
| Anastase Nakassis, Compression Algorithms |
The Mathematical and Computational Sciences Division provides technical leadership within NIST in modern analytical and computational methods for solving mathematical problems of interest to American industry. A program of advanced research focuses on selected areas of applied and computational mathematics and collaboration with technical experts in other NIST divisions, industry, and academia. The scope includes the development and analysis of theoretical descriptions of phenomena (mathematical modeling); the design and analysis of requisite computational methods and experiments; the transformation of these methods into efficient numerical algorithms for high performance computers; the implementation of these methods in high-quality mathematical software; and the distribution of this software to potential clients, both within NIST and to the external community.
The work of the division can be grouped into three broad areas: testing and evaluation methodology, mathematical modeling, and tools for high performance computing.
Highlights of the Year's Work
B. Alpert (ITL) and colleagues M. Francis (Electronics and Electrical Engineering Laboratory [EEEL]) and R. Wittmann (EEEL) won a Department of Commerce Bronze Medal for their work in developing an algorithm for the processing of antenna measurements corrupted by probe position errors. The method exploits position information available during the measurement procedure to compute far fields as accurately as when no position errors are present, and at a computational cost which is acceptable even for electrically very large antennas. The algorithm has been implemented and the software has been distributed to antenna calibration laboratories in government and industry.
Roldan Pozo used funding from his 1996 Presidential Early Career Award for Scientists and Engineers to develop JazzNet, a dedicated cluster of PCs for use in scientific computing applications. The goal was to build an inexpensive "personal supercomputer" using off-the-shelf components, capable of achieving more than 1 Gflop (one billion floating-point operations per second) for under $30,000. Initial performance studies indicate that this milestone has been surpassed. We expect such systems to become increasingly common at NIST and other scientific computing sites in the future. JazzNet was featured in a variety of external publications including Government Computer News, the Montgomery Business Gazette, and New Technology Week, sparking wide interest, and many queries from industries such as Gillette Corp., Dean Witter-Reynolds, and Sunshine Medical Electronics. (See http://math.nist.gov/jazznet/.)
Overall the staff of the division continued its high level of productivity and professional activity. We produced 56 refereed publications, gave 72 talks of which 37 were invited, served on the editorial boards of five journals, served as Editor-in-Chief of one journal, participated on five review panels, refereed for numerous journals and funding agencies, and obtained one patent.
Testing and Evaluation Methodology
We made significant new efforts on the development of testing and evaluation methodology and infrastructural services for computational science, including the design and analysis of testing methodology, evaluation criteria, reference data, and reference algorithms. Customers for this work are industrial developers of software products for scientific computing, as well as the computational science research communities in industry, government, and academia.
The Matrix Market provides online access to a large collection of test data for use in comparative studies of algorithms and software for numerical linear algebra. Done in collaboration with Boeing, the Matrix Market features some 500 large sparse matrices from a variety of applications as well as matrix generation tools and services. In its second year of operation, the service has been accessed by more than 7,500 distinct Internet hosts and has distributed more than 4 Gbytes of matrix data. (See http://math.nist.gov/MatrixMarket/.)
In a related effort, we are working with the BLAS Technical Forum on the development of community standards for basic linear algebra software components to promote portability and high performance of applications. The Forum brings together researchers in government laboratories and academia with vendors such as Cray, HP/Convex, NEC, Intel, Tera, and NAG. As leader of the Sparse BLAS Subcommittee, we developed a working proposal for the Sparse BLAS, as well as reference implementations in C and Fortran. (See http://math.nist.gov/spblas/.)
This year we inaugurated work on the NIST Digital Library of Mathematical Functions (DLMF), a joint project of ITL, the NIST Physics Laboratory, and the NIST Standard Reference Data Program. The DLMF is envisioned as a modern replacement for the NBS Handbook of Mathematical Functions, which was first issued in 1964. The Handbook contains technical information, such as formulas, graphs, and tables, on a variety of mathematical functions of widespread use in the sciences and engineering. The new DLMF would revise and expand this core data and would make use of advanced communications and computational resources to disseminate the information in ways not possible using static print media. In July 1997, we held an invitational workshop for well-known experts in special functions and their application to begin planning for the project. Substantial external funding is now being sought to permit the participation of outside technical experts in the many subfields of mathematical functions. (See http://math.nist.gov/DigitalMathLib/.)
There is a large demand for micromagnetic modeling results, both for industrial design processes and for the materials science and physics of magnetism. Currently there are weaknesses in the physical and computational models and a public modeling code is unavailable. In collaboration with NIST's Materials Science and Engineering Laboratory, we are developing standard test problems for micromagnetic modeling. We presented the first standard problem results this year. We collected solutions from several researchers (submitted anonymously) and found that the results showed little agreement. (See http://www.ctcms.nist.gov/%7Erdm/std1/prob1report.html.) We are also developing a public code for micromagnetics, whose initial release will be in FY 98.
Work has also been initiated in the development of metrics for use in the evaluation of image transformations such as compression. Initial studies are attempting to model images as parameterized surfaces plus error components. Such modeling seeks to combine earlier results on metrics with recent findings of division staff and others on the contrast sensitivity function associated with human vision. Initial results show that a vast category of models can be obtained through linear filtering and that improved metrics can be obtained by using the L2 norm of the residual vector.
Finally, we are cooperating with the Statistical Engineering Division on the development of reference data for the evaluation of statistical software. The Statistical Reference Datasets service, which was formally unveiled this year, contains a collection of test problems for nonlinear least squares developed by division staff.
Mathematical Modeling
Mathematical modeling is an interdisciplinary effort requiring close collaboration between scientists inside and outside the division. Our researchers cooperate with the outside scientists to develop specific mathematical models that capture the essence of the phenomena under study. They analyze the model, propose and develop numerical algorithms, and produce a computer program. The resulting program is run to provide simulations that are compared with experimental results to validate the entire process and to provide the basis for further refinements and enhancements. This process provides more cost-effective, quicker, and better information than experimentation alone. Indeed, such modeling and simulation activities are augmenting and, in many cases, replacing the need to do experiments or used to guide the experimental process into more fertile areas. The information allows the outside scientists to gain understanding or to predict behavior of a complex system, and thus forms the basis for techniques to improve the performance of the system under study.
The customers for our work include our collaborating ITL and NIST scientists and engineers, and through these collaborators, industrial scientists and engineers; other customers are the larger community of researchers in computational science and engineering. Our aim is to work on a spectrum of tasks, including engineering and advanced development, to conduct both short- and long-term research, and to accelerate the adoption of advanced modeling and simulation techniques.
We are active in an inter-laboratory NIST competence project, "Measurement Science for Optical Reflectance and Scattering." This project seeks to identify and measure the physical and optical characteristics of surfaces with desirable appearance characteristics. The results of this work could lead to a computer-based foundation for the virtual design of a surface and a tool for predicting its appearance. This past year, we formed a consultative and collaborative relationship with researchers in the very rapidly expanding computer graphics industry. We organized a two-day meeting attended by researchers in industry and academia for the purposes of discussing and critiquing the proposed ITL research program; scientists from Silicon Graphics Inc. and IBM strongly supported the research proposal, as did participants from Cornell University and the Massachusetts Institute of Technology. Our research will help to move the field of computer graphics rendering from a craft to an engineering discipline so that rendered surfaces have more of the visual properties of real surfaces.
Composite materials with complicated microstructures, such as ceramics, are important in many industrial applications. The microstructure is determined by the processing of the material, but it is the macroscopic properties that are relevant in applications. Previous analysis and computer simulations of the macroscopic properties have been based on idealized simplifications of the microstructure. In collaboration with the Materials Science and Engineering Laboratory (MSEL), we constructed a finite-element computer model that has as input a digitized image of a real microstructure. This program, while still two-dimensional, models elasticity and fracture much more realistically than previous programs, and will help material scientists determine the properties of actual composite materials.
High-speed machining processes are becoming increasingly important in modern manufacturing, but such processes can lead to discontinuous chip formation that is strongly correlated with increased tool wear, degradation of the workpiece surface finish, and less accuracy in the machined part. In an ongoing collaboration with the Automated Production Technology Division in the Manufacturing Engineering Laboratory, a new approach to modeling some high-speed machining processes is being developed that has the potential to predict the onset of discontinuities. One of the main objectives of this effort is to provide improved mathematical models for computer simulations of manufacturing processes which involve high-speed cutting of materials. This information can then be used to control and improve the machining processes.
The use of Monte Carlo simulations to explore phenomena continues to be an important topic of research. It was known that the number of different dimer coverings of a cubic lattice grows exponentially with the size of a lattice. We calculated the exponent, which is a physical constant whose determination has resisted theoretical and computational efforts for over 40 years. We did this by conducting a very large scale Monte Carlo calculation. The method extends to the monomer-dimer case and should enable the first ever computation of the partition function for monomer-dimer systems. This result is of interest in chemistry and materials science because it explains how energy states are distributed. The core computation has been parallelized.
The nondestructive testing of welds is a common problem in many manufacturing industries. Special "test blocks" are used to calibrate transducers used in the nondestructive testing of materials and welds. The problem, however, is that these blocks have not been easy to calibrate. At the request of the American Society for Testing and Materials (ASTM) and in collaboration with MSEL, we developed two- and three-dimensional models for elastic wave propagation that were used to study elastic wave behavior in various test block geometries. The computer model was used as part of a sensitivity study to determine the effect of source and test block geometry. International and U.S. standards committees devoted to the use of ultrasonics in welding are using these results to help the committees improve the methods used to calibrate transducers for nondestructive testing. The International Institute of Welding has expressed interest in this work.
Tools for High Performance Computing
Division consulting and collaboration activities often lead to the development of general purpose tools that can be reused in other NIST and external applications.
A primary tool for the dissemination of information about mathematical software tools is the NIST Guide to Available Mathematical Software. GAMS indexes more than 10,000 software components from 110 libraries and packages. These are components which were either developed at NIST, selected for use at NIST, or archived at netlib, an external repository of the numerical analysis community. GAMS serves the needs of local users for information about software available on local systems; however, the information is of wide interest and has been made available to the public. GAMS is accessed by 10,000 external users each month, and the Web server which hosts GAMS, the Matrix Market, and other division project pages is now averaging more than 400,000 "hits" per month, having exceeded seven million hits since it started operation in 1994.
Our work in support of distributed memory parallel computing led to the development of PHAML, a parallel hierarchical adaptive multilevel software package for elliptic boundary-value problems. Work on PHAML has resulted in fundamental advances in multigrid methods and automated load balancing for adaptive computations. A public release of PHAML is expected this year. Work on PHAML led to the need for portable interactive graphics accessible in Fortran. To solve this problem, f90gl, a Fortran binding, along with a reference implementation, for the OpenGL graphics interface was developed and submitted to the OpenGL Architecture Review Board (ARB). The ARB is composed of representatives of a variety of manufacturers with OpenGL products, including Digital, Evans & Sutherland, Hewlett-Packard, IBM, Intergraph, Intel, Microsoft, and Silicon Graphics. Favorable initial reviews were obtained from the ARB and the J3 Fortran standards committee, and f90gl is now under consideration as the official Fortran binding for OpenGL. (See http://math.nist.gov/f90gl/.)
Our work in image analysis and metrics also led to a variety of new methods and tools. For example, we studied progressive transmission techniques, with applications to downsampling/upsampling schemes, and are in the process of developing prototype software. We developed software to generate optimal biorthogonal wavelets, as well as prototype software to apply wavelet transforms to color pictures. We also generalized the lifting schemes used for ordinary biorthogonal wavelets to multiwavelets. Lifting is the process of starting with a wavelet pair and generating a new one that satisfies some property such as vanishing moments.
To Top
TO CONTENTS PAGE
| Chief: | Kevin Mills |
| Chief (Acting): | Craig Hunt |
| Group Managers: | David Su, High Speed Network Technologies |
| Nader Moayeri, Wireless Communication Technologies | |
| Jean-Philippe Favreau, Multimedia & Digital Video Technologies | |
| Craig Hunt, Internetworking Technologies |
Information technology trends indicate an ongoing move toward a future of universal, continuous access to information. Three barriers to realizing this future are interoperability problems, scaling problems, and security problems. The Advanced Network Technologies Division concentrates primarily on overcoming the first two barriers and on eliminating their detrimental effect on the development of a global network infrastructure. Our contributions to overcoming these barriers are focused on developing test methods, such as testbeds and reference implementations for interoperability testing, and simulation analysis of protocol interactions and scaling limits. Each group's projects and accomplishments are described below.
High Speed Network Technologies
The High Speed Network Technologies Group continues its leadership role in the development of Asynchronous Transfer Mode (ATM) network protocols by active participation in the ATM Forum. The ATM Forum is an industry standards consortium that develops standards for high speed digital network technology. We develop abstract test suites (ATS) for conformance and interoperability testing of the ATM network protocols through participation in the Testing Working Group of the Forum. Our efforts concentrate on testing of call control signaling, routing, and traffic management. During 1997, the group developed conformance tests for the user-network interface (UNI) version 3.1 signaling layer for user side equipment, and interoperability tests for the Private Network-to-Network Interface (PNNI). Currently, the group is developing conformance tests for PNNI and for the Available Bit Rate (ABR) service.
At the international level, we participated in the ITU-T Study Group 13 in the development of the Protocol Implementation Conformance Statement (PICS) Proforma for ATM Adaption Layer (AAL) Type 2 and related service parts. This work contributed to defining the real requirements of the AAL Type 2 protocol and helped the members reach agreement on the ITU-T Recommendations. An important application of AAL2 is the transport of voice over ATM networks.
There is an increasing need for residential broadband access network standards to provide high-speed Internet access from homes that can support the new multimedia applications. The industry standards group IEEE 802.14 is developing protocols for high speed bi-directional data communication over the Hybrid Fiber/Coaxial (HFC) networks currently being used or deployed by the cable TV industry.
For the past three years, the IEEE 802.14 group has been working on a draft specification that will include protocols for the Physical Media (PHY) and Medium Access Control (MAC) protocol layers. As an unbiased third party, ITL participated in the evaluation of several MAC proposals submitted to the 802.14 group. We produced several reports to the group based on the results of computer simulations conducted in our laboratory. The subjects of these reports included performance comparison of MAC proposals, analysis of MAC frame formats, comparison of contention resolution algorithms, and evaluation of bandwidth allocation methods. These reports helped the standards group achieve several important agreements in arriving at a final MAC protocol. In addition, our work on the HFC MAC protocol was extended to include performance evaluation of Transmission & Control Protocol/Internet Protocol (TCP/IP) and ATM ABR service in an asymmetrical network environment such as HFC. The results were presented at technical conferences and in journal publications.
A Video-on-Demand (VoD) Interoperability Testing Laboratory was established jointly by the High Speed Network Technologies Group and the Multimedia and Digital Video Group. In this laboratory, a VoD system was developed based on the DAVIC (Digital Audio-Visual Council) specifications. The system implemented DAVIC's control and data flow. The data flow delivers MPEG2 (Moving Picture Experts Group [MPEG]) data directly over ATM network while the control flow carries control data such as video selection and VCR-type controls via the DAVIC standard protocol stack, or alternatively, through an Internet Web browser.
Test tools are being developed for conformance testing of protocols involved in the laboratory testbed. In 1997 we developed a conformance test suite for the Digital Storage Media Command and Control (DSMCC) protocol which is to be included as part of the International Organization for Standardization (ISO) MPEG2 standards. This facility offers vendors the opportunity to test product interoperability with other vendors' products as well as with NIST's VoD implementation.
In addition to the interoperability tests of VoD applications, we conducted experiments on the transport of digital video information over ATM networks. In a collaborative effort with Bellcore and Bell Atlantic, we used a Bellcore prototype Video Dial Tone system to test transfer of video over multiple wide area networks. ATM networks between New Jersey and Maryland were used in these tests, which specifically addressed the impact of ATM quality of service (QoS) parameters on the performance of the digital video application.
Wireless Technologies
Wireless technology is emerging as an important new area of network research. To meet the interoperability and scaling challenges of the effective use of wireless technology, we are strengthening the Wireless Technologies Group to give it a stronger research focus.
Multimedia and Digital Video
The Multimedia and Digital Video Group works with industry to promote the development of cost-effective, interoperable, distributed multimedia applications and to enable the development of digital video technologies for broadcast, interactive television, video-on-demand, and video conferencing. The group focuses on three areas: measurement techniques for characterization of distributed multimedia technologies and digital video devices and services; techniques for integrating multimedia services with network technologies; and industry-driven standards for multimedia technologies and digital video devices and services. The video-on-demand work is done in collaboration with the High Speed Network Technologies Group.
In collaboration with MITRE and Carnegie Mellon University (CMU), the group works on the Defense Advanced Research Projects Agency (DARPA) Intelligent Collaboration and Visualization Program (IC&V) program. The goal is to identify and apply an evaluation and benchmarking approach to the collaboration infrastructure and applications that will be developed with DARPA funding.
As the Internet has gained popularity over the past decade, the need for collaborative multimedia conferencing and application sharing systems has risen significantly. Application sharing allows participants to view and interact with the same application (e.g., spreadsheet) during their conference. These systems are beginning to play large roles in research, education (e.g., distance learning), and business.
In collaboration with Old Dominion University, ITL is designing and implementing an adaptable and extensible architecture for platform-independent multimedia conferencing and collaborative application sharing. The JCE (Java Collaborative Environment) uses Java-based collaboration mechanisms that provide solutions to overcome the platform-dependency problems for collaborative computing in heterogeneous systems. The Java programming language produces bytecodes that can be run on any platform which has a Java Virtual Machine. This enables application developers to write the application once and have it run anywhere.
Internetworking Technologies
Division personnel actively participate in the design, standardization, development, and testing of next- generation internetworking technology. These activities focus on current design and standardization efforts within the Internet Engineering Task Force (IETF) to add significant new functionality to the Internet Protocol Suite (IPS). Our efforts concentrate in areas that hold promise for the most significant improvements to the capabilities of the IPS infrastructure: network security technology, the next-generation internetwork protocol (IPv6), and protocols and architectures to support integrated services.
In the area of network security technology, staff members took a leadership role in the IETF and vendor community in the design and standardization of internetwork layer security protocols, known as IPsec. IPsec protocols are designed to provide authentication, integrity, and confidentiality services to both the current IP protocol (IPv4) and IPv6. We concentrate our current efforts on IPv4 because of the high level of interest in fielding Internet security technology as quickly as possible.
At the request of IETF directors, ITL staff collaborated with key industry partners to develop several specifications for emerging IPsec protocols. Our staff co-authored IPsec protocol specifications with Cisco Systems, Inc., Bay Networks, IBM T. J. Watson Research Center, the National Security Agency, and Sable Systems.
In addition to providing leadership in IETF standards development, we designed and developed Cerberus, a leading- edge prototype and reference implementation of the latest IPsec specifications. Cerberus serves as a publicly available reference implementation and a platform for ongoing research on advanced issues in IPsec technology.
To answer an industry call for more frequent and accessible interoperability testing for emerging commercial implementations of IPsec technology, we developed the NIST IPsec WWW-based Interoperability Tester (IPsec-WIT). IPsec-WIT is built around the Cerberus prototype and ubiquitous WWW technology and allows implementers to remotely execute series of interoperability tests against the NIST reference implementation. IPsec-WIT also serves as an experiment in test system architectures and technologies. The novel use of WWW technology allows IPsec-WIT to provide interoperability testing services anytime and anywhere without requiring any distribution of test system software or relocation of the systems under test. Work is currently underway to expand Cerberus and IPsec-WIT to support emerging key management protocols and to address IPv6 in addition to IPv4.
In the area of integrated services, we focused our initial efforts on test and instrumentation tools to foster early experimentation with emerging IETF signaling, routing, and transport protocols for real-time traffic. Our tools address the gap between the capabilities of new network services and the requirements and capabilities of new and existing applications. Two of the most difficult aspects of engineering performance sensitive applications are being able to realistically test and measure the behavior of such applications in a controlled laboratory environment, and understanding how to map application requirements to new, real-time network services.
To address the first issue, we developed the NIST Network Emulation Tool (NIST Net) that enables experimentation with arbitrary IP network performance dynamics in a simple laboratory setting. The tool allows developers to use inexpensive PC components to experiment with bandwidth, delay, congestion, and corruption dynamics commonly experienced in large, wide area networks (e.g., the public Internet) and new subnetwork technologies (e.g., xDSL, cable modems).
To address the second issue, staff members developed the Integrated Services Protocol Instrument (ISPI). The ISPI tool enables one to measure the performance of real-time data streams transmitted over IP networks and to experiment with resource reservations without modifying existing applications. ISPI is capable of measuring in real-time the performance seen by applications distributed at various points in the network and to dynamically adjust the real-time attributes of the underlying network service.
To Top
TO CONTENTS PAGE
| Chief : | Stuart Katzke |
| Group Managers: | Miles Smid, Security Technology |
| Tim Grance, Systems and Network Security |
As information technology (IT) and electronic commerce become integral to all aspects of the government and private sectors, the security and protection of that technology becomes critical. Both industry and government require swift, seamless, and secure computer systems and networks to compete in the global marketplace. Our Computer Security Division emphasizes the development of vital tests and test methods, reference data, proof-of-concept implementations, and technical analyses needed by government and industry. These products, tools, and techniques enable security technology developers, vendors, and integrators to deliver high-quality, reliable products into the marketplace.
The NIST Computer Security program has six primary focus areas:
Advanced Encryption Standard
In anticipation of future needs for the next generation of high-quality cryptography, the division initiated the development of an Advanced Encryption Standard (AES) that will provide a strong cryptoalgorithm for use by the public and private sectors in protecting sensitive unclassified information for the next 20 to 30 years. A Federal Register notice of January 2, 1997, announced NIST's intent to develop an AES and proposed minimum acceptability requirements and evaluation factors. Draft submission requirements for candidate algorithms were also announced. An AES Requirements Workshop attracted about 80 participants from industry, government, and academia, including representatives from Canada, the UK, Belgium, and Japan.
In September 1997, NIST issued a public call for submission of candidate algorithms. After the call period closes on June 15, 1998, NIST will make all submissions available for public review and analysis. Through a series of open workshops and public review periods, NIST will select the best algorithm for the AES based on its ability to provide the required level of security first, then on cost and flexibility considerations. For more information, visit NIST's Computer Security Resource Clearinghouse Web site at http://csrc.nist.gov/encryption.
Cryptographic Module Validation Program
The Cryptographic Module Validation (CMV) Program has resulted in the validation of four cryptographic modules as complying with Federal Information Processing Standard (FIPS) 140-1, Security Requirements for Cryptographic Modules. This included the validation of Netscape Security Module 1, by Netscape Communications Corporation, for use by federal agencies. In addition to Netscape's software module, Entrust Technologies has received two validations for software modules used in their Entrust family of products. These modules received an overall rating of Level 1, which may be sufficient for many user's security needs. Four hardware cryptomodules have also received validations, including Fortezza PC Cards from National Semiconductor, Mykotronx, and SPYRUS, in addition to a module used in some of Motorola's radios. The radio module received a security rating of Level 1, while all three Fortezza cards have been validated at Level 2.
FIPS 140-1 specifies four separate levels of security provided by Cryptographic Modules with each level providing increased security and assurance. The Netscape Module was validated for secure email, certificate management, and password management and received an overall rating at Level 2. These validations expand choices for federal agencies in securing sensitive information over the Internet.
The CMV program is a joint effort between NIST and the Communications Security Establishment (CSE) of the Government of Canada. ITL and CSE serve as validation authorities for the program. Currently, there are three National Voluntary Laboratory Accreditation Program (NVLAP) accredited laboratories that test cryptographic modules. Currently, the laboratories are testing approximately two dozen varying types of cryptomodules, which will be posted to the Cryptographic Modules Validation List as they are validated by NIST and CSE.
For more information on FIPS 140-1, validated modules, and the accredited laboratories, visit the Web site at http://csrc.nist.gov/cryptval.
Public Key Infrastructure (PKI)
Without a common infrastructure to support the issuance, management, distribution, and verification of public key certificates, the full benefits of cryptographic services will not be achievable. NIST is leading efforts to develop such a public key infrastructure (PKI). In July 1997, we coordinated a Public Forum on Certificate Authorities and Digital Signatures: Enhancing Global Electronic Commerce sponsored by the Department of Commerce. The forum provided an opportunity for the public to comment on various issues of the public key infrastructure related to certificate authority and digital signatures.
Many of the U.S. Government's PKI activities are coordinated through the Federal PKI Steering Committee. NIST chairs the Technical Working Group of the steering committee and hosts meetings of the committee itself. The working group has produced several technical documents on PKI issues, including an overall PKI Concept of Operations and PKI architectural and policy analyses. Industry participation in the PKI development process showed a marked increase in 1997.
In a related activity, ITL developed the Minimum Interoperability Specification for Public Key Infrastructure (PKI) Components (MISPC) with the cooperation of ten industry partners under Cooperative Research and Development Agreements. The MISPC provides a basis for interoperation between PKI components from different vendors. The specification supports interoperability for a large-scale PKI that issues, revokes, and manages public key certificates that bind public keys used for digital signatures to their owners. Interoperable PKIs are essential as more and more business transactions are carried out electronically. The work is published as NIST Special Publication 800-15.
Key features introduced by the MISPC have been incorporated into industry standards, such as the Internet Engineering Task Force (IETF) PKIX (Public Key Infrastructure Using X.509) documents. The convergence of PKI specifications increases vendor and consumer confidence and encourages the development and procurement of PKI components.
Key Recovery Standard
NIST serves as secretariat for the Technical Advisory Committee for the development of a Federal Key Management Infrastructure. This advisory committee, which met six times in 1997, was established to obtain private-sector assistance in the development of needed cryptographic key management services for the government. As participants in an Interagency Key Recovery Demonstration Project, we established the Pilot Root Certification Authority (CA) and developed Pilot Tests. We also conducted a Broad Agency Announcement for Key Recovery System Components.
National Information Assurance Partnership (NIAP)
To promote the independent evaluation of security products, NIST and the National Security Agency (NSA) formed the National Information Assurance Partnership (NIAP). This new federal initiative focuses on ensuring the security of information technology systems and networks through cost-effective testing, evaluation, and certification programs. The partnership encourages the availability of objective measures and test methods for evaluating the quality of information technology security products.
The NIAP will develop tools, test methods, and tests for specification-based information technology security products. This means that the security functionality and assurance requirements of a product or system must be formally described or specified. These specifications then form the basis for the development and conduct of tests for the product or for a class of product (e.g., for a firewall, an access control device, or even a network router).
In addition, NIAP will promote the development of commercial testing laboratories to provide testing and evaluation services that will meet the demands of both producers and users. NIST's National Voluntary Laboratory Accreditation Program (NVLAP) will be the basis for much of the NIAP test laboratory accreditation efforts. The program should help producers increase the value and competitiveness of their products (in the U.S. and abroad) through the availability of formal, independent testing and certification. NIAP efforts will help users in both public and private sectors by providing a sound and reliable basis for the evaluation, comparison, and selection of security products.
Initial industry reaction to the new partnership has been very positive. NIST and NSA are committed to the National Performance Review goal of transferring methodologies and techniques to private-sector laboratories. Accordingly, the agencies registered the NIAP as a National Performance Review Reinvention Laboratory through the Department of Defense in September 1997. Visit the NIAP home page at http://niap.nist.gov/.
Common Criteria (CC)
The internationally developed Common Criteria (CC) are the focus of much of NIAP's work. The CC provides a comprehensive, rigorous method for specifying security functionality and assurance requirements for products (or classes of products), usually in the form of protection profiles (PPs). The CC provides an internationally recognized basis for specifying and testing a wide range of security technology, from components to products and systems. For more information on the CC, visit our Web site at http://csrc.nist.gov/nistpubs/cc/.
Role Based Access Control (RBAC)
U.S. industry is increasingly using ITL's RBAC research in new product development. Corporations using the RBAC mechanism include the Secure Computing Corporation and Sybase, Inc. Citing the fact that in the software industry there is a premium on bringing new functionality to market quickly, Sybase was able to reduce significantly the development time of the next version of the SQL Server, Adaptive Server Enterprise 11.5 that will include a robust RBAC mechanism based on ITL research.
Under a CRADA, SETA Corporation and NIST are working together to modify the RBAC administrative tool, for use in an SQL/RDBMS environment. Open Group, a consortium that includes nearly every major software and computer vendor, is developing a new security system, Adage, that will be used in a variety of Internet applications. Adage is being designed to include a number of security features that have been derived from the RBAC security model. As a result of NIST's success in transferring its RBAC technology, the ITL RBAC research and development team won the 1998 Federal Laboratory Consortium Award for Excellence in Technology Transfer.
ITL developed the first formal, general model for RBAC, which provides access to IT resources based on a user's role in an organization. A joint project of the Computer Security Division and the Software Diagnostics and Conformance Testing Division, researchers are developing a technical specification, including a formal description, of RBAC on the Web. Also being developed is a proof-of-concept model implementing RBAC. Three patents on this research are in process.
FedCIRC
The number and complexity of Internet-related incidents increased in the past year. With start-up funding from the Government Information Technology Services (GITS) Innovation Fund Committee, ITL coordinated the establishment of the Federal Computer Incident Response Capability (FedCIRC). FedCIRC provides a capability to assist federal civilian agencies in their incident handling efforts by providing proactive and reactive computer security-related services, cost-reimbursable technical assistance, and technical support. To date FedCIRC has six subscribers: the Department of Justice, the Department of State, the U.S. Customs Service, the Department of Agriculture National Finance Center, the General Services Administration Federal Supply Systems, and the Department of the Treasury Bureau of Alcohol, Tobacco, and Firearms.
Security Management and Support
Computer Security Resource Clearinghouse - To facilitate access to NIST publications and guidance, as well as a wide variety of other sources of computer security information, we maintain the Computer Security Resource Clearinghouse (CSRC). The CSRC is a World Wide Web (WWW) site containing references to or electronic copies of many NIST computer security documents as well as links to many other valuable resources available on the Web. The address of the CSRC is http://csrc.nist.gov.
National Information Systems Security Conference - NIST co-sponsors, with the National Security Agency, the annual National Information Systems Security Conference (formerly the National Computer Security Conference). The conference, one of the largest of its kind, provides a forum for the government, commercial, and academic communities to come together to discuss the latest developments in information security technology.
Agency and Public Input - In addition, NIST serves as secretariat for advisory committees and other groups designed to further discussion, cooperation, and coordination among the key communities in the information security field. Two notable groups are the Computer Systems Security and Privacy Advisory Board (CSSPAB), which met four times in 1997, and the Federal Computer Security Program Managers Forum, which met six times in 1997.
Agency Assistance and Collaboration - The NIST Computer Security Division is frequently called upon by other agencies to assist with or collaborate in IT security-related analyses, program development, or technology implementation. In FY 1997, we undertook a number of such projects. NIST participated in an interagency Collaborations in Internet Security (CIS) project to enable multiple-agency use of advances in IT security technology and products. NIST was also involved in supporting standards and guidance development for the Health Insurance Portability and Accountability Act (HIPAA) of 1997. As in previous years, NIST also produced several ITL Bulletins that addressed IT security topics of current interest.
To Top
TO CONTENTS PAGE
| Chief (Acting): | Martin Herman |
| Group Managers: | David Pallett, Spoken Natural Language Processing |
| Donna Harman, Natural Language Processing and Information Retrieval | |
| Charles Wilson, Visual Image Processing | |
| Sharon Laskowski, Visualization and Virtual Reality | |
The Information Access and User Interfaces Division accelerates the development of technologies that allow intuitive, efficient access, manipulation, and exchange of complex information by developing and coordinating measurement methods, evaluation methodologies, test suites and corpora, prototypes, workshops, and standards.
Spoken Natural Language Processing
The Spoken Natural Language Processing Group advances the state of the art of spoken language processing technologies, which serve as an alternative modality for the human-computer interface. They are also used to provide transcripts from speech input that can be searched to provide relevant excerpts of audio. We develop test procedures, coordinate community-wide benchmark tests, provide reference materials (speech corpora) used by the research and development community, and build prototype testbed systems.
The group has worked with the Defense Advanced Research Projects Agency (DARPA) spoken language community since 1984, and has played a key role in the development of speech corpora (databases of speech, transcriptions, and related materials) for the research community to use in both system development and benchmark tests. Over 200 CD-ROMs containing these corpora have been produced by ITL and are used throughout the worldwide speech research community.
Benchmark tests, which we have implemented within this community since 1987, are used to track the development of several speech technologies including English and multi-lingual speech recognition and understanding, spotting technologies, language identification, and most recently, spoken document retrieval. These tests, which provide diagnostic information that helps to identify strengths and weaknesses in the technology, have facilitated increased accuracy and robustness of the technology over time.
The scope of speech recognition technologies under development and test within our community now includes recognition of conversational telephone-channel speech in several foreign languages, including Spanish, Mandarin, Japanese, Arabic, and German. In FY 1997, we implemented large vocabulary continuous speech recognition benchmark tests for DARPA's Human Language Systems Program and other Department of Defense agencies with participation of U.S. and international government-sponsored organizations.
ITL's work has helped to move the research community into real application domains by designing new benchmark tests and test materials using "found" speech from television and radio news broadcasts involving background noise and music, foreign dialects, spontaneous and conversational speech, varying recording/channel effects, and unconstrained vocabularies. Continuous speech recognition benchmark tests now include audio recordings from radio and television news broadcasts in Spanish and Mandarin as well as English. Error rates for the task decreased substantially over the last three years.
In conjunction with the Natural Language Processing and Information Retrieval Group, our group organized the first benchmark test of spoken document retrieval (SDR) technology which involves the retrieval of excerpts from collections of audio recordings. Since SDR is implemented by applying information retrieval techniques to the output of speech recognition systems, the benchmark test provided the opportunity for speech research organizations to work with information retrieval organizations. Fourteen organizations participated in the SDR test and 3 of the 13 submitted systems were joint efforts between speech recognition and information retrieval organizations. Several sites had SDR results which were nearly comparable to their retrieval on human-generated control transcripts and demonstrated that successful retrieval can be performed with current speech recognition technology.
We developed a portable UNIX-based software package (SCLITE) for scoring the performance of speech recognition systems, and we use this software to score, tabulate, and analyze results for the speech recognition benchmark tests we administer. Publicly available, SCLITE is used internally by many research organizations to evaluate their speech recognition systems. It produces a variety of performance reports and includes implementations of several paired-comparison statistical tests. These tests, developed in part with the assistance of the Statistical Engineering Division, quantify the significance of performance differences across systems.
We initiated the development of an interactive spoken document retrieval evaluation testbed. When complete, the modular system will permit us to explore measurement and evaluation techniques for speech recognition and understanding, spoken document retrieval, adaptive language modeling, information acquisition, and human-computer interaction.
Natural Language Processing and Information Retrieval
This group promotes the use of more effective and efficient techniques for manipulating unstructured textual information, especially the browsing, searching, and presentation of that information. The work is accomplished via three complementary mechanisms: the creation of testing materials and metrics for text retrieval, the sponsorship of the annual Text REtrieval Conference (TREC), and the ongoing investigation of specific issues in text access using in-house prototypes.
In past years we developed testing material for evaluating text retrieval in English (over 5 gigabytes of test documents and 300 test questions), created the first test collections ever built for retrieval in Spanish and Chinese, and produced new metrics and testing material for evaluating retrieval using input produced by degraded optical character recognition (OCR). All these testing materials and metrics were first used as part of the evaluation of participating systems in TREC, and then distributed for use by researchers in industry and academia.
During 1997 we expanded this effort into new areas. In cooperation with the Swiss Federal Institute of Technology (ETH), we produced the first test collection for evaluation of cross-language retrieval, i.e., the retrieval of documents in different languages than the input query language. This testing material is comprised mainly of newswire data in three languages: English, French, and German. The 25 test questions built are also in these languages, to enable testing in both a monolingual and a cross-lingual mode.
Additionally, in cooperation with the Spoken Natural Language Processing Group, we created testing material for the evaluation of speech retrieval against 50 hours of broadcast news. New metrics and evaluation methodologies were developed for using this material in TREC-6. As a third project for TREC-6, the group provided extensive guidance to the Australian National University during their creation of a very large (20-gigabyte) corpus for use in testing the efficiency of search engines.
A final project for TREC-6 involved the design and implementation of a new testing methodology for evaluating interactive text retrieval. With major help from the Statistical Engineering Division, we created an experimental design involving the use of a control system to help separate the various factors in the performance of interactive systems. NIST's ZPRISE retrieval system was slightly modified for use as the control system, and nine TREC-6 groups participated in this experimental evaluation test.
The TREC-6 conference, the latest in the ongoing series of conferences designed to improve text retrieval algorithms, took place in November 1997. This conference, co-sponsored by NIST and DARPA, attracts international participation from information retrieval researchers in industry, academia, and government. The conference, which has grown from 25 systems in 1992 to 51 systems in 1997, serves as a major technology-transfer mechanism in the field. Participating groups worked with large (NIST-built) test collections, used the same evaluation procedures, and met for a three-day workshop to compare techniques and results. In 1997 there were ten areas of testing: retrieval in English in both new query and standing query modes, retrieval in Chinese, cross-language retrieval, speech retrieval, filtering, interactive retrieval, high-precision retrieval, retrieval using natural language processing techniques, and retrieval using a 20-gigabyte corpus.
Other areas of research during 1997 included successful experiments building a simple German stemmer and decompounder for use in the TREC-6 cross-language task, the initial design work for a digital library for access to archival publications in information retrieval, and the user testing of a pilot search system (EAMATE) in conjunction with the Social Security Administration (SSA). The group's prototype search engine (PRISE) was used in an experiment to develop better search algorithms based on the type of documents being searched. This experiment led to a revision in the current algorithm. A client/server version of this search engine, called ZPRISE, continued to be distributed as public domain software, and was sent to 30 additional institutions in 1997.
The group worked on two projects particularly oriented towards improved hypermedia access, including the Web. The HyperIndexer project, with SSA, continued its investigation into the building of automatic links, including a small user test of the effectiveness of these links. Along with changes to the prototype based on this testing, new work has started on the automatic creation of embedded links. The second project involves the design and building of test metrics for usability evaluation of Web sites remotely. This project, called WebMetrics, is being conducted jointly with the Visualization and Virtual Reality Group, and has already resulted in experimental testing of three very different Web sites.
Visual Image Processing
ITL supports the technology of image recognition in government and industry by developing new image recognition methods, developing techniques for the evaluation of existing methods, and providing technology transfer to the commercial imaging and document conversion industry.
In cooperation with the Federal Bureau of Investigation, we developed methods for the evaluation of fingerprint applications and mugshot standards. The goal of the mugshot standards project is to develop a standard method for acquiring electronic mugshots which is usable at all levels of law enforcement. Also in 1997, we initiated a project to explore combining a fingerprint biometric with existing digital signature standards. This project will allow electronic law enforcement records to be authenticated with a digital signature accessible only after a matched fingerprint has been used to verify the identity of the sender.
In 1997 researchers in the optical information processing project began to develop evaluation methods for system components of pattern recognition and holographic memory systems. The goal of this project is to develop the metrology needed to industrialize optical information processing using a real commercial application as a test bed. Initially, we explored the feasibility of optical methods of image storage, 3D holography, and combining optical correlation and neural networks for fingerprint matching.
These efforts showed that properly characterized 3D analog holographic memory has capabilities for image storage which is sufficient to support various correlation methods of pattern recognition. In the initial phase of the project, we also showed that a combination of local optical correlation and neural network matching can be used to provide the first advance on minutia matching in 20 years. If properly combined, these methods should allow a new class of optical pattern matching system to be developed. This will require development on new measurement methods to characterize components of the system, such as the input and output devices, and the development of new phase coherent optical computer-aided design (CAD) methods. A researcher from Carnegie Mellon University worked with the group on these new measurement methods.
The expertise developed in this project allowed us to specify a high impact commercial application which could use this technology. This is real-time fingerprint matching for user verification for financial use, credit, law enforcement, and Internet security. The Financial Services Technology Consortium is interested in the first application and several small companies are pursuing the network access market. In both applications, fingerprint matching, retinal scan matching, and face recognition have all been suggested. The projected costs, the input device, and user inconvenience make fingerprint and face matching more attractive candidates than retinal scanning. At the present state of the technology, fingerprint matching provides much higher levels of security. In samples of a few thousand, look-alike faces can usually be found. In 30 million fingerprint samples, no matches have been found between different individuals.
In 1997, the group began an initial investigation of new pattern recognition methods based on the statistical learning theories of V. N. Vapnik. This project may provide a more accurate method of pattern recognition and should allow the performance effects associated with the distributions functions of the testing and training sets to be more accurately evaluated.
Work continues on cost-effective document conversion technology in cooperation with the National Security Agency (NSA). Commercial off-the-shelf (COTS) technology for many areas of document conversion is widely used for tasks such as universal library conversion but this technology does not address the need for large-scale, timely conversion of low-quality documents and the impact of this type of conversion on information retrieval. Studies at NIST which evaluate COTS solutions to this problem show that for high-quality images, COTS packages all perform well; on medium-quality images, some COTS packages perform well enough to be useful; and on low-quality documents, no COTS package performs well enough to be useful.
In 1997, our group collaborated with the Natural Language Processing and Information Retrieval Group and NSA to organize a committee to run the METTREC Conference (Metadata Text Retrieval Conference). Conferences of this type have been used in the case of OCR and TREC to define and focus both commercial and academic research efforts on specific problem areas. These conferences provide high leverage for potential government users and aid in technology evaluation by encouraging exchange of research ideas and by demonstrating the strengths and identifying areas requiring future work in the selected technologies.
Initially, METTREC will use 67,000 pages of the Federal Register (the entire year 1994) which have full typesetting instructions and paper documents. This allows comparison of images generated by typesetting with real images on scanned paper and enables assessment of OCR techniques in generating images. Initially, this comparison took place in a subset of the data in the 1996 TREC Conference. The results demonstrated that word error rates from OCR in the 5 percent range had little effect on retrieval. Future tests in METTREC should demonstrate the effects of 40 to 50 percent error rates that COTS packages produce on low-quality documents.
Visualization and Virtual Reality
Information visualization is receiving much attention within the human-computer interaction and graphics research communities because it holds promise as a technology that will enable the display and exploration of large, complex information spaces. The Visualization and Virtual Reality Group advances the state of the art in information visualization and virtual environment technology through the development of evaluation methodologies, benchmarks, and metrics that address the usability and scalability of three-dimensional visualization approaches and through the creation of proof-of-concept prototypes, benchmark data sets, and formats for simplifying the integration of visualization tools with applications. Secondarily, the group seeks to develop evaluation methodologies for leading-edge human computer interaction technologies that support information access.
In FY 1997, ITL developed several three-dimensional interfaces to the NIST PRISE information retrieval system to support easier access to document collections. These systems now form the basis of experiments to support the evaluation of the effectiveness of such interfaces. The group continues to pursue opportunities within the information retrieval and digital library communities to supply appropriate evaluation methodologies and guidelines that will leverage the technology to support innovative uses of visual displays of information for information retrieval.
A second project, begun three years ago, centers around investigating the appropriateness of virtual environments and the Virtual Reality Modeling Language (VRML) for manufacturing applications. The project is affiliated with users and developers through a collaboration with NIST's Manufacturing Engineering Laboratory and the Systems Integration for Manufacturing Applications program. The goal is to assist the manufacturing community in exploring how visualization can improve the manufacturing process. This led to the software modeling of factory floor assembly lines, machine tools, and parts with all associated multimedia information as a virtual environment in VRML.
The VIM (Visual Interface for Manufacturing) prototype was constructed in FY 1996 and supported an investigation of the feasibility and usability of Web-based virtual environments for modeling manufacturing processes. This was followed by a successful effort in translating various 3D designs and physical models into VRML2 and integrating into one VRML2 model. We also acquired data from the Consumer Product Safety Commission consisting of detailed infant and child measurements. This database was placed into electronic form and converted to VRML models of children via a human body motion simulation package. The VRML models and data developed by ITL have been integrated into several commercial packages.
We continued collaborating with researchers who have formed a group to develop large datasets to support the visualization and data mining community. The goal is to create large, public datasets in which researchers can experiment with and evaluate the effectiveness of visualization techniques to support information exploration. This is similar to efforts in the machine learning community, but with an emphasis on very large, timely datasets. In FY 1997, we successfully participated in a comparison of visualization and mining techniques for the first dataset.
We also continued our work with the DARPA Intelligent Collaboration and Visualization Program. As part of an evaluation effort designed to provide evaluation tools for DARPA-funded researchers, this year we developed a framework for evaluation of collaborative systems and associated scenario templates and test scenarios. This framework will be valuable as there currently exists a large body of knowledge about collaborative systems and computer-supported cooperative work, but no easy way to structure and apply this knowledge across the many dimensions of group work and technologies.
To Top
Chief: Dean Collins
Group Managers: Gordon Lyon, Scalable Parallel Systems and Applications
Jack Newmeyer, High Performance Systems Usage
John Antonishek, Network and Telecommunications Systems
Developers and users of high performance computing technologies need tests, test methods, and innovative measurement standards to produce high-quality products and to evaluate the products. The division assesses the functionality, interoperability, and operational characteristics of high performance systems to assist industry and users. It also provides high performance computing and telecommunications services to NIST researchers.
S-Check
S-Check is NIST's novel tool for assaying and improving performance of parallel and networked programs. This year researchers expanded the capability of S-Check by developing extensions of the tool. S-Check augmentations now allow users to acquire the tool with either syntax-checking of C code or with a capacity to handle multi-languages without syntax-checking (variant S-Check ML). R&D Magazine selected S-Check (r) as a R&D 100 Award Winner for 1997. The tool is available at: http://cmr.ncsl.nist.gov/scheck/scheck.html.
S-Check is especially suited for improving code running on parallel systems, where component interactions are common but difficult to evaluate with conventional profiler tools. Displayed in color graphical formats, results from S-Check provide programmers with quantitative predictions on the effect of code improvements in their programs without actually requiring the changes to be made, saving a lot of time. With the added extensions, the tool is stable and mature for extended marketing to a wide variety of users.
Distributed Systems Technologies
ITL is working with industry to apply MultiKron boards for assessing the performance of systems of various sizes. The MultiKron series of Very Large Scale Integration (VLSI) instrumentation chips and interface boards are measurement tools that promote the development of high performance computing and flexible, scalable systems. The Microsoft Corporation expressed an interest in applying the board to normal applications on multi-processor machines, to video servers where Peripheral Component Interconnect (PCI) utilization and synchronization are important, and to client server and cluster performance applications where a synchronized timebase is important. ITL provided four MultiKron boards to Microsoft and is working with the company to enhance the functionality of the technology, particularly in the area of clustering.
Work continues on our public server asynchronous transfer mode (ATM) cluster "Buffet," a demonstration testbed for distributed clustered computing for the NIST technical community. We added and tested representative programs on the cluster, and installed 16 Pentium Pro machines on the cluster to provide substantial compute power to message passing interface (MPI) users on the ATM network. Many MPI programmers use a Single-Program Multiple-Data (SPMD) model that assumes a homogeneous set of processors; the Pentiums provide this base set.
Time Synchronization for Distributed Computing
Synchronized clocks are important in the distributed computing market, where they support commerce, security,
monitoring, and billing. The NIST Physics Laboratory (PL) provides a synchronization service on the Internet with 10ms precision; the service receives about two million synchronization requests per day. NIST would like to increase the precision of the service by overcoming the problems of variability of delays, to one microsecond.
ITL is developing a hardware supplement employing our MultiKron (r) clocking chip and low-cost commercial global positioning system (GPS) receivers. We are testing the hardware implementation for this application. Although such hardware support is sufficient to achieve higher precision time synchronization, the cost and availability of such specialized hardware limit its use. PL and ITL will investigate distributed software time synchronization algorithms targeted to achieve one microsecond precision and use this NIST hardware support instrumentation to evaluate the precision achieved and identify the impediments encountered. These low-cost techniques and guidelines can then be transferred to industry.
Interoperable Message Passing Interface
ITL and industry are working together to define an interoperable message passing interface (IMPI) standard.
In September 1997, ITL hosted the third Interoperable Message Passing Interface (IMPI) Workshop. All major U.S. computer vendors participate in this effort; representatives of these vendors as well as representatives from the embedded computing community attended the workshop. The group agreed that implementation of IMPI by the vendors would go on in parallel with the standard definition in order to get the most information on the choices made. The release of the first standard, IMPI-1, is expected by December 1998.
WebSubmit
ITL researchers continued to enhance WebSubmit, an advanced Intranet application tool that provides a Web page interface to supercomputing applications. It differs from other Web applications because it allows interaction with a user's data files and directories on the target supercomputer as if the user were logged on. The advantage of a Web-based interface is that it is hardware- and software-independent; it depends only on whatever Web browser the user has available.
The current implementation of WebSubmit provides an interface to an IBM SP2 running the IBM LoadLeveler job scheduler. It supports general LoadLeveler use, utility operations on the SP2, and an application-specific interface
to Gaussian94, a computational chemistry package. All of the Web pages are dynamically generated with common gateway interface (CGI) scripts written in Tcl. The Tcl code is modular, making the addition of new interfaces very easy, and simple to customize for a particular SP2 site. Future enhancements will include the addition of a NIST-developed cryptography toolkit for security, a generalized module constructor, as well as extension to other computer environments. Software developers and Web-page designers have requested information about this tool.
Parallel Library for NIST Staff
Work continued on the development of a portable parallel library for NIST users. We wrote a simple version of a "reproducible parallel Random Number Generator" called rrandom_array(). It produces the same array or the number of processors the array is distributed across.
For heterogeneous cluster computing based upon MPI, we added "unions" to the AutoMap/AutoLink utility set, enhancing the ability of NIST users to send data structures over network connections. Similarly, the NIST tool PADE can now manage DPARLIB code (the NIST parallel library) on multiple platforms.
Due in part to library development by ITL staff, the SP2 is now being used as a true parallel machine. The SP2's 16-node queue is in greatest demand by users, indicating that NIST users are adopting parallel coding techniques.
CD-Electronic Book
The CD-Electronic Book project demonstrated for the NIST staff the use of Microsoft Word and Powerpoint to simulate the features of an electronic book with touch-screen display. The software simulated electronic pagination, annotation, and electronic dictionary functions. Work is underway to use Visual Basic for the two-page version of the book.
Flat Panel Display
ITL played a significant role in the development of several recent standards within Video Electronics Standards Association (VESA), an industry standards association which develops display-related voluntary industry standards. Flat panel displays are an essential enabling technology for portable computers, desktop monitors, and instruments. Panel architecture makes proper timing much more critical than for cathode ray tube (CRT) displays; failure to meet the timing requirements can cause permanent damage to the panel. The standard timings and "plug and play" compatibility of CRTs have not been available for flat panel displays, an omission that VESA is moving to correct.
At VESA's request, researchers analyzed flat panel display interface timing characteristics, based on product information submitted by manufacturers to ITL which conducted an impartial analysis. The necessary timing characteristics were derived, analyzed, and reported to VESA as general information without brand names or product identification. This information will be used as the technical foundation for a new standard for interfacing flat panels and other digital input displays to computers.
Co-sponsored by ITL and VESA, the Display Forum '97 Workshop in October 1997 attracted 85 representatives, mostly from industry. Future trends of the display industry were discussed and recommendations were made on ways in which ITL can continue to support industry in this work.
Advanced Information Processing, Recognition and Storage Systems
Work continued on developing simulation models for ATM network architectures that include Network-Attached Storage Devices. We also established an optical tape testing laboratory, led an industry effort to define dynamic optical tape testing methods, and initiated the development of a digital media error test system with remote testing capabilities.
We initiated work under a Cooperative Research and Development Agreement (CRADA) with Calimetrics, Inc., for the investigation of test methods, standards, and High Performance Distributed Computing applications for CD-ROM and Digital Versatile Disc (DVD)-based optical data storage subsystems. We led an industry effort to specify prototype metadata for the portability of sequential storage media (including optical tape) between File Storage Management Systems (FSMSs). A staff member, Fernando Podio, was appointed by the Association of Information and Image Management (AIIM) as the chair of the FSMS Subcommittee to lead the effort of developing an industry standard based on the metadata specification. This effort received wide industrial support among FSMS vendors and mass storage and high performance computing users.
ITL worked with the AIIM Optical Tape Study Group to develop a preliminary set of media specifications and identified a methodology for dynamic testing of optical tape. These proposals were submitted to AIIM for the development of industry optical tape standards: Write-Once Read Many Times (WORM) Optical Tape Cartridges in a "3480" form factor; and "Media Error Monitoring and Reporting Information (MEMRI) to Verify the Integrity of Stored Data on Tape Media."
The NIST Scientific Computing Environment
The continued upgrading of the IBM SP2 expanded the capabilities previously provided for NIST researchers for scalable parallel processing. Upgrades to the system included the addition of IBM new switch technology to the existing 31 nodes; 6 additional nodes to create a 37-node system; an IBM Parallel System Support Package and Performance Toolbox Software upgrade for the SP2 control workstation; and a 256 MB memory upgrade for the SP2 control workstation. In addition, we acquired an SGI Origin 2000 system with 8 CPUs, 8 GB memory, 96 GB disk storage, and operator and system administrator consoles. The security of the SP2 system was also enhanced.
Networking and Telecommunications
We continued to upgrade the PEPnet/Eznet network infrastructure at NIST. The MAC watch (NIST developed) and Arp watch software are now running in an automated environment. This software has proved to be a real time saver in the PEPnet operation. The software has been ported to the NIST Boulder Eznet environment where it is running successfully. Further enhancements are planned.
A switched CAT5 network was installed in NIST's Radiation Physics Building to support PEPnet users in the Physics Laboratory. We also provided switched FastEthernet to ITL's Scalable Parallel Systems and Applications Group for their cluster computing experiments. The upgrade of Eznet to CAT5 cabling continued. All NIST Administration Building networks were migrated to the FDDI-based Cabletron switches.
Our network monitoring programs were modified to page our staff if a subnet becomes unreachable or show a packet loss of greater that 15 percent. This is determined by testing a single host on the subnet that was supplied by the subnet administrator as "always available."
To provide state-of-the-art telecommunications services to the NIST staff, we are identifying the requirements of future telephone services at NIST, determining needed features for a telephone switch upgrade. We also started a trial of test users for Bell Atlantic's Asymmetric Digital Subscriber Line (ADSL) service.
To Top
The Distributed Computing and Information Services Division provides the information technology resources, supporting infrastructure, applied research, and assistance to NIST staff, collaborators, and clients in the conduct of NIST's scientific, engineering, and administrative applications and in the dissemination of information, including:
In FY 1997, the Distributed Processing and Operating Systems Support Group continued to provide the NIST staff with high-quality computing support services. One major project was to simplify and streamline the administration of the NIST-wide software checkout service. This service provides NIST scientists and researchers the option to access and use expensive, licensed data processing software packages. Updated versions of these software packages were obtained and installed on the server to ensure that the users had the latest versions. The administration of the software was improved and automated where possible.
The NIST electronic mail servers operated by the group in Gaithersburg and Boulder continue to provide highly reliable service to over 2,500 users, with over 3,000 electronic mail messages processed per day. These servers have been reprogrammed to provide the electronic mail alias translation service for NIST staff, so that e-mail addressed to firstname.lastname@nist.gov is delivered no matter where the staff member actually receives mail. A third e-mail service, administering over 200 mailing lists used by NIST staff, was also updated. Three of these lists are large and are updated automatically overnight -- that for Gaithersburg staff, that for Boulder staff, and that for all NIST staff.
During the year the group assumed responsibility for another NIST-wide service, the administration of the Usenet News server. Usenet is an informal collection of news servers supporting over 25,000 special-interest groups. The news servers allow individuals to read and post messages to those groups, and to periodically transfer messages to other servers supporting the same news groups. The group manages the news server that communicates with servers external to NIST, distributes postings to all NIST users, and supports the NIST-specific special-interest groups.
The upgrade of hardware continued, to support the increasing demand for service by the NIST staff and to improve system reliability. A new server, a Sun Microsystems Ultra Enterprise 4000, was acquired. Once configured, this server will replace five currently used computers and provide better and faster service.
Information Processing Support
In FY 1997, the responsibilities of the Information Processing Support Group continued to grow. Currently, the group hosts a total of 28 World Wide Web (WWW) servers that provide Web services to 7 of the 11 NIST Measurement and Standards Laboratories and Programs. These are the Information Technology Laboratory (ITL), Technology Services (TS), the Chemical Science and Technology Laboratory (CSTL), the Manufacturing Engineering Laboratory (MEL), the Physics Laboratory (PL), the Advanced Technology Program (ATP), and the National Quality Program. The group also constructed an operating prototype of an electronic commerce system for limited use by Technology Services to sell standard reference materials and databases through the Internet.
System performance and security are continually monitored by the staff and by special software installed on the servers. The software is programmed to page the staff responsible in case of a malfunction or in case it appears that an intruder is trying to obtain access to any part of the system. This approach of centralized monitoring by staff who are experienced in this area results in a more secure system and improved operation over the previous approach of having each Operating Unit (OU) run its own system.
The group also provided consulting and programming services to NIST OUs. Help with the creation of Web pages, with access to databases from the Web, and with the programming of special OU-specific WWW applications was provided to NIST staff. Assistance was also provided to OUs interested in developing intranets for use by their staff. Additionally, applications that generally support NIST staff, improve communication between NIST offices, and provide monitoring of Web access and traffic were developed. Improved services included development of a Web forum capability, development of electronic forms, and generation of statistics of Internet users that access NIST WWW servers. The group also participated in the development of policies for NIST Web pages. CGI scripts to warn Internet users when they exit NIST were developed with the cooperation of the NIST Information Coordinators.
Additional information on all group services may be found at
http://webservices.nist.gov/
Administrative Computing Support
In FY 1997, the Administrative Computing Support Group completed an inventory of the administrative systems at NIST. The inventory consisted of one hundred and thirteen systems with over one million lines of code, written in COBOL, dBase, FOCUS, and REXX. The majority of these systems support the three primary business units at NIST: the Chief Financial Office, the Office of Human Resource Management, and the Acquisition and Assistance Division. This inventory served as the basis for determining which systems may have Year 2000 problems, prioritizing which systems need to be addressed first, and reporting progress to NIST senior management, DoC, and the Office of Management and Budget (OMB).
The group also completed a series of pilot conversion projects. These pilot projects allowed the staff involved to learn about the Year 2000 problems and to implement solutions. These projects also served as templates for future conversion projects. During a four-month period, 16 systems were converted and made Year 2000-compliant.
The group supported the Department of Commerce Administrative Management Systems (CAMS) project. The CAMS effort may be viewed as the Core Financial System (CFS) (General Ledger, Accounts Payable, etc.) and supplementary modules to be developed by NIST and other DoC agencies. The testing of the CFS required a considerable amount of staff resources. Insufficient documentation, changes in configuration management, and suspect program design required extensive code review while testing and debugging.
NIST agreed to take the lead to develop two supplemental modules, a Personal Property module and a Time and Attendance (T&A)/Estimated Labor module. For the Personal Property module, after reviewing commercial, off-the-shelf (COTS) packages, Oracle Assets was selected in April 1997 as the base package, and the implementation began at NIST in May 1997. A model of the proposed T&A/Estimated Labor module was completed and was demonstrated to yield estimates accurate to 99 percent from the T&A records.
In addition to these two initiatives, the group continued to support the business operations of the 100+ administrative computing systems at NIST. This included revising DOS-based systems to be Windows 95-compatible, responding to auditors' requests, developing Web pages for the administrative officers and secretaries, assisting in automating the telephone billing system, and supporting the ATP Survey project.
PC Support
The event that had a major impact on NIST staff was the approval by the NIST O-Board of ITL's recommendation to change the current collection of software office automation packages to a modern, integrated suite of software. The cutoff date for the switch to the new package was set as October 1, 1997. In preparation for the changeover, a number of training sessions in the use of the new software were offered by group members to NIST staff. In addition, seminars that highlighted the differences between the previous group of NIST office automation software and the new integrated suite were held in Gaithersburg and Boulder.
To augment the Help Desk support staff, a contract for telephone support for the new software products was let to Battelle Memorial Institute at the Pacific Northwest Laboratories (PNL) in Richland, Washington. Battelle currently runs this Department of Energy laboratory. Initial statistics on this service indicate that, although the staff that used it considered it very helpful, very few calls were made. If the call rate remains low, current NIST staff would be able to handle this service and the contract with Battelle will not be renewed.
In the past year, steps were taken to improve communications between group members and PC support staff employed by other OUs at NIST with the goal of improving service level across NIST. The PC Assistance Group was revived and meetings to improve staff "networking" and to discuss installation, operation, and maintenance of the new software suite were and continue to be held frequently. A new NIST-wide group, the PC Liaison Group, was also formed to facilitate staff networking and discuss hardware problems.
Another development in 1997 was the replacement of the DOS- based virus detector/remover with a Windows-based virus detector/remover whose virus list is updated automatically through the Web and which can detect virus in e-mail. A site license for the new detector was purchased for NIST and group members developed a self-installing package that can be accessed through the NIST intranet.
Another new service added to group responsibilities was to provide installation and maintenance support for NIST's Travel Manager System. Travel Manager grew from a small pilot project to a system that is currently servicing more than 300 users. Many of the NIST laboratories plan to make the use of Travel Manager mandatory on January 1, 1998.
Staff continue to work on methods to improve software distribution to NIST users. Prototypes are being developed to make installation of new software through the Web much easier.
To Top
Activities in the Software Diagnostics and Conformance Testing Division focus on the development of software evaluation technology, conformance tests, and standards that can be used to assist U.S. industry in the development of high-quality software. In this role, the division develops software testing tools and methods, participates with industry in the development of forward-looking information technology standards, and leads efforts for the development of conformance tests even at the early development stage of standards.
Software Analysis Tools
Division researchers participate in the development of tools for static and dynamic analysis of software, focused on measuring conformance to specifications and diagnosis of the causes for deviations from specifications. Included is work on static analysis tools for program slicing and generation of paths for basis testing, and the extension of object-oriented languages to allow for the detection of pre- and post-condition violations.
Building on previous work in role based access control (RBAC), we developed a prototype of the first RBAC extension to World Wide Web servers. The software developed in this project can be incorporated into existing Web servers to provide access control based on users' roles in the organization, simplifying security for both administrators and users. Our formal model for RBAC, developed jointly with the Computer Security Division, was adopted by two major software vendors for incorporation into their own products.
A unique graphical search engine, Reference Information for Software Quality (RISQ), was developed in cooperation with the Software Engineering Institute at Carnegie-Mellon University. The RISQ search tool uses a graphical taxonomy- based query system to simplify and speed up user searches for information on software quality. Although developed for searching software quality information, the RISQ facility is ideally suited to other types of taxonomy-based data.
For the Nuclear Regulatory Commission, ITL developed the Unravel Program Slicer which computes "slices" of C programs, where a slice is a subset of the program that contains all lines of code that can directly or indirectly affect the value of a particular variable at a particular point. The tool assists in the effort to debug or test a program since it allows the programmer to focus on those parts of the program that are relevant to the logic in question. Other front ends for the Unravel Program Slicer are being considered for development for other programming languages such as C++ or Java.
To allow software developers to measure the effectiveness of development techniques and compare their results with others, a unique database of software errors, faults, and failures is being developed. In addition, to assist industry in the evaluation of the methods and tools they use to improve software quality, we are developing a library of standard reference materials consisting of software with known errors. This library can be used by software developers to determine the effectiveness of the test tools and techniques they use in developing their systems. As part of this effort, we plan to develop statistical methods of evaluating testing procedures and software development tools and techniques.
We initiated a project to develop methods and tools for automatic generation of tests from software specifications. Successful automated testing methods promise significantly more economical means for testing software than are currently available. This will reduce time to market for software products and benefit consumers by making quality software less costly to produce.
Software Conformance Testing
Division researchers develop test methods and tests to determine whether implementations of public specifications conform to that specification. The division is also working closely with outside organizations to help them establish certification test services based on division tests, for the issuing of certificates of conformance.
Working with industry, we are developing conformance tests and test tools for the Virtual Reality Modeling Language (VRML), Java, Computer Graphics Metafile (CGM), and Programmer's Imaging Kernel System (PIKS). For VRML, we developed a reference parser and initial VRML Test Suite (VTS). We are continuing to augment the VTS, by extending the parser and using it to automatically generate VRML test files. To develop the VTS, ITL researchers work in cooperation with the VRML community to validate completed tests and to develop new tests.
In the area of Java, we initiated a project to develop conformance tests, tools, and methods for the areas of Java technology that need test metrics. An ITL-sponsored workshop on Java conformity assessment provided a forum for interested parties to discuss testing philosophies and methodologies for the varying Java technologies and to formulate a direction forward. Working with industry, we will build Java tests and tools that will ensure the consistency and accurate use of the Java specification, applications, and applets.
In cooperation with the Air Transport Association (ATA), ITL is building Computer Graphics Metafile (CGM) conformance test suites and is helping ATA establish its certification service. Also cooperating in this effort is the Aerospace Industries Association (AIA).
Additionally, we are working with several private sector companies and federal agencies to assist them in the establishment of conformance testing programs. In support of this technology transfer, we are developing a certification system framework which describes the processes and procedures for establishing, administering, and operating a conformance testing service. Our goal is to transition current operational testing and certification programs from ITL to private industry, as well as help to establish new testing programs in the private sector. In support of this effort, we developed a Web-based Directory of Conformance Testing Programs, Products, and Services for information technology standards. This directory provides a source for linking providers of conformance testing materials and services with users of these materials and services.
As a research project, ITL is currently investigating new and more efficient approaches to the development of conformance tests. Current conformance tests are developed through the extremely time-consuming process of developing falsification tests. Such testing, while extremely useful, can never cover all requirements and thus never actually provides total proof of correctness. We are investigating new methods for software testing based on stochastic processes and statistical measures in order to improve the quality of software and to provide quantitative measures for determining the probability that software correctly adheres to its specifications. In addition, we are investigating the effectiveness of using automated test generation methods to develop conformance tests for specifications of standards. This research could lead to faster ways of developing conformance tests, which would then result in an increased capability for product developers to determine if their products work according to specifications.
Forward-Looking Standards
ITL researchers seek to make significant technical contributions to standards which are on the cutting edge of software technology. In this effort, we pursue work in areas that have a research component for either the technology itself or for the concept of conformance testing, and for which vendors preferably do not yet have implementations or other vested interests in the work. In performing this work, ITL researchers are working both within the traditional standards community, such as Information Technology Technical Committees that operate under the auspices of the American National Standards Institute, and within other standards related organizations such as the World Wide Web Consortium.
Users of standards are often faced with the daunting task of trying to locate and access standards that are appropriate to their work. To simplify this task, we are working with other standards-related organizations in the development and application of a taxonomy and framework for standards that can be used to both coordinate the development of forward-looking standards and to assist potential users of standards in finding and applying those standards that are applicable to their particular requirements. As part of this effort, an online standards locating and retrieval capability, offering the user multiple interfaces, was implemented to provide access to standards from both government and commercial sources. This retrieval capability is being expanded to provide World Wide Web availability for retrieval of correct values for specific instances of data codes in those standards containing large numbers of uniquely identified data codes.
In cooperation with the Environmental Protection Agency, ITL continued its work on a project that focuses on development of several specific infrastructure components needed for the intelligent integration of database information with intelligent information services techniques. Particular attention is being paid to the application of classification taxonomies and ontologies, as related to development of an Environmental Data Registry, in order to provide for testable, high-quality access interfaces for multiple types of software that serve as information search engines. As part of this task, we are investigating the application of specified infrastructure components in the research area of network Object Registration.
Also in the area of Object-Oriented technology and testing, ITL is working with other organizations in developing and testing object-oriented technology components that can be specifically applied to the concept of "distance learning." In this area, ITL researchers are working on the complete network system from interoperability standards to repository object exchange protocols, including metadata fields for educational objects, and finally to the concept of conformance testing of educational objects.
To Top
Chief (Acting): Keith Eberhardt
Group Managers: M. Carroll Croarkin, Measurement Process Evaluation
Keith Eberhardt, Statistical Modeling and Analysis
The Statistical Engineering Division seeks to catalyze experimentation, enhance research and improve communication of results by working collaboratively with, and developing effective statistical methods for, NIST scientists and our partners in industry. To accomplish this mission, the division develops strong collaborative research relationships with NIST staff in all fields, maintains expertise in the development of statistical methods relevant to measurement science and technology, and ensures that NIST staff have access to information on the latest statistical modeling and analysis techniques needed for their research.
Statistical Consulting
The division collaborates with NIST staff on research projects where optimal experiment design, statistical modeling, and data analysis can play a significant role in improving measurement processes and gaining scientific insight. Staff members also provide general statistical consulting services to NIST scientists working in all aspects of NIST's mission. Specific contributions include:
Statistical Planning for a Neutron Lifetime Experiment
ITL statisticians contributed their expertise in planning an experiment at the NIST Cold Neutron Research Facility (CNRF), where researchers are developing a dramatic new technology to trap polarized ultracold neutrons in a three-dimensional magnetic trap filled with superfluid helium. With this technology, the mean lifetime of the neutron will be determined. The present experimental value of the neutron lifetime is 887.4 s. The associated estimated standard deviation is 1.7 s. The planned experiment should reduce the experimental error by a factor ranging from 10 to 100. Along with other experimental data, this measurement allows one to test the consistency of the standard model of electroweak interactions. Further, the mean lifetime of the neutron is an important parameter in astrophysical theories.
At the CNRF, many runs of a two-stage experiment are planned. In the first stage of each run cycle, the trap is filled with neutrons to a desired level. In the second stage, scintillation events due to neutron decay or background processes are observed. The statistical properties of the lifetime estimate depend on the time allocation between the fill and decay stages, as well as the particular nonlinear algorithm used for estimation of the mean lifetime from the time sequence of events. In collaboration with NIST's Physics Laboratory staff, our staff developed a stochastic model for the data. Based on this model, a strategy for maximizing the precision of the lifetime estimate was formulated.
Optical Fibers
In another collaboration, ITL statisticians achieved a more precise measurement of polarization mode dispersion (PMD), which arises in single-mode communication fibers when there is imperfect circular symmetry in the fiber core. An optical pulse input to a fiber is split into two orthogonally polarized pulses. Distortion arises as a result of a differential group-delay time between these two orthogonally polarized pulses at the output. This differential group-delay can have a limiting effect on the speed of digital communication systems and therefore is a good indicator of the performance of a lightwave system. PMD is routinely measured both at the manufacturing stage and in installed systems.
Among the methods of PMD measurement, the fixed analyzer technique is perhaps the simplest to use. Division scientists collaborated with NIST's Electronics and Electrical Engineering Laboratory to improve PMD measurements using the fixed analyzer technique. A new value for the polarization mode coupling factor of 0.805 (a 2 percent discrepancy with the old value of 0.824) was found. Systematic biases due to sampling density and extrema thresholding were quantified (6 to 12 percent for typical measurement conditions), and a simple correction algorithm was developed which removes the effects of these biases to within 1.7 percent.
Ballistic Imaging Interoperability Test Methods
To facilitate interoperability between existing ballistic imaging systems, the Office of the National Drug Control Policy, the Federal Bureau of Investigation, and the Bureau of Alcohol, Tobacco, and Firearms executed a memorandum of understanding recognizing that the two ballistic image systems currently in use should be interoperable. Under this memorandum, NIST, as a neutral third party, was charged to develop a standard for interoperability and to develop and oversee interoperability conformance tests. The purpose of ballistic imaging systems is to permit forensic evidence (cartridge cases and bullets) recovered at a crime scene to be imaged and compared to an existing database of thousands of images to identify possible links between crimes previously unsuspected as being related. However, due to differences in software, image acquisition, and networking capabilities, the images captured on either one of the two systems cannot be used on the other, thus denying crime laboratories full access to all image databases.
NIST has developed a specification for interoperability between the two image systems that requires the capability of creating cartridge-case images on either system in such a manner that the image can be correlated to the database on the dissimilar system. With respect to image acquisition and matching, the concept of interoperability is not a yes/no characteristic but rather is a matter of degree of interoperability. That is, it is recognized that acquiring images on a non-native system may produce subtle differences in image "quality" that could result in changes match probability, relative to images acquired on the native system.
To address the problem of experimental evaluation of interoperability for image-matching, division staff developed measures of disarray for comparing ranked ballistic images from a native database to the array of images obtained by matching the corresponding non-native test image. Initially, ordinary rank correlation coefficients were suggested for measuring and testing interoperability, but these were thought to be inadequate since only the top few rank positions are believed to have any practical significance. So, we developed the statistical theory for two new rank coefficients of disarray that assign greater importance to the higher ranks. Furthermore, by contrast to common rank correlation procedures, these procedures are designed to measure the agreement of a novice "judge" with a known standard ranking, rather than the mutual agreement between equally weighted rank vectors. A limited interoperability test program is presently underway, and an analysis based on our interoperability measures will be used to evaluate and refine system modifications in advance of more extensive testing.
Intrusion Detection in Computer Networks
Network vulnerability is widely recognized as a key problem for the technological future. Hack attacks or performance anomalies have, in a number of high-profile cases, crippled key industries and services. This makes it urgently necessary to develop tools that identify aberrant behavior as early as possible, and flag those problems and apparent causes for the system administrator.
The Defense Advanced Research Projects Agency (DARPA) awarded a three-year grant for joint work in our division and Carnegie Mellon University on intrusion detection in computer networks, and, more generally, anomaly detection in complex datasets. The work entails the combination of methods from time series analysis, cluster analysis, multivariate analysis, and pattern recognition. The ultimate aim is to develop a system monitor that combines high sensitivity, low false-alarm rates, and the capability to respond in real-time to apparent threats to system integrity.
The project takes advantage of an existing cross-ITL group of researchers who are studying the junction of statistics, data mining, and visualization. The group established competence in this inter-disciplinary area by participating in a network intrusion detection contest. The contest was designed to explore the relationship between data mining and visualization. The ITL group produced one of the most successful entries in the contest.
Sharpness of Scanning Electron Microscopes
Fully automated or semi-automated scanning electron microscopes (SEM) are commonly used in semiconductor production and other forms of manufacturing. Sharpness is an important characteristic of SEMs, and at the present time, no self-tests are incorporated into these instruments to test this characteristic even though the goal of industry is to have the instruments perform without human intervention for long periods of time. Nien-fan Zhang of our division and Michael Postek of NIST's Precision Engineering Division collaborated on identifying suitable test objects for this purpose and developed a statistical measure of sharpness, based on a multivariate kurtosis statistic with an appropriate analytic algorithm. This year, a small company that develops software for precision instruments implemented the algorithm in software and demonstrated its effectiveness at the 1997 SPIE Conference; and IVS, Inc., requested assistance in implementing the technique.
Random Number Generation for Evaluation of Cryptosecurity
Ensuring the integrity of information technology is now a national issue whose importance has been highlighted by many recent security compromises. A crucial component of system security is the encryption of data and text. The standard format for encrypted transmissions depends upon large, nominally random, and balanced binary bit streams, which are used as keystreams in stream ciphers or in block cipher algorithms. Their randomness and balance are critical. Failure of either quality provides an exploitable compromise in system security.
Statisticians in our division, together with ITL's Computer Security Division, are collecting, inventing, analyzing, and organizing algorithms designed to test the integrity of the random binary generators that are the key components of security systems. The study seeks to clarify the interdependencies among the tests and to identify tests to cover deficiencies among those already in use. The final product will be a highly portable, user-friendly package that will enable a wide spectrum of government, commercial, and private users to validate the cryptosecurity of their systems.
Engineering Statistics Handbook
We continue our collaboration with the SEMATECH Statistical Methods Group on the production of a Web-based statistics handbook to help physical scientists and engineers incorporate statistics into their work more efficiently. The material in the handbook will include case studies from the semiconductor industry and NIST laboratories. The handbook will also be integrated with statistical software, allowing users to reproduce examples from the handbook interactively and to perform similar analyses on other data.
Over the last year, prototype handbook designs, navigation, and technical materials have been favorably critiqued by several audiences, including engineers, statisticians and managers from SEMATECH member companies, academia, and other industries. Current work includes formal usability testing as well as development of new technical material. The usability testing is being done in collaboration with ITL's Visualization and Virtual Reality Group and will examine both the organizational structure and content of the handbook. The project is supported in part by the NIST Systems Integration for Manufacturing Applications program which is part of a government-wide effort in High Performance Computing and Communications (HPCC).
TO CONTENTS PAGE
TO CONTENTS PAGE
Distributed Processing and Operating Systems Support
TO CONTENTS PAGE
TO CONTENTS PAGE
The following examples of statistical applications are typical of the work of our division.