To Beginning of NISTIR 5372
To Section 3 

2 General procedures

2.1 Validation by testing

2.1.1 Overview

The process of validation by testing consists of a test laboratory conducting the conformance tests on a client's implementation and reporting the results of that testing in a Validation Summary Report (VSR). For details of the interpreter testing process see Section 3.

If the validation procedures are followed and the VSR shows that the IUT demonstrated conformance to the standard and the application profile, then a certificate is issued to the client. Thus, a certificate is only issued if there are no errors detected by the validation process. A Registered Report (i.e., VSR) without a certificate will be issued for interpreters that have been tested and contain errors. Generally, a client must receive a Certificate of Validation or Registered Report in order for the client's implementation to be procured by Federal agencies.

A certificate received as a result of interpreter testing is valid for two years. A Registered Report, without a certificate, is valid for one year.

2.1.2 Validation Test Software and Interpreter Test Suites

MetaCheck1 with the MetaCALS Option has been designated as the official Validation Test Software. The IUT is tested using the latest version of the Validation Test Software or the Interpreter Test Suite, whichever are applicable. New releases or versions of the test software and/or test suites will be issued to correct existing errors, enhance test routines, and reflect changes made to the FIPS or application profile.

The Interpreter Test Suite consists of a collection of CGM files, operator test script, and set of reference pictures. The client is required to abide by the conditions of the site license for the Interpreter Test Suite.

2.1.3 Renewal of a Certificate of Validation A Certificate of Validation may be renewed for additional years, if the following conditions are met:

2.1.4 Test Report

The Test Report presents the results of the validation. The Test Report contains information about the client, validation test software and test suite versions, as well as errors that may have been detected. Additionally, the VSR may contain other information gathered during the validation process, such as the profile conformance statement.

A draft Test Report and Notification of Conformance form is sent to the client. The client should review the report and return the signed notification form to NIST. Once the signed notification form is received, the Test Report becomes final, is designated as a Registered Report, and is entered on the Validated Products List.
Until the Test Report is finalized, all information concerning the validation is considered to be confidential. If the client does not wish to release the Test Report information, the notification form should not be signed or returned to NIST. No further action regarding the validation will be taken.

2.1.5 Validated Product List

NIST/ITL publishes, on a quarterly basis, a Validated Products List that is a collection of registers describing implementations of Federal Information Processing Standards that have been validated for conformance to FIPS. The VPL contains information about the organizations, test methods, and procedures that support the validation programs for the FIPS identified in this document. The VPL is intended to serve as an index to more detailed information.

For CGM, the VPL will list those implementations that have completed conformance testing and received a Certificate of Validation and/or Registered Test Report. The VPL entries are a limited extract from the Registered Test Report. It is recommended that the Registered Test Report, available from NIST or the testing laboratories, be obtained for the complete test results. The Validated Products List may be obtained by request from:

The VPL may be accessed on the World Wide Web at 2.1.6 Role of the procuring agency

The procuring agency has the responsibility of reviewing the CGM validations listed in the VPL and determining the applicability of these validations to the hardware/software environment involved in a specific procurement. The criteria for applicability of a Certificate or Registered Test Report should be appointed to the size and timing of the procurement.

2.2 Registration

2.2.1 Overview

As an alternative to NIST formally validating each and every environment, validation by registration allows the client to self-test additional environments. Validation by registration provides the client a low cost method for testing these additional environments and registering them in the VPL.

Validation by registration is only available for clients whose interpreters have been formally validated by NIST. The self-tested environment is validated against the formally validated implementation. Compliance is demonstrated only if the output from the self-tested environment is identical to the output from the formally validated implementation.

The process of validation by registration consists of a client conducting the conformance tests, evaluating the results, and sending the results to NIST for inspection. In order to perform the self-testing, the client must have a copy of the latest version of the appropriate test suite and test software. If the registration criteria are met, the additional environments are added to the Validated Products List as a registered environment.

All self-tested environments are subject to challenge by NIST and other interested parties (such as procuring Federal agencies). If the NIST inspection reveals that a self-tested environment does not behave in accordance with the submitted validation material, all entries in the VPL for self-tested environments dependent on the formally validated implementation are stricken.

2.2.2 Registration Criteria

1. The client's certificate of validation or registered report has not expired.

2. The client submits a request for registration.

3. The client conducts the conformance testing.

4. The client submits a signed statement along with all testing outputs, affirming that the results were obtained from the self-tested environment and are identical to the results obtained from the formally validated implementation.

2.3 Administrative Procedures

2.3.1 Submitting implementations for validation

The FIRMR refers to the Index that provides terminology for agencies to use when incorporating FIPS in Government procurement. This terminology requires that CGM implementations entering the Federal Government inventory be validated. This requirement may be satisfied by supplying a current Certificate of Validation or Registered Test Report for the CGM Implementation; or at the option of the procuring agency, temporarily satisfy this requirement by submitting the CGM implementation for validation.

The phrase "submission for validation" used in the Index means that a letter has been received by the test laboratory requesting that the CGM implementation be validated for the purposes of offering the implementation to the Government. When such a request is received, the test laboratory will send the requester a letter acknowledging receipt of the request and indicate the month in which validation testing is scheduled to take place. This letter may be offered to Government departments and agencies as proof of submission for validation.

2.3.2 Requests for Validation

A request for validation services shall be in the form of a letter to any of the approved test laboratories. NIST may be contacted at the following address to receive a list of the approved test laboratories:

2.3.3 Disputed and withdrawn tests

Questions regarding the interpretation of the standard and the validity of the tests should be forwarded to the testing laboratory at NIST, along with associated rationale and detailed documentation. The testing laboratory and client will attempt to resolve these issues informally. If no resolution is reached, the question is referred to the Control Board for a ruling.

All test results issued by the testing laboratory remain in force unless and until reversed by the Control Board. If the test is judged to be invalid, the offending test will be corrected or withdrawn, and the VSR altered to reflect the ruling.

All test results issued by the testing laboratory remain in force unless and until reversed by the Control Board. If the test is judged to be invalid, the offending test will be corrected or withdrawn, and the Test Report altered to reflect the ruling.

2.3.4 Pricing

The CGM Test Service validations are to be performed on a cost-reimbursable basis. Pricing information can be found in the following program section or by contacting the NIST CGM Test Service.

2.3.5 Cancellation

Once the validation process has begun, the client agrees to reimburse NIST/ITL for the expenses incurred in preparation for performance of the validation. In the event that NIST/ITL cancels the validation due to nonsupport by the client or failure of the client to perform in a reasonable manner, the client agrees to pay NIST/ITL for all validation expenses.

2.3.6 Release of validation information

Until a Registered Test Report is finalized and a Notification of Conformance form is received, NIST will treat all information concerning the validation to be confidential.

In general, NIST shall have the right to use all information gathered in the course of developing and administering a validation for any governmental purpose but shall not release such information publicly except: (1) When reporting on the results of testing, NIST may provide information, subject to the provisions of the subparagraphs below; and (2) as required pursuant to a request under the Freedom of Information Act (5 U.S.C. Section 552).

The client shall place a Proprietary notice on all information delivered to NIST that the client asserts is proprietary. Any information designated as proprietary that is furnished to NIST, shall be used by NIST only for the purpose of carrying out validations. Information designated as proprietary shall not be disclosed, copied, reproduced or otherwise made available in any form to any other person firm, corporation, partnership, association, or other entity without the consent of the client except as such information may be subject to disclosure under the Freedom of Information Act (5U.S.C.522). NIST will use its best efforts to protect information designated as proprietary from unauthorized disclosure.

2.3.7 Publication

Registered Test Reports completed by NIST shall be made available to the public. In no event, however, shall the name of the client or any of its trademarks and trade names be used in NIST publications without the client's prior written consent. With respect to publication in the VPL, the Notification of Conformance shall contain the client's written consent.

NIST and the client shall agree to confer and consult prior to the publication of data to assure that no proprietary data is released and that patent rights are not jeopardized. Prior to publishing a Registered Test Report, the client shall be offered an opportunity to review such proposed publication.

In general, NIST shall have the right to use all information gathered in the course of developing and administering a conformance testing program for any governmental purpose. Registered Reports completed by NIST shall be made available to the public upon request. 

1 MetaCheck is a licensed product of CGM Technology Software, P.O. Box 648, Gales Ferry, CT 06335. MetaCheck with the CALS Option is referred to as MetaCALS.

To Top of Page
To Section 3