CGM Conformance Testing Basics - Why and How

by Lynne Rosenthal and Mark Skall

CGM conformance testing benefits both vendors and users.

This document is reprinted from the Winter 1993 issue of the CALS Journal and was converted into HTML and includes editorial statements [ed - marked like this].

The primary purpose of the Computer Graphics Metafile (CGM) standard is to allow organizations to store and exchange graphical data (i.e., pictures) among different computers, graphical devices, and applications both within their organizations, as well as with other organizations. The CGM is widely used and available for a broad range of computer systems and applications. However, a CGM picture that is created by one CGM product may not be fully rendered by another CGM product. Successful graphics interchange can only be achieved if commercially available products faithfully implement 1) the CGM standard and 2) the formally recognized profiles.

CGM conformance testing is a way of determining if a CGM product correctly implements the CGM standard and its associated application profile. [ed - CGM is a large and complex standard. Vendors do not implement the entire standard. They implement functional subsets. These subsets are know as profiles.] The Conformance Testing Group at the National Institute of Standards and Technology, Computer Systems Laboratory (NIST/CSL) is developing CGM test suites and has established a validation test service for testing conformance of CGM.

Why test CGM?

Testing and certification is an essential step towards achieving interoperability. It ensures that a CGM product correctly implements the standard.

Conformance testing is a functional and fiscal advantage to the vendor of CGM products as well as to the user of CGM products. Testing conformance benefits users by providing an independent, objective method for evaluating CGM products. The availability of conforming products provides users with the freedom of choice to purchase a product suited to individual requirements, and not become locked-in to a single vendor. Users must be informed about the benefits of testing if any real progress in interoperability is to be made. The availablity of standards and conformance tests are meaningless unless users demand that vendors get tested and obtain a certificate. Vendors benefit by using the conformance tests to debug their implementations and by using their certificate of conformance as a potential selling point for their products.

Testing CGMs for standard conformance

Strictly speaking, CGM is solely a file format, that is, a graphics data stream. However, the term "CGM" is often used to include a generator (a program which produces the CGM) and an interpreter (a program which reads the CGM), as well as a metafile. Together, the generator, metafile, and interpreter form a total CGM system.

Conformance of CGM is defined in terms of conformance to a particular profile of CGM. Thus, the CGM standard in conjunction with an application profile is necessary in order to test conformance of a total CGM system.

Testing CGM for conformance entails: 1. verifying that the metafile is syntactically correct, 2. verifying that a generator produces conforming metafiles which accurately and correctly represent the intended picture, and 3. verifying that an interpreter can correctly and completely read the metafile and produce the intended picture.

NIST CGM Test Services

The Conformance Testing Group has developed test suites to test implementations for conformance to either the CALS profile or the ATA profile. Additionally, a CGM test service was established to test implementations of CGM. The test service is divided into three testing programs: metafile testing, generator testing, and interpreter testing. For metafiles, the testing focuses on testing an instance of a CGM, that is, a CGM data stream for compliance to Version 1 CGM, binary encoding as specified in either the CALS profile or the ATA profile. For generators, the testing focuses on testing the ability of the CGM generator and its operating environment to produce graphically correct metafiles which conform to the profile. For interpreters, the testing focuses on testing the interpreter and its operating environment to render the correct picture, according to the requirements of the profile. Implementors who successfully undergo the validation process receive certificates of validation or registered reports, and the results are published in the Validated Products List (VPL), which is available from NIST. Generally, an implementor must receive a certificate of validation or a registered report in order for that product to be procured by Federal agencies.

How is a vendor product tested?

The validation processes for metafiles, generators and interpreters are distinct processes. For metafiles, the client (i.e., user or vendor) submits the metafiles to the Graphics Software Group for testing. Testing takes place at NIST/CSL. A certificate of validation is issued for all compliant metafiles. Generator and interpreter validation, on the other hand, is a multi-step process conducted at both the vendor site and at NIST/CSL. The entire validation process may take several weeks to complete. A certificate of validation is issued if the generator or interpreter under test passes all tests with no errors. If errors are detected during the validation, a registered report is issued in place of a certificate.

The Test Service Procedures provide specific information about how a metafile, generator and/or an interpreter is tested.

Which test service should you choose?

The interpreter test suite may be used by either vendors or users. The test files can be used to debug implementations as well as for users to evaluate different products. The test suite consists of a collection of CGM files, reference pictures, and evaluation criteria. Using the test suite, a vendor or user can process these CGMs and compare the resulting picture to the expected or reference picture. If the pictures are the same, the implementation passes that specific test. Users can apply the test files to different CGM implementations (e.g., graphics software, wordprocessing package) and evaluate for themselves whether the implementation can correctly interprete the CGM. Vendors of CGM implementations may also use the files to check their implementations. The test suite provides both the vendor and user a means of assessing whether the implementation adheres to the standard, and most importantly, provides a level of confidence in the products themselves. While both vendors and users may use the test suite, typically vendors of interpreter products would request a formal validation as provided by the interpreter test service. This is because, the CGM implementation may need to be changed (e.g., correct the software). Users who intend on rendering CGM graphics should insist on using validated CGM implementations. Validated implementations provide the user a level of confidence as to the quality of the product and its ability to produce correct pictures.

Although both the metafile and generater test services require the client to submit CGMs to NIST/CSL for analysis, the objective and applicability of each test service are different.

The metafile test service focuses on testing individual metafiles. The objective is to analyze each metafile and verify that it is syntactically correct. The picture represented by the metafile, as well as the generator product, the hardware and the operating system are all irrelevant to metafile testing. Moreover, the conformance of a metafile does not necessarily imply conformance of other CGMs created by the same CGM generator. The metafile test service is applicable when specific pictures or illustrations represented as a CGM are being acquired (e.g., a set of illustrations which will be included in a technical manual, or a library of clip art pictures).

The generator test service focuses on testing a generator product to determine if it produces conforming metafiles. While metafile testing concentrates on syntax checking, generator testing checks the semantics of the CGMs generated to determine if the generator is producing CGMs that correctly describe the picture. The metafile, the picture represented by the metafile, as well as the hardware, operating system, and all supporting software are an integral part of the testing and, consequently, the conformance. An error in any one of these components may cause the generator under test to fail the conformance tests.

Typically, vendors of generator products, rather than users, would request validation. This is because an in-depth knowledge about the internal workings of the generator is required in order to complete the validation. However, users who intend to create CGMs from graphics or other application programs, should insist upon using programs with a validated generator. With a validated generator, the user may assume that 1) only conforming metafiles are produced, 2) the intended picture is accurately and correctly defined, and 3) successful picture exchange can be achieved. 

[Software Diagnostic and Conformance Testing Division] [Conformance Testing Group] 

Conformance Testing Group
Comments and suggestions:
Last modified: March 20, 1998