Mark Levenson, Stefan Leigh
John Libert, Charles Fenimore, Peter Roitman
Statisticians are assisting the Electricity Division of EEEL in the analysis of a multi-laboratory, multi-national examination of objective methods for the measurement of video quality through the Video Quality Experts Group (VQEG), a joint working party of the International Telecommunications Union (ITU). Traditionally, subjective measurement studies are used in the television industry for evaluation of video systems, however such methods involve the assembly of groups of viewers to evaluate carefully selected video sequences displayed on special equipment and under controlled conditions. Naturally, such tests are costly and impractical for the frequent, or even continuous ``in-service'' measurement of video quality. Accordingly, the VQEG has undertaken a study of proposed computational models that mimic subjective tests. The study consisted of assembling appropriate source material, rating that material using the ten proposed objective models, and analyzing the correlation of the model scores to previously determined subjective scores. NIST, as an impartial participant, was responsible for the study logistics and data analysis. The results from the study showed that many of the models were not statistically distinguishable from each other, a conclusion that surprised many.
The accompanying figure displays frames from a high and low compression of a video clip. In the high compression frame, artifacts associated with the compression algorithm are apparent. VQEG seeks objective measures of such video degradations.
Figure 12: Frames from a high and low compression of a video clip.
Date created: 7/20/2001