**3.2 Analysis and findings**

The data analysis progresses through several stages: verification and quality control, exploratory analysis to uncover interesting relationships, and statistical modeling to validate findings and formally test hypotheses of interest. The initial analysis examined the data for anomalies or outliers. None were found in this case.

Next, we calculated an overall interpretability rating from each analyst for each clip. The method for calculating these ratings was as follows: Each of the three criteria used to rate each clip was calibrated (on a 0-100 scale) in terms of interpretability, where this calibration was derived from an earlier evaluation (Irvine *et al.* 2007c). Multiplying the interpretability level by the IA's confidence rating produces a score for each criterion. The final interpretability score (Equation 1) was the maximum of the three scores for a given clip.

$$\text{Interprestability Score(j, k)} = \max \left\{ \mathbf{C}\_{i,j,k} \, \mathbf{I}\_{i,k} \colon i = 1, 2, 3 \right\} / \ 100 \tag{1}$$

Where Ci,j,k is the confidence rating by the jth IA on the kth clip for the ith criterion and Ii,k is the calibrated interpretability level for that criterion. All subsequent analysis presented below is based on this final interpretability score. The remaining analysis is divided into two sections: interframe compression and intraframe compression. Ultimately, we compared the

effect.

Quantifying Interpretability Loss due to Image Compression 43

varies by clip, the pattern is clear for all clips. Statistical analysis shows that bitrate, modeled as a covariate, is the primary factor affecting interpretability. For interframe compression, the differences between H.264 and MPEG-2 are small but statistically significant (Table 3). The pattern holds across all scenes, as indicated by the lack of a codec-by-scene interaction

> Intercept 1 6.2 0.0033 BitRate 1 59.7 0.00003 Codec 2 5.7 0.015 Scene 4 15.4 0.00028 Codec \* Scene 8 0.5 0.82

In the case of intraframe compression, all three codecs yielded products for the evaluation, although MPEG-2 would not support extreme compression rates. The findings in this case are slightly different than interframe compression. Bitrate remained the dominant factor, but more pronounced differences among the codecs emerged (Figure 4). At modest compression rates, MPEG-2 exhibited a substantially loss in interpretability compared to either H.264

**0 1000 2000 3000 4000 5000 6000 7000**

**Bit Rate**

**Freedom F-statistic Significance**

**<0.025** 

**JPEG2000 MPEG2 H.264**

**Source Deg. Of**

Table 3. Analysis of Covariance for Interframe Comparisons

Fig. 5. Summary Comparison Across Codec and Bitrate

**0**

**10**

**20**

**30**

**40**

**50**

**Interpretability Score**

**60**

**70**

**80**

**90**

**100**

**3.4 Intraframe compression** 

analyst derived utility measures, as a NIIRS surrogate, to the automated computational values.

All three codecs yielded products for the evaluation. However, MPEG-2 would not support extreme compression rates. Bitrate was the dominant factor, but pronounced differences among the codecs emerged too (Fig. 6 and Fig. 7). At modest compression rates, MPEG-2 exhibited a substantial loss in interpretability compared to either H.264 or JPEG-2000. Only JPEG-2000 supported more extreme intraframe compression. A computational model was developed to characterize the significance among codec, scene, and bitrate's effect on data quality. There were systematic differences across the clips, as expected, but the effects of the codecs and bitrates were consistent. When modeled as a covariate, the effects of bitrate dominate. The effect due to codec is modest, but still significant. As expected, there is a significant main effect due to scene, but no scene-by-codec interaction.

#### **3.3 Interframe compression**

Analysis of the interframe ratings shows a loss in image interpretability for both MPEG-2 and H.264 as a function of bitrate (Fig. 4). The initial compression from the native rate to 3 MB per second corresponds to a modest loss in interpretability. This finding is consistent with previous work. At extreme compression levels (below 1 MB per second), the interpretability loss is substantial. H.264 generally supported more extreme compression levels, but the interpretability degrades accordingly. Although the exact compression level

Fig. 4. Summary Comparison Across Codec and Bitrate

varies by clip, the pattern is clear for all clips. Statistical analysis shows that bitrate, modeled as a covariate, is the primary factor affecting interpretability. For interframe compression, the differences between H.264 and MPEG-2 are small but statistically significant (Table 3). The pattern holds across all scenes, as indicated by the lack of a codec-by-scene interaction effect.


Table 3. Analysis of Covariance for Interframe Comparisons

#### **3.4 Intraframe compression**

42 Video Compression

analyst derived utility measures, as a NIIRS surrogate, to the automated computational

All three codecs yielded products for the evaluation. However, MPEG-2 would not support extreme compression rates. Bitrate was the dominant factor, but pronounced differences among the codecs emerged too (Fig. 6 and Fig. 7). At modest compression rates, MPEG-2 exhibited a substantial loss in interpretability compared to either H.264 or JPEG-2000. Only JPEG-2000 supported more extreme intraframe compression. A computational model was developed to characterize the significance among codec, scene, and bitrate's effect on data quality. There were systematic differences across the clips, as expected, but the effects of the codecs and bitrates were consistent. When modeled as a covariate, the effects of bitrate dominate. The effect due to codec is modest, but still significant. As expected, there is a

Analysis of the interframe ratings shows a loss in image interpretability for both MPEG-2 and H.264 as a function of bitrate (Fig. 4). The initial compression from the native rate to 3 MB per second corresponds to a modest loss in interpretability. This finding is consistent with previous work. At extreme compression levels (below 1 MB per second), the interpretability loss is substantial. H.264 generally supported more extreme compression levels, but the interpretability degrades accordingly. Although the exact compression level

**0 1000 2000 3000 4000 5000 6000 7000**

**MPEG2**

**H.264**

**Bit Rate**

significant main effect due to scene, but no scene-by-codec interaction.

Fig. 4. Summary Comparison Across Codec and Bitrate

**0**

**10**

**20**

**30**

**40**

**50**

**Interpretability Score**

**60**

**70**

**80**

**90**

**100**

values.

**3.3 Interframe compression** 

In the case of intraframe compression, all three codecs yielded products for the evaluation, although MPEG-2 would not support extreme compression rates. The findings in this case are slightly different than interframe compression. Bitrate remained the dominant factor, but more pronounced differences among the codecs emerged (Figure 4). At modest compression rates, MPEG-2 exhibited a substantially loss in interpretability compared to either H.264

Fig. 5. Summary Comparison Across Codec and Bitrate

Quantifying Interpretability Loss due to Image Compression 45

(a) (b)

(c) (d)

(e) (f)

Fig. 6. (a) Original, (b) Compressed version, (c,d) original edge images, (e) edge images displayed together where Red, Blue, Magenta are from the original, the compressed, both edge images respectively (f) edge intensities, and (g) is the SSIM image darker areas

*Imagery extracted from the VIVID Public Release Data set provided by Air Force Research Laboratory* 

(g)

represent more noticeable differences.

or JPEG-2000. Only JPEG-2000 supported more extreme intraframe compression and highly compressed renditions were produced from all of the parent clips. As with the interframe comparisons, there were systematic differences across the clips, as expected, but the effects of the codecs and bitrates were consistent. The analysis of covariance confirms these statistical effects (Table 4). When modeled as a covariate, effects of bitrate dominate. The effect due to codec is modest, but still significant. As expected, there is a significant main effect due to scene, but no scene-by-codec interaction.


Table 4. Analysis of Variance for Interframe Comparisons
