Image Compression Example

We shall represent a monochrome (luminance) image by a matrix x whose elements are x(n), where n=(

n1

n2

) is the integer vector of row and column indexes. The energy of x is defined as

Energy of x=∑(x²(n))(1)

where the sum is performed over all n in x.

Figure 1: The basic block diagram of an image coding system.

Figure 1 shows the main blocks in any image coding system. The decoder is the inverse of the encoder. The three encoder blocks perform the following tasks:

• Energy compression - This is usually a transformation or filtering process which aims to concentrate a high proportion of the energy of the image x into as few samples (coefficients) of yas possible while preserving the total energy of x in y. This minimises the number of non-zero samples of y which need to be transmitted for a given level of distortion in the reconstructed image x.
• Quantisation - This represents the samples of y to a given level of accuracy in the integer matrix q. The quantiser step size controls the tradeoff between distortion and bit rate and may be adapted to take account of human visual sensitivities. The inverse quantiser reconstructs y, the best estimate of y from q.
• Entropy coding - This encodes the integers in q into a serial bit stream d, using variable-length entropy codes which attempt to minimise the total number of bits in d, based on the statistics (PDFs) of various classes of samples in q.

The energy compression / reconstruction and the entropy coding / decoding processes are normally all lossless. Only the quantiser introduces loss and distortion: y is a distorted version of y, and hence x is a distorted version of x. In the absence of quantisation, if y=y, then x=x.

Use of Laplacian PDFs in Image Compression

It is found to be appropriate and convenient to model the distribution of many types of transformed image coefficients by Laplacian distributions. It is appropriate because much real data is approximately modeled by the Laplacian probability density function (PDF), and it is convenient because the mathematical form of the Laplacian PDF is simple enough to allow some useful analytical results to be derived.

A Laplacian PDF is a back-to-back pair of exponential decays and is given by:

p(x)=

1

2x0

 ⅇ−(

|x|

x0

 )(1)

where x₀ is the equivalent of a time constant which defines the width of the PDF from the centre to the 

1

ⅇ

 points. The initial scaling factor ensures that the area under p(x) is unity, so that it is a valid PDF. Figure 1 shows the shape of p(x).

Figure 1: Laplacian PDF, p(x), and typical quantiser decision thresholds, shown for the case when the quantiser step size Q=2x₀

The mean of this PDF is zero and the variance is given by:

v(x0)	=	∫−∞∞x2p(x)dx
	=	2∫ x2 2x0  ⅇ−( x x0  )dx
	=	2x02

(2)

(using integration by parts twice).

Hence the standard deviation is:

σ(x0)	=	\v(x0)
	=	\2x0

(3)

Given the variance (power) of a subimage of transformed pels, we may calculate x₀ and hence determine the PDF of the subimage, assuming a Laplacian shape. We now show that, if we quantise the subimage using a uniform quantiser with step size Q, we can calculate the entropy of the quantised samples and thus estimate the bit rate needed to encode the subimage in bits/pel. This is a powerful analytical tool as it shows how the compressed bit rate relates directly to the energy of a subimage. The vertical dashed lines in Figure 1 show the decision thresholds for a typical quantiser for the case when Q=2x₀.

First we analyse the probability of a pel being quantised to each step of the quantiser. This is given by the area under p(x) between each adjacent pair of quantiser thresholds.

• Probability of being at step 0, p₀=Pr[−(

  1

  2

   Q)<x<

  1

  2

   Q]=2Pr[0<x<

  1

  2

   Q]
• Probability of being at step k, p_k=Pr[(k−

  1

  2

   )Q<x<(k+

  1

  2

   )Q]

First, for x₂≥x₁≥0, we calculate: Pr[x₁<x<x₂]=∫_x1^x₂p(x)dx=(−(

1

2

 ))ⅇ−(

x

x0

 )|_x1^x₂=

1

2

 (ⅇ−(

x1

x0

 )−ⅇ−(

x2

x0

 )) Therefore,

p₀=1−ⅇ−(

Q

2x0

 )(4)

and, for k≥1,

pk	=	1 2  (ⅇ−( (k− 1 2 )Q x0  )−ⅇ−( (k+ 1 2 )Q x0  ))
	=	sinh( Q 2x0  )ⅇ−( kQ x0  )

(5)

By symmetry, if k is nonzero, p_−k=p_k=sinh(

Q

2x0

 )ⅇ−(

|k|Q

x0

 )

Now we can calculate the entropy of the subimage:

H	=	−( ∞ ∑ k=−∞  (pklog2pk))
	=	−(p0log2p0)−2 ∞ ∑ k=1  (pklog2pk)

(6)

To make the evaluation of the summation easier when we substitute for p_k, we let p_k=αr^k where α=sinh(

Q

2x0

 ) and r=ⅇ−(

Q

x0

 ). Therefore,

∞ ∑ k=1  (pklog2pk)	=	∞ ∑ k=1  (αrklog2(αrk))
	=	∞ ∑ k=1  (αrk(log2α+klog2r))
	=	αlog2α ∞ ∑ k=1  (rk)+αlog2r ∞ ∑ k=1  (krk)

(7)

Now 

∞

∑

k=1

 (r^k)=

r

1−r

 and, differentiating by r: 

∞

∑

k=1

 (kr^k−1)=

1

(1−r)2

 . Therefore,

∞ ∑ k=1  (pklog2pk)	=	αlog2α r 1−r  +αlog2r r (1−r)2
	=	αr 1−r  (log2α+ log2r 1−r  )

(8)

and

p₀log₂p₀=(1−\r)log₂(1−\r)(9)

Hence the entropy is given by:

H=−((1−\r)log₂(1−\r))−

2αr

1−r

 (log₂α+

log2r

1−r

 )(10)

Because both α and r are functions of 

Q

x0

 , then H is a function of just 

Q

x0

 too. We expect that, for constant Q, as the energy of the subimage increases, the entropy will also increase approximately logarithmically, so we plot H against 

x0

Q

 in dB in Figure 2. This shows that our expectations are born out.

Figure 2: Entropy H and approximate entropy H_a of a quantised subimage with Laplacian PDF, as a function of 

x0

Q

 in dB.

We can show this in theory by considering the case when (

x0

Q

 ,≫,1), when we find that: α≈

Q

2x0

 r≈1−

Q

x0

 ≈1−2α \r≈1−α Using the approximation log₂(1−ε)≈−(

ε

ln2

 ) for small ε, it is then fairly straightforward to show that H≈−(log₂α)+

1

ln2

 ≈log₂(

2ⅇx0

Q

 ) We denote this approximation as H_a in Figure 2, which shows how close to H the approximation is, for x₀>Q (i.e. for 

x0

Q

 >0 dB).

We can compare the entropies calculated using Equation 10 with those that were calculated from the bandpass subimage histograms, as given in these figures describing Haar transform energies and entropies; level 1 energies, level 2 energies, level 3 energies, and level 4 energies. (The Lo-Lo subimages have PDFs which are more uniform and do not fit the Laplacian model well.) The values of x₀ are calculated from: x₀=

std.  dev.

\2

 =\

subimage  energy

2  (no  of  pels  in  subimage)

 The following table shows this comparison:

TABLE 1

Transform level	Subimage type	Energy (× 106)	No of pels	x0	Laplacian entropy	Measured entropy
1	Hi-Lo	4.56	16384	11.80	2.16	1.71
1	Lo-Hi	1.89	16384	7.59	1.58	1.15
1	Hi-Hi	0.82	16384	5.09	1.08	0.80
2	Hi-Lo	7.64	4096	30.54	3.48	3.00
2	Lo-Hi	2.95	4096	18.98	2.81	2.22
2	Hi-Hi	1.42	4096	13.17	2.31	1.75
3	Hi-Lo	13.17	1024	80.19	4.86	4.52
3	Lo-Hi	3.90	1024	43.64	3.99	3.55
3	Hi-Hi	2.49	1024	34.87	3.67	3.05
4	Hi-Lo	15.49	256	173.9	5.98	5.65
4	Lo-Hi	6.46	256	112.3	5.35	4.75
4	Hi-Hi	3.29	256	80.2	4.86	4.38

We see that the entropies calculated from the energy via the Laplacian PDF method (second column from the right) are approximately 0.5 bit/pel greater than the entropies measured from the Lenna subimage histograms. This is due to the heavier tails of the actual PDFs compared with the Laplacian exponentially decreasing tails. More accurate entropies can be obtained if x₀ is obtained from the mean absolute values of the pels in each subimage. For a Laplacian PDF we can show that

Mean  absolute  value	=	∫−∞∞|x|p(x)dx
	=	2∫0∞ x 2x0  ⅇ−( x x0  )dx
	=	x0

(11)

This gives values of x₀ that are about 20% lower than those calculated from the energies and the calculated entropies are then within approximately 0.2 bit/pel of the measured entropies.