Adaptive image compression using local pattern information

Feng Pan

School of EEE, Centre for Signal Processing, Nanyang Technological University, 639798 Singapore, Singapore

Received 2 August 2001; received in revised form 9 February 2002

Abstract

This paper describes a new adaptive coding technique to the coding of transform coefficients used in block based

image compression schemes. The presence and orientation of the edge information in a sub-block are used to select

different quantization tables and zigzag scan paths to cater for the local image pattern. Measures of the edge presence and

edge orientation in a sub-block are calculated out of their DCT coefficients, and each sub-block can be classified into four

different edge patterns. Experimental results show that compared to JPEG and the improved HVS-based coding, the new

scheme has significantly increased the compression ratio without sacrificing the reconstructed image quality.

� 2002 Elsevier Science B.V. All rights reserved.

Keywords: Image compression; JPEG; Block-based coding; Adaptive quantization

1. Introduction

Image/video data compression plays an impor-tant role in many emerging applications in thefield of visual communication and broadcasting.There are currently two major compression tech-niques: the block based discrete cosine transform(DCT) coding (JPEG, MPEG, and H.26x) (Raoand Hwang, 1996; Shi and Sun, 1999), and waveletbased coding (JPEG-2000) (Skodras et al., 2001).Although wavelet based coding has been the focusof research recently, JPEG-2000 does not intend toreplace the existing JPEG standard, but to providea set of new features, such as lossless coding, goodlow bit-rate performance, large image size etc. thatare important to many high end and emergingapplications. It also addresses areas where current

standards fail to produce the best performance.However, its additional complexity, and its in-compatibility with the existing techniques (most ofthem are DCT based) are perceived as a disad-vantage for its deployment in many applications.Block-based DCT coding, on the other hand, is themost popular one among available image/videocompression approaches. Actually it is used inmost of the current standards. It is strongly be-lieved that DCT based coding will retain its dom-inant role in many applications, especially in lossycoding of video sequences such as MPEG-4 videocoding. Therefore, continuous improvements toDCT based coding are very important and havebeen the focus of many researchers (Ratnakar andLivny, 2000; Kondo and Oishi, 2000; Grosse et al.,2000).

In block-based DCT coding, DCT coefficients arecalculated over small non-overlapping sub-blocks

Pattern Recognition Letters 23 (2002) 1837–1845

www.elsevier.com/locate/patrec

E-mail address: edwardpan@ieee.org (F. Pan).

0167-8655/02/$ - see front matter � 2002 Elsevier Science B.V. All rights reserved.

PII: S0167-8655 (02 )00156-3

(usually 8� 8 in size). This block-by-block pro-cessing takes advantage of the local spatial cor-relation properties of the image and it furthersimplifies the VLSI implementation of the algo-rithm. The DCT process produces many 2D blocksof transform coefficients that are quantized to se-lectively eliminate some of the trivial coefficientsthat are likely masked by edges. The quantizedcoefficients are then zigzag scanned to output thedata in an efficient way. The final step in thecoding process uses variable length coding (Huff-man coding or arithmetic coding) to further reducethe entropy. It is well known that the choice ofquantization tables significantly affects the com-pression ratio and reconstructed image quality.Many adaptive quantization methods have beendiscussed to cater for the given bit rate and picturepatterns (Wang et al., 2001; Puri and Aravind,1991; Rosenholtz and Watson, 1996), though theydid not use the block edge pattern to adjust thequantization table; an adaptive zigzag scanningalgorithm has also been discussed (Grosse et al.,2000). It uses a neural network to determine thezigzag-reordering scan, and the adaptation de-pended only on the resultant non-zero coefficientspattern, not on edge pattern of the sub-block.

In this paper, we describe a new adaptive imagecompression scheme guided by pattern classifica-tion of the sub-blocks. Measures of the edge pres-ence and edge orientation in a sub-block arecalculated out of their DCT coefficients, and eachsub-block can be classified into four different blockpatterns, namely, horizontal edge pattern, verti-cal edge pattern, diagonal edge pattern and non-edge pattern. Four sets of quantization tables andzigzag scan paths are designed to best fit to the fourdifferent sub-block patterns. For example, in ahorizontal edge pattern, compared to the standardJPEG table, finer quantization is used for the lefthalf of the coefficients than that of the right half;also, the zigzag scan path is adjusted such that ittraverses the left half of the coefficients with higherpriority. On the other hand, a penalty of 2 bits perblock will have to be included in the header of thebitstream in order for the decoder to choose theproper quantization table and zigzag scan path.These 2 bits can be further compressed by usingvariable length coding. Experimental results show

that compared to JPEG and the improved HVS-based coding, the new scheme has significantly re-duced the resultant data bits without sacrificing thereconstructed image quality in terms of subjectviewpoint and objective measurement of PSNR.The improvements are more obvious to the imageswith high complexity, and when coded at a higheramount of bits per pixel.

This paper is organized as follows. In Section 2,the classification of the sub-blocks is explainedin detail. In Section 3, the new set of quantizationtables and zigzag scan paths are introduced withdetailed explanation of the rationales behind thesenew approaches. Section 4 reports experimentalresults of a number of standard test images toelaborate the effectiveness of the adaptive com-pression algorithm. Conclusions will be presentedin Section 5.

2. Block pattern classification

The edge pattern in a sub-block can be fullycaptured from its DCT coefficients in frequencydomain. Fig. 1 shows the structural decompositionof the DCT coefficients (Rao and Hwang, 1996). Itis well known that the top left coefficients represent

Fig. 1. Structural decomposition of DCT coefficients.

1838 F. Pan / Pattern Recognition Letters 23 (2002) 1837–1845

the mean intensity of the (8� 8) spatial block. Theremaining basis coefficients on the first few rows(columns) represent the vertical (horizontal) edgesetc. Therefore the energy distribution in the DCTdomain determines the edge patterns in the spatialblock (Nam and Rao, 1991).

2.1. Measures of different edge patterns

Based on the above observation, three energyintensity measures of different edge patterns, HE,VE, and DE are calculated from the DCT coeffi-cients of the sub-block. These measures are used toindicate whether this sub-block has strong edgepresence, and if so, which edge pattern this sub-block has.

HE ¼X7;7

u;v¼0;uþv 6¼0ðv

"þ 1Þ2 � F 2ðu; vÞ

#1=2

ð1Þ

VE ¼X7;7

u;v¼0;uþv 6¼0ðu

"þ 1Þ2 � F 2ðu; vÞ

#1=2ð2Þ

DE ¼X7;7

u;v¼0;uþv 6¼0ðu

"þ 1Þ � ðvþ 1Þ � F 2ðu; vÞ

#1=2

ð3Þwhere F ðu; vÞ is the 2D DCT coefficients of the8� 8 sub-block, and HE, VE, DE are the energyintensity measures of horizontal, vertical and di-agonal edges in this sub-block respectively. Afourth measure as in Eq. (4) has been used to de-cide whether the sub-block contains mainly hori-zontal, vertical or diagonal edge,

HV ¼ arctanVE

HE

� �ð4Þ

HV is an angle measure ranging from 0� to 90�.From its expression we know that 0� means ahorizontal edge, 45� means a diagonal edge and90� a vertical edge.

2.2. Classifications of edge patterns

We have applied the measures discussed inSection 2.1 to many standard test images, including

Baboon, Barbara, Boats, Bridge, Cameraman,Couple, Lena, Peppers, Plane and Columbia. Basedon the experiments, the following criterions areproven to be the most effective for the classificationof edge patterns in a sub-block in terms of com-pression ratio and quality measure.

Fig. 2 shows some of the typical values of theabove four different edge patterns in the Lenaimage. Their classifications are obvious. It can benoted that HV is a very effective measure ofpresence of different edge patterns, while DE canbe used to tell the difference between a diagonaledge pattern and a non-edge pattern. It is notedthat the threshold values stated above are not toosensitive to the image characteristics, but thesevalues are the best with the test images. Amongall the 10 test images in our experiment, we havefound that, based on the above criteria, the verticaland horizontal sub-blocks count between 10% and30%, while the majority is diagonal sub-block(between 50% and 85%). The image Peppers hasthe least number of vertical and horizontal sub-blocks, and the image Couple has the most, onthe other hand. Based on these statistics, variable

Horizontal edge block: HV6 32:5�;Vertical edge block: HVP 57:5�;Diagonal edge block: 32:5� < HV < 57:5�,

DE > 200;Non-edge block: 32:5� < HV < 57:5�,

DE6 200.

Fig. 2. Different edge patterns and their HV, DE values.

F. Pan / Pattern Recognition Letters 23 (2002) 1837–1845 1839

length coding can be used to code the 2 penaltybits that are used in decoder to choose the sub-block pattern. The result of VLC coding this 2 bitsproduces between 1.15 and 1.65 bits per block.

3. Adaptive quantization and zigzag reordering

3.1. Adaptive quantization in MPEG

In general, the process of adaptive quantization(e.g. MPEG) can be described in the followingformula (5),

Fqðu; vÞ ¼ int8� F ðu; vÞqp � Qðu; vÞ

�þ k2

�ð5Þ

where Fqðu; vÞ is the matrix of the quantized DCTcoefficients, Qðu; vÞ is a matrix of quantizationweights, and qp is the quantization scaling factorwhich varies between 1 and 31 on a macro-blockto macro-block basis. It is this variability thatpermits adaptive quantization, though the changeof qp does not depend on the edge pattern of thesub-block. This paper describes an algorithm thatimproves the ways of changing qp by taking intoaccount of the edge masking effect of human visionsystem. It optimises the visual quality of the imageaccording to the variations of the edge patterns.

Furthermore, many efforts have been made tofind a better quantization table besides the stan-dard JPEG or MPEG quantization tables (Wanget al., 2001; Puri and Aravind, 1991; Rosenholtzand Watson, 1996). It is noted that the HVS-basedquantization table presented in (Wang et al., 2001)(as shown in Fig. 3) has significant improvementsin PSNR of reconstructed images. Thus this HVS-based quantization table is adopted in this exper-iment, with simple variations to cater for differentedge patterns of the sub-block.

3.2. Edge pattern adaptive quantization

One of the important characteristics of humanvision system is called edge masking, which saysthat the human vision system is less sensitive to theerrors (variations) along a prominent edge in an

image. In another word, a dominant edge patternwill obscure the perception of other lower contrastvariations in this sub-block.

We take Block b in Fig. 2 as an example. It isobviously a vertical edge sub-block, with a HVvalue of 66.4�. The energy of its DCT coefficients isconcentrated on the top region of the coefficientsblock. Based on this finding, the coefficients in thetop region contain the most important informa-tion regarding this block. It is natural that wequantize this group of coefficients with a finer stepthan these of the bottom region.

Therefore the new quantization tables for dif-ferent edge patterns are derived from Fig. 3 basedon the importance of the DCT coefficients in theregion. They are basically a linear weighted trans-form of the table in Fig. 3, and are again decidedexperimentally based on results of the 10 test im-ages. These weighting factors are to keep thecompression ratio of the reconstructed horizontaland vertical sub-block the same as that of usingthe non-weighted table, though with much higherPSNR values.

QFðu; vÞ ¼ QHVSðu; vÞ � 0:7

QHðu; vÞ ¼ QHVSðu; vÞ � ½0:09� ðv� 7Þ þ 1:3QVðu; vÞ ¼ QHVSðu; vÞ � ½0:09� ðu� 7Þ þ 1:3QDðu; vÞ ¼ QHVSðu; vÞ � ½0:09� ðju� vj � 7Þ þ 1:3

ð6Þwhere QHVSðu; vÞ is the HVS-based quantizationtable as shown in Fig. 3, and QFðu; vÞ, QHðu; vÞ,QVðu; vÞ, QDðu; vÞ are the modified quantization

Fig. 3. HVS-based quantization table.

1840 F. Pan / Pattern Recognition Letters 23 (2002) 1837–1845

tables for flat, horizontal, vertical and diagonaledge patterns respectively. The four new adaptivequantization tables are shown in Fig. 4.

It is noted that from these new tables, thequantization levels have been adjusted to highlightthe edge information in three directions (hori-zontal, vertical and diagonal), and to suppress thenoises elsewhere. These tables comply with thestructural decomposition of DCT coefficients asshown in Fig. 1.

3.3. Edge pattern adaptive zigzag scan

Edge masking can also be used to adaptivelyadjust the zigzag scan paths in a similar way. Withthe presence of prominent edge pattern in a sub-block, the energy concentration of DCT coeffi-cients is not uniform. The energy concentrated intop horizontal region indicates a vertical edge inthe sub-block; the energy concentrated in left ver-tical region indicates a horizontal edge in the sub-block; and the energy concentrated in diagonal

region indicates a diagonal edge in the sub-block;It is our interests to traverse through the energyconcentrated region first so as to produce muchshort non-zero coefficients sequence without caus-ing information loss.

For the diagonal edge pattern, and the non-edge block, we stick to the standard zigzag scanpath as the energy of their DCT coefficients con-centrate in the top left region; for horizontal andvertical edge pattern, we have tested the MPEG-2alternate zigzag san path, which is designed to beoptimal in interlaced picture structure. We havefine-tuned it to best suite for the horizontal andvertical edge pattern. These two new adaptivezigzag scan paths are as shown in Fig. 5(a) and(b). Note that by introducing the adaptive zigzagscan paths, the length of the runs of non-zerocoefficients, as well as the number of differentlength of zero runs are reduced significantly. It isalso noted that these paths are not too sensitiveto the image characteristics, but are the best withthe test images.

Fig. 4. Edge pattern adaptive quantization tables.

F. Pan / Pattern Recognition Letters 23 (2002) 1837–1845 1841

3.4. The penalty bits

In order for the decoder to choose the properquantization table and zigzag scan path, a penaltyof 2 bits per sub-block will have to be includedin the header of the bitstream. Based on blockpattern statistics, variable length coding is usedto code the 2 penalty bits, which can be reducedto between 1.15 and 1.65 bits. It can be shown inSection 4 that these extra bits are worthwhile com-paring the benefits they have achieved.

4. Experimental results

We have implemented the algorithms basedon portable C code for JPEG compression thatis available at ftp.simtel.net:/pub/simtelnet/msdos/graphics/jpegsr6b.zip from the Independent JPEGGroup. The first part of this section presents thenumerical results of applying the new adaptivecoding algorithm to four different edge patterns.The second part will then present the practicalcoding results of this new technique by applying itto all the selected test images. The third part lists

the average results applying the new algorithms toall the 10 test images used in this paper. In order toshow the robustness of this algorithm, additionalresults with standard images other than the 10 testimages are presented in Section 4.4.

4.1. Numerical results of different edge patterns

Here we use the four blocks highlighted in Fig.2 as the examples to elaborate the detailed codingprocess of the new adaptive coding algorithm de-scribed in this paper. Firstly we take Block b as theexample. The DCT coefficients of Block b are il-lustrated in Fig. 6(a), and 6(b) shows the quantizedcoefficients using the table in Fig. 3. When thecoefficients are scanned with the adaptive zigzagscan path it results in the following 1D sequence(length 34),

103; 57;�8; 0; 1;�17; 6; 6; 0; 0; 1;�1; 0; 0; 3; 0; 0; 0;� 1;�1; 0; 0; 2;�1; 1; 0; 0; 1;�1; 0; 0; 0; 0; 1;EOB

Applying Huffman coding (ISO/IEC JTC1 10918-1, 1994) to the above 1D sequence will generate110 bits, thus compression ratio for this block is

Fig. 6. DCT coefficients of Block b in Fig. 2 (a) and their

adaptively quantized results (b).

Fig. 5. Adaptive zigzag scan path for (a) horizontal and

(b) vertical edge pattern.

1842 F. Pan / Pattern Recognition Letters 23 (2002) 1837–1845

4.57, including the 2 penalty bits for this block.The PSNR of the reconstructed Block b is ap-proximately 30.99 dB.

Coding Block b using standard JPEG quanti-zation table and zigzag scan path, for the samenumber of bits (by adjusting a multiplier to thequantization table, for the comparison purpose),we would have a PSNR of 27.28 dB; while for thesame PSNR value, it needs 156 bits. Coding Blockb using HVS-based quantization table (Wang et al.,2001), for the same number of bits, it would have aPSNR of 27.79 dB; while for the same PSNRvalue, it needs 127 bits.

Tables 1 and 2 illustrate the comparisons be-tween the new algorithm (‘Adaptive’ is used todenote this method in the table), the HVS-based,and the standard JPEG coding using the fourblocks of Fig. 2 as the examples. It is noted fromthese tables that the advantage of the new algo-rithm is obvious.

4.2. Numerical results of different images

We have applied the new adaptive codingscheme to the following test images: Baboon, Bar-bara, Boats, Bridge, Cameraman, Couple, Lena,Peppers, Plane and Columbia. Figs. 7 and 8 illus-trate the comparisons of the results from the twoimages Lena and Baboon by applying the stan-

dard JPEG coding, the HVS-based coding and theadaptive coding scheme presented in this paper. Itis noted that when comparing to the HVS-basedmethod (Wang et al., 2001), the new algorithm havereduced between 0.04 and 0.17 in terms of bits perpixel at the same PSNR level; and it improves thePSNR of the overall images in between 0.25 and0.89 dB using the same amount of bits. The im-provement is more obvious at higher bits per pixel.

The reconstructed images of Lena and Baboonare shown in Figs. 9 and 10. It is noted that thereconstructed image Lena does not suffer notice-able impairment under the subject criteria, andhence are considered of good quality. However theimage Baboon has noticeable block effect in thenose area, due to its high complexity and thus lowPSNR at this bits per pixel.

Table 1

Bit rate (bits) under the same PSNR

Block

a b c d

PSNR (dB) 31.66 30.99 33.73 39.76

HVS-based 111 127 71 17

JPEG 108 156 77 17

Adaptive 116 112 62 19

Table 2

PSNR (dB) under the same bit rate

Block

a b c d

Bit rate (bits) 116 112 62 19

HVS-based 31.18 27.79 30.24 39.38

JPEG 31.75 27.28 29.41 39.42

Adaptive 31.66 30.99 33.73 39.76

Fig. 7. Performance comparisons for Lena.

Fig. 8. Performance comparisons for Baboon.

F. Pan / Pattern Recognition Letters 23 (2002) 1837–1845 1843

4.3. Average results of 10 selected test images

Table 3 lists the PSNR results for all the 10 testimages coded at 0.5 bpp. Comparing to JPEG aswell as the HVS-based method, we can see thatsignificant improvements in PSNR (0.07–1.63 dB)have been achieved by using this adaptive algo-

rithm. It is also noted that that more gain could beachieved to the images with high complexity, es-pecially when these images have strong horizontalor vertical edges, such as Couple or Baboon.

4.4. Average results of other standard test images

Table 4 lists the PSNR results for eight stan-dard test images coded at 0.5 bpp. Note that theseimages are not used during the derivation of theadaptive quantization tables and the adaptivezigzag scans. It can be seen from the table thatsimilar improvements could be achieved.

5. Conclusions

Adaptive image compression using local patterninformation improves the traditional JPEG-likecompression schemes in the following ways. Firstly,it significantly reduces the quantization noise ofthe important coefficients. Secondly it effectivelyeliminates the trivial coefficients that are likelymasked by the presence of the edge pattern in the

Table 3

PSNRs (dB) of the test images coded at 0.5 bpp

JPEG HVS Adaptive

Baboon 25.00 25.02 26.03

Barbara 28.63 29.63 30.48

Boats 34.40 34.77 35.54

Bridge 25.84 25.70 25.92

Cameraman 28.67 29.15 29.54

Columbia 36.05 36.08 36.90

Couple 30.46 30.65 32.28

Lena 34.65 35.02 35.86

Peppers 33.84 34.09 34.16

Plane 34.03 34.29 34.46

Table 4

PSNRs (dB) of other standard test images coded at 0.5 bpp

JPEG HVS Adaptive

Crowd 31.54 31.55 31.83

Goldhill 32.28 32.39 32.94

Lake 30.01 30.29 30.54

Lax 25.70 25.83 26.51

Man 30.82 30.92 31.47

Milkdrop 37.73 37.98 38.16

Woman 1 31.13 31.54 32.35

Woman 2 40.20 40.21 40.27

Fig. 9. Reconstructed Lena, 0.5 bpp, 35.86 dB.

Fig. 10. Reconstructed Baboon, 0.5 bpp, 26.03 dB.

1844 F. Pan / Pattern Recognition Letters 23 (2002) 1837–1845

sub-block. Thirdly, the run-length of non-zerocoefficients is shortened, and finally the number ofdifferent length of zero runs is also reduced sig-nificantly. Experiments on different tests imageshave shown that comparing to the HVS basedmethod, the new algorithm have reduced between0.04 and 0.17 in terms of bits per pixel at the samePSNR level; and it improves the PSNR of theoverall images in between 0.25 and 1.44 dB whenthe same amount of bits are used. These im-provements exhibit two features, i.e., good im-provements at higher bits per pixel, and to theimages with higher complexity.

It must be noted that a penalty of 2 bits per blockwill have to be included in the header of the bit-stream in order for the decoder to choose the properquantization table and zigzag scan path. However,by Huffman coding these two penalty bits, it canbe reduced to 1.15–1.65 bits per block, which isworthwhile for the improvements it brings in.

Although the discussion of this paper has beenfocused on DCT based still image coding, thismethod could easily be applied to the DCT basedcoding of video sequences. While the extra com-putational expense introduced by this method doesnot cause much difficulty for still image compres-sion due to increasing computing power of PC, itmight be a problem for coding of video signal inreal time. One possible solution is to use the abso-lute value instead of square operation Eqs. (1)–(3)in video coding.

References

Grosse, H.-J., Varley, M.R., Terrell, T.J., Chan, Y.K., 2000.

Improved coding of transform coefficients in JPEG-like

image compression schemes. Pattern Recognition Lett. 21,

1061–1069.

ISO/IEC 10918-1: ITU-T Rec. T81, 1994. Information Tech-

nology––Digital Compression and Coding of Continuous-

tone Still Images: Requirements and Guidelines.

Kondo, H., Oishi, Y., 2000. Digital image compression using

directional sub-block DCT. In: Internat. Conf. on Comm.

Technol. Proc., vol. 1, pp. 985–992.

Nam, J.Y., Rao, K.R., 1991. Image coding using a classified

DCT/VQ based on two channel conjugate vector quantiza-

tion. IEEE Trans. Circuits Systems Video Technol. 1, 325–

336.

Puri, A., Aravind, R., 1991. Motion-compensated video coding

with adaptive perceptual quantization. IEEE Trans. Circuits

Systems Video Technol. 1, 351–361.

Rao, K.R., Hwang, J.J., 1996. Techniques and Standards for

Image, Video and Audio Coding. Prentice Hall PTR.

Ratnakar, V., Livny, M., 2000. An efficient algorithm for

optimizing DCT quantization. IEEE Trans. Image Process.

9, 267–270.

Rosenholtz, R., Watson, A.B., 1996. Perceptual adaptive JPEG

coding. In: Internat. Conf. on Image Process., vol. 1, pp.

901–904.

Shi, Y.Q., Sun, H., 1999. Image and Video Compression for

Multimedia Engineering––Fundamentals, Algorithms and

Standards. CRC Press, Boca Raton.

Skodras, A., Christopoulos, C., Ebrahimi, T., 2001. The JPEG

2000 still image compression standard. IEEE Signal Pro-

cessing Magazine, September.

Wang, C.-Y., Lee, S.-M., Chang, L.-W., 2001. Designing JPEG

quantization tables based on human visual systems. Signal

Process.: Image Comm. 16, 501–506.

F. Pan / Pattern Recognition Letters 23 (2002) 1837–1845 1845