**Using Digital Watermarking for Copyright Protection**

Charlie Obimbo and Behzad Salami *University of Guelph Canada* 

#### **1. Introduction**

136 Watermarking – Volume 2

Ziener, D. & Teich, J. (2008). Power Signature Watermarking of IP Cores for FPGAs. *Journal* 

Ziener, D.; Baueregger, F. & Teich, J. (2010). Using the Power Side Channel of FPGAs for

Communication. *In Proceedings of the 18th Annual International IEEE Sympo- sium on Field-Programmable Custom Computing Machines (FCCM 2010),* pp. 237-244, ISBN

*of Signal Processing Systems,* VOL. 51, 2008, pp.123-136

978-0-7695-4056-6, Carolina USA, May, 2010

Without a doubt, the Internet has revolutionized the way we access information and share our ideas via tools such as Facebook, twitter, email, forums, blogs and instant messaging. The Internet is also an excellent distribution system for digital media. It is inexpensive, eliminates warehousing and delivery, and is almost instantaneous. Together with the advances of compression techniques such as JPEG, MP3 and MPEG; the Internet has become even faster, easier and more cost effective to distribute digital media such as audio, video, images and documents over the World Wide Web.

In addition to existing web sites and shared networks, the recent development of peer-topeer (P2P) file distribution tools such as Kazaa, Limewire, Exceem or eMule enables a copious number of web users to easily access and share terabytes of digital media across the globe. These technologies also significantly reduce the efforts of pirates to illegally record, sell, copy and distribute copyright-protected material without compensating the legal copyright owners.

Today, content owners are eagerly seeking technologies that promise to protect their rights and secure their content from piracy, unauthorized usage and enable the tracking and conviction of media pirates. Cryptography is probably the most common method of protecting digital content [Koch & Zhao, 1995], where the content is encrypted prior to delivery and a decryption key is provided to those who have purchased legitimate copies. However, cryptography cannot help the content providers monitor their goods after the decryption process; a pirate could easily purchase a legit copy and then re-sell it or distribute it for free over a shared network.

It is therefore important to find a way to protect these digital media with a more stringent method, which would enable the vendors and artists / photographers / directors get confidence in placing and distributing their material over the Internet. Watermarking could be such a vehicle.

#### **2. Overview**

Digital watermarking is a field that refers to the process of embedding digital data directly onto multimedia objects such that it can be detected or extracted later.

Using Digital Watermarking for Copyright Protection 139

As a result various other domains have been proposed. In current literature, the watermark is added to the image either in the spatial domain or in a transform domain [Leighton et. al. 1997]. Example of transform domains are discrete Fourier transform (DFT), the full-image discrete cosine transform (DCT) [Bartolini et. al., 2001], the block-wise DCT [Wolfgang et. al, 1999], the discrete wavelet domain (DWT) [Cappellini et. al., 1998], fractal domain [Puate et. al., 1996 & Shahraeini and Yaghoobi, 2012], the redundant contourlet transform [Leighton et. al., 1997], the Hadamard domain, Fourier- Mellin domain or the Radon domain [Lie et. al., 2006]. It has been shown that embedding the mark in the mid-frequencies of a transform domain is advantageous in terms of visibility and security over the spatial domain [Cheng

In the embedding stage visibility artifacts must be avoided and thus the Human Visual System (HVS) must be taken into account. The watermark is generally shaped using spatial or spectral shaping to reduce it's energy in areas where the mark would become visible [Lie et. al.]. An image adaptive watermarking scheme uses the local or global characteristics of the original image to determine the maximum strength that can be achieved in each area without introducing visible artifacts [Cappellini et. al., 1998]. Image-adaptive watermarking Algorithms have been proposed in [Hartung et. al., 1999, Podilchuk et. al., 1998, Cappelini

Watermarking techniques that do not require the original image for verification or extraction of the watermark are called "blind" watermarking as opposed to "informed" watermarking [Liu & Innoue, 2003, Cappellini et. al., 1998, Anderson & Petitcolas, 1998,

The functions of the digital watermarking technology can be classified in four broad

Each individual application area desires its own set of special requirements with regards to

In spite of the fact that digital watermarking has been an active area of research for decades, there is still a lot of room for improvements. One main reason for this is the limitations associated with each technique and the need to find the best balance between the three

Robustness calls for the watermark to be as strong as possible where the fidelity

It is difficult to satisfy all the requirements to their maximum at the same time. In current systems image watermarks are typically a pseudo-random signal with much lower amplitude, compared to the original image amplitude and usually with distribution of each bit into a group of pixels [Wolfgang et. al]. The pseudorandom signal is generally generated with Gaussian, uniform or bipolar probability density distribution using a secret seed.

et.al., 1999].

et. al., 1998].

Koch & Luo, 1998.].

b. Monitoring,

categories [Miller et. al.]: a. Copyright Protection,

c. Authentication, and

d. Secure and Invisible Communications.

robustness, fidelity and capacity [Lie et. al.].

requirement asks the watermark to be invisible.

conflicting requirements (robustness, fidelity and capacity).

It has three unique advantages over other techniques such as cryptography. First of all, it is imperceptible and does not affect the aesthetic of the digital data. Secondly, watermarks become fused with the actual bits of the work, unlike headers they do not get removed when the work is displayed, copied or during format changes. Lastly, they undergo the same transformation as the work itself and sometimes the extracted mark can be used to learn about the history of transformations that the work has undergone.

In general any watermarking system consists of three components


Watermarking can be applied to various digital multimedia such as images [Wolfgang et. al, 1999 & Hartung & Kutter, 1999], videos [Ren-Hou et. al., 2005 & Lie et. al., 2006], audio [Liu & Innoue, 2003 & Berghel, 1997], or text [Huang & Wu, 2004]. Image watermarking is either perceptible or imperceptible to the human eye and can be designed to be robust, fragile or semi-fragile [Koch & Zhao, 1995].

An example of a basic visible watermark would be placing a text or logo onto an image to identify it's rightful copyright owner (see Figure 1). As seen in Figure 1, an image can be placed on the web in low resolution as an advertisement. The purchaser would then receive a copy minus the watermark, on completion of the purchase, from the vendor.

Fig. 1. Example of a visible watermark

Visible marks are usually embedded in the spatial domain, that is, directly onto the pixel values of an image. Clearly, this method is fragile and can easily be compromised by cropping or replacing the text using either a basic image processing tool such as Microsoft Paint, advanced software such as Adobe Photoshop or sophisticated Algorithms such as Huang & Wu's [Huang & Wu, 2004 & Baaziz, 2005].

It has three unique advantages over other techniques such as cryptography. First of all, it is imperceptible and does not affect the aesthetic of the digital data. Secondly, watermarks become fused with the actual bits of the work, unlike headers they do not get removed when the work is displayed, copied or during format changes. Lastly, they undergo the same transformation as the work itself and sometimes the extracted mark can be used to

Watermarking can be applied to various digital multimedia such as images [Wolfgang et. al, 1999 & Hartung & Kutter, 1999], videos [Ren-Hou et. al., 2005 & Lie et. al., 2006], audio [Liu & Innoue, 2003 & Berghel, 1997], or text [Huang & Wu, 2004]. Image watermarking is either perceptible or imperceptible to the human eye and can be designed to be robust, fragile or

An example of a basic visible watermark would be placing a text or logo onto an image to identify it's rightful copyright owner (see Figure 1). As seen in Figure 1, an image can be placed on the web in low resolution as an advertisement. The purchaser would then receive

Visible marks are usually embedded in the spatial domain, that is, directly onto the pixel values of an image. Clearly, this method is fragile and can easily be compromised by cropping or replacing the text using either a basic image processing tool such as Microsoft Paint, advanced software such as Adobe Photoshop or sophisticated Algorithms such as

a copy minus the watermark, on completion of the purchase, from the vendor.

learn about the history of transformations that the work has undergone.

In general any watermarking system consists of three components

a. Watermark generation stage,

semi-fragile [Koch & Zhao, 1995].

Fig. 1. Example of a visible watermark

Huang & Wu's [Huang & Wu, 2004 & Baaziz, 2005].

b. encoding and c. decoding [12]. As a result various other domains have been proposed. In current literature, the watermark is added to the image either in the spatial domain or in a transform domain [Leighton et. al. 1997]. Example of transform domains are discrete Fourier transform (DFT), the full-image discrete cosine transform (DCT) [Bartolini et. al., 2001], the block-wise DCT [Wolfgang et. al, 1999], the discrete wavelet domain (DWT) [Cappellini et. al., 1998], fractal domain [Puate et. al., 1996 & Shahraeini and Yaghoobi, 2012], the redundant contourlet transform [Leighton et. al., 1997], the Hadamard domain, Fourier- Mellin domain or the Radon domain [Lie et. al., 2006]. It has been shown that embedding the mark in the mid-frequencies of a transform domain is advantageous in terms of visibility and security over the spatial domain [Cheng et.al., 1999].

In the embedding stage visibility artifacts must be avoided and thus the Human Visual System (HVS) must be taken into account. The watermark is generally shaped using spatial or spectral shaping to reduce it's energy in areas where the mark would become visible [Lie et. al.]. An image adaptive watermarking scheme uses the local or global characteristics of the original image to determine the maximum strength that can be achieved in each area without introducing visible artifacts [Cappellini et. al., 1998]. Image-adaptive watermarking Algorithms have been proposed in [Hartung et. al., 1999, Podilchuk et. al., 1998, Cappelini et. al., 1998].

Watermarking techniques that do not require the original image for verification or extraction of the watermark are called "blind" watermarking as opposed to "informed" watermarking [Liu & Innoue, 2003, Cappellini et. al., 1998, Anderson & Petitcolas, 1998, Koch & Luo, 1998.].

The functions of the digital watermarking technology can be classified in four broad categories [Miller et. al.]:


Each individual application area desires its own set of special requirements with regards to robustness, fidelity and capacity [Lie et. al.].

In spite of the fact that digital watermarking has been an active area of research for decades, there is still a lot of room for improvements. One main reason for this is the limitations associated with each technique and the need to find the best balance between the three conflicting requirements (robustness, fidelity and capacity).

Robustness calls for the watermark to be as strong as possible where the fidelity requirement asks the watermark to be invisible.

It is difficult to satisfy all the requirements to their maximum at the same time. In current systems image watermarks are typically a pseudo-random signal with much lower amplitude, compared to the original image amplitude and usually with distribution of each bit into a group of pixels [Wolfgang et. al]. The pseudorandom signal is generally generated with Gaussian, uniform or bipolar probability density distribution using a secret seed.

Using Digital Watermarking for Copyright Protection 141

Digital watermarking used as covert communication adds an extra level of security compared to cryptography. In cryptography, the data is encrypted and can only be decrypted using a secret key. However, the attacker is aware of the existence of such data and can be certain that with enough time, he can decrypt the data, where as in digital watermarking, the attacker can never be certain that secret information is being transmitted. Another advantage of digital watermarks is that it continues to exist even after the receiver obtains the information. Digital watermarking combined with cryptography is highly

In this Chapter we will describe a watermarking algorithm for digital images for the

In general any watermarking system consists of three components Watermark generation

The watermark signal is typically a pseudo-random signal with much lower amplitude, compared to that of the original image and usually with distribution of each bit into a group of pixels [Hartung & Kutter, 1999]. The pseudo-random signal is generally generated with Gaussian, uniform or bipolar probability density distribution using a secret seed. Watermarks could also be a string of bits or a pseudo-randomly generated set of real

The general idea is to embed a unique mark into a digital image such that it cannot be perceived by the Human Visual System but can be extracted at a later time using the content owner's secret key to prove ownership. Figure 2 shows the general example of encoding and decoding of a 4096 bit mark into the image "Lena". The mark is a binary image that has been

The cover image is first transformed into a domain that facilitates data embedding. The watermark can be embedded or encoded generally by adding or multiplying the signal to the cover image's luminance channel, the colour channels or both. For increased security and invisibility a spread spectrum coding with combination of a shaping technique is applied. In spread spectrum coding the watermark signal is spread over another known signal and then added to the image. Shaping can be done by increasing and decreasing the watermark's energy in some areas to adapt (become less visible) to the original work. In the DCT-Block domain, coefficients are modified according to the watermark content either by re-quantization, substitution or modification to impose a relationship [Bartolini et. al., 2001],

In the extraction stage some watermarking techniques need the original host image for subtracting the watermarked images, such techniques are referred to as "Informed" or

[Koch & Zhao, 1995]. A General Watermarking encoding is described in Figure 3.

desired.

purpose of copyright protection.

**3. The watermarking process** 

numbers or a small image such as a logo.

uniquely generated by the watermarking system.

**3.1 Watermark generation** 

**3.2 Watermark encoding** 

**Watermark decoding** 

stage, encoding and decoding [Bartolini et. al., 2001].

Watermarks could also be a string of bits or a pseudo-randomly generated set of real numbers or a small image such as a company logo.

These watermarks however, often carry no extra information and are not very useful. On the other hand, multi-bit watermarks typically include a second signal used as error correction and thus decrease the amount of useful information or the payload that can be embedded.

Below are some Watermarking applications.

#### **2.1 Watermark applications**

Copyright Protection: Content providers such as individual artists or large-scale broadcast companies are interested in enforcing copyright protection of digital media [Koch & Luo, 1998, Berghel, 1997]. Authors wish to be ensured that their products are not commercially used without the payment of royalties. Another branch of this technology is fingerprinting. A product is marked with a unique label or fingerprint and then distributed to the rightful customer. Fingerprinting and Copyright applications require a high degree of robustness, and should be imperceptible but may have low capacity.

#### **2.2 Monitoring**

Digital watermarks can also be used to track and monitor digital content. In medical applications, watermarks might be used for identification and accessing of individual patient records. This particular application may prevent human errors such as record mismatching therefore preventing fatal mistakes [Koch & Luo, 1998]. In broadcast monitoring, companies like to confirm that their advertisements receive the full amount of airtime purchased. They have a desire to ensure that their product is broadcasted with the full duration, at the most optimal time of the day, and at preferred strategic frequencies [Cox, 2008]. Also, companies may wish to monitor the advertisement of the competition to predict future business strategies or explore competitive marketing techniques.

#### **2.3 Authentication**

For proof of authentication watermarks can be used not only to identify if a digital file has been tampered with, but also to determine how it has been tampered with. Such information can possibly give clues on how to reverse the malicious tampering to recover the original data. Authentication of surveillance cameras can be of importance if authorities question the reliability of such evidence in courts [Koch & Luo, 1998].

#### **2.4 Communication**

The idea of covert or secret communication is as old as communication itself [Hartung & Kutter, 1999] and is used frequently by defence and intelligence sectors. Digital watermarking continue to exist even after the receiver has obtained the information. If sensitive data is leaked out to unauthorized personal, the digital watermark contained in them can be used to trace back to the original owner or the intended receiver [Koch & Luo, 1998].

Watermarks could also be a string of bits or a pseudo-randomly generated set of real

These watermarks however, often carry no extra information and are not very useful. On the other hand, multi-bit watermarks typically include a second signal used as error correction and thus decrease the amount of useful information or the payload that can be embedded.

Copyright Protection: Content providers such as individual artists or large-scale broadcast companies are interested in enforcing copyright protection of digital media [Koch & Luo, 1998, Berghel, 1997]. Authors wish to be ensured that their products are not commercially used without the payment of royalties. Another branch of this technology is fingerprinting. A product is marked with a unique label or fingerprint and then distributed to the rightful customer. Fingerprinting and Copyright applications require a high degree of robustness,

Digital watermarks can also be used to track and monitor digital content. In medical applications, watermarks might be used for identification and accessing of individual patient records. This particular application may prevent human errors such as record mismatching therefore preventing fatal mistakes [Koch & Luo, 1998]. In broadcast monitoring, companies like to confirm that their advertisements receive the full amount of airtime purchased. They have a desire to ensure that their product is broadcasted with the full duration, at the most optimal time of the day, and at preferred strategic frequencies [Cox, 2008]. Also, companies may wish to monitor the advertisement of the competition to

For proof of authentication watermarks can be used not only to identify if a digital file has been tampered with, but also to determine how it has been tampered with. Such information can possibly give clues on how to reverse the malicious tampering to recover the original data. Authentication of surveillance cameras can be of importance if authorities

The idea of covert or secret communication is as old as communication itself [Hartung & Kutter, 1999] and is used frequently by defence and intelligence sectors. Digital watermarking continue to exist even after the receiver has obtained the information. If sensitive data is leaked out to unauthorized personal, the digital watermark contained in them can be used to trace back to the original owner or the intended receiver [Koch & Luo,

predict future business strategies or explore competitive marketing techniques.

question the reliability of such evidence in courts [Koch & Luo, 1998].

numbers or a small image such as a company logo.

and should be imperceptible but may have low capacity.

Below are some Watermarking applications.

**2.1 Watermark applications** 

**2.2 Monitoring** 

**2.3 Authentication** 

**2.4 Communication** 

1998].

Digital watermarking used as covert communication adds an extra level of security compared to cryptography. In cryptography, the data is encrypted and can only be decrypted using a secret key. However, the attacker is aware of the existence of such data and can be certain that with enough time, he can decrypt the data, where as in digital watermarking, the attacker can never be certain that secret information is being transmitted.

Another advantage of digital watermarks is that it continues to exist even after the receiver obtains the information. Digital watermarking combined with cryptography is highly desired.

In this Chapter we will describe a watermarking algorithm for digital images for the purpose of copyright protection.

#### **3. The watermarking process**

In general any watermarking system consists of three components Watermark generation stage, encoding and decoding [Bartolini et. al., 2001].

#### **3.1 Watermark generation**

The watermark signal is typically a pseudo-random signal with much lower amplitude, compared to that of the original image and usually with distribution of each bit into a group of pixels [Hartung & Kutter, 1999]. The pseudo-random signal is generally generated with Gaussian, uniform or bipolar probability density distribution using a secret seed. Watermarks could also be a string of bits or a pseudo-randomly generated set of real numbers or a small image such as a logo.

#### **3.2 Watermark encoding**

The general idea is to embed a unique mark into a digital image such that it cannot be perceived by the Human Visual System but can be extracted at a later time using the content owner's secret key to prove ownership. Figure 2 shows the general example of encoding and decoding of a 4096 bit mark into the image "Lena". The mark is a binary image that has been uniquely generated by the watermarking system.

The cover image is first transformed into a domain that facilitates data embedding. The watermark can be embedded or encoded generally by adding or multiplying the signal to the cover image's luminance channel, the colour channels or both. For increased security and invisibility a spread spectrum coding with combination of a shaping technique is applied. In spread spectrum coding the watermark signal is spread over another known signal and then added to the image. Shaping can be done by increasing and decreasing the watermark's energy in some areas to adapt (become less visible) to the original work. In the DCT-Block domain, coefficients are modified according to the watermark content either by re-quantization, substitution or modification to impose a relationship [Bartolini et. al., 2001], [Koch & Zhao, 1995]. A General Watermarking encoding is described in Figure 3.

#### **Watermark decoding**

In the extraction stage some watermarking techniques need the original host image for subtracting the watermarked images, such techniques are referred to as "Informed" or

Using Digital Watermarking for Copyright Protection 143

\* ( , \*) \* \* *W W SIM W W*

The output of a verification system is a yes/no answer. Extraction of the watermark is performed by reconstructing the watermark bit by bit from a potential watermarked image and comparing it with the original watermark. A threshold is defined for the percentage of similarity (Bit Error Rate) between the two. The basic process is depicted in Figure 4. An image marked with a watermark and a secret key are used by the watermark decoder to

Watermarks can take many shapes such as a company logo, image of a text or a pseudorandomly generated sequence of bits or real numbers. We propose a new watermark with properties of self-correction. The Error Correction stage performs without any additional sources or reference marks. The author of the host image has the ability to specify personal information such as name, creation date, transaction ID or image ID as a human readable

The provided information string is denoted as *S*, where *Si* represents the *i*th characters in the string. First, each character *Si* is converted to it's binary representation *Bi* and all *Bi*'s are concatenated to form a sequence of bits denoted as B. For example, the binary representation of the string "Ben" is "01000010 01100101 01101110", where the spaces are

Fig. 5. Personal watermark of the string "Salami06" before encryption

extract the original watermark signal.

Fig. 4. General watermark decoding diagram

**3.3 Watermark generation algorithm** 

only added for ease of visual distinction.

string of characters.

*W W* <sup>+</sup> <sup>+</sup>

(1)

Fig. 2. General example of watermarking an image

Fig. 3. General watermark encoding diagram

private watermarking [Miller et. al., 2002]. Other techniques do not need the original host image but need a secret seed to generate the original watermark for comparison. Such systems are referred to as "blind" watermarking. A watermarking system is "semi-blind" if it relies on some data or features derived from the original host image.

It is important to distinguish between watermark verification and watermark extraction. In most of literature the watermark is only verified, that is a correlation between the potential watermarked image and the original watermarked image is performed using the normalized correlation defined in Equation 1.

private watermarking [Miller et. al., 2002]. Other techniques do not need the original host image but need a secret seed to generate the original watermark for comparison. Such systems are referred to as "blind" watermarking. A watermarking system is "semi-blind" if

It is important to distinguish between watermark verification and watermark extraction. In most of literature the watermark is only verified, that is a correlation between the potential watermarked image and the original watermarked image is performed using the

it relies on some data or features derived from the original host image.

Fig. 2. General example of watermarking an image

Fig. 3. General watermark encoding diagram

normalized correlation defined in Equation 1.

$$SIM(\mathcal{W}, \mathcal{W}^\*) = \frac{\mathcal{W}^\* \cdot \mathcal{W}}{\sqrt{\mathcal{W}^\* \cdot \mathcal{W}^\*}}\tag{1}$$

The output of a verification system is a yes/no answer. Extraction of the watermark is performed by reconstructing the watermark bit by bit from a potential watermarked image and comparing it with the original watermark. A threshold is defined for the percentage of similarity (Bit Error Rate) between the two. The basic process is depicted in Figure 4. An image marked with a watermark and a secret key are used by the watermark decoder to extract the original watermark signal.

Fig. 4. General watermark decoding diagram

#### **3.3 Watermark generation algorithm**

Watermarks can take many shapes such as a company logo, image of a text or a pseudorandomly generated sequence of bits or real numbers. We propose a new watermark with properties of self-correction. The Error Correction stage performs without any additional sources or reference marks. The author of the host image has the ability to specify personal information such as name, creation date, transaction ID or image ID as a human readable string of characters.

The provided information string is denoted as *S*, where *Si* represents the *i*th characters in the string. First, each character *Si* is converted to it's binary representation *Bi* and all *Bi*'s are concatenated to form a sequence of bits denoted as B. For example, the binary representation of the string "Ben" is "01000010 01100101 01101110", where the spaces are only added for ease of visual distinction.

Fig. 5. Personal watermark of the string "Salami06" before encryption

Using Digital Watermarking for Copyright Protection 145

Decryption method is very similar except that the shuffling is performed in the reverse

We embed the watermark into the DCT-Block domain of the host image. The DCT-Block has the advantage of revealing the local image characteristics [Cox & Li, 2005] and unlike using the full frame DCT, the watermark strength can be adapted to each local frequency content. This method proves to achieve maximum watermark fidelity [De Rosa et. al.,

At first, the general encoding procedure is briefly described to allow the reader a broad conceptual view of the algorithm. Then in subsequent sections the algorithm is disassembled in individual components and each is further described in greater detail. The algorithm can be divided in three general stages. Image Preparation: The image is segmented into individual non-overlapping blocks, the colour space is converted from RGB to YCrCb (YUV) and each 8 × 8 block is transformed from the spatial to the frequency

**Watermark Encoding:** The properties of the Human Visual System is explored and image adaptive strengths are determined for each block, the blocks are checked for potential edges before the pixels of the watermark image can be embedded. A testing mechanism ensures that the pixel was correctly embedded. Image Finalization: This stage is exactly the same as "Image Preparation" only in the reverse order. Each block is transformed back from the frequency to the spatial domain and the colour space is converted back from YCrCb to RGB.

A pseudo-code of the encoding method is described in Algorithm 3 and for a more visual

Lastly, all blocks are re-assembled to form the final watermarked image.

**Algorithm 1 (Encrypt)** *Encrypts the Watermark using Mersenne Twister*

**Algorithm 2 (Decrypt)** *Decrypts the Watermark using Mersenne Twister* 

order, for more details see Algorithm 2.

*1* R ! GENERATERANDOMS*(N,Kmark)*

! *Wi*

ENCRYPT-MARK*(W, Kmark, N)*

! *WRi*

> ! *Temp*

DECRYPT-MARK*(W, Kmark, N)*

! *WRi*

> ! *Temp*

!

3 **do** *Temp* 

4 *Wi*

5 *WRi*

6 DELETE(R)

1 R ! *GENERATERANDOMS(N,Kmark)*

**3.4 Watermark encoding algorithm** 

representation please refer to Figure 7.

 *N 1 to 0*

! *Wi*

! *0 to N*

*3* **do** *Temp* 

*4 Wi*

*5 WRi*

*6 DELETE(R)*

*2* **for** *i* 

2 **for** *i* 

2000].

domain.

In the next stage the sequence B is converted to a binary image, where a "0" represents a white pixel and a "1" represents a black pixel. The sequence is repeated vertically generating a barcode like image, illustrated in Figure 5. The mark uses a 64 bit information, duplicated 64 times, resulting in 4096 individual bits or 4 kilobytes. The vertical dimension of the mark depends on the height of the host image I , the larger the image dimensions the more the string can be repeated, thus increasing the robustness.

The dimensions of the image is determined by the Equation

$$\text{W}\_{h} = \frac{I\_{h} \times I\_{w}}{\text{Strlen(S)} \times 8} \tag{2}$$

where *Wh* is the height of the watermark image, *Ih* and *Iw* are the dimensions of the host image and Strlen(*S*) is a function that returns the number of characters in the string *S* provided by the owner or author.

The watermark image is further encrypted using a user specified seed *Kmark* into a fast uniform pseudo-random number generator called "Mersenne Twister" with a period of 219937 1. The algorithm was developed by M. Matsumoto and T. Nishimura [Matsumoto & Nishimura, 1998] in 1998 and improved in 2002 [Matsumoto & Nishimura, 2002]. The generator is implemented to generate fast output by completely avoiding divisions and multiplications. It generates an array at one time and takes the full advantage of cache memory and pipeline processing if supported. Figure 6 depicts an example of an encrypted watermark

Fig. 6. Example of an encrypted watermark

Experts consider this an excellent random number generator. Using the seed Kmark, a sequence of *N* long-integer values ranging from 0 to *N* 1 is generated where *N* = *Wh* × *Ww*. The result is a 1-Dimensional array of pseudo-randomly generated values denoted as *R*, where *Ri* denotes the ith value in the list.

In order to encrypt the watermark the forward-scrambling Algorithm 1 is used, where each individual pixel *Wi* is exchanged with the corresponding pixel *WRi* defined by *Ri*. The

In the next stage the sequence B is converted to a binary image, where a "0" represents a white pixel and a "1" represents a black pixel. The sequence is repeated vertically generating a barcode like image, illustrated in Figure 5. The mark uses a 64 bit information, duplicated 64 times, resulting in 4096 individual bits or 4 kilobytes. The vertical dimension of the mark depends on the height of the host image I , the larger the image dimensions the more the

*h w*

where *Wh* is the height of the watermark image, *Ih* and *Iw* are the dimensions of the host image and Strlen(*S*) is a function that returns the number of characters in the string *S*

The watermark image is further encrypted using a user specified seed *Kmark* into a fast uniform pseudo-random number generator called "Mersenne Twister" with a period of 219937 1. The algorithm was developed by M. Matsumoto and T. Nishimura [Matsumoto & Nishimura, 1998] in 1998 and improved in 2002 [Matsumoto & Nishimura, 2002]. The generator is implemented to generate fast output by completely avoiding divisions and multiplications. It generates an array at one time and takes the full advantage of cache memory and pipeline processing if supported. Figure 6 depicts an example of an encrypted

Experts consider this an excellent random number generator. Using the seed Kmark, a sequence of *N* long-integer values ranging from 0 to *N* 1 is generated where *N* = *Wh* × *Ww*. The result is a 1-Dimensional array of pseudo-randomly generated values denoted as *R*,

In order to encrypt the watermark the forward-scrambling Algorithm 1 is used, where each individual pixel *Wi* is exchanged with the corresponding pixel *WRi* defined by *Ri*. The

*S*

% % (2)

*h I I <sup>W</sup>*

string can be repeated, thus increasing the robustness.

The dimensions of the image is determined by the Equation

Strlen( ) 8

provided by the owner or author.

Fig. 6. Example of an encrypted watermark

where *Ri* denotes the ith value in the list.

watermark

Decryption method is very similar except that the shuffling is performed in the reverse order, for more details see Algorithm 2.

**Algorithm 1 (Encrypt)** *Encrypts the Watermark using Mersenne Twister*

ENCRYPT-MARK*(W, Kmark, N) 1* R ! GENERATERANDOMS*(N,Kmark) 2* **for** *i* ! *0 to N 3* **do** *Temp* ! *Wi 4 Wi* ! *WRi 5 WRi* ! *Temp 6 DELETE(R)*

**Algorithm 2 (Decrypt)** *Decrypts the Watermark using Mersenne Twister* 

```
DECRYPT-MARK(W, Kmark, N)
1 R ! GENERATERANDOMS(N,Kmark)
2 for i ! N  1 to 0
3 do Temp ! Wi
4 Wi ! WRi
5 WRi ! Temp
```
6 DELETE(R)

#### **3.4 Watermark encoding algorithm**

We embed the watermark into the DCT-Block domain of the host image. The DCT-Block has the advantage of revealing the local image characteristics [Cox & Li, 2005] and unlike using the full frame DCT, the watermark strength can be adapted to each local frequency content. This method proves to achieve maximum watermark fidelity [De Rosa et. al., 2000].

At first, the general encoding procedure is briefly described to allow the reader a broad conceptual view of the algorithm. Then in subsequent sections the algorithm is disassembled in individual components and each is further described in greater detail. The algorithm can be divided in three general stages. Image Preparation: The image is segmented into individual non-overlapping blocks, the colour space is converted from RGB to YCrCb (YUV) and each 8 × 8 block is transformed from the spatial to the frequency domain.

**Watermark Encoding:** The properties of the Human Visual System is explored and image adaptive strengths are determined for each block, the blocks are checked for potential edges before the pixels of the watermark image can be embedded. A testing mechanism ensures that the pixel was correctly embedded. Image Finalization: This stage is exactly the same as "Image Preparation" only in the reverse order. Each block is transformed back from the frequency to the spatial domain and the colour space is converted back from YCrCb to RGB. Lastly, all blocks are re-assembled to form the final watermarked image.

A pseudo-code of the encoding method is described in Algorithm 3 and for a more visual representation please refer to Figure 7.

Using Digital Watermarking for Copyright Protection 147

The image I is first segmented into 16 × 16 non-overlapping *RGB* blocks, and for each the colout space is converted from *RGB* to *YCrCb* with a subsampling ratio of (4:1:1) obtaining *IYcrcb*. The chrominance *Icrcb* and luminance *IY* components are separated. Then the Block-DCT is applied to transform each 8 × 8 luminance block from the spatial domain to the frequency domain, followed by a lossy quantization step similar to JPEG compression.

The final quantized luminance blocks are saved in the set Y for the embedding procedure. In the next subsection and the discrete cosine transform (DCT) is described in more detail.

After the colour conversion, the luminance (Y) component is extracted and a 2- Dimensional Discrete Cosine Transform (DCT) is performed on every 8 × 8 (Y) block. The DCT is an invertible function that transforms the data from the spatial domain to the frequency domain and helps to separate the image into parts (spectral sub-bands) of differing importance with respect to the image's quality. The JPEG, MPEG-1, MPEG-2 and MPEG-7 encodings use the DCT domain to discard high frequency information that are not important to the human perception. The Forward DCT is defined in Equation 3 and it's


2 1 2 1 , , , cos cos

where *C*(*u*, *v*) is the resulting DCT coefficient at the coordinates (*u*, *v*), (*u*, *v*) is defined by Equation 5, *S* is the two dimensional square array of size *N* × *N* and in this case *N* = 8.


 1 / for 0 and 0 , 2 / otherwise *Nu v*

<sup>3</sup> <sup>4</sup>

*N*

2 1 2 1 , , , cos cos

1 1

*C uv uv Sxy N N*

1 1

0 ; <sup>0</sup> ; 1 < <sup>1</sup> <sup>&</sup>lt; 2 = <sup>2</sup> <sup>=</sup> , , (3)

0 ; <sup>0</sup> ; 1 < <sup>1</sup> <sup>&</sup>lt; 2 = <sup>2</sup> <sup>=</sup> , , (4)



*N N*

*x u y v*

(5)

2 2

*x u y v*

2 2

**Algorithm 4 (Prepare)** *Prepares image I for encoding*

**3.4.1 Discrete cosine transform / quantization** 

inverse in Equation 3.3.





*N N*

*x y*

where *N*, *S* and *C* are as described in Equation 3.

0 0

*Sxy uvC uv*


*u v*


*N N*

*x y*

0 0


5

Fig. 7. Proposed watermark encoding diagram

**Algorithm 3 (Encode)** *Encodes the watermark W in image I*


In Algorithm 3 *W* is the encrypted watermark and I is the original host image. *Kimage* is used for shuffling of blocks, is the user defined watermark strength, Target is a value that can be toggle between 0 and 255 to minimize the number of changes that the encoding algorithm must perform. ET is the edge threshold used in edge classification of blocks. The image I is first segmented via a call to Prepare-Image(*I*, *N*, *Y*, *B*, *Icrcb*) where *N* luminance blocks denoted as *Y* together with the chrominance components *Icrcb* are extracted.

In Algorithm 3 *W* is the encrypted watermark and I is the original host image. *Kimage* is used for shuffling of blocks, is the user defined watermark strength, Target is a value that can be toggle between 0 and 255 to minimize the number of changes that the encoding algorithm must perform. ET is the edge threshold used in edge classification of blocks. The image I is first segmented via a call to Prepare-Image(*I*, *N*, *Y*, *B*, *Icrcb*) where *N* luminance blocks denoted as *Y* together with the chrominance components *Icrcb* are

Fig. 7. Proposed watermark encoding diagram

extracted.

**Algorithm 3 (Encode)** *Encodes the watermark W in image I*

**Algorithm 4 (Prepare)** *Prepares image I for encoding*

The image I is first segmented into 16 × 16 non-overlapping *RGB* blocks, and for each the colout space is converted from *RGB* to *YCrCb* with a subsampling ratio of (4:1:1) obtaining *IYcrcb*. The chrominance *Icrcb* and luminance *IY* components are separated. Then the Block-DCT is applied to transform each 8 × 8 luminance block from the spatial domain to the frequency domain, followed by a lossy quantization step similar to JPEG compression.

The final quantized luminance blocks are saved in the set Y for the embedding procedure. In the next subsection and the discrete cosine transform (DCT) is described in more detail.

#### **3.4.1 Discrete cosine transform / quantization**

After the colour conversion, the luminance (Y) component is extracted and a 2- Dimensional Discrete Cosine Transform (DCT) is performed on every 8 × 8 (Y) block. The DCT is an invertible function that transforms the data from the spatial domain to the frequency domain and helps to separate the image into parts (spectral sub-bands) of differing importance with respect to the image's quality. The JPEG, MPEG-1, MPEG-2 and MPEG-7 encodings use the DCT domain to discard high frequency information that are not important to the human perception. The Forward DCT is defined in Equation 3 and it's inverse in Equation 3.3.

$$\mathcal{C}(u,v) = \alpha(u,v) \sum\_{x=0}^{N-1} \sum\_{y=0}^{N-1} S(x,y) \cos\left[\frac{\pi(2x+1)u}{2N}\right] \cos\left[\frac{\pi(2y+1)v}{2N}\right] \tag{3}$$

where *C*(*u*, *v*) is the resulting DCT coefficient at the coordinates (*u*, *v*), (*u*, *v*) is defined by Equation 5, *S* is the two dimensional square array of size *N* × *N* and in this case *N* = 8.

$$S(x,y) = \sum\_{x=0}^{N-1} \sum\_{y=0}^{N-1} a(u,v)C(u,v)\cos\left[\frac{\pi(2x+1)u}{2N}\right] \cos\left[\frac{\pi(2y+1)v}{2N}\right] \tag{4}$$

where *N*, *S* and *C* are as described in Equation 3.

$$\alpha(u,v) = \begin{cases} 1/N & \text{for } u = 0 \text{ and } v = 0\\ 2/N & \text{otherwise} \end{cases} \tag{5}$$

Using Digital Watermarking for Copyright Protection 149

In Algorithm 5, Y is the set of *N* luminance blocks, W is the encrypted watermark, *ET* is the threshold used for edge detection, is the maximum watermark strength and Target is a value that can be toggled between 0 and 255 for minimizing the number of changes a single

The watermark decoding stage is very similar to the procedures described in the encoding except that now the original image and the original watermark are not available. The watermark bits are constructed bit by bit using the watermarked image. In order to extract the watermark successfully, several requirements must be met. One of the requirements is that the author's key file must be present. A key file is an encrypted binary file that has been written at the time of watermark encoding. This file contains the secret keys used by the author, the target value used for embedding, several templates such as image patches that facilitates in synchronization of the image in the spatial domain and several indices of rejected DCT blocks that can be included for a more robust

Furthermore, if the dimensions of the image have been altered, the image must be rescaled to the exact same dimensions as when it was marked. The algorithm will also need to be informed by a human user if a potential cropping has occurred, which in that case the synchronization templates provided in the key file will be matched against

 *with strength*

**Algorithm 5 (Encode-Watermark)** *Encodes W in Luminance component Y*

**Algorithm 6 (Encode-InBlock)** *Encodes one DCT block* 

run of embedding creates.

decoding.

**3.5 Watermark decoding algorithm** 

The first transform coefficient in the block is the average value of the sample so at location (0, 0) in the two dimensional 8 × 8 block the value for (u, v) is 1/*N*. This value is referred to as the "DC" coefficient. All other transform coefficients are called the "AC" coefficients and have (*u*, *v*) equal to 2/*N*.

The lossy JPEG compression uses an 8×8 quantization matrix of step sizes (quantums), one element for each DCT coefficient, to further increase the compression ratio by discarding the high frequency coefficients. In this watermarking algorithm it is important to ensure that the coefficients used for embedding are not affected by JPEG's lossy-quantization step, therefore the embedding will occur after the lossy-quantization. Great care was taken not to affect the compression ratio. The luminance quantization matrix "Q" obtained from the (Independent Jpeg Group) IJG JPEG library is shown below.


Each DCT coefficient in the block is divided by the associated element in the quantization matrix and rounded to the nearest integer. The higher frequency coefficients which are located towards the lower part of the block are divided by higher values forcing them to become 0's. The lower frequencies (upper left) which are the perceptually significant part of the image are divided by smaller values, maintaining their accuracy. After quantization, usually more than half of the DCT coefficients are equal to zero. Therefore, it is impractical to modify any of the high frequency coefficients during watermark embedding for two main reasons. The first and most important reason is that JPEG compression will wipe out these values, destroying the watermark completely. The second reason is that it will disallow JPEG to perform an optimal compression using run-length coding of the zero coefficients.

#### **3.4.2 Pixel encoding**

The next step in the encoding process is the call to ShuffleBlocks(Y,Kimage) in which quantized luminance blocks are shuffled using a user defined key, very similar to the algorithm described in Algorithm 3.1. This step adds extra security to protect the watermark from intentional removal so that it will be very difficult for an attacker to guess or statistically show in which block which pixel of the encrypted watermark has been embedded.

Finally the encoding algorithms are given below. Algorithm 5 embeds the mark into the entire image and Algorithm 6 embeds on bit into one DCT block.

The first transform coefficient in the block is the average value of the sample so at location (0, 0) in the two dimensional 8 × 8 block the value for (u, v) is 1/*N*. This value is referred to as the "DC" coefficient. All other transform coefficients are called the "AC" coefficients and

The lossy JPEG compression uses an 8×8 quantization matrix of step sizes (quantums), one element for each DCT coefficient, to further increase the compression ratio by discarding the high frequency coefficients. In this watermarking algorithm it is important to ensure that the coefficients used for embedding are not affected by JPEG's lossy-quantization step, therefore the embedding will occur after the lossy-quantization. Great care was taken not to affect the compression ratio. The luminance quantization matrix "Q" obtained from the (Independent

Each DCT coefficient in the block is divided by the associated element in the quantization matrix and rounded to the nearest integer. The higher frequency coefficients which are located towards the lower part of the block are divided by higher values forcing them to become 0's. The lower frequencies (upper left) which are the perceptually significant part of the image are divided by smaller values, maintaining their accuracy. After quantization, usually more than half of the DCT coefficients are equal to zero. Therefore, it is impractical to modify any of the high frequency coefficients during watermark embedding for two main reasons. The first and most important reason is that JPEG compression will wipe out these values, destroying the watermark completely. The second reason is that it will disallow JPEG to perform an optimal compression using run-length coding of the zero

The next step in the encoding process is the call to ShuffleBlocks(Y,Kimage) in which quantized luminance blocks are shuffled using a user defined key, very similar to the algorithm described in Algorithm 3.1. This step adds extra security to protect the watermark from intentional removal so that it will be very difficult for an attacker to guess or statistically show in which block which pixel of the encrypted watermark has been

Finally the encoding algorithms are given below. Algorithm 5 embeds the mark into the

entire image and Algorithm 6 embeds on bit into one DCT block.

have (*u*, *v*) equal to 2/*N*.

coefficients.

embedded.

**3.4.2 Pixel encoding** 

Jpeg Group) IJG JPEG library is shown below.

**Algorithm 5 (Encode-Watermark)** *Encodes W in Luminance component Y*

**Algorithm 6 (Encode-InBlock)** *Encodes one DCT block with strength*

In Algorithm 5, Y is the set of *N* luminance blocks, W is the encrypted watermark, *ET* is the threshold used for edge detection, is the maximum watermark strength and Target is a value that can be toggled between 0 and 255 for minimizing the number of changes a single run of embedding creates.

#### **3.5 Watermark decoding algorithm**

The watermark decoding stage is very similar to the procedures described in the encoding except that now the original image and the original watermark are not available. The watermark bits are constructed bit by bit using the watermarked image. In order to extract the watermark successfully, several requirements must be met. One of the requirements is that the author's key file must be present. A key file is an encrypted binary file that has been written at the time of watermark encoding. This file contains the secret keys used by the author, the target value used for embedding, several templates such as image patches that facilitates in synchronization of the image in the spatial domain and several indices of rejected DCT blocks that can be included for a more robust decoding.

Furthermore, if the dimensions of the image have been altered, the image must be rescaled to the exact same dimensions as when it was marked. The algorithm will also need to be informed by a human user if a potential cropping has occurred, which in that case the synchronization templates provided in the key file will be matched against

Using Digital Watermarking for Copyright Protection 151

Watermarked images are usually posted on web sites (internet) or distributed to individual customers sometime after the encoding process. Between the time of encoding and the time of decoding the watermarked image may undergo many possible manipulations. Some of these attacks are intentional such as cropping and others are unintentional like the collection of channel noise. In addition, the lossy quantization step during JPEG compression and decompression is a major source for error in the decoding process. Therefore, the decoded watermark may not always appear 100% identical to the original embedded mark. One method of determining the similarity of two given signals is known as Bit Error Rate (BER). The BER is the ratio of the total bit error to the total number of bits embedded and is

> <sup>1</sup> \* 0

*BER B B N* 

For a perfect decoding step the BER would equal to 0, indicating no lost bits. Since no algorithm can claim to be perfect it is important for every watermarking system to expect

This Section presents the results found using the methodology described in Section 3. One Thousand color images are used to test and obtain results from the proposed system "Digital Image Copyright Protector" (DIGI-COP). In Section 4.1 we explore the fidelity of the marked images, in Section 4.2 Error Rates are analyzed and in the subsequent sections

*I I*

 and *B B I I* are the bits at the *i*th position of the decoded watermark and the original watermark respectively. The symbol ) is the binary XOR operation and *N* is the total

*N*

*I*

such bit errors and facilitate a method of Error Correction.

/


**Algorithm 7 (Decode)** *Extracts the embedded watermark from image I*

given by:

where \*

number of bits in *B*.

**4. Experiments and results** 

various attacks are explored as listed below.

the cropped image. If the matching has been successful, the remaining (uncropped) image is pasted against a black background in the exact same position that the templates suggest.

Fig. 8. Proposed watermark decoding diagram

The watermark is constructed pixel by pixel according to the relationship between specific DCT coefficients in a block. Before the relationships can be tested for, the image is segmented into blocks of 16 × 16 for a colour space conversion from RGB to YCrCb (YUV). Next, all blocks from the luminance (Y) component are extracted and a Forward DCT is performed on each 8 × 8 non-overlapping block using Equation 3.

The DCT blocks are shuffled using a secret key *Kimage* obtained from the key file. Now the DCT blocks are ready for the watermarking extraction routine outlined in Algorithm 7.

The Target value is read from the key file and can either be 0 or 255, the value for AntiTarget is always calculated to be the opposite of the Target. The decoding procedure in Algorithm 7 traverses each block in the luminance component *Y*, choosing the same two random coefficients *C1* and *C2* as used in the encoding procedure described in Algorithm 6. The structure Bucket read from is used to check if the current block has been previously discarded by the encoding algorithm, in which case the sign of the value in *Bucketj* will decide if the pixel *Wi* is equal to 255 or 0. After all blocks have been processed, the watermark *W* has to be decrypted to reveal the original Bar-code like image created by the author.

the cropped image. If the matching has been successful, the remaining (uncropped) image is pasted against a black background in the exact same position that the templates

The watermark is constructed pixel by pixel according to the relationship between specific DCT coefficients in a block. Before the relationships can be tested for, the image is segmented into blocks of 16 × 16 for a colour space conversion from RGB to YCrCb (YUV). Next, all blocks from the luminance (Y) component are extracted and a Forward DCT is

The DCT blocks are shuffled using a secret key *Kimage* obtained from the key file. Now the DCT blocks are ready for the watermarking extraction routine outlined in Algorithm 7.

The Target value is read from the key file and can either be 0 or 255, the value for AntiTarget is always calculated to be the opposite of the Target. The decoding procedure in Algorithm 7 traverses each block in the luminance component *Y*, choosing the same two random coefficients *C1* and *C2* as used in the encoding procedure described in Algorithm 6. The structure Bucket read from is used to check if the current block has been previously discarded by the encoding algorithm, in which case the sign of the value in *Bucketj* will decide if the pixel *Wi* is equal to 255 or 0. After all blocks have been processed, the watermark *W* has to be decrypted to reveal the original Bar-code like image created by the

suggest.

author.

Fig. 8. Proposed watermark decoding diagram

performed on each 8 × 8 non-overlapping block using Equation 3.

**Algorithm 7 (Decode)** *Extracts the embedded watermark from image I*

Watermarked images are usually posted on web sites (internet) or distributed to individual customers sometime after the encoding process. Between the time of encoding and the time of decoding the watermarked image may undergo many possible manipulations. Some of these attacks are intentional such as cropping and others are unintentional like the collection of channel noise. In addition, the lossy quantization step during JPEG compression and decompression is a major source for error in the decoding process. Therefore, the decoded watermark may not always appear 100% identical to the original embedded mark. One method of determining the similarity of two given signals is known as Bit Error Rate (BER). The BER is the ratio of the total bit error to the total number of bits embedded and is given by:

$$BER = \left(\sum\_{I=0}^{N-1} B\_I^\* \oplus B\_I \right) \; / \; N \tag{6}$$

where \* and *B B I I* are the bits at the *i*th position of the decoded watermark and the original watermark respectively. The symbol ) is the binary XOR operation and *N* is the total number of bits in *B*.

For a perfect decoding step the BER would equal to 0, indicating no lost bits. Since no algorithm can claim to be perfect it is important for every watermarking system to expect such bit errors and facilitate a method of Error Correction.

#### **4. Experiments and results**

This Section presents the results found using the methodology described in Section 3. One Thousand color images are used to test and obtain results from the proposed system "Digital Image Copyright Protector" (DIGI-COP). In Section 4.1 we explore the fidelity of the marked images, in Section 4.2 Error Rates are analyzed and in the subsequent sections various attacks are explored as listed below.

Using Digital Watermarking for Copyright Protection 153

It is clear from the results that DIGI-COP achieves higher PSNR values than the BWM even when the watermark strength () is increased. Next, the PSNR results for classical images marked with = 40 are illustrated in Table 2 and Figure 10. The PSNR values are calculated

in the YCrCb domain and reflects the similarities in the luminance components.

Fig. 9. PSNR values of 1000 watermarked images with = 10

Table 2. PSNR values of classical images with = 40

Fig. 10. PSNR values of selected watermarked images with = 40

#### **4.1 Fidelity**

One of the major requirements of digital watermarking systems is the ability to hide the mark within the cover work such that it becomes perceptually invisible to a human observer. The proposed image watermarking method (DIGI-COP) presented in this Chapter makes use of local characteristics of the image to achieve higher invisibility rates than it's base algorithm (BWM). Consider an image I and it's watermarked version IW, then the standard deviation between I and IW is defined by the MSE (Mean Square Error):

$$\text{MSE}\ (I, I^W) = \frac{1}{M \times N} \sum\_{i=0}^{M-1} \sum\_{j=0}^{N-1} ||I(i, j) - I^W(i, j)||^2 \tag{7}$$

Peak Signal-to-Noise Ratio (PSNR) is often used for global evaluation of the quality of reconstruction in image compression techniques. It is expressed in terms of the logarithmic decibel (dB) scale and defined as:

$$\text{PSNR} \ (I, I^W) = 10 \log\_{10} \left( \frac{255^2}{MSE} \right) = 20 \log\_{10} \left( \frac{255}{\sqrt{MSE}} \right) \text{dB} \tag{8}$$

Thus PSNR is the ratio between the maximum possible power of a signal and power of corrupting noise that affects the fidelity of its representation. The original image and the watermarked image are denoted by *I* and by *IW* respectively, and *M* and *N* are the dimensions of the images. A lower value of MSE means less distortion, and therefore the higher the PSNR value is, the better or closer the watermarked image is to the original. Generally a PSNR larger than 32 dB means invisible visual degradation and a human observer perceives both images as indistinguishable [Wilson & Martinez 97].

For the fidelity test 1000 RGB images were marked with various watermark strengths, \* {10, 20, 30, 40}, using both the base method (BWM) and the proposed watermarking method (DIGI-COP). The PSNR values obtained are graphically produced in Figure 9 and the average PSNR values together with the standard deviations are presented in Table 1.


Table 1. Average PSNR of 1000 watermarked images DIGI-COP BWM

One of the major requirements of digital watermarking systems is the ability to hide the mark within the cover work such that it becomes perceptually invisible to a human observer. The proposed image watermarking method (DIGI-COP) presented in this Chapter makes use of local characteristics of the image to achieve higher invisibility rates than it's base algorithm (BWM). Consider an image I and it's watermarked version IW, then the

Peak Signal-to-Noise Ratio (PSNR) is often used for global evaluation of the quality of reconstruction in image compression techniques. It is expressed in terms of the logarithmic

Thus PSNR is the ratio between the maximum possible power of a signal and power of corrupting noise that affects the fidelity of its representation. The original image and the watermarked image are denoted by *I* and by *IW* respectively, and *M* and *N* are the dimensions of the images. A lower value of MSE means less distortion, and therefore the higher the PSNR value is, the better or closer the watermarked image is to the original. Generally a PSNR larger than 32 dB means invisible visual degradation and a human

For the fidelity test 1000 RGB images were marked with various watermark strengths, \* {10, 20, 30, 40}, using both the base method (BWM) and the proposed watermarking method (DIGI-COP). The PSNR values obtained are graphically produced in Figure 9 and the average PSNR values together with the standard deviations are presented in Table 1.

> 10 41.76 2.37 39.97 2.75 20 41.24 2.07 38.71 2.22 30 40.72 1.92 37.39 1.75 40 40.35 1.83 36.12 1.38

observer perceives both images as indistinguishable [Wilson & Martinez 97].

PSNR (dB)

Table 1. Average PSNR of 1000 watermarked images DIGI-COP BWM

(strength) Digi-Cop

(7)

(8)

BWM PSNR (dB)

standard deviation between I and IW is defined by the MSE (Mean Square Error):

**4.1 Fidelity** 

decibel (dB) scale and defined as:

Fig. 9. PSNR values of 1000 watermarked images with = 10

It is clear from the results that DIGI-COP achieves higher PSNR values than the BWM even when the watermark strength () is increased. Next, the PSNR results for classical images marked with = 40 are illustrated in Table 2 and Figure 10. The PSNR values are calculated in the YCrCb domain and reflects the similarities in the luminance components.


Table 2. PSNR values of classical images with = 40

Fig. 10. PSNR values of selected watermarked images with = 40

Using Digital Watermarking for Copyright Protection 155

The second type of a decoding error is a false-positive, where the watermark decoder incorrectly detects the presence of a watermark in an image. There are two types of falsepositives, the first type (FP-I) occurs when a watermark decoder extracts a watermark in an image I that has not been marked previously. The second type (FP-II) of false positives is when a decoder extracts watermark W from an image I that has been marked with a different mark W . Both types of false-positives are undesired in a reliable system. The

In Figure 13, the FP-I results suggests that the BWM has a higher accuracy in determining unwatermarked or incorrectly marked images. On the other hand, FP-II results show clearly that both watermark decoders can correctly extract the watermark W from an image marked with W placed at location 500. Further, BWM and DIGI-COP show high BER values when attempting to extract watermark W from images marked with different watermarks such as W\*.

JPEG compression is one of the main reasons for the success of the internet and must be taken into account when designing an image watermarking system. JPEG compression will attempt to remove the perceptual unimportant elements from an image and may render the imperceptible watermark undetectable. Image compressions are considered one of the

The JPEG compression resistance test presented in this thesis is performed on 1000 color images over 11 different JPEG quality factors ranging from 100% to 0%. The BER value is

results for both types of false-positives tests are presented in Figure 4.9.

Fig. 13. False positive rates types I and II

strongest enemies of digital watermarking techniques today.

**4.3 Image processing attacks** 

**4.3.1 JPEG compression** 

In literature [Meerwald 2001, Jellinke 2000, Mohanty 1999, Guo 2003], it has been previously reported that the base watermarking method (BWM) occasionally produces small spatial defects around edges and an undesired blocking effect in smooth regions of the image. Figure 11 demonstrates that DIGI-COP protects the image from such unwanted artifacts.

Fig. 11. Smooth and edge regions of the image "Boys" after watermarking

In Figure 11 the shaded blue and green regions in the image represent the selected locations of smooth and edge regions. The enlarged views of the smooth and edge region are presented towards the right of the image and compared to the original unwatermarked areas.

#### **4.2 Error rates**

There are two types of errors that can occur during the watermark extraction stage. The first error is a false-negative (FN), in which a watermark decoder fails to identify a watermarked image as a legit or "marked" copy. The decoder's ability of correctly extracting the watermark *W* from a marked image *I* is calculated in terms of a BER (Bit Error Rate) value described in Equation 6 on Page 17. The FN test was performed on 1000 watermarked images with of 20 using the decoder of BWM and DIGI-COP. The BER values are illustrated in Figure 12.

Fig. 12. False negative rate

The graph indicates that DIGI-COP achieves a lower BER in the decoding process and extracts the watermark with higher precision.

In literature [Meerwald 2001, Jellinke 2000, Mohanty 1999, Guo 2003], it has been previously reported that the base watermarking method (BWM) occasionally produces small spatial defects around edges and an undesired blocking effect in smooth regions of the image. Figure 11 demonstrates that DIGI-COP protects the image from such unwanted artifacts.

Fig. 11. Smooth and edge regions of the image "Boys" after watermarking

**4.2 Error rates** 

Fig. 12. False negative rate

extracts the watermark with higher precision.

towards the right of the image and compared to the original unwatermarked areas.

using the decoder of BWM and DIGI-COP. The BER values are illustrated in Figure 12.

In Figure 11 the shaded blue and green regions in the image represent the selected locations of smooth and edge regions. The enlarged views of the smooth and edge region are presented

There are two types of errors that can occur during the watermark extraction stage. The first error is a false-negative (FN), in which a watermark decoder fails to identify a watermarked image as a legit or "marked" copy. The decoder's ability of correctly extracting the watermark *W* from a marked image *I* is calculated in terms of a BER (Bit Error Rate) value described in Equation 6 on Page 17. The FN test was performed on 1000 watermarked images with of 20

The graph indicates that DIGI-COP achieves a lower BER in the decoding process and

The second type of a decoding error is a false-positive, where the watermark decoder incorrectly detects the presence of a watermark in an image. There are two types of falsepositives, the first type (FP-I) occurs when a watermark decoder extracts a watermark in an image I that has not been marked previously. The second type (FP-II) of false positives is when a decoder extracts watermark W from an image I that has been marked with a different mark W . Both types of false-positives are undesired in a reliable system. The results for both types of false-positives tests are presented in Figure 4.9.

In Figure 13, the FP-I results suggests that the BWM has a higher accuracy in determining unwatermarked or incorrectly marked images. On the other hand, FP-II results show clearly that both watermark decoders can correctly extract the watermark W from an image marked with W placed at location 500. Further, BWM and DIGI-COP show high BER values when attempting to extract watermark W from images marked with different watermarks such as W\*.

Fig. 13. False positive rates types I and II

#### **4.3 Image processing attacks**

#### **4.3.1 JPEG compression**

JPEG compression is one of the main reasons for the success of the internet and must be taken into account when designing an image watermarking system. JPEG compression will attempt to remove the perceptual unimportant elements from an image and may render the imperceptible watermark undetectable. Image compressions are considered one of the strongest enemies of digital watermarking techniques today.

The JPEG compression resistance test presented in this thesis is performed on 1000 color images over 11 different JPEG quality factors ranging from 100% to 0%. The BER value is

Using Digital Watermarking for Copyright Protection 157

Extensive results show that DIGI-COP is preferred over the classical method in terms of it's fidelity and robustness. The method can embed large quantities of data into the cover image without any noticeable changes. The new embedding technique facilitates embedding the right amount of watermark at the most advantageous locations in the image without causing visual artifacts. The improvement is achieved by exploiting the characteristics of the cover image in the DCT-Block domain, as well as the sensitivity of the HVS to small changes in smooth regions and edges of the image. The fidelity test of DIGI-COP achieved on average 41 dB over one thousand encodings where the classical method achieves on average

In addition, both false-negative and false-positive rates of the two algorithms were compared over a set of thousand images. DIGI-COP's false-negative rates show to be more reliable than the classical algorithm. It correctly extracts 99.9% of the data with a standard deviation of 0.26 as compared to the classical method with 97.2% decoding rate and a deviation of 1.8. Falsepositive rates were compared and both algorithms show good performance. Although both algorithms can identify a legit watermark from a set of thousand randomly marked images and deny all unmarked or incorrectly marked images, the classical algorithm shows a slight better performance. This is due to DIGI-COP's feature of storing discarded bits in the key file

Anderson, R. J. & Petitcolas, F. On The Limits Of Steganography. (1998). IEEE J. Select. Areas

Baaziz, N. (2005). Adaptive Watermarking Schemes Based On A Redundant Contourlet

Bartolini, F.; Barni, M.; Podilchuk, C. I. & Delp, E. J. Watermark Embedding: Hiding A Signal

Berghel. (1997). Watermarking Cyberspace. Communications of the ACM, 40, 1997, pp 19–24. Cappellini, V.; Barni, M.; Bartolini F.; & Piva, A. (1998). A DCT Domain System For Robust

Cappellini, V.; Piva, A.; Barni, M.; Bartolini F. & Rigacci, F. (1998). A M.A.P. Identification

Cheng, L.L.; Ng, K.S.; Cheng L. M. & Wong, M.K. (1999). Adaptive Watermarking By Using

Cox and Q. Li. Using Perceptual Models To Improve Fidelity And Provide Invariance To

De Rosa M. Barni, F. Bartolini and A. Piva. Capacity of Full Frame DCT Image Watermarks. In IEEE Transactions on Image Processing, vol. 9, August 2000, pp 1450–1455.

Cox, I. J. Digital Watermarking and Steganography. Morgan Kaufmann, 2008.

Communication. (Special Issue on Copyright and Privacy Protection), 16, 474–481,

Transform. IEEE International Conference on Image Processing ICIP 2005, 1, , 11–14

Within A Cover Image. IEEE Communications Magazine, 39, August 2001. pp 102–108,

Image Watermarking, Signal Processing (Special Issue on Watermarking), 66, May

Criterion For DCT-Based Watermarking, Proceedings Europe: Signal Processing

Pixel Position Shifting Technique IEEE Transactions on Consumer Electronics, 45,

Value-Metric Scaling For Quantization Index Modulation Watermarking. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal

and assuming it's presence in unmarked or incorrectly marked images.

38 dB on the same set of images.

**6. References** 

May 1998.

1998. pp 357–372

September 2005, pp 221–224.

November 1999. pp 1057–1064.

Conference (EUSIPCOŠ98), September 1998.

Processing 2005' (ICASSP), March 18-23 2005.


calculated for each individual image and averaged over the entire image set for that particular quality as illustrated in Table 3.

Table 3. Effects of JPEG compression on BER

Both the BWM and DIGI-COP decoders show high resilience against this attack up to a JPEG quality factor of 40. In addition, an Error Correction (EC) technique is used to achieve a much lower BER value than the original proposed method in it's essence. The illustration in Figure 14 is the direct result of a total of 50,000 individual decoding operations and shows the advantage of the Error Correction technique.

Fig. 14. JPG Compression with Error Correction (EC)

The Error Correction was set to 10%, 20% and 30% to see the changing effects of the BER value over the entire image set. In Figure 4.10 the BER value is plotted against the various JPEG qualities, and it can be seen that as images are compressed at higher rates (lower qualities), the Bit Error Rate also increases. However, DIGICOP's Error Correction feature significantly reduced the decoding BER.

#### **5. Conclusion and future research**

A new adaptive and invisible digital watermarking system ("DIGICOP") in the DCT-Block domain is discussed as a method for protecting copyright for digital images. The performances of DIGI-COP and the classical DCT-Block technique are compared.

Extensive results show that DIGI-COP is preferred over the classical method in terms of it's fidelity and robustness. The method can embed large quantities of data into the cover image without any noticeable changes. The new embedding technique facilitates embedding the right amount of watermark at the most advantageous locations in the image without causing visual artifacts. The improvement is achieved by exploiting the characteristics of the cover image in the DCT-Block domain, as well as the sensitivity of the HVS to small changes in smooth regions and edges of the image. The fidelity test of DIGI-COP achieved on average 41 dB over one thousand encodings where the classical method achieves on average 38 dB on the same set of images.

In addition, both false-negative and false-positive rates of the two algorithms were compared over a set of thousand images. DIGI-COP's false-negative rates show to be more reliable than the classical algorithm. It correctly extracts 99.9% of the data with a standard deviation of 0.26 as compared to the classical method with 97.2% decoding rate and a deviation of 1.8. Falsepositive rates were compared and both algorithms show good performance. Although both algorithms can identify a legit watermark from a set of thousand randomly marked images and deny all unmarked or incorrectly marked images, the classical algorithm shows a slight better performance. This is due to DIGI-COP's feature of storing discarded bits in the key file and assuming it's presence in unmarked or incorrectly marked images.

#### **6. References**

156 Watermarking – Volume 2

calculated for each individual image and averaged over the entire image set for that

Both the BWM and DIGI-COP decoders show high resilience against this attack up to a JPEG quality factor of 40. In addition, an Error Correction (EC) technique is used to achieve a much lower BER value than the original proposed method in it's essence. The illustration in Figure 14 is the direct result of a total of 50,000 individual decoding operations and shows

The Error Correction was set to 10%, 20% and 30% to see the changing effects of the BER value over the entire image set. In Figure 4.10 the BER value is plotted against the various JPEG qualities, and it can be seen that as images are compressed at higher rates (lower qualities), the Bit Error Rate also increases. However, DIGICOP's Error Correction feature

A new adaptive and invisible digital watermarking system ("DIGICOP") in the DCT-Block domain is discussed as a method for protecting copyright for digital images. The

performances of DIGI-COP and the classical DCT-Block technique are compared.

particular quality as illustrated in Table 3.

Table 3. Effects of JPEG compression on BER

the advantage of the Error Correction technique.

Fig. 14. JPG Compression with Error Correction (EC)

significantly reduced the decoding BER.

**5. Conclusion and future research** 


**8** 

**2D Watermarking:** 

Hassen Seddik

*Tunisia* 

**Non Conventional Approaches** 

*ESSTT Higher Sciences and Technical School of Tunisia, Tunis* 

The growth of new image technologies and data exchanges, in addition to the everincreasing use of multimedia content through online services, has created the need for new techniques capable of assuring copyright protection and data owner identification. Watermarking is now considered as an efficient means for resolving these problems. Watermark embedding techniques depend on the representation domain of the image (spatial, frequency, and multiresolution). Every domain has its specific advantages and limitations. Moreover, each technique in a chosen domain is found to be robust to specific sets of attack types. In addition all the techniques developed in theses domain are widely known and can be defeated to break the used algorithm and target the embedded watermark to destroy it or to put it out. So we need to develop another robust domain that defeats these limitations and respects all the watermarking criterions (capacity, invisibility and robustness). In this chapter, new watermarking methods are presented using new domains for the image representation and watermark embedding. These domains are based on different mathematical transformations of the image matrix. The applied transformations that process the image coefficients must dispose of three indispensable proprieties: no data loss, reversibility and preservation. Theses domains are found to be robust against a wide range of synchronous and asynchronous STIRMARK attacks. The robustness of the techniques in preserving and extracting the embedded watermark is proved after various attacks types. It is also improved when compared with other methods in use. In addition, the proposed methods are blind and the use of the host image is not needed in the

**2. A Blind image watermarking method based on the Hessenberg** 

The advent of the Internet brought about a sudden increase in the use of digital media in electronic commerce and various other services. Because of the ease of reproducing or falsifying digital media, it's very easy for the manufacturer to incur financial losses. To counter such problems, watermarking methods have gained significantly in popularity as they protect the ownership rights and simplify proprietor identification. To that end,

**1. Introduction** 

watermark detection process.

**transformation 2.1 Introduction** 


## **2D Watermarking: Non Conventional Approaches**

Hassen Seddik

*ESSTT Higher Sciences and Technical School of Tunisia, Tunis Tunisia* 

#### **1. Introduction**

158 Watermarking – Volume 2

Guo, H. Digital Image Watermarking for Ownership verifciation. PhD thesis, University of

Hartung, F. & M. Kutter, Multimedia Watermarking Techniques. (1999). Proceedings of the

Huang, C.-H. and Wu, J.-L. Attacking visible watermarking schemes. In IEEE Transactions

Jellinek, B. Invisible watermarking of digital images for copyright protection. Master's

Koch, E. & Zhao, J. (1995) Towards Robust And Hidden Image Copyright Labeling.

Koch E.; Zhao, J. & Luo, C. (1998). In business today and tomorrow Communications of the

Leighton, F. T.; Cox, I. J.; Kilian, J. & Shamoon, T. (1997). Secure Spread Spectrum

Lie, W. & Chang, L. (1997). Robust And High-Quality Time-Domain Audio Watermarking

Liu, Z. & Inoue, A. (2003). Audio Watermarking Techniques Using Sinusoidal Patterns

Matsumoto, M. & Nishimura, T. (1998). Mersenne twister: A 623-dimensionally

Matsumoto, M. & Nishimura, T. (2002). A nonempirical test on the weight of pseudorandom

Meerwald, P. Digital image watermarking in the wavelet transform domain. Master's

Podilchuk, C. I.; & Zeng, W. (1998) Image-Adaptive Watermarking Using Visual Models, IEEE Journal on Selected Areas in Communications, 16, May 1998. pp 525–539. Puate, J. and Jordan, F. "Using fractal compression scheme to embed a digital signature into

Ren-Hou, L.; Lian-Shan, L. & Qi, G. (2005). A Robust Video Watermarking Scheme Based On

Shahraeini, S. and Yaghoobi, M. A Robust Digital Image Watermarking Approach against

Wilson, D. R. and Martinez, T. R. Improved heterogeneous distance functions. In Journal of

Wolfgang, R. B.; Podilchuk, C. I. & Delp, E. J. (1999). Perceptual Watermarks for Digital

Available http://iswww.epfl.ch/~jordan/watermarking.html.

Cybernetics, 8, 5176-5180, August 2005. pp 18–21.

Images and Video, vol 87:7, July 1999, pp 1108–1126.

Miller, M. L.; Cox, I. J. & J. A. Bloom. (2002). Digital Watermarking. Academic Press, 2002. Mohanty, S. P. Watermarking of digital images. Master's Thesis, University of Salzburg,

Modeling and Computer Simulation, vol 8, January 1998, pp 3–30.

Proceedings of 1995 IEEE Workshop on Nonlinear Signal and Image Processing,

Watermarking For Multimedia. IEEE Transactions Image Processing, v6, December

Based On Low-Frequency Amplitude Modification, IEEE Transactions on

Based On Pseudorandom Sequences. IEEE Transactions on Circuits and Systems

equidistributed uniform pseudorandom number generator. ACM Transactions on

number generators. Monte Carlo and Quasi-Monte Carlo methods, 2002, pp 381–395.

image," in Proc. SPIE Photonics East Symp., Boston, MA, Nov. 18–22, 1996.

DCT. Proceedings of 2005 International Conference on Machine Learning and

JPEG Compression Attack Based on Hybrid Fractal-Wavelet. Advanced Materials

Ottawa, 2003.

1999 IEEE, 87, July 1999, pp 1079–1107.

Thesis, University of Salzburg, January 2000.

pages 452–455. Halkidiki, Greece, June 1995.

Multimedia, vol 8, February 2006. pp 46 – 59.

Thesis, University of Salzburg, January 2001.

for Video Technology, vol. 13, August 2003, pp 801–812.

ACM, 41, July 1998. pp 66–72.

1997. pp 1673–1687.

January 1999.

Research, 403-408, 2012.

Artificial Intelligence Research., 1997.

on Multimedia, volume 6, pages 16–30, February 2004.

The growth of new image technologies and data exchanges, in addition to the everincreasing use of multimedia content through online services, has created the need for new techniques capable of assuring copyright protection and data owner identification. Watermarking is now considered as an efficient means for resolving these problems. Watermark embedding techniques depend on the representation domain of the image (spatial, frequency, and multiresolution). Every domain has its specific advantages and limitations. Moreover, each technique in a chosen domain is found to be robust to specific sets of attack types. In addition all the techniques developed in theses domain are widely known and can be defeated to break the used algorithm and target the embedded watermark to destroy it or to put it out. So we need to develop another robust domain that defeats these limitations and respects all the watermarking criterions (capacity, invisibility and robustness). In this chapter, new watermarking methods are presented using new domains for the image representation and watermark embedding. These domains are based on different mathematical transformations of the image matrix. The applied transformations that process the image coefficients must dispose of three indispensable proprieties: no data loss, reversibility and preservation. Theses domains are found to be robust against a wide range of synchronous and asynchronous STIRMARK attacks. The robustness of the techniques in preserving and extracting the embedded watermark is proved after various attacks types. It is also improved when compared with other methods in use. In addition, the proposed methods are blind and the use of the host image is not needed in the watermark detection process.

#### **2. A Blind image watermarking method based on the Hessenberg transformation**

#### **2.1 Introduction**

The advent of the Internet brought about a sudden increase in the use of digital media in electronic commerce and various other services. Because of the ease of reproducing or falsifying digital media, it's very easy for the manufacturer to incur financial losses. To counter such problems, watermarking methods have gained significantly in popularity as they protect the ownership rights and simplify proprietor identification. To that end,

2D Watermarking: Non Conventional Approaches 161

exceeds the length of the fixed interval, it becomes impossible to describe it. To overcome this problem, the discrete wavelet transform is necessary by the use of base functions and windowing operations. In the case of data compression, the implementation of the DWT is similar to that of sub-band coding, where at each stage, a coarse overall shape and the details of the data obtained from the previous stage are derived. When encoding is performed in the DWT domain, two processes are applied: decomposition by separating data into frequency bands using high-pass and low-pass filtering, and down-sampling, which consists in removing unneeded data for future reconstruction. Decoding for its part, involves up sampling in order to adjust dimensionality and recombine data from different bands. Many methods are developed based on DWT schemes. One of the interesting is the use of the SA-DWT (Shape Adaptive Discrete Wavelets Transform) for developing a blind watermarking algorithm and HVS characteristics to achieve the best trade-off between invisibility and robustness. This method is found to be robust against some attacks such as

As explained, different domains are used for watermark embedding with various developed techniques. The spatial domain is found to be more robust against different kinds of asynchronous attacks such as rotation, rescaling, affine transforms etc. and less than others, whereas the frequency domains is well known for its robustness against synchronous attacks such as lossy compression (using DCT transform), filtering, noise adding or a specific kind of geometrical transforms such as scaling, rotation and translation (using Fourier-Melin transform or the multi-resolution domain: DWT). These limitations created the necessity to develop specific techniques in each domain to cover the robustness loss face to some attacks. In the following, we propose a new image watermarking domain: called the Hessenberg domain, which is able to counter a large set of different attacks kinds with high robustness, by the use of a substituting technique. We show that this domain can cover the limitations of the spatial, frequency and DWT domains. In addition, the robustness of this watermarking technique is improved when

Any image can be transformed by an orthogonal transformation; we will illustrate the relationship between our proposed method and other transformations. In fact an orthogonal transform applies a rotation to the representation space. The data of the image passes from a space where they are highly correlated into a space where this correlation is minimized. The less correlated coefficients of the transformed image gather the image characteristics. If this information is classified in way of significance, the data can be compressed by eliminating the data which valuesare null or near to be null and by quantization of the selected

*T* its complex conjugate. An image I of size M×N has a transformed form

, , Where 0 1 *k M* and 0 1 *l N* (1)

, *j* a set of function of orthogonal discrete

lossy compression (JPEG, JPEG2000 and MPEG-4), scaling and filtering.

compared with many recent techniques in use.

coefficients to be transmitted. Let's note *T i k l*, -

1 1

0 0 ( ,) (, ) (, ) *M N T k l i j I kl Ii j T i j* 

,

bases and -

*TI* given:

\* , , *<sup>i</sup> <sup>j</sup> k l*

**2.2 Proposed Hessenberg watermarking technique 2.2.1 Mathematical Hessenberg transform overview** 

various techniques have been developed, each ultimately aiming at pinpointing equilibrium between imperceptibility and robustness of the watermark against wide attacks kinds, depending on the image domain representation. Many researchers have focused their efforts on security and robustness, as well as on the watermarking capacity that are essential to obtain an irremovable and inappreciable watermark with regards to the image processing domain. Each domain presents its robustness face to particular kind of attacks and its limitation to others, but no one is able to resist to a wide set of synchronous and asynchronous attacks gathering the robustness of the different domains. In addition, many of these techniques require the presence of the original image to read the inserted watermark. To satisfy these watermarking obligations, the necessity of either finding a blind watermarking method robust to a large set of attacks kinds, or a new image domain representation more robust than the known domains, is more and more urgent. In this chapter, these two constraints are satisfied. In fact we propose a new watermarking method using a new domain of the image representation based on the mathematical Hessenberg transformation. Using this method, both robustness and security criteria are fully met, and the embedded mark is fully invisible and present in all cases after different signal processing distortions. The Hessenberg Image Transformation brings a new representation domain with remarkable watermarking possibilities exceeding the limits of the domains cited above face to different attacks. The image is represented by the triangular part of the Hessenberg matrix used in the watermarking process. A study of this matrix is conducted with a view to identify the sectors or zones that can hold the watermark according to the three criteria mentioned above. Once the watermark is embedded in a chosen sector, inverse transformation is applied to return to the spatial representation holding the embedded mark.

If we explore the field of watermarking we find that the most commonly used watermarking technique domains are Spatial, DFT, DCT and Wavelet domains. There are many ways the spatial domain can be used in watermark embedding, for example: substituting the least significant bit in a chosen image pixel, coding by texture blocks, changing paired pixels, etc. However, various approaches that defeat the limitations of the spatial domains have been developed and can be used in the frequency domain. These include, the spread spectrum, content-based approaches, JPEG-based approaches, etc. For this purpose, the transformations used are the Discrete Cosine Transform (DCT), the Discrete Fourier Transform (DFT), and the Discrete Wavelet Transform (DWT). The DCT is the main transform used in JPEG image compression. It eliminates DFT high frequency components induced by the sharp discontinuities at the boundary between the consecutive periods in the time. To represent sharp value changes, it needs non-zero high frequency DFT components. For purpose of compression, all high frequency DFT components are deleted, causing a distortion of the original image. To overcome this difficulty, the DCT concatenates a period with the mirrored image of its an-adjacent period. The common DCT form is derived from a class of discrete Chebyshev polynomials. Whereas the advantage of DFT is its ability to describe the frequency responses of a signal even as allowing the possibility of extracting different signal characteristics from this frequency domain. While it's notable disadvantage is the absence of any information concerning the occurrence time of these frequency components. However, a particular frequency response that occurs in a certain interval can be detected with the Short Time Fourier Transform which splits the signal into fixed-length intervals where the Fourier analysis is applied. But if the cycle of the frequency response

various techniques have been developed, each ultimately aiming at pinpointing equilibrium between imperceptibility and robustness of the watermark against wide attacks kinds, depending on the image domain representation. Many researchers have focused their efforts on security and robustness, as well as on the watermarking capacity that are essential to obtain an irremovable and inappreciable watermark with regards to the image processing domain. Each domain presents its robustness face to particular kind of attacks and its limitation to others, but no one is able to resist to a wide set of synchronous and asynchronous attacks gathering the robustness of the different domains. In addition, many of these techniques require the presence of the original image to read the inserted watermark. To satisfy these watermarking obligations, the necessity of either finding a blind watermarking method robust to a large set of attacks kinds, or a new image domain representation more robust than the known domains, is more and more urgent. In this chapter, these two constraints are satisfied. In fact we propose a new watermarking method using a new domain of the image representation based on the mathematical Hessenberg transformation. Using this method, both robustness and security criteria are fully met, and the embedded mark is fully invisible and present in all cases after different signal processing distortions. The Hessenberg Image Transformation brings a new representation domain with remarkable watermarking possibilities exceeding the limits of the domains cited above face to different attacks. The image is represented by the triangular part of the Hessenberg matrix used in the watermarking process. A study of this matrix is conducted with a view to identify the sectors or zones that can hold the watermark according to the three criteria mentioned above. Once the watermark is embedded in a chosen sector, inverse transformation is applied to return to the spatial representation

If we explore the field of watermarking we find that the most commonly used watermarking technique domains are Spatial, DFT, DCT and Wavelet domains. There are many ways the spatial domain can be used in watermark embedding, for example: substituting the least significant bit in a chosen image pixel, coding by texture blocks, changing paired pixels, etc. However, various approaches that defeat the limitations of the spatial domains have been developed and can be used in the frequency domain. These include, the spread spectrum, content-based approaches, JPEG-based approaches, etc. For this purpose, the transformations used are the Discrete Cosine Transform (DCT), the Discrete Fourier Transform (DFT), and the Discrete Wavelet Transform (DWT). The DCT is the main transform used in JPEG image compression. It eliminates DFT high frequency components induced by the sharp discontinuities at the boundary between the consecutive periods in the time. To represent sharp value changes, it needs non-zero high frequency DFT components. For purpose of compression, all high frequency DFT components are deleted, causing a distortion of the original image. To overcome this difficulty, the DCT concatenates a period with the mirrored image of its an-adjacent period. The common DCT form is derived from a class of discrete Chebyshev polynomials. Whereas the advantage of DFT is its ability to describe the frequency responses of a signal even as allowing the possibility of extracting different signal characteristics from this frequency domain. While it's notable disadvantage is the absence of any information concerning the occurrence time of these frequency components. However, a particular frequency response that occurs in a certain interval can be detected with the Short Time Fourier Transform which splits the signal into fixed-length intervals where the Fourier analysis is applied. But if the cycle of the frequency response

holding the embedded mark.

exceeds the length of the fixed interval, it becomes impossible to describe it. To overcome this problem, the discrete wavelet transform is necessary by the use of base functions and windowing operations. In the case of data compression, the implementation of the DWT is similar to that of sub-band coding, where at each stage, a coarse overall shape and the details of the data obtained from the previous stage are derived. When encoding is performed in the DWT domain, two processes are applied: decomposition by separating data into frequency bands using high-pass and low-pass filtering, and down-sampling, which consists in removing unneeded data for future reconstruction. Decoding for its part, involves up sampling in order to adjust dimensionality and recombine data from different bands. Many methods are developed based on DWT schemes. One of the interesting is the use of the SA-DWT (Shape Adaptive Discrete Wavelets Transform) for developing a blind watermarking algorithm and HVS characteristics to achieve the best trade-off between invisibility and robustness. This method is found to be robust against some attacks such as lossy compression (JPEG, JPEG2000 and MPEG-4), scaling and filtering.

As explained, different domains are used for watermark embedding with various developed techniques. The spatial domain is found to be more robust against different kinds of asynchronous attacks such as rotation, rescaling, affine transforms etc. and less than others, whereas the frequency domains is well known for its robustness against synchronous attacks such as lossy compression (using DCT transform), filtering, noise adding or a specific kind of geometrical transforms such as scaling, rotation and translation (using Fourier-Melin transform or the multi-resolution domain: DWT). These limitations created the necessity to develop specific techniques in each domain to cover the robustness loss face to some attacks. In the following, we propose a new image watermarking domain: called the Hessenberg domain, which is able to counter a large set of different attacks kinds with high robustness, by the use of a substituting technique. We show that this domain can cover the limitations of the spatial, frequency and DWT domains. In addition, the robustness of this watermarking technique is improved when compared with many recent techniques in use.

#### **2.2 Proposed Hessenberg watermarking technique**

#### **2.2.1 Mathematical Hessenberg transform overview**

Any image can be transformed by an orthogonal transformation; we will illustrate the relationship between our proposed method and other transformations. In fact an orthogonal transform applies a rotation to the representation space. The data of the image passes from a space where they are highly correlated into a space where this correlation is minimized. The less correlated coefficients of the transformed image gather the image characteristics. If this information is classified in way of significance, the data can be compressed by eliminating the data which valuesare null or near to be null and by quantization of the selected coefficients to be transmitted. Let's note *T i k l*, - , *j* a set of function of orthogonal discrete bases and - \* , , *<sup>i</sup> <sup>j</sup> k l T* its complex conjugate. An image I of size M×N has a transformed form *TI* given:

$$I\_T(k,l) = \sum\_{i=0}^{M-1} \sum\_{j=0}^{N-1} I(i,j)T\_{k,l}(i,j) \text{ Where } \ 0 \le k \le M-1 \text{ and } 0 \le l \le N-1 \tag{1}$$

2D Watermarking: Non Conventional Approaches 163

is easy to see that H is unitarily similar to A throughout the course of this iteration. The iteration is continued until the sub-diagonal elements of H converge to zero, i.e., until the Schur decomposition has been (approximately) obtained. To summarize these mathematical steps, we can say that in finding the eigenvalues of a matrix using the QR algorithm, the matrix is first transformed by a unitary similarity transformation to upper Hessenberg form. The QR algorithm then iteratively generates a sequence of upper Hessenberg matrices by unitary similarity transformations. For general real matrices, \* Q becomes <sup>T</sup> Q and the routine reduces the real general matrix A to the upper Hessenberg form H by this

*I* ). It

(8)

T Q Q I ( ) *size A* (7)

In this scheme, V is unitary and R is upper triangular (i.e., the QR factorization of H

<sup>T</sup> A Q H Q , where Q is a unitary matrix so that -

I: denotes the identity matrix and *<sup>T</sup> Q* represents the unitary matrix transpose. The general matrix structure produced by the routine is presented by the following equations as

11 12 1n

<sup>0</sup> ; <sup>1</sup> <sup>&</sup>lt; <sup>1</sup> <sup>&</sup>lt;

. . .. 0 . 0a

<sup>1</sup> <sup>&</sup>lt; <sup>1</sup> <sup>&</sup>lt; <sup>1</sup> <sup>&</sup>lt; <sup>1</sup> <sup>&</sup>lt; <sup>2</sup> <sup>=</sup>

The form of H will be tri-diagonal if the processed matrix is symmetric or Hermetian. In general, the mathematical application of this transformation serves two purposes: the first is to obtain a matrix having the same eigenvalues as the original one, but which requires less computation to reveal them; conversely, the second is the packed storage. In fact, a triangular matrix may be stored more compactly, if the relevant triangle is packed by

In this work, the produced triangular Hessenberg matrix is exploited as a new representation domain of the image where the watermark is embedded. By analyzing this matrix, we discovered the existence of an exploitable zone. Embedding a watermark in this zone produces no effect on the original matrix values after applying the inverse Hessenberg transform. This zone provides imperceptibility and robustness to the watermark. To take advantage of this characteristic, we apply this transformation to the image matrix to obtain the Hessenberg triangular representation of the image. This matrix is processed to find in which zones we can embed a watermark without any change produced on the original image. This means that we can change and increase the values of this zone in the Hessenberg matrix, without any effect being produced on the original image after the inverse Hessenberg transform is applied. The following section presents the study applied

a a . . a 0 a a

H . ..

22 2n

nn

1 2 i j *ij a W i j j - / for* (9)

orthogonal similarity transformation:

*H(i, j) i j i j -* 0 unless or 1 and:

columns in a one-dimensional array.

The aij elements of H are stored in the one-dimensional array W as:

to this matrix to identify this zone called insensitive zone.


$$I(i,j) = \sum\_{i=0}^{M-1} \sum\_{j=0}^{N-1} I\_T(k,l) T\_{k,l}^\*(i,j) \text{ Where } 0 \le i \le M-1 \text{ and } 0 \le j \le N-1 \tag{2}$$

From these transformations an important characteristic must be illustrated which is the energy distribution. In Figure 1, we show an example of the energy distribution in the spatial and frequency DCT domains. We will illustrate the importance of the energy distribution of the image through the Hessenberg transform and its affect in the watermarking process.

The proposed method uses the mathematical Hessenberg transformation. The used algorithm is based on the LAPACK routines for computing the Hessenberg form of the processed matrix. Perhaps the most successful numerical algorithm for computing the complete eigensystem of a general square matrix *A* is the implicitly shifted QR algorithm. One of the keys to the success of this method is its relationship to the Schur decomposition:

Fig. 1. Energy distribution in the spatial and transformed domains.

This well-known decomposition asserts that every square matrix A is unitarily similar to an upper triangular matrix T. The QR algorithm produces a sequence of unitary similarity transformations that iteratively reduce A to upper triangular form. In other words, it computes the Schur decomposition. A practical implementation of the QR algorithm begins with an initial unitary similarity transformation of A to the condensed form \* Q A Q H where H is upper Hessenberg (almost upper triangular') and Q is unitary. Then the following iterations to set the lower triangular to zero is performed as follows:

$$\mathbf{Q}^\* \mathbf{A} \mathbf{Q} = \mathbf{H} \quad \text{is first characterized} \tag{4}$$

For j=1, 2, 3… until convergence. Select a shift µ then the QR factorization is as follows:

$$\text{VR} = \text{H} - \mu I \tag{5}$$

Then the matrices Q and H are assigned the following values:

$$\begin{cases} \mathbf{H} = \mathbf{V}^\* \text{ H} \mathbf{V} \\ \mathbf{Q} = \mathbf{Q} \mathbf{V} \end{cases} \tag{6}$$

From these transformations an important characteristic must be illustrated which is the energy distribution. In Figure 1, we show an example of the energy distribution in the spatial and frequency DCT domains. We will illustrate the importance of the energy distribution of the image through the Hessenberg transform and its affect in the

The proposed method uses the mathematical Hessenberg transformation. The used algorithm is based on the LAPACK routines for computing the Hessenberg form of the processed matrix. Perhaps the most successful numerical algorithm for computing the complete eigensystem of a general square matrix *A* is the implicitly shifted QR algorithm. One of the keys to the success of this method is its relationship to the Schur decomposition:

This well-known decomposition asserts that every square matrix A is unitarily similar to an upper triangular matrix T. The QR algorithm produces a sequence of unitary similarity transformations that iteratively reduce A to upper triangular form. In other words, it computes the Schur decomposition. A practical implementation of the QR algorithm begins with an initial unitary similarity transformation of A to the condensed form \* Q A Q H where H is upper Hessenberg (almost upper triangular') and Q is unitary. Then the

, , Where 0 1 *i M* and 0 1 *j N* (2)

\* A U T U (3)

\* Q A Q H is first factorized (4)

*I* (5)

(6)

1 1 \*

*T kl*

Fig. 1. Energy distribution in the spatial and transformed domains.

Then the matrices Q and H are assigned the following values:

following iterations to set the lower triangular to zero is performed as follows:

For j=1, 2, 3… until convergence. Select a shift µ then the QR factorization is as follows:

VR H

\* H V H V Q QV <sup>36</sup> <sup>4</sup> 6 5

0 0 (, ) ( ,) (, )

*M N*

*i j Ii j I klT i j* 

watermarking process.

,

In this scheme, V is unitary and R is upper triangular (i.e., the QR factorization of H *I* ). It is easy to see that H is unitarily similar to A throughout the course of this iteration. The iteration is continued until the sub-diagonal elements of H converge to zero, i.e., until the Schur decomposition has been (approximately) obtained. To summarize these mathematical steps, we can say that in finding the eigenvalues of a matrix using the QR algorithm, the matrix is first transformed by a unitary similarity transformation to upper Hessenberg form. The QR algorithm then iteratively generates a sequence of upper Hessenberg matrices by unitary similarity transformations. For general real matrices, \* Q becomes <sup>T</sup> Q and the routine reduces the real general matrix A to the upper Hessenberg form H by this orthogonal similarity transformation:

$$\mathbf{A} = \mathbf{Q} \,\mathrm{H} \mathbf{Q}^{\mathrm{T}}\text{, where } \mathbf{Q} \text{ is a unitary matrix so that } \quad \mathbf{Q}^{\mathrm{T}} \mathbf{Q} = \mathrm{I}\left(\mathrm{size}(\mathbf{A})\right) \tag{7}$$

I: denotes the identity matrix and *<sup>T</sup> Q* represents the unitary matrix transpose. The general matrix structure produced by the routine is presented by the following equations as *H(i, j) i j i j -* 0 unless or 1 and:

$$\mathbf{H} = \begin{bmatrix} \mathbf{a}\_{11} & \mathbf{a}\_{12} & \cdot & \cdot & \mathbf{a}\_{1n} \\ 0 & \mathbf{a}\_{22} & & \mathbf{a}\_{2n} \\ \cdot & & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot \\ 0 & \cdot & 0 & \mathbf{a}\_{\text{rm}} \end{bmatrix} \tag{8}$$

The form of H will be tri-diagonal if the processed matrix is symmetric or Hermetian. In general, the mathematical application of this transformation serves two purposes: the first is to obtain a matrix having the same eigenvalues as the original one, but which requires less computation to reveal them; conversely, the second is the packed storage. In fact, a triangular matrix may be stored more compactly, if the relevant triangle is packed by columns in a one-dimensional array.

The aij elements of H are stored in the one-dimensional array W as:

$$a\_{ij} = \mathcal{W}\{i + j\left(j - 1\right)/2\} \quad \text{for} \quad \text{i} \le \text{j} \tag{9}$$

In this work, the produced triangular Hessenberg matrix is exploited as a new representation domain of the image where the watermark is embedded. By analyzing this matrix, we discovered the existence of an exploitable zone. Embedding a watermark in this zone produces no effect on the original matrix values after applying the inverse Hessenberg transform. This zone provides imperceptibility and robustness to the watermark. To take advantage of this characteristic, we apply this transformation to the image matrix to obtain the Hessenberg triangular representation of the image. This matrix is processed to find in which zones we can embed a watermark without any change produced on the original image. This means that we can change and increase the values of this zone in the Hessenberg matrix, without any effect being produced on the original image after the inverse Hessenberg transform is applied. The following section presents the study applied to this matrix to identify this zone called insensitive zone.

2D Watermarking: Non Conventional Approaches 165

exploit the previous detailed characteristics to determine different Hessenberg matrix bands that can characterize the Hessenberg transform with respect to their effect on the image quality if a watermark is embedded. In this study the processed blocks are rectangles of 5 pixels high and 20 pixels wide. They are respectively examined with an overlap of 10 pixels from left to right and 2 pixels from top to bottom. In order to determine the bands or zones that produces the same effect on the image if they are processed, the study consists of substituting each selected block B by a supposed small watermark W and then applies the inverse Hessenberg transform in order to come back to the spatial representation and view the effects of each block change on the image distortion. After sliding over the entire matrix and testing all the blocks, we set the limits of the insensitive zone. This zone represents the matrix sectors for which values change or increase does not affect the original image. It is exploited for watermarking purpose. This study revealed the existence of other three zones

detailed in the next section.

Fig. 4. A 3D distribution of the H matrix values

Fig. 5. Representation of the decreasing H values.

#### **2.2.2 Hessenberg matrix study**

Successive iterations as shown in the following figure decreases the original matrix values from upper region to the lower one.

Fig. 2. Successive iterations of the H matrix.

These iterations tends to set the values of the lower triangular part of the Hessenberg matrix that we will call H matrix to zero. As a consequence, the values of the upper part of the Hessenberg matrix follow the sense of this decrease. The values of the matrix H decrease from the upper left sub-diagonal to the lower right sub-diagonal as shown in Figure 3, 4 and 5, containing a pick in the upper left part which values are in the range of 2 to 3.104 as shown in figure 6. The first figures 3, 4 and 5 are illustrated without this pick.

Fig. 3. Direction of decreasing matrix values.

The unidirectional value decrease is an important characteristic of this transformation. In fact, it is helpful in the matrix zones study. It allows the upper triangular part of the matrix to be divided into different blocks, in the way of the decreasing values. In analogy with the image matrix, the energy of the image is concentrated in the mid-high and high part of the transformed Hessenberg matrix. The mid-low and lower part of this transformed matrix contains a low energy of the image while its values are the weaker in the matrix. While the Hessenberg transformation is an orthogonal transformation similarly to DCT, we will

Successive iterations as shown in the following figure decreases the original matrix values

These iterations tends to set the values of the lower triangular part of the Hessenberg matrix that we will call H matrix to zero. As a consequence, the values of the upper part of the Hessenberg matrix follow the sense of this decrease. The values of the matrix H decrease from the upper left sub-diagonal to the lower right sub-diagonal as shown in Figure 3, 4 and 5, containing a pick in the upper left part which values are in the range of 2 to 3.104 as

The unidirectional value decrease is an important characteristic of this transformation. In fact, it is helpful in the matrix zones study. It allows the upper triangular part of the matrix to be divided into different blocks, in the way of the decreasing values. In analogy with the image matrix, the energy of the image is concentrated in the mid-high and high part of the transformed Hessenberg matrix. The mid-low and lower part of this transformed matrix contains a low energy of the image while its values are the weaker in the matrix. While the Hessenberg transformation is an orthogonal transformation similarly to DCT, we will

shown in figure 6. The first figures 3, 4 and 5 are illustrated without this pick.

**2.2.2 Hessenberg matrix study** 

from upper region to the lower one.

Fig. 2. Successive iterations of the H matrix.

Fig. 3. Direction of decreasing matrix values.

exploit the previous detailed characteristics to determine different Hessenberg matrix bands that can characterize the Hessenberg transform with respect to their effect on the image quality if a watermark is embedded. In this study the processed blocks are rectangles of 5 pixels high and 20 pixels wide. They are respectively examined with an overlap of 10 pixels from left to right and 2 pixels from top to bottom. In order to determine the bands or zones that produces the same effect on the image if they are processed, the study consists of substituting each selected block B by a supposed small watermark W and then applies the inverse Hessenberg transform in order to come back to the spatial representation and view the effects of each block change on the image distortion. After sliding over the entire matrix and testing all the blocks, we set the limits of the insensitive zone. This zone represents the matrix sectors for which values change or increase does not affect the original image. It is exploited for watermarking purpose. This study revealed the existence of other three zones detailed in the next section.

Fig. 4. A 3D distribution of the H matrix values

Fig. 5. Representation of the decreasing H values.

2D Watermarking: Non Conventional Approaches 167

To preserve the image quality and to guarantee that the embedded watermark is kept imperceptible, it is important to choose "very well" the matrix zone where blocks are substituted. Indeed from this zones study, a partition of the Hessenberg matrix in four zones is carried out with respect to the image quality change due to the block substitution, as

*'*

Fig. 7. a) The matrix blocks substitution.

Fig. 7. b) The matrix partition.

Fig. 8. Matrix delimited zones.

shown in Figures 7 and 8.

 <sup>123</sup> *' k*

*H B , B , B , ..., B , ..., B <sup>n</sup>* (14)

Fig. 6. Representation of the pick in the H matrix.

#### **2.2.3 The watermarking process**

The watermarking process consists in substituting one or multiple blocks in the Hessenberg matrix with a watermark. Once these blocks are substituted, the inverse transform is applied to the watermarked matrix in order to return back to the original image hiding the inserted watermark. To simplify the watermarking process, the watermark consists in applying a non-random permutation function on the chosen block values. The same function turns the obtained values to the nearest integer. The embedded watermark is then multiplied by a gain factor. This factor is chosen with respect to the limits of the image quality preserve. The substitution procedure is shown in Figure 6 and in equations (10)-(14).

Let B be the designed block, H the Hessenberg matrix, n the number of blocks that partitions the matrix and B'k the watermark block:

$$H = \begin{Bmatrix} B\_i \end{Bmatrix} \text{ as } i \in \left[1, n\right] \tag{10}$$

H is composed by a set of n associated blocks:

$$\mathbf{H} = \left\{ \mathbf{B}\_{1'} \, \mathbf{B}\_{2'} \, \mathbf{B}\_{3'} \, \dots \, \mathbf{B}\_{k'} \dots \mathbf{B}\_{n'} \right\} \tag{11}$$

$$\mathcal{W} = A \begin{pmatrix} B\_k \end{pmatrix} \tag{12}$$

A is a function used to apply a non-random mixture on the block values and turning them to the nearest integer value, W is the obtained watermark having the same size as Bk.

$$B\_k^\top = \mathbb{K} \ w$$

K is a gain factor used to increase the watermark values in order to add more resistance against attacks. The transformed Hessenberg matrix *' H* that holds the substituted watermark becomes:

The watermarking process consists in substituting one or multiple blocks in the Hessenberg matrix with a watermark. Once these blocks are substituted, the inverse transform is applied to the watermarked matrix in order to return back to the original image hiding the inserted watermark. To simplify the watermarking process, the watermark consists in applying a non-random permutation function on the chosen block values. The same function turns the obtained values to the nearest integer. The embedded watermark is then multiplied by a gain factor. This factor is chosen with respect to the limits of the image quality preserve. The

Let B be the designed block, H the Hessenberg matrix, n the number of blocks that partitions

*W A B*  -

A is a function used to apply a non-random mixture on the block values and turning them

K is a gain factor used to increase the watermark values in order to add more resistance against attacks. The transformed Hessenberg matrix *' H* that holds the substituted

to the nearest integer value, W is the obtained watermark having the same size as Bk. *'*

*H B as i n* \* *<sup>i</sup>* 1, (10)

*<sup>k</sup>* (12)

*B K w k* (13)

H B , B , B , ..., B , ..., B 1 2 3 k n (11)

substitution procedure is shown in Figure 6 and in equations (10)-(14).

Fig. 6. Representation of the pick in the H matrix.

**2.2.3 The watermarking process** 

the matrix and B'k the watermark block:

watermark becomes:

H is composed by a set of n associated blocks:

$$\boldsymbol{H}^{'} = \left\{ \boldsymbol{B}\_{\;1\prime}, \boldsymbol{B}\_{\;2\prime}, \boldsymbol{B}\_{\;3\prime\cdots}, \boldsymbol{B}\_{\;\;1\prime}^{'}, \dots, \boldsymbol{B}\_{\;n\prime} \right\} \tag{14}$$

To preserve the image quality and to guarantee that the embedded watermark is kept imperceptible, it is important to choose "very well" the matrix zone where blocks are substituted. Indeed from this zones study, a partition of the Hessenberg matrix in four zones is carried out with respect to the image quality change due to the block substitution, as shown in Figures 7 and 8.

Fig. 7. a) The matrix blocks substitution.

Fig. 7. b) The matrix partition.

Fig. 8. Matrix delimited zones.

2D Watermarking: Non Conventional Approaches 169

 -' T -

is the watermarked image, H' the watermarked Hessenberg matrix and Q is the

*I inv Q <sup>W</sup>* H inv Q (15)


unitary matrix. The watermark-embedding algorithm is presented in Figure 5.


Fig. 9. The embedded watermark in the Hessenberg matrix.

Fig. 10. Block watermark substituted in the fourth zone of the H matrix.

Where *IW* ---

The Hessenberg transform is an entirely reversible transform that brings the image from the spatial representation to the H matrix and come back without any information loss or change. If this transform is applied on the image several times the same matrices are generated. This data preservation allows us to apply it on the image and use its transformed matrix for watermarking purposes. The Hessenberg matrix is divided in four zones. Each zone produces a specific distortion effect on the image if it is modified. The first zone containing the highest values in the transformed matrix and then contains the main part of the image energy. This zone is represented by the first twenty lines from the top and 150 columns from the left. Substituting blocks in this region of the Hessenberg matrix, affects considerably the image quality by adding to it a distortion look as presented in section 4. The second zone is situated between the first twenty lines from the top and a width between columns 150 and 256; this is a sensitive zone for image watermarking because any change in its values damages the original image considerably by adding to it a blurred effect. The intensity of the blur depends on the gain factor strength used. The first and second zones at the top and the fourth zone at the bottom delimit the third zone shown in Figure 8. In this zone, any watermark embedding followed by an inverse Hessenberg transform, damages the image by adding a noise effect to it. In fact the forms and shapes in the obtained image do no change, but a noise speck appears locally or in the entire image depending on the position of the substituted blocks. The intensity of this noise depends essentially on the magnitude of the gain factor used. Of course in these three mentioned zones if a low gain factor is used as it preserves the range of the watermark values near to the original matrix coefficients no change appears in the image. The fourth zone, which is red, is delimited between lines 160 to 256 in width and between columns 155 to 256 in height, excluding the lower triangle zero values. This zone contains the lower values of the matrix as presented in figure 1c; week energy of the image is transformed and spread in this zone. Because of the decreasing values direction in the Hessenberg matrix, this zone has the smallest values in the entire matrix. It is found to have an insignificant effect on the image quality whatever the change in its values and the increase in the gain factor. In this zone, different sectors can be substituted by a watermark. In our simulations, the sector delimited between lines 220 to 253 and columns 230 to 256 is used. As a result, the watermark insertion will consist in embedding a watermark block in this fourth Hessenberg zone as shown in figure 9 and 10. The gains factor increases up to 35 without any distortions to the watermarked image. Then, embedding a watermark in this zone allows us to increase the watermark robustness against different attacks. The visual imperceptibility threshold is not exceeded and the image quality is preserved. To supply more security, a secret key is provided to the copyright owner, used to determine the position of the watermarked sector.

Based on the detailed previous study, the choice of the embedding zone and the selected block to be substituted with the watermark block is chosen. It must increase the robustness of the embedded watermark against a large set of attacks and decreases the distortions introduced to the watermarked image. The fourth zone in the Hessenberg matrix is found as it satisfies these essential constraints to develop the watermarking algorithm. After watermarking the Hessenberg matrix, an inverse process shown by the following equation is applied to come back to the spatial representation of the image as follows:

The Hessenberg transform is an entirely reversible transform that brings the image from the spatial representation to the H matrix and come back without any information loss or change. If this transform is applied on the image several times the same matrices are generated. This data preservation allows us to apply it on the image and use its transformed matrix for watermarking purposes. The Hessenberg matrix is divided in four zones. Each zone produces a specific distortion effect on the image if it is modified. The first zone containing the highest values in the transformed matrix and then contains the main part of the image energy. This zone is represented by the first twenty lines from the top and 150 columns from the left. Substituting blocks in this region of the Hessenberg matrix, affects considerably the image quality by adding to it a distortion look as presented in section 4. The second zone is situated between the first twenty lines from the top and a width between columns 150 and 256; this is a sensitive zone for image watermarking because any change in its values damages the original image considerably by adding to it a blurred effect. The intensity of the blur depends on the gain factor strength used. The first and second zones at the top and the fourth zone at the bottom delimit the third zone shown in Figure 8. In this zone, any watermark embedding followed by an inverse Hessenberg transform, damages the image by adding a noise effect to it. In fact the forms and shapes in the obtained image do no change, but a noise speck appears locally or in the entire image depending on the position of the substituted blocks. The intensity of this noise depends essentially on the magnitude of the gain factor used. Of course in these three mentioned zones if a low gain factor is used as it preserves the range of the watermark values near to the original matrix coefficients no change appears in the image. The fourth zone, which is red, is delimited between lines 160 to 256 in width and between columns 155 to 256 in height, excluding the lower triangle zero values. This zone contains the lower values of the matrix as presented in figure 1c; week energy of the image is transformed and spread in this zone. Because of the decreasing values direction in the Hessenberg matrix, this zone has the smallest values in the entire matrix. It is found to have an insignificant effect on the image quality whatever the change in its values and the increase in the gain factor. In this zone, different sectors can be substituted by a watermark. In our simulations, the sector delimited between lines 220 to 253 and columns 230 to 256 is used. As a result, the watermark insertion will consist in embedding a watermark block in this fourth Hessenberg zone as shown in figure 9 and 10. The gains factor increases up to 35 without any distortions to the watermarked image. Then, embedding a watermark in this zone allows us to increase the watermark robustness against different attacks. The visual imperceptibility threshold is not exceeded and the image quality is preserved. To supply more security, a secret key is provided to the copyright owner, used to determine the position of the watermarked

Based on the detailed previous study, the choice of the embedding zone and the selected block to be substituted with the watermark block is chosen. It must increase the robustness of the embedded watermark against a large set of attacks and decreases the distortions introduced to the watermarked image. The fourth zone in the Hessenberg matrix is found as it satisfies these essential constraints to develop the watermarking algorithm. After watermarking the Hessenberg matrix, an inverse process shown by the following equation

is applied to come back to the spatial representation of the image as follows:

sector.

$$\dddot{I}\_W = \text{inv}(\mathbf{Q}) \text{ H}^\prime \text{inv}(\mathbf{Q}^\text{T}) \tag{15}$$

Where *IW* -- is the watermarked image, H' the watermarked Hessenberg matrix and Q is the unitary matrix. The watermark-embedding algorithm is presented in Figure 5.

Fig. 9. The embedded watermark in the Hessenberg matrix.

Fig. 10. Block watermark substituted in the fourth zone of the H matrix.

2D Watermarking: Non Conventional Approaches 171

total number of pixels in the watermarked image and T is the detection threshold over which the watermark is set as detected. While the location of the watermark in the Hessenberg matrix is known by the use of the secret key, the second measure based on the normalized correlation is more appropriate in this work. This measure is presented in (17),

1 1

,,

11 11 


10 255 255 *PSNR* 10log *MSE*

1

1 *<sup>N</sup>*

*i MSE I I N* , -

Where MSE is the mean square error, N is the total number of pixels in the image, I and *IW*

are the original and watermarked image. With K is used equal to 35 and embedding in the fort zone. The PSNR of the watermarked image is maintained in the range of 40–50 dB (so

 2

(18)


% (19)

*i Wi*


To test the robustness and the improvement that our method offers, different STIRMARK attacks are applied on the watermarked image. Once the image is attacked, a correlation is computed between the original watermark block and the attacked one. A threshold Th fixed equal to 0.85 is applied to decide whether the watermark is detected or not. This threshold is chosen as the mean of the total correlation values corresponding to all synchronous and asynchronous attacks applied on the watermarked image in the simulation study. Some of these tests that provide week results corresponding to some geometrical attacks are not displayed in Table 1. The used attacks and gathered correlation results are detailed in Table 1. The chosen sector and applied gain factor, giving the best correlation result without causing any visible distortions between the original image and the watermarked one, are shown in the same table. The results prove the high resistance of this method against different attacks, especially JPEG compression, noise adding and convolutions filtering Stirmark attacks. In addition, an improvement against other attacks is noted when compared with other current techniques as shown in figures 18 and 19. Various examples of those attacks are simulated below on the original cameraman image followed by the corresponding peak detection of the watermark block between 1000 other random blocks. The watermark embedding capacity depends on the required embedding procedure. For a robust watermark embedding only the fourth Hessenberg zone is used. We can embed in the other zones for a fragile watermark embedding method using a low gain factor that avoids damaging the image quality. To establish a more quantitative measure of imperceptibility, we make use of the peak signal to noise ration (PSNR) metric. This measure serves generally as a good rule for the watermark visibility estimation, given by:

,, ,,

*nm nm*

*n m '*

2 '2

*W W*


*W W*

I are the watermark and image variances, N is the

(17)

Where -

 <sup>2</sup> <sup>1</sup> -t /2 2 e dt *x*

where n and m are the watermark block size.

<sup>7</sup> ,

*<sup>w</sup>* and

*Corr*

*erfc <sup>x</sup>*

Fig. 11. The Hessenberg watermark embedding algorithm.

#### **2.3 Watermark recovery and tests**

In order to test the watermark presence, the Hessenberg transformation is applied on the watermarked image to handle the transformed Hessenberg matrix. The secret key is used to detect the position of the watermarked sector. Once this sector is located, the similarity between the extracted and the original watermark (W' and W) is determined. The watermark extraction procedure is detailed in figure 6. Two measures of performance can be computed, the first is the (BER) bit error rate as false positive or false negative detection errors. The false positive detection is a false alarm of an incorrect detected watermark. While the false negative detection error consists in a missed detection of an existent watermark. The probabilities of these two types of errors are derived based on a first-order autoregressive image model as shown by:

$$P\_{fp} = \frac{1}{2} \operatorname{erfc} \left( \frac{T \sqrt{N}}{\sigma\_w \, \sigma\_1 \sqrt{2}} \right) \\ \text{And } \; P\_{fp} = \frac{1}{2} \operatorname{erfc} \left( \frac{(\sigma\_w^2 - T) \sqrt{N}}{\sigma\_1 \sqrt{2}} \right) \tag{16}$$

*Zones Selection*

*Watermark Block* 

*insertion* **Applied** 

*Block Selection*

*Some randomized elements* 

> **embedding strength**

*Hessenberg Matrix* 

> **Changed matrix**

*Inverse Hessenberg Transformation* 

> *Watermarked Image*

Fig. 11. The Hessenberg watermark embedding algorithm.

I

. 9 / :

In order to test the watermark presence, the Hessenberg transformation is applied on the watermarked image to handle the transformed Hessenberg matrix. The secret key is used to detect the position of the watermarked sector. Once this sector is located, the similarity between the extracted and the original watermark (W' and W) is determined. The watermark extraction procedure is detailed in figure 6. Two measures of performance can be computed, the first is the (BER) bit error rate as false positive or false negative detection errors. The false positive detection is a false alarm of an incorrect detected watermark. While the false negative detection error consists in a missed detection of an existent watermark. The probabilities of these two types of errors are derived based on a first-order

And -

*fp*

1 

*P erfc*

<sup>2</sup>

*T*

. 9 / : (16)

2 2 *w*

*N*


I

**2.3 Watermark recovery and tests** 

*Hessenberg Transformation* 

**Unitary matrixes**

**Matrix partitioning** 

autoregressive image model as shown by:

1 

<sup>2</sup> 2 *fp w*


*T N P erfc* 

Where - <sup>2</sup> <sup>1</sup> -t /2 2 e dt *x erfc <sup>x</sup>* <sup>7</sup> , *<sup>w</sup>* and I are the watermark and image variances, N is the

total number of pixels in the watermarked image and T is the detection threshold over which the watermark is set as detected. While the location of the watermark in the Hessenberg matrix is known by the use of the secret key, the second measure based on the normalized correlation is more appropriate in this work. This measure is presented in (17), where n and m are the watermark block size.

$$Corr = \frac{\sum\_{1}^{n} \sum\_{1}^{m} \text{W } \mathbf{W'}}{\sqrt{\left(\sum\_{1}^{n} \sum\_{1}^{m} \mathbf{W'^2}\right) \sum\_{1}^{n} \left(\sum\_{1}^{m} \mathbf{W'^2}\right)}} \tag{17}$$

To test the robustness and the improvement that our method offers, different STIRMARK attacks are applied on the watermarked image. Once the image is attacked, a correlation is computed between the original watermark block and the attacked one. A threshold Th fixed equal to 0.85 is applied to decide whether the watermark is detected or not. This threshold is chosen as the mean of the total correlation values corresponding to all synchronous and asynchronous attacks applied on the watermarked image in the simulation study. Some of these tests that provide week results corresponding to some geometrical attacks are not displayed in Table 1. The used attacks and gathered correlation results are detailed in Table 1. The chosen sector and applied gain factor, giving the best correlation result without causing any visible distortions between the original image and the watermarked one, are shown in the same table. The results prove the high resistance of this method against different attacks, especially JPEG compression, noise adding and convolutions filtering Stirmark attacks. In addition, an improvement against other attacks is noted when compared with other current techniques as shown in figures 18 and 19. Various examples of those attacks are simulated below on the original cameraman image followed by the corresponding peak detection of the watermark block between 1000 other random blocks. The watermark embedding capacity depends on the required embedding procedure. For a robust watermark embedding only the fourth Hessenberg zone is used. We can embed in the other zones for a fragile watermark embedding method using a low gain factor that avoids damaging the image quality. To establish a more quantitative measure of imperceptibility, we make use of the peak signal to noise ration (PSNR) metric. This measure serves generally as a good rule for the watermark visibility estimation, given by:

$$MSE = \frac{1}{N} \sum\_{i=1}^{N} \left( I\_i - \dddot{I}\_{\text{Vi}i} \right)^2 \tag{18}$$

$$PSNR = 10\log\_{10}\frac{255 \times 255}{MSE} \tag{19}$$

Where MSE is the mean square error, N is the total number of pixels in the image, I and *IW* --- are the original and watermarked image. With K is used equal to 35 and embedding in the fort zone. The PSNR of the watermarked image is maintained in the range of 40–50 dB (so

2D Watermarking: Non Conventional Approaches 173

Fig. 12. The watermark blind detection algorithm after the applied attacks.

The results prove the high resistance of this method against different kinds of attacks such as: JPEG compression, noise, rotation, affine transform and other Stirmark attacks. Using this method we exploit the advantages of being robust face to different kind of attacks in the same time we have the guaranty of a secure and undetectable watermark with a rapid embedding and extraction algorithm. After applying the Hessenberg transform on different attacked images we note the values of the H matrix are nearly invariant if the fixed gain factor in the embedding procedure is not exceeded. This explains the fact that the watermark is always detected. The maximum error values resulting from the difference between the H matrix of a watermarked image and this of a watermarked and attacked image by JPEG compression indexed 40 are contained in the range between 7 and 9 in the entire matrix excluding the upper left matrix corner. The matrix resulting of this difference is shown in figures 13. This error pick value varies in the range of 2.103. With regard to this error value occurring in the upper part of the first zone, the error resulting in the zones 2 and 3 represents a ratio limited between 4 and 5.10-3. The fourth zone presents an error band between the watermarked image and the attacked one contained in the range between -2 and 3 as shown in figure 14. This means that the numerical difference resulting between the watermarked image and the watermarked attacked one in this zone is too week. This difference presents the error resulting from the applied attack. This resulting error presents a ratio of 1.10-3 when compared with the error pick in the first zone. This week variation in the embedding zone with regards to the other zones composing the H matrix preserves the watermark from loss. On the other hand, the mathematical characteristics concerning the successive iterations of the Hessenberg transform bringing the values of the lower triangular part to zero and transforms the image matrix into a triangular matrix which absolute values varies from 2.104 to 0. We can say that the energy of the image is concentrated in the upper and middle zones. That is why embedding in the fourth zone introduces a week energy to

**2.4 Experimental results and discussion** 



Table 1. Watermark detection responses values (Substituted Block dimension and location: [220:253,230:256], Gain factor: K = 35).


Table 2. PSNR variation against different JPEG quality factor attacks.


Table 3. PSNR variation against Median filtering attack.


Table 4. PSNR variation against different noise level adding attack.

The PSNR is also computed between the original image and the different watermarked attacked images. The PSNR variation against different attacks level as JPEG compression, median filtering and noise adding are present in tables 2, 3 and 4. The watermark extracting using the inverse procedure is shown by figure 12.

 Convolution filter 1 0.9138 **2** Adding Noise 0 1 Convolution filter 2 0.9994 **4** Adding Noise 20 0.9662 Compression JPEG 20 0.9603 **6** Adding Noise 40 0.8811 Compression JPEG 40 0.9704 **8** Adding Noise 80 0.7671 Compression JPEG 60 0.9802 **10** PSNR 10 1 Compression JPEG 70 0.9891 **12** PSNR 50 1 Compression JPEG 80 0.9925 **14** Remove lines 60 0.8412 Compression JPEG 90 0.9999 **16** Rotation 2° 0.7789 Compression JPEG 100 1 **18** Rotation 45° 0.8641 MEDIANCUT 5 0.7538 **20** Rotation 90° 0.9124 MEDIANCUT 7 0.8634 **22** Affine 7 0.9876 MEDIANCUT 9 0.7565 **24** Affine 5 0.9941 Table 1. Watermark detection responses values (Substituted Block dimension and location:

image, 49.70 dB using the image door and 44.62 dB using the image blood.

STIRMARK ATTACKS Correlation

[220:253,230:256], Gain factor: K = 35).

JPEG quality factor

 is visually indistinguishable from I). Generally if the distortions between two images output a PSNR higher than 35 dB, no differences are visually detected. In this work, the computed PSNR from the experimental study is equal to 44.55 dB using the cameraman

20 40 60 70 80 90 100

PSNR (dB) 23.943 24.581 24.979 25.168 25.373 25.607 25.760

Median Filter size 5×5 7×7 9×9 PSNR (dB) 21.528 20.831 20.381

Noise level 20 40 60 80 PSNR (dB) 25.763 8.374 6.826 6.015

The PSNR is also computed between the original image and the different watermarked attacked images. The PSNR variation against different attacks level as JPEG compression, median filtering and noise adding are present in tables 2, 3 and 4. The watermark extracting

Table 2. PSNR variation against different JPEG quality factor attacks.

Table 4. PSNR variation against different noise level adding attack.

Table 3. PSNR variation against Median filtering attack.

using the inverse procedure is shown by figure 12.

values STIRMARK ATTACKS Correlation

values

that *IW* ---

Fig. 12. The watermark blind detection algorithm after the applied attacks.

#### **2.4 Experimental results and discussion**

The results prove the high resistance of this method against different kinds of attacks such as: JPEG compression, noise, rotation, affine transform and other Stirmark attacks. Using this method we exploit the advantages of being robust face to different kind of attacks in the same time we have the guaranty of a secure and undetectable watermark with a rapid embedding and extraction algorithm. After applying the Hessenberg transform on different attacked images we note the values of the H matrix are nearly invariant if the fixed gain factor in the embedding procedure is not exceeded. This explains the fact that the watermark is always detected. The maximum error values resulting from the difference between the H matrix of a watermarked image and this of a watermarked and attacked image by JPEG compression indexed 40 are contained in the range between 7 and 9 in the entire matrix excluding the upper left matrix corner. The matrix resulting of this difference is shown in figures 13. This error pick value varies in the range of 2.103. With regard to this error value occurring in the upper part of the first zone, the error resulting in the zones 2 and 3 represents a ratio limited between 4 and 5.10-3. The fourth zone presents an error band between the watermarked image and the attacked one contained in the range between -2 and 3 as shown in figure 14. This means that the numerical difference resulting between the watermarked image and the watermarked attacked one in this zone is too week. This difference presents the error resulting from the applied attack. This resulting error presents a ratio of 1.10-3 when compared with the error pick in the first zone. This week variation in the embedding zone with regards to the other zones composing the H matrix preserves the watermark from loss. On the other hand, the mathematical characteristics concerning the successive iterations of the Hessenberg transform bringing the values of the lower triangular part to zero and transforms the image matrix into a triangular matrix which absolute values varies from 2.104 to 0. We can say that the energy of the image is concentrated in the upper and middle zones. That is why embedding in the fourth zone introduces a week energy to

2D Watermarking: Non Conventional Approaches 175

Fig. 15. Watermark presence in the H matrix after JPEG 40 attack

Fig. 16. Watermark presence after Convolution filtering 2 attack.

JPEG 100 to JPEG 20.

As shown in figure 18, the proposed method is also more robust to JPEG compression than recent algorithms. From JPEG 70 to JPEG 50 the CHEN algorithm is slightly higher then our algorithm, Nevertheless, when using high compression rates such as JPEG 20 and JPEG 10 the proposed method maintain its robustness by preserving the embedded watermark. The other methods lose their resistance face to this destructive attack. In the same time, when compared with other methods using the DWT domain, high robustness against noise adding is presented. The additional robustness to convolution filtering and different geometrical distortions is also presented. The proposed Hessenberg method provides also a better robustness face to median filtering then many other techniques as shown in figure 19. Figure 17, presents the correlations values computed between the extracted watermark after seven JPEG compression attacks and the original one, with different quality factors from

the image and then a high embedding strength is needed to introduce distortions to the watermarked image.

Fig. 13. Difference resultant between two *H* matrices belonging respectively to a Watermarked image and JPEG 40 lossy compression attacked image.

Figure 15 and 16 illustrate the presence of the watermark in the fourth zones after the watermarked image has been attacked by JPEG 40 compression and convolution 2 filtering. Because of the very week variation of the Hessenberg values in the watermarking zones if compared with the others matrix zones, the watermark is always preserved from loss after attacks.

Fig. 14. Error band between the H matrices corresponding to zones 2, 3 and 4.

the image and then a high embedding strength is needed to introduce distortions to the

Fig. 13. Difference resultant between two *H* matrices belonging respectively to a

Fig. 14. Error band between the H matrices corresponding to zones 2, 3 and 4.

Figure 15 and 16 illustrate the presence of the watermark in the fourth zones after the watermarked image has been attacked by JPEG 40 compression and convolution 2 filtering. Because of the very week variation of the Hessenberg values in the watermarking zones if compared with the others matrix zones, the watermark is always preserved from loss after

Watermarked image and JPEG 40 lossy compression attacked image.

watermarked image.

attacks.

Fig. 15. Watermark presence in the H matrix after JPEG 40 attack

Fig. 16. Watermark presence after Convolution filtering 2 attack.

As shown in figure 18, the proposed method is also more robust to JPEG compression than recent algorithms. From JPEG 70 to JPEG 50 the CHEN algorithm is slightly higher then our algorithm, Nevertheless, when using high compression rates such as JPEG 20 and JPEG 10 the proposed method maintain its robustness by preserving the embedded watermark. The other methods lose their resistance face to this destructive attack. In the same time, when compared with other methods using the DWT domain, high robustness against noise adding is presented. The additional robustness to convolution filtering and different geometrical distortions is also presented. The proposed Hessenberg method provides also a better robustness face to median filtering then many other techniques as shown in figure 19.

Figure 17, presents the correlations values computed between the extracted watermark after seven JPEG compression attacks and the original one, with different quality factors from JPEG 100 to JPEG 20.

2D Watermarking: Non Conventional Approaches 177

against synchronous attacks than asynchronous ones presenting high degrees and levels of distortions. Using the Hessenberg domain, we can provide more robustness against different sets of intentional and malicious possible attacks. Various examples of these attacks are simulated below on the original cameraman image followed by the corresponding correlation values of the detected watermark block between 1000 other matrix blocks. For a robust watermark embedding only the fourth Hessenberg zone is used. We can embed in the other zones for a fragile watermark embedding method using a low gain factor to avoid damaging the image quality. In addition the image appearance can be changed by modifying the

Figures 20a, 20b, 21a, and 21b show applied convolution filters attack and the responses of the watermark detector to 1000 random blocks of the watermarked Hessenberg image matrix. The positive response due to the correct watermark block is much stronger than with incorrect blocks. The detection rates of these attacks are very high when compared

Hessenberg blocks matrix with respect to the zone location.

Fig. 19. Robustness against Median Filtering.

Fig. 20. a) Convolution 1 attack.

**2.4.1 Experiment 1: Convolution filtering 1 and 2 attacks** 

Fig. 17. Robustness of the Hessenberg technique against JPEG compression quality.

Fig. 18. Robustness variation face to JPEG compression.

As we shown in table 1 this technique is visibly more robust against synchronous attacks than geometrical ones. In fact the synchronous attacks applied on the image modify the values of its intensity values. After applying the Hessenberg transform on it we will extract the same H matrix with a variation of its values from a zone to another. Of course as we demonstrated in the previous section, the fourth zone has the less variation in the entire matrix and that is why we find high robustness if we embed in this zone. Conversely, if an asynchronous attack is applied on the watermarked image a change in the pixels position will happen. Since the Hessenberg transformation is a block-based orthogonal transform, the change in the image pixels position will be reflected on the Hessenberg matrix. The position of the values of the H matrix corresponding to the location of the watermark will be modified by this geometrical transformation increasing the error on the watermark extraction. As the watermark block will be extracted from the same location in the H matrix, it will contain some wrong values introduced by the change of the values position. For this reason, this method is more robust

Fig. 17. Robustness of the Hessenberg technique against JPEG compression quality.

As we shown in table 1 this technique is visibly more robust against synchronous attacks than geometrical ones. In fact the synchronous attacks applied on the image modify the values of its intensity values. After applying the Hessenberg transform on it we will extract the same H matrix with a variation of its values from a zone to another. Of course as we demonstrated in the previous section, the fourth zone has the less variation in the entire matrix and that is why we find high robustness if we embed in this zone. Conversely, if an asynchronous attack is applied on the watermarked image a change in the pixels position will happen. Since the Hessenberg transformation is a block-based orthogonal transform, the change in the image pixels position will be reflected on the Hessenberg matrix. The position of the values of the H matrix corresponding to the location of the watermark will be modified by this geometrical transformation increasing the error on the watermark extraction. As the watermark block will be extracted from the same location in the H matrix, it will contain some wrong values introduced by the change of the values position. For this reason, this method is more robust

Fig. 18. Robustness variation face to JPEG compression.

against synchronous attacks than asynchronous ones presenting high degrees and levels of distortions. Using the Hessenberg domain, we can provide more robustness against different sets of intentional and malicious possible attacks. Various examples of these attacks are simulated below on the original cameraman image followed by the corresponding correlation values of the detected watermark block between 1000 other matrix blocks. For a robust watermark embedding only the fourth Hessenberg zone is used. We can embed in the other zones for a fragile watermark embedding method using a low gain factor to avoid damaging the image quality. In addition the image appearance can be changed by modifying the Hessenberg blocks matrix with respect to the zone location.

Fig. 19. Robustness against Median Filtering.

#### **2.4.1 Experiment 1: Convolution filtering 1 and 2 attacks**

Figures 20a, 20b, 21a, and 21b show applied convolution filters attack and the responses of the watermark detector to 1000 random blocks of the watermarked Hessenberg image matrix. The positive response due to the correct watermark block is much stronger than with incorrect blocks. The detection rates of these attacks are very high when compared

Fig. 20. a) Convolution 1 attack.

2D Watermarking: Non Conventional Approaches 179

second a sharpening filters. The parameters of the filters applied on the cameraman image as shown in Fig.20a and Fig.21a, is given as the following: CONV 1 filter = 3, 3, 9. Where the two first numbers corresponds to the filter width and high and the third number is the

1 2

2 4

1 2 1

0

1

.

Figures 22a and 22b show the results obtained after respectively applying a JPEG 20 and JPEG 60 compression on the image. This technique is found to be robust against this kind of signal processing distortion. A series of different JPEG compression rates are applied, as shown in Table 1, with high rates of watermark block detection. Using this method, the watermarked images are safe face to these unintentional signal processing attacks and the watermark can be entirely get back after this lossy

0 1

1 5

**2.4.2 Experiment 2: JPEG compression attacks** 

Fig. 22. a) JPEG 20-compression attack.

0 1 0

0 ; 1 < 1 < 1 < 1 < 1 < 1 < 1 < 12 <=

0 ; 1 < 1 < 1 < 1 < 1 < 1 < 1 < 12 <=

1

2

. The parameters of CONV 2 filter are

division factor. The matrix filter is

the same, and its matrix is:

compression.

Fig. 20. b) Watermark detection result.

Fig. 21. a) Convolution 2 attack.

Fig. 21. b) Watermark detection result.

with other watermarking domains, such as the spatial domain [8]. Two filtering attacks are proposed: the convolution 1 and 2 filters, where the first represents a gaussian filter and the

with other watermarking domains, such as the spatial domain [8]. Two filtering attacks are proposed: the convolution 1 and 2 filters, where the first represents a gaussian filter and the

Fig. 20. b) Watermark detection result.

Fig. 21. a) Convolution 2 attack.

Fig. 21. b) Watermark detection result.

second a sharpening filters. The parameters of the filters applied on the cameraman image as shown in Fig.20a and Fig.21a, is given as the following: CONV 1 filter = 3, 3, 9. Where the two first numbers corresponds to the filter width and high and the third number is the

#### **2.4.2 Experiment 2: JPEG compression attacks**

Figures 22a and 22b show the results obtained after respectively applying a JPEG 20 and JPEG 60 compression on the image. This technique is found to be robust against this kind of signal processing distortion. A series of different JPEG compression rates are applied, as shown in Table 1, with high rates of watermark block detection. Using this method, the watermarked images are safe face to these unintentional signal processing attacks and the watermark can be entirely get back after this lossy compression.

Fig. 22. a) JPEG 20-compression attack.

2D Watermarking: Non Conventional Approaches 181

Different geometrical distortions are applied as attacks to the watermarked image such as rotations and affine transforms. The rotation attacks change the position of the image pixels and break the correlation between the image and the embedded watermark. Many rotation degrees are applied between which three are showed in the table above: two, forty-five and ninety degrees rotations attacks. The second kind of geometrical distortions attacks are the affine transform. Two kinds of affine attacks are applied indexed as affine 5 and affine

 *b X*

The variables a, b, c, d, e changes with the affine transform index and fixed by the Stirmark

The proposed technique is found to be robust against this kind of signal processing distortion as shown by the correlation results in the table 1. Figures 24a and 25b show the response value of the watermark detection corresponding to the computed correlation between the original watermark block and the detected one after the applied attack, and the attacked watermarked image, respectively. The watermark block resists this image processing with a high correlation value, compared with the other matrix blocks used in the

*d*

*e*

(20)

**2.4.4 Experiment 5: Geometrical distortions** 

tool.

test.

Fig. 24. a) Affine 7 transform attack.

7.This geometrical transform is given by the equation (14).

'

*X a*

**2.4.5 Effect of changing blocks in different Hessenberg matrix zones** 

As detailed in section 2, the Hessenberg matrix is divided in different zones. The choice of the zone in which the watermark block is substituted, is very important in order to avoid a possible image characteristics and appearance change. In this section, the influence of each matrix zone is illustrated. The fourth zone, which is indifferent with regard to the image quality when a watermark is embedded in, is also shown, and different images are

' .

*<sup>Y</sup> c d Y*

Fig. 23. b) Watermark detection result.

#### **2.4.3 Experiment 4: Noise attacks**

Gaussian noise with zero mean and varied variances .Different noise magnitudes are added to the watermarked image from (0 to 80), as shown in Table 1, with the corresponding watermark detector responses. In all the cases, the watermark was not removed, and the correlations with the real watermark block were higher. Figures 23a and 23b illustrate two noise attacks (80), with the corresponding normalized watermark detector responses.

Fig. 23. a) NOISE 80 attack.

Fig. 23. b) Watermark detection result.

to the watermarked image from (0 to 80), as shown in Table 1, with the corresponding watermark detector responses. In all the cases, the watermark was not removed, and the correlations with the real watermark block were higher. Figures 23a and 23b illustrate two

noise attacks (80), with the corresponding normalized watermark detector responses.

.Different noise magnitudes are added

Fig. 23. b) Watermark detection result.

**2.4.3 Experiment 4: Noise attacks** 

Fig. 23. a) NOISE 80 attack.

Fig. 23. b) Watermark detection result.

Gaussian noise with zero mean and varied variances

#### **2.4.4 Experiment 5: Geometrical distortions**

Different geometrical distortions are applied as attacks to the watermarked image such as rotations and affine transforms. The rotation attacks change the position of the image pixels and break the correlation between the image and the embedded watermark. Many rotation degrees are applied between which three are showed in the table above: two, forty-five and ninety degrees rotations attacks. The second kind of geometrical distortions attacks are the affine transform. Two kinds of affine attacks are applied indexed as affine 5 and affine 7.This geometrical transform is given by the equation (14).

$$
\begin{vmatrix} X \\ Y \end{vmatrix} = \begin{vmatrix} a & b \\ & \\ c & d \end{vmatrix} \dots \begin{vmatrix} X \\ & \\ Y \end{vmatrix} \quad + \begin{vmatrix} d \\ & \\ e \end{vmatrix} \tag{20}
$$

The variables a, b, c, d, e changes with the affine transform index and fixed by the Stirmark tool.

The proposed technique is found to be robust against this kind of signal processing distortion as shown by the correlation results in the table 1. Figures 24a and 25b show the response value of the watermark detection corresponding to the computed correlation between the original watermark block and the detected one after the applied attack, and the attacked watermarked image, respectively. The watermark block resists this image processing with a high correlation value, compared with the other matrix blocks used in the test.

Fig. 24. a) Affine 7 transform attack.

#### **2.4.5 Effect of changing blocks in different Hessenberg matrix zones**

As detailed in section 2, the Hessenberg matrix is divided in different zones. The choice of the zone in which the watermark block is substituted, is very important in order to avoid a possible image characteristics and appearance change. In this section, the influence of each matrix zone is illustrated. The fourth zone, which is indifferent with regard to the image quality when a watermark is embedded in, is also shown, and different images are

2D Watermarking: Non Conventional Approaches 183

"Blood" image. Some other images require higher gain factors to be affected by certain changes. Figures 31h and 31i show the "Door" and "Rice" images, where a gain factor of 210 and 250 respectively, is applied. Until the gain factor reaches these high values, some visible changes begin to appear, as indicated by the arrows on the regions affected by the changes. It is clear that the gain factor used, and which is capable of causing some damage or changes to the watermarked image differs with the image type and characteristics. Of course, using a high gain factor implies higher correlation values and watermark detection between the attacked watermarked image and the original one. The Table 5 presents the different PSNR corresponding to the figures from 31a to 31i. The computed PSNR shows the distortion magnitude introduced to these test images watermarked in the fourth Hessenberg zone.

number 21a 21b 21c 21d 21e 21f 21g 21h 21i PSNR (dB) 44.72 44.68 44.60 49.70 44.41 44.32 49.65 42.72 43.12 Table 5. PSNR variation against different watermarked test images with variable embedding

Figures

strength.

Fig. 25. Original image "Blood".

Fig. 26. Original image "Door".

Fig. 24. b) Watermark detection result.

watermarked with various gain factors. All figures belonging to these simulations are detailed below in different sets. In all the sets of figures, the limits of the blocks processed are presented by [lines-limits, Columns-limits], and K represents the gain factor used.

Figures 25: "Image BLOOD", 26: "Image DOOR" and 27: "Image RICE" are the original images used throughout the simulation experiments, in addition to the "Cameraman" image shown in section 3. The images, from 28a to 28h, illustrate the results of changing blocks in the first zone applied to the "cameraman, door and blood" images with different gain factors, varying from 5 up to 35. The examples show the effect of operating in different sectors of zone 1. The damage caused to these images by changing blocks in this first zone results in local or general variable image blurring. In the set of figures, from 29a to 29f, the second zone of the transformed image matrix is changed. The simulation is applied to the different used images with different gain factors, as shown below. When changing a sector belonging to this zone, a string effect appears locally or on the entire image. The intensity of this string effect varies with the level's value of the gain used. Figures from 30a to 30i represent the results of simulations where the third zone is processed. In fact, this is an interesting zone. The result of changing blocks in zone 3 is shown in Figures 30a, 30b, 30c, 30d and 30g. Embedding a watermark in this zone affects these images by adding a non-uniform noise appearance. The intensity of this noise varies from one image region to another. Figures 30e, 30f, 30h and 30i show the effect of a uniformly distributed noise by changing the sectors detailed with the images.

In all these simulations, dealing with the fourth zone is the most interesting in this proposed Hessenberg watermarking method. From Figures 31a to 31i, the fourth zone of the transformed image matrix is changed. In fact, as we will detail in these figures, it is clear that this zone is the least sensitive to watermark embedding, and can be totally insensitive in some cases to the blocks changing. Figures 31a, 31b, 31c and 31d clearly show that changing the blocks in the fourth Hessenberg matrix zone does not affect the image quality where no visible changes are observed in the watermarked image even though the gain increases from 1 up to 35. As shown in Figure 31e when using the "Cameraman" image, some visible changes begin to appear in the upper left corner of the watermarked image as indicated by the arrow if the applied gain factor reaches the value 38. The same distortions are shown in Figure 31f, with a gain factor that reaches 50. In the figure 31g, the same gain factors is used, and the same block is changed, we note no visible changes appear in the watermarked

watermarked with various gain factors. All figures belonging to these simulations are detailed below in different sets. In all the sets of figures, the limits of the blocks processed are presented by [lines-limits, Columns-limits], and K represents the gain factor used. Figures 25: "Image BLOOD", 26: "Image DOOR" and 27: "Image RICE" are the original images used throughout the simulation experiments, in addition to the "Cameraman" image shown in section 3. The images, from 28a to 28h, illustrate the results of changing blocks in the first zone applied to the "cameraman, door and blood" images with different gain factors, varying from 5 up to 35. The examples show the effect of operating in different sectors of zone 1. The damage caused to these images by changing blocks in this first zone results in local or general variable image blurring. In the set of figures, from 29a to 29f, the second zone of the transformed image matrix is changed. The simulation is applied to the different used images with different gain factors, as shown below. When changing a sector belonging to this zone, a string effect appears locally or on the entire image. The intensity of this string effect varies with the level's value of the gain used. Figures from 30a to 30i represent the results of simulations where the third zone is processed. In fact, this is an interesting zone. The result of changing blocks in zone 3 is shown in Figures 30a, 30b, 30c, 30d and 30g. Embedding a watermark in this zone affects these images by adding a non-uniform noise appearance. The intensity of this noise varies from one image region to another. Figures 30e, 30f, 30h and 30i show the effect of a uniformly

In all these simulations, dealing with the fourth zone is the most interesting in this proposed Hessenberg watermarking method. From Figures 31a to 31i, the fourth zone of the transformed image matrix is changed. In fact, as we will detail in these figures, it is clear that this zone is the least sensitive to watermark embedding, and can be totally insensitive in some cases to the blocks changing. Figures 31a, 31b, 31c and 31d clearly show that changing the blocks in the fourth Hessenberg matrix zone does not affect the image quality where no visible changes are observed in the watermarked image even though the gain increases from 1 up to 35. As shown in Figure 31e when using the "Cameraman" image, some visible changes begin to appear in the upper left corner of the watermarked image as indicated by the arrow if the applied gain factor reaches the value 38. The same distortions are shown in Figure 31f, with a gain factor that reaches 50. In the figure 31g, the same gain factors is used, and the same block is changed, we note no visible changes appear in the watermarked

distributed noise by changing the sectors detailed with the images.

Fig. 24. b) Watermark detection result.

"Blood" image. Some other images require higher gain factors to be affected by certain changes. Figures 31h and 31i show the "Door" and "Rice" images, where a gain factor of 210 and 250 respectively, is applied. Until the gain factor reaches these high values, some visible changes begin to appear, as indicated by the arrows on the regions affected by the changes. It is clear that the gain factor used, and which is capable of causing some damage or changes to the watermarked image differs with the image type and characteristics. Of course, using a high gain factor implies higher correlation values and watermark detection between the attacked watermarked image and the original one. The Table 5 presents the different PSNR corresponding to the figures from 31a to 31i. The computed PSNR shows the distortion magnitude introduced to these test images watermarked in the fourth Hessenberg zone.


Table 5. PSNR variation against different watermarked test images with variable embedding strength.

Fig. 25. Original image "Blood".

Fig. 26. Original image "Door".

2D Watermarking: Non Conventional Approaches 185

Fig. 28. c) [1:10,10:20], K=5 (Blood).

Fig. 28. d) [1:10,10:20], K=35 (Blood).

Fig. 28. e) [1:10,10:20], K=20 (door).

Original image "Rice".

Fig. 27. The original images used in the simulations.

Fig. 28. a) [1:10,10:20], K = 5.

Fig. 28. b) [1:10,10:20], K=20.

 Original image "Rice". Fig. 27. The original images used in the simulations.

Fig. 28. a) [1:10,10:20], K = 5.

Fig. 28. b) [1:10,10:20], K=20.

Fig. 28. c) [1:10,10:20], K=5 (Blood).

Fig. 28. d) [1:10,10:20], K=35 (Blood).

Fig. 28. e) [1:10,10:20], K=20 (door).

2D Watermarking: Non Conventional Approaches 187

Fig. 29. a) [1:10,230:256], K=5.

Fig. 29. b) [1:10,230:256], K=5.

Fig. 29. c) [1:10,230:256], K=20.

Fig. 28. f) [1:10, 50:100], K=5.

Fig. 28. g) [1:10, 50:100], K=5 (blood

Fig. 28. h) [1:10, 50:100], K=35. Fig. 28. Result of changing blocks in the first zone on the image.

Fig. 28. f) [1:10, 50:100], K=5.

Fig. 28. g) [1:10, 50:100], K=5 (blood

Fig. 28. h) [1:10, 50:100], K=35.

Fig. 28. Result of changing blocks in the first zone on the image.

Fig. 29. a) [1:10,230:256], K=5.

Fig. 29. b) [1:10,230:256], K=5.

Fig. 29. c) [1:10,230:256], K=20.

2D Watermarking: Non Conventional Approaches 189

Fig. 30. a) [30:60, 60:100], K=5.

Fig. 30. b) [30:60,60:100], K=25.

Fig. 30. c) [30:60,150:256] K=5.

Fig. 29. d) [1:10,230:256], K=20.

Fig. 29. e) [1:10,230:256], K=35.

Fig. 29. f) [1:10,230:256], K=35. Fig. 29. Result of changing blocks in the second zone on the image.

Fig. 29. d) [1:10,230:256], K=20.

Fig. 29. e) [1:10,230:256], K=35.

Fig. 29. f) [1:10,230:256], K=35.

Fig. 29. Result of changing blocks in the second zone on the image.

Fig. 30. a) [30:60, 60:100], K=5.

Fig. 30. b) [30:60,60:100], K=25.

Fig. 30. c) [30:60,150:256] K=5.

2D Watermarking: Non Conventional Approaches 191

Fig. 30. g) [100:180,100:180], K=20.

Fig. 30. h) [100:180,100:180], K=35

Fig. 30. i) [100:180,100:180], K=35.

Fig. 30. Results of changing blocks in the third zone on the image.

Fig. 30. d) [30:60,150:256], K=35

Fig. 30. e) [100:180,100:180], K=5.

Fig. 30. f) [100:180,100:180], K=20.

Fig. 30. d) [30:60,150:256], K=35

Fig. 30. e) [100:180,100:180], K=5.

Fig. 30. f) [100:180,100:180], K=20.

Fig. 30. g) [100:180,100:180], K=20.

Fig. 30. h) [100:180,100:180], K=35

Fig. 30. i) [100:180,100:180], K=35. Fig. 30. Results of changing blocks in the third zone on the image.

2D Watermarking: Non Conventional Approaches 193

Fig. 31. d) K=35.

Fig. 31. e) K=38.

Fig. 31. f) K=50.

Fig. 31. a) K=5.

Fig. 31. b) K=10.

Fig. 31. c) K=20.

Fig. 31. d) K=35.

192 Watermarking – Volume 2

Fig. 31. a) K=5.

Fig. 31. b) K=10.

Fig. 31. c) K=20.

Fig. 31. e) K=38.

Fig. 31. f) K=50.

2D Watermarking: Non Conventional Approaches 195

Different constraints are required in a watermarking method, such imperceptibility and robustness. Besides, lossy JPEG compression remains the most unintended used attacks with data exchange in Internet, for size reduction. It can seriously affect the embedded watermark if the compression rate is high and the used scheme presents a weakness against this attack. So, the best solution resides in exploiting the DCT domain used in the JPEG algorithm in order to dispose of the robustness against this compression or the multiresolution domain as in. But acting to be robust against this attack reveals automatically the domains of watermark embedding and than increase the possibility of its detection. In this section, a novel watermarking method is proposed. It consists in using the parametric space of the mathematical Hough transform as a watermarking domain. The technique consists in selecting specific maximums in the Hough matrix with respect to a secret key. The peaks are found to be invariant points in the proposed Hough domain especially against lossy JPEG compression. Two signatures are considered; the first is hold in the Hough domain by the transformed space and consists in the locations of the specific chosen invariant points. Whereas, the second is represented by the use of end points of the correspondent detected lines. These end points are used as centers to embed similarities blocks in. The watermarking in this domain is found to be extremely robust against JPEG compression and some geometrical transforms. All these attacks are generated by the STIRMARK tools. This section is organized as the following: In section 2, an overview of the Hough transform is presented. Section 3 details the proposed method in the Hough domains: the carried study and the proposed solutions. In section 4, we study the robustness of this technique against different STIRMARK attacks by testing its capacity to detect the embedded watermark. The privileges offered by this approach are also detailed, and finally we conclude this work.

The Hough transform is a mathematical algorithm used in images processing to detect the presence of parametric forms as ellipses or lines in the image. This technique uses the principle of evidences accumulating to prove the existence of a particular form in the image. For this aim, this transform uses a parametric domain or space to characterize these forms. Each form is represented by its proper parameters in this space. In our work, this transform is coded to be used as lines detector where its parametric space is exploited. It's important to note that the Hough transform is found to have the capacity to detect the same segments or broken lines in the image, before and after being compressed. This detection invariance is due to the fact of the invariance properties of its parametric space in the case where the image is subjected to JPEG compression and some asynchronous transforms. In the case of

y = a . x + b (1)

x sin + + y cos (2)

**4. A new watermarking method using the parametric hough transform** 

**domain** 

**4.1 Introduction** 

**4.2 Hough transform overview** 

lines detecting, the Hough transform is presented as follows:

Each line can be described in the orthonormal space by the equation (1) or (2)

$$\text{Fig. 31. g} \text{ K=50.}$$

Fig. 31. h) K=210.

#### **4. A new watermarking method using the parametric hough transform domain**

#### **4.1 Introduction**

194 Watermarking – Volume 2

Fig. 31. g) K=50.

Fig. 31. h) K=210.

Fig. 31. i) K=250.

Fig. 31. Result of changing blocks in the fourth zone on the image.

Different constraints are required in a watermarking method, such imperceptibility and robustness. Besides, lossy JPEG compression remains the most unintended used attacks with data exchange in Internet, for size reduction. It can seriously affect the embedded watermark if the compression rate is high and the used scheme presents a weakness against this attack. So, the best solution resides in exploiting the DCT domain used in the JPEG algorithm in order to dispose of the robustness against this compression or the multiresolution domain as in. But acting to be robust against this attack reveals automatically the domains of watermark embedding and than increase the possibility of its detection. In this section, a novel watermarking method is proposed. It consists in using the parametric space of the mathematical Hough transform as a watermarking domain. The technique consists in selecting specific maximums in the Hough matrix with respect to a secret key. The peaks are found to be invariant points in the proposed Hough domain especially against lossy JPEG compression. Two signatures are considered; the first is hold in the Hough domain by the transformed space and consists in the locations of the specific chosen invariant points. Whereas, the second is represented by the use of end points of the correspondent detected lines. These end points are used as centers to embed similarities blocks in. The watermarking in this domain is found to be extremely robust against JPEG compression and some geometrical transforms. All these attacks are generated by the STIRMARK tools. This section is organized as the following: In section 2, an overview of the Hough transform is presented. Section 3 details the proposed method in the Hough domains: the carried study and the proposed solutions. In section 4, we study the robustness of this technique against different STIRMARK attacks by testing its capacity to detect the embedded watermark. The privileges offered by this approach are also detailed, and finally we conclude this work.

#### **4.2 Hough transform overview**

The Hough transform is a mathematical algorithm used in images processing to detect the presence of parametric forms as ellipses or lines in the image. This technique uses the principle of evidences accumulating to prove the existence of a particular form in the image. For this aim, this transform uses a parametric domain or space to characterize these forms. Each form is represented by its proper parameters in this space. In our work, this transform is coded to be used as lines detector where its parametric space is exploited. It's important to note that the Hough transform is found to have the capacity to detect the same segments or broken lines in the image, before and after being compressed. This detection invariance is due to the fact of the invariance properties of its parametric space in the case where the image is subjected to JPEG compression and some asynchronous transforms. In the case of lines detecting, the Hough transform is presented as follows:

Each line can be described in the orthonormal space by the equation (1) or (2)

$$\mathbf{y} = \mathbf{a} \cdot \mathbf{x} + \mathbf{b} \tag{1}$$

$$\mathbf{p} = \mathbf{x} \cdot \sin \theta + \mathbf{y} \cdot \cos \theta \tag{2}$$

2D Watermarking: Non Conventional Approaches 197

Consider an image with size N×M, can vary in the interval range of [0, 2
], the value of is maximum when it's computed in the image diagonal. The steps and variation domains of


<sup>0</sup> ; \* <sup>1</sup> <sup>&</sup>lt; <sup>1</sup> <sup>&</sup>lt; <sup>2</sup> <sup>=</sup>

<sup>2</sup> N M 22 2 *MAX*

2 2 0 , 2 *MAX*

More the quantification steps are decreased, better the resolution is; but the Hough matrix size increase. In order to attend equilibrium between: resolution, computing time and parametric space dimension, we will fix the orientation step depending on the image size. If

2 0 ,

 0 ; \* + 1 < <sup>2</sup> <sup>=</sup>

> 100 *MAX*

2 2 M 200 *N*

These steps provide an acceptable precision to browse the entire image as shown in

*N M* 

2 2 0 , ; and ; 2 <sup>2</sup> *MAX N with*

2 2 2 2

*N M*

*N M*

.9 . 9 /: / : (4)

<sup>0</sup> ; \* <sup>1</sup> <sup>&</sup>lt; <sup>2</sup> <sup>+</sup> <sup>=</sup> (6)

 

(8)

(9)

(5)

(7)

Fig. 2. The parametric space map.

varies as

Figure 3.

The quantification steps are computed as follow:

are than described by the equations (4, 5, 6 and 7):

If the image is square the resolution will be as:

In the following, we will consider the step as:

*N* 

The parametric space is than composed by two parameters: and that forme a space matrix as shown in figure 1.

Fig. 1. The parametric space (, ).

An infinity of lines can pass through a fixed point called P having (x,y) as coordonates. But if we consider a second point P1 having (x1,y1) as coordinates, only one line can pass through P and P1 satisfying the same couple of ( and ). If this principle is applied to the image, the Hough transform of an image generates a parametric space matrix as presented in (3):

$$H \text{ (} M \text{) } = \text{A} (\rho, \theta) \tag{3}$$

Where H is the Hough transform, M is the image matrix and A is the space parametric resulting matrix. Since this matrix contains a limited number of elements, the number of possible detected lines is with respect to the quantization step of and in their respective variation domains. Peaks contained in this space represent an accumulation of evidences indicating the possibility of lines presence with respect to a specific position and orientation.

#### **4.3 The proposed method**

In this work, we propose to apply the philosophy of the Hough transform on the image in order to process and manipulate it in the parametric Hough space, and use it as a watermark-embedding domain. If we consider an (N×M) image; in order to accumulate evidences and define the parametric space, the information source is gathered from the pixels composing the image. More the evidences are accumulated and put in the parametric space matrix; more the chance to identify a real line in the image is high. In the following, we will define the parametric space matrix generated by the Hough transform as the Hough space or Hough domain. The first step consists in applying a high pass filter in order to extract the image edges. In each point belonging to this edge, infinity of lines can pass through it. Accumulating evidences in the parametric space provides the unique position and orientation ( and ) for witch one line can pass through this point. In our case the positions and the orientations are quantified by a step computed with respect to the required precision. The Hough space is than a two-dimensional matrix or map. The size of this map depends on the quantification step as shown in Figure 2.

The parametric space is than composed by two parameters: and that forme a space

An infinity of lines can pass through a fixed point called P having (x,y) as coordonates. But if we consider a second point P1 having (x1,y1) as coordinates, only one line can pass through P and P1 satisfying the same couple of ( and ). If this principle is applied to the image, the

A , -

Where H is the Hough transform, M is the image matrix and A is the space parametric resulting matrix. Since this matrix contains a limited number of elements, the number of possible detected lines is with respect to the quantization step of and in their respective variation domains. Peaks contained in this space represent an accumulation of evidences indicating the possibility of lines presence with respect to a specific position

In this work, we propose to apply the philosophy of the Hough transform on the image in order to process and manipulate it in the parametric Hough space, and use it as a watermark-embedding domain. If we consider an (N×M) image; in order to accumulate evidences and define the parametric space, the information source is gathered from the pixels composing the image. More the evidences are accumulated and put in the parametric space matrix; more the chance to identify a real line in the image is high. In the following, we will define the parametric space matrix generated by the Hough transform as the Hough space or Hough domain. The first step consists in applying a high pass filter in order to extract the image edges. In each point belonging to this edge, infinity of lines can pass through it. Accumulating evidences in the parametric space provides the unique position and orientation ( and ) for witch one line can pass through this point. In our case the positions and the orientations are quantified by a step computed with respect to the required precision. The Hough space is than a two-dimensional matrix or map. The size of

 

(3)

Hough transform of an image generates a parametric space matrix as presented in (3):

*H M*-

this map depends on the quantification step as shown in Figure 2.

matrix as shown in figure 1.

Fig. 1. The parametric space (, ).

and orientation.

**4.3 The proposed method** 

Fig. 2. The parametric space map.

The quantification steps are computed as follow:

Consider an image with size N×M, can vary in the interval range of [0, 2
], the value of is maximum when it's computed in the image diagonal. The steps and variation domains of are than described by the equations (4, 5, 6 and 7):

$$
\rho\_{\text{MAX}}^2 = \left(\frac{\text{N}}{2}\right)^2 + \left(\frac{\text{M}}{2}\right)^2 = \frac{\sqrt{\text{N}^2 + \text{M}^2}}{2} \tag{4}
$$

$$\rho\_{\text{MAX}} \in \left[0, \frac{\sqrt{N^2 + M^2}}{2}\right] \tag{5}$$

More the quantification steps are decreased, better the resolution is; but the Hough matrix size increase. In order to attend equilibrium between: resolution, computing time and parametric space dimension, we will fix the orientation step depending on the image size. If varies as

$$\theta \in \left[0, \frac{2\pi}{\sqrt{N \cdot M}}\right] \tag{6}$$

If the image is square the resolution will be as:

$$\theta \in \left[0, \frac{2\pi}{N}\right]; \text{and } \rho\_{\text{MAX}} = \frac{\sqrt{2}}{2} \cdot N; \text{ with } \Delta \text{ } \rho = \sqrt{2} \tag{7}$$

In the following, we will consider the step as:

$$
\Delta\_{\perp} \rho\_{\perp} = \frac{\rho\_{\text{MAX}}}{100} \tag{8}
$$

$$
\Lambda \rho = \frac{\sqrt{N^2 + \mathbf{M}^2}}{200} \tag{9}
$$

These steps provide an acceptable precision to browse the entire image as shown in Figure 3.

2D Watermarking: Non Conventional Approaches 199

this image into a binary image. An edge extractor filter is then applied to extract the image edges. Once theses edges are taken out, the Hough transform is coded and then applied on the resultant image to browse it and then accumulates evidences in the transformed parametric matrix to decide witch maximums corresponds to real lines in the image. In this matrix, the number of chosen peaks and their respective position represents the secret key used to select the ends of the correspondent lines where the similarities blocks are embedded. The position of the peaks are returned and saved to be compared with the same peaks position after the attacks are applied on the image and view if they can be considered as unvaried points with respect to the applied attack. The peaks are defined as the entire matrix maximum that exceeds a fixed threshold. In this work, in order to obtain a better

*PK* + T max H <sup>h</sup> -

Where Pk represents the returned peaks, *H* is the Hough matrix. Different peaks can be selected and then view theirs corresponding lines and end points in the image as shown in

In the following, the number of peaks is chosen equal to one. The location of the peaks in the Hough parametric space is represented by the respective position in the matrix lines and

The first peak is selected and its position is returned as (-23, 89.2472) in the Hough matrix

peaks in the Hough space and their respective positions shown in Table 1. The correspondent detected lines and respectively their end points where the similarities blocks are embedded are shown in these figures. The Table 1 presents the attacks applied on the cameraman image; the JPEG compression and the rot-scale transform. The extracted peaks after the attacks have been applied presented. A total invariance is remarked concerning the

(12)

89.2472 . Figures from 8 to 11 present the detected

precision, the threshold is fixed as Th = 0.7 and then:

Figures 4, 5, 6 and 7.

columns as (L, C).

**4.5 Simulation results** 

space. That means that -23

peaks positions against lossy JPEG compression.

Fig. 4. The first pick in the parametric Hough space.

" and

Fig. 3. Image browsed by and variation.

The operation of accumulating evidences in the Hough matrix for potential presence of lines in the image is characterized by the appearance of maximums in the matrix. A threshold is previously chosen to characterize since witch values we can consider maximums in the space parametric matrix as peaks. The number of picks and their position in the Hough map is used as secret key. By finding and fixing the peaks number, we extract the correspondent lines and respectively their end points. These end points are used as centres of blocks similarities embedding. In fact in each end point we extract a block of size -2 1 2 1 *n n* % - . All these blocks are substituted with similarities as shown by the equations (10) and (11). If we consider Bi as the chosen block, Wi the watermark and Bwi the watermarked block:

$$\mathcal{W}\_{i} = \text{ dyn (B}\_{i}) = \frac{B\_{i} \cdot \stackrel{\cdots}{\text{B}}\_{i}}{\max\left(B\_{i} \cdot \stackrel{\cdots}{\text{B}}\_{i}\right)} \tag{10}$$

$$\mathbf{B}\_{\rm wi} = \mathbf{B}\_i \cdot \left( \begin{array}{c} \mathbf{1} \ + \ \ \ \ \ \mathbf{a} \ \cdot \end{array} \right) \tag{11}$$

Where B*<sup>i</sup>* -- is the indexed block mean and is the watermark embedding strength. In the experimental results, as will be shown in the next section, the selected peaks (maximums) in the Hough space matrix corresponds to the embedded watermark location in the image. The peaks position in the Hough space and their respective end points in the spatial representation, are completely invariant when the image is attacked by JPEG compression or some geometrical transforms.

#### **4.4 Experimental configuration**

In our experiments, the cameraman image is used to simulate the applied method and the chosen attacks. This image is chosen because of its content variation. In fact it contains lines in addition to homogeneous and textured zones. A binarizing method is applied to convert

The operation of accumulating evidences in the Hough matrix for potential presence of lines in the image is characterized by the appearance of maximums in the matrix. A threshold is previously chosen to characterize since witch values we can consider maximums in the space parametric matrix as peaks. The number of picks and their position in the Hough map is used as secret key. By finding and fixing the peaks number, we extract the correspondent lines and respectively their end points. These end points are used as centres of blocks similarities embedding. In fact in each end point we extract a block of size

equations (10) and (11). If we consider Bi as the chosen block, Wi the watermark and Bwi the


B 1 wi + + *B W i i* -

experimental results, as will be shown in the next section, the selected peaks (maximums) in the Hough space matrix corresponds to the embedded watermark location in the image. The peaks position in the Hough space and their respective end points in the spatial representation, are completely invariant when the image is attacked by JPEG compression

In our experiments, the cameraman image is used to simulate the applied method and the chosen attacks. This image is chosen because of its content variation. In fact it contains lines in addition to homogeneous and textured zones. A binarizing method is applied to convert


*<sup>B</sup> <sup>W</sup>*

*i*

is the indexed block mean and

or some geometrical transforms.

**4.4 Experimental configuration** 

. All these blocks are substituted with similarities as shown by the


*i i*


*<sup>B</sup>* (10)

(11)

is the watermark embedding strength. In the

*i i*

 --

i

Fig. 3. Image browsed by and variation.


Where B*<sup>i</sup>* ---

2 1 2 1 *n n* % -

watermarked block:

this image into a binary image. An edge extractor filter is then applied to extract the image edges. Once theses edges are taken out, the Hough transform is coded and then applied on the resultant image to browse it and then accumulates evidences in the transformed parametric matrix to decide witch maximums corresponds to real lines in the image. In this matrix, the number of chosen peaks and their respective position represents the secret key used to select the ends of the correspondent lines where the similarities blocks are embedded. The position of the peaks are returned and saved to be compared with the same peaks position after the attacks are applied on the image and view if they can be considered as unvaried points with respect to the applied attack. The peaks are defined as the entire matrix maximum that exceeds a fixed threshold. In this work, in order to obtain a better precision, the threshold is fixed as Th = 0.7 and then:

$$P\_{\mathcal{K}} = \mathcal{T}\_{\mathcal{h}} \cdot \max \left( \mathcal{H} \right) \tag{12}$$

Where Pk represents the returned peaks, *H* is the Hough matrix. Different peaks can be selected and then view theirs corresponding lines and end points in the image as shown in Figures 4, 5, 6 and 7.

#### **4.5 Simulation results**

In the following, the number of peaks is chosen equal to one. The location of the peaks in the Hough parametric space is represented by the respective position in the matrix lines and columns as (L, C).

The first peak is selected and its position is returned as (-23, 89.2472) in the Hough matrix space. That means that -23 " and 89.2472 . Figures from 8 to 11 present the detected peaks in the Hough space and their respective positions shown in Table 1. The correspondent detected lines and respectively their end points where the similarities blocks are embedded are shown in these figures. The Table 1 presents the attacks applied on the cameraman image; the JPEG compression and the rot-scale transform. The extracted peaks after the attacks have been applied presented. A total invariance is remarked concerning the peaks positions against lossy JPEG compression.

Fig. 4. The first pick in the parametric Hough space.

2D Watermarking: Non Conventional Approaches 201

Fig. 9. Detected segments corresponding to the three first selected peaks presented in Fig.8

The figures from 8 to 11 show respectively the cameraman image attacked by the JPEG 10, 20 and 40 compression and the obtained result of the detected peaks position leading to the watermark detection. The proposed method based on the Hough parametric space is found to be highly robust against lossy compression. A series of different JPEG compression rates from JPEG 100 to JPEG 10 are applied as shown in Table 1. The distortion caused to the watermarked image by all these attacks hasn't changed the position of the selected peaks in

Fig. 8. The three first selected picks.

**4.5.1 Experiment 1: JPEG compression** 

the Hough space.

Fig. 5. Three detected segments corresponding to the first selected peak presented in Fig.4.

Fig. 6. Two first picks in the parametric Hough space.

Fig. 7. Detected segments corresponding to the first selected peak presented in fig.6.

Fig. 5. Three detected segments corresponding to the first selected peak presented in Fig.4.

Fig. 7. Detected segments corresponding to the first selected peak presented in fig.6.

Fig. 6. Two first picks in the parametric Hough space.

Fig. 8. The three first selected picks.

Fig. 9. Detected segments corresponding to the three first selected peaks presented in Fig.8

#### **4.5.1 Experiment 1: JPEG compression**

The figures from 8 to 11 show respectively the cameraman image attacked by the JPEG 10, 20 and 40 compression and the obtained result of the detected peaks position leading to the watermark detection. The proposed method based on the Hough parametric space is found to be highly robust against lossy compression. A series of different JPEG compression rates from JPEG 100 to JPEG 10 are applied as shown in Table 1. The distortion caused to the watermarked image by all these attacks hasn't changed the position of the selected peaks in the Hough space.

2D Watermarking: Non Conventional Approaches 203

Fig. 12. The position of the detected peak in the JPEG 10 compressed image.

Fig. 13. End points of the detected lines correspondent to the peak in Fig.12.

Fig. 14. The position of the detected peak in the ROT-SCALE 0.5 attacked image.

Figures 14 and 16 show the rotation and scale attack, and the correspondent detected peaks position in the Hough space. As shown in the Table 1 and the figures below. The use of this space provides high robustness against these attacks and grant invariance properties to the selected peaks if the image is attacked, especially when dealing with small distortions the

**4.5.2 Experiment 2: Asynchronous attacks** 

invariance of the peaks position is kept unchanged.


Table 1. Invariance of the selected peak position against applied attacks.

Fig. 10. The position of the detected peak in the JPEG 30 compressed image.

Fig. 11. End points of the detected lines correspondent to the peak in Fig.10.

APPLIED ATTACK PEAK POSITION IN

JPEG 100 (-23, 89.2472) JPEG 90 (-23, 89.2472) JPEG 80 (-23, 89.2472) JPEG 70 (-23, 89.2472) JPEG 60 (-23, 89.2472) JPEG 40 (-23, 89.2472) JPEG 30 (-23, 89.2472) JPEG 20 (-23, 89.2472) JPEG 10 (-23, 89.2472) ROTSCALE –0.25 (-23, 89.2472) ROTSCALE –0.5 (-23, 89.2472) PSNR 100 (-23, 89.2472)

Table 1. Invariance of the selected peak position against applied attacks.

Fig. 10. The position of the detected peak in the JPEG 30 compressed image.

Fig. 11. End points of the detected lines correspondent to the peak in Fig.10.

THE HOUGH SPACE

Fig. 12. The position of the detected peak in the JPEG 10 compressed image.

Fig. 13. End points of the detected lines correspondent to the peak in Fig.12.

#### **4.5.2 Experiment 2: Asynchronous attacks**

Figures 14 and 16 show the rotation and scale attack, and the correspondent detected peaks position in the Hough space. As shown in the Table 1 and the figures below. The use of this space provides high robustness against these attacks and grant invariance properties to the selected peaks if the image is attacked, especially when dealing with small distortions the invariance of the peaks position is kept unchanged.

Fig. 14. The position of the detected peak in the ROT-SCALE 0.5 attacked image.

2D Watermarking: Non Conventional Approaches 205

Figure 18 shows the robustness of the proposed Hough algorithm when compared with the well known and most robust algorithms proposed in the DCT domain to defeat the JPEG compression attacks. It's evident that our algorithm is the most robust. This method is found to be better than those actually in use, due to the fact that the image is not processed similarly to the methods in use that embed the watermark by modifying the image either in

A new domain and watermarking techniques are proposed in this work. In the first presented approach, sing the mathematical Hessenberg transformation, the original image is transformed in the Hessenberg domain as a triangular matrix which values present specific characteristics. The embedding procedure is applied to a transformed image in a nonsensitive zone that has no effect on the image quality after an inverse transformation is applied. Processing the lowest values in the entire matrix, by modifying them we impose a low variation on the original coefficients of the image and no distortions appears on the watermarked image. A study was carried out to show how the Hessenberg matrix can be

the spatial domain or in the frequency and multi-resolution domains.

Fig. 17. Peaks detected corresponding to Fig.16.

Fig. 18. Robustness of the Hough algorithms.

**5. Conclusion** 

#### **4.6 Evaluation and comments**

In this section, we comment the results and compare our proposed methods to other ones. As shown previously, this method is highly robust against JPEG compression. The Peaks positions are unvaried whatever the compression rate used. The image is represented by the Hough parametric space as a new representation domain where the signature is characterized by the unvaried positions of the selected peaks and hold in it. The robustness of this method is picked out from the robustness of this new Hough domain face to JPEG and other attacks. In fact, the parametric Hough space holding the signature cannot be modified by synchronous attacks as JPEG, filtering, i.e. it doesn't modify the pixels position and then the ends of lines. As a result, the detected peaks after the Hough transform is applied remain unvaried. Conversely, the asynchronous attacks that modify vastly the pixels position change the position of the lines ends and then modify the positions of the Hough space peaks. Figures 16 and 17 show the asynchronous rot-scale attack with two degrees and the correspondent detected peaks.

Fig. 15. End points of the detected lines correspondent to the peak in Fig.14.

Fig. 16. Rot\_scale 2° peak position.

In this section, we comment the results and compare our proposed methods to other ones. As shown previously, this method is highly robust against JPEG compression. The Peaks positions are unvaried whatever the compression rate used. The image is represented by the Hough parametric space as a new representation domain where the signature is characterized by the unvaried positions of the selected peaks and hold in it. The robustness of this method is picked out from the robustness of this new Hough domain face to JPEG and other attacks. In fact, the parametric Hough space holding the signature cannot be modified by synchronous attacks as JPEG, filtering, i.e. it doesn't modify the pixels position and then the ends of lines. As a result, the detected peaks after the Hough transform is applied remain unvaried. Conversely, the asynchronous attacks that modify vastly the pixels position change the position of the lines ends and then modify the positions of the Hough space peaks. Figures 16 and 17 show the asynchronous rot-scale attack with two

**4.6 Evaluation and comments** 

degrees and the correspondent detected peaks.

Fig. 16. Rot\_scale 2° peak position.

Fig. 15. End points of the detected lines correspondent to the peak in Fig.14.

Fig. 17. Peaks detected corresponding to Fig.16.

Fig. 18. Robustness of the Hough algorithms.

Figure 18 shows the robustness of the proposed Hough algorithm when compared with the well known and most robust algorithms proposed in the DCT domain to defeat the JPEG compression attacks. It's evident that our algorithm is the most robust. This method is found to be better than those actually in use, due to the fact that the image is not processed similarly to the methods in use that embed the watermark by modifying the image either in the spatial domain or in the frequency and multi-resolution domains.

#### **5. Conclusion**

A new domain and watermarking techniques are proposed in this work. In the first presented approach, sing the mathematical Hessenberg transformation, the original image is transformed in the Hessenberg domain as a triangular matrix which values present specific characteristics. The embedding procedure is applied to a transformed image in a nonsensitive zone that has no effect on the image quality after an inverse transformation is applied. Processing the lowest values in the entire matrix, by modifying them we impose a low variation on the original coefficients of the image and no distortions appears on the watermarked image. A study was carried out to show how the Hessenberg matrix can be

2D Watermarking: Non Conventional Approaches 207

L.H. Chen and J.J. Lin, "Mean quantization based image watermarking", Image and vision

P. Chun Chen, Y. Sheng Chen, and W. Hsing Hsu, "A communication system model for

L.Diane, Cours de traitement d'images, Laboratoire I3S Informatique, Signaux et Systèmes,

A. Fabien, P. Peticolas, "Watermarking schemes evaluation", IEEE Signal Processing

A. Fabien, P. Petitcolas, M. Steinebach, F. Raynal, J. Dittmann, C. Fontaine and N. Fatès, "A

G. H. Golub, and C. F. Van Loan, "Matrix Computation", Johns Hopkins University Press,

M. Van.droo. Genbroeck, "Acquisition et traitement d'image", Université de Liège

H. Guo and N. D. Georganas, "Multi-resolution Image Watermarking Scheme in the

F. Hartung and M. Kutter, Multimedia watermarking techniques, Proc. IEEE Vol. 87, No. 7,

J. Huang, Yun Q. Shi and Yi Shi, "Embedding image watermarks in DC components", IEEE

N. Kaewakamnerd and K.R. Rao, Wavelet based image adaptive scheme, Electronics letters

S. Katzenbeisser, F.A.P. Petitcolas, Information hiding techniques for steganography and

E. Koch and J. Zhao, Towards robust and hidden image copyright labeling, Proceedings of

X .Kong, Y. Liu, H. Liu and D. Yang, "Object watermark for digital image and video", Image and vision computing journal, Vol.22, Issue.8, August 2004, pp. 583-595. D. Kundur and D. Hatzinakos, "Diversity and Attack Characterization for Improved Robust Watermarking", IEEE Transactions on Signal Processing, Vol. 49, no. 10, Oct 2001.

the IEEE International Workshop on Nonlinear Signal and Image Processing, pp.

contents III, vol. 4314, San Jose, California, U.S.A., 20-26 January 2001. J. Fridrich, Combining low-frequency and spread spectrum watermarking, In Proceedings of

Systems Analysis and Synthesis , ISAS, Vol.6, pages 2935, USA,1999. I. Cox, J. Kilian, T. Leighton, T. Shamoon, "Secure spread spectrum watermarking for multimedia", IEEE Trans. Image Process. No. 6, Vol. 12, June 1997, pp.1673-1687. F. Davoine and S. Pateux, "Tatouage de documents audiovisuels numériques", Hermes

digital image watermarking problems", International conference on Information

Université de Nice Sophia Antipolice, Rapport de recherche ISRN I3S/ RR, 22

public automated web-based evaluation service for watermarking schemes: StirMark Benchmark", In Ping Wah Wong and Edward J. Delp, editors, proceedings of electronic imaging, security and watermarking of multimedia

the SPIE Symposium on Optical Science, Engineering and Instrumentation, San

publication, Institut Montefiori, Service de télécommunication et d'imagerie,

Spectrum Domain", IEEE Canadian conference on electrical and computer

computing vol. 21, No. 8, 1 August 2003, pp. 717-727.

Magazine, Vol. 17, no. 5, pp. 58-64, September 2000.

Trans. Consumer Electron. 46 3 (2000), pp. 415-421.

digital watermarking, Artech house, December 1999.

452-455, Halkidiki, Marmaras, Greece, June 1995.

science, Lavoisier 2004.

Diego, USA, July 1998.

Septembre 2001, version 4.14.

engineering, pp125, may 2002.

1999, pp. 1079-1107.

Vol. 36, 2000, pp. 312-313.

1983, pp. 384.

janvier 2005.

partitioned, and the effect of each matrix zone on the image perception. Many advantages are proposed by the use of this method. In fact it allows the use of a high embedding strength which allows being more robust against attacks than DCT, DFT or spatial methods. Its robustness against lossy JPEG compression exceeds this allowed by the well known and used until now DCT domain. In addition, by choosing the appropriate zone and changing some of its values, this technique is able to give the appearance of a noised, banded or blurred image without really applying these signal processing operations on the image. This technique is found to be very resistant against simultaneous a large set of synchronous and asynchronous signal processing attacks, and the watermark is always present in the entire set of the attacked image. The watermark detection process and the similarities computing presented in this approach are obtained from tests applied on the "Cameraman" image with a gain factor of 35. Evidently, the embedding strength can be highly augmented without exceeding the watermark imperceptibility when dealing with certain kinds of other images presenting different characteristics that allow high gains value without any changes, as shown in Figures 31h and 31i. Markedly, this high gain increases the robustness of the embedded watermark and improves vastly the results of the proposed method. We finally note that we propose a blind watermarking technique; the presence of the host image is not required for watermark detection procedure.

In the second approach, a new watermarking method is presented. Based on the mathematical hough transform, a parametric space matrix is obtained and used as a new space where the image is processed. A secret key is characterized by the number of selected peaks in this space matrix is chosen. These peaks are found to be invariant and robust against jpeg compression and some asynchronous transforms. They are also used to determine the correspondent lines end points where similarities blocks are embedded. The embedded watermark is carried in hough space by the invariant peaks and their position that corresponds to the embedded similarities. This method proposes higher resistance against lossy compression than the previous algorithms based essentially in the DCT domain.

#### **6. References**


partitioned, and the effect of each matrix zone on the image perception. Many advantages are proposed by the use of this method. In fact it allows the use of a high embedding strength which allows being more robust against attacks than DCT, DFT or spatial methods. Its robustness against lossy JPEG compression exceeds this allowed by the well known and used until now DCT domain. In addition, by choosing the appropriate zone and changing some of its values, this technique is able to give the appearance of a noised, banded or blurred image without really applying these signal processing operations on the image. This technique is found to be very resistant against simultaneous a large set of synchronous and asynchronous signal processing attacks, and the watermark is always present in the entire set of the attacked image. The watermark detection process and the similarities computing presented in this approach are obtained from tests applied on the "Cameraman" image with a gain factor of 35. Evidently, the embedding strength can be highly augmented without exceeding the watermark imperceptibility when dealing with certain kinds of other images presenting different characteristics that allow high gains value without any changes, as shown in Figures 31h and 31i. Markedly, this high gain increases the robustness of the embedded watermark and improves vastly the results of the proposed method. We finally note that we propose a blind watermarking technique; the presence of the host image is not

In the second approach, a new watermarking method is presented. Based on the mathematical hough transform, a parametric space matrix is obtained and used as a new space where the image is processed. A secret key is characterized by the number of selected peaks in this space matrix is chosen. These peaks are found to be invariant and robust against jpeg compression and some asynchronous transforms. They are also used to determine the correspondent lines end points where similarities blocks are embedded. The embedded watermark is carried in hough space by the invariant peaks and their position that corresponds to the embedded similarities. This method proposes higher resistance against lossy compression than the previous algorithms based essentially in the DCT

E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A.

M. Barni, F. Bartolini, A. De Rosa and A. Piva, " Capacity of the watermark channel: how

P. Bas, J.M. Chassery and B. Macq, "Image watermarking: an evolution to content-based

P. Bas and J.M. Chassery, Tatouage d'image résistant aux transformées géométriques, 17éme

G. Caronni, "Assuring ownership rights for digital images", Proceeding of reliable IT

L. Chang, "Issues in Information Hiding Transform Techniques", Storming Media,

Guide", Third Edition, SIAM, Philadelphia, 1999.

approaches", Pattern Recognition 35 (2002), pp.545-561.

colloque GRETSI, Vannes, France, 13-17 Septembre 1999.

Systems, VIS' 95, Viewveg Publishing Company, Germany, 1995.

Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, "LAPACK User's

many bits can be hidden within a digital image", Proc. SPIE 3657, 1999, pp. 437-448.

Computers: Cybernetics, 20 May, 2002, http://www.stormingmedia.us/84/8491/

required for watermark detection procedure.

domain.

**6. References** 

A849104.html.


**9** 

*Ukraine* 

**Audio Watermarking for Automatic** 

**in VHF Maritime Communication** 

Oleksandr V. Shishkin and Vitaliy M. Koshevyy

*Odessa National Maritime Academy* 

**Identification of Radiotelephone Transmissions** 

Audio watermarking (AW) corresponds to digital information imperceptibly embedded into the audio signal. AW for maritime VHF (Very High Frequency) communication is inspired first of all by the ability of implementation an automatic identification of radiotelephone transmissions in the channels of maritime (156…174) MHz mobile radio communication service. Applied to VHF radiotelephony, a watermarking system could overcome existing limitations, and ultimately increase safety and efficiency of maritime communication. The same application of AW may be implemented in the aeronautical (118…136) MHz mobile service. In the mentioned services analogue broadcasting channels with frequency/phase

For the meanwhile the identification of the sea vessels is realized by means of verbal calling of ship's call sign or numerical identification. However on account of different reasons such verbal identification may be absent, transmitted with delay, or understood with errors. This problem is illustrated in Fig. 1. Motor vessel "Arcona" transmits a certain message to all stations. But one of the receiving vessels missed the name and call sign of the transmitting ship, and another ship interpreted the name of transmitting ship as "Gargona" instead

It is obvious that false, incorrectly interpreted or delayed verbal identification negatively affects maritime navigation. Automatic identification could avoid misidentification and call sign confusion. Strictly speaking from the time the Global Maritime Distress and Safety System (GMDSS) (Brehaut, 2009) came in force in 1999, each radiotelephone exchange should be preceded by digital selective calling (DSC) on special calling channel 70 and appropriate acknowledge by means of DSC. After such calling procedures the radiotelephone transmission should be started on the assigned in DSC working channel. Meanwhile DSC and radiotelephone exchange are two independently executed by the navigational officer operations. Correct execution of these operations under Radio Regulation completely depends on human factor. In practice, however, DSC is often ignored, and navigators at once use radiotelephone channel 16. Especially this is typical for urgent communication. In such circumstances timely, clear and authentic identification is extremely necessary. Automatic identification would exclude the human factor and increase

an efficiency of VHF radiocommunication and maritime safety in the whole.

and amplitude modulation correspondingly are utilized.

**1. Introduction** 

"Arcona".


## **Audio Watermarking for Automatic Identification of Radiotelephone Transmissions in VHF Maritime Communication**

Oleksandr V. Shishkin and Vitaliy M. Koshevyy *Odessa National Maritime Academy Ukraine* 

#### **1. Introduction**

208 Watermarking – Volume 2

P. Lan, "Robust transparent image watermarking system with spatial mechanisms", Journal

G.C. Langelaar, I. Setyawan, and R.L. Lagendijk, "Watermarking in digital image and data:

M. Laug, "Traitement optique du signal et des images", Ecole Nationale Supérieure de l'Aéronautique et de l'Espace SUP'AERO, Edition Cépaduès, France, 1980. P. Moulin, M.K. Mihcak, "A framework for evaluating the data-hiding capacity of image

A. Natarajan, "Discrete cosine transform", IEEE Trans. on Computers, 1974, Vol. c-23, pp 90-

N. Nikolaidis and I. Oitas, "Robust image watermarking in the spatial domain", Signal

I. Pitas, T. Kaskalis, "Applying signatures on digital image", Workshop on Nonlinear Signal and Image Processing, IEEE, Neos Marmaras, June 1995, pp 460-463. C.I. Podichuck and W. Zeng, image adaptive watermarking using visual models, IEEE

V. Rouilly, Présentation de la transformée de Hough, Rapport interne, ENST, Raris, France,

J. O'Ruanaidh and T. Pun, "Rotation, Scale and Translation Invariant Digital Image

K. Sayood, "Data compression", Maurgan Kaufmann Publishers, San Francisco, CA, 2000. H. Seddik, M. Sayadi and F. Fnaiech, "A New Watermarking method using the parametric

H.Seddik, E.Ben.Braiek, "Color Medical Images Watermarking" ICGST International Journal

J.S. Seo and C.D. Chang Yoo, Localized image watermarking based on features points of scalespace representation, Pattern Recognition, Vol. 37, No. 7, July 2004, pp. 1365-1375. F.Y. Shih, S.Y.T. Wu, "Combinational image watermarking in the spatial and frequency domains", Pattern Recognition, vol. 36, Issue 4, April 2003, pp. 969-975. P. Su, C.J. Kuo and H.M Wang, Blind digital watermarking for cartoon and map images,

T.L. WANG and W.B. GRAGG, "Convergence of the shifted QR algorithm for unitary

R. Wolfgang, E. Delp, "A watermarking technique for digital imagery: further studies",

K.E. Zhao, "Embedding robust labels into images for copyright protection", Technical Report, Fraunhofer Institute for Computer Graphics, Darmatadt, Germany, 1994.

A state of the art overview", IEEE Signal Processing magazine, September (2000),

sources", IEEE Transactions on Image Processing, September 2002, vol. 11, no. 9,

journal on selected area in communication, Special Issue on Copyright and Privacy

http://www.tsi.enst.fr/tsi/enseignement/ressour ces/mti/ellipses/Hough.html .

Watermarking", Proc. IEEE international conference on image processing, Vol. 1,

Hough Transform Domain", WSEAS Trans. on Information Science and

on Graphics, Vision and Image Processing, Vol.6 Special Issue on Medical Image

SPIE conference on security and watermarking of multimedia contents, San Jose,

Hessenberg matrices", Mathematics of computation, Volume 71, Number 240, pp.

International Conference on Imaging Science, Systems and Technology, Los Vegas,

of systems and software, 15 February 2000, 107-116.

Processing, vol. 66, no. 3 (1998), pp. 385-403.

Application, no. 9, Vol. 2, Sep. 2005, pp. 1277-1284.

Protection, Vol. 16, 1998, pp. 525-538.

Processing, pp.81-86, March 2006.

CA, USA, January 1999 pp. 296-305.

1473-1496,November 30, 2001.

Nevada, July, 1997.

pp. 20-40.

pp. 1029-1042.

pp. 536-539, 1997.

93.

Audio watermarking (AW) corresponds to digital information imperceptibly embedded into the audio signal. AW for maritime VHF (Very High Frequency) communication is inspired first of all by the ability of implementation an automatic identification of radiotelephone transmissions in the channels of maritime (156…174) MHz mobile radio communication service. Applied to VHF radiotelephony, a watermarking system could overcome existing limitations, and ultimately increase safety and efficiency of maritime communication. The same application of AW may be implemented in the aeronautical (118…136) MHz mobile service. In the mentioned services analogue broadcasting channels with frequency/phase and amplitude modulation correspondingly are utilized.

For the meanwhile the identification of the sea vessels is realized by means of verbal calling of ship's call sign or numerical identification. However on account of different reasons such verbal identification may be absent, transmitted with delay, or understood with errors. This problem is illustrated in Fig. 1. Motor vessel "Arcona" transmits a certain message to all stations. But one of the receiving vessels missed the name and call sign of the transmitting ship, and another ship interpreted the name of transmitting ship as "Gargona" instead "Arcona".

It is obvious that false, incorrectly interpreted or delayed verbal identification negatively affects maritime navigation. Automatic identification could avoid misidentification and call sign confusion. Strictly speaking from the time the Global Maritime Distress and Safety System (GMDSS) (Brehaut, 2009) came in force in 1999, each radiotelephone exchange should be preceded by digital selective calling (DSC) on special calling channel 70 and appropriate acknowledge by means of DSC. After such calling procedures the radiotelephone transmission should be started on the assigned in DSC working channel. Meanwhile DSC and radiotelephone exchange are two independently executed by the navigational officer operations. Correct execution of these operations under Radio Regulation completely depends on human factor. In practice, however, DSC is often ignored, and navigators at once use radiotelephone channel 16. Especially this is typical for urgent communication. In such circumstances timely, clear and authentic identification is extremely necessary. Automatic identification would exclude the human factor and increase an efficiency of VHF radiocommunication and maritime safety in the whole.

Audio Watermarking for Automatic Identification of

**2. Watermarking as communication problem** 

be imperceptible on the background of carrier signal x .

encoder, is defined by formula:

is defined by the formula:

technologies acquired in the field of conventional communication.

**2.1 Watermarked communication over a channel with side information** 

Fig. 2. Watermarked communication over a channel with side information

<sup>1</sup> C log 1 <sup>2</sup>

Practically 2 2 w x , and capacity is limited mainly by the host itself.

Radiotelephone Transmissions in VHF Maritime Communication 211

Watermarking may be examined in the frame of common communication problem (Cox et al., 2008), especially taking into account numerous algorithms of signal processing and

Model of communication system with additive embedding of digital watermarks is presented in Fig. 2. Watermark signal w is formed on the base of embedded data m . Encoder may use information about carrier signal (or host signal) x , that is reflected with dotted line. Then carrier signal x is added by watermark w . Power <sup>2</sup> w of w is limited by the acceptable level of introduced distortions of carrier signal because watermark w should

In the channel two interferences act against watermark w : the first interference is itself the carrier signal with the power <sup>2</sup> <sup>x</sup> , and the second one – a noise n with the power <sup>2</sup> <sup>n</sup> . Watermarking channel is characterised by its capacity - the maximum achievable code rate. Assuming the both interferences are white Gaussian noises the capacity C (bit/sample) of watermarked channel with noniformed encoder, when host signal is not available to

At the same time using information about the carrier signal x , it is possible to increase C (this is a case of informed encoder). An idea of informed encoder goes back to Kuznetsov & Tsybakov, 1974 and is known as writing in memory with defective cells. Gel'fand & Pinsker, 1980 and osta, 1983 shown that assuming the host is known at the transmitter, the capacity

<sup>1</sup> C log 1 <sup>2</sup>

The Eq. (2) shows that carrier signal doesn't influence on watermark transmission and the capacity is determined only by the second noise, which is unknown at the encoder. Capacity

2 <sup>w</sup> <sup>2</sup> 2 2 x n

> 2 <sup>w</sup> <sup>2</sup> <sup>2</sup> n


/ : . (1)

/ : . (2)


Identification in DSC is produced by means of so called maritime mobile service identity (MMSI). MMSI is a unique combination of nine decimal digits. In binary representation MMSI occupies 36 bits sequence in DSC format. The same identification by means of MMSI may be applied to radiotelephone identification.

Fig. 1. Audio watermarking makes possible guaranteed identification of VHF maritime radiotelephony

Verbal identification doesn't protect against illegal radio transmission. Illegal transmissions are especially harmful on the VHF distress channel 16. Of course, violators are transmitting anonymously. Reliable identification of such transmissions could avoid the violation of radiotelephone regulation.

Another advantage of automatic identification becomes apparent in the ability of digital information inputting to another ships' navigational and information systems, for example ECDIS (Electronic Chart Display and Information System). ECDIS makes visualization of neighboring vessels in the range of VHF radio (i. e. approximately 30 nautical miles). However the transmitting vessel by no means is marked in an electronic map. Automatic identification would enable to mark on the electronic chart the transmitting vessel. Such an innovative function of ECDIS would be useful for clearing of current navigational environment. It is obvious that in such visual presentation navigator officer decision could be accepted more quickly and correctly. Such application of AW would again reduce risks of human factor demonstration.

One more application of AW is a covered information transmission in the special applications (for example, facing the threat of terrorist aggression).

It is essential that AW doesn't require altering an existing radio installation and operational procedures. AW identification keeps standard equipment and procedures. Only new telephone receiver (or headset) with embedded processor at the transmitter side and processor with mini-display switched to common audio output at the receiver side are to be mounted. Automatic identification starts right away press-to-talk switching and runs during all transmitting period independently from verbal signal occurrence. No additional time and frequency channel recourses are required.

#### **2. Watermarking as communication problem**

210 Watermarking – Volume 2

Identification in DSC is produced by means of so called maritime mobile service identity (MMSI). MMSI is a unique combination of nine decimal digits. In binary representation MMSI occupies 36 bits sequence in DSC format. The same identification by means of MMSI

*All stations* 

*I'm going …*

Transmitting vessel

*This is ARCONA/URAC* 

MMSI

Fig. 1. Audio watermarking makes possible guaranteed identification of VHF maritime

Verbal identification doesn't protect against illegal radio transmission. Illegal transmissions are especially harmful on the VHF distress channel 16. Of course, violators are transmitting anonymously. Reliable identification of such transmissions could avoid the violation of

*All stations This is GARGONA I'm going …*

> Receiving vessel

Another advantage of automatic identification becomes apparent in the ability of digital information inputting to another ships' navigational and information systems, for example ECDIS (Electronic Chart Display and Information System). ECDIS makes visualization of neighboring vessels in the range of VHF radio (i. e. approximately 30 nautical miles). However the transmitting vessel by no means is marked in an electronic map. Automatic identification would enable to mark on the electronic chart the transmitting vessel. Such an innovative function of ECDIS would be useful for clearing of current navigational environment. It is obvious that in such visual presentation navigator officer decision could be accepted more quickly and correctly. Such application of AW would again reduce risks of

One more application of AW is a covered information transmission in the special

It is essential that AW doesn't require altering an existing radio installation and operational procedures. AW identification keeps standard equipment and procedures. Only new telephone receiver (or headset) with embedded processor at the transmitter side and processor with mini-display switched to common audio output at the receiver side are to be mounted. Automatic identification starts right away press-to-talk switching and runs during all transmitting period independently from verbal signal occurrence. No additional time and

applications (for example, facing the threat of terrorist aggression).

may be applied to radiotelephone identification.

*All stations This is ??? ??? I'm going …*

Receiving vessel

radiotelephony

radiotelephone regulation.

human factor demonstration.

frequency channel recourses are required.

Watermarking may be examined in the frame of common communication problem (Cox et al., 2008), especially taking into account numerous algorithms of signal processing and technologies acquired in the field of conventional communication.

#### **2.1 Watermarked communication over a channel with side information**

Model of communication system with additive embedding of digital watermarks is presented in Fig. 2. Watermark signal w is formed on the base of embedded data m . Encoder may use information about carrier signal (or host signal) x , that is reflected with dotted line. Then carrier signal x is added by watermark w . Power <sup>2</sup> w of w is limited by the acceptable level of introduced distortions of carrier signal because watermark w should be imperceptible on the background of carrier signal x .

Fig. 2. Watermarked communication over a channel with side information

In the channel two interferences act against watermark w : the first interference is itself the carrier signal with the power <sup>2</sup> <sup>x</sup> , and the second one – a noise n with the power <sup>2</sup> <sup>n</sup> . Watermarking channel is characterised by its capacity - the maximum achievable code rate. Assuming the both interferences are white Gaussian noises the capacity C (bit/sample) of watermarked channel with noniformed encoder, when host signal is not available to encoder, is defined by formula:

$$\mathbf{C} = \frac{1}{2} \log\_2 \left( 1 + \frac{\sigma\_{\mathbf{w}}^2}{\sigma\_{\mathbf{x}}^2 + \sigma\_{\mathbf{n}}^2} \right). \tag{1}$$

Practically 2 2 w x , and capacity is limited mainly by the host itself.

At the same time using information about the carrier signal x , it is possible to increase C (this is a case of informed encoder). An idea of informed encoder goes back to Kuznetsov & Tsybakov, 1974 and is known as writing in memory with defective cells. Gel'fand & Pinsker, 1980 and osta, 1983 shown that assuming the host is known at the transmitter, the capacity is defined by the formula:

$$\mathbf{C} = \frac{1}{2} \log\_2 \left( 1 + \frac{\sigma\_{\mathbf{w}}^2}{\sigma\_{\mathbf{n}}^2} \right). \tag{2}$$

The Eq. (2) shows that carrier signal doesn't influence on watermark transmission and the capacity is determined only by the second noise, which is unknown at the encoder. Capacity

Audio Watermarking for Automatic Identification of

Fig. 3. Watermarking channel model

**2.2.1 Intersymbol interference (ISI)** 

Watermarking channel model is presented in Fig. 3.

Radiotelephone Transmissions in VHF Maritime Communication 213

Watermarking is concerned with the reliable transmission of information embedded into a host signal. The main difference from classical communication situation comes from the restriction of the host signal distortion. Digital watermarking can be viewed as a communication problem: information m to be sent from point A to point B is encoded into a signal w using information on host signal x . It is clear that making use of x calls for some time delay in x transmission. The time delay depends on complexity of processing at the encoder. But practically delay less then 100 – 200 msec in the master channel determines nothing but gives the ability to eliminate interfering influence of host on watermark signal

Watermarked signal sxw is then passing through a common channel, which is unknown by nature. The embedded watermark should be reliably decodable even after further processing of the marked signal, which is also denoted as attack against the

Channel

Linear filtering

from mic to telephone

*C D*

*w m*

Encoder + Decoder *<sup>m</sup> <sup>s</sup>*

*A B*

*y*

+

The attacks result unavoidable signal degradation. For robust watermarking we need restoration of embedded information m until speech communication in the master channel is possible. Another words watermarks robustness should monotonically fall with the

ISI forms distortions of a signal in which current symbol interferes with the previous one. Previous symbol has influence on the current symbol like noise, thus making communication less reliable. In the radio (or wireless) channels ISI is usually caused by multipath propagation. The transmitting medium in VHF radio communication is the atmosphere, in which radio signal is transferred by means of electromagnetic waves. The received electromagnetic signal is usually a superposition of a line-of-sight path signal and multiple waves coming from different directions. This phenomenon is known as multipath propagation. It is clear that reflected waves have to pass a longer distance and therefore arrive with a time-delay compared to the line-of-sight signal. The received signal is spread in time and the channel is said to be time dispersive. The time delays correspond to phase shifts in between superimposed waves. The phase shifts vary depending on frequency and signal frequency component may be cancelled or reinforced. This effect is known as

w . Standard audio-radiotelephone channel in Fig. 3 is denoted C – D.

*x*

Delay

*n*

+

embedded watermark. Consider that malicious attacks are absent.

quality of radio transmission in the master channel.

for such channel is increasing very much. The signal doesn't act as a noise source, that's why an informed encoding (i.e. "writing on dirty paper") is an attractive method for watermarking on account of potential capacity.

Noninformed encoder watermarking techniques exploit spread spectrum (SS) methods. In SS schemes the embedded bit flow is modulated by an SS sequence and added to the signal in the time or frequency domain. In schemes using SS the signal itself is seen as a source of interference and for reliable watermark restoration at the receiver the length of spreading sequence should be rather long to accumulate sufficient watermark energy. SS methods are traditionally considered as the most resistant against various attacks.

Capacity according Eq. (1) for SS watermarking is valid for assumption that the host signal is additive white Gaussian (AWGN) process. In practice, speech signal is highly correlated process and characteristics of SS watermarking may be remarkably improved. In paragraph 6 we consider adaptive whitening procedure for decreasing interfering influence of the host signal on watermark detection.

Informed encoding techniques are based on quantization the host signal directly or its certain transformation. The most popular method for this mode of embedding is quantization index modulation (QIM), proposed by Chen & Wornell, 2001. QIM-methods are free from the host signal interference, but commonly are more sensitive to attacks in watermarking channel.

Our investigations are based on classical communication approaches, applied to watermarking. It is known another view to the problem, for instance Hofbauer, et al., 2009 presented a blind speech watermarking algorithm that embeds the watermark data in the phase of non-voiced speech by replacing a certain voice segments on watermarked signal. However, the proposed method is based only on speech as a carrier signal and cannot involve any audio signals.

A comprehensive review of state-of-art methods for watermarking and data hiding is done by Moulin & Koetter, 2005. From the variety of watermarking method we focused on three candidates: SS, QIM and improved SS (ISS). The main goal of the paper consists in applying the modern communication technologies to audio watermarking.

#### **2.2 Interferences in VHF radio channel**

In this section we consider interferences which are important for watermarking in VHF analog radiochannel.

AW uses the common radiotelephone channel. The main interferences in through audioradio-audio channel that affect AW are:


Hofbauer & Kubin, 2006 proposed to take into account also Doppler effect that is actual for aeronautical applications.

Watermarking channel model is presented in Fig. 3.

212 Watermarking – Volume 2

for such channel is increasing very much. The signal doesn't act as a noise source, that's why an informed encoding (i.e. "writing on dirty paper") is an attractive method for

Noninformed encoder watermarking techniques exploit spread spectrum (SS) methods. In SS schemes the embedded bit flow is modulated by an SS sequence and added to the signal in the time or frequency domain. In schemes using SS the signal itself is seen as a source of interference and for reliable watermark restoration at the receiver the length of spreading sequence should be rather long to accumulate sufficient watermark energy. SS methods are

Capacity according Eq. (1) for SS watermarking is valid for assumption that the host signal is additive white Gaussian (AWGN) process. In practice, speech signal is highly correlated process and characteristics of SS watermarking may be remarkably improved. In paragraph 6 we consider adaptive whitening procedure for decreasing interfering influence of the host

Informed encoding techniques are based on quantization the host signal directly or its certain transformation. The most popular method for this mode of embedding is quantization index modulation (QIM), proposed by Chen & Wornell, 2001. QIM-methods are free from the host signal interference, but commonly are more sensitive to attacks in

Our investigations are based on classical communication approaches, applied to watermarking. It is known another view to the problem, for instance Hofbauer, et al., 2009 presented a blind speech watermarking algorithm that embeds the watermark data in the phase of non-voiced speech by replacing a certain voice segments on watermarked signal. However, the proposed method is based only on speech as a carrier signal and cannot

A comprehensive review of state-of-art methods for watermarking and data hiding is done by Moulin & Koetter, 2005. From the variety of watermarking method we focused on three candidates: SS, QIM and improved SS (ISS). The main goal of the paper consists in applying

In this section we consider interferences which are important for watermarking in VHF

AW uses the common radiotelephone channel. The main interferences in through audio-

1. intersymbol interference (ISI) caused by low frequency circuits in the transceiver and

Hofbauer & Kubin, 2006 proposed to take into account also Doppler effect that is actual for

watermarking on account of potential capacity.

signal on watermark detection.

watermarking channel.

involve any audio signals.

analog radiochannel.

2. flat amplitude fading; 3. external additive noise;

aeronautical applications.

**2.2 Interferences in VHF radio channel** 

radio-audio channel that affect AW are:

4. nonlinear distortions (clipping); 5. resampling and desinchronization.

multipass radio waves propagation;

traditionally considered as the most resistant against various attacks.

the modern communication technologies to audio watermarking.

Watermarking is concerned with the reliable transmission of information embedded into a host signal. The main difference from classical communication situation comes from the restriction of the host signal distortion. Digital watermarking can be viewed as a communication problem: information m to be sent from point A to point B is encoded into a signal w using information on host signal x . It is clear that making use of x calls for some time delay in x transmission. The time delay depends on complexity of processing at the encoder. But practically delay less then 100 – 200 msec in the master channel determines nothing but gives the ability to eliminate interfering influence of host on watermark signal w . Standard audio-radiotelephone channel in Fig. 3 is denoted C – D.

Fig. 3. Watermarking channel model

Watermarked signal sxw is then passing through a common channel, which is unknown by nature. The embedded watermark should be reliably decodable even after further processing of the marked signal, which is also denoted as attack against the embedded watermark. Consider that malicious attacks are absent.

The attacks result unavoidable signal degradation. For robust watermarking we need restoration of embedded information m until speech communication in the master channel is possible. Another words watermarks robustness should monotonically fall with the quality of radio transmission in the master channel.

#### **2.2.1 Intersymbol interference (ISI)**

ISI forms distortions of a signal in which current symbol interferes with the previous one. Previous symbol has influence on the current symbol like noise, thus making communication less reliable. In the radio (or wireless) channels ISI is usually caused by multipath propagation. The transmitting medium in VHF radio communication is the atmosphere, in which radio signal is transferred by means of electromagnetic waves. The received electromagnetic signal is usually a superposition of a line-of-sight path signal and multiple waves coming from different directions. This phenomenon is known as multipath propagation. It is clear that reflected waves have to pass a longer distance and therefore arrive with a time-delay compared to the line-of-sight signal. The received signal is spread in time and the channel is said to be time dispersive. The time delays correspond to phase shifts in between superimposed waves. The phase shifts vary depending on frequency and signal frequency component may be cancelled or reinforced. This effect is known as

Audio Watermarking for Automatic Identification of

resampling attacks are not reflected in Fig. 3.

Radiotelephone Transmissions in VHF Maritime Communication 215

watermark restoration beginning of watermark should be first detected and then all decision points are counted from the starting point. Analog radiotelephone channel by all means leads to resampling and loss of the watermark beginning. Desynchronization and

a)

b)

c)

Fig. 4. OFDM application for watermarking: a) noninformed encoding;

Orthogonal frequency division multiplexing (OFDM) is well known multi-carrier modulation method and is used in various wireless communication systems (Ipatov, 2005). It has been shown to be an effective technique to combat multipath fading in wireless

In basic OFDM scheme data symbols modulate a parallel collection of regularly spaced subcarriers. OFDM is simple to use on channels with time delay spread or, equivalently, frequency selectivity. OFDM converts one frequency selective channel into a parallel collection of frequency flat sub-channels. Techniques that are appropriate for flat fading

channels can then be applied in a straight forward fashion to every sub-channel.

b) informed encoding; c) decoding

**3.1 OFDM based watermarking schemes** 

**3. OFDM for watermarking** 

channels.

frequency selective fading and gives rise to notches in the frequency response of the channel.

Another physical cause of ISI is nonuniformity of frequency response of a channel. Analog low-frequency circuits of the transceiver are composed from reactive elements. These elements (including spurious effects) are bases for channel frequency band limitation. Frequency dependent elements cause nonuniformity of frequency response within audio signal spectrum. When frequency response is explicitly nonuniform within signal spectrum output signal highly differs from input one. Distortions caused by bandlimited lowfrequency channel also represent ISI.

From the signal processing point of view the two physically different causes (presence of reactive elements in audio circuits and multipath radio wave propagation) lead to the same final result in the form of ISI.

For watermarking ISI may be simulated by linear filtering with appropriate frequency or impulse response. In Fig. 3 the two above mentioned sources of ISI are incorporated in one block denoted "Linear filtering".

#### **2.2.2 Flat fading**

Coming back to multipath propagation, one can analyze a variant when the different path lengths are very similar compared to the wavelengths of the signal components. Then the phase variations between components will be small and they will all undergo very similar amounts of cancellation or reinforcement. This case is usually termed flat fading.

In watermarking flat fading is simulated by amplitude scaling attacks. In Fig. 3 flat fading is shown as multiplicative interference .

#### **2.2.3 Additive noise**

Additive noise is imposed onto the signal during transmission. The noise results from thermal noise in electronic circuits, from atmospheric noise or from other radio stations. Quantization noise from analog-to-digit converter may be attributed to additive noise. Commonly recognized model of an additive noise is additive white Gaussian noise, denoted in Fig. 3 by n .

#### **2.2.4 Nonlinear distortions**

Nonlinear distortions appear in amplitude limitations caused, for example, by the overload in audio circuits. Overload arises from redundant power of transmitting station. The simplest model of nonlinear distortions is clipping. AW in any case should be resistant against such distortions. Source of nonlinear distortion is not shown in Fig. 3.

#### **2.2.5 Desynchronization and resampling**

At the transmitter and receiver sampling processes are not synchronized. It means that sampling instants are mutually shifted. Consider sampling frequencies are equal at the transmitter and receiver. At the receiver beginning of watermark is unknown. For

frequency selective fading and gives rise to notches in the frequency response of the

Another physical cause of ISI is nonuniformity of frequency response of a channel. Analog low-frequency circuits of the transceiver are composed from reactive elements. These elements (including spurious effects) are bases for channel frequency band limitation. Frequency dependent elements cause nonuniformity of frequency response within audio signal spectrum. When frequency response is explicitly nonuniform within signal spectrum output signal highly differs from input one. Distortions caused by bandlimited low-

From the signal processing point of view the two physically different causes (presence of reactive elements in audio circuits and multipath radio wave propagation) lead to the same

For watermarking ISI may be simulated by linear filtering with appropriate frequency or impulse response. In Fig. 3 the two above mentioned sources of ISI are incorporated in one

Coming back to multipath propagation, one can analyze a variant when the different path lengths are very similar compared to the wavelengths of the signal components. Then the phase variations between components will be small and they will all undergo very similar amounts of cancellation or reinforcement. This case is usually termed

In watermarking flat fading is simulated by amplitude scaling attacks. In Fig. 3 flat fading is

Additive noise is imposed onto the signal during transmission. The noise results from thermal noise in electronic circuits, from atmospheric noise or from other radio stations. Quantization noise from analog-to-digit converter may be attributed to additive noise. Commonly recognized model of an additive noise is additive white Gaussian noise, denoted

Nonlinear distortions appear in amplitude limitations caused, for example, by the overload in audio circuits. Overload arises from redundant power of transmitting station. The simplest model of nonlinear distortions is clipping. AW in any case should be resistant

At the transmitter and receiver sampling processes are not synchronized. It means that sampling instants are mutually shifted. Consider sampling frequencies are equal at the transmitter and receiver. At the receiver beginning of watermark is unknown. For

against such distortions. Source of nonlinear distortion is not shown in Fig. 3.

channel.

frequency channel also represent ISI.

final result in the form of ISI.

block denoted "Linear filtering".

shown as multiplicative interference .

**2.2.2 Flat fading** 

**2.2.3 Additive noise** 

in Fig. 3 by n .

**2.2.4 Nonlinear distortions** 

**2.2.5 Desynchronization and resampling** 

flat fading.

watermark restoration beginning of watermark should be first detected and then all decision points are counted from the starting point. Analog radiotelephone channel by all means leads to resampling and loss of the watermark beginning. Desynchronization and resampling attacks are not reflected in Fig. 3.

Fig. 4. OFDM application for watermarking: a) noninformed encoding; b) informed encoding; c) decoding

#### **3. OFDM for watermarking**

Orthogonal frequency division multiplexing (OFDM) is well known multi-carrier modulation method and is used in various wireless communication systems (Ipatov, 2005). It has been shown to be an effective technique to combat multipath fading in wireless channels.

#### **3.1 OFDM based watermarking schemes**

In basic OFDM scheme data symbols modulate a parallel collection of regularly spaced subcarriers. OFDM is simple to use on channels with time delay spread or, equivalently, frequency selectivity. OFDM converts one frequency selective channel into a parallel collection of frequency flat sub-channels. Techniques that are appropriate for flat fading channels can then be applied in a straight forward fashion to every sub-channel.

Audio Watermarking for Automatic Identification of


samples. Then we obtain so called OFDM symbol:

It obvious that Ni i x x , i 0,1, ,P 1.

while keeping introduced distortions.

 -.

, k 1, ,N /2 1 and X ,X 0 N/2 -

N/2

n kk k 1

2 22 a


In Eq. (7) it is supposed that watermarked coefficients are from 1 to *B* .


0

and sequence the n x is assured to be real also.

and N unknowns Xk

implemented in MatLab.

X X k Nk

 --

to be real.

form:

X a jb kk k -

Coefficients *Xk*


watermarked coefficients:

where Xk -

N 1 k n n 0

N 1 n k k 0


Radiotelephone Transmissions in VHF Maritime Communication 217

2 nk <sup>Y</sup> <sup>y</sup> exp <sup>j</sup> , k 0,1, ,N 1. <sup>N</sup>

According to OFDM principle each harmonic should be cyclically extended with P

<sup>1</sup> 2 nk x X exp <sup>j</sup> , n 0,1, ,N P 1 N N

However we couldn't insert N P samples instead of N samples without sampling frequency alteration. We are forced to utilize N P samples n x from Eq. (5) instead of N samples n x from Eq. (3) plus subsequent P samples from the future block. Prefix results in the host signal distortions and serves as a penalty for ISI decreasing. The length of prefix comes from the channel impulse response. Reasonable P may be accepted to suppress ISI

Mathematically equations (5) presents an overdetermined system with N P equations

For the least squares criteria min x x n n mathematic methods are well designed and

Solution of system (5) is inaccurate in general and consists from complex numbers Xk


k 0,1, ,N 1. Therefore samples n x may come up to complex values also. Only if

To obtain real values n x let us represent system (5) in trigonometric form. Denote

<sup>x</sup> a cos kn b sin kn , n 0,1, ,N P 1 N2 N N - 8 . 9

As before we have N P equations and N unknowns: 0 kk N/2 a , (a , b ), k 1,2,...,N / 2 1, a . Least squared solution will be in real field numbers

*kkk* ,...,2,1, *BkWXS*-

in Eq. (5). After evident transformation one may get the system in required

, or part of them, are subjected for watermarking. In general we get the

/ : , (6)



, (4)

, (5)

are both real, the sequence n x is guarantied

. (7)

 -,

A general system for SS watermarking based on OFDM principle is shown in Fig. 4 a). Sequence of message bits m first is split in serial-to-parallel (S/P) block into some "slow" flows. Relation between a dimension of inverse fast Fourier transform (IFFT) and a number of slow flows depends on what Fourier coefficients are subjected for watermarking. So that some of inputs of IFFT block may be set to zeros. Then parallel-to-serial (P/S) block combines slow flows into one sequence which represents watermark it the time domain. Commonly SS methods use a certain type of transform (discrete cosine transform (DCT), discrete Fourier transform (DFT), Wavelet, etc.) for subsequent watermarking of transformation coefficients.

Application of any transform demands storing of carrier signal and strictly speaking leads to some delay in signal transmission. OFDM scheme produces inherently the same watermarking of fast Fourier transform (FFT) coefficients without any delay. Encoder turns out really noninformed.

General system for informed OFDM encoding is presented in Fig. 4 b). Signal x is splited into slow flows which are subjected to FFT. Message flow m is splited into some, suppose B , (B N /2) slow flows. Channel encoders use B Fourier coefficients for watermarking independently in each channel. In the simple case one bit may watermark one coefficient. For more complex variants one embedded bit may be distributed among L 1 coefficients. In general NL samples of x are needed for embedding B message bits. Algorithms for each channel encoders are identical. Watermarked coefficients and all the rest undisturbed coefficients are then used for IFFT. Again P/S block combines slow flows into one sequence s in the time domain.

Watermarked signal s to resist against intersymbol interference may be added with prefix ps (see below) so the watermarked signal becomes p [s , s] . Prefixed signal presents so called OFDM symbol and is then transmitted through the channel.

At the receiver (Fig. 4 c) OFDM symbol is primarily cleared from the prefix, which is mostly corrupted by ISI. This operation is not shown in the figure. Then signal y is splited into N flows which are transformed in Fourier coefficients. These coefficients are processed according to demodulation algorithm for extracting a watermark message bits mˆ . Receiving scheme is general for informed and noninformed encoding.

Attractive features of OFDM-like watermarking are good vectorisation for encoding and decoding algorithms and application of standard FFT and IFFT procedures.

#### **3.2 Resistance to ISI**

Let us show how to apply the main principles of OFDM for resistant to ISI watermarking.

According to inverse DFT signal block n x , n 0,1, ,N 1 may be composed from N harmonics with complex amplitudes X , k 0,1, ,N 1. <sup>k</sup> -

$$\mathbf{x}\_n = \frac{1}{N} \sum\_{k=0}^{N-1} \dot{\mathbf{X}}\_k \exp\left(j\frac{2\pi nk}{N}\right) \quad n = 0, 1, \ldots, N-1. \tag{3}$$

At the decoder on account of ISI we have harmonics on the same frequency grid, but with another complex amplitudes *XY kk*- - ' , where *Yk* are obtained from DFT:

$$\dot{\mathbf{Y}}\_{\mathbf{k}} = \sum\_{\mathbf{n}=0}^{\text{N}-1} \mathbf{y}\_{\mathbf{n}} \exp\left(-\mathbf{j}\frac{2\pi \mathbf{n}\mathbf{k}}{\mathbf{N}}\right), \quad \mathbf{k} = \mathbf{0}, \mathbf{1}, \dots, \mathbf{N} - 1. \tag{4}$$

According to OFDM principle each harmonic should be cyclically extended with P samples. Then we obtain so called OFDM symbol:

$$\mathbf{x}'\_{\mathbf{n}} = \frac{\mathbf{1}}{\mathbf{N}} \sum\_{\mathbf{k}=0}^{\mathbf{N}-1} \dot{\mathbf{X}}'\_{\mathbf{k}} \exp\left(\mathbf{j} \frac{2\pi \mathbf{m} \mathbf{k}}{\mathbf{N}}\right) \quad \mathbf{n} = \mathbf{0}, \mathbf{1}, \dots, \mathbf{N} + \mathbf{P} - \mathbf{1} \tag{5}$$

where Xk -- complex amplitudes undetermined for the present.

It obvious that Ni i x x , i 0,1, ,P 1.

216 Watermarking – Volume 2

A general system for SS watermarking based on OFDM principle is shown in Fig. 4 a). Sequence of message bits m first is split in serial-to-parallel (S/P) block into some "slow" flows. Relation between a dimension of inverse fast Fourier transform (IFFT) and a number of slow flows depends on what Fourier coefficients are subjected for watermarking. So that some of inputs of IFFT block may be set to zeros. Then parallel-to-serial (P/S) block combines slow flows into one sequence which represents watermark it the time domain. Commonly SS methods use a certain type of transform (discrete cosine transform (DCT), discrete Fourier transform (DFT), Wavelet, etc.) for subsequent watermarking of

Application of any transform demands storing of carrier signal and strictly speaking leads to some delay in signal transmission. OFDM scheme produces inherently the same watermarking of fast Fourier transform (FFT) coefficients without any delay. Encoder turns

General system for informed OFDM encoding is presented in Fig. 4 b). Signal x is splited into slow flows which are subjected to FFT. Message flow m is splited into some, suppose B , (B N /2) slow flows. Channel encoders use B Fourier coefficients for watermarking independently in each channel. In the simple case one bit may watermark one coefficient. For more complex variants one embedded bit may be distributed among L 1 coefficients. In general NL samples of x are needed for embedding B message bits. Algorithms for each channel encoders are identical. Watermarked coefficients and all the rest undisturbed coefficients are then used for IFFT. Again P/S block combines slow flows into one sequence

Watermarked signal s to resist against intersymbol interference may be added with prefix ps (see below) so the watermarked signal becomes p [s , s] . Prefixed signal presents so called

At the receiver (Fig. 4 c) OFDM symbol is primarily cleared from the prefix, which is mostly corrupted by ISI. This operation is not shown in the figure. Then signal y is splited into N flows which are transformed in Fourier coefficients. These coefficients are processed according to demodulation algorithm for extracting a watermark message bits mˆ .

Attractive features of OFDM-like watermarking are good vectorisation for encoding and

Let us show how to apply the main principles of OFDM for resistant to ISI watermarking.

According to inverse DFT signal block n x , n 0,1, ,N 1 may be composed from N

<sup>8</sup> . /

*N*

At the decoder on account of ISI we have harmonics on the same frequency grid, but with


*<sup>k</sup>* 

.1,,1,0, <sup>2</sup>

<sup>9</sup> :

*n N*

are obtained from DFT:

(3)

OFDM symbol and is then transmitted through the channel.

harmonics with complex amplitudes X , k 0,1, ,N 1. <sup>k</sup> -

*x*

another complex amplitudes *XY kk*-

1 <sup>1</sup> 0

*N k <sup>n</sup>* -



Receiving scheme is general for informed and noninformed encoding.

decoding algorithms and application of standard FFT and IFFT procedures.

exp

' , where *Yk*

*nk jX <sup>N</sup>*

transformation coefficients.

out really noninformed.

s in the time domain.

**3.2 Resistance to ISI** 

However we couldn't insert N P samples instead of N samples without sampling frequency alteration. We are forced to utilize N P samples n x from Eq. (5) instead of N samples n x from Eq. (3) plus subsequent P samples from the future block. Prefix results in the host signal distortions and serves as a penalty for ISI decreasing. The length of prefix comes from the channel impulse response. Reasonable P may be accepted to suppress ISI while keeping introduced distortions.

Mathematically equations (5) presents an overdetermined system with N P equations and N unknowns Xk -.

For the least squares criteria min x x n n mathematic methods are well designed and implemented in MatLab.

Solution of system (5) is inaccurate in general and consists from complex numbers Xk - , k 0,1, ,N 1. Therefore samples n x may come up to complex values also. Only if X X k Nk - - , k 1, ,N /2 1 and X ,X 0 N/2 - are both real, the sequence n x is guarantied to be real.

To obtain real values n x let us represent system (5) in trigonometric form. Denote X a jb kk k in Eq. (5). After evident transformation one may get the system in required form:

$$\mathbf{x}'\_{n} = \frac{2}{\mathbf{N}} \left( \frac{\mathbf{a}\_{0}}{2} + \sum\_{k=1}^{\mathbf{N}/2} \mathbf{a}\_{k} \cos \frac{2\pi}{\mathbf{N}} \mathbf{k} \mathbf{n} - \mathbf{b}\_{k} \sin \frac{2\pi}{\mathbf{N}} \mathbf{k} \mathbf{n} \right) \quad \text{n} = 0, 1, \dots, \mathbf{N} + \mathbf{P} - 1 \tag{6}$$

As before we have N P equations and N unknowns: 0 kk N/2 a , (a , b ), k 1,2,...,N / 2 1, a . Least squared solution will be in real field numbers and sequence the n x is assured to be real also.

Coefficients *Xk* - , or part of them, are subjected for watermarking. In general we get the watermarked coefficients:

$$
\dot{S}\_k = \dot{X}\_k' + \dot{W}\_{k'} \quad k = 1, 2, \dots, B \; . \tag{7}
$$

In Eq. (7) it is supposed that watermarked coefficients are from 1 to *B* .

Audio Watermarking for Automatic Identification of

may be presented in the form:

In Eq. (10) denoted:

operation.

numbers.

where j 1 .

Step of quantization

1

2

0


c) partition to decision areas


expressed through the quantization function s Qx -


Scalar QIM translates real numbers x into lattices <sup>m</sup>

closest to y and forms the estimation of extracted bit

quantization function for QIM2 is written in the form:

where ..., 2, 1,0, 1, 2,... - set of integer numbers.

Quantization step is chosen depends on distortions-robustness trade-off.

Radiotelephone Transmissions in VHF Maritime Communication 219

 xm m Q x,m round 2 2 - 8

The QIM decoder operates as a minimum-distance decoder. It finds the quantizer node

Let us introduce quantization on the complex plain and denote it QIM2. Complex


 . 9 / :

xm m s round 1 j 1 j 2 2

Complex quantization QIM2 is illustrated in Fig. 5. QIM2 uses two-dimensional lattices

, 1 

m 0,1 m argmin y Q y,m ˆ

\*



magnitude is done independently according to real and imagynary axes.

 <sup>0</sup>

1

2

0


 a) b) c) Fig. 5. Lattices on the complex plain: a) lattice for *m* 1 , b) lattice for *m* 0 ,

 . 9 / :




, (12)

represents in general a complex number. Rounding of complex

10 2 -1 10 2

1

2

0


, where scalar quantization function

. (10)

Z m /2 , Z - is set of integer

. (11)

( 1 2) , (13)

+ - rounding

First N samples of watermarked sequence are calculated by means IDFT: s IDFT S <sup>n</sup> - , n 0,1, ,N 1 and supplemented with repeated prefix.

Because of ISI received block y [y , y ,...,y ] 0 1 PN1 even providing zero additive noise differs from transmitted block. Initial samples are especially corrupted by ISI. For processing decoder removes the first P samples and computes coefficients Y DFT <sup>k</sup> <sup>y</sup> - , utilising the last samples y [y , y ,..., y ] P P1 PN1 , which are free from ISI.

Prefix appending results in matching linear y sh and cyclic y sh convolutions at interval P 1, N P . Here s,h - are periodic sequences which are formed from the sequences 0 1 N1 [s ,s ,...,s ] , 0 1 P1 [h ,h ,...,h ,0, 0,...,0] correspondingly and the second sequence is added by zeros up to N samples.

If Y,S,H - - -- are DFTs of y,s,h , then

$$
\dot{\mathbf{Y}} = \dot{\mathbf{S}} \,\, \dot{\mathbf{H}} \,. \tag{8}
$$

In the last Eq. (8) H-- is the frequency response of general channel.

According to basic idea of OFDM an audio signal can be split into the some of sub carriers on frequencies 0 1 N1 f ,f ,...,f . Every sub carrier has its own complex amplitude, say Xi - . The amplitudes vary from slot to slot and are constant within every time slot.

From point of view of watermarking every sub carrier forms separate sub channel, and every channel operates independently from each other. Amplitudes Xi are subjected to watermarking. Assume they are altered to i S according certain algorithm. Since sub carriers are orthogonal the amplitudes i S will not have influence on each other. Composed watermarked audio may by presented in the form:

$$\dot{\mathbf{S}}(\mathbf{t}) = \sum\_{i=1}^{M} \dot{\mathbf{S}}\_{i} \exp\left(\mathbf{j} \,\mathbf{2}\pi \mathbf{f}\_{i} \mathbf{t}\right) \,\tag{9}$$

Every sub channel occupies a narrow frequency band. It is reasonable to assume that within one sub channel frequency response of the general channel is constant.

Thanks to prefix ISI is eliminated and received complex amplitude in sub channel will be defined by Eq. (8).

For slow fading it is considered *H j f const* - 2 *<sup>i</sup>* within one time slot.

#### **4. Quantization Index Modulation (QIM)**

Chen & Wornell, 2001 introduced a class of data-hiding codes known as dither modulation codes, or quantization index modulation (QIM) codes. These methods are based on quantization techniques.

#### **4.1 Scalar and complex QIM**

The simplest implementation of quantization-based watermarking employs scalar quantizer for embedding one bit into one host sample. The watermarking rule for this case is expressed through the quantization function s Qx - , where scalar quantization function may be presented in the form:

$$\mathbf{Q(x,m)} = \Lambda \text{round}\left(\frac{\mathbf{x}}{\Lambda} + \frac{\mathbf{m}}{2}\right) - \Lambda \frac{\mathbf{m}}{2} \,. \tag{10}$$

In Eq. (10) denoted: - quantization step, m 0, 1 - embedded bit, round-+ - rounding operation.

Quantization step is chosen depends on distortions-robustness trade-off.

Scalar QIM translates real numbers x into lattices <sup>m</sup> Z m /2 , Z - is set of integer numbers.

The QIM decoder operates as a minimum-distance decoder. It finds the quantizer node closest to y and forms the estimation of extracted bit

$$\hat{\mathbf{m}} = \underset{\mathbf{m} \in \{0, 1\}}{\arg\min} \left\| \mathbf{y} - \mathbf{Q}(\mathbf{y}, \mathbf{m}) \right\|\,. \tag{11}$$

Let us introduce quantization on the complex plain and denote it QIM2. Complex quantization function for QIM2 is written in the form:

$$\dot{\mathbf{s}} = \tilde{\Lambda} \mathbf{r} \mathbf{u} \text{und} \left( \frac{\dot{\mathbf{x}}}{\tilde{\Lambda}} + \frac{\mathbf{m}}{2} (\mathbf{1} + \dot{\mathbf{j}}) \right) - \tilde{\Lambda} \frac{\mathbf{m}}{2} (\mathbf{1} + \dot{\mathbf{j}}) \,\tag{12}$$

where j 1 .

218 Watermarking – Volume 2

First N samples of watermarked sequence are calculated by means IDFT: s IDFT S <sup>n</sup> -

Because of ISI received block y [y , y ,...,y ] 0 1 PN1 even providing zero additive noise differs from transmitted block. Initial samples are especially corrupted by ISI. For processing decoder removes the first P samples and computes coefficients Y DFT <sup>k</sup> <sup>y</sup> -

Prefix appending results in matching linear y sh and cyclic y sh convolutions at interval P 1, N P . Here s,h - are periodic sequences which are formed from the sequences 0 1 N1 [s ,s ,...,s ] , 0 1 P1 [h ,h ,...,h ,0, 0,...,0] correspondingly and the second

> Y SH - --

According to basic idea of OFDM an audio signal can be split into the some of sub carriers on frequencies 0 1 N1 f ,f ,...,f . Every sub carrier has its own complex amplitude, say Xi

From point of view of watermarking every sub carrier forms separate sub channel, and


Every sub channel occupies a narrow frequency band. It is reasonable to assume that within

Thanks to prefix ISI is eliminated and received complex amplitude in sub channel will be

Chen & Wornell, 2001 introduced a class of data-hiding codes known as dither modulation codes, or quantization index modulation (QIM) codes. These methods are based on

The simplest implementation of quantization-based watermarking employs scalar quantizer for embedding one bit into one host sample. The watermarking rule for this case is


M

i 1 S t S exp j2 ft 


i i

within one time slot.


utilising the last samples y [y , y ,..., y ] P P1 PN1 , which are free from ISI.

amplitudes vary from slot to slot and are constant within every time slot.

every channel operates independently from each other. Amplitudes Xi



one sub channel frequency response of the general channel is constant.

 2*<sup>i</sup>* -


n 0,1, ,N 1 and supplemented with repeated prefix.

sequence is added by zeros up to N samples.


watermarking. Assume they are altered to i S

watermarked audio may by presented in the form:

For slow fading it is considered *H j f const* -

**4. Quantization Index Modulation (QIM)** 

carriers are orthogonal the amplitudes i S

If Y,S,H - --

In the last Eq. (8) H-

defined by Eq. (8).

quantization techniques.

**4.1 Scalar and complex QIM** 

,

,


are subjected to

. (8)


according certain algorithm. Since sub

will not have influence on each other. Composed

, . (9)

Step of quantization represents in general a complex number. Rounding of complex magnitude is done independently according to real and imagynary axes.

Complex quantization QIM2 is illustrated in Fig. 5. QIM2 uses two-dimensional lattices

$$
\Lambda\_0 = \tilde{\Delta} \, Z \,, \, \Lambda\_1 = \tilde{\Delta} \, \left( Z + 1/2 \right) \, \,, \, \tag{13}
$$

where ..., 2, 1,0, 1, 2,... - set of integer numbers.

Fig. 5. Lattices on the complex plain: a) lattice for *m* 1 , b) lattice for *m* 0 , c) partition to decision areas

Audio Watermarking for Automatic Identification of

where

Radiotelephone Transmissions in VHF Maritime Communication 221

k 1 k i i 1 1 l - 8 . 9 / :

The same ralations shold be used at the receiver with the substitution vector **Y**k i instead of **S**k i . Note that all bold-marked vectors have length B . At the receiver detection of


Relations (16) - (20) make possible to eliminate flat fading and realize QIM process fully

From Eq. (8) it is clear that invariant amplitude scaling and phase shift procedure, abbriviate it as IAP, can be applied to each narrowband channel. Multi channel encoder

k k1 k

**m YG m Y** . (20)

Vector for step quantization in Eq. (12) is then given by the relation

is certaine stepsize chosen apriory.

imperceptible to slowly amplitude scaling and phase shift.

implementation is presented in Fig. 6.

a) transmitter; b) receiver

embedded bits is performed by using a minimum Euclidian distance rule, i.e.

0,1 ˆ arg min Q , ,

\*

**m**

 a) b) Fig. 6. OFDM invariant to amplitude scaling and phase shift QIM2 system:

Coming through the channel signal is subjected to linear filtering. Thanks to prefix the received signal after linear filtering may be considered in the frequency domain according to

1

**g S** , (17)

,**<sup>S</sup>** (18)

k1 k1 **G** . (19)

<sup>l</sup> <sup>p</sup> <sup>p</sup>

l k 1 k i i 1 arg - 8 . 9 / :

Advantage of QIM2 over scalar QIM consist in increasing the of distance between the nearest concurring decision points. It is possible to show that, assuming equal distortions the minimal distance is 2 / 2 times greater for QIM2 procedure comparatively to scalar QIM (see Fig. 5).

#### **4.2 Invariance to amplitude scaling and phase shift**

The main drawback of QIM is sensitivity to amplitude scaling and filtering.

Perez-Gonzalez et al., 2005 proposed techique for images processing against value-metric scaling attack, named as rational dither modulation (RDM). The main idea of RDM is snaping of quantization steps to watermarked and received signals at the transmitter and receiver correspondingly. For the first order for RDM scheme steps quantization for the current samples Xk and Yk are produced on the base of previous samples k 1 S and Yk 1 . Assuming noise absence, the influence of constant multiplier in the channel is fully eliminated on decoding process.

This scheme works until k 1 S 0 ' . Performance of the first order RDM scheme may by improved by high order schemes. Stepsize estimation at the transmitter is produced on the base of l previous watermarked samples and is given by the p norm of vector S :

$$\mathbf{g} = \left(\frac{1}{1}\sum\_{i=1}^{1} |\mathbf{S}\_{\mathbf{k}-\mathbf{i}}|^{\mathbf{P}}\right)^{\frac{1}{\mathbf{P}}}.\tag{14}$$

Analogous procedure is produced at the receiver with vector Y .

Let us extend idea of double stepsize calibration on complex quantization that will be useful for enhancing watermark resistance to filtering. For this purpose apply RDM approach on the case of complex multiplicative interference - exp j - <sup>0</sup> . This interference adds constant unknown phase shift <sup>0</sup> . We want to quantize complex amplitudes on a plain lattice invariantly to this shift. For that it is necessary to make QIM process transparent not only to amplitude scaling but to phase alterations also.

Suppose vector S presents complex amplitudes of narrowband signal. Passing the channel all amplitudes S changes their amplitudes by : Y S i i - and corresponding phases by <sup>0</sup> : arg Y arg S i i0 - - -. Phase of the resulting vector at the transmitter is given by

$$\boldsymbol{\phi}\_{\rm S} = \arg \left( \sum\_{i=1}^{l} \dot{\mathbf{S}}\_{i} \right). \tag{15}$$

Thanks to narrowband nature phase of resulting vector Y at the receiver will be turned on phase 0 compared to transmitter: Y S0 . Adding phase multiplier - <sup>S</sup> exp j in Eq. (14) we get complex estimation for quantization step.

Appropriate equations in vector form are presented below:

$$\mathbf{G}\_{\mathbf{k}-1} = \mathbf{g}\_{\mathbf{k}-1} \exp(\mathbf{j} \boldsymbol{\Phi}\_{\mathbf{k}-1}) \,\prime \tag{16}$$

Advantage of QIM2 over scalar QIM consist in increasing the of distance between the nearest concurring decision points. It is possible to show that, assuming equal distortions the minimal distance is 2 / 2 times greater for QIM2 procedure comparatively to scalar

Perez-Gonzalez et al., 2005 proposed techique for images processing against value-metric scaling attack, named as rational dither modulation (RDM). The main idea of RDM is snaping of quantization steps to watermarked and received signals at the transmitter and receiver correspondingly. For the first order for RDM scheme steps quantization for the current samples Xk and Yk are produced on the base of previous samples k 1 S and Yk 1 . Assuming noise absence, the influence of constant multiplier in the channel is fully

This scheme works until k 1 S 0 ' . Performance of the first order RDM scheme may by improved by high order schemes. Stepsize estimation at the transmitter is produced on the

> i 1 <sup>1</sup> g S <sup>l</sup> - 8 . 9

Let us extend idea of double stepsize calibration on complex quantization that will be useful for enhancing watermark resistance to filtering. For this purpose apply RDM approach on

constant unknown phase shift <sup>0</sup> . We want to quantize complex amplitudes on a plain lattice invariantly to this shift. For that it is necessary to make QIM process transparent not

changes their amplitudes by : Y S i i -

phase 0 compared to transmitter: Y S0 . Adding phase multiplier -

**G g** k1 k1 k1 exp j -

1

exp j -

presents complex amplitudes of narrowband signal. Passing the channel

. Phase of the resulting vector at the transmitter is given by

l S i i 1 arg S - 8 . 9

/ : ,-


/ : , . (14)

<sup>0</sup> . This interference adds

and corresponding phases by

at the receiver will be turned on

<sup>S</sup> exp j in Eq.

. (15)

, (16)

<sup>l</sup> <sup>p</sup> <sup>p</sup> k i

base of l previous watermarked samples and is given by the p norm of vector S :

Analogous procedure is produced at the receiver with vector Y .

the case of complex multiplicative interference -

only to amplitude scaling but to phase alterations also.

Thanks to narrowband nature phase of resulting vector Y-

Appropriate equations in vector form are presented below:

(14) we get complex estimation for quantization step.

QIM (see Fig. 5).

Suppose vector S

all amplitudes S

<sup>0</sup> : arg Y arg S -


 i i0 - --


eliminated on decoding process.

**4.2 Invariance to amplitude scaling and phase shift** 

The main drawback of QIM is sensitivity to amplitude scaling and filtering.

$$\mathbf{g}\_{\mathbf{k}-1} = \left(\frac{1}{1}\sum\_{i=1}^{1} \left|\mathbf{S}\_{\mathbf{k}-i}\right|^{\mathbf{P}}\right)^{\frac{1}{\mathbf{P}}} \tag{17}$$

$$\mathbf{OP}\_{\mathbf{k}-1} = \arg\left(\sum\_{i=1}^{l} \mathbf{S}\_{\mathbf{k}-i}\right) \tag{18}$$

Vector for step quantization in Eq. (12) is then given by the relation

$$
\bar{\Delta}\_{\mathbf{k}-1} = \Delta \, \mathbf{G}\_{\mathbf{k}-1} \,. \tag{19}
$$

where is certaine stepsize chosen apriory.

The same ralations shold be used at the receiver with the substitution vector **Y**k i instead of **S**k i . Note that all bold-marked vectors have length B . At the receiver detection of embedded bits is performed by using a minimum Euclidian distance rule, i.e.

$$\hat{\mathbf{m}} = \operatorname\*{arg\,min}\_{\mathbf{m} \in \mathcal{0}, 1} \left\| \mathbf{Q} (\mathbf{Y}\_{\mathbf{k}}, \mathbf{G}\_{\mathbf{k}-1}, \mathbf{m}) - \mathbf{Y}\_{\mathbf{k}} \right\|. \tag{20}$$

Relations (16) - (20) make possible to eliminate flat fading and realize QIM process fully imperceptible to slowly amplitude scaling and phase shift.

From Eq. (8) it is clear that invariant amplitude scaling and phase shift procedure, abbriviate it as IAP, can be applied to each narrowband channel. Multi channel encoder implementation is presented in Fig. 6.

Fig. 6. OFDM invariant to amplitude scaling and phase shift QIM2 system: a) transmitter; b) receiver

Coming through the channel signal is subjected to linear filtering. Thanks to prefix the received signal after linear filtering may be considered in the frequency domain according to

Audio Watermarking for Automatic Identification of

w mx is distributed along the carrier signal.

where symbol + denotes unit vector norm.

i

Detected bit is restored by means of sign function

nonblind watermarking are shown with dashed lines.

within (15 … 20) dB in comparison to SS Fig. 8 a)).

Characteristics in Fig. 8 are given for zero prefix.

**6. Adaptive cancelling of carrier signal at the receiver** 

5

following rules:

according Eq. (21).

where, y , -

**y u** .

Radiotelephone Transmissions in VHF Maritime Communication 223

more than if m 1 . It is obvious that there are situations when inserting of watermark signal is not needed at all. In this case distortions due to watermarking are absent. When do the modification of carrier signal is necessary, total correction expressed by the relation

Taking into account that after embedding process according to Eq. (21) the watermarked amplitude may take negative value, the following algorithm for calculating **w** was applied:

Sequences 0,1 **u** and 0, 1 **u** are composed from sequence **u** according the

i

Eq. (23), (24) eliminate in any case a negative amplitude values for watermarked signal

m si ˆ gn-

Comparative detection characteristics for ISS and SS according Shishkin, June 2008 are presented in Fig. 8. Characteristics are plotted as error probability function versus watermark-to-signal ratio (WSR) and watermark-to-noise ratio (WNR) both expressed in dB. Graphics are plotted for identical parameters signal-to-noise ratio (SNR) and WSR accordingly. Teoretical bounds, when host signal is completely availablee at the receiver, i.e.

Almost vertical slope of p (WSR, SNR const) er is explained by the independency of watermarking channel capacity from the carrier signal (see Eq. (2)). Gain for ISS method is

It is worth to notice that OFDM-like processing is completely applicable to ISS.

SS-based watermarking algorithm embeds one bit of information in a vector **s** of L samples:

<sup>i</sup>

5

<sup>u</sup> 0, otherwise 3 <sup>4</sup>

1, if u 1,

**u u**

**u u**

 <sup>3</sup> \$ <sup>6</sup> <sup>4</sup>

6 5 i

1, if u 1, <sup>u</sup> 0, otherwise <sup>3</sup> <sup>4</sup>

**w**

w / , if w 0,

. (23)

y (25)

**sx u** <sup>w</sup> m , (26)

(24)

w / , otherwise

Eq. (8). At the receiver prefix y ,y ,...,y is removed and the remaining samples 12 P y ,y ,...,y are used for performing DFT. Th 12 N en decoding IAP procedures are performed under coefficients Y ,Y ,...,Y 12 B -- for detecting embedded bits i mˆ , i 1,...,B in each channel

In Fig. 7 simulation results are presented for filtering attack. For channel simulation Butterworth low pass filter of order 2 an cut off frequency 0.3 was chosen. Function representing that filter in MatLab is: [b,a]=butter(2,.3). Length of impulse response for that filter is 8. Additive noise is absent. Number of watermarked channel is B 8 .

It is seen that even for zero prefix received complex amplitudes Y are scattered because of ISI influence (Fig. 7 a). Increasing P leads to concentration of Y around centroids S - , and for P 8 dispersion is nearly zero: Y S - -.

Fig. 7. Influence of prefix on scattering for OFDM IAP QIM2 algorithm: a) no prefix *P* 0 ; b) moderate prefix *P* 4 ; c) full prefix *P* 8

#### **5. Improved Spread Spectrum (ISS)**

Malvar & Florencio, 2003 introduced ISS method for robust watermarking. In spite of the title ISS radically distinguishes from common SS due to utilization of information about carrier signal. When compared with traditional SS, the signal doesn't act as noise source. Thanks to that ISS encoding algorithm refers to informed encoding and may be treated as binary QIM. The main idea of ISS is to look ahead across the carrier signal and relying on it generate the appropriate watermark signal. Just as for standard SS chip sequence **u** is used but with the coefficient x,b :

$$\mathbf{s} = \mathbf{x} + \mu(\tilde{\mathbf{x}}, \mathbf{m})\mathbf{u}\prime \tag{21}$$

where *x* **x u**, , *m* 1,1 - embedded bit.

Here inner product is defined as:

$$\mathbf{u}(\mathbf{x}, \mathbf{u}) = \sum\_{i=1}^{L} \mathbf{x}\_i \mathbf{u}\_i \tag{22}$$

The main idea of ISS is to generate watermark vector **w u** x,m in such mode that inner product -**s u**, would give the value not less than certain threshold if m 1 and not more than if m 1 . It is obvious that there are situations when inserting of watermark signal is not needed at all. In this case distortions due to watermarking are absent. When do the modification of carrier signal is necessary, total correction expressed by the relation w mx is distributed along the carrier signal.

Taking into account that after embedding process according to Eq. (21) the watermarked amplitude may take negative value, the following algorithm for calculating **w** was applied:

$$\mathbf{w} = \begin{cases} \tilde{\mathbf{w}} \mathbf{u}^- / \left\| \mathbf{u}^- \right\|\_{\prime} & \text{if} \quad \tilde{\mathbf{w}} \ge 0, \\\tilde{\mathbf{w}} \mathbf{u}^+ / \left\| \mathbf{u}^+ \right\|\_{\prime} & \text{otherwise} \end{cases} \tag{23}$$

where symbol + denotes unit vector norm.

Sequences 0,1 **u** and 0, 1 **u** are composed from sequence **u** according the following rules:

$$\mathbf{u}\_{\mathbf{i}}^{+} = \begin{cases} \mathbf{1}, & \text{if} \quad \mathbf{u}\_{\mathbf{i}} = \mathbf{1}, \\ \mathbf{0}, & \text{otherwise} \end{cases} \quad \mathbf{u}\_{\mathbf{i}}^{-} = \begin{cases} -\mathbf{1}, & \text{if} \quad \mathbf{u}\_{\mathbf{i}} = -\mathbf{1}, \\ \mathbf{0}, & \text{otherwise} \end{cases} \tag{24}$$

Eq. (23), (24) eliminate in any case a negative amplitude values for watermarked signal according Eq. (21).

Detected bit is restored by means of sign function

$$\hat{\mathbf{m}} = \text{sign}(\tilde{\mathbf{y}}) \tag{25}$$

where, y , **y u** .

222 Watermarking – Volume 2

Eq. (8). At the receiver prefix y ,y ,...,y is removed and the remaining samples 12 P y ,y ,...,y are used for performing DFT. Th 12 N en decoding IAP procedures are performed

In Fig. 7 simulation results are presented for filtering attack. For channel simulation Butterworth low pass filter of order 2 an cut off frequency 0.3 was chosen. Function representing that filter in MatLab is: [b,a]=butter(2,.3). Length of impulse response for that

a) b) c)


Malvar & Florencio, 2003 introduced ISS method for robust watermarking. In spite of the title ISS radically distinguishes from common SS due to utilization of information about carrier signal. When compared with traditional SS, the signal doesn't act as noise source. Thanks to that ISS encoding algorithm refers to informed encoding and may be treated as binary QIM. The main idea of ISS is to look ahead across the carrier signal and relying on it generate the appropriate watermark signal. Just as for standard SS chip sequence **u** is used

**sx u** -

i i i 1 , xu 

**s u**, would give the value not less than certain threshold if m 1 and not


The main idea of ISS is to generate watermark vector **w u** -

filter is 8. Additive noise is absent. Number of watermarked channel is B 8 .

 -.

It is seen that even for zero prefix received complex amplitudes Y-

Fig. 7. Influence of prefix on scattering for OFDM IAP QIM2 algorithm:

a) no prefix *P* 0 ; b) moderate prefix *P* 4 ; c) full prefix *P* 8


x,b :

**x u**, , *m* 1,1 - embedded bit.

ISI influence (Fig. 7 a). Increasing P leads to concentration of Y-

for detecting embedded bits i mˆ , i 1,...,B in each channel



around centroids S


x,m , (21)

x,m in such mode that inner

**x u** , . (22)

under coefficients Y ,Y ,...,Y 12 B --

for P 8 dispersion is nearly zero: Y S -

**5. Improved Spread Spectrum (ISS)** 


but with the coefficient -

Here inner product is defined as:

where *x* -


product -


Comparative detection characteristics for ISS and SS according Shishkin, June 2008 are presented in Fig. 8. Characteristics are plotted as error probability function versus watermark-to-signal ratio (WSR) and watermark-to-noise ratio (WNR) both expressed in dB. Graphics are plotted for identical parameters signal-to-noise ratio (SNR) and WSR accordingly. Teoretical bounds, when host signal is completely availablee at the receiver, i.e. nonblind watermarking are shown with dashed lines.

Almost vertical slope of p (WSR, SNR const) er is explained by the independency of watermarking channel capacity from the carrier signal (see Eq. (2)). Gain for ISS method is within (15 … 20) dB in comparison to SS Fig. 8 a)).

It is worth to notice that OFDM-like processing is completely applicable to ISS. Characteristics in Fig. 8 are given for zero prefix.

#### **6. Adaptive cancelling of carrier signal at the receiver**

SS-based watermarking algorithm embeds one bit of information in a vector **s** of L samples:

$$\mathbf{s} = \mathbf{x} + \sigma\_{\mathbf{w}} \mathbf{m} \,\mathrm{u}\,\mathrm{u}\,\tag{26}$$

Audio Watermarking for Automatic Identification of

between signal samples and theirs predicted values.

step i is expressed through the preceding samples:

Prediction error comes to

time series y based on past samples.



Radiotelephone Transmissions in VHF Maritime Communication 225

Verbal signal is a nonstationary random process and it requires adaptation of transfer function. For WF implementation let us apply linear prediction method. Basic principle of linear prediction consists in presentation of predicting sample through the linear combination of previous samples. Weighting coefficients in the linear combination are calculated on the basis of mean squared error minimization for prediction, i.e. differences

Linear prediction method in the view of so called linear predictive coding (LPC) is widely used in the audio compressing algorithms. LPC algorithm assumes partition of audio signal on frames of duration approximately 20 msec. For every segment weighting coefficients of WF, which would minimize mean squared error of prediction are calculated. Prediction at

y i h y i 1 h y i 2 ... h y i p pr 1 -

e i y i y i h y i k


Coefficients hk , k 1,2,...,p in Eq. (29) are subject of adaptation, h 1 <sup>0</sup> .

Fig. 9. Optimal receiver for watermark on the background of correlated noise

Algorithm for finding coefficients hi is well elaborated mathematically. It is based on Yule-Walker equation and computing procedure by the Levinson-Durbin algorithm (O'Shaughnessy, 2000). In MatLab function lpc(y,p) finds the coefficients of a p -order linear predictor (finite impulse response filter) that predicts the current value of the real-valued

LPC algorithm is block and applies the samples from fixed time interval. During filtration coefficients of WF doesn't vary within the frame. Coming to next frame the coefficients

 -<sup>2</sup> <sup>p</sup> -


p pr k k 0

, . (29)

. (28)

where w is wanted root mean square deviation of watermark, m 1, 1 - information bit to be embedded, **u** u ,u ,...,u 12 L - binary pseudo random sequence, u 1, 1 <sup>i</sup> . If the channel modeled as additive noise, the received signal is: **ysn** .

Fig. 8. Comparative error probability functions for ISS and SS methods: a) *er p* versus *WSR* , *SNR* 20 dB, b) *er p* versus *WNR* , *WSR* -30 dB

Detection at the receiver is performed by the correlator that computes inner product

$$\mathbf{y} = (\mathbf{y}, \mathbf{u}) = (\mathbf{x}, \mathbf{u}) + \sigma\_{\mathbf{w}} \, \mathbf{m} \, \mathrm{L} + (\mathbf{n}, \mathbf{u}) \, \mathrm{.} \tag{27}$$

Detected bit is estimated according to Eq. (25). One can see that interferences for watermark are terms x , - **x u** and n , **n u** in Eq. (27). Suppose vectors **x** and **n** are uncorrelated random processes. Therefore terms in Eq. (27) x , n grows proportionally L , meanwhile useful term <sup>w</sup> mL is proportional to L . Processing gain is L . It is possible to yield desired probability of detection through the appropriate chip length L .

Another possibility for achieving the goal appears when **x** represents a correlated process. Practically x is a really highly correlated process. And just this process has a maximal influence on watermark.

In communication optimal receiving algorithms in the presence of correlated noise are well developed (Van Trees, 1968). Scheme for optimal receiver on the background correlated noise is based on whitening filter (WF) and presented in Fig. 9. WF is based on predictor and forms at the output an error signal e i between an actual sample and predicted one. For - good predictor e i looks like white noise with less power compared to power of input - signal y i . -

Verbal signal is a nonstationary random process and it requires adaptation of transfer function. For WF implementation let us apply linear prediction method. Basic principle of linear prediction consists in presentation of predicting sample through the linear combination of previous samples. Weighting coefficients in the linear combination are calculated on the basis of mean squared error minimization for prediction, i.e. differences between signal samples and theirs predicted values.

Linear prediction method in the view of so called linear predictive coding (LPC) is widely used in the audio compressing algorithms. LPC algorithm assumes partition of audio signal on frames of duration approximately 20 msec. For every segment weighting coefficients of WF, which would minimize mean squared error of prediction are calculated. Prediction at step i is expressed through the preceding samples:

$$\mathbf{y}\_{\text{pr}}\left(\mathbf{i}\right) = \mathbf{h}\_1\mathbf{y}\left(\mathbf{i}-\mathbf{1}\right) + \mathbf{h}\_2\mathbf{y}\left(\mathbf{i}-\mathbf{2}\right) + \dots + \mathbf{h}\_p\mathbf{y}\left(\mathbf{i}-\mathbf{p}\right). \tag{28}$$

Prediction error comes to

224 Watermarking – Volume 2

where w is wanted root mean square deviation of watermark, m 1, 1 - information bit to be embedded, **u** u ,u ,...,u 12 L - binary pseudo random sequence, u 1, 1 <sup>i</sup> .

10-3

10-2

Error Probability

10-1

100

Fig. 8. Comparative error probability functions for ISS and SS methods: a) *er p* versus *WSR* ,

y , , mL , -

Detected bit is estimated according to Eq. (25). One can see that interferences for watermark

random processes. Therefore terms in Eq. (27) x , n grows proportionally L , meanwhile useful term <sup>w</sup> mL is proportional to L . Processing gain is L . It is possible to yield

Another possibility for achieving the goal appears when **x** represents a correlated process. Practically x is a really highly correlated process. And just this process has a maximal

In communication optimal receiving algorithms in the presence of correlated noise are well developed (Van Trees, 1968). Scheme for optimal receiver on the background correlated noise is based on whitening filter (WF) and presented in Fig. 9. WF is based on predictor and forms at the output an error signal e i between an actual sample and predicted one. For -

good predictor e i looks like white noise with less power compared to power of input -

<sup>w</sup> -

**n u** in Eq. (27). Suppose vectors **x** and **n** are uncorrelated

**n u** . (27)


WNR

**ISS**

**SS**

Detection at the receiver is performed by the correlator that computes inner product

**yu xu** -

desired probability of detection through the appropriate chip length L .

a) b)

**ISS SS**

*SNR* 20 dB, b) *er p* versus *WNR* , *WSR* -30 dB


WSR

**x u** and n , -

are terms x , -

10-3

10-2

Error Probability

10-1

100

signal y i . -

influence on watermark.

If the channel modeled as additive noise, the received signal is: **ysn** .

$$\mathbf{e}\left(\mathbf{i}\right) = \mathbf{y}\left(\mathbf{i}\right) - \mathbf{y}\_{\text{pr}}\left(\mathbf{i}\right) = \sum\_{\mathbf{k}=0}^{\mathbf{P}} \mathbf{h}\_{\text{k}} \mathbf{y}\left(\mathbf{i} - \mathbf{k}\right) \,. \tag{29}$$

Coefficients hk , k 1,2,...,p in Eq. (29) are subject of adaptation, h 1 <sup>0</sup> .

Fig. 9. Optimal receiver for watermark on the background of correlated noise

Algorithm for finding coefficients hi is well elaborated mathematically. It is based on Yule-Walker equation and computing procedure by the Levinson-Durbin algorithm (O'Shaughnessy, 2000). In MatLab function lpc(y,p) finds the coefficients of a p -order linear predictor (finite impulse response filter) that predicts the current value of the real-valued time series y based on past samples.

LPC algorithm is block and applies the samples from fixed time interval. During filtration coefficients of WF doesn't vary within the frame. Coming to next frame the coefficients

Audio Watermarking for Automatic Identification of

frequency domain without any delay for transmitting signal.

*on Information Theory*, Vol. 47, No. 4, pp. 1423-1443

*Control and Information Theory*, Vol. 9, No.1, pp. 19 – 31

Ltd, ISBN 0-470-09178-9, Chichester, England

*Peredachi Inf,* Vol. 10, No.2, pp. 52-60, UDC 621.391.15

978-1408114933, London

372585-1, Burlington, MA, USA

1624 - 1637, ISSN 1558-7916

No.4, pp.898 – 905

pp. 2083 – 2126

pp. 439 – 441

(coefficient).

method.

**8. References** 

Radiotelephone Transmissions in VHF Maritime Communication 227

one bit – many samples (or coefficients) instead of principle one bit – one sample

Improved SS watermarking inherently resist to amplitude scaling. Noise immunity may be achieved by means of exchange embedding rate and noise robustness. For existing radiotelephone channel of frequency band (300 … 3000) Hz and signal-to-noise ratio about 15 dB realistic embedding rate forms about 60 bit/sec. This rate is quite sufficient for reliable identification. Presently ISS watermarking appears to be the most acceptable

Traditional SS method with additional processing at the receiver gives about 40 bit/sec rate. Absolute advantage of this method is absence of delay and simple encoding at the transmitter. OFDM-like pre-processing procedure makes possible to watermark in the

Brehaut, D. (2009). *GMDSS: A User's Handbook* (Forth edition), Adlard Coles Nautical, ISBN

Chen, B. & Wornell, G. (2001). Quantization index modulation: a class of provably good

osta, M. (1983). Writing on dirty paper*. IEEE Transactions on Information Theory*, Vol. IT-29,

Cox, I.; Miller, M.; Bloom, J.; Fridrich, J. & Kalker, T. (2008). *Digital Watermarking and* 

Gel'fand, S. & Pinsker, M. (1980). Coding for channel with random parameters. *Problems of* 

Hofbauer, K. & Kubin, G. (2006). Aeronautical voice radio channel modelling and

Ipatov, V. (2005). *Spread Spectrum and CDMA: Principles and Applications,* John Wiley & Sons,

Kuznetsov, A. & Tsybakov, B. (1974). Coding in a Memory with Random Parameters. *Probl.* 

Malvar, H. & Florencio, D. (2003). Improved Spread Spectrum: A New Modulation

Moulin, P. & Koetter, R. (2005). Data-Hiding Codes. *Proceedings of the IEEE*, Vol.93, No.12,

*Research in Air Transportation (ICRAT)*, Belgrade, Serbia, Available from http://www3.spsc.tugraz.at/people/hofbauer/papers/ Hofbauer\_ICRAT\_2006.pdf Hofbauer, K. et al. (2009). Speech watermarking for analog flat-fading bandpass channels,

methods for digital watermarking and information embedding. *IEEE Transactions* 

*Steganography*, (Second edition), Morgan Kaufmann Publishers, ISBN 978-0-12-

simulation—a tutorial review, In: *Proceedings of the International Conference on* 

*IEEE Transactions on Audio, Speech, and Language Processing*, 2009, Vol.17, No.8, pp.

Technique for Robast Watermarking, *IEEE Transactions on Signal Processing*, Vol.51,

should be recalculated. Other prediction procedures may use continuous adaptation algorithms, for example, Least Mean Square (LMS) or Recursive Least Square (RLS) algorithm. Shishkin, October 2008 simulated above mentioned algorithms for SSwatermarked speech signals (Fig. 10 a)). In Fig. 10 a) time axes are marked in sample numbers, assuming sampling frequency F 22050 Hz <sup>s</sup> , WF order p 7 .

Waveforms shows that prediction error is less for LPC algorithm compared to LMS and RLS. Time segments for samples 1 – 2000, 2000 – 4000 and 8000 – 10000 correspond to vowel sounds and have quite negligible prediction error because appropriate signal is highly correlated. On the other hand consonant speech sound (samples 4000 – 6000) is closer to white noise and that's why it is predicted worse.

Comparative detection characteristic for SS watermarking in AWGN channel are presented in Fig. 10 b). Watermarking was executed in the time domain. It is seen that additional processing at the receiver effectively suppress correlated speech signal. Processing gain makes approximately 15 dB.

Fig. 10. Simulation results for adaptive carrier signal cancellation: a) signal waveforms after whitening according LPC, LMS, RLS algorithms; b) error probability as a function of WSR (Shishkin, October 2008)

#### **7. Conclusion**

Proposed OFDM technology application to QIM watermarking makes possible to resist against intersymbol interference, that is induced by multipath propagation and bandlimited nature of VHF radiotelephone channel. OFDM-QIM integration makes QIM watermarked signal to be invariant against amplitude scaling and phase shift. Reasonable prefix length about P 2...4 to cancel ISI is acceptable for the total OFDM symbol size about 500 samples. The main restricted factor is reliable estimation for step quantization on the background of additive noise. One of the possible ways to resist against noise is one bit distribution on numerous samples, i.e. application of the principle one bit – many samples (or coefficients) instead of principle one bit – one sample (coefficient).

Improved SS watermarking inherently resist to amplitude scaling. Noise immunity may be achieved by means of exchange embedding rate and noise robustness. For existing radiotelephone channel of frequency band (300 … 3000) Hz and signal-to-noise ratio about 15 dB realistic embedding rate forms about 60 bit/sec. This rate is quite sufficient for reliable identification. Presently ISS watermarking appears to be the most acceptable method.

Traditional SS method with additional processing at the receiver gives about 40 bit/sec rate. Absolute advantage of this method is absence of delay and simple encoding at the transmitter. OFDM-like pre-processing procedure makes possible to watermark in the frequency domain without any delay for transmitting signal.

#### **8. References**

226 Watermarking – Volume 2

should be recalculated. Other prediction procedures may use continuous adaptation algorithms, for example, Least Mean Square (LMS) or Recursive Least Square (RLS) algorithm. Shishkin, October 2008 simulated above mentioned algorithms for SSwatermarked speech signals (Fig. 10 a)). In Fig. 10 a) time axes are marked in sample

Waveforms shows that prediction error is less for LPC algorithm compared to LMS and RLS. Time segments for samples 1 – 2000, 2000 – 4000 and 8000 – 10000 correspond to vowel sounds and have quite negligible prediction error because appropriate signal is highly correlated. On the other hand consonant speech sound (samples 4000 – 6000) is closer to

Comparative detection characteristic for SS watermarking in AWGN channel are presented in Fig. 10 b). Watermarking was executed in the time domain. It is seen that additional processing at the receiver effectively suppress correlated speech signal. Processing gain

numbers, assuming sampling frequency F 22050 Hz <sup>s</sup> , WF order p 7 .

a) b)

Fig. 10. Simulation results for adaptive carrier signal cancellation: a) signal waveforms after whitening according LPC, LMS, RLS algorithms; b) error probability as a function of WSR

10-4

10-3

Âåðî ÿòí î ñòü î ø èáêè

Error Probability

10-2

10-1

100


WSR

Ñ î áðàáî òêî é Áåç î áðàáî òêè

LPC processing No processing

, äÁ

N\*sigmaW / sigmaX

Proposed OFDM technology application to QIM watermarking makes possible to resist against intersymbol interference, that is induced by multipath propagation and bandlimited nature of VHF radiotelephone channel. OFDM-QIM integration makes QIM watermarked signal to be invariant against amplitude scaling and phase shift. Reasonable prefix length about P 2...4 to cancel ISI is acceptable for the total OFDM symbol size about 500 samples. The main restricted factor is reliable estimation for step quantization on the background of additive noise. One of the possible ways to resist against noise is one bit distribution on numerous samples, i.e. application of the principle

white noise and that's why it is predicted worse.

0 2000 4000 6000 8000 10000

0 2000 4000 6000 8000 10000

0 2000 4000 6000 8000 10000

0 2000 4000 6000 8000 10000

makes approximately 15 dB.





(Shishkin, October 2008)

**7. Conclusion** 


**10** 

Jiunn-Lin Wu

*Taiwan* 

**Robust Watermarking Framework for High Dynamic Range Images Against** 

As digital cameras become more and more popular recently, it is very easy for us to take many digital photos. Unfortunately, they are rarely true measurements of relative radiance in the scene due to the limited dynamic range in the image acquisition devices. High dynamic range (HDR) images emphasis in image processing fields because they can accommodate a greater dynamic range of luminance between the brightest and darkest parts of an image. Dynamic range is the ratio between the brightest and darkest luminance values of a scene. In general, human eyes can handle a very large dynamic range of approximately 100000:1 in a single view. However, a standard photo taken with a standard camera with film or an electronic imaging array always has a limited dynamic range [1]. A standard image, called a LDR image, cannot reproduce the luminance ratio observed in the real world. A scene containing very bright highlights and deep shadows always loses some detail if the exposure time is not suitably determined. Over the past decade, many researchers have developed HDR imaging techniques (Debevec & Malik, 1997)(Reinhard *et al*, 2005) (Reinhard *et al*, 2007). Debevec and Malik proposed a method to recover the single high dynamic range radiance map from multiple images with different exposure times (Debevec & Malik, 1997), this method has been implemented in

The reconstruction of a high dynamic range image is a complex process. Producing an HDR image is by capturing multiple images of the same scene with different exposure levels and merging them into a single HDR image (Debevec & Malik, 1997). The photographers can use a tripod in order to capture the same scene and avoid image registration problems. However, if the differently exposed image sequences are took hand-held, an image registration method which is robust to the illumination changes and moving objects should be used to align the multiple input images before HDR image composition. In addition, the user must be able to use the exposure bracketing technique to ensure the pictures are

**1. Introduction** 

many HDR software.

properly exposed.

**1.1 Background explanation** 

**Tone-Mapping Attacks** 

*Dept. of Computer Science and Engineering National Chung Hsing University, Taichung* 


## **Robust Watermarking Framework for High Dynamic Range Images Against Tone-Mapping Attacks**

#### Jiunn-Lin Wu

*Dept. of Computer Science and Engineering National Chung Hsing University, Taichung Taiwan* 

#### **1. Introduction**

228 Watermarking – Volume 2

O'Shaughnessy, D. (2000). *Speech communication: human and machine,* (Second edition), IEEE,

Perez-Gonzalez, F. et al. (2005). Rational Dither Modulation: A High Rate Data-Hiding

Shishkin, A. (June 2008). Digital Watermarks with Spectrum Spreading for Audio Signals

Shishkin, A. (October 2008). Adaptive Algorithms Application in Sound Steganographic

Van Trees, H. (1968). *Detection, Estimation, and Modulation Theory*, (First edition), John Wiley

Method Invariant to Gain Attacks. *IEEE Transactions on Signal Processing*, Vol.53,

Using the Signal Carrier Information. *Radioelectronics and Communication Systems,*

Systems with Signal Spectrum Broadening. *Radioelectronics and Communication* 

Inc. New York, ISBN 0-7803-3449-3

Vol. 51, No.6, pp.308-315, ISSN 0735-2727

& Sons Inc, ISBN 978-0471899556

*Systems,* Vol.51, No.10, pp.524-530, ISSN 0735-2727

No.10, pp.3960–3975

#### **1.1 Background explanation**

As digital cameras become more and more popular recently, it is very easy for us to take many digital photos. Unfortunately, they are rarely true measurements of relative radiance in the scene due to the limited dynamic range in the image acquisition devices. High dynamic range (HDR) images emphasis in image processing fields because they can accommodate a greater dynamic range of luminance between the brightest and darkest parts of an image. Dynamic range is the ratio between the brightest and darkest luminance values of a scene. In general, human eyes can handle a very large dynamic range of approximately 100000:1 in a single view. However, a standard photo taken with a standard camera with film or an electronic imaging array always has a limited dynamic range [1]. A standard image, called a LDR image, cannot reproduce the luminance ratio observed in the real world. A scene containing very bright highlights and deep shadows always loses some detail if the exposure time is not suitably determined. Over the past decade, many researchers have developed HDR imaging techniques (Debevec & Malik, 1997)(Reinhard *et al*, 2005) (Reinhard *et al*, 2007). Debevec and Malik proposed a method to recover the single high dynamic range radiance map from multiple images with different exposure times (Debevec & Malik, 1997), this method has been implemented in many HDR software.

The reconstruction of a high dynamic range image is a complex process. Producing an HDR image is by capturing multiple images of the same scene with different exposure levels and merging them into a single HDR image (Debevec & Malik, 1997). The photographers can use a tripod in order to capture the same scene and avoid image registration problems. However, if the differently exposed image sequences are took hand-held, an image registration method which is robust to the illumination changes and moving objects should be used to align the multiple input images before HDR image composition. In addition, the user must be able to use the exposure bracketing technique to ensure the pictures are properly exposed.

Robust Watermarking Framework for High

Tone Mapping Operator

HDR Image

images.

Dynamic Range Images Against Tone-Mapping Attacks 231

version of HDR original image is accompanied by restorative information in the standard 24-bit RGB format. This sub-band in JPEG format contains a compressed ratio image, which can be used to recovers the original high dynamic range image by multiplying the tonemapped foreground by the ratio image. Figure 1 illustrates the flowchart of the proposed

Image Robust

Fig. 1. Illustration of the proposed watermark embedding process for high dynamic range

A tone-mapped LDR image is first generated by a tone-mapping operator from the original high dynamic range image. A ratio image is then obtained by dividing the HDR image by the tone-mapped LDR image. Any conventional watermarking technique for LDR images can be applied to embed the watermark bits into the tone-mapped LDR image. Finally, by multiplying the watermarked image by the ratio image, the HDR image with watermark is then obtained. Due to the watermark is embedded in the tone mapped LDR image, the

Discrete cosine transform (DCT) is widely used in signal and image processing for lossy data compression. To demonstrate the powerfulness of the proposed HDR image watermarking scheme, a simple DCT-based watermarking method is adopted to embed the watermark into the tone-mapped LDR image. We transformed the LDR image into the frequency domain and embedded the watermark into the lower middle-frequency blocks, for example the coefficient *DCT*3,3 of each 8 8 % image block. We used a neighboring difference-based method to embed the watermark as shown in Table 1, where *X* denotes a block *DCT*3,3 , *Y* denotes its neighboring block *DCT*2,4 , and *X* denotes the *X* block after the bit is embedded. If the watermark bit is 1, *X* is set to be larger than or equal to *Y* . If the bit is 0, *X* is set to be smaller than *Y* . After embedding the watermark, the watermarked LDR image is obtained by inversely transforming from the modified DCT image. It is obvious that the modification is invisible and the watermark is robust to various common attacks, such as blurring and noising and cropping. Finally multiplying the watermarked LDR image by the ratio image produces the watermarked high dynamic range image.

To improve the security to the proposed watermarking technique, the index table of the watermark is disturbed by a 1-D binary pseudo-random sequence using the private key as seed. The encrypted watermark is then embedded into the tone mapped LDR image and

yields the watermarked HDR image using the procedure shown in Fig. 1.

proposed HDR watermarking scheme is robust against the tone mapping attacks.

Watermarking Encoding

Ratio Image

Watermarked Image

Watermark

watermark embedding framework for high dynamic range images.

24-bits True Color


#### **1.2 Research motivation**

Obviously, it is quite an achievement to create a high dynamic range image containing pixel values that span the whole tonal range of real world scenes. It takes efforts not only to capture the differently exposed photographs as input, but also to reconstruct high dynamic range image by the techniques of image registration and image composition. The copyright protection for HDR images has become increasingly important.

Image watermarking is a common method of proving ownership or determining origin (Tsang & Au, 2001). Unintentionally destroyed watermarks happen when transmitting an image. Since pirates may also seek to remove the watermark or make it undetectable, the watermark must be robust to common attacks. Some of the common problems include noising, blurring, cropping and geometric distortions. Several watermarking schemes embed the watermark in the transformed domain (Piva *et al*, 1997)(Barni *et al*, 1999)(Wang *et al*, 2002) (Suhail. & Obaidat, 2003), which is robust to common image processing attacks, such as low-pass filtering or JPEG compression. However, the watermarking methods in the literature paid attention on the conventional LDR images, they can not be applied to the high dynamic range images.

#### **1.3 The purpose of research**

The challenge of the watermarking techniques for high dynamic range images is the tone mapping operators in which we usually use them to convert a high dynamic range image to the conventional low dynamic range image. Tone mapping is necessary for rendering an HDR image on low dynamic range devices such as standard screens or printers. This chapter presents a new watermarking framework for HDR images to alleviate the problem of tone mapping distortions. To demonstrate the powerfulness of the proposed method, a simple DCT-based watermarking technique for conventional LDR images is used. We embed the watermarking in the middle frequency DCT coefficients of the tone-mapped LDR image, the ratio image is then multiplied to recover the HDR values, where the ratio image is computed by dividing the original HDR image at each pixel by the tone-mapped luminance. Experimental results shows the watermarked HDR image keeps high visual quality and the embedded watermark using the proposed technique is robust to varying degree to tone mapping distortions, low-pass filtering, noise contamination and cropping.

#### **2. HDR watermarking technique for HDR images**

This section presents an efficient and robust watermarking algorithm for high dynamic range images. Using this blind watermarking algorithm, the watermark extraction is without the original image. The most common process for HDR images is tone mapping. Tone mapping HDR images to LDR images reveals highlights and shadow details on standard LDR devices. The aim of the proposed method is to develop a watermarking scheme against the process of tone mapping.

#### **2.1 Watermark embedding**

The key idea of the proposed method is triggered by a sub-band encoding algorithm for high dynamic range images (Ward & Simmons, (2004), the new lossy HDR high dynamic range image format is backwards compatible with existing JPEG software. A tone-mapped

Obviously, it is quite an achievement to create a high dynamic range image containing pixel values that span the whole tonal range of real world scenes. It takes efforts not only to capture the differently exposed photographs as input, but also to reconstruct high dynamic range image by the techniques of image registration and image composition. The copyright

Image watermarking is a common method of proving ownership or determining origin (Tsang & Au, 2001). Unintentionally destroyed watermarks happen when transmitting an image. Since pirates may also seek to remove the watermark or make it undetectable, the watermark must be robust to common attacks. Some of the common problems include noising, blurring, cropping and geometric distortions. Several watermarking schemes embed the watermark in the transformed domain (Piva *et al*, 1997)(Barni *et al*, 1999)(Wang *et al*, 2002) (Suhail. & Obaidat, 2003), which is robust to common image processing attacks, such as low-pass filtering or JPEG compression. However, the watermarking methods in the literature paid attention on the conventional LDR images, they can not be applied to the

The challenge of the watermarking techniques for high dynamic range images is the tone mapping operators in which we usually use them to convert a high dynamic range image to the conventional low dynamic range image. Tone mapping is necessary for rendering an HDR image on low dynamic range devices such as standard screens or printers. This chapter presents a new watermarking framework for HDR images to alleviate the problem of tone mapping distortions. To demonstrate the powerfulness of the proposed method, a simple DCT-based watermarking technique for conventional LDR images is used. We embed the watermarking in the middle frequency DCT coefficients of the tone-mapped LDR image, the ratio image is then multiplied to recover the HDR values, where the ratio image is computed by dividing the original HDR image at each pixel by the tone-mapped luminance. Experimental results shows the watermarked HDR image keeps high visual quality and the embedded watermark using the proposed technique is robust to varying degree to tone mapping distortions, low-pass filtering, noise contamination and cropping.

This section presents an efficient and robust watermarking algorithm for high dynamic range images. Using this blind watermarking algorithm, the watermark extraction is without the original image. The most common process for HDR images is tone mapping. Tone mapping HDR images to LDR images reveals highlights and shadow details on standard LDR devices. The aim of the proposed method is to develop a watermarking

The key idea of the proposed method is triggered by a sub-band encoding algorithm for high dynamic range images (Ward & Simmons, (2004), the new lossy HDR high dynamic range image format is backwards compatible with existing JPEG software. A tone-mapped

protection for HDR images has become increasingly important.

**2. HDR watermarking technique for HDR images** 

scheme against the process of tone mapping.

**2.1 Watermark embedding** 

**1.2 Research motivation** 

high dynamic range images.

**1.3 The purpose of research** 

version of HDR original image is accompanied by restorative information in the standard 24-bit RGB format. This sub-band in JPEG format contains a compressed ratio image, which can be used to recovers the original high dynamic range image by multiplying the tonemapped foreground by the ratio image. Figure 1 illustrates the flowchart of the proposed watermark embedding framework for high dynamic range images.

Fig. 1. Illustration of the proposed watermark embedding process for high dynamic range images.

A tone-mapped LDR image is first generated by a tone-mapping operator from the original high dynamic range image. A ratio image is then obtained by dividing the HDR image by the tone-mapped LDR image. Any conventional watermarking technique for LDR images can be applied to embed the watermark bits into the tone-mapped LDR image. Finally, by multiplying the watermarked image by the ratio image, the HDR image with watermark is then obtained. Due to the watermark is embedded in the tone mapped LDR image, the proposed HDR watermarking scheme is robust against the tone mapping attacks.

Discrete cosine transform (DCT) is widely used in signal and image processing for lossy data compression. To demonstrate the powerfulness of the proposed HDR image watermarking scheme, a simple DCT-based watermarking method is adopted to embed the watermark into the tone-mapped LDR image. We transformed the LDR image into the frequency domain and embedded the watermark into the lower middle-frequency blocks, for example the coefficient *DCT*3,3 of each 8 8 % image block. We used a neighboring difference-based method to embed the watermark as shown in Table 1, where *X* denotes a block *DCT*3,3 , *Y* denotes its neighboring block *DCT*2,4 , and *X* denotes the *X* block after the bit is embedded. If the watermark bit is 1, *X* is set to be larger than or equal to *Y* . If the bit is 0, *X* is set to be smaller than *Y* . After embedding the watermark, the watermarked LDR image is obtained by inversely transforming from the modified DCT image. It is obvious that the modification is invisible and the watermark is robust to various common attacks, such as blurring and noising and cropping. Finally multiplying the watermarked LDR image by the ratio image produces the watermarked high dynamic range image.

To improve the security to the proposed watermarking technique, the index table of the watermark is disturbed by a 1-D binary pseudo-random sequence using the private key as seed. The encrypted watermark is then embedded into the tone mapped LDR image and yields the watermarked HDR image using the procedure shown in Fig. 1.

Robust Watermarking Framework for High

 1 ( , )1 

A larger

Dynamic Range Images Against Tone-Mapping Attacks 233

where *W* and *W* denote the embedded watermark and the extracted watermark respectively and *NW* is the length of the watermark. The coefficient is bounded by

2 2

 ,

Figure 4 shows a high dynamic range image –hot spring, which is recovered from six LDR photographs with different exposure time, the image size is 640 480 % . It is used for experiments to prove the robustness of the proposed method against common signal processing attacks on the watermarked HDR image. We first converted the original HDR image to its corresponding LDR image using a tone mapping operation and then computed the ratio image. The dynamic range compression algorithm based on fast bilateral filtering (Durand & Dorsey, 2002) is used as the tone mapping operator in the watermark embedding procedure. A simple DCT- based watermarking method is then used to embed the logo

Figure 4 shows a high dynamic range image –hot spring, which is recovered from six LDR photographs with different exposure time, the image size is 640 480 % . It is used for experiments to prove the robustness of the proposed method against common signal processing attacks on the watermarked HDR image. We first converted the original HDR image to its corresponding LDR image using a tone mapping operation and then computed the ratio image. The dynamic range compression algorithm based on fast bilateral filtering [10] is used as the tone mapping operator in the watermark embedding procedure. A simple DCT- based watermarking method is then used to embed the logo watermark shown in Fig.

Figure 5(a) depicts the tone mapped LDR image, and the watermarked LDR image is shown in Fig. 5(b). This figure shows that the two images are visually indistinguishable and the peak signal-to-noise ratio (PSNR) value between them is 41.48 dB. Finally the watermarked HDR image is produced by multiplying the watermarked LDR image by the ratio image.

Table 2 shows the result of the watermarked high dynamic range image corrupted by cropping, blurring and noising attacks. The normalized correlation coefficient between the extracted and original watermark are all higher than 0.3, and the extracted watermarks are distinguishable. It shows the effectiveness of the proposed HDR watermarking algorithm.

Mean square error (MSE) is not a good performance index to measure the difference between the original high dynamic range image and the watermarked one, it is because that the intensity range of the high dynamic range radiance map recovered by different approaches or programs are quite varying. In order to provide a fair measurement to the

*w wN*

( )

*m*

*m m*

*w*

*N*

*w w*

, , (2)

(3)

*m mw*

*W W* . Since the watermark is a binary sequence of #1 , we have

*N N w w*

*m m*

The normalized correlation coefficient can also be written as

indicates a better retrieval performance.

watermark shown in Fig. 3 into the tone mapped LDR image.

3 into the tone mapped LDR image.

1 1

(, )

*W W*


Table 1. DCT-based watermark embedding algorithm.

#### **2.2 Watermark extraction**

Figure 2 shows the flowchart for extracting the watermark. We first use the tone mapping operator to convert the corrupted watermarked HDR image to the watermarked LDR image. The watermark detection method for conventional LDR image is then used to blindly extract the watermark bits without the original HDR image. The watermark is decrypted by the secrete private key.

Fig. 2. The flowchart of the HDR image watermark retrieving procedure.

#### **3. Experimental results**

Several HDR images are used in the experiments to verify the proposed HDR image watermarking algorithm. The watermark used in this chapter is the 80 60 % logo watermark as shown in Fig. 3. We applied several common signal processing attacks to the watermarked HDR images to evaluate the proposed watermarking scheme.

Fig. 3. The logo watermark used in the experiments.

To quantify the robustness of the proposed algorithm, we computed the value of the normalized correlation (NC) coefficient between the original watermark and the extracted one to measure the quality of the retrieved watermark bits, the formula is shown below:

$$\rho(\mathcal{W}, \mathcal{W}') = \frac{\sum\_{m=1}^{N\_w} w\_m w\_m'}{\sqrt{\sum\_{m=1}^{N\_w} w\_m^2 \sum\_m w\_m'^2}},\tag{1}$$

if *X Y*\$ *X Y diff* if *X Y*\$ *X X diff* if *X Y X X diff* if *X Y X Y diff*

Figure 2 shows the flowchart for extracting the watermark. We first use the tone mapping operator to convert the corrupted watermarked HDR image to the watermarked LDR image. The watermark detection method for conventional LDR image is then used to blindly extract the watermark bits without the original HDR image. The watermark is decrypted by

> 24bits True Color

Several HDR images are used in the experiments to verify the proposed HDR image watermarking algorithm. The watermark used in this chapter is the 80 60 % logo watermark as shown in Fig. 3. We applied several common signal processing attacks to the

To quantify the robustness of the proposed algorithm, we computed the value of the normalized correlation (NC) coefficient between the original watermark and the extracted one to measure the quality of the retrieved watermark bits, the formula is shown below:

1

, ,

*w*

,

*N*

1

*w*

*m N*

2 2

*m m m m*

*w w*

*m m*

*w w*

Image Watermark

Detection Watermark

, (1)

Table 1. DCT-based watermark embedding algorithm.

Tone Mapping Operator

Fig. 3. The logo watermark used in the experiments.

Fig. 2. The flowchart of the HDR image watermark retrieving procedure.

watermarked HDR images to evaluate the proposed watermarking scheme.

(, )

*W W*

**2.2 Watermark extraction** 

the secrete private key.

**3. Experimental results** 

Attacked HDR Images

W = 0 W = 1

where *W* and *W* denote the embedded watermark and the extracted watermark respectively and *NW* is the length of the watermark. The coefficient is bounded by 1 ( , )1 *W W* . Since the watermark is a binary sequence of #1 , we have

$$\sum\_{m=1}^{N\_w} w\_m^2 = \sum\_{m=1}^{N\_w} \left(w\_m'\right)^2 = N\_w \tag{2}$$

The normalized correlation coefficient can also be written as

$$\rho \rho (\mathcal{W}, \mathcal{W}') = \frac{\sum w\_m w\_m'}{N\_w} \tag{3}$$

A larger indicates a better retrieval performance.

Figure 4 shows a high dynamic range image –hot spring, which is recovered from six LDR photographs with different exposure time, the image size is 640 480 % . It is used for experiments to prove the robustness of the proposed method against common signal processing attacks on the watermarked HDR image. We first converted the original HDR image to its corresponding LDR image using a tone mapping operation and then computed the ratio image. The dynamic range compression algorithm based on fast bilateral filtering (Durand & Dorsey, 2002) is used as the tone mapping operator in the watermark embedding procedure. A simple DCT- based watermarking method is then used to embed the logo watermark shown in Fig. 3 into the tone mapped LDR image.

Figure 4 shows a high dynamic range image –hot spring, which is recovered from six LDR photographs with different exposure time, the image size is 640 480 % . It is used for experiments to prove the robustness of the proposed method against common signal processing attacks on the watermarked HDR image. We first converted the original HDR image to its corresponding LDR image using a tone mapping operation and then computed the ratio image. The dynamic range compression algorithm based on fast bilateral filtering [10] is used as the tone mapping operator in the watermark embedding procedure. A simple DCT- based watermarking method is then used to embed the logo watermark shown in Fig. 3 into the tone mapped LDR image.

Figure 5(a) depicts the tone mapped LDR image, and the watermarked LDR image is shown in Fig. 5(b). This figure shows that the two images are visually indistinguishable and the peak signal-to-noise ratio (PSNR) value between them is 41.48 dB. Finally the watermarked HDR image is produced by multiplying the watermarked LDR image by the ratio image.

Table 2 shows the result of the watermarked high dynamic range image corrupted by cropping, blurring and noising attacks. The normalized correlation coefficient between the extracted and original watermark are all higher than 0.3, and the extracted watermarks are distinguishable. It shows the effectiveness of the proposed HDR watermarking algorithm.

Mean square error (MSE) is not a good performance index to measure the difference between the original high dynamic range image and the watermarked one, it is because that the intensity range of the high dynamic range radiance map recovered by different approaches or programs are quite varying. In order to provide a fair measurement to the

Robust Watermarking Framework for High

Cropping 1

Cropping 2

Gaussian blur (radius=0.8)

Addaptive Gaussian noise (variance=9)

Table 2. The extracted watermark and the correlation coefficient

processing attacks – cropping, blurring and noising.

Dynamic Range Images Against Tone-Mapping Attacks 235

Distortion attack Tone-mapped LDR image Extracted watermark

*-*= 0.78

*-*= 0.75

*-*= 0.55

*-*= 0.71

upon common signal

Fig. 4. A high dynamic range image, hot spring, reconstructed from six differently exposed photographs.

(a) (b)

Fig. 5. Tone mapped LDR image and its watermarked version. (a) The tone mapped LDR image using the bilateral-filter based algorithm. (b) The watermarked LDR image by the DCT-based watermarking method.

Fig. 4. A high dynamic range image, hot spring, reconstructed from six differently exposed

(a) (b)

Fig. 5. Tone mapped LDR image and its watermarked version. (a) The tone mapped LDR image using the bilateral-filter based algorithm. (b) The watermarked LDR image by the

photographs.

DCT-based watermarking method.


Table 2. The extracted watermark and the correlation coefficient upon common signal processing attacks – cropping, blurring and noising.

Robust Watermarking Framework for High

proposed method.

**4. Conclusion** 

into account.

Dynamic Range Images Against Tone-Mapping Attacks 237

The algorithms that preparing HDR images for display on LDR devices are called tone mapping operators or tone reproduction. Three famous and popular tone mapping algorithms include the fast bilateral filter approach proposed by Durand and Dorsey [10], photographic method by Reihard. (Reihard. *et al*., 2002), and the gradient domain compression (GDC) algorithm by Fattal. (Fattal *et al*., 2002). Just as described in previous section, tone mapping is the most often used attack for high dynamic range images. To evaluate of the robustness of the proposed method against the tone mapping attacks, all three approaches mentioned above are used to test the proposed watermarking algorithm. In this experiment, a high dynamic range image - Belgium house as shown in Fig. 6 is used , which is obtained from the work of Fattal (Fattal *et al*., 2002), and its size is 1024 by 760.

The PSNR of the watermarked HDR image using the bilateral filtering method [10], photographic method (Reihard. *et al*., 2002), and the gradient domain compression (GDC) algorithm in the watermark embedding procedure are 77.58, 77.36 and 73.00 respectively. We observed that the watermarked high dynamic range image by using the GDC as the tone mapping operator in the watermark embedding step produced higher distortion compared two other tome mapping approaches. However, the visual quality is still satisfied and the PSNR is much higher than 55dB. Table 3 shows the performance comparison of the different tone mapping operators are used in the watermark embedding and retrieving procedures. It is worthy to notice that the watermarked HDR image using GDC in the watermark

Table 4 shows the robustness comparison of the watermarked HDR image when different tone mapping methods are used in the watermark embedding procedure. Two common signal processing attacks- blurring and cropping are corrupted on the watermarked HDR image. It shows the watermarking method used GDC method achieves the highest normalized correlation coefficient. In the following experiments, we adopt GDC as the tone

Finally, two another high dynamic range images obtained from Debevec's work (Debevec & Malik, 1997) are used to verify the proposed watermarking method, as shown in Fig. 7. Table 5 and 6 show the results, they demosntrate the efffectivenss and robust ness of the

Researching the watermarking scheme for high dynamic range images is an important task in image processing and computational photography fields. This chapter presents a new blind watermarking algorithm for HDR images. It achieves the robustness by embedding the watermark bits into a tone mapped version of the original HDR image. Experimental results show that the proposed algorithm is robust against various tone mapping operations, which is an inherent problem in watermarking HDR images. A simple DCT-based watermarking method for the derived tone-mapped LDR image is used in the watermark embedding procedure, it protects the watermarked HDR image from several common signal processing attacks, including noising, blurring and cropping. In the future work, the geometric attack invariant features will be put into analysis to enhance the robustness. Meanwhile, the capacity and fidelity are also taken

embedding procedure perform best against the various tone mapping attacks.

mapping operator in the proposed HDR watermark embedding procedure.

quality of the watermarked high dynamic range image, we propose to normalize the original high dynamic range image to the range 0, 255 in the experiments, the peak signalto-noise ratio (PSNR) is then calculated to measure the distortion between the original HDR image and the watermarked one, the formula is shown in Eqs. (4) and (5).

$$PSNR = 10 \cdot \log\_{10} \left(\frac{255^2}{MSE}\right) \text{ dB} \tag{4}$$

and

$$MSE = \frac{1}{3\text{V}\text{NH}} \sum\_{x=1}^{W} \sum\_{y=1}^{H} \sum\_{c \in \{R, G, B\}} \left( O\_c(\mathbf{x}, \mathbf{y}) - I\_c(\mathbf{x}, \mathbf{y}) \right)^2 \tag{5}$$

where *W* and *H* are the total number of pixels in the horizontal and the vertical dimensions of the images; *O x <sup>c</sup>* - ,*y* and *I x <sup>c</sup>* - ,*y* denote the pixels of the original and watermarked image respectively. According to our experience, the distortion for the high dynamic range images is invisible if the PSNR of the normalized HDR image is higher than 55 dB.

Fig. 6. A high dynamic range image- Belgium house (Fattal, *et al*., 2002), the image size is 1024 760 % .

quality of the watermarked high dynamic range image, we propose to normalize the original high dynamic range image to the range 0, 255 in the experiments, the peak signalto-noise ratio (PSNR) is then calculated to measure the distortion between the original HDR

> 10 <sup>255</sup> *PSNR* 10 log *MSE* - 8 + . 9

> > -

<sup>1</sup> (,) (,) <sup>3</sup>

1 1 { , ,}

where *W* and *H* are the total number of pixels in the horizontal and the vertical

watermarked image respectively. According to our experience, the distortion for the high dynamic range images is invisible if the PSNR of the normalized HDR image is higher than

Fig. 6. A high dynamic range image- Belgium house (Fattal, *et al*., 2002), the image size is

*x y c RGB MSE O x <sup>y</sup> I x <sup>y</sup> WH* \*

*W H*

,*y* and *I x <sup>c</sup>* -

2

*c c*

,, , (5)

dB (4)

2

,*y* denote the pixels of the original and

/ :

image and the watermarked one, the formula is shown in Eqs. (4) and (5).

and

55 dB.

1024 760 % .

dimensions of the images; *O x <sup>c</sup>* -

The algorithms that preparing HDR images for display on LDR devices are called tone mapping operators or tone reproduction. Three famous and popular tone mapping algorithms include the fast bilateral filter approach proposed by Durand and Dorsey [10], photographic method by Reihard. (Reihard. *et al*., 2002), and the gradient domain compression (GDC) algorithm by Fattal. (Fattal *et al*., 2002). Just as described in previous section, tone mapping is the most often used attack for high dynamic range images. To evaluate of the robustness of the proposed method against the tone mapping attacks, all three approaches mentioned above are used to test the proposed watermarking algorithm. In this experiment, a high dynamic range image - Belgium house as shown in Fig. 6 is used , which is obtained from the work of Fattal (Fattal *et al*., 2002), and its size is 1024 by 760.

The PSNR of the watermarked HDR image using the bilateral filtering method [10], photographic method (Reihard. *et al*., 2002), and the gradient domain compression (GDC) algorithm in the watermark embedding procedure are 77.58, 77.36 and 73.00 respectively. We observed that the watermarked high dynamic range image by using the GDC as the tone mapping operator in the watermark embedding step produced higher distortion compared two other tome mapping approaches. However, the visual quality is still satisfied and the PSNR is much higher than 55dB. Table 3 shows the performance comparison of the different tone mapping operators are used in the watermark embedding and retrieving procedures. It is worthy to notice that the watermarked HDR image using GDC in the watermark embedding procedure perform best against the various tone mapping attacks.

Table 4 shows the robustness comparison of the watermarked HDR image when different tone mapping methods are used in the watermark embedding procedure. Two common signal processing attacks- blurring and cropping are corrupted on the watermarked HDR image. It shows the watermarking method used GDC method achieves the highest normalized correlation coefficient. In the following experiments, we adopt GDC as the tone mapping operator in the proposed HDR watermark embedding procedure.

Finally, two another high dynamic range images obtained from Debevec's work (Debevec & Malik, 1997) are used to verify the proposed watermarking method, as shown in Fig. 7. Table 5 and 6 show the results, they demosntrate the efffectivenss and robust ness of the proposed method.

## **4. Conclusion**

Researching the watermarking scheme for high dynamic range images is an important task in image processing and computational photography fields. This chapter presents a new blind watermarking algorithm for HDR images. It achieves the robustness by embedding the watermark bits into a tone mapped version of the original HDR image. Experimental results show that the proposed algorithm is robust against various tone mapping operations, which is an inherent problem in watermarking HDR images. A simple DCT-based watermarking method for the derived tone-mapped LDR image is used in the watermark embedding procedure, it protects the watermarked HDR image from several common signal processing attacks, including noising, blurring and cropping. In the future work, the geometric attack invariant features will be put into analysis to enhance the robustness. Meanwhile, the capacity and fidelity are also taken into account.

Robust Watermarking Framework for High

(a)

(b)

Dynamic Range Images Against Tone-Mapping Attacks 239

Fig. 7. Two high dynamic range images (Debevec & Malik, 1997). (a) indoor; (b) church

window.


Table 3. The comparison of the different tone mapping operators are used in the HDR watermark embedding and retrieving procedures.

(a)

(b)

238 Watermarking – Volume 2

Retrieving Photographic Bilateral GDC

Table 3. The comparison of the different tone mapping operators are used in the HDR

watermark embedding and retrieving procedures.

0.87 0.82 0.90

0.83 0.83 0.87

0.88 0.87 0.92

Embedding

Photographic

Bilateral

GDC

Fig. 7. Two high dynamic range images (Debevec & Malik, 1997). (a) indoor; (b) church window.

Robust Watermarking Framework for High

ISBN 978-0-12-585263-0.

Vol. 11, No. 2, pp. 77-88.

ACM Press, pp.83-90.

1647.

265.

Tone mapping by

Gaussian blurring

**5. References** 

Distortion PSNR of the corrupted

Dynamic Range Images Against Tone-Mapping Attacks 241

Debevec, P. & Malik, J. (1997). Recovering High Dynamic Range Radiance Maps From

Reinhard, E.; Ward, G. & Pattanaik, S. (2005) High Dynamic Range Imaging: Acquisition,

Reinhard,E.; Kunkel, T.; Marion, Y.; Brouillat, J.; Cozot, R. & Bouatouch, K. (2007) Image

Tsang, K. F. & Au, O. C. (2001) A review on attacks, problems and weakness of digital

Piva,A.; Barni, M.; F. Bartolini & Cappellini, V. (1997) DCT-Based Watermark Recovering

Barni, M.; Bartolini, F.; Cappellini, V.; Lippi, A. & Piva, A. (1999) DWT-Based Technique for

Wang, Y.; Doherty, J. F. & Van Dyck, R. E. (2002) A Wavelet-Based Watermarking Algorithm

Suhail, M. A. & Obaidat, M. S. (2003) Digital Watermarking-Based DCT and JPEG Model,

Ward, G. & Simmons, M. (2004) Subband Encoding of High Dynamic Range Imagery,

Durand, F. & Dorsey, J. (2002) Fast Bilateral Filtering for the Display of High

Reihard,E.; Stark, P.; Shirley, M. & Ferwerda, J. (2002) Photographic Tone Reproduction for Digital Images, *ACM Transactions on Graphics*, Vol. 21, No.3, pp.267-276.

Display and Image-Based Lighting with CDROM, Morgan Kaufmann Publishers

Display algorithms for High and Low Dynamic Range Display Devices, *Journal of* 

watermarking and the pixel reallocation attack, *Proceedings of the SPIE, Security and* 

without Resorting to The Uncorrupted Original Image, *Proceedings of International* 

Spatio-Frequency Masking of Digital Signatures, *Proceedings of SPIE, Security and* 

for Ownership Verification of Digital Images, *IEEE Transactions on Image Processing*,

*IEEE Transactions on Instrumentation and Measurement*, Vol. 52, No. 5, pp. 1640-

*Proceeding Fist Symposium Applied Perception in Graphics and Visualization (APGV)*,

Dynamic Range Image, *ACM Transactions on Graphics*, Vol. 21, No.3, pp. 257-

Correlation coefficient

extracted watermark

of the

HDR image (dB)

photographic method 43.34 0.94

(radius=0.8) 28.31 0.74 Cropping 31.33 0.74 Blurring + cropping 26.61 0.55

Table 6. The watermarking result using the HDR image-church window.

Photographs, *ACM Transactions on Graphics*, pp. 369-378.

*the Society for Information Display*, Vol. 15, No. 2, pp. 997-1014..

*Watermarking of Multimedia Contents III*, Vol. 4314, pp. 385-393.

*Conference on Image Processing*, Vol. 1, pp. 520-523.

*Watermarking of Multimedia Contents*, Vol. 3657, pp. 31-39.


Table 4. The comparison of the robustness of the watermakre HDR image when different tone mapping methods are used in the watermark embedding procedure under bluring and cropping attacks.


Table 5. The watermarking result using the HDR image-indoor.


Table 6. The watermarking result using the HDR image-church window.

#### **5. References**

240 Watermarking – Volume 2

PhotoGraphic Bilateral GDC

40.18 40.18 40.18

=0.72 =0.68 =0.78

35.46 35.46 35.46

=0.72 =0.69 =0.75

Correlation coefficient

extracted watermark

of the

Table 4. The comparison of the robustness of the watermakre HDR image when different tone mapping methods are used in the watermark embedding procedure under bluring and

HDR image (dB)

Bilateral approach 61.51 0.86

photographic method 59.78 0.83

(radius=0.8) 36.19 0.71 Blurring + cropping 54.40 0.80

Distortion PSNR of the corrupted

Table 5. The watermarking result using the HDR image-indoor.

Embedding

Tone mapped corrupted image

PSNR of distorted HDR image

NC of the retrieved watermark

Tone mapped corrupted image

PSNR of distorted HDR image

NC of the retrieved watermark

cropping attacks.

Tone mapping by

Tone mapping by

Gaussian blurring

Distortion

Bluring

Local cropping


**1. Introduction** 

Aauml, 2003).

long as the embedding key is kept unknown.

**11** 

*China* 

**Improve Steganalysis by** 

**MWM Feature Selection** 

B. B. Xia, X. F. Zhao and D. G. Feng

*Institute of Software Chinese Academy of Sciences* 

Steganography is the art of invisible communication. It is derived from the ancient Greece thousands of years ago (Johnson & Jajodia, 1998; Kahn, 1996), and grow rapidly in the past few years along with the development of digital technology and the internet. Modern steganography has been widely used in various scenarios such as secret communication, digital rights management, data temper detection and recovery, etc (Provos & Honeyman, 2003). The main goal of modern steganography is to hide some secret messages into the socalled cover data, e.g. images, videos, audios, documents…, which produced the so-called stego data, and make the existence of the hidden messages unnoticeable to everyone expect the prospective receiver (Provos, 2001; Fridrich & Goljan, 2002; Fridrich, 2005). Usually an embedding key is also involved in the steganography scheme to provide security. The malicious can never tamper, remove, nor obtain the secret messages in the stego data, as

In contrast to steganography, steganalysis is developed to detect the presence of the secret messages, and furthermore estimate the length or even extract the content of the embedded messages. Although the secret message in stego data is always imperceptible to human's visual, the embedding process changes some statistics of the cover medium nevertheless. This can be utilized by steganalysis methods to distinguish stego mediums containing secret messages from the clean cover mediums (Lyu & Farid, 2002; Fridrich et al., 2002; Fridrich, 2005; Harmsen & Pearlman, 2003; Ker, 2005; Tzschoppe & Aauml, 2003; Xuan et al., 2005). Steganalysis methods can be roughly divided into two categories: the targeted (or specific) steganalysis which detects a particular known steganography method, and the blind (or universal) steganalysis which can deal with a wide variety of steganography methods. Though the targeted methods often have slightly better accuracy and efficiency than the blind ones, it is quit reasonable to assume that the nature of the cover data and the embedding method used for steganography is unknown to the analysers beforehand. Therefore, blind steganalysis based on learning and classifying are more valuable from a practical point of view (Fridrich & Goljan, 2002; Provos & Honeyman, 2003; Tzschoppe &

The typical framework of blind steganalysis is a procedure of two-class classification, which consists of training and classifying. First, a set of statistics called steganalysis features is

Fattal, R.; Lischinski, D. & Werman, M. (2002) Gradient Domain High Dynamic Range Compression, *ACM Transactions on Graphics*, Vol. 21, No.3*,* pp.249- 256.

## **Improve Steganalysis by MWM Feature Selection**

B. B. Xia, X. F. Zhao and D. G. Feng *Institute of Software Chinese Academy of Sciences China* 

#### **1. Introduction**

242 Watermarking – Volume 2

Fattal, R.; Lischinski, D. & Werman, M. (2002) Gradient Domain High Dynamic

256.

Range Compression, *ACM Transactions on Graphics*, Vol. 21, No.3*,* pp.249-

Steganography is the art of invisible communication. It is derived from the ancient Greece thousands of years ago (Johnson & Jajodia, 1998; Kahn, 1996), and grow rapidly in the past few years along with the development of digital technology and the internet. Modern steganography has been widely used in various scenarios such as secret communication, digital rights management, data temper detection and recovery, etc (Provos & Honeyman, 2003). The main goal of modern steganography is to hide some secret messages into the socalled cover data, e.g. images, videos, audios, documents…, which produced the so-called stego data, and make the existence of the hidden messages unnoticeable to everyone expect the prospective receiver (Provos, 2001; Fridrich & Goljan, 2002; Fridrich, 2005). Usually an embedding key is also involved in the steganography scheme to provide security. The malicious can never tamper, remove, nor obtain the secret messages in the stego data, as long as the embedding key is kept unknown.

In contrast to steganography, steganalysis is developed to detect the presence of the secret messages, and furthermore estimate the length or even extract the content of the embedded messages. Although the secret message in stego data is always imperceptible to human's visual, the embedding process changes some statistics of the cover medium nevertheless. This can be utilized by steganalysis methods to distinguish stego mediums containing secret messages from the clean cover mediums (Lyu & Farid, 2002; Fridrich et al., 2002; Fridrich, 2005; Harmsen & Pearlman, 2003; Ker, 2005; Tzschoppe & Aauml, 2003; Xuan et al., 2005).

Steganalysis methods can be roughly divided into two categories: the targeted (or specific) steganalysis which detects a particular known steganography method, and the blind (or universal) steganalysis which can deal with a wide variety of steganography methods. Though the targeted methods often have slightly better accuracy and efficiency than the blind ones, it is quit reasonable to assume that the nature of the cover data and the embedding method used for steganography is unknown to the analysers beforehand. Therefore, blind steganalysis based on learning and classifying are more valuable from a practical point of view (Fridrich & Goljan, 2002; Provos & Honeyman, 2003; Tzschoppe & Aauml, 2003).

The typical framework of blind steganalysis is a procedure of two-class classification, which consists of training and classifying. First, a set of statistics called steganalysis features is

Improve Steganalysis by MWM Feature Selection 245

Miche et al. (Miche et al., 2006) uses a fast classifier called K-Nearest-Neighbors combined with a forward selection method to achieve feature selection, which is still limited to deal with the relative low dimension feature sets. Dong et al. (Dong et al., 2008) make use of the Boosting Feature Selection (BFS) algorithm as the fusion tool to select a subset of original statistic features. Each dimension of the original features is treated as a weak classifier, and the output of BFS classification is calculated for each of them as an evaluation indicator. The final classifier is then constructed by every weak classifier with the evaluation indicator as their weight. Gul & Kurugollu (Gul & Kurugollu, 2011) also establish an evaluation indicator for each single dimension of the original features by means of calculating the covariance between features and the embedding rates. After that, the original features are sorted corresponding to their co-variance in the decreasing order. The best K features are then determined to form the final classifier, by adding all the features one by one to the classifier and test the performance. Fridrich et al. (Fridrich et al., 2011) propose another method of ensemble classifier to reduce the dimension of steganalysis classifier. The ensemble classifier consist of weak base learners which constructed by randomly choose subsets of the original features. The dimensionality of each base learner is significantly smaller than the full dimensionality of the original feature set. The final decision is obtained

In this chapter, we present a novel methodology called MMD-weighted-MI (MMD, Maximum-Mean-Discrepancy; MI, Mutual-Information) feature selection to deal with highdimensional steganalysis features. Before training the classifier, a MMD-weighted-MI (MWM) value is calculated and assigned to each dimension of the original features by evaluating the distribution of the extracted features using the MI and MMD indicators. The MI and MMD are both efficient measurements used exclusively in steganography benchmarking, but focus on different aspects of the feature distribution (Pevný & Fridrich, 2008). The MMD gives an overall view of a subset of the features, evaluates the difference of feature distribution between cover and stego training sets by means of generate a set of functions from the kernel ones and then calculate the maximum mean discrepancy. On the other hand, the MI, which is calculated only for single feature dimension due to its unacceptable complexity introduced by estimation of high-dimensional distribution, gives more details about how each dimension contributes to the classifier in steganalysis. When combined together as MWM values, these two indicators can give us a more comprehensive impression about difference between features extracted from the cover and stego training sets. After the MWM values are assigned, feature selection is simply implemented by

The organization of this chapter is as follows: Section 2 introduces some basic concepts of MI and MMD, as well as elaborates their different effect in feature selection briefly. Section 3 describes the proposed approach of MWM feature selection. Experimental results are

In this section, we explain the basic concepts of MI and MMD. These two measurements have been used in steganography benchmarking due to their characteristic of evaluating the difference between two distributions, which makes them natural candidates for steganalysis

by fuse the result of all base learners together under certain voting rule.

choosing feature dimensions with high value.

**2. Basic concepts of MI and MMD** 

feature selection.

presented in Section 4. The chapter is finally concluded in Section 5.

extracted from a pair of training set which contains cover and stego mediums respectively. A classifier is then trained by these extracted features. Then, given the medium under test, the steganalysis features is extracted similarly and input to the classifier to decide whether it contains hidden messages.

The choice of steganalysis features is crucial to the classification accuracy. As mentioned earlier, embedding messages in cover medium will change some of the statistics. It is obvious that choosing statistics which are sensitive to the steganography embedding process will provide better steganalysis accuracy. The statistic moments and transition probabilities have been proved to be more efficient than other choice for a wide variety of steganography methods, and so are used frequently in modern steganalysis (Davidson & Jalan, 2010; Fridrich et al., 2002; Lyu & Farid, 2002; Pevný et al., 2010a; Pevný & Fridrich, 2007).

To improve the steganography security, some embedding methods attempt to maintain the statistics of cover medium by means of minimizing the embedding distortion. Encoding methods such as matrix encoding or wet paper encoding are implemented to steganography process to reduce the embedding distortion (Fridrich et al., 2004; Westfeld, 2001). LSB(Least Significant Bit) matching method solves the imbalance problem introduced to the sample value histogram by the original LSB replacement method, and thus provide good security against steganalysis based on 1*st* order statistic features (Mielikainen, 2006). An adaptive steganography called HUGO (Highly Undetectable Steganography) is proposed recently. Before embedding secret messages into a cover image, HUGO determines a distortion measure for each pixel by calculating a weighted sum of difference between the features derived from cover and stego images. 1*st* and 2*nd* order transition probabilities of SPAM steganalysis features are chose as the features (Pevný et al., 2010a), which makes HUGO undetectable using steganalysis methods based on 1*st* and 2*nd* order statistic features. The details of this algorithm can be found in (Pevný et al., 2010b).

As steganography methods try to reduce embedding distortion and preserve representation of covers approximately for low-order statistic features, it is natural that steganalysis takes one more step in the same direction. That is to say, higher-order statistic features should be used as steganalysis features for better classification accuracy. However, this leads to a catastrophic growth of the amount of feature dimensions. Recently, Gul & Kurugollu propose a 1237 dimension feature set constructed by k-variate probability distribution function (PDF) estimates (Gul & Kurugollu, 2011). Fridrich et al. suggest a final HOLMES feature set that consists of 33,963 features to obtain better accuracy against HUGO (Fridrich et al., 2011).

The increasing dimensions of features bring new challenges to steganalysis. Training classifiers in high dimensions requires relatively large number of samples. With the significant growth of the feature dimensions, it becomes harder or even impossible to obtain sufficient samples. Furthermore, the computational complexity of training the classifier on a large-scale training set in high-dimensional spaces also becomes prohibitive.

Feature selection is a typical method to deal with the excessive feature dimensions. Feature selection not only reduces the number of dimensions, but also removes the inefficient or redundant features, leaves the efficient ones to the classifier for better training and classification. The theoretical ideal way of feature selection is exhaustive searching all the possible combination of feature dimensions, which can not be achieved in practical scenario.

extracted from a pair of training set which contains cover and stego mediums respectively. A classifier is then trained by these extracted features. Then, given the medium under test, the steganalysis features is extracted similarly and input to the classifier to decide whether it

The choice of steganalysis features is crucial to the classification accuracy. As mentioned earlier, embedding messages in cover medium will change some of the statistics. It is obvious that choosing statistics which are sensitive to the steganography embedding process will provide better steganalysis accuracy. The statistic moments and transition probabilities have been proved to be more efficient than other choice for a wide variety of steganography methods, and so are used frequently in modern steganalysis (Davidson & Jalan, 2010;

To improve the steganography security, some embedding methods attempt to maintain the statistics of cover medium by means of minimizing the embedding distortion. Encoding methods such as matrix encoding or wet paper encoding are implemented to steganography process to reduce the embedding distortion (Fridrich et al., 2004; Westfeld, 2001). LSB(Least Significant Bit) matching method solves the imbalance problem introduced to the sample value histogram by the original LSB replacement method, and thus provide good security against steganalysis based on 1*st* order statistic features (Mielikainen, 2006). An adaptive steganography called HUGO (Highly Undetectable Steganography) is proposed recently. Before embedding secret messages into a cover image, HUGO determines a distortion measure for each pixel by calculating a weighted sum of difference between the features derived from cover and stego images. 1*st* and 2*nd* order transition probabilities of SPAM steganalysis features are chose as the features (Pevný et al., 2010a), which makes HUGO undetectable using steganalysis methods based on 1*st* and 2*nd* order statistic features. The

As steganography methods try to reduce embedding distortion and preserve representation of covers approximately for low-order statistic features, it is natural that steganalysis takes one more step in the same direction. That is to say, higher-order statistic features should be used as steganalysis features for better classification accuracy. However, this leads to a catastrophic growth of the amount of feature dimensions. Recently, Gul & Kurugollu propose a 1237 dimension feature set constructed by k-variate probability distribution function (PDF) estimates (Gul & Kurugollu, 2011). Fridrich et al. suggest a final HOLMES feature set that consists of 33,963 features to obtain better accuracy against HUGO (Fridrich

The increasing dimensions of features bring new challenges to steganalysis. Training classifiers in high dimensions requires relatively large number of samples. With the significant growth of the feature dimensions, it becomes harder or even impossible to obtain sufficient samples. Furthermore, the computational complexity of training the classifier on a

Feature selection is a typical method to deal with the excessive feature dimensions. Feature selection not only reduces the number of dimensions, but also removes the inefficient or redundant features, leaves the efficient ones to the classifier for better training and classification. The theoretical ideal way of feature selection is exhaustive searching all the possible combination of feature dimensions, which can not be achieved in practical scenario.

large-scale training set in high-dimensional spaces also becomes prohibitive.

Fridrich et al., 2002; Lyu & Farid, 2002; Pevný et al., 2010a; Pevný & Fridrich, 2007).

details of this algorithm can be found in (Pevný et al., 2010b).

contains hidden messages.

et al., 2011).

Miche et al. (Miche et al., 2006) uses a fast classifier called K-Nearest-Neighbors combined with a forward selection method to achieve feature selection, which is still limited to deal with the relative low dimension feature sets. Dong et al. (Dong et al., 2008) make use of the Boosting Feature Selection (BFS) algorithm as the fusion tool to select a subset of original statistic features. Each dimension of the original features is treated as a weak classifier, and the output of BFS classification is calculated for each of them as an evaluation indicator. The final classifier is then constructed by every weak classifier with the evaluation indicator as their weight. Gul & Kurugollu (Gul & Kurugollu, 2011) also establish an evaluation indicator for each single dimension of the original features by means of calculating the covariance between features and the embedding rates. After that, the original features are sorted corresponding to their co-variance in the decreasing order. The best K features are then determined to form the final classifier, by adding all the features one by one to the classifier and test the performance. Fridrich et al. (Fridrich et al., 2011) propose another method of ensemble classifier to reduce the dimension of steganalysis classifier. The ensemble classifier consist of weak base learners which constructed by randomly choose subsets of the original features. The dimensionality of each base learner is significantly smaller than the full dimensionality of the original feature set. The final decision is obtained by fuse the result of all base learners together under certain voting rule.

In this chapter, we present a novel methodology called MMD-weighted-MI (MMD, Maximum-Mean-Discrepancy; MI, Mutual-Information) feature selection to deal with highdimensional steganalysis features. Before training the classifier, a MMD-weighted-MI (MWM) value is calculated and assigned to each dimension of the original features by evaluating the distribution of the extracted features using the MI and MMD indicators. The MI and MMD are both efficient measurements used exclusively in steganography benchmarking, but focus on different aspects of the feature distribution (Pevný & Fridrich, 2008). The MMD gives an overall view of a subset of the features, evaluates the difference of feature distribution between cover and stego training sets by means of generate a set of functions from the kernel ones and then calculate the maximum mean discrepancy. On the other hand, the MI, which is calculated only for single feature dimension due to its unacceptable complexity introduced by estimation of high-dimensional distribution, gives more details about how each dimension contributes to the classifier in steganalysis. When combined together as MWM values, these two indicators can give us a more comprehensive impression about difference between features extracted from the cover and stego training sets. After the MWM values are assigned, feature selection is simply implemented by choosing feature dimensions with high value.

The organization of this chapter is as follows: Section 2 introduces some basic concepts of MI and MMD, as well as elaborates their different effect in feature selection briefly. Section 3 describes the proposed approach of MWM feature selection. Experimental results are presented in Section 4. The chapter is finally concluded in Section 5.

#### **2. Basic concepts of MI and MMD**

In this section, we explain the basic concepts of MI and MMD. These two measurements have been used in steganography benchmarking due to their characteristic of evaluating the difference between two distributions, which makes them natural candidates for steganalysis feature selection.

Improve Steganalysis by MWM Feature Selection 247

fundamental quantity that estimates the difference between the distributions of the features obtained from the cover and stego set. In this way, the MI establishes a measurement of how much the features contribute to the final classifier. These properties and the computing symmetry shown in (4) make the MI an appropriate choice in evaluating the value of feature

The calculation of the MI relies on the estimation of the distributions of *P* and *Q* , which is quite difficult or even impossible to achieve for high dimensional features in a practical point of view. Thus, we treat each dimension of the original features as a single feature and calculate MI separately. Histogram estimates are applied to each single feature to provide

Given two distributions *P* and *Q* defined on the whole set of images *X* , the disparity of *P* and *Q* can be evaluated by a statistic called Maximum Mean Discrepancy (MMD) (Gretton et al., 2007). The main idea behind MMD is based on the statement that *P* and *Q* are the same distribution if and only if their probability distribution functions (pdf) *p* and


+ denotes the expectation for the random variable *x* with *p* and *q* as the pdf

The number of functions in *C X*( ) is infinite, but only part of the functions in *C X*( ) can be utilized because of the finite number of samples in the training sets in practical steganalysis scenarios. Denote a subset of *C X*( ) , then the difference between distributions *P* and *Q*

, () *Ex p x q f x E* \* *f x f C X* (6)



1 1

*D D*

1 1

\* *D D*

*D D i i f i i*

, 0 . (8)

*Ex p* + and

estimation of their distributions. The details can be found in Section 3.

~ ~ --



*MMD X Y fx fy*

where *Xxx D D* <sup>1</sup> , , and *YD D y y* <sup>1</sup> , , are observations of the cover and stego

The choice of affects the performance of MMD significantly. It has to be rich enough to make *p* and *q* distinguishable, while still under the restriction of the finite number of images in cover and stego training sets. A typical construction of is the Reproducing Kernel Hilbert Spaces (RKHS) built from the so-called *kernel* function. The kernel function is a symmetric, positive definite function used to generate the RKHS. Gaussian kernel has been

> <sup>2</sup> <sup>2</sup> *k xy x y* , exp

In this case, the MMD values corresponding to are obtained by an unbiased estimate

 

, , sup

where *C X*( ) denotes the set of all continuous bounded functions on *X* , ~ -

dimensions in steganalysis feature selection.

**2.2 Maximum Mean Discrepancy (MMD)** 

is evaluated by MMD values corresponding to as

proved to be a valuable choice (Pevný & Fridrich, 2008) as



distributions *P* and *Q* respectively.

based on U-statistics as

*q* satisfy that

respectively.

*Ex q* <sup>~</sup> -

Without loss of generality, images are chose as the cover medium in the following discussion. The same result holds for other form of mediums such as videos, audios and documents.

#### **2.1 Mutual Information (MI)**

Denote *X* the whole set of images corresponding to the steganalysis system. *X* can be divided into two non-overlap subsets, namely cover set *C* and stego set *S* respectively. Denote *P* and *Q* the distribution of the cover and stego set, with *p* and *q* as the probability distribution function (pdf) respectively. Then the difficulty of distinguishing stego images from cover ones can be measured using statistic called Kullback-Leibler divergence (Cachin, 1998)

$$KL(P \mid \mid Q) = \int\_{\mathbf{x}} p(\mathbf{x}) \log \frac{p(\mathbf{x})}{q(\mathbf{x})} d\mathbf{x} \tag{1}$$

where *x* denotes the sample medium drawn from the whole set of image *X* .

The KL divergence is a fundamental quantity for steganography benchmarking, which provides good estimate to the difference between cover and stego sets for certain features (Cover & Thomas, 2001). However, the asymmetry in calculating the KL divergence becomes a main drawback. From (1), it is obvious that

$$KL(P \mid \mid Q) \neq KL(Q \mid \mid P) \,. \tag{2}$$

This computing asymmetry, without carefully treatment, could cause inconsistent in the quantitative evaluation for feature dimensions, and thus leads to inconveniency and ambiguity in feature selection. To overcome this difficulty, we use Mutual Information (MI) to substitute KL divergence

$$I(P,Q) = \sum\_{i} \sum\_{j} \phi(\mathbf{x}\_i, \mathbf{y}\_j) \log \frac{\phi(\mathbf{x}\_i, \mathbf{y}\_j)}{p(\mathbf{x}\_i)q(\mathbf{y}\_j)} \tag{3}$$

where *<sup>i</sup> x* and *<sup>i</sup> y* denote the steganalysis features extracted from images in cover and stego set respectively, (,) *i i x y* denote the joint probability distribution function, ( )*<sup>i</sup> p x* and ( )*<sup>i</sup> q y* denote the marginal probability distribution functions respectively. It is obvious that the definition of MI is symmetric

$$I(P,Q) = I(Q,P) \,. \tag{4}$$

The MI can be represented as an expectation of KL divergence as below

$$I(X:Y) = E\_Y\left(KL\left(p(x\mid y)\mid p(x)\right)\right) \tag{5}$$

where *p*(|) *x y* denotes the conditional probability of image *x* drawn from the cover set, given the image *y* from the stego set, and -+ *EY* denotes the expectation for the random variable *y* . The relationship between the MI and the KL divergence in (5) suggests that MI maintains the characteristics of KL divergence in steganography benchmarking, provides a

Without loss of generality, images are chose as the cover medium in the following discussion. The same result holds for other form of mediums such as videos, audios and

Denote *X* the whole set of images corresponding to the steganalysis system. *X* can be divided into two non-overlap subsets, namely cover set *C* and stego set *S* respectively. Denote *P* and *Q* the distribution of the cover and stego set, with *p* and *q* as the probability distribution function (pdf) respectively. Then the difficulty of distinguishing stego images from cover ones can be measured using statistic called Kullback-Leibler

> ( ) ( || ) ( )log ( ) *<sup>x</sup> p x KL P Q <sup>p</sup> x dx*

The KL divergence is a fundamental quantity for steganography benchmarking, which provides good estimate to the difference between cover and stego sets for certain features (Cover & Thomas, 2001). However, the asymmetry in calculating the KL divergence

This computing asymmetry, without carefully treatment, could cause inconsistent in the quantitative evaluation for feature dimensions, and thus leads to inconveniency and ambiguity in feature selection. To overcome this difficulty, we use Mutual Information (MI)

(, ) ( , ) ( , )log ( )( )

where *<sup>i</sup> x* and *<sup>i</sup> y* denote the steganalysis features extracted from images in cover and stego

denote the marginal probability distribution functions respectively. It is obvious that the

 *p*-

where *p*(|) *x y* denotes the conditional probability of image *x* drawn from the cover set,

variable *y* . The relationship between the MI and the KL divergence in (5) suggests that MI maintains the characteristics of KL divergence in steganography benchmarking, provides a

*<sup>x</sup> <sup>y</sup> IPQ x y*

 ,,

The MI can be represented as an expectation of KL divergence as below

*I X Y E KL* -: *<sup>Y</sup>* -

*i j i j i j*

where *x* denotes the sample medium drawn from the whole set of image *X* .

becomes a main drawback. From (1), it is obvious that

*q x* 7 (1)

*KL P Q KL Q P* ( || ) ( || ) ' . (2)

*i j*

(3)

*IPQ IQP* (, ) (,) . (4)

+ *EY* denotes the expectation for the random

*x* (5)

*p x q y* 

*x y* denote the joint probability distribution function, ( )*<sup>i</sup> p x* and ( )*<sup>i</sup> q y*

*x*| || *y p*-

documents.

**2.1 Mutual Information (MI)** 

divergence (Cachin, 1998)

to substitute KL divergence

set respectively, (,) *i i*

definition of MI is symmetric

given the image *y* from the stego set, and -

fundamental quantity that estimates the difference between the distributions of the features obtained from the cover and stego set. In this way, the MI establishes a measurement of how much the features contribute to the final classifier. These properties and the computing symmetry shown in (4) make the MI an appropriate choice in evaluating the value of feature dimensions in steganalysis feature selection.

The calculation of the MI relies on the estimation of the distributions of *P* and *Q* , which is quite difficult or even impossible to achieve for high dimensional features in a practical point of view. Thus, we treat each dimension of the original features as a single feature and calculate MI separately. Histogram estimates are applied to each single feature to provide estimation of their distributions. The details can be found in Section 3.

#### **2.2 Maximum Mean Discrepancy (MMD)**

Given two distributions *P* and *Q* defined on the whole set of images *X* , the disparity of *P* and *Q* can be evaluated by a statistic called Maximum Mean Discrepancy (MMD) (Gretton et al., 2007). The main idea behind MMD is based on the statement that *P* and *Q* are the same distribution if and only if their probability distribution functions (pdf) *p* and *q* satisfy that

$$E\_{\mathbf{x}\sim p}\left(f\left(\mathbf{x}\right)\right) = E\_{\mathbf{x}\sim q}\left(f\left(\mathbf{x}\right)\right), \forall f \in \mathcal{C}(X) \tag{6}$$

where *C X*( ) denotes the set of all continuous bounded functions on *X* , ~ - *Ex p* + and *Ex q* <sup>~</sup> -+ denotes the expectation for the random variable *x* with *p* and *q* as the pdf respectively.

The number of functions in *C X*( ) is infinite, but only part of the functions in *C X*( ) can be utilized because of the finite number of samples in the training sets in practical steganalysis scenarios. Denote a subset of *C X*( ) , then the difference between distributions *P* and *Q* is evaluated by MMD values corresponding to as

$$\text{MMD}\left[\Gamma, X\_{D}, Y\_{D}\right] = \sup\_{f \in \Gamma} \left(\frac{1}{D} \sum\_{i=1}^{D} f(\mathbf{x}\_{i}) - \frac{1}{D} \sum\_{i=1}^{D} f(y\_{i})\right) \tag{7}$$

where *Xxx D D* <sup>1</sup> , , and *YD D y y* <sup>1</sup> , , are observations of the cover and stego distributions *P* and *Q* respectively.

The choice of affects the performance of MMD significantly. It has to be rich enough to make *p* and *q* distinguishable, while still under the restriction of the finite number of images in cover and stego training sets. A typical construction of is the Reproducing Kernel Hilbert Spaces (RKHS) built from the so-called *kernel* function. The kernel function is a symmetric, positive definite function used to generate the RKHS. Gaussian kernel has been proved to be a valuable choice (Pevný & Fridrich, 2008) as

$$k\left(\mathbf{x}, y\right) = \exp\left(-\boldsymbol{\gamma} \left\|\mathbf{x} - y\right\|\_2^2\right), \boldsymbol{\gamma} > 0 \; . \tag{8}$$

In this case, the MMD values corresponding to are obtained by an unbiased estimate based on U-statistics as

Improve Steganalysis by MWM Feature Selection 249

selection methods. As they focus on different aspects of the feature distributions, we combine these two indicators into MMD-Weighted-MI for more comprehensive feature selection.

As shown in Section 2.1, MI is a fundamental quantity for evaluating the value of feature dimensions in steganalysis feature selection. However, the calculation of MI relies on accurate estimation of high-dimensional distribution of the original features, which is difficult or even impossible to achieve from a practical point of view. To solve this problem, we calculate MI for each single dimension of the original features separately instead of

extracted from the images in *X* , *d* is the total number of dimensions. Denote *<sup>k</sup> P* and *<sup>k</sup> Q* the marginal distribution of the *k* -th dimension in original features extracted from the cover and stego set respectively, the MI value for the *k* -th dimension of the original features is

(,) ( , ) ( , )log , 1,2, ( )( )

*ki j k k i j ki k j x y MI I P Q x y k d*

*x y* denote the joint probability distribution function of the *k* -th dimension of

*n*

*x y* is estimated by the joint histogram similarly. The choice of *h* affects

*k kk k k ki j*


where *nj* is the frequency fell into the *j* -th category, and *h* denotes the interval of

the discrepancy and variance of the histogram estimation. Higher value of *h* leads to larger discrepancy and smaller variance, or vice versa. To achieve balance between discrepancy and variance, we set intervals dynamically corresponding to the dynamic range of each

Fig. 2 gives an example of MI calculated for each single dimension. The training set consists of cover images in jpg format and corresponding stego images generated by F5 steganography algorithm (Westfeld, 2001) with embedding rate at 0.05 bpac1. The Merging Markov and DCT features (Pevný & Fridrich, 2007) are chose as original steganalysis features. Fig. 2 shows that MI value for each single dimension varies significantly, and the feature dimensions with higher MI values contribute more than others in steganalysis

The calculation of MI values of single feature dimensions treats each dimension as an independent feature. However, the correlation of different dimensions also plays an important

*k*

*k k*

(10)

*<sup>k</sup> <sup>j</sup> q y* denote the marginal distributions respectively.

*p x nh* (11)

*pxqy*

*<sup>k</sup> <sup>j</sup> q y* are both distributions of single random variables, it is simple to

1 2 ,,, *<sup>d</sup> x xx x* the original feature

**3.1 MI values for single feature dimension** 

defined as

where ( , ) *k k k i j* 

categories. ( , ) *k k*

Since ( ) *<sup>k</sup>*

classifier.

the cover and stego set, ( ) *<sup>k</sup>*

*k i p x* and ( ) *<sup>k</sup>*

*k i j* 

**3.2 MMD-Weighted-MI (MWM)** 

1 bpac, bit per AC coefficients

estimate their pdf by histogram estimation as

treating them as high-dimensional features. Denote -

,,

feature dimension in the proposed algorithm in Section 3.3.

*k i p x* and ( ) *<sup>k</sup>*

$$\text{MMD}\_{u}\left[\Gamma, X\_{D'}Y\_{D}\right] = \left[\frac{1}{D\left(D-1\right)\_{i\neq j}}\sum\_{i\neq j}\left(k\left(\mathbf{x}\_{i},\mathbf{x}\_{j}\right) + k\left(y\_{i},y\_{j}\right) - k\left(\mathbf{x}\_{i},y\_{j}\right) - k\left(\mathbf{x}\_{j},y\_{i}\right)\right)\right]^{\frac{1}{2}}.\tag{9}$$

Fig. 1 shows the effectiveness of MMD values as steganography benchmarking. Training sets generated by various embedding methods (Fridrich et al., 2004; Westfeld, 2001; Mielikainen, 2006) and different embedding rates are applied to calculate MMD values. The false rates of the classifier corresponding to each training sets are also obtained, and normalized to be comparable to the MMD values. Note that the original MMD values are replaced by 10 log ( ) *MMD* for better visual. The result shows that MMD values are good estimations of the performance of classifiers in steganalysis.

Fig. 1. Comparison between the MMD values (square marked) and the false rate of the classifiers (triangle marked)

The computational complexity of MMD with Gaussian kernel is - <sup>2</sup> *O D* , where *D* is the number of sample images. It is far more efficient in comparison to Support Vector Machines (SVM), which is a commonly used classifier in modern steganalysis. Further more, the MMD converges with error 1 *D* , yet almost independently on feature dimensions, which means that a sample set with roughly <sup>3</sup> 10 images is sufficient for MMD to provide accurate estimations despite the feature dimensions. These advantages make MMD a natural choice to achieve feature selection for high-dimensional steganalysis features.

#### **3. MMD-weighted-MI feature selection**

The MI and MMD are both efficient measurements of evaluating the difficulty of distinguishing stego images from cover ones. Therefore, we apply them to steganalysis feature


0 ;

<sup>2</sup> <sup>1</sup> , , ,,,, <sup>1</sup> *u DD ij ij ij ji*

Fig. 1 shows the effectiveness of MMD values as steganography benchmarking. Training sets generated by various embedding methods (Fridrich et al., 2004; Westfeld, 2001; Mielikainen, 2006) and different embedding rates are applied to calculate MMD values. The false rates of the classifier corresponding to each training sets are also obtained, and normalized to be comparable to the MMD values. Note that the original MMD values are replaced by 10 log ( ) *MMD* for better visual. The result shows that MMD values are good

<sup>1</sup> <sup>&</sup>lt; <sup>1</sup> <sup>&</sup>lt; <sup>2</sup> <sup>=</sup>

0 2 4 6 8 10 12 14

Serial numbers

number of sample images. It is far more efficient in comparison to Support Vector Machines (SVM), which is a commonly used classifier in modern steganalysis. Further more, the MMD converges with error 1 *D* , yet almost independently on feature dimensions, which means that a sample set with roughly <sup>3</sup> 10 images is sufficient for MMD to provide accurate estimations despite the feature dimensions. These advantages make MMD a natural choice

The MI and MMD are both efficient measurements of evaluating the difficulty of distinguishing stego images from cover ones. Therefore, we apply them to steganalysis feature

Fig. 1. Comparison between the MMD values (square marked) and the false rate of the

The computational complexity of MMD with Gaussian kernel is -

to achieve feature selection for high-dimensional steganalysis features.

**3. MMD-weighted-MI feature selection** 


, . (9)


<sup>2</sup> *O D* , where *D* is the

1



*i j MMD X Y kx x ky y kx y kx y D D* '

> MMD values False rate


0

classifiers (triangle marked)

0.2

0.4

Normalized values

0.6

0.8

1

estimations of the performance of classifiers in steganalysis.

selection methods. As they focus on different aspects of the feature distributions, we combine these two indicators into MMD-Weighted-MI for more comprehensive feature selection.

#### **3.1 MI values for single feature dimension**

As shown in Section 2.1, MI is a fundamental quantity for evaluating the value of feature dimensions in steganalysis feature selection. However, the calculation of MI relies on accurate estimation of high-dimensional distribution of the original features, which is difficult or even impossible to achieve from a practical point of view. To solve this problem, we calculate MI for each single dimension of the original features separately instead of treating them as high-dimensional features. Denote - 1 2 ,,, *<sup>d</sup> x xx x* the original feature extracted from the images in *X* , *d* is the total number of dimensions. Denote *<sup>k</sup> P* and *<sup>k</sup> Q* the marginal distribution of the *k* -th dimension in original features extracted from the cover and stego set respectively, the MI value for the *k* -th dimension of the original features is defined as

$$\mathbf{M}^{k} = \mathbf{I}(\mathbf{P}^{k}, \mathbf{Q}^{k}) = \sum\_{i} \sum\_{j} \phi\_{k}(\mathbf{x}\_{i}^{k}, \mathbf{y}\_{j}^{k}) \log \frac{\phi\_{k}(\mathbf{x}\_{i}^{k}, \mathbf{y}\_{j}^{k})}{p\_{k}(\mathbf{x}\_{i}^{k}) q\_{k}(\mathbf{y}\_{j}^{k})}, k = 1, 2, \cdots, d \tag{10}$$

where ( , ) *k k k i j x y* denote the joint probability distribution function of the *k* -th dimension of the cover and stego set, ( ) *<sup>k</sup> k i p x* and ( ) *<sup>k</sup> <sup>k</sup> <sup>j</sup> q y* denote the marginal distributions respectively. Since ( ) *<sup>k</sup> k i p x* and ( ) *<sup>k</sup> <sup>k</sup> <sup>j</sup> q y* are both distributions of single random variables, it is simple to estimate their pdf by histogram estimation as

$$
\vec{p}\_k \left( \mathbf{x} \right) = \frac{n\_j}{n\hbar} \tag{11}
$$

where *nj* is the frequency fell into the *j* -th category, and *h* denotes the interval of categories. ( , ) *k k k i j x y* is estimated by the joint histogram similarly. The choice of *h* affects the discrepancy and variance of the histogram estimation. Higher value of *h* leads to larger discrepancy and smaller variance, or vice versa. To achieve balance between discrepancy and variance, we set intervals dynamically corresponding to the dynamic range of each feature dimension in the proposed algorithm in Section 3.3.

Fig. 2 gives an example of MI calculated for each single dimension. The training set consists of cover images in jpg format and corresponding stego images generated by F5 steganography algorithm (Westfeld, 2001) with embedding rate at 0.05 bpac1. The Merging Markov and DCT features (Pevný & Fridrich, 2007) are chose as original steganalysis features. Fig. 2 shows that MI value for each single dimension varies significantly, and the feature dimensions with higher MI values contribute more than others in steganalysis classifier.

#### **3.2 MMD-Weighted-MI (MWM)**

The calculation of MI values of single feature dimensions treats each dimension as an independent feature. However, the correlation of different dimensions also plays an important

<sup>1</sup> bpac, bit per AC coefficients

Improve Steganalysis by MWM Feature Selection 251

Merge POMM

Merge POMM

0 100 200 300 400 500

Feature dimensions

Fig. 3. Comparison of MI values between the Merging Markov and DCT features ('Merge', cross marked ones) and the Partially Ordered Markov Model features ('POMM', square

Fig. 4 shows the MWM values derived from the two feature sets, namely the Merging Markov and DCT features and the Partially Ordered Markov Model features which is the

0 100 200 300 400 500

Feature dimensions

Fig. 4. Comparison of MI values between the Merging Markov and DCT features ('Merge', cross marked ones) and the Partially Ordered Markov Model ('POMM', square marked ones)

0

0

2

4

MWM values

6

8

10

0.01

0.02

MI values

marked ones)

0.03

0.04

Fig. 2. MI values of Merging Markov and DCT features

role in training and classification. The absence of the information of feature correlation will bring troubles to the feature selection. Furthermore, the MI values of features drawn from different original feature sets have different dynamic range, which leads to an unfair comparison between feature sets if only MI values are used for feature selection. Fig. 3 shows the comparison of two feature sets as an example, where the MI values of most feature dimensions from the Partially Ordered Markov Model features sets (Davidson & Jalan, 2010) are relatively low than the Merging Markov and DCT features. The MI values of Y-Axis are limited to 0.04 for the purpose of clear observation though.

The raw MI values of single feature dimension are defective for feature selection due to the lack of correlation information. To overcome this difficulty, we introduce MMD as well for evaluating the feature dimensions. The MMD is numerically stable even in highdimensional spaces, which makes it an excellent choice for providing information about correlation between feature dimensions. The computational complexity is also relatively low so that calculating MMD values for high-dimensional feature sets is feasible.

The combination of MI and MMD values provides a more comprehensive impression about the difference between the distribution of features extracted from the cover and stego training sets, which results in a new indicator called MMD-Weighted-MI (WMW) for better feature selection. Denote *MIij* (, ) the MI value of the *j* -th dimension in the *i* -th feature set, and *MMD i*( ) the MMD value of the *i* -th feature set. Then a WMW value is assigned to each feature dimension as

$$\text{VMMV}(i, j) = \frac{MI(i, j)}{\text{MMD}(i)} \tag{12}$$

0 50 100 150 200 250 300

Feature dimensions

role in training and classification. The absence of the information of feature correlation will bring troubles to the feature selection. Furthermore, the MI values of features drawn from different original feature sets have different dynamic range, which leads to an unfair comparison between feature sets if only MI values are used for feature selection. Fig. 3 shows the comparison of two feature sets as an example, where the MI values of most feature dimensions from the Partially Ordered Markov Model features sets (Davidson & Jalan, 2010) are relatively low than the Merging Markov and DCT features. The MI values of

The raw MI values of single feature dimension are defective for feature selection due to the lack of correlation information. To overcome this difficulty, we introduce MMD as well for evaluating the feature dimensions. The MMD is numerically stable even in highdimensional spaces, which makes it an excellent choice for providing information about correlation between feature dimensions. The computational complexity is also relatively low

The combination of MI and MMD values provides a more comprehensive impression about the difference between the distribution of features extracted from the cover and stego training sets, which results in a new indicator called MMD-Weighted-MI (WMW) for better feature selection. Denote *MIij* (, ) the MI value of the *j* -th dimension in the *i* -th feature set, and *MMD i*( ) the MMD value of the *i* -th feature set. Then a WMW value is assigned to


(, ) , ( )

*MI i <sup>j</sup> WMW i j MMD i* (12)

0

Fig. 2. MI values of Merging Markov and DCT features

Y-Axis are limited to 0.04 for the purpose of clear observation though.

so that calculating MMD values for high-dimensional feature sets is feasible.

0.02

each feature dimension as

0.04

0.06

MI values

0.08

0.1

0.12

Fig. 3. Comparison of MI values between the Merging Markov and DCT features ('Merge', cross marked ones) and the Partially Ordered Markov Model features ('POMM', square marked ones)

Fig. 4 shows the MWM values derived from the two feature sets, namely the Merging Markov and DCT features and the Partially Ordered Markov Model features which is the

Fig. 4. Comparison of MI values between the Merging Markov and DCT features ('Merge', cross marked ones) and the Partially Ordered Markov Model ('POMM', square marked ones)

Improve Steganalysis by MWM Feature Selection 253

converse them into JPEG format with a 98 JPEG quality factor. The stego images are

The embedding rates vary from 0.05 bpac to 0.15 bpac. We randomly choose 1000 pair of

Three typical steganalysis feature sets are chose to provide original features for WMW

a. Markov Transition Probability features by Shi et al. (Shi et al., 2007), with 900

b. Merging Markov and DCT features by Pevný & Fridrich. (Pevný & Fridrich, 2007), with

c. Partially Ordered Markov Model features by Davidson & Jalan (Davidson & Jalan,

The total number of dimensions involved in our experiments is 900+274+448=1622. We gradually increase the number of the chosen dimensions by WMW feature selection and test the performance of the corresponding classifiers by TR 2. The intervals of the feature numbers are set differently because of the uneven density of the distribution of the MWM values. For the purpose of a clear view, we set interval to 5 dimensions within the first 150 features and 100 dimensions for the rest of them. The TR of the classifiers are tested and shown in Fig. 6, 7 and 8 for different embedding rate (0.05bpac, 0.1bpac and 0.15bpac)

0 10 20 30 40 50

F5, raw MI F5, MWM PQ, raw MI PQ, MWM

Serial numbers

Fig. 6. Comparison of the performance between the classifier using WMW values (solid lines) and the raw MI values (dotted lines) for feature selection, with embedding rate 0.05bpac.

2 TR, True Rate, the average of the True Positive rate (TP) and True Negative rate (TN)

b. Perturbed Quantization (PQ) steganography by Fridrich et al. (Fridrich et al., 2004)

cover/stego images as the training sets and 300 other pairs as the testing sets.

generated using the following two typical steganography algorithms:

a. F5 steganography by Westfeld (Westfeld, 2001)

feature selection:

respectively.

dimensions.

274 dimensions.

0.5

0.6

0.7

TR of the classifier

0.8

0.9

1

2010), with 448 dimensions.

same as used in Fig. 3. Compared to the result of the raw MI values, MWM values achieve a fair comparison between different feature sets; make the dynamic range of the two feature sets comparable. This leads to a better feature selection for steganalysis and thus better accuracy of the classifier, which is supported by experiment results in Section 4.

#### **3.3 Feature selection approach**

The feature selection is implemented based on the MWM values and new steganalysis classifiers are constructed by the selected features. Fig. 5 shows the overview of our approach, and the details are presented as follows.


Fig. 5. Overview of the MWM feature selection

#### **4. Experiment results**

In this section, we experimentally investigate the performance of our WMW feature selection. We choose images of JPEG format as the cover images without loss of generality. The extensively used BOSSbase image database (Bas et al., 2010) is used as the source of cover images. Because the original images from BOSSbase are in the RAW format, we

same as used in Fig. 3. Compared to the result of the raw MI values, MWM values achieve a fair comparison between different feature sets; make the dynamic range of the two feature sets comparable. This leads to a better feature selection for steganalysis and thus better

The feature selection is implemented based on the MWM values and new steganalysis classifiers are constructed by the selected features. Fig. 5 shows the overview of our

& Step 1. We choose several different steganalysis methods and extract the corresponding

& Step 2. MI values are calculated and assigned to each feature dimension, and MMD values are calculated for each feature set as high-dimensional features. The MWM

& Step 3. Feature dimensions with their MWM values larger than a given threshold are selected to assemble the new fused features. The serial numbers of the selected features are recorded as well. A classifier is trained with the fused features for steganalysis.

In this section, we experimentally investigate the performance of our WMW feature selection. We choose images of JPEG format as the cover images without loss of generality. The extensively used BOSSbase image database (Bas et al., 2010) is used as the source of cover images. Because the original images from BOSSbase are in the RAW format, we

accuracy of the classifier, which is supported by experiment results in Section 4.

**3.3 Feature selection approach** 

approach, and the details are presented as follows.

values are then generated based on MI and MMD values.

feature sets from the training set.

Fig. 5. Overview of the MWM feature selection

**4. Experiment results** 

converse them into JPEG format with a 98 JPEG quality factor. The stego images are generated using the following two typical steganography algorithms:


The embedding rates vary from 0.05 bpac to 0.15 bpac. We randomly choose 1000 pair of cover/stego images as the training sets and 300 other pairs as the testing sets.

Three typical steganalysis feature sets are chose to provide original features for WMW feature selection:


The total number of dimensions involved in our experiments is 900+274+448=1622. We gradually increase the number of the chosen dimensions by WMW feature selection and test the performance of the corresponding classifiers by TR 2. The intervals of the feature numbers are set differently because of the uneven density of the distribution of the MWM values. For the purpose of a clear view, we set interval to 5 dimensions within the first 150 features and 100 dimensions for the rest of them. The TR of the classifiers are tested and shown in Fig. 6, 7 and 8 for different embedding rate (0.05bpac, 0.1bpac and 0.15bpac) respectively.

Fig. 6. Comparison of the performance between the classifier using WMW values (solid lines) and the raw MI values (dotted lines) for feature selection, with embedding rate 0.05bpac.

<sup>2</sup> TR, True Rate, the average of the True Positive rate (TP) and True Negative rate (TN)

Improve Steganalysis by MWM Feature Selection 255

From Fig. 6, 7 and 8, we can observe that WMW feature selection provides higher TR of the classifier than feature selections using only the raw MI values. The reason of this has been discussed in Section 3.2. Note that the last TR value of each curve represents the performance of the classifier consist of all features without selection approach. It is then obvious that we can always achieve better accuracy of steganalysis using WMW feature selections than the original feature sets, whereas feature selection with raw MI values fail in some cases, e.g. F5 steganography with embedding rate 0.1 bpac in Fig. 7, and Perturbed Quantization steganography with embedding rate 0.15 bpac in Fig. 8. Table 1 shows the optimal accuracy of each case, and the TR of classifiers consist of the original feature set are

Embedding Cases MPB Merge POMM Total Raw MWM F5, 0.05bpac 60.5% 80.7% 67.6% 79.7% 80.4% **81.9**% F5, 0.10bpac 73.1% 93.9% 85.5% 94.4% 95.2% **96.0**% F5, 0.15bpac 84.2% 98.3% 94.0% 98.9% **99.2**% **99.2**% PQ, 0.05bpac 68.8% 88.4% 73.3% 89.9% 89.9% **90.5**% PQ, 0.10bpac 72.6% 89.2% 76.2% 91.2% 92.9% **93.4**% PQ, 0.15bpac 71.4% 90.9% 78.0% 92.4% 92.7% **93.9**%

The best accuracy for each embedding case is marked in bold in Table 1, and from that we can assert that the MWM feature selection is always the better choice for steganalysis

In this chapter, we present a new approach of feature selection in steganalysis involving MI and MMD, which are both efficient indicators for evaluating the difference between cover and stego sets. Although the MI values are well understood theoretically, the computational difficulty of estimating the distribution of high-dimensional features makes it inconvenient in steganalysis feature selection. Thus, we treat each dimension as a single feature and

This approach, however, abandons the correlation between feature dimensions, which makes raw MI values defective for feature selection. To solve this problem, MMD values are introduced in our approach as well. The MMD values are numerically stable even in highdimensional spaces, and the computational complexity is relatively low. These advantages

Table 1. Optimal accuracy of steganalysis for different embedding cases using different feature sets: Markov Transition Probability features ('MPB'), Merging Markov and DCT features ('Merge'), Partially Ordered Markov Model features ('POMM'), fused features contain all feature dimensions without feature selection ('Total'), feature selection using only raw MI values ('Raw'), and MWM feature selection

also listed for comparison.

('MWM').

**5. Conclusion** 

comparing to other methods.

calculate MI values separately.

Fig. 7. Comparison of the performance between the classifier using WMW values (solid lines) and the raw MI values (dotted lines) for feature selection, with embedding rate 0.1bpac.

Fig. 8. Comparison of the performance between the classifier using WMW values (solid lines) and the raw MI values (dotted lines) for feature selection, with embedding rate 0.15bpac.

0 10 20 30 40 50

F5, raw MI F5, MWM PQ, raw MI PQ, MWM

F5, raw MI F5, MWM PQ, raw MI PQ, MWM

Serial numbers

0 10 20 30 40 50

Serial numbers

Fig. 8. Comparison of the performance between the classifier using WMW values (solid lines) and the raw MI values (dotted lines) for feature selection, with embedding rate

Fig. 7. Comparison of the performance between the classifier using WMW values (solid lines) and the raw MI values (dotted lines) for feature selection, with embedding rate 0.1bpac.

0.65

0.65

0.7

0.75

0.8

TR of the classifier

0.15bpac.

0.85

0.9

0.95

1

0.7

0.75

0.8

TR of the classifier

0.85

0.9

0.95

1

From Fig. 6, 7 and 8, we can observe that WMW feature selection provides higher TR of the classifier than feature selections using only the raw MI values. The reason of this has been discussed in Section 3.2. Note that the last TR value of each curve represents the performance of the classifier consist of all features without selection approach. It is then obvious that we can always achieve better accuracy of steganalysis using WMW feature selections than the original feature sets, whereas feature selection with raw MI values fail in some cases, e.g. F5 steganography with embedding rate 0.1 bpac in Fig. 7, and Perturbed Quantization steganography with embedding rate 0.15 bpac in Fig. 8. Table 1 shows the optimal accuracy of each case, and the TR of classifiers consist of the original feature set are also listed for comparison.


Table 1. Optimal accuracy of steganalysis for different embedding cases using different feature sets: Markov Transition Probability features ('MPB'), Merging Markov and DCT features ('Merge'), Partially Ordered Markov Model features ('POMM'), fused features contain all feature dimensions without feature selection ('Total'), feature selection using only raw MI values ('Raw'), and MWM feature selection ('MWM').

The best accuracy for each embedding case is marked in bold in Table 1, and from that we can assert that the MWM feature selection is always the better choice for steganalysis comparing to other methods.

## **5. Conclusion**

In this chapter, we present a new approach of feature selection in steganalysis involving MI and MMD, which are both efficient indicators for evaluating the difference between cover and stego sets. Although the MI values are well understood theoretically, the computational difficulty of estimating the distribution of high-dimensional features makes it inconvenient in steganalysis feature selection. Thus, we treat each dimension as a single feature and calculate MI values separately.

This approach, however, abandons the correlation between feature dimensions, which makes raw MI values defective for feature selection. To solve this problem, MMD values are introduced in our approach as well. The MMD values are numerically stable even in highdimensional spaces, and the computational complexity is relatively low. These advantages

Improve Steganalysis by MWM Feature Selection 257

Gul, G. & Kurugollu, F. (2011). A New Methodology in Steganalysis: Breaking Highly

Gretton, A.; Borgwardt, K.; Rasch, M.; Scholkopf, B. & Smola, A. (2007). A Kernel Method

Harmsen, J. J. & Pearlman, W.A. (2003). Steganalysis of Additive Noise Modelable

Johnson, N. F. & Jajodia, S. (1998). Exploring Steganography: Seeing the Unseen. In: *IEEE* 

Kahn, D. (1996). The History of Steganography. *Information Hiding, Lecture Notes in Computer* 

Ker, A. D. (2005). Steganalysis of LSB Matching in Grayscale Images. Signal Processing Letters, IEEE, Vol. 12, No. 6, (June 2005), pp. 441-444, ISSN: 1070-9908 Lyu S. & Farid, H. (2002). Detecting Hidden Messages Using Higher-order Statistics and

Miche, Y.; Roue, B.; Lendasse, A. & Bas, B. (2006). A Feature Selection Methodology for

Mielikainen, J. (2006). LSB matching revisited. *Signal Processing Letters, IEEE*, Vol. 13, No. 5,

Pevný, T.; Bas, P. & Fridrich, J. (2010). Steganalysis by Subtractive Pixel Adjacency Matrix.

Pevný, T.; Filler, T. & Bas, P. (2010). Using High-dimensional Image Models to Perform

*Watermarking of Multimedia Contents IX*, Vol. 6505, pp. 3-1 – 3-14, 2007 Pevný, T. & Fridrich, J. (2008). Benchmarking for Steganography. *Information Hiding, Lecture* 

Provos, N. (2001) Defending Against Statistical Steganalysis. In: *Proceedings of the 10th* 

Provos, N. & Honeyman, P. (2003). Hide and seek: an introduction to steganography.

Shi, Y. Q.; Chen, C. H. & Chen, W. (2007). A Markov Process Based Approach to Effective

*Notes in Computer Science*, Vol. 5284, (2008), pp. 251-267

*USENIX Security Symposium*, pp. 323–336, 2001

*Science*, Vol. 6958, pp. 71-84, 2011

19, (2007), pp. 513–520, MIT Press, Cambridge

*Multimedia Contents VI*, (2003), pp. 131–142

*Computer Society*, Vol. 31, pp. 26–34, 1998

*Notes in Computer Science*, Vol. 4105, (2006), pp. 49-56

(May 2006), pp. 285-287, ISSN: 1070-9908.

215-224, ISSN: 1556-6013

*Science*, Vol. 1174, (1996), pp. 1-5

340-354, 2002

7993

249-264

Undetectable Steganography (HUGO). *Information Hiding, Lecture Notes in Computer* 

for the Two-sample-problem. *Advances in Neural Information Processing Systems*, Vol.

Information Hiding. In: *Proc. SPIE, Security, Steganography, and Watermarking of* 

Support Vector Machines. In: *5th International Workshop on Information Hiding,* pp.

Steganalysis. *Multimedia Content Representation, Classification and Security Lecture* 

*IEEE Transaction on Information Forensics and Security*, Vol. 5, No. 2, (June 2010), pp.

Highly Undetectable Steganography. *Information Hiding, 12th International Workshop, Lecture Notes in Computer Science*, Calgary, Canada, June 28–30, 2010 Pevný, T. & Fridrich, J. (2007). Merging Markov and DCT features for Multi-class JPEG

steganalysis. Proc. *SPIE Electronic Imaging, Security, Steganography, and* 

*Security & Privacy, IEEE*, Vol. 1(3), (May-June 2003), pp. 32-44, ISSN 1540-

Attacking JPEG Steganography. Proc. *Information Hiding '07*, vol.4437, (2007), pp.

make MMD a natural complementary to MI values, and thus leads to our proposed approach of feature selection based on MMD-weighted-MI values.

To test the performance of the MWM feature selection, we apply our method to three typical steganalysis feature sets to generate new classifiers, and estimate the accuracy of these fused classifiers against two widely used steganography algorithms. Experimental results shows that the MWM feature selection approach outperforms the feature selections with raw MI values, and guarantees better accuracy comparing to the original feature sets.

#### **6. Acknowledgment**

This work was supported by the NSF of China under 61170281, the NSF of Beijing under 4112063, the Strategic and Pilot Project of Chinese Academy of Sciences (CAS) under XDA06030600, and the Project of Institute of Information Engineering, CAS, under Y1Z0051101 and Y1Z0041101.

#### **7. References**

Bas, P.; Filler, T. & Pevný, T. (2010). BOSS.

http://boss.gipsa-lab.grenobleinp.fr/BOSSRank/, July 2010


make MMD a natural complementary to MI values, and thus leads to our proposed

To test the performance of the MWM feature selection, we apply our method to three typical steganalysis feature sets to generate new classifiers, and estimate the accuracy of these fused classifiers against two widely used steganography algorithms. Experimental results shows that the MWM feature selection approach outperforms the feature selections with raw MI values, and guarantees better accuracy comparing to the original

This work was supported by the NSF of China under 61170281, the NSF of Beijing under 4112063, the Strategic and Pilot Project of Chinese Academy of Sciences (CAS) under XDA06030600, and the Project of Institute of Information Engineering, CAS, under

Cachin, C. (1998). An information-theoretic Model for Steganography. *Information Hiding 1998, Lecture Notes in Computer Science*, Vol. 1525, (1998), pp. 306–318 Cover, T. M. & Thomas, J. A. (2001). *Frontmatter and Index, in Elements of Information Theory*.

Davidson, J. & Jalan, J. (2010). Steganalysis Using Partially Ordered Markov Models. Proc.

Dong, J.; Chen, X.; Guo, L. & Tan, T. (2008). Fusion Based Blind Image Steganalysis by

Fridrich, J. (2005). Feature-Based Steganalysis for JPEG Images and Its Implications for

Fridrich, J. & Goljan, M. (2002). Practical Steganalysis of Digital Images: State of the Art. In:

Fridrich, J.; Goljan, M. & Hogea, D. (2002). Steganalysis of JPEG Images: Breaking the F5

Fridrich, J.; Goljan, M. & Soukal, D. (2004). Perturbed Quantization Steganography with Wet

Fridrich, J.; Kodovský, J.; Holub, V. & Goljan, M. (2011) Steganalysis of Content-adaptive

Paper Codes. *Proc. ACM Multimedia Workshop'04,* pp. 4-15, 2004

*Information Hiding 2010, Lecture Notes in Computer Science*, Vol. 6387, (2010), pp. 118-

Boosting Feature Selection. *Digital Watermarking, Lecture Notes in Computer Science*,

Future Design of Steganographic Schemes. *Information Hiding, Lecture Notes in* 

*Security and Watermarking of Multimedia Contents*, Vol. SPIE-4675, pp. 1–13,

Algorithm. *5th International Workshop on Information Hiding*, Vol. 2578, pp. 310-323,

Steganography in Spatial Domain. *Information Hiding, Lecture Notes in Computer* 

approach of feature selection based on MMD-weighted-MI values.

http://boss.gipsa-lab.grenobleinp.fr/BOSSRank/, July 2010

John Wiley & Sons, Inc., ISBN 9780471062592, New York

feature sets.

**7. References** 

132

2002

Oct 07-09, 2002

*Science*, 2011

**6. Acknowledgment** 

Y1Z0051101 and Y1Z0041101.

Bas, P.; Filler, T. & Pevný, T. (2010). BOSS.

Vol. 5041, (2008), pp. 87-98

*Computer Science*, Vol. 3200, (2005), pp. 67-81


**12** 

Xin Yan and Yang Wu

*P. R. China* 

**The Digital Watermarking Techniques** 

*Department of Computer Science, Wuhan University of Technology* 

Power supply in the 21st century is facing more and more challenges, e.g., environment protection, energy shortage etc, such that the techniques related to power supply imminently need to be promoted. Complying with the development of green and lowcarbon economy, the concept of Smart Grid has been proposed. By and large, Smart Grid may be regarded as an important application of the techniques of wireless sensor networks and Internet of Things etc in power grid. Smart Grid should be a complicated integrated system with high security, which involves many aspects, e.g., the security of power generation equipments, the security of power transmission equipments, the security of data communications, and so on. Nonetheless, Smart Grid is an open and inclusive system,

The traditional security methods use cryptography to encrypt data for transmissions, for instance, data encryption, data integrity protection, and two-way authentication etc. The data communication networks employed by Smart Grid involve cable and wireless communication networks. Here wireless communication networks usually refer to wireless sensor networks (Akyildiz et al., 2002). Due to the limited resources at sensor nodes, cryptography methods will seriously abate the life time of sensor nodes. The reason is that encryption algorithms usually need to consume more energy, time, and memory space to compute and store data (Kleider et al., 2004; Zia & Zomaya, 2006). Anyway, the traditional encryption methods are not suitable for handling the security issue of data communications in Smart Grid. Thus, this chapter will investigate how to apply a digital watermarking technique to solve the security problem of data communications for wireless sensor

Smart Grid is an intelligent network built in some integrated, high-speed, two-way communication networks. Its objective is to implement the power reliability, security, and efficiency, as well as clean energy supply by using advanced sensor technology, measurement technology and advanced decision support systems. Smart Grid transmits a wide variety of data, including the key equipment operation parameters, the power facility information, the power distribution and scheduling information, the electricity usage state,

**1. Introduction** 

which makes it unsafe inevitably.

networks in Smart Grid.

**2. Smart Grid and wireless sensor networks** 

**Applied to Smart Grid Security** 


## **The Digital Watermarking Techniques Applied to Smart Grid Security**

Xin Yan and Yang Wu

*Department of Computer Science, Wuhan University of Technology P. R. China* 

#### **1. Introduction**

258 Watermarking – Volume 2

Tzschoppe, R. & Aauml, R.B. (2003). Steganographic System based on Higher-order

Westfeld A. (2001). F5 - A Steganographic Algorithm High Capacity Despite Better

Xuan, G. R.; Shi, Y. Q. & Gao, J. J. (2005). Steganalysis based on Multiple Features formed by

(2003), Vol. 5020

27, 2001), pp. 289-302

*2005*, Vol. 3727, p.p. 262-277, 2005

Statistics. In: *Proceedings of SPIE, Security and Watermarking of Multimedia Contents V*,

Steganalysis. *4th International Workshop on Information Hiding*, Vol. 2137, (April 25-

Statistical Moments of Wavelet Characteristic Functions. *Proc. Information Hiding* 

Power supply in the 21st century is facing more and more challenges, e.g., environment protection, energy shortage etc, such that the techniques related to power supply imminently need to be promoted. Complying with the development of green and lowcarbon economy, the concept of Smart Grid has been proposed. By and large, Smart Grid may be regarded as an important application of the techniques of wireless sensor networks and Internet of Things etc in power grid. Smart Grid should be a complicated integrated system with high security, which involves many aspects, e.g., the security of power generation equipments, the security of power transmission equipments, the security of data communications, and so on. Nonetheless, Smart Grid is an open and inclusive system, which makes it unsafe inevitably.

The traditional security methods use cryptography to encrypt data for transmissions, for instance, data encryption, data integrity protection, and two-way authentication etc. The data communication networks employed by Smart Grid involve cable and wireless communication networks. Here wireless communication networks usually refer to wireless sensor networks (Akyildiz et al., 2002). Due to the limited resources at sensor nodes, cryptography methods will seriously abate the life time of sensor nodes. The reason is that encryption algorithms usually need to consume more energy, time, and memory space to compute and store data (Kleider et al., 2004; Zia & Zomaya, 2006). Anyway, the traditional encryption methods are not suitable for handling the security issue of data communications in Smart Grid. Thus, this chapter will investigate how to apply a digital watermarking technique to solve the security problem of data communications for wireless sensor networks in Smart Grid.

#### **2. Smart Grid and wireless sensor networks**

Smart Grid is an intelligent network built in some integrated, high-speed, two-way communication networks. Its objective is to implement the power reliability, security, and efficiency, as well as clean energy supply by using advanced sensor technology, measurement technology and advanced decision support systems. Smart Grid transmits a wide variety of data, including the key equipment operation parameters, the power facility information, the power distribution and scheduling information, the electricity usage state,

The Digital Watermarking Techniques Applied to Smart Grid Security 261

the large magnitude of energy-constrained sensor nodes and the high network dynamics caused by the node mobility or node failure, there still exist a lot of potential threats to the

1. The unauthorized interception of information. A sensor node transmits information to others by broadcasting, so any of communication devices within its RF radius may

2. Sensor nodes are vulnerable to be captured easily. We must take into account what measurements should be taken to fight against, while a sensor node is captured and

3. In the practical environments, we must also consider which routing schemes should be adopted, in the case that some of sensor nodes do not work because of failures or

4. Tampering with information is usually regarded as the most dangerous attack. The tampered information can be spread throughout networks like normal messages, which

These potential threats to wireless sensor networks cause unsafe data communications in Smart Grid. To obtain safe communication services from Smart Grid, we must solve the security issues about wireless sensor networks. However, because of the differences between wireless sensor networks and traditional networks, the security policies for wireless sensor networks should not be borrowed directly from the existing mature security solutions for traditional networks. The security policies should be more suitable for wireless sensor networks. Data encryption methods are widely used in traditional networks, where the information needed to be protected is generated to cipher-text information without readability or obvious correlation. Nevertheless, the resources of computation and storage at sensor nodes are scarce and limited. The traditional data encryption methods will seriously consume the expensive resources at sensor nodes, because they require more power and memory space to accomplish the data encryption procedure. Therefore, we need to use digital watermarking methods to implement the security policies in wireless sensor networks, because digital watermarking needs much less resources at sensor nodes than

security of wireless sensor networks (Wang et al., 2006), e.g.:

used as a pseudo terminal to launch malicious attacks.

can attack or even control the whole networks.

Fig. 2. A network architecture of wireless sensor networks

traditional data encryption (Xiao et al., 2008).

receive and intercept the information.

attacks.

early warning information, and so on (Divan & Johal, 2006). By using the rich information, Smart Grid can efficiently control the power generation, transmission, distribution, scheduling, and sub-time pricing, as well as timely error check etc (Amin & Wollenberg, 2005). The hierarchical model of information flows in Smart Grid is shown in Fig. 1.

Fig. 1. The hierarchical model of information flows in Smart Grid

The communication networks related to Smart Grid consist of cable networks and wireless networks. The wireless networks mainly refer to wireless sensor networks that are usually used in some places where cable networks are not applicable to deploy or wireless sensor networks is more suitable. Smart Grid has a remarkable feature that its networks must be safer than other networks for general purposes. That is to say, Smart Grid must withstand the physical destructions and malicious network attacks without blackouts or a high cost of recovery (Perrig et al., 2004). Smart Grid security involves many aspects, where the data transmission security is one of the most important issues. Since the security mechanisms and techniques in cable networks are already quite rich and mature, we focus on trying to improve the security of the data in wireless sensor networks for Smart Grid.

Wireless sensor networks are a multi-hop self-organized network system, which contains a large number of miniaturized sensor nodes. These sensor nodes are distributed in a monitored area, and communicate in a multi-hop ad hoc way. They collaborate with each other to collect the sensitive information of monitored objects, and send them to a decision support center. The functions of wireless sensor networks consist of data collection, data transmission, and data analysis and processing. A sensor node, the smallest logical unit of wireless sensor networks, is a micro-system, which is integrated by sensor modules, data processing modules and communication modules. Sensor nodes build up wireless links to form a self-organized and distributed network architecture, depending on a certain network routing protocol that can fuse and aggregate the collected data and transmit them to the information processing centre (Chen et al., 2009). A network architecture of wireless sensor networks is shown in Fig. 2.

Smart Grid involves a large number of wireless sensor networks, so the data transmission security is an important issue in Smart Grid. However, due to wireless sensor networks with

early warning information, and so on (Divan & Johal, 2006). By using the rich information, Smart Grid can efficiently control the power generation, transmission, distribution, scheduling, and sub-time pricing, as well as timely error check etc (Amin & Wollenberg,

The communication networks related to Smart Grid consist of cable networks and wireless networks. The wireless networks mainly refer to wireless sensor networks that are usually used in some places where cable networks are not applicable to deploy or wireless sensor networks is more suitable. Smart Grid has a remarkable feature that its networks must be safer than other networks for general purposes. That is to say, Smart Grid must withstand the physical destructions and malicious network attacks without blackouts or a high cost of recovery (Perrig et al., 2004). Smart Grid security involves many aspects, where the data transmission security is one of the most important issues. Since the security mechanisms and techniques in cable networks are already quite rich and mature, we focus on trying to

Wireless sensor networks are a multi-hop self-organized network system, which contains a large number of miniaturized sensor nodes. These sensor nodes are distributed in a monitored area, and communicate in a multi-hop ad hoc way. They collaborate with each other to collect the sensitive information of monitored objects, and send them to a decision support center. The functions of wireless sensor networks consist of data collection, data transmission, and data analysis and processing. A sensor node, the smallest logical unit of wireless sensor networks, is a micro-system, which is integrated by sensor modules, data processing modules and communication modules. Sensor nodes build up wireless links to form a self-organized and distributed network architecture, depending on a certain network routing protocol that can fuse and aggregate the collected data and transmit them to the information processing centre (Chen et al., 2009). A network architecture of wireless sensor

Smart Grid involves a large number of wireless sensor networks, so the data transmission security is an important issue in Smart Grid. However, due to wireless sensor networks with

2005). The hierarchical model of information flows in Smart Grid is shown in Fig. 1.

Fig. 1. The hierarchical model of information flows in Smart Grid

improve the security of the data in wireless sensor networks for Smart Grid.

networks is shown in Fig. 2.

the large magnitude of energy-constrained sensor nodes and the high network dynamics caused by the node mobility or node failure, there still exist a lot of potential threats to the security of wireless sensor networks (Wang et al., 2006), e.g.:


Fig. 2. A network architecture of wireless sensor networks

These potential threats to wireless sensor networks cause unsafe data communications in Smart Grid. To obtain safe communication services from Smart Grid, we must solve the security issues about wireless sensor networks. However, because of the differences between wireless sensor networks and traditional networks, the security policies for wireless sensor networks should not be borrowed directly from the existing mature security solutions for traditional networks. The security policies should be more suitable for wireless sensor networks. Data encryption methods are widely used in traditional networks, where the information needed to be protected is generated to cipher-text information without readability or obvious correlation. Nevertheless, the resources of computation and storage at sensor nodes are scarce and limited. The traditional data encryption methods will seriously consume the expensive resources at sensor nodes, because they require more power and memory space to accomplish the data encryption procedure. Therefore, we need to use digital watermarking methods to implement the security policies in wireless sensor networks, because digital watermarking needs much less resources at sensor nodes than traditional data encryption (Xiao et al., 2008).

The Digital Watermarking Techniques Applied to Smart Grid Security 263

The electric current on electric transmission line is alternating, which means its current value and direction change periodically. In addition, the electric current is a monotonic function of time, a sine trigonometric function. That is to say, both current intensity and orientation are a unique value at any given time within a cycle. These features of alternating electric current are ideal for watermark generation (McDaniel & Mclaughlin, 2009). We use the alphabet *I* to represent the electric current. Physically, it is a vector that contains the information of both its value and direction. As electric current is periodic, the value may be equal although the current direction is different at different times. In order to generate diverse watermark information, we make some special changes to the reverse electric

Suppose the format of a sent packet is Packet = (Head, Send\_Data), where Head is the packet's head including routing information, data type, and packet length etc. Send\_Data is the data which the sensor node sends at a time. It is also the buffer content at the sensor node when its buffer is full. Send\_Data contains a variety of collected data items, and the current at the moment of the data item acquisition. Send\_Data = (Data[1], Data[2], Data[3], …, Data[*n*]), where Data[*i*] (*i* = 1, 2, …, *n*) represents one of the data items collected by the

/\*The current at the moment of the data item acquisition, a vector\*/

Suppose that a packet consists of Head and *m* data items in a collection cycle. The

2. Using single hash function to compute its hash value, according to the key and electric current *I* at the moment of data collection. This step can be described as a program

/\*Boolean value, which identifies whether this data item has

In this chapter, based on alternating electric current and time window respectively, we propose two digital watermarking algorithms in wireless sensor networks for the data

**4. Digital watermarking algorithm based on alternating electric current** 

transmission security of Smart Grid.

current before watermark generation (Cox et al., 2007).

sensor node. Its data type definition is shown in Fig. 4.

/\*Kernel data, which is the protected data\*/

1. Typedef struct Data\_info {

2. *I*;

4. Flag;

watermark\*/ 5. } Data[*i*]

Fig. 4. The data type definition of Send\_Data

1. Taking out each data item from Send\_Data.

statement hsh[*i*] = Hash(Key, Data[*i*].*I*).

watermark generation algorithm is described as follows:

3. Kernal\_data;

**4.1 Algorithmic process 4.1.1 Watermark generation** 

#### **3. Digital watermarking**

Digital watermarking is a special kind of information hiding techniques, which is used to detect piracies or illegal copies. The watermark is transmitted with the information embedded identity in a digital form. Digital watermarking technique is suitable for the data-centric wireless sensor networks. Reasonable watermarking algorithms can ensure the data security at a low cost of operation, and tolerate effectively the impacts from data processing. Using digital watermarking techniques to solve the security issues in the wireless sensor networks for Smart Grid is a practical and effective solution (Xiao et al., 2007).

Fig. 3. The operation procedures of watermarking

Digital watermarking algorithms consist of three basic procedures: watermark generation, watermark embedding, and watermark extraction or detection. The main idea of watermarking algorithms is that watermarks are generated by watermark generation algorithms, and then are embedded into the data collected by sensor nodes. The watermark information is stored in the memory at a node before the data in this node is transmitted. The destination nodes operate the watermark detection in terms of the designated keys and parameters. Only the data with correct watermarks can be considered reliable, meanwhile, it must be eligible for storing and forwarding. Otherwise, it is considered counterfeit or damaged, and discarded directly (Feng & Potkonjak, 2003). The detailed operation procedures about watermarking are shown in Fig. 3.

In this chapter, based on alternating electric current and time window respectively, we propose two digital watermarking algorithms in wireless sensor networks for the data transmission security of Smart Grid.

#### **4. Digital watermarking algorithm based on alternating electric current**

#### **4.1 Algorithmic process**

262 Watermarking – Volume 2

Digital watermarking is a special kind of information hiding techniques, which is used to detect piracies or illegal copies. The watermark is transmitted with the information embedded identity in a digital form. Digital watermarking technique is suitable for the data-centric wireless sensor networks. Reasonable watermarking algorithms can ensure the data security at a low cost of operation, and tolerate effectively the impacts from data processing. Using digital watermarking techniques to solve the security issues in the wireless sensor networks for Smart Grid is a practical and effective solution

Digital watermarking algorithms consist of three basic procedures: watermark generation, watermark embedding, and watermark extraction or detection. The main idea of watermarking algorithms is that watermarks are generated by watermark generation algorithms, and then are embedded into the data collected by sensor nodes. The watermark information is stored in the memory at a node before the data in this node is transmitted. The destination nodes operate the watermark detection in terms of the designated keys and parameters. Only the data with correct watermarks can be considered reliable, meanwhile, it must be eligible for storing and forwarding. Otherwise, it is considered counterfeit or damaged, and discarded directly (Feng & Potkonjak, 2003). The detailed operation

**3. Digital watermarking** 

(Xiao et al., 2007).

Fig. 3. The operation procedures of watermarking

procedures about watermarking are shown in Fig. 3.

#### **4.1.1 Watermark generation**

The electric current on electric transmission line is alternating, which means its current value and direction change periodically. In addition, the electric current is a monotonic function of time, a sine trigonometric function. That is to say, both current intensity and orientation are a unique value at any given time within a cycle. These features of alternating electric current are ideal for watermark generation (McDaniel & Mclaughlin, 2009). We use the alphabet *I* to represent the electric current. Physically, it is a vector that contains the information of both its value and direction. As electric current is periodic, the value may be equal although the current direction is different at different times. In order to generate diverse watermark information, we make some special changes to the reverse electric current before watermark generation (Cox et al., 2007).

Suppose the format of a sent packet is Packet = (Head, Send\_Data), where Head is the packet's head including routing information, data type, and packet length etc. Send\_Data is the data which the sensor node sends at a time. It is also the buffer content at the sensor node when its buffer is full. Send\_Data contains a variety of collected data items, and the current at the moment of the data item acquisition. Send\_Data = (Data[1], Data[2], Data[3], …, Data[*n*]), where Data[*i*] (*i* = 1, 2, …, *n*) represents one of the data items collected by the sensor node. Its data type definition is shown in Fig. 4.

> 1. Typedef struct Data\_info { 2. *I*; /\*The current at the moment of the data item acquisition, a vector\*/ 3. Kernal\_data; /\*Kernel data, which is the protected data\*/ 4. Flag; /\*Boolean value, which identifies whether this data item has watermark\*/ 5. } Data[*i*]

Fig. 4. The data type definition of Send\_Data

Suppose that a packet consists of Head and *m* data items in a collection cycle. The watermark generation algorithm is described as follows:


The Digital Watermarking Techniques Applied to Smart Grid Security 265

3. Generate\_W (Data[*i*], Key, Num) /\*Generating watermarks\*/

/\*Determining which data item will be embedded watermark\*/

The structure of received packet is the same as that of sent data. In order to illustrate it clearly, we describe a received packet as Packet\_R = (Head, Receive\_Data), where Receive\_Data is the content of received packet with watermark information. The watermark

2. Retrieving the state of each data item's flag bit. If the flag is 1, the function Get\_LSB (Data[*i*]) obtains the data item's watermark *W'*, and compare *W'* to *W* = Generate\_W (Receive\_Data[*i*], Key, Num). If they are same, it means the data item

However, the security of a data item does not ensure the packet is safe. In order to measure the security of a packet, we introduce a threshold parameter *P*. It represents the correct watermark rate of all data items in a packet, which shows the authentic level of all data items in a packet. If the watermark detection rate of all data items in a packet is larger than *P*, we say that the credibility of this packet's contents is fully consistent with the requirements. The packet is correct and acceptable; conversely, it should be dropped by the corresponding node. The meta-code of the watermark detection algorithm is shown

We employ Matlab7.0 as our experimental network environment. The coordinate area of simulation configuration is 40m \* 100m, and a total of 50 sensor nodes are distributed uniformly. We draw out 300 packets to analyze, and initialize each node's energy to 2 joules. In order to facilitate and simplify the simulation, the electric current value is measured as follows. The watermarking algorithm makes use of its value directly if the current value is positive. When it is negative, we multiply the current value by a constant, then use the

/\*Selecting some fixed binary bits from LSB as the embedded

1. Embed\_W (Send\_Data , Key, Num, *u*) {

4. MSB\_Data = MSB(Data[*i*].Kernal\_data) 5. rd = random (Key, Data[*i*].*I*, MSB\_Data)

7. Select\_Bits (LSB(Data[i].Kernal\_data))

Fig. 6. The meta-code of the watermark embedding algorithm

1. The node reads each data item in a received packet.

8. Embed watermark WM[*i*] in the fixed bits

2. For *i* =1 to *m*

positions\*/

9. Flag = 1 10. End if 11. End for 12. }

**4.1.3 Watermark detection algorithm** 

detection process is as follows:

Data[*i*] is safe.

**4.2 Performance analysis** 

transformed values to generate watermark.

in Fig. 7.

6. If (rd mod *u* = 0) then


The detailed steps in the watermark generation algorithm are shown in Fig. 5.

1. Generate\_W (Data[*i*], Key, Num) { 2. If (Data[*i*].*I* > 0) then 3. *I'* = Data[*i*].*I* 4. Else 5. *I'* = Translate(Data[*i*].*I*) /\*Making some changes to the reverse current in order to generate a variety of watermark\*/ 6. End if 7. hsh[*i*] = Hash(Key, *I'* ); 8. *W*[*i*] = Produce\_W(MSB(hsh[*i*]), Num) /\*MSB(hsh[*i*]) means obtaining the most significant bit, and the function Produce\_W means XOR to generate watermark of Data[*i*].\*/ 9. Data[*i*].Flag = 0 /\*Initializing the value of the flag bit before watermark embedding\*/ 10. }

Fig. 5. The meta-code of the watermark generation algorithm

#### **4.1.2 Watermark embedding**

To minimize the varying range of data, only the watermark at the least significant bit of data item is embedded. Considering the fact that the energy at sensor nodes is limited, the watermark algorithm should be designed concisely, so we take the following two measures:


The embedding algorithm uses the same key as the generation algorithm. The scaling parameter *u* is selected in terms of the requirements to security, which is used to control the percentage of data items needed to be embedded watermark. We only insert watermark into the data items whose random numbers can divided by *u*. Macroscopically the value of *u* reflects the dense degree of data items embedded watermark in a packet. Larger the value of *u* is, and smaller the probability of related data items inserted watermark is. After determining which data item should be inserted into watermark, we can get the watermark information by using the algorithm in Fig. 5. Next we insert it into the fixed position of the data item's LSB (the least significant bit). The detailed steps of the watermark embedding algorithm are shown in Fig. 6.

3. Getting the most significant bit in hsh[*i*]. The corresponding statement is MSB(hsh[*i*]). 4. Taking Num binary bits from MSB(hsh[*i*]), then XOR them. The result *W*[*i*] is the

/\*Making some changes to the reverse current in order to generate a

/\*MSB(hsh[*i*]) means obtaining the most significant bit, and the function

/\*Initializing the value of the flag bit before watermark embedding\*/

To minimize the varying range of data, only the watermark at the least significant bit of data item is embedded. Considering the fact that the energy at sensor nodes is limited, the watermark algorithm should be designed concisely, so we take the following two

1. Selecting some items randomly from the data items (i.e., Data[1], Data[2], Data[3], …, Data[*m*]) to embed watermark, which can reduce the computational complexity. 2. Deriving the least significant bit of data item Data[*i*], which will be embedded watermark; and selecting some fixed binary bits of the least significant bit, which are the watermark embedding positions. That can simplify the watermark extraction. The embedding algorithm uses the same key as the generation algorithm. The scaling parameter *u* is selected in terms of the requirements to security, which is used to control the percentage of data items needed to be embedded watermark. We only insert watermark into the data items whose random numbers can divided by *u*. Macroscopically the value of *u* reflects the dense degree of data items embedded watermark in a packet. Larger the value of *u* is, and smaller the probability of related data items inserted watermark is. After determining which data item should be inserted into watermark, we can get the watermark information by using the algorithm in Fig. 5. Next we insert it into the fixed position of the data item's LSB (the least significant bit). The detailed steps of the watermark embedding

Produce\_W means XOR to generate watermark of Data[*i*].\*/

The detailed steps in the watermark generation algorithm are shown in Fig. 5.

watermark of data item Data[*i*].

2. If (Data[*i*].*I* > 0) then 3. *I'* = Data[*i*].*I*

variety of watermark\*/

7. hsh[*i*] = Hash(Key, *I'* );

9. Data[*i*].Flag = 0

5. *I'* = Translate(Data[*i*].*I*)

4. Else

6. End if

10. }

measures:

**4.1.2 Watermark embedding** 

algorithm are shown in Fig. 6.

1. Generate\_W (Data[*i*], Key, Num) {

8. *W*[*i*] = Produce\_W(MSB(hsh[*i*]), Num)

Fig. 5. The meta-code of the watermark generation algorithm

1. Embed\_W (Send\_Data , Key, Num, *u*) { 2. For *i* =1 to *m* 3. Generate\_W (Data[*i*], Key, Num) /\*Generating watermarks\*/ 4. MSB\_Data = MSB(Data[*i*].Kernal\_data) 5. rd = random (Key, Data[*i*].*I*, MSB\_Data) 6. If (rd mod *u* = 0) then /\*Determining which data item will be embedded watermark\*/ 7. Select\_Bits (LSB(Data[i].Kernal\_data)) /\*Selecting some fixed binary bits from LSB as the embedded positions\*/ 8. Embed watermark WM[*i*] in the fixed bits 9. Flag = 1 10. End if 11. End for 12. }

Fig. 6. The meta-code of the watermark embedding algorithm

#### **4.1.3 Watermark detection algorithm**

The structure of received packet is the same as that of sent data. In order to illustrate it clearly, we describe a received packet as Packet\_R = (Head, Receive\_Data), where Receive\_Data is the content of received packet with watermark information. The watermark detection process is as follows:


However, the security of a data item does not ensure the packet is safe. In order to measure the security of a packet, we introduce a threshold parameter *P*. It represents the correct watermark rate of all data items in a packet, which shows the authentic level of all data items in a packet. If the watermark detection rate of all data items in a packet is larger than *P*, we say that the credibility of this packet's contents is fully consistent with the requirements. The packet is correct and acceptable; conversely, it should be dropped by the corresponding node. The meta-code of the watermark detection algorithm is shown in Fig. 7.

#### **4.2 Performance analysis**

We employ Matlab7.0 as our experimental network environment. The coordinate area of simulation configuration is 40m \* 100m, and a total of 50 sensor nodes are distributed uniformly. We draw out 300 packets to analyze, and initialize each node's energy to 2 joules. In order to facilitate and simplify the simulation, the electric current value is measured as follows. The watermarking algorithm makes use of its value directly if the current value is positive. When it is negative, we multiply the current value by a constant, then use the transformed values to generate watermark.

The Digital Watermarking Techniques Applied to Smart Grid Security 267

correspondingly, the less amount of watermark information the packet contains. But when *u* takes a smaller value, the detection rate is still quite large (nearly above 95%) regardless of the number of packets increasing. From the experimental results, we are able to anticipate that this algorithm can efficiently operate with a high security when the proper value of *u* is

Watermarks in Packet 1 Watermarks in Packet 2 Watermarks in Packet 3 Original Received Original Received Original Received 1010 1010 1110 1110 1011 1011 1101 1101 1001 1000 1001 1001 1000 1001 1111 1011 0000 0000 0110 0110 0001 0001 1110 0110 1011 1011 0011 0011 1010 1010 1010 1010 1011 1011 0010 1010 1011 1000 1001 1001 1000 1000 1001 1001 1000 1000 1101 1101 0110 0110 1110 0111 0111 0110 0111 0011 0000 0101 1111 1111

Table 1. The comparison received watermarks to original watermarks in 3 data packets

<sup>0</sup> <sup>30</sup> <sup>60</sup> <sup>90</sup> <sup>120</sup> <sup>150</sup> <sup>180</sup> <sup>210</sup> <sup>240</sup> <sup>270</sup> <sup>300</sup> 10%

An important advantage of digital watermarking is that it does not increase the burden of network transmission. In this algorithm, we replace the most important part of the carrier with the watermark through the least significant bit (LSB) method, which does not import an additional data for the original data. Therefore, digital watermarking technique can

In general, the throughput of the networks without watermark information is slightly larger than that of ones with embedded watermark. At the beginning, the number of the nodes forwarding packets is smaller, such that the network is unimpeded and faster. Thus, the network throughput increases rapidly. But as more and more nodes begin to transmit

maintain the throughput of the original network well, as shown in Fig. 9.

Packets number

Eembedding parameter u=2 Eembedding parameter u=8 Eembedding parameter u=16

20% 30% 40% 50% 60% 70% 80% 90% 100%

Fig. 8. The comparison of the securities

**4.2.2 Network throughput** 

Detection rate

chosen.

1. Detect\_W (Receive\_Data, Key, Num, *P*) { 2. Right\_count = W\_count = 0 /\*Right\_count is the total number of data items that can be correctly detected the watermark in a packet. W\_count is the total number of data items that contains watermark in a packet\*/ 3. For *i* = 1 to *m* 4. If (Receive\_Data[*i*].Flag = 1) then 5. W\_count = W\_count + 1 6. *W'* = Get\_LSB (Receive\_Data[*i*]) 7. *W* = Generate\_W (Receive\_Data[*i*], Key, Num) 8. End if 9. If ( *W'* = *W* ) then /\*The watermark information is correct\*/ 10. Right\_count = Right\_count +1 11. End if 12. End for 13. If (Right\_count/W\_count > *P*) then 14. Receive this reliable packet and forward it 15. Else 16. Drop this packet 17. End if 18. }

Fig. 7. The meta-code of the watermark detection algorithm

Before evaluating the performance of this watermarking algorithm, it is necessary to verify that it is reasonable and viable for data security by experiments. Here we can reach the experimental goal by comparing the received watermarked message from a certain data packet to its original watermark message, as shown in Table 1 (three data packets are selected, i.e., Packet 1, Packet 2, and Packet 3). Next, we analyzed the algorithm's performance from three aspects: the security of algorithm, the network throughput, and the node energy consumption.

#### **4.2.1 The security of algorithm**

We evaluate this algorithm's security according to the statistics of its probability of handling the data correctly. For this purpose, we introduce the formula of algorithm detection rate.

$$P\\_Decive = \frac{\text{Decitive\\_Sum}}{\text{Receive\\_Sum}} \tag{1}$$

Wherein *P\_Dective* is the detection rate of packets, and *Dective\_Sum* is the number of packets whose watermark are correctly detected. *Receive\_Sum* is the total number of received packets. In this experiment, we add 7 attacking nodes, 3 camouflage nodes, and assign different values to the embedding parameter *u* at the same time. The results are shown in Fig. 8. Different values of parameter *u* have different impacts on the detection rate to some extent. Larger the value of parameter is, and less the detection rate is. The reason is that the larger value of *u*, the smaller probability of embedding watermark in data items,

/\*Right\_count is the total number of data items that can be correctly detected the watermark in a packet. W\_count is the total number of data

9. If ( *W'* = *W* ) then /\*The watermark information is correct\*/

Before evaluating the performance of this watermarking algorithm, it is necessary to verify that it is reasonable and viable for data security by experiments. Here we can reach the experimental goal by comparing the received watermarked message from a certain data packet to its original watermark message, as shown in Table 1 (three data packets are selected, i.e., Packet 1, Packet 2, and Packet 3). Next, we analyzed the algorithm's performance from three aspects: the security of algorithm, the network throughput, and the

We evaluate this algorithm's security according to the statistics of its probability of handling the data correctly. For this purpose, we introduce the formula of algorithm detection rate.

> \_ \_ Re \_ *Dective Sum P Dective*

Wherein *P\_Dective* is the detection rate of packets, and *Dective\_Sum* is the number of packets whose watermark are correctly detected. *Receive\_Sum* is the total number of received packets. In this experiment, we add 7 attacking nodes, 3 camouflage nodes, and assign different values to the embedding parameter *u* at the same time. The results are shown in Fig. 8. Different values of parameter *u* have different impacts on the detection rate to some extent. Larger the value of parameter is, and less the detection rate is. The reason is that the larger value of *u*, the smaller probability of embedding watermark in data items,

*ceive Sum* (1)

1. Detect\_W (Receive\_Data, Key, Num, *P*) {

items that contains watermark in a packet\*/

7. *W* = Generate\_W (Receive\_Data[*i*], Key, Num)

4. If (Receive\_Data[*i*].Flag = 1) then 5. W\_count = W\_count + 1 6. *W'* = Get\_LSB (Receive\_Data[*i*])

10. Right\_count = Right\_count +1

13. If (Right\_count/W\_count > *P*) then

Fig. 7. The meta-code of the watermark detection algorithm

14. Receive this reliable packet and forward it

2. Right\_count = W\_count = 0

3. For *i* = 1 to *m*

8. End if

11. End if 12. End for

15. Else

17. End if 18. }

node energy consumption.

**4.2.1 The security of algorithm** 

16. Drop this packet

correspondingly, the less amount of watermark information the packet contains. But when *u* takes a smaller value, the detection rate is still quite large (nearly above 95%) regardless of the number of packets increasing. From the experimental results, we are able to anticipate that this algorithm can efficiently operate with a high security when the proper value of *u* is chosen.


Table 1. The comparison received watermarks to original watermarks in 3 data packets

Fig. 8. The comparison of the securities

#### **4.2.2 Network throughput**

An important advantage of digital watermarking is that it does not increase the burden of network transmission. In this algorithm, we replace the most important part of the carrier with the watermark through the least significant bit (LSB) method, which does not import an additional data for the original data. Therefore, digital watermarking technique can maintain the throughput of the original network well, as shown in Fig. 9.

In general, the throughput of the networks without watermark information is slightly larger than that of ones with embedded watermark. At the beginning, the number of the nodes forwarding packets is smaller, such that the network is unimpeded and faster. Thus, the network throughput increases rapidly. But as more and more nodes begin to transmit

The Digital Watermarking Techniques Applied to Smart Grid Security 269

In the experiment, the energy consumption at the nodes that communicate with lots of neighbours is higher. On the contrary, the energy consumption at the nodes with less traffic is lower. Overall, the differences of energy consumption are quite slight in spite of the node

According to the characteristics of time zone storage format of packets in wireless sensor networks and the digital watermarking, we proposed another new digital watermarking algorithm based on time window. At first, we defined the format of packets with encapsulated format, and divide the packet into eleven parts. The contents of each part are

1 2 3 4 5 6 7 8 9 10 11

We use one byte to store the sending time of the packet, and set it to the float type. If we operate the lowest bit of this byte with 0 or 1, its value will be certainly changed. However, this change is quite slight, only between -0.3% and 0.3%. The higher accuracy we use, the smaller impact it will be. The offset value is always at the range of the sensor's tolerance deviation. Time information is recorded in the 6th fixed field of the packet format, so it is the lowest bit of the value. In this case, the sensor for different usages will not be affected even if

And then, we introduce the watermark embedding and detection algorithms based on time storage format. At first, we deal with the data area (i.e., the 9th field) of a packet by the MD5 algorithm, and get a unique mapping. After that, let the value which is generated by this mapping XOR the watermark. At last, embed the result into the hidden bit. The mapping is one-way and irreversible, such that the watermark adding to this mapping can ensure the

The watermark embedding procedure consists of the following four steps (see also Table 4

1. Firstly, we select the 4th field of a packet as the original information *M*, and then operate the original information *M* with the key *K* and the watermark generation algorithm *G*.

better reliability of the transmission (Katzenbeisser & Petitcolas, 2000).

7: Group ID 8: Length of data 9: Content of data

11: End mark of message

10: CRC

**5. Digital watermarking algorithm based on time window** 

1: Beginning mark of message

3: Destination address 4: Source address 5: Packet type

6: Time of sending packet

the lowest bit of the time information is changed.

Table 3. The definition of packet format

**5.1.1 The watermark embedding** 

Then we get the watermark *W*.

in detail):

with watermark or not.

**5.1 Algorithmic process** 

described in Table 3.

2: ACK

packets, the number of sent packets increases, which leads to a slight decline in the throughput of the networks with watermarks because of the watermark embedding and the data operation frequently. At last, with the end of data collection, forwarding, transportation, and processing etc, the network throughput becomes less and less. From the experimental results, the digital watermarking technique can effectively protect the packet transmission. Moreover, it does not increase the burden of network throughput.

Fig. 9. The comparison of the network throughputs

#### **4.2.3 The node energy consumption**

Since the complexity of this watermarking algorithm is *O*(*m*), it does not increase the energy cost at sensor nodes, when processing the watermark information. In addition, the watermark is directly embedded into the data item, which does not take up additional storage space at the node. So the node energy is mainly consumed on data transmission process. Therefore, the digital watermarking technique can well meet the requirement that the energy at sensor nodes is limited in wireless sensor networks. Table 2 is the energy consumption statistics of some nodes.


Table 2. The energy consumption at a part of nodes (unit: micro joule)

In the experiment, the energy consumption at the nodes that communicate with lots of neighbours is higher. On the contrary, the energy consumption at the nodes with less traffic is lower. Overall, the differences of energy consumption are quite slight in spite of the node with watermark or not.

## **5. Digital watermarking algorithm based on time window**

#### **5.1 Algorithmic process**

268 Watermarking – Volume 2

packets, the number of sent packets increases, which leads to a slight decline in the throughput of the networks with watermarks because of the watermark embedding and the data operation frequently. At last, with the end of data collection, forwarding, transportation, and processing etc, the network throughput becomes less and less. From the experimental results, the digital watermarking technique can effectively protect the packet

<sup>0</sup> <sup>20</sup> <sup>40</sup> <sup>60</sup> <sup>80</sup> <sup>100</sup> <sup>120</sup> <sup>140</sup> <sup>160</sup> <sup>180</sup> <sup>200</sup> <sup>220</sup> <sup>240</sup> <sup>260</sup> <sup>280</sup> <sup>300</sup> <sup>0</sup>

Throughput of Network with watermark information Throughput of Network without watermark information

> Node energy consumption (without watermark)

Time s -

Since the complexity of this watermarking algorithm is *O*(*m*), it does not increase the energy cost at sensor nodes, when processing the watermark information. In addition, the watermark is directly embedded into the data item, which does not take up additional storage space at the node. So the node energy is mainly consumed on data transmission process. Therefore, the digital watermarking technique can well meet the requirement that the energy at sensor nodes is limited in wireless sensor networks. Table 2 is the energy

transmission. Moreover, it does not increase the burden of network throughput.

**4.2.3 The node energy consumption** 

consumption statistics of some nodes.

Fig. 9. The comparison of the network throughputs

Node number Node energy consumption

Table 2. The energy consumption at a part of nodes (unit: micro joule)

(with watermark)

1 90 85 3 35 31 6 88 86 7 20 20 9 80 65 12 78 75 16 105 96 23 43 39 25 57 50 33 113 100

Throughput(Number of packets)

According to the characteristics of time zone storage format of packets in wireless sensor networks and the digital watermarking, we proposed another new digital watermarking algorithm based on time window. At first, we defined the format of packets with encapsulated format, and divide the packet into eleven parts. The contents of each part are described in Table 3.


Table 3. The definition of packet format

We use one byte to store the sending time of the packet, and set it to the float type. If we operate the lowest bit of this byte with 0 or 1, its value will be certainly changed. However, this change is quite slight, only between -0.3% and 0.3%. The higher accuracy we use, the smaller impact it will be. The offset value is always at the range of the sensor's tolerance deviation. Time information is recorded in the 6th fixed field of the packet format, so it is the lowest bit of the value. In this case, the sensor for different usages will not be affected even if the lowest bit of the time information is changed.

And then, we introduce the watermark embedding and detection algorithms based on time storage format. At first, we deal with the data area (i.e., the 9th field) of a packet by the MD5 algorithm, and get a unique mapping. After that, let the value which is generated by this mapping XOR the watermark. At last, embed the result into the hidden bit. The mapping is one-way and irreversible, such that the watermark adding to this mapping can ensure the better reliability of the transmission (Katzenbeisser & Petitcolas, 2000).

#### **5.1.1 The watermark embedding**

The watermark embedding procedure consists of the following four steps (see also Table 4 in detail):

1. Firstly, we select the 4th field of a packet as the original information *M*, and then operate the original information *M* with the key *K* and the watermark generation algorithm *G*. Then we get the watermark *W*.

The Digital Watermarking Techniques Applied to Smart Grid Security 271

Input: The document embedded information packet', and the key *K*

1. Using CRC algorithm to calculate the designated data in packet'

2. Compare its value with the content in its CRC field

13. The watermark is right, and this packet is accepted

Table 5. The process of watermark extraction and detection based on time window

packets are drawn out for analysis, and the size of each packet is set to 128 bit. In addition, each node's energy is initialized to 2 joules. We take the embedded value as the source node's ID, and regard the collecting time of data as its sending time approximately. When the parameter configuration is ready, we start to embed watermark and to transmit data. During the transmission, the energy consumption, the processing speed, the time consumption, and all received data at the base station are recorded in different document

Similarly, by comparing the received watermark from a received data packet to its original watermark, as shown in Table 6, it can be seen that this watermark algorithm is also reasonable and viable for data security, because it is able to identify those malicious packets. At the end, we probe its performance from four aspects: the security of algorithm, the

The main objective of this algorithm is to find the counterfeit or damaged data and discarded them directly when there are malicious node attacks during network transmissions. Fig. 10 is the comparison of packet loss between the transmissions with embedded watermark and ones without watermarking in a simulation network environment. In this algorithm, the packet loss in the base station consists of two parts: one is the packet loss in the network communication, and the other is the received packets that are malicious and discarded directly. Seen from Fig. 10, the number of packet loss with embedding watermark is more than that without digital watermarking. We should also note

network throughput, the network delay, and the node energy consumption.

Output: The information packet

3. If (they are same) then

8. ' \_ () *W get data T* ! *lsb* 9. *W GMK* ! ( ,)

6. The packet loss is marked

10. *X'* message\_hash (*X*,*K*)

15. The packet loss is marked

4. Go to loop 8

11. '' ' *W XW* ) 12. If '' ' *W W* then

17. Output packet

5. Else

7. End if

14. Else

16. End if

**5.2.1 The security of algorithm** 

files.

Input: Information packet, and the key *K* Output: The document embedded information, packet' 1. *W G* (*M*,*K*) 2. If (the data buffer is not full) then 3. Continue collecting to fill the data buffer 4. Else 5. *X'* message\_hash (*X*,*K*) 6. End if 7. For *i* = 0 to 8 8. If (the *i*'s value is less than the size of the data buffer) then 9. '' ' *X XW iii* ) 10. 1 *i i* 11. Else 12. Go to loop 15 13. End if 14. End for 15. ( '', ) *T Em X K lsb* 16. Using CRC algorithm to calculate the designated data in the packet 17. Output packet'

Table 4. The process of the embedding watermark based on time window


#### **5.1.2 The watermark extraction and detection**

After transmitting through the relay nodes, packets will reach the base station. We will extract and detect the watermark.

As shown in Table 5, we use CRC algorithm to check from the 3rd to the 9th field data in the packets, and compare the results with the content in its 10th field. If they are not same, the packet should be discarded. Otherwise, we get the embedded data from the lowest bit in the time information, and then extract the watermark *W'* with the watermark extraction algorithm. At last, we compare *W'* with *W* '' . If they are equal, the packet is accepted; if not, it will be discarded.

#### **5.2 Performance analysis**

We investigate the efficiency of the algorithm and its network performance by simulation experiments. The experimental configuration in Matlab7.0 is described as follows. The coordinates area is 40m \* 100m, and a total of 50 sensor nodes are distributed. There are 300

Input: Information packet, and the key *K*

3. Continue collecting to fill the data buffer

2. If (the data buffer is not full) then

5. *X'* message\_hash (*X*,*K*)

1. *W G* (*M*,*K*)

9. '' ' *X XW iii* ) 10. 1 *i i* 11. Else

12. Go to loop 15

15. ( '', ) *T Em X K lsb*

17. Output packet'

(i.e., the 6th field of the packet).

the results into the 10th field in it.

extract and detect the watermark.

**5.1.2 The watermark extraction and detection** 

4. Else

6. End if 7. For *i* = 0 to 8

13. End if 14. End for

the packet.

it will be discarded.

**5.2 Performance analysis** 

Output: The document embedded information, packet'

8. If (the *i*'s value is less than the size of the data buffer) then

16. Using CRC algorithm to calculate the designated data in the packet

2. Secondly, calculate the hashing value of the data items of the packet with the MD5 algorithm, and then get a hashing value hsh which is mapping with the data items of

3. Third, let hsh XOR *W*, and embed the results into the lowest bit of the time information

4. Finally, use CRC algorithm to check from the 3rd to the 9th field in the packet, and put

After transmitting through the relay nodes, packets will reach the base station. We will

As shown in Table 5, we use CRC algorithm to check from the 3rd to the 9th field data in the packets, and compare the results with the content in its 10th field. If they are not same, the packet should be discarded. Otherwise, we get the embedded data from the lowest bit in the time information, and then extract the watermark *W'* with the watermark extraction algorithm. At last, we compare *W'* with *W* '' . If they are equal, the packet is accepted; if not,

We investigate the efficiency of the algorithm and its network performance by simulation experiments. The experimental configuration in Matlab7.0 is described as follows. The coordinates area is 40m \* 100m, and a total of 50 sensor nodes are distributed. There are 300

Table 4. The process of the embedding watermark based on time window


Table 5. The process of watermark extraction and detection based on time window

packets are drawn out for analysis, and the size of each packet is set to 128 bit. In addition, each node's energy is initialized to 2 joules. We take the embedded value as the source node's ID, and regard the collecting time of data as its sending time approximately. When the parameter configuration is ready, we start to embed watermark and to transmit data. During the transmission, the energy consumption, the processing speed, the time consumption, and all received data at the base station are recorded in different document files.

Similarly, by comparing the received watermark from a received data packet to its original watermark, as shown in Table 6, it can be seen that this watermark algorithm is also reasonable and viable for data security, because it is able to identify those malicious packets. At the end, we probe its performance from four aspects: the security of algorithm, the network throughput, the network delay, and the node energy consumption.

#### **5.2.1 The security of algorithm**

The main objective of this algorithm is to find the counterfeit or damaged data and discarded them directly when there are malicious node attacks during network transmissions. Fig. 10 is the comparison of packet loss between the transmissions with embedded watermark and ones without watermarking in a simulation network environment. In this algorithm, the packet loss in the base station consists of two parts: one is the packet loss in the network communication, and the other is the received packets that are malicious and discarded directly. Seen from Fig. 10, the number of packet loss with embedding watermark is more than that without digital watermarking. We should also note

The Digital Watermarking Techniques Applied to Smart Grid Security 273

It is shown in Fig. 11 that the comparison of network throughput between the wireless sensor networks with embedded watermark and that without watermarking in a simulation network environment. The packets which the whole network can send are changed as the simulation time increases. And the network throughput without watermarking is slightly higher than that containing watermarking, with the maximum throughput difference 20. The reason is that the nodes that transmit and forward packets are less at the beginning, and the network is smooth, fast, and less delay such that the data throughput increases slowly. However, as more nodes join into the transmission of packets, the whole network can send more and more packets. Due to embedding the watermark frequently, the throughput with watermarking algorithm will be slightly slower than that without watermarking. Finally, due to the end of data collection, the number of nodes joining transmission and forwarding gradually become less such that the whole network send less and less packets. And the

Generally speaking, the network delay is the interval between the sending time and the receiving time of packets in end-to-end network communication, which consists of the propagation delay, the transmission delay, the queuing delay, and the routing execution delay etc. Fig. 12 is the comparison of network delay between the wireless sensor networks with embedded watermark and that without watermarking. As the simulation time increases, the number of packets which the whole network can send is increasing. At this time, the network delay with watermarking is slightly more than that without watermarking. This reason is that the nodes that transmit and forward packets are less at the beginning, and the network is smooth and fast with less delay. The more nodes join into the transmission, the more packets the base station receives. Due to embedding the watermark frequently in wireless sensor networks with digital watermarking, the network throughput decreases, on the contrary, the network delay increases. Finally, since the transmission

network becomes smooth with less delay and unaffected data throughput.

comes to a close, the network recovers with less delay.

Fig. 11. The comparison of network throughput

**5.2.2 Network throughput** 

**5.2.3 Network delay** 


Table 6. The original watermarks and received watermarks in 15 data packets

that the number of packet loss without watermarking algorithm only is the number of the lost packets during the network communication. However, the number of packet loss in wireless sensor networks with watermarking algorithm not only contains the lost packets during the network communication, but only includes the packet loss during the data processing at the base station. In short, from the experiments we can conclude that this watermarking algorithm for wireless sensor networks can implement the function of identifying and discarding the malicious packets.

Fig. 10. The comparison of packet loss

Fig. 11. The comparison of network throughput

#### **5.2.2 Network throughput**

272 Watermarking – Volume 2

Table 6. The original watermarks and received watermarks in 15 data packets

identifying and discarding the malicious packets.

Fig. 10. The comparison of packet loss

that the number of packet loss without watermarking algorithm only is the number of the lost packets during the network communication. However, the number of packet loss in wireless sensor networks with watermarking algorithm not only contains the lost packets during the network communication, but only includes the packet loss during the data processing at the base station. In short, from the experiments we can conclude that this watermarking algorithm for wireless sensor networks can implement the function of It is shown in Fig. 11 that the comparison of network throughput between the wireless sensor networks with embedded watermark and that without watermarking in a simulation network environment. The packets which the whole network can send are changed as the simulation time increases. And the network throughput without watermarking is slightly higher than that containing watermarking, with the maximum throughput difference 20. The reason is that the nodes that transmit and forward packets are less at the beginning, and the network is smooth, fast, and less delay such that the data throughput increases slowly. However, as more nodes join into the transmission of packets, the whole network can send more and more packets. Due to embedding the watermark frequently, the throughput with watermarking algorithm will be slightly slower than that without watermarking. Finally, due to the end of data collection, the number of nodes joining transmission and forwarding gradually become less such that the whole network send less and less packets. And the network becomes smooth with less delay and unaffected data throughput.

#### **5.2.3 Network delay**

Generally speaking, the network delay is the interval between the sending time and the receiving time of packets in end-to-end network communication, which consists of the propagation delay, the transmission delay, the queuing delay, and the routing execution delay etc. Fig. 12 is the comparison of network delay between the wireless sensor networks with embedded watermark and that without watermarking. As the simulation time increases, the number of packets which the whole network can send is increasing. At this time, the network delay with watermarking is slightly more than that without watermarking. This reason is that the nodes that transmit and forward packets are less at the beginning, and the network is smooth and fast with less delay. The more nodes join into the transmission, the more packets the base station receives. Due to embedding the watermark frequently in wireless sensor networks with digital watermarking, the network throughput decreases, on the contrary, the network delay increases. Finally, since the transmission comes to a close, the network recovers with less delay.

The Digital Watermarking Techniques Applied to Smart Grid Security 275

This chapter begins with a general introduction to Smart Grid, wireless sensor networks, and their security issues. Next it is followed up by the basic principle of digital watermarking applied to Smart Grid. The chapter focus on two digital watermarking schemes based on alternating electric current and time window, respectively. Both of them consist of watermark generation, watermark embedding, and watermark extraction or detection algorithms. Afterward, we evaluate the two watermarking schemes from their security, network throughput and energy consumption etc by lots of simulation experiments. The results show that it is reasonable and beneficial to apply digital watermarking to handle the data security in Smart Grid. The watermarking schemes we propose fully take into account the characteristics of both Smart Grid and wireless sensor networks. With the development of wireless sensor networks and digital watermarking techniques, we believe that digital watermarking would play a more and more important

The data communication security in Smart Grid is a comprehensive and complicated research topic. Although some research fruits are obtained in this chapter, there still remain some problems needed to solve. The digital watermarking schemes proposed in this chapter could bring some distortions for data when considering the interference from communication noise. In addition, the robustness of watermark is not explored yet so far.

This work is supported by the National Natural Science Foundation of China under Grant No. 61171075, the PhD Program Foundation of Ministry of Education of China under Grant No. 200804971030, and the Visiting Scholarship of State Key Laboratory of Power Transmission Equipment & System Security and New Technology (Chongqing University,

Akyildiz, I. F.; Su, W. & Sankar, Y. (2002). Wireless Sensor Networks: a Survey. *The* 

Amin, S. M. & Wollenberg, B. F. (2005). Toward a Smart Grid: Power Delivery for the 21st

Cox, I.; Matthew, M. & Bloom, J. (2007). *Digital Watermarking and Steganography (2nd)*, USA: Morgan Kaufman Publishers, ISBN 978-0-12-372585-1, San Francisco, CA Divan, D. & Johal, H. (2006). A Smarter Grid for Improving System Reliability and Asset

Feng, J. & Potkonjak, M. (2003). Real-Time Watermarking Techniques for Sensor Networks,

*International Journal of Computer and Telecommunications Networking*, Vol.38, No.4,

Century. *IEEE Power and Energy Magazine*, Vol.3, No.5, (Sept.-Oct. 2005), pp. 34-41,

Utilization, *Proceedings of Power Electronics and Motion Control Conference*, ISBN 1-

*Proceedings of IEEE Int. Conf. on Security and Watermarking of Multimedia Contents*,

How to design a robust watermarking scheme without distortion is our future work.

**6. Conclusion** 

role in Smart Grid.

**7. Acknowledgments** 

**8. References** 

China) under Grant No. 2007DA10512710407.

ISSN 1540-7977

(2002), pp. 393-442, ISSN 1389-1286

4244-0448-7, Shanghai, Aug. 2006

Santa Clara, CA, USA, Jan 2003

Fig. 12. The comparison of network delay

#### **5.2.4 The node energy consumption**

It is shown in Fig. 13 that the comparison of node energy consumption between wireless sensor networks with embedded watermark and that without watermarking in a simulation network environment. The nodes sending data can select their neighbour nodes according to the routing and calculated hop-count to transmit and forward packets. This figure shows that the nodes that frequently use the same path will consume more energy. When a sensor node is failure, the node will automatically select the other neighbor nodes to transmit. However, it will prolong the survival of the entire network. From the figure, we can find that the node energy consumption in wireless sensor networks with watermarking algorithm does not differ much from that without watermarking. Therefore, the digital watermarking based on time window can well meet the requirement that the energy consumption at sensor nodes is limited in wireless sensor networks.

Fig. 13. The comparison of node energy consumption

#### **6. Conclusion**

274 Watermarking – Volume 2

It is shown in Fig. 13 that the comparison of node energy consumption between wireless sensor networks with embedded watermark and that without watermarking in a simulation network environment. The nodes sending data can select their neighbour nodes according to the routing and calculated hop-count to transmit and forward packets. This figure shows that the nodes that frequently use the same path will consume more energy. When a sensor node is failure, the node will automatically select the other neighbor nodes to transmit. However, it will prolong the survival of the entire network. From the figure, we can find that the node energy consumption in wireless sensor networks with watermarking algorithm does not differ much from that without watermarking. Therefore, the digital watermarking based on time window can well meet the requirement that the energy

consumption at sensor nodes is limited in wireless sensor networks.

Fig. 13. The comparison of node energy consumption

Fig. 12. The comparison of network delay

**5.2.4 The node energy consumption** 

This chapter begins with a general introduction to Smart Grid, wireless sensor networks, and their security issues. Next it is followed up by the basic principle of digital watermarking applied to Smart Grid. The chapter focus on two digital watermarking schemes based on alternating electric current and time window, respectively. Both of them consist of watermark generation, watermark embedding, and watermark extraction or detection algorithms. Afterward, we evaluate the two watermarking schemes from their security, network throughput and energy consumption etc by lots of simulation experiments. The results show that it is reasonable and beneficial to apply digital watermarking to handle the data security in Smart Grid. The watermarking schemes we propose fully take into account the characteristics of both Smart Grid and wireless sensor networks. With the development of wireless sensor networks and digital watermarking techniques, we believe that digital watermarking would play a more and more important role in Smart Grid.

The data communication security in Smart Grid is a comprehensive and complicated research topic. Although some research fruits are obtained in this chapter, there still remain some problems needed to solve. The digital watermarking schemes proposed in this chapter could bring some distortions for data when considering the interference from communication noise. In addition, the robustness of watermark is not explored yet so far. How to design a robust watermarking scheme without distortion is our future work.

#### **7. Acknowledgments**

This work is supported by the National Natural Science Foundation of China under Grant No. 61171075, the PhD Program Foundation of Ministry of Education of China under Grant No. 200804971030, and the Visiting Scholarship of State Key Laboratory of Power Transmission Equipment & System Security and New Technology (Chongqing University, China) under Grant No. 2007DA10512710407.

#### **8. References**


Kleider, J. E.; Gifford, S. & Chuprun, S. (2004). Radio Frequency Watermarking for OFDM

Katzenbeisser, S. & Petitcolas F. A. P. (2000). *Information Hiding Techniques for Stegonagraphy and Digital Watermarking*, Artech Print on Demand, ISBN 1-58053-035-4, London McDaniel, P. & McLaughlin, S. (2009). Security and Privacy Challenges in the Smart Grid. *IEEE Security & Privacy*, Vol. 7, No. 3, (May-Jun 2009), pp. 75-77, ISSN 1540-7993 Perrig, A.; Stankovic, J. & Wagner, D. (2004). Security in Wireless Sensor Networks. *The* 

Wang, Y.; Attebury, G. & Ramamurthy, B. (2006). A Survey of Security Issues in Wireless

Xiao, R.; Sun, X. & Yang, Y. (2008). Copyright Protection in Wireless Sensor Networks by

Xiao, X.; Sun, X.; Yang, L. & Chen, M. (2007). Secure Data Transmission of Wireless Sensor

Chen, Y. C.; Chuang, C. C.; Chang, R. I.; Lin, J. S. & Wang, T. C. (2009). Integrated Wireless

Zia, P. & Zomaya, A. (2006). Security Issues in Wireless Sensor Networks, *Proceedings of the* 

Sensor Networks. *IEEE Communications Surveys & Tutorials*, Vol.8, No.2, (Feb. 2006),

Watermarking, *Proceedings of IEEE International Conference*, ISBN 978-0-7695-3278-3,

Network Based on Information Hiding, *Proceedings of 4th Annual International Conference on Mobile and Ubiquitous Systems: Computing Networking and Services*,

Access Point Architecture for Wireless Sensor Networks, *Proceedings of ICACT 2009*,

*Int. Conf. on System and Network Communications*, ISBN 0-7695-2699-3, French, Oct.

*Processing*, ISBN 0-7803-8484-9, USA, May 2004

pp. 2-23, ISSN 1553-877X

New Zealand, Aug 2008

2006

*ACM Communications*, Vol.47, No.6, (June 2004), pp. 53-57

ISBN 978-1-4244-1024-8, Philadelphia, PA, USA, Aug 2007

ISBN 978-89-5519-138-7, South Korea, Feb. 2009

Wireless Networks, *Proceedings of IEEE Int. Conf. on Acoustics, Speech, and Signal* 

## *Edited by Mithun Das Gupta*

This collection of books brings some of the latest developments in the field of watermarking. Researchers from varied background and expertise propose a remarkable collection of chapters to render this work an important piece of scientific research. The chapters deal with a gamut of fields where watermarking can be used to encode copyright information. The work also presents a wide array of algorithms ranging from intelligent bit replacement to more traditional methods like ICA. The current work is split into two books. Book one is more traditional in its approach dealing mostly with image watermarking applications. Book two deals with audio watermarking and describes an array of chapters on performance analysis of algorithms.

Photo by 8vFanI / iStock

Watermarking - Volume 2

Watermarking

Volume 2

*Edited by Mithun Das Gupta*