**1.1 CGHs for two-dimensional image display**

Presenting visual information using a phase-only holographic approach provides a significant efficiency advantage compared to conventional video projection techniques. Unlike conventional projection displays, which utilize amplitude-modulating microdisplays to selectively block incident optical energy to form the desired image, a holographic display employing an ideal dynamic phase-modulating CGH has a transmission of near unity. Significant efficiency gains could therefore be realized compared to conventional LCOS or DLP-based projectors, in which the illumination is set at a level sufficient to produce a peak white value regardless of the average pixel level (APL) of the scene. Furthermore, the use of an LCOS display as the dynamic modulating element in a laser-based holographic projector allows the removal of front polarizer which serves to waste an additional 50% of the available light in LED-illuminated systems.

The properties of diffraction potentially also allow for projection angles several times greater than currently possible in conventional LCOS-based systems. Conventional LCOS-based projection systems are limited by the necessity for a relatively large projection lens, since the function of the projection lens assembly is to enlarge an already sizeable image; to miniaturize the projection optics, then, the resultant image size must be shrunk concomitantly or be subject to severe aberrations, which can only be reduced through the use of highly complex and expensive lens systems. A phase-only holographic projector, on the other hand, is able to exert control over the entire optical field and consequently Buckley et al. (2009) was able to demonstrate that ultra-wide projection angles and novel projection geometries could be achieved without residual optical aberrations.

In addition to the compact, simple opto-mechanical assembly of a CGH-based projector, the use of solid-state light sources and LCOS-based light modulators would result in a system containing no moving parts. Fault tolerance of the optical system, which is achieved since the hologram pattern is decoupled from the desired image by a Fourier relationship, is also an attractive property in some applications where display integrity is required and "dead pixels" are unacceptable.

Although there have been plenty of examples of using fixed holograms for 2D image formation by Heggarty & Chevalier (1998); Kirk et al. (1992); Lesem & Hirsch (1969); Taghizadeh (1998; 2000); Takaki & Hojo (1999), previous attempts at real-time image projection and display using CGHs have been mainly limited to the 3D case and the demonstrations by Ito et al. (2005); Ito & Okano (2004); Ito & Shimobaba (2004); Sando et al. (2004) have required significant computational resources. The few attempts at an implementation of real-time 2D holographic projection by, for example, Mok et al. (1986); Papazoglou et al. (2002); Poon et al. (1993) have been affected by critical limitations imposed both by the computational complexity of the hologram generation algorithms required, and by the poor quality of images produced by the binary holograms they generate.

Recently, a great deal of progress has been made in using binary-phase CGHs for projection as detailed in Buckley (2008a;b; 2011a) and a new approach to hologram generation and display, based on a psychometrically-determined perceptual measure of image quality, has been shown to overcome both of these problems and has resulted in the commercialization of a real-time 2D holographic projector. This chapter will bring together, for the first time, the recent theoretical and practical advances in realizing 2D and 3D holographic projection systems based on binary phase CGHs.

### **2. Motivation**

2 Will-be-set-by-IN-TECH

(1981); Gu et al. (1986); Roux (1991; 1993); Stuff & Cederquist (1990), self-adjusting CGHs Lofving (1997), aspheric testing Tang & Chang (1992), and wavelength discrimination for wavelength-division multiplexing (WDM) applications Dong et al. (1998; 1996); Layet et al.

Despite the obvious benefits of computer-generated holography for a wide range of applications, however, it is only recently that CGHs have been demonstrated for the projection and display of two dimensional video-style images. Indeed, such a method of image projection and display has long been desired, but was never previously realized, due to high

Presenting visual information using a phase-only holographic approach provides a significant efficiency advantage compared to conventional video projection techniques. Unlike conventional projection displays, which utilize amplitude-modulating microdisplays to selectively block incident optical energy to form the desired image, a holographic display employing an ideal dynamic phase-modulating CGH has a transmission of near unity. Significant efficiency gains could therefore be realized compared to conventional LCOS or DLP-based projectors, in which the illumination is set at a level sufficient to produce a peak white value regardless of the average pixel level (APL) of the scene. Furthermore, the use of an LCOS display as the dynamic modulating element in a laser-based holographic projector allows the removal of front polarizer which serves to waste an additional 50% of the available

The properties of diffraction potentially also allow for projection angles several times greater than currently possible in conventional LCOS-based systems. Conventional LCOS-based projection systems are limited by the necessity for a relatively large projection lens, since the function of the projection lens assembly is to enlarge an already sizeable image; to miniaturize the projection optics, then, the resultant image size must be shrunk concomitantly or be subject to severe aberrations, which can only be reduced through the use of highly complex and expensive lens systems. A phase-only holographic projector, on the other hand, is able to exert control over the entire optical field and consequently Buckley et al. (2009) was able to demonstrate that ultra-wide projection angles and novel projection geometries could be

In addition to the compact, simple opto-mechanical assembly of a CGH-based projector, the use of solid-state light sources and LCOS-based light modulators would result in a system containing no moving parts. Fault tolerance of the optical system, which is achieved since the hologram pattern is decoupled from the desired image by a Fourier relationship, is also an attractive property in some applications where display integrity is required and "dead pixels"

Although there have been plenty of examples of using fixed holograms for 2D image formation by Heggarty & Chevalier (1998); Kirk et al. (1992); Lesem & Hirsch (1969); Taghizadeh (1998; 2000); Takaki & Hojo (1999), previous attempts at real-time image projection and display using CGHs have been mainly limited to the 3D case and the demonstrations by Ito et al. (2005); Ito & Okano (2004); Ito & Shimobaba (2004); Sando et al. (2004) have required significant computational resources. The few attempts at an implementation of real-time 2D holographic projection by, for example, Mok et al. (1986); Papazoglou et al. (2002); Poon et al. (1993) have been affected by critical limitations imposed both by the computational

computational complexity and poor quality of the resultant images.

**1.1 CGHs for two-dimensional image display**

light in LED-illuminated systems.

are unacceptable.

achieved without residual optical aberrations.

(1999); Yang et al. (1994).

For video display applications, in which the APL is significantly less than the full-white maximum, a projection display based on phase-only computer generated holography could offer a significant efficiency advantage compared to amplitude-modulating LCOS displays, since light is not blocked from the desired image pixels. Quantifying this benefit has proven difficult, however, since there is widespread disagreement in the published literature from, for example, Bhatia et al. (2009; 2007); Buckley et al. (2008); Lee et al. (2009); Weber (2005) as to an acceptable value to use for the APL. The variation in reported values appears to result from the point at which the APL measurement is defined.

In a generalized display, the light intensity produced *Lout* is related to the video signal voltage *V* by *Lout* ∝ *V<sup>γ</sup>* , where *γ* is the display gamma. To obtain a display intensity response *Lout* which is linear with respect to the video image *P*, the transmit video signal *V* is encoded by an inverse gamma correction function so that *V* ∝ *P*1/*γ*. To ensure a uniform perceptual response, the display gamma is typically set to *γ* = 2.2 to match the approximate lightness sensitivity of a human viewer.

In a projection architecture in which the light sources can be modulated in response to average scene or per-pixel brightness, the resultant efficiency benefit is directly related to the mean value of *Lout*, E [*Lout*], which is clearly not equal to E [*V*] when *γ* � 1. In order to calculate this mean value, and since neither the form of *Lout* nor *V* are known a priori, we must derive a statistical model for the pixel distribution pre- and post-gamma.

Consider an image pixel *P* that can take a value in the range [0, *p*), quantized into *n* bins of size *<sup>b</sup>* so that *<sup>b</sup>* = *<sup>p</sup>*/*n*. The number of occurrences of a pixel value within the bin [*pi*−1, *<sup>p</sup>*) is *ki*, so that the total number of occurrences of that pixel value is

$$\sum\_{i=1}^{\infty} k\_i = k \tag{1}$$

and the total number of occurrences *k* is fixed, so that

$$\frac{1}{k}\sum\_{i=1}^{\infty}k\_i = \mathfrak{e} \tag{2}$$

where *�* is some positive constant.

We define Pr*n*(*b*) to be the probability that the pixel value will fall into the *bth* bin *n* times. Since each pixel has an equal probability of taking a value in the range [0, *p*), the probability

polarizer and careful étendue matching, the efficiency gain could approach one order of magnitude and clearly motivates the investigation of a projection system based on phase-only

Computer-Generated Phase-Only Holograms for Real-Time Image Display 281

A holographic display employs a phase-modulating display element in combination with a coherent light source to form images by diffraction, rather than projection. A Fraunhofer (or far-field) holographic display is based on the result that, when a hologram *h*(*u*, *v*) is illuminated by coherent collimated light of wavelength *λ*, the complex field *F*(*x*, *y*) formed in the back focal plane of the lens of focal length *f* due to Fraunhofer diffraction from the pattern *h*(*u*, *v*) is the two-dimensional spatial Fourier transform of the hologram pattern:

*<sup>h</sup>*(*u*, *<sup>v</sup>*) exp <sup>2</sup>*j<sup>π</sup>*

*Fxy huv*

*f*

Fig. 1. The relationship between hologram *h*(*u*, *v*) and image *F*(*x*, *y*) present at the back focal plane of a lens of focal length *f* , when illuminated by coherent monochromatic light of

If the continuous hologram pattern is then replaced by an element with pixel size Δ then the image *Fxy* formed (or replayed) in the focal plane of the lens is related to the pixellated

> *Q*−1 ∑ *v*=0

Despite the potential advantages of a holographic display, previous attempts at constructing such a system as detailed by Georgiou et al. (2008); Heggarty & Chevalier (1998); Mok et al. (1986); Papazoglou et al. (2002); Poon et al. (1993) have been unable to overcome two

The first difficulty is that of calculating a hologram *huv* such that, when illuminated by coherent light, a high quality image *Fxy* is formed. It is not possible to simply invert the Fourier transform relationship of equation 10 to obtain the desired hologram *huv*, since the result of this calculation would be fully complex and there is no material in existence that can independently modulate both amplitude *Auv* and phase *ϕuv* where *huv* = *Auv* exp *jϕuv*. Even if such a material became available, the result contains amplitude components which would

exp 2*jπ*

 *ux <sup>P</sup>* <sup>+</sup> *vy Q* 

hologram pattern *huv* by the discrete Fourier transform F [·], and is written as

*P*−1 ∑ *u*=0

*Fxy* = F [*huv*] =

*<sup>λ</sup> <sup>f</sup>* (*ux* <sup>+</sup> *vy*)

*f*

*dudv* (9)

(10)

holography.

**3. 2D Fourier holography**

fundamental technical problems.

wavelength *λ*.

*<sup>F</sup>*(*x*, *<sup>y</sup>*) = <sup>∞</sup>

The relationship of equation 9 is illustrated in Figure 1.

−∞

that a pixel is addressed once with a value in the *bth* bin is Pr1(*b*) = *λb*, where *λ* is a constant, and the probability that a pixel is not addressed is Pr0(*b*) = 1 − *λb*.

We wish to find Pr(*P* > *p*), where *P* is the smallest pixel value, which is equivalent to finding the probability that a pixel is not addressed with any value in the range (0, *p*). If we suppose further that the pixel value probabilities in any bin are independent of each other, then we obtain

$$\Pr(P > p) = \Pr\_0^n(b) = \left(1 - \frac{\lambda p}{n}\right)^n \tag{3}$$

From elementary calculus,

$$\lim \left( 1 - \frac{\lambda p}{n} \right)^n = \exp \left( -\lambda p \right) \tag{4}$$

so that

$$\Pr(P > p) = \lim\_{b \to 0} \Pr\_{\mathbb{H}}(P > p) = \exp(-\lambda p) \tag{5}$$

and it follows that the corresponding probability density function (PDF) *fP*(*p*) is

$$f\_P(p) = -\frac{d}{dp}\Pr(P > p) = \lambda \exp(-\lambda p) \tag{6}$$

where *p* > 0, thus completing the proof that the pixel values are exponentially distributed with mean *λ*.

Let the image pixels *P* be subject to a gamma encoding process with value *γ* such that *<sup>V</sup>* <sup>∝</sup> *<sup>P</sup>*1/*γ*. If *<sup>P</sup>* is exponentially-distributed with parameter *<sup>λ</sup>*, written as *<sup>P</sup>* <sup>∼</sup> exp(*λ*), then Leemis & McQueston (2008) provides the standard result that the PDF of the random variable *V*, *fV*(*v*), is Weibull distributed *V* ∼ Weibull[*α*, *β*], or

$$f\_{V}(v; \alpha, \beta) = \frac{\alpha}{\beta} \left(\frac{v}{\beta}\right)^{\alpha - 1} \exp\left(-\frac{v}{\beta}\right)^{\alpha} \tag{7}$$

with mean value given by

$$\mathbb{E}\left[V\right] = a\Gamma\left(1 + \frac{1}{\beta}\right) \tag{8}$$

where Γ is the Gamma function, *α* = *λ* 1 *<sup>γ</sup>* and *β* = *γ*. A number of measurements of *V* for typical TV content are provided by Jones & Harrison (2007); Lee et al. (2009); Stobbe et al. (2008); Weber (2005) and Jones & Harrison (2007) presents curves for experimentally-measured APL data by country, to which a Weibull-distributed variable *V* ∼ Weibull[*α* = 0.43, *β* = 2.2] with mean of approximately 38% is an excellent fit to the average measured APL.

Since we know from experimentally-measured transmission data and equation 6 that the average pixel value in a video image is E [*P*] = *λ* = *α<sup>γ</sup>* = 16%, then we can reasonably state that, due to the nature of a typical video image, the average optical utilization efficiency of a holographic projector should be a factor of six greater than an LCOS-based system excluding all other inefficiencies. When comparing to LED-illuminated systems which require a front polarizer and careful étendue matching, the efficiency gain could approach one order of magnitude and clearly motivates the investigation of a projection system based on phase-only holography.

### **3. 2D Fourier holography**

4 Will-be-set-by-IN-TECH

that a pixel is addressed once with a value in the *bth* bin is Pr1(*b*) = *λb*, where *λ* is a constant,

We wish to find Pr(*P* > *p*), where *P* is the smallest pixel value, which is equivalent to finding the probability that a pixel is not addressed with any value in the range (0, *p*). If we suppose further that the pixel value probabilities in any bin are independent of each other, then we

<sup>0</sup> (*b*) =

*<sup>n</sup>*

where *p* > 0, thus completing the proof that the pixel values are exponentially distributed

Let the image pixels *P* be subject to a gamma encoding process with value *γ* such that *<sup>V</sup>* <sup>∝</sup> *<sup>P</sup>*1/*γ*. If *<sup>P</sup>* is exponentially-distributed with parameter *<sup>λ</sup>*, written as *<sup>P</sup>* <sup>∼</sup> exp(*λ*), then Leemis & McQueston (2008) provides the standard result that the PDF of the random

*<sup>α</sup>*−<sup>1</sup>

 1 + 1 *β* 

exp − *v β α*

 <sup>1</sup> <sup>−</sup> *<sup>λ</sup><sup>p</sup> n*

*<sup>n</sup>*

= exp (−*λp*) (4)

Pr*n*(*P* > *p*) = exp(−*λp*) (5)

*<sup>γ</sup>* and *β* = *γ*. A number of measurements of

*dp* Pr(*<sup>P</sup>* <sup>&</sup>gt; *<sup>p</sup>*) = *<sup>λ</sup>* exp(−*λp*) (6)

(3)

(7)

(8)

and the probability that a pixel is not addressed is Pr0(*b*) = 1 − *λb*.

Pr(*P* > *p*) = Pr*<sup>n</sup>*

lim <sup>1</sup> <sup>−</sup> *<sup>λ</sup><sup>p</sup> n*

Pr(*P* > *p*) = lim

*fP*(*p*) = <sup>−</sup> *<sup>d</sup>*

variable *V*, *fV*(*v*), is Weibull distributed *V* ∼ Weibull[*α*, *β*], or

*fV*(*v*; *<sup>α</sup>*, *<sup>β</sup>*) = *<sup>α</sup>*

*β v β*

E [*V*] = *α*Γ

1

*V* for typical TV content are provided by Jones & Harrison (2007); Lee et al. (2009); Stobbe et al. (2008); Weber (2005) and Jones & Harrison (2007) presents curves for experimentally-measured APL data by country, to which a Weibull-distributed variable *V* ∼ Weibull[*α* = 0.43, *β* = 2.2] with mean of approximately 38% is an excellent fit to the

Since we know from experimentally-measured transmission data and equation 6 that the average pixel value in a video image is E [*P*] = *λ* = *α<sup>γ</sup>* = 16%, then we can reasonably state that, due to the nature of a typical video image, the average optical utilization efficiency of a holographic projector should be a factor of six greater than an LCOS-based system excluding all other inefficiencies. When comparing to LED-illuminated systems which require a front

*b*→0

and it follows that the corresponding probability density function (PDF) *fP*(*p*) is

obtain

so that

with mean *λ*.

with mean value given by

average measured APL.

where Γ is the Gamma function, *α* = *λ*

From elementary calculus,

A holographic display employs a phase-modulating display element in combination with a coherent light source to form images by diffraction, rather than projection. A Fraunhofer (or far-field) holographic display is based on the result that, when a hologram *h*(*u*, *v*) is illuminated by coherent collimated light of wavelength *λ*, the complex field *F*(*x*, *y*) formed in the back focal plane of the lens of focal length *f* due to Fraunhofer diffraction from the pattern *h*(*u*, *v*) is the two-dimensional spatial Fourier transform of the hologram pattern:

$$F(x,y) = \int\_{-\infty}^{\infty} h(u,v) \exp\left\{\frac{2j\pi}{\lambda f} \left(u\mathbf{x} + vy\right)\right\} du dv \tag{9}$$

The relationship of equation 9 is illustrated in Figure 1.

Fig. 1. The relationship between hologram *h*(*u*, *v*) and image *F*(*x*, *y*) present at the back focal plane of a lens of focal length *f* , when illuminated by coherent monochromatic light of wavelength *λ*.

If the continuous hologram pattern is then replaced by an element with pixel size Δ then the image *Fxy* formed (or replayed) in the focal plane of the lens is related to the pixellated hologram pattern *huv* by the discrete Fourier transform F [·], and is written as

$$F\_{\mathbf{x}\mathbf{y}} = \mathcal{F}\left[\mathbb{I}\_{\mathbf{u}\mathbf{y}}\right] = \sum\_{\mu=0}^{P-1} \sum\_{\upsilon=0}^{Q-1} \exp 2j\pi \left(\frac{\mu\mathbf{x}}{P} + \frac{\upsilon\mathbf{y}}{Q}\right) \tag{10}$$

Despite the potential advantages of a holographic display, previous attempts at constructing such a system as detailed by Georgiou et al. (2008); Heggarty & Chevalier (1998); Mok et al. (1986); Papazoglou et al. (2002); Poon et al. (1993) have been unable to overcome two fundamental technical problems.

The first difficulty is that of calculating a hologram *huv* such that, when illuminated by coherent light, a high quality image *Fxy* is formed. It is not possible to simply invert the Fourier transform relationship of equation 10 to obtain the desired hologram *huv*, since the result of this calculation would be fully complex and there is no material in existence that can independently modulate both amplitude *Auv* and phase *ϕuv* where *huv* = *Auv* exp *jϕuv*. Even if such a material became available, the result contains amplitude components which would

An effective demonstration of the deficiency of the MSE measure is provided in the following

Computer-Generated Phase-Only Holograms for Real-Time Image Display 283

change in contrast is given by *c* and the mean value of the image pixels is *μ* , then the MSE

The resultant images are shown in Figure 3, together with MSE figures calculated using

the MSE metrics in fact indicate the opposite. It is clear from equation 12 and Figure 3 that MSE is in fact dominated by mean image errors caused by the contrast change, rather than the

<sup>2</sup> + *σ*<sup>2</sup>

*xy* exhibits the highest perceptual image quality, and *F*<sup>3</sup>

*J* = (*μ* + *c*)

additive Gaussian noise which corresponds with poor perceptual image quality.

Fig. 3. The poor correspondence of the MSE metric with image quality. Whilst *F*<sup>1</sup>

of the complex Fourier Transform information by a phase only hologram.

*Fx*

*P*−1 ∑ *u*=0

In order to determine an improved optimization metric, it is necessary to derive the properties of noise in holographic replay and, in particular, the noise resulting from the approximation

Without loss of generality, we consider the one-dimensional image *Fx*, which is the discrete Fourier transform (DFT) of the corresponding *P*-pixel hologram *hu*, and is termed the replay

*hu* exp

Since the form of *hu* is not known a priori, because it is the result of some unspecified calculation, it can only be assumed that *hu* is a random variable with some, as yet unknown, distribution. The quest to determine the properties of holographic replay therefore begins by considering the properties of samples of the DFT of a random sequence *hu*, proceeding

−2*jπxu P*

*xy* are generated from a target image *Txy*. Image

*xy* exhibits both a change in contrast and additive noise. If the

*xy*: *J* = 81 (c) *F*<sup>3</sup>

*xy* the worst, the MSE metric *J* indicates the reverse.

*xy* contains additive Gaussian

*xy*: *J* = 67

*xy* exhibits

(13)

*xy* the lowest,

*<sup>n</sup>* (12)

*xy* and *F*<sup>3</sup>

*xy*, *F*<sup>2</sup>

*xy* is equivalent to *Txy* except for a small contrast change, *F*<sup>2</sup>

*xy*: *J* = 103 (b) *F*<sup>2</sup>

example, in which three images *F*<sup>1</sup>

*<sup>n</sup>*, and *F*<sup>3</sup>

metric for each of the images can be shown to be

noise of variance *σ*<sup>2</sup>

equation 12. Although *F*<sup>1</sup>

(a) *F*<sup>1</sup>

field (RPF):

the highest image quality, and *F*<sup>3</sup>

**3.2 General properties of holographic replay**

*F*1

absorb incident light and reduce system efficiency. A much better approach is to restrict the hologram *huv* to a set of phase only values exp *jϕuv*. Performing this operation on *huv* whilst maintaining high image quality in *Fxy* is absolutely non trivial, and requires computation to mitigate the effects of information lost in the quantization.

The second problem is one of computation. Until recently, there was no hologram-generation method in existence that could simultaneously produce images of sufficient quality for video style images, whilst calculating the holograms quickly enough to allow real-time image display. Figure 2 provides a good example; the 512 × 512-pixel hologram *huv* of Figure 2(b) took 10 hours to compute using the standard direct binary search (DBS) algorithm proposed in Dames et al. (1991); Seldowitz et al. (1987) and the resultant reconstruction *Fxy*, shown in Figure 2(c), is a very poor representation of the desired image *Txy* of Figure 2(a).

In this section it is shown that the twin barriers to the realization of a real-time, high quality holographic display can be overcome by defining a new, psychometrically-determined, measure of image quality that is matched to human visual perception. A method of displaying phase holograms that is optimized with respect to this new measure is presented, and is shown to result in high-quality image reproduction.

Fig. 2. Image |*Fxy*| <sup>2</sup> resulting from the reconstruction of a desired image *Txy* from a binary phase-only hologram *huv* calculated using the DBS algorithm.

### **3.1 An improved method for hologram generation**

Conventional hologram generation algorithms such as DBS, and the Gerchberg-Saxton (GS) described in Gerchberg & Saxton (1972), attempt to exhaustively optimize a hologram to minimize some metric *J*, which is calculated by comparing the projected image *Fxy* with respect to a target image *Txy* within some region Ω. Typically, such algorithms employ the mean-squared error (MSE) measure where

$$J = \sum\_{\Omega} |F\_{xy} - \gamma T\_{xy}|^2 \tag{11}$$

and *γ* is a normalizing constant chosen to minimize equation 11 - which seems intuitively satisfying, since zero MSE implies a perfect reconstruction. Unfortunately, this metric is particularly insensitive for the low MSE values typically encountered in holographically-generated images.

6 Will-be-set-by-IN-TECH

absorb incident light and reduce system efficiency. A much better approach is to restrict the hologram *huv* to a set of phase only values exp *jϕuv*. Performing this operation on *huv* whilst maintaining high image quality in *Fxy* is absolutely non trivial, and requires computation to

The second problem is one of computation. Until recently, there was no hologram-generation method in existence that could simultaneously produce images of sufficient quality for video style images, whilst calculating the holograms quickly enough to allow real-time image display. Figure 2 provides a good example; the 512 × 512-pixel hologram *huv* of Figure 2(b) took 10 hours to compute using the standard direct binary search (DBS) algorithm proposed in Dames et al. (1991); Seldowitz et al. (1987) and the resultant reconstruction *Fxy*, shown in

In this section it is shown that the twin barriers to the realization of a real-time, high quality holographic display can be overcome by defining a new, psychometrically-determined, measure of image quality that is matched to human visual perception. A method of displaying phase holograms that is optimized with respect to this new measure is presented, and is

Figure 2(c), is a very poor representation of the desired image *Txy* of Figure 2(a).

(a) *Txy* (b) *huv* = exp *jφuv* (c)

*J* = ∑ Ω

Conventional hologram generation algorithms such as DBS, and the Gerchberg-Saxton (GS) described in Gerchberg & Saxton (1972), attempt to exhaustively optimize a hologram to minimize some metric *J*, which is calculated by comparing the projected image *Fxy* with respect to a target image *Txy* within some region Ω. Typically, such algorithms employ the


and *γ* is a normalizing constant chosen to minimize equation 11 - which seems intuitively satisfying, since zero MSE implies a perfect reconstruction. Unfortunately, this metric is particularly insensitive for the low MSE values typically encountered in

<sup>2</sup> resulting from the reconstruction of a desired image *Txy* from a binary

*Fxy* 

<sup>2</sup> (11)

<sup>2</sup> <sup>=</sup> |F [*huv*]<sup>|</sup>

2

mitigate the effects of information lost in the quantization.

shown to result in high-quality image reproduction.

phase-only hologram *huv* calculated using the DBS algorithm.

**3.1 An improved method for hologram generation**

mean-squared error (MSE) measure where

holographically-generated images.

Fig. 2. Image |*Fxy*|

An effective demonstration of the deficiency of the MSE measure is provided in the following example, in which three images *F*<sup>1</sup> *xy*, *F*<sup>2</sup> *xy* and *F*<sup>3</sup> *xy* are generated from a target image *Txy*. Image *F*1 *xy* is equivalent to *Txy* except for a small contrast change, *F*<sup>2</sup> *xy* contains additive Gaussian noise of variance *σ*<sup>2</sup> *<sup>n</sup>*, and *F*<sup>3</sup> *xy* exhibits both a change in contrast and additive noise. If the change in contrast is given by *c* and the mean value of the image pixels is *μ* , then the MSE metric for each of the images can be shown to be

$$J = (\mu + \mathfrak{c})^2 + \sigma\_n^2 \tag{12}$$

The resultant images are shown in Figure 3, together with MSE figures calculated using equation 12. Although *F*<sup>1</sup> *xy* exhibits the highest perceptual image quality, and *F*<sup>3</sup> *xy* the lowest, the MSE metrics in fact indicate the opposite. It is clear from equation 12 and Figure 3 that MSE is in fact dominated by mean image errors caused by the contrast change, rather than the additive Gaussian noise which corresponds with poor perceptual image quality.

Fig. 3. The poor correspondence of the MSE metric with image quality. Whilst *F*<sup>1</sup> *xy* exhibits the highest image quality, and *F*<sup>3</sup> *xy* the worst, the MSE metric *J* indicates the reverse.

In order to determine an improved optimization metric, it is necessary to derive the properties of noise in holographic replay and, in particular, the noise resulting from the approximation of the complex Fourier Transform information by a phase only hologram.

### **3.2 General properties of holographic replay**

Without loss of generality, we consider the one-dimensional image *Fx*, which is the discrete Fourier transform (DFT) of the corresponding *P*-pixel hologram *hu*, and is termed the replay field (RPF):

$$F\_X \stackrel{\Delta}{=} \sum\_{\mu=0}^{P-1} h\_{\mu} \exp\left(-\frac{2j\pi xu}{P}\right) \tag{13}$$

Since the form of *hu* is not known a priori, because it is the result of some unspecified calculation, it can only be assumed that *hu* is a random variable with some, as yet unknown, distribution. The quest to determine the properties of holographic replay therefore begins by considering the properties of samples of the DFT of a random sequence *hu*, proceeding

algorithm used to generate the hologram - will always be Rayleigh distributed and dependent

Computer-Generated Phase-Only Holograms for Real-Time Image Display 285

It follows that a holographically generated image will consist of a desired signal component of average value *V* plus additive noise *Exy* due to the hologram quantization, and therefore

> *V*, *σ*<sup>2</sup> *n*

 0, *σ*<sup>2</sup> *n*

 exp

Equation 20 is the crucial result for deriving an improved hologram generation algorithm, because it describes the statistical properties of the images produced by any holographic display. Surprisingly, equation 20 shows that holographically-generated images can be

create the hologram. By appropriate manipulation of these parameters, therefore, it is possible

Although equation 20 characterizes the statistics of holographic replay with just two parameters, the relationship between the choice of values for each parameter and the resultant perceived image quality is not clear. Since it is not obvious what values a human viewing an

proceed is to characterize the perceptual degradation of image quality with respect to these parameters by performing a suitably-designed psychometric test on a representative sample

The general question of the comparative perceptual importance of artifacts in images is too broad to consider in this chapter. Instead, we deal with the more tractable problem of the relative perceptual significance of noise (that is, the deviation of the RPF from the target) that is inevitably present in any holographic reproduction, and how the statistical parameters of

The psychometric test was designed to present the subject with 300 sequential stimuli, examples of which are shown in Figure 4. Each stimuli comprises a pair of images, which have each been generated from a set of basis images, and the images are presented and random positions with random intensities. To simulate the effect of holographic replay, intensity noise

A subject was placed in front of a monitor screen displaying such stimuli, which in combination are termed the 'veridical field'. To give the impression of a video image, the stimuli were updated 20 times per second. The subject was then asked to record their subjective interpretation of the most pleasing image or, if no distinction was possible, to

*<sup>n</sup>* was added to each image pair, according to equation 19.

= *V*<sup>2</sup> + 2*σ*<sup>2</sup>

<sup>−</sup> *<sup>x</sup>*<sup>2</sup> <sup>+</sup> *<sup>V</sup>*<sup>2</sup> 2*σ*<sup>2</sup> *n*

(18)

*<sup>n</sup>* (20)

*<sup>n</sup>*, regardless of the algorithm used to

*<sup>n</sup>*, the only logical way to

(19)

�[*Fxy*] ∼ N

�[*Fxy*] ∼ N

so that the magnitude of the image |*Fxy*| is Ricean distributed and described by

*σ*2 *n Io xV σ*2 *n*

E *Fxy* 2 

*n*.

*<sup>f</sup>*|F|(*x*) = *<sup>x</sup>*

completely characterized by just two parameters, *V* and *σ*<sup>2</sup>

**3.2.2 Perceptual significance of noise in holographic replay**

image with Ricean distributed pixel values would assign to *V* and *σ*<sup>2</sup>

to control the noise properties of a holographic display.

of the population as shown in Cable et al. (2004).

the noise affect perception.

<sup>2</sup> with mean *μ* and variance *σ*<sup>2</sup>


the samples of the total complex image amplitude *Fxy* are distributed as

only upon the noise variance *σ*<sup>2</sup>

with energy

to determine the properties of the absolute value of the signal and noise components of the image as would be detected by the eye.

Consider a *P* × 1 vector of independent identically distributed (i.i.d) random variables *h*1, *h*2,..., *hP*, each of which has the same arbitrary probability distribution function (PDF) *fhi* (*u*), *i* = 1 ··· *P*. The central limit theorem (CLT) states that the sum of these i.i.d random variables will tend to the Normal distribution which, remarkably, holds true even if the random variables are not themselves Normally distributed, provided that the sample size *P* is large enough.

Since equation 13 shows that the DFT of *hu* is merely a weighted average of *hu*, with the weights being complex exponential factors, the samples resulting from the DFT operation will therefore be governed by the CLT. Hence, regardless of the distribution of the samples *hu*, the real and imaginary parts of the DFT will be Normally distributed provided that *P* is large enough. This is an important result in determining the properties of noise occurring in holographic replay.

We consider further a *P* × *Q* set of complex random samples *huv* which can be written as

$$h\_{\rm u\overline{\nu}} = \Re[h\_{\rm u\overline{\nu}}] + j \,\odot[h\_{\rm u\overline{\nu}}] \tag{14}$$

where the real and imaginary parts of *huv* have mean and variance (*μr*, *σ*<sup>2</sup> *<sup>r</sup>* ) and (*μi*, *σ*<sup>2</sup> *i* ) respectively. The DFT of these samples, obtained from equation 10, is therefore Normally-distributed in real and imaginary parts

$$F\_{xy} = \Re[F\_{xy}] + j \circledast[F\_{xy}] \tag{15}$$

and, following some lengthy calculations, the samples of the DFT are found to be distributed as

$$\begin{split} \mathfrak{R}[F\_{\rm xy}(0)] &\sim \mathcal{N}[\mu\_{I}P\_{\prime}\sigma\_{r}^{2}PQ] \\ \mathfrak{G}[F\_{\rm xy}(0)] &\sim \mathcal{N}[\mu\_{I}P\_{\prime}\sigma\_{I}^{2}PQ] \\ \mathfrak{R}[F\_{\rm xy}] &\sim \mathcal{N}[0\_{\prime}(\sigma\_{r}^{2}+\sigma\_{i}^{2})PQ/2] \\ \mathfrak{G}[F\_{\rm xy}] &\sim \mathcal{N}[0\_{\prime}(\sigma\_{r}^{2}+\sigma\_{i}^{2})PQ/2]. \end{split} \tag{16}$$

where *Fxy* ∼ N[·] indicates that the samples *Fxy* are Normally distributed.

#### **3.2.1 Effect of hologram quantization upon the image**

In order to determine the properties of noise in holographic replay, it is necessary to determine the effects of quantizing the hologram *hu*. Let the samples *eu* represent the error introduced into the hologram by quantization, and *Ex* = F [*eu*] be the resultant noise introduced into the image. It is clear from equations 16 that, regardless of the PDF of the error in the samples *eu*, the image error samples *Ex* are always Normally distributed in real and imaginary parts and, hence, the amplitude of this error is Rayleigh distributed and is given by

$$f\_{|\mathcal{E}|}(\mathbf{x}) = \frac{\mathbf{x}}{\sigma\_n^2} \exp\left(-\frac{\mathbf{x}^2}{2\sigma\_n^2}\right) \tag{17}$$

where *σ*<sup>2</sup> *<sup>n</sup>* = *σ*2 *<sup>r</sup>* + *σ*<sup>2</sup> *i PQ*/2 and depends upon the nature of the quantization performed. It follows that the noise amplitude in any holographically-formed image - regardless of the algorithm used to generate the hologram - will always be Rayleigh distributed and dependent only upon the noise variance *σ*<sup>2</sup> *n*.

It follows that a holographically generated image will consist of a desired signal component of average value *V* plus additive noise *Exy* due to the hologram quantization, and therefore the samples of the total complex image amplitude *Fxy* are distributed as

$$\begin{aligned} \Re[F\_{\rm xy}] &\sim \mathbf{N}\left[V\_{\prime}\sigma\_{\eta}^{2}\right] \\ \Re[F\_{\rm xy}] &\sim \mathbf{N}\left[0, \sigma\_{\eta}^{2}\right] \end{aligned} \tag{18}$$

so that the magnitude of the image |*Fxy*| is Ricean distributed and described by

$$f\_{|\mathbf{F}|}(\mathbf{x}) = \frac{\mathbf{x}}{\sigma\_n^2} I\_o \left(\frac{\mathbf{x}V}{\sigma\_n^2}\right) \exp\left(-\frac{\mathbf{x}^2 + V^2}{2\sigma\_n^2}\right) \tag{19}$$

with energy

8 Will-be-set-by-IN-TECH

to determine the properties of the absolute value of the signal and noise components of the

Consider a *P* × 1 vector of independent identically distributed (i.i.d) random variables *h*1, *h*2,..., *hP*, each of which has the same arbitrary probability distribution function (PDF)

Since equation 13 shows that the DFT of *hu* is merely a weighted average of *hu*, with the weights being complex exponential factors, the samples resulting from the DFT operation will therefore be governed by the CLT. Hence, regardless of the distribution of the samples *hu*, the real and imaginary parts of the DFT will be Normally distributed provided that *P* is large enough. This is an important result in determining the properties of noise occurring in

We consider further a *P* × *Q* set of complex random samples *huv* which can be written as

respectively. The DFT of these samples, obtained from equation 10, is therefore

and, following some lengthy calculations, the samples of the DFT are found to be distributed

In order to determine the properties of noise in holographic replay, it is necessary to determine the effects of quantizing the hologram *hu*. Let the samples *eu* represent the error introduced into the hologram by quantization, and *Ex* = F [*eu*] be the resultant noise introduced into the image. It is clear from equations 16 that, regardless of the PDF of the error in the samples *eu*, the image error samples *Ex* are always Normally distributed in real and imaginary parts and,

*<sup>r</sup> PQ*]

*<sup>i</sup> PQ*]

*<sup>i</sup>* )*PQ*/2]

*<sup>i</sup>* )*PQ*/2].

*PQ*/2 and depends upon the nature of the quantization performed.

*<sup>r</sup>* + *<sup>σ</sup>*<sup>2</sup>

*<sup>r</sup>* + *<sup>σ</sup>*<sup>2</sup>

where the real and imaginary parts of *huv* have mean and variance (*μr*, *σ*<sup>2</sup>

�[*Fxy*(0)] <sup>∼</sup> <sup>N</sup>[*μrP*, *<sup>σ</sup>*<sup>2</sup>

�[*Fxy*(0)] <sup>∼</sup> <sup>N</sup>[*μiP*, *<sup>σ</sup>*<sup>2</sup>

�[*Fxy*] <sup>∼</sup> <sup>N</sup>[0,(*σ*<sup>2</sup>

�[*Fxy*] <sup>∼</sup> <sup>N</sup>[0,(*σ*<sup>2</sup>

where *Fxy* ∼ N[·] indicates that the samples *Fxy* are Normally distributed.

hence, the amplitude of this error is Rayleigh distributed and is given by

*<sup>f</sup>*|E|(*x*) = *<sup>x</sup>*

*σ*2 *n* exp − *x*2 2*σ*<sup>2</sup> *n* 

It follows that the noise amplitude in any holographically-formed image - regardless of the

Normally-distributed in real and imaginary parts

**3.2.1 Effect of hologram quantization upon the image**

*huv* = �[*huv*] + *j* �[*huv*] (14)

*Fxy* = �[*Fxy*] + *j* �[*Fxy*] (15)

*<sup>r</sup>* ) and (*μi*, *σ*<sup>2</sup>

*i* )

(16)

(17)

(*u*), *i* = 1 ··· *P*. The central limit theorem (CLT) states that the sum of these i.i.d random variables will tend to the Normal distribution which, remarkably, holds true even if the random variables are not themselves Normally distributed, provided that the sample size

image as would be detected by the eye.

*fhi*

as

where *σ*<sup>2</sup>

*<sup>n</sup>* = *σ*2 *<sup>r</sup>* + *σ*<sup>2</sup> *i* 

*P* is large enough.

holographic replay.

$$\mathbb{E}\left[\left|F\_{xy}\right|^2\right] = V^2 + 2\sigma\_n^2 \tag{20}$$

Equation 20 is the crucial result for deriving an improved hologram generation algorithm, because it describes the statistical properties of the images produced by any holographic display. Surprisingly, equation 20 shows that holographically-generated images can be completely characterized by just two parameters, *V* and *σ*<sup>2</sup> *<sup>n</sup>*, regardless of the algorithm used to create the hologram. By appropriate manipulation of these parameters, therefore, it is possible to control the noise properties of a holographic display.

### **3.2.2 Perceptual significance of noise in holographic replay**

Although equation 20 characterizes the statistics of holographic replay with just two parameters, the relationship between the choice of values for each parameter and the resultant perceived image quality is not clear. Since it is not obvious what values a human viewing an image with Ricean distributed pixel values would assign to *V* and *σ*<sup>2</sup> *<sup>n</sup>*, the only logical way to proceed is to characterize the perceptual degradation of image quality with respect to these parameters by performing a suitably-designed psychometric test on a representative sample of the population as shown in Cable et al. (2004).

The general question of the comparative perceptual importance of artifacts in images is too broad to consider in this chapter. Instead, we deal with the more tractable problem of the relative perceptual significance of noise (that is, the deviation of the RPF from the target) that is inevitably present in any holographic reproduction, and how the statistical parameters of the noise affect perception.

The psychometric test was designed to present the subject with 300 sequential stimuli, examples of which are shown in Figure 4. Each stimuli comprises a pair of images, which have each been generated from a set of basis images, and the images are presented and random positions with random intensities. To simulate the effect of holographic replay, intensity noise |*Exy*| <sup>2</sup> with mean *μ* and variance *σ*<sup>2</sup> *<sup>n</sup>* was added to each image pair, according to equation 19.

A subject was placed in front of a monitor screen displaying such stimuli, which in combination are termed the 'veridical field'. To give the impression of a video image, the stimuli were updated 20 times per second. The subject was then asked to record their subjective interpretation of the most pleasing image or, if no distinction was possible, to

experiment suggests that a hologram generation algorithm which employs an error metric

Computer-Generated Phase-Only Holograms for Real-Time Image Display 287

as far higher in quality than the equivalent RPF obtained from other metrics, such as MSE

The conclusion that noise variance is an improved determinant of the perceptual significance of noise in a video image suggests a method for perceptual reduction of noise by exploiting temporal averaging. Consider a holographic display which generates *N* video subframes which are the result of some, as yet unspecified, hologram generation algorithm. The intensity

> *N* ∑ *i*=1

Var[*Vxy*] = *<sup>σ</sup>*<sup>2</sup>

reduction in the noise variance of a video frame can be achieved by displaying the average of *N* noisy subframes. This property precisely fulfils the requirements suggested by the

A simple method for the creation of the time averaged percept of equation 21 relies upon the properties of the human visual system. The eye, because it responds to intensity, is a square-law detector and due to its composition has a finite response time. Kelly & van Norren (1977) performed a series of experiments using flickering veridical fields to deduce the temporal frequency characteristics of the eye, which resulted in the frequency response curves of Figure 6. Since the rod and cone structures respond slightly differently to flicker, there are disparities between pure luminous and chromatic (red-green) flicker responses - nevertheless, the frequency response of the human eye can be well approximated by a brick-wall filter

Using this approximation and accounting for the square-law response, the time-averaged

If a suitable microdisplay is used to show *N* subframes within this 40 ms period, then the integral of equation 23 becomes the summation of equation 21. Hence, by displaying *N* frames quickly enough to exploit the limited temporal bandwidth of the eye, a human subject will


intensity percept *Vxy* is approximately equal to the integral of the veridical field

 *t t*−0.04

*Vxy* =


minimization, which attempts to minimize noise energy *μ*<sup>2</sup> + *σ*<sup>2</sup>

 *F*(*i*) *xy* 2

the average of all such subframes is displayed, the time-averaged percept is

which is *N* times smaller than the variance of each individual subframe

function with a temporal bandwidth of approximately 25 Hz.

a 40 ms time window, and can be expressed as

*Vxy* <sup>=</sup> <sup>1</sup> *N*

and, from the CLT, it follows that the variance of this time-averaged field is given by

*<sup>n</sup>* is likely to produce RPFs that are subjectively regarded

*n*.

, and has mean *<sup>μ</sup>* and variance *<sup>σ</sup>*<sup>2</sup> and *<sup>i</sup>* <sup>=</sup> 1, ··· , *<sup>N</sup>*. If

<sup>2</sup> (21)

*<sup>N</sup>* (22)

*Fxy* 2

> *Fxy* <sup>2</sup> within

<sup>2</sup>*dτ* (23)

. Hence, a

that minimizes noise variance *σ*<sup>2</sup>

**3.3 Reduction of noise variance**

*th* displayed image is *I* =

analytical and psychometric test results.

**3.4 Practical implementation**

of the *i*

record no preference. This is known as the three-alternative forced choice (3AFC) paradigm, described in Greene & d'Oliveira (1999). To ensure that the subjective choice of image quality was made instinctively, as it would be for a typical video stream, a time limit of four seconds per image was imposed and, if the response time of the subject was longer, the result was discarded.

Fig. 4. Sample psychometric test stimuli. Each of the left and right images contains additive noise of mean *μ* and variance *σ*<sup>2</sup> *<sup>n</sup>*, updated 20 times per second as per a video image.

The results were analyzed by constructing a scatter plot of Figure 4 indicating, for each sample, the user's preference and demarcating the scatter plot into regions where the subject considers the left image to be superior ("left preferred"), reverse ("right preferred") or has no preference ("cannot tell"). Boundaries of best fit between these three regions were then constructed using a linear least-squares measure.

Fig. 5. Psychometric test results showing mean difference and variance difference between left and right stimuli, and the associated viewer preference. Key: ■ - Left preferred, ◆ - Right preferred, ▲ - No preference.

The results contained in Figures 5(a) and 5(b) clearly show, as indicated by the dominant horizontal component in the boundary lines, that noise variance in holographic replay is far more significant than the mean as a determinant of the perceptual significance of noise. This experiment suggests that a hologram generation algorithm which employs an error metric that minimizes noise variance *σ*<sup>2</sup> *<sup>n</sup>* is likely to produce RPFs that are subjectively regarded as far higher in quality than the equivalent RPF obtained from other metrics, such as MSE minimization, which attempts to minimize noise energy *μ*<sup>2</sup> + *σ*<sup>2</sup> *n*.

### **3.3 Reduction of noise variance**

10 Will-be-set-by-IN-TECH

record no preference. This is known as the three-alternative forced choice (3AFC) paradigm, described in Greene & d'Oliveira (1999). To ensure that the subjective choice of image quality was made instinctively, as it would be for a typical video stream, a time limit of four seconds per image was imposed and, if the response time of the subject was longer, the result was

Fig. 4. Sample psychometric test stimuli. Each of the left and right images contains additive

The results were analyzed by constructing a scatter plot of Figure 4 indicating, for each sample, the user's preference and demarcating the scatter plot into regions where the subject considers the left image to be superior ("left preferred"), reverse ("right preferred") or has no preference ("cannot tell"). Boundaries of best fit between these three regions were then constructed using

Fig. 5. Psychometric test results showing mean difference and variance difference between left and right stimuli, and the associated viewer preference. Key: ■ - Left preferred, ◆ - Right

The results contained in Figures 5(a) and 5(b) clearly show, as indicated by the dominant horizontal component in the boundary lines, that noise variance in holographic replay is far more significant than the mean as a determinant of the perceptual significance of noise. This

*<sup>n</sup>*, updated 20 times per second as per a video image.



Right

σ2

σ2

*n*

*n* - left

0

0.05

0.1


Right μ - left μ

(b) Females

discarded.

noise of mean *μ* and variance *σ*<sup>2</sup>

a linear least-squares measure.


preferred, ▲ - No preference.


Right

σ2

σ2

*n*

*n* - left

0

0.05

0.1


Right μ - left μ

(a) Males

The conclusion that noise variance is an improved determinant of the perceptual significance of noise in a video image suggests a method for perceptual reduction of noise by exploiting temporal averaging. Consider a holographic display which generates *N* video subframes which are the result of some, as yet unspecified, hologram generation algorithm. The intensity of the *i th* displayed image is *I* = *F*(*i*) *xy* 2 , and has mean *<sup>μ</sup>* and variance *<sup>σ</sup>*<sup>2</sup> and *<sup>i</sup>* <sup>=</sup> 1, ··· , *<sup>N</sup>*. If the average of all such subframes is displayed, the time-averaged percept is

$$V\_{xy} = \frac{1}{N} \sum\_{i=1}^{N} |F\_{xy}^{(i)}|^2 \tag{21}$$

and, from the CLT, it follows that the variance of this time-averaged field is given by

$$\text{Var}[V\_{\text{xy}}] = \frac{\sigma^2}{N} \tag{22}$$

which is *N* times smaller than the variance of each individual subframe *Fxy* 2 . Hence, a reduction in the noise variance of a video frame can be achieved by displaying the average of *N* noisy subframes. This property precisely fulfils the requirements suggested by the analytical and psychometric test results.

### **3.4 Practical implementation**

A simple method for the creation of the time averaged percept of equation 21 relies upon the properties of the human visual system. The eye, because it responds to intensity, is a square-law detector and due to its composition has a finite response time. Kelly & van Norren (1977) performed a series of experiments using flickering veridical fields to deduce the temporal frequency characteristics of the eye, which resulted in the frequency response curves of Figure 6. Since the rod and cone structures respond slightly differently to flicker, there are disparities between pure luminous and chromatic (red-green) flicker responses - nevertheless, the frequency response of the human eye can be well approximated by a brick-wall filter function with a temporal bandwidth of approximately 25 Hz.

Using this approximation and accounting for the square-law response, the time-averaged intensity percept *Vxy* is approximately equal to the integral of the veridical field *Fxy* <sup>2</sup> within a 40 ms time window, and can be expressed as

$$V\_{xy} = \int\_{t-0.04}^{t} |F\_{xy}(\tau)|^2 d\tau \tag{23}$$

If a suitable microdisplay is used to show *N* subframes within this 40 ms period, then the integral of equation 23 becomes the summation of equation 21. Hence, by displaying *N* frames quickly enough to exploit the limited temporal bandwidth of the eye, a human subject will

the results of equation 16 can be applied to show that the real and imaginary parts of the reconstruction error *Exy* - given by the Fourier transform of the quantization error *euv*, F [*euv*] - are independently distributed with zero mean and a variance which depends on the second moment of the reconstruction error only. It follows from equation 17 that the magnitude of the reconstruction error has a Raleigh distribution and that we can ensure that each of the *N*

Computer-Generated Phase-Only Holograms for Real-Time Image Display 289

A simple expression for maximum diffraction efficiency *η* of a phase-only quantized hologram

interval [0, 2*π*]. For binary phase devices with *M* = 2, the maximum achievable diffraction efficiency is just 41%. This figure can be further refined to account for the desired image pattern, per Wyrowski (1991), the computation algorithm - as shown in Mait (1990) - and

It is relatively straightforward to determine the maximum achievable diffraction efficiency when the OSPR algorithm is used to calculate the hologram patterns *huv*. We first consider

reconstructs to form an image *Fxy* with total energy *σ*<sup>2</sup> by equations 16 and Parseval's theorem. A quantization operation is applied to the random variable *muv* to obtain a quantized random

where *q* is the quantization threshold. Restricting the analysis to one dimension for a moment, then the noise *eu* introduced into the hologram by quantization pixel values about a point *q* at

*a muv* < *q*

*<sup>u</sup>* <sup>−</sup> *a u* <sup>&</sup>lt; *<sup>q</sup>*

 ∞ *q*

> ∞ *q*

0, *σ*2/*PQ*

 1 *M* 

*<sup>π</sup><sup>x</sup>* and *M* is the number of phase levels uniformly distributed in the

(24)

resulting from Algorithm 1, which

*e*2 *u* of

<sup>2</sup> ; since *a* = −*b* then

(28)

(27)

*b muv* <sup>≥</sup> *<sup>q</sup>* (25)

*<sup>u</sup>* <sup>−</sup> *b u* <sup>≥</sup> *<sup>q</sup>* (26)

(*u* − *a*)*fm*(*u*) *du*

(*<sup>u</sup>* <sup>−</sup> *<sup>a</sup>*)<sup>2</sup> *fm*(*u*) *du*


*η* = sinc<sup>2</sup>

spatial quantization effects as covered by Arrizón & Testorf (1997); Wyrowski (1992).

*huv* =

*eu* =

the mean hologram quantization noise E [*eu*] is minimized for *q* = *<sup>a</sup>*+*<sup>b</sup>*

which results in a mean quantization noise E [*eu*] and quantization noise energy E

(*u* − *b*)*fm*(*u*) *du* +

(*<sup>u</sup>* <sup>−</sup> *<sup>b</sup>*)<sup>2</sup> *fm*(*u*) *du* <sup>+</sup>

F[*muv*] − F[*huv*]

where *fm*(*u*) is the PDF of the random variable *mu*. In the case of binary phase holography,

*q* −→ 0 as and E [*eu*] � 0 previously shown. *Exy* = F[*euv*] then represents an upper bound

 <sup>≥</sup>

holograms generated will exhibit i.i.d. noise in its RPF if each *ϕxy* in step 1 is i.i.d.

**3.4.2 Diffraction efficiency**

where sinc(*x*) *sin*(*πx*)

variable *huv* such that

is provided by Goodman & Silvestri (1970) and is

an unquantized hologram pattern *muv* <sup>∼</sup> <sup>N</sup>

reconstruction points *a*, *b* can be calculated using

E [*eu*] =

E *e* 2 *u* = *q* −∞

 *q* −∞

for the RPF noise resulting from hologram quantization, since

*Exy* <sup>=</sup> |F [*euv*]<sup>|</sup> <sup>=</sup>

Fig. 6. Temporal frequency response of the eye to luminous and chromatic flicker (after Kelly & van Norren (1977)). The curves show that the eye can be modeled as a brick-wall filter function with temporal bandwidth of approximately 25 Hz.

perceive an image which is the average of *N* noisy subframes which is, from equation 22, substantially noise-free.

### **3.4.1 The one-step phase retrieval (OSPR) algorithm**

What remains is to design a hologram-generation algorithm that has the capability to generate *N* sets of holograms both efficiently and in real time. A simple and computationally-simple method for generating *i* = 1, ··· , *N* holograms, each of which gives rise to a reconstruction *F*(*i*) *xy* 2 containing the desired image *Txy* in addition to i.i.d noise *E*(*i*) *xy* 2 , is provided in the OSPR Algorithm 1 below.

**inputs** : *P* × *Q* pixel target image *Txy*, *N*

**output**: *N* binary phase holograms *h* (*n*) *uv* , *<sup>n</sup>* <sup>=</sup> 1, ··· , *<sup>N</sup>*, of size *<sup>P</sup>* <sup>×</sup> *<sup>Q</sup>* pixels

$$\begin{array}{l|l} \text{for } n \leftarrow 1 \text{ to } N/2 \text{ do} \\ \hline \end{array} \quad \begin{array}{l} \text{for } T\_{xy}^{(n)} = \sqrt{I\_{xy}} \exp j \rho\_{xy}^{(n)} \text{ where } \rho\_{xy}^{(n)} \text{ is uniformly distributed in the interval } [0, 2\pi] \\ \text{Let } g\_{uv}^{(n)} = \mathcal{F}^{-1} \left[ T\_{xy}^{(n)} \right] \\ \text{Let } m\_{uv}^{(n)} = \mathfrak{R} \left\{ g\_{uv}^{(n)} \right\} \text{ where } \mathfrak{R} \{ \cdot \} \text{ represents the real part} \\ \quad \text{Let } m\_{uv}^{(n + N/2)} = \bigotimes \left\{ g\_{uv}^{(n)} \right\} \text{ where } \mathfrak{I} \{ \cdot \} \text{ represents the imaginary part} \\ \quad \text{Let } h\_{uv}^{(n)} = \begin{cases} -1 \text{ if } m\_{uv}^{(n)} < 0 \\ 1 \text{ if } m\_{uv}^{(n)} \ge 0 \end{cases} \end{array}$$

**end**

**Algorithm 1:** The OSPR algorithm for calculating *N P* × *Q* pixel binary phase holograms *h* (*n*) *uv* , *<sup>n</sup>* <sup>=</sup> 1, ··· , *<sup>N</sup>* from a *<sup>P</sup>* <sup>×</sup> *<sup>Q</sup>* pixel target image *Txy*.

The results of the previous sections allow us to verify that Algorithm 1 generates holograms with the correct properties. Provided that mean quantization error (introduced into the hologram in the last step of Algorithm 1 is zero, which follows from thresholding about zero, the results of equation 16 can be applied to show that the real and imaginary parts of the reconstruction error *Exy* - given by the Fourier transform of the quantization error *euv*, F [*euv*] - are independently distributed with zero mean and a variance which depends on the second moment of the reconstruction error only. It follows from equation 17 that the magnitude of the reconstruction error has a Raleigh distribution and that we can ensure that each of the *N* holograms generated will exhibit i.i.d. noise in its RPF if each *ϕxy* in step 1 is i.i.d.

#### **3.4.2 Diffraction efficiency**

12 Will-be-set-by-IN-TECH

Fig. 6. Temporal frequency response of the eye to luminous and chromatic flicker (after Kelly & van Norren (1977)). The curves show that the eye can be modeled as a brick-wall filter

perceive an image which is the average of *N* noisy subframes which is, from equation 22,

What remains is to design a hologram-generation algorithm that has the capability to generate *N* sets of holograms both efficiently and in real time. A simple and computationally-simple method for generating *i* = 1, ··· , *N* holograms, each of which gives rise to a reconstruction

where �{·} represents the real part

**Algorithm 1:** The OSPR algorithm for calculating *N P* × *Q* pixel binary phase holograms

The results of the previous sections allow us to verify that Algorithm 1 generates holograms with the correct properties. Provided that mean quantization error (introduced into the hologram in the last step of Algorithm 1 is zero, which follows from thresholding about zero,

Chromatic flicker Luminous flicker Brick-wall response

> *E*(*i*) *xy* 2

*xy* is uniformly distributed in the interval [0; 2*π*]

(*n*) *uv* , *<sup>n</sup>* <sup>=</sup> 1, ··· , *<sup>N</sup>*, of size *<sup>P</sup>* <sup>×</sup> *<sup>Q</sup>* pixels

where �{·} represents the imaginary part

, is provided in the

0.1 1 10 100 Frequency (Hz)

containing the desired image *Txy* in addition to i.i.d noise

*xy* where *<sup>ϕ</sup>*(*n*)

function with temporal bandwidth of approximately 25 Hz.

**3.4.1 The one-step phase retrieval (OSPR) algorithm**

**inputs** : *P* × *Q* pixel target image *Txy*, *N* **output**: *N* binary phase holograms *h*

*xy* <sup>=</sup> *Ixy* exp *<sup>j</sup>ϕ*(*n*)

 *g* (*n*) *uv* 

 *T*(*n*) *xy* 

> *g* (*n*) *uv*

<sup>−</sup>1 if *<sup>m</sup>*(*n*) *uv* <sup>&</sup>lt; <sup>0</sup> 1 if *<sup>m</sup>*(*n*) *uv* <sup>≥</sup> <sup>0</sup>

(*n*) *uv* , *<sup>n</sup>* <sup>=</sup> 1, ··· , *<sup>N</sup>* from a *<sup>P</sup>* <sup>×</sup> *<sup>Q</sup>* pixel target image *Txy*.

0.01

0.1

Normalised response

substantially noise-free.

OSPR Algorithm 1 below.

**for** *n* ← 1 **to** *N*/2 **do** Let *T*(*n*)

Let *<sup>m</sup>*(*n*) *uv* <sup>=</sup> �

(*n*) *uv* <sup>=</sup> <sup>F</sup>−<sup>1</sup>

Let *<sup>m</sup>*(*n*+*N*/2) *uv* <sup>=</sup> �

(*n*) *uv* <sup>=</sup>

Let *g*

Let *h*

**end**

*h*

 *F*(*i*) *xy* 2 1

A simple expression for maximum diffraction efficiency *η* of a phase-only quantized hologram is provided by Goodman & Silvestri (1970) and is

$$\eta = \text{sinc}^2\left(\frac{1}{M}\right) \tag{24}$$

where sinc(*x*) *sin*(*πx*) *<sup>π</sup><sup>x</sup>* and *M* is the number of phase levels uniformly distributed in the interval [0, 2*π*]. For binary phase devices with *M* = 2, the maximum achievable diffraction efficiency is just 41%. This figure can be further refined to account for the desired image pattern, per Wyrowski (1991), the computation algorithm - as shown in Mait (1990) - and spatial quantization effects as covered by Arrizón & Testorf (1997); Wyrowski (1992).

It is relatively straightforward to determine the maximum achievable diffraction efficiency when the OSPR algorithm is used to calculate the hologram patterns *huv*. We first consider an unquantized hologram pattern *muv* <sup>∼</sup> <sup>N</sup> 0, *σ*2/*PQ* resulting from Algorithm 1, which reconstructs to form an image *Fxy* with total energy *σ*<sup>2</sup> by equations 16 and Parseval's theorem. A quantization operation is applied to the random variable *muv* to obtain a quantized random variable *huv* such that

$$h\_{\text{uv}} = \begin{cases} a \ m\_{\text{uv}} < q \\ b \ m\_{\text{uv}} \ge q \end{cases} \tag{25}$$

where *q* is the quantization threshold. Restricting the analysis to one dimension for a moment, then the noise *eu* introduced into the hologram by quantization pixel values about a point *q* at reconstruction points *a*, *b* can be calculated using

$$e\_{ll} = \begin{cases} u - a \ u < q \\ u - b \ u \ge q \end{cases} \tag{26}$$

which results in a mean quantization noise E [*eu*] and quantization noise energy E *e*2 *u* of

$$\begin{aligned} \mathrm{E}\left[e\_{\mathrm{u}}\right] &= \int\_{-\infty}^{q} (u-b) f\_{\mathrm{m}}(u) \, du + \int\_{q}^{\infty} (u-a) f\_{\mathrm{m}}(u) \, du \\ \mathrm{E}\left[e\_{\mathrm{u}}^{2}\right] &= \int\_{-\infty}^{q} (u-b)^{2} f\_{\mathrm{m}}(u) \, du + \int\_{q}^{\infty} (u-a)^{2} f\_{\mathrm{m}}(u) \, du \end{aligned} \tag{27}$$

where *fm*(*u*) is the PDF of the random variable *mu*. In the case of binary phase holography, the mean hologram quantization noise E [*eu*] is minimized for *q* = *<sup>a</sup>*+*<sup>b</sup>* <sup>2</sup> ; since *a* = −*b* then *q* −→ 0 as and E [*eu*] � 0 previously shown. *Exy* = F[*euv*] then represents an upper bound for the RPF noise resulting from hologram quantization, since

$$E\_{xy} = |\mathcal{F}\left[\mathcal{e}\_{\rm uv}\right]| = \left|\mathcal{F}[m\_{\rm uv}] - \mathcal{F}[h\_{\rm uv}]\right| \ge \left||\mathcal{F}[m\_{\rm uv}]| - |\mathcal{F}[h\_{\rm uv}]|\right|\tag{28}$$

where Γ(·) is the complete Gamma function. Since the mean of the Gamma distribution is *Nσ<sup>n</sup>* then it follows from equation 21 that the mean noise energy in a veridical field *Vxy* composed

Computer-Generated Phase-Only Holograms for Real-Time Image Display 291

The noise energy present in the field *Vxy*, and hence the SNR, is clearly independent of the number of subframes *N* and, as for the diffraction efficiency, is defined by the number of

If we further define a fractional coverage value *η* to be the ratio of the sum of the normalized

*Txy*

*<sup>S</sup>* � 1.75

Since typical video images exhibit *η* = 0.24, giving *S* � 7 independent of the number of subframes *N*, this immediately highlights an obvious limitation of binary phase holographic video projection. There are several algorithmic methods capable of improving the contrast ratio of a holographically-generated image, each of which depend upon quantizing the hologram in such a way that noise can be selectively placed in the RPF. The Gerchberg-Saxton (GS) and Direct Binary Search (DBS) hologram generation algorithms can both be modified so that each algorithm attempts to minimize the quantization noise energy within a predefined signal window within the RPF, thereby obtaining a local signal-to-noise ratio (SNR) improvement as Brauer et al. (1991); Meister & Winfield (2002); Wyrowksi et al. (1986); Wyrowski & Bryngdahl (1988) previously found. However, both algorithms generate RPFs of insufficient quality and impose computational burdens that are incompatible with a

The error diffusion (ED) algorithm, whilst not capable of generating holograms, was shown to be able to quantize holograms to generate RPFs with this useful characteristic by Kirk et al. (1992). As demonstrated in Buckley (2011b), it is possible to employ a multiple subframe approach, using OSPR to calculate holograms which are subsequently binarized using ED, to combine the benefits of image uniformity and high contrast. By implementing a parallel-processor design, the ED algorithm can be realized at the rate required by a

The requirements imposed upon the microdisplay used in the holographic projection system described previously are very different to those for the equivalent imaging system in terms of the liquid crystal material, backplane circuitry and pixel geometry. For a microdisplay employed in an imaging system, the choice of pixel size is usually chosen to represent a compromise between maintaining an adequate aperture ratio whilst minimizing diffractive effects - in a projection system which exploits diffraction, however, such a restriction does

= *σ*<sup>2</sup>

 <sup>=</sup> *<sup>V</sup>*2/*<sup>η</sup> σ*2 *n*

*PQ* ≤ *η* ≤ 1, then, since the quantization noise is determined by

*<sup>n</sup>* (34)

*<sup>n</sup>*, then equation 30 can be used to show that

*<sup>η</sup>* (36)

(35)

E *Vxy*

hologram phase levels and choice of hologram computation algorithm.

the number of phase levels and is constant, then the SNR *S* can be defined as

*<sup>S</sup>* <sup>E</sup>

E *Exy* 2

of *N* frames is

pixel values to *PQ*, so that <sup>1</sup>

and, since the total RPF energy *σ*<sup>2</sup> = *V*2/*η* + *σ*<sup>2</sup>

high-quality real-time holographic display.

multiple-subframe holographic projection system.

**3.4.4 Choice of microdisplay**

by the triangle inequality.

The noise in the RPF due to quantization can be determined by evaluation of equations 27 at the appropriate reconstruction points *a* and *b*. For binary phase quantization, the points lie on the centroids of the positive and negative real axes respectively so that

$$a = \int\_{q \longrightarrow 0}^{\infty} u f\_m(u) du = \sigma \sqrt{\frac{2}{\pi}} \tag{29}$$

Using equations 27 it can further be shown that the RPF noise due to binary phase quantization is

$$\mathbb{E}\left[e\_{\mu\upsilon}^2\right] = \frac{\sigma^2}{PQ}\left(1 - \frac{2}{\pi}\right) = \frac{\sigma\_n^2}{PQ} \tag{30}$$

so that the reconstruction *Exy* <sup>∼</sup> <sup>N</sup> � 0, *σ*<sup>2</sup> *n* � from equations 16 and it follows that *σ*<sup>2</sup> *<sup>n</sup>*/*σ*<sup>2</sup> � 36% of the reconstruction energy resulting from a binary phase hologram generated by the OSPR algorithm is noise. The diffraction efficiency *η*, defined as the proportion of usable energy directed into the first-order intensity samples in the presence of a RPF noise energy *σ*<sup>2</sup> *<sup>n</sup>*, is then

$$\eta = \begin{cases} \frac{1}{2} \left( 1 - \frac{\sigma\_u^2}{\sigma^2} \right) & M = 2\\ 1 - \frac{\sigma\_u^2}{\sigma^2} & \text{otherwise} \end{cases} \tag{31}$$

and is approximately 32% for binary phase holograms generated using OSPR. A similar calculation by Buckley & Wilkinson (2007) results in a figure of 88% for OSPR-generated continuous phase holograms.

#### **3.4.3 Signal-to-noise ratio**

Signal-to-noise ratio (SNR) is an important metric for image display applications since it defines the maximum achievable contrast ratio. In a holographically formed two-dimensional image, the SNR is defined as the ratio of the mean signal energy to the mean noise energy, where the RPF, |*Fxy*| 2, contains the desired target image *Txy* with mean value *V*<sup>2</sup> in addition to additive noise |*Exy*| <sup>2</sup> caused by hologram quantization.

If each subframe *i* contains noise components *E*(*i*) *xy* with amplitudes � � � *E*(*i*) *xy* ] � � � that are Rayleigh distributed as per equation 17, it is simple to show that the noise intensity � � � *E*(*i*) *xy* � � � 2 is distributed as the exponential distribution

$$f\_{|E|^2}(\mathbf{x}) = \frac{1}{\sigma\_n^2} \exp\left(-\frac{\mathbf{x}}{\sigma\_n^2}\right) \tag{32}$$

In an OSPR-based holographic display system, the overall noise field � �*Exy* � � <sup>2</sup> is the time average of *N* such contributions due to the square-law detection properties of the eye as shown by equation 21. Using a standard result it can be shown that the sum of *N* such independent, identically distributed exponential random variables is distributed according to the Gamma distribution

$$f\_{|E|^2}(\mathbf{x}) = \frac{\mathbf{x}^{n-1}}{\Gamma\left(N\right)\sigma\_n^{2N}} \exp\left(-\frac{\mathbf{x}}{\sigma\_n^2}\right) \tag{33}$$

14 Will-be-set-by-IN-TECH

The noise in the RPF due to quantization can be determined by evaluation of equations 27 at the appropriate reconstruction points *a* and *b*. For binary phase quantization, the points lie on

Using equations 27 it can further be shown that the RPF noise due to binary phase

� <sup>1</sup> <sup>−</sup> <sup>2</sup> *π* � <sup>=</sup> *<sup>σ</sup>*<sup>2</sup> *n*

of the reconstruction energy resulting from a binary phase hologram generated by the OSPR algorithm is noise. The diffraction efficiency *η*, defined as the proportion of usable energy directed into the first-order intensity samples in the presence of a RPF noise energy *σ*<sup>2</sup>

and is approximately 32% for binary phase holograms generated using OSPR. A similar calculation by Buckley & Wilkinson (2007) results in a figure of 88% for OSPR-generated

Signal-to-noise ratio (SNR) is an important metric for image display applications since it defines the maximum achievable contrast ratio. In a holographically formed two-dimensional image, the SNR is defined as the ratio of the mean signal energy to the mean noise energy,

*u fm*(*u*)*du* = *σ*

� 2

from equations 16 and it follows that *σ*<sup>2</sup>

*M* = 2

2, contains the desired target image *Txy* with mean value *V*<sup>2</sup> in addition

*xy* with amplitudes

� � � *E*(*i*) *xy* ] � �

� � � *E*(*i*) *xy* � � � 2

> �*Exy* � �

*<sup>σ</sup>*<sup>2</sup> otherwise

*<sup>π</sup>* (29)

*PQ* (30)

*<sup>n</sup>*/*σ*<sup>2</sup> � 36%

*<sup>n</sup>*, is then

� that are Rayleigh

is distributed

<sup>2</sup> is the time

(32)

(33)

(31)

the centroids of the positive and negative real axes respectively so that

*a* = � ∞ *q*−→0

E � *e* 2 *uv* � <sup>=</sup> *<sup>σ</sup>*<sup>2</sup> *PQ*

*η* =

If each subframe *i* contains noise components *E*(*i*)

⎧ ⎨ ⎩

1 2 � <sup>1</sup> <sup>−</sup> *<sup>σ</sup>*<sup>2</sup> *n σ*2 �

<sup>2</sup> caused by hologram quantization.

*<sup>f</sup>*|*E*|<sup>2</sup> (*x*) = <sup>1</sup>

In an OSPR-based holographic display system, the overall noise field �

*<sup>f</sup>*|*E*|<sup>2</sup> (*x*) = *<sup>x</sup>n*−<sup>1</sup>

*σ*2 *n* exp � − *x σ*2 *n* �

average of *N* such contributions due to the square-law detection properties of the eye as shown by equation 21. Using a standard result it can be shown that the sum of *N* such independent, identically distributed exponential random variables is distributed according

> Γ (*N*) *σ*2*<sup>N</sup> n*

exp � − *x σ*2 *n* �

distributed as per equation 17, it is simple to show that the noise intensity

<sup>1</sup> <sup>−</sup> *<sup>σ</sup>*<sup>2</sup> *n*

0, *σ*<sup>2</sup> *n* �

by the triangle inequality.

so that the reconstruction *Exy* <sup>∼</sup> <sup>N</sup> �

continuous phase holograms.

**3.4.3 Signal-to-noise ratio**

where the RPF, |*Fxy*|

to additive noise |*Exy*|

as the exponential distribution

to the Gamma distribution

quantization is

where Γ(·) is the complete Gamma function. Since the mean of the Gamma distribution is *Nσ<sup>n</sup>* then it follows from equation 21 that the mean noise energy in a veridical field *Vxy* composed of *N* frames is

$$\to \left[V\_{\mathbf{x}\mathbf{y}}\right] = \sigma\_{\mathbf{n}}^2\tag{34}$$

The noise energy present in the field *Vxy*, and hence the SNR, is clearly independent of the number of subframes *N* and, as for the diffraction efficiency, is defined by the number of hologram phase levels and choice of hologram computation algorithm.

If we further define a fractional coverage value *η* to be the ratio of the sum of the normalized pixel values to *PQ*, so that <sup>1</sup> *PQ* ≤ *η* ≤ 1, then, since the quantization noise is determined by the number of phase levels and is constant, then the SNR *S* can be defined as

$$\mathcal{S} \stackrel{\triangle}{=} \frac{\mathbb{E}\left[T\_{\text{xy}}\right]}{\mathbb{E}\left[\left|E\_{\text{xy}}\right|^2\right]} = \frac{V^2/\eta}{\sigma\_n^2} \tag{35}$$

and, since the total RPF energy *σ*<sup>2</sup> = *V*2/*η* + *σ*<sup>2</sup> *<sup>n</sup>*, then equation 30 can be used to show that

$$S \simeq \frac{1.75}{\eta} \tag{36}$$

Since typical video images exhibit *η* = 0.24, giving *S* � 7 independent of the number of subframes *N*, this immediately highlights an obvious limitation of binary phase holographic video projection. There are several algorithmic methods capable of improving the contrast ratio of a holographically-generated image, each of which depend upon quantizing the hologram in such a way that noise can be selectively placed in the RPF. The Gerchberg-Saxton (GS) and Direct Binary Search (DBS) hologram generation algorithms can both be modified so that each algorithm attempts to minimize the quantization noise energy within a predefined signal window within the RPF, thereby obtaining a local signal-to-noise ratio (SNR) improvement as Brauer et al. (1991); Meister & Winfield (2002); Wyrowksi et al. (1986); Wyrowski & Bryngdahl (1988) previously found. However, both algorithms generate RPFs of insufficient quality and impose computational burdens that are incompatible with a high-quality real-time holographic display.

The error diffusion (ED) algorithm, whilst not capable of generating holograms, was shown to be able to quantize holograms to generate RPFs with this useful characteristic by Kirk et al. (1992). As demonstrated in Buckley (2011b), it is possible to employ a multiple subframe approach, using OSPR to calculate holograms which are subsequently binarized using ED, to combine the benefits of image uniformity and high contrast. By implementing a parallel-processor design, the ED algorithm can be realized at the rate required by a multiple-subframe holographic projection system.

### **3.4.4 Choice of microdisplay**

The requirements imposed upon the microdisplay used in the holographic projection system described previously are very different to those for the equivalent imaging system in terms of the liquid crystal material, backplane circuitry and pixel geometry. For a microdisplay employed in an imaging system, the choice of pixel size is usually chosen to represent a compromise between maintaining an adequate aperture ratio whilst minimizing diffractive effects - in a projection system which exploits diffraction, however, such a restriction does

that this requirement can effectively halve the maximum achievable optical efficiency. In a phase-modulating system employing the OSPR algorithm, however, since the holograms can be chosen to be automatically DC balanced and because the hologram and its inverse both result in the same image, the device can be illuminated during both the valid and

Computer-Generated Phase-Only Holograms for Real-Time Image Display 293

Figure 7 shows the simplest optical architecture for a holographic projector. The lens pair of *L*<sup>1</sup> and *L*<sup>2</sup> form a telescope, which expands the laser beam to capture the entire hologram pattern so that low-pass filtering of the resultant intensity RPF *Vxy* does not result. The reverse arrangement is used for the lens pair of *L*<sup>3</sup> and *L*4, which acts to demagnify the hologram pixels and consequently increase the diffraction angle Δ as described by Buckley et al. (2006). The demagnification *D* is set by the ratio of focal lengths *f*<sup>3</sup> to *f*<sup>4</sup> and, due to the properties of

Hologram

Fig. 7. Optical design for a simple holographic projector. Beam-expansion of the laser diode

The realization of a color holographic projector is relatively straightforward. A desired image is converted into sets of holograms and displayed on a phase modulating microdisplay illuminated by red, green and blue coherent light. Color images can be formed either by spatially segmenting the microdisplay as per Ito & Okano (2004), designing multi-focal CGHs by the method of Makowski et al. (2008), or by employing the frame-sequential color of Buckley (2008a; 2011a), which has the advantage of maximizing the output resolution. In

wavelength *λr*, *λ<sup>g</sup>* and *λb*, with the RPF scaled to account for the wavelength-dependent diffraction angle. The subsequent diffraction patterns pass through the simple lens pair *L*<sup>3</sup> and *L*4, which increases the projection angle by demagnifying the microdisplay pixel size Δ. Since the color planes are displayed and illuminated at the subframe rate, the color-sequential

Figure 8 shows an image obtained from a phase-only holographic projection system, employing the techniques described in this paper, manufactured by Light Blue Optics Ltd. The projector was imaged onto a commercially available rear-projection screen and the resultant image was captured by a digital camera. The nominal resolution at the projection screen was approximately WVGA (850 × 480 pixels.) It is clear that the image exhibits the

*uv* , *i* = 1, ··· , *N*, are calculated and displayed for each

Replay field

*R*

*f*<sup>1</sup> *f*<sup>2</sup> *f*<sup>3</sup> *f*<sup>4</sup>

*L*<sup>1</sup> *L*<sup>2</sup> *L*<sup>3</sup> *L*<sup>4</sup>

compensating fields resulting in the maximum optical efficiency.

Fraunhofer diffraction, the images remain in focus at all distances from *L*4.

is performed by lenses *L*<sup>1</sup> and *L*2, and demagnification by lenses *L*<sup>3</sup> and *L*4.

(*i*)

**3.4.5 Optical system**

Laser wavelength

**3.4.6 Color architecture**

the latter case, sets of holograms *h*

approach does not suffer from color breakup.

not apply. It is a standard result, given in Hecht (1998), that the diffraction angle *θ* from a hologram pattern of pixel size Δ placed behind a lens and illuminated with coherent collimated light of wavelength *λ*, is given by

$$\theta = \arctan\left(\frac{\lambda}{2\Delta}\right) \tag{37}$$

This inverse relationship between diffraction angle and feature size suggests that the pixel size in a microdisplay employed in a holographic projection system should be as small as possible, so that subsequent lens power to achieve the desired projection angle is minimized.

It is also of paramount importance to provide predictable phase modulation over a wide temperate range and, because multiple subframes are displayed per video frame for the purposes of noise reduction, a high frame rate is required. These requirements can be fulfilled by the use of a ferroelectric Liquid Crystal on Silicon (LCOS) device operating as a phase-only modulator, as shown by O'Brien et al. (1994).

In phase modulating mode, a ferroelectric LCOS device with a cell gap providing optical retardation Γ can act as a pixellated binary phase hologram in which each of the pixels can independently impose a phase shift of either 0 or *π* radians. To achieve phase modulation, the direction of polarization or the incident light (with components *Ex* and *Ey*) is aligned to bisect the switching angle 2*θ* of the two LC states. The resultant modulated light components *E*� *<sup>x</sup>* and *E*� *<sup>y</sup>* in switched and unswitched states can be written in Jones Matrix notation, and are given by

$$
\begin{bmatrix} E\_x' \\ E\_y' \end{bmatrix} = \begin{bmatrix} 1 \ 0 \\ 0 \ 0 \end{bmatrix} \begin{bmatrix} e^{-j\frac{\Gamma}{2}} \cos^2 \theta + e^{j\frac{\Gamma}{2}} \sin^2 \theta & \pm j \sin\frac{\Gamma}{2} \sin 2\theta \\\ \pm j \sin\frac{\Gamma}{2} \sin 2\theta & e^{-j\frac{\Gamma}{2}} \cos^2 \theta + e^{j\frac{\Gamma}{2}} \sin^2 \theta \end{bmatrix} \begin{bmatrix} E\_x \\ E\_y \end{bmatrix} \tag{38}
$$

which reduces to

$$
\begin{bmatrix} E\_x' \\ E\_y' \end{bmatrix} = \begin{bmatrix} \pm j \sin\frac{\Gamma}{2} \sin 2\theta \\ 0 \end{bmatrix} \begin{bmatrix} E\_x \\ E\_y \end{bmatrix} \tag{39}
$$

It follows that the diffraction efficiency of the FLC material is determined by

$$
\eta\_{FLC} = \sin^2(2\theta) \sin^2\left(\frac{\Gamma}{2}\right) \tag{40}
$$

where the optical retardation <sup>Γ</sup> <sup>=</sup> <sup>2</sup>*πd*Δ*<sup>n</sup> <sup>λ</sup>* , with *<sup>d</sup>* the thickness of the LC layer and <sup>Δ</sup>*<sup>n</sup>* its birefringence. It is clear from equations 39 and 40 that in order to maximize the diffraction efficiency then the LC material switching angle must be 2*θ* = *π* radians; the pixels of a microdisplay employing such a material could then independently impose phase shifts of either 0 or *π* radians, giving *ϕuv* ∼ [0, *π*] as required.

It is clear from equation 39 that, given a LC material switching angle of *π* radians, the pixels of such a device can independently impose a phase shift of either 0 or *π* radians, giving *ϕuv* ∼ [0, *π*] as required. Development devices with a switching angle of 88◦ in the smectic C\* phase (SmC\*) at operating temperature have previously been demonstrated by Heggarty et al. (2004) and have been deployed as phase modulators in optical switching applications.

A commonly encountered issue with ferroelectric LC devices is the need to DC balance the device by displaying inverse compensating images, during which time the device cannot be illuminated. When used in an imaging architecture, O'Callaghan et al. (2009) has shown that this requirement can effectively halve the maximum achievable optical efficiency. In a phase-modulating system employing the OSPR algorithm, however, since the holograms can be chosen to be automatically DC balanced and because the hologram and its inverse both result in the same image, the device can be illuminated during both the valid and compensating fields resulting in the maximum optical efficiency.

### **3.4.5 Optical system**

16 Will-be-set-by-IN-TECH

not apply. It is a standard result, given in Hecht (1998), that the diffraction angle *θ* from a hologram pattern of pixel size Δ placed behind a lens and illuminated with coherent

This inverse relationship between diffraction angle and feature size suggests that the pixel size in a microdisplay employed in a holographic projection system should be as small as possible,

It is also of paramount importance to provide predictable phase modulation over a wide temperate range and, because multiple subframes are displayed per video frame for the purposes of noise reduction, a high frame rate is required. These requirements can be fulfilled by the use of a ferroelectric Liquid Crystal on Silicon (LCOS) device operating as a phase-only

In phase modulating mode, a ferroelectric LCOS device with a cell gap providing optical retardation Γ can act as a pixellated binary phase hologram in which each of the pixels can independently impose a phase shift of either 0 or *π* radians. To achieve phase modulation, the direction of polarization or the incident light (with components *Ex* and *Ey*) is aligned to bisect the switching angle 2*θ* of the two LC states. The resultant modulated light components

*<sup>y</sup>* in switched and unswitched states can be written in Jones Matrix notation, and are

<sup>2</sup> sin 2*θ* 0

<sup>2</sup> sin 2*<sup>θ</sup> <sup>e</sup>*−*<sup>j</sup>* <sup>Γ</sup>

<sup>±</sup>*<sup>j</sup>* sin <sup>Γ</sup>

*ηFLC* = sin2(2*θ*) sin2

where the optical retardation <sup>Γ</sup> <sup>=</sup> <sup>2</sup>*πd*Δ*<sup>n</sup> <sup>λ</sup>* , with *<sup>d</sup>* the thickness of the LC layer and <sup>Δ</sup>*<sup>n</sup>* its birefringence. It is clear from equations 39 and 40 that in order to maximize the diffraction efficiency then the LC material switching angle must be 2*θ* = *π* radians; the pixels of a microdisplay employing such a material could then independently impose phase shifts of

It is clear from equation 39 that, given a LC material switching angle of *π* radians, the pixels of such a device can independently impose a phase shift of either 0 or *π* radians, giving *ϕuv* ∼ [0, *π*] as required. Development devices with a switching angle of 88◦ in the smectic C\* phase (SmC\*) at operating temperature have previously been demonstrated by Heggarty et al. (2004)

A commonly encountered issue with ferroelectric LC devices is the need to DC balance the device by displaying inverse compensating images, during which time the device cannot be illuminated. When used in an imaging architecture, O'Callaghan et al. (2009) has shown

and have been deployed as phase modulators in optical switching applications.

<sup>2</sup> sin2 *<sup>θ</sup>* <sup>±</sup>*<sup>j</sup>* sin <sup>Γ</sup>

<sup>2</sup> sin 2*θ*

<sup>2</sup> sin<sup>2</sup> *θ*

 *Ex Ey* 

<sup>2</sup> cos2 *θ* + *e<sup>j</sup>* <sup>Γ</sup>

 *Ex Ey* 

 Γ 2 

 *λ* 2Δ 

(37)

(38)

(39)

(40)

*θ* = arctan

so that subsequent lens power to achieve the desired projection angle is minimized.

<sup>2</sup> cos2 *θ* + *e<sup>j</sup>* <sup>Γ</sup>

It follows that the diffraction efficiency of the FLC material is determined by

<sup>±</sup>*<sup>j</sup>* sin <sup>Γ</sup>

collimated light of wavelength *λ*, is given by

modulator, as shown by O'Brien et al. (1994).

 *e*−*<sup>j</sup>* <sup>Γ</sup>

either 0 or *π* radians, giving *ϕuv* ∼ [0, *π*] as required.

 *E*� *x E*� *y* = 

*E*� *<sup>x</sup>* and *E*�

given by

 *E*� *x E*� *y* = 1 0 0 0

which reduces to

Figure 7 shows the simplest optical architecture for a holographic projector. The lens pair of *L*<sup>1</sup> and *L*<sup>2</sup> form a telescope, which expands the laser beam to capture the entire hologram pattern so that low-pass filtering of the resultant intensity RPF *Vxy* does not result. The reverse arrangement is used for the lens pair of *L*<sup>3</sup> and *L*4, which acts to demagnify the hologram pixels and consequently increase the diffraction angle Δ as described by Buckley et al. (2006). The demagnification *D* is set by the ratio of focal lengths *f*<sup>3</sup> to *f*<sup>4</sup> and, due to the properties of Fraunhofer diffraction, the images remain in focus at all distances from *L*4.

Fig. 7. Optical design for a simple holographic projector. Beam-expansion of the laser diode is performed by lenses *L*<sup>1</sup> and *L*2, and demagnification by lenses *L*<sup>3</sup> and *L*4.

### **3.4.6 Color architecture**

The realization of a color holographic projector is relatively straightforward. A desired image is converted into sets of holograms and displayed on a phase modulating microdisplay illuminated by red, green and blue coherent light. Color images can be formed either by spatially segmenting the microdisplay as per Ito & Okano (2004), designing multi-focal CGHs by the method of Makowski et al. (2008), or by employing the frame-sequential color of Buckley (2008a; 2011a), which has the advantage of maximizing the output resolution. In the latter case, sets of holograms *h* (*i*) *uv* , *i* = 1, ··· , *N*, are calculated and displayed for each wavelength *λr*, *λ<sup>g</sup>* and *λb*, with the RPF scaled to account for the wavelength-dependent diffraction angle. The subsequent diffraction patterns pass through the simple lens pair *L*<sup>3</sup> and *L*4, which increases the projection angle by demagnifying the microdisplay pixel size Δ. Since the color planes are displayed and illuminated at the subframe rate, the color-sequential approach does not suffer from color breakup.

Figure 8 shows an image obtained from a phase-only holographic projection system, employing the techniques described in this paper, manufactured by Light Blue Optics Ltd. The projector was imaged onto a commercially available rear-projection screen and the resultant image was captured by a digital camera. The nominal resolution at the projection screen was approximately WVGA (850 × 480 pixels.) It is clear that the image exhibits the

where

and

*F*(1)

so that the dimensions of the RPF are *<sup>λ</sup><sup>z</sup>*

geometry is illustrated in Figure 9.

found to be distributed as

*<sup>x</sup>* and *Q*� = *Q*Δ<sup>2</sup>

which encodes the equivalent lens power *z* = *f*3.

*y*.

where *P*� = *P*Δ<sup>2</sup>

*xy* <sup>=</sup> <sup>Δ</sup>*x*Δ*<sup>y</sup>*

*<sup>j</sup>λ<sup>z</sup>* exp *<sup>j</sup>* <sup>2</sup>*π<sup>z</sup>*

(2) *uv* <sup>=</sup> exp *<sup>j</sup><sup>π</sup>*

*f*

*<sup>λ</sup>* exp *<sup>j</sup><sup>π</sup> λz*

Computer-Generated Phase-Only Holograms for Real-Time Image Display 295

*λz u*2Δ<sup>2</sup>

<sup>Δ</sup>*<sup>x</sup>* <sup>×</sup> *<sup>λ</sup><sup>z</sup>*

Fraunhofer diffraction regime as per Schnars & Juptner (2002). The Fresnel diffraction

Fig. 9. Fresnel diffraction geometry. When the hologram *huv* is illuminated by coherent light,

As previously shown by Dorsch et al. (1994); Fetthauer et al. (1995), it is straightforward to generalize hologram generation algorithms to the case of calculating Fresnel holograms. Here, the OSPR algorithm 1 is employed, replacing the conventional Fourier transform step by the discrete Fresnel transform of equation 43. The samples of the discrete Fresnel transform are

> *<sup>r</sup> P*� *Q*� ]

*<sup>i</sup> P*� *Q*� ]

*<sup>r</sup>* + *<sup>σ</sup>*<sup>2</sup> *<sup>i</sup>* )*P*� *Q*� /2]

*<sup>r</sup>* + *<sup>σ</sup>*<sup>2</sup> *<sup>i</sup>* )*P*� *Q*� /2].

The use of Fresnel holography has in two beneficial effects. Firstly, the diffracted near-field at the propagation distance *z* does not contain the conjugate image evident in the Fraunhofer region, in which *z* is necessarily greater than the Goodman (1996) distance. Second, because Fresnel propagation is characterized by a distance *z*, it is evident that the hologram incorporates lens power determined by the properties of the computed hologram, rather than the optical system. It therefore follows that the lens count in a holographic projection system could be reduced simply by removing *L*<sup>3</sup> of Figure 7, employing instead a Fresnel hologram

the RPF *Fxy* at a distance *z* is determined by Fresnel (or near-field) diffraction.

�[*Fxy*(0)] <sup>∼</sup> <sup>N</sup>[*μrP*, *<sup>σ</sup>*<sup>2</sup>

�[*Fxy*(0)] <sup>∼</sup> <sup>N</sup>[*μiP*, *<sup>σ</sup>*<sup>2</sup>

�[*Fxy*] <sup>∼</sup> <sup>N</sup>[0,(*σ*<sup>2</sup>

�[*Fxy*] <sup>∼</sup> <sup>N</sup>[0,(*σ*<sup>2</sup>

*huv Fxy*

 *<sup>x</sup> P*Δ*<sup>x</sup>*

> *<sup>x</sup>* + *<sup>v</sup>*2Δ<sup>2</sup> *y*

> > *z*

<sup>2</sup> + *y Q*Δ*<sup>y</sup>* <sup>2</sup>

<sup>Δ</sup>*<sup>y</sup>* , consistent with the size of RPF in the

. (45)

(44)

(46)

highly saturated primaries associated with laser-based display system, but that the speckle artefacts traditionally associated with this method of projection are substantially suppressed.

Fig. 8. Projected image at WVGA resolution resulting from a phase-only holographic projection system, employing the techniques described in this chapter, manufactured by Light Blue Optics Ltd. In this instance, *λ<sup>r</sup>* = 642 nm, *λ<sup>g</sup>* = 532 nm and *λ<sup>b</sup>* = 445 nm.

Several methods can be combined in a holographic projector in order to reduce speckle. In particular, the use of multiple holograms per video frame is beneficial to the speckle contrast; since *N* phase-independent subframes per video frame are shown within the eye's integration period, then the eye acts to add *N* independent speckle patterns on an intensity basis, and the contrast of the low-frequency components of the speckle in the field *Vxy* falls as *N*1/2. Due to computational and LC switching speed limitations, *N* cannot be increased indefinitely so additional methods can be combined to further reduce the speckle contrast. The presence of an intermediate image plane between the lens pair *L*<sup>3</sup> and *L*<sup>4</sup> makes it straightforward to employ optical speckle reduction techniques, as previously presented by Buckley (2008c).
