**2. Pointer Locator Using Stereovision (PLUS)**

The first metheod is to use two digital cameras simultaneously. It is called Pointer Locator Using Sterovision (PLUS) system.

#### **2.1 System set up**

118 Virtual Reality and Environments

to transform an acrylic plastic to a touch screen (Wilson, 2004). However, these two methods require the cameras to be placed on the opposite side of the screen from the user. They inevitably increased the thickness of the display and are not suitable for household, office,

Another popular approach is to put the camera at the user's side and use it to track a laser pointer or a fingertip (Cheng & Takatsuka, 2006). Meanwhile, a popular do-it-yourself project hacking the infrared camera of a WII remote to track the motion of an infrared pointer (Lee, 2007) have won applauses in the Internet. Nevertheless, the view of the user side cameras can be easily blocked by the body of the users and needs calibrations whenever the display is moved. Except for the fixed large projector screens, these methods are difficult

Meanwhile, stereovision has long been used in robot navigation to control manipulation or maneuver (Hager, 1995). A virtual touch screen using stereovision and see-through head mounted display was also proposed (Koh et al, 2008). However, the set up of the approaches is not suitable for daily application and the image processing is complicate due to the

In this article, we will describe the concept of using stereo- or mono-vision from the corners of the displays and track the motions of the pointer from sides. As we will explain later, this approach is simple to implement, inexpensive to equip, and not sensitive to the wear and tear of the screen. Comparing to existing touch screen technologies, it is also easy to scale up or down for different sizes or height/width ratios. Further, we can even generate virtual forces in the active input space and provide more vivid and intuitive user experiences. What's more, one of the most outstanding capabilities of the optical method is that it will superimpose but not obstruct the existing surface. This makes it ideal to be used on nontraditional displaying applications, such as a whiteboard, desktop, or even a regular writing

Further, we will briefly introduce our application of touch screen in the next generation Classroom Response System (CRS) (Langman & Fies, 2010; Suchman et al, 2006). It takes advantage of the superimposing capability of the optical touch screen on a regular writing surface and can obtain instantaneous feedback from the students beyond multi-choice questions provided by traditional Clickers (Nicol et al, 2003; Siau et al, 2006). That is, the students can write or sketch their answers using touch screen devices or touch screen

The structure of this article is as follows. After this introduction, we will provide two approaches of optical touch screens, one based on stereovision and the other one on pseudostereovision. Then we will introduce the idea of virtual force to be used in touch screen input and further the superimposed optical touch screen as a next generation CRS. After

The first metheod is to use two digital cameras simultaneously. It is called Pointer Locator

modified writing pads employing the technology described in this chapter.

that, a quick conclusion and discussion will be followed in the final section.

or classroom use.

to be adopted on regular desktops or laptops.

pad, which is a staple stationary in a student's backpack.

**2. Pointer Locator Using Stereovision (PLUS)** 

Using Sterovision (PLUS) system.

unstructured and noisy background.

In a PLUS system, each of the two cameras comes with a viewing angle that is equal to or greater than 90 degrees. As shown in Figure 1, a supporting structure encloses the flat screen display. At two adjacent corners, two cameras are mounted just above the surface of the display. They are positioned toward the center of the display such that the overlapped viewing field covers the entire screen surface. For convenience of illustration, we assume the cameras are located at the lower left and lower right corners as depicted in Figure 2. In the real-life applications as we will show later in this artical, the cameras are generally put on the top two corners to prevent occlusions by hand and to keep the camera lenses from collecting dust.

Fig. 1. Set up of the pointer locator using stereovision.

For convenience of analysis we first build a coordinate system. As seen in Figure 2, the origin of the system is at the focal point of the lower left camera. Its X and Y axes are parallel to the horizontal and vertical edges of the screen respectively. At this section, we only analyze the pointer positions. That is, we will only study the planar (X-Y) movement of the pointer projected at the surface of the display. Therefore, we assume a linear camera method

Fig. 2. Coordinate System for Pointer Locater Using Stereovision.

Optical Touch Screen and Its Application as a Next Generation Classroom Response System 121

 

 

*CB CB <sup>P</sup> AB AB*

 

> 

 

 

*AC AC <sup>P</sup> AB AB*

1 1 11 1 2 2 22 2 1 1 11 1 2 2 22 2

The corresponding cursor position of the pointer is obtained as Ps=[Psx, Psy]T=[(Px-s1x)/spx, (Py-s1y)/spy]T where spx and spy are pixel sizes of the display unit at both X and Y directions. Please note that the values of s1x, s1y, d2x, d2y can be obtained from initial calibration. However, they will not change even if the display is moved since the cameras and the screen

In normal stereovision, the surroundings of the interested features is often full of image noises. However, the scenes observed by the point-detecting cameras are simple and constructed.

sin cos , sin cos , cos sin , cos sin ,

*P P dN*

sin cos cos sin *xy y xy x PPd*

*x*

*y*

1 2 22 22

are both fixed to the same supporting structure.

**3.1 Pseudo-stereovision** 

**3. Pointer Locator Using Monovision (PLUM)** 

0, . *x y*

Therefore, we can use one camera instead of two to achieve the same result.

Fig. 3. One configuration of Pointer Locator Using Pseudo-Stereovision.

*A N A N B N B N C C d Nd*

where

By solving Equations 1 and 2 we can get the original position of [Px, Py]T as

2 22 2 2 22 2

 

 

. (6)

(7)

at this point and ignore the Z-coordinate. The focal point of the camera is offset by -s1x and -s1y at horizontal and vertical directions respectively from the lower left corner of the screen. Then the position of the corner is **s**1=[s1x, s1y]T. The lower right corner is **s**2=[s2x, s2y]T =[LH+s1x, s1y]T where LH is the width of the screen. Likewise, the upper right corner is **s**3=[s3x, s3y]T =[LH+s1x, LV+s1y]T where LV is the height of the screen.

#### **2.2 Pointer location using stereovision**

Assuming the optical axis of the left camera is rotated from X-axis by angle θ1, we can obtain the rotation matrix of the camera from screen-based coordinate system to camera-based coordinate system as

$$A\_1 = \begin{bmatrix} \cos \theta\_1 & -\sin \theta\_1 \\ \sin \theta\_1 & \cos \theta\_1 \end{bmatrix}. \tag{1}$$

Likewise, assuming the focal point of the right camera is located at **d**2=[d2x, d2y]T with angle θ2, the transformation (rotation and translation) matrix is

$$A\_2 = \begin{bmatrix} \cos \theta\_2 & -\sin \theta\_2 & d\_{2x} \\ \sin \theta\_2 & \cos \theta\_2 & d\_{2x} \\ 0 & 0 & 1 \end{bmatrix}. \tag{2}$$

Please note that we implied the use of homogeneous coordinates in Eq. 2. We will change back and forth between [x, y]T and [x, y, 1]T in calculations and in expressions whenever necessary. Therefore any point P=[Px, Py]T in the screen can be transformed to

$$P\_1 = \begin{bmatrix} P\_{1x} \\ P\_{1y} \end{bmatrix} = A\_1 P = \begin{bmatrix} P\_x \cos \theta\_1 - P\_y \sin \theta\_1 \\ P\_x \sin \theta\_1 + P\_y \cos \theta\_1 \end{bmatrix} \tag{3}$$

at camera 1 and

$$P\_1 = \begin{bmatrix} P\_{2x} \\ P\_{2y} \end{bmatrix} = A\_2 P = \begin{bmatrix} P\_x \cos \theta\_2 - P\_y \sin \theta\_2 \\ P\_x \sin \theta\_2 + P\_y \cos \theta\_2 \end{bmatrix} \tag{4}$$

at camera 2. We assume a pinpoint camera model with the focal length as λ1.

For camera 1, the projection of the point P on the image plane can be easily measured from the image as N1=n1\*cpx1 where n1 is the pixel position at the image plane and cpx1 is the size of each pixel for camera 1. Hence, we can easily obtain the following relationship using similar triangles:

$$\frac{P\_x \sin \theta\_1 + P\_y \cos \theta\_1}{P\_x \cos \theta\_1 - P\_y \sin \theta\_1} = \frac{-\lambda\_1}{N\_1} \tag{5}$$

Likewise, we can find similar result for camera 2 as:

$$\frac{P\_x \sin \theta\_2 + P\_y \cos \theta\_2 + d\_{2y}}{P\_x \cos \theta\_2 - P\_y \sin \theta\_2 + d\_{2x}} = \frac{-\lambda\_2}{N\_2} \quad . \tag{6}$$

By solving Equations 1 and 2 we can get the original position of [Px, Py]T as

$$\begin{cases} P\_x = \frac{C\_1 B\_2 - C\_2 B\_1}{A\_1 B\_2 - A\_2 B\_1} \\ P\_y = \frac{A\_1 C\_2 - A\_2 C\_1}{A\_1 B\_2 - A\_2 B\_1} \end{cases} \tag{7}$$

where

120 Virtual Reality and Environments

at this point and ignore the Z-coordinate. The focal point of the camera is offset by -s1x and -s1y at horizontal and vertical directions respectively from the lower left corner of the screen. Then the position of the corner is **s**1=[s1x, s1y]T. The lower right corner is **s**2=[s2x, s2y]T =[LH+s1x, s1y]T where LH is the width of the screen. Likewise, the upper right corner is

Assuming the optical axis of the left camera is rotated from X-axis by angle θ1, we can obtain the rotation matrix of the camera from screen-based coordinate system to camera-based

> cos sin sin cos

Likewise, assuming the focal point of the right camera is located at **d**2=[d2x, d2y]T with angle

2 2 22 cos sin sin cos

Please note that we implied the use of homogeneous coordinates in Eq. 2. We will change back and forth between [x, y]T and [x, y, 1]T in calculations and in expressions whenever

*A d* 

necessary. Therefore any point P=[Px, Py]T in the screen can be transformed to

1 1

1 2

Likewise, we can find similar result for camera 2 as:

at camera 2. We assume a pinpoint camera model with the focal length as λ1.

1 1

. (1)

. (2)

(3)

(4)

*x x d*

1 1

2 22

 

 

0 01

1 1 1

*x x y y x y P P P P AP <sup>P</sup> P P*

 

cos sin sin cos

cos sin sin cos

 (5)

1 1 1

2 2 2

*x x y y x y P P P P AP <sup>P</sup> P P*

For camera 1, the projection of the point P on the image plane can be easily measured from the image as N1=n1\*cpx1 where n1 is the pixel position at the image plane and cpx1 is the size of each pixel for camera 1. Hence, we can easily obtain the following relationship using

> sin cos cos sin *x y x y P P*

*PP N*

 

2 2 2

1 1 1 1 11

 

  **s**3=[s3x, s3y]T =[LH+s1x, LV+s1y]T where LV is the height of the screen.

1

*A*

θ2, the transformation (rotation and translation) matrix is

**2.2 Pointer location using stereovision** 

coordinate system as

at camera 1 and

similar triangles:

$$\begin{aligned} A\_1 &= N\_1 \sin \theta\_1 + \dot{\lambda}\_1 \cos \theta\_1, & A\_2 &= N\_2 \sin \theta\_2 + \dot{\lambda}\_2 \cos \theta\_2, \\ B\_1 &= N\_1 \cos \theta\_1 - \dot{\lambda}\_1 \sin \theta\_1, & B\_2 &= N\_2 \cos \theta\_2 - \dot{\lambda}\_2 \sin \theta\_2, \\ C\_1 &= 0, & C\_2 &= -\dot{\lambda}\_2 d\_{2\times} - N\_2 d\_{2\times} \end{aligned}$$

The corresponding cursor position of the pointer is obtained as Ps=[Psx, Psy]T=[(Px-s1x)/spx, (Py-s1y)/spy]T where spx and spy are pixel sizes of the display unit at both X and Y directions. Please note that the values of s1x, s1y, d2x, d2y can be obtained from initial calibration. However, they will not change even if the display is moved since the cameras and the screen are both fixed to the same supporting structure.

#### **3. Pointer Locator Using Monovision (PLUM)**

#### **3.1 Pseudo-stereovision**

In normal stereovision, the surroundings of the interested features is often full of image noises. However, the scenes observed by the point-detecting cameras are simple and constructed. Therefore, we can use one camera instead of two to achieve the same result.

Fig. 3. One configuration of Pointer Locator Using Pseudo-Stereovision.

Optical Touch Screen and Its Application as a Next Generation Classroom Response System 123

First, using the same set up of the coordinate system, the position of P1=[P1x, P1y]T can be found as [2L-Px, Py]T where L is the horizontal distance from the origin O to the mirror.

> sin cos cos sin *x y x y P P*

(2 )sin cos (2 )cos sin *x y x y*

*LP P*

*PP N*

*LP P N* 

where λ is the focal length of the physical camera, θ is the angle of the camera, and N1 and

2 2 2 1 2 2 1

 

1 2 1 2

*y x*

Note that the result is simpler compare to the true stereovision method due to the equality of focal length λ and rotation angle θ between the physical and virtual cameras. Meanwhile, we need to cover the combined surface of both the physical screen and the reflected one, so we can calibrate the angle θ to be π/4. In this case, the result can be

2

 

*x*

( )sin 2 2 sin 2 cos ( )sin 2 ( ) cos2 sin cos sin cos

*NN N N P L NN N N <sup>N</sup> P P N*

 

1

1 2

( )( )

 

*N N P L N N*

*y x*

*<sup>N</sup> P P N*

Sometime, the pointer is away from the vicinity of the screen but is still close enough to be picked up by the cameras. This can easily happen when a user is pondering for the next action or is talking in front of the screen. As shown in Figure 5A, in true stereovision, a virtual boundary can be set up in the algorithm to filter out the unwanted movements. Any point P within the virtual boundary is considered legitimate and is processed accordingly. Its motion will be reflected as the movement of the cursor on the screen. If the pointer is outside the virtual boundary, like the position P' in Figure 5A, it can be considered idle and an update is not necessary. However, in the background, the video stream is constantly processed to extract the positions of P at both cameras. The distance of the pointer from the screen surface needed to be constantly calculated and compared with the virtual boundary.

 

 

1

 

 

> 

> 

1

2

(10)

 

. (11)

(12)

 

(9)

Then we can obtain the equations similar to Equations 3 and 4:

N2 are the projected positions of P and P1 on the image plane.

2

 

*x*

Solve for Px and Py, we have

further simplified as

**4.1 Virtual boundary** 

**4. Application considerations** 

As seen in Figure 3, the camera on the right is removed. Instead, a mirror is put on the edge of the screen. For the convenience of illustration, we cut the width of the screen to half of that is shown in Figure 2. The mirror is configured to be parallel to the left edge of the screen, or the Y-axis of the screen-based coordinate system. Therefore we can have a mirror image of the camera (virtual camera) with the exact same physical characters as the real one. By combining the physical camera and its reflection (the virtual camera) together, we can obtain a pseudo-stereovision.

From the set up shown in Figure 3, it is obvious that

$$\begin{cases} \theta\_2 = -\theta\_1\\ \lambda\_2 = \lambda\_1 \end{cases} \tag{8}$$

The focal point or origin of the virtual camera frame is at [d2x, d2y]T=[2LH+2s1x, 0]T. Meanwhile, as seen from Figure 3, the mirror image of the pointer P=[Px, Py]T is P1=[P1x, P1y]T. Its projection to the image plane of the physical camera is N2. As depicted from Figure 3, the mirror image of the projection of P1 on the physical camera is the projection of the point P on the virtual camera. It is clear that N'2=-N2.

If we substitute the above values to Equation 5, we can easily find the position of pointer P=[Px, Py]T and its corresponding cursor position Ps=[Psx, Psy]T.

#### **3.2 Monovision with mirror image**

Alternatively, we can use a different approach. That is, instead of using the virtual camera and convert back to the case of stereovision, we can process the mirror image of the pointer at the physical camera directly.

Fig. 4. An alternative configuration of Pointer Localization Using Monovision with mirror image.

First, using the same set up of the coordinate system, the position of P1=[P1x, P1y]T can be found as [2L-Px, Py]T where L is the horizontal distance from the origin O to the mirror. Then we can obtain the equations similar to Equations 3 and 4:

$$\frac{P\_x \sin \theta + P\_y \cos \theta}{P\_x \cos \theta - P\_y \sin \theta} = \frac{-\lambda}{N\_1} \tag{9}$$

$$\frac{(2L - P\_x)\sin\theta + P\_y\cos\theta}{(2L - P\_x)\cos\theta - P\_y\sin\theta} = \frac{-\lambda}{N\_2} \tag{10}$$

where λ is the focal length of the physical camera, θ is the angle of the camera, and N1 and N2 are the projected positions of P and P1 on the image plane.

Solve for Px and Py, we have

122 Virtual Reality and Environments

As seen in Figure 3, the camera on the right is removed. Instead, a mirror is put on the edge of the screen. For the convenience of illustration, we cut the width of the screen to half of that is shown in Figure 2. The mirror is configured to be parallel to the left edge of the screen, or the Y-axis of the screen-based coordinate system. Therefore we can have a mirror image of the camera (virtual camera) with the exact same physical characters as the real one. By combining the physical camera and its reflection (the virtual camera) together, we can

> 2 1 2 1

The focal point or origin of the virtual camera frame is at [d2x, d2y]T=[2LH+2s1x, 0]T. Meanwhile, as seen from Figure 3, the mirror image of the pointer P=[Px, Py]T is P1=[P1x, P1y]T. Its projection to the image plane of the physical camera is N2. As depicted from Figure 3, the mirror image of the projection of P1 on the physical camera is the projection of the

If we substitute the above values to Equation 5, we can easily find the position of pointer

Alternatively, we can use a different approach. That is, instead of using the virtual camera and convert back to the case of stereovision, we can process the mirror image of the pointer

Fig. 4. An alternative configuration of Pointer Localization Using Monovision with mirror

 

(8)

 

obtain a pseudo-stereovision.

From the set up shown in Figure 3, it is obvious that

point P on the virtual camera. It is clear that N'2=-N2.

**3.2 Monovision with mirror image** 

at the physical camera directly.

image.

P=[Px, Py]T and its corresponding cursor position Ps=[Psx, Psy]T.

$$\begin{cases} P\_x = \frac{(\lambda^2 - N\_1 N\_2) \sin 2\theta + 2N\_2 \lambda \sin^2 \theta - 2N\_1 \lambda \cos^2 \theta}{(\lambda^2 - N\_1 N\_2) \sin 2\theta - (N\_1 + N\_2) \lambda \cos 2\theta} L \\\ P\_y = \frac{N\_1 \sin \theta + \lambda \cos \theta}{\lambda \sin \theta - N\_1 \cos \theta} P\_x \end{cases} \tag{11}$$

Note that the result is simpler compare to the true stereovision method due to the equality of focal length λ and rotation angle θ between the physical and virtual cameras. Meanwhile, we need to cover the combined surface of both the physical screen and the reflected one, so we can calibrate the angle θ to be π/4. In this case, the result can be further simplified as

$$\begin{cases} P\_{\mathbf{x}} = \frac{(\mathcal{L} - N\_1)(\mathcal{A} + N\_2)}{\mathcal{A}^2 - N\_1 N\_2} L \\\\ P\_{\mathbf{y}} = \frac{\mathcal{A} + N\_1}{\mathcal{A} - N\_1} P\_{\mathbf{x}} \end{cases} \tag{12}$$

#### **4. Application considerations**

#### **4.1 Virtual boundary**

Sometime, the pointer is away from the vicinity of the screen but is still close enough to be picked up by the cameras. This can easily happen when a user is pondering for the next action or is talking in front of the screen. As shown in Figure 5A, in true stereovision, a virtual boundary can be set up in the algorithm to filter out the unwanted movements. Any point P within the virtual boundary is considered legitimate and is processed accordingly. Its motion will be reflected as the movement of the cursor on the screen. If the pointer is outside the virtual boundary, like the position P' in Figure 5A, it can be considered idle and an update is not necessary. However, in the background, the video stream is constantly processed to extract the positions of P at both cameras. The distance of the pointer from the screen surface needed to be constantly calculated and compared with the virtual boundary.

Optical Touch Screen and Its Application as a Next Generation Classroom Response System 125

*z x <sup>M</sup> P P* 

With the 3D image features obtained, we can easily construct a virtual force function at both

where Ks and Kp are to be obtained from experiment and to be further adapted using self learning algorithms, *a*x and *a*y are accelerations of pointer at x and y directions, while *a*z is its acceleration at the z direction. The variables *a*x, *a*y and *a*z can all be calculated through second

We should point out that a virtual force can be considered as a function of any subset of the position variables and their derivatives. For example, a simple position based force feedback

<sup>1</sup> *F KPp <sup>z</sup>*

*F KP KV*

*ai bi pi i vi i*

3

1

where Kpi, Pi, ai, Kvi, Vi, and bi should be determined by the specific program needs.

*i*

where M is the vertical position of the pointer projected on the image plane.

(13)

2 2 *F Ka a s sx <sup>y</sup>* (14)

*F Ka p p <sup>z</sup>* (15)

(16)

(17)

Fig. 6. 3D trajectory in the action space.

Then the z position Pz can be easily obtained as

directions. A virtual stroke force can be obtained as

and a virtual push or pull can be considered as

order finite differentiation from Px, Py, and Pz.

while a more sophiscated one may look like

can be obtained as

On the other hand, with the method of monovision using mirror, we can set up a virtual boundary by combining the height of the mirror and the viewing angle of the camera. As shown in Figure 5B, when the pointer is above the virtual boundary, the image of the pointer P'1 will not be picked up by the camera even the pointer P' itself is within the range. That is, only one image feature will be obtained instead of two. A quick on-off logic can be easily used to dismiss this single point without calculating the position and distance first. Since the duration of the pointer in use is often far less than when it is not, demand on computing power will be greatly reduced.

Fig. 5. Virtual boundary with pseudo-stereovision.

## **4.2 Action space and virtual force**

In previous methods, only the X-direction projection (parallel to the screen) on the camera image plane is used. It only provides information on the position of the pointer. Move one step forward, we can take advantage of the Z-direction projection (perpendicular to the screen) of the image and the time derivatives of the projections for more advanced features. One such application is to set up an action space immediately outside the touch screen. In the action space, we can construct virtual forces and use them to achieve the results such as push, pull, stroke, or other similar motions.

First, let us expand the aforementioned model to a 3-dimensional space with the Z-axis perpendicular to the display screen and pointing away from it. This 3D action space is enclosed by the display screen on bottom and the virtual boundary on top. As seen in Figure 6, the pointer can freely move in this space. The projection of the pointer gives a planar trajectory on the camera image plane.

Fig. 6. 3D trajectory in the action space.

On the other hand, with the method of monovision using mirror, we can set up a virtual boundary by combining the height of the mirror and the viewing angle of the camera. As shown in Figure 5B, when the pointer is above the virtual boundary, the image of the pointer P'1 will not be picked up by the camera even the pointer P' itself is within the range. That is, only one image feature will be obtained instead of two. A quick on-off logic can be easily used to dismiss this single point without calculating the position and distance first. Since the duration of the pointer in use is often far less than when it is not, demand on

In previous methods, only the X-direction projection (parallel to the screen) on the camera image plane is used. It only provides information on the position of the pointer. Move one step forward, we can take advantage of the Z-direction projection (perpendicular to the screen) of the image and the time derivatives of the projections for more advanced features. One such application is to set up an action space immediately outside the touch screen. In the action space, we can construct virtual forces and use them to achieve the results such as

First, let us expand the aforementioned model to a 3-dimensional space with the Z-axis perpendicular to the display screen and pointing away from it. This 3D action space is enclosed by the display screen on bottom and the virtual boundary on top. As seen in Figure 6, the pointer can freely move in this space. The projection of the pointer gives a planar

computing power will be greatly reduced.

Fig. 5. Virtual boundary with pseudo-stereovision.

**4.2 Action space and virtual force** 

push, pull, stroke, or other similar motions.

trajectory on the camera image plane.

Then the z position Pz can be easily obtained as

$$P\_z = \frac{M}{\lambda} P\_x \tag{13}$$

where M is the vertical position of the pointer projected on the image plane.

With the 3D image features obtained, we can easily construct a virtual force function at both directions. A virtual stroke force can be obtained as

$$F\_s = K\_s \sqrt{a\_x^2 + a\_y^2} \tag{14}$$

and a virtual push or pull can be considered as

$$F\_p = K\_p a\_z \tag{15}$$

where Ks and Kp are to be obtained from experiment and to be further adapted using self learning algorithms, *a*x and *a*y are accelerations of pointer at x and y directions, while *a*z is its acceleration at the z direction. The variables *a*x, *a*y and *a*z can all be calculated through second order finite differentiation from Px, Py, and Pz.

We should point out that a virtual force can be considered as a function of any subset of the position variables and their derivatives. For example, a simple position based force feedback can be obtained as

$$F = \mathbb{K}\_p P\_z^{-1} \tag{16}$$

while a more sophiscated one may look like

$$F = \sum\_{i=1}^{3} K\_{pi} P\_i^{-ai} + K\_{vi} V\_i^{-bi} \tag{17}$$

where Kpi, Pi, ai, Kvi, Vi, and bi should be determined by the specific program needs.

Optical Touch Screen and Its Application as a Next Generation Classroom Response System 127

answers. The answers then are instantly transmitted to the host computer operated by the teacher. A program will collect and summarize the answers and present the aggregated

Both teachers and students welcomed the use of CRS. The engagement is bi-directional. The teacher can continuously assess the level of student understanding or mis-understanding. They can then give targeted clarifications or adjust the course material or pace to address the problems arisen. At the same time, the students can quickly apply the knowledge they just learned and answer the questions without raising hands, which can be intimidating to many. Since the answers are transmitted anonymously, the students will be free from the

However, still more features are desired for the Clickers. One major drawback of the current Clickers is its dependence on multi-choice or true/false questions (Siau et al, 2006). The teachers make up generally three or four possible wrong answers and mix them with the right one. As a legacy from the early computer-based standardized test, it is easy to grade. However, multi-choice or true/false questions are not the best or the only assessing tools. The number of choices is limited; the result can be skewed by test-taking techniques; and the teachers cannot directly know what the mistakes the students make from the answers.

Meanwhile, since the students can only choose from a limited list of options, their creativities are often ignored or discouraged. In any subject, there are always multiple solutions to a question, especially an open-ended question. In Science, Technology, Engineering, Art & Mathematics (STEAM) and many other fields, we often encourage students to focus on problem-solving progress instead of the final answer. Unfortunately, the multi-choice questions provide no room for the students to derive their own answers. It would be great if we can have a better tool that will encourage open-ended questions and

One attempt of improvement over clicker is to use touch sensitive tablets (Berque et al, 2004) or touch screen tablets. A touch sensitive tablet digitizes the strokes of the stylus on its surface and send the data stream to the host computer. However, due to the seperation of writing and displaying, the use of this kind device is unnatural and did not gain wide

The touch screen pads include Tablet PCs (Koile & Singer, 2006; Willis & Miertschin, 2004; Zurn & Frolik, 2004) or iPad and its competitors (we denote them as xPads). A user can write on the touch screen of the tablet with a stylus or a finger. Then the hardware and software will convert the writing to either the text or graphics, and then display them back on the screen. Once networked together, tablets can be a great tool for a teacher to get instant feedback from the students. The feedback will not be limited to multiple choices, but on every part of the knowledge assessment that can be conducted with a regular paper test. However, there are still several major drawbacks of these touch screen tablets. First is its cost. For example, the price of a tablet PC starts from \$1k and costs much more with rugged design. xPads are generally more affordable, but still costs hundred dollars each. The prices can easily rule out most classroom use, especially when they are under the stress of young students at their active years. Second is the different platforms of the tablet devices in the market. It is able to recommend but difficult to require all the students using the same

result to the teacher using a chart or graphic.

fear of peer pressure and public embarrassment.

promote problem-solving skills.

acceptance in the classrooms.

Comparing with the force sensing of traditional touch screens, the virtual force can give a much greater force range. The depth of the action space can be easily adjusted by the virtual boundary. By using cameras with wide viewing angles, the action space can be large enough to yield many interesting 3D motions, which can be important for active computer games or on screen calligraphy or painting. Currently, the pull, push, or similar motions are absent in many applications. If used, they can greatly improve the user experiences.

Further, the users do not even need to physically touch the screen. As long as the pointer or finger is in the action space, its movement will be monitored and reflected on the movement of the corresponding cursor or object in the program.
