**Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon Web Services (AWS)**

Kelly Bennett and James Robertson

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/58895

### **1. Introduction**

The popularity of social networking has allowed access to staggering amounts of unique data, which has created new possibilities for data analysis and exploitation. Such data has proven useful in marketing, decision making, destabilizing terrorist networks, behavior evolution, and determining future social trends [1].

Increased usage of social networking sites has also been observed during events related to natural disasters; significant political, sporting, and social events; and other crises. Twitter users provide status updates through tweets. Since tweets are typically short and always less than 140 characters, they may need to undergo additional analysis to provide contextual clues. Applying traditional natural language processing algorithms on such data is challenging. Before making use of this data, it must first be extracted, processed and analyzed appropriately using certain algorithms and theories.

According to a 2011 study from the International Data Corporation (IDC)–a marketing firm specializing in information technology and other consumer technologies–unstructured data, that is, data that does not have a pre-defined data model or is not organized in a pre-defined manner, is growing at a faster rate than structured data. Within the next decade, unstructured data will account for 90 percent of all data created.

A large driving factor in the increase in unstructured data is social networking data, such as tweets. It is estimated that more than 80 percent of all potentially useful data is unstructured [2].

© 2014 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The success of businesses in the coming decade will likely rely on their ability to successfully analyze data from social networks.

into one variable so they are not separated and lost. In addition, some phrases are commonly abbreviated, such as "NYC" for "New York City," or reduced, such as "Vegas" for "Las Vegas." A study, using algorithms to account for these abnormalities, was performed that determined trendy and bursty keywords on Twitter with 92% accuracy when compared with Google

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

5

Karandikar [4] described an approach to determine the most suitable topic model to cluster tweets by analyzing the effect of change in topic model parameters such as training data size and type and the number of topics on its clustering performance. The model was able to cluster tweets, using this approach, with an accuracy of greater than 64 percent for the two specific

Perera [5] developed a software architecture using a Twitter application program interface (API) to collect tweets sent to specific users. The extracted data were processed to characterize the inter-arrival times between tweets and the number of re-tweets. Analysis revealed that the arrival process of new tweets to a user can be modeled as a Poisson process while the number

Turner and Malleson [6] presented an exploratory geographical analysis of a sample of Twitter post data from June 2011 to March 2012 in the city of Leeds, England. Geographical cluster detection methods were used in combination with text mining techniques to identify patterns. Preliminary results suggested that the data could be used as a means of exploring daily spatial-

The following presents insight in to the MATLAB toolboxes that can be used in detecting

MATLABTM is a high-level language and interactive environment for numerical computation, visualization, and programming. MATLAB's product family includes numerous toolboxes for parallel computing, math, statistics and optimization, control system design and analysis, signal processing and communication, image processing and computer vision, test and measurement, and other areas [7]. A key feature of these products is the ability to research a

MATLAB's interactive environment allows end users to quickly prototype their own algo‐ rithms if existing toolboxes do not provide the desired functionality. A well-established community of users supports the exchange of algorithms for those wishing to share their

This research took advantage of the MATLAB community and the existing toolboxes. From the file exchange, Vladimir Bondarenko's contribution of the class to the Twitter REST API v1.1 was used to interface with Twitter to search for and extract tweets for specific topics and geographic regions [8]. Existing MATLAB Statistics and Neural Network Toolboxes were also

trends [3].

events.

temporal behavior.

research and prototypes.

of retweets follows a geometric distribution.

anomalies, that is, deviation from the normal pattern of life.

**3. MATLAB toolboxes and classes for anomaly detection**

number of algorithms quickly without having to design and code the algorithm.

In addition to the increase in social networking, cloud computing technologies continue to increase in popularity and provide end users the ability to quickly spin up powerful servers with analytical capabilities, making analysis of large amounts of data more affordable and practical than in previous decades.

#### **Main topics addressed in this chapter include:**


#### **2. Previous work**

Muhammad Mahbubur Rahman, author of "Mining Social Data to Extract Intellectual Knowledge," was able to extract data from Facebook using various data mining techniques. He first determined the most frequent terms that were being used, such as birthday, about me, gender, and music, and then created a database using these terms and the appropriate data types. [1] From the "about me" information, he was able to use an Euclidian distance formula to put each user into a certain "class", for example, aggressive, dishonest, romantic, eager to learn, or lazy. Grouping classes by age and gender creates a method of comparing various attributes by age and gender. This type of research demonstrates the ability to use data mining techniques to provide intellectual knowledge that can be used to possibility predict human behavior, provide insight into human decision making, or provide a method of determining anomalous events.

Data mining has proven useful in Twitter as well, with much research performed into detecting trends and bursty keywords. One has to pay attention to some abnormalities with Twitter, such as many people use smart phones to tweet and it is common for people to make mistakes while typing on smart phones. One would have to account for misspellings and merge them

<sup>1</sup> Registered trademark of Mathworks Corporation.

into one variable so they are not separated and lost. In addition, some phrases are commonly abbreviated, such as "NYC" for "New York City," or reduced, such as "Vegas" for "Las Vegas." A study, using algorithms to account for these abnormalities, was performed that determined trendy and bursty keywords on Twitter with 92% accuracy when compared with Google trends [3].

The success of businesses in the coming decade will likely rely on their ability to successfully

In addition to the increase in social networking, cloud computing technologies continue to increase in popularity and provide end users the ability to quickly spin up powerful servers with analytical capabilities, making analysis of large amounts of data more affordable and

**•** Explore viable approaches for extracting social networking data sets using tools compatible

**•** Use existing data mining and statistical tools within MATLAB to conduct analysis on social

**•** Discuss potential cost savings and document approaches for implementing social network‐ ing site data analysis via the cloud through vendors such as Amazon Web Services (AWS).

**•** Provide a tutorial on the use of MATLAB tools to analyze unstructured data with an

**•** Provide an overview and example on the use of MATLAB in a commercial cloud environ‐

Muhammad Mahbubur Rahman, author of "Mining Social Data to Extract Intellectual Knowledge," was able to extract data from Facebook using various data mining techniques. He first determined the most frequent terms that were being used, such as birthday, about me, gender, and music, and then created a database using these terms and the appropriate data types. [1] From the "about me" information, he was able to use an Euclidian distance formula to put each user into a certain "class", for example, aggressive, dishonest, romantic, eager to learn, or lazy. Grouping classes by age and gender creates a method of comparing various attributes by age and gender. This type of research demonstrates the ability to use data mining techniques to provide intellectual knowledge that can be used to possibility predict human behavior, provide insight into human decision making, or provide a method of determining

Data mining has proven useful in Twitter as well, with much research performed into detecting trends and bursty keywords. One has to pay attention to some abnormalities with Twitter, such as many people use smart phones to tweet and it is common for people to make mistakes while typing on smart phones. One would have to account for misspellings and merge them

and cloud technologies and infrastructures.

analyze data from social networks.

4 MATLAB Applications for the Practical Engineer

practical than in previous decades.

with MATLAB1

networking site data.

ment, such as AWS.

**2. Previous work**

anomalous events.

1 Registered trademark of Mathworks Corporation.

**Main topics addressed in this chapter include:**

emphasis on social networking data.

Karandikar [4] described an approach to determine the most suitable topic model to cluster tweets by analyzing the effect of change in topic model parameters such as training data size and type and the number of topics on its clustering performance. The model was able to cluster tweets, using this approach, with an accuracy of greater than 64 percent for the two specific events.

Perera [5] developed a software architecture using a Twitter application program interface (API) to collect tweets sent to specific users. The extracted data were processed to characterize the inter-arrival times between tweets and the number of re-tweets. Analysis revealed that the arrival process of new tweets to a user can be modeled as a Poisson process while the number of retweets follows a geometric distribution.

Turner and Malleson [6] presented an exploratory geographical analysis of a sample of Twitter post data from June 2011 to March 2012 in the city of Leeds, England. Geographical cluster detection methods were used in combination with text mining techniques to identify patterns. Preliminary results suggested that the data could be used as a means of exploring daily spatialtemporal behavior.

The following presents insight in to the MATLAB toolboxes that can be used in detecting anomalies, that is, deviation from the normal pattern of life.

#### **3. MATLAB toolboxes and classes for anomaly detection**

MATLABTM is a high-level language and interactive environment for numerical computation, visualization, and programming. MATLAB's product family includes numerous toolboxes for parallel computing, math, statistics and optimization, control system design and analysis, signal processing and communication, image processing and computer vision, test and measurement, and other areas [7]. A key feature of these products is the ability to research a number of algorithms quickly without having to design and code the algorithm.

MATLAB's interactive environment allows end users to quickly prototype their own algo‐ rithms if existing toolboxes do not provide the desired functionality. A well-established community of users supports the exchange of algorithms for those wishing to share their research and prototypes.

This research took advantage of the MATLAB community and the existing toolboxes. From the file exchange, Vladimir Bondarenko's contribution of the class to the Twitter REST API v1.1 was used to interface with Twitter to search for and extract tweets for specific topics and geographic regions [8]. Existing MATLAB Statistics and Neural Network Toolboxes were also used in this research to provide statistical algorithms and unsupervised learning methods, such as cluster analysis, for exploring data to discover hidden patterns and groupings in the data.

#### **3.1. TWITTY — Twitter REST API interface**

Twitty is a useful interface that runs within MATLAB for communicating with Twitter. Methods used in Twitty are essentially wrapper functions that call the Twitter API [9]. The API caller function, callTwitterAPI(), does the main work.

Key steps to successfully using the Twitter API include obtaining Twitter credentials and using JavaScript Object Notation (JSON) parsers. The MATLAB file exchange provides a JSON parser developed by Joel Feenstra [10]. The JSON parser parses a JSON string and returns a MATLAB cell array with the parsed data. JSON objects are converted to structures and JSON arrays are converted to cell arrays. Twitter credentials are easily created by registering at the Twitter site, creating an application, and retrieving consumer and access keys. These keys are required for running Twitty and include specific values for: also used in this research to provide statistical algorithms and unsupervised learning methods, such as cluster analysis, for exploring data to discover hidden patterns and groupings in the data. **3.1 TWITTY – TWITTER REST API INTERFACE**  Twitty is a useful interface that runs within MATLAB for communicating with Twitter. Methods used in Twitty are essentially wrapper functions that call the Twitter API [9]. The API caller function, callTwitterAPI(), does the main work.


To use these keys in a MATLAB script, users assign the values to their credential structure. Then, a twitty instance can be created and methods, such as search, can be called as shown in Figure 1. ConsumerKey ConsumerSecret AccessToken AccessTokenSecret

created and methods, such as search, can be called as shown in Figure 1.

To use these keys in a MATLAB script, users assign the values to their credential structure. Then, a twitty instance can be

search(text string) tw.search('World Series') Search all public tweets

updateStatus() tw.updateStatus('Watching the World Series tonight') Twit the text "Watching the

sampleStatuses() tw.sampleStatuses() Return a continuous stream

tw.search('NFL', 'count',20, 'include\_entities','true',

95.724660,1700mi','since\_id',LastID)

% Create credentials credentials.ConsumerKey = 'YourConsumerKey' credentials.ConsumerSecret = 'YourConsumerSecret' credentials.AccessToken = 'YourAccessToken' credentials.AccessTokenSecret = 'YourAccessTokenSecret' % Create Twitty Instance tw = twitty(credentials); % Search for World Series Related Tweets tw.search('World Series');

Figure 1. Twitty search method example. **Figure 1.** Twitty search method example.

search(multiple parameters)

Table 1. Twitty method examples.

Twitty provides a number of useful methods for interacting with the Twitter API. Example calls, along with a brief description, are shown in Table 1. Additional methods and detailed descriptions are found by typing *twitty.API* at the MATLAB prompt. **Method Example Call Description** Twitty provides a number of useful methods for interacting with the Twitter API. Example calls, along with a brief description, are shown in Table 1. Additional methods and detailed descriptions are found by typing *twitty.API* at the MATLAB prompt.

'geocode','39.051300,-

3

**Method Example Call Description**

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

tw.search('NFL', 'count',20, 'include\_entities','true', 'geocode','39.051300,-95.724660,1700mi','since\_id',LastI

The Statistics Toolbox™ provides statistical and machine learning algorithms and tools for organizing, analyzing, and modeling data. Key features include data organization and management via data set arrays and categorical arrays, exploratory data analysis with the use of interactive graphics and multivariate statistics, and regression or classification for predictive modeling [11]. The toolbox also includes functions that allow users to test hypotheses more effectively by checking for autocorrelation and randomness as well as other tests, for example, t-tests, one-sample tests, and distribution tests such as Chi-square and Kolmogorov-Smirnov.

Clustering algorithms available within MATLAB's Statistical Toolbox include *k*-means and hierarchical approaches. Clustering algorithms are particularly useful for analyzing social networking data as they help identify natural groupings that can then be further analyzed to determine similarities or differences and make business, marketing, or other decisions.

A *k*-means clustering algorithm forms *k* clusters by minimizing the mean between all cluster members. Clusters are defined by the centroid or center of each cluster. The algorithm works by moving data between clusters until the sum of the distances between each member and the

MATLAB allows different distance measures to be selected. Table 2 lists the available distance

Search all public tweets containing the words "World

http://dx.doi.org/10.5772/58895

7

Search public tweets having a identifier greater than LastID within 1700 miles of Topeka, Kansas containing the word "NFL" . Return entity data also (e.g., hashtags, URL

Twit the text "Watching the World Series tonight" from your account to the public

Return a continuous stream of random public tweets

Series"

mentions)

search(text string) tw.search('World Series')

D)

sampleStatuses() tw.sampleStatuses()

**3.2. MATLAB — Statistics TOOLBOXTM**

**Table 1.** Twitty method examples.

centroid is minimized.

measures and a brief description and example call.

updateStatus() tw.updateStatus('Watching the World Series tonight')

search(multiple parameters)

containing the words "World

Search public tweets having a identifier greater than LastID within 1700 miles of Topeka, Kansas containing the word "NFL" . Return entity data also (e.g., hashtags, URL mentions)

World Series tonight" from your account to the public

of random public tweets

Series"

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 7


JSON arrays are converted to cell arrays. Twitter credentials are easily created by registering at the Twitter site, creating an application, and retrieving consumer and access keys. These keys are required for running Twitty and include specific values **Table 1.** Twitty method examples.

3

containing the words "World

Search public tweets having a identifier greater than LastID within 1700 miles of Topeka, Kansas containing the word "NFL" . Return entity data also (e.g., hashtags, URL mentions)

World Series tonight" from your account to the public

of random public tweets

Series"

used in this research to provide statistical algorithms and unsupervised learning methods, such as cluster analysis, for exploring data to discover hidden patterns and groupings in the

Twitty is a useful interface that runs within MATLAB for communicating with Twitter. Methods used in Twitty are essentially wrapper functions that call the Twitter API [9]. The

Key steps to successfully using the Twitter API include obtaining Twitter credentials and using JavaScript Object Notation (JSON) parsers. The MATLAB file exchange provides a JSON parser developed by Joel Feenstra [10]. The JSON parser parses a JSON string and returns a MATLAB cell array with the parsed data. JSON objects are converted to structures and JSON arrays are converted to cell arrays. Twitter credentials are easily created by registering at the Twitter site, creating an application, and retrieving consumer and access keys. These keys are required for

**Method Example Call Description**

search(text string) tw.search('World Series') Search all public tweets

updateStatus() tw.updateStatus('Watching the World Series tonight') Twit the text "Watching the

sampleStatuses() tw.sampleStatuses() Return a continuous stream

tw.search('NFL', 'count',20, 'include\_entities','true',

95.724660,1700mi','since\_id',LastID)

To use these keys in a MATLAB script, users assign the values to their credential structure. Then, a twitty instance can be created and methods, such as search, can be called as shown in

created and methods, such as search, can be called as shown in Figure 1.

credentials.AccessTokenSecret = 'YourAccessTokenSecret'

Twitty provides a number of useful methods for interacting with the Twitter API. Example calls, along with a brief description, are shown in Table 1. Additional methods and detailed

'geocode','39.051300,-

credentials.ConsumerKey = 'YourConsumerKey' credentials.ConsumerSecret = 'YourConsumerSecret' credentials.AccessToken = 'YourAccessToken'

% Search for World Series Related Tweets

data.

**•** ConsumerKey

**•** AccessToken

Figure 1.

**•** ConsumerSecret

**•** AccessTokenSecret

for:

**3.1. TWITTY — Twitter REST API interface**

6 MATLAB Applications for the Practical Engineer

running Twitty and include specific values for:

 ConsumerKey ConsumerSecret AccessToken AccessTokenSecret

% Create credentials

% Create Twitty Instance tw = twitty(credentials);

tw.search('World Series');

Figure 1. Twitty search method example.

descriptions are found by typing *twitty.API* at the MATLAB prompt.

search(multiple parameters)

**Figure 1.** Twitty search method example.

Table 1. Twitty method examples.

API caller function, callTwitterAPI(), does the main work.

#### **3.2. MATLAB — Statistics TOOLBOXTM**

To use these keys in a MATLAB script, users assign the values to their credential structure. Then, a twitty instance can be The Statistics Toolbox™ provides statistical and machine learning algorithms and tools for organizing, analyzing, and modeling data. Key features include data organization and management via data set arrays and categorical arrays, exploratory data analysis with the use of interactive graphics and multivariate statistics, and regression or classification for predictive modeling [11]. The toolbox also includes functions that allow users to test hypotheses more effectively by checking for autocorrelation and randomness as well as other tests, for example, t-tests, one-sample tests, and distribution tests such as Chi-square and Kolmogorov-Smirnov.

> Clustering algorithms available within MATLAB's Statistical Toolbox include *k*-means and hierarchical approaches. Clustering algorithms are particularly useful for analyzing social networking data as they help identify natural groupings that can then be further analyzed to determine similarities or differences and make business, marketing, or other decisions.

Twitty provides a number of useful methods for interacting with the Twitter API. Example calls, along with a brief description, are shown in Table 1. Additional methods and detailed descriptions are found by typing *twitty.API* at the MATLAB prompt. A *k*-means clustering algorithm forms *k* clusters by minimizing the mean between all cluster members. Clusters are defined by the centroid or center of each cluster. The algorithm works by moving data between clusters until the sum of the distances between each member and the centroid is minimized.

> MATLAB allows different distance measures to be selected. Table 2 lists the available distance measures and a brief description and example call.


The closer the value of the cophenetic correlation coefficient is to 1, the more accurately the

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

Distance measures, similar to the *k*-means clustering, need to be selected to generate the linkages of the tree. Figure 2 shows the code to create a tree based on the Euclidean distance for a matrix named 'meas' and displays the results in a tree and then calculates the cophenetic

and then calculates the cophenetic correlation.

eucD = pdist(meas,'euclidean'); clustTreeEuc = linkage(eucD,'average'); [h,nodes] = dendrogram(clustTreeEuc,0); measCophenet = cophenet(clustTreeEuc,eucD);

Figure 2. MATLAB code to calculate cophenetic correlation. **3.3 MATLAB – NEURAL NETWORK TOOLBOXTM**

normalizing the mean and standard deviation of the training set.

The Neural Network ToolboxTM provides functions and apps for modeling complex nonlinear systems that are not easily modeled with a closed-form equation. Primary features include data fitting, clustering, and pattern recognition to forecast future events, both supervised and unsupervised network architectures, and training algorithms, such as gradient descent and conjugate gradient methods, to help automatically adjust the network's weights and biases [12]. Also included are preprocessing and post-processing functions that improve the efficien‐ cy of neural network training and enable detailed analysis of network performance by reducing the dimensions of the input vectors using principal component analysis and nor‐

MATLAB's Neural Network Toolbox provides self-organizing maps for both unsupervised and supervised clustering. Self-organizing maps retain topological information for similar classes and provide reasonable classifiers. Self-organizing maps can be used for multidimen‐ sional data with complex features, making them attractive for analysis of social networking data. The self-organizing maps functionality within MATLAB can be used to cluster tweets

Self-organizing maps learn to classify input vectors according to how they are grouped in the input space. The neurons in a self-organizing map can be arranged based on specific topologies. Within MATLAB, topologies for gridtop, hextop and random are possible. Results are not affected by the choice of topology used; however, visual inspection of the data space is best achieved by use of a hexagonal grid (hextop) [13]. A gridtop topology has neurons evenly spaced in matrix of specific dimensions. For example, an 80-neuron self-organizing map may be arranged in an 8 × 10 gridtop topology. A hextop topology is similar, but the shape of the grid can best be described as a hexagon. A random topology has the neurons randomly located.

Figure 3 illustrates gridtop, hextop and random topologies for a 3 × 4 set of neurons.

Distance measures, similar to the *k*-means clustering, need to be selected to generate the linkages of the tree. Figure 2 shows the code to create a tree based on the Euclidean distance for a matrix named 'meas' and displays the results in a tree

http://dx.doi.org/10.5772/58895

9

The Neural Network Toolbox provides functions and apps for modeling complex nonlinear systems that are not easily modeled with a closed-form equation. Primary features include data fitting, clustering, and pattern recognition to forecast future events, both supervised and unsupervised network architectures, and training algorithms, such as gradient descent and conjugate gradient methods, to help automatically adjust the network's weights and biases [12]. Also included are preprocessing and post-processing functions that improve the efficiency of neural network training and enable detailed analysis of network performance by reducing the dimensions of the input vectors using principal component analysis and

MATLAB's Neural Network Toolbox provides self-organizing maps for both unsupervised and supervised clustering. Selforganizing maps retain topological information for similar classes and provide reasonable classifiers. Self-organizing maps can be used for multidimensional data with complex features, making them attractive for analysis of social networking data. The self-organizing maps functionality within MATLAB can be used to cluster tweets based on the Twitter-extracted fields. Self-organizing maps learn to classify input vectors according to how they are grouped in the input space. The neurons in a self-organizing map can be arranged based on specific topologies. Within MATLAB, topologies for gridtop, hextop and random are possible. Results are not affected by the choice of topology used; however, visual inspection of the data space is best achieved by use of a hexagonal grid (hextop) [13]. A gridtop topology has neurons evenly spaced in matrix of specific dimensions. For example, an 80-neuron self-organizing map may be arranged in an 8 × 10 gridtop topology. A hextop topology is similar, but the shape of the grid can best be described as a hexagon. A random topology has the neurons

randomly located. Figure 3 illustrates gridtop, hextop and random topologies for a 3 × 4 set of neurons.

5

clustering solution reflects your data.

**Figure 2.** MATLAB code to calculate cophenetic correlation.

**3.3. MATLAB – Neural Network TOOLBOXTM**

based on the Twitter-extracted fields.

malizing the mean and standard deviation of the training set.

correlation.

**Table 2.** MATLAB *k*-means clustering distance measures.

Visually displaying the results for multidimensional data can be challenging. However; the silhouette plot, available within the MATLAB Statistics Toolbox, displays a measure between 0 and 1, representing how close each point in one cluster is to the points in the neighboring clusters. Values close to 1 indicate points are distant from neighboring clusters whereas values close to 0 indicate points are not distinctly different from one cluster or another. Negative values indicate points that are most likely assigned to the wrong cluster.

In the following example, cidx2 represents the cluster index for each given sample of data. Assuming cidx2 was returned from the call to the *k*-means function, an example call of the silhouette function is:

```
silhouette(meas,cidx2,'sqeuclidean');
```
Hierarchical clustering groups data to create a tree structure consisting of multiple levels. Users can prune parts of the tree depending upon the application and level of detail required. The links between data are represented as upside-down U-shaped lines with the height of each line indicating the distance between the data. This height is known as the cophenetic correla‐ tion distance between the two objects. MATLAB has a function that measures this distance. The closer the value of the cophenetic correlation coefficient is to 1, the more accurately the clustering solution reflects your data.

Distance measures, similar to the *k*-means clustering, need to be selected to generate the linkages of the tree. Figure 2 shows the code to create a tree based on the Euclidean distance for a matrix named 'meas' and displays the results in a tree and then calculates the cophenetic correlation. Distance measures, similar to the *k*-means clustering, need to be selected to generate the linkages of the tree. Figure 2 shows the code to create a tree based on the Euclidean distance for a matrix named 'meas' and displays the results in a tree

and then calculates the cophenetic correlation.

eucD = pdist(meas,'euclidean'); clustTreeEuc = linkage(eucD,'average'); [h,nodes] = dendrogram(clustTreeEuc,0); measCophenet = cophenet(clustTreeEuc,eucD);

**3.3 MATLAB – NEURAL NETWORK TOOLBOXTM**

Figure 2. MATLAB code to calculate cophenetic correlation. **Figure 2.** MATLAB code to calculate cophenetic correlation.

**Distance Measure Description Example call**

kmeans(meas,3,'dist','sqeuclidean');

kmeans(meas,3,'dist','cityblock');

kmeans(meas,3,'dist','cosine');

kmeans(meas,3,'dist','correlation');

kmeans(meas,3,'dist','hamming');

Visually displaying the results for multidimensional data can be challenging. However; the silhouette plot, available within the MATLAB Statistics Toolbox, displays a measure between 0 and 1, representing how close each point in one cluster is to the points in the neighboring clusters. Values close to 1 indicate points are distant from neighboring clusters whereas values close to 0 indicate points are not distinctly different from one cluster or another. Negative

In the following example, cidx2 represents the cluster index for each given sample of data. Assuming cidx2 was returned from the call to the *k*-means function, an example call of the

Hierarchical clustering groups data to create a tree structure consisting of multiple levels. Users can prune parts of the tree depending upon the application and level of detail required. The links between data are represented as upside-down U-shaped lines with the height of each line indicating the distance between the data. This height is known as the cophenetic correla‐ tion distance between the two objects. MATLAB has a function that measures this distance.

points in that cluster.

Each centroid is the componentwise median of the points in that

included angle between. Each centroid is the mean of the points in that cluster.

correlation between points. Each centroid is the component-wise mean of the points in that

binary data. Each centroid is the component-wise median of points in that cluster.

values indicate points that are most likely assigned to the wrong cluster.

silhouette(meas,cidx2,'sqeuclidean');

Squared Euclidean Each centroid is the mean of the

8 MATLAB Applications for the Practical Engineer

City Block Sum of absolute differences.

cluster.

Cosine One minus the cosine of the

Correlation One minus the sample

cluster.

Hamming Percentage of bits that differ in

**Table 2.** MATLAB *k*-means clustering distance measures.

silhouette function is:

#### The Neural Network Toolbox provides functions and apps for modeling complex nonlinear systems that are not easily **3.3. MATLAB – Neural Network TOOLBOXTM**

future events, both supervised and unsupervised network architectures, and training algorithms, such as gradient descent and conjugate gradient methods, to help automatically adjust the network's weights and biases [12]. Also included are preprocessing and post-processing functions that improve the efficiency of neural network training and enable detailed analysis of network performance by reducing the dimensions of the input vectors using principal component analysis and normalizing the mean and standard deviation of the training set. MATLAB's Neural Network Toolbox provides self-organizing maps for both unsupervised and supervised clustering. Selforganizing maps retain topological information for similar classes and provide reasonable classifiers. Self-organizing maps can be used for multidimensional data with complex features, making them attractive for analysis of social networking data. The self-organizing maps functionality within MATLAB can be used to cluster tweets based on the Twitter-extracted fields. Self-organizing maps learn to classify input vectors according to how they are grouped in the input space. The neurons in a self-organizing map can be arranged based on specific topologies. Within MATLAB, topologies for gridtop, hextop and The Neural Network ToolboxTM provides functions and apps for modeling complex nonlinear systems that are not easily modeled with a closed-form equation. Primary features include data fitting, clustering, and pattern recognition to forecast future events, both supervised and unsupervised network architectures, and training algorithms, such as gradient descent and conjugate gradient methods, to help automatically adjust the network's weights and biases [12]. Also included are preprocessing and post-processing functions that improve the efficien‐ cy of neural network training and enable detailed analysis of network performance by reducing the dimensions of the input vectors using principal component analysis and nor‐ malizing the mean and standard deviation of the training set.

modeled with a closed-form equation. Primary features include data fitting, clustering, and pattern recognition to forecast

random are possible. Results are not affected by the choice of topology used; however, visual inspection of the data space is best achieved by use of a hexagonal grid (hextop) [13]. A gridtop topology has neurons evenly spaced in matrix of specific dimensions. For example, an 80-neuron self-organizing map may be arranged in an 8 × 10 gridtop topology. A hextop topology is similar, but the shape of the grid can best be described as a hexagon. A random topology has the neurons randomly located. Figure 3 illustrates gridtop, hextop and random topologies for a 3 × 4 set of neurons. MATLAB's Neural Network Toolbox provides self-organizing maps for both unsupervised and supervised clustering. Self-organizing maps retain topological information for similar classes and provide reasonable classifiers. Self-organizing maps can be used for multidimen‐ sional data with complex features, making them attractive for analysis of social networking data. The self-organizing maps functionality within MATLAB can be used to cluster tweets based on the Twitter-extracted fields.

Self-organizing maps learn to classify input vectors according to how they are grouped in the input space. The neurons in a self-organizing map can be arranged based on specific topologies. Within MATLAB, topologies for gridtop, hextop and random are possible. Results are not affected by the choice of topology used; however, visual inspection of the data space is best achieved by use of a hexagonal grid (hextop) [13]. A gridtop topology has neurons evenly spaced in matrix of specific dimensions. For example, an 80-neuron self-organizing map may be arranged in an 8 × 10 gridtop topology. A hextop topology is similar, but the shape of the grid can best be described as a hexagon. A random topology has the neurons randomly located. Figure 3 illustrates gridtop, hextop and random topologies for a 3 × 4 set of neurons.

5

**Figure 3.** Gridtop, hextop, and randtop self-organizing map topologies.

The self-organization map routine can be run in MATLAB by calling the selforgmap function. Once a map is set up, it can be trained using specific input, and results can be displayed visually. Figure 4 shows a code example to generate a 3 × 4 neuron self-organizing map, conduct training, and display the results. Figure 3. Gridtop, hextop, and randtop self-organizing map topologies. The self-organization map routine can be run in MATLAB by calling the selforgmap function. Once a map is set up, it can be

self-organizing map, conduct training, and display the results.

trained using specific input, and results can be displayed visually. Figure 4 shows a code example to generate a 3 × 4 neuron

```
net = selforgmap([3 4]); 
net = train(net,meas); 
view(net); 
y = net(meas); 
classes = vec2ind(y);
```
Figure 4. MATLAB code example to generate a 3 × 4 neuron self-organizing map, conduct training, and display the results. The Neural Network Toolbox also contains the nnstart tool, which provides an interactive tool for loading data and selecting **Figure 4.** MATLAB code example to generate a 3 × 4 neuron self-organizing map, conduct training, and display the results.

algorithms and training iterations. As shown in Figure 5, the interactive tool allows the user to select run options and then train for a specific number of epochs. The Neural Network Toolbox also contains the nnstart tool, which provides an interactive tool for loading data and selecting algorithms and training iterations. As shown in Figure 5, the interactive tool allows the user to select run options and then train for a specific number of epochs.

6

**Figure 5.** MATLAB's nnstart interactive interface.

resources to purchase hardware and software.

**3.4. Deploying MATLAB on AWS**

The addition of the MATLAB's Parallel Computing Toolbox™ allows the user to speed up training and handle large data sets by distributing computations and data across multiple processors and graphic processing units (GPUs). The ability to spin up multiple central processing units (CPUs) and GPUs from cloud services such as AWS makes this toolbox attractive. The user may be limited by the availability of MATLAB licenses for this option. MATLAB and AWS are currently working out a process to make this more feasible with a payas-you-go license feature that could be quite attractive for businesses that do not have the

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

11

Cloud service providers such as AWS can significantly reduce the startup and maintenance costs for high-performance computers needed for efficient running of complex math, statistics, and optimization algorithms. AWS has free and pay tier options that will fit most budgets. Previous research has shown the cost of running multiple high-performance servers for many Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 11


trained using specific input, and results can be displayed visually. Figure 4 shows a code example to generate a 3 × 4 neuron **Figure 5.** MATLAB's nnstart interactive interface.

**Figure 3.** Gridtop, hextop, and randtop self-organizing map topologies.

conduct training, and display the results.

10 MATLAB Applications for the Practical Engineer

results.

epochs.

The self-organization map routine can be run in MATLAB by calling the selforgmap function. Once a map is set up, it can be trained using specific input, and results can be displayed visually. Figure 4 shows a code example to generate a 3 × 4 neuron self-organizing map,

> net = selforgmap([3 4]); net = train(net,meas);

**Figure 4.** MATLAB code example to generate a 3 × 4 neuron self-organizing map, conduct training, and display the

The Neural Network Toolbox also contains the nnstart tool, which provides an interactive tool for loading data and selecting algorithms and training iterations. As shown in Figure 5, the interactive tool allows the user to select run options and then train for a specific number of

view(net); y = net(meas); classes = vec2ind(y);

train for a specific number of epochs.

Figure 3. Gridtop, hextop, and randtop self-organizing map topologies.

self-organizing map, conduct training, and display the results.

The self-organization map routine can be run in MATLAB by calling the selforgmap function. Once a map is set up, it can be

algorithms and training iterations. As shown in Figure 5, the interactive tool allows the user to select run options and then

Figure 4. MATLAB code example to generate a 3 × 4 neuron self-organizing map, conduct training, and display the results. The addition of the MATLAB's Parallel Computing Toolbox™ allows the user to speed up training and handle large data sets by distributing computations and data across multiple processors and graphic processing units (GPUs). The ability to spin up multiple central processing units (CPUs) and GPUs from cloud services such as AWS makes this toolbox attractive. The user may be limited by the availability of MATLAB licenses for this option. MATLAB and AWS are currently working out a process to make this more feasible with a payas-you-go license feature that could be quite attractive for businesses that do not have the resources to purchase hardware and software.

#### The Neural Network Toolbox also contains the nnstart tool, which provides an interactive tool for loading data and selecting **3.4. Deploying MATLAB on AWS**

6

Cloud service providers such as AWS can significantly reduce the startup and maintenance costs for high-performance computers needed for efficient running of complex math, statistics, and optimization algorithms. AWS has free and pay tier options that will fit most budgets. Previous research has shown the cost of running multiple high-performance servers for many hours was just a few dollars [14]. As long as users have a valid MATLAB license, they can install MATLAB on the AWS machine(s) and run the data and analysis.

iteration to gather only recent tweets beyond a specific ID. Appendix A shows the MATLAB

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

13

Statistics and analysis were run at the end of the data collection using existing MATLAB tools and functions provided in the Statistics and Neural Network Toolboxes. Within the Statistics Toolbox, the *k*-means and hierarchical clustering approaches and associated visual displays

The distance measure used for all *k*-means calculations was the default selection of Squared Euclidian. An example call used for three clusters with Squared Euclidian distance is as

The self-organizing map functions within the Neural Network Toolbox were used to cluster

In addition to the existing Statistical and Neural Network functions, some simple data manipulation algorithms were used to extract the time between tweets. As shown in Figure 6, the following code example determines and stores the time between tweets found in

time1 = [str2num(DatesQ2{i}(26:30)) 10 str2num(DatesQ2{i}(9:10)) str2num(DatesQ2{i}(12:13))

time2 = [str2num(DatesQ2{i+1}(26:30)) 10 str2num(DatesQ2{i+1}(9:10)) str2num(DatesQ2{i+1}(12:13))

In this code, time {i} and time {i+1} values are constructed using the default time structure of the Twitter collection and converted to a MATLAB time structure. The dates are then differenced using MATLAB's built-in elapsed time function (etime). This process continues to calculate the time delta for all consecutive tweets for all search queries.

In this code, time {i} and time {i+1} values are constructed using the default time structure of the Twitter collection and converted to a MATLAB time structure. The dates are then differ‐ enced using MATLAB's built-in elapsed time function (etime). This process continues to

> Results were provided using MATLAB's visual and plotting functions. To better understand the frequency, or popularity, of the two search terms for each time period during the experiment, some baseline descriptive statistics were calculated.

Boxplots are a convenient way for visualizing the interquartile range, average and outlier data. Figures 7–10 represent boxplots for the 3-hour experiment windows from Saturday 5 PM to 8 PM, Saturday 8 PM to 11 PM , Sunday 1 PM to 4 PM,

Results were provided using MATLAB's visual and plotting functions. To better understand the frequency, or popularity, of the two search terms for each time period during the experi‐

Figure 7. Boxplot for NFL versus World Series time between tweets, Saturday 5 PM to 8 PM.

9

m file that was run for the experiment.

kmeans(meas,3,'dist','sqeuclidean');

str2num(DatesQ2{i}(15:16)) str2num(DatesQ2{i}(18:19))];

deltaWorldSeries(i) = abs(etime(time1,time2));

Figure 6. MATLAB code to determine time between tweets.

**5. INITIAL RESULTS AND DISCUSSION** 

**Figure 6.** MATLAB code to determine time between tweets.

and Sunday 5 PM to 8 PM, respectively.

ment, some baseline descriptive statistics were calculated.

Appendix B shows the Data analysis m file used for this experiment.

calculate the time delta for all consecutive tweets for all search queries.

Appendix B shows the Data analysis m file used for this experiment.

str2num(DatesQ2{i+1}(15:16)) str2num(DatesQ2{i+1}(18:19))];

were used.

follows:

the Twitter collected data.

end

**5.1 BOXPLOTS** 

**5. Initial results and discussion**

MATLAB cell array named DatesQ2.

for i=1:length(DatesQ2)-1

Once the runs are complete, users can disable the MATLAB license on the AWS machine. If users need to run the experiment again, they can spin up the server and enable the license to continue the experiment.

### **4. Experiment**

The experiment consisted of running MATLAB code on commercial cloud architecture (AWS) to collect publically available tweets on common keywords, such as NFL and World Series, during specific time intervals of a weekend over a large geographic region when both NFL games and World Series games were being played. Various MATLAB tools and functions were used to gather statistical information and analyze the data by use of unsupervised learning and clustering methods.

For this experiment, a server running Windows2 2008 operating system was spun up in the AWS free tier. A MATLAB license was installed that included the Statistics and Neural Network toolboxes. The twitty.m and parse\_json.m files were also uploaded to allow MATLAB to call the Twitter API.

After successful installation, a MATLAB script file (m file) was created to provide the proper Twitter credentials, and then search for tweets related to two different sports events, the World Series and NFL games. The specific search strings used included "NFL" and "World Series".

Four different 3-hour data collections were made over a period of two days. Collection times included Saturday from 5 PM to 8 PM and 8 PM to 11 PM and Sunday from 1 PM to 4 PM and 5 PM to 8 PM during the weekend of the 2013 World Series. During this time frame, a World Series event took place on both Saturday and Sunday evenings, and a series of NFL games occurred on Sunday afternoon and evening.

A large geographic collection area, centered over a 1700 mile radius about Topeka, Kansas, was used for this experiment. This geographic radius included most of the continental United States and parts of Canada and Mexico. The search parameters included the Twitter entities for storing User-, URL-, and hashtag-mentions. Sampling occurred every 60 seconds over the 3-hour window with Twitter ID information being used to eliminate duplicate tweets in the time frame. A sample call for the search is as follows:

S=tw.search('NFL', 'count',20, 'include\_entities','true','geocode','39.051300,-95.724660,1700mi', 'since\_id',LastID);

In this call, Twitter is being searched for up to 20 tweets that include the text "NFL" that originated within 1700 miles of Topeka, Kansas. The LastID variable was updated after each

<sup>2</sup> Registered trademark of Microsoft Corporation.

iteration to gather only recent tweets beyond a specific ID. Appendix A shows the MATLAB m file that was run for the experiment.

Statistics and analysis were run at the end of the data collection using existing MATLAB tools and functions provided in the Statistics and Neural Network Toolboxes. Within the Statistics Toolbox, the *k*-means and hierarchical clustering approaches and associated visual displays were used.

The distance measure used for all *k*-means calculations was the default selection of Squared Euclidian. An example call used for three clusters with Squared Euclidian distance is as follows:

```
kmeans(meas,3,'dist','sqeuclidean');
```
hours was just a few dollars [14]. As long as users have a valid MATLAB license, they can

Once the runs are complete, users can disable the MATLAB license on the AWS machine. If users need to run the experiment again, they can spin up the server and enable the license to

The experiment consisted of running MATLAB code on commercial cloud architecture (AWS) to collect publically available tweets on common keywords, such as NFL and World Series, during specific time intervals of a weekend over a large geographic region when both NFL games and World Series games were being played. Various MATLAB tools and functions were used to gather statistical information and analyze the data by use of unsupervised learning

AWS free tier. A MATLAB license was installed that included the Statistics and Neural Network toolboxes. The twitty.m and parse\_json.m files were also uploaded to allow MATLAB

After successful installation, a MATLAB script file (m file) was created to provide the proper Twitter credentials, and then search for tweets related to two different sports events, the World Series and NFL games. The specific search strings used included "NFL" and "World Series".

Four different 3-hour data collections were made over a period of two days. Collection times included Saturday from 5 PM to 8 PM and 8 PM to 11 PM and Sunday from 1 PM to 4 PM and 5 PM to 8 PM during the weekend of the 2013 World Series. During this time frame, a World Series event took place on both Saturday and Sunday evenings, and a series of NFL games

A large geographic collection area, centered over a 1700 mile radius about Topeka, Kansas, was used for this experiment. This geographic radius included most of the continental United States and parts of Canada and Mexico. The search parameters included the Twitter entities for storing User-, URL-, and hashtag-mentions. Sampling occurred every 60 seconds over the 3-hour window with Twitter ID information being used to eliminate duplicate tweets in the

In this call, Twitter is being searched for up to 20 tweets that include the text "NFL" that originated within 1700 miles of Topeka, Kansas. The LastID variable was updated after each

S=tw.search('NFL', 'count',20, 'include\_entities','true','geoco-

2008 operating system was spun up in the

install MATLAB on the AWS machine(s) and run the data and analysis.

continue the experiment.

12 MATLAB Applications for the Practical Engineer

and clustering methods.

to call the Twitter API.

For this experiment, a server running Windows2

occurred on Sunday afternoon and evening.

2 Registered trademark of Microsoft Corporation.

time frame. A sample call for the search is as follows:

de','39.051300,-95.724660,1700mi', 'since\_id',LastID);

**4. Experiment**

The self-organizing map functions within the Neural Network Toolbox were used to cluster the Twitter collected data.

In addition to the existing Statistical and Neural Network functions, some simple data manipulation algorithms were used to extract the time between tweets. As shown in Figure 6, the following code example determines and stores the time between tweets found in MATLAB cell array named DatesQ2.

Figure 6. MATLAB code to determine time between tweets. **Figure 6.** MATLAB code to determine time between tweets.

converted to a MATLAB time structure. The dates are then differenced using MATLAB's built-in elapsed time function (etime). This process continues to calculate the time delta for all consecutive tweets for all search queries. Appendix B shows the Data analysis m file used for this experiment. **5. INITIAL RESULTS AND DISCUSSION**  Results were provided using MATLAB's visual and plotting functions. To better understand the frequency, or popularity, of In this code, time {i} and time {i+1} values are constructed using the default time structure of the Twitter collection and converted to a MATLAB time structure. The dates are then differ‐ enced using MATLAB's built-in elapsed time function (etime). This process continues to calculate the time delta for all consecutive tweets for all search queries.

In this code, time {i} and time {i+1} values are constructed using the default time structure of the Twitter collection and

Boxplots are a convenient way for visualizing the interquartile range, average and outlier data. Figures 7–10 represent boxplots for the 3-hour experiment windows from Saturday 5 PM to 8 PM, Saturday 8 PM to 11 PM , Sunday 1 PM to 4 PM,

the two search terms for each time period during the experiment, some baseline descriptive statistics were calculated. Appendix B shows the Data analysis m file used for this experiment.

#### **5. Initial results and discussion**

and Sunday 5 PM to 8 PM, respectively.

**5.1 BOXPLOTS** 

Results were provided using MATLAB's visual and plotting functions. To better understand the frequency, or popularity, of the two search terms for each time period during the experi‐ ment, some baseline descriptive statistics were calculated.

Figure 7. Boxplot for NFL versus World Series time between tweets, Saturday 5 PM to 8 PM.

9

#### **5.1. Boxplots**

Boxplots are a convenient way for visualizing the interquartile range, average and outlier data. Figures 7–10 represent boxplots for the 3-hour experiment windows from Saturday 5 PM to 8 PM, Saturday 8 PM to 11 PM, Sunday 1 PM to 4 PM, and Sunday 5 PM to 8 PM, respectively.

**Figure 9.** Boxplot for NFL versus World Series time between tweets, Sunday 1 PM to 4 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

15

**Figure 10.** Boxplot for NFL versus World Series time between tweets, Sunday, 5 PM to 8 PM.

Reviewing the boxplots in Figures 7–10 reveals a couple of interesting trends. In most cases, outliers outside of the interquartile ranges exist; however, as a sporting event gets closer to start time, the frequency of tweets increase. Note the narrowing of the boxes for both the NFL

**Figure 7.** Boxplot for NFL versus World Series time between tweets, Saturday 5 PM to 8 PM.

**Figure 8.** Boxplot for NFL versus World Series time between tweets, Saturday 8 PM to 11 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 15

**Figure 9.** Boxplot for NFL versus World Series time between tweets, Sunday 1 PM to 4 PM.

**5.1. Boxplots**

14 MATLAB Applications for the Practical Engineer

Boxplots are a convenient way for visualizing the interquartile range, average and outlier data. Figures 7–10 represent boxplots for the 3-hour experiment windows from Saturday 5 PM to 8 PM, Saturday 8 PM to 11 PM, Sunday 1 PM to 4 PM, and Sunday 5 PM to 8 PM, respectively.

**Figure 7.** Boxplot for NFL versus World Series time between tweets, Saturday 5 PM to 8 PM.

**Figure 8.** Boxplot for NFL versus World Series time between tweets, Saturday 8 PM to 11 PM.

**Figure 10.** Boxplot for NFL versus World Series time between tweets, Sunday, 5 PM to 8 PM.

Reviewing the boxplots in Figures 7–10 reveals a couple of interesting trends. In most cases, outliers outside of the interquartile ranges exist; however, as a sporting event gets closer to start time, the frequency of tweets increase. Note the narrowing of the boxes for both the NFL and World Series plots during or approaching the actual game events. By Sunday late afternoon, there was significant activity in both the NFL and World Series tweets in terms of frequency. The descriptive statistics were very similar, making it difficult to see any difference, in terms of tweet frequency between NFL and World Series fans.

#### **5.2. XY plots**

MATLAB was also used to plot the time between tweets for each collection period and search query. This view, although somewhat cluttered, can be used to quickly compare the number of tweets and any other patterns obvious in an xy plot. The time between tweets is plotted in Figures 11–14, for each of the four collection periods. The most revealing information from these plots is the noticeable increase in the number of tweets as the events draw closer. On Saturday, the number of tweets related to the World Series was higher than NFL tweets. This seems reasonable since a World Series game took place on Saturday but no NFL games occurred until Sunday.

On Sunday, with multiple NFL games taking place, the number of NFL tweets was larger than World Series-related tweets. However, the number and frequency of World Series tweets was also high due to the Sunday evening World Series event, but not as significant as the NFLrelated tweet activity.

**Figure 12.** XY plot for NFL versus World Series time between tweets, Saturday 8 PM to 11 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

17

**Figure 13.** XY plot for NFL versus World Series time between tweets, Sunday 1 PM to 4PM.

**Figure 11.** XY plot for NFL versus World Series time between tweets, Saturday 5 PM to 8 PM.

**Figure 12.** XY plot for NFL versus World Series time between tweets, Saturday 8 PM to 11 PM.

and World Series plots during or approaching the actual game events. By Sunday late afternoon, there was significant activity in both the NFL and World Series tweets in terms of frequency. The descriptive statistics were very similar, making it difficult to see any difference,

MATLAB was also used to plot the time between tweets for each collection period and search query. This view, although somewhat cluttered, can be used to quickly compare the number of tweets and any other patterns obvious in an xy plot. The time between tweets is plotted in Figures 11–14, for each of the four collection periods. The most revealing information from these plots is the noticeable increase in the number of tweets as the events draw closer. On Saturday, the number of tweets related to the World Series was higher than NFL tweets. This seems reasonable since a World Series game took place on Saturday but no NFL games

On Sunday, with multiple NFL games taking place, the number of NFL tweets was larger than World Series-related tweets. However, the number and frequency of World Series tweets was also high due to the Sunday evening World Series event, but not as significant as the NFL-

**Figure 11.** XY plot for NFL versus World Series time between tweets, Saturday 5 PM to 8 PM.

in terms of tweet frequency between NFL and World Series fans.

**5.2. XY plots**

16 MATLAB Applications for the Practical Engineer

occurred until Sunday.

related tweet activity.

**Figure 13.** XY plot for NFL versus World Series time between tweets, Sunday 1 PM to 4PM.

**Figure 15.** Histogram plot for NFL versus World Series time between tweets, Saturday 5 PM to 8 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

19

**Figure 16.** Histogram plot for NFL versus World Series time between tweets, Saturday 8 PM to 11 PM.

**Figure 14.** XY plot for NFL versus World Series Time between tweets, Sunday 5 PM to 8 PM.

#### **5.3. Histograms**

The histogram functionality within MATLAB was used to get a better idea of the quantity and length of the time between tweets for each of the collection periods. As shown in Figures 15 and 16, on Saturday, the number of tweets with time difference values less than 10 seconds was much greater for World Series than NFL tweets. In all histograms, a handful of tweets had a time between tweets of greater than 100 seconds. The shift to NFL interest is very visible on Sunday in the histograms shown in Figures 17 and 18. Note the significant increase in the both the number and quantity of tweets occurring in less than 10 second intervals for NFL tweets. However, in Figure 18, the histograms for Sunday late afternoon and evening reveal similar results for the time between tweets for both NFL and World Series texts.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 19

**Figure 15.** Histogram plot for NFL versus World Series time between tweets, Saturday 5 PM to 8 PM.

**Figure 14.** XY plot for NFL versus World Series Time between tweets, Sunday 5 PM to 8 PM.

results for the time between tweets for both NFL and World Series texts.

The histogram functionality within MATLAB was used to get a better idea of the quantity and length of the time between tweets for each of the collection periods. As shown in Figures 15 and 16, on Saturday, the number of tweets with time difference values less than 10 seconds was much greater for World Series than NFL tweets. In all histograms, a handful of tweets had a time between tweets of greater than 100 seconds. The shift to NFL interest is very visible on Sunday in the histograms shown in Figures 17 and 18. Note the significant increase in the both the number and quantity of tweets occurring in less than 10 second intervals for NFL tweets. However, in Figure 18, the histograms for Sunday late afternoon and evening reveal similar

**5.3. Histograms**

18 MATLAB Applications for the Practical Engineer

**Figure 16.** Histogram plot for NFL versus World Series time between tweets, Saturday 8 PM to 11 PM.

**5.4. Scatter plots**

Twitter queries provide additional data beyond the time and text of the tweet. For example, the number of friends, number of followers, user screen name, date of account creation, and time zone of user are also available and easily extract from the queries. This information can be used to determine if any significant differences or similarities exist among users of Twitter.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

21

Within MATLAB, scatter plots can be used to visually identify clusters of data. Figures 19–22 illustrate the use of scatter plots to compare the number of friends and number of followers for NFL (blue Xs) and World Series (red circles) tweeters during each of the four data collection periods. Although a significant amount of overlap exists in the groups, particularly for small numbers of friend and followers, outliers can be identified in all four plots. Additional

variables and analysis may be needed to further isolate these groups.

**Figure 19.** Scatter plot for friends versus followers, Saturday 5 PM to 8 PM.

**Figure 17.** Histogram plot for NFL versus World Series time between tweets, Sunday 1 PM to 4 PM.

**Figure 18.** Histogram plot for NFL versus World Series time between tweets, Sunday 5 PM to 8 PM.

#### **5.4. Scatter plots**

**Figure 17.** Histogram plot for NFL versus World Series time between tweets, Sunday 1 PM to 4 PM.

20 MATLAB Applications for the Practical Engineer

**Figure 18.** Histogram plot for NFL versus World Series time between tweets, Sunday 5 PM to 8 PM.

Twitter queries provide additional data beyond the time and text of the tweet. For example, the number of friends, number of followers, user screen name, date of account creation, and time zone of user are also available and easily extract from the queries. This information can be used to determine if any significant differences or similarities exist among users of Twitter.

Within MATLAB, scatter plots can be used to visually identify clusters of data. Figures 19–22 illustrate the use of scatter plots to compare the number of friends and number of followers for NFL (blue Xs) and World Series (red circles) tweeters during each of the four data collection periods. Although a significant amount of overlap exists in the groups, particularly for small numbers of friend and followers, outliers can be identified in all four plots. Additional variables and analysis may be needed to further isolate these groups.

**Figure 19.** Scatter plot for friends versus followers, Saturday 5 PM to 8 PM.

**Figure 20.** Scatter plot for friends versus followers, Saturday 8 PM to 11 PM.

**Figure 22.** Scatter plot for friends versus followers, Sunday 5 PM to 8 PM.

Running the *k*-means algorithm in MATLAB for the Twitter feature sets further reveals the similarity in both NFL and World Series sets of tweets for all four time collection periods. Figures 23–26 illustrate the results of the applying the *k*-means algorithm for three clusters.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

23

In all cases, one very large cluster with silhouette values very close to 1 is observed. However, the other two clusters are very small with both negative and lower positive silhouette values. This indicates that the separation into unique clusters is difficult for this set of data and features. Additional selection of features and analysis would be needed to better identify clusters of

**5.5. Silhouette plots**

similar Twitter users.

**Figure 21.** Scatter plot for friends versus followers, Sunday 1 PM to 4 PM.

**Figure 22.** Scatter plot for friends versus followers, Sunday 5 PM to 8 PM.

#### **5.5. Silhouette plots**

**Figure 20.** Scatter plot for friends versus followers, Saturday 8 PM to 11 PM.

22 MATLAB Applications for the Practical Engineer

**Figure 21.** Scatter plot for friends versus followers, Sunday 1 PM to 4 PM.

Running the *k*-means algorithm in MATLAB for the Twitter feature sets further reveals the similarity in both NFL and World Series sets of tweets for all four time collection periods. Figures 23–26 illustrate the results of the applying the *k*-means algorithm for three clusters.

In all cases, one very large cluster with silhouette values very close to 1 is observed. However, the other two clusters are very small with both negative and lower positive silhouette values. This indicates that the separation into unique clusters is difficult for this set of data and features. Additional selection of features and analysis would be needed to better identify clusters of similar Twitter users.

**Figure 25.** Silhouette plot of NFL versus World Series tweet features, Sunday 1 PM to 4 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

25

**Figure 26.** Silhouette plot of NFL versus World Series tweet features, Sunday 5 PM to 8 PM.

**Figure 23.** Silhouette plot of NFL versus World Series tweet features, Saturday 5 PM to 8PM.

**Figure 24.** Silhouette plot of NFL versus World Series tweet features, Saturday 8 PM to 11 PM.

**Figure 25.** Silhouette plot of NFL versus World Series tweet features, Sunday 1 PM to 4 PM.

**Figure 23.** Silhouette plot of NFL versus World Series tweet features, Saturday 5 PM to 8PM.

24 MATLAB Applications for the Practical Engineer

**Figure 24.** Silhouette plot of NFL versus World Series tweet features, Saturday 8 PM to 11 PM.

**Figure 26.** Silhouette plot of NFL versus World Series tweet features, Sunday 5 PM to 8 PM.

#### **5.6. Hierarchical plots**

Hierarchical clustering provides an additional visualization of possible cluster separation for the tweet data. In all cases, the number of cluster and representatives within that cluster matches the *k*-means clustering results. Specifically, one large cluster was identified along with at most one or two very small clusters as illustrated in Figures 27–30 where the left portion of the tree has many nodes whereas the right portion just has one or two. The size of the cluster grows over the collection periods with the final collection period on Sunday resulting in the largest cluster with the least amount of additional clusters. For example, on Sunday, the tree's one small branch is evident on the right side of the graph with the larger branch on the left side containing most observations.

**Figure 28.** Hierarchical clustering of NFL versus World Series tweet features, Saturday 8 PM to 11 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

27

**Figure 29.** Hierarchical clustering of NFL versus World Series tweet features, Sunday 1 PM to 4 PM.

**Figure 27.** Hierarchical clustering of NFL versus World Series tweet features, Saturday 5 PM to 8 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 27

**Figure 28.** Hierarchical clustering of NFL versus World Series tweet features, Saturday 8 PM to 11 PM.

**5.6. Hierarchical plots**

26 MATLAB Applications for the Practical Engineer

side containing most observations.

Hierarchical clustering provides an additional visualization of possible cluster separation for the tweet data. In all cases, the number of cluster and representatives within that cluster matches the *k*-means clustering results. Specifically, one large cluster was identified along with at most one or two very small clusters as illustrated in Figures 27–30 where the left portion of the tree has many nodes whereas the right portion just has one or two. The size of the cluster grows over the collection periods with the final collection period on Sunday resulting in the largest cluster with the least amount of additional clusters. For example, on Sunday, the tree's one small branch is evident on the right side of the graph with the larger branch on the left

**Figure 27.** Hierarchical clustering of NFL versus World Series tweet features, Saturday 5 PM to 8 PM.

**Figure 29.** Hierarchical clustering of NFL versus World Series tweet features, Sunday 1 PM to 4 PM.

*5.7.1. SOM weight distance plots*

**1.** Neurons are represented by blue hexagons

relatively large distances between the member records.

**Figure 32.** SOM weight distances, Saturday 5 PM to 8 PM.

**3.** Dark-colored regions represent larger distances between neurons

**4.** Light-colored regions represent smaller distances between neurons

**2.** Red lines connect neighboring neurons

interpreting these plots:

The SOM neighbor weight distances plot, using multiple dimensions, provides more infor‐ mation. In Figures 32–35 SOM neighbor weight distance plots are displayed for each of the four time collections. The following diagram colors and description should be used when

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

29

Figures 32–35 show one large cluster, with small distances between the member records, present in all four collection periods. One or two very small clusters are also present, with

**Figure 30.** Hierarchical clustering of NFL versus World Series tweet features, Sunday 5 PM to 8 PM.

#### **5.7. Self-Organizing Maps (SOM) plots**

MATLAB's Neural Network Toolbox was also used to cluster the tweets through the selforganizing maps function. A hextop topology of 10 ×10 nodes was selected with 200 training epochs. After training, several visualizations are available to help determine how the input is distributed across the nodes. The locations of the data points and the weight vectors are shown by selecting the weight positions plot. With this display, only two weights can be shown at one time. Figure 31 shows that most of the data points cluster in one area and they are not very well distributed. This type of clustering was observed in both the hierarchical and *k*-means results.

**Figure 31.** SOM weight positions, Saturday 5 PM to 8 PM.

#### *5.7.1. SOM weight distance plots*

**Figure 30.** Hierarchical clustering of NFL versus World Series tweet features, Sunday 5 PM to 8 PM.

MATLAB's Neural Network Toolbox was also used to cluster the tweets through the selforganizing maps function. A hextop topology of 10 ×10 nodes was selected with 200 training epochs. After training, several visualizations are available to help determine how the input is distributed across the nodes. The locations of the data points and the weight vectors are shown by selecting the weight positions plot. With this display, only two weights can be shown at one time. Figure 31 shows that most of the data points cluster in one area and they are not very well distributed. This type of clustering was observed in both the hierarchical and *k*-means

**5.7. Self-Organizing Maps (SOM) plots**

28 MATLAB Applications for the Practical Engineer

**Figure 31.** SOM weight positions, Saturday 5 PM to 8 PM.

results.

The SOM neighbor weight distances plot, using multiple dimensions, provides more infor‐ mation. In Figures 32–35 SOM neighbor weight distance plots are displayed for each of the four time collections. The following diagram colors and description should be used when interpreting these plots:


Figures 32–35 show one large cluster, with small distances between the member records, present in all four collection periods. One or two very small clusters are also present, with relatively large distances between the member records.

**Figure 32.** SOM weight distances, Saturday 5 PM to 8 PM.

**Figure 35.** SOM weight distances, Sunday 5 PM to 8 PM.

these maps and to perform better clustering.

The SOM weight plane plots are used to visualize the strength of weights that connect each input to each of the neurons. For our experiment, five inputs were used; therefore, five subplots were generated for each experiment. The five input features included, for each tweet, the number of user mentions, URL mentions, hashtag mentions, followers, and friends. Lighter colors in the plots represent larger weights whereas darker colors represent smaller weights.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

31

Weight plane plots are shown for each of the collection times in Figures 36–39. Inputs 4 and 5 appeared to be similar in all collection times and were interpreted as highly correlated. Input from variables 1, 2, and 3 seemed to contribute the smallest amount of cluster separation in the data sets as they appear to be the least similar and less correlated. This seems reasonable because the number of friends and followers do seem to be correlated with Twitter users as a large number of friends also have a large number of followers. The information from the user-, URL-and hashtag-mentions shows some promise as the maps show these as not being highly correlated. Additional features and analysis are recommended to enhance the differences in

Similar connection patterns of the inputs indicate a high correlation.

*5.7.2. SOM weight plane plots*

**Figure 33.** SOM weight distances, Saturday 8 PM to 11 PM.

**Figure 34.** SOM weight distances, Sunday 1 PM to 4 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 31

**Figure 35.** SOM weight distances, Sunday 5 PM to 8 PM.

#### *5.7.2. SOM weight plane plots*

**Figure 33.** SOM weight distances, Saturday 8 PM to 11 PM.

30 MATLAB Applications for the Practical Engineer

**Figure 34.** SOM weight distances, Sunday 1 PM to 4 PM.

The SOM weight plane plots are used to visualize the strength of weights that connect each input to each of the neurons. For our experiment, five inputs were used; therefore, five subplots were generated for each experiment. The five input features included, for each tweet, the number of user mentions, URL mentions, hashtag mentions, followers, and friends. Lighter colors in the plots represent larger weights whereas darker colors represent smaller weights. Similar connection patterns of the inputs indicate a high correlation.

Weight plane plots are shown for each of the collection times in Figures 36–39. Inputs 4 and 5 appeared to be similar in all collection times and were interpreted as highly correlated. Input from variables 1, 2, and 3 seemed to contribute the smallest amount of cluster separation in the data sets as they appear to be the least similar and less correlated. This seems reasonable because the number of friends and followers do seem to be correlated with Twitter users as a large number of friends also have a large number of followers. The information from the user-, URL-and hashtag-mentions shows some promise as the maps show these as not being highly correlated. Additional features and analysis are recommended to enhance the differences in these maps and to perform better clustering.

**Figure 38.** SOM weight planes, Sunday 1 PM to 4 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

33

**Figure 39.** SOM weight planes, Sunday 5 PM to 8 PM.

**Figure 36.** SOM weight planes, Saturday 5 PM to 8 PM.

**Figure 37.** SOM weight planes, Saturday 8 PM to 11 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 33

**Figure 38.** SOM weight planes, Sunday 1 PM to 4 PM.

**Figure 36.** SOM weight planes, Saturday 5 PM to 8 PM.

32 MATLAB Applications for the Practical Engineer

**Figure 37.** SOM weight planes, Saturday 8 PM to 11 PM.

**Figure 39.** SOM weight planes, Sunday 5 PM to 8 PM.

#### *5.7.3. SOM sample hits plots*

One additional and useful visual plot provided by the MATLAB SOM function is the SOM sample hits plot. The sample hits plot counts the number of data records associated with each neuron. In an ideal situation, a relatively even distribution across the neurons is desired.

Figures 40–43 show the distribution across the neurons was not even in our experiment. In most cases, the distribution was clustered in one area, which indicates very similar data without much separation. Even though the location of concentration seems to shift from one area of the map to another for each time frame, the results are essentially the same for each collection time. The exception is the increase in concentration of data around the "heavyhitting" neurons on Sunday as both NFL and World Series events were scheduled.

**Figure 41.** SOM sample hits, Saturday 8 PM to 11 PM.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

35

**Figure 42.** SOM sample hits, Sunday 1 PM to 4 PM.

**Figure 40.** SOM sample hits, Saturday 5 PM to 8 PM.

**Figure 41.** SOM sample hits, Saturday 8 PM to 11 PM.

*5.7.3. SOM sample hits plots*

34 MATLAB Applications for the Practical Engineer

**Figure 40.** SOM sample hits, Saturday 5 PM to 8 PM.

One additional and useful visual plot provided by the MATLAB SOM function is the SOM sample hits plot. The sample hits plot counts the number of data records associated with each neuron. In an ideal situation, a relatively even distribution across the neurons is desired.

Figures 40–43 show the distribution across the neurons was not even in our experiment. In most cases, the distribution was clustered in one area, which indicates very similar data without much separation. Even though the location of concentration seems to shift from one area of the map to another for each time frame, the results are essentially the same for each collection time. The exception is the increase in concentration of data around the "heavy-

hitting" neurons on Sunday as both NFL and World Series events were scheduled.

**Figure 42.** SOM sample hits, Sunday 1 PM to 4 PM.

AWS provided cost savings, including all server-related hardware and software. Since MATLAB was used, the development costs for some very specialized analysis visualization tools were also reduced. Further cost savings are envisioned as MATLAB provides cloud options for running its software for short durations using a pay-as-you-go cost model.

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

37

At the time of publishing, access to MATLAB Distributed Computing Server on the cloud is available as part of an Early Adopter Program for MATLAB Distributed Computing Server on Elastic Compute Cloud (EC2). Future efforts could take advantage of this option to speed up implementations, run algorithms in parallel, and possibly further reduce licensing costs.

The twitty.m and parse\_json.m community files were successfully used to interface with the Twitter API to search for tweets related to specific queries including "NFL" and "World Series" sporting events. In addition to the tweet texts, other information used in the experiment included user data such as number of friends, number of followers, time between tweets, number of URL mentions, number of Hashtag mentions, and number of user mentions in each

The MATLAB Statistics and Neural Network Toolboxes were then used to extract and display descriptive statistics and perform unsupervised clustering using *k*-means, hierarchical, and self-organized maps. Initial analysis of the visualization and data output revealed that sports events and the associated popularity and frequency of tweets increases as the time of the event

Tweet users were also shown to have similar characteristics and profiles and would be difficult to separate based on just a few features. Referencing the SOM weight distance plots, smaller clusters were observed in all the plots, however, the distance among the members of the cluster were large compared to the one large cluster observed in each plot. Additional research with

This research demonstrated the use of MATLAB for social networking site analysis using AWS servers to reduce costs is feasible, cost-effective, and efficient. Further research is recommend‐ ed to Investigate MATLAB's AWS Parallel computing license options and costs. It is believed even more cost-savings could be realized with a pay-as-you-go model. This is particularly attractive for smaller companies and start-ups that might have limited financial resources, yet

This initial experiment collected data from four different, but relatively short, time periods for two sports-related tweets. Only a handful of features were used to identify clusters and perform statistical analysis. We recommend expanding the time frame, number of tweet queries, and features to be able to provide further insight and understanding from the

gets closer. The frequency of tweets also persisted throughout the event.

have the personnel skills to conduct some excellent research.

information available from social networking sites, such as Twitter.

larger data sets and more robust features are needed to form predicative models.

text.

**7. Recommendations**

**Figure 43.** SOM sample hits, Sunday 5 PM to 8 PM.

#### **6. Conclusion**

This research used MATLAB tools to extract and analyze social networking data sets and leverage cloud technologies and infrastructures. AWS was used to spin up a Windows 2008 server and install MATLAB and its associated toolboxes for data mining and statistics. In addition, a community MATLAB m file interfaced with the Twitter API to search specific text queries and retrieve associated data. The setup process on AWS was straightforward and provided a cost-effective and free solution for the hardware and operating system. The MATLAB license was still needed in this implementation to be able to use the toolboxes and associated m files.

AWS provided cost savings, including all server-related hardware and software. Since MATLAB was used, the development costs for some very specialized analysis visualization tools were also reduced. Further cost savings are envisioned as MATLAB provides cloud options for running its software for short durations using a pay-as-you-go cost model.

At the time of publishing, access to MATLAB Distributed Computing Server on the cloud is available as part of an Early Adopter Program for MATLAB Distributed Computing Server on Elastic Compute Cloud (EC2). Future efforts could take advantage of this option to speed up implementations, run algorithms in parallel, and possibly further reduce licensing costs.

The twitty.m and parse\_json.m community files were successfully used to interface with the Twitter API to search for tweets related to specific queries including "NFL" and "World Series" sporting events. In addition to the tweet texts, other information used in the experiment included user data such as number of friends, number of followers, time between tweets, number of URL mentions, number of Hashtag mentions, and number of user mentions in each text.

The MATLAB Statistics and Neural Network Toolboxes were then used to extract and display descriptive statistics and perform unsupervised clustering using *k*-means, hierarchical, and self-organized maps. Initial analysis of the visualization and data output revealed that sports events and the associated popularity and frequency of tweets increases as the time of the event gets closer. The frequency of tweets also persisted throughout the event.

Tweet users were also shown to have similar characteristics and profiles and would be difficult to separate based on just a few features. Referencing the SOM weight distance plots, smaller clusters were observed in all the plots, however, the distance among the members of the cluster were large compared to the one large cluster observed in each plot. Additional research with larger data sets and more robust features are needed to form predicative models.

### **7. Recommendations**

**Figure 43.** SOM sample hits, Sunday 5 PM to 8 PM.

36 MATLAB Applications for the Practical Engineer

This research used MATLAB tools to extract and analyze social networking data sets and leverage cloud technologies and infrastructures. AWS was used to spin up a Windows 2008 server and install MATLAB and its associated toolboxes for data mining and statistics. In addition, a community MATLAB m file interfaced with the Twitter API to search specific text queries and retrieve associated data. The setup process on AWS was straightforward and provided a cost-effective and free solution for the hardware and operating system. The MATLAB license was still needed in this implementation to be able to use the toolboxes and

**6. Conclusion**

associated m files.

This research demonstrated the use of MATLAB for social networking site analysis using AWS servers to reduce costs is feasible, cost-effective, and efficient. Further research is recommend‐ ed to Investigate MATLAB's AWS Parallel computing license options and costs. It is believed even more cost-savings could be realized with a pay-as-you-go model. This is particularly attractive for smaller companies and start-ups that might have limited financial resources, yet have the personnel skills to conduct some excellent research.

This initial experiment collected data from four different, but relatively short, time periods for two sports-related tweets. Only a handful of features were used to identify clusters and perform statistical analysis. We recommend expanding the time frame, number of tweet queries, and features to be able to provide further insight and understanding from the information available from social networking sites, such as Twitter.

#### **Appendix A: Example MATLAB Code**

```
% Test Twitty call account
% Clear variables
clear;
% Set up the credentials based on Twitter account
credentials.ConsumerKey = 'YourKey';
credentials.ConsumerSecret = 'Yoursecret';
credentials.AccessToken = yourAccess;
credentials.AccessTokenSecret = 'youraccesssecret';
% Create Tweet instance based on credentials
% tw = twitty(credentials);
% Create a Tweet example
% Pick Topeka Kansas for Center of Continental U.S.
% Set the last ID to Any value to start
LastID = 123456;
LastIDQ2 = 123456;
NumIts = 180;
TweetCnt=0; % Initialize to 0 counts
TweetCntQ2=0;
PauseTime = 60;
% Start a timer
Tic;
% Loop several times gathering data
for j=1:NumIts
j
% clear old data
clear S;
clear Q2;
% Time out issues establish new tw each iteration
tw = twitty(credentials);
% Run the First Query
S = tw.search('NFL', 'count',20, 'include_entities','true','geoco-
de','39.051300,-95.724660,1700mi','since_id',LastID);
% Let us see how many we got
StatCnt = length(cellfun('ndims',S{1,1}.statuses));
for i=1:StatCnt
 TweetCnt = TweetCnt+1;
 MyTweets=S{1,1}.statuses{1,i}.text ;
 % Put the data in
 Tweets{TweetCnt}=MyTweets;
 % Put the dates/time in
 Dates{TweetCnt} = S{1,1}.statuses{1,i}.created_at;
 % Gather other stuff
 FollowerCount(TweetCnt) = S{1,1}.statuses{1,i}.user.followers_count;
 FriendsCount(TweetCnt) = S{1,1}.statuses{1,i}.user.friends_count;
 TZones = S{1,1}.statuses{1,i}.user.time_zone;
 TimeZones{TweetCnt} = TZones;
 % Get screen name
 SName = S{1,1}.statuses{1,i}.user.screen_name;
```
 ScreenName{TweetCnt} = SName; % Get the User Mentions Count

% Get the lastID to eliminate Dups

% Run the World Series Query

% Let us see how many we got

TweetCntQ2 = TweetCntQ2+1;

% Put the dates/time in

% Gather other stuff

% Get screen name

tities.urls));

if (StatCntQ2 > 0)

 % Pause a few seconds pause(PauseTime);

end

end

end

TweetsQ2{TweetCntQ2}=MyTweetsQ2;

TimeZonesQ2{TweetCntQ2} = TZonesQ2;

 ScreenNameQ2{TweetCntQ2} = SNameQ2; % Get the User Mentions Count

tuses{1,i}.entities.user\_mentions));

tuses{1,i}.entities.hashtags));

% Get the lastID to eliminate Dups

LastIDQ2 = Q2{1,1}.statuses{1,1}.id\_str;

LastID = S{1,1}.statuses{1,1}.id\_str;

MyTweetsQ2=Q2{1,1}.statuses{1,i}.text ;

de','39.051300,-95.724660,1700mi','since\_id',LastIDQ2);

StatCntQ2 = length(cellfun('ndims',Q2{1,1}.statuses));

DatesQ2{TweetCntQ2} = Q2{1,1}.statuses{1,i}.created\_at;

TZonesQ2 = Q2{1,1}.statuses{1,i}.user.time\_zone;

SNameQ2 = Q2{1,1}.statuses{1,i}.user.screen\_name;

ties.user\_mentions));

tities.hashtags));

if (StatCnt > 0)

for i=1:StatCntQ2

% Put the data in

ties.urls));

end

end

UserMentionCnt{TweetCnt} = length(cellfun('ndims',S{1,1}.statuses{1,i}.enti-

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

39

URLMentionCnt{TweetCnt} = length(cellfun('ndims',S{1,1}.statuses{1,i}.enti-

HashTagsMentionCnt{TweetCnt} = length(cellfun('ndims',S{1,1}.statuses{1,i}.en-

Q2 = tw.search('World Series', 'count',20, 'include\_entities','true','geoco-

 FollowerCountQ2(TweetCntQ2) = Q2{1,1}.statuses{1,i}.user.followers\_count; FriendsCountQ2(TweetCntQ2) = Q2{1,1}.statuses{1,i}.user.friends\_count;

UserMentionCntQ2{TweetCntQ2} = length(cellfun('ndims',Q2{1,1}.sta-

URLMentionCntQ2{TweetCntQ2} = length(cellfun('ndims',Q2{1,1}.statuses{1,i}.en-

HashTagsMentionCntQ2{TweetCntQ2} = length(cellfun('ndims',Q2{1,1}.sta-

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 39

**Appendix A: Example MATLAB Code**

% Test Twitty call account

% tw = twitty(credentials); % Create a Tweet example

credentials.ConsumerKey = 'YourKey'; credentials.ConsumerSecret = 'Yoursecret'; credentials.AccessToken = yourAccess;

% Set up the credentials based on Twitter account

credentials.AccessTokenSecret = 'youraccesssecret';

% Pick Topeka Kansas for Center of Continental U.S.

% Time out issues establish new tw each iteration

de','39.051300,-95.724660,1700mi','since\_id',LastID);

StatCnt = length(cellfun('ndims',S{1,1}.statuses));

Dates{TweetCnt} = S{1,1}.statuses{1,i}.created\_at;

TZones = S{1,1}.statuses{1,i}.user.time\_zone;

SName = S{1,1}.statuses{1,i}.user.screen\_name;

 FollowerCount(TweetCnt) = S{1,1}.statuses{1,i}.user.followers\_count; FriendsCount(TweetCnt) = S{1,1}.statuses{1,i}.user.friends\_count;

S = tw.search('NFL', 'count',20, 'include\_entities','true','geoco-

% Create Tweet instance based on credentials

% Set the last ID to Any value to start

TweetCnt=0; % Initialize to 0 counts

% Loop several times gathering data

% Clear variables

38 MATLAB Applications for the Practical Engineer

LastID = 123456; LastIDQ2 = 123456; NumIts = 180;

TweetCntQ2=0; PauseTime = 60; % Start a timer

for j=1:NumIts

% clear old data

for i=1:StatCnt

% Put the data in

% Gather other stuff

% Get screen name

tw = twitty(credentials); % Run the First Query

% Let us see how many we got

 Tweets{TweetCnt}=MyTweets; % Put the dates/time in

TimeZones{TweetCnt} = TZones;

MyTweets=S{1,1}.statuses{1,i}.text ;

TweetCnt = TweetCnt+1;

Tic;

clear S; clear Q2;

j

clear;

```
 ScreenName{TweetCnt} = SName;
 % Get the User Mentions Count
 UserMentionCnt{TweetCnt} = length(cellfun('ndims',S{1,1}.statuses{1,i}.enti-
ties.user_mentions));
 URLMentionCnt{TweetCnt} = length(cellfun('ndims',S{1,1}.statuses{1,i}.enti-
ties.urls));
 HashTagsMentionCnt{TweetCnt} = length(cellfun('ndims',S{1,1}.statuses{1,i}.en-
tities.hashtags));
 end
 % Get the lastID to eliminate Dups
 if (StatCnt > 0)
 LastID = S{1,1}.statuses{1,1}.id_str;
 end
 % Run the World Series Query 
Q2 = tw.search('World Series', 'count',20, 'include_entities','true','geoco-
de','39.051300,-95.724660,1700mi','since_id',LastIDQ2);
% Let us see how many we got
StatCntQ2 = length(cellfun('ndims',Q2{1,1}.statuses));
for i=1:StatCntQ2
 TweetCntQ2 = TweetCntQ2+1;
 MyTweetsQ2=Q2{1,1}.statuses{1,i}.text ;
 % Put the data in
 TweetsQ2{TweetCntQ2}=MyTweetsQ2;
 % Put the dates/time in
 DatesQ2{TweetCntQ2} = Q2{1,1}.statuses{1,i}.created_at;
 % Gather other stuff
 FollowerCountQ2(TweetCntQ2) = Q2{1,1}.statuses{1,i}.user.followers_count;
 FriendsCountQ2(TweetCntQ2) = Q2{1,1}.statuses{1,i}.user.friends_count;
 TZonesQ2 = Q2{1,1}.statuses{1,i}.user.time_zone;
 TimeZonesQ2{TweetCntQ2} = TZonesQ2;
 % Get screen name
 SNameQ2 = Q2{1,1}.statuses{1,i}.user.screen_name;
 ScreenNameQ2{TweetCntQ2} = SNameQ2;
 % Get the User Mentions Count
 UserMentionCntQ2{TweetCntQ2} = length(cellfun('ndims',Q2{1,1}.sta-
tuses{1,i}.entities.user_mentions));
 URLMentionCntQ2{TweetCntQ2} = length(cellfun('ndims',Q2{1,1}.statuses{1,i}.en-
tities.urls));
 HashTagsMentionCntQ2{TweetCntQ2} = length(cellfun('ndims',Q2{1,1}.sta-
tuses{1,i}.entities.hashtags));
 end
 % Get the lastID to eliminate Dups
 if (StatCntQ2 > 0)
 LastIDQ2 = Q2{1,1}.statuses{1,1}.id_str;
 end 
 % Pause a few seconds
 pause(PauseTime);
end
```
#### **Appendix B: Example Data Analysis MATLAB M-File**

```
% Combine the Data into a vector
measWS = [cell2mat(UserMentionCntQ2)' cell2mat(URLMentionCntQ2)' cell2mat(Hash-
TagsMentionCntQ2)' FollowerCountQ2' FriendsCountQ2'];
measNFL= [cell2mat(UserMentionCnt)' cell2mat(URLMentionCnt)' cell2mat(HashTags-
MentionCnt)' FollowerCount' FriendsCount'];
% Combine
meas = [measWS;measNFL];
% Now let's do some Stats and Clustering
[cidx2,cmeans2] = kmeans(meas,2,'dist','sqeuclidean');
figure;
[silh2,h] = silhouette(meas,cidx2,'sqeuclidean');
% Look for maximum number of clusters
[cidx3,cmeans3] = kmeans(meas,3,'display','iter');
% Now do some clustering an
figure;
[silh3,h] = silhouette(meas,cidx3,'sqeuclidean');
% Some plots Might want to plot with NFL versus World Series
% Hierarchical Clustering
eucD = pdist(meas,'euclidean');
clustTreeEuc = linkage(eucD,'average');
myCop = cophenet(clustTreeEuc,eucD)
figure;
[h,nodes] = dendrogram(clustTreeEuc,0);
set(gca,'TickDir','out','TickLength',[.002 0],'XTickLabel',[]);
% Reduce nodes
figure;
[h,nodes] = dendrogram(clustTreeEuc,12);
toc
% World Series tweet Gaps
for i=1:length(Dates)-1
time1 = [str2num(Dates{i}(26:30)) 10 str2num(Dates{i}(9:10)) str2num(Dates{i}
(12:13)) str2num(Dates{i}(15:16)) str2num(Dates{i}(18:19))];
time2 = [str2num(Dates{i+1}(26:30)) 10 str2num(Dates{i+1}(9:10)) 
str2num(Dates{i+1}(12:13)) str2num(Dates{i+1}(15:16)) str2num(Dates{i+1}
(18:19))];
deltaWorldSeries(i) = abs(etime(time1,time2));
end
% NFLtweet gaps
for i=1:length(DatesQ2)-1
time1 = [str2num(DatesQ2{i}(26:30)) 10 str2num(DatesQ2{i}(9:10)) str2num(Da-
tesQ2{i}(12:13)) str2num(DatesQ2{i}(15:16)) str2num(DatesQ2{i}(18:19))];
time2 = [str2num(DatesQ2{i+1}(26:30)) 10 str2num(DatesQ2{i+1}(9:10)) 
str2num(DatesQ2{i+1}(12:13)) str2num(DatesQ2{i+1}(15:16)) str2num(DatesQ2{i+1}
(18:19))];
deltaNFL(i) = abs(etime(time1,time2));
end
figure (1)
subplot(2,1,1)
plot (deltaNFL,'r');
```
ylabel('Time between Tweets') xlabel ('Tweet Number')

plot(deltaWorldSeries, 'b'); ylabel('Time between Tweets') xlabel ('Tweet Number')

grid;

grid; % Stats

grid;

grid;

grid;

grid;

figure(3) subplot(1,2,1) boxplot(deltaNFL)

subplot(1,2,2)

% Group Scatter

B1 = FollowerCount; C1 = FriendsCount;

B2 = FollowerCountQ2; C2 = FriendsCountQ2;

TagMention'};

figure (2) subplot (1,2,1)

subplot(2,1,2)

xvalues = -1:2:800;

hist(deltaNFL,xvalues);

axis ([-1 200 0 200]); subplot (1,2,2)

axis ([-1 200 0 200]);

boxplot(deltaWorldSeries)

% Put everything together

MyGroups = [MyGroup1 MyGroup2]; A1 = cell2mat(HashTagsMentionCnt);

D1 = cell2mat(UserMentionCnt); E1 = cell2mat(URLMentionCnt); A2 = cell2mat(HashTagsMentionCntQ2);

D2 = cell2mat(UserMentionCntQ2); E2 = cell2mat(URLMentionCntQ2);

title('"NFL" Time between Tweets');

title('"World Series" Time between Tweets');

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

41

title ('Time between "World Series" Tweets')

title ('Time between "World Series" Tweets')

MyGroup1 = zeros(1,length(HashTagsMentionCnt)); MyGroup2 = ones(1,length(HashTagsMentionCntQ2));

MyVars = [A1' B1' C1' D1' E1'; A2' B2' C2' D2' E2'];

varNames = {'HashTagMention' 'FollowerCount' 'FriendCount' 'URLMention' 'Hash-

% Box plots for Descriptive visualization

title ('Time between "NFL" Tweets')

% Quick histograms for the 2 delta times

title ('Time between "NFL" Tweets')

hist(deltaWorldSeries,xvalues);

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon… http://dx.doi.org/10.5772/58895 41

**Appendix B: Example Data Analysis MATLAB M-File**

MentionCnt)' FollowerCount' FriendsCount'];

% Now let's do some Stats and Clustering

% Look for maximum number of clusters

clustTreeEuc = linkage(eucD,'average'); myCop = cophenet(clustTreeEuc,eucD)

[h,nodes] = dendrogram(clustTreeEuc,0);

[h,nodes] = dendrogram(clustTreeEuc,12);

deltaWorldSeries(i) = abs(etime(time1,time2));

deltaNFL(i) = abs(etime(time1,time2));

TagsMentionCntQ2)' FollowerCountQ2' FriendsCountQ2'];

[cidx2,cmeans2] = kmeans(meas,2,'dist','sqeuclidean');

[silh2,h] = silhouette(meas,cidx2,'sqeuclidean');

[cidx3,cmeans3] = kmeans(meas,3,'display','iter');

[silh3,h] = silhouette(meas,cidx3,'sqeuclidean');

% Some plots Might want to plot with NFL versus World Series

set(gca,'TickDir','out','TickLength',[.002 0],'XTickLabel',[]);

(12:13)) str2num(Dates{i}(15:16)) str2num(Dates{i}(18:19))];

time1 = [str2num(Dates{i}(26:30)) 10 str2num(Dates{i}(9:10)) str2num(Dates{i}

time1 = [str2num(DatesQ2{i}(26:30)) 10 str2num(DatesQ2{i}(9:10)) str2num(DatesQ2{i}(12:13)) str2num(DatesQ2{i}(15:16)) str2num(DatesQ2{i}(18:19))]; time2 = [str2num(DatesQ2{i+1}(26:30)) 10 str2num(DatesQ2{i+1}(9:10)) str2num(DatesQ2{i+1}(12:13)) str2num(DatesQ2{i+1}(15:16)) str2num(DatesQ2{i+1}

time2 = [str2num(Dates{i+1}(26:30)) 10 str2num(Dates{i+1}(9:10)) str2num(Dates{i+1}(12:13)) str2num(Dates{i+1}(15:16)) str2num(Dates{i+1}

measWS = [cell2mat(UserMentionCntQ2)' cell2mat(URLMentionCntQ2)' cell2mat(Hash-

measNFL= [cell2mat(UserMentionCnt)' cell2mat(URLMentionCnt)' cell2mat(HashTags-

% Combine the Data into a vector

meas = [measWS;measNFL];

% Now do some clustering an

% Hierarchical Clustering eucD = pdist(meas,'euclidean');

% World Series tweet Gaps for i=1:length(Dates)-1

for i=1:length(DatesQ2)-1

% Combine

40 MATLAB Applications for the Practical Engineer

figure;

figure;

figure;

toc

end

end figure (1) subplot(2,1,1) plot (deltaNFL,'r');

% Reduce nodes figure;

(18:19))];

(18:19))];

% NFLtweet gaps

```
ylabel('Time between Tweets')
xlabel ('Tweet Number')
title('"NFL" Time between Tweets');
grid;
subplot(2,1,2)
plot(deltaWorldSeries, 'b');
ylabel('Time between Tweets')
xlabel ('Tweet Number')
title('"World Series" Time between Tweets');
grid;
% Stats
% Quick histograms for the 2 delta times
xvalues = -1:2:800;
figure (2)
subplot (1,2,1)
hist(deltaNFL,xvalues);
grid;
title ('Time between "NFL" Tweets')
axis ([-1 200 0 200]);
subplot (1,2,2)
hist(deltaWorldSeries,xvalues);
grid;
title ('Time between "World Series" Tweets')
axis ([-1 200 0 200]);
% Box plots for Descriptive visualization
figure(3)
subplot(1,2,1)
boxplot(deltaNFL)
title ('Time between "NFL" Tweets')
grid;
subplot(1,2,2)
boxplot(deltaWorldSeries)
title ('Time between "World Series" Tweets')
grid;
% Group Scatter
% Put everything together
MyGroup1 = zeros(1,length(HashTagsMentionCnt));
MyGroup2 = ones(1,length(HashTagsMentionCntQ2));
MyGroups = [MyGroup1 MyGroup2];
A1 = cell2mat(HashTagsMentionCnt);
B1 = FollowerCount;
C1 = FriendsCount;
D1 = cell2mat(UserMentionCnt);
E1 = cell2mat(URLMentionCnt);
A2 = cell2mat(HashTagsMentionCntQ2);
B2 = FollowerCountQ2;
C2 = FriendsCountQ2;
D2 = cell2mat(UserMentionCntQ2);
E2 = cell2mat(URLMentionCntQ2);
MyVars = [A1' B1' C1' D1' E1'; A2' B2' C2' D2' E2'];
varNames = {'HashTagMention' 'FollowerCount' 'FriendCount' 'URLMention' 'Hash-
TagMention'};
```

```
figure(4)
gplotmatrix(MyVars(:,2), MyVars(:,3),MyGroups,['r' 'b'],['O' 'X'],
[],'off','hist',['Followers'],['Friends']);
grid;
```
v1.1. http://www.mathworks.com/matlabcentral/fileexchange/34837-twitty/content/

Knowledge Discovery and Information Extraction on the Open Internet Using MATLAB and Amazon…

http://dx.doi.org/10.5772/58895

43

[10] MathWorks. MATLAB Central. File Exchange. JSON Parser. JSON Parser by Joel Feenstra. 3 July 2008 (Updated 18 June 2009). http://www.mathworks.com/matlab‐

[11] MathWorks. Statistics Toolbox. Overview. http://www.mathworks.com/products/

[12] MathWorks. Neural Network Toolbox Overview. http://www.mathworks.com/prod‐

[13] Kohonen T. Self-Organizing Maps. 3rd ed. Springer Series in Information Sciences.

[14] Bennett K, Robertson J. Signal and image processing algorithm performance in a vir‐ tual and elastic computing environment, Proc. SPIE Vol. 8734, Active and Passive

[9] Twitter API. https://dev.twitter.com/docs/api/1.1 (accessed 12 July 2013).

central/fileexchange/20565-json-parser. (accessed 23 September 2013).

twitty.m. (accessed 21 September 2013).

statistics/ (accessed 1 October 2013).

Signatures IV, 87340B (2013).

ucts/neural-network/ (accessed 18 September 2013).

Springer-Verlag Heidelberg: New York; 2001.

### **Author details**

Kelly Bennett1 and James Robertson2

1 U.S. Army Research Laboratory, Sensors and Electron Devices Directorate, Adelphi, MD, USA

2 Clearhaven Technologies LLC, Severna Park, MD, USA

#### **References**


v1.1. http://www.mathworks.com/matlabcentral/fileexchange/34837-twitty/content/ twitty.m. (accessed 21 September 2013).

[9] Twitter API. https://dev.twitter.com/docs/api/1.1 (accessed 12 July 2013).

figure(4)

42 MATLAB Applications for the Practical Engineer

grid;

**Author details**

Kelly Bennett1

**References**

2012.10.02

Home 2013;7(1) 209-220.

Convention Center, CA; 2010.

September 2013).

USA

[],'off','hist',['Followers'],['Friends']);

and James Robertson2

2 Clearhaven Technologies LLC, Severna Park, MD, USA

ry\_/papers/518.pdf (accessed 29 October 2013).

works.com/products/matlab/ (accessed 8 October 2013).

gplotmatrix(MyVars(:,2), MyVars(:,3),MyGroups,['r' 'b'],['O' 'X'],

1 U.S. Army Research Laboratory, Sensors and Electron Devices Directorate, Adelphi, MD,

[1] Rahman, M. Mining social data to extract intellectual knowledge. doi:10.5815/ijisa.

[2] Das TK., Kumar P. BIG Data Analytics: A Framework for Unstructured Data Analy‐ sis. International Journal of Engineering Science and Technology 2013;5(2) 153-156. [3] Daehoon K., Daeyong K., Seungmin R., Eenjun H. Detecting Trend and Bursty Key‐ words Using Characteristics of Twitter Stream Data. International Journal of Smart

[4] Karandikar A. Clustering short status messages: A topic model based approach. Master's thesis. University of Maryland; 2010. http://ebiquity.umbc.edu/\_file\_directo‐

[5] Perera R. Twitter Analytics: Architecture, Tools and Analysis, Proc. The 2010 Military Communications Conference, MILCOM2010, Oct. 31, 2010-Nov. 3, 2010, San Jose

[6] Turner and Malleson (2011). Applying geographical clustering methods to analyze geo-located open micro-blog posts: A case study of Tweets around Leeds since June 2011, https://docs.google.com/document/d/1yaRBUwJy8Cb3JWlNfJOZZRN1JnXvpdWzhRDstqwsoI/edit?pli=1#heading=h.rx5m14o2e6yw (accessed 30

[7] MathWorks. MATLAB Products and Services. Overview. http://www.math‐

[8] MathWorks. MATLAB Central. File Exchange. twitty by Vladimir Bondarenko. 30 January 2012 (Updated 12 July 2013). Interface-class to access the Twitter REST API


**Section 2**

**Control, System, and Design Applications**

**Control, System, and Design Applications**

**Chapter 2**

**A MATLAB-based Microscope**

Additional information is available at the end of the chapter

microscope are the optical layout and computer software.

When someone intends to build a laboratorial prototype of a microscope there are two major tasks. One is in the optical side which is the selection of adequate optical and mechanical components of the optical setup taking into account budget restrictions. However the major challenge is to find the adequate overall environment that enables a easy and effective integration of the different parts of the microscope in order to arrange an efficient instrument.

This is a consequence of the great evolution in microscopy field in the last decades that naturally follows the progress in science and particularly in electronics and computer science. Currently the microscopes have little similarity with the general concept of an ancient microscope. In fact those former stand-alone microscopes that were used in biology laborato‐ ries at school have moved on to a complete instrumentation system. Different areas such as optics, mechanics, electronics and software have now to be integrated in order to get a digital image of an object. Ultimately a modern microscope is a user computer-controlled instrument.

Surely this configures an instrumentation field where it is very attractive to use MATLAB. In this chapter it is presented a practical example of MATLAB application as the fundamental tool in a three-dimensional (3D) microscopy platform. The first stage of this research project consisted on the selection of a one-dimensional (1D) array CCD/CMOS sensor and the subsequent development of the sensor readout module. Afterwards the laboratory platform has been built. Besides the sensor readout module the main components of this bench-

The choice of an efficient computer software is fundamental as the configuration of acquisition parameters, control of data acquisition, visualization of microscope images and data process‐ ing are to be performed by the user in a computer. The versatility of this platform is probably

> © 2014 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

the most important feature in order to accommodate such different tasks as:

Milton P. Macedo

**1. Introduction**

http://dx.doi.org/10.5772/58532

### **Chapter 2**

## **A MATLAB-based Microscope**

Milton P. Macedo

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/58532

### **1. Introduction**

When someone intends to build a laboratorial prototype of a microscope there are two major tasks. One is in the optical side which is the selection of adequate optical and mechanical components of the optical setup taking into account budget restrictions. However the major challenge is to find the adequate overall environment that enables a easy and effective integration of the different parts of the microscope in order to arrange an efficient instrument.

This is a consequence of the great evolution in microscopy field in the last decades that naturally follows the progress in science and particularly in electronics and computer science. Currently the microscopes have little similarity with the general concept of an ancient microscope. In fact those former stand-alone microscopes that were used in biology laborato‐ ries at school have moved on to a complete instrumentation system. Different areas such as optics, mechanics, electronics and software have now to be integrated in order to get a digital image of an object. Ultimately a modern microscope is a user computer-controlled instrument.

Surely this configures an instrumentation field where it is very attractive to use MATLAB. In this chapter it is presented a practical example of MATLAB application as the fundamental tool in a three-dimensional (3D) microscopy platform. The first stage of this research project consisted on the selection of a one-dimensional (1D) array CCD/CMOS sensor and the subsequent development of the sensor readout module. Afterwards the laboratory platform has been built. Besides the sensor readout module the main components of this benchmicroscope are the optical layout and computer software.

The choice of an efficient computer software is fundamental as the configuration of acquisition parameters, control of data acquisition, visualization of microscope images and data process‐ ing are to be performed by the user in a computer. The versatility of this platform is probably the most important feature in order to accommodate such different tasks as:

© 2014 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**•** Acquisition of data from stand-alone sensor readout module-implementation of a data communication channel from sensor readout module to the computer;

*Micro-Manager* is a complete microscopy software that include control of microscope itself and associated hardware. A long list of microscope models as well as cameras, stages and other peripherals can be controlled. It runs as a plugin to *ImageJ* and provides an easy to use software

*OME* aims at the storage and manipulation of biological data. It comprises a client-server software (*OMERO*) for visualization, management and analysis of images and a *Java* library (*Bio-Formats*) for reading and writing biological image files. This library can be used as an *ImageJ* plugin, MATLAB toolbox or in our own software. *ImageJ* is a public domain *Java* image processing program. It is a very complete image tool that can be used with many image formats

As this is a research project a fast and easy access to hardware in order to test other acquisition and control configurations would be important. Surely if a software platform is developed from the zero it is more versatile and flexible. On the other hand the graphical user interface (GUI) was adapted from the one used in the preliminary tests which had been developed in C/C++language. These two issues together with other particularities listed in table 1 were decisive to the choice of developing in MATLAB an entirely original and dedicated software

**Micro-Manager OME Original MATLAB applications**

not supported Do not perform microscope control Versatility and flexibility in hardware

Optimized to work with biological data

No previous experience in using ImageJ

The purpose of this work was to build a platform to develop and test algorithms able to obtain 3D images. The own microscope optics is based on a linear-array image sensor. At an initial stage an hardware previously developed in our research team, named PAF (*Photodiode Array Fluorometer*), had been used. After the implementation of a few and plain adjustments in the PAF software the first tests were performed. A scheme of this system is shown in figure 1.

**Table 1.** Advantages and disadvantages of using an open source microscopy software or develop original Matlab

**Disadvantages Advantages**

configuration

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 49

GUI layout previously developed in C/C++ language

Experience in programming languages / MATLAB

to control microscope experiments.

as well as raw-images.

for this project.

Sensor and stage actuators devices

Time consuming and energy investment in Micro-Manager

No previous experience in using ImageJ

**3. Description of bench-microscope**

applications for this project.


A brief reference has to be made to the use of the Borland C compiler as computer software development tool at an initial stage of this work. Namely when testing a different acquisition hardware using a different sensor and to test this sensor readout module. But next the integration of those different tasks enumerated above had to be accomplished. At this point MATLAB had been naturally considered owing to its versatility given by the functionalities from its core and toolboxes.

In this chapter the focus is the description of MATLAB applications. However for a deeper comprehension of MATLAB functionalities implemented in these applications some details of the bench-microscope prototype have to be stated. Then firstly some hardware features as the sensor readout module and object stage positioning are reported. The particularities in image build and visualization owing to the use of a linear image sensor in this bench-microscope are also covered.

On the other hand the best mode of evaluating the effectiveness of the MATLAB applications is showing the results obtained with this bench-microscope. Four applications have been developed for the implementation of image acquisition and visualization as well as for the assessment of image quality and image processing in some practical applications of this platform in materials science field.

Lastly a summary of the overall functionalities of these different MATLAB applications and a discussion of the advantages of a platform that use such a diversity of integrated tools is presented.

#### **2. Microscope implementation**

The challenge of assembling a microscope with the diversity of areas of knowledge that it demands had led to the development of open-source microscopy software. In this manner different research groups in universities as well as in industry work together in order to build software platforms that make easier the implementation of the different tasks in order to accomplish the acquisition of an image on a microscope.

Obviously the use of an open source microscopy software should be considered whenever a new microscope setup is assembled. Amongst these open source microscopy software, *Micro-Manager* and *OME* (*Open Microscopy Environment*) are probably the better established.

*Micro-Manager* is a complete microscopy software that include control of microscope itself and associated hardware. A long list of microscope models as well as cameras, stages and other peripherals can be controlled. It runs as a plugin to *ImageJ* and provides an easy to use software to control microscope experiments.

*OME* aims at the storage and manipulation of biological data. It comprises a client-server software (*OMERO*) for visualization, management and analysis of images and a *Java* library (*Bio-Formats*) for reading and writing biological image files. This library can be used as an *ImageJ* plugin, MATLAB toolbox or in our own software. *ImageJ* is a public domain *Java* image processing program. It is a very complete image tool that can be used with many image formats as well as raw-images.

As this is a research project a fast and easy access to hardware in order to test other acquisition and control configurations would be important. Surely if a software platform is developed from the zero it is more versatile and flexible. On the other hand the graphical user interface (GUI) was adapted from the one used in the preliminary tests which had been developed in C/C++language. These two issues together with other particularities listed in table 1 were decisive to the choice of developing in MATLAB an entirely original and dedicated software for this project.


**Table 1.** Advantages and disadvantages of using an open source microscopy software or develop original Matlab applications for this project.

### **3. Description of bench-microscope**

**•** Acquisition of data from stand-alone sensor readout module-implementation of a data

**•** Positioning control of the object stage – implementation of the interface of stage actuators

A brief reference has to be made to the use of the Borland C compiler as computer software development tool at an initial stage of this work. Namely when testing a different acquisition hardware using a different sensor and to test this sensor readout module. But next the integration of those different tasks enumerated above had to be accomplished. At this point MATLAB had been naturally considered owing to its versatility given by the functionalities

In this chapter the focus is the description of MATLAB applications. However for a deeper comprehension of MATLAB functionalities implemented in these applications some details of the bench-microscope prototype have to be stated. Then firstly some hardware features as the sensor readout module and object stage positioning are reported. The particularities in image build and visualization owing to the use of a linear image sensor in this bench-microscope are

On the other hand the best mode of evaluating the effectiveness of the MATLAB applications is showing the results obtained with this bench-microscope. Four applications have been developed for the implementation of image acquisition and visualization as well as for the assessment of image quality and image processing in some practical applications of this

Lastly a summary of the overall functionalities of these different MATLAB applications and a discussion of the advantages of a platform that use such a diversity of integrated tools is

The challenge of assembling a microscope with the diversity of areas of knowledge that it demands had led to the development of open-source microscopy software. In this manner different research groups in universities as well as in industry work together in order to build software platforms that make easier the implementation of the different tasks in order to

Obviously the use of an open source microscopy software should be considered whenever a new microscope setup is assembled. Amongst these open source microscopy software, *Micro-*

*Manager* and *OME* (*Open Microscopy Environment*) are probably the better established.

communication channel from sensor readout module to the computer;

**•** Development of Graphical User Interfaces (GUI) for the different applications.

**•** Visualization of data in real-time in a computer display;

**•** Development of 3D reconstruction algorithms;

to computer;

48 MATLAB Applications for the Practical Engineer

from its core and toolboxes.

platform in materials science field.

**2. Microscope implementation**

accomplish the acquisition of an image on a microscope.

also covered.

presented.

The purpose of this work was to build a platform to develop and test algorithms able to obtain 3D images. The own microscope optics is based on a linear-array image sensor. At an initial stage an hardware previously developed in our research team, named PAF (*Photodiode Array Fluorometer*), had been used. After the implementation of a few and plain adjustments in the PAF software the first tests were performed. A scheme of this system is shown in figure 1.

**3. Description of bench-microscope** 

applications for this project.

industry work together in order to build software platforms that make easier the implementation of the different tasks in

Obviously the use of an open source microscopy software should be considered whenever a new microscope setup is assembled. Amongst these open source microscopy software, *Micro-Manager* and *OME* (*Open Microscopy* 

*Micro-Manager* is a complete microscopy software that include control of microscope itself and associated hardware. A long list of microscope models as well as cameras, stages and other peripherals can be controlled. It runs as a plugin to

*OME* aims at the storage and manipulation of biological data. It comprises a client-server software (*OMERO*) for visualization, management and analysis of images and a *Java* library (*Bio-Formats*) for reading and writing biological image files. This library can be used as an *ImageJ* plugin, MATLAB toolbox or in our own software. *ImageJ* is a public domain *Java* image processing program. It is a very complete image tool that can be used with many image formats as

As this is a research project a fast and easy access to hardware in order to test other acquisition and control configurations would be important. Surely if a software platform is developed from the zero it is more versatile and flexible. On the other hand the graphical user interface (GUI) was adapted from the one used in the preliminary tests which had been developed in C/C++ language. These two issues together with other particularities listed in table 1 were

**Micro-Manager OME Original MATLAB** 

Do not perform microscope control

Optimized to work with biological data

No previous experience in using ImageJ

Table 1. Advantages and disadvantages of using an open source microscopy software or develop original Matlab

microscope optics is based on a linear-array image sensor. At an initial stage an hardware previously developed in our research team, named PAF (*Photodiode Array Fluorometer*), had been used. After the implementation of a few and plain

adjustments in the PAF software the first tests were performed. A scheme of this system is shown in figure 1.

**Disadvantages Advantages** 

**applications** 

Versatility and flexibility in hardware configuration

GUI layout previously developed in C/C++ language

Experience in programming languages / MATLAB

the overall specifications of *PIC16F876* are more adequate to the system needs namely because

readout module and stage actuators both controlled from MATLAB. The optical layout is beyond the scope of this chapter

but for clarity a brief description of sensor-readout module and positioning of the object platform is essential.

**PC**

Figure 3 shows the block diagram of the module. Besides the sensor and microcontroller it contains a RS-232 *driver* (*MAX242 from Maxim*) that receive/transmit signals from/to PIC serial port. This driver also put data in electric format of RS-232 standard and manage control signals for data communication with the computer. As serial ports have been gradually disappearing of computers in recent years there is the possibility of using an USB to RS232 adapter and

Its weakness lies in communication options. It only has a USART for RS232 communication. So sensor data is transferred to the computer through its serial port (RS232). However the optimization of the system regarding acquisition speed is not a goal of this project. Otherwise other PIC microcontroller, *PIC16C745*, with an USB serial port would be the right choice. But the overall specifications of *PIC16F876* are more adequate to the systems needs namely because of its

Figure 3 shows the block diagram of the module. Besides the sensor and microcontroller it contains a RS-232 *driver* (*MAX242 from Maxim*) that receive/transmit signals from/to PIC serial port. This driver also put data in electric format of RS-232 standard and manage control signals for data communication with the computer. As serial ports have been gradually disappearing of computers in recent years there is the possibility of using an USB to RS232 adapter and

> RS-232 Driver

Sensor data Commands

In-Circuit Serial Programming (ICSP)

**PC**

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 51

Another important functionality of this PIC microcontroller, shown in this block diagram, is called ICSP (*In-Circuit Serial Programming*). This enables an easy mode for programming the PIC, changing its configurations, namely the sensor

Another important functionality of this PIC microcontroller, shown in this block diagram, is called ICSP (*In-Circuit Serial Programming*). This enables an easy mode for programming the PIC, changing its configurations, namely the sensor readout mode, without the need of take

The ICSP functionality also enables a fast and easy configuration of the sensor operation mode taking advantage of its flexibility that comes out from CMOS technology. It is very useful as the more adequate sensor operation mode depends on the type of application in which the

The sensor readout mode normally used was the destructive Dynamic Pixel Reset (DPR). The reset of each pixel is executed as the respective readout is concluded. This assures an equal integration time for every pixel. To ensure the correct timing of the readout start of the next

The verification of timing specifications for sensor readout is ensured by the three *timers* of the PIC. Therefore the internal clock of the sensor with a duty-cycle of 50%, the readout of each

set of 1024 pixels, the PIC waits by a specific control signal from the sensor.

This stand-alone module is based on a microcontroller of PIC family (*PIC16F876*) from *Microchip*. It had been selected amongst a set of similar devices as it completely meets the predefined specifications, namely: a 10-bit ADC, three timers

of its 10-bit ADC in comparison to the 8-bit ADC in *PIC16C745*.

**RS 232** 

**RS 232** 

and an high versatility owing to an interrupt structure with thirteen interrupt sources.

Figure 2. Block diagram and a photo of the bench-microscope.

10-bit ADC in comparison to the 8-bit ADC in *PIC16C745*.

Sensor

connect this module to a more modern and powerful computer.

readout mode, without the need of take the hardware module out of optical layout.

**PIC16F876** 

3 Timers Memory ICSP

ADC

10-bit USART

Figure 3. Block diagram of the sensor readout module.

**Figure 3.** Block diagram of the sensor readout module.

the hardware module out of optical layout.

*3.1.1. Sensor readout*

bench-microscope is used.

Control signals

 **3.1. Sensor-readout module** 

Sensor

**Sensor-readout module**

> **Óptical Layout**

Three-axis Translation Stage Positioning

connect this module to a more modern and powerful computer.

**Sensor readout module** 

decisive to the choice of developing in MATLAB an entirely original and dedicated software for this project.

order to accomplish the acquisition of an image on a microscope.

Sensor and stage actuators devices not supported

Time consuming and energy investment in Micro-Manager

No previous experience in using ImageJ

*ImageJ* and provides an easy to use software to control microscope experiments.

*Environment*) are probably the better established.

well as raw-images.

Figure 1. Test system using PAF. PAFPC e PAFDET are PAF hardware modules in computer bus and with detector, respectively; OTS – Object Translation Stage; PDU – Piezoelectric Drive Unit. Owing to the excessive pixel width in the linear-array CCD used in PAF other sensor must be searched. A new sensor in **Figure 1.** Test system using PAF. PAFPC e PAFDET are PAF hardware modules in computer bus and with detector, re‐ spectively; OTS – Object Translation Stage; PDU – Piezoelectric Drive Unit.

the market (*LIS-1024 from Photon-Vision Systems*) with 1024 pixels of 7,8 μm width had been selected. Afterwards the development of a sensor readout had also enabled to assemble the sensor in the optical bench. The block diagram showing microscope architecture is presented in figure 2 as well as a photo of the optical layout showing the sensor-Owing to the excessive pixel width in the linear-array CCD used in PAF other sensor must be searched. A new sensor in the market (*LIS-1024 from Photon-Vision Systems*) with 1024 pixels of 7,8 μm width had been selected. Afterwards the development of a sensor readout had also enabled to assemble the sensor in the optical bench. The block diagram showing microscope architecture is presented in figure 2 as well as a photo of the optical layout showing the sensorreadout module and stage actuators both controlled from MATLAB. The optical layout is beyond the scope of this chapter but for clarity a brief description of sensor-readout module and positioning of the object platform is essential.

but for clarity a brief description of sensor-readout module and positioning of the object platform is essential. Figure 2. Block diagram and a photo of the bench-microscope. **Figure 2.** Block diagram and a photo of the bench-microscope.

connect this module to a more modern and powerful computer.

readout mode, without the need of take the hardware module out of optical layout.

**PIC16F876** 

3 Timers Memory ICSP

ADC

**Sensor readout module** 

10-bit USART

Figure 3. Block diagram of the sensor readout module.

Control signals

Sensor

#### **3.1. Sensor-readout module 3.1. Sensor-readout module**

This stand-alone module is based on a microcontroller of PIC family (*PIC16F876*) from *Microchip*. It had been selected amongst a set of similar devices as it completely meets the predefined specifications, namely: a 10-bit ADC, three timers and an high versatility owing to an interrupt structure with thirteen interrupt sources. Its weakness lies in communication options. It only has a USART for RS232 communication. So sensor data is transferred to the computer through its serial port (RS232). However the optimization of the system regarding acquisition This stand-alone module is based on a microcontroller of PIC family *(PIC16F876)* from Microchip. It had been selected amongst a set of similar devices as it completely meets the predefined specifications, namely: a 10-bit ADC, three timers and an high versatility owing to an interrupt structure with thirteen interrupt sources.

speed is not a goal of this project. Otherwise other PIC microcontroller, *PIC16C745*, with an USB serial port would be the

readout module and stage actuators both controlled from MATLAB. The optical layout is beyond the scope of this chapter

right choice. But the overall specifications of *PIC16F876* are more adequate to the systems needs namely because of its 10-bit ADC in comparison to the 8-bit ADC in *PIC16C745*. Figure 3 shows the block diagram of the module. Besides the sensor and microcontroller it contains a RS-232 *driver* (*MAX242 from Maxim*) that receive/transmit signals from/to PIC serial port. This driver also put data in electric format of RS-232 standard and manage control signals for data communication with the computer. As serial ports have been gradually disappearing of computers in recent years there is the possibility of using an USB to RS232 adapter and Its weakness lies in communication options. It only has a USART for RS232 communication. So sensor data is transferred to the computer through its serial port (RS232). However the optimization of the system regarding acquisition speed is not a goal of this project. Otherwise other PIC microcontroller*, PIC16C745*, with an USB serial port would be the right choice. But

> RS-232 Driver

Sensor data Commands

In-Circuit Serial Programming (ICSP)

**PC**

Another important functionality of this PIC microcontroller, shown in this block diagram, is called ICSP (*In-Circuit Serial Programming*). This enables an easy mode for programming the PIC, changing its configurations, namely the sensor the overall specifications of *PIC16F876* are more adequate to the system needs namely because of its 10-bit ADC in comparison to the 8-bit ADC in *PIC16C745*.  **3.1. Sensor-readout module**  This stand-alone module is based on a microcontroller of PIC family (*PIC16F876*) from *Microchip*. It had been selected

**PC**

**RS 232** 

**RS 232** 

Figure 3 shows the block diagram of the module. Besides the sensor and microcontroller it contains a RS-232 *driver* (*MAX242 from Maxim*) that receive/transmit signals from/to PIC serial port. This driver also put data in electric format of RS-232 standard and manage control signals for data communication with the computer. As serial ports have been gradually disappearing of computers in recent years there is the possibility of using an USB to RS232 adapter and connect this module to a more modern and powerful computer. amongst a set of similar devices as it completely meets the predefined specifications, namely: a 10-bit ADC, three timers and an high versatility owing to an interrupt structure with thirteen interrupt sources. Its weakness lies in communication options. It only has a USART for RS232 communication. So sensor data is transferred to the computer through its serial port (RS232). However the optimization of the system regarding acquisition speed is not a goal of this project. Otherwise other PIC microcontroller, *PIC16C745*, with an USB serial port would be the right choice. But the overall specifications of *PIC16F876* are more adequate to the systems needs namely because of its 10-bit ADC in comparison to the 8-bit ADC in *PIC16C745*. Figure 3 shows the block diagram of the module. Besides the sensor and microcontroller it contains a RS-232 *driver* (*MAX242 from Maxim*) that receive/transmit signals from/to PIC serial port. This driver also put data in electric format of

RS-232 standard and manage control signals for data communication with the computer. As serial ports have been gradually disappearing of computers in recent years there is the possibility of using an USB to RS232 adapter and

Figure 3. Block diagram of the sensor readout module. **Figure 3.** Block diagram of the sensor readout module.

Figure 2. Block diagram and a photo of the bench-microscope.

Sensor

**Sensor-readout module**

> **Óptical Layout**

Three-axis Translation Stage Positioning

Another important functionality of this PIC microcontroller, shown in this block diagram, is called ICSP (*In-Circuit Serial Programming*). This enables an easy mode for programming the PIC, changing its configurations, namely the sensor readout mode, without the need of take the hardware module out of optical layout. Another important functionality of this PIC microcontroller, shown in this block diagram, is called ICSP (*In-Circuit Serial Programming*). This enables an easy mode for programming the PIC, changing its configurations, namely the sensor readout mode, without the need of take the hardware module out of optical layout.

#### *3.1.1. Sensor readout*

industry work together in order to build software platforms that make easier the implementation of the different tasks in

Obviously the use of an open source microscopy software should be considered whenever a new microscope setup is assembled. Amongst these open source microscopy software, *Micro-Manager* and *OME* (*Open Microscopy* 

*Micro-Manager* is a complete microscopy software that include control of microscope itself and associated hardware. A long list of microscope models as well as cameras, stages and other peripherals can be controlled. It runs as a plugin to

*OME* aims at the storage and manipulation of biological data. It comprises a client-server software (*OMERO*) for visualization, management and analysis of images and a *Java* library (*Bio-Formats*) for reading and writing biological image files. This library can be used as an *ImageJ* plugin, MATLAB toolbox or in our own software. *ImageJ* is a public domain *Java* image processing program. It is a very complete image tool that can be used with many image formats as

As this is a research project a fast and easy access to hardware in order to test other acquisition and control configurations would be important. Surely if a software platform is developed from the zero it is more versatile and flexible. On the other hand the graphical user interface (GUI) was adapted from the one used in the preliminary tests which had been developed in C/C++ language. These two issues together with other particularities listed in table 1 were

**Micro-Manager OME Original MATLAB** 

Do not perform microscope control

Optimized to work with biological data

No previous experience in using ImageJ

Table 1. Advantages and disadvantages of using an open source microscopy software or develop original Matlab

The purpose of this work was to build a platform to develop and test algorithms able to obtain 3D images. The own microscope optics is based on a linear-array image sensor. At an initial stage an hardware previously developed in our research team, named PAF (*Photodiode Array Fluorometer*), had been used. After the implementation of a few and plain

**CCD** 

**OPTICAL** 

Figure 1. Test system using PAF. PAFPC e PAFDET are PAF hardware modules in computer bus and with detector,

**Figure 1.** Test system using PAF. PAFPC e PAFDET are PAF hardware modules in computer bus and with detector, re‐

Owing to the excessive pixel width in the linear-array CCD used in PAF other sensor must be searched. A new sensor in the market (*LIS-1024 from Photon-Vision Systems*) with 1024 pixels of 7,8 μm width had been selected. Afterwards the development of a sensor readout had also enabled to assemble the sensor in the optical bench. The block diagram showing microscope architecture is presented in figure 2 as well as a photo of the optical layout showing the sensor-

Owing to the excessive pixel width in the linear-array CCD used in PAF other sensor must be searched. A new sensor in the market (*LIS-1024 from Photon-Vision Systems*) with 1024 pixels of 7,8 μm width had been selected. Afterwards the development of a sensor readout had also enabled to assemble the sensor in the optical bench. The block diagram showing microscope architecture is presented in figure 2 as well as a photo of the optical layout showing the sensorreadout module and stage actuators both controlled from MATLAB. The optical layout is beyond the scope of this chapter but for clarity a brief description of sensor-readout module

readout module and stage actuators both controlled from MATLAB. The optical layout is beyond the scope of this chapter

This stand-alone module is based on a microcontroller of PIC family (*PIC16F876*) from *Microchip*. It had been selected amongst a set of similar devices as it completely meets the predefined specifications, namely: a 10-bit ADC, three timers

This stand-alone module is based on a microcontroller of PIC family *(PIC16F876)* from Microchip. It had been selected amongst a set of similar devices as it completely meets the predefined specifications, namely: a 10-bit ADC, three timers and an high versatility owing to

Its weakness lies in communication options. It only has a USART for RS232 communication. So sensor data is transferred to the computer through its serial port (RS232). However the optimization of the system regarding acquisition speed is not a goal of this project. Otherwise other PIC microcontroller, *PIC16C745*, with an USB serial port would be the right choice. But the overall specifications of *PIC16F876* are more adequate to the systems needs namely because of its

Figure 3 shows the block diagram of the module. Besides the sensor and microcontroller it contains a RS-232 *driver* (*MAX242 from Maxim*) that receive/transmit signals from/to PIC serial port. This driver also put data in electric format of RS-232 standard and manage control signals for data communication with the computer. As serial ports have been gradually disappearing of computers in recent years there is the possibility of using an USB to RS232 adapter and

Its weakness lies in communication options. It only has a USART for RS232 communication. So sensor data is transferred to the computer through its serial port (RS232). However the optimization of the system regarding acquisition speed is not a goal of this project. Otherwise other PIC microcontroller*, PIC16C745*, with an USB serial port would be the right choice. But

> RS-232 Driver

Sensor data Commands

In-Circuit Serial Programming (ICSP)

**PC**

Another important functionality of this PIC microcontroller, shown in this block diagram, is called ICSP (*In-Circuit Serial Programming*). This enables an easy mode for programming the PIC, changing its configurations, namely the sensor

but for clarity a brief description of sensor-readout module and positioning of the object platform is essential.

**PC**

adjustments in the PAF software the first tests were performed. A scheme of this system is shown in figure 1.

**Disadvantages Advantages** 

**applications** 

Versatility and flexibility in hardware configuration

GUI layout previously developed in C/C++ language

Experience in programming languages / MATLAB

**SYSTEM OTS PDU** 

decisive to the choice of developing in MATLAB an entirely original and dedicated software for this project.

order to accomplish the acquisition of an image on a microscope.

Sensor and stage actuators devices not supported

Time consuming and energy investment in Micro-Manager

No previous experience in using ImageJ

respectively; OTS – Object Translation Stage; PDU – Piezoelectric Drive Unit.

spectively; OTS – Object Translation Stage; PDU – Piezoelectric Drive Unit.

**PAFPC PAFDET** 

and positioning of the object platform is essential.

**RS 232** 

Figure 2. Block diagram and a photo of the bench-microscope.

**Figure 2.** Block diagram and a photo of the bench-microscope.

10-bit ADC in comparison to the 8-bit ADC in *PIC16C745*.

Sensor

an interrupt structure with thirteen interrupt sources.

connect this module to a more modern and powerful computer.

readout mode, without the need of take the hardware module out of optical layout.

**PIC16F876** 

3 Timers Memory ICSP

ADC

**Sensor readout module** 

10-bit USART

Figure 3. Block diagram of the sensor readout module.

Control signals

and an high versatility owing to an interrupt structure with thirteen interrupt sources.

**RS 232** 

 **3.1. Sensor-readout module** 

**3.1. Sensor-readout module**

Sensor

**Sensor-readout module**

> **Óptical Layout**

Three-axis Translation Stage Positioning

**3. Description of bench-microscope** 

50 MATLAB Applications for the Practical Engineer

*ImageJ* and provides an easy to use software to control microscope experiments.

*Environment*) are probably the better established.

well as raw-images.

applications for this project.

**PC**

The ICSP functionality also enables a fast and easy configuration of the sensor operation mode taking advantage of its flexibility that comes out from CMOS technology. It is very useful as the more adequate sensor operation mode depends on the type of application in which the bench-microscope is used.

The sensor readout mode normally used was the destructive Dynamic Pixel Reset (DPR). The reset of each pixel is executed as the respective readout is concluded. This assures an equal integration time for every pixel. To ensure the correct timing of the readout start of the next set of 1024 pixels, the PIC waits by a specific control signal from the sensor.

The verification of timing specifications for sensor readout is ensured by the three *timers* of the PIC. Therefore the internal clock of the sensor with a duty-cycle of 50%, the readout of each pixel data in the second half of the respective readout time window and the acquisition time in accordance with precision specifications of the ADC are easily implemented.

The acquisition cycle is based on the appropriate interrupt structure of the PIC. Figure 4 presents the timing diagram of the acquisition cycle. It is also shown the timing of data transfer to computer through RS-232 serial port.

No external memory exists in this stand-alone module. As there is no way to store the data in the module memory, each pixel value of 10 bits, the result of the ADC conversion of the analog value read from each sensor pixel, has to be sent to computer till the end of the timeslot. In this case the timeslot is the time lapse from one ADC conversion to the next one.

These 10 bits value from each pixel is packed in a frame with three words of 8 bits (bytes) as it is shown in figure 5. Thus each timeslot must be long enough for the USART complete the transfer of this frame.

Using the *Instrument Control Toolbox* the configuration of the computer serial port was performed. Preliminary tests had shown that using the maximum baud rate of 59200 bps the communication errors were very scarce. In spite of this it had been considered that a baud rate of 19200 bps was the best compromise between speed and reliability. The option had been to completely avoid these error relaxing the speed goals.

This lower baud rate imposes a rate of pixel sensor readout slightly above 1 kHz. This is achieved from a timeslot width (*Tcycle*) of nearly 900 μs. Therefore the acquisition of all 1024 sensor pixels takes around one second.

In many applications of this bench microscope it is unnecessary to perform the acquisition of the 1024 pixels. Owing to the easiness of programming the PIC that arises from the ICSP functionality described above it is plain to change the sensor readout configuration namely the timing issues in order to adequate it to the specific needs of each situation.

the computer display. Also these positioning products use a direct drive system for a simplified mechanical design with

**Figure 4.** Timing diagram of the acquisition cycle of sensor data (time intervals not in scale). Pixel acquisition time

the computer display. Also these positioning products use a direct drive system for a simplified mechanical design with

Likewise the specifications of this linear actuators in terms of resolution and repeatability comply with the purposes of the

Pixel 1 and 2 readout

Pixel 1 and 2 readout

pixel 2

pixel 1 pixel 3

frame 1 frame 2

frame 1 frame 2

Not Used

Not Used

Byte 2 Byte 3

Byte 2 Byte 3

*Tconv Tconv Tconv* 

*Tconv Tconv Tconv* 

pixel 2

pixel 1 pixel 3

*Tcycle Tcycle Tcycle*

*Tacq Tacq Tacq* 

*Tacq Tacq Tacq* 

*Ttimer1* 

*Ttimer1* 

count 1 count 2 count 3 to 1024

*Tcycle Tcycle Tcycle*

count 1 count 2 count 3 to 1024

*Ttimer1* 

*Ttimer1* 

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 53

 Pixel 3 to 1024 readout

 Pixel 3 to 1024 readout

> End Of Frame (6-bit)

End Of Frame (6-bit)

Likewise the specifications of this linear actuators in terms of resolution and repeatability comply with the purposes of the

Figure 4. Timing diagram of the acquisition cycle of sensor data (time intervals not in scale). Pixel acquisition time (*Tcycle*)

Figure 4. Timing diagram of the acquisition cycle of sensor data (time intervals not in scale). Pixel acquisition time (*Tcycle*)

The resolution (or addressability) is the distance equivalent to the smallest incremental move the device can be instructed to make. In other words, it is the linear or rotational displacement corresponding to a single microstep of

There are a wide range of positioning devices that use, e.g., stepper motors, acousto-optic deflectors (AOD), galvanometric mirrors or piezoelectric drivers. The selection of the actuators to control the positioning of the object translation stage had been based on the following issues:

The repeatability is the maximum deviation in the position of the device when attempting to return to a position after moving to a different position. The typical repeatability for T-LA actuators is about 0.3 mm. Also the typical backlash, which is the deviation of the final position that results from reversing the direction of approach, is 2.2 mm. T-LA devices have built in anti-backlash routines that do not affect motion in the positive direction (increasing absolute position, plunger extending). For negative motion, however, the device will overshoot the desired position by roughly 600

The resolution (or addressability) is the distance equivalent to the smallest incremental move the device can be instructed to make. In other words, it is the linear or rotational displacement corresponding to a single microstep of

A/D Low Byte (8-bit)

**S O F** 9 8 7 6 5 4 3 2 1 0 **E O F** 

A/D Low Byte (8-bit)

**S O F** 9 8 7 6 5 4 3 2 1 0 **E O F** 

The repeatability is the maximum deviation in the position of the device when attempting to return to a position after moving to a different position. The typical repeatability for T-LA actuators is about 0.3 mm. Also the typical backlash, which is the deviation of the final position that results from reversing the direction of approach, is 2.2 mm. T-LA devices have built in anti-backlash routines that do not affect motion in the positive direction (increasing absolute position, plunger extending). For negative motion, however, the device will overshoot the desired position by roughly 600

**•** Easiness to accommodate in the three-axis translation stage (*Melles Griot 17 AMB 003*);

bench-microscope thence the option by the linear actuator T-LA 28, with a 28 mm range.

of 888 s, A/D acquisition time (*Tacq*) of 30 s and conversion time (*Tconv*) of 20 s.

(*Tcycle*) of 888 μs, A/D acquisition time (*Tacq*) of 30 μs and conversion time (*Tconv*) of 20 μs.

bench-microscope thence the option by the linear actuator T-LA 28, with a 28 mm range.

movement. The resolution for T-LA products is 0.09921875 mm (or approximately 0.1 mm).

**•** Cost effective (nice compromise between cost and performance).

movement. The resolution for T-LA products is 0.09921875 mm (or approximately 0.1 mm).

of 888 s, A/D acquisition time (*Tacq*) of 30 s and conversion time (*Tconv*) of 20 s.

A/D High Byte (2-bit)

A/D High Byte (2-bit)

no coupling, gear, belt or other expensive components.

no coupling, gear, belt or other expensive components.

Byte 1

Byte 1

Start Of Frame (6-bit)

Start Of Frame (6-bit)

*Timer0* interrupt

A/D\_Go/Done bit Status

A/D\_Go/Done bit Status

A/D interrupt

A/D interrupt

Vo (sensor output/ A/D input)

Vo (sensor output/ A/D input)

> RS-232 data link

RS-232 data link

*Timer1* interrupt

*Timer0* interrupt *Tdelay\_timer1* 

*Tdelay\_timer1* 

*CLK*  (sensor input)

*CLK*  (sensor input)

> *Timer1* interrupt

Figure 5. Format of the *frame* used in RS-232 communication.

Figure 5. Format of the *frame* used in RS-232 communication.

**Figure 5.** Format of the *frame* used in RS-232 communication.

microsteps and return, approaching the requested position from below.

microsteps and return, approaching the requested position from below.

**3.2.1. Positioning control** 

**3.2.1. Positioning control** 

**•** Computer-controlled;

In these cases instead of wasting time for the acquisition of data without relevant information it is clearly useful the definition of a region of interest (ROI). The dimension of this ROI in image plane depends on the magnification of the objective used. One of the objective lenses in this bench microscope is 40x / 0.65 NA (numerical aperture is a number related to the width of the cone of light gathered by the lens). So the image with 1024 pixels corresponds to *200 μm* in the sample (in object plane). If in one specific application this is excessive a ROI should be de‐ fined and consequently the acquisition rate is increased. For a ROI with 256 or 128 pixels the acquisition rate is increased by a factor of 4 or 8, respectively.

#### **3.2. Positioning actuators**

This bench microscope is intended to be used in reflection mode. Its optical layout is an epiillumination configuration, typical of confocal microscopes. In this case light travelling from the light source to the sample has a fraction of its path in common to light reflected by the sample. Due to budget constraints it has been made the option by a stage-scanning instead of the beam-scanning configuration.

End Of Frame (6-bit)

Not Used

count 1 count 2 count 3 to 1024

the computer display. Also these positioning products use a direct drive system for a simplified mechanical design with no coupling, gear, belt or other expensive components. Likewise the specifications of this linear actuators in terms of resolution and repeatability comply with the purposes of the **Figure 4.** Timing diagram of the acquisition cycle of sensor data (time intervals not in scale). Pixel acquisition time (*Tcycle*) of 888 μs, A/D acquisition time (*Tacq*) of 30 μs and conversion time (*Tconv*) of 20 μs. Likewise the specifications of this linear actuators in terms of resolution and repeatability comply with the purposes of the bench-microscope thence the option by the linear actuator T-LA 28, with a 28 mm range. Figure 4. Timing diagram of the acquisition cycle of sensor data (time intervals not in scale). Pixel acquisition time (*Tcycle*)

A/D Low Byte

(8-bit) High Byte (2-bit) (6-bit) Figure 5. Format of the *frame* used in RS-232 communication. The resolution (or addressability) is the distance equivalent to the smallest incremental move the device can be **Figure 5.** Format of the *frame* used in RS-232 communication.

Start Of Frame

A/D

Figure 5. Format of the *frame* used in RS-232 communication. The resolution (or addressability) is the distance equivalent to the smallest incremental move the device can be instructed to make. In other words, it is the linear or rotational displacement corresponding to a single microstep of movement. The resolution for T-LA products is 0.09921875 mm (or approximately 0.1 mm). The repeatability is the maximum deviation in the position of the device when attempting to return to a position after movement. The resolution for T-LA products is 0.09921875 mm (or approximately 0.1 mm). The repeatability is the maximum deviation in the position of the device when attempting to return to a position after moving to a different position. The typical repeatability for T-LA actuators is about 0.3 mm. Also the typical backlash, which is the deviation of the final position that results from reversing the direction of approach, is 2.2 mm. T-LA devices have built in anti-backlash routines that do not affect motion in the positive direction (increasing absolute position, There are a wide range of positioning devices that use, e.g., stepper motors, acousto-optic deflectors (AOD), galvanometric mirrors or piezoelectric drivers. The selection of the actuators to control the positioning of the object translation stage had been based on the following issues:

instructed to make. In other words, it is the linear or rotational displacement corresponding to a single microstep of


pixel data in the second half of the respective readout time window and the acquisition time

The acquisition cycle is based on the appropriate interrupt structure of the PIC. Figure 4 presents the timing diagram of the acquisition cycle. It is also shown the timing of data transfer

No external memory exists in this stand-alone module. As there is no way to store the data in the module memory, each pixel value of 10 bits, the result of the ADC conversion of the analog value read from each sensor pixel, has to be sent to computer till the end of the timeslot. In this

These 10 bits value from each pixel is packed in a frame with three words of 8 bits (bytes) as it is shown in figure 5. Thus each timeslot must be long enough for the USART complete the

Using the *Instrument Control Toolbox* the configuration of the computer serial port was performed. Preliminary tests had shown that using the maximum baud rate of 59200 bps the communication errors were very scarce. In spite of this it had been considered that a baud rate of 19200 bps was the best compromise between speed and reliability. The option had been to

This lower baud rate imposes a rate of pixel sensor readout slightly above 1 kHz. This is achieved from a timeslot width (*Tcycle*) of nearly 900 μs. Therefore the acquisition of all 1024

In many applications of this bench microscope it is unnecessary to perform the acquisition of the 1024 pixels. Owing to the easiness of programming the PIC that arises from the ICSP functionality described above it is plain to change the sensor readout configuration namely

In these cases instead of wasting time for the acquisition of data without relevant information it is clearly useful the definition of a region of interest (ROI). The dimension of this ROI in image plane depends on the magnification of the objective used. One of the objective lenses in this bench microscope is 40x / 0.65 NA (numerical aperture is a number related to the width of the cone of light gathered by the lens). So the image with 1024 pixels corresponds to *200 μm* in the sample (in object plane). If in one specific application this is excessive a ROI should be de‐ fined and consequently the acquisition rate is increased. For a ROI with 256 or 128 pixels the

This bench microscope is intended to be used in reflection mode. Its optical layout is an epiillumination configuration, typical of confocal microscopes. In this case light travelling from the light source to the sample has a fraction of its path in common to light reflected by the sample. Due to budget constraints it has been made the option by a stage-scanning instead of

the timing issues in order to adequate it to the specific needs of each situation.

in accordance with precision specifications of the ADC are easily implemented.

case the timeslot is the time lapse from one ADC conversion to the next one.

to computer through RS-232 serial port.

52 MATLAB Applications for the Practical Engineer

completely avoid these error relaxing the speed goals.

acquisition rate is increased by a factor of 4 or 8, respectively.

sensor pixels takes around one second.

**3.2. Positioning actuators**

the beam-scanning configuration.

transfer of this frame.

**3.2.1. Positioning control •** Cost effective (nice compromise between cost and performance).

bench-microscope thence the option by the linear actuator T-LA 28, with a 28 mm range.

of 888 s, A/D acquisition time (*Tacq*) of 30 s and conversion time (*Tconv*) of 20 s.

T-series positioning products from ZaberTM and in particular linear actuators are ready to mount in the translation stage. These computer controlled positioning products use stepper motors to achieve open loop position control. These devices turn by a constant angle called a step for every electrical impulse sent to them. This allows a system to be built without feedback, reducing total system cost.

All instructions consist of a group of 6 bytes. They must be transmitted with less than 10 ms between each byte. If the unit has received less than 6 bytes and then a period of more than 10 ms passes, itignores the bytes already received. The table 2 below shows the instruction format:

The first byte is the unit number in the chain. Unit number 1 is the unit closest to the computer, unit number 2 is next and so forth. If the number 0 is used, all the units in the chain will execute

The second byte is the command number. Bytes 3, 4, 5 and 6 are data in long integer, 2's complement format with the least significant byte transmitted first. How the data bytes are

Most instructions cause the unit to reply with a return code. It is also a group of 6 bytes. The first byte is the device #. Byte 2 is the instruction just completed or 255 if an error occurs. Bytes 3, 4, 5 and 6 are data bytes in the same format as the instruction data byte. For some instructions

Therefore the communication between the computer and T-LA 28 is executed in both direc‐ tions. From the graphical user interface (GUI) of the MATLAB application the user gives an order to perform a command, e.g., reset, home, renumber, move absolute, move relative, using

The slow data communication between these linear actuators and the computer, 9600 *baud*, is one of the most important weaknesses of these actuators. Image acquisition rate is conse‐ quently low but it is assumed that in this work the concern is not to optimize this rate. The aim of this project is to obtain microscope images from which a three-dimensional reconstruction is made. Therefore with this goal in mind the easiness in the integration of this actuators in the optical layout combined with programming versatility largely compensate the imposed low

Raw-data provided in one sensor readout consists of 1024 analogue values that the A/D in microcontroller converts in 10-bit digital values. And these values compose a linear image of a region-of-interest (ROI) in the object with a length that depends on the objective magnifica‐

Usually the object plane is considered as the XY plane. Hence the optical axis which is perpendicular to this plane is the z-axis. When using a linear sensor normally the x-axis

tion. It corresponds to 200 μm and 400 μm for 40X and 20X objectives, respectively,

Data Data Data (most

significant byte)

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 55

**Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6**

significant byte)

Unit #; Command # Data (least

the accompanying command simultaneously.

in this reply it is sent the actual effective position.

the instruction format presented in table 1.

interpreted depends on the command.

**Table 2.** Instruction format.

acquisition rate.

**4. Image visualization**

However, being incremental (as opposed to absolute) in nature, the stepper motor must initially be zeroed by going to a home sensor. As there is no encoder, the actual position of the device will become different from the position shown in the computer display. Also these positioning products use a direct drive system for a simplified mechanical design with no coupling, gear, belt or other expensive components.

Likewise the specifications of this linear actuators in terms of resolution and repeatability comply with the purposes of the bench-microscope thence the option by the linear actuator T-LA 28, with a 28 mm range.

The resolution (or addressability) is the distance equivalent to the smallest incremental move the device can be instructed to make. In other words, it is the linear or rotational displacement corresponding to a single microstep of movement. The resolution for T-LA products is 0.09921875 mm (or approximately 0.1 mm).

The repeatability is the maximum deviation in the position of the device when attempting to return to a position after moving to a different position. The typical repeatability for T-LA actuators is about 0.3 mm. Also the typical backlash, which is the deviation of the final position that results from reversing the direction of approach, is 2.2 mm. T-LA devices have built in anti-backlash routines that do not affect motion in the positive direction (increasing absolute position, plunger extending). For negative motion, however, the device will overshoot the desired position by roughly 600 microsteps and return, approaching the requested position from below.

#### *3.2.1. Positioning control*

Image acquisition with this bench-microscope requires to control the positioning of the object stage. Automatic scanning in the three-axis (XYZ) is performed. One T-LA 28 actuator controlled through the RS232 port of a computer is used to implement the scanning in each axis. However the three units are connected in a daisy-chained mode thus sharing the same serial port in the computer. The configuration of the computer serial port and the control of RS232 communication was implemented in a MATLAB application.

Communications settings must be: 9600 baud, no hand shaking, no parity, one stop bit. After power-up, the units in the chain will each initialize itself as unit #1 and thus each will execute the same instructions. To assign each unit a unique identifier, a renumber instruction must be issued after all the units in the chain are powered up and every time a unit is added or removed from the chain. Instructions must not be transmitted while the chain is renumbering or the renumbering routine may be corrupted. Renumbering takes less than a second, after which instructions may start to be issued over the RS232 connection.

All instructions consist of a group of 6 bytes. They must be transmitted with less than 10 ms between each byte. If the unit has received less than 6 bytes and then a period of more than 10 ms passes, itignores the bytes already received. The table 2 below shows the instruction format:


**Table 2.** Instruction format.

T-series positioning products from ZaberTM and in particular linear actuators are ready to mount in the translation stage. These computer controlled positioning products use stepper motors to achieve open loop position control. These devices turn by a constant angle called a step for every electrical impulse sent to them. This allows a system to be built without feedback,

However, being incremental (as opposed to absolute) in nature, the stepper motor must initially be zeroed by going to a home sensor. As there is no encoder, the actual position of the device will become different from the position shown in the computer display. Also these positioning products use a direct drive system for a simplified mechanical design with no

Likewise the specifications of this linear actuators in terms of resolution and repeatability comply with the purposes of the bench-microscope thence the option by the linear actuator T-

The resolution (or addressability) is the distance equivalent to the smallest incremental move the device can be instructed to make. In other words, it is the linear or rotational displacement corresponding to a single microstep of movement. The resolution for T-LA products is

The repeatability is the maximum deviation in the position of the device when attempting to return to a position after moving to a different position. The typical repeatability for T-LA actuators is about 0.3 mm. Also the typical backlash, which is the deviation of the final position that results from reversing the direction of approach, is 2.2 mm. T-LA devices have built in anti-backlash routines that do not affect motion in the positive direction (increasing absolute position, plunger extending). For negative motion, however, the device will overshoot the desired position by roughly 600 microsteps and return, approaching the requested position

Image acquisition with this bench-microscope requires to control the positioning of the object stage. Automatic scanning in the three-axis (XYZ) is performed. One T-LA 28 actuator controlled through the RS232 port of a computer is used to implement the scanning in each axis. However the three units are connected in a daisy-chained mode thus sharing the same serial port in the computer. The configuration of the computer serial port and the control of

Communications settings must be: 9600 baud, no hand shaking, no parity, one stop bit. After power-up, the units in the chain will each initialize itself as unit #1 and thus each will execute the same instructions. To assign each unit a unique identifier, a renumber instruction must be issued after all the units in the chain are powered up and every time a unit is added or removed from the chain. Instructions must not be transmitted while the chain is renumbering or the renumbering routine may be corrupted. Renumbering takes less than a second, after which

RS232 communication was implemented in a MATLAB application.

instructions may start to be issued over the RS232 connection.

reducing total system cost.

54 MATLAB Applications for the Practical Engineer

LA 28, with a 28 mm range.

from below.

*3.2.1. Positioning control*

coupling, gear, belt or other expensive components.

0.09921875 mm (or approximately 0.1 mm).

The first byte is the unit number in the chain. Unit number 1 is the unit closest to the computer, unit number 2 is next and so forth. If the number 0 is used, all the units in the chain will execute the accompanying command simultaneously.

The second byte is the command number. Bytes 3, 4, 5 and 6 are data in long integer, 2's complement format with the least significant byte transmitted first. How the data bytes are interpreted depends on the command.

Most instructions cause the unit to reply with a return code. It is also a group of 6 bytes. The first byte is the device #. Byte 2 is the instruction just completed or 255 if an error occurs. Bytes 3, 4, 5 and 6 are data bytes in the same format as the instruction data byte. For some instructions in this reply it is sent the actual effective position.

Therefore the communication between the computer and T-LA 28 is executed in both direc‐ tions. From the graphical user interface (GUI) of the MATLAB application the user gives an order to perform a command, e.g., reset, home, renumber, move absolute, move relative, using the instruction format presented in table 1.

The slow data communication between these linear actuators and the computer, 9600 *baud*, is one of the most important weaknesses of these actuators. Image acquisition rate is conse‐ quently low but it is assumed that in this work the concern is not to optimize this rate. The aim of this project is to obtain microscope images from which a three-dimensional reconstruction is made. Therefore with this goal in mind the easiness in the integration of this actuators in the optical layout combined with programming versatility largely compensate the imposed low acquisition rate.

#### **4. Image visualization**

Raw-data provided in one sensor readout consists of 1024 analogue values that the A/D in microcontroller converts in 10-bit digital values. And these values compose a linear image of a region-of-interest (ROI) in the object with a length that depends on the objective magnifica‐ tion. It corresponds to 200 μm and 400 μm for 40X and 20X objectives, respectively,

Usually the object plane is considered as the XY plane. Hence the optical axis which is perpendicular to this plane is the z-axis. When using a linear sensor normally the x-axis represents the direction of the sensor pixels. Thus the acquisition of a two-dimensional (2D) image implies the scanning of the object on the translation-stage in the other lateral direction perpendicular to sensor (y-axis).

sampling rate in the two axes generally is different, the MATLAB function *imresize* is used. One of its parameters is exactly the ratio of sampling rate values in both axes. This parameter acts as a multiplicative factor to be used for the values in the axis with lower sampling rate. If the 20x objective is used together with the minimum scanning step in y-axis a multiplicative factor of 2,56 should be used in *imresize* function. Other parameter is the interpolation method that may be chosen from the following three: nearest-neighbor,

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 57

This process for 2D image build is illustrated in the scheme of figure 7. It is presented as example one image of the USAF resolution target used in this work for image quality assess‐

(2) (3) (4)

**Figure 7.** Diagram depicting the 2D image build in the case of the USAF resolution target. (1) Linear image (data from one sensor readout); (2) 3D representation of the intensity values in the XY plane (after the scanning along y-axis); (3) 2D image using data from sensor readout (no data processing); (4) 2D image with equal scales in both lateral axes

In some microscope applications the field-of-view from each sensor readout is insufficient. However this microscope is able to increase the field-of-view through the scanning also along

(output of *imresize* function). In (3) and (4): horizontal direction – y-axis; vertical direction – x-axis.

bilinear or bicubic.

(1)

ment.

#### **4.1. 2D image view**

Two-dimensional image build is schematically shown in figure 6. This diagram illustrates its dependence on spatial sampling rate in both lateral axes.

Sampling rate in x-axis is imposed by pixel width which is 7,8 μm. It is possible to join contiguous pixels and consequently the correspondent photoelectrons are added. This process is denominated by binning. The advantages are a faster image acquisition rate, reduced exposition time or an improved signal-to-noise ratio (SNR). However it is achieved at expense of a degradation in image resolution [1]. Spatial sampling rate in object space also depends on objective magnification. For the objectives 40x and 20x these values are 195 nm and 390 nm, respectively, without binning. Consequently putting together e.g., four contiguous pixels, the sampling rate is decreased by the same factor.

Spatial sampling in y-axis is imposed by the minimum length in XYZ stage movements in that direction. Based on the specifications of the positioning devices the option was to use the minimum scanning step of 1 μm. On the other hand in microscope applications that demand a larger field-of-view (FOV), wider scanning steps, till the maximum of 20 μm, were used at expense of image lateral resolution.

**Figure 6.** Schematic of 2D image build. Pixel width is 7,8 μm at image space. Sampling rate in y-axis at object space range from one to twenty microns.

To build a 2D image that is a faithful representation of the relative dimensions of the object in the two lateral directions it is necessary to have an equal scale in both axes. As spatial

sampling rate in the two axes generally is different, the MATLAB function *imresize* is used. One of its parameters is exactly the ratio of sampling rate values in both axes. This parameter acts as a multiplicative factor to be used for the values in the axis with lower sampling rate. If the 20x objective is used together with the minimum scanning step in y-axis a multiplicative factor of 2,56 should be used in *imresize* function. Other parameter is the interpolation method that may be chosen from the following three: nearest-neighbor, bilinear or bicubic.

represents the direction of the sensor pixels. Thus the acquisition of a two-dimensional (2D) image implies the scanning of the object on the translation-stage in the other lateral direction

Two-dimensional image build is schematically shown in figure 6. This diagram illustrates its

Sampling rate in x-axis is imposed by pixel width which is 7,8 μm. It is possible to join contiguous pixels and consequently the correspondent photoelectrons are added. This process is denominated by binning. The advantages are a faster image acquisition rate, reduced exposition time or an improved signal-to-noise ratio (SNR). However it is achieved at expense of a degradation in image resolution [1]. Spatial sampling rate in object space also depends on objective magnification. For the objectives 40x and 20x these values are 195 nm and 390 nm, respectively, without binning. Consequently putting together e.g., four contiguous pixels, the

Spatial sampling in y-axis is imposed by the minimum length in XYZ stage movements in that direction. Based on the specifications of the positioning devices the option was to use the minimum scanning step of 1 μm. On the other hand in microscope applications that demand a larger field-of-view (FOV), wider scanning steps, till the maximum of 20 μm, were used at

*image 1* 

**Figure 6.** Schematic of 2D image build. Pixel width is 7,8 μm at image space. Sampling rate in y-axis at object space

To build a 2D image that is a faithful representation of the relative dimensions of the object in the two lateral directions it is necessary to have an equal scale in both axes. As spatial

*2D image* 

*image i* 

*image n* 

perpendicular to sensor (y-axis).

56 MATLAB Applications for the Practical Engineer

dependence on spatial sampling rate in both lateral axes.

sampling rate is decreased by the same factor.

x-axis

range from one to twenty microns.

*pixel 1 pixel 1024* 

expense of image lateral resolution.

y-axis

**4.1. 2D image view**

This process for 2D image build is illustrated in the scheme of figure 7. It is presented as example one image of the USAF resolution target used in this work for image quality assess‐ ment.

**Figure 7.** Diagram depicting the 2D image build in the case of the USAF resolution target. (1) Linear image (data from one sensor readout); (2) 3D representation of the intensity values in the XY plane (after the scanning along y-axis); (3) 2D image using data from sensor readout (no data processing); (4) 2D image with equal scales in both lateral axes (output of *imresize* function). In (3) and (4): horizontal direction – y-axis; vertical direction – x-axis.

In some microscope applications the field-of-view from each sensor readout is insufficient. However this microscope is able to increase the field-of-view through the scanning also along sensor direction (x-axis). The overall width of the 1024 sensor pixels gives a total field-of-view of approximately 400 μm and 200 μm, for 40x and 20x objective lenses, respectively. In order to build the 2D image with a larger field-of-view it is necessary to use an adequate method to mount contiguous images in x-axis. The MATLAB function *montage* performs this action assuring a correct position alignment. In the overall image these transitions are undistinguish‐ able.

#### **4.2. Extraction and visualization of 3D information**

Linear images from each sensor readout of its 1024 pixels as well as 2D images obtained in result of the scanning in y-axis are blurred by light collected from reflection on points of the object at different depths. Then this blur in an ideally focused image is a result of light coming from reflection on points in planes in front or behind the focal plane.

**Figure 8.** *Best-focus* image (bonding wire). FOV: 620 μm x 400 μm (horizontal direction – y-axis; vertical direction – x-

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 59

The image acquisition as well as its visualization is configured by the computer user. Even in the early stage of this bench-microscope in which preliminary tests were performed with PAF hardware and software it was accomplished. Using PAF system as well as in the initial tests using this sensor readout module the software development tool had been Borland C. This occurred mainly because the software already developed with this tool for PAF was in an easy

However a point had been reached in which the need of improving the quality of the graphical user interface and of creating a more user-friendly interface had demanded the change to other software environment. Borland C had been a great tool to develop code for someone that has learned C language in the first step of programming learning. Nevertheless it was not directed for the development of graphical user interfaces (GUI). Consequently it was a time consuming

As already mentioned the option had been to use MATLAB. In fact it is a very complete tool as through the use of its objects it is easy and much faster to create a graphical user interface, to add or remove any functionality at any time. But in this bench-microscope there is also the necessity of implement the acquisition of sensor data as well as the control of the positioning of the object stage. Yet MATLAB covers this scope through its toolboxes namely *Instrument Control* or *Data Acquisition*. The mode how this functionalities had been easily implemented in

*CompleteGUI* had been the first MATLAB application developed for this bench-microscope. It is the more general in the sense it is used anytime the user intends to get a microscope image. Besides the initialization of the sensor and positioners the user has to define the acquisition

axis).

**5. Results**

**5.1. Image acquisition**

and fast mode applied to this new hardware module.

task and the graphical interface had not reached the desired quality.

MATLAB had been probably the major proof that it was the best choice.

parameters in terms of the scanning axes, range and steps.

Scanning optical microscopes (SOM) and particularly the confocal microscope, which is the most widely used, are suitable to get three-dimensional (3D) information. To achieve an image with 3D information the acquisition of different optical sections must be per‐ formed. It consists on images of object planes at different depths. Its acquisition is performed through the scanning along the direction of the optical axis, usually known as the axial direction.

Thus as this bench-microscope is intended for the acquisition of images with 3D information, scanning of the XYZ object translation stage is also performed along optical axis (z-axis). Spatial sampling rate in this axis is similar to the used in y-axis. Sampling intervals range from the minimum of 1 μm till 20 μm. The axial resolution depends on the numerical aperture (NA) of the objective. So the selection of the spatial sampling rate must have in consideration which objective is used, namely the 0.4 NA or 0.65 NA.

Amongst the different modes of visualization of 3D information the following two are the more usual:


A more detailed description of the methods implemented in this microscope will be presented together with the presentation of several microscope applications. However to illustrate how 3D information can be visualized and simultaneously to evidence the ability of this benchmicroscope to achieve this goal see figure 8. It is the result of the application of a particular non-automatic method. In this case the user select for each lateral position the best focus on the bonding wire amongst the overall axial positions. It is clearly distinguishable the wire inclination that departs from the pad on the integrated circuit and increases its height from the left to the right.

**Figure 8.** *Best-focus* image (bonding wire). FOV: 620 μm x 400 μm (horizontal direction – y-axis; vertical direction – xaxis).

#### **5. Results**

sensor direction (x-axis). The overall width of the 1024 sensor pixels gives a total field-of-view of approximately 400 μm and 200 μm, for 40x and 20x objective lenses, respectively. In order to build the 2D image with a larger field-of-view it is necessary to use an adequate method to mount contiguous images in x-axis. The MATLAB function *montage* performs this action assuring a correct position alignment. In the overall image these transitions are undistinguish‐

Linear images from each sensor readout of its 1024 pixels as well as 2D images obtained in result of the scanning in y-axis are blurred by light collected from reflection on points of the object at different depths. Then this blur in an ideally focused image is a result of light coming

Scanning optical microscopes (SOM) and particularly the confocal microscope, which is the most widely used, are suitable to get three-dimensional (3D) information. To achieve an image with 3D information the acquisition of different optical sections must be per‐ formed. It consists on images of object planes at different depths. Its acquisition is performed through the scanning along the direction of the optical axis, usually known as the axial

Thus as this bench-microscope is intended for the acquisition of images with 3D information, scanning of the XYZ object translation stage is also performed along optical axis (z-axis). Spatial sampling rate in this axis is similar to the used in y-axis. Sampling intervals range from the minimum of 1 μm till 20 μm. The axial resolution depends on the numerical aperture (NA) of the objective. So the selection of the spatial sampling rate must have in consideration which

Amongst the different modes of visualization of 3D information the following two are the more

**•** Auto-focus images – images with three-dimensional information as points in image are

A more detailed description of the methods implemented in this microscope will be presented together with the presentation of several microscope applications. However to illustrate how 3D information can be visualized and simultaneously to evidence the ability of this benchmicroscope to achieve this goal see figure 8. It is the result of the application of a particular non-automatic method. In this case the user select for each lateral position the best focus on the bonding wire amongst the overall axial positions. It is clearly distinguishable the wire inclination that departs from the pad on the integrated circuit and increases its height from

, *yi*

), of each point

**•** Topographic maps – a three-dimensional representation of the height, *h* (*xi*

**4.2. Extraction and visualization of 3D information**

58 MATLAB Applications for the Practical Engineer

objective is used, namely the 0.4 NA or 0.65 NA.

focused at different depths.

from reflection on points in planes in front or behind the focal plane.

able.

direction.

usual:

in XY plane.

the left to the right.

#### **5.1. Image acquisition**

The image acquisition as well as its visualization is configured by the computer user. Even in the early stage of this bench-microscope in which preliminary tests were performed with PAF hardware and software it was accomplished. Using PAF system as well as in the initial tests using this sensor readout module the software development tool had been Borland C. This occurred mainly because the software already developed with this tool for PAF was in an easy and fast mode applied to this new hardware module.

However a point had been reached in which the need of improving the quality of the graphical user interface and of creating a more user-friendly interface had demanded the change to other software environment. Borland C had been a great tool to develop code for someone that has learned C language in the first step of programming learning. Nevertheless it was not directed for the development of graphical user interfaces (GUI). Consequently it was a time consuming task and the graphical interface had not reached the desired quality.

As already mentioned the option had been to use MATLAB. In fact it is a very complete tool as through the use of its objects it is easy and much faster to create a graphical user interface, to add or remove any functionality at any time. But in this bench-microscope there is also the necessity of implement the acquisition of sensor data as well as the control of the positioning of the object stage. Yet MATLAB covers this scope through its toolboxes namely *Instrument Control* or *Data Acquisition*. The mode how this functionalities had been easily implemented in MATLAB had been probably the major proof that it was the best choice.

*CompleteGUI* had been the first MATLAB application developed for this bench-microscope. It is the more general in the sense it is used anytime the user intends to get a microscope image. Besides the initialization of the sensor and positioners the user has to define the acquisition parameters in terms of the scanning axes, range and steps.

As the sensor readout is performed with an acquisition rate of about one frame of 1024 pixels per second and it is a linear image sensor it is necessary to complete the acquisi‐ tion of a set of frames to build a two-dimensional (2D) image. It usually takes a few tens of seconds. This image is then built from a set of one-dimensional (1D) images which are also displayed in real-time. One functionality of this application is to build this 2D image giving the possibility of image visualization immediately after its acquisition is complet‐ ed. This GUI is shown in figure 9 with an example of real-time visualization of raw-data from sensor which is the 1D image.

**•** Sensor data readout

**•** Image visualization

completed **•** Data-files creation

assessment:

respectively.

mination of PSF would be implemented.

*5.2.1. Contrast measurement (determination of MTF)*

normally called contrast, has the following definition:

sensor raw-data;

**◦** Open Excel data-files

**5.2. Image quality assessment**

**◦** Store data files in the computer hard disk.

computer serial port).

**◦** Send a command to the sensor readout module to signal acquisition start (through a

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 61

**◦** Receive the sensor data, 10-bit values of the 1024 pixels (through a computer serial port).

**◦** Real-time visualization (preview) of each sensor readout, i.e., a 1D image that shows

**◦** Visualization of each 2D image as soon as the acquisition of the set of sensor data is

One of the most important tasks to be performed had been image quality assessment. After the implementation of overall system architecture that includes the optical layout it was necessary to know whether the images obtained with this bench-microscope are in good agreement with theoretical expressions. In fact the performance of a microscope is deter‐ mined by the quality of its output image. There are two different approaches for this

**•** Quantitative assessment through the determination of the modulation transfer function (MTF) and point spread function (PSF) which are a measure of contrast and resolution,

Usually only one of these two different methods is selected. It depends on the specificity of the system. Owing to practical reasons a method for quantitative assessment was used. Namely because this is a bright-field microscope in reflection mode and a new method for the deter‐

Firstly it was measured the MTF which is an usual representation of the performance of imaging systems. It is used as a measure of image contrast. The general definition of MTF is given by the ratio of modulation depth in the output and input of an optical system in function of the spatial frequency when a sinusoidal target is used as input [3]. Modulation depth (*M*),

**•** Qualitative assessment using images of specific objects with known dimensions [2]

**Figure 9.** Graphical User Interface – *CompleteGUI* – developed for configuration settings and acquisition control and visualization.

The functionalities of *CompleteGUI* application are summarized below:

	- **◦** User definition of the order of scanning in the three axes (through the GUI);
	- **◦** User definition of the region-of-interest limits in the three axes (through the GUI);
	- **◦** Send commands to positioners through a computer serial port.

**•** Sensor data readout

As the sensor readout is performed with an acquisition rate of about one frame of 1024 pixels per second and it is a linear image sensor it is necessary to complete the acquisi‐ tion of a set of frames to build a two-dimensional (2D) image. It usually takes a few tens of seconds. This image is then built from a set of one-dimensional (1D) images which are also displayed in real-time. One functionality of this application is to build this 2D image giving the possibility of image visualization immediately after its acquisition is complet‐ ed. This GUI is shown in figure 9 with an example of real-time visualization of raw-data

**Figure 9.** Graphical User Interface – *CompleteGUI* – developed for configuration settings and acquisition control and

**◦** User definition of the order of scanning in the three axes (through the GUI);

**◦** User definition of the region-of-interest limits in the three axes (through the GUI);

The functionalities of *CompleteGUI* application are summarized below:

**◦** Send commands to positioners through a computer serial port.

from sensor which is the 1D image.

60 MATLAB Applications for the Practical Engineer

visualization.

**•** Object-stage positioning

	- **◦** Real-time visualization (preview) of each sensor readout, i.e., a 1D image that shows sensor raw-data;
	- **◦** Visualization of each 2D image as soon as the acquisition of the set of sensor data is completed
	- **◦** Open Excel data-files
	- **◦** Store data files in the computer hard disk.

#### **5.2. Image quality assessment**

One of the most important tasks to be performed had been image quality assessment. After the implementation of overall system architecture that includes the optical layout it was necessary to know whether the images obtained with this bench-microscope are in good agreement with theoretical expressions. In fact the performance of a microscope is deter‐ mined by the quality of its output image. There are two different approaches for this assessment:


Usually only one of these two different methods is selected. It depends on the specificity of the system. Owing to practical reasons a method for quantitative assessment was used. Namely because this is a bright-field microscope in reflection mode and a new method for the deter‐ mination of PSF would be implemented.

#### *5.2.1. Contrast measurement (determination of MTF)*

Firstly it was measured the MTF which is an usual representation of the performance of imaging systems. It is used as a measure of image contrast. The general definition of MTF is given by the ratio of modulation depth in the output and input of an optical system in function of the spatial frequency when a sinusoidal target is used as input [3]. Modulation depth (*M*), normally called contrast, has the following definition:

$$M = \frac{I\_{\text{max}} - I\_{\text{min}}}{I\_{\text{max}} + I\_{\text{min}}},\tag{1}$$

**•** Image files

urements.

**◦** Parameter acquisition

**•** Selection of a ROI containing one line;

**◦** Store image files in JPEG format in the computer hard disk;

**◦** Determination of maximum and minimum value in that ROI;

value (usually average of maximum and minimum values).

**◦** Determination of line width given in number of pixels using a predetermined threshold

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 63

**Figure 10.** Graphical User Interface – *USAF\_image* – developed for image quality assessment namely contrast meas‐

CTF values were then calculated for each spatial frequency, using the groups of three lines in USAF target images as shown in figure 11. Different optical configurations had been used in the acquisition of these two images. Images on the left and right correspond to wide-field and line-illumination, respectively. Despite the importance of the illumination mode for the success of this bench-microscope, it is out of the scope of this chapter. Just for completeness the

comparison of image contrast on these two illumination modes will be presented.

where *Imax* and *Imin* are the maximum and minimum intensity values, respectively. It is relative either to irradiance emitted by the object or collected in the image, which are system output and input, respectively. Thus MTF definition is:

$$MTF(\xi) = \frac{M\_{output}\left(\xi\right)}{M\_{\sin\,usoidal}\left(\xi\right)},\tag{2}$$

whereξ is the spatial frequency (rigorously it is its component along sinusoidal grid direction)

From the different methods for MTF determination [4], the choice had been the scanning method. It consists on the measurement of the dependence of the contrast on spatial frequency using a sinusoidal grid as object. Although the MTF concept is applied to sinusoidal grids, for practical reasons, it is much easier to use square grids.

A widely used square grid is the USAF (*United-States Air Force*) resolution target. Then MTF determination was accomplished through contrast measurement for each group / element in images of the USAF target. This target consists on groups of three lines with different densities. The separation of the lines ranges from a minimum of approximately 2 μm (that corresponds to a maximum spatial frequency of 228 lp/mm) to tens of microns (one line pair/mm).

In this manner this method consists on the measurement of the system response to an input which is a square and not a sinusoidal grid. This is the definition similar to MTF but designated by contrast transfer function (CTF), in accordance to:

$$CTF(\xi\_f) = \frac{M\_{output} \left(\xi\_f\right)}{M\_{square} \left(\xi\_f\right)},\tag{3}$$

where*ξ f* is the fundamental spatial frequency.

To materialize this implementation the MATLAB application *USAF\_image* had been devel‐ oped. Its graphical user interface is presented in figure 10.

The functionalities of *USAF\_image* application are summarized below:

	- **◦** User definition of ROI limits in the two axes (through the GUI);
	- **◦** ROI visualization;

**•** Image files

max min max min , - <sup>=</sup> <sup>+</sup> *I I <sup>M</sup>*

where *Imax* and *Imin* are the maximum and minimum intensity values, respectively. It is relative either to irradiance emitted by the object or collected in the image, which are system output

sin ( ) ( ) ,

whereξ is the spatial frequency (rigorously it is its component along sinusoidal grid direction)

From the different methods for MTF determination [4], the choice had been the scanning method. It consists on the measurement of the dependence of the contrast on spatial frequency using a sinusoidal grid as object. Although the MTF concept is applied to sinusoidal grids, for

A widely used square grid is the USAF (*United-States Air Force*) resolution target. Then MTF determination was accomplished through contrast measurement for each group / element in images of the USAF target. This target consists on groups of three lines with different densities. The separation of the lines ranges from a minimum of approximately 2 μm (that corresponds

In this manner this method consists on the measurement of the system response to an input which is a square and not a sinusoidal grid. This is the definition similar to MTF but designated

( ) ( ) ,

*M*

<sup>=</sup> *output f*

To materialize this implementation the MATLAB application *USAF\_image* had been devel‐

*square f input*

x

*CTF*

The functionalities of *USAF\_image* application are summarized below:

**◦** User definition of ROI limits in the two axes (through the GUI);

*f*

( )

x

x

to a maximum spatial frequency of 228 lp/mm) to tens of microns (one line pair/mm).

 <sup>=</sup> *output usoidal input*

*M*

x

*MTF*

( )

x

x

and input, respectively. Thus MTF definition is:

62 MATLAB Applications for the Practical Engineer

practical reasons, it is much easier to use square grids.

by contrast transfer function (CTF), in accordance to:

where*ξ f* is the fundamental spatial frequency.

**•** Region-of-interest (ROI)

**◦** ROI visualization;

oped. Its graphical user interface is presented in figure 10.

*I I* (1)

*M* (2)

*<sup>M</sup>* (3)

	- **◦** Determination of maximum and minimum value in that ROI;
	- **◦** Determination of line width given in number of pixels using a predetermined threshold value (usually average of maximum and minimum values).

**Figure 10.** Graphical User Interface – *USAF\_image* – developed for image quality assessment namely contrast meas‐ urements.

CTF values were then calculated for each spatial frequency, using the groups of three lines in USAF target images as shown in figure 11. Different optical configurations had been used in the acquisition of these two images. Images on the left and right correspond to wide-field and line-illumination, respectively. Despite the importance of the illumination mode for the success of this bench-microscope, it is out of the scope of this chapter. Just for completeness the comparison of image contrast on these two illumination modes will be presented.

**Figure 11.** Images of USAF target showing some elements (sets of three lines) of group 7 (higher density). (a) widefield illumination; (b) line-illumination. Scale bar is 5 μm (horizontal direction – y-axis; vertical direction – x-axis).

This is also the reason why a detailed explanation of how the measured CTF (response to a square input) is converted to MTF (response to a sinusoidal input) is not presented in this chapter. The mathematical expression used for this conversion is:

$$MTF(\xi\_f) = \frac{\pi}{4} \sum\_{n=0}^{N} B\_{2n+1} \frac{CTF\left[2n + 1 \left(\xi\_f\right)\right]}{2n+1} \quad para \ \frac{\xi\_f}{2N+3} < \xi \le \frac{\xi\_f}{2N+1} \tag{4}$$

**Figure 12.** Comparison of wide-field (WF) to line-illumination (LI) for experimental data and calculated diffraction-

Images of this bench-microscope have contributions of light coming from planes in front and behind the focal plane. A mathematical model describing the image formation process is used frequently in these cases to remove this blur. From a compromise between data processing

where *I* is the image collected in sensor, *H* is the sampled PSF (matrix representing blur)

To use this model it is necessary to calculate the PSF. In this method that is performed using as object one line of USAF target. It obeys the fundamental condition that object spatial parameters are well-known. So it was applied using lines of higher spatial frequencies in the

For the implementation of this method some code was developed in MATLAB and included in *USAF\_image*. It consists on the steps presented in table 3. Owing to anisotropy imposed by the use of a linear image sensor, the PSF in x-axis and y-axis, respectively, parallel and perpendicular to sensor, are different. Thus the method was applied separately for lines in

This algorithm was applied to obtain *PSFxx* using a line along x-axis in the group 6 of USAF target. This one-dimensional function was used as image (*I*). The object (*O*), built in MATLAB,

I = + *HO b* (5)

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 65

*5.2.2. Lateral resolution measurement (new method for the determination of PSF)*

sometimes space variant, *O* is a discrete object and *b* is background light.

range from 64 to 228 lp/mm, as those shown previously in figure 11.

was a step function. Its width is 8 μm according to target specifications.

demands and required accuracy it had been used one simplified model as follows:

limited MTF.

both directions.

where *B2n+1* is equal to-*1*, *0* or *1*.

Thus the selection of the line is made in an interactive mode using *USAF\_image* GUI. When the ROI is exactly as desired the user gives the order to calculate the parameters namely *Imax* and *Imin*, already described and previously shown in equation 1. The range of spatial frequen‐ cies used for the determination of MTF is much lower than the range of frequencies the system is able to pass. However this is considered adequate in several applications [4,5,6].

Results are presented in figure 12. In wide-field and line-illumination experimental data range from 32 lp/mm to 228 lp/mm and 128 lp/mm to 228 lp/mm, respectively. Experimen‐ tal cutoff frequencies (maximum spatial frequency the system is able to pass) are 719 *lp/mm* and 975 lp/mm for wide-field and line-illumination, respectively. It had been used a 40X 0.65 NA objective lens which diffraction-limited (ideal system with no aberrations) cutoff frequency is *1136 lp/mm*.

**Figure 12.** Comparison of wide-field (WF) to line-illumination (LI) for experimental data and calculated diffractionlimited MTF.

#### *5.2.2. Lateral resolution measurement (new method for the determination of PSF)*

(a) (b)

**Figure 11.** Images of USAF target showing some elements (sets of three lines) of group 7 (higher density). (a) widefield illumination; (b) line-illumination. Scale bar is 5 μm (horizontal direction – y-axis; vertical direction – x-axis).

This is also the reason why a detailed explanation of how the measured CTF (response to a square input) is converted to MTF (response to a sinusoidal input) is not presented in this

( )

x

 x

(4)

 x

x

é ù <sup>+</sup> ë û <sup>=</sup> < £

( ) 4 21 2 3 2 1

+

Thus the selection of the line is made in an interactive mode using *USAF\_image* GUI. When the ROI is exactly as desired the user gives the order to calculate the parameters namely *Imax* and *Imin*, already described and previously shown in equation 1. The range of spatial frequen‐ cies used for the determination of MTF is much lower than the range of frequencies the system

Results are presented in figure 12. In wide-field and line-illumination experimental data range from 32 lp/mm to 228 lp/mm and 128 lp/mm to 228 lp/mm, respectively. Experimen‐ tal cutoff frequencies (maximum spatial frequency the system is able to pass) are 719 *lp/mm* and 975 lp/mm for wide-field and line-illumination, respectively. It had been used a 40X 0.65 NA objective lens which diffraction-limited (ideal system with no aberrations) cutoff

+ ++ å *<sup>N</sup> <sup>f</sup> f f*

2 1

*MTF <sup>B</sup> para n NN*

is able to pass. However this is considered adequate in several applications [4,5,6].

*CTF n*

chapter. The mathematical expression used for this conversion is:

2 1 0

p

*f n n*

=

x

64 MATLAB Applications for the Practical Engineer

where *B2n+1* is equal to-*1*, *0* or *1*.

frequency is *1136 lp/mm*.

Images of this bench-microscope have contributions of light coming from planes in front and behind the focal plane. A mathematical model describing the image formation process is used frequently in these cases to remove this blur. From a compromise between data processing demands and required accuracy it had been used one simplified model as follows:

$$\text{I} = HO + b \tag{5}$$

where *I* is the image collected in sensor, *H* is the sampled PSF (matrix representing blur) sometimes space variant, *O* is a discrete object and *b* is background light.

To use this model it is necessary to calculate the PSF. In this method that is performed using as object one line of USAF target. It obeys the fundamental condition that object spatial parameters are well-known. So it was applied using lines of higher spatial frequencies in the range from 64 to 228 lp/mm, as those shown previously in figure 11.

For the implementation of this method some code was developed in MATLAB and included in *USAF\_image*. It consists on the steps presented in table 3. Owing to anisotropy imposed by the use of a linear image sensor, the PSF in x-axis and y-axis, respectively, parallel and perpendicular to sensor, are different. Thus the method was applied separately for lines in both directions.

This algorithm was applied to obtain *PSFxx* using a line along x-axis in the group 6 of USAF target. This one-dimensional function was used as image (*I*). The object (*O*), built in MATLAB, was a step function. Its width is 8 μm according to target specifications.


**5.3. Integrated circuit and printed circuit board inspection**

dimensions namely its width and thickness.

One inspection task in the manufacturing of integrated circuits occurs after the bonding process in which a metallic wire (copper, gold or aluminum) is used to connect a silicon die to package terminals. The diameter of this bonding wire depends on the required specifications of each integrated circuit (IC). Typically it is of the order of magnitude of a few tens of microns.

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 67

Another inspection procedure is relative to quality assurance of a printed circuit board (PCB) in order to comply with customer specifications. It consists on the measurement of track

These inspection processes were performed with this bench-microscope. The graphical user

**Figure 14.** Graphical User Interface – IC\_image-developed for the inspection of bonding wire in integrated circuits.

The functionalities of *IC\_image* application are summarized below:

**◦** User definition of contrast adjustment (through the GUI)

**◦** User definition of ROI limits in the two axes (through the GUI);

**•** Image settings

interface of the developed MATLAB application, *IC\_image*, is presented in figure 14.

The PSFyy was calculated in a similar mode. In this case an image of a line along y-axis in the group 7 was used with a step function of 3.1 μm width.

For the 40X 0.65NA objective the Nyquist limit is 258 nm. Therefore the minimum sampling interval of 1 μm used in y-axis, which corresponds to the used unit space in the scanning of object stage, is above Nyquist limit. This impairs the result concerning PSFxx.

Apart from sampling issues results agree with theory in the sense that *PSFyy* is narrower than *PSFxx*. Ratio of lateral resolution in both axes is near the 1.4 factor which is the typical im‐ provement of confocal microscopy. The full width of half maximum of *PSFyy* is close to 300 nm.

To illustrate and close this subject, in figure 13 it is represented the two-dimensional point spread function in the XY plane. It had been built putting together the one-dimensional PSF curves in each axis.

**Figure 13.** Representation of two-dimensional point spread function (PSF) built from one-dimensional PSF in both xand y-axis (different scales in the two axes)

#### **5.3. Integrated circuit and printed circuit board inspection**

One inspection task in the manufacturing of integrated circuits occurs after the bonding process in which a metallic wire (copper, gold or aluminum) is used to connect a silicon die to package terminals. The diameter of this bonding wire depends on the required specifications of each integrated circuit (IC). Typically it is of the order of magnitude of a few tens of microns.

Another inspection procedure is relative to quality assurance of a printed circuit board (PCB) in order to comply with customer specifications. It consists on the measurement of track dimensions namely its width and thickness.

These inspection processes were performed with this bench-microscope. The graphical user interface of the developed MATLAB application, *IC\_image*, is presented in figure 14.

**Figure 14.** Graphical User Interface – IC\_image-developed for the inspection of bonding wire in integrated circuits.

The functionalities of *IC\_image* application are summarized below:

**•** Image settings

The PSFyy was calculated in a similar mode. In this case an image of a line along y-axis in the

For the 40X 0.65NA objective the Nyquist limit is 258 nm. Therefore the minimum sampling interval of 1 μm used in y-axis, which corresponds to the used unit space in the scanning of

Apart from sampling issues results agree with theory in the sense that *PSFyy* is narrower than *PSFxx*. Ratio of lateral resolution in both axes is near the 1.4 factor which is the typical im‐ provement of confocal microscopy. The full width of half maximum of *PSFyy* is close to 300

To illustrate and close this subject, in figure 13 it is represented the two-dimensional point spread function in the XY plane. It had been built putting together the one-dimensional PSF

**Figure 13.** Representation of two-dimensional point spread function (PSF) built from one-dimensional PSF in both x-

object stage, is above Nyquist limit. This impairs the result concerning PSFxx.

**Step # Description Function** 1. Selection of one line from the USAF target *I*(*Image*)

3. Calculation of its FFT (*Fast Fourier Transform*) *fft*(*I* −*b*) 4. Definition of correspondent theoretical line as object *O* (*Object*) 5. Calculation of FFT of the object *fft* (*O*) fft(O)

2. Subtraction of background *I* −*b* , *b* (*background*)

6. Determination of FFT of the PSF (application of model) *fft*(*H*)= *fft*(*I* −*b*) / *fft*(*O*)

7. Calculation of its inverse FFT (corresponds to PSF) *H* =*ifft*( *fft*(*H*))

group 7 was used with a step function of 3.1 μm width.

66 MATLAB Applications for the Practical Engineer

**Table 3.** Description of the algorithm executed for the determination of the PSF.

nm.

curves in each axis.

and y-axis (different scales in the two axes)

	- **◦** User selection of the set of sensor images previously acquired (through the GUI);
	- **◦** User definition of the *imresize* method and parameter (through the GUI);
	- **◦** User selection of the visualization mode (through the GUI);
	- **◦** Visualization of 2D or 3D images

#### *5.3.1. Determination of bonding wire diameter*

As it will be shown later, for the determination of wire diameter as well as PCB track dimen‐ sions it was necessary to implement algorithms for the reconstruction of 3D images. Then besides the operations for image visualization it had been necessary to include these algorithms in the *IC\_image* application.

Two of these algorithms will be presented below. Algorithm #1 is more simplified and consists on the determination of the maximum of intensity for each pixel through a stack of 2D images. Then an image, called *auto-focus*, is built using these pixel values which means that presumably the most focused XY plane is find for each pixel. Therefore these images contain 3D information through the application of:

$$I(\mathbf{x}\_i, \mathbf{y}\_i) = \mathbf{M} \overset{N}{\underset{k=1}{\mathbf{M}}} X \text{ } I(\mathbf{x}\_i, \mathbf{y}\_i, \mathbf{z}\_k) \tag{6}$$

**•** *Inda* and *indb* are adjustable and obey the requirement of −0.5≤*inda*, *indb* ≤0. After some tests, in this inspection task middle values had been used (*inda* =*indb* = −0.25). *S* is the amount

The output of this algorithm #2 provide data that may be presented through three visualization

Image in visualization mode (2) is also an *auto-focus* image as the image obtained by equation 6. These two images are shown in figure 15 for one example where six two-dimensional images (parameter S in equation 6 and equation 7) and a ROI (defined by *N* x *M* in equation 7) equal to

(a) (b)

**Figure 15.** Auto-focus images: (a) algorithm #1 (from Equation 6); (b) algorithm #2 (from Equation 7) using visualiza‐

In a very similar mode in figure 16 are presented the respective 3D maps. It means that the data shown in part (a) had been obtained from algorithm #1 and the visualization is similar to

For the determination of the gold wire diameter the following three images had been used, for

, *yi*

, *yi*

pixel values are maxima in each pixel of the processed image *Iext*(*xi*

, *yi*

, *zk* ) =*h* (*xi*

, *yi*

pixel values are maxima in each pixel of the raw-data image *I*(*xi*

) (matrix values obtained directly with the mathematical

, *zk* ) where *zk* is the position along z-axis in which the pixel values

, *yi* ).

) representing the *zk* values already described in the

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 69

, *zk* ) where *zk* is the position along z-axis in which the

, *zk* ) where *zk* is the position along z-axis in which the

) where *h* (*xi*

, *yi* ).

> , *yi* ).

, *yi*

) represents the *zk*

modes. So the user may select in *IC\_image* GUI one of the following modes:

, *yi*

, *yi*

are maxima in each pixel of the processed image, *Iext*(*xi*

, *yi*

tion mode (2). FOV: 460 μm x 210 μm (horizontal – y-axis; vertical – x-axis)

of 2D images to be used.

**1.** Processed image *Iext*(*xi*

**2.** Raw-data image *I*(*xi*

previous mode.

17 x 1 had been used.

mode (3).

comparison purposes:

**1.** Raw-data auto-focus image *I*(*xi*

**2.** Processed auto-focus image *I*(*xi*

**3.** Processed height image such as *I*(*xi*

values already described for the previous image.

expression in equation 7)

**3.** Processed topographic map *h* (*xi*

Algorithm #2 adapted from literature [7] had been also used to remove light from planes in front and behind the focal plane. It is based on the assumption that ideally at the focal plane the light is focused on one point and as it departs from that plane it spreads over an increasing area. The mathematical expressions are:

$$I\_{ex}(\mathbf{x}\_i, \mathbf{y}\_i) = \max\_{k=1}^{S} X \left[ \frac{\sum\_{a=-(N-1)/2}^{(N-1)/2} \sum\_{b=-(M-1)/2}^{(M-1)/2} \left( I\_m(\mathbf{x}\_i, \mathbf{y}\_i, \mathbf{z}\_k) - I(\mathbf{x}\_i + a, \mathbf{y}\_i + b, \mathbf{z}\_k) \right)^2}{I\_m^{-2}(\mathbf{x}\_i, \mathbf{y}\_i, \mathbf{z}\_k)} \right] \tag{7}$$
 
$$I\_m(\mathbf{x}\_i, \mathbf{y}\_i, \mathbf{z}\_k) = \frac{1}{(N \cdot M)} \sum\_{a=-(N-1)/2}^{(N-1)/2} \sum\_{b=-(M-1)/2}^{(M-1)/2} I(\mathbf{x}\_i + a, \mathbf{y}\_i + b, \mathbf{z}\_k)$$

where:

**•** *N* and *M* are spatially parameters used for the definition of a ROI to be considered around each pixel. These values had been established from specific geometrical and sampling parameters of this bench-microscope. *N* and *M* represent the number of pixels and linear images, respectively.

**•** *Inda* and *indb* are adjustable and obey the requirement of −0.5≤*inda*, *indb* ≤0. After some tests, in this inspection task middle values had been used (*inda* =*indb* = −0.25). *S* is the amount of 2D images to be used.

**•** Image visualization

68 MATLAB Applications for the Practical Engineer

in the *IC\_image* application.

through the application of:

( )

images, respectively.

*ext i i*

where:

=

area. The mathematical expressions are:

1

=

*MAX*

**◦** Visualization of 2D or 3D images

*5.3.1. Determination of bonding wire diameter*

**◦** User selection of the set of sensor images previously acquired (through the GUI);

As it will be shown later, for the determination of wire diameter as well as PCB track dimen‐ sions it was necessary to implement algorithms for the reconstruction of 3D images. Then besides the operations for image visualization it had been necessary to include these algorithms

Two of these algorithms will be presented below. Algorithm #1 is more simplified and consists on the determination of the maximum of intensity for each pixel through a stack of 2D images. Then an image, called *auto-focus*, is built using these pixel values which means that presumably the most focused XY plane is find for each pixel. Therefore these images contain 3D information

> ( ) ( ) 1 , , , = <sup>=</sup> *<sup>N</sup> i i iik k*

> > ( )

( ) ( ) ( )

=- - =- -

<sup>1</sup> , , , ,

*N M m iik i ik aN bM*


å å

( )

*I xyz I x ay bz N M*

( )

( )

( )

1 /2 1 /2

1 /2 1 /2

<sup>=</sup> + + <sup>×</sup>

**•** *N* and *M* are spatially parameters used for the definition of a ROI to be considered around each pixel. These values had been established from specific geometrical and sampling parameters of this bench-microscope. *N* and *M* represent the number of pixels and linear

2

*m iik i i k <sup>S</sup> aN bM*

( )

1 /2 1 /2

*k m iik*

1 /2 1 /2


*N M*

å å

( )

*I xy I xyz*

, , ,

=- - =- -

( )

Algorithm #2 adapted from literature [7] had been also used to remove light from planes in front and behind the focal plane. It is based on the assumption that ideally at the focal plane the light is focused on one point and as it departs from that plane it spreads over an increasing

*Ixy Ixyz MAX* (6)

( ( ) ( ))

*I x y z I x ay bz*

,, , ,

2

(7)

( )

é ù ê ú -++

ë û

**◦** User definition of the *imresize* method and parameter (through the GUI);

**◦** User selection of the visualization mode (through the GUI);

The output of this algorithm #2 provide data that may be presented through three visualization modes. So the user may select in *IC\_image* GUI one of the following modes:


Image in visualization mode (2) is also an *auto-focus* image as the image obtained by equation 6. These two images are shown in figure 15 for one example where six two-dimensional images (parameter S in equation 6 and equation 7) and a ROI (defined by *N* x *M* in equation 7) equal to 17 x 1 had been used.

**Figure 15.** Auto-focus images: (a) algorithm #1 (from Equation 6); (b) algorithm #2 (from Equation 7) using visualiza‐ tion mode (2). FOV: 460 μm x 210 μm (horizontal – y-axis; vertical – x-axis)

In a very similar mode in figure 16 are presented the respective 3D maps. It means that the data shown in part (a) had been obtained from algorithm #1 and the visualization is similar to mode (3).

For the determination of the gold wire diameter the following three images had been used, for comparison purposes:


(1) and (2) it calculates the number of pixels with intensity lower, and in image (3) higher, than

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 71

Using a threshold value of 50% it corresponds to the calculation of FWHm (Full Width at Half minimum) and FWHM, respectively. A geometric correction factor should be used as the wire

The results of the application of this algorithm on the three images are presented in table 4. Besides average values of wire diameter its standard deviation was also calculated. The diameter from image (2) is considerably larger than from the other two images. The definition of a threshold different than 50% would mitigate this difference induced by lower image

(a) (c)

(b) (d)

**Figure 17.** (a) Complete image (3). FOV: 460 μm x 210 μm. (b) to (d) images of the ROI extracted from images (1) to (3), respectively. FOV: 220 μm x 117 μm (horizontal – y-axis; vertical – x-axis). Image in part (a) uses a different scale.

> (1) 43,06 4,51 (2) 61,78 6,83 (3) 36,24 7,97

**Table 4.** Wire diameter average and standard deviation values calculated from images (1) to (3) using the ROIs shown

Both track orientations, parallel and perpendicular to sensor, had been used. However it was accomplished using a very similar method. For illustration it is presented the parallel one in

**Wire diameter (μm) Average Std. deviation**

is not horizontal in image because it is not perpendicular to sensor.

a pre-defined threshold.

contrast in this particular image.

**Image**

in figure 17. (a) to (c), respectively.

*5.3.2. Determination of width and thickness of PCB tracks*

which twelve two-dimensional images had been acquired.

**Figure 16.** 3D maps achieved using visualization mode (3): (a) algorithm #1; (b) algorithm #2. Dashed oval in part (a) surrounds a linear structure with the same orientation of the structure that is extremely clear in part (b).

In this image (3) shown in figure 17. (a) as well as in images (1) and (2) both shown in figure 15 a ROI with 300 pixels (117 μm) and twelve linear images (220 μm) had been defined. This ROI should contain a part of the wire which is away from the package terminal. In this manner the diameter measurement is not affected by the stress imposed in the wire to make the connection.

However this implies that the MATLAB application must use a 3D reconstruction algorithm to obtain a focused image of this part of the wire because its height is not constant. This ROI in the three images (1) to (3) is shown in figure 17 (b) to (d).

The aim is to develop a method that for each linear image in the ROI is able to find the amount of pixels that belong to wire. So this algorithm is adjustable depending on the image. For images (1) and (2) it calculates the number of pixels with intensity lower, and in image (3) higher, than a pre-defined threshold.

Using a threshold value of 50% it corresponds to the calculation of FWHm (Full Width at Half minimum) and FWHM, respectively. A geometric correction factor should be used as the wire is not horizontal in image because it is not perpendicular to sensor.

The results of the application of this algorithm on the three images are presented in table 4. Besides average values of wire diameter its standard deviation was also calculated. The diameter from image (2) is considerably larger than from the other two images. The definition of a threshold different than 50% would mitigate this difference induced by lower image contrast in this particular image.

**Figure 17.** (a) Complete image (3). FOV: 460 μm x 210 μm. (b) to (d) images of the ROI extracted from images (1) to (3), respectively. FOV: 220 μm x 117 μm (horizontal – y-axis; vertical – x-axis). Image in part (a) uses a different scale.


**Table 4.** Wire diameter average and standard deviation values calculated from images (1) to (3) using the ROIs shown in figure 17. (a) to (c), respectively.

#### *5.3.2. Determination of width and thickness of PCB tracks*

**Figure 16.** 3D maps achieved using visualization mode (3): (a) algorithm #1; (b) algorithm #2. Dashed oval in part (a)

In this image (3) shown in figure 17. (a) as well as in images (1) and (2) both shown in figure 15 a ROI with 300 pixels (117 μm) and twelve linear images (220 μm) had been defined. This ROI should contain a part of the wire which is away from the package terminal. In this manner the diameter measurement is not affected by the stress imposed in the wire to make the

However this implies that the MATLAB application must use a 3D reconstruction algorithm to obtain a focused image of this part of the wire because its height is not constant. This ROI

The aim is to develop a method that for each linear image in the ROI is able to find the amount of pixels that belong to wire. So this algorithm is adjustable depending on the image. For images

surrounds a linear structure with the same orientation of the structure that is extremely clear in part (b).

in the three images (1) to (3) is shown in figure 17 (b) to (d).

connection.

70 MATLAB Applications for the Practical Engineer

Both track orientations, parallel and perpendicular to sensor, had been used. However it was accomplished using a very similar method. For illustration it is presented the parallel one in which twelve two-dimensional images had been acquired.

Using the equation 6 it was calculated the *auto-focus* image, designated previously by image (1). This image is shown in figure 18. Also the topographic (3D) map *h* (*xi* , *yi* ) representing the *zk* values that correspond to the position along z-axis in which the pixel values are maxima in each pixel of the raw-data image *I*(*xi* , *yi* ). To represent a two-dimensional (2D) profile it was selected the pixel *n* of the sensor and *h* (*xn*, *yi* ) is the height of that pixel *n* for each sensor line.

**5.** Definition of other ROI including the complete field of view along y-axis (perpendicular

**6.** Calculation of the number of lines (position in y-axis) in which the height lays in the range

**7.** Determination of track width mean and standard deviation expressed in number of lines;

is easily calculated through the determination of the height of the exterior side plane and its subsequent subtraction. This calculation had been made for a track perpendicular to sensor but this example is not covered in this chapter. Table 5 shows the results of the implementation

**Track parameter Mean Std. deviation**

Height 4,2890\* 0,5963\*

**Table 5.** Mean and standard deviation values of the track height and width obtained using a ROI inside the track and

Owing to its depth discrimination ability, optical techniques as confocal microscopy have been used in profilometry. It consists on build three-dimensional profiles in order to measure different surface or geometrical dimensions such as roughness, height / depth or width.

Another MATLAB application, *Profilometry\_SiFrame*, which GUI is presented in figure 19 had

**◦** User selection of the set of sensor images previously acquired (through the GUI);

**◦** User definition of the scanning sequence to be used in 2D visualization (through the GUI);

**◦** User selection of the visualization mode (2D or 3D) (through the GUI);

**◦** User selection of the 3D reconstruction algorithm (through the GUI);

**◦** Interactive visualization of a sequence of 2D profiles (stop by an user order).

over the complete y-axis, respectively. (\*) rigorously it indicates only the axial plane and not the height.

This MATLAB application has the following functionalities:

**◦** User definition of ROI limits in the two axes (through the GUI);

Width 120,40 μm (4,74 mils) 9,74 μm (0,38 mils)

) represents the axial position of the track. Therefore the track height

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 73

*h* min ≤*h* ≤*h* max (over all the pixels inside the ROI);

to the track);

The height value *h* (*xi*

of this algorithm.

**8.** Conversion to microns and mils.

*5.3.3. Application in profilometry*

been developed.

**•** Profile settings

**•** Profile visualization

, *yi*

**Figure 18.** PCB track parallel to the sensor. (a) raw-data *auto-focus* image. FOV: 190 μm x 115 μm (horizontal – y-axis; vertical – x-axis). Same scale in both axes. (b) raw-data topographic (3D) map and a 2D profile drawn for pixel *n*.

In spite of the ringing effects present at track borders, that hamper the measurement of its width and reduce precision and accuracy, an algorithm had been developed for its calculation. This algorithm implemented in *IC\_image* application consists on the following steps:


Using the equation 6 it was calculated the *auto-focus* image, designated previously by image

*zk* values that correspond to the position along z-axis in which the pixel values are maxima in

**Figure 18.** PCB track parallel to the sensor. (a) raw-data *auto-focus* image. FOV: 190 μm x 115 μm (horizontal – y-axis; vertical – x-axis). Same scale in both axes. (b) raw-data topographic (3D) map and a 2D profile drawn for pixel *n*.

In spite of the ringing effects present at track borders, that hamper the measurement of its width and reduce precision and accuracy, an algorithm had been developed for its calculation.

**4.** Definition of the lower (*hmin*) and upper (*hmax)* height limits inside the track. The considered

This algorithm implemented in *IC\_image* application consists on the following steps:

, *yi*

**3.** Determination of height mean (*hmean*) and standard deviation (*hstd*).

=*h mean* ± *hstd* .

pixel values are maxima for each pixel of the raw-data image *I*(*xi*

**1.** Definition of a ROI completely inside the track;

min

**2.** Determination of height *h* (*xi*

limits had been: *h* max

, *yi*

). To represent a two-dimensional (2D) profile it was

) = *zk* where *zk* is the position along z-axis in which the

, *yi* ).

) is the height of that pixel *n* for each sensor line.

) representing the

(1). This image is shown in figure 18. Also the topographic (3D) map *h* (*xi*

, *yi*

each pixel of the raw-data image *I*(*xi*

72 MATLAB Applications for the Practical Engineer

(a)

(b)

selected the pixel *n* of the sensor and *h* (*xn*, *yi*

The height value *h* (*xi* , *yi* ) represents the axial position of the track. Therefore the track height is easily calculated through the determination of the height of the exterior side plane and its subsequent subtraction. This calculation had been made for a track perpendicular to sensor but this example is not covered in this chapter. Table 5 shows the results of the implementation of this algorithm.


**Table 5.** Mean and standard deviation values of the track height and width obtained using a ROI inside the track and over the complete y-axis, respectively. (\*) rigorously it indicates only the axial plane and not the height.

#### *5.3.3. Application in profilometry*

Owing to its depth discrimination ability, optical techniques as confocal microscopy have been used in profilometry. It consists on build three-dimensional profiles in order to measure different surface or geometrical dimensions such as roughness, height / depth or width.

Another MATLAB application, *Profilometry\_SiFrame*, which GUI is presented in figure 19 had been developed.

This MATLAB application has the following functionalities:

	- **◦** User definition of ROI limits in the two axes (through the GUI);
	- **◦** User selection of the set of sensor images previously acquired (through the GUI);
	- **◦** User selection of the visualization mode (2D or 3D) (through the GUI);
	- **◦** User definition of the scanning sequence to be used in 2D visualization (through the GUI);
	- **◦** User selection of the 3D reconstruction algorithm (through the GUI);
	- **◦** Interactive visualization of a sequence of 2D profiles (stop by an user order).

#### **◦** Visualization of 3D profiles

(b) (c)

profile showing the results for frame dimensions (not to scale).

in order to achieve this computer-controlled platform.

application in materials science field.

**6. Conclusion**

**Figure 20.** (a) Scheme of the silicon frame illustrating its dimensions (reproduced from [8]). (b) experimental 3D pro‐ file of the frame (x-and y-axis represent parallel and perpendicular directions relative to sensor, respectively). (c) 2D

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 75

The challenge of building a laboratorial prototype of a microscope using a linear image sensor had been successfully attained. It had been shown its ability to perform the acquisition and visualization of images containing three-dimensional information as well as its potential

MATLAB has played a fundamental role for this outcome. Apart from the optical layout every task since sensor readout through communication of sensor data to the computer for image visualization to image reconstruction algorithms had been implemented in MATLAB appli‐ cations. In this manner the result is a bench-microscope completely controlled by a computer user. Figure 21 summarizes overall functions implemented in the four MATLAB applications

The essential graphical user interface had been developed in a much faster and easier mode using functions of MATLAB core than previously with Borland C. Besides it is important to emphasize the versatility provided by its toolboxes. Particularly *Instrument Control toolbox* that made the implementation of all acquisition and control tasks, commanded from the computer through its serial port, look like a trivial task. Also *Image Processing toolbox* had shown how the implementation of reconstruction algorithms and of any image related operation may be

(a)

**Figure 19.** Graphical User Interface – *Profilometry\_SiFrame* – developed for building and visualization of microma‐ chined component profiles.

A micromachined component that contains a three-dimensional silicon frame had been the test object. For the assessment of the quality of the results, a scheme with frame specification is also presented in figure 20.

Experimental profiles shown in figure 20 are a raw-data topographic (3D) map and a 2D profile where height is represented versus x-axis (sensor orientation). This profile had been drawn from topographic map in (b) representing one of the lines across the frame. Experimental values of width, height or slope of silicon frame are superimposed in the 2D profile.

The lateral side of the silicon frame had been aligned with y-axis. Thus lateral walls of the frame are perpendicular to sensor. On the other hand three different ROI were defined over x-axis. In ROI separation the slope of the frame wall is higher than its maximum value considering the particular objective used. Owing to this slope the light reflected in frame surface is not gathered by the objective and consequently it is not collected in the sensor.

**Figure 20.** (a) Scheme of the silicon frame illustrating its dimensions (reproduced from [8]). (b) experimental 3D pro‐ file of the frame (x-and y-axis represent parallel and perpendicular directions relative to sensor, respectively). (c) 2D profile showing the results for frame dimensions (not to scale).

#### **6. Conclusion**

**◦** Visualization of 3D profiles

74 MATLAB Applications for the Practical Engineer

chined component profiles.

sensor.

is also presented in figure 20.

**Figure 19.** Graphical User Interface – *Profilometry\_SiFrame* – developed for building and visualization of microma‐

A micromachined component that contains a three-dimensional silicon frame had been the test object. For the assessment of the quality of the results, a scheme with frame specification

Experimental profiles shown in figure 20 are a raw-data topographic (3D) map and a 2D profile where height is represented versus x-axis (sensor orientation). This profile had been drawn from topographic map in (b) representing one of the lines across the frame. Experimental

The lateral side of the silicon frame had been aligned with y-axis. Thus lateral walls of the frame are perpendicular to sensor. On the other hand three different ROI were defined over x-axis. In ROI separation the slope of the frame wall is higher than its maximum value considering the particular objective used. Owing to this slope the light reflected in frame surface is not gathered by the objective and consequently it is not collected in the

values of width, height or slope of silicon frame are superimposed in the 2D profile.

The challenge of building a laboratorial prototype of a microscope using a linear image sensor had been successfully attained. It had been shown its ability to perform the acquisition and visualization of images containing three-dimensional information as well as its potential application in materials science field.

MATLAB has played a fundamental role for this outcome. Apart from the optical layout every task since sensor readout through communication of sensor data to the computer for image visualization to image reconstruction algorithms had been implemented in MATLAB appli‐ cations. In this manner the result is a bench-microscope completely controlled by a computer user. Figure 21 summarizes overall functions implemented in the four MATLAB applications in order to achieve this computer-controlled platform.

The essential graphical user interface had been developed in a much faster and easier mode using functions of MATLAB core than previously with Borland C. Besides it is important to emphasize the versatility provided by its toolboxes. Particularly *Instrument Control toolbox* that made the implementation of all acquisition and control tasks, commanded from the computer through its serial port, look like a trivial task. Also *Image Processing toolbox* had shown how the implementation of reconstruction algorithms and of any image related operation may be this computer-controlled platform.

showing the results for frame dimensions (not to scale).

information as well as its potential application in materials science field.

(b) (c)

(a)

performed in an easy way despite the complexity that people might assume because of matrix approach. In summary, with every sense, it is a MATLAB-based microscope. MATLAB has played a fundamental role for this outcome. Apart from the optical layout every task since sensor readout through communication of sensor data to the computer for image visualization to image reconstruction algorithms had been implemented in MATLAB applications. In this manner the result is a bench-microscope completely controlled by a computer user. Figure 21 summarizes overall functions implemented in the four MATLAB applications in order to achieve

[4] Williams TL. The Optical Transfer Function of Imaging Systems. USA: IoP; 1999.

imaging. Applied Optics 1999; 38 (34) 7133-7144.

Jose, USA. Washington: SPIE Press; 1997. p190-198.

Publications; 1998. p563-568.

Press; 2002

[5] Williams CS, Becklund OA. Introduction to the Optical Transfer Function. USA: SPIE

A MATLAB-based Microscope http://dx.doi.org/10.5772/58532 77

[6] Sabharwal S et al. Slit-scanning confocal microendoscope for high-resolution in vivo

[7] Tympel V. Three dimensional animation with a conventional light microscopy. In: Cogswell CJ, Conchello JA, Wilson T. (eds) BiOS1997: proceedings of three-dimen‐ sional microscopy: image acquisition and processing IV, 12-13 February 1997, San

[8] Correia JH, Bartek M, Wolffenbuttel RF. Load-deflection of a low-stress SiN-mem‐ brane/Si-frame composite diaphragm. In: First International Conference on Modeling and Simulation of Microsystems, Semiconductors, Sensors and Actuators: conference proceedings, 6-8 April 1998, Santa Clara Marriott, CA, USA. Boston: Computational

The challenge of building a laboratorial prototype of a microscope using a linear image sensor had been successfully attained. It had been shown its ability to perform the acquisition and visualization of images containing three-dimensional

Figure 20. (a) Scheme of the silicon frame illustrating its dimensions (reproduced from [8]). (b) experimental 3D profile of the frame (x- and y-axis represent parallel and perpendicular directions relative to sensor, respectively). (c) 2D profile

Figure 21. Diagram of the functionalities of the MATLAB applications. **Figure 21.** Diagram of the functionalities of the MATLAB applications.

### **Author details**

Milton P. Macedo1,2

1 Instituto Politécnico de Coimbra, ISEC, DFM, Coimbra, Portugal

2 Centro de Instrumentação, Department of Physics, University of Coimbra, Coimbra, Portugal

#### **References**


[4] Williams TL. The Optical Transfer Function of Imaging Systems. USA: IoP; 1999.

performed in an easy way despite the complexity that people might assume because of matrix

MATLAB has played a fundamental role for this outcome. Apart from the optical layout every task since sensor readout through communication of sensor data to the computer for image visualization to image reconstruction algorithms had been implemented in MATLAB applications. In this manner the result is a bench-microscope completely controlled by a computer user. Figure 21 summarizes overall functions implemented in the four MATLAB applications in order to achieve

MATLAB applications

**MATLAB Core**

**Creation of Excel data files Real-time visualization of sensor readout** 

**Image Processing Toolbox**

> *Image build 2D/3D*

**Image contrast adjustment Image size adjustment Test of algorithms for image quality improvement** 

The challenge of building a laboratorial prototype of a microscope using a linear image sensor had been successfully attained. It had been shown its ability to perform the acquisition and visualization of images containing three-dimensional

Figure 20. (a) Scheme of the silicon frame illustrating its dimensions (reproduced from [8]). (b) experimental 3D profile of the frame (x- and y-axis represent parallel and perpendicular directions relative to sensor, respectively). (c) 2D profile

approach. In summary, with every sense, it is a MATLAB-based microscope.

*Serial port GUI Development* 

showing the results for frame dimensions (not to scale).

**Instrument Control Toolbox** 

*management* 

**Sensor data reception Transmission of commands to the three actuators** 

information as well as its potential application in materials science field.

(b) (c)

(a)

Figure 21. Diagram of the functionalities of the MATLAB applications.

**Figure 21.** Diagram of the functionalities of the MATLAB applications.

1 Instituto Politécnico de Coimbra, ISEC, DFM, Coimbra, Portugal

microscope. Optics Express 2003; 11(12) 1436-1451.

four bar target, Applied Optics supplement 1995; 8050-8052.

2 Centro de Instrumentação, Department of Physics, University of Coimbra, Coimbra,

[1] Murphy DB. Fundamentals of light microscopy and electronic imaging. USA: Wiley-

[2] Lee J, Rogers JD, Descour MR. Imaging quality assessment of multi-modal miniature

[3] Boreman GD, Yang S. Modulation transfer function measurement using three-and

**6. Conclusion** 

**Author details**

Milton P. Macedo1,2

Portugal

**References**

Liss; 2001.

this computer-controlled platform.

76 MATLAB Applications for the Practical Engineer


**Chapter 3**

**Provisional chapter**

**Obstacle Avoidance Task for a Wheeled Mobile Robot –**

Hernández-Guzmán3, J. R. García-Sánchez1, H. Taud1, M. Marciano-Melchor1, and J. A.

In recent years, the development of wheeled mobile robots (WMR) has allowed for significant progress in various fields of science due to the wide scope of applications in which such robots can be employed [1]. These fields include medical assistance, space and marine exploration, leisure, entertainment, technology research and development, metal-mechanics and chemical industries, military research, cleaning in diverse surroundings, agriculture, inspection, security, and transportation, among others. The discussion regarding mobile robotics has been undergoing a slowly-evolving transition and continues even now to be a

Mobile robots can be classified according to their locomotion method into three types [1]: wheeled, legged, and caterpillar track. Although both legged and caterpillar track locomotion have been widely researched, most mobile robots that have been built, evaluated, and reported use wheels to move. This reflects their increasing use in applications such as planetary exploration, mining, inspection, security, rescue operations, hazardous waste clean-up, and medical assistance. A review associated with WMR can be found in [1]; although much of the research that has been carried out thus far has provided a significant contribution to the topic, the development and conception of a mobile robot that has total

In order to improve the autonomy of mobile robots, experiments examining control have generally focused on solving the following problems: (1) mobile robot positioning,

> ©2012 Silva-Ortigoz et al., licensee InTech. This is an open access chapter distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2014 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**A Matlab-Simulink-Based Didactic Application**

**Robot – A Matlab-Simulink-Based Didactic**

R. Silva-Ortigoza1, C. Márquez-Sánchez1, F. Carrizosa-Corral2, V. M.

**Obstacle Avoidance Task for a Wheeled Mobile**

R. Silva-Ortigoza, C. Márquez-Sánchez,

J. R. García-Sánchez, H. Taud,

**Application**

Álvarez-Cedillo1

10.5772/58392

**1. Introduction**

http://dx.doi.org/10.5772/58392

F. Carrizosa-Corral, V. M. Hernández-Guzmán,

M. Marciano-Melchor and J. A. Álvarez-Cedillo

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

topic of great interest at the international level.

autonomy of operation remains a distant prospect.

**Provisional chapter**

#### **Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application**

R. Silva-Ortigoza, C. Márquez-Sánchez,

F. Carrizosa-Corral, V. M. Hernández-Guzmán, R. Silva-Ortigoza1, C. Márquez-Sánchez1, F. Carrizosa-Corral2, V. M.

J. R. García-Sánchez, H. Taud, Hernández-Guzmán3, J. R. García-Sánchez1, H. Taud1, M. Marciano-Melchor1, and J. A. Álvarez-Cedillo1

M. Marciano-Melchor and J. A. Álvarez-Cedillo

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/58392 10.5772/58392

**1. Introduction**

In recent years, the development of wheeled mobile robots (WMR) has allowed for significant progress in various fields of science due to the wide scope of applications in which such robots can be employed [1]. These fields include medical assistance, space and marine exploration, leisure, entertainment, technology research and development, metal-mechanics and chemical industries, military research, cleaning in diverse surroundings, agriculture, inspection, security, and transportation, among others. The discussion regarding mobile robotics has been undergoing a slowly-evolving transition and continues even now to be a topic of great interest at the international level.

Mobile robots can be classified according to their locomotion method into three types [1]: wheeled, legged, and caterpillar track. Although both legged and caterpillar track locomotion have been widely researched, most mobile robots that have been built, evaluated, and reported use wheels to move. This reflects their increasing use in applications such as planetary exploration, mining, inspection, security, rescue operations, hazardous waste clean-up, and medical assistance. A review associated with WMR can be found in [1]; although much of the research that has been carried out thus far has provided a significant contribution to the topic, the development and conception of a mobile robot that has total autonomy of operation remains a distant prospect.

In order to improve the autonomy of mobile robots, experiments examining control have generally focused on solving the following problems: (1) mobile robot positioning,

Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2014 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

©2012 Silva-Ortigoz et al., licensee InTech. This is an open access chapter distributed under the terms of the

(2) stabilization, (3) trajectory tracking control, (4) trajectory planning, and (5) obstacle avoidance. In this respect, significant progress (albeit not total) has been achieved. The present work focuses on the obstacle avoidance problem, whose objective is providing a mobile robot with collision-free navigation through a workspace in which obstacles have been predeterminately distributed. In particular, we develop, step-by-step, a Matlab-Simulink application that describes the experimental implementation of a WMR controller which allows collision-free navigation.

10.5772/58392

81

http://dx.doi.org/10.5772/58392

**2. Artificial potential field**

a local minimum.

generated by the obstacle.

obstacle, respectively, where

and the operator ∇ is defined as *<sup>∂</sup>*

point *q*, collision-free, towards the goal.

This section describes the artificial potential field method, which in conjunction with a controller allows the accomplishment of the obstacle avoidance task. Although the method has been modified by various authors in order to solve some of its inherent problems, these modifications account for individual problems separately and not necessarily in an optimal manner. Therefore, the classic method is employed herein. The artificial potential field method consists of the creation of artificial potential fields, with the goal being the development of an attractive pole and obstacles acting as repulsive surfaces for the mobile robot. Whereas the attractive force produced by the artificial potential field associated with the goal generates a continuous trajectory towards it, the repulsive force produced by the fields associated with the obstacles move the mobile robot away from them. When combining both forces a third force is produced which enables an effective control. It is worth noting that when the attractive and repulsive forces are equal in magnitude and opposite in orientation, as generated by a specific goal and obstacle distribution, the mobile robot cannot accomplish the obstacle avoidance task. When this occurs, the mobile robot is said to be trapped within

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

The present study addresses a problem involving a WMR, an obstacle, and the goal which is directly extendable to the general case, namely, the *n* obstacles problem. Consider a bidimensional workspace and a fixed coordinate system *X-Y* in which the coordinates of a point associated with the WMR are determined by *q* = (*x*1, *y*1) (see Figure 4), the obstacle by *qobs* = (*xo*, *yo*), and the goal by *qm* = (*xm*, *ym*). Moreover, suppose that the obstacle and the goal separately exert a force on the WMR, generating a resultant force, *Ftotal*, given by

where *Fat* (*q*) is the attractive force produced by the goal and *Frep* (*q*) is the repulsive force

The resultant force is considered to be exerted by an artificial potential field determined by

with *Uat* (*q*) and *Urep* (*q*) being the artificial potential fields associated with the goal and the

by the negative gradient of the potential fields, as expressed in (3) and (4). Hence, depending on the positions of the obstacle and the goal, the resultant force will lead the mobile robot's

*<sup>∂</sup><sup>x</sup>* , *<sup>∂</sup> ∂y* 

*Ftotal* = *Fat*(*q*) + *Frep*(*q*), (1)

*U* (*q*) = *Uat* (*q*) + *Urep* (*q*), (2)

*Fat* (*q*) = −∇*Uat* (*q*), (3) *Frep* (*q*) = −∇*Urep* (*q*), (4)

. The attractive and repulsive forces are represented

According to the robotics literature, a variety of different methods are available with which to perform the obstacle avoidance task, the most relevant being: edge detection [2], cell decomposition [3], map building [4], and artificial potential fields [5]; a description of all of these methods can be found in [1], with the artificial potential field method notably the most cited. In the present work the artificial potential field technique is employed to carry out an obstacle avoiding task with a WMR. Developed by Khatib [5], in this method the mobile robot can be considered as a particle, the obstacles presented within the workspace as particles that exert a repulsive force on the mobile robot, and the goal as a particle that exerts an attractive force on the mobile robot. Thus, a resultant potential field is achieved that leads the trajectory that the mobile robot must follow in order to evade the obstacles –which might be considered previously in the method, or else detected via a sensor– and reach the goal. It is worth mentioning that applying the artificial potential field method does not require a great deal of computational complexity because it is based on the implementation of mathematical functions representing the attractive and repulsive forces exerted on the mobile robot. Finally, since the initial establishment of the artificial potential field method, a number of different modifications and implementations have been developed in conjunction with other techniques. Hence, some variations of the method have been reported in [6–13].

Whereas the works cited above introduce research examining the obstacle avoidance task using the artificial potential field method with WMRs, the present document describes the step-by-step experimental implementation of a hierarchical control which is performed along with the artificial potential field method in carrying out the obstacle avoidance task with the differentially-driven WMR built and reported in [14]. This work is aimed at helping students integrate theoretical and practical knowledge through the use of a relevant and modern open-architecture testbed that allows rapid prototyping [15]. On the one hand, simulations are carried out using Matlab-Simulink. On the other hand, Matlab-Simulink, ControlDesk, and the DS1104 electronic board (dSPACE) are employed for the real-time experiments since the graphical environment provided by Simulink facilitates the analysis, design, and construction of dynamic systems.

The present work is structured as follows. Section 2 describes the basis of the artificial potential field method for the obstacle avoidance task. The hierarchical control applied to the WMR in performing the obstacle avoidance task is detailed in Section 3. In Section 4, a description of the block diagrams programmed in Matlab-Simulink, along with the simulation results associated with the closed-loop system, are presented. The subsystems of the prototype employed are then described in Section 5, with Section 6 containing a description of the blocks developed in Matlab-Simulink for the experimental implementation of the hierarchical control via Matlab-Simulink, ControlDesk, and the DS1104 in the prototype. Finally, conclusions drawn from the study are presented in Section 7.

### **2. Artificial potential field**

2

allows collision-free navigation.

design, and construction of dynamic systems.

(2) stabilization, (3) trajectory tracking control, (4) trajectory planning, and (5) obstacle avoidance. In this respect, significant progress (albeit not total) has been achieved. The present work focuses on the obstacle avoidance problem, whose objective is providing a mobile robot with collision-free navigation through a workspace in which obstacles have been predeterminately distributed. In particular, we develop, step-by-step, a Matlab-Simulink application that describes the experimental implementation of a WMR controller which

According to the robotics literature, a variety of different methods are available with which to perform the obstacle avoidance task, the most relevant being: edge detection [2], cell decomposition [3], map building [4], and artificial potential fields [5]; a description of all of these methods can be found in [1], with the artificial potential field method notably the most cited. In the present work the artificial potential field technique is employed to carry out an obstacle avoiding task with a WMR. Developed by Khatib [5], in this method the mobile robot can be considered as a particle, the obstacles presented within the workspace as particles that exert a repulsive force on the mobile robot, and the goal as a particle that exerts an attractive force on the mobile robot. Thus, a resultant potential field is achieved that leads the trajectory that the mobile robot must follow in order to evade the obstacles –which might be considered previously in the method, or else detected via a sensor– and reach the goal. It is worth mentioning that applying the artificial potential field method does not require a great deal of computational complexity because it is based on the implementation of mathematical functions representing the attractive and repulsive forces exerted on the mobile robot. Finally, since the initial establishment of the artificial potential field method, a number of different modifications and implementations have been developed in conjunction with other techniques. Hence, some variations of the method have been reported in [6–13]. Whereas the works cited above introduce research examining the obstacle avoidance task using the artificial potential field method with WMRs, the present document describes the step-by-step experimental implementation of a hierarchical control which is performed along with the artificial potential field method in carrying out the obstacle avoidance task with the differentially-driven WMR built and reported in [14]. This work is aimed at helping students integrate theoretical and practical knowledge through the use of a relevant and modern open-architecture testbed that allows rapid prototyping [15]. On the one hand, simulations are carried out using Matlab-Simulink. On the other hand, Matlab-Simulink, ControlDesk, and the DS1104 electronic board (dSPACE) are employed for the real-time experiments since the graphical environment provided by Simulink facilitates the analysis,

The present work is structured as follows. Section 2 describes the basis of the artificial potential field method for the obstacle avoidance task. The hierarchical control applied to the WMR in performing the obstacle avoidance task is detailed in Section 3. In Section 4, a description of the block diagrams programmed in Matlab-Simulink, along with the simulation results associated with the closed-loop system, are presented. The subsystems of the prototype employed are then described in Section 5, with Section 6 containing a description of the blocks developed in Matlab-Simulink for the experimental implementation of the hierarchical control via Matlab-Simulink, ControlDesk, and the DS1104 in the

prototype. Finally, conclusions drawn from the study are presented in Section 7.

This section describes the artificial potential field method, which in conjunction with a controller allows the accomplishment of the obstacle avoidance task. Although the method has been modified by various authors in order to solve some of its inherent problems, these modifications account for individual problems separately and not necessarily in an optimal manner. Therefore, the classic method is employed herein. The artificial potential field method consists of the creation of artificial potential fields, with the goal being the development of an attractive pole and obstacles acting as repulsive surfaces for the mobile robot. Whereas the attractive force produced by the artificial potential field associated with the goal generates a continuous trajectory towards it, the repulsive force produced by the fields associated with the obstacles move the mobile robot away from them. When combining both forces a third force is produced which enables an effective control. It is worth noting that when the attractive and repulsive forces are equal in magnitude and opposite in orientation, as generated by a specific goal and obstacle distribution, the mobile robot cannot accomplish the obstacle avoidance task. When this occurs, the mobile robot is said to be trapped within a local minimum.

The present study addresses a problem involving a WMR, an obstacle, and the goal which is directly extendable to the general case, namely, the *n* obstacles problem. Consider a bidimensional workspace and a fixed coordinate system *X-Y* in which the coordinates of a point associated with the WMR are determined by *q* = (*x*1, *y*1) (see Figure 4), the obstacle by *qobs* = (*xo*, *yo*), and the goal by *qm* = (*xm*, *ym*). Moreover, suppose that the obstacle and the goal separately exert a force on the WMR, generating a resultant force, *Ftotal*, given by

$$F\_{total} = F\_{at}(q) + F\_{rep}(q)\_{\prime} \tag{1}$$

where *Fat* (*q*) is the attractive force produced by the goal and *Frep* (*q*) is the repulsive force generated by the obstacle.

The resultant force is considered to be exerted by an artificial potential field determined by

$$\mathcal{U}\left(\boldsymbol{q}\right) = \mathcal{U}\_{\rm at}\left(\boldsymbol{q}\right) + \mathcal{U}\_{\rm rep}\left(\boldsymbol{q}\right),\tag{2}$$

with *Uat* (*q*) and *Urep* (*q*) being the artificial potential fields associated with the goal and the obstacle, respectively, where

$$F\_{\rm at} \left( \boldsymbol{q} \right) = -\nabla \mathcal{U}\_{\rm at} \left( \boldsymbol{q} \right) \,, \tag{3}$$

$$F\_{rep}\left(q\right) = -\nabla \mathcal{U}\_{rep}\left(q\right),\tag{4}$$

and the operator ∇ is defined as *<sup>∂</sup> <sup>∂</sup><sup>x</sup>* , *<sup>∂</sup> ∂y* . The attractive and repulsive forces are represented by the negative gradient of the potential fields, as expressed in (3) and (4). Hence, depending on the positions of the obstacle and the goal, the resultant force will lead the mobile robot's point *q*, collision-free, towards the goal.

#### **2.1. Attractive potential**

One of the most commonly-used attractive potential fields was presented in [16], and is determined by

$$\mathcal{U}\_{\rm at}\left(q\right) = \frac{1}{2}\xi\rho^{k}\left(q, q\_{\rm m}\right),\tag{5}$$

10.5772/58392

83

http://dx.doi.org/10.5772/58392

(8)

(9)

<sup>2</sup> 1.5 <sup>1</sup>

  1 2 *η* � 1

  *η* � 1

0 0.5 1 1.5 2 2.5 3 3.5 4

**Figure 2.** Attractive potential field for *k* = 2.

*Urep* (*q*) =

*Frep*(*q*) = −∇*Urep* (*q*) =

of these coordinates in real-time.

**2.2. Repulsive potential**

by

0.5 <sup>0</sup> <sup>0</sup> 0.5 <sup>1</sup> 1.5 <sup>2</sup> 2.5 <sup>3</sup> 3.5 <sup>4</sup> 2.5 <sup>3</sup> 3.5 <sup>4</sup>

Y X

One commonly-used repulsive potential function takes the following form [16]:

*<sup>ρ</sup>* (*q*, *qobs*) <sup>−</sup> <sup>1</sup>

where *<sup>η</sup>* is a positive scale factor, *<sup>ρ</sup>* (*q*, *qobs*) = �*<sup>q</sup>* − *qobs*� is the shortest distance between the mobile robot and the obstacle, and *ρ*<sup>0</sup> is a positive constant that represents the distance of the obstacle's influence. The graphical representation associated with the obstacle defined by (8) is shown in Figure 3. Hence, the repulsive force, *Frep* (*q*), associated with (8), is determined

*<sup>ρ</sup>* (*q*, *qobs*) <sup>−</sup> <sup>1</sup>

Finally, it is worth mentioning that the described artificial potential field method depends upon the relative position of the mobile robot to the obstacle, unlike other methods such as that of Krogh [17] in which the potential field is sensitive to the impact time. Furthermore, implementing the presented method requires knowledge of the coordinates of the mobile robot, the obstacle, and the goal, i.e., *q* = (*x*1, *y*1), *qobs* = (*xo*, *yo*), and *qm* = (*xm*, *ym*), respectively. In the present work, it is assumed that these coordinates are already known; nevertheless, the employed method can also be extended to sensors that allow the acquisition

*ρ*0

� *q* − *qo ρ*<sup>3</sup> (*q*, *qobs*)

0, for *ρ* (*q*, *qobs*) > *ρ*0.

*ρ*0 �2

0, for *ρ* (*q*, *qobs*) > *ρ*0,

, for *<sup>ρ</sup>* (*q*, *qobs*) ≤ *<sup>ρ</sup>*0,

, for *<sup>ρ</sup>* (*q*, *qobs*) ≤ *<sup>ρ</sup>*0,

Uat(q)

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

where *ξ* is a positive scale factor, *q* is the mobile robot's reference point, *qm* is the goal's coordinate, *ρ* (*q*, *qm*) = �*q* − *qm*� is the distance between *q* and *qm*, and *k* = 1, 2.

• For *k* = 1, the attractive potential field presents a conic shape, as shown in Figure 1. The attractive force generated by *Uat* has constant amplitude except at the goal, where *Uat* is non-differentiable, as given by

$$F\_{\rm at}(q) = -\nabla \mathcal{U}\_{\rm at}\left(q\right) = -\frac{1}{2}\xi \frac{\left(q - q\_m\right)}{||q - q\_m||}. \tag{6}$$

• For *k* = 2, the attractive potential field presents a parabolic shape, as shown in Figure 2. The corresponding attractive force is determined by the negative gradient of the attractive potential, i.e.,

$$F\_{at}(q) = -\nabla \mathcal{U}\_{at}\left(q\right) = -\xi \left(q - q\_m\right),\tag{7}$$

which converges linearly to zero as the mobile robot approaches the goal.

**Figure 1.** Attractive potential field for *k* = 1.

**Figure 2.** Attractive potential field for *k* = 2.

#### **2.2. Repulsive potential**

4

**2.1. Attractive potential**

non-differentiable, as given by

determined by

potential, i.e.,

1.5

1

0.5

0

<sup>4</sup> 3.5 <sup>3</sup> 2.5 <sup>2</sup>

**Figure 1.** Attractive potential field for *k* = 1.

One of the most commonly-used attractive potential fields was presented in [16], and is

2

where *ξ* is a positive scale factor, *q* is the mobile robot's reference point, *qm* is the goal's

• For *k* = 1, the attractive potential field presents a conic shape, as shown in Figure 1. The attractive force generated by *Uat* has constant amplitude except at the goal, where *Uat* is

• For *k* = 2, the attractive potential field presents a parabolic shape, as shown in Figure 2. The corresponding attractive force is determined by the negative gradient of the attractive

2

Uat(q)

1.5 <sup>1</sup> 0.5 <sup>0</sup> <sup>0</sup> 0.5 <sup>1</sup> 1.5 <sup>2</sup> 2.5 <sup>3</sup> 3.5 <sup>4</sup>

Y X

*<sup>ξ</sup>* (*<sup>q</sup>* <sup>−</sup> *qm*) �*q* − *qm*�

*Fat*(*q*) = −∇*Uat* (*q*) = −*ξ* (*q* − *qm*), (7)

*ξρ<sup>k</sup>* (*q*, *qm*), (5)

. (6)

*Uat* (*q*) <sup>=</sup> <sup>1</sup>

coordinate, *ρ* (*q*, *qm*) = �*q* − *qm*� is the distance between *q* and *qm*, and *k* = 1, 2.

*Fat*(*q*) = −∇*Uat* (*q*) <sup>=</sup> <sup>−</sup><sup>1</sup>

which converges linearly to zero as the mobile robot approaches the goal.

One commonly-used repulsive potential function takes the following form [16]:

$$\mathcal{U}\_{\text{rep}}\left(q\right) = \left\{ \frac{1}{2}\eta \left[ \frac{1}{\rho \left( q\_{\prime} q\_{\text{obs}} \right)} - \frac{1}{\rho\_{0}} \right]^{2} , \text{for } \rho \left( q\_{\prime} q\_{\text{obs}} \right) \le \rho\_{0\prime} \tag{8}$$
 
$$\text{for } \rho \left( q\_{\prime} q\_{\text{obs}} \right) > \rho\_{0\prime}$$

where *<sup>η</sup>* is a positive scale factor, *<sup>ρ</sup>* (*q*, *qobs*) = �*<sup>q</sup>* − *qobs*� is the shortest distance between the mobile robot and the obstacle, and *ρ*<sup>0</sup> is a positive constant that represents the distance of the obstacle's influence. The graphical representation associated with the obstacle defined by (8) is shown in Figure 3. Hence, the repulsive force, *Frep* (*q*), associated with (8), is determined by

$$F\_{\rm rep}(q) = -\nabla \mathcal{U}\_{\rm rep} \left( q \right) = \begin{cases} \eta \left[ \frac{1}{\rho \left( q\_{\prime} q\_{\rm obs} \right)} - \frac{1}{\rho\_{0}} \right] \frac{q - q\_{\rho}}{\rho^{3} \left( q\_{\prime} q\_{\rm obs} \right)}, & \text{for } \rho \left( q\_{\prime} q\_{\rm obs} \right) \le \rho\_{0}, \\ 0, & \text{for } \rho \left( q\_{\prime} q\_{\rm obs} \right) > \rho\_{0}. \end{cases} \tag{9}$$

Finally, it is worth mentioning that the described artificial potential field method depends upon the relative position of the mobile robot to the obstacle, unlike other methods such as that of Krogh [17] in which the potential field is sensitive to the impact time. Furthermore, implementing the presented method requires knowledge of the coordinates of the mobile robot, the obstacle, and the goal, i.e., *q* = (*x*1, *y*1), *qobs* = (*xo*, *yo*), and *qm* = (*xm*, *ym*), respectively. In the present work, it is assumed that these coordinates are already known; nevertheless, the employed method can also be extended to sensors that allow the acquisition of these coordinates in real-time.

10.5772/58392

85

http://dx.doi.org/10.5772/58392

angular velocities of the right and left wheels, respectively, *r* is the wheel ratio, and 2*l* is the separation between the wheels. Parameters and variables associated with the WMR are

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

Whereas the derivative with respect to time *t*, associated with the state variables, is denoted by a dot in equations (10) and (12), in the rest of the chapter it is represented explicitly, i.e.,

Since the mathematical model described by (10) presents a noninvertible relationship between the controls, (*ωr*, *ωl*), and the outputs, (*x*, *y*), it is not possible to propose a control via input-output linearization. For simplicity, another reference point associated with the WMR that can therefore be considered is the front part which has the coordinates *q* = (*x*1, *y*1) (see Figure 4). The coordinates of *q* expressed in terms of *x*, *y*, and *ϕ* are determined by

*x*<sup>1</sup> = *x* + *L* cos *ϕ*,

where *L* is the distance from the mid-point of the wheel shaft, (*x*, *y*), to the point *q* in the direction perpendicular to the shaft. Deriving system (11) with respect to time obtains the

= *A* (*ϕ*)

 cos *ϕ* −*L* sin *ϕ* sin *ϕ L* cos *ϕ*

 *ω<sup>r</sup> ωl* 

> *<sup>r</sup>* <sup>2</sup> *<sup>r</sup>* <sup>2</sup> *<sup>r</sup>* <sup>2</sup>*<sup>l</sup>* <sup>−</sup> *<sup>r</sup>* 2*l* .

<sup>2</sup>*<sup>l</sup>* �<sup>=</sup> 0, it is clear that an input-output linearization scheme can be

WMR kinematic model associated with point *q* = (*x*1, *y*1), which is given by

 *x*˙1 *y*˙1 

*A* (*ϕ*) =

*<sup>y</sup>*<sup>1</sup> <sup>=</sup> *<sup>y</sup>* <sup>+</sup> *<sup>L</sup>* sin *<sup>ϕ</sup>*, (11)

, (12)

r

shown in Figure 4. This WMR configuration is known as differential traction.

**Figure 4.** WMR diagram.

*d dt* .

with

Since det *A* (*ϕ*) = − *Lr*<sup>2</sup>

l

**Figure 3.** Repulsive potential field.

#### **3. Control for obstacle avoidance of WMR**

This section presents the hierarchical control developed in [18], and analyzed in [26] for a trailer-like vehicle. Using the kinematic model associated with the differentially-driven WMR, an input-output linearization control is proposed that, in conjunction with the artificial potential field method, enables the accomplishment of the obstacle avoidance task. A PI control is then proposed for each DC motor, allowing the WMR to move. Finally, with the ultimate aim of experimentally accomplishing the obstacle avoidance task with a WMR prototype, a hierarchical control is proposed which merges the said controls, similar to the structure presented in [14, 19–23].

#### **3.1. Control of the kinematic model**

The mobile robot under study is a vehicle comprising two traction wheels, left and right. These two wheels are identical, parallel to each other, non-deformable, and joined by a shaft. The robot also comprises two omnidirectional wheels, front and rear, that ensure the robot platform remains on a plane. Supposing movement is restricted on an *XY* plane and that there is no wheel slip, existing literature (see [24]) describe the WMR kinematics as given by

$$\begin{aligned} \dot{x} &= \frac{\left(\omega\_r + \omega\_l\right)r}{2} \cos \varphi, \\ \dot{y} &= \frac{\left(\omega\_r + \omega\_l\right)r}{2} \sin \varphi, \\ \dot{\varphi} &= \frac{\left(\omega\_r - \omega\_l\right)r}{2l}, \end{aligned} \tag{10}$$

where (*x*, *y*) denotes the position of the mid-point of the shaft that joins the wheels, *ϕ* is the angle formed by the WMR symmetry axis and the positive *X*-axis, *ω<sup>r</sup>* and *ω<sup>l</sup>* are the angular velocities of the right and left wheels, respectively, *r* is the wheel ratio, and 2*l* is the separation between the wheels. Parameters and variables associated with the WMR are shown in Figure 4. This WMR configuration is known as differential traction.

**Figure 4.** WMR diagram.

6

**Figure 3.** Repulsive potential field.

structure presented in [14, 19–23].

**3.1. Control of the kinematic model**

**3. Control for obstacle avoidance of WMR**

Urep(q)

Y X

This section presents the hierarchical control developed in [18], and analyzed in [26] for a trailer-like vehicle. Using the kinematic model associated with the differentially-driven WMR, an input-output linearization control is proposed that, in conjunction with the artificial potential field method, enables the accomplishment of the obstacle avoidance task. A PI control is then proposed for each DC motor, allowing the WMR to move. Finally, with the ultimate aim of experimentally accomplishing the obstacle avoidance task with a WMR prototype, a hierarchical control is proposed which merges the said controls, similar to the

The mobile robot under study is a vehicle comprising two traction wheels, left and right. These two wheels are identical, parallel to each other, non-deformable, and joined by a shaft. The robot also comprises two omnidirectional wheels, front and rear, that ensure the robot platform remains on a plane. Supposing movement is restricted on an *XY* plane and that there is no wheel slip, existing literature (see [24]) describe the WMR kinematics as given by

*<sup>x</sup>*˙ <sup>=</sup> (*ω<sup>r</sup>* <sup>+</sup> *<sup>ω</sup>l*)*<sup>r</sup>*

*<sup>y</sup>*˙ <sup>=</sup> (*ω<sup>r</sup>* <sup>+</sup> *<sup>ω</sup>l*)*<sup>r</sup>*

*<sup>ϕ</sup>*˙ <sup>=</sup> (*ω<sup>r</sup>* <sup>−</sup> *<sup>ω</sup>l*)*<sup>r</sup>*

<sup>2</sup> cos *<sup>ϕ</sup>*,

<sup>2</sup>*<sup>l</sup>* ,

where (*x*, *y*) denotes the position of the mid-point of the shaft that joins the wheels, *ϕ* is the angle formed by the WMR symmetry axis and the positive *X*-axis, *ω<sup>r</sup>* and *ω<sup>l</sup>* are the

<sup>2</sup> sin *<sup>ϕ</sup>*, (10)

Whereas the derivative with respect to time *t*, associated with the state variables, is denoted by a dot in equations (10) and (12), in the rest of the chapter it is represented explicitly, i.e., *d dt* .

Since the mathematical model described by (10) presents a noninvertible relationship between the controls, (*ωr*, *ωl*), and the outputs, (*x*, *y*), it is not possible to propose a control via input-output linearization. For simplicity, another reference point associated with the WMR that can therefore be considered is the front part which has the coordinates *q* = (*x*1, *y*1) (see Figure 4). The coordinates of *q* expressed in terms of *x*, *y*, and *ϕ* are determined by

$$\begin{aligned} x\_1 &= x + L \cos \varphi\_{\prime} \\ y\_1 &= y + L \sin \varphi\_{\prime} \end{aligned} \tag{11}$$

where *L* is the distance from the mid-point of the wheel shaft, (*x*, *y*), to the point *q* in the direction perpendicular to the shaft. Deriving system (11) with respect to time obtains the WMR kinematic model associated with point *q* = (*x*1, *y*1), which is given by

$$A\begin{pmatrix} \dot{\mathbf{x}}\_1\\ \dot{\mathbf{y}}\_1 \end{pmatrix} = A\begin{pmatrix} \boldsymbol{\varrho} \end{pmatrix} \begin{pmatrix} \boldsymbol{\omega}\_r\\ \boldsymbol{\omega}\_l \end{pmatrix},\tag{12}$$

with

$$A\left(\boldsymbol{\varrho}\right) = \begin{pmatrix} \cos\boldsymbol{\varrho} \ -L\sin\boldsymbol{\varrho} \\ \sin\boldsymbol{\varrho} \ \boldsymbol{L}\cos\boldsymbol{\varrho} \end{pmatrix} \begin{pmatrix} \frac{r}{2} & \frac{r}{2} \\ \frac{r}{2l} & -\frac{r}{2l} \end{pmatrix}.$$

Since det *A* (*ϕ*) = − *Lr*<sup>2</sup> <sup>2</sup>*<sup>l</sup>* �<sup>=</sup> 0, it is clear that an input-output linearization scheme can be proposed for (*ωr*, *<sup>ω</sup>l*) − (*x*1, *<sup>y</sup>*1). According to [18], an input-output linearization control that allows the WMR to accomplish the obstacle avoidance task and reach the goal can be written as follows:

$$
\begin{pmatrix} \omega\_{\mathbb{T}} \\ \omega\_{\mathbb{I}} \end{pmatrix} = \frac{v\_{d}}{\sqrt{f\_{\mathfrak{x}}^{2} + f\_{\mathfrak{y}}^{2} + \varepsilon}} \frac{1}{Lr} \begin{pmatrix} L\cos\varphi - l\sin\varphi & l\cos\varphi + L\sin\varphi \\ L\cos\varphi + l\sin\varphi & -(l\cos\varphi - L\sin\varphi) \end{pmatrix} \begin{pmatrix} f\_{\mathfrak{x}} \\ f\_{\mathfrak{y}} \end{pmatrix}, \tag{13}
$$

where *υ<sup>d</sup>* is a desired constant velocity, *ε* is a constant value close to zero, and *fx* and *fy* are the components of *Ftotal* in directions *X* and *Y*, respectively.

When there are *n* obstacles located within the workspace, *fx* and *fy* are determined by

$$
\begin{pmatrix} f\_{\mathbf{x}} \\ f\_{\mathbf{y}} \end{pmatrix} = \begin{pmatrix} f\_{\mathbf{x}\,\mathrm{at}} + f\_{\mathbf{x}\,1\mathrm{rep}} + f\_{\mathbf{x}\,2\mathrm{rep}} + \dots + f\_{\mathbf{x}\,n\mathrm{rep}} \\ f\_{\mathbf{y}\,\mathrm{at}} + f\_{\mathbf{y}\,1\mathrm{rep}} + f\_{\mathbf{y}\,2\mathrm{rep}} + \dots + f\_{\mathbf{y}\,n\mathrm{rep}} \end{pmatrix},\tag{14}
$$

10.5772/58392

87

(22)

(26)

. (21)

http://dx.doi.org/10.5772/58392

Therefore, for a scenario involving one obstacle, (14) is given by

� *fx fy* � =

*<sup>ρ</sup>*(*q*,*qobs*2) <sup>−</sup> <sup>1</sup>

*<sup>ρ</sup>*(*q*,*qobs*2) <sup>−</sup> <sup>1</sup>

*<sup>ρ</sup>*(*q*,*qobs*2) <sup>−</sup> <sup>1</sup>

� *fx fy* � =

be increased depending on the number of obstacles.

controller is implemented for each motor.

*ρ*02 �2

*<sup>ρ</sup>*<sup>02</sup> � � <sup>1</sup> *ρ*<sup>3</sup>(*q*,*qobs*2)

*<sup>ρ</sup>*<sup>02</sup> � � <sup>1</sup> *ρ*<sup>3</sup>(*q*,*qobs*2)

Thus, in a scenario two obstacles, (14) adopts the following expression:

0, for *<sup>ρ</sup>*(*q*, *qobs*2) > *<sup>ρ</sup>*02.

• Two obstacles:

following force components:

 

� *η* � 1

� *η* � 1

1 2 *η* � 1

*U*2*rep*(*q*) =

*fx* <sup>2</sup>*rep* =

*fy* <sup>2</sup>*rep* =

**3.2. DC motor control**

� *fx at* + *fx* <sup>1</sup>*rep fy at* <sup>+</sup> *fy* <sup>1</sup>*rep* �

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

, for *<sup>ρ</sup>*(*q*, *qobs*2) ≤ *<sup>ρ</sup>*02,

0, for *<sup>ρ</sup>*(*q*, *qobs*2) <sup>&</sup>gt; *<sup>ρ</sup>*02. (23)

0, for *<sup>ρ</sup>*(*q*, *qobs*2) <sup>&</sup>gt; *<sup>ρ</sup>*02. (24)

(*x*<sup>1</sup> − *xo*2), for *<sup>ρ</sup>*(*q*, *qobs*2) ≤ *<sup>ρ</sup>*02,

(*y*<sup>1</sup> − *yo*2), for *<sup>ρ</sup>*(*q*, *qobs*2) ≤ *<sup>ρ</sup>*02,

. (25)

�

�

� *fx at* + *fx* <sup>1</sup>*rep* + *fx* <sup>2</sup>*rep fy at* <sup>+</sup> *fy* <sup>1</sup>*rep* <sup>+</sup> *fy* <sup>2</sup>*rep* �

Finally, it is worth mentioning that for each obstacle present within the workspace there will exist a repulsive potential field; as a consequence, the terms associated with *fx* and *fy* will

In order to execute the obstacle avoidance task experimentally, the angular velocity profiles *ω<sup>r</sup>* and *ωl*, determined by the upper hierarchy (13), must be reproduced by the DC motors associated with the WMR prototype employed in the present work (see [14]). Thus for the right and left angular velocities of the DC motors to approach *ω<sup>r</sup>* and *ωl*, respectively, a PI

A DC motor mathematical model expressed in terms of the motor shaft speed *̟* is given by

*dt* <sup>=</sup> *<sup>u</sup>* <sup>−</sup> *Raia* <sup>−</sup> *ke̟*,

*dt* <sup>=</sup> <sup>−</sup>*b̟* <sup>+</sup> *kmia*,

*La dia*

> *J d̟*

The attractive force components associated with the goal, *fx at* and *fy at*, are determined by (16) and (17), respectively. Whereas the repulsive force components associated with one of the obstacles, *fx* <sup>1</sup>*rep* and *fy* <sup>1</sup>*rep*, are determined by (20) and (20), respectively, the second obstacle, which is associated with the repulsive potential field *U*2*rep*(*q*), has the

with *fx at* the attractive force component associated with the goal in direction *X*, and *fx* <sup>1</sup>*rep*, *fx* <sup>2</sup>*rep*, ..., *fx nrep* the repulsive force components in direction *X* associated with obstacles 1, 2, . . . , *n*, respectively. The description of the terms associated with *fy* is highly similar to that for the terms associated with *fx*.

Here we present the equations explicitly associated with (14) when it is supposed that, within the workspace, there are one and two obstacles, respectively.

• One obstacle:

In this situation, in accordance with (5) and (7), the attractive potential fields associated with the goal and its force components in directions *X* and *Y*, respectively, are given by

$$\mathcal{U}\_{dt}(q) = \frac{1}{2}\xi \left[ (\mathbf{x}\_1 - \mathbf{x}\_m)^2 + (y\_1 - y\_m)^2 \right],\tag{15}$$

$$f\_{\mathbf{x}\text{ at}} = -\mathfrak{F}(\mathbf{x}\_1 - \mathbf{x}\_{\text{ll}}),\tag{16}$$

$$f\_{y\,at} = -\mathfrak{F}(y\_1 - y\_m). \tag{17}$$

Whereas, in accordance with (8) and (9), the repulsive potential fields associated with the obstacle and its force components in directions *X* and *Y*, respectively, are determined by

$$\mathcal{U}\_{1\text{rep}}(q) = \begin{cases} \frac{1}{2}\eta \left[ \frac{1}{\rho(q, q\_{\text{obs1}})} - \frac{1}{\rho \text{u}} \right]^2, & \text{for} \quad \rho(q, q\_{\text{obs1}}) \le \rho\_{01\text{\textquotedblleft}}\\ 0, & \text{for} \quad \rho(q, q\_{\text{obs1}}) > \rho\_{01}. \end{cases} \tag{18}$$

$$f\_{\mathbf{x}\,1\,rep} = \begin{cases} \eta \left[ \frac{1}{\rho(\boldsymbol{q}, \boldsymbol{q}\_{\mathrm{obs1}})} - \frac{1}{\rho\_{\mathrm{01}}} \right] \left[ \frac{1}{\rho^3(\boldsymbol{q}, \boldsymbol{q}\_{\mathrm{obs1}})} \right] (\mathbf{x}\_{\mathrm{I}} - \mathbf{x}\_{\mathrm{o1}}) \, \text{ } & \text{for} \quad \rho(\boldsymbol{q}, \boldsymbol{q}\_{\mathrm{obs1}}) \le \rho\_{\mathrm{01}\,\prime} \\ 0, & \text{for} \quad \rho(\boldsymbol{q}, \boldsymbol{q}\_{\mathrm{obs1}}) > \rho\_{\mathrm{01}\,\prime} \end{cases} \tag{19}$$

$$f\_{y\text{ 1rep}} = \begin{cases} \eta \left[ \frac{1}{\rho(\eta.\rho\_{\text{obs1}})} - \frac{1}{\rho\_{\text{01}}} \right] \left[ \frac{1}{\rho^3(\eta.\rho\_{\text{obs1}})} \right] (y\_1 - y\_{o1}) \, & \text{ for } \quad \rho(\eta.\rho\_{\text{obs1}}) \le \rho\_{\text{01}}\\ 0, & \text{ for } \quad \rho(\eta.\rho\_{\text{obs1}}) > \rho\_{\text{01}}. \end{cases} \tag{20}$$

Therefore, for a scenario involving one obstacle, (14) is given by

$$
\begin{pmatrix} f\_{\mathbf{x}} \\ f\_{\mathbf{y}} \end{pmatrix} = \begin{pmatrix} f\_{\mathbf{x} \text{ at }} + f\_{\mathbf{x} \text{ 1} rep} \\ f\_{\mathbf{y} \text{ at }} + f\_{\mathbf{y} \text{ 1} rep} \end{pmatrix}. \tag{21}
$$

• Two obstacles:

8

as follows:

� *ω<sup>r</sup> ωl* � <sup>=</sup> �

• One obstacle:

*U*1*rep*(*q*) =

*fx* <sup>1</sup>*rep* =

*fy* <sup>1</sup>*rep* =

 

� *η* � 1

� *η* � 1

1 2 *η* � 1

*υd*

1 *Lr*

the components of *Ftotal* in directions *X* and *Y*, respectively.

the workspace, there are one and two obstacles, respectively.

*Uat*(*q*) = <sup>1</sup>

*<sup>ρ</sup>*(*q*,*qobs*1) <sup>−</sup> <sup>1</sup>

*<sup>ρ</sup>*(*q*,*qobs*1) <sup>−</sup> <sup>1</sup>

*<sup>ρ</sup>*(*q*,*qobs*1) <sup>−</sup> <sup>1</sup>

*ρ*01 �2

*ρ*01

*ρ*01

2 *ξ* �

*f* 2 *<sup>x</sup>* + *f* <sup>2</sup> *<sup>y</sup>* + *ε*

� *fx fy* � =

similar to that for the terms associated with *fx*.

proposed for (*ωr*, *<sup>ω</sup>l*) − (*x*1, *<sup>y</sup>*1). According to [18], an input-output linearization control that allows the WMR to accomplish the obstacle avoidance task and reach the goal can be written

where *υ<sup>d</sup>* is a desired constant velocity, *ε* is a constant value close to zero, and *fx* and *fy* are

with *fx at* the attractive force component associated with the goal in direction *X*, and *fx* <sup>1</sup>*rep*, *fx* <sup>2</sup>*rep*, ..., *fx nrep* the repulsive force components in direction *X* associated with obstacles 1, 2, . . . , *n*, respectively. The description of the terms associated with *fy* is highly

Here we present the equations explicitly associated with (14) when it is supposed that, within

In this situation, in accordance with (5) and (7), the attractive potential fields associated with the goal and its force components in directions *X* and *Y*, respectively, are given by

(*x*<sup>1</sup> − *xm*)

Whereas, in accordance with (8) and (9), the repulsive potential fields associated with the obstacle and its force components in directions *X* and *Y*, respectively, are determined by

0, for *<sup>ρ</sup>*(*q*, *qobs*1) > *<sup>ρ</sup>*01.

� � 1 *ρ*<sup>3</sup>(*q*,*qobs*1)

� � 1 *ρ*<sup>3</sup>(*q*,*qobs*1) <sup>2</sup> + (*y*<sup>1</sup> − *ym*)

, for *<sup>ρ</sup>*(*q*, *qobs*1) ≤ *<sup>ρ</sup>*01,

0, for *<sup>ρ</sup>*(*q*, *qobs*1) <sup>&</sup>gt; *<sup>ρ</sup>*01. (19)

0, for *<sup>ρ</sup>*(*q*, *qobs*1) <sup>&</sup>gt; *<sup>ρ</sup>*01. (20)

�

�

*fx at* = −*ξ*(*x*<sup>1</sup> − *xm*), (16) *fy at* = −*ξ*(*y*<sup>1</sup> − *ym*). (17)

2 �

(*x*<sup>1</sup> − *xo*1), for *<sup>ρ</sup>*(*q*, *qobs*1) ≤ *<sup>ρ</sup>*01,

(*y*<sup>1</sup> − *yo*1), for *<sup>ρ</sup>*(*q*, *qobs*1) ≤ *<sup>ρ</sup>*01,

� *fx at* + *fx* <sup>1</sup>*rep* + *fx* <sup>2</sup>*rep* + ... + *fx nrep fy at* + *fy* <sup>1</sup>*rep* + *fy* <sup>2</sup>*rep* + ... + *fy nrep*

When there are *n* obstacles located within the workspace, *fx* and *fy* are determined by

� *L* cos *ϕ* − *l* sin *ϕ l* cos *ϕ* + *L* sin *ϕ L* cos *ϕ* + *l* sin *ϕ* − (*l* cos *ϕ* − *L* sin *ϕ*)

� � *fx fy* �

, (14)

, (15)

(18)

�

, (13)

The attractive force components associated with the goal, *fx at* and *fy at*, are determined by (16) and (17), respectively. Whereas the repulsive force components associated with one of the obstacles, *fx* <sup>1</sup>*rep* and *fy* <sup>1</sup>*rep*, are determined by (20) and (20), respectively, the second obstacle, which is associated with the repulsive potential field *U*2*rep*(*q*), has the following force components:

$$\mathcal{U}\_{2\text{rep}}(q) = \begin{cases} \frac{1}{2}\eta \left[ \frac{1}{\rho(q, q\_{\text{obs2}})} - \frac{1}{\rho\_{\text{02}}} \right]^2, & \text{for} \quad \rho(q, q\_{\text{obs2}}) \le \rho\_{\text{02}}, \\ 0, & \text{for} \quad \rho(q, q\_{\text{obs2}}) > \rho\_{\text{02}}. \end{cases} \tag{22}$$

$$f\_{\mathbf{x}\,2\text{rep}} = \begin{cases} \eta \left[ \frac{1}{\rho(\boldsymbol{q},\boldsymbol{q}\_{\text{obs2}})} - \frac{1}{\rho\_{\text{02}}} \right] \left[ \frac{1}{\rho^3(\boldsymbol{q},\boldsymbol{q}\_{\text{obs2}})} \right] \left( \mathbf{x}\_1 - \mathbf{x}\_{\text{o2}} \right), & \text{for} \quad \rho(\boldsymbol{q},\boldsymbol{q}\_{\text{obs2}}) \le \rho\_{\text{02}}. \\ 0, & \text{for} \quad \rho(\boldsymbol{q},\boldsymbol{q}\_{\text{obs2}}) > \rho\_{\text{02}}. \end{cases} \tag{23}$$

$$f\_{y\cdot 2rep} = \begin{cases} \eta \left[ \frac{1}{\rho(\boldsymbol{q}, \rho\_{\rm obs2})} - \frac{1}{\rho\_{\rm ox}} \right] \left[ \frac{1}{\rho^3(\boldsymbol{q}, \rho\_{\rm obs2})} \right] \left( y\_1 - y\_{o2} \right), & \text{for} \quad \rho(\boldsymbol{q}, \boldsymbol{q}\_{\rm obs2}) \le \rho\_{02\prime} \\ 0, & \text{for} \quad \rho(\boldsymbol{q}, \boldsymbol{q}\_{\rm obs2}) > \rho\_{02\prime} \end{cases} \tag{24}$$

Thus, in a scenario two obstacles, (14) adopts the following expression:

$$
\begin{pmatrix} f\_x \\ f\_y \end{pmatrix} = \begin{pmatrix} f\_{x\ at} + f\_{x\ 1rep} + f\_{x\ 2rep} \\ f\_{y\ at} + f\_{y\ 1rep} + f\_{y\ 2rep} \end{pmatrix}.\tag{25}
$$

Finally, it is worth mentioning that for each obstacle present within the workspace there will exist a repulsive potential field; as a consequence, the terms associated with *fx* and *fy* will be increased depending on the number of obstacles.

#### **3.2. DC motor control**

In order to execute the obstacle avoidance task experimentally, the angular velocity profiles *ω<sup>r</sup>* and *ωl*, determined by the upper hierarchy (13), must be reproduced by the DC motors associated with the WMR prototype employed in the present work (see [14]). Thus for the right and left angular velocities of the DC motors to approach *ω<sup>r</sup>* and *ωl*, respectively, a PI controller is implemented for each motor.

A DC motor mathematical model expressed in terms of the motor shaft speed *̟* is given by

$$\begin{aligned} L\_a \frac{di\_a}{dt} &= \mu - \mathcal{R}\_d i\_a - k\_\epsilon \mathcal{O}\_\prime \\ J \frac{d\mathcal{O}}{dt} &= -b\mathcal{O} + k\_m i\_a \end{aligned} \tag{26}$$

where *u* is the motor armature voltage, *ia* is the armature current, *ke* is the back-electromotive force constant, *km* is the motor torque constant, *La* is the armature inductance, *Ra* is the armature resistance, *J* is the rotor and load inertia, and *b* is the viscous friction constant due to both motor and load.

Since motor manufacturers do not generally provide all of the parameter values associated with (26), these values can be simply obtained via the reduction of (26) to a first order system that relates *̟* to *u*. This is accomplished by assuming that *La* ≈ 0 in (26). Hence, the simplified model is determined by

$$\frac{d\mathcal{\boldsymbol{\sigma}}}{dt} = -\boldsymbol{\alpha}\boldsymbol{\sigma} + \boldsymbol{\beta}\boldsymbol{u}.\tag{27}$$

10.5772/58392

89

http://dx.doi.org/10.5772/58392

(31)

(32)

At the lower level, two PI controllers for the DC motors were considered, ensuring that the actual wheel velocities followed the desired velocity profiles generated at the upper level. In accordance with (28), the mathematical models associated with the right and left motors

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

*dt* <sup>=</sup> <sup>−</sup>10.20*̟<sup>r</sup>* <sup>+</sup> 5.51*ur*,

*dt* <sup>=</sup> <sup>−</sup>10.20*̟<sup>l</sup>* <sup>+</sup> 5.99*ul*,

Thus, two controls, *ur* and *ul*, are required; in accordance with (29), these are given

0

0

*<sup>r</sup>* − *̟r*,

*erdτ*, (33)

*eldτ*, (34)

*<sup>l</sup>* <sup>−</sup> *̟l*, (35)

*<sup>l</sup>* ) <sup>=</sup> (*ωr*, *<sup>ω</sup>l*). (36)

are determined by (*ωr*, *ωl*),

*̟*∗ *<sup>r</sup>* , *̟*<sup>∗</sup> *l* 

*ur* <sup>=</sup> *Kprer* <sup>+</sup> *Kir <sup>t</sup>*

*ul* <sup>=</sup> *Kplel* <sup>+</sup> *Kil <sup>t</sup>*

*er* <sup>=</sup> *̟*<sup>∗</sup>

*el* <sup>=</sup> *̟*<sup>∗</sup>

Using Matlab-Simulink, which allows the programming of mathematical models by means of blocks that facilitate the establishment of equations in a transparent and simple manner, this section presents the numerical simulations associated with the obstacle avoidance task developed previously, with the general results then applied to a scenario involving three obstacles within the WMR workspace. As mentioned earlier, the positions of the obstacles

Finally, *er* and *el* represent the angular velocity tracking errors defined by

(*̟*<sup>∗</sup> *<sup>r</sup>* , *̟*<sup>∗</sup>

This means that the desired angular velocity trajectories

which are obtained from (13).

**4. Simulations**

where *ur* and *ul* are the control voltages for the right and left motors, respectively, and *Kpr*, *Kir*, *Kpl*, and *Kil* are the constant gains (proportional and integral) associated with each motor.

*d̟<sup>r</sup>*

*d̟<sup>l</sup>*

*yr* = *̟r*,

*yl* = *̟l*.

are expressed as follows:

and

by

with

Characterization of *α* and *β* associated with the prototype's DC motors –two Engel GNM3150s (24 V, 55 W) with G2.6 gearboxes– was previously carried out in [14]. In this latter study it was found that the simplified model (27) of the right and left motors can be respectively expressed by

$$\begin{aligned} \frac{d\sigma\_r}{dt} &= -10.20\sigma\_r + 5.51u\_{r\prime} \\ \frac{d\sigma\_l}{dt} &= -10.20\sigma\_l + 5.99u\_{l\prime} \end{aligned} \tag{28}$$

where *̟<sup>r</sup>* and *̟<sup>l</sup>* are the right and left angular velocity of the motors, and *ur* and *ul* represent the armature voltages of the right and left motors, respectively. Hence, a PI control for (27) that achieves *̟* → *̟*<sup>∗</sup> is determined by

$$
\mu = K\_p e + K\_{\bar{i}} \int\_0^t e d\tau,\tag{29}
$$

with

$$
\sigma = \varpi^\* - \varpi,\tag{30}
$$

where *<sup>e</sup>* is the tracking error, *̟*<sup>∗</sup> is the desired angular velocity trajectory, *Kp* is the proportional gain, and *Ki* is the integral gain. Next, (29) is applied to the DC motors of the WMR.

#### **3.3. Hierarchical control**

This subsection presents the connection of the control laws developed via a hierarchical control, a block diagram of which is shown in Figure 5. At the upper level, an input-output linearization control was used for the WMR model, generating the desired velocity profiles, *<sup>ω</sup><sup>r</sup>* and *<sup>ω</sup>l*, for the robot wheels to track. These velocity profiles ensure that the WMR moves from the starting point to the goal point while avoiding the obstacles placed in between. At the lower level, two PI controllers for the DC motors were considered, ensuring that the actual wheel velocities followed the desired velocity profiles generated at the upper level.

In accordance with (28), the mathematical models associated with the right and left motors are expressed as follows:

$$\begin{array}{lcl}\frac{d\boldsymbol{\sigma}\_{r}}{dt} = -10.20\boldsymbol{\sigma}\_{r} + 5.51\boldsymbol{u}\_{r} \\ \boldsymbol{y}\_{r} = \boldsymbol{\sigma}\_{r} \end{array} \tag{31}$$

and

10

to both motor and load.

respectively expressed by

with

the WMR.

**3.3. Hierarchical control**

that achieves *̟* → *̟*<sup>∗</sup> is determined by

simplified model is determined by

where *u* is the motor armature voltage, *ia* is the armature current, *ke* is the back-electromotive force constant, *km* is the motor torque constant, *La* is the armature inductance, *Ra* is the armature resistance, *J* is the rotor and load inertia, and *b* is the viscous friction constant due

Since motor manufacturers do not generally provide all of the parameter values associated with (26), these values can be simply obtained via the reduction of (26) to a first order system that relates *̟* to *u*. This is accomplished by assuming that *La* ≈ 0 in (26). Hence, the

Characterization of *α* and *β* associated with the prototype's DC motors –two Engel GNM3150s (24 V, 55 W) with G2.6 gearboxes– was previously carried out in [14]. In this latter study it was found that the simplified model (27) of the right and left motors can be

*dt* <sup>=</sup> <sup>−</sup>10.20*̟<sup>r</sup>* <sup>+</sup> 5.51*ur*,

*dt* <sup>=</sup> <sup>−</sup>10.20*̟<sup>l</sup>* <sup>+</sup> 5.99*ul*,

where *̟<sup>r</sup>* and *̟<sup>l</sup>* are the right and left angular velocity of the motors, and *ur* and *ul* represent the armature voltages of the right and left motors, respectively. Hence, a PI control for (27)

where *<sup>e</sup>* is the tracking error, *̟*<sup>∗</sup> is the desired angular velocity trajectory, *Kp* is the proportional gain, and *Ki* is the integral gain. Next, (29) is applied to the DC motors of

This subsection presents the connection of the control laws developed via a hierarchical control, a block diagram of which is shown in Figure 5. At the upper level, an input-output linearization control was used for the WMR model, generating the desired velocity profiles, *<sup>ω</sup><sup>r</sup>* and *<sup>ω</sup>l*, for the robot wheels to track. These velocity profiles ensure that the WMR moves from the starting point to the goal point while avoiding the obstacles placed in between.

 *<sup>t</sup>* 0

*u* = *Kpe* + *Ki*

*dt* <sup>=</sup> <sup>−</sup>*α̟* <sup>+</sup> *<sup>β</sup>u*. (27)

*edτ*, (29)

*<sup>e</sup>* = *̟*<sup>∗</sup> − *̟*, (30)

(28)

*d̟*

*d̟<sup>r</sup>*

*d̟<sup>l</sup>*

$$\begin{array}{lcl}\frac{d\sigma\_l}{dt} = -10.20\sigma\_l + 5.99u\_{l\prime} \\ y\_l = \varpi\_l. \end{array} \tag{32}$$

Thus, two controls, *ur* and *ul*, are required; in accordance with (29), these are given by

$$
\mu\_r = K\_{pr} e\_r + K\_{ir} \int\_0^t e\_r d\tau\_r \tag{33}
$$

$$
\mu\_l = K\_{pl}e\_l + K\_{il} \int\_0^t e\_l d\tau\_\prime \tag{34}
$$

where *ur* and *ul* are the control voltages for the right and left motors, respectively, and *Kpr*, *Kir*, *Kpl*, and *Kil* are the constant gains (proportional and integral) associated with each motor. Finally, *er* and *el* represent the angular velocity tracking errors defined by

$$\begin{aligned} \boldsymbol{e}\_{r} &= \boldsymbol{\varpi}\_{r}^{\*} - \boldsymbol{\varpi}\_{r} \\ \boldsymbol{e}\_{l} &= \boldsymbol{\varpi}\_{l}^{\*} - \boldsymbol{\varpi}\_{l} \end{aligned} \tag{35}$$

with

$$\left(\left(\sigma\_{r\prime}^{\*}\sigma\_{l}^{\*}\right) = \left(\omega\_{r\prime}\omega\_{l}\right).\tag{36}$$

This means that the desired angular velocity trajectories *̟*∗ *<sup>r</sup>* , *̟*<sup>∗</sup> *l* are determined by (*ωr*, *ωl*), which are obtained from (13).

#### **4. Simulations**

Using Matlab-Simulink, which allows the programming of mathematical models by means of blocks that facilitate the establishment of equations in a transparent and simple manner, this section presents the numerical simulations associated with the obstacle avoidance task developed previously, with the general results then applied to a scenario involving three obstacles within the WMR workspace. As mentioned earlier, the positions of the obstacles

10.5772/58392

91

http://dx.doi.org/10.5772/58392

*q*(*x*1, *y*1) *qm*(*xm*, *ym*) *qobs* <sup>1</sup>(*xo*1, *yo*1) *qobs* <sup>2</sup>(*xo*2, *yo*2) *qobs* <sup>3</sup>(*xo*3, *yo*3) *x*<sup>1</sup> = 0 m *xm* = 2.2 m *xo*<sup>1</sup> = 0.2 m *xo*<sup>2</sup> = 1.2 m *xo*<sup>3</sup> = 1.5 m *y*<sup>1</sup> = 0 m *ym* = 1.7 m *yo*<sup>1</sup> = 0.4 m *yo*<sup>2</sup> = 0.4 m *yo*<sup>3</sup> = 1.6 m

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

**Table 2.** Coordinates associated with the goal and the obstacles.

**Figure 6.** Implementation of the hierarchical control in Matlab-Simulink for the WMR.

**4.2. Implementing the control via Matlab-Simulink**

*motors*, and *Kinematic model of the WMR*.

closed-loop system.

Once the values of the parameters associated with the WMR and the control have been defined, the hierarchical control can be programmed in Matlab-Simulink to simulate the

Figure 6 (bottom part) shows the three blocks associated with the hierarchical control programmed using Matlab-Simulink: *Input-output linearization control*, *PI controllers/DC*

1.- *Input-output linearization control* block. In this block, the control determined by (13), which accomplishes the obstacle avoidance task, is programmed. The inputs are the variables

*<sup>r</sup>* , *̟*<sup>∗</sup>

*<sup>l</sup>* ), the latter

(*x*1, *<sup>y</sup>*1, *<sup>ϕ</sup>*), and the outputs the desired velocity profiles (*ωr*, *<sup>ω</sup>l*)=(*̟*<sup>∗</sup>

*ρ*<sup>01</sup> = 0.5 m *ρ*<sup>02</sup> = 0.5 m *ρ*<sup>03</sup> = 0.5 m

**Figure 5.** Block diagram of the WMR hierarchical control.

within the workspace is assumed to be already known. The following subsections outline the stages of the simulation process, namely: the definition of the parameters of the system and of the hierarchical control, the implementation of the control via Matlab-Simulink, and finally the obtained results.

#### **4.1. Definition of parameters**

When implementing simulations associated with the closed-loop system, both WMR and control parameters must first be considered in order to use them later to program (via Matlab-Simulink) the block associated with the hierarchical control. In this case, the parameters associated with the WMR are *l*, *r*, and *L*, and those associated with the hierarchical control being *<sup>υ</sup>d*, *<sup>ε</sup>*, *<sup>η</sup>*, *<sup>ξ</sup>*, and *<sup>ρ</sup>*<sup>0</sup> due to the use of the artificial potential field method. The values of these parameters are shown in Table 1. It is worth mentioning that the values associated with the WMR, i.e., *l*, *r*, and *L*, correspond to the prototype reported in [14]. Finally, the coordinates associated with the goal, *qm*, and the obstacles, *qobs*, (for a three obstacle scenario) are presented in Table 2, where *ρ*<sup>0</sup> = *ρ*<sup>01</sup> = *ρ*<sup>02</sup> = *ρ*03. The declarations elaborated in Matlab-Simulink for all the above-mentioned parameters are shown in the upper part of Figure 6.


**Table 1.** Parameters employed in the development of the simulations.

<sup>90</sup> MATLAB Applications for the Practical Engineer Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application 13 10.5772/58392 Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application http://dx.doi.org/10.5772/58392 91


**Table 2.** Coordinates associated with the goal and the obstacles.

12

**Figure 5.** Block diagram of the WMR hierarchical control.

finally the obtained results.

upper part of Figure 6.

**Table 1.** Parameters employed in the development of the simulations.

**4.1. Definition of parameters**

within the workspace is assumed to be already known. The following subsections outline the stages of the simulation process, namely: the definition of the parameters of the system and of the hierarchical control, the implementation of the control via Matlab-Simulink, and

When implementing simulations associated with the closed-loop system, both WMR and control parameters must first be considered in order to use them later to program (via Matlab-Simulink) the block associated with the hierarchical control. In this case, the parameters associated with the WMR are *l*, *r*, and *L*, and those associated with the hierarchical control being *<sup>υ</sup>d*, *<sup>ε</sup>*, *<sup>η</sup>*, *<sup>ξ</sup>*, and *<sup>ρ</sup>*<sup>0</sup> due to the use of the artificial potential field method. The values of these parameters are shown in Table 1. It is worth mentioning that the values associated with the WMR, i.e., *l*, *r*, and *L*, correspond to the prototype reported in [14]. Finally, the coordinates associated with the goal, *qm*, and the obstacles, *qobs*, (for a three obstacle scenario) are presented in Table 2, where *ρ*<sup>0</sup> = *ρ*<sup>01</sup> = *ρ*<sup>02</sup> = *ρ*03. The declarations elaborated in Matlab-Simulink for all the above-mentioned parameters are shown in the

> **Constant Definition Value** *l* Distance from (*x*, *y*) to the wheels 0.220 m *r* Wheel ratio 0.075 m *L* Distance from (*x*, *y*) to *q* 0.250 m *υ<sup>d</sup>* Desired constant velocity 0.5 m/s *ε* Constant value close to zero 0.1 *η* Positive scale factor (repulsive) 2 *ξ* Positive scale factor (attractive) 1 *ρ*<sup>0</sup> Distance of the obstacle's influence 0.5 m

**Figure 6.** Implementation of the hierarchical control in Matlab-Simulink for the WMR.

Once the values of the parameters associated with the WMR and the control have been defined, the hierarchical control can be programmed in Matlab-Simulink to simulate the closed-loop system.

#### **4.2. Implementing the control via Matlab-Simulink**

Figure 6 (bottom part) shows the three blocks associated with the hierarchical control programmed using Matlab-Simulink: *Input-output linearization control*, *PI controllers/DC motors*, and *Kinematic model of the WMR*.

1.- *Input-output linearization control* block. In this block, the control determined by (13), which accomplishes the obstacle avoidance task, is programmed. The inputs are the variables (*x*1, *<sup>y</sup>*1, *<sup>ϕ</sup>*), and the outputs the desired velocity profiles (*ωr*, *<sup>ω</sup>l*)=(*̟*<sup>∗</sup> *<sup>r</sup>* , *̟*<sup>∗</sup> *<sup>l</sup>* ), the latter being tracked by the angular velocities of the right and left DC motors, respectively. Figure 7 presents the block's constituent sub-blocks.

10.5772/58392

93

http://dx.doi.org/10.5772/58392

(a) (b)

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

(c) (d)

(e) (f)

**Figure 8.** Simulation results in the three obstacles scenario.


**Figure 7.** Input-output linearization control block.

#### **4.3. Simulation results**

Finally, this subsection presents the simulation results associated with the obstacle avoidance task for the differentially-driven WMR. The simulations consider the presence of three obstacles within the workspace which have an influence over the WMR during its journey to the goal. Table 2 presents the coordinates associated with the goal and the three obstacles, with the distribution depicted in Figure 8(a) and Figure 14. The results obtained for the variables of interest to the WMR are shown in Figure 8; in this figure one can observe how the WMR successfully evades the obstacles and reaches the goal.

### **5. Description of the employed prototype**

This section provides a general description of the WMR prototype reported in [14], which was built to carry out various control tasks associated with WMRs. Figure 9 displays a block diagram describing the existent connections between the different stages of the employed WMR, namely: *Subsystems*, *Power system*, and *Data acquisition and control system*.

Here we present a summarized description of the blocks composing the WMR prototype, including its connection with the DS1104 board, as shown in Figure 9.

• Stage 1: *Subsystems*. This stage (see Figure 10) comprises the mechanical subsystems **a** and **b**, which include the actuators, sensors, and the mechanical structure of the WMR. Subsystem **a**, *actuators and sensors*, generates the movement of the WMR wheels in a specified workspace, and discrete position sensing, respectively. Subsystem **b**, *mechanical*

**Figure 8.** Simulation results in the three obstacles scenario.

being tracked by the angular velocities of the right and left DC motors, respectively.

respectively, with the outputs being the controls *ur* and *ul* in such a way that (*̟r*, *̟l*) →

3.- *Kinematic model of the WMR* block. In this block, the kinematic model of the WMR associated with the point *q*, determined by (12), is programmed. The inputs are (*̟r*, *̟l*),

Finally, this subsection presents the simulation results associated with the obstacle avoidance task for the differentially-driven WMR. The simulations consider the presence of three obstacles within the workspace which have an influence over the WMR during its journey to the goal. Table 2 presents the coordinates associated with the goal and the three obstacles, with the distribution depicted in Figure 8(a) and Figure 14. The results obtained for the variables of interest to the WMR are shown in Figure 8; in this figure one can observe how

This section provides a general description of the WMR prototype reported in [14], which was built to carry out various control tasks associated with WMRs. Figure 9 displays a block diagram describing the existent connections between the different stages of the employed

Here we present a summarized description of the blocks composing the WMR prototype,

• Stage 1: *Subsystems*. This stage (see Figure 10) comprises the mechanical subsystems **a** and **b**, which include the actuators, sensors, and the mechanical structure of the WMR. Subsystem **a**, *actuators and sensors*, generates the movement of the WMR wheels in a specified workspace, and discrete position sensing, respectively. Subsystem **b**, *mechanical*

WMR, namely: *Subsystems*, *Power system*, and *Data acquisition and control system*.

including its connection with the DS1104 board, as shown in Figure 9.

the WMR successfully evades the obstacles and reaches the goal.

**5. Description of the employed prototype**

*<sup>l</sup>* ). The gains associated with the PI controls were here selected as *Kpr* = 2, *Kir* =

*<sup>l</sup>* ) and the angular velocities produced by the motors (*̟r*, *̟l*),

2.- *PI controllers/DC motors* block. In this block, the PI controls, associated with the right and left motors, respectively determined by (33) and (34), are implemented. The inputs

Figure 7 presents the block's constituent sub-blocks.

*<sup>r</sup>* , *̟*<sup>∗</sup>

and the outputs the variables (*x*1, *y*1, *ϕ*).

are determined by (*̟*<sup>∗</sup>

50, *Kpl* = 2, and *Kil* = 50.

**Figure 7.** Input-output linearization control block.

**4.3. Simulation results**

(*̟*<sup>∗</sup> *<sup>r</sup>* , *̟*<sup>∗</sup>

10.5772/58392

95

http://dx.doi.org/10.5772/58392

**Figure 10.** Bottom view of the mechanical structure.

**Figure 11.** Block diagram of the power system.

sensor was used for the WMR in the present study.

was employed here to provide an estimation of the WMR position, since no localization

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

The Matlab-Simulink program associated with the hierarchical control used to obtain the experimental results for the closed-loop system is shown in Figure 13, which consists of the following blocks: *System parameters*, *Input-output linearization control*, *PI controllers/DC motors*, and *Kinematic model of the WMR*. Figure 13 also depicts the connections between the WMR and the hierarchical control. A comparison of Figures 6 and 13 reveals that the two are highly similar, with the only block experiencing any significant changes being the one associated with the *PI controllers/DC motors*. Hence, we can focus our attention solely on this block.

• *PI controllers/DC motors* block. This block describes the implementation of the PI controls, *ur* and *ul*, associated with the right and left motors, respectively, which allow (*̟r*, *̟l*) →

**Figure 9.** General block diagram of the WMR prototype.

*structure*, corresponds to the mechanical topology associated with the differentially driven WMR.


Figure 12 shows pictures of the real WMR, including the employed instrumentation.

#### **6. Real-time experiments**

In order to validate the data obtained via numeric simulation presented in Section 4, here we present the experimental results obtained using the WMR in real-time. These experiments were performed using Matlab-Simulink, ControlDesk, and a DS1104 board (dSPACE), with the distribution of the obstacles and the goal identical to that in the simulations.

#### **6.1. Real-time control of the WMR via Matlab-Simulink**

Whereas the mathematical models associated with the WMR and the DC motors were used to obtain the simulation results, these models were replaced by the WMR prototype in obtaining the experimental results. However, it is worth mentioning that the robot kinematic model <sup>94</sup> MATLAB Applications for the Practical Engineer Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application 17 10.5772/58392 Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application http://dx.doi.org/10.5772/58392 95

**Figure 10.** Bottom view of the mechanical structure.

16

**Figure 9.** General block diagram of the WMR prototype.

the WMR within the workspace.

the DS1104 board and the WMR.

**6. Real-time experiments**

WMR.

*structure*, corresponds to the mechanical topology associated with the differentially driven

• Stage 2: *Power system*. This stage enables interaction between the electronic control interface (stage 3) and the mechanical subsystems (stage 1). A block diagram of stage 2 is shown in Figure 11. This block comprises the following four substages, numbered from 1 to 4: *source circuit*, *optoisolator circuit*, *H-bridge circuit*, and *encoder circuit*. In substage 1, the power supply, via the source circuit, distributes the different voltages to the general electronic system. Substage 2 enables electrical signal isolation between the DS1104 electronic board and substage 3. Substage 3 enables the direction of rotation of the DC motors to be controlled via the use of a positive or a negative voltage, which is determined by the control exerted by the DS1104 board. Finally, substage 4 involves the acquisition of the encoders' signals, which can then be used to estimate the position of

• Stage 3: *Data acquisition and control system*. The main device involved in this stage is the DS1104 board, which performs the acquisition of the variables of interest to the experimental implementation of the WMR hierarchical control. This board was selected due to the potential for integration between Matlab-Simulink and the board's firmware. Moreover, the high programming level available in Simulink makes it a practical selection for the programming of complex control strategies in a graphical environment. Stage 3 also includes an *interface circuit* (see Figure 9) which establishes communication between

Figure 12 shows pictures of the real WMR, including the employed instrumentation.

the distribution of the obstacles and the goal identical to that in the simulations.

**6.1. Real-time control of the WMR via Matlab-Simulink**

In order to validate the data obtained via numeric simulation presented in Section 4, here we present the experimental results obtained using the WMR in real-time. These experiments were performed using Matlab-Simulink, ControlDesk, and a DS1104 board (dSPACE), with

Whereas the mathematical models associated with the WMR and the DC motors were used to obtain the simulation results, these models were replaced by the WMR prototype in obtaining the experimental results. However, it is worth mentioning that the robot kinematic model

**Figure 11.** Block diagram of the power system.

was employed here to provide an estimation of the WMR position, since no localization sensor was used for the WMR in the present study.

The Matlab-Simulink program associated with the hierarchical control used to obtain the experimental results for the closed-loop system is shown in Figure 13, which consists of the following blocks: *System parameters*, *Input-output linearization control*, *PI controllers/DC motors*, and *Kinematic model of the WMR*. Figure 13 also depicts the connections between the WMR and the hierarchical control. A comparison of Figures 6 and 13 reveals that the two are highly similar, with the only block experiencing any significant changes being the one associated with the *PI controllers/DC motors*. Hence, we can focus our attention solely on this block.

• *PI controllers/DC motors* block. This block describes the implementation of the PI controls, *ur* and *ul*, associated with the right and left motors, respectively, which allow (*̟r*, *̟l*) →

10.5772/58392

97

http://dx.doi.org/10.5772/58392

**Figure 13.** Experimental implementation of the controller via Matlab-Simulink and its connection with the WMR.

videos are available online at: www.controlautomatico.com.mx/trabajos.html

**Figure 14.** Position and orientation of the starting point of the WMR.

the workspace are presented in Videos S2 and S3, respectively, (see [25]). The aforementioned

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

**Figure 12.** WMR prototype.

(*̟*<sup>∗</sup> *<sup>r</sup>* , *̟*<sup>∗</sup> *<sup>l</sup>* ). In order to start the DC motors, the *ur* and *ul* voltages must be conditioned through PWM blocks, which are implemented via the DS1104 board. The PWM signals then go through the *power system* stage illustrated in Figure 9. For the acquisition of the angular velocities *̟<sup>r</sup>* and *̟l*, two E50S8 incremental encoders (Autonics) were employed in combination with Matlab-Simulink blocks.

#### **6.2. Experimental results**

This subsection presents the experimental results associated with the distribution of the obstacles and the goal mentioned in Table 2. Likewise, the locations of the WMR, obstacles, and goal within the workspace are illustrated in Figure 14. The corresponding experimental results for such a scenario, in which the parameters of the WMR and the gains of the controls were the same as those employed in the simulations, are shown in Figure 15. Additionally, supplementary material associated with these results can be seen through Video S1 (see [25]). Also, experimental results for the scenarios involving two and zero obstacles within <sup>96</sup> MATLAB Applications for the Practical Engineer Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application 19 10.5772/58392 Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application http://dx.doi.org/10.5772/58392 97

**Figure 13.** Experimental implementation of the controller via Matlab-Simulink and its connection with the WMR.

the workspace are presented in Videos S2 and S3, respectively, (see [25]). The aforementioned videos are available online at: www.controlautomatico.com.mx/trabajos.html

**Figure 14.** Position and orientation of the starting point of the WMR.

18

**Figure 12.** WMR prototype.

**6.2. Experimental results**

in combination with Matlab-Simulink blocks.

*<sup>l</sup>* ). In order to start the DC motors, the *ur* and *ul* voltages must be conditioned through PWM blocks, which are implemented via the DS1104 board. The PWM signals then go through the *power system* stage illustrated in Figure 9. For the acquisition of the angular velocities *̟<sup>r</sup>* and *̟l*, two E50S8 incremental encoders (Autonics) were employed

This subsection presents the experimental results associated with the distribution of the obstacles and the goal mentioned in Table 2. Likewise, the locations of the WMR, obstacles, and goal within the workspace are illustrated in Figure 14. The corresponding experimental results for such a scenario, in which the parameters of the WMR and the gains of the controls were the same as those employed in the simulations, are shown in Figure 15. Additionally, supplementary material associated with these results can be seen through Video S1 (see [25]). Also, experimental results for the scenarios involving two and zero obstacles within

(*̟*<sup>∗</sup> *<sup>r</sup>* , *̟*<sup>∗</sup>

10.5772/58392

99

http://dx.doi.org/10.5772/58392

*<sup>l</sup>* , respectively. Hence, the obstacle avoidance

**6.3. Discussion of the results**

by the WMR kinematic model, *̟*<sup>∗</sup>

**7. Conclusions**

developed hierarchical control performed successfully.

in order to reduce or eliminate said error.

**Acknowledgments**

**Author details**

Similar results were obtained in the simulation and experimental tests; in both cases the angular velocities of the DC motors, *̟<sup>r</sup>* and *̟l*, tracked the desired velocity profiles imposed

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

task was successfully carried out by the WMR, as verified experimentally in Figure 15(a). However, it is worth mentioning that an error arose that could only be appreciated visually and which thus does not appear in Figure 15(a): a gap of approximately 10 cm was observed between point *q* and point *qm*. This discrepancy likely occurred due to the fact that the position of *q* within the workspace was calculated indirectly via the WMR kinematic model. It was also observed that the voltages *ur* and *ul*, associated with the DC motors, did not surpass the (−24 V, +24 V) voltage interval: this was convenient since the nominal voltage of the employed DC motors is in the range of ±24 V. As a result it can be confirmed that the

Based on the artificial potential field approach, the present work has provided a solution to the obstacle avoidance task for a differentially-driven WMR via the design of a hierarchical control, for which a step-by-step guide has been presented that details the theory, simulation, and experimental implementation. Simulation results were obtained by developing a Matlab-Simulink program, since Simulink provides a graphical environment that facilitates the analysis, design, and construction of dynamic systems. In order to obtain experimental results, a Matlab-Simulink program was again employed, this time alongside ControlDesk and the DS1104 board. A comparison of the simulation and experimental results associated with the obstacle avoidance task revealed the good performance of the developed hierarchical control. However, as mentioned earlier, an error between the WMR and the goal arises that can only be observed visually, likely due to the fact that the position of the robot is calculated via the kinematic model of the WMR. Future work will aim to locate the WMR via a sensor

R. Silva-Ortigoza, H. Taud, M. Marciano-Melchor, and J. A. Álvarez-Cedillo acknowledge financial support from the Secretaría de Investigación y Posgrado del Instituto Politécnico Nacional (SIP-IPN), SNI-México, and the IPN programs EDI and COFAA. The work of C. Márquez-Sánchez was supported by CONACYT and BEIFI scholarships. V. M. Hernández-Guzmán acknowledges financial support from SNI-México. Finally, Rhomina and Joserhamón deserve a special mention from R. Silva-Ortigoza for their moral support

R. Silva-Ortigoza1, C. Márquez-Sánchez1, F. Carrizosa-Corral2, V. M. Hernández-Guzmán3,

and for being the inspiration for his introduction to the thrilling world of robotics.

J. R. García-Sánchez1, H. Taud1, M. Marciano-Melchor1, and J. A. Álvarez-Cedillo1

*<sup>r</sup>* and *̟*<sup>∗</sup>

**Figure 15.** Experimental results for the three obstacles scenario.

#### **6.3. Discussion of the results**

20

(a) (b)

(c) (d)

(e) (f)

**Figure 15.** Experimental results for the three obstacles scenario.

Similar results were obtained in the simulation and experimental tests; in both cases the angular velocities of the DC motors, *̟<sup>r</sup>* and *̟l*, tracked the desired velocity profiles imposed by the WMR kinematic model, *̟*<sup>∗</sup> *<sup>r</sup>* and *̟*<sup>∗</sup> *<sup>l</sup>* , respectively. Hence, the obstacle avoidance task was successfully carried out by the WMR, as verified experimentally in Figure 15(a). However, it is worth mentioning that an error arose that could only be appreciated visually and which thus does not appear in Figure 15(a): a gap of approximately 10 cm was observed between point *q* and point *qm*. This discrepancy likely occurred due to the fact that the position of *q* within the workspace was calculated indirectly via the WMR kinematic model. It was also observed that the voltages *ur* and *ul*, associated with the DC motors, did not surpass the (−24 V, +24 V) voltage interval: this was convenient since the nominal voltage of the employed DC motors is in the range of ±24 V. As a result it can be confirmed that the developed hierarchical control performed successfully.

### **7. Conclusions**

Based on the artificial potential field approach, the present work has provided a solution to the obstacle avoidance task for a differentially-driven WMR via the design of a hierarchical control, for which a step-by-step guide has been presented that details the theory, simulation, and experimental implementation. Simulation results were obtained by developing a Matlab-Simulink program, since Simulink provides a graphical environment that facilitates the analysis, design, and construction of dynamic systems. In order to obtain experimental results, a Matlab-Simulink program was again employed, this time alongside ControlDesk and the DS1104 board. A comparison of the simulation and experimental results associated with the obstacle avoidance task revealed the good performance of the developed hierarchical control. However, as mentioned earlier, an error between the WMR and the goal arises that can only be observed visually, likely due to the fact that the position of the robot is calculated via the kinematic model of the WMR. Future work will aim to locate the WMR via a sensor in order to reduce or eliminate said error.

#### **Acknowledgments**

R. Silva-Ortigoza, H. Taud, M. Marciano-Melchor, and J. A. Álvarez-Cedillo acknowledge financial support from the Secretaría de Investigación y Posgrado del Instituto Politécnico Nacional (SIP-IPN), SNI-México, and the IPN programs EDI and COFAA. The work of C. Márquez-Sánchez was supported by CONACYT and BEIFI scholarships. V. M. Hernández-Guzmán acknowledges financial support from SNI-México. Finally, Rhomina and Joserhamón deserve a special mention from R. Silva-Ortigoza for their moral support and for being the inspiration for his introduction to the thrilling world of robotics.

### **Author details**

R. Silva-Ortigoza1, C. Márquez-Sánchez1, F. Carrizosa-Corral2, V. M. Hernández-Guzmán3, J. R. García-Sánchez1, H. Taud1, M. Marciano-Melchor1, and J. A. Álvarez-Cedillo1

1 Instituto Politécnico Nacional, CIDETEC, Área de Mecatrónica, Unidad Profesional Adolfo López Mateos, México, DF, Mexico

10.5772/58392

101

http://dx.doi.org/10.5772/58392

[11] K. Valanavis, T. Hebert, R. Kolluru, and N. Tsourveloudis, "Mobile robot navigation in 2-D dynamic enviroments using an electrostatic potential field," *IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans*, vol. 30, no. 2, pp. 187–196,

Obstacle Avoidance Task for a Wheeled Mobile Robot – A Matlab-Simulink-Based Didactic Application

[12] F. Xu, H. Van Brussel, M. Nuttin, and R. Moreas, "Concepts for dynamic obstacle avoidance and their extended application in underground navigation," *Robotics and*

[13] W. Huang, B. Fajen, J. Fink, and W. Warren, "Visual navigation and obstacle avoidance using a steering potential function," *Robotics and Autonomous Systems*, vol. 54, no. 4, pp.

[14] R. Silva-Ortigoza, C. Márquez-Sánchez, M. Marcelino-Aranda, M. Marciano-Melchor, G. Silva-Ortigoza, R. Bautista-Quintero, E. R. Ramos-Silvestre, J. C. Rivera-Díaz, and D. Muñoz-Carrillo, "Construction of a WMR for trajectory tracking control: Experimental results," *The Scientific World Journal*, vol. 2013, Article ID 723645, pp. 1–17, 2013.

[15] V. M. Hernández-Guzmán, R. Silva-Ortigoza y R. V. Carrillo-Serrano, *Control Automático: Teoría de Diseño, Construcción de Prototipos, Modelado, Identificación y Pruebas Experimentales*, Colección CIDETEC–IPN, Mexico, DF, Mexico, 2013. Available at

[16] S. S. Ge and Y. J. Cui, "New potential functions for mobile robot path planning," *IEEE*

[17] B. H. Krogh, "A generalized potential field approach to obstacle avoidance control,"

[18] J. R. García Sánchez, *Diseño y construcción de un robot móvil, aplicando el método de campos potenciales en la evasión de obstáculos*. Tesis de Maestría. CIDETEC del Instituto Politécnico

[19] A. W. Divelbiss and J. T. Wen, "Trajectory tracking control of a car-trailer system," *IEEE*

[20] R. Silva-Ortigoza, G. Silva-Ortigoza, V. M. Hernández-Guzmán, V. R. Barrientos-Sotelo, J. M. Albarrán-Jiménez, and V. M. Silva-García, "Trajectory tracking in a mobile robot without using velocity measurements for control of wheels," *IEEE Latin America Transactions*, vol. 6, no. 7, pp. 598–607, 2008. Available at

[21] R. Silva-Ortigoza, J. R. García-Sánchez, J. M. Alba-Martínez, V. M. Hernández-Guzmán, M. Marcelino-Aranda, H. Taud, and R. Bautista-Quintero, "Two-stage control design of a Buck converter/DC motor system without velocity measurements via a Σ − ∆-modulator," *Mathematical Problems in Engineering*, vol. 2013, Article ID 929316, pp.

*Transactions on Robotics and Automation*, vol. 16, no. 5, pp. 615–620, 2000.

*International Robotics Research Conference*, Bethlehem, PA, August 1984.

*Transactions on Control Systems Technology*, vol. 5, no. 3, pp. 269–278, 1997.

1–11, 2013. Available at http://dx.doi.org/10.1155/2013/929316

*Autonomous Systems*, vol. 42, no. 1, pp. 1–15, 2003.

Available at http://dx.doi.org/10.1155/2013/723645

http://www.controlautomatico.com.mx

Nacional, Mexico City, Mexico, 2008.

http://www.ewh.ieee.org/reg/9/etrans/eng/

2000.

288–299, 2006.

2 Instituto Tecnológico de Culiacán, Departamento de Metal-Mecánica, Culiacán, Sin, Mexico

3 Universidad Autónoma de Querétaro, Facultad de Ingeniería, Querétaro, Qro, Mexico

#### **References**


[11] K. Valanavis, T. Hebert, R. Kolluru, and N. Tsourveloudis, "Mobile robot navigation in 2-D dynamic enviroments using an electrostatic potential field," *IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans*, vol. 30, no. 2, pp. 187–196, 2000.

22

Mexico

**References**

1969.

pp. 566–571, May 1990.

López Mateos, México, DF, Mexico

http://www.ewh.ieee.org/reg/9/etrans/eng/

*Mathematics*, vol. 4, no. 3, pp. 298–351, 1983.

*and Automation*, pp. 1398–1404, April 1991.

vol. 11, no. 2, pp. 247–254, 1995.

*Automation*, pp. 674–680, May 1989.

1 Instituto Politécnico Nacional, CIDETEC, Área de Mecatrónica, Unidad Profesional Adolfo

2 Instituto Tecnológico de Culiacán, Departamento de Metal-Mecánica, Culiacán, Sin,

[1] R. Silva-Ortigoza, M. Marcelino-Aranda, G. Silva-Ortigoza, V. M. Hernández-Guzmán, M. A. Molina-Vilchis, G. Saldaña-González, J. C. Herrera-Lozada, and M. Olguín-Carbajal, "Wheeled mobile robots: A review," *IEEE Latin America Transactions*, vol. 10, no. 6, pp. 2209–2217, 2012. Available at

[2] J. L. Crowley, "World modeling and position estimation for a mobile robot using ultrasonic rangin," *in Proceedings of the IEEE International Conference on Robotics and*

[3] J. T. Schwartz and M. Sharir, "On the piano movers problem. II. General techniques for computing topological properties of real algebraic manifolds," *Advances in Applied*

[4] N. J. Nilsson, "A mobile automaton: An application of artificial intelligence techniques," *in Proceedings of the International Joint Conference on Artificial Intelligence*, pp. 509–520, May

[5] O. Khatib, "Real-Time obstacle avoidance for manipulators and mobile robots, " *The*

[6] J. Borenstein and Y. Koren, "Obstacle avoidance with ultrasonic sensors," *IEEE Journal*

[7] J. Borenstein and Y. Koren, "Real-Time obstacle avoidance for fast mobile robots, " *IEEE Transactions on Systems, Man and Cybernetics*, vol. 19, no. 5, pp. 1179–1187, 1989.

[8] R. Tilove, "Local obstacle avoidance for mobile robots based on the method of artificial potentials," *in Proceedings of the IEEE International Conference on Robotics and Automation*,

[9] J. Borenstein and Y. Koren, "Potential field methods and their inherent limitations for mobile robot navigation," *in Proceedings of the IEEE International Conference on Robotics*

[10] J. Guldner and V. I. Utkin, "Sliding mode control for gradient tracking and robot navigation usig artificial potential fields," *IEEE Transactions on Robotics and Automation*,

*international Journal of Robotics Research*, vol. 5, no. 2, pp. 90–98, 1986.

*of Robotics and Automation*, vol 4, no. 2, pp. 213–218, 1988.

3 Universidad Autónoma de Querétaro, Facultad de Ingeniería, Querétaro, Qro, Mexico


[22] R. Silva-Ortigoza, C. Márquez-Sánchez, F. Carrizosa-Corral, M. Antonio-Cruz, J. M. Alba-Martínez, and G. Saldaña-González, "Hierarchical velocity control based on differential flatness for a DC/DC Buck converter-DC motor System," *Mathematical Problems in Engineering*, vol. 2014, Article ID 912815, pp. 1–12, 2014. Available at http://dx.doi.org/10.1155/2014/912815

**Chapter 4**

**Dual Heuristic Neural Programming Controller for**

This chapter describes the development of voltage control system of a synchronous generator based on neural networks. Recurrent (dynamic) neural networks (RNN) are used, as a type that has great capabilities in approximation of dynamic systems [1]. Two algorithms are used for training – Dual Heuristic Programming (DHP) and Globalized Dual Heuristic Program‐ ming (GDHP). The algorithms have been developed for the optimal control of nonlinear

Neural voltage controller is developed in MATLAB and Simulink environment. For training purposes a mathematical model of synchronous generator is designed and applied in Simu‐ link. DHP and GDHP algorithms are designed in Simulink, with matrix calculations in Sfunctions. Algorithms are used for offline training of neural networks (NN). In the second part, the same functions are redesigned as real time controllers, based on the Real Time Windows

DHP or GDHP algorithms provide a significant improvement over conventional PI controller with, in some cases, power system stabilizer (PSS). Conventional linear controller can only provide optimal control in single operating point. Algorithms described in the chapter provide significantly better control over the whole operating range by minimization of the user defined

Digital voltage controller is implemented on real system in two basic steps. First step is neural network design and training in Matlab environment on desktop computer. Second step consists of transfer of controller with computed coefficients to the digital control system

> © 2014 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**Synchronous Generator**

http://dx.doi.org/10.5772/58377

**1. Introduction**

Target Toolbox.

hardware.

Mato Miskovic, Ivan Miskovic and Marija Mirosevic

Additional information is available at the end of the chapter

systems using dynamic programming principles.

cost function during the training of the neural network.

