**4. Hidden Markov models for behavioral modeling of smart cities IoT data**

As discussed, the main tasks of City4Age analytic framework are recognition of behavioral patterns, behavior changes (transitions) in time and anomaly detection. Additionally, models derived from data should be interpretable in order to integrate data driven insights with domain knowledge expertise. Hidden Markov models (HMMs) provide a framework for all main tasks and thus we employed these models for behavior variation analyses. Additionally, HMMs allow prediction of identified behavioral patterns in future and this adds predictive and preventive component in analytic framework. Here we will consider first order HMMs where each temporal state depends only on one previous state. This is strong assumption, but allows development of scalable models and real-time inference. **Figure 3** describes first order Markov chain where each state *x* depends on previous state (*x*-1) and observed data *(y)*.

Hidden Markov models can be explained as total probability of *X* and *Y* by following formulae:

$$p(\mathbf{X}, \mathbf{Y}) = \left. p(\mathbf{x}\_{\iota}) \prod\_{l=1}^{t+1} p\left(\mathbf{x}\_{\iota+1} \mid \mathbf{x}\_{\iota}\right) \prod\_{l'}^{t+1} p\left(y\_{r'} \mid \mathbf{x}\_{r'}\right) \tag{1}$$

Based on HMM definition, we can work on following tasks [10]:

,y2 ,…,yT

This task is solved by Viterbi (backward algorithm).

ing sequence of observations y1

**Figure 4.** Behavior modeling with Gaussian HMMs.

groups of care recipients.

**detection**

Training—Learning parameters of HMM (A, B, and the prior distribution π), given a train-

Decoding—given an observation sequence and an HMM, determine the most probable hidden state (behavior) sequence. We used this task for state prediction and model evaluation.

Likelihood—Calculation of probability that given sequence originates from given HMM model. In this research, we did not work on this task, since we built personalized behavioral models, but it will be used in later stages of the project when we will model behavior of

Based on definition of behavior as pattern of sequences of activities and corresponding measure values, clustering algorithms emerge as natural algorithmic approach for behavioral pattern recognition and change detection. In City4Age setting, inputs for clustering algorithms are time series. These time series can be represented by values of activity measures, GES, GEF or Geriatric Score of care recipients. Based on time series, temporal clustering algorithms can identify patterns (similar time series values in consecutive time-steps) that are repeated over time. We characterize these patterns as behaviors and transition between patterns, behavior changes. Very important component in derivation of GES and GEF from activity measures are numerical indicators (NUIs). NUIs represent aggregations (e.g., mean, std., trend, etc.) of activity measure values on monthly level. This granularity level is convenient since it allows direct

behavioral patterns (distributions). This task is solved by forward-backward algorithm.

**5. Framework for behavioral pattern recognition and change** 

. By solving this task, we will be able to characterize

Temporal Clustering for Behavior Variation and Anomaly Detection from Data Acquired…

http://dx.doi.org/10.5772/intechopen.75203

123

where *p*(*yt*′ |*xt* ) represents observation probability, while *p*(*xt*+1|*xt* ) prepresents transition probability.

In our case observations are series of IoT sensory data while hidden states represent categorized, homogenous series parts (that wil.l be characterized as behavioral patterns or behaviors). This is why we use Gaussian HMMs that characterize states with Gaussian distributions. This is depicted on **Figure 4**.

Each HMM model is thus constituted from three elements:


**Figure 3.** First-order Markov chain.

Temporal Clustering for Behavior Variation and Anomaly Detection from Data Acquired… http://dx.doi.org/10.5772/intechopen.75203 123

**Figure 4.** Behavior modeling with Gaussian HMMs.

**4. Hidden Markov models for behavioral modeling of smart cities** 

As discussed, the main tasks of City4Age analytic framework are recognition of behavioral patterns, behavior changes (transitions) in time and anomaly detection. Additionally, models derived from data should be interpretable in order to integrate data driven insights with domain knowledge expertise. Hidden Markov models (HMMs) provide a framework for all main tasks and thus we employed these models for behavior variation analyses. Additionally, HMMs allow prediction of identified behavioral patterns in future and this adds predictive and preventive component in analytic framework. Here we will consider first order HMMs where each temporal state depends only on one previous state. This is strong assumption, but allows development of scalable models and real-time inference. **Figure 3** describes first order Markov chain where each state *x* depends on previous state (*x*-1) and observed data *(y)*.

Hidden Markov models can be explained as total probability of *X* and *Y* by following formulae:

In our case observations are series of IoT sensory data while hidden states represent categorized, homogenous series parts (that wil.l be characterized as behavioral patterns or behaviors). This is why we use Gaussian HMMs that characterize states with Gaussian distributions.

**1.** Prior probability distribution of hidden states (vector *π*) that describes how frequently

**2.** Transition matrix (*Ai,j*) that describe the transition probabilities from one state to another. **3.** Probability distribution functions (one for each state) with corresponding parameters. In our case Gaussian distributions are modeled and thus means and standard deviations are used for definition of hidden state (behavior) probability distribution. HMMs allow modeling of discrete data too, but in that case probability distributions are represented by con-

*p*(*xt*+1 | *xt*)∏

*T t*′=1 *p*(*yt*′ | *xt*′

) (1)

) prepresents transition probability.

*T*−1 *t*=1

) represents observation probability, while *p*(*xt*+1|*xt*

Each HMM model is thus constituted from three elements:

*p*(*X*,*Y*) = *p*(*x*1)∏

**IoT data**

122 Recent Applications in Data Clustering

where *p*(*yt*′


This is depicted on **Figure 4**.

ditional distributions.

**Figure 3.** First-order Markov chain.

each state occurs in general.

Based on HMM definition, we can work on following tasks [10]:

Training—Learning parameters of HMM (A, B, and the prior distribution π), given a training sequence of observations y1 ,y2 ,…,yT . By solving this task, we will be able to characterize behavioral patterns (distributions). This task is solved by forward-backward algorithm.

Decoding—given an observation sequence and an HMM, determine the most probable hidden state (behavior) sequence. We used this task for state prediction and model evaluation. This task is solved by Viterbi (backward algorithm).

Likelihood—Calculation of probability that given sequence originates from given HMM model. In this research, we did not work on this task, since we built personalized behavioral models, but it will be used in later stages of the project when we will model behavior of groups of care recipients.
