Novel Fluorescence Imaging Techniques and Methodologies

### **Chapter 5**

## Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review

*Yahui Li, Lixin Liu, Dong Xiao, Hang Li, Natakorn Sapermsap, Jinshou Tian, Yu Chen and David Day-Uei Li*

### **Abstract**

Fluorescence lifetime imaging (FLIM) is powerful for monitoring cellular microenvironments, protein conformational changes, and protein interactions. It can facilitate metabolism research, drug screening, DNA sequencing, and cancer diagnosis. Lifetime determination algorithms (LDAs) adopted in FLIM analysis can influence biological interpretations and clinical diagnoses. Herein, we discuss the commonly used and advanced time-domain LDAs classified in fitting and non-fitting categories. The concept and explicit mathematical forms of LDAs are reviewed. The output lifetime parameter types are discussed, including lifetime components, average lifetimes, and graphic representation. We compare their performances, identify trends, and provide suggestions for end users in terms of multi-exponential decay unmixing ability, lifetime estimation precision, and processing speed.

**Keywords:** fluorescence lifetime imaging, lifetime determination algorithm, fitting method, non-fitting method, deep learning

### **1. Introduction**

Fluorescence lifetime imaging (FLIM) is a vital and versatile technique for assessing molecular microenvironments of fluorophores in living cells, such as pH, O2, viscosity, temperature, or ion concentrations [1, 2]. FLIM can be a powerful "quantum ruler" to measure subnanometer protein conformational changes and interactions by quantifying the occurrence of Förster Resonance Energy Transfer (FRET) [3–7]. FLIM has been used in diverse disciplines, including biology, chemistry and biophysics [8–12]; however, it is an indirect imaging technique that needs sophisticated data analysis to deliver meaningful information. FLIM analysis can profoundly impact the interpretations of biochemical and physical phenomena.

In time-domain approaches, FLIM usually measures a three-dimensional data cube (*x*-*y*-*t*), obtained with a time-correlated single-photon counting (TCSPC) system [13–16], a time-gated camera [17–21], or a streak camera [22–28]. A time-resolved histogram *hm* at (*x*, *y*) in a measured data cube can be expressed as:

$$h\_m = \sum\_{k=0}^{m} i \boldsymbol{\eta} \boldsymbol{f}\_{k-m} \cdot \boldsymbol{f}\_m + c\_m,\tag{1}$$

$$\dot{\imath}\circ f\_m = \int\_{m\Delta t}^{(m+1)\Delta t} \dot{\imath}\circ f(t)dt,\\ f\_m = \int\_{m\Delta t}^{(m+1)\Delta t} f(t)dt,\\ m = 0, \ldots, M-1,\tag{2}$$

where *M* is the number of time-bins and Δ*t* is the bin width.

*f*(*t*) in Eq. (1) is the underlying fluorescence decay, usually following a multiexponential decay model,

$$f(t) = A \sum\_{p=0}^{p-1} q\_p \exp\left(-t/\tau\_p\right), \sum\_{p=0}^{p-1} q\_p = \mathbf{1},\tag{3}$$

where *A* is the amplitude, *qp* and *τ<sup>p</sup>* are the fraction and lifetime of the *pth* component (*p* = 0, … , *P*-1).

*irf*(*t*) is the instrument response function (IRF), often measured using a sample with a much shorter lifetime than the width of the excitation pulse or a scattering solution. The full width at half maximum (FWHM) of IRF, Δ*t*IRF, is given by [16, 29].

$$
\Delta t\_{\rm IRF}^2 = \Delta t\_{\rm optical}^2 + \Delta t\_{\rm tts}^2 + \Delta t\_{\rm jitter}^2,\tag{4}
$$

where Δ*t*optical is the optical pulse width, Δ*t*tts is the detector transit time spread, and Δ*t*jitter is the detection and timing electronics jitter. With *h* = (*h*0, … , *hM*-1) and *irf* = (*irf*0, … , *irfM*-1) already measured, *A*, *q* = (*q*0, … , *qP*-1) and *τ* = (*τ*0, … , *τP*-1) can be extracted with a lifetime determination algorithm (LDA).

LDAs can be divided into two categories: fitting and non-fitting methods.

Fitting methods solve a nonlinear minimization problem *argmin χ* 2 , where *χ* <sup>2</sup> is a merit function revealing the goodness of fit. *χ* <sup>2</sup> is determined by statistics models, such as least square estimation (LSE) or maximum likelihood estimation (MLE), using pixelwise or global fitting modes [30]. Fitting methods suffer from slow analysis due to extensive intrinsic convolutions. The Laguerre expansion method converts the nonlinear-fitting problem to a linear-fitting problem [31, 32], and speeds up deconvolution procedures.

Non-fitting methods can provide lifetime parameters much faster than fitting methods, but some can only provide average lifetimes or graphic representation rather than specific lifetime components. Non-fitting methods should be used carefully according to applications. As discussed in [33–35], the intensity-weighted average lifetime, *<sup>τ</sup><sup>I</sup>* <sup>¼</sup> <sup>P</sup>*<sup>P</sup>*�<sup>1</sup> *<sup>p</sup>*¼<sup>0</sup> *qpτ*<sup>2</sup> *p=* P*<sup>P</sup>*�<sup>1</sup> *<sup>p</sup>*¼<sup>0</sup> *qpτp*, can estimate the average collisional constant *kq* from the Stern–Volmer constant *KD*, whereas the amplitude-weighted average lifetime, *<sup>τ</sup><sup>A</sup>* <sup>¼</sup> <sup>P</sup>*<sup>P</sup>*�<sup>1</sup> *<sup>p</sup>*¼<sup>0</sup> *qpτp*, can estimate FRET efficiency and assess dynamic quenching behaviors, as described by the Stern–Volmer equation. *τ<sup>I</sup>* was previously misused in FRET efficiency estimation, introducing significant bias when the decay follows a multi-exponential decay model.

The practical use of diverse LDAs depends on applications. Due to the timeconsuming estimation procedure, fitting methods are generally used for offline analysis, providing more decay information, including lifetime components and average

#### *Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

lifetimes, than most non-fitting methods. Meanwhile, non-fitting methods are suitable for real-time FLIM applications, as they are much faster and more hardware-friendly.

This work attempts to review widely used and newly developed cutting-edge LDAs for time-domain FLIM analysis. The subsequent sections are arranged as follows. Section 2 reviews fitting methods, such as the least squares estimation, maximum likelihood estimation, global fitting using iterative convolution and variable projection approaches, and Laguerre expansion deconvolution methods. Section 3 reviews nonfitting methods, including rapid lifetime determination, center-of-mass, integral extraction, phasor, *τA*/*τI*, deep learning, and histogram clustering methods. Finally, we conclude this review in Section 4 and speculate on future research directions.

### **2. Fitting methods**

Fitting methods use iterative constrained optimization algorithms to estimate fitting parameters based on a specified decay kinetics model. Feedback is provided at each iteration for updating or terminating the process with a criterion. The weighted least squares estimation (LSE) and the maximum likelihood estimation (MLE) are commonly used for FLIM analysis, and these methods have been compared using synthetic and experimental data [36–39]. Although it is well known that MLE is more efficient and accurate than LSE, as FLIM data is Poisson distributed rather than Gaussian distributed [40–44], LSE is still more popular than MLE in FLIM analysis. Attempts have been made to provide robust MLE fitting algorithms, which are counterparts of the Levenberg–Marquardt procedure for LSE [45–47].

The global fitting method can accelerate analysis by assuming spatial lifetime invariances to reduce the degree of freedom significantly. The Laguerre expansion method can accelerate analysis by converting the nonlinear-fitting problem to a linear-fitting problem to speed up deconvolution procedures.

#### **2.1 Weighted least squares estimation (LSE)**

The weighted LSE defines a merit function as [48, 49],

$$\chi^2 = \sum\_{m} \frac{\left(\hat{h}\_m - \hat{h}\_m\right)^2}{\sigma\_m^2},\tag{5}$$

where *hm* is the measured counts, ^ *hm* is the predicted counts, and *σ<sup>m</sup>* is the count deviation in Bin *m*. By minimizing *χ*2, lifetime parameters can be obtained. For Poisson distributed data, the variance is equal to the actual value *hm true* of Bin *m*, that is, *σ<sup>m</sup>* ¼ ffiffiffiffiffiffiffiffi *htrue m* q . As this actual value is not available, *χ*<sup>2</sup> can be approximated as:

$$\chi\_P^2 = \sum\_m \frac{\left(h\_m - \hat{h}\_m\right)^2}{h\_m}, \chi\_N^2 = \sum\_m \frac{\left(h\_m - \hat{h}\_m\right)^2}{m\omega\left(h\_m, 1\right)},\tag{6}$$

where *χ*<sup>2</sup> *<sup>P</sup>* and *χ*<sup>2</sup> *<sup>N</sup>* are Pearson's and modified Neyman's *χ*2. Studies show that Neyman weighting underestimates the target answer, whereas Pearson weighting affords an acceptable answer when the total count is more than 1000 [39].

#### **2.2 Maximum likelihood estimation (MLE)**

MLE maximizes the probability that the data can occur given a model and a set of parameters. The likelihood *L* can be expressed as [50, 51].

$$L = \prod\_{m} P\left(h\_m; \hat{h}\_m\right),\tag{7}$$

where *P hm*; ^ *hm* � � is the probability that Bin *<sup>m</sup>* has *hm* counts if the actual value is ^ *hm*

. Because the measurements are Poisson distributed, the Poisson likelihood becomes.

$$L\_P = \prod\_m \frac{\hat{h}\_m^{\ldots h\_m}}{h\_m!} \mathbf{e}^{-\hat{h}\_m}.\tag{8}$$

If Eq. (8) is divided by the maximum possible likelihood *LP*ð Þ *hm*; *hm* , then

$$
\lambda = L\_P \left( h\_m; \hat{h}\_m \right) / L\_P(h\_m; h\_m). \tag{9}
$$

A merit function can be defined as:

$$\chi^2\_{\text{mle}} = -2\ln\lambda = 2\sum\_{m} \left(\hat{h}\_m - h\_m\right) - 2\sum\_{m,h\_m \neq 0} h\_m \ln\left(\hat{h}\_m / h\_m\right). \tag{10}$$

Based on the Poisson likelihood function, MLE can ensure unbiased estimations.

#### **2.3 Global fitting (GF)**

The global fitting method uses a least squares estimate *χ*<sup>2</sup> *GF* for all histograms as the merit function,

$$\chi^2\_{GF} = \sum\_{s} \sum\_{m} \frac{\left(h\_m^{(s)} - \hat{h}\_m^{(s)}\right)^2}{\sigma\_m^{(s)2}},\tag{11}$$

where the superscript *s* represents Histogram *s*. GF treats the lifetime components *τ* as constants, but the amplitude *A* and fraction *q* as variables for all histograms. There are two strategies for implementing GF, the iterative convolution and the variable projection approaches. The variable projection approach appears to be faster than the iterative convolution method, as investigated in [52].

#### *2.3.1 Iterative convolution*

The underlying decay is estimated with.

$$\hat{f}\_{m}^{(\circ)} = \hat{A}^{(\circ)} \sum\_{p=0}^{P-1} \hat{q}\_{p}^{(\circ)} e^{\frac{-t\_{m}}{\hat{q}\_{p}}},\tag{12}$$

where ^*τ<sup>p</sup>* are estimated constant lifetimes for all histograms with *<sup>A</sup>*^ð Þ*<sup>s</sup>* and ^*qp* ð Þ*s* being the parameters for Histogram *s*.

*Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

Then the estimated signal can be expressed as:

$$\hat{h}\_{m}^{(\boldsymbol{\ell})} = \sum\_{k=0}^{m} i \boldsymbol{\eta} \mathbf{f}\_{k-m} \cdot \hat{\boldsymbol{f}}\_{m}^{(\boldsymbol{\ell})}, m = \mathbf{0}, \ \dots, M-\mathbf{1}.\tag{13}$$

Using Eq. (13), we can minimize Eq. (11) with constrained LSE. The analysis speed is significantly affected by chosen initial conditions. S. Pelet *et al.* compared different strategies for initial guesses [30] and proposed an efficient image segmentation method.

#### *2.3.2 Variable projection*

The idea of global fitting with variable projection is to minimize a projection function that depends only on nonlinear parameters *τ* and obtain linear parameters *A*(s) and *q*(s). A matrix whose columns only depend on *τ* is constructed,

$$\Phi(\hat{\mathbf{r}}) = [\mathfrak{p}\_0(\hat{\mathbf{r}}\_0), \dots, \mathfrak{p}\_{P-1}(\hat{\mathbf{r}}\_{P-1})],\tag{14}$$

where *φ<sup>p</sup>* ^*τ<sup>p</sup>* � � <sup>¼</sup> *<sup>φ</sup><sup>p</sup>* ^*τp*; *<sup>t</sup>*<sup>0</sup> � �, … , *<sup>φ</sup><sup>p</sup>* ^*τp*; *tM*�<sup>1</sup> � � h i*<sup>T</sup>* , *φ<sup>p</sup>* ^*τp*; *tm* � � <sup>¼</sup> <sup>P</sup>*<sup>m</sup> <sup>k</sup>*¼<sup>0</sup>*irf <sup>k</sup>*�*<sup>m</sup>* � exp � *tm* ^*τp* � �, and *<sup>p</sup>* = 0, … , *<sup>P</sup>*-1.

Then the estimated signal can be written as:

$$\hat{h}\_{m}^{(\boldsymbol{\epsilon})} = \sum\_{p=0}^{P-1} \hat{a}\_{p}^{(\boldsymbol{\epsilon})} \rho\_{p} \left( \hat{\tau}\_{p}; t\_{m} \right), \tag{15}$$

where *a*^*<sup>p</sup>* ð Þ*<sup>s</sup>* <sup>¼</sup> *<sup>A</sup>*^ð Þ*<sup>s</sup>* ^*qp* ð Þ*<sup>s</sup>* . Eq. (11) becomes.

$$\chi\_{GF}^{2} = \sum\_{s} \sum\_{m} \frac{\left(h\_{m}^{(s)} - \sum\_{p=0}^{P-1} \hat{a}\_{p}^{(s)} \rho\_{p} \left(\hat{\mathbf{r}}\_{p}; t\_{m}\right)\right)^{2}}{\sigma\_{m}^{(s)2}},\tag{16}$$

$$\chi^2\_{\rm GF} = \sum\_{\mathfrak{s}} \|\overline{\mathfrak{h}}^{(\mathfrak{s})} - \overline{\mathfrak{sp}}(\mathfrak{r})\widehat{a}^{(\mathfrak{s})}\|^2,\tag{17}$$

where *h* ð Þ*<sup>s</sup>* is *<sup>h</sup>*ð Þ*<sup>s</sup>* weighted by *<sup>σ</sup>*ð Þ*<sup>s</sup> <sup>m</sup>* , *<sup>Φ</sup>*ð Þ^*<sup>τ</sup>* is *<sup>Φ</sup>*ð Þ^*<sup>τ</sup>* weighted by *<sup>σ</sup>*ð Þ*<sup>s</sup> <sup>m</sup>* , and *<sup>a</sup>*^ð Þ*<sup>s</sup>* <sup>¼</sup> *a*^0 ð Þ*<sup>s</sup>* , … , *<sup>a</sup>*^*<sup>P</sup>*�<sup>1</sup> ð Þ*<sup>s</sup>* h i*<sup>T</sup>* .

For a given ^*τ*, Eq. (17) is minimized when *<sup>a</sup>*^ð Þ*<sup>s</sup>* <sup>¼</sup> *<sup>Φ</sup>*ð Þ^*<sup>τ</sup>* �*<sup>h</sup>* ð Þ*s* , where *Φ*ð Þ^*τ* � is the symmetric generalized inverse of *Φ*ð Þ^*τ* . Then Eq. (17) can be rewritten as:

$$\chi^2\_{\rm GF} = \sum\_{s} \|P^\perp\_{\Phi(\vec{t})} \overline{h}^{(s)}\|^2,\tag{18}$$

where *P*<sup>⊥</sup> *<sup>Φ</sup>*ð Þ^*<sup>τ</sup>* <sup>¼</sup> *<sup>I</sup>* � *<sup>Φ</sup>*ð Þ^*<sup>τ</sup> <sup>Φ</sup>*ð Þ^*<sup>τ</sup>* � [52], which can be calculated by matrix decomposition of *Φ*ð Þ^*τ* using the QR method. Eq. (18) only has the nonlinear parameter, reducing the minimization parameter space considerably. Once ^*τ* is obtained by minimizing Eq. (18), the linear parameter *<sup>a</sup>*^ð Þ*<sup>s</sup>* can be obtained as a solution of *<sup>h</sup>*ð Þ*<sup>s</sup>* <sup>¼</sup> *<sup>Φ</sup>*ð Þ^*<sup>τ</sup> <sup>a</sup>*^ð Þ*<sup>s</sup>* . The implementation can be achieved with the VARP2 code [53].

#### **2.4 Laguerre expansion (LE)**

Due to the IRF introduced in FLIM data, iterative fitting methods include an enormous amount of convolutions, which is time-consuming [54]. Furthermore, iterative deconvolution methods require acquiring a considerable number of counts, which would increase the acquisition time. Numerous mathematical tools have been devised for deconvolution [55–59], and LE is faster and more robust than others. LE estimates the underlying fluorescence decay *f* with an ordered set of discrete-time Laguerre basis functions (LBFs) [60, 61],

$$\hat{f}\_m = \sum\_{l=0}^{L-1} \hat{c}\_l b\_l(m; a), \tag{19}$$

where *L* and *α* are the basic parameters and ^*cl* is the estimated *l* th expansion coefficient. The *l* th discrete-time LBF is defined as:

$$b\_l(m;a) = a^{\frac{(m-l)}{2}}(1-a)^{\frac{l}{2}}\sum\_{i=0}^l (-1)^i \binom{m}{i} \binom{l}{i} a^{l-i}(1-a)^i,\tag{20}$$

where 0 < *α* < 1 and *l* = 0, … , *L*-1.

Substituting Eq. (19) for *fm* in Eq. (1), the estimated signal becomes:

$$\hat{h}\_m = \sum\_{i=0}^m \sum\_{l=0}^{L-1} \hat{c}\_l \cdot i \circ f\_{m-i} \cdot b\_l(i; a) = \sum\_{l=0}^{L-1} \hat{c}\_l \cdot \nu\_l(m; a), \tag{21}$$

where *<sup>υ</sup>l*ð Þ¼ *<sup>m</sup>*; *<sup>α</sup>* <sup>P</sup>*<sup>m</sup> <sup>i</sup>*¼<sup>0</sup>*irf <sup>m</sup>*�*<sup>i</sup>* � *bl*ð Þ *<sup>i</sup>*; *<sup>α</sup>* . Then, Eq. (5) becomes.

$$\chi^2\_{\rm LE} = \sum\_{m} \frac{\left(h\_m - \sum\_{l=0}^{L-1} \hat{c}\_l \cdot \nu\_l(m; a)\right)^2}{\sigma\_m^2},\tag{22}$$

where *V* = [*v*0, … , *vL*-1] and *v<sup>l</sup>* = [*vl*(0, *α*), … , *vl*(*M*-1; *α*)]*<sup>T</sup>* . Eq. (22) can be minimized with the ordinary and constrained LSE, as demonstrated in [61]. Once ^*c* ¼ ½ � ^*c*0, … , ^*cL*�<sup>1</sup> *<sup>T</sup>* is determined, ^*f* can be recovered with Eq. (19). Then decay parameters can be extracted from ^*f* using the fitting methods mentioned above or the following non-fitting methods.

Setting proper *L* and *α* depends on the lifetime dynamic range and the measurement window *T* = *M*Δ*t* [32]*.* Automated Laguerre deconvolution methods have been reported to optimize *L* and *α* during the deconvolution routine [62, 63].

### **3. Non-fitting methods**

Fitting methods usually require experienced users to set proper initial conditions and require time-consuming iterative computations, making unsupervised non-fitting methods attractive for robust, real-time FLIM analysis. We review the rapid lifetime determination, center-of-mass, integral extraction, phasor, *τA*/*τI*, deep learning, and histogram clustering methods.

*Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

#### **3.1 Rapid lifetime determination (RLD)**

RLD needs two time-gated signals for the lifetime estimation of mono-exponential decays. The initial RLD method utilizes two consecutive time-gates of equal widths [64], as shown in **Figure 1(a)**. By integrating the signal within the time-gated windows, the lifetime can be determined using,

$$
\pi\_{RLD} = \frac{-\Delta t\_{TG}}{\ln \left( D\_1 / D\_0 \right)},
\tag{23}
$$

where *D*<sup>0</sup> and *D*<sup>1</sup> denote the integrated signal in Gates 1 and 2, and Δ*tTG* denotes the time-gate width. This calculation is simple and fast; however, it has limited estimation accuracy and precision in a narrow lifetime range determined by the gate width. To optimize RLD, several strategies with overlapping and unequal time gates, as shown in **Figure 1(b)**, have been proposed [65–67], suggesting optimized gating schemes. In optimized RLD schemes, the ratio of *D*<sup>1</sup> and *D*<sup>0</sup> can be expressed as:

$$DR = \frac{D\_1}{D\_0} = \frac{\exp\left(-Y\frac{\Delta t\_{\rm TC}}{\tau}\right) - \exp\left[-\left(Y + \mathcal{W}\right)\frac{\Delta t\_{\rm TC}}{\tau}\right]}{1 - \exp\left(-\frac{\Delta t\_{\rm TC}}{\tau}\right)},\tag{24}$$

where Δ*tTG* is the width of Gate 1 and *Y* and *W* are the rising edge and width coefficient of Gate 2. Lifetimes *τRLD* can be solved by applying Newton's method.

RLD can deal with multiple types of FLIM data, including TCSPC, streak camera, time-gated counting, and time-gated imaging data. However, RLD performs differently for uncorrelated and correlated datasets, as analyzed in ref. [66]. For multiexponential decays, RLD can provide average lifetimes which are neither intensifynor amplitude-weighted. Additionally, with three-time gates, the lifetime estimation dynamic range of RLD can be expanded for mono-exponential decays [21]. With four time-gates, we can extract lifetime components of bi-exponential decays [68, 69]. IRF is usually neglected in RLD, introducing estimation bias, especially for short lifetimes comparable with IRF's width.

**Figure 1.** *Rapid lifetime determination (RLD) schemes with (a) equal and (b) unequal time-gate widths.*

#### **3.2 Center-of-mass method (CMM)**

CMM provides intensity-weighted average lifetimes for multi-exponential decays [70–72],

$$\tau\_{\rm CMM} = \frac{\int\_0^\infty t \cdot h(t)dt}{\int\_0^\infty h(t)dt} - \frac{\int\_0^\infty t \cdot i\dot{r}f(t)dt}{\int\_0^\infty i\dot{r}f(t)dt} = \frac{\sum\_{p=0}^{P-1} q\_p \tau\_p^2}{\sum\_{p=0}^{P-1} q\_p \tau\_p} \approx \frac{\sum\_{m=0}^{M-1} t\_m \cdot h\_m}{\sum\_{m=0}^{M-1} h\_m} - \frac{\sum\_{m=0}^{M-1} t\_m \cdot i\dot{r}f\_m}{\sum\_{m=0}^{M-1} i\dot{r}f\_m}.\tag{25}$$

Li et al. proposed two versions of bi-exponential CMM (BCMM) that provide lifetime components information [73].

#### **3.3 Integral extraction method (IEM)**

For IEM, deconvolution is required to obtain ^*f* with which the average lifetime can be determined as [74, 75].

$$\tau\_{\text{IEM}} = -\frac{\int\_0^\infty g(t)dt}{\int\_0^\infty \dot{g}'(t)dt} \approx \sum\_{p=0}^{P-1} q\_p \tau\_p \approx -\frac{\sum\_{m=0}^{M-1} \mathcal{S}\_m \cdot \hat{f}\_m}{\sum\_{m=0}^{M-1} \frac{\hat{f}\_m - \hat{f}\_{m-1}}{\Delta t}} = -\frac{\Delta t \sum\_{m=0}^{M-1} \mathcal{S}\_m \cdot \hat{f}\_m}{\hat{f}\_{M-1} - \hat{f}\_0},\tag{26}$$

where *Sm* = [1/3, 4/3, 2/3, … , 4/3, 1/3] are the coefficients for numerical integration based on Simpson's rule,

$$\log(t) = A \sum\_{p=0}^{P-1} q\_p \tau\_p e^{-\frac{t}{\tau\_p}} \left[1 - e^{-\frac{4t}{\tau\_p}}\right]. \tag{27}$$

*τIEM* is an estimator of amplitude-weighted lifetimes.

#### **3.4 Phasor method**

The phasor method transforms each histogram into a phasor, like a vector. The sine-cosine transforms of decays are represented in a phasor plot as a two-dimensional histogram [76–80]. Phasor components *g* and *s* for time-domain FLIM can be expressed as:

$$\mathbf{g} = \frac{\int\_0^\infty f(t) \cdot \cos\left(\alpha t\right) dt}{\int\_0^\infty f(t) dt} = \frac{\sum\_{p=0}^{P-1} \frac{q\_p \tau\_p}{1 + \omega^2 \tau\_p}}{\sum\_{p=0}^{P-1} q\_p \tau\_p} = \frac{R e + s \bullet B}{A},\tag{28}$$

$$s = \frac{\int\_0^\infty f(t) \cdot \sin\left(at\right)dt}{\int\_0^\infty f(t)dt} = \frac{\sum\_{p=0}^{P-1} \frac{\alpha q\_p \tau\_p^2}{1 + \alpha^2 \tau\_p^2}}{\sum\_{p=0}^{P-1} q\_p \tau\_p} = \frac{A \bullet Im - B \bullet Re}{A^2 + B^2},\tag{29}$$

*Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

where

$$Re = \frac{\int\_0^\infty h(t) \cdot \cos\left(\alpha t\right) dt}{\int\_0^\infty h(t) dt} \approx \frac{\sum\_{m=0}^{M-1} h\_m \cdot \cos\left(\alpha t\_m\right)}{\sum\_{m=0}^{M-1} h\_m},\tag{30}$$

$$Im = \frac{\int\_0^\infty h(t) \cdot \sin\left(\alpha t\right) dt}{\int\_0^\infty h(t) dt} \approx \frac{\sum\_{m=0}^{M-1} h\_m \cdot \sin\left(\alpha t\_m\right)}{\sum\_{m=0}^{M-1} h\_m},\tag{31}$$

$$A = \frac{\int\_0^\infty i\eta f(t) \cdot \cos\left(\alpha t\right) dt}{\int\_0^\infty i\eta f(t) dt} \approx \frac{\sum\_{m=0}^{M-1} i\eta f\_m \cdot \cos\left(\alpha t\_m\right)}{\sum\_{m=0}^{M-1} i\eta f\_m},\tag{32}$$

$$B = \frac{\int\_0^\infty i\eta f(t) \cdot \sin\left(\alpha t\right) dt}{\int\_0^\infty i\eta f(t) dt} \approx \frac{\sum\_{m=0}^{M-1} i\eta f\_m \cdot \sin\left(\alpha t\_m\right)}{\sum\_{n=0}^{M-1} i\eta f\_m}.\tag{33}$$

Phasor plots' interpretation is usually user-dependent by manually selecting regions of interest in a phasor plot to find corresponding regions in fluorescence images. Based on the feature that pixels with similar fluorescence decays tend to congregate and form a cluster in a phasor plot, machine learning techniques have been developed with clustering methods [81] to automatically organize phasors into sensible groupings. Up to four-lifetime components can be resolved from a phasor plot using the rule of a linear combination of phasors [82], a graphical approach [83], and a computational nonlinear minimization algorithm [83].

A weighted average lifetime, *τ*Phasor, can also be derived using phasors,

$$\tau\_{\text{Phasor}} = \frac{\mathfrak{s}}{\mathfrak{g}o} = \frac{\sum\_{p=0}^{P-1} \frac{q\_p \mathfrak{r}\_p^2}{1 + a^2 \mathfrak{r}\_p^2}}{\sum\_{p=0}^{P-1} \frac{q\_p \mathfrak{r}\_p}{1 + a^2 \mathfrak{r}\_p^2}},\tag{34}$$

where *ω* = 2π/*T*,*T* = *M*Δ*t* is the measurement window. The weights of *τ*Phasor are *qpτ<sup>p</sup>* <sup>1</sup>þ*ω*2*τp*2. If *<sup>τ</sup><sup>p</sup>* < < *<sup>T</sup>*, then the weights are approximately equal to *qpτp*, i.e. *<sup>τ</sup>*Phasor is close to intensity-weighted lifetimes.

### **3.5** *τA***/***τ<sup>I</sup>* **method**

We proposed the *τA/τ<sup>I</sup>* method, a multi-exponential decay visualization method using two types of average lifetimes, *τ<sup>I</sup>* and *τA*. As mentioned previously (discussed in ref. [35]), a fluorescence decay can be approximated by a bi-exponential decay model, so that the ratio of *τ<sup>A</sup>* and *τ<sup>I</sup>* can be expressed as:

$$\frac{\tau\_A}{\tau\_I} = \frac{\left(\mathbf{1} + q\_1(\mathbf{R} - \mathbf{1})\right)^2}{\mathbf{1} + q\_1(\mathbf{R}^2 - \mathbf{1})},\tag{35}$$

### **Figure 2.**

*Distribution of* τA*/*τ<sup>I</sup> *with* q*<sup>1</sup> = 0 1 and* R *= 0 1 [35].*

where *R* = *τ*1/*τ*2. The distribution of *τA*/*τ<sup>I</sup>* (**Figure 2**) shows that when *R* ≈ 1 or *q*<sup>1</sup> ≈ 0 or 1, *τA*/*τ<sup>I</sup>* ≈ 1. With a decrease in *R* or an increase of *q*1, *τA*/*τ<sup>I</sup>* decreases. Therefore, the ranges of *q*<sup>1</sup> and *R* of a pixel can be determined by *τA*/*τI*.

With a TCSPC dataset of tSA201 cells, **Figures 3** and **4** show the results of the selected pixels within different *τA*/*τ<sup>I</sup>* ranges in (a) *τI*, (b) *τA*, (c) *τA*/*τ<sup>I</sup>* images, (d) histograms, (e) phasor plots, and (f) *τA*/*τ<sup>I</sup>* plots. For the pixels within *τA*/*τ<sup>I</sup>* = 0.2 0.5 (**Figure 3**), the histograms clearly show that *τ<sup>A</sup>* is much smaller than *τI*, which means the difference between *τ*<sup>1</sup> and *τ*<sup>2</sup> is significant. **Figure 3** (f) shows that the ranges of *q*<sup>1</sup> and *R* are approximately 0.5 1 and 0 0.2, respectively. For the pixels with *τA*/ *τ<sup>I</sup>* = 0.5 1 (**Figure 4**), *τ<sup>A</sup>* is closer to *τI*, meaning the pixels have decays close to mono-exponential.

*τA*/*τ<sup>I</sup>* is an intuitive tool for visualizing multi-exponential decays in a lifetime image. Separating the average lifetime images with *τA*/*τ<sup>I</sup>* is easier than phasor plots because *τA*/*τ<sup>I</sup>* is one-dimensional, and phasors are two-dimensional. Furthermore, *τA*/ *τ<sup>I</sup>* can intuitively show the *q*<sup>1</sup> and *R* ranges.

### **3.6 Deep learning (DL)**

Recently, DL-based FLIM analysis methods have been reported. DL features hierarchical representation learning by extracting high-level features through multiple nonlinear transformations of low-level features. DL shows a powerful ability to learn complex data and functions. The advent of DL breaks the conventional "modeldriven" paradigm and offers us a new "data-driven" approach to solving general optimization problems. Given sufficient labeled training data, DL algorithms can directly map the raw input data to their corresponding results, thus avoiding timeconsuming iterative optimization processes. Wu et al., first employed DL with a multi-layer perception (MLP) network using one input layer, one output layer, and two hidden layers [84]. The input layer has 57 entries depending on the number of time-bins in a histogram. The output layer has four neurons mapping four-lifetime parameters for bi-exponential decays. The investigation reveals that MLP can provide *Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

#### **Figure 3.**

*(a)* τI*-intensity image, (b)* τA*-intensity image, (c)* τA*/*τ<sup>I</sup> *ratio image, (d) histograms of* τ<sup>I</sup> *(yellow) and* τ<sup>A</sup> *(blue), (e) phasor plot, and (f) distribution of* τA*/*τ<sup>I</sup> *of the selected pixels in (c) with* τA*/*τ<sup>I</sup> *= 0.2 0.5 [35].*

#### **Figure 4.**

*(a)* τI*-intensity image, (b)* τA*-intensity image, (c)* τA*/*τ<sup>I</sup> *ratio image, (d) histograms of* τ<sup>I</sup> *(yellow) and* τ<sup>A</sup> *(blue), (e) phasor plot, and (f) distribution of* τA*/*τ<sup>I</sup> *of the selected pixels in (c) with* τA*/*τ<sup>I</sup> *= 0.5 1 [35].*

comparable or better performances and generate lifetime images at least 180-fold faster than conventional LSE. However, IRF was not considered in their work. Smith et al. proposed a 3D convolutional neural network (CNN) architecture named

fluorescence lifetime imaging network (FLI-Net) to quantify fluorescence decays at fast speeds [85]. FLI-Net is designed and trained for TCSPC FLIM, and gated intensity charged-coupled device (ICCD) based FLIM. FLI-Net's input is a 3D data cube (*x*, *y*, *t*), and output is bi-exponential decay parameters. Synthetic data is used for training, avoiding acquiring massive training datasets experimentally. FLI-Net is about 30-fold faster than SPCImage, the widely used FLIM processing software. FLI-Net outperforms LSE and MLP, especially with low photon counts range from 25 to 100. Xiao et al. proposed an easier and faster trained 1D CNN architecture named 1D-ConvResNet [86]. Compared with 2D or 3D CNNs, 1D-ConvResNet is more hardware-friendly and can be implemented on field-programmable gate array (FPGA) devices. Synthetic data is used for training and the training time with a CPU (Intel i7–4790) is about 0.5 hours, 8-fold faster than FLI-Net with a GPU (NVIDIA TITAN Xp GPU). Experimental FLIM datasets with an intensity threshold of 100 counts per pixel were used for validation, and analyzing a 256 256 image takes several seconds on a laptop. To generate highquality FLIM images under photon-starved conditions (50 counts per pixel), Chen et al. introduced a method called flimGANE [87]. It was derived from the Wasserstein GAN algorithm, where an "artificial" high-photon-count fluorescence decay histogram can be produced with a generator with a low-photon-count input. Using a well-trained generator and an estimator, a low-quality decay histogram can be mapped to a highquality counterpart and provides bi-exponential lifetime parameters within 0.32 ms/ pixel using a CPU, which is 258-fold faster than LSE and 2800-fold faster than MLE in generating a 512 512 FLIM image. However, it takes up to 500 h to fully train the network. Additionally, using time-resolved single-pixel datasets, a deep CNN named Net-FLICS (fluorescence lifetime imaging with compressed sensing) was reported [88].

DL methods for spatial resolution enhancement of FLIM images are also developed, including SRI-FLIMnet for reconstructing high-resolution images from lowresolution 3D FLIM data [89] and CNN-based denoising method removing noise in phasor plots after the K-means clustering segmentation [90].

Although DL is promising in real-time FLIM analysis even under starve photonstarved conditions and needs no user-dependent initial conditions, it suffers from long training times (hours to days) when retaining is required due to the change of IRF. Whenever the laser source for excitation and the detector are changed, the IRF can also vary, and the network should be retrained, reducing the universality. Recently, many attempts have been made to alleviate this disadvantage. For example, Zang proposed an online neural network training method using the extreme learning machine (ELM) [91]. ELM does not require a back-propagation process in the training phase. Therefore, it provides a much faster training speed, enabling online network training for any system configuration. The phasor coordinates of a fluorescence decay can also serve as the network's inputs, which can be viewed as a simple feature engineering to reduce the training time significantly [92]. Despite some trade-offs in current DL algorithms, it is no doubt that DL has excellent potential in a wide range of FLIM-related applications. DL algorithms can also be implemented on edge-computing platforms like FPGAs and smartphones to develop intelligent and portable FLIM devices [93].

### **3.7 Histogram clustering (HC)**

The HC method devised by Li et al. can improve FLIM analysis speed and accuracy by sorting histograms with similar profiles in a dataset into several clusters and significantly reducing the number of histograms to be analyzed [94]. HC implements


*Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

**Table 1.**

t*exe and MSE evaluated by lifetime determination algorithms without and with histogram clustering [94].*

clustering with two features of a histogram. It is a preprocessing method that can work with the LDAs mentioned above. The performances for producing decay parameter images without and with HC using synthetic and experimental datasets were investigated [94]. The execution time *t*exe and the mean squared error (MSE) of a FLIM dataset following a bi-exponential decay model with 150 150 pixels and 256 timebins are shown in **Table 1**. LE-LSE and LE-IEM stand for LSE and IEM with the Laguerre expansion method; IC and VP represent iterative convolution and variable expansion global fitting approaches.

For different output types, we suggested the fastest FLIM analysis methods: 1) LE-LSE with HC for all lifetime component images with *t*exe = 3.36 s, 116-fold shorter than *t*exe without HC; 2) VP with HC for constant lifetimes, *q*1, *τA*, and *τ<sup>I</sup>* images with *t*exe = 0.31 s, 10-fold shorter than *t*exe without HC; and 3) LE-IEM with HC as the second choice for *τ<sup>A</sup>* with *t*exe = 0.63 s, 98-fold shorter than *t*exe without HC, and CMM as the second choice for *τ<sup>I</sup>* with *t*exe = 0.2 s without or with HC (biased if the most significant lifetime > *T*/4). The analysis was conducted in MATLAB, and it can be translated to C or other environments to speed up the analysis.

HC can benefit applications demanding real-time FLIM, such as clinical diagnosis and fast screening. In the future, deep learning methods can be employed for unsupervised histogram clustering.

### **4. Conclusions**

Lifetime determination algorithms (LDAs) play a vital role in FLIM analysis. The results provided by diverse LDAs profoundly impact the interpretation of observed phenomena. Numerous approaches have been developed with notable features in terms of photon efficiency, estimation speed, accuracy, and precision. Fast FLIM

analysis under low-photon conditions is universally desirable, especially for real-time and single-molecular FLIM. This mini review focuses on the popular and cutting-edge LDAs used in time-domain FLIM. The progress in LDAs has shown possibilities for real-time FLIM under low photon conditions.

Fitting methods estimate lifetime parameters by solving an iterative minimization problem. The least squares estimation (LSE) introduces bias even with high photon counts (>1000), while the maximum likelihood estimation (MLE) is unbiased and photon efficient, as FLIM data is Poisson distributed. With robust algorithms, MLE instead of LSE can be used as a standard. The iterative convolution of IRF with exponential decays in LSE and MLE is time-consuming. The global fitting and Laguerre expansion methods have been proposed to accelerate analysis.

Non-fitting methods are attractive, as they can be much faster and more hardware-friendly than fitting methods because of simple calculations. However, non-fitting methods for a specific application should be careful. RLD using two timegates is suitable for mono-exponential decays but may introduce a bias for lifetimes comparable with IRF's width. CMM and IEM deliver intensity- and amplitudeweighted average lifetimes, respectively. The phasor method can graphically show exponential decays in a phasor plot. The *τA*/*τ<sup>I</sup>* method is a multi-exponential decay visualization method that uses two types of average lifetimes, which can intuitively show lifetime components' fraction and lifetime ratio. The deep learning method employs fully connected networks, 3D CNN, 1D CNN, and GAN algorithms in rapid FLIM analysis under photon-starved conditions, down to 50 counts per pixel. However, its long retraining time (hours to days) limits its universality when IRF is changed. The histogram clustering (HC) method suggests that histograms having similar profiles can be clustered for one calculation. By embedding HC, the algorithms mentioned above can be significantly accelerated with an execution time down to 30 μs/pixel.

In summary, the existing LDAs reveal different features in FLIM analysis, and it is hard to judge which one is replaceable. We believe that software with multiple embedded LDAs is desirable for end-users, providing comprehensive and complementary information for robust analysis. In addition, the newly developed "datadriven" DL-based LDAs and LDA accelerating method HC are promising, as they can not only enhance speed but also accuracy, especially for photon-starved applications. More efforts are expected to shrink retraining time for DL and increase clustering performance for HC via deep learning.

### **Acknowledgements**

Supported by the Scientific Instrument Developing Project of the Chinese Academy of Sciences (GJJSTD20220006), Science Foundation of the Chinese Academy of Sciences (CXJJ-21S006), China Scholarship Council, Biotechnology and Biological Sciences Research Council (BB/V019643/1) and Medical Research Scotland.

### **Conflict of interest**

The authors declare no conflict of interest.

*Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

### **Abbreviations**


## **Author details**

Yahui Li<sup>1</sup> \*, Lixin Liu<sup>2</sup> , Dong Xiao3 , Hang Li<sup>1</sup> , Natakorn Sapermsap<sup>4</sup> , Jinshou Tian<sup>1</sup> , Yu Chen<sup>4</sup> and David Day-Uei Li<sup>3</sup>

1 Key Laboratory of Ultra-fast Photoelectric Diagnostics Technology, Xi'an Institute of Optics and Precision Mechanics, Xi'an, Shaanxi, China

2 Xidian University, School of Optoelectronic Engineering, Xi'an Shaanxi, China

3 Department of Biomedical Engineering, University of Strathclyde, Glasgow, United Kingdom

4 Department of Physics, University of Strathclyde, Glasgow, United Kingdom

\*Address all correspondence to: liyahui@opt.ac.cn

© 2022 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

### **References**

[1] Lakowicz JR. Principles of Fluorescence Spectroscopy. 3rd ed. New York: Springer; 2006

[2] Priessner M, Summers PA, Lewis BW, Sastre M, Ying L, Kuimova MK, et al. Selective detection of Cu <sup>+</sup> ions in live cells via fluorescence lifetime imaging microscopy. Angewandte Chemie, International Edition. 2021;**60**:23148-23153. DOI: 10.1002/anie.202109349

[3] Gadella TWJ, editor. FRET and FLIM techniques. 1st ed. Amsterdam ; Boston: Elsevier; 2009

[4] Wallrabe H, Periasamy A. Imaging protein molecules using FRET and FLIM microscopy. Current Opinion in Biotechnology. 2005;**16**:19-27. DOI: 10.1016/j.copbio.2004.12.002

[5] Bücherl CA, Bader AN, Westphal AH, Laptenok SP, Borst JW. FRET-FLIM applications in plant systems. Protoplasma. 2014;**251**:383-394. DOI: 10.1007/s00709-013-0595-7

[6] Summers PA, Lewis BW, Gonzalez-Garcia J, Porreca RM, Lim AHM, Cadinu P, et al. Visualizing Gquadruplex DNA dynamics in live cells by fluorescence lifetime imaging microscopy. Nature Communications. 2021;**12**:162. DOI: 10.1038/s41467-020- 20414-7

[7] Levchenko SM, Pliss A, Peng X, Prasad PN, Qu J. Fluorescence lifetime imaging for studying DNA compaction and gene activities. Light Science Applications. 2021;**10**:224. DOI: 10.1038/ s41377-021-00664-w

[8] Pliss A, Zhao L, Ohulchanskyy TY, Qu J, Prasad PN. Fluorescence lifetime of fluorescent proteins as an intracellular

environment probe sensing the cell cycle progression. ACS Chemical Biology. 2012;**7**:1385-1392. DOI: 10.1021/ cb300065w

[9] Fornasiero EF, Mandad S, Wildhagen H, Alevra M, Rammner B, Keihani S, et al. Precisely measured protein lifetimes in the mouse brain reveal differences across tissues and subcellular fractions. Nature Communications. 2018;**9**:4230. DOI: 10.1038/s41467-018-06519-0

[10] Hirmiz N, Tsikouras A, Osterlund EJ, Richards M, Andrews DW, Fang Q. Highly multiplexed confocal fluorescence lifetime microscope designed for screening applications. IEEE Journal of Selected Topics in Quantum Electronics. 2021;**27**:1-9. DOI: 10.1109/JSTQE.2020.2997834

[11] Lukina M, Yashin K, Kiseleva EE, Alekseeva A, Dudenkova V, Zagaynova EV, et al. Label-free macroscopic fluorescence lifetime imaging of brain tumors. Frontiers in Oncology. 2021;**11**:666059. DOI: 10.3389/fonc.2021.666059

[12] Chen J, Han G, Liu Z, Wang H, Wang D, Zhao J, et al. Recovery mechanism of endoplasmic reticulum revealed by fluorescence lifetime imaging in live cells. Analytical Chemistry. 2022;**94**:5173-5180. DOI: 10.1021/acs.analchem.2c00216

[13] Becker W. Advanced Time-Correlated Single Photon Counting Techniques. Berlin ; New York: Springer; 2005

[14] Hirvonen LM, Suhling K. Wide-field TCSPC: Methods and applications. Measurement Science and Technology.

*Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

2017;**28**:012003. DOI: 10.1088/ 1361-6501/28/1/012003

[15] Liu X, Lin D, Becker W, Niu J, Yu B, Liu L, et al. Fast fluorescence lifetime imaging techniques: A review on challenge and development. Journal of Innovative Optical Health Sciences. 2019;**12**:1930003. DOI: 10.1142/ s1793545819300039

[16] Hirvonen LM, Suhling K. Fast timing techniques in FLIM applications. Frontiers of Physics. 2020;**8**:161. DOI: 10.3389/fphy.2020.00161

[17] Dowling K, Hyde SCW, Dainty JC, French PMW, Hares JD. 2-D fluorescence lifetime imaging using a time-gated image intensifier. Optics Communication. 1997; **135**:27-31. DOI: 10.1016/S0030-4018(96) 00618-9

[18] Elangovan M, Day RN, Periasamy A. Nanosecond fluorescence resonance energy transfer-fluorescence lifetime imaging microscopy to localize the protein interactions in a single living cell: NANOSECOND FRET-FLIM MICROSCOPY. Journal of Microscopy. 2002;**205**:3-14. DOI: 10.1046/ j.0022-2720.2001.00984.x

[19] Grant DM, McGinty J, Ewan J. McGhee, McGhee EJ, Bunney TD, Owen DM, et al. High speed optically sectioned fluorescence lifetime imaging permits study of live cell signaling events. Optics Express 2007;15: 15656–15673. DOI: 10.1364/oe.15.015656.

[20] Ehn A, Johansson O, Arvidsson A, Aldén M, Bood J. Single-laser shot fluorescence lifetime imaging on the nanosecond timescale using a dual image and modeling evaluation algorithm. Optics Express. 2012;**20**:3043-3056. DOI: 10.1364/oe.20.003043

[21] Li Y, Jia H, Chen S, Tian J, Liang L, Yuan F, et al. Single-shot time-gated fluorescence lifetime imaging using three-frame images. Optics Express. 2018;**26**:17936-17947. DOI: 10.1364/ OE.26.017936

[22] Fleming G, Morris J, Robinson H. Picosecond fluorescence spectroscopy with a streak camera. Australian Journal of Chemistry. 1977;**30**:2337. DOI: 10.1071/CH9772337

[23] Krishnan RV, Biener E, Zhang J, Heckel R, Herman B. Probing subtle fluorescence dynamics in cellular proteins by streak camera based fluorescence lifetime imaging microscopy. Applied Physics Letters. 2003;**83**:4658-4660. DOI: 10.1063/ 1.1630154

[24] Biskup C, Zimmer T, Benndorf K. FRET between cardiac Na+ channel subunits measured with a confocal microscope and a streak camera. Nature Biotechnology. 2004;**22**:220-224. DOI: 10.1038/nbt935

[25] Faust S, Dreier T, Schulz C. Temperature and bath gas composition dependence of effective fluorescence lifetimes of toluene excited at 266nm. Chemical Physics. 2011;**383**:6-11. DOI: 10.1016/j.chemphys.2011.03.013

[26] Camborde L, Jauneau A, Brière C, Deslandes L, Dumas B, Gaulin E. Detection of nucleic acid–protein interactions in plant leaves using fluorescence lifetime imaging microscopy. Nature Protocols. 2017;**12**: 1933-1950. DOI: 10.1038/nprot.2017.076

[27] Qu J, Liu L, Chen D, Lin Z, Xu G, Guo B, et al. Temporally and spectrally resolved sampling imaging with a specially designed streak camera. Optics Letters. 2006;**31**:368. DOI: 10.1364/ OL.31.000368

[28] Liu L, Qu J, Lin Z, Wang L, Fu Z, Guo B, et al. Simultaneous time- and spectrum-resolved multifocal multiphoton microscopy. Applied Physics B: Lasers and Optics. 2006;**84**: 379-383. DOI: 10.1007/s00340-006- 2314-y

[29] Birch DJS, Imhof RE. Time-domain fluorescence spectroscopy using timecorrelated single-photon counting. In: Lakowicz JR, editor. Topics in Fluorescence Spectroscopy. Boston: Kluwer Academic Publishers; 2002. pp. 1-95. DOI: 10.1007/0-306-47057-8\_1

[30] Pelet S, Previte MJ, Laiho LH, So PT. A fast global fitting algorithm for fluorescence lifetime imaging microscopy based on image segmentation. Biophysical Journal. 2004; **87**:2807-2817. DOI: 10.1529/ biophysj.104.045492

[31] Jo JA, Fang Q, Papaioannou T, Marcu L. Fast model-free deconvolution of fluorescence decay for analysis of biological systems. Journal of Biomedical Optics. 2004;**9**:743-752. DOI: 10.1117/ 1.1752919

[32] Zhang Y, Chen Y, Li DD-U. Optimizing Laguerre expansion based deconvolution methods for analyzing bi-exponential fluorescence lifetime images. Optics Express. 2016;**24**:13894. DOI: 10.1364/OE.24.013894

[33] Sillen A, Engelborghs Y. The correct use of "average" fluorescence parameters. Photochemistry and Photobiology. 1998;**67**:475-486. DOI: 10.1111/j.1751-1097.1998.tb09082.x

[34] Fišerová E, Kubala M. Mean fluorescence lifetime and its error. Journal of Luminescence. 2012;**132**: 2059-2064. DOI: 10.1016/j. jlumin.2012.03.038

[35] Li Y, Natakorn S, Chen Y, Safar M, Cunningham M, Tian J, et al. Investigations on average fluorescence lifetimes for visualizing multiexponential decays. Frontiers of Physics. 2020;**8**:576862. DOI: 10.3389/ fphy.2020.576862

[36] Hauschild T, Jentschel M. Comparison of maximum likelihood estimation and chi-square statistics applied to counting experiments. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment. 2001;**457**: 384-401. DOI: 10.1016/S0168-9002(00) 00756-7

[37] Maus M, Cotlet M, Hofkens J, Gensch T, De Schryver FC, Schaffer J, et al. An experimental comparison of the maximum likelihood estimation and nonlinear least-squares fluorescence lifetime analysis of single molecules. Analytical Chemistry. 2001;**73**: 2078-2086. DOI: 10.1021/ac000877g

[38] Turton DA, Reid GD, Beddard GS. Accurate analysis of fluorescence decays from single molecules in photon counting experiments. Analytical Chemistry. 2003;**75**:4182-4187. DOI: 10.1021/ac034325k

[39] Santra K, Zhan J, Song X, Smith EA, Vaswani N, Petrich JW. What is the best method to fit time-resolved data? A comparison of the residual minimization and the maximum likelihood techniques As applied to experimental timecorrelated, single-photon counting data. The Journal of Physical Chemistry. B. 2016;**120**:2484-2490. DOI: 10.1021/acs. jpcb.6b00154

[40] Hall P, Selinger B. Better estimates of exponential decay parameters. The Journal of Physical Chemistry. 1981;**85**: 2941-2946. DOI: 10.1021/j150620a019

*Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

[41] Hall P, Selinger B. Better estimates of multiexponential decay parameters. Z Für Physics Chemistry. 1984;**141**:77-89. DOI: 10.1524/zpch.1984.141.1.077

[42] Köllner M, Wolfrum J. How many photons are necessary for fluorescencelifetime measurements? Chemical Physics Letters. 1992;**200**:199-204. DOI: 10.1016/0009-2614(92)87068-Z

[43] Tellinghuisen J, Wilkerson CW. Bias and precision in the estimation of exponential decay parameters from sparse data. Analytical Chemistry. 1993; **65**:1240-1246. DOI: 10.1021/ ac00057a022

[44] Kim J, Seok J. Statistical properties of amplitude and decay parameter estimators for fluorescence lifetime imaging. Optics Express. 2013;**21**:6061. DOI: 10.1364/OE.21.006061

[45] Laurence TA, Chromy BA. Efficient maximum likelihood estimator fitting of histograms. Nature Methods. 2010;**7**: 338-339. DOI: 10.1038/nmeth0510-338

[46] Chessel A, Waharte F, Salamero J, Kervrann C. A maximum likelihood method for lifetime estimation in photon counting-based fluorescence lifetime imaging microscopy. In: 21st European Signal Processing Conference (EUSIPCO 2013); 09–13 September 2013. Marrakech, Morocco: IEEE; 2014. pp. 1-5

[47] Marquardt DW. An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics. 1963;**11**:431-441. DOI: 10.1137/0111030

[48] Johnson ML, Faunt LM. Parameter estimation by least-squares methods. Methods in Enzymology. 1992; **210**:1-37. DOI: 10.1016/0076-6879(92) 10003-v

[49] Johnson ML. Why, when, and how biochemists should use least squares. Analytical Biochemistry. 1992;**206**: 215-225. DOI: 10.1016/0003-2697(92) 90356-c

[50] Bajzer Ž, Therneau TM, Sharp JC, Prendergast FG. Maximum likelihood method for the analysis of time-resolved fluorescence decay curves. European Biophysics Journal. 1991;**20**:247-262. DOI: 10.1007/BF00450560

[51] Bajzer Ž, Prendergast FG. [10] maximum likelihood analysis of fluorescence data. Methods in Enzymology. Elsevier. 1992;**210**:200-237. DOI: 10.1016/0076-6879(92)10012-3

[52] Warren SC, Margineanu A, Alibhai D, Kelly DJ, Talbot C, Alexandrov Y, et al. Rapid global fitting of large fluorescence lifetime imaging microscopy datasets. Degtyar VE, editor. PLoS One. 2013;**8**:e70687. DOI: 10.1371/ journal.pone.0070687

[53] Golub GH, LeVeque RJ. Extensions and uses of the variable projection Algorith for solving nonlinear least squares problems. In: Proceedings of the 1979 Numerical Analsysis and Computers Conference. Washington, DC: University of Washington; 1979. pp. 79-83. DOI: 10.5281/zenodo.852682. Available from: http://faculty. washington.edu/rjl/pubs/ GolubLeVeque1979/index.htm

[54] Ware WR, Doemeny LJ, Nemzek TL. Deconvolution of fluorescence and phosphorescence decay curves. A Least-Squares Method. 1973;**77**:2038-2048. DOI: 10.1021/J100636A003

[55] Gafni A, Modlin RL, Brand L. Analysis of fluorescence decay curves by means of the Laplace transformation. Biophysical Journal. 1975;**15**:263-280. DOI: 10.1016/S0006-3495(75)85817-6

[56] O'Connor DV, Ware WR, Andre JC. Deconvolution of fluorescence decay curves. A critical comparison of techniques. The Journal of Physical Chemistry. 1979;**83**:1333-1343. DOI: 10.1021/j100473a019

[57] Apanasovich VV, Novikov EG. Deconvolution method for fluorescence decays. Optics Communication. 1990;**78**: 279-282. DOI: 10.1016/0030-4018(90) 90361-V

[58] Zhang Z, Grattan KTV, Hu Y, Palmer AW, Meggitt BT. Prony's method for exponential lifetime estimations in fluorescence-based thermometers. The Review of Scientific Instruments. 1996; **67**:2590-2594. DOI: 10.1063/1.1147219

[59] Fu CY, Ng BK, Razul SG. Fluorescence lifetime discrimination using expectation-maximization algorithm with joint deconvolution. Journal of Biomedical Optics. 2009;**14**: 064009. DOI: 10.1117/1.3258835

[60] Jo JA, Fang Q, Marcu L. Ultrafast method for the analysis of fluorescence lifetime imaging microscopy data based on the Laguerre expansion technique. IEEE Journal of Selected Topics in Quantum Electronics. 2005;**11**: 835-845. DOI: 10.1109/ JSTQE.2005.857685

[61] Liu J, Sun Y, Qi J, Marcu L. A novel method for fast and robust estimation of fluorescence decay dynamics using constrained least-squares deconvolution with Laguerre expansion. Physics in Medicine and Biology. 2012;**57**:843-865. DOI: 10.1088/0031-9155/57/4/843

[62] Dabir AS, Trivedi CA, Ryu Y, Pande P, Jo JA. Fully automated deconvolution method for on-line analysis of time-resolved fluorescence spectroscopy data based on an iterative Laguerre expansion technique. Journal of Biomedical Optics. 2009;**14**: 024030-024030. DOI: 10.1117/1.3103342

[63] Pande P, Jo JA. Automated analysis of fluorescence lifetime imaging microscopy (FLIM) data based on the Laguerre deconvolution method. IEEE Transactions on Biomedical Engineering. 2011;**58**:172-181. DOI: 10.1109/ tbme.2010.2084086

[64] Ballew RM, Demas JN. An error analysis of the rapid lifetime determination method for the evaluation of single exponential decays. Analytical Chemistry. 1989;**61**:30-33. DOI: 10.1021/ ac00176a007

[65] Chan SP, Fuller ZJ, Demas JN, DeGraff BA. Optimized gating scheme for rapid lifetime determinations of single-exponential luminescence lifetimes. Analytical Chemistry. 2001;**73**: 4486-4490. DOI: 10.1021/ac0102361

[66] Li DD-U, Ameer-Beg S, Arlt J, Tyndall D, Walker R, Matthews DR, et al. Time-domain fluorescence lifetime imaging techniques suitable for solidstate imaging sensor arrays. Sensors. 2012;**12**:5650-5669. DOI: 10.3390/ s120505650

[67] Collier BB, McShane MJ. Dynamic windowing algorithm for the fast and accurate determination of luminescence lifetimes. Analytical Chemistry. 2012;**84**: 4725-4731. DOI: 10.1021/ac300023q

[68] Sharman KK, Periasamy A, Ashworth H, Demas JN. Error analysis of the rapid lifetime determination method for double-exponential decays and new windowing schemes. Analytical Chemistry. 1999;**71**:947-952. DOI: 10.1021/ac981050d

[69] Silva SF, Domingues JP, Morgado AM. Can we use rapid lifetime *Lifetime Determination Algorithms for Time-Domain Fluorescence Lifetime Imaging: A Review DOI: http://dx.doi.org/10.5772/intechopen.106423*

determination for fast, fluorescence lifetime based, metabolic imaging? Precision and accuracy of doubleexponential decay measurements with low total counts. PLoS One. 2019;**14**: 1-20. DOI: 10.1371/journal.pone.0216894

[70] Li DD-U, Arlt J, Tyndall D, Walker R, Richardson J, Stoppa D, et al. Video-rate fluorescence lifetime imaging camera with CMOS single-photon avalanche diode arrays and high-speed imaging algorithm. Journal of Biomedical Optics. 2011;**16**:096012. DOI: 10.1117/ 1.3625288

[71] Li DD-U, Rae BR, Andrews R, Arlt J, Henderson R. Hardware implementation algorithm and error analysis of highspeed fluorescence lifetime sensing systems using center-of-mass method. Journal of Biomedical Optics. 2010;**15**: 017006. DOI: 10.1117/1.3309737

[72] Poland SP, Erdogan AT, Krstajić N, Levitt J, Devauges V, Walker RJ, et al. New high-speed Centre of mass method incorporating background subtraction for accurate determination of fluorescence lifetime. Optics Express. 2016;**24**:6899. DOI: 10.1364/ OE.24.006899

[73] Li DD-U, Yu H, Chen Y. Fast bi-exponential fluorescence lifetime imaging analysis methods. Optics Letters. 2015;**40**:336. DOI: 10.1364/ OL.40.000336

[74] Li DD-U, Bonnist E, Renshaw D, Henderson R. On-chip, time-correlated, fluorescence lifetime extraction algorithms and error analysis. Journal of the Optical Society of America. A. 2008; **25**:1190. DOI: 10.1364/JOSAA.25.001190

[75] Li DD-U, Walker R, Richardson J, Rae B, Buts A, Renshaw D, et al. Hardware implementation and

calibration of background noise for an integration-based fluorescence lifetime sensing algorithm. Journal of the Optical Society of America. A. 2009;**26**:804. DOI: 10.1364/JOSAA.26.000804

[76] Digman MA, Caiolfa VR, Zamai M, Gratton E. The phasor approach to fluorescence lifetime imaging analysis. Biophysical Journal. 2008;**94**:L14-L16. DOI: 10.1529/biophysj.107.120154

[77] Fereidouni F, Esposito A, Blab GA, Gerritsen HC. A modified phasor approach for analyzing time-gated fluorescence lifetime images. Journal of Microscopy. 2011;**244**:248-258. DOI: 10.1111/j.1365-2818.2011.03533.x

[78] Ranjit S, Malacrida L, Jameson DM, Gratton E. Fit-free analysis of fluorescence lifetime imaging data using the phasor approach. Nature Protocols. 2018;**13**:1979-2004. DOI: 10.1038/ s41596-018-0026-5

[79] Sorrells JE, Iyer RR, Yang L, Bower AJ, Spillman DR, Chaney EJ, et al. Real-time pixelwise phasor analysis for video-rate two-photon fluorescence lifetime imaging microscopy. Biomedical Optics Express. 2021;**12**:4003. DOI: 10.1364/BOE.424533

[80] Michalet X. Continuous and discrete phasor analysis of binned or time-gated periodic decays. AIP Advances. 2021;**11**: 035331-035331. DOI: 10.1063/5.0027834

[81] Vallmitjana A, Torrado B, Gratton E. Phasor-based image segmentation: Machine learning clustering techniques. Biomedical Optics Express. 2021;**12**: 3410-3422. DOI: 10.1364/boe.422766

[82] Vallmitjana A, Dvornikov A, Torrado B, Jameson DM, Ranjit S, Gratton E. Resolution of 4 components in the same pixel in FLIM images using the phasor approach. Methods

Applications Fluorescope IOP Publishing. 2020;**8**:035001. DOI: 10.1088/2050-6120/ab8570

[83] Vallmitjana A, Torrado B, Dvornikov A, Ranjit S, Gratton E. Blind resolution of lifetime components in individual pixels of fluorescence lifetime images using the phasor approach. The Journal of Physical Chemistry. B. 2020; **124**:10126-10137. DOI: 10.1021/acs. jpcb.0c06946

[84] Wu G, Nowotny T, Zhang Y, Yu H-Q, Li DD-U. Artificial neural network approaches for fluorescence lifetime imaging techniques. Optics Letters. 2016;**41**:2561. DOI: 10.1364/ OL.41.002561

[85] Smith JT, Yao R, Sinsuebphon N, Rudkouskaya A, Un N, Mazurkiewicz J, et al. Fast fit-free analysis of fluorescence lifetime imaging via deep learning. Proceedings of the National Academy of Sciences. 2019;**116**: 24019-24030. DOI: 10.1073/ pnas.1912707116

[86] Xiao D, Chen Y, Li DD-U. Onedimensional deep learning architecture for fast fluorescence lifetime imaging. IEEE Journal of Selected Topics in Quantum Electronics. 2021;**27**:7000210. DOI: 10.1109/JSTQE.2021.3049349

[87] Chen Y-I, Chang Y-J, Liao S-C, Nguyen TD, Yang J, Kuo Y-A, et al. Generative adversarial network enables rapid and robust fluorescence lifetime image analysis in live cells. Communications Biology. Springer US. 2022;**5**:18. DOI: 10.1038/s42003-021- 02938-w

[88] Yao R, Ochoa M, Yan P, Intes X. Net-FLICS: Fast quantitative wide-field fluorescence lifetime imaging with compressed sensing – A deep learning approach. Light Science Application.

Springer US. 2019;**8**:26. DOI: 10.1038/ s41377-019-0138-x

[89] Xiao D, Zang Z, Xie W, Sapermsap N, Chen Y, Li DD-U. Spatial resolution improved fluorescence lifetime imaging via deep learning. Optics Express. 2022;**30**:11479. DOI: 10.1364/OE.451215

[90] Mannam V, Zhang Y, Yuan X, Hato T, Dagher PC, Nichols EL, et al. Convolutional neural network denoising in fluorescence lifetime imaging microscopy (FLIM). In: Periasamy A, So PT, König K, editors. Multiphoton Microscopy in the Biomedical Sciences XXI. Vol. 11648. Bellingham WA USA: SPIE; 2021:116481C. DOI: 10.1117/12.2578574

[91] Zang Z, Xiao D, Wang Q, Li Z, Xie W, Chen Y, et al. Fast analysis of time-domain fluorescence lifetime imaging via extreme learning machine. Sensors. 2022;**22**:3758. DOI: 10.3390/ s2210375

[92] Héliot L, Leray A. Simple phasorbased deep neural network for fluorescence lifetime imaging microscopy. Science Reports Nature Publishing Group UK. 2021;**11**:23858. DOI: 10.1038/s41598-021-03060-x

[93] Xiao D, Zang Z, Sapermsap N, Wang Q, Xie W, Chen Y, et al. Dynamic fluorescence lifetime sensing with CMOS single-photon avalanche diode arrays and deep learning processors. Biomedical Optics Express. 2021;**12**: 3450. DOI: 10.1364/BOE.425663

[94] Li Y, Sapermsap N, Yu J, Tian J, Chen Y, Li DD-U. Histogram clustering for rapid time-domain fluorescence lifetime image analysis. Biomed. Optics Express. 2021;**12**:4293-4307. DOI: 10.1038/s41598-021-03060-x

## *Edited by Raffaello Papadakis*

Fluorescence imaging is widely used in scientific fields ranging from biology to biomedicine and even materials science. The development of novel fluorescent labels and microscopy techniques has rendered fluorescence imaging profoundly useful. Particularly in bioscience, fluorescence imaging empowers the study of the intracellular distribution, dynamics, gene expression, protein–protein interactions, and protein localization and enables the identification and tracking of lysosomes. Fluorescence imaging is applicable in cells and tissues and is constantly gaining attention in medicine too, in the fields of fluorescence-guided surgery and robotic-assisted fluorescence surgery. Acknowledging all these important new trends, this book provides an overview of the recent advances and applications in fluorescence imaging.

Published in London, UK © 2023 IntechOpen © colnihko / iStock

Fluorescence Imaging - Recent Advances and Applications

Fluorescence Imaging

Recent Advances and Applications

*Edited by Raffaello Papadakis*