**7. Performance estimates**

186 Applications of Digital Signal Processing

**(a)** 

**(b)** 

Fig. 12. Frequency and amplitude errors,Vf and Va, as a function of the noise standard deviation V for OMP (blue) and OMP-NLS (red) for a signal composed of a two sinusoids with *N* = 128, *M* = 20 and *N*f = 4 averaged over 100 trials with randomly chosen input

frequencies. (a) Frequency error. (b) Amplitude error.

As discussed above, this study is based on experimental or empirical evaluation (i.e. numerical simulations) of a proposed technique for recovering compressively sensed signals. The weakness of such a study is that calculations alone do not provide performance guarantees while the strength of such a study is that calculations can evaluate practical cases that would be encountered in real applications. Regardless, it is necessary to know when and how an algorithm fails for it to be of much use, and we can use prior work on performance guarantees for CS, OMP and NLS to help us.

Consider first the noise-free case in which the number of sinusoids is known. Here the difference between success and failure is computationally obvious. If the recovery is successful, the residual after extraction of the known number of sinusoids collapses to near the machine precision. If it fails, the residual remains at about the level of the initial measurement vector *y*. In the presence of noise the situation is similar except the collapse is to the system noise level. If the number of sinusoids is unknown, then recovery proceeds until the system noise level is reached, but statistical testing must be used to determine if the residual at this threshold is due to noise or incorrectly recovered signals.

Practical use of the OMP/NLS algorithm requires at a minimum empirical knowledge of where the algorithm fails and ultimately, performance guarantees and complexity estimates (operation counts). Since this algorithm couples two well known algorithms, in principle we can rely on previous work. The problem can be divided into 3 parts. First, one has to assess the compressive sensing part of the problem. Does the mixing matrix **Ʒ** satisfy the appropriate conditions? Is the value of *M* large enough to recover the *K* unknowns? Are the measurements really sparse in the chosen model or even is the model applicable to the signal of interest? Our empirical observations suggest that it is difficult for a random number generator to pick a bad mixing matrix. Observations also suggest that the requirement on *M* for recovery is on the same order as that derived for grid-based CS, *M* **~**  *K* log**(***N***/***K***).** Second, the sampling in the overcomplete dictionary must be fine enough that the first frequency found by the argmax of *G***(***f***,***r***)** in (7) is near a true solution. If this is not the case due to insufficient granularity, multiple frequencies too close together, or high noise levels, the OMP cannot start. This issue is not restricted to our work but common to all matching pursuit algorithms. While we do not have performance guarantees here, we have noted empirically that lack of convergence is very easy to determine for a known number or sinusoids and known noise floor. Finally, the NLS must be able to converge. Here we can rely on the results given by (Stoica et al., 2000; Li et al., 2000; Chan and So, 2004; Christensen and Jensen, 2006) that the NLS achieves the Cramer Rao Bound. Empirically, we observe that the dictionary must be sufficiently overcomplete that the NLS is looking for a frequency solution in one local minimum.

Applications of the Orthogonal Matching Pursuit/ Nonlinear

Approx., Vol.13, pp. 57-98.

*Trans. Signal Processing*.

1306, Sept. 2006.

pp. 301-321.

CA. Vol.1, pp. 40-44.

arXiv:1103.0414v1, pp. 1-29.

Prentice Hall, pp. 146-151.

334.

352.

Least Squares Algorithm to Compressive Sensing Recovery 189

Davenport, M.; Wakin, M., and Baraniuk, R., (2006). Detection and estimation with compressive measurements, Rice ECE Department Technical Report, TREE 0610. Davis, G.; Mallat, S., & Avellaneda, M., (1997). Greedy adaptive approximation, Const.

Donoho, D. L.; (2006). Compressed sensing*, IEEE Trans. Inform. Theo*ry, Vol.52, pp. 1289-

Duarte, M. F.: & Baraniuk, R. G., (2010). Spectral Compressive Sensing, submitted to *IEEE* 

Hormati, A.; & M. Vetterli, (2007). Annihilating filter-based decoding in the compressed

Huang, S.; & J. Zhu, (2011). Recovery of sparse signals using OMP and its variants: convergence analysis based on RIP, Inverse Problems, Vol.27, 035003 (14pp). Jacques, L.; & C. De Vleeschouwer, (2008). A Geometrical study of matching pursuit

Li, H.; Stoica, P., & Li, J., (2000). Computationally efficient parameter estimation for harmonic sinusoidal signals, *Signal Processing*, Vol.80, pp. 1937-1944. Li, H.; Sun, W., Stoica, P. & Li, J., (2001). 2-D sinusoidal amplitude estimation with

Loris, I.; (2008). L1Packv2: A *Mathematica* package for minimizing an ell-1-penalized

Mallat, S. G.; & Zhang, Z., (1993). Matching pursuits with time-frequency dictionaries, *IEEE* 

Needell, D.; & Tropp, J. A., (2009). CoSaMP: Iterative signal recovery from incomplete and

Needell, D.; & Vershynin, R., (2009). Uniform uncertainty principle and signal recovery via

Pati, Y.C.; Rezaifar, R., & Krishnaprasad, P.S., (1993). Orthogonal matching pursuit:

Salzo, S.; & Villa, S., (2011). Convergence analysis of a proximal Gauss-Newton method,

Shaw, T. J.; & Valley, G. C., (2010). Angle of arrival detection in sparse systems using

Stoica, P.; & Moses, R. L., (1997). *Introduction to Spectral Analysis*, Upper Saddle River, NJ:

Stoica, P.; Li, H., and Li, J. (2000). Amplitude estimation of sinusoidal signals: survey, new

Denmark, 23-27Aug. 2010, *EUSIPCO 2010 Digest*, pp. 1424-1428.

application to 2-D system identification, IEEE International Conf. On Acoustics,

inaccurate samples, Applied and Computational Harmonic Analysis, Vol.26, No.3,

regularized orthogonal matching pursuit, Found. Comput. Math., Vol.9, pp. 317-

Recursive function approximation with applications to wavelet decomposition, Proc. 27th Annu. Asilomar Conf. Signals. Systems, and Computers, Pacific Grove,

compressed sensing, European Signal Processing Conference (EUSIPCO), Aalburg,

results, and an application, *IEEE Trans. Signal Processing*, Vol.48, No.2, pp. 338-

parameterization, IEEE Trans. Signal Proc., Vol.56, pp. 2835-2848.

sensing framework, *Proc. SPIE*, Vol.6701, pp. 1-10.

Speech, and Signal Proc., ICASSP, Vol.3, pp. 1921-1924.

functional, *Computer Phys. Comm.*, Vol.79, pp. 895-902.

*Trans. Signal Processing*, Vol.41, No.12, pp. 3397-3415.
