Pencils of Semi-Infinite Matrices and Orthogonal Polynomials

*Sergey Zagorodnyuk*

## **Abstract**

Semi-infinite matrices, generalized eigenvalue problems, and orthogonal polynomials are closely related subjects. They connect different domains in mathematics—matrix theory, operator theory, analysis, differential equations, etc. The classical examples are Jacobi and Hessenberg matrices, which lead to orthogonal polynomials on the real line (OPRL) and orthogonal polynomials on the unit circle (OPUC). Recently there turned out that pencils (i.e., operator polynomials) of semiinfinite matrices are related to various orthogonal systems of functions. Our aim here is to survey this increasing subject. We are mostly interested in pencils of symmetric semi-infinite matrices. The corresponding polynomials are defined as generalized eigenvectors of the pencil. These polynomials possess special orthogonality relations. They have physical and mathematical applications that will be discussed. Examples show that there is an unclarified relation to Sobolev orthogonal polynomials. This intriguing connection is a challenge for further investigations.

**Keywords:** semi-infinite matrix, pencil, orthogonal polynomials, Sobolev orthogonality, difference equation

## **1. Introduction**

In this section, we will introduce the main objects of this chapter along with some brief historical notes.

By operator pencils or operator polynomials one means polynomials of a complex variable *λ* whose coefficients are linear bounded operators acting in a Banach space *X*:

$$L(\lambda) = \sum\_{j=0}^{m} \lambda^j A\_j,\tag{1}$$

where *A <sup>j</sup>* : *X* ! *X* (*j* ¼ 0, … , *m*), see, for example, [1, 2]. Parlett in ref. [3, p. 339] stated that the term *pencil* was introduced by Gantmacher in ref. [4] for matrix expressions, and Parlett explained how this term came from optics and geometry. In this chapter, we shall be mainly interested in pencils of banded semi-infinite matrices that are related to different kinds of scalar orthogonal polynomials. The classical example of such a relation is the case of orthogonal polynomials on the real line

(OPRL) and Jacobi matrices, see, for example, refs. [5, 6]. If *pn*ð Þ *<sup>x</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> is a set of orthonormal OPRL and *J* is the corresponding Jacobi matrix, then the following relation holds:

$$(J - \varkappa E)\overrightarrow{p}(\varkappa) = 0,\tag{2}$$

where *p* !ð Þ¼ *<sup>x</sup> <sup>p</sup>*0ð Þ *<sup>x</sup>* , *<sup>p</sup>*1ð Þ *<sup>x</sup>* , … � �*<sup>T</sup>* , is a vector of polynomials (here the superscript *T* means the transposition), and *E* is the identity matrix (having units on the main diagonal and zeros elsewhere). In other words, *p* ! is an eigenfunction of the pencil *J* � *xE*. It is surprising that mathematicians rarely talked about the relation (2) in such a manner. The next classical example is the case of orthogonal polynomials on the unit circle (OPUC) and the corresponding three-term recurrence relation, see ref. [7, p. 159]. More recently there appeared CMV matrices, which are also related to OPUC, see, for example, ref. [8]. We should notice that besides orthogonal polynomials, there are other systems of functions that are closely related to semi-infinite matrices. Here we can mention biorthogonal polynomials and rational functions, see, for example, [9, 10] and references therein.

A natural generalization of OPRL is matrix orthogonal polynomials on the real line (MOPRL). MOPRL was introduced by Krein in 1949 [11]. They satisfy the relation of type (2), with *J* replaced by a block Jacobi matrix, and with *p* ! replaced by a vector of matrix polynomials. It turned out that MOPRL is closely related to orthogonal polynomials on the radial rays in the complex plane, see refs. [12, 13]. We shall discuss this case in Section 2.

Another possible generalization of relation (2) is the following one:

$$(\!(f\_5 - \!\!xJ\_3)\overrightarrow{p}(\mathbf{x}) = \mathbf{0},\tag{3}$$

where *J*<sup>3</sup> is a Jacobi matrix, and *J*<sup>5</sup> is a real symmetric semi-infinite five-diagonal matrix with positive numbers on the second subdiagonal, see ref. [14]. These polynomials contain OPRL as a proper subclass. In general, they possess some special orthogonality relations. These polynomials will be discussed in Section 3.

Another natural generalization of OPRL is Sobolev orthogonal polynomials, see a recent survey in ref. [15]. During last years there appeared several examples of Sobolev polynomials, which are eigenfunctions of pencils of differential or difference operators. This subject will be discussed in Section 4.

**Notations.** As usual, we denote by , ,, þ, the sets of real numbers, complex numbers, positive integers, integers, and nonnegative integers, respectively. By *<sup>m</sup>*,*<sup>n</sup>* we mean a set of all complex matrices of size ð Þ *m* � *n* . By we denote the set of all polynomials with complex coefficients. The superscript *T* means the transposition of a matrix.

By *<sup>l</sup>*<sup>2</sup> we denote the usual Hilbert space of all complex sequences *<sup>c</sup>* <sup>¼</sup> ð Þ *cn* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> ¼ ð Þ *<sup>c</sup>*0,*c*1,*c*2, … *<sup>T</sup>* with the finite norm <sup>∥</sup>*c*∥*<sup>l</sup>*<sup>2</sup> <sup>¼</sup> ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup>j j *cn* <sup>2</sup> q . The scalar product of two sequences *<sup>c</sup>* <sup>¼</sup> ð Þ *cn* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup>, *<sup>d</sup>* <sup>¼</sup> ð Þ *dn* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> <sup>∈</sup>*l*<sup>2</sup> is given by ð Þ *<sup>c</sup>*, *<sup>d</sup> <sup>l</sup>*<sup>2</sup> <sup>¼</sup> <sup>P</sup><sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup>*cndn*. We denote *e* ! *<sup>m</sup>* <sup>¼</sup> ð Þ *<sup>δ</sup><sup>n</sup>*,*<sup>m</sup>* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> <sup>∈</sup>*l*2, *<sup>m</sup>* <sup>∈</sup> þ. By *<sup>l</sup>*2,*fin* we denote the set of all finite vectors from *<sup>l</sup>*2, that is, vectors with all, but finite number, components being zeros.

By Bð Þ we denote the set of all Borel subsets of . If *σ* is a (non-negative) bounded measure on <sup>B</sup>ð Þ then by *<sup>L</sup>*<sup>2</sup> *<sup>σ</sup>* we denote a Hilbert space of all (classes of equivalences of) complex-valued functions *f* on with a finite norm ∥*f* ∥*L*<sup>2</sup> *<sup>σ</sup>* ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Ð j j *f x*ð Þ <sup>2</sup> *dσ* q . The scalar product of two functions *f*, *g* ∈*L*<sup>2</sup> *<sup>σ</sup>* is given by ð Þ *f*, *g <sup>L</sup>*<sup>2</sup> *σ* ¼ Ð *f x*ð Þ*g x*ð Þ*dσ*. By ½ � *<sup>f</sup>* we denote the class of equivalence in *<sup>L</sup>*<sup>2</sup> *<sup>σ</sup>*, which contains the representative *f*. By P we denote a set of all (classes of equivalence which contain) polynomials in *L*<sup>2</sup> *<sup>σ</sup>*. As usual, we sometimes use the representatives instead of their classes in formulas. Let *B* be an arbitrary linear operator in *L*<sup>2</sup> *<sup>σ</sup>* with the domain <sup>P</sup>. Let *<sup>f</sup>*ð Þ*<sup>λ</sup>* <sup>∈</sup> be nonzero and of degree *<sup>d</sup>*∈þ, *<sup>f</sup>*ð Þ¼ *<sup>λ</sup>* <sup>P</sup>*<sup>d</sup> <sup>k</sup>*¼<sup>0</sup>*dkλk*, *dk* <sup>∈</sup> . We set

$$f(B) = \sum\_{k=0}^{d} d\_k B^k; \quad B^0 \coloneqq E \Big|\_{\mathcal{P}}.$$

If *f* � 0, then *f B*ð Þ≔0j P.

If H is a Hilbert space then ð Þ �, � *<sup>H</sup>* and ∥ � ∥*<sup>H</sup>* mean the scalar product and the norm in *H*, respectively. Indices may be omitted in obvious cases. For a linear operator *A* in *H*, we denote by *D A*ð Þ its domain, by *R A*ð Þ its range, by Ker A its null subspace (kernel), and *A*<sup>∗</sup> means the adjoint operator if it exists. If *A* is invertible then *A*�<sup>1</sup> means its inverse. *A* means the closure of the operator, if the operator is closable. If *A* is bounded then ∥*A*∥ denotes its norm.

## **2. Pencils** *<sup>J</sup>***2***N*þ**<sup>1</sup>** � *<sup>λ</sup>NE* **and orthogonal polynomials on radial rays in the complex plane**

Throughout this section *N* will denote a fixed natural number. Let *J*2*N*þ<sup>1</sup> be a complex Hermitian semi-infinite 2ð Þ *<sup>N</sup>* <sup>þ</sup> <sup>1</sup> -diagonal matrix. Let *pn*ð Þ*<sup>λ</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup>, deg*pn* <sup>¼</sup> *<sup>n</sup>* be a set of complex polynomials, which satisfy the following relation:

$$(J\_{2N+1} - \vec{\lambda}^\nu E)\vec{p}(\lambda) = \mathbf{0},\tag{4}$$

where *p* !ð Þ¼ *<sup>λ</sup> <sup>p</sup>*0ð Þ*<sup>λ</sup>* , *<sup>p</sup>*1ð Þ*<sup>λ</sup>* , … � �*<sup>T</sup>* , is a vector of polynomials, and *E* is the identity matrix. Polynomials, which satisfy (4) with real coefficients, were first studied by Durán in ref. [16], following a suggestion of Marcellán. As it was already noticed in the Introduction, these polynomials are related to MOPRL. Namely, the following polynomials:

$$P\_n(\mathbf{x}) = \begin{pmatrix} R\_{N,0}(p\_{nN})(\mathbf{x}) & R\_{N,1}(p\_{nN})(\mathbf{x}) & \cdots & R\_{N,N-1}(p\_{nN})(\mathbf{x}) \\ R\_{N,0}(p\_{nN+1})(\mathbf{x}) & R\_{N,1}(p\_{nN+1})(\mathbf{x}) & \cdots & R\_{N,N-1}(p\_{nN+1})(\mathbf{x}) \\ \vdots & \vdots & \ddots & \vdots \\ R\_{N,0}(p\_{nN+N-1})(\mathbf{x}) & R\_{N,1}(p\_{nN+N-1})(\mathbf{x}) & \cdots & R\_{N,N-1}(p\_{nN+N-1})(\mathbf{x}) \end{pmatrix} \\ \text{(5)}$$

are orthonormal MOPRL [12, Theorem]. Here

$$R\_{N,m}(p)(t) = \sum\_{n} \frac{p^{(nN+m)}(\mathbf{0})}{(nN+m)!} t^n, \qquad p \in \mathbb{P}, \quad \mathbf{0} \le m \le N-1. \tag{6}$$

Conversely, from a given set *Pn*ð Þ¼ *x Pn*,*m*,*<sup>j</sup>* � �*N*�<sup>1</sup> *m*,*j*¼0 n o<sup>∞</sup> *n*¼0 of orthonormal MOPRL (suitably normed) one can construct scalar polynomials:

$$p\_{nN+m}(\mathbf{x}) = \sum\_{j=0}^{N-1} \mathbf{x}^j P\_{n,m,j}(\mathbf{x}^N), \qquad n \in \mathbb{N}, \ 0 \le m \le N-1,\tag{7}$$

which satisfy relation (4) [12]. Writing the corresponding matrix orthonormality conditions for *Pn* and equating the entries on both sides, one immediately gets orthogonality conditions for *pn*:

$$\begin{aligned} \left\{ \begin{array}{l} \left( \boldsymbol{R}\_{N,0} \left( \boldsymbol{p}\_{n} \right) (\boldsymbol{\pi}), \boldsymbol{R}\_{N,1} \left( \boldsymbol{p}\_{n} \right) (\boldsymbol{\pi}), \dots, \boldsymbol{R}\_{N,N-1} \left( \boldsymbol{p}\_{n} \right) (\boldsymbol{\pi}) \right) d\mu \\ \end{array} \; \begin{aligned} \overline{\left( \begin{array}{l} \boldsymbol{R}\_{N,0} \left( \boldsymbol{p}\_{m} \right) (\boldsymbol{\pi}) \\ \boldsymbol{R}\_{N,1} \left( \boldsymbol{p}\_{m} \right) (\boldsymbol{\pi}) \\ \vdots \\ \boldsymbol{R}\_{N,N-1} \left( \boldsymbol{p}\_{m} \right) (\boldsymbol{\pi}) \end{array} \right)} \end{aligned} \right. \end{aligned} \right. \end{cases}$$

where *μ* is a ð Þ *N* � *N* matrix measure. In the case of real coefficients in (4), this property was obtained by Durán in ref. [17].

Polynomials *pn*ð Þ*<sup>λ</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> also satisfy the following orthogonality relations on radial rays in the complex plane [13]:

$$\begin{aligned} \left\{ \begin{aligned} \left( p\_n(\boldsymbol{\lambda}), p\_n(\boldsymbol{\lambda}\boldsymbol{\varepsilon}), \dots, p\_n(\boldsymbol{\lambda}\boldsymbol{\varepsilon}^{N-1}) \right) dW(\boldsymbol{\lambda}) \begin{pmatrix} p\_m(\boldsymbol{\lambda}) \\ p\_m(\boldsymbol{\lambda}\boldsymbol{\varepsilon}) \\ \vdots \\ p\_m(\boldsymbol{\lambda}\boldsymbol{\varepsilon}^{N-1}) \end{pmatrix} + \\ + \left( p\_n(\mathbf{0}), p'\_n(\mathbf{0}), \dots, p'\_n(\mathbf{0}), \dots, p\_n^{(N-1)}(\mathbf{0}) \right) \mathbf{M} \begin{pmatrix} p\_m(\boldsymbol{\lambda}) \\ p'\_m(\boldsymbol{\lambda}\boldsymbol{\varepsilon}^{N-1}) \\ \vdots \\ p'\_m(\boldsymbol{\lambda}\boldsymbol{\varepsilon}) \end{pmatrix} = \delta\_{n,m}, \qquad n, m \in \mathbb{Z}\_+, \end{aligned} \tag{9}$$

where *W*ð Þ*λ* is a non-decreasing matrix-valued function on *LN*nf g0 ; *M* ∈ *<sup>N</sup>*,*<sup>N</sup>*, *<sup>M</sup>* <sup>≥</sup>0; *LN* <sup>¼</sup> *<sup>λ</sup>*<sup>∈</sup> : *<sup>λ</sup><sup>N</sup>* <sup>∈</sup> � �; *<sup>ε</sup>* is a primitive *<sup>N</sup>*-th root of unity. At *<sup>λ</sup>* <sup>¼</sup> 0 the integral is understood as improper. Relation (9) can be derived from a Favard-type theorem in ref. [12, Theorem], but in ref. [13] we proceeded in another way. Relation (9) easily shows that the following classes of polynomials are included in the class of polynomials from (4):

A. OPRL;


A detailed investigation of polynomials in the case ð Þ *B* was done by Milovanovi'c, see ref. [18] and references therein. In particular, interesting examples of orthogonal polynomials were constructed and zero distribution of polynomials was studied. Discrete Sobolev polynomials from the case ð Þ *C* may possess higher-order differential equations. This subject has a long history, see historical remarks in recent papers [19, 20]. For polynomials (9) some simple general properties of zeros were studied in ref. [21], while a Christoffel type formula was constructed in ref. [22]. In ref. [12] there was studied a more general case of relation (4), with a polynomial *h*ð Þ*λ* instead of *λN*.

## **3. Pencils** *J***<sup>5</sup>** � *xJ***<sup>3</sup> and orthogonal polynomials**

Let *J*<sup>3</sup> be a Jacobi matrix and *J*<sup>5</sup> be a semi-infinite real symmetric five-diagonal matrix with positive numbers on the second subdiagonal. A set Θ ¼ *J*3, *J*<sup>5</sup> ð Þ , *α*, *β* , where *α* >0, *β* ∈ , is said to be *a Jacobi-type pencil (of matrices)* [14]. With a Jacobi-type pencil of matrices <sup>Θ</sup> one associates a system of polynomials *pn*ð Þ*<sup>λ</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup>, which satisfies the following relations:

$$p\_0(\lambda) = \mathbf{1}, \quad p\_1(\lambda) = a\lambda + \beta,\tag{10}$$

and

$$(\vec{J}\_5 - \lambda \vec{J}\_3)\vec{\tilde{p}}(\lambda) = \mathbf{0},\tag{11}$$

where *p* !ð Þ¼ *<sup>λ</sup> <sup>p</sup>*0ð Þ*<sup>λ</sup>* , *<sup>p</sup>*1ð Þ*<sup>λ</sup>* , *<sup>p</sup>*2ð Þ*<sup>λ</sup>* , <sup>⋯</sup> � �*<sup>T</sup>* . Polynomials *pn*ð Þ*<sup>λ</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> are said to be *associated with the Jacobi-type pencil of matrices* Θ.

Observe that for each system of OPRL with *p*<sup>0</sup> ¼ 1 one can take *J*<sup>3</sup> to be the corresponding Jacobi matrix, *J*<sup>5</sup> ¼ *J* 2 3, and *α*, *β* being the coefficients of *p*<sup>1</sup> (*p*1ð Þ¼ *λ αλ* þ *β*). Then, this system is associated with Θ ¼ *J*3, *J*<sup>5</sup> ð Þ , *α*, *β* . Let us mention two other circumstances where Jacobi-type pencils arise in a natural way.

## **1. Discretization of a** 4**-th order differential operator.** Ben Amara, Vladimirov, and Shkalikov investigated the following linear pencil of differential operators [23]:

$$
\lambda \left( p \mathbf{y}^{\prime\prime} \right) - \lambda \left( -\mathbf{y}^{\prime\prime} + c \mathbf{y} \mathbf{y} \right) = \mathbf{0}.\tag{12}
$$

The initial conditions are: *y*ð Þ¼ 0 *y*<sup>0</sup> ð Þ¼ 0 *y*ð Þ¼ 1 *y*<sup>0</sup> ð Þ¼ 1 0, or *y*ð Þ¼ 0 *y*<sup>0</sup> ð Þ¼ 0 *y*0 ð Þ¼ <sup>1</sup> *py*<sup>00</sup> � �<sup>0</sup> ð Þþ 1 *λαy*ð Þ¼ 1 0. Here *p*,*r*∈*C*½ � 0, 1 are uniformly positive, while the parameters *c* and *α* are real. Eq. (12) has several physical applications, which include a motion of a partially fixed bar with additional constraints in the elasticity theory [23]. The discretization of this equation leads to a Jacobi-type pencil, see ref. [24].

*2.* **Partial sums of series of OPRL.** Let *gn*ð Þ *<sup>x</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> (deg*gn* <sup>¼</sup> *<sup>n</sup>*) be orthonormal OPRL with positive leading coefficients. Let f g *ck* <sup>∞</sup> *<sup>k</sup>*¼<sup>0</sup> be a set of arbitrary positive numbers. Then polynomials

$$p\_n(\mathbf{x}) \coloneqq \frac{1}{c\_{\mathbf{0}} \mathbf{g}\_0} \sum\_{j=0}^n c\_{j\mathbf{g}} \mathbf{g}\_j(\mathbf{x}), \qquad n \in \mathbb{Z}\_+,\tag{13}$$

are associated with a Jacobi-type pencil [25, Theorem 1]. Polynomials *pn* are normed partial sums of the following formal power series:

$$\sum\_{j=0}^{\ast} c\_j \mathfrak{g}\_j(\mathfrak{x}).$$

We shall return to such sums below.

From the definition of a Jacobi type pencil we see that matrices *J*<sup>3</sup> and *J*<sup>5</sup> have the following form:

$$J\_3 = \begin{pmatrix} b\_0 & a\_0 & 0 & 0 & 0 & \cdots \\ a\_0 & b\_1 & a\_1 & 0 & 0 & \cdots \\ 0 & a\_1 & b\_2 & a\_2 & 0 & \cdots \\ \cdot & \cdot & \cdot & \cdot & \cdot \end{pmatrix}, \qquad a\_k > 0, \ b\_k \in \mathbb{R}, \ k \in \mathbb{Z}\_+;\tag{14}$$

$$J\_5 = \begin{pmatrix} \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \cdots & \vdots \\ a\_0 & \beta\_0 & \gamma\_0 & \mathbf{0} & \mathbf{0} & \mathbf{0} & \cdots & \\ \beta\_0 & a\_1 & \beta\_1 & \gamma\_1 & \mathbf{0} & \mathbf{0} & \cdots & \\ \gamma\_0 & \beta\_1 & a\_2 & \beta\_2 & \gamma\_2 & \mathbf{0} & \cdots & \\ \mathbf{0} & \gamma\_1 & \beta\_2 & a\_3 & \beta\_3 & \gamma\_3 & \cdots \\ \vdots & \vdots & \vdots & \vdots & \ddots & \\ \end{pmatrix}, \quad a\_n, \beta\_n \in \mathbb{R}, \quad \gamma\_n > 0, \ \ n \in \mathbb{Z}\_+. \tag{15}$$

Set

$$
\mu\_n \coloneqq \vec{l}\_3 \vec{e}\_n = a\_{n-1}\vec{e}\_{n-1} + b\_n \vec{e}\_n + a\_n \vec{e}\_{n+1},\tag{16}
$$

$$w\_n \coloneqq \mathbf{J}\_5 \overrightarrow{e}\_n = \chi\_{n-2} \overrightarrow{e}\_{n-2} + \beta\_{n-1} \overrightarrow{e}\_{n-1} + a\_n \overrightarrow{e}\_n + \beta\_n \overrightarrow{e}\_{n+1} + \chi\_n \overrightarrow{e}\_{n+2}, \qquad n \in \mathbb{Z}\_+. \tag{17}$$

Here and in what follows by *e* ! *<sup>k</sup>* with negative *k* we mean (vector) zero. The following operator:

$$Af = \frac{\zeta}{\alpha} \left( \overrightarrow{e}\_1 - \beta \overrightarrow{e}\_0 \right) + \sum\_{n=0}^{\infty} \xi\_n w\_n,$$

$$f = \zeta \overrightarrow{e}\_0 + \sum\_{n=0}^{\infty} \xi\_n u\_n \in l\_{2\hat{\beta}n}, \quad \zeta, \xi\_n \in \mathbb{C}, \tag{18}$$

with *D A*ð Þ¼ *l*2,*fin* is called *the associated operator for the Jacobi-type pencil* Θ*.* In the sums in (18), only a finite number of *ξ<sup>n</sup>* are nonzero. In what follows we shall always assume this in the case of elements from the linear span. In particular, the following relation holds:

> *AJ*<sup>3</sup> *e* ! *<sup>n</sup>* ¼ *J*<sup>5</sup> *e* ! *<sup>n</sup>*, *n*∈þ*:*

Then

$$A\mathbf{J}\_3 = \mathbf{J}\_5.\tag{19}$$

The matrices *J*<sup>3</sup> and *J*<sup>5</sup> define linear operators with the domain *l*2,*fin*, which we denote by the same letters.

For an arbitrary nonzero polynomial *<sup>f</sup>*ð Þ*<sup>λ</sup>* <sup>∈</sup> of degree *<sup>d</sup>*<sup>∈</sup> þ, *<sup>f</sup>*ð Þ¼ *<sup>λ</sup>* <sup>P</sup>*<sup>d</sup> <sup>k</sup>*¼<sup>0</sup>*dkλk*, *dk* <sup>∈</sup> , we set *f A*ð Þ¼ <sup>P</sup>*<sup>d</sup> <sup>k</sup>*¼<sup>0</sup>*dkAk* . Here *A*<sup>0</sup>≔*E* � � *l*2,*fin* . For *f*ð Þ� *λ* 0, we set *f A*ð Þ¼ 0j *<sup>l</sup>*2,*fin* . The following relations hold [14]:

$$
\overrightarrow{\boldsymbol{e}}\_{n} = \boldsymbol{p}\_{n}(\boldsymbol{A})\overrightarrow{\boldsymbol{e}}\_{0}, \qquad n \in \mathbb{Z}\_{+}; \tag{20}
$$

$$\left(p\_n(A)\stackrel{\rightarrow}{e}\_0, p\_m(A)\stackrel{\rightarrow}{e}\_0\right)\_{l\_2} = \delta\_{n,m}, \qquad n, m \in \mathbb{Z}\_+.\tag{21}$$

Denote by f g *rn*ð Þ*<sup>λ</sup>* <sup>∞</sup> *<sup>n</sup>*¼0, *<sup>r</sup>*0ð Þ¼ *<sup>λ</sup>* 1, the system of polynomials satisfying

$$f\_3 \overrightarrow{r}(\lambda) = \lambda \overrightarrow{r}(\lambda), \quad \overrightarrow{r}(\lambda) = (r\_0(\lambda), r\_1(\lambda), r\_2(\lambda), \dots)^T. \tag{22}$$

These polynomials are orthonormal on the real line with respect to a (possibly nonunique) nonnegative finite measure *σ* on the Borel subsets of (Favard's theorem). Consider the following operator:

$$U\sum\_{n=0}^{\infty}\xi\_n\overrightarrow{e}\_n = \left[\sum\_{n=0}^{\infty}\xi\_n r\_n(\varkappa)\right], \qquad \xi\_n \in \mathbb{R}, \tag{23}$$

which maps *l*2,*fin* onto P. Here, by P we denote a set of all (classes of equivalence which contain) polynomials in *L*<sup>2</sup> *<sup>σ</sup>*. Denote

$$\mathcal{A} = \mathcal{A}\_{\sigma} = UAU^{-1}. \tag{24}$$

The operator <sup>A</sup> <sup>¼</sup> <sup>A</sup>*<sup>σ</sup>* is said to be *the model representation in L*<sup>2</sup> *<sup>σ</sup> of the associated operator A*.

Theorem 1.1 ([14]) Let <sup>Θ</sup> <sup>¼</sup> *<sup>J</sup>*3, *<sup>J</sup>*<sup>5</sup> ð Þ , *<sup>α</sup>*, *<sup>β</sup>* be a Jacobi-type pencil. Let f g *rn*ð Þ*<sup>λ</sup>* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup>, *r*0ð Þ¼ *λ* 1, be a system of polynomials satisfying (22) and *σ* be their (arbitrary) orthogonality measure on <sup>B</sup>ð Þ . The associated polynomials *pn*ð Þ*<sup>λ</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> satisfy the following relations:

$$\int\_{\mathbb{R}} p\_n(\mathcal{A})(\mathbf{1}) \overline{p\_m(\mathcal{A})(\mathbf{1})} d\sigma = \delta\_{n,m}, \qquad n, m \in \mathbb{Z}\_+,\tag{25}$$

where A is the model representation in *L*<sup>2</sup> *<sup>σ</sup>* of the associated operator *A*.

There appears a natural question: *what are the characteristic properties of the operator* A*?* The answer is given by the following theorem.

Theorem 1.2 ([24, Corollary 1]) Let *σ* be a nonnegative measure on Bð Þ with all finite power moments, Ð *d<sup>σ</sup>* <sup>¼</sup> 1, <sup>Ð</sup> j j *g x*ð Þ <sup>2</sup> *dσ* > 0, for any nonzero complex polynomial *g*. A linear operator A in *L*<sup>2</sup> *<sup>σ</sup>* is a model representation in *L*<sup>2</sup> *<sup>σ</sup>* of the associated operator of a Jacobi-type pencil if and only if the following properties hold:

$$\text{i. } D(\mathcal{A}) = \mathcal{P};$$

ii. For each *k*∈ ℤ<sup>þ</sup> it holds:

$$\mathcal{A}\mathfrak{x}^{k} = \mathfrak{f}\_{k,k+1}\mathfrak{x}^{k+1} + \sum\_{j=0}^{k} \mathfrak{f}\_{k,j}\mathfrak{x}^{j},\tag{26}$$

where *ξk*,*k*þ<sup>1</sup> >0, *ξk*,*<sup>j</sup>* ∈ (0≤ *j*≤*k*);

iii. The operator AΛ<sup>0</sup> is symmetric. Here, by Λ<sup>0</sup> we denote the operator of the multiplication by an independent variable in *L*<sup>2</sup> *<sup>σ</sup>* restricted to P.

There is a general subclass of Jacobi-type pencils, for which elements much more can be said about their associated operators and models [24]. Here we used some ideas from the general theory of operator pencils, see ref. [1, Chapter IV, p. 163].

Let <sup>Θ</sup> <sup>¼</sup> *<sup>J</sup>*3, *<sup>J</sup>*<sup>5</sup> ð Þ , *<sup>α</sup>*, *<sup>β</sup>* be a Jacobi-type pencil and <sup>A</sup> be a model representation in *<sup>L</sup>*<sup>2</sup> *σ* of the associated operator of Θ. By Theorem 1.2 we see that AΛ<sup>0</sup> is symmetric:

$$(\mathcal{A}\Lambda\_0[\mu(\lambda)], [\nu(\lambda)])\_{L^2\_\sigma} = ([\mu(\lambda)], \mathcal{A}\Lambda\_0[\nu(\lambda)])\_{L^2\_\sigma}, \qquad \mu, \nu \in \mathcal{P}. \tag{27}$$

Suppose that the measure *σ* is supported inside a finite real segment ½ � *a*, *b* , 0<*a*<*b*< þ ∞, that is, *σ*ð Þ¼ n½ � *a*, *b* 0. In this case, the operator Λ of the multiplication by an independent variable has a bounded inverse on the whole *L*<sup>2</sup> *<sup>σ</sup>*. Using (27) we may write:

$$\left(\Lambda^{-1}\mathcal{A}[\lambda u(\lambda)], [\lambda v(\lambda)]\right)\_{L^2\_\sigma} = \left(\Lambda^{-1}[\lambda u(\lambda)], \mathcal{A}[\lambda v(\lambda)]\right)\_{L^2\_\sigma}, \qquad u, v \in \mathcal{P}.\tag{28}$$

Denote P<sup>0</sup> ¼ ΛP and A<sup>0</sup> ¼ Aj <sup>P</sup><sup>0</sup> . Then

$$(\Lambda^{-1}\mathcal{A}\mathfrak{g}f,\mathfrak{g})\_{L^{2}\_{\sigma}}=(\Lambda^{-1}f,\mathcal{A}\mathfrak{g})\_{L^{2}\_{\sigma}},\qquad f,\mathfrak{g}\in\mathcal{P}\_{0}.\tag{29}$$

Then A<sup>0</sup> is symmetric with respect to the form Λ�<sup>1</sup> �, � � � *L*2 *σ* . Thus, *in this case, the operator* A *is a perturbation of a symmetric operator*.

Consider two examples of Jacobi-type pencils which show that Sobolev orthogonality is close to them.

**Example 3.1.** ([26]). Let *σ* be a nonnegative measure on Bð Þ with all finite power moments, Ð *d<sup>σ</sup>* <sup>¼</sup> 1, <sup>Ð</sup> j j *g x*ð Þ <sup>2</sup> *dσ* >0, for any nonzero complex polynomial *g*. The following operator:

$$\mathcal{A}[p(\lambda)] = \Lambda\_0[p(\lambda)] + p(\mathbf{0})[c\lambda + d], \qquad p \in \mathbb{P}, \tag{30}$$

where *c*> � 1 and *d*∈ , satisfies properties ð Þ*i* -ð Þ *iii* of Theorem 1.2. Let *J*<sup>3</sup> be the Jacobi matrix, corresponding to the measure *σ*, and *J*<sup>5</sup> ¼ *J* 2 3. Define *α*, *β* in the following way:

$$a = \frac{1}{\xi\_{0,1}\sqrt{\Delta\_1}}, \quad \beta = -\frac{\xi\_{0,1}\epsilon\_1 + \xi\_{0,0}}{\xi\_{0,1}\sqrt{\Delta\_1}}.\tag{31}$$

Here *sj* are the power moments of *σ*, while Δ*n*≔detð Þ *sk*þ*<sup>l</sup> n <sup>k</sup>*,*l*¼<sup>0</sup>, *<sup>n</sup>* <sup>∈</sup>þ, <sup>Δ</sup>�<sup>1</sup>≔1 are the Hankel determinants. The coefficients *ξ<sup>k</sup>*,*<sup>j</sup>* are taken from property (ii) of Theorem 1.2. Let <sup>Θ</sup> <sup>¼</sup> *<sup>J</sup>*3, *<sup>J</sup>*<sup>5</sup> ð Þ , *<sup>α</sup>*, *<sup>β</sup>* . Denote by *pn*ð Þ*<sup>λ</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> the associated polynomials to the pencil <sup>Θ</sup>, and denote by f g *rn*ð Þ*<sup>λ</sup>* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> the orthonormal polynomials (with positive leading coefficients) with respect to the measure *σ*. Then

$$p\_n(\lambda) = \frac{1}{c+1}r\_n(\lambda) - \frac{d}{c+1}\frac{r\_n(\lambda) - r\_n(0)}{\lambda} + \frac{c}{c+1}r\_n(0), \qquad n \in \mathbb{Z}\_+;\tag{32}$$

$$r\_n(\lambda) = (c+1)p\_n(\lambda) + (c+1)d\frac{p\_n(\lambda) - p\_n(d)}{\lambda - d} - cp\_n(d), \qquad n \in \mathbb{Z}\_+. \tag{33}$$

In (32), (33) we mean the limit values at *λ* ¼ 0 and *λ* ¼ *d*, respectively. The following recurrence relation, involving three subsequent associated polynomials, holds:

$$dp\_n(\lambda) = \frac{p\_n(d)}{c+1}(c\lambda + d) + a\_{n-1}p\_{n-1}(\lambda) + b\_n p\_n(\lambda) + a\_n p\_{n+1}(\lambda), \ n \in \mathbb{Z}\_+, \ (\lambda \in \mathbb{C}).\tag{34}$$

The following orthogonality relations hold:

$$\begin{pmatrix} \int\_{\mathbb{R}\backslash\{d\}} \left( p\_n(\lambda), p\_n(d) \right) \\\\ \left( \begin{matrix} (c+1)^2 \left( \frac{\lambda}{\lambda-d} \right)^2 & (-c-1) \frac{\lambda(c\lambda+d)}{(\lambda-d)^2} \\\\ (-c-1) \frac{\lambda(c\lambda+d)}{(\lambda-d)^2} & \frac{\left( \frac{c\lambda+d}{\lambda-d} \right)^2}{(\lambda-d)} \end{matrix} \right) \begin{pmatrix} p\_m(\lambda) \\ p\_m(d) \end{pmatrix} d\sigma + \\ + \left( p\_n(d), p\_n'(d) \right) \begin{pmatrix} 1 & (c+1)d \\\\ (c+1)d & (c+1)^2 d^2 \end{pmatrix} \begin{pmatrix} p\_m(d) \\ p\_m'(d) \end{pmatrix} \sigma(\{d\}) = \delta\_{n,m}, \ n, m \in \mathbb{Z}\_+. \end{cases} \tag{35}$$

Polynomials *pn*ð Þ*λ* can have multiple or complex roots.

Suppose additionally that *σ* and *J*<sup>3</sup> correspond to orthonormal Jacobi polynomials *rn*ð Þ¼ *λ Pn*ð Þ *λ*; *a*, *b* (*a*, *b*> � 1) and *c* ¼ 0; *d* ¼ 1. In this case, the associated polynomial *pn* (*n* ∈þ):

$$p\_n(\lambda) = r\_n(\lambda) - \frac{r\_n(\lambda) - r\_n(0)}{\lambda},\tag{36}$$

is a unique, up to a constant multiple, real *n*-th degree polynomial solution of the following 4-th order differential equation:

$$\begin{aligned} &-(t+1)t(t-1)^2y^{(4)}(t)+(t-1)\left(-(a+b+10)t^2+(b-a)t+4\right)y^{(3)}(t) \\ &++\left(-3(2a+2b+8)t^2+(a+9b+22)t+3a-3b\right)y'(t) \\ &+(-6(a+b+2)t+2a+6b+8)y'(t) \\ &++\lambda\_n(t(t-1)y''(t)+2(2t-1)y'(t)+2y(t)) \\ &=0, \end{aligned} \tag{37}$$

where *λ<sup>n</sup>* ¼ *n n*ð Þ þ *a* þ *b* þ 1 .

Moreover, there exists a unique *λ<sup>n</sup>* ∈ , such that differential Eq. (37) has a real *n*th degree polynomial solution.

**Example 3.2.** ([25]). Recall that Jacobi polynomials *<sup>P</sup>*ð Þ *<sup>α</sup>*,*<sup>β</sup> <sup>n</sup>* ð Þ *<sup>x</sup>* :

$$P\_n^{(a,\beta)}(\mathbf{x}) = \binom{n+a}{n}\_2 F\_1\left(-n, n+a+\beta+1; a+1; \frac{1-\mathbf{x}}{2}\right), \ n \in \mathbb{Z}\_+, \ \beta$$

are orthogonal on ½ � �1, 1 with respect to the weight *w x*ð Þ¼ ð Þ <sup>1</sup> � *<sup>x</sup> <sup>α</sup>* ð Þ <sup>1</sup> <sup>þ</sup> *<sup>x</sup> <sup>β</sup>* , *α*, *β* > � 1. Orthonormal polynomials have the following form:

$$\widehat{P}\_0^{(a,\beta)}(\mathbf{x}) = \frac{\mathbf{1}}{\sqrt{2^{a+\beta+1}\mathbf{B}(a+\mathbf{1},\beta+\mathbf{1})}},$$

$$\widehat{P}\_n^{(a,\beta)}(\mathbf{x}) = \sqrt{\frac{(2n+a+\beta+1)n!\Gamma(n+a+\beta+1)}{2^{a+\beta+1}\Gamma(n+a+1)\Gamma(n+\beta+1)}}P\_n^{(a,\beta)}(\mathbf{x}), \quad n \in \mathbb{N}.$$

Let *c*>0 be an arbitrary positive number. Set

$$D\_{a,\beta,\epsilon} \coloneqq \left(\mathbf{x}^2 - \mathbf{1}\right) \frac{d^2}{d\mathbf{x}^2} + \left[ (\alpha + \beta + 2)\mathbf{x} + a - \beta \right] \frac{d}{d\mathbf{x}} + c,\tag{38}$$

$$l\_{n, \varepsilon} \coloneqq \varepsilon + n(n + a + \beta + \mathbf{1}). \tag{39}$$

Define the following polynomials:

$$P\_n(a,\beta,c,t\_0;\mathbf{x}) \coloneqq \sum\_{k=0}^n \frac{1}{l\_{k,c}} \widehat{P}\_k^{(a,\beta)}(t\_0) \widehat{P}\_k^{(a,\beta)}(\mathbf{x}), \qquad n \in \mathbb{Z}\_+,\tag{40}$$

where *t*<sup>0</sup> ≥1 is an arbitrary parameter. Notice that normed by eigenvalues polynomial kernels of some Sobolev orthogonal polynomials appeared earlier in literature, see ref. [27].

Theorem 1.3. Let *α*, *β* > � 1; *c*>0, and *t*<sup>0</sup> ≥1, be arbitrary parameters. Polynomials *Pn*ð Þ¼ *x Pn*ð Þ *α*, *β*,*c*, *t*0; *x* , from (40), are Sobolev orthogonal polynomials on :

$$\begin{split} \int\_{-1}^{1} \left( P\_{n}(\mathbf{x}), P\_{n'}(\mathbf{x}), P\_{n'}(\mathbf{x}) \right) M\_{n, \beta, \varepsilon}(\mathbf{x}) \begin{pmatrix} P\_{m}(\mathbf{x}) \\ P\_{m}'(\mathbf{x}) \\ P\_{m}''(\mathbf{x}) \\ P\_{m}''(\mathbf{x}) \end{pmatrix} (t\_{0} - \mathbf{x}) (1 - \mathbf{x})^{n} (1 + \mathbf{x})^{\beta} d\mathbf{x} = \\ \quad \quad \quad \quad \quad \quad n, m \in \mathbb{Z}\_{+}, \end{split} \tag{41}$$

where *An* are some positive numbers and

$$\mathcal{M}\_{a,\beta,\varepsilon} = \equiv \begin{pmatrix} c \\ (a+\beta+2)\mathfrak{x}+a-\beta \\ \mathfrak{x}^2-\mathfrak{1} \end{pmatrix} \Big( c, (a+\beta+2)\mathfrak{x}+a-\beta, \mathfrak{x}^2-\mathfrak{1} \Big). \tag{42}$$

For *Pn*ð Þ *α*, *β*,*c*, 1; *x* the following differential equation holds:

$$D\_{a+1,\beta,0}D\_{a,\beta,\varepsilon}P\_n(a,\beta,\varepsilon,1;\infty) = l\_{n,0}D\_{a,\beta,\varepsilon}P\_n(a,\beta,\varepsilon,1;\infty), \quad n \in \mathbb{Z}\_+,\tag{43}$$

where *D<sup>α</sup>*,*β*,*<sup>c</sup>*, *ln*,*<sup>c</sup>* are defined by (38), (39).

## **4. Pencils of banded matrices and Sobolev orthogonality**

Let K denote the real line or the unit circle. The following problem was stated in ref. [28], see also ref. [29]:

**Problem 1.** *To describe all Sobolev orthogonal polynomials yn*ð Þ*<sup>z</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> *on* <sup>K</sup>*, satisfying the following two properties:*

a. *Polynomials yn*ð Þ*z satisfy the following differential equation:*

$$R\mathbf{y}\_n(\mathbf{z}) = \lambda\_n \mathbf{S} \mathbf{y}\_n(\mathbf{z}), \qquad n = \mathbf{0}, \mathbf{1}, \mathbf{2}, \dots, \tag{44}$$

where *R*, *S* are linear differential operators of finite orders, having complex polynomial coefficients not depending on *n*; *λ<sup>n</sup>* ∈ ;

b. *Polynomials yn*ð Þ*z obey the following difference equation:*

$$L\overrightarrow{\boldsymbol{y}}(\boldsymbol{z}) = \boldsymbol{z}\boldsymbol{M}\overrightarrow{\boldsymbol{y}}(\boldsymbol{z}), \quad \overrightarrow{\boldsymbol{y}}(\boldsymbol{z}) = \begin{pmatrix} \boldsymbol{y}\_{0}(\boldsymbol{z}), \boldsymbol{y}\_{1}(\boldsymbol{z}), \dots \end{pmatrix}^{T}, \tag{45}$$

where *L*, *M* are semi-infinite complex banded (i.e., having a finite number of nonzero diagonals) matrices.

Relation (44) shows that *yn* is an eigenfunction of the operator pencil *R* � *λS*, while relation (45) means that vectors of *yn*ð Þ*z* are eigenfunctions of the operator pencil *L* � *zM*. We emphasize that in Problem 1 we do not exclude OPRL or OPUC. They are formally considered as Sobolev orthogonal polynomials with the derivatives of order 0. In this way, we may view systems from Problem 1 as generalizations of systems of classical orthogonal polynomials (see, e.g., the book [30], and papers [20, 31, 32] for more recent developments on this subject, as well as references therein). Related topics are also studied for systems of biorthogonal rational functions, see, for example, ref. [33]. Conditions ð Þ *a* ,ð Þ *b* of Problem 1 are close to bispectral problems, and in particular, to the Bochner-Krall problem (see refs. [31, 34–36] and papers cited therein).

One example of Sobolev orthogonal polynomials, which satisfy conditions of Problem 1, we have already met in Example 3.2. In ref. [37] there was proposed a way to construct such systems of polynomials. Let *pn*ð Þ *<sup>x</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> (*pn* has degree *<sup>n</sup>* and real coefficients) be orthogonal polynomials on ½ � **a**, **b** ⊆ with respect to a weight function *w x*ð Þ:

$$\int\_{\mathfrak{a}}^{\mathfrak{b}} p\_n(\mathfrak{x}) p\_m(\mathfrak{x}) w(\mathfrak{x}) d\mathfrak{x} = A\_n \delta\_{n,m}, \qquad A\_n > 0, \quad n, m \in \mathbb{Z}\_+. \tag{46}$$

The weight *w* is supposed to be continuous on ð Þ **a**, **b** . Denote

$$D\_{\xi}y(\mathbf{x}) = \sum\_{k=0}^{\xi} d\_{k}(\mathbf{x}) y^{(k)}(\mathbf{x}),\tag{47}$$

where *dk* and *y* are real polynomials of *x*: *d<sup>ξ</sup>* 6¼ 0. Let us fix a positive integer *ξ*, and consider the following differential equation:

$$D\_{\xi}y(\mathbf{x}) = p\_n(\mathbf{x}),\tag{48}$$

where *D<sup>ξ</sup>* is defined as in Eq. (47), and *n* ∈þ. The following assumption plays a key role here.

**Condition 1.** *Suppose that for each n*∈þ*, the differential Eq. (48) has a real n-th degree polynomial solution y x*ð Þ¼ *yn*ð Þ *x .*

If Condition 1 is satisfied, by relations (46),(48) we immediately obtain that *yn*ð Þ *<sup>x</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> are Sobolev orthogonal polynomials:

$$\left(\Box^{\mathsf{b}}\left(\boldsymbol{y}\_{n}(\boldsymbol{\mathsf{x}}),\boldsymbol{y}\_{n}^{\prime}(\boldsymbol{\mathsf{x}}),\ldots,\boldsymbol{y}\_{n}^{\prime(\boldsymbol{\xi})}(\boldsymbol{\mathsf{x}})\right)\mathcal{M}(\boldsymbol{\mathsf{x}})\begin{pmatrix}\boldsymbol{y}\_{m}(\boldsymbol{\mathsf{x}})\\\boldsymbol{y}\_{m}^{\prime}(\boldsymbol{\mathsf{x}})\\\vdots\\\boldsymbol{y}\_{m}^{\prime(\boldsymbol{\xi})}(\boldsymbol{\mathsf{x}})\\\boldsymbol{y}\_{m}^{(\boldsymbol{\xi})}(\boldsymbol{\mathsf{x}})\end{pmatrix}\boldsymbol{w}(\boldsymbol{x})d\boldsymbol{x}=\boldsymbol{A}\_{n}\delta\_{n,m},\tag{49}$$

where

$$M(\mathbf{x}) \coloneqq \begin{pmatrix} d\_0(\mathbf{x}) \\ d\_1(\mathbf{x}) \\ \vdots \\ d\_{\vec{\xi}}(\mathbf{x}) \end{pmatrix} (d\_0(\mathbf{x}), d\_1(\mathbf{x}), \dots, d\_{\vec{\xi}}(\mathbf{x})), \qquad \mathbf{x} \in (\mathbf{a}, \mathbf{b}). \tag{50}$$

Moreover, if *pn* satisfy a differential equation, then *yn* satisfy a differential equation as well. Question: when Condition 1 is satisfied? An answer is given by the following proposition.

Proposition 1 ([28, Proposition 2.1]) Let *D* be a linear differential operator of order *r*∈ , with complex polynomial coefficients:

$$D = \sum\_{k=0}^{r} d\_k(z) \frac{d^k}{dz^k}, \quad d\_k(z) \in \mathbb{P}.$$

Let f g *un*ð Þ*<sup>z</sup>* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup>, deg*un* <sup>¼</sup> *<sup>n</sup>*, be an arbitrary set of complex polynomials. The following statements are equivalent:

(A) The following equation:

$$D\mathbf{y}(\mathbf{z}) = \mathfrak{u}\_n(\mathbf{z}),\tag{51}$$

for each *n*∈ ℤþ, has a complex polynomial solution *y z*ð Þ¼ *yn*ð Þ*z* of degree *n*;

(B) *Dz<sup>n</sup>* is a complex polynomial of degree *<sup>n</sup>*, <sup>∀</sup>*<sup>n</sup>* <sup>∈</sup> <sup>ℤ</sup>þ;

(C) The following conditions hold:

$$
\deg d\_k \le k, \qquad 0 \le k \le r; \tag{52}
$$

$$\sum\_{j=0}^{r} [n]\_j d\_{jj} \neq 0, \qquad n \in \mathbb{Z}\_+,\tag{53}$$

where *d <sup>j</sup>*,*<sup>l</sup>* means the coefficient by *z<sup>l</sup>* of the polynomial *d <sup>j</sup>*.

If one of the statements ð Þ *A* ,ð Þ *B* ,ð Þ *C* holds true, then for each *n* ∈þ, the solution of (51) is unique.

Observe that condition (53) holds true, if the following simple condition holds:

$$d\_{0,0} > 0, \quad d\_{j,j} \ge 0, \qquad j \in \mathbb{Z}\_{1,r}.\tag{54}$$

Thus, *there exists a big variety of linear differential operators with polynomial coefficients that have property A*ð Þ. This leads to various Sobolev orthogonal polynomials.

In ref. [37] there were constructed families of Sobolev orthogonal polynomials on the real line, depending on an arbitrary finite number of complex parameters. Namely, we considered the following hypergeometric polynomials:

$$\begin{split} \mathbf{L}\_{n}(\mathbf{x}) &= \mathbf{L}\_{n}(\mathbf{x}; a, \kappa\_{1}, \dots, \kappa\_{\delta}) = \\ = & \boldsymbol{\kappa}\_{\delta+1} F\_{\delta+1}(-n, \mathbf{1}, \dots, \mathbf{1}; a+\mathbf{1}, \kappa\_{1}+\mathbf{1}, \dots, \kappa\_{\delta}+\mathbf{1}; \mathbf{x}), \end{split} \tag{55}$$
 
$$\begin{split} \mathbf{P}\_{n}(\mathbf{x}) &= \mathbf{P}\_{n}(\mathbf{x}; a, \beta, \kappa\_{1}, \dots, \kappa\_{\delta}) = \\ = & \boldsymbol{\kappa}\_{\delta+2} F\_{\delta+1}(-n, n+a+\beta+\mathbf{1}, \mathbf{1}, \dots, \mathbf{1}; a+\mathbf{1}, \kappa\_{1}+\mathbf{1}, \dots, \kappa\_{\delta}+\mathbf{1}; \mathbf{x}), \\ & \qquad a, \beta, \kappa\_{1}, \dots, \kappa\_{\delta} > -\mathbf{1}, \quad n \in \mathbb{Z}\_{+}. \end{split} \tag{56}$$

Here *pFq* is a usual notation for the generalized hypergeometric function, and *δ* is a positive integer. These families obey differential equations. As for recurrence relations, they were only constructed for the case *δ* ¼ 1.

In ref. [29] a family of hypergeometric Sobolev orthogonal polynomials on the unit circle was considered:

$$\mathcal{Y}\_n(\mathbf{x}) = \frac{(-\mathbf{1})^\rho}{n!} \mathfrak{x}^n \,\_2F\_0\left( \begin{matrix} -n, \rho; \ - \end{matrix}; -\frac{\mathbf{1}}{\mathbf{x}} \right),$$

depending on a parameter *ρ*∈ . Observe that the reversed polynomials to *yn* appeared in numerators of some biorthogonal rational functions, see [38].

Let *gn*ð Þ*<sup>t</sup>* � �<sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> be a system of OPRL or OPUC, having a generating function of the following form:

$$G(t, w) = f(w)e^{tu(w)} = \sum\_{n=0}^{\infty} \mathcal{g}\_n(t) \frac{w^n}{n!}, \qquad t \in \mathbb{C}, \quad |w| < R\_0, \quad (R\_0 > 0), \tag{57}$$

where *f*, *u* are analytic functions in the circle f g j*w*j<*R*<sup>0</sup> , *u*ð Þ¼ 0 0. Such generating functions for OPRL were studied by Meixner, see, for instance, ref. [39, p. 273]. In the case of OPUC, we do not know any such a system, besides *<sup>z</sup><sup>n</sup>* f g<sup>∞</sup> *<sup>n</sup>*¼0. Consider the following function:

$$\begin{aligned} F(t, w) &= \frac{1}{p(u(w))} G(t, w) = \frac{1}{p(u(w))} f(w) e^{\mu(w)}, \\ &\quad t \in \mathbb{R}, \quad |w| < R\_1 < R\_0, \quad (R\_1 > 0), \end{aligned} \tag{58}$$

where *p* ∈: *p*ð Þ 0 6¼ 0. In the case *u z*ð Þ¼ *z*, one should take *R*<sup>1</sup> ≤ ∣*z*0∣, where *z*<sup>0</sup> is a root of *p* with the smallest modulus. This ensures that *F t*ð Þ , *w* is an analytic function of two variables in any polydisk *CT*1,*R*<sup>1</sup> <sup>¼</sup> ð Þ *<sup>t</sup>*, *<sup>w</sup>* <sup>∈</sup> <sup>2</sup> : <sup>j</sup>*t*j<*T*1, *<sup>w</sup>*j<sup>&</sup>lt; *<sup>R</sup>*1<sup>j</sup> � �, *<sup>T</sup>*<sup>1</sup> <sup>&</sup>gt;0. In the general case, since *p u*ð Þ¼ ð Þ 0 *p*ð Þ 0 6¼ 0, there also exists a suitable *R*1, which guarantees that *F* is analytic in *CT*1,*R*<sup>1</sup> . Expand the function *F t*ð Þ , *w* in Taylor's series by *w* with a fixed *t*:

$$F(t, w) = \sum\_{n=0}^{\infty} \varphi\_n(t) \frac{w^n}{n!}, \qquad (t, w) \in C\_{T\_1, R\_1}, \tag{59}$$

where *φn*ð Þ*t* are some complex-valued functions. Then the function *φn*ð Þ*t* is a complex polynomial of degree *n*, ∀*n*∈þ, see [28, Lemma 3.5]. Suppose that deg*p*≥ 1, and

$$p(\mathbf{z}) = \sum\_{k=0}^{d} c\_k \mathbf{z}^k, \qquad c\_k \in \mathbb{C}, \ c\_d \neq \mathbf{0}; \ c\_0 \neq \mathbf{0}; \ d \in \mathbb{N}.\tag{60}$$

Theorem 1.4 ([28, Theorem 3.7]) Let *<sup>d</sup>*<sup>∈</sup> , and *p z*ð Þ be as in (60). Let *gn*ð Þ*<sup>t</sup>* � �<sup>∞</sup> *n*¼0 be a system of OPRL or OPUC, having a generating function *G t*ð Þ , *w* from (57) and *F t*ð Þ , *w* be given by (58). Fix some positive *T*1, *R*1, such that *F t*ð Þ , *w* is analytic in the polydisk *CT*1,*R*<sup>1</sup> . Polynomials

$$\rho\_n(\mathbf{z}) = \sum\_{j=0}^n \binom{n}{j} b\_j \mathbf{g}\_{n-j}(t), \qquad n \in \mathbb{Z}\_+,\tag{61}$$

where *<sup>b</sup> <sup>j</sup>* <sup>¼</sup> <sup>1</sup> *puw* ð Þ ð Þ � �ð Þ*<sup>j</sup>* ð Þ 0 , have the following properties:

i. Polynomials *φ<sup>n</sup>* are Sobolev orthogonal polynomials:

$$\int \left(\rho\_n(t), \rho'\_n(t), \dots, \rho\_n^{(d)}(t)\right) \check{M} \begin{pmatrix} \rho\_m(t) \\ \rho'\_m(t) \\ \vdots \\ \rho'\_m(t) \\ \rho\_m^{(d)}(t) \end{pmatrix} d\mu\_g = \tau\_n \delta\_{n,m},$$
 
$$\tau\_n > 0, \quad n, m \in \mathbb{Z}\_+,$$

where

$$
\tilde{M} = \begin{pmatrix} c\_0, c\_1, \dots, c\_d \end{pmatrix}^T \begin{pmatrix} \overline{c\_0}, \overline{c\_1}, \dots, \overline{c\_d} \end{pmatrix}.
$$

Here *dμ<sup>g</sup>* is the measure of orthogonality of *gn*.

i. Polynomials *φ<sup>n</sup>* have the generating function *F t*ð Þ , *w* , and relation (59) holds.

ii. Polynomials *φ<sup>n</sup>* have the following integral representation:

$$\rho\_n(t) = \frac{n!}{2\pi i} \oint\_{|w|=R\_2} \frac{1}{p(u(w))} f(w) e^{\mu(w)} w^{-n-1} dw, \qquad n \in \mathbb{Z}\_+,\tag{62}$$

where *R*<sup>2</sup> is an arbitrary number, satisfying 0< *R*<sup>2</sup> <*R*1.

There are two cases of *gn*, which lead to additional properties of *φn*, namely, to differential equations and recurrence relations. The next two corollaries are devoted to them.

Corollary 1 ([28]) In conditions of Theorem 1.4 suppose that *gn*ðÞ¼ *t t <sup>n</sup>*, *<sup>n</sup>* <sup>∈</sup>þ; *f w*ð Þ¼ 1, *u w*ð Þ¼ *<sup>w</sup>*. Polynomials *<sup>φ</sup><sup>n</sup>* f g ð Þ*<sup>t</sup>* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> satisfy the following recurrence relation: *Pencils of Semi-Infinite Matrices and Orthogonal Polynomials DOI: http://dx.doi.org/10.5772/intechopen.102422*

$$\begin{split} &(n+1) \sum\_{k=0}^{d} \rho\_{n+1-k}(t) \frac{c\_k}{(n+1-k)!} = \\ &= t \left( \sum\_{k=0}^{d} \rho\_{n-k}(t) \frac{c\_k}{(n-k)!} \right), \qquad n \in \mathbb{Z}\_+, \end{split} \tag{63}$$

where *φr*≔0, *r*!≔1, for *r*∈ : *r*<0. Polynomials *<sup>φ</sup><sup>n</sup>* f g ð Þ*<sup>t</sup>* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> obey the following differential equation:

$$t\sum\_{k=0}^{d} c\_k \rho\_n^{(k+1)}(t) = n \left(\sum\_{k=0}^{d} c\_k \rho\_n^{(k)}(t)\right), \qquad n \in \mathbb{Z}\_+.$$

Corollary 2 ([28]) In conditions of Theorem 1.4 suppose that *gn*ðÞ¼ *t Hn*ð Þ*t* , *n* ∈þ, are Hermite polynomials; *f w*ð Þ¼ *<sup>e</sup>*�*w*<sup>2</sup> , *u w*ð Þ¼ <sup>2</sup>*w*. Polynomials *<sup>φ</sup><sup>n</sup>* f g ð Þ*<sup>t</sup>* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> satisfy the following recurrence relation:

$$\begin{split} \rho(n+1) \sum\_{k=0}^{d} \rho\_{n+1-k}(t) \frac{c\_k \mathfrak{L}^k}{(n+1-k)!} &+ 2 \sum\_{k=0}^{d} \rho\_{n-1-k}(t) \frac{c\_k \mathfrak{L}^k}{(n+1-k)!} = \\ &= 2t \left( \sum\_{k=0}^{d} \rho\_{n-k}(t) \frac{c\_k \mathfrak{L}^k}{(n-k)!} \right), \qquad n \in \mathbb{N}, \end{split} \tag{64}$$

where *φr*≔0, *r*!≔1, for *r*∈ : *r*<0; and

$$
\omega\_0 \rho\_1(t) + 2c\_1 \rho\_0(t) = 2c\_0 t \rho\_0(t). \tag{65}
$$

Polynomials *<sup>φ</sup><sup>n</sup>* f g ð Þ*<sup>t</sup>* <sup>∞</sup> *<sup>n</sup>*¼<sup>0</sup> obey the following differential equation:

$$\sum\_{k=0}^{d} c\_k \rho\_n^{(k)}(t) - 2t \sum\_{k=0}^{d} c\_k \rho\_n^{(k+1)}(t) = -2n \left( \sum\_{k=0}^{d} c\_k \rho\_n^{(k)}(t) \right), \qquad n \in \mathbb{Z}\_+. \tag{66}$$

Observe that polynomials *φ<sup>n</sup>* from the last two corollaries fit into the scheme of Problem 1.

## **5. Conclusion**

The theory of orthogonal polynomials is closely related to semi-infinite matrices, as well as to their finite truncations. This interplay has shown its productivity in classical results. Nowadays there appeared new kinds of orthogonality, such as Sobolev orthogonality. It is not yet clear what kind of matrices can be attributed to them. One of candidates is a pencil of matrices, since it appeared in examples. In Section 3 there appeared a pencil of semi-infinite symmetric matrices, while in Section 4 it was a pencil of some banded matrices. In Section 2 we also met a pencil, but it was more close to classical eigenvalue problems of single operators.

The above-mentioned examples of Sobolev orthogonal polynomials also showed that pencils of differential equations appeared here in a natural way. Moreover, there is a large number of differential operators, which have polynomial solutions with

Sobolev orthogonality. This fact promises that Sobolev orthogonal polynomials can find their applications in mathematical physics.

We think that Problem 1 is an appropriate framework for a search and a construction of new Sobolev orthogonal polynomials having nice properties. Notice that one can produce such systems using classical OPRL or OPUC. The differential equation, if it existed, is inherited by new systems of polynomials. The more complicated question is the existence of a recurrence relation.

Besides new families of Sobolev orthogonal polynomials, it is of a big interest finding classes of systems of Sobolev orthogonal polynomials, having recurrence relations. One such a class (orthogonal polynomials on radial rays) was described in Section 2. Thus, it looks reasonable to start not only from Sobolev orthogonality, but from the other side, i.e., from recurrent relations. One such an example of derivation was given by orthogonal polynomials on radial rays from Section 2.

Another possible way was given in Section 3, where we described Jacobi-type pencils. The associated polynomials of a Jacobi type pencil have special orthogonality relations. The associated operator yet has not a suitable functional calculus. As we have seen, under some conditions this operator is a perturbation of a symmetric operator. However, it is not clear how to calculate effectively a polynomial of this operator.

In general, it is a classical situation that the operator theory stands behind special classes of semi-infinite matrices and related objects. The operator theory of single operators is well promoted and it is well recognized by any mathematician. It seems that the theory of operator pencils is less known to the mathematical community. This fact can explain the situation that pencils of semi-infinite matrices and related polynomials appeared on a mathematical scene just recently. We hope that, as in the classical case, these new orthogonal polynomial systems will shed some new light on the theory of operator pencils.

## **Acknowledgements**

The author is grateful to Professors Zolotarev and Yantsevich for their permanent support.

## **Author details**

Sergey Zagorodnyuk V.N. Karazin Kharkiv National University, Kharkiv, Ukraine

\*Address all correspondence to: sergey.m.zagorodnyuk@gmail.com

© 2022 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

*Pencils of Semi-Infinite Matrices and Orthogonal Polynomials DOI: http://dx.doi.org/10.5772/intechopen.102422*

## **References**

[1] Markus AS. Introduction to the Spectral Theory of Polynomial Operator Pencils. With an appendix by M. V. Keldysh. In: Translations of Mathematical Monographs. Vol. 71. Providence, RI: American Mathematical Society; 1988. pp. iv+250

[2] Rodman L. An Introduction to Operator Polynomials. In: Operator Theory: Advances and Applications. Vol. 38. Basel: Birkhåuser Verlag; 1989. pp. xii +389

[3] Parlett BN. The symmetric eigenvalue problem. In: Corrected reprint of the 1980 original. Classics in Applied Mathematics. Vol. 20. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM); 1998 xxiv+398 pp

[4] Gantmacher FR. The Theory of Matrices. Vol. 2. New York: Chelsea Publishing Co; 1959. pp. ix+276

[5] Szegö G. Orthogonal Polynomials. Fourth ed. Providence, R.I: American Mathematical Society, Colloquium Publications, Vol. XXIII; 1975. pp. xiii+432

[6] Akhiezer NI. The Classical Moment Problem and Some Related Questions in Analysis. New York: Hafner Publishing Co.; pp. 1965 x+253

[7] Geronimus JL. Polynomials, Orthogonal on a Circumference and on an Interval. Estimates, Asymptotic Formulas, Orthogonal Series (in Russian), Sovremennye Problemy Matematiki, Gosudarstv. Moscow: Izdat. Fiz.-Mat. Lit; 1958. pp. 240

[8] Simon B. Orthogonal Polynomials on the Unit Circle. Part 1. Classical Theory. Vol. 54, Part 1. Providence, RI: American Mathematical Society Colloquium Publications; 2005. pp. xxvi+466

[9] Bultheel A, González-Vera P, Hendriksen E, Njåstad O. Orthogonal Rational Functions. Cambridge Monographs on Applied and Computational Mathematics. Vol. 5. Cambridge: Cambridge University Press; 1999. pp. xiv+407

[10] Zhedanov A. Biorthogonal rational functions and the generalized eigenvalue problem. Journal of Approximation Theory. 1999;**101**(2):303-329

[11] Krein M. Infinite J-matrices and a matrix-moment problem. (Russian). Doklady Akad. Nauk SSSR (N.S.). 1949; **69**:125-128

[12] Durán AJ, Van Assche W. Orthogonal matrix polynomials and higher-order recurrence relations. Linear Algebra and its Applications. 1995;**219**: 261-280

[13] Zagorodnyuk SM. On generalized Jacobi matrices and orthogonal polynomials. New York Journal of Mathematics. 2003;**9**:117-136

[14] Zagorodnyuk SM. Orthogonal polynomials associated with some Jacobitype pencils (Russian). Ukraïn. Mathematical. Journal. 2017;**68**(9): 1353-1365 translation in Ukrainian Math. J

[15] Marcellán F. Xu, Yuan.: On Sobolev orthogonal polynomials. Expositiones Mathematicae. 2015;**33**(3):308-352

[16] Durán AJ. A generalization of Favard's theorem for polynomials satisfying a recurrence relation. Journal of Approximation Theory. 1993;**74**(1): 83-109

[17] Durán AJ. On orthogonal polynomials with respect to a positive definite matrix of measures. Canadian Journal of Mathematics. 1995;**47**(1): 88-112

[18] Milovanović, GV. Orthogonal polynomials on the radial rays in the complex plane and applications. Proceedings of the Fourth International Conference on Functional Analysis and Approximation Theory. Vol. I(Suppl. 2) (Potenza, 2000). Rend. Circ. Mat. Palermo. 2002, no. 68, part I, 65–94

[19] Durán AJ, de la Iglesia MD. Differential equations for discrete Laguerre-Sobolev orthogonal polynomials. Journal of Approximation Theory. 2015;**195**:70-88

[20] Durán AJ, de la Iglesia MD. Differential equations for discrete Jacobi-Sobolev orthogonal polynomials. Journal of Spectral Theory. 2018;**8**(1): 191-234

[21] Zagorodnyuk SM. Orthogonal polynomials on rays: properties of zeros, related moment problems and symmetries. Zh. Mat. Fiz. Anal. Geom. 2008;**4**(3):395-419

[22] Choque Rivero AE, Zagorodnyuk SM. Orthogonal polynomials on rays: Christoffel's formula. Bol. Soc. Mat. Mexicana. 2009; **15**(2):149-164

[23] Ben AJ, Vladimirov AA, Shkalikov AA. Spectral and oscillatory properties of a linear pencil of fourthorder differential operators. Mathematical Notes. 2013;**94**(1):49-59

[24] Zagorodnyuk SM. The inverse spectral problem for Jacobi-type pencils. SIGMA Symmetry Integrability Geom. Methods Appl. 2017;**13**. Paper No. 085, 16 pp

[25] Zagorodnyuk SM. On series of orthogonal polynomials and systems of classical type polynomials. Ukr. Math. J. 2021;**73**(6):799-810 translation from Ukr. Mat. Zh

[26] Zagorodnyuk SM. Difference equations related to Jacobi-type pencils. J. Difference Equ. Appl. 2018;**24**(10): 1664-1684

[27] Littlejohn LL, Mañas-Mañas JF, Moreno-Balcázar JJ, Wellman R. Differential operator for discrete Gegenbauer-Sobolev orthogonal polynomials: Eigenvalues and asymptotics. Journal of Approximation Theory. 2018;**230**:32-49

[28] Zagorodnyuk SM. On some Sobolev spaces with matrix weights and classical type Sobolev orthogonal polynomials. J. Difference Equ. Appl. 2021;**27**(2): 261-283

[29] Zagorodnyuk SM. On a family of hypergeometric Sobolev orthogonal polynomials on the unit circle. Constr. Math. Anal. 2020;**3**(2):75-84

[30] Koekoek R, Lesky PA, Swarttouw RF. Hypergeometric Orthogonal Polynomials and their q-Analogues. With a foreword by Tom H. Koornwinder. In: Springer Monographs in Mathematics. Berlin: Springer-Verlag; 2010. pp. xx+578

[31] Horozov E. Automorphisms of algebras and Bochner's property for vector orthogonal polynomials. SIGMA Symmetry Integrability Geom. Methods Appl. 2016;**12**. Paper No. 050, 14 pp

[32] Horozov E. d-orthogonal analogs of classical orthogonal polynomials. SIGMA Symmetry Integrability Geom. Methods Appl. 2018;**14**. Paper No. 063, 27 pp

[33] Spiridonov V, Zhedanov A. Classical biorthogonal rational functions on elliptic grids. Comptes Rendus

*Pencils of Semi-Infinite Matrices and Orthogonal Polynomials DOI: http://dx.doi.org/10.5772/intechopen.102422*

Mathématiques des l'Académie des Sciences. 2000;**22**(2):70-76

[34] Duistermaat JJ, Grünbaum FA. Differential equations in the spectral parameter. Communications in Mathematical Physics. 1986;**103**(2): 177-240

[35] Everitt WN, Kwon KH, Littlejohn LL, Wellman R. Orthogonal polynomial solutions of linear ordinary differential equations. Proceedings of the Fifth International Symposium on Orthogonal Polynomials, Special Functions and their Applications (Patras, 1999). Journal of Computational and Applied Mathematics. 2001;**133**(1-2): 85-109

[36] Horozov E. Vector orthogonal polynomials with Bochner's property. Constructive Approximation. 2018; **48**(2):201-234

[37] Zagorodnyuk SM. On some classical type Sobolev orthogonal polynomials. Journal of Approximation Theory. 2020; **250**(105337) 14 pp

[38] Hendriksen E, van Rossum H. Orthogonal Laurent polynomials. Nederl. Akad. Wetensch. Indag. Math. 1986;**48**(1):17-36

[39] Erdélyi A, Magnus W, Oberhettinger F, Tricomi FG. Higher Transcendental Functions. Vol. III. Based, in part, on notes left by Harry Bateman. New York-Toronto-London: McGraw-Hill Book Company, Inc.; 1955. pp. xvii+292

## **Chapter 3**
