Generalized eigenvector

This example is simple but clearly illustrates the point. This type of matrix is used frequently in textbooks.[39][40][41] Suppose

${\displaystyle A={\begin{pmatrix}1&1\\0&1\end{pmatrix}}.}$

Then there is only one eigenvalue, ${\displaystyle \lambda =1}$, and its algebraic multiplicity is m = 2.

Notice that this matrix is in Jordan normal form but is not diagonal. Hence, this matrix is not diagonalizable. Since there is one superdiagonal entry, there will be one generalized eigenvector of rank greater than 1 (or one could note that the vector space ${\displaystyle V}$ is of dimension 2, so there can be at most one generalized eigenvector of rank greater than 1). Alternatively, one could compute the dimension of the nullspace of ${\displaystyle A-\lambda I}$ to be p = 1, and thus there are m – p = 1 generalized eigenvectors of rank greater than 1.

The ordinary eigenvector ${\displaystyle \mathbf {v} _{1}={\begin{pmatrix}1\\0\end{pmatrix}}}$ is computed as usual (see the eigenvector page for examples). Using this eigenvector, we compute the generalized eigenvector ${\displaystyle \mathbf {v} _{2}}$ by solving

${\displaystyle (A-\lambda I)\mathbf {v} _{2}=\mathbf {v} _{1}.}$

Writing out the values:

${\displaystyle \left({\begin{pmatrix}1&1\\0&1\end{pmatrix}}-1{\begin{pmatrix}1&0\\0&1\end{pmatrix}}\right){\begin{pmatrix}v_{21}\\v_{22}\end{pmatrix}}={\begin{pmatrix}0&1\\0&0\end{pmatrix}}{\begin{pmatrix}v_{21}\\v_{22}\end{pmatrix}}={\begin{pmatrix}1\\0\end{pmatrix}}.}$

This simplifies to

${\displaystyle v_{22}=1.}$

The element ${\displaystyle v_{21}}$ has no restrictions. The generalized eigenvector of rank 2 is then ${\displaystyle \mathbf {v} _{2}={\begin{pmatrix}a\\1\end{pmatrix}}}$, where a can have any scalar value. The choice of a = 0 is usually the simplest.

Note that

${\displaystyle (A-\lambda I)\mathbf {v} _{2}={\begin{pmatrix}0&1\\0&0\end{pmatrix}}{\begin{pmatrix}a\\1\end{pmatrix}}={\begin{pmatrix}1\\0\end{pmatrix}}=\mathbf {v} _{1},}$

so that ${\displaystyle \mathbf {v} _{2}}$ is a generalized eigenvector,

${\displaystyle (A-\lambda I)\mathbf {v} _{1}={\begin{pmatrix}0&1\\0&0\end{pmatrix}}{\begin{pmatrix}1\\0\end{pmatrix}}={\begin{pmatrix}0\\0\end{pmatrix}}=\mathbf {0} ,}$

so that ${\displaystyle \mathbf {v} _{1}}$ is an ordinary eigenvector, and that ${\displaystyle \mathbf {v} _{1}}$ and ${\displaystyle \mathbf {v} _{2}}$ are linearly independent and hence constitute a basis for the vector space ${\displaystyle V}$.

This example is more complex than Example 1. Unfortunately, it is a little difficult to construct an interesting example of low order.[42] The matrix

${\displaystyle A={\begin{pmatrix}1&0&0&0&0\\3&1&0&0&0\\6&3&2&0&0\\10&6&3&2&0\\15&10&6&3&2\end{pmatrix}}}$

has eigenvalues ${\displaystyle \lambda _{1}=1}$ and ${\displaystyle \lambda _{2}=2}$ with algebraic multiplicities ${\displaystyle \mu _{1}=2}$ and ${\displaystyle \mu _{2}=3}$, but geometric multiplicities ${\displaystyle \gamma _{1}=1}$ and ${\displaystyle \gamma _{2}=1}$.

The generalized eigenspaces of ${\displaystyle A}$ are calculated below. ${\displaystyle \mathbf {x} _{1}}$ is the ordinary eigenvector associated with ${\displaystyle \lambda _{1}}$. ${\displaystyle \mathbf {x} _{2}}$ is a generalized eigenvector associated with ${\displaystyle \lambda _{1}}$. ${\displaystyle \mathbf {y} _{1}}$ is the ordinary eigenvector associated with ${\displaystyle \lambda _{2}}$. ${\displaystyle \mathbf {y} _{2}}$ and ${\displaystyle \mathbf {y} _{3}}$ are generalized eigenvectors associated with ${\displaystyle \lambda _{2}}$.

${\displaystyle (A-1I)\mathbf {x} _{1}={\begin{pmatrix}0&0&0&0&0\\3&0&0&0&0\\6&3&1&0&0\\10&6&3&1&0\\15&10&6&3&1\end{pmatrix}}{\begin{pmatrix}0\\3\\-9\\9\\-3\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\0\\0\end{pmatrix}}=\mathbf {0} ,}$

${\displaystyle (A-1I)\mathbf {x} _{2}={\begin{pmatrix}0&0&0&0&0\\3&0&0&0&0\\6&3&1&0&0\\10&6&3&1&0\\15&10&6&3&1\end{pmatrix}}{\begin{pmatrix}1\\-15\\30\\-1\\-45\end{pmatrix}}={\begin{pmatrix}0\\3\\-9\\9\\-3\end{pmatrix}}=\mathbf {x} _{1},}$

${\displaystyle (A-2I)\mathbf {y} _{1}={\begin{pmatrix}-1&0&0&0&0\\3&-1&0&0&0\\6&3&0&0&0\\10&6&3&0&0\\15&10&6&3&0\end{pmatrix}}{\begin{pmatrix}0\\0\\0\\0\\9\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\0\\0\end{pmatrix}}=\mathbf {0} ,}$

${\displaystyle (A-2I)\mathbf {y} _{2}={\begin{pmatrix}-1&0&0&0&0\\3&-1&0&0&0\\6&3&0&0&0\\10&6&3&0&0\\15&10&6&3&0\end{pmatrix}}{\begin{pmatrix}0\\0\\0\\3\\0\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\0\\9\end{pmatrix}}=\mathbf {y} _{1},}$

${\displaystyle (A-2I)\mathbf {y} _{3}={\begin{pmatrix}-1&0&0&0&0\\3&-1&0&0&0\\6&3&0&0&0\\10&6&3&0&0\\15&10&6&3&0\end{pmatrix}}{\begin{pmatrix}0\\0\\1\\-2\\0\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\3\\0\end{pmatrix}}=\mathbf {y} _{2}.}$

This results in a basis for each of the generalized eigenspaces of ${\displaystyle A}$. Together the two chains of generalized eigenvectors span the space of all 5-dimensional column vectors.

${\displaystyle \left\{\mathbf {x} _{1},\mathbf {x} _{2}\right\}=\left\{{\begin{pmatrix}0\\3\\-9\\9\\-3\end{pmatrix}}{\begin{pmatrix}1\\-15\\30\\-1\\-45\end{pmatrix}}\right\},\left\{\mathbf {y} _{1},\mathbf {y} _{2},\mathbf {y} _{3}\right\}=\left\{{\begin{pmatrix}0\\0\\0\\0\\9\end{pmatrix}}{\begin{pmatrix}0\\0\\0\\3\\0\end{pmatrix}}{\begin{pmatrix}0\\0\\1\\-2\\0\end{pmatrix}}\right\}.}$

An "almost diagonal" matrix ${\displaystyle J}$ in Jordan normal form, similar to ${\displaystyle A}$ is obtained as follows:

${\displaystyle M={\begin{pmatrix}\mathbf {x} _{1}&\mathbf {x} _{2}&\mathbf {y} _{1}&\mathbf {y} _{2}&\mathbf {y} _{3}\end{pmatrix}}={\begin{pmatrix}0&1&0&0&0\\3&-15&0&0&0\\-9&30&0&0&1\\9&-1&0&3&-2\\-3&-45&9&0&0\end{pmatrix}},}$

${\displaystyle J={\begin{pmatrix}1&1&0&0&0\\0&1&0&0&0\\0&0&2&1&0\\0&0&0&2&1\\0&0&0&0&2\end{pmatrix}},}$

where ${\displaystyle M}$ is a generalized modal matrix for ${\displaystyle A}$, the columns of ${\displaystyle M}$ are a canonical basis for ${\displaystyle A}$, and ${\displaystyle AM=MJ}$.[43]

The matrix

${\displaystyle A={\begin{pmatrix}5&1&-2&4\\0&5&2&2\\0&0&5&3\\0&0&0&4\end{pmatrix}}}$

has an eigenvalue ${\displaystyle \lambda _{1}=5}$ of algebraic multiplicity ${\displaystyle \mu _{1}=3}$ and an eigenvalue ${\displaystyle \lambda _{2}=4}$ of algebraic multiplicity ${\displaystyle \mu _{2}=1}$. We also have ${\displaystyle n=4}$. For ${\displaystyle \lambda _{1}}$ we have ${\displaystyle n-\mu _{1}=4-3=1}$.

${\displaystyle (A-5I)={\begin{pmatrix}0&1&-2&4\\0&0&2&2\\0&0&0&3\\0&0&0&-1\end{pmatrix}},\qquad \operatorname {rank} (A-5I)=3.}$

${\displaystyle (A-5I)^{2}={\begin{pmatrix}0&0&2&-8\\0&0&0&4\\0&0&0&-3\\0&0&0&1\end{pmatrix}},\qquad \operatorname {rank} (A-5I)^{2}=2.}$

${\displaystyle (A-5I)^{3}={\begin{pmatrix}0&0&0&14\\0&0&0&-4\\0&0&0&3\\0&0&0&-1\end{pmatrix}},\qquad \operatorname {rank} (A-5I)^{3}=1.}$

The first integer ${\displaystyle m_{1}}$ for which ${\displaystyle (A-5I)^{m_{1}}}$ has rank ${\displaystyle n-\mu _{1}=1}$ is ${\displaystyle m_{1}=3}$.

We now define

${\displaystyle \rho _{3}=\operatorname {rank} (A-5I)^{2}-\operatorname {rank} (A-5I)^{3}=2-1=1,}$

${\displaystyle \rho _{2}=\operatorname {rank} (A-5I)^{1}-\operatorname {rank} (A-5I)^{2}=3-2=1,}$

${\displaystyle \rho _{1}=\operatorname {rank} (A-5I)^{0}-\operatorname {rank} (A-5I)^{1}=4-3=1.}$

Consequently, there will be three linearly independent generalized eigenvectors; one each of ranks 3, 2 and 1. Since ${\displaystyle \lambda _{1}}$ corresponds to a single chain of three linearly independent generalized eigenvectors, we know that there is a generalized eigenvector ${\displaystyle \mathbf {x} _{3}}$ of rank 3 corresponding to ${\displaystyle \lambda _{1}}$ such that

${\displaystyle (A-5I)^{3}\mathbf {x} _{3}=\mathbf {0} }$

(3)

but

${\displaystyle (A-5I)^{2}\mathbf {x} _{3}\neq \mathbf {0} .}$

(4)

Equations (3) and (4) represent linear systems that can be solved for ${\displaystyle \mathbf {x} _{3}}$. Let

${\displaystyle \mathbf {x} _{3}={\begin{pmatrix}x_{31}\\x_{32}\\x_{33}\\x_{34}\end{pmatrix}}.}$

Then

${\displaystyle (A-5I)^{3}\mathbf {x} _{3}={\begin{pmatrix}0&0&0&14\\0&0&0&-4\\0&0&0&3\\0&0&0&-1\end{pmatrix}}{\begin{pmatrix}x_{31}\\x_{32}\\x_{33}\\x_{34}\end{pmatrix}}={\begin{pmatrix}14x_{34}\\-4x_{34}\\3x_{34}\\-x_{34}\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\0\end{pmatrix}}}$

and

${\displaystyle (A-5I)^{2}\mathbf {x} _{3}={\begin{pmatrix}0&0&2&-8\\0&0&0&4\\0&0&0&-3\\0&0&0&1\end{pmatrix}}{\begin{pmatrix}x_{31}\\x_{32}\\x_{33}\\x_{34}\end{pmatrix}}={\begin{pmatrix}2x_{33}-8x_{34}\\4x_{34}\\-3x_{34}\\x_{34}\end{pmatrix}}\neq {\begin{pmatrix}0\\0\\0\\0\end{pmatrix}}.}$

Thus, in order to satisfy the conditions (3) and (4), we must have ${\displaystyle x_{34}=0}$ and ${\displaystyle x_{33}\neq 0}$. No restrictions are placed on ${\displaystyle x_{31}}$ and ${\displaystyle x_{32}}$. By choosing ${\displaystyle x_{31}=x_{32}=x_{34}=0,x_{33}=1}$, we obtain

${\displaystyle \mathbf {x} _{3}={\begin{pmatrix}0\\0\\1\\0\end{pmatrix}}}$

as a generalized eigenvector of rank 3 corresponding to ${\displaystyle \lambda _{1}=5}$. Note that it is possible to obtain infinitely many other generalized eigenvectors of rank 3 by choosing different values of ${\displaystyle x_{31}}$, ${\displaystyle x_{32}}$ and ${\displaystyle x_{33}}$, with ${\displaystyle x_{33}\neq 0}$. Our first choice, however, is the simplest.[47]

Now using equations (1), we obtain ${\displaystyle \mathbf {x} _{2}}$ and ${\displaystyle \mathbf {x} _{1}}$ as generalized eigenvectors of rank 2 and 1, respectively, where

${\displaystyle \mathbf {x} _{2}=(A-5I)\mathbf {x} _{3}={\begin{pmatrix}-2\\2\\0\\0\end{pmatrix}},}$

and

${\displaystyle \mathbf {x} _{1}=(A-5I)\mathbf {x} _{2}={\begin{pmatrix}2\\0\\0\\0\end{pmatrix}}.}$

The simple eigenvalue ${\displaystyle \lambda _{2}=4}$ can be dealt with using standard techniques and has an ordinary eigenvector

${\displaystyle \mathbf {y} _{1}={\begin{pmatrix}-14\\4\\-3\\1\end{pmatrix}}.}$

A canonical basis for ${\displaystyle A}$ is

${\displaystyle \left\{\mathbf {x} _{3},\mathbf {x} _{2},\mathbf {x} _{1},\mathbf {y} _{1}\right\}=\left\{{\begin{pmatrix}0\\0\\1\\0\end{pmatrix}}{\begin{pmatrix}-2\\2\\0\\0\end{pmatrix}}{\begin{pmatrix}2\\0\\0\\0\end{pmatrix}}{\begin{pmatrix}-14\\4\\-3\\1\end{pmatrix}}\right\}.}$

${\displaystyle \mathbf {x} _{1},\mathbf {x} _{2}}$ and ${\displaystyle \mathbf {x} _{3}}$ are generalized eigenvectors associated with ${\displaystyle \lambda _{1}}$, while ${\displaystyle \mathbf {y} _{1}}$ is the ordinary eigenvector associated with ${\displaystyle \lambda _{2}}$.

This is a fairly simple example. In general, the numbers ${\displaystyle \rho _{k}}$ of linearly independent generalized eigenvectors of rank ${\displaystyle k}$ will not always be equal. That is, there may be several chains of different lengths corresponding to a particular eigenvalue.[48]

Find a matrix in Jordan normal form that is similar to

${\displaystyle A={\begin{pmatrix}0&4&2\\-3&8&3\\4&-8&-2\end{pmatrix}}.}$

Solution: The characteristic equation of ${\displaystyle A}$ is ${\displaystyle (\lambda -2)^{3}=0}$, hence, ${\displaystyle \lambda =2}$ is an eigenvalue of algebraic multiplicity three. Following the procedures of the previous sections, we find that

${\displaystyle \operatorname {rank} (A-2I)=1}$

and

${\displaystyle \operatorname {rank} (A-2I)^{2}=0=n-\mu .}$

Thus, ${\displaystyle \rho _{2}=1}$ and ${\displaystyle \rho _{1}=2}$, which implies that a canonical basis for ${\displaystyle A}$ will contain one linearly independent generalized eigenvector of rank 2 and two linearly independent generalized eigenvectors of rank 1, or equivalently, one chain of two vectors ${\displaystyle \left\{\mathbf {x} _{2},\mathbf {x} _{1}\right\}}$ and one chain of one vector ${\displaystyle \left\{\mathbf {y} _{1}\right\}}$. Designating ${\displaystyle M={\begin{pmatrix}\mathbf {y} _{1}&\mathbf {x} _{1}&\mathbf {x} _{2}\end{pmatrix}}}$, we find that

${\displaystyle M={\begin{pmatrix}2&2&0\\1&3&0\\0&-4&1\end{pmatrix}},}$

and

${\displaystyle J={\begin{pmatrix}2&0&0\\0&2&1\\0&0&2\end{pmatrix}},}$

where ${\displaystyle M}$ is a generalized modal matrix for ${\displaystyle A}$, the columns of ${\displaystyle M}$ are a canonical basis for ${\displaystyle A}$, and ${\displaystyle AM=MJ}$.[54] Note that since generalized eigenvectors themselves are not unique, and since some of the columns of both ${\displaystyle M}$ and ${\displaystyle J}$ may be interchanged, it follows that both ${\displaystyle M}$ and ${\displaystyle J}$ are not unique.[55]

In Example 3, we found a canonical basis of linearly independent generalized eigenvectors for a matrix ${\displaystyle A}$. A generalized modal matrix for ${\displaystyle A}$ is

${\displaystyle M={\begin{pmatrix}\mathbf {y} _{1}&\mathbf {x} _{1}&\mathbf {x} _{2}&\mathbf {x} _{3}\end{pmatrix}}={\begin{pmatrix}-14&2&-2&0\\4&0&2&0\\-3&0&0&1\\1&0&0&0\end{pmatrix}}.}$

A matrix in Jordan normal form, similar to ${\displaystyle A}$ is

${\displaystyle J={\begin{pmatrix}4&0&0&0\\0&5&1&0\\0&0&5&1\\0&0&0&5\end{pmatrix}},}$

so that ${\displaystyle AM=MJ}$.

Three of the most fundamental operations which can be performed on square matrices are matrix addition, multiplication by a scalar, and matrix multiplication.[56] These are exactly those operations necessary for defining a polynomial function of an n × n matrix ${\displaystyle A}$.[57] If we recall from basic calculus that many functions can be written as a Maclaurin series, then we can define more general functions of matrices quite easily.[58] If ${\displaystyle A}$ is diagonalizable, that is

${\displaystyle D=M^{-1}AM,}$

with

${\displaystyle D={\begin{pmatrix}\lambda _{1}&0&\cdots &0\\0&\lambda _{2}&\cdots &0\\\vdots &\vdots &\ddots &\vdots \\0&0&\cdots &\lambda _{n}\end{pmatrix}},}$

then

${\displaystyle D^{k}={\begin{pmatrix}\lambda _{1}^{k}&0&\cdots &0\\0&\lambda _{2}^{k}&\cdots &0\\\vdots &\vdots &\ddots &\vdots \\0&0&\cdots &\lambda _{n}^{k}\end{pmatrix}}}$

and the evaluation of the Maclaurin series for functions of ${\displaystyle A}$ is greatly simplified.[59] For example, to obtain any power k of ${\displaystyle A}$, we need only compute ${\displaystyle D^{k}}$, premultiply ${\displaystyle D^{k}}$ by ${\displaystyle M}$, and postmultiply the result by ${\displaystyle M^{-1}}$.[60]

Using generalized eigenvectors, we can obtain the Jordan normal form for ${\displaystyle A}$ and these results can be generalized to a straightforward method for computing functions of nondiagonalizable matrices.[61] (See Matrix function#Jordan decomposition.)

Consider the problem of solving the system of linear ordinary differential equations

${\displaystyle \mathbf {x} '=A\mathbf {x} ,}$

(5)

where

${\displaystyle \mathbf {x} ={\begin{pmatrix}x_{1}(t)\\x_{2}(t)\\\vdots \\x_{n}(t)\end{pmatrix}},\quad \mathbf {x} '={\begin{pmatrix}x_{1}'(t)\\x_{2}'(t)\\\vdots \\x_{n}'(t)\end{pmatrix}},}$      and      ${\displaystyle A=(a_{ij}).}$

If the matrix ${\displaystyle A}$ is a diagonal matrix so that ${\displaystyle a_{ij}=0}$ for ${\displaystyle i\neq j}$, then the system (5) reduces to a system of n equations which take the form

${\displaystyle x_{1}'=a_{11}x_{1}}$ ${\displaystyle x_{2}'=a_{22}x_{2}}$ ${\displaystyle \vdots }$ ${\displaystyle x_{n}'=a_{nn}x_{n}.}$

(6)

In this case, the general solution is given by

${\displaystyle x_{1}=k_{1}e^{a_{11}t}}$

${\displaystyle x_{2}=k_{2}e^{a_{22}t}}$

${\displaystyle \vdots }$

${\displaystyle x_{n}=k_{n}e^{a_{nn}t}.}$

In the general case, we try to diagonalize ${\displaystyle A}$ and reduce the system (5) to a system like (6) as follows. If ${\displaystyle A}$ is diagonalizable, we have ${\displaystyle D=M^{-1}AM}$, where ${\displaystyle M}$ is a modal matrix for ${\displaystyle A}$. Substituting ${\displaystyle A=MDM^{-1}}$, equation (5) takes the form ${\displaystyle M^{-1}\mathbf {x} '=D(M^{-1}\mathbf {x} )}$, or

${\displaystyle \mathbf {y} '=D\mathbf {y} ,}$

(7)

where

${\displaystyle \mathbf {x} =M\mathbf {y} .}$

(8)

The solution of (7) is

${\displaystyle y_{1}=k_{1}e^{\lambda _{1}t}}$

${\displaystyle y_{2}=k_{2}e^{\lambda _{2}t}}$

${\displaystyle \vdots }$

${\displaystyle y_{n}=k_{n}e^{\lambda _{n}t}.}$

The solution ${\displaystyle \mathbf {x} }$ of (5) is then obtained using the relation (8).[62]

On the other hand, if ${\displaystyle A}$ is not diagonalizable, we choose ${\displaystyle M}$ to be a generalized modal matrix for ${\displaystyle A}$, such that ${\displaystyle J=M^{-1}AM}$ is the Jordan normal form of ${\displaystyle A}$. The system ${\displaystyle \mathbf {y} '=J\mathbf {y} }$ has the form

${\displaystyle {\begin{aligned}y_{1}'&=\lambda _{1}y_{1}+\epsilon _{1}y_{2}\\&\vdots \\y_{n-1}'&=\lambda _{n-1}y_{n-1}+\epsilon _{n-1}y_{n}\\y_{n}'&=\lambda _{n}y_{n},\end{aligned}}}$

(9)

where the ${\displaystyle \lambda _{i}}$ are the eigenvalues from the main diagonal of ${\displaystyle J}$ and the ${\displaystyle \epsilon _{i}}$ are the ones and zeros from the superdiagonal of ${\displaystyle J}$. The system (9) is often more easily solved than (5). We may solve the last equation in (9) for ${\displaystyle y_{n}}$, obtaining ${\displaystyle y_{n}=k_{n}e^{\lambda _{n}t}}$. We then substitute this solution for ${\displaystyle y_{n}}$ into the next to last equation in (9) and solve for ${\displaystyle y_{n-1}}$. Continuing this procedure, we work through (9) from the last equation to the first, solving the entire system for ${\displaystyle \mathbf {y} }$. The solution ${\displaystyle \mathbf {x} }$ is then obtained using the relation (8).[63]