Eigenvalue, Eigenvector, and Matrix Diagonalization

Eigenvalue, Eigenvector, and Matrix Diagonalization

Motivation:

Given an n n matrix A, how do you find the inverse of the matrix if it exists? In other words, you are trying to find a matrix Q such that

AQ=I_n

We can think in the following way. First, the matrix Q can be written in the form by listing all its columns as follows:

So, AQ= I_n means

A =

Ac₁ = , Ac₂= , …, Ac_n=

In other words, to find the right inverse matrix of A, we need to solve the problem Ax=d by setting d to different column vector of I_n :

x= , d=

The problem is to solve x by setting d to different matrix.

A =

When = , the solution to the problem is c₁ ;

…

= , the solution to the problem is c_n .

Of course, this problem can be solved by using Gauss Elimination. We can do the Gauss Elimination at the same time for all problems by aligning A and I_n together:

In the meanwhile, when one elementary operation is applied on A, we also apply it on I_n . If our target to perform a series of elementary row operations to change A to and identity matrix, then the I_n on the right will turn out to be the right inverse matrix of A. So, Q can be found by this approach.

After doing so many times of Gauss Elimination on solving different system equations, it seems that transforming a matrix A into a diagonal form that the elements in that matrix are all 0 if the elements are not in the diagonal line.

Definition ( Eigenvalue and Eigenvector )

For an n n matrix A, if there exists a non-zero n 1 matrix v such that

Av = v , where is a non-zero scalar

Then v is known as an eigenvector of the matrix A ( right eigenvector ) and is eigenvalue corresponding to the eigenvector v .

Is there any advantage to find the eigenvector and eigenvalue of the matrix? For an n n matrix A, if it has n eigenvectors v₁, v₂, … , v_n with corresponding eigenvalues , …, , we will have

A =

And recall that the elementary column operation introduced previously,

Thus,

A =

Let’s use the following notation:

Q= , D=

Then

AQ=QD

If the inverse of Q exits, then

A=QDQ^-1 and Q^-1AQ=D

So, this is a method to transform A into a diagonal matrix. Please notice that

D^-1=

If we set B=QD^-1Q, then

AB=(QDQ^-1)(QD^-1Q^-1)

=QDQ^-1QD^-1Q^-1

=QDD^-1Q^-1

=QQ^-1

=I_n

So, B is the inverse matrix of A. But using this approach to find the inverse matrix is not appropriate because you need to find the inverse of Q at first. However, let’s check the following:

A² = AA = (QDQ^-1)(QDQ^-1) = QDQQDQ^-1 = QD²Q^-1

A³=A²A = QD³Q^-1

A^k= QD^kQ^-1

And D is a diagonal matrix. To find D^k, it can be easily done by manipulating the diagonal elements of D:

D^k =

So, this method does have some advantages to find A^k as long as we know how to find eigenvalues and eigenvectors of it. We summarize the result in the following theorem.

Theorem ( Matrix Diagonalization):

A is an n n matrix. If A has distinct eigenvectors v₁, v₂, …, v_n with corresponding eigenvalues , …, , then we have

A=QDQ^-1

where

Q= , D=

With this result, we have

A^-1 = QD^-1Q^-1

A^k = QD^kQ^-1

Now, we might wonder how to find the eigenvectors and eigenvalues of a matrix. However, there are several questions associated with it:

1. For an n n matrix A, what is the condition that it has n eigenvectors ?

2. If the number of eigenvectors of A is less than n, is there any other way that we can do with A such that it is something similar to diagonal matrix?

3. Can we find other simpler result for this diagonalization if A has some other nice properties? For example, A is symmetric .

Those questions will be explored in some college courses like “Linear Algebra”. And because of those questions, this area is with a lot of things to explore. We will start with how to find the eigenvectors and eigenvalues.

If Ax= x, we can have

Ax= I_n x

Ax - I_nx = 0

(A- I_n) x = 0

where x=

But x is a non-zero vector, that means the columns of (A- I_n) are linearly dependent. In more advanced math, it will be introduced that the columns of a matrix is linearly dependent if and only if its determinant is 0. In other words,

det(A- I_n) = 0

With this, an equation of is obtained. If is solved, we can find the corresponding eigenvector by properly using Gauss Elimination. At this moment, we just focus on 2 2 matrix.

For 2 2 matrix, it is easy to determine if its rows or columns are linearly dependent. If they are linearly dependent, one can by multiplied by a scalar to get the other.

Example: Find the eigenvalue(s) and eigenvector(s) of the following matrix

Sol:

We consider the matrix (A- I) :

If you have no idea about determinant, you can think in this way: the two columns are linearly independent, it means one can be represented by the other by multiplying a scalar. In other words,

( same as the result from using determinant )

, =6, -3

When =6:

So, we can choose as the eigenvector.

Actually, multiplying this vector by any non-zero scalar can also be used for the eigenvector corresponding to this eigenvalue.

When =-3:

We can choose as the corresponding eigenvector.

Thus, we can have

Q= , D=

Q^-1 =

and A=QDQ^-1 :

A= =

A^k =