# Theorems and definitions in linear algebra

This article collects the main **theorems and definitions in linear algebra**.

## Vector spaces

* A vector space( or linear space) *

**V**

*over a number field²*x

**F**consists of a set on which two operations (called**addition**and**scalar multiplication**, respectively) are defined so, that for each pair of elements*,*y

*, in*

**V**

*there is a unique element*x

*+*y

*in*

**V**

*, and for each element a in F and each element x in*

**V**

*there is a unique element ax in*

**V**

*, such that the following conditions hold.*

- (VS 1)
*For all in***V***, (***commutativity of addition**). - (VS 2)
*For all in***V***, (***associativity of addition**). - (VS 3)
*There exists an element in***V***denoted by such that for each in***V***.* - (VS 4)
*For each element in***V***there exists an element in***V***such that .* - (VS 5)
*For each element in***V***, .* - (VS 6)
*For each pair of element in***F**and each element in**V***, .* - (VS 7)
*For each element in***F**and each pair of elements in**V***, .* - (VS 8)
*For each pair of elements in***F**and each pair of elements in**V***, .*

### Subspaces

* A subspace *

**W**

*of a*

**vector space****V**

*over a field F is a*

**subset**of**V**

*which also has the properties that*

**W**

*is closed under scalar addition and multiplication. That is, For all*x

*,*y

*in*

**W**

*,*x

*and*y

*are in*

**V**

*and for any*c

*in F, is in*

**W**

*.*

### Linear combinations

### Systems of linear equations

### Linear dependence

### Linear independence

### Bases

### Dimension

## Linear transformations and matrices

Change of coordinate matrix

Clique

Coordinate vector relative to a basis

Dimension theorem

Dominance relation

Identity matrix

Identity transformation

Incidence matrix

Inverse of a linear transformation

Inverse of a matrix

Invertible linear transformation

Isomorphic vector spaces

Isomorphism

Kronecker delta

Left-multiplication transformation

Linear operator

Linear transformation

Matrix representing a linear transformation

Nullity of a linear transformation

Null space

Ordered basis

Product of matrices

Projection on a subspace

Projection on the x-axis

Range

Rank of a linear transformation

Reflection about the x-axis

Rotation

Similar matrices

Standard ordered basis for

Standard representation of a vector space with respect to a basis

Zero transformation

P.S. coefficient of the differential equation, differentiability of complex function,vector space of functionsdifferential operator, auxiliary polynomialTemplate:Disambiguation needed, to the power of a complex number, exponential function.

### *N*(*T*) and *R*(*T*) are subspaces

Let *V* and *W* be vector spaces and *I*: *V* → *W* be linear. Then *N*(*T*) and *R*(*T*) are subspaces of *V* and *W*, respectively.

### R(T)= span of T(basis in V)

Let V and W be vector spaces, and let T: V→W be linear. If is a basis for V, then

### Dimension theorem

Let V and W be vector spaces, and let T: V → W be linear. If V is finite-dimensional, then

### one-to-one ⇔ N(T) = {0}

Let V and W be vector spaces, and let T: V→W be linear. Then T is one-to-one if and only if N(T)={0}.

### one-to-one ⇔ onto ⇔ rank(*T*) = dim(*V*)

Let V and W be vector spaces of equal (finite) dimension, and let *T*:*V* → *W* be linear. Then the following are equivalent.

- (a)
*T*is one-to-one. - (b)
*T*is onto. - (c) rank(
*T*) = dim(*V*).

### ∀ exactly one T (basis),

Let V and W be vector space over F, and suppose that is a basis for V. For in W, there exists exactly one linear transformation T: V→W such that for

**Corollary.**
Let V and W be vector spaces, and suppose that V has a finite basis . If U, T: V→W are linear and for then U=T.

### T is vector space

Let V and W be vector spaces over a field F, and let T, U: V→W be linear.

- (a) For all ∈
*F*, is linear. - (b) Using the operations of addition and scalar multiplication in the preceding definition, the collection of all linear transformations form V to W is a vector space over F.

### linearity of matrix representation of linear transformation

Let V and W be finite-dimensional vector spaces with ordered bases β and γ, respectively, and let T, U: V→W be linear transformations. Then

### composition law of linear operators

Let V,w, and Z be vector spaces over the same field f, and let T:V→W and U:W→Z be linear. then UT:V→Z is linear.

### law of linear operator

Let v be a vector space. Let T, U_{1}, U_{2} ∈ (V). Then

(a) T(U_{1}+U_{2})=TU_{1}+TU_{2} and (U_{1}+U_{2})T=U_{1}T+U_{2}T

(b) T(U_{1}U_{2})=(TU_{1})U_{2}

(c) TI=IT=T

(d) (U_{1}U_{2})=(U_{1})U_{2}=U_{1}(U_{2}) for all scalars .

### [UT]_{α}^{γ}=[U]_{β}^{γ}[T]_{α}^{β}

Let V, W and Z be finite-dimensional vector spaces with ordered bases α β γ, respectively. Let T: V⇐W and U: W→Z be linear transformations. Then

**Corollary**. Let V be a finite-dimensional vector space with an ordered basis β. Let T,U∈(V). Then [UT]_{β}=[U]_{β}[T]_{β}.

### law of matrix

Let A be an m×n matrix, B and C be n×p matrices, and D and E be q×m matrices. Then

- (a) A(B+C)=AB+AC and (D+E)A=DA+EA.
- (b) (AB)=(A)B=A(B) for any scalar .
- (c) I
_{m}A=AI_{m}. - (d) If V is an n-dimensional vector space with an ordered basis β, then [I
_{v}]_{β}=I_{n}.

**Corollary.** Let A be an m×n matrix, B_{1},B_{2},...,B_{k} be n×p matrices, C_{1},C_{1},...,C_{1} be q×m matrices, and be scalars. Then

and

### law of column multiplication

Let A be an m×n matrix and B be an n×p matrix. For each let and denote the jth columns of AB and B, respectively. Then

(a)

(b) , where is the jth standard vector of F^{p}.

### [T(u)]_{γ}=[T]_{β}^{γ}[u]_{β}

Let V and W be finite-dimensional vector spaces having ordered bases β and γ, respectively, and let T: V→W be linear. Then, for each u ∈ V, we have

### laws of L_{A}

Let A be an m×n matrix with entries from F. Then the left-multiplication transformation L_{A}: F^{n}→F^{m} is linear. Furthermore, if B is any other m×n matrix (with entries from F) and β and γ are the standard ordered bases for F^{n} and F^{m}, respectively, then we have the following properties.

(a) .

(b) L_{A}=L_{B} if and only if A=B.

(c) L_{A+B}=L_{A}+L_{B} and L_{$a$A}=L_{A} for all ∈F.

(d) If T:F^{n}→F^{m} is linear, then there exists a unique m×n matrix C such that T=L_{C}. In fact, .

(e) If W is an n×p matrix, then L_{AE}=L_{A}L_{E}.

(f ) If m=n, then .

### A(BC)=(AB)C

Let A,B, and C be matrices such that A(BC) is defined. Then A(BC)=(AB)C; that is, matrix multiplication is associative.

### T^{-1}is linear

Let V and W be vector spaces, and let T:V→W be linear and invertible. Then T^{−1}: W
→V is linear.

### [T^{-1}]_{γ}^{β}=([T]_{β}^{γ})^{-1}

Let V and W be finite-dimensional vector spaces with ordered bases β and γ, respectively. Let T:V→W be linear. Then T is invertible if and only if is invertible. Furthermore,

**Lemma.** Let T be an invertible linear transformation from V to W. Then V is finite-dimensional if and only if W is finite-dimensional. In this case, dim(V)=dim(W).

**Corollary 1.** Let V be a finite-dimensional vector space with an ordered basis β, and let T:V→V be linear. Then T is invertible if and only if [T]_{β} is invertible. Furthermore, [T^{−1}]_{β}=([T]_{β})^{−1}.

**Corollary 2.** Let A be an n×n matrix. Then A is invertible if and only if L_{A} is invertible. Furthermore, (L_{A})^{−1}=L_{A−1}.

### V is isomorphic to W ⇔ dim(V)=dim(W)

Let W and W be finite-dimensional vector spaces (over the same field). Then V is isomorphic to W if and only if dim(V)=dim(W).

**Corollary.** Let V be a vector space over F. Then V is isomorphic to F^{n} if and only if dim(V)=n.

### ??

Let V and W be finite-dimensional vector spaces over F of dimensions n and m, respectively, and let β and γ be ordered bases for V and W, respectively. Then the function : (V,W)→M_{m×n}(F), defined by for T∈(V,W), is an isomorphism.

**Corollary.** Let V and W be finite-dimensional vector spaces of dimension n and m, respectively. Then (V,W) is finite-dimensional of dimension mn.

### *Φ*_{β} is an isomorphism

_{β}

For any finite-dimensional vector space V with ordered basis β, *Φ _{β}* is an isomorphism.

### ??

Let β and β' be two ordered bases for a finite-dimensional vector space V, and let . Then

(a) is invertible.

(b) For any V, .

### [T]_{β'}=Q^{-1}[T]_{β}Q

Let T be a linear operator on a finite-dimensional vector space V,and let β and β' be two ordered bases for V. Suppose that Q is the change of coordinate matrix that changes β'-coordinates into β-coordinates. Then

**Corollary.** Let A∈M_{n×n}(*F*), and le t γ be an ordered basis for F^{n}. Then [L_{A}]_{γ}=Q^{−1}AQ, where Q is the n×n matrix whose jth column is the jth vector of γ.

### *p*(D)(x)=0 (*p*(D)∈C^{∞})⇒ x^{(k)}exists (k∈N)

Any solution to a homogeneous linear differential equation with constant coefficients has derivatives of all orders; that is, if is a solution to such an equation, then exists for every positive integer k.

### {solutions}= N(p(D))

The set of all solutions to a homogeneous linear differential equation with constant coefficients coincides with the null space of p(D), where p(t) is the auxiliary polynomial with the equation.

**Corollary**. The set of all solutions to s homogeneous linear differential equation with constant coefficients is a subspace of .

### derivative of exponential function

For any exponential function .

### {e^{-at}} is a basis of N(*p*(D+aI))

The solution space for the differential equation,

is of dimension 1 and has as a basis.

**Corollary.** For any complex number c, the null space of the differential operator D-cI has {} as a basis.

### is a solution

Let p(t) be the auxiliary polynomial for a homogeneous linear differential equation with constant coefficients. For any complex number c, if c is a zero of p(t), then to the differential equation.

### dim(N(*p*(D)))=n

For any differential operator p(D) of order n, the null space of p(D) is an n_dimensional subspace of C^{∞}.

**Lemma 1**. The differential operator D-cI: C^{∞} to C^{∞} is onto for any complex number c.

**Lemma 2** Let V be a vector space, and suppose that T and U are linear operators on V such that U is onto and the null spaces of T and U are finite-dimensional, Then the null space of TU is finite-dimensional, and

- dim(N(TU))=dim(N(U))+dim(N(U)).

**Corollary**. The solution space of any nth-order homogeneous linear differential equation with constant coefficients is an n-dimensional subspace of C^{∞}.

### e^{cit} is linearly independent with each other (c_{i} are distinct)

Given n distinct complex numbers , the set of exponential functions is linearly independent.

**Corollary**. For any nth-order homogeneous linear differential equation with constant coefficients, if the auxiliary polynomial has n distinct zeros , then is a basis for the solution space of the differential equation.

**Lemma**. For a given complex number c and positive integer n, suppose that (t-c)^n is athe auxiliary polynomial of a homogeneous linear differential equation with constant coefficients. Then the set

is a basis for the solution space of the equation.

### general solution of homogeneous linear differential equation

Given a homogeneous linear differential equation with constant coefficients and auxiliary polynomial

where are positive integers and are distinct complex numbers, the following set is a basis for the solution space of the equation:

## Elementary matrix operations and systems of linear equations

### Elementary matrix operations

1. Matrix rows can be interchanged 2. Matrix rows can be multiplied by a non-zero real number(i.e. number can be either positive or negative) 3. Any row can be changed by adding or subtracting corresponding row elements with another row

### Elementary matrix

### Rank of a matrix

The rank of a matrix A is the number of pivot columns after the reduced row echelon form of A.

### Matrix inverses

### System of linear equations

## Determinants

*If*

*is a* 2×2* matrix with entries form a field F, then we define the determinant of A, denoted *det(

*A*)

*or |A|, to be the scalar .*

＊Theorem 1: linear function for a single row.

＊Theorem 2: nonzero determinant ⇔ invertible matrix

**Theorem 1:**
* The function *det: **M**_{2×2}(*F*)* → F is a linear function of each row of a *2×2* matrix when the other row is held fixed. That is, if and are in ***F**²* and is a scalar, then*

*and*

**Theorem 2**:
*Let A *M_{2×2}(*F*)*. Then thee deter minant of A is nonzero if and only if A is invertible. Moreover, if A is invertible, then*

## Diagonalization

Characteristic polynomial of a linear operator/matrix

### diagonalizable⇔basis of eigenvector

A linear operator T on a finite-dimensional vector space V is diagonalizable if and only if there exists an ordered basis β for V consisting of eigenvectors of T. Furthermore, if T is diagonalizable, is an ordered basis of eigenvectors of T, and *D* = [T]_{β} then D is a diagonal matrix and is the eigenvalue corresponding to for .

### eigenvalue⇔det(*A*-λ*I*n)=0

Let *A*∈M_{n×n}(*F*). Then a scalar λ is an eigenvalue of *A* if and only if det(*A*-λ*I*_{n})=0

### characteristic polynomial

Let A∈Mn×n(*F*).

(a) The characteristic polynomial of A is a polynomial of degree n with leading coefficient(-1)n.

(b) A has at most n distinct eigenvalues.

### υ to λ⇔υ∈N(T-λI)

Let T be a linear operator on a vector space V, and let λ be an eigenvalue of T.

A vector υ∈V is an eigenvector of T corresponding to λ if and only if υ≠0 and υ∈N(T-λI).

### vi to λi⇔vi is linearly independent

Let T be a linear operator on a vector space V, and let be distinct eigenvalues of T. If are eigenvectors of t such that corresponds to (), then {} is linearly independent.

### characteristic polynomial splits

The characteristic polynomial of any diagonalizable linear operator splits.

### 1 ≤ dim(E*λ*) ≤ *m*

Let T be alinear operator on a finite-dimensional vectorspace V, and let λ be an eigenvalue of T having multiplicity . Then .

### *S* = S_{1} ∪ *S*_{2} ∪ ...∪ *S*_{k} is linearly independent

Let T be a linear operator on a vector space V, and let be distinct eigenvalues of T. For each let be a finite linearly independent subset of the eigenspace . Then is a linearly independent subset of V.

### ⇔T is diagonalizable

Let T be a linear operator on a finite-dimensional vector space V that the characteristic polynomial of T splits. Let be the distinct eigenvalues of T. Then

(a) T is diagonalizable if and only if the multiplicity of is equal to for all .

(b) If T is diagonalizable and is an ordered basis for for each , then is an ordered for V consisting of eigenvectors of T.

**Test for diagonlization**

## Inner product spaces

Inner product, standard inner product on F^{n}, conjugate transpose, adjointTemplate:Disambiguation needed, Frobenius inner product, complex/real inner product space, norm, length, conjugate linear, orthogonal, perpendicular, orthogonal, unit vector, orthonormal, normalization.

### properties of linear product

Let V be an inner product space. Then for x,y,z\in V and c \in f, the following staements are true.

(a)

(b)

(c)

(d) if and only if

(e) If for all V, then .

### law of norm

Let V be an inner product space over F. Then for all x,y\in V and c\in F, the following statements are true.

(a) .

(b) if and only if . In any case, .

(c)(**Cauchy-Schwarz In equality**).

(d)(**Triangle Inequality**).

orthonormal basis, Gram–Schmidt process, Fourier coefficients, orthogonal complement, orthogonal projection

### span of orthogonal subset

Let V be an inner product space and be an orthogonal subset of V consisting of nonzero vectors. If ∈span(S), then

### Gram-Schmidt process

Let V be an inner product space and S= be a linearly independent subset of V. DefineS'=, where and

Then S' is an orhtogonal set of nonzero vectors such that span(S')=span(S).

### orthonormal basis

Let V be a nonzero finite-dimensional inner product space. Then V has an orthonormal basis β. Furthermore, if β = and x∈V, then

**Corollary.** Let V be a finite-dimensional inner product space with an orthonormal basis β =. Let T be a linear operator on V, and let A=[T]_{β}. Then for any and , .

### W^{⊥} by orthonormal basis

Let W be a finite-dimensional subspace of an inner product space V, and let ∈V. Then there exist unique vectors ∈W and ∈W^{⊥} such that . Furthermore, if is an orthornormal basis for W, then

S=\{v_1,v_2,\ldots,v_k\}
**Corollary.** In the notation of Theorem 6.6, the vector is the unique vector in W that is "closest" to ; thet is, for any ∈W, , and this inequality is an equality if and onlly if .

### properties of orthonormal set

Suppose that is an orthonormal set in an -dimensional inner product space V. Than

(a) S can be extended to an orthonormal basis for V.

(b) If W=span(S), then is an orhtonormal basis for W^{⊥}(using the preceding notation).

(c) If W is any subspace of V, then dim(V)=dim(W)+dim(W^{⊥}).

Least squares approximation, Minimal solutions to systems of linear equations

### linear functional representation inner product

Let V be a finite-dimensional inner product space over F, and let :V→F be a linear transformation. Then there exists a unique vector ∈ V such that for all ∈ V.

### definition of T*

Let V be a finite-dimensional inner product space, and let T be a linear operator on V. Then there exists a unique function T*:V→V such that for all ∈ V. Furthermore, T* is linear

### [T*]_{β}=[T]*_{β}

Let V be a finite-dimensional inner product space, and let β be an orthonormal basis for V. If T is a linear operator on V, then

### properties of T*

Let V be an inner product space, and let T and U be linear operators onV. Then

(a) (T+U)*=T*+U*;

(b) (T)*= T* for any c∈ F;

(c) (TU)*=U*T*;

(d) T**=T;

(e) I*=I.

**Corollary.** Let A and B be n×nmatrices. Then

(a) (*A*+*B*)*=*A**+*B**;

(b) (*A*)*= *A** for any ∈ F;

(c) (*AB*)*=*B***A**;

(d) *A***=*A*;

(e) *I**=*I*.

### Least squares approximation

Let *A* ∈ M_{m×n}(*F*) and ∈F^{m}. Then there exists ∈ F^{n} such that and for all x∈ F^{n
}

**Lemma 1.** let *A *∈ M_{m×n}(*F*), ∈F^{n}, and ∈F^{m}. Then

**Lemma 2.** Let *A *∈ M_{m×n}(*F*). Then rank(*A*A*)=rank(*A*).

**Corollary.**(of lemma 2) If *A* is an m×n matrix such that rank(*A*)=n, then *A*A* is invertible.

### Minimal solutions to systems of linear equations

Let *A *∈ M_{m×n}(*F*) and b∈ F^{m}. Suppose that is consistent. Then the following statements are true.

(a) There existes exactly one minimal solution of , and ∈R(L_{A*}).

(b) The vector is the only solution to that lies in R(L_{A*}); that is, if satisfies , then .

## Canonical forms

## References

- Linear Algebra 4th edition, by Stephen H. Friedberg Arnold J. Insel and Lawrence E. spence ISBN 7-04-016733-6
- Linear Algebra 3rd edition, by Serge Lang (UTM) ISBN 0-387-96412-6
- Linear Algebra and Its Applications 4th edition, by Gilbert Strang ISBN 0-03-010567-6