MATH347DS

MATH347DS L05: The FTLA

Overview

FTLA-associated definitions and results
Main problems of linear algebra
Homework 1 review - see sol01.tm

Preparing the FTLS

Set $S$ partition = collection $P$ of disjoint subsets covering the entire set
$P = {S_{i} | S_{i} \subset P, S_{i} \neq \emptyset .}, \forall x \in S, \exists! S_{i} : x \in S_{i}$
Partitioning of vector spaces: similar allow $𝟎$ as a common element
$𝒇 : ℝ^{n} \to ℝ^{m}$ , $𝒇 (a 𝒙 + b 𝒚) = a 𝒇 (𝒙) + b 𝒇 (𝒚)$ ,
$𝑨 = [\begin{array}{cccc} 𝒇 (𝒆_{1}) & 𝒇 (𝒆_{2}) & \dots & 𝒇 (𝒆_{n}) \end{array}] = [\begin{array}{cccc} 𝒂_{1} & 𝒂_{2} & \dots & 𝒂_{n} \end{array}]$
Relationships of subspaces associated with $𝑨$
$\begin{array}{cccc} C (𝑨) \leq ℝ^{m}, & N (𝑨^{T}) \leq ℝ^{m}, & C (𝑨) ⊥ N (𝑨^{T}) & C (𝑨) + N (𝑨^{T}) \leq ℝ^{m} \end{array} .$

When is a vector space sum a direct sum?

Lemma 1. Let $𝒰, 𝒱$ , be subspaces of vector space $𝒲$ . Then $𝒲 = 𝒰 \oplus 𝒱$ if and only if

$𝒲 = 𝒰 + 𝒱$ , and

$𝒰 \cap 𝒱 = {𝟎}$ .

Proof. $𝒲 = 𝒰 \oplus 𝒱 \Rightarrow 𝒲 = 𝒰 + 𝒱$ by definition of direct sum, sum of vector subspaces. To prove that $𝒲 = 𝒰 \oplus 𝒱 \Rightarrow 𝒰 \cap 𝒱 = {𝟎}$ , consider $𝒘 \in 𝒰 \cap 𝒱$ . Since $𝒘 \in 𝒰$ and $𝒘 \in 𝒱$ write

$\begin{array}{llll} 𝒘 = 𝒘 + 𝟎 & (𝒘 \in 𝒰, 𝟎 \in 𝒱), & 𝒘 = 𝟎 + 𝒘 & (𝟎 \in 𝒰, 𝒘 \in 𝒱), \end{array}$

and since expression $𝒘 = 𝒖 + 𝒗$ is unique, it results that $𝒘 = 𝟎$ . Now assume (i),(ii) and establish an unique decomposition. Assume there might be two decompositions of $𝒘 \in 𝒲$ , $𝒘 = 𝒖_{1} + 𝒗_{1}$ , $𝒘 = 𝒖_{2} + 𝒗_{2}$ , with $𝒖_{1}, 𝒖_{2} \in 𝒰$ , $𝒗_{1}, 𝒗_{2} \in 𝒱 .$ Obtain $𝒖_{1} + 𝒗_{1} = 𝒖_{2} + 𝒗_{2}$ , or $𝒙 = 𝒖_{1} - 𝒖_{2} = 𝒗_{2} - 𝒗_{1}$ . Since $𝒙 \in 𝒰$ and $𝒙 \in 𝒱$ it results that $𝒙 = 𝟎$ , and $𝒖_{1} = 𝒖_{2}$ , $𝒗_{1} = 𝒗_{2}$ , i.e., the decomposition is unique. $□$

The FTLA

Theorem. Given the linear mapping associated with matrix $𝑨 \in ℝ^{m \times n}$ we have:

$C (𝑨) \oplus N (𝑨^{T}) = ℝ^{m}$ , the direct sum of the column space and left null space is the codomain of the mapping

$C (𝑨^{T}) \oplus N (𝑨) = ℝ^{n}$ , the direct sum of the row space and null space is the domain of the mapping

$C (𝑨) ⊥ N (𝑨^{T})$ and $C (𝑨) \cap N (𝑨^{T}) = {𝟎}$ , the column space is orthogonal to the left null space, and they are orthogonal complements of one another,
$\begin{array}{ll} C (𝑨) = N {(𝑨^{T})}^{⊥}, & N (𝑨^{T}) = C {(𝑨)}^{⊥} \end{array} .$

$C (𝑨^{T}) ⊥ N (𝑨)$ and $C (𝑨^{T}) \cap N (𝑨) = {𝟎}$ , the row space is orthogonal to the null space, and they are orthogonal complements of one another,
$\begin{array}{ll} C (𝑨^{T}) = N {(𝑨)}^{⊥}, & N (𝑨) = C {(𝑨^{T})}^{⊥} \end{array} .$

FTLA: a graphical representation

Figure 1. Graphical representation of the Fundamental Theorem of Linear Algebra, Gil Strang, Amer. Math. Monthly 100, 848-855, 1993.

FTLA: proof

FTLA proof highlights the properties of a norm, especially $|| 𝒖 || = 0 \Rightarrow 𝒖 = 𝟎$ .

$C (𝑨) ⊥ N (𝑨^{T})$ (column space is orthogonal to left null space).

Proof. Consider arbitrary $𝒖 \in C (𝑨), 𝒗 \in N (𝑨^{T})$ . By definition of $C (𝑨)$ , $\exists 𝒙 \in ℝ^{n}$ such that $𝒖 = 𝑨 𝒙$ , and by definition of $N (𝑨^{T})$ , $𝑨^{T} 𝒗 = 𝟎$ . Compute $𝒖^{T} 𝒗 = {(𝑨 𝒙)}^{T} 𝒗 = 𝒙^{T} 𝑨^{T} 𝒗 = 𝒙^{T} (𝑨^{T} 𝒗) = 𝒙^{T} 𝟎 = 0$ , hence $𝒖 ⊥ 𝒗$ for arbitrary $𝒖, 𝒗$ , and $C (𝑨) ⊥ N (𝑨^{T})$ . $□$
$C (𝑨) \cap N (𝑨^{T}) = {𝟎}$ ( $𝟎$ is the only vector both in $C (𝑨)$ and $N (𝑨^{T})$ ).

Proof. (By contradiction, reductio ad absurdum). Assume there might be $𝒃 \in C (𝑨)$ and $b \in N (𝑨^{T})$ and $𝒃 \neq 𝟎$ . Since $𝒃 \in C (𝑨)$ , $\exists 𝒙 \in ℝ^{n}$ such that $𝒃 = 𝑨 𝒙$ . Since $𝒃 \in N (𝑨^{T})$ , $𝑨^{T} 𝒃 = 𝑨^{T} (𝑨 𝒙) = 𝟎$ . Note that $𝒙 \neq 0$ since $𝒙 = 0 \Rightarrow 𝒃 = 0$ , contradicting assumptions. Multiply equality $𝑨^{T} 𝑨 𝒙 = 𝟎$ on left by $𝒙^{T}$ ,
$𝒙^{T} 𝑨^{T} 𝑨 𝒙 = 𝟎 \Rightarrow {(𝑨 𝒙)}^{T} (𝑨 𝒙) = 𝒃^{T} 𝒃 = {|| 𝒃 ||}^{2} = 0,$
thereby obtaining $𝒃 = 0$ , using norm property 3. Contradiction.

$□$

FTLA: proof

established that $C (𝑨), N (𝑨^{T})$ are orthogonal complements
$C (𝑨) = N {(𝑨^{T})}^{⊥}, N (𝑨^{T}) = C {(𝑨)}^{⊥} .$
By Lemma 2 it results that $C (𝑨) \oplus N (𝑨^{T}) = ℝ^{m}$ .
Second part of FTLA: as above for $𝑩 = 𝑨^{T}$

FTLA: useful concepts and results

FTLA in a mathematical nutshell: matrix $𝑨 \in ℝ^{m \times n}$ represents linear mapping $f : ℝ^{n} \to ℝ^{m}$
$\begin{array}{lll} C (𝑨) \oplus N (𝑨^{T}) = ℝ^{m} & C (𝑨) ⊥ N (𝑨^{T}) & C (𝑨) \cap N (𝑨^{T}) = {𝟎} \\ C (𝑨^{T}) \oplus N (𝑨) = ℝ^{n} & C (𝑨^{T}) ⊥ N (𝑨) & C (𝑨^{T}) \cap N (𝑨) = {𝟎} \end{array}$

Definition. The rank of matrix $𝑨 \in ℝ^{m \times n}$ is the dimension of the column space, $r = \dim$ $C (𝑨)$

Definition. The nullity of $𝑨 \in ℝ^{m \times n}$ is the dimension of the null space, $z = \dim$ $C (𝑨)$

Proposition. The dimension of the column space equals the dimension of the row space.

Corollary. The system $𝑨 𝒙 = 𝒃$ , $𝑨 \in ℝ^{m \times n}$ , $𝒙 \in ℝ^{n}$ , $𝒃 \in ℝ^{m}$ has a solution if $𝒃 \in C (𝑨)$ . The solution is unique if $𝑵 (𝑨) = {𝟎}$ .

Main problems of linear algebra

FTLA asserts that the framework of linear algebra is complete:
$C (𝑨^{T}) \oplus N (𝑨) = ℝ^{n}, C (𝑨) \oplus N (𝑨^{T}) = ℝ^{m}$
What types of questions can be posed within this framework?
1. Least squares problem. Given $𝒃 \in ℝ^{m}$ in the $𝑰$ basis, find the linear combination of the vectors $𝒂_{1}, \dots, 𝒂_{n} \in ℝ^{m}$ that is “as close as possible' to $𝒃$ .
  ${min}_{𝒙 \in ℝ^{n}} {|| 𝒃 - 𝑨 𝒙 ||}_{2}$
2. Solving a linear system. Given $𝒃 \in ℝ^{m}$ in the $𝑰$ basis, what are the coordinates in another vector set $𝑨 = [\begin{array}{lll} 𝒂_{1} & \dots & 𝒂_{n} \end{array}] \in ℝ^{m \times n}$ (which might not be a basis for $ℝ^{m}$ )
  $𝑨 𝒙 = 𝒃 = 𝑰 𝒃$
  Very often the matrix $𝑨$ is square $m = n$ .
3. Eigenproblem. Given a square matrix $𝑨 \in ℂ^{m \times m}$ are there linear combinations that leave the direction of a matrix-vector product unchanged?
  $Given 𝑨 \in ℂ^{m \times m}, find 𝒙 \in ℂ^{m}, λ \in ℂ such that 𝑨 𝒙 = λ 𝒙 .$