MATH347

MATH347 L11: Vector subspaces of a matrix

New concepts:
- Span of a set of vectors, matrix column space
- Matrix null space
- Matrix row space
- Matrix left null space
- Vector space basis
- Vector space sums

Vector span, matrix column space

Definition. The span of vectors $𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n} \in 𝒱,$ is the set of vectors reachable by linear combination

$span {𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n}} = {𝒃 \in 𝒱 | \exists x_{1}, \dots, x_{n} \in 𝒮 such that 𝒃 = x_{1} 𝒂_{1} + \dots x_{n} 𝒂_{n}} .$

The notation used for set on the right hand side is read: “those vectors $𝒃$ in $𝒱$ with the property that there exist $n$ scalars $x_{1}, \dots, x_{n}$ to obtain $𝒃$ by linear combination of $𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n}$ .

A linear combination is conveniently expressed as a matrix-vector product leading to a different formulation of the same concept

Definition. The column space (or range) of matrix $𝑨 \in ℝ^{m \times n}$ is the set of vectors reachable by linear combination of the matrix column vectors

$C (𝑨) = range (𝑨) = {𝒃 \in ℝ^{m} | \exists 𝒙 \in ℝ^{n} such that 𝒃 = 𝑨 𝒙} \subseteq ℝ^{m}$

Null space

Introduce a characterization of the column vectors of a matrix related to linear dependence

Definition. The null space of a matrix $𝑨 \in ℝ^{m \times n}$ is the set

$N (𝑨) = null (𝑨) = {𝒙 \in ℝ^{n} | 𝑨 𝒙 = 𝟎} \subseteq ℝ^{n}$

If $null (𝑨) = {𝟎}$ then the column vectors of $𝑨$ are linearly independent, since the only way to satisfy (?) is by the trivial solution $𝒙 = 𝟎$

For example $𝑨 = [\begin{array}{lll} 𝒂_{1} & 𝒂_{2} & 𝒂_{3} \end{array}]$ below, $c (𝒂_{1} + 𝒂_{2} - 𝒂_{3}) = 𝟎$ for any scalar $c$ , hence

𝑨 = [\begin{array}{lll} 1 & 2 & 3 \\ 0 & 1 & 1 \\ 1 & 2 & 3 \end{array}] \Rightarrow C (𝑨) = span {𝒂_{1}, 𝒂_{2}}, N (𝑨) = span {(\begin{array}{c} 1 \\ 1 \\ - 1 \end{array})}

.

Column, null spaces as subspaces of codomain, domain of

𝑨

Recall definitions of column space, null space of $𝑨 \in ℝ^{m \times n}$

\begin{array}{rcl} C (𝑨) & = & {𝒃 \in ℝ^{m} | \exists 𝒙 \in ℝ^{n} such that 𝒃 = 𝑨 𝒙} \subseteq ℝ^{m} \\ N (𝑨) & = & {𝒙 \in ℝ^{n} | 𝑨 𝒙 = 𝟎} \subseteq ℝ^{n} \end{array}

Note that $C (𝑨) \subseteq ℝ^{m}$ , $N (𝑨) \subseteq ℝ^{n}$ means that $C (𝑨), N (𝑨)$ are subsets of $ℝ^{m}, ℝ^{n}$ respectively. In fact, we can make a stronger statement, that they are vector subspaces

C (𝑨) \leq ℝ^{m}, N (𝑨) \leq ℝ^{n}

Proof. Let $𝒖, 𝒗 \in C (𝑨)$ , $α, β \in 𝒮$ . By defintion of $C (𝑨)$ there exist $𝒙, 𝒚 \in ℝ^{n}$ such that $𝒖 = 𝑨 𝒙$ and $𝒗 = 𝑨 𝒚$ . Using vector space properties

α 𝒖 + β 𝒗 = α 𝑨 𝒙 + β 𝑨 𝒚 = 𝑨 (α 𝒙 + β 𝒚),

hence $α 𝒖 + β 𝒗 \in C (𝑨)$ (it is obtained as the image through the linear mapping $𝑨$ of $α 𝒙 + β 𝒚$ )

Orthogonality implies linear independence

Recall that if $𝒖^{T} 𝒗 = 0$ , with $𝒖, 𝒗 \in ℝ^{m}$ then $𝒖 ⊥ 𝒗$ (orthogonal)

Proposition. If $𝒖_{1}, 𝒖_{2}, \dots, 𝒖_{n} \in ℝ_{}^{m}$ are non-zero ( $𝒖_{i} \neq 𝟎$ ) and pairwise orthogonal, $𝒖_{i}^{T} 𝒖_{j} = 0$ for $i \neq j$ then they form a linearly independent set of vectors.

Proof. Consider the equation equating the linear combination $c_{1} 𝒖_{1} + \dots + c_{n} 𝒖_{n}$ to the zero vector

$c_{1} 𝒖_{1} + \dots + c_{n} 𝒖_{n} = 𝟎$ (1)

Multiply on the left by $𝒖_{i}^{T}$ and use orthogonality to obtain $c_{i} = 0$ for $i = 1, \dots, n$ . The only solution to (1) is $c_{1} = c_{2} = \dots = c_{n} = 0$ , hence ${𝒖_{1}, 𝒖_{2}, \dots, 𝒖_{n}}$ is a linearly independent set.

Row space, left null space

For $𝑨 \in ℝ^{m \times n}$ , seen as a linear mapping $𝑨 : ℝ^{n} \mapsto ℝ^{m}$ , that given input vector $𝒙 \in ℝ^{n}$ returns output vector $𝒃 \in ℝ^{m}, 𝒃 = 𝑨 𝒙$ , we have defined the vector space of possible outputs, the column space of $𝑨$
$C (𝑨) = {𝒃 \in ℝ^{m} | \exists 𝒙 \in ℝ^{n} such that 𝒃 = 𝑨 𝒙} \subseteq ℝ^{m}$
The transpose $𝑨^{T} \in ℝ^{n \times m}$ can also be seen as a linear mapping. Given some input vector $𝒚 \in ℝ^{m}$ the mapping returns the output vector $𝒄 \in ℝ^{n}$ , $𝒄 = 𝑨^{T} 𝒚$ . The set of possible outputs is the column space of $𝑨^{T}$ . Since columns of $𝑨^{T}$ are rows of $𝑨$ , we can define the row space of $𝑨$ as
$R (𝑨) = C (𝑨^{T}) = {𝒄 \in ℝ^{n} | \exists 𝒚 \in ℝ^{m} such that 𝒄 = 𝑨^{T} 𝒚} \subseteq ℝ^{n}$
Left null space, $N (𝑨^{T}) = {𝒚 \in ℝ^{m} | 𝑨^{T} 𝒚 = 0} \subseteq ℝ^{m}$ , the part of $ℝ^{m}$ not reachable by linear combination of columns of $𝑨$

Basis of a vector space

Definition. A set of vectors $𝒖_{1}, \dots, 𝒖_{n} \in 𝒱$ is a basis for vector space $𝒱$ if:

$𝒖_{1}, \dots, 𝒖_{n}$ are linearly independent;

$span {𝒖_{1}, \dots, 𝒖_{n}} = 𝒱$ .

Definition. The number of vectors $𝒖_{1}, \dots, 𝒖_{n} \in 𝒱$ within a basis is the dimension of the vector space $𝒱$ .

Direct sum, intersection of vector spaces

Definition. Given two vector subspaces $(𝒰, 𝒮, +)$ , $(𝒱, 𝒮, +)$ of the space $(𝒲, 𝒮, +)$ , the sum is the set $𝒰 + 𝒱 = {𝒖 + 𝒗 | 𝒖 \in 𝒰, 𝒗 \in 𝒱} .$

Definition. Given two vector subspaces $(𝒰, 𝒮, +)$ , $(𝒱, 𝒮, +)$ of the space $(𝒲, 𝒮, +)$ , the direct sum is the set $𝒰 \oplus 𝒱 = {𝒖 + 𝒗 | \exists! 𝒖 \in 𝒰, \exists! 𝒗 \in 𝒱} .$ (unique decomposition)

Definition. Given two vector subspaces $(𝒰, 𝒮, +)$ , $(𝒱, 𝒮, +)$ of the space $(𝒲, 𝒮, +)$ , the intersection is the set

$𝒰 \cap 𝒱 = {𝒙 | 𝒙 \in 𝒰, 𝒙 \in 𝒱} .$

Definition. Two vector subspaces $(𝒰, 𝒮, +)$ , $(𝒱, 𝒮, +)$ of the space $(𝒲, 𝒮, +)$ are orthogonal subspaces, denoted $𝒰 ⊥ 𝒱$ if $𝒖^{T} 𝒗 = 0$ for any $𝒖 \in 𝒰, 𝒗 \in 𝒱$ .

Definition. Two vector subspaces $(𝒰, 𝒮, +)$ , $(𝒱, 𝒮, +)$ of the space $(𝒲, 𝒮, +)$ are orthogonal complements, denoted $𝒰 = 𝒱^{⊥}$ , $𝒱 = 𝒰^{⊥}$ if they are orthogonal subspaces and $𝒰 \cap 𝒱 = {𝟎}$ , i.e., the null vector is the only common element of both subspaces.