MATH347DS

MATH347DS L03: Matrix Vector Subspaces

Overview

Span of a vector set
Vector subspaces
Vector subspace composition
Vector subspace of a linear mapping and its associated matrix
- Column space
- Left null space
Geometric interpretation of subspaces of Euclidean spaces
Applications of concept of vector subspace

Recall: vector space definition

Formalize linear combinations by explicit definition of allowed operations (“algebra”)

Addition rules for	$\forall 𝒂, 𝒃, 𝒄 \in V$
$𝒂 + 𝒃 \in V$	Closure
$𝒂 + (𝒃 + 𝒄) = (𝒂 + 𝒃) + 𝒄$	Associativity
$𝒂 + 𝒃 = 𝒃 + 𝒂$	Commutativity
$𝟎 + 𝒂 = 𝒂$	Zero vector
$𝒂 + (- 𝒂_{}) = 𝟎$	Additive inverse
Scaling rules for	$\forall 𝒂, 𝒃 \in V$ , $\forall x, y \in S$
$x 𝒂 \in V$	Closure
$x (𝒂 + 𝒃) = x 𝒂 + x 𝒃$	Distributivity
$(x + y) 𝒂 = x 𝒂 + y 𝒂$	Distributivity
$x (y 𝒂) = (x y) 𝒂$	Composition
$1 𝒂 = 𝒂$	Scalar identity

Table 1. Vector space properties

Example: $V = ℝ^{m}, S = ℝ$

Span of a set of vectors

Definition. The span of vectors $𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n} \in 𝒱,$ is the set of vectors reachable by linear combination

$span {𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n}} = {𝒃 \in 𝒱 : \exists x_{1}, \dots, x_{n} \in 𝒮 such that 𝒃 = x_{1} 𝒂_{1} + \dots x_{n} 𝒂_{n}} .$

The notation used for set on the right hand side is read: “those vectors $𝒃$ in $𝒱$ with the property that there exist $n$ scalars $x_{1}, \dots, x_{n}$ to obtain $𝒃$ by linear combination of $𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n}$ .

Example 1. The $ℝ^{2}$ plane is described as

ℝ^{2} = {[\begin{array}{l} a \\ b \end{array}] : a, b \in ℝ} = span {[\begin{array}{l} 1 \\ 0 \end{array}], [\begin{array}{l} 0 \\ 1 \end{array}]} .

Example 2. Within $ℝ^{2}$ , the $x_{1}$ axis is described as

𝕏_{1} = {[\begin{array}{l} a \\ 0 \end{array}] : a \in ℝ} = span {[\begin{array}{l} 1 \\ 0 \end{array}]} .

Vector space composition

Definition. (Vector Subspace) . $𝒰 = (U, S, +, \cdot)$ with $U \neq \emptyset$ is a vector subspace of vector space $𝒱 = (V, S, +, \cdot)$ over the same field of scalars S if $U \subseteq V$ and $\forall a, b \in S$ , $\forall 𝒖, 𝒗 \in U$ , the linear combination $a 𝒖 + b 𝒗 \in U$ .

Definition. Given two vector subspaces $(U, S, +, \cdot)$ , $(V, S, +, \cdot)$ of the space $(W, S, +, \cdot)$ , the sum is the set $U + V = {𝒖 + 𝒗 : 𝒖 \in U, 𝒗 \in V} .$

Definition. Given two vector subspaces $(U, S, +, \cdot)$ , $(V, S, +, \cdot)$ of the space $(W, S, +, \cdot)$ , the direct sum is the set $U \oplus V = {𝒖 + 𝒗 : \exists! 𝒖 \in U, \exists! 𝒗 \in V} .$ (unique decomposition)

Definition. Given two vector subspaces $(U, S, +, \cdot)$ , $(V, S, +, \cdot)$ of the space $(W, S, +, \cdot)$ , the intersection is the set

$𝒰 \cap 𝒱 = {𝒙 : 𝒙 \in 𝒰, 𝒙 \in 𝒱} .$

Definition. Two vector subspaces $𝒰 = (U, S, +, \cdot)$ , $𝒱 = (V, S, +, \cdot)$ of the space $𝒲 = (W, S, U +, \cdot)$ are orthogonal subspaces, denoted $𝒰 ⊥ 𝒱$ if $𝒖^{T} 𝒗 = 0$ for any $𝒖 \in U, 𝒗 \in V$ .

Definition. Two vector subspaces $𝒰 = (U, S, +, \cdot)$ , $𝒱 = (V, S, +, \cdot)$ of the space $𝒲 = (W, S, U +, \cdot)$ are orthogonal complements, denoted $𝒰 = 𝒱^{⊥}$ , $𝒱 = 𝒰^{⊥}$ if they are orthogonal subspaces and $𝒰 \cap 𝒱 = {𝟎}$ , i.e., the null vector is the only common element of both subspaces.

Vector subspaces of a linear mapping

Recall that a matrix $𝑨$ can be associated to the linear mapping $𝒇 : ℝ^{n} \to ℝ^{m}$ through
$𝑨 = [\begin{array}{llll} 𝒂_{1} & 𝒂_{2} & \dots & 𝒂_{n} \end{array}] = [\begin{array}{llll} 𝒇 (𝒆_{1}) & 𝒇 (𝒆_{2}) & \dots & 𝒇 (𝒆_{n}) \end{array}], 𝒇 (𝒙) = 𝑨 𝒙$

Definition. The column space (or range) of matrix $𝑨 \in ℝ^{m \times n}$ is the set of vectors reachable by linear combination of the matrix column vectors

$C (𝑨) = range (𝑨) = {𝒃 \in ℝ^{m} : \exists 𝒙 \in ℝ^{n} such that 𝒃 = 𝑨 𝒙} \subseteq ℝ^{m} .$

Definition. The left null space of a matrix $𝑨 \in ℝ^{m \times n}$ is the set

$N (𝑨^{T}) = null (𝑨^{T}) = {𝒙 \in ℝ^{n} : 𝑨 𝒙 = 𝟎} \subseteq ℝ^{n} .$

Definition. The row space (or corange) of a matrix $𝑨 \in ℝ^{m \times n}$ is the set

$R (𝑨) = C (𝑨^{T}) = range (𝑨^{T}) = {𝒄 \in ℝ^{n} : \exists 𝒚 \in ℝ^{m} 𝒄 = 𝑨^{T} 𝒚} \subseteq ℝ^{n}$

Definition. The null space of a matrix $𝑨 \in ℝ^{m \times n}$ is the set

$N (𝑨) = null (𝑨) = {𝒙 \in ℝ^{n} : 𝑨 𝒙 = 𝟎} \subseteq ℝ^{n}$

Geometry of subspaces

The $ℝ^{2}$ plane
$ℝ^{2} = {[\begin{array}{l} x_{1} \\ x_{2} \end{array}] : x_{1}, x_{2} \in ℝ} .$
A line in the $ℝ^{2}$ plane
$L_{p, q} = {[\begin{array}{l} a p \\ a q \end{array}] : a \in ℝ} .$
The $x_{1}$ , $x_{2}$ axes
$L_{1, 0} = {[\begin{array}{l} a \\ 0 \end{array}] : a \in ℝ}, L_{0, 1} = {[\begin{array}{l} 0 \\ a \end{array}] : a \in ℝ},$
$ℝ^{3}$ , three-dimensional space
$ℝ^{2} = {[\begin{array}{l} x_{1} \\ x_{2} \\ x_{3} \end{array}] : x_{1}, x_{2}, x_{3} \in ℝ} .$
Lines in $ℝ^{3}$
$L_{p, q, r} = {[\begin{array}{l} a p \\ a q \\ a r \end{array}] : a \in ℝ}$
Planes in $ℝ^{3}$
$P_{𝒏} = {[\begin{array}{l} x_{1} \\ x_{2} \\ x_{3} \end{array}] : 𝒏^{T} 𝒙 = n_{1} x_{1} + n_{2} x_{2} + n_{3} x_{3} = 0, x_{1}, x_{2}, x_{3} \in ℝ}$

Spanning sets for column, null spaces

Define a function to give a spanning set for the column space

∴	function colspace(A,p=6) return round.(Matrix(qr(A).Q)[:,1:rank(A)],digits=p) end;

∴	short(x) = round(x,digits=6);

∴

short(pi)

$3.141593$

∴	colspace([1; 0; 0])

$[\begin{array}{c} 1.0 \\ 0.0 \\ 0.0 \end{array}]$ (1)

∴

Julia already has a function to give a spanning set for the null set

∴	nullspace([1 0 0])

$[\begin{array}{cc} 0.0 & 0.0 \\ 1.0 & 0.0 \\ 0.0 & 1.0 \end{array}]$ (2)

Example 1

𝑨 = [\begin{array}{c} 1 \\ 0 \\ 0 \end{array}], 𝑨^{T} = [\begin{array}{ccc} 1 & 0 & 0 \end{array}],

The column space $C (𝑨)$ is the $y_{1}$ -axis, and the left null space $N (𝑨^{T})$ is the $y_{2} y_{3}$ -plane

∴	A=[1; 0; 0]; colspace(A)

$[\begin{array}{c} 1.0 \\ 0.0 \\ 0.0 \end{array}]$ (3)

∴	nullspace(A')

$[\begin{array}{cc} 0.0 & 0.0 \\ 1.0 & 0.0 \\ 0.0 & 1.0 \end{array}]$ (4)

∴	[colspace(A) nullspace(A')]

$[\begin{array}{ccc} 1.0 & 0.0 & 0.0 \\ 0.0 & 1.0 & 0.0 \\ 0.0 & 0.0 & 1.0 \end{array}]$ (5)

∴

Example 2

𝑨 = [\begin{array}{cc} 1 & - 1 \\ 0 & 0 \\ 0 & 0 \end{array}] = [\begin{array}{cc} 𝒂_{1} & 𝒂_{2} \end{array}], 𝑨^{T} = [\begin{array}{ccc} 1 & 0 & 0 \\ - 1 & 0 & 0 \end{array}],

The columns of $𝑨$ are colinear, $𝒂_{2} = - 𝒂_{1}$ , and the column space $C (𝑨)$ is the $y_{1}$ -axis, and the left null space $N (𝑨^{T})$ is the $y_{2} y_{3}$ -plane, as before.

∴	A=[1 -1; 0 0; 0 0]; CA=colspace(A)

$[\begin{array}{c} 1.0 \\ 0.0 \\ 0.0 \end{array}]$ (6)

∴	NAt=short.(nullspace(A'))

$[\begin{array}{cc} 0.0 & 0.0 \\ 1.0 & 0.0 \\ 0.0 & 1.0 \end{array}]$ (7)

∴

[CA NAt]

$[\begin{array}{ccc} 1.0 & 0.0 & 0.0 \\ 0.0 & 1.0 & 0.0 \\ 0.0 & 0.0 & 1.0 \end{array}]$ (8)

∴

Example 3

𝑨 = [\begin{array}{cc} 1 & 0 \\ 0 & 1 \\ 0 & 0 \end{array}], 𝑨^{T} = [\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \end{array}],

The column space $C (𝑨)$ is the $y_{1} y_{2}$ -plane, and the left null space $N (𝑨^{T})$ is the $y_{3}$ -axis.

∴	A=[1 0; 0 1; 0 0]; CA=colspace(A)

$[\begin{array}{cc} 1.0 & 0.0 \\ 0.0 & 1.0 \\ 0.0 & 0.0 \end{array}]$ (9)

∴	NAt=short.(nullspace(A'))

$[\begin{array}{c} 0.0 \\ 0.0 \\ 1.0 \end{array}]$ (10)

∴

[CA NAt]

$[\begin{array}{ccc} 1.0 & 0.0 & 0.0 \\ 0.0 & 1.0 & 0.0 \\ 0.0 & 0.0 & 1.0 \end{array}]$ (11)

∴

Example 4

𝑨 = [\begin{array}{cc} 1 & 1 \\ 1 & - 1 \\ 0 & 0 \end{array}], 𝑨^{T} = [\begin{array}{ccc} 1 & 1 & 0 \\ 1 & - 1 & 0 \end{array}],

the same $C (𝑨)$ , $N (𝑨^{T})$ are obtained, albeit with a different set of spanning vectors returned by colspace.

∴	A=[1 1; 1 -1; 0 0]; CA=colspace(A)

$[\begin{array}{cc} - 0.707107 & - 0.707107 \\ - 0.707107 & 0.707107 \\ 0.0 & 0.0 \end{array}]$ (12)

∴	NAt=short.(nullspace(A'))

$[\begin{array}{c} 0.0 \\ 0.0 \\ 1.0 \end{array}]$ (13)

∴

[CA NAt]

$[\begin{array}{ccc} - 0.707107 & - 0.707107 & 0.0 \\ - 0.707107 & 0.707107 & 0.0 \\ 0.0 & 0.0 & 1.0 \end{array}]$ (14)

∴

Example 5

𝑨 = [\begin{array}{ccc} 1 & 1 & 3 \\ 1 & - 1 & - 1 \\ 1 & 1 & 3 \end{array}] = [\begin{array}{ccc} 𝒂_{1} & 𝒂_{2} & 𝒂_{3} \end{array}], 𝑨^{T} = [\begin{array}{ccc} 1 & 1 & 1 \\ 1 & - 1 & 1 \\ 3 & - 1 & 3 \end{array}] = [\begin{array}{c} 𝒂_{1}^{T} \\ 𝒂_{2}^{T} \\ 𝒂_{3}^{T} \end{array}], 𝑨^{T} 𝒚 = [\begin{array}{c} 𝒂_{1}^{T} 𝒚 \\ 𝒂_{2}^{T} 𝒚 \\ 𝒂_{3}^{T} 𝒚 \end{array}] .

$𝒂_{3} = 𝒂_{1} + 2 𝒂_{2} \Rightarrow$ $𝑨^{T} 𝒚 = 𝟎$ is satisfied by vectors of form $𝒚 = [\begin{array}{ccc} a & 0 & - a \end{array}]$ , $\forall a \in ℝ$ .

∴	A=[1 1 3; 1 -1 -1; 1 1 3]; CA=colspace(A)

$[\begin{array}{cc} - 0.5773502691896257 & 0.40824829046386313 \\ - 0.5773502691896257 & - 0.816496580927726 \\ - 0.5773502691896257 & 0.40824829046386313 \end{array}]$ (15)

∴	NAt=short.(nullspace(A'))

$[\begin{array}{c} - 0.707107 \\ 0.0 \\ 0.707107 \end{array}]$ (16)

∴

[CA NAt]

$[\begin{array}{ccc} - 0.5773502691896257 & 0.40824829046386313 & - 0.707107 \\ - 0.5773502691896257 & - 0.816496580927726 & 0.0 \\ - 0.5773502691896257 & 0.40824829046386313 & 0.707107 \end{array}]$ (17)

∴