MATH383

Overview

Algebraic structure of a vector space, $(ℝ^{n}, ℝ, +, \cdot)$
Linear combination, matrix-vector product
Linear independence, linear dependence
Range and null space

After groups and fields, an additional algebraic structure of particular relevance to differential equations is now introduced, that of a vector space

A vector space $(𝒱, (𝒮, +, \cdot), \oplus, ⊙)$ is formed by a set of vectors $𝒱$ , a set of scalars $(𝒮, +, \cdot)$ with a field structure, the operation of vector addition $\oplus$ , and the operation of multiplication of a vector by a scalar $⊙$ , with the properties of

a commutative group for $(𝒱, \oplus)$ : $\forall 𝒖, 𝒗, 𝒘 \in 𝒱$ , $\forall α, β \in 𝒮$

$𝒖 \oplus 𝒗 \in 𝒱$	$𝒖 \oplus (𝒗 \oplus 𝒘) = (𝒖 \oplus 𝒗) \oplus 𝒘$
$\exists 𝟎 \in 𝒱$ , $𝒖 \oplus 𝟎 = 𝟎 \oplus 𝒖 = 𝒖$	$\exists (- 𝒖)$ , $𝒖 \oplus (- 𝒖) = 0$
$𝒖 \oplus 𝒗 = 𝒗 \oplus 𝒖$

closure with respect to multiplication by a scalar: $α 𝒖 \in 𝒱$

Distributivity properties:

$α ⊙ (𝒖 \oplus 𝒗) = α ⊙ 𝒖 \oplus α ⊙ 𝒗$	$(α + β) ⊙ 𝒖 = α ⊙ 𝒖 \oplus β ⊙ 𝒖$
$α ⊙ (β ⊙ 𝒖) = (α \cdot β) 𝒖$

Scalar identity element: $1 ⊙ 𝒖 = 𝒖$

$(𝒫_{n}, +, ℝ, \cdot)$ is the vector space of polynomials of degree at most $n$ over the reals. For $𝒑, 𝒒 \in 𝒫_{n}$ $α \in ℝ$ , the operations $+, \cdot$ are defined as
$\begin{array}{ll} 𝒑 = a_{0} + a_{1} x + \dots + a_{n} x^{n}, & 𝒒 = b_{0} + b_{1} x + \dots + b_{n} x^{n} \\ 𝒑 + 𝒒 = a_{0} + b_{0} + (a_{1} + b_{1}) x + \dots + (a_{n} + b_{n}) x^{n}, \\ α 𝒑 = α a_{0} + α a_{1} x + \dots + α a_{n} x^{n}, \end{array}$
with $a_{n}, \dots a_{0} \in ℝ$ , $b_{n}, \dots b_{0} \in ℝ$ . Note that the variable $x$ is irrelevant to the specification of a particular polynomial by its coefficients, such that it is convenient to identify $𝒑 \equiv (a_{0}, a_{1}, \dots, a_{n})$ and the operations become
$\begin{array}{ll} 𝒑 + 𝒒 = (a_{0} + b_{0}, a_{1} + b_{1}, \dots, a_{n} + b_{n}) & α 𝒑 = (α a_{0}, \dots, α a_{n}) \end{array}$
$(ℝ, \oplus, ℝ, \cdot)$ with $a, b \in 𝒱 = ℝ$ , $a \oplus b = a^{b}$ is not a vector space, since $a^{b} \neq b^{a}$ in general. $(𝒫_{n}, +, ℕ, \cdot)$ is not a vector space since $ℕ$ is not a field.

The Euclidean vector space $(ℝ^{n}, ℝ, +, \cdot)$ with operations for $𝒙, 𝒚 \in ℝ^{n}$ , $α \in ℝ$
$𝒙 = (\begin{array}{l} x_{1} \\ ⋮ \\ x_{n} \end{array}), 𝒚 = (\begin{array}{l} y_{1} \\ ⋮ \\ y_{n} \end{array}), 𝒙 + 𝒚 = (\begin{array}{l} x_{1} + y_{1} \\ ⋮ \\ x_{n} + y_{n} \end{array}), α 𝒙 = (\begin{array}{l} α x_{1} \\ ⋮ \\ α x_{n} \end{array})$
has many applications:
- position vector of a point in 3D, $\overset{⇀}{r} = x \overset{⇀}{i} + y \overset{⇀}{i} + z \overset{⇀}{k}$
- polynomials, $𝒑 = a_{0} + a_{1} x + \dots + a_{n} x^{n}$ , $𝒑 = (a_{0}, a_{1}, \dots, a_{n})$
- trigonometric sums, $𝒇 = a_{0} + a_{1} \cos t + b_{1} \sin t + \dots + a_{n} \cos t + b_{n} \sin t$ , $𝒇 \equiv (a_{0}, a_{1}, b_{1}, \dots, a_{n}, b_{n}) \in ℝ^{2 n + 1}$
- exponential sums, $𝒇 = a_{0} + a_{1} e^{t} + \dots + a_{n} e^{n t}$

In vector space $(𝒱, +, 𝒮, \cdot)$ , from $𝒂_{1}, 𝒃_{2} \in 𝒱$ , $x_{1}, x_{2} \in 𝒮$ , obtain a new vector $𝒘$

𝒃 = x_{1} 𝒂_{1} + x_{2} 𝒂_{2}

(1)

Question: can all vectors within $𝒱$ be obtained this way?

Answer: in general, no since in $(ℝ^{3}, +, ℝ, \cdot)$ , $𝒃 = (0, 0, 1)$ cannot be obtained from $𝒂_{1} = (1, 0, 0)$ , $𝒂_{2} = (0, 1, 0)$ .
Question: can one somehow generalize (1) so as to describe all vectors in $𝒱$ ?

Answer: yes, through the following

Definition. In vector space $(𝒱, +, 𝒮, \cdot)$

$𝒃 = x_{1} 𝒂_{1} + x_{2} 𝒂_{2} + \dots + x_{n} 𝒂_{n}$

is the linear combination of $𝒂_{1}, \dots, 𝒂_{n} \in 𝒱$ with coefficients $x_{1}, \dots, x_{n} \in 𝒮$ .

Linear combinations have very many applications. It is convenient to define notation to efficiently carry out linear combinations, especially for the widely used Euclidean space $(ℝ^{m}, +, ℝ, \cdot)$ (we use $m$ for number of components, $n$

Group the vectors $𝒂_{1}, \dots, 𝒂_{n} \in ℝ^{m}$ together to define a matrix, group the scalar coefficients $x_{1}, x_{2}, \dots, x_{n} \in ℝ$ together into a vector $𝒙 \in ℝ^{n}$
$𝑨 = (\begin{array}{lll} 𝒂_{1} & \dots & 𝒂_{n} \end{array}) = (\begin{array}{llll} a_{11} & a_{12} & \dots & a_{1 n} \\ a_{21} & a_{22} & \dots & a_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{m 1} & a_{m 2} & \dots & a_{m n} \end{array}), 𝒙 = (\begin{array}{l} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{array}) .$
Define matrix-vector multiplication to carry out the linear combination
$𝑨 𝒙 = x_{1} 𝒂_{1} + x_{2} 𝒂_{2} + \dots + x_{n} 𝒂_{n}$
Note: $m$ denotes the number of components in a vector, $n$ the number of vectors in the linear combination.

Denote set of real-component matrices with $m$ rows and $n$ columns by $ℝ^{m \times n}$
Notation:
- $𝑨, 𝑩, \dots$ are matrices, $𝒙, 𝒃$ are vectors, $u, v$ are scalars
- Column vectors of $𝑩 \in ℝ^{m \times n}$ are $𝒃_{1}, \dots, 𝒃_{n}$
- Components of $𝒂_{j} \in ℝ^{m}$ are $a_{1 j}, \dots, a_{m j}$
Matrix-vector product can also be computed by “row over columns rule”
$𝑨 𝒙 = (\begin{array}{llll} a_{11} & a_{12} & \dots & a_{1 n} \\ a_{21} & a_{22} & \dots & a_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{m 1} & a_{m 2} & \dots & a_{m n} \end{array}) (\begin{array}{l} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{array}) = (\begin{array}{l} x_{1} a_{11} + x_{2} a_{12} + \dots + x_{n} a_{1 n} \\ x_{1} a_{21} + x_{2} a_{22} + \dots + x_{n} a_{2 n} \\ ⋮ \\ x_{1} a_{m 1} + x_{2} a_{m 2} + \dots + x_{n} a_{m n} \end{array})$

Having defined $𝑨 𝒙$ as a concise way of defining the linear combination of $n$ vectors $𝒂_{1}, \dots, 𝒂_{n} \in ℝ^{m}$ with scalar coefficients $x_{1}, \dots, x_{n} \in ℝ$ , consider now the original question: can all vectors within a vector space $𝒱$ be reached by linear combination of some choice of vectors in $𝒂_{1}, \dots, 𝒂_{n} \in 𝒱$ ?

Definition. The vectors $𝒂_{1}, \dots, 𝒂_{n} \in 𝒱$ are said to be linearly dependent if there exists a solution $𝒙 \neq 𝟎$ to the equation $𝑨 𝒙 = x_{1} 𝒂_{1} + \dots + x_{n} 𝒂_{n} = 𝟎 .$

In the vector space $(ℝ^{2}, +, ℝ, \cdot)$ , $𝑨 = (\begin{array}{ll} 𝒂_{1} & 𝒂_{2} \end{array}) = (\begin{array}{ll} 1 & 2 \\ 2 & 4 \end{array})$ are linearly dependent. The solutions to $𝑨 𝒙 = 𝟎$ are $x_{1} = - 2 α$ , $x_{2} = α$ , for any $α \in ℝ$
In the vector space $(𝒫_{1}, +, ℝ, \cdot)$ , $𝑨 = (\begin{array}{lll} 1 & t & 1 - t \end{array})$ are linearly dependent since $𝒂_{3} = 1 - t = 1 \cdot 𝒂_{1} - 2 \cdot 𝒂_{2} \Rightarrow 1 \cdot 𝒂_{1} - 2 \cdot 𝒂_{2} - 𝒂_{3} = 𝟎$ , and $𝒙 = (1, - 2, - 1)$ is a solution to $𝑨 𝒙 = 𝟎$ with $𝒙 \neq 𝟎$ .

1. Note: the matrix notation has been extended to allow for arbitrary vectors (in this case polynomials) to be placed in columns. This is a very useful convention for computational applications.

¹

Definition. The vectors $𝒂_{1}, \dots, 𝒂_{n} \in 𝒱$ are said to be linearly independent if $𝒙 = 𝟎$ is the only solution to the equation $𝑨 𝒙 = x_{1} 𝒂_{1} + \dots + x_{n} 𝒂_{n} = 𝟎 .$

In the vector space $(ℝ^{2}, +, ℝ, \cdot)$ , $𝑨 = (\begin{array}{ll} 𝒂_{1} & 𝒂_{2} \end{array}) = (\begin{array}{ll} 1 & 0 \\ 0 & 1 \end{array})$ are linearly independent. The only solution to $𝑨 𝒙 = 𝟎$ is $x_{1} = 0$ , $x_{2} = 0$
In the vector space $(𝒫_{1}, +, ℝ, \cdot)$ , $𝑨 = (\begin{array}{ll} 𝟏 & 𝒕 \end{array})$ are linearly independent since

$𝑨 𝒙 = x_{1} \cdot 𝟏 + x_{2} \cdot 𝒕 = 𝟎$ (2)

is an equality between vectors that are in this case the polynomials $𝟏, 𝒕 \in 𝒫_{1}$ , and (2) must be satisfied for all $t$ . Choosing $t = 0$ implies $x_{1} = 0$ , and choosing $t = 1$ subsequently implies $x_{2} = 0$

Linear dependence of vectors $𝒂_{1}, \dots, 𝒂_{n}$ indicates that some vectors are redundant, they can be expressed as linear combinations of other vectors in the set.
Linear independence of vectors $𝒂_{1}, \dots, 𝒂_{n}$ indicates that there are no redundant vectors in the set, but it is not yet established that all vectors in the $𝒱$ can be reached by linear combination of $𝒂_{1}, \dots, 𝒂_{n}$ .

Definition. The range of $𝑨 = (\begin{array}{lll} 𝒂_{1} & \dots & 𝒂_{n} \end{array})$ (also known as the span of vectors $𝒂_{1}, \dots, 𝒂_{n}$ ) is the set

$ℛ (𝑨) = {𝒃 : \exists 𝒙 \in ℝ^{n}, 𝒃 = 𝑨 𝒙} .$

The range or the span is the set of all vectors reachable by linear combination of columns of $𝑨$ .

Also introduce a definition to characterize whether a set of vectors (represented as columns of a matrix) are linearly independent or not.

Definition. The null space of $𝑨 = (\begin{array}{lll} 𝒂_{1} & \dots & 𝒂_{n} \end{array})$ (also known as the kernel of $𝑨$ ) is the set

$𝒩 (𝑨) = {𝒙 : 𝑨 𝒙 = 𝟎} .$

If the null space only contains $𝟎$ , $𝒩 (𝑨) = {𝟎}$ , then the vectors $𝒂_{1}, \dots, 𝒂_{n}$ , $𝑨 = (\begin{array}{lll} 𝒂_{1} & \dots & 𝒂_{n} \end{array})$ are linearly independent.

In vector space $(ℝ^{2}, +, ℝ, \cdot)$ , the null space of $𝑨 = (\begin{array}{ll} 𝒂_{1} & 𝒂_{2} \end{array}) = (\begin{array}{ll} 1 & 2 \\ 2 & 4 \end{array})$ is
$𝒩 (𝑨) = {α (\begin{array}{l} - 2 \\ 1 \end{array}) : α \in ℝ}, 𝑨 (\begin{array}{l} - 2 α \\ α \end{array}) = (\begin{array}{ll} 1 & 2 \\ 2 & 4 \end{array}) (\begin{array}{l} - 2 α \\ α \end{array}) = (\begin{array}{l} 0 \\ 0 \end{array})$