MATH383

Overview

Basis, dimension
Norm, scalar product, orthogonality
Change of basis in $ℝ^{m}$ , $𝑨 𝒙 = 𝑰 𝒃$ , and $𝒫_{n},$ $𝑩 (t) 𝒄 = 𝑴 (t) 𝒂$

Ask what is the minimal set of vectors $𝒂_{1}, 𝒂_{2}, \dots \in 𝒱$ needed to reach all other vectors in $𝒱$ . This leads to the following:

Definition. A basis for a vector space $(𝒱, +, 𝒮, \cdot)$ is a set of vectors $𝒂_{1}, 𝒂_{2}, \dots \in 𝒱$ that are linearly independent and whose range is the entire vector space, $ℛ (𝑨) = 𝒱$ , with $𝑨 = (\begin{array}{lll} 𝒂_{1} & 𝒂_{2} & \dots \end{array})$

Example 1. The column vectors of the identity matrix $𝑰 \in ℝ^{m \times m}$ are a basis for $ℝ^{m}$

𝑰 = (\begin{array}{llll} 𝒆_{1} & 𝒆_{2} & \dots & 𝒆_{m} \end{array}) = (\begin{array}{llll} 1 & 0 & \dots & 0 \\ 0 & 1 & \dots & 0 \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & \dots & 1 \end{array})

Example 2. ${1, t, t^{2}, \dots, t^{n}}$ is a basis for $𝒫_{n}$ , polynomials up to degree $n$ .

The notion of a basis allows a precise definition of dimension, i.e., “the size of a vector space”.

Definition. Two sets $𝒜, ℬ$ are said to have the same cardinality (intuitively, the same size), denoted as $| 𝒜 | = | ℬ |$ if there is some one-to-one function (a bijection) between $𝒜$ and $ℬ$ .

Definition. A set of the same cardinality as the ${1, 2, \dots, n}$ subset of the naturals $ℕ$ is said to be a finite set.

Definition. The dimension of a vector space $(𝒱, +, 𝒮, \cdot)$ is the cardinality of a basis $𝑨 = (\begin{array}{lll} 𝒂_{1} & 𝒂_{2} & \dots \end{array})$ of $𝒱$ .

Note that the above sequence of definitions allows for definition of infinite-dimensional vector spaces, whose definition gets to be a bit technical, i.e., an infinite set is one that has a subset of cardinality $n$ , for any $n \in ℕ$ .

Vectors were introduced for quantities with both magnitude and direction.
The norm function $|| || : 𝒱 \to ℝ_{+}$ extracts the magnitude of a vector

Definition. A function $|| || : 𝒱 \to ℝ_{+}$ is a norm on vector space $(𝒱, +, 𝒮, \cdot)$ if for any vectors $𝒖, 𝒗 \in 𝒱$ and any scalar $α \in 𝒮$

$|| 𝒖 || ⩾ 0$ , and $|| 𝒖 || = 0$ if and only if $𝒖 = 𝟎$ (only the zero vector has zero norm)

$|| α 𝒖 || = | α | || 𝒖 ||$ (vectors can be scaled)

$|| 𝒖 + 𝒗 || ⩽ || 𝒖 || + || 𝒗 ||$ (triangle inequality)

Example. In the Euclidean vector space $(ℝ^{m}, +, ℝ, \cdot)$ , $|| 𝒗 || = {(\sum_{i = 1}^{m} v_{i}^{2})}^{1 / 2}$

is a norm (the Euclidean norm or the $2$ -norm).

The norm can be used to measure magnitude, but another tool is needed to measure direction, such as the relative orientation between two vectors.

Definition. A function $(,) : 𝒱 \times 𝒱 \to ℝ$ is a scalar product over the vector space $(𝒱, +, ℝ, \cdot)$ if $\forall 𝒖, 𝒗, 𝒘 \in 𝒱$ , $\forall α, β \in 𝒮$ :

$(𝒖, 𝒗) = (𝒗, 𝒖)$ (symmetry of relative orientation)

$(α 𝒖 + β 𝒗, 𝒘) = α (𝒖, 𝒘) + β (𝒗, 𝒘)$ (linearity in first argument)

$(𝒖, 𝒖) > 0$ if $𝒖 \neq 𝟎$ .

Note that a scalar product can be used to define a norm through $|| 𝒖 || = {(𝒖, 𝒖)}^{1 / 2}$ .

Example. In $(ℝ^{m}, +, ℝ, \cdot)$ , $(𝒖, 𝒗) = \sum_{i = 1}^{m} u_{i} v_{i}$ is a scalar product. It can be computed through matrix multiplication as $(𝒖, 𝒗) = 𝒖^{T} 𝒗$ .

One special relative orientation between vectors is of great interest in linear algebra because it implies linear independence.

Definition. Vectors $𝒖, 𝒗$ are said to be orthogonal if their scalar product is null, $(𝒖, 𝒗) = 0$ .

Orthogonal vectors are linearly independent. Consider $α 𝒖 + β 𝒗 = 𝟎$ and $(𝒖, 𝒗) = 0$ .

Take the scalar product of $α 𝒖 + β 𝒗$ and $𝒖$ : $(α 𝒖 + β 𝒗, 𝒖) = α (𝒖, 𝒖) + β (𝒗, 𝒖) = α (𝒖, 𝒖) + β (𝒖, 𝒗) = α (𝒖, 𝒖) = (𝟎, 𝒖) = 0$ . Since $(𝒖, 𝒖) > 0$ it results that $α = 0$
Repeat for scalar product of $α 𝒖 + β 𝒗$ and $𝒗$ to find that $β = 0$
Since $α 𝒖 + β 𝒗 = 𝟎$ implies both $α = 0$ and $β = 0$ , $𝒖, 𝒗$ are linearly independent.

The above framework can be extended to discuss relative orientation and orthogonality (and hence, linear independence) of functions
$(𝒞_{[a, b]}, +, ℝ, \cdot)$ is the vector space of continuous, real-valued functions defined on $[a, b]$ , $𝒇 \in 𝒞_{[a, b]} \Rightarrow f : [a, b] \to ℝ$
$(𝒇, 𝒈) = \int_{a}^{b} f (t) g (t) d t$ is a scalar product for $(𝒞_{[a, b]}, +, ℝ, \cdot)$
If $(𝒇, 𝒈) = 0$ the functions $𝒇, 𝒈 \in 𝒞_{[a, b]}$ are said to be orthogonal and they are linearly independent

Example. $\sin t, \cos t$ are linearly independent in $(𝒞_{[0, 2 π]}, +, ℝ, \cdot)$ since

(\sin t, \cos t) = \int_{0}^{2 π} \sin t \cos t d t = \frac{1}{2} \int_{0}^{2 π} \sin 2 t d t = \frac{- 1}{4} {[\cos 2 t]}_{0}^{2 π} = 0 .

A vector from the Euclidean vector space $(ℝ^{m}, +, ℝ, \cdot)$ is typically given in terms of its components with respect to the $𝑰 = (\begin{array}{llll} 𝒆_{1} & 𝒆_{2} & \dots & 𝒆_{m} \end{array})$ basis
$𝒃 = b_{1} 𝒆_{1} + b_{2} 𝒆_{2} + \dots + b_{m} 𝒆_{m} = 𝑰 𝒃$
Ask whether the vector can be expressed as a linear combination $𝒃 = 𝑨 𝒙$ of another set of vectors $𝑨 = (\begin{array}{lll} 𝒂_{1} & 𝒂_{2} & \dots 𝒂_{n} \end{array})$ , with coefficients $x_{1}, x_{2}, \dots, x_{n}$ .
$(\begin{array}{ll} 𝑨 & 𝒃 \end{array}) = (\begin{array}{llll} 1 & 2 & 2 & 15 \\ 2 & 1 & 0 & 4 \\ 1 & 2 & 1 & 10 \end{array}) \sim (\begin{array}{llll} 1 & 2 & 2 & 15 \\ 0 & - 3 & - 4 & - 26 \\ 0 & 0 & - 1 & - 5 \end{array}) \sim (\begin{array}{llll} 1 & 2 & 2 & 15 \\ 0 & 3 & 4 & 26 \\ 0 & 0 & 1 & 5 \end{array}) \sim$ $(\begin{array}{llll} 1 & 2 & 0 & 5 \\ 0 & 3 & 0 & 6 \\ 0 & 0 & 1 & 5 \end{array}) \sim (\begin{array}{llll} 1 & 0 & 0 & 1 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 5 \end{array}) \Rightarrow (\begin{array}{lll} 1 & 2 & 2 \\ 2 & 1 & 0 \\ 1 & 2 & 1 \end{array}) (\begin{array}{l} 1 \\ 2 \\ 5 \end{array}) = (\begin{array}{l} 15 \\ 4 \\ 10 \end{array})$

A similar procedure can be used for other base changes for example from the basis $(\begin{array}{lll} 1 & t & t^{2} \end{array})$ to $𝑨 = (\begin{array}{lll} 2 & t - 1 & t^{2} - 1 \end{array})$ for $𝒫_{2}$ . Consider $b (t) = 1 + t + t^{2}$
$𝑨 𝒙 = 𝒃 \Rightarrow (\begin{array}{lll} 2 & t - 1 & t^{2} - 1 \end{array}) (\begin{array}{l} x_{1} \\ x_{2} \\ x_{3} \end{array}) = (\begin{array}{lll} 1 & t & t^{2} \end{array}) (\begin{array}{l} 1 \\ 1 \\ 1 \end{array}) = 1 + t + t^{2} \Rightarrow$ ${\begin{cases} 2 x_{1} - x_{2} - x_{3} & = & 1 \\ x_{2} & = & 1 \\ x_{3} & = & 1 \end{cases} \Rightarrow x_{1} = \frac{3}{2} .$