"Maymester MATH547 Linear Algebra for Applications in Data Science"

1.Linear dependence

For the simple scalar mapping $f : ℝ \to ℝ$ , $f (x) = a x$ , the condition $f (x) = 0$ implies either that $a = 0$ or $x = 0$ . Note that $a = 0$ can be understood as defining a zero mapping $f (x) = 0$ . Linear mappings between vector spaces, $𝒇 : U \to V$ , can exhibit different behavior, and the condtion $𝒇 (𝒙) = 𝑨 𝒙 = 𝟎$ , might be satisfied for both $𝒙 \neq 𝟎$ , and $𝑨 \neq 𝟎$ . Analogous to the scalar case, $𝑨 = 𝟎$ can be understood as defining a zero mapping, $𝒇 (𝒙) = 𝟎$ .

In vector space $𝒱 = (V, S, +, \cdot)$ , vectors $𝒖, 𝒗 \in V$ related by a scaling operation, $𝒗 = a 𝒖$ , $a \in S$ , are said to be colinear, and are considered to contain redundant data. This can be restated as $𝒗 \in span {𝒖}$ , from which it results that $span {𝒖} = span {𝒖, 𝒗}$ . Colinearity can be expressed only in terms of vector scaling, but other types of redundancy arise when also considering vector addition as expressed by the span of a vector set. Assuming that $𝒗 \notin span {𝒖}$ , then the strict inclusion relation $span {𝒖} \subset span {𝒖, 𝒗}$ holds. This strict inclusion expressed in terms of set concepts can be transcribed into an algebraic condition.

Definition. The vectors $𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n} \in V,$ are linearly dependent if there exist $n$ scalars, $x_{1}, \dots, x_{n} \in S$ , at least one of which is different from zero such that

$x_{1} 𝒂_{1} + \dots + x_{n} 𝒂_{n} = 𝟎 .$

Introducing a matrix representation of the vectors

𝑨 = [\begin{array}{cccc} 𝒂_{1} & 𝒂_{2} & \dots & 𝒂_{n} \end{array}]; 𝒙 = [\begin{array}{c} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{array}]

allows restating linear dependence as the existence of a non-zero vector, $\exists 𝒙 \neq 𝟎$ , such that $𝑨 𝒙 = 𝟎$ . Linear dependence can also be written as $𝑨 𝒙 = 𝟎 ⇏ 𝒙 = 𝟎$ , or that one cannot deduce from the fact that the linear mapping $𝒇 (𝒙) = 𝑨 𝒙$ attains a zero value that the argument itself is zero. The converse of this statement would be that the only way to ensure $𝑨 𝒙 = 𝟎$ is for $𝒙 = 𝟎$ , or $𝑨 𝒙 = 𝟎 \Rightarrow 𝒙 = 𝟎$ , leading to the concept of linear independence.

Definition. The vectors $𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n} \in V,$ are linearly independent if the only $n$ scalars, $x_{1}, \dots, x_{n} \in S$ , that satisfy

x_{1} 𝒂_{1} + \dots + x_{n} 𝒂_{n} = 𝟎,

(1)

are $x_{1} = 0$ , $x_{2} = 0$ ,…, $x_{n} = 0$ .

2.Basis and dimension

Vector spaces are closed under linear combination, and the span of a vector set

ℬ = {𝒂_{1}, 𝒂_{2}, \dots}

defines a vector subspace. If the entire set of vectors can be obtained by a spanning set,

V = span ℬ

, extending

ℬ

by an additional element

𝒞 = ℬ \cup {𝒃}

would be redundant since

span ℬ = span 𝒞

. This is recognized by the concept of a basis, and also allows leads to a characterization of the size of a vector space by the cardinality of a basis set.

Definition. A set of vectors $𝒖_{1}, \dots, 𝒖_{n} \in V$ is a basis for vector space $𝒱 = (V, S, +, \cdot)$ if

Definition. The number of vectors $𝒖_{1}, \dots, 𝒖_{n} \in V$ within a basis is the dimension of the vector space $𝒱 = (V, S, +, \cdot)$ .

3.Dimension of matrix spaces

The domain and co-domain of the linear mapping

𝒇 : U \to V

𝒇 (𝒙) = 𝑨 𝒙

, are decomposed by the spaces associated with the matrix

𝑨

. When

U = ℝ^{n}

V = ℝ^{m}

, the following vector subspaces associated with the matrix

𝑨 \in ℝ^{m \times n}

have been defined:

Definition. The rank of a matrix $𝑨 \in ℝ^{m \times n}$ is the dimension of its column space and is equal to the dimension of its row space.

Definition. The nullity of a matrix $𝑨 \in ℝ^{m \times n}$ is the dimension of its null space.

Data Redundancy

1.Linear dependence

2.Basis and dimension

3.Dimension of matrix spaces