MATH347

MATH347 L2: Matrices and problems of linear algebra

Previous concept synopsis:
- Vectors are groupings of scalars
- Vector addition and multiplication of a vector by a scalar have been defined
- Linear combination defined
- Matrices are groupings of vectors
- Matrix-vector product expresses a linear combination in a concise notation
New concepts:
- Scalar product of two vectors
- Orthogonal vectors
- Norm of a vector

Matrices

Definition. An $m$ by $n$ matrix is a grouping of $n$ vectors,

$𝑨 = [\begin{array}{llll} 𝒂_{1} & 𝒂_{2} & \dots & 𝒂_{n} \end{array}] \in S^{m \times n},$

where each vector has $m$ scalar components $𝒂_{1}, 𝒂_{2}, \dots, 𝒂_{n} \in S^{m}$ .

Notation conventions:
- scalars: normal face, Latin or Greek letters, $a, b, α, β, u_{1}, a_{11}$ , $A_{11}, I_{12}$
- vectors: bold face, lower case Latin letters, $𝒖, 𝒗, 𝒂_{1}$
- matrices: bold face, upper case Latin letters, $𝑨, 𝑩, 𝑳_{1}$

Matrices

Matrix components
$𝒂_{1} = [\begin{array}{l} a_{11} \\ a_{21} \\ ⋮ \\ a_{m 1} \end{array}], 𝒂_{2} = [\begin{array}{l} a_{12} \\ a_{22} \\ ⋮ \\ a_{m 2} \end{array}], \dots, 𝒂_{n} = [\begin{array}{l} a_{1 n} \\ a_{2 n} \\ ⋮ \\ a_{m n} \end{array}] \Rightarrow$ $𝑨 = [\begin{array}{llll} a_{11} & a_{12} & \dots & a_{1 n} \\ a_{21} & a_{22} & \dots & a_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{m 1} & a_{m 2} & \dots & a_{m n} \end{array}]$
A real-valued matrix with $m$ lines and $n$ : $𝑨 \in ℝ^{m \times n}$
>>

A=[3 1 2; -1 0 1; 3 4 1]; disp(A)
3 1 2 -1 0 1 3 4 1

Matrix components

Instead of explicitly writing out components, it is often convenient to specify a matrix by a rule to construct each component
$𝑨 \in ℝ^{m \times n},, 𝑨 = [a_{i j}]$
with indices taking values $i \in {1, \dots, m}, j \in {1, \dots, n}$ .

Example: A Hilbert matrix $𝑯_{m}$ is defined as

𝑯_{m} \in ℚ^{m \times m},, 𝑯_{m} = [\frac{1}{i + j - 1}]

Note that a vector is a matrix with a single column. The notation $𝒗 \in ℝ^{m},$ is a customary shorter form of $𝒗 \in ℝ^{m \times 1}$ .

Scalar product, innner product

Scalar products are useful in many other contexts than real-valued vectors.

Definition. Consider vectors $𝒖, 𝒗, 𝒘 \in 𝒱$ and scalar $a \in ℝ .$ The function

$⟨, ⟩ : 𝒱 \times 𝒱 \to S$

is a scalar product if:
$⟨ 𝒖, 𝒗 ⟩ = ⟨ 𝒗, 𝒖 ⟩$ (Symmetry)

$⟨ a 𝒖, 𝒗 ⟩ = a ⟨ 𝒖, 𝒗 ⟩, ⟨ 𝒖 + 𝒗, 𝒘 ⟩ = ⟨ 𝒖, 𝒘 ⟩ + ⟨ 𝒗, 𝒘 ⟩$ (Linearity in first argument)

$⟨ 𝒖, 𝒖 ⟩ ⩾ 0, ⟨ 𝒖, 𝒖 ⟩ = 0 \Rightarrow 𝒖 = 𝟎$ (Positive definiteness)
Inner product of $𝒖, 𝒗 \in ℝ^{m}$ , $𝒖 \cdot 𝒗 = u_{1} v_{1} + u_{2} v_{2} + \dots + u_{m} v_{m}$

Vector norm

Definition. The norm of a vector is a function that takes a vector argument, returns a positive real number, $|| || : 𝒱 \to ℝ_{+}$ , and for $𝒖, 𝒗 \in 𝒱$ , $a \in S$ ,satisfies properties:

$|| a 𝒖 || = | a | || 𝒖 ||$

$|| 𝒖 + 𝒗 || ⩽ || 𝒖 || + || 𝒗 ||$

$|| 𝒖 || = 0 \Rightarrow 𝒖 = 𝟎$ .

A norm embodies the concept of measurement of the magnitude of a vector

Different ways of measuring the magnitude of a vector are most appropriate in various applications, resulting in different definitions of a vector norm for $𝒖 \in ℝ^{m}$

1-norm	${\|\| 𝒖 \|\|}_{1} = \sum_{i = 1}^{m} \| u_{i} \|$
2-norm (Euclidean norm)	${\|\| 𝒖 \|\|}_{2} = {(\sum_{i = 1}^{m} u_{i}^{2})}^{1 / 2} = {(𝒖^{T} 𝒖)}^{1 / 2}$
inf-norm	${\|\| 𝒖 \|\|}_{\infty} = {max}_{i \in {1, 2, \dots, m}} \| u_{i} \|$

Different norms are distinguished by subscripts as above. The most commonly used norm is the Euclidean norm that corresponds to the square root of the scalar product, i.e. $|| 𝒖 || = {(𝒖 \cdot 𝒖)}^{1 / 2}$ , in which case the subscript is often suppressed to simplify notation $|| 𝒖 || = {|| 𝒖 ||}_{2}$

Angle between two vectors

Law of cosines
${|| 𝒗 - 𝒘 ||}^{2} = {|| 𝒗 ||}^{2} + {|| 𝒘 ||}^{2} - 2 || 𝒗 || || 𝒘 || \cos (θ)$
Dot product properties
${|| 𝒗 - 𝒘 ||}^{2} = {|| 𝒗 ||}^{2} + {|| 𝒘 ||}^{2} - 2 (𝒗 \cdot 𝒘)$
Combine, and define
$\cos (θ) = \frac{𝒗 \cdot 𝒘}{|| 𝒗 || || 𝒘 ||}$
In particular if $\cos (θ) = 0 \Rightarrow 𝒗 \cdot 𝒘 = 0$ , $𝒗, 𝒘$ are said to be orthogonal

Example: Column vectors of a matrix

𝑨 = [\begin{array}{lll} 3 & 1 & 2 \\ - 1 & 0 & 1 \\ 3 & 4 & 1 \end{array}] = [\begin{array}{lll} 𝒂_{1} & 𝒂_{2} & 𝒂_{3} \end{array}]

>>	A=[3 1 2; -1 0 1; 3 4 1]; disp(A)

3 1 2 -1 0 1 3 4 1

>>

A(:,2)

$(\begin{array}{c} 1 \\ 0.0 \\ 4 \end{array})$

Example: Row vectors of a matrix

𝑨 = [\begin{array}{lll} 3 & 1 & 2 \\ - 1 & 0 & 1 \\ 3 & 4 & 1 \end{array}] = [\begin{array}{lll} 𝒂_{1} & 𝒂_{2} & 𝒂_{3} \end{array}]

>>	A=[3 1 2; -1 0 1; 3 4 1]; disp(A)

3 1 2 -1 0 1 3 4 1

>>

A(2,:)

$(\begin{array}{c} 1 \\ 0.0 \\ 4 \end{array})$

Example: Block within a matrix

𝑨 = [\begin{array}{lll} 3 & 1 & 2 \\ - 1 & 0 & 1 \\ 3 & 4 & 1 \end{array}] = [\begin{array}{lll} 𝒂_{1} & 𝒂_{2} & 𝒂_{3} \end{array}]

>>	A=[3 1 2; -1 0 1; 3 4 1]; disp(A)

3 1 2 -1 0 1 3 4 1

>>

A(:,2:3)

$(\begin{array}{cc} 1 & 2 \\ 0.0 & 1 \\ 4 & 1 \end{array})$

Example: Inner product of two vectors

>>	u=[1 2 3]; v=[3 0 1]; dot(u,v)

$6$

>>	e1=[1 0 0]; e2=[0 1 0]; e3=[0 0 1]; dot(e1,e2)

$0.0$

>>	dot(e1,e3)

$0.0$

>>	dot(e2,e3)

$0.0$

>>	dot(e1,e1)

$1$

>>	dot(e2,e2)

$1$

>>	dot(e3,e3)

$1$

Example: Vector norm

>>	u=[1 2 3]; v=[3 0 1]; norm(u)

$3.7417$

>>	sqrt(dot(u,u))

$3.7417$

>>

norm(v)

$3.1623$

>>	sqrt(dot(v,v))

$3.1623$

>>	e1=[1 0 0]; e2=[0 1 0]; e3=[0 0 1]; norm(e1)

$1$

>>

norm(e2)

$1$

>>

norm(e3)

$1$

Example: Angle between two vectors

\cos θ = \frac{𝒖 \cdot 𝒗}{|| 𝒖 || || 𝒗 ||}

>>	dot(e1,e2)/norm(e1)/norm(e2)

$0.0$

>>

acos(0.0)

$1.5708$

>>	dot(e1,e1)/norm(e1)/norm(e1)

$1$

>>

acos(1)

$0.0$

>>	u=[1 0]; v=[1 1]; dot(u,v)/norm(u)/norm(v)

$0.7071$

>>	acos(0.7071)/pi*180

$45.001$