1.1.Relations

A general procedure to relate input values from set $X$ to output values from set $Y$ is to first construct the set of all possible instances of $x\in X$ and $y\in Y$, which is the Cartesian product of $X$ with $Y$, denoted as $X\times Y=\{(x,y)|.x\in X,y\in Y\}$. Usually only some associations of inputs to outputs are of interest leading to the following definition.

Definition. (Relation) . A relation $R$ between two sets $X,Y$ is a subset of the Cartesian product $X\times Y$, $R\subseteq X\times Y$.

Associating an output to an input is also useful, leading to the definition of an inverse relation as $R^{-1}\subseteq Y\times X$, $R^{-1}=\{(y,x)|(x,y)\in R.\}$. Note that an inverse exists for any relation, and the inverse of an inverse is the original relation, ${\left(R^{-1}\right)}^{-1}=R$.

Homogeneous relations.

Reflection

Relation $H$ is reflexive if $(a,a)\in H$ for any $a\in A$. The equality relation $E\subseteq ℝ\times ℝ$ is reflexive, $\forall a\in A,a=a$, the less than relation $L\subseteq ℝ\times ℝ$ is not, $1\in R,1\nless 1$.

Symmetry

Relation $H$ is symmetric if $(a,b)\in H$ implies that $(b,a)\in H$, $(a,b)\in H\Rightarrow (b,a)\in H$. The equality relation $E\subseteq ℝ\times ℝ$ is symmetric, $a=b\Rightarrow b=a$, the less than relation $L\subseteq ℝ\times ℝ$ is not, $a<b\nRightarrow b<a$.

Anti-symmetry

Relation $H$ is anti-symmetric if $(a,b)\in H$ for $a\ne b$, then $(b,a)\notin H$. The less than relation $L\subseteq ℝ\times ℝ$ is antisymmetric, $a<b\Rightarrow b\nless a$.

Transitivity

Relation $H$ is transitive if $(a,b)\in H$ and $(b,c)\in H$ implies $(a,c)\in H$. for any $a\in A$. The equality relation $E\subseteq ℝ\times ℝ$ is transitive, $a=b\wedge b=c\Rightarrow a=c$, as is the less than relation $L\subseteq ℝ\times ℝ$, $a<b\wedge b<c\Rightarrow a<c$.

Certain combinations of properties often arise. A homogeneous relation that is reflexive, symmetric, and transitive is said to be an equivalence relation. Equivalence relations include equality among the reals, or congruence among triangles. A homogeneous relation that is reflexive, anti-symmetric and transitive is a partial order relation, such as the less than or equal relation between reals. Finally, a homogeneous relation that is anti-symmetric and transitive is an order relation, such as the less than relation between reals.

1.2.Functions

Functions between sets $X$ and $Y$ are a specific type of relationship that often arise in science. For a given input $x\in X$, theories that predict a single possible output $y\in Y$ are of particular scientific interest.

Definition. (Function) . A function from set $X$ to set $Y$ is a relation $F\subseteq X\times Y$, that associates to $x\in X$ a single $y\in Y$.

The above intuitive definition can be transcribed in precise mathematical terms as $F\subseteq X\times Y$ is a function if $(x,y)\in F$ and $(x,z)\in F$ implies $y=z$. Since it's a particular kind of relation, a function is a triplet of sets $(X,Y,F)$, but with a special, common notation to denote the triplet by $f:X\to Y$, with $F=\left\{(x,f\left(x\right))|x\in X,f\left(x\right)\in Y.\right\}$ and the property that $(x,y)\in F\Rightarrow y=f\left(x\right)$. The set $X$ is the domain and the set $Y$ is the codomain of the function $f$. The value from the domain $x\in X$ is the argument of the function associated with the function value $y=f\left(x\right)$. The function value $y$ is said to be returned by evaluation $y=f\left(x\right)$.

As seen previously, a Euclidean space $E_m=({ℝ}^m,ℝ,+,\cdot )$ can be used to suggest properties of more complex spaces such as the vector space of continuous functions ${𝒞}^0\left(ℝ\right)$. A construct that will be often used is to interpret a vector within $E_m$ as a function, since $𝒗\in {ℝ}^m$ with components $𝒗={\left[\begin{array}{llll}{v}_1& v_2& \dots & v_m\end{array}\right]}^T$ also defines a function $v:\{1,2,\dots ,m\}\to ℝ$, with values $v\left(i\right)=v_i$. As the number of components grows the function $v$ can provide better approximations of some continuous function $f\in {𝒞}^0\left(ℝ\right)$ through the function values $v_i=v\left(i\right)=f\left(x_i\right)$ at distinct sample points $x_1,x_2,\dots ,x_m$.

The above function examples are all defined on a domain of scalars or naturals and returned scalar values. Within linear algebra the particular interest is on functions defined on sets of vectors from some vector space $𝒱=(V,S,+,\cdot )$ that return either scalars $f:V\to S$, or vectors from some other vector space $𝒲=(W,S,+,\cdot )$, $𝒈:V\to W$. The codomain of a vector-valued function might be the same set of vectors as its domain, $𝒉:V\to V$. The fundamental operation within linear algebra is the linear combination $a𝒖+b𝒗$ with $a,b\in S$, $𝒖,𝒗\in V$. A key aspect is to characterize how a function behaves when given a linear combination as its argument, for instance $f(a𝒖+b𝒗)$ or $𝒈(a𝒖+b𝒗).$

1.3.Linear functionals

Consider first the case of a function defined on a set of vectors that returns a scalar value. These can be interpreted as labels attached to a vector, and are very often encountered in applications from natural phenomena or data analysis.

Definition. (Functional) . A functional on vector space $𝒱=(V,S,+,\cdot )$ is a function from the set of vectors $V$ to the set of scalars $S$ of the vector space $𝒱$.

Definition. (Linear Functional) . The functional $f:V\to S$ on vector space $𝒱=(V,S,+,\cdot )$ is a linear functional if for any two vectors $𝒖,𝒗\in V$ and any two scalars $a,b$

Many different functionals may be defined on a vector space $𝒱=(V,S,+,\cdot )$, and an insightful alternative description is provided by considering the set of all linear functionals, that will be denoted as $V^{\ast }=\left\{f|f:V\to S.\right\}$. These can be organized into another vector space ${𝒱}^{\ast }=(V^{\ast },S,+,\cdot )$ with vector addition of linear functionals $f,g\in V^{\ast }$ and scaling by $a\in S$ defined by

Definition. (Dual Vector Space) . For some vector space $𝒱$, the vector space of linear functionals ${𝒱}^{\ast }$ is called the dual vector space.

As is often the case, the above abstract definition can better be understood by reference to the familiar case of Euclidean space. Consider ${ℛ}_2=({ℝ}^2,ℝ,+,\cdot )$, the set of vectors in the plane with $𝒙\in {ℝ}^2$ the position vector from the origin $(0,0)$ to point $X$ in the plane with coordinates $(x_1,x_2)$. One functional from the dual space ${ℛ}_2^{\ast }$ is $f_2\left(𝒙\right)=x_2$, i.e., taking the second coordinate of the position vector. The linearity property is readily verified. For $𝒙,𝒚\in {ℛ}_2$, $a,b\in ℝ$,

Given some constant value $h\in ℝ$, the curves within the plane defined by $f_2\left(𝒙\right)=h$ are called the contour lines or level sets of $f_2$. Several contour lines and position vectors are shown in Figure 1. The utility of functionals and dual spaces can be shown by considering a simple example from physics. Assume that $x_2$ is the height above ground level and a vector $𝒙$ is the displacement of a body of mass $m$ in a gravitational field. The mechanical work done to lift the body from ground level to height $h$ is $W=mgh$ with $g$ the gravitational acceleration. The mechanical work is the same for all displacements $𝒙$ that satisfy the equation $f_2\left(𝒙\right)=h$. The work expressed in units $mg\Delta h$ can be interpreted as the number of contour lines $f_2\left(𝒙\right)=n\Delta h$ intersected by the displacement vector $𝒙$. This concept of duality between vectors and scalar-valued functionals arises throughout mathematics, the physical and social sciences and in data science. The term “duality” itself comes from geometry. A point $X$ in ${ℝ}^2$ with coordinates $(x_1,x_2)$ can be defined either as the end-point of the position vector $𝒙$, or as the intersection of the contour lines of two functionals $f_1\left(𝒙\right)=x_1$ and $f_2\left(𝒙\right)=x_2$. Either geometric description works equally well in specifying the position of $X$, so it might seem redundant to have two such procedures. It turns out though that many quantities of interest in applications can be defined through use of both descriptions, as shown in the computation of mechanical work in a gravitational field.

Figure 1. Vectors in $E_2$ and contour lines of the functional $f\left(𝒙\right)=x_2$

1.4.Linear mappings

Consider now functions $𝒇:V\to W$ from vector space $𝒱=(V,S,+,\cdot )$ to another vector space $𝒲=(W,T,+,\cdot )$. As before, the action of such functions on linear combinations is of special interest.

Definition. (Linear Mapping) . A function $𝒇:V\to W$, from vector space $𝒱=(V,S,+,\cdot )$ to vector space $𝒲=(W,S,\oplus ,\odot )$ is called a linear mapping if for any two vectors $𝒖,𝒗\in V$ and any two scalars $a,b\in S$

The image of a linear combination $a𝒖+b𝒗$ through a linear mapping is another linear combination $a𝒇\left(𝒖\right)+b𝒇\left(𝒗\right)$, and linear mappings are said to preserve the structure of a vector space, and called homomorphisms in mathematics. The codomain of a linear mapping might be the same as the domain in which case the mapping is said to be an endomorphism.

Matrix-vector multiplication has been introduced as a concise way to specify a linear combination

with ${𝒂}_1,\dots ,{𝒂}_n$ the columns of the matrix, $𝑨=\left[\begin{array}{llll}{𝒂}_1& {𝒂}_2& \dots & {𝒂}_n\end{array}\right]$. This is a linear mapping between the real spaces ${ℛ}_m$, ${ℛ}_n$, $𝒇:{ℝ}^m\to {ℝ}^n$, and indeed any linear mapping between real spaces can be given as a matrix-vector product.

2.Measurements

Vectors within the real space ${ℛ}_m$ can be completely specified by $m$ real numbers, even though $m$ is large in many realistic applications. A vector within ${𝒞}^0\left(ℝ\right)$, i.e., a continuous function defined on the reals, cannot be so specified since it would require an infinite, non-countable listing of function values. In either case, the task of describing the elements of a vector space $𝒱=(V,S,+,\cdot )$ by simpler means arises. Within data science this leads to classification problems in accordance with some relevant criteria.

2.1.Equivalence classes

Many classification criteria are scalars, defined as a scalar-valued function $f:𝒱\to S$ on a vector space, $𝒱=(V,S,+,\cdot )$. The most common criteria are inspired by experience with Euclidean space. In a Euclidean-Cartesian model $({ℝ}^2,ℝ,+,\cdot )$ of the geometry of a plane $\Pi$, a point $O\in \Pi$ is arbitrarily chosen to correspond to the zero vector $𝟎={\left[\begin{array}{ll}0& 0\end{array}\right]}^T$, along with two preferred vectors ${𝒆}_1,{𝒆}_2$ grouped together into the identity matrix $𝑰$. The position of a point $X\in \Pi$ with respect to $O$ is given by the linear combination

Several possible classifications of points in the plane are depicted in Figure 2: lines, squares, circles. Intuitively, each choice separates the plane into subsets, and a given point in the plane belongs to just one in the chosen family of subsets. A more precise characterization is given by the concept of a partition of a set.

Definition. (Partition) . A partition of a set is a grouping of its elements into non-empty subsets such that every element is included in exactly one subset.

In precise mathematical terms, a partition of set $S$ is $P=\{S_i|S_i\subset P,S_i\ne \varnothing .,i\in I\}$ such that $\forall x\in S$, $\exists !j\in I$ for which $x\in S_j$. Since there is only one set ($\exists !$ signifies “exists and is unique”) to which some given $x\in S$ belongs, the subsets $S_i$ of the partition $P$ are disjoint, $i\ne j\Rightarrow S_i\cap S_j=\varnothing$. The subsets $S_i$ are labeled by $i$ within some index set $I$. The index set might be a subset of the naturals, $I\subset \mathbb{N}$ in which case the partition is countable, possibly finite. The partitions of the plane suggested by Figure 2 are however indexed by a real-valued label, $i\in ℝ$ with $I\subset ℝ$.

A technique which is often used to generate a partition of a vector space $𝒱=(V,S,+,\cdot )$ is to define an equivalence relation between vectors, $H\subseteq V\times V$. For some element $𝒖\in V$, the equivalence class of $𝒖$ is defined as all vectors $𝒗$ that are equivalent to $𝒖$, $\{𝒗|.(𝒖,𝒗)\in H\}$. The set of equivalence classes of is called the quotient set and denoted as $V/H$, and the quotient set is a partition of $V$. Figure 2 depicts four different partitions of the plane. These can be interpreted geometrically, such as parallel lines or distance from the origin. With wider implications for linear algebra, the partitions can also be given in terms of classification criteria specified by functions.

Figure 2. Equivalence classes within the plane

2.2.Norms

The partition of ${ℝ}^2$ by circles from Figure 2 is familiar; the equivalence classes are sets of points whose position vector has the same size, $\{𝒙={\left[\begin{array}{ll}{x}_1& x_2\end{array}\right]}^T|{(x_1^2+x_2^2)}^{1/2}=r.\}$, or is at the same distance from the origin. Note that familiarity with Euclidean geometry should not obscure the fact that some other concept of distance might be induced by the data. A simple example is statement of walking distance in terms of city blocks, in which the distance from a starting point to an address $x_1=3$ blocks east and $x_2=4$ blocks north is $x_1+x_2=7$ city blocks, not the Euclidean distance ${(x_1^2+x_2^2)}^{1/2}=5$ since one cannot walk through the buildings occupying a city block.

The above observations lead to the mathematical concept of a norm as a tool to evaluate vector magnitude. Recall that a vector space is specified by two sets and two operations, $𝒱=(V,S,+,\cdot )$, and the behavior of a norm with respect to each of these components must be defined. The desired behavior includes the following properties and formal definition.

Unique value

The magnitude of a vector $𝒗\in V$ should be a unique scalar, requiring the definition of a function. The scalar could have irrational values and should allow ordering of vectors by size, so the function should be from $V$ to $ℝ$, $f:V\to ℝ$. On the real line the point at coordinate $x$ is at distance $\left|x\right|$ from the origin, and to mimic this usage the norm of $𝒗\in V$ is denoted as $||𝒗||$, leading to the definition of a function $||||:V\to {ℝ}_{+}$, ${ℝ}_{+}=\left\{a|a\in ℝ,a⩾0.\right\}$.

Null vector case

Provision must be made for the only distinguished element of $V$, the null vector $𝟎$. It is natural to associate the null vector with the null scalar element, $||𝟎||=0$. A crucial additional property is also imposed namely that the null vector is the only vector whose norm is zero, $||𝒗||=0\Rightarrow 𝒗=𝟎$. From knowledge of a single scalar value, an entire vector can be determined. This property arises at key junctures in linear algebra, notably in providing a link to another branch of mathematics known as analysis, and is needed to establish the fundamental theorem of linear algbera or the singular value decomposition encountered later.

Scaling

Transfer of the scaling operation $𝒗=a𝒖$ property leads to imposing $||𝒗||=\left|a\right|||𝒖||$. This property ensures commensurability of vectors, meaning that the magnitude of vector $𝒗$ can be expressed as a multiple of some standard vector magnitude $||𝒖||$.

Vector addition

Position vectors from the origin to coordinates $x,y>0$ on the real line can be added and $|x+y|=\left|x\right|+\left|y\right|$. If however the position vectors point in different directions, $x>0$, $y<0$, then $|x+y|<\left|x\right|+\left|y\right|$. For a general vector space the analogous property is known as the triangle inequality, $||𝒖+𝒗||⩽||𝒖||+||𝒗||$ for $𝒖,𝒗\in V$.

Definition. (Norm) . A norm on the vector space $𝒱=(V,S,+,\cdot )$ is a function $||||:V\to {ℝ}_{+}$ that for $𝒖,𝒗\in V$, $a\in S$ satisfies:

Note that the norm is a functional, but the triangle inequality implies that it is not generally a linear functional. Returning to Figure 2, consider the functions $f_i:{ℝ}^2\to {ℝ}_{+}$ defined for $𝒙={\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]}^T$ through values

Sets of constant value of the above functions are also equivalence classes induced by the equivalence relations $E_i$ for $i=1,2,3,4$.

1. $f_1\left(𝒙\right)=c\Rightarrow \left|x_1\right|=c$, $E_1=\left\{(𝒙,𝒚)|f_1\left(𝒙\right)=f_1\left(𝒚\right)\iff \left|x_1\right|=\left|y_1\right|.\right\}\subseteq {ℝ}^2\times {ℝ}^2$;

2. $f_2\left(𝒙\right)=c\Rightarrow \left|x_2\right|=c$, $E_2=\left\{(𝒙,𝒚)|f_2\left(𝒙\right)=f_2\left(𝒚\right)\iff \left|x_2\right|=\left|y_2\right|.\right\}\subseteq {ℝ}^2\times {ℝ}^2$;

3. $f_3\left(𝒙\right)=c\Rightarrow \left|x_1\right|+\left|x_2\right|=c$, $E_3=\left\{(𝒙,𝒚)|f_3\left(𝒙\right)=f_3\left(𝒚\right)\iff \left|x_1\right|+\left|x_2\right|=\left|y_1\right|+\left|y_2\right|.\right\}\subseteq {ℝ}^2\times {ℝ}^2$;

4. $f_4\left(𝒙\right)=c\Rightarrow {({\left|x_1\right|}^2+{\left|x_2\right|}^2)}^{1/2}=c$, $E_4=\left\{(𝒙,𝒚)|f_4\left(𝒙\right)=f_4\left(𝒚\right)\iff {({\left|x_1\right|}^2+{\left|x_2\right|}^2)}^{1/2}={({\left|y_1\right|}^2+{\left|y_2\right|}^2)}^{1/2}.\right\}\subseteq {ℝ}^2\times {ℝ}^2$.

These equivalence classes correspond to the vertical lines, horizontal lines, squares, and circles of Figure 2. Not all of the functions $f_i$ are norms since $f_1\left(𝒙\right)$ is zero for the non-null vector $𝒙={\left[\begin{array}{cc}0& 1\end{array}\right]}^T$, and $f_2\left(𝒙\right)$is zero for the non-null vector $𝒙={\left[\begin{array}{cc}1& 0\end{array}\right]}^T$. The functions $f_3$ and $f_4$ are indeed norms, and specific cases of the following general norm.

Definition. ($p$-Norm in ${ℛ}_m$) . The $p$-norm on the real vector space ${ℛ}_m=({ℝ}^m,ℝ,+,\cdot )$ for $p⩾1$ is the function ${||||}_p:V\to {ℝ}_{+}$ with values ${||𝒙||}_p={({\left|x_1\right|}^p+{\left|x_2\right|}^p+\cdots +{\left|x_m\right|}^p)}^{1/p}$, or

Denote by $x_i$ the largest component in absolute value of $𝒙\in {ℝ}^m$. As $p$ increases, ${\left|x_i\right|}^p$ becomes dominant with respect to all other terms in the sum suggesting the definition of an inf-norm by

This also works for vectors with equal components, since the fact that the number of components is finite while $p\to \infty$ can be used as exemplified for $𝒙={\left[\begin{array}{cccc}a& a& \dots & a\end{array}\right]}^T$, by ${||𝒙||}_p={(m{\left|a\right|}^p)}^{1/p}=m^{1/p}\left|a\right|$, with $m^{1/p}\to 1$.

Note that the Euclidean norm corresponds to $p=2$, and is often called the $2$-norm. The analogy between vectors and functions can be exploited to also define a $p$-norm for ${𝒞}^0[a,b]=(C\left([a,b]\right),ℝ,+,\cdot )$ , the vector space of continuous functions defined on $[a,b]$.

Definition. ($p$-Norm in ${𝒞}^0[a,b]$) . The $p$-norm on the vector space of continuous functions $$${𝒞}^0[a,b]$ for $p⩾1$ is the function ${||||}_p:V\to {ℝ}_{+}$ with values

The integration operation ${\int }_a^b$ can be intuitively interpreted as the value of the sum ${\sum }_{i=1}^m$ from equation (4) for very large $m$ and very closely spaced evaluation points of the function $f\left(x_i\right)$, for instance $|x_{i+1}-x_i|=(b-a)/m$. An inf-norm can also be define for continuous functions by

where sup, the supremum operation can be intuitively understood as the generalization of the max operation over the countable set $\{1,2,\dots ,m\}$ to the uncountable set $[a,b]$.

Figure 3. Regions within ${ℝ}^2$ for which ${||𝒙||}_p⩽1$, for $p=1,2,3,\infty$.

∴ 

x=[1 1 1]; [norm(x) sqrt(3.0)]

∴ 

m=9; x=ones(m); [norm(x) sqrt(m)]

∴ 

m=4; x=ones(m); [norm(x,1) m]

∴ 

[norm(x,1) norm(x,2) norm(x,4) norm(x,8) norm(x,16) norm(x,Inf)]

∴ 

2.3.Inner product

Norms are functionals that define what is meant by the size of a vector, but are not linear. Even in the simplest case of the real line, the linearity relation $|x+y|=\left|x\right|+\left|y\right|$ is not verified for $x>0$, $y<0$. Nor do norms characterize the familiar geometric concept of orientation of a vector. A particularly important orientation from Euclidean geometry is orthogonality between two vectors. Another function is required, but before a formal definition some intuitive understanding is sought by considering vectors and functionals in the plane, as depicted in Figure 4. Consider a position vector $𝒙={\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]}^T\in {ℝ}^2$ and the previously-encountered linear functionals

The $x_1$ component of the vector $𝒙$ can be thought of as the number of level sets of $f_1$ times it crosses; similarly for the $x_2$ component. A convenient labeling of level sets is by their normal vectors. The level sets of $f_1$ have normal ${𝒆}_1^T=\left[\begin{array}{cc}1& 0\end{array}\right]$, and those of $f_2$ have normal vector ${𝒆}_2^T=\left[\begin{array}{cc}0& 1\end{array}\right]$. Both of these can be thought of as matrices with two columns, each containing a single component. The products of these matrices with the vector $𝒙$ gives the value of the functionals $f_1,f_2$

Figure 4. Euclidean space $E_2$ and its dual $E_2^{\ast }$.

In general, any linear functional $f$ defined on the real space ${ℛ}_m$ can be labeled by a vector

and evaluated through the matrix-vector product $f\left(𝒙\right)={𝒂}^T𝒙$. This suggests the definition of another function $s:{ℝ}^m\times {ℝ}^m\to ℝ$,

The function $s$ is called an inner product, has two vector arguments from which a matrix-vector product is formed and returns a scalar value, hence is also called a scalar product. The definition from an Euclidean space can be extended to general vector spaces. For now, consider the field of scalars to be the reals $S=ℝ$.

Definition. (Inner Product) . An inner product in the vector space $𝒱=(V,ℝ,+,\cdot )$ is a function $s:V\times V\to ℝ$ with properties

The inner product $s(𝒂,𝒙)$ returns the number of level sets of the functional labeled by $𝒂$ crossed by the vector $𝒙$, and this interpretation underlies many applications in the sciences as in the gravitational field example above. Inner products also provide a procedure to evaluate geometrical quantities and relationships.

Vector norm

In ${ℛ}_m$ the number of level sets of the functional labeled by $𝒙$ crossed by $𝒙$ itself is identical to the square of the 2-norm

${\displaystyle s(𝒙,𝒙)={𝒙}^T𝒙={||𝒙||}_2^2.}$

In general, the square root of $s(𝒙,𝒙)$ satisfies the properties of a norm, and is called the norm induced by an inner product

${\displaystyle ||𝒙||=s{(𝒙,𝒙)}^{1/2}.}$

A real space together with the scalar product $s(𝒙,𝒚)={𝒙}^T𝒚$ and induced norm $||𝒙||=s{(𝒙,𝒙)}^{1/2}$ defines an Euclidean vector space ${ℰ}_m$.

Orientation

In ${ℰ}_2$ the point specified by polar coordinates $(r,\theta )$ has the Cartesian coordinates $x_1=r\cos \theta$, $x_2=r\sin \theta$, and position vector $𝒙={\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]}^T$. The inner product

${\displaystyle {𝒆}_1^T𝒙=\left[\begin{array}{cc}1& 0\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=1\cdot x_1+0\cdot x_2=r\cos \theta ,}$

is seen to contain information on the relative orientation of $𝒙$ with respect to ${𝒆}_1$. In general, the angle $\theta$ between two vectors $𝒙,𝒚$ with any vector space with a scalar product can be defined by

${\displaystyle \cos \theta =\frac{s(𝒙,𝒚)}{{[s(𝒙,𝒙)s(𝒚,𝒚)]}^{1/2}}=\frac{s(𝒙,𝒚)}{||𝒙||||𝒚||},}$

which becomes

${\displaystyle \cos \theta =\frac{{𝒙}^T𝒚}{||𝒙||||𝒚||},}$

in a Euclidean space, $𝒙,𝒚\in {ℝ}^m$.

Orthogonality

In ${ℰ}_2$ two vectors are orthogonal if the angle between them is such that $\cos \theta =0$, and this can be extended to an arbitrary vector space $𝒱=(V,ℝ,+,\cdot )$ with a scalar product by stating that $𝒙,𝒚\in V$ are orthogonal if $s(𝒙,𝒚)=0$. In ${ℰ}_m$ vectors $𝒙,𝒚\in {ℝ}^m$ are orthogonal if ${𝒙}^T𝒚=0$.

3.Linear mapping composition

3.1.Matrix-matrix product

From two functions $f:A\to B$ and $g:B\to C$, a composite function, $h=g\circ f$, $h:A\to C$ is defined by

Consider linear mappings between Euclidean spaces $𝒇:{ℝ}^n\to {ℝ}^m$, $𝒈:{ℝ}^m\to {ℝ}^p$. Recall that linear mappings between Euclidean spaces are expressed as matrix vector multiplication

The composite function $𝒉=𝒈\circ 𝒇$ is $𝒉:{ℝ}^n\to {ℝ}^p$, defined by

Note that the intemediate vector $𝒖=𝑨𝒙$ is subsequently multiplied by the matrix $𝑩$. The composite function $𝒉$ is itself a linear mapping

so it also can be expressed a matrix-vector multiplication

Using the above, $𝑪$ is defined as the product of matrix $𝑩$ with matrix $𝑨$

The columns of $𝑪$ can be determined from those of $𝑨$ by considering the action of $𝒉$ on the the column vectors of the identity matrix $𝑰=\left[\begin{array}{cccc}{𝒆}_1& {𝒆}_2& \dots & {𝒆}_n\end{array}\right]\in {ℝ}^{n\times n}$. First, note that

The above can be repeated for the matrix $𝑪=\left[\begin{array}{cccc}{𝒄}_1& {𝒄}_2& \dots & {𝒄}_n\end{array}\right]$ giving

Combining the above equations leads to ${𝒄}_j=𝑩{𝒂}_j$, or

∴ 

a1=[1; 2]; a2=[3; 4]; A=[a1 a2]

∴ 

b1=[-1; 1; 3]; b2=[2; -2; 3]; B=[b1 b2]

∴ 

C=B*A

∴ 

c1=B*a1; c2=B*a2; [c1 c2]

∴