"Maymester MATH547 Linear Algebra for Applications in Data Science"

1.MATH547 Homework 2

Topic:	Math@UNC environment
Post date:	May 18, 2020
Due date:	May 19, 2020

1.1.Background

This homework investigates the matrix fundamental spaces, and the concept of linear dependence and independence.

1.2.Theoretical questions

Is the zero vector a linear combination of any non-empty set of vectors?
If $S \subseteq V$ , with $V$ a vector space then span( $S$ ) equals the intersection of all subspaces of $V$ that contain $S$ .
Can a vector space have more than one basis?
Must a vector space have a finite basis.

1.3.Fundamental spaces and linear dependence in image processing

1.3.1.Data input

Images are often processed using techniques from linear algebra. In this assignment a database of facial images from MIT will be used to investigate the fundamental vector subspaces associated with a matrix (linear mapping) and concepts of linear dependence. The database is available in a format readily loaded into Octave, with gray-scale images stored as column vectors of a matrix $𝑨 \in ℝ^{m \times n} .$ There are $n$ images, each with $m$ pixels, and the two-dimensional image size in pixels is $p_{x} \times p_{y}$ . The images have been pre-processed to remove non-uniform background illumination, noise, off-centering, different face size, and also been scaled such that the norm of each column vector is equal to one. The resulting images represent common facial features, but may seem distant from the much more detailed processing of visual information that leads a human to recognize a face. After loading the database, a directory is created for this assignment and set as the current directory

octave]

cd /home/student/courses/MATH547ML/data/faces; load faces

octave]

[m,n]=size(A); px=floor(sqrt(m)); py=m/px; disp([m n px py])

16384 99 128 128

octave]

cd /home/student/courses/MATH547ML;

octave]

mkdir homework; cd homework; mkdir hw02; cd hw02

octave]

pwd

ans = /home/student/courses/MATH547ML/homework/hw02

octave]

1.3.2.Utility functions

Define functions to display a facial image, and save an image to a file.

octave]

function shwface(a,px,py,save)
  im=reshape(-a,px,py)'; colormap(gray);
  imagesc(im);
end

octave]

shwface(A(:,51),px,py)

octave]

function savface(a,px,py,fname)
  im=reshape(-a,px,py)'; colormap(gray);
  imagesc(im); print(fname,"-deps");
end

octave]

savface(A(:,1),px,py,"face1");

octave]

savface(A(:,2),px,py,"face2");

octave]

savface(A(:,3),px,py,"face3")

octave]

Figure 1.

1.3.3.Questions to investigate

Functionals to assess data variability.

Define an average of the data

𝒃 = \frac{1}{n} \sum_{j = 1}^{n} 𝒂_{j}

, with

𝑨 = [\begin{array}{cccc} 𝒂_{1} & 𝒂_{2} & \dots & 𝒂_{n} \end{array}]

. Compute the norms

u_{j} = {|| 𝒂_{j} - 𝒃 ||}_{p}

and angle cosines

c_{j} = 𝒃^{T} 𝒂_{j} / || 𝒃 ||

(recall that preprocessing ensured

|| 𝒂_{j} || = 1

). Plot the results and comment on information provided by different norms. Assess data variability.

Functional to assess data redundancy.

If vectors

𝒙, 𝒚 \in ℝ^{m}

are colinear

𝒙^{T} 𝒚 / || 𝒙 || / || 𝒚 || = \pm 1

, one vector can be recovered from the other through a scaling operation

𝒙 = a 𝒚

, and the data is redundant. If the vectors are orthogonal,

𝒙^{T} 𝒚 = 0

, data in

𝒙

is independent of data in

𝒚

. Assess whether facial data in

𝑨

is independent by computing the angle cosines

s_{j k} = 𝒂_{j}^{T} 𝒂_{k}

Column space to assess data sampling.

The Octave orth function returns an orthognal set of vectors that span the column space of a matrix given as its argument. Display a few such vectors and comment on their utility by comparison to the column vectors of

𝑨

Left null space to assess missing data.

The Octave null function returns an orthognal set of vectors that span the null space of a matrix given as its argument. Display a few vectors of

N (𝑨^{T})

and comment on what data cannot be obtained by linear combination of columns of

𝑨

Linear combinations.

Define a few new faces from linear combinations of

p = 2, 3, \dots, 8

columns of

𝑨

Linear dependence.

Is the facial data within

𝑨

linearly dependent or independent?