1.MATH547 Homework 1

1.1.Background

This homework is meant as a tutorial on matrix computations, in particular:

• vector addition and scaling;

• vector norms;

• inner products;

• matrix-vector products;

• matrix-matrix products.

1.2.Theoretical questions

1.2.1.Vector addition and scaling

Problem

Consider $n$ random walks on the real line with $m$ unit steps starting from the origin. What is the mean distance and mean squared distance from the origin?

Answer

Denote distance by $d$, and the random walk steps as $u_1,u_2,\dots$

${\displaystyle d=u_1+u_2+\cdots +u_m=\sum _{i=1}^m{u}_i,u_i\in \{-1,1\}.}$

Denote by $\mu \left(d\right)$ the average distance over $n$ walks which is the sum of the average displacement in each step

${\displaystyle \mu \left(d\right)=\sum _{i=1}^m\mu \left(u_i\right)}$

Assuming that both left and right steps are equiprobable, $\mu \left(u_i\right)=0$ and $\mu \left(d\right)=0$.

The squared distance is

${\displaystyle d^2={(u_1+u_2+\cdots +u_m)}^2=\sum _{i=1}^m{u}_i^2+2\sum _{i=1}^m\sum _{j=i+1}^m{u}_i{u}_j}$

The average is

${\displaystyle \mu \left(d^2\right)=\sum _{i=1}^m\mu \left(u_i^2\right)+2\sum _{i=1}^m\sum _{j=i+1}^m\mu (u_i{u}_j).}$

Since $u_i^2=1$ the average is $\mu \left(u_i^2\right)=1$. The possible values of $u_i{u}_j=\pm 1$ with equal probability $\mu \left(u_i{u}_j\right)=0$. The average squared distance is

${\displaystyle \mu \left(d^2\right)=m.}$ ${\displaystyle 𝑨\in {ℝ}^{m\times n},m\gg 1,n}$

octave]

m=100; n=500; U=2*(rand(m,n)<0.5)-1; mean(mean(U))

ans = -0.0063600

octave]

function md=meand(m,n) U=2*(rand(m,n)<0.5)-1; md=mean(mean(U)); end

octave]

Nmax=1000; md=zeros(50,1); for k=1:50 md(k) = meand(m,100*k); end;

octave]

plot(100:100:5000,abs(md),'o')

octave]

pwd

ans = /home/student/courses/MATH547ML

octave]

mkdir homework

ans = 1

octave]

cd homework

octave]

pwd

ans = /home/student/courses/MATH547ML/homework

octave]

mkdir hw01;

octave]

cd hw01

octave]

print -deps hw01fig01.eps

octave]

pwd

ans = /home/student/courses/MATH547ML/homework/hw01

octave]

ls

hw01fig01.eps untitled.ofig

Figure 1. Experiment on statsitc to evaluate mean of random walk distance $\mu \left(d\right)$

1.2.2.Norms in $E_m$

Problem

Show that for $a_1,a_2>0$, $f:{ℝ}^2\to {ℝ}_{+}$ defined by

${\displaystyle f\left(𝒙\right)=a_1{x}_1^2+a_2{x}_2^2,}$

is a norm in $E_2$.

Answer

Let $y_1=\sqrt{a_1}{x}_1$, $y_2=\sqrt{a_2}{x}_2$, or in vector form $f\left(𝒙\right)=g\left(𝒚\right)=y_1^2+y_2^2$.

${\displaystyle 𝒚=𝑨𝒙,𝑨=\left[\begin{array}{cc}\sqrt{a_1}& 0\\ 0& \sqrt{a_2}\end{array}\right],}$

Since $g\left(𝒚\right)$ does not satisfy scaling property $||a𝒖||=\left|a\right|||𝒖||$, neither if $f$ a norm. However $f=\sqrt{g}$ would be.

1.2.3.Norms in $E_m$

Problem

Show that for $a_1,a_2>0$, $f:{ℝ}^2\to {ℝ}_{+}$ defined by

${\displaystyle f\left(𝒙\right)=a_1{x}_1^2-a_2{x}_2^2}$

is not a norm in $E_2$.

Answer

No $f\left(𝒙\right)=0$ for $𝒙=(1,\sqrt{a_1/a_2})\ne 𝟎$

1.2.4.Inner product in $E_m$

Problem

Show that for $a_1,a_2>0$, $s:{ℝ}^2\times {ℝ}^2\to ℝ$ defined by

${\displaystyle s(𝒙,𝒚)=a_1{x}_1{y}_1+a_2{x}_2{y}_2,}$

is a scalar product in $E_2$.

Answer

Recall that

${\displaystyle t(𝒙,𝒚)=\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2\end{array}\right]=x_1{y}_1+x_2{y}_2}$

is a scalar product. From

${\displaystyle s(𝒙,𝒚)=t(𝑨𝒙,𝒚)=t(𝒙,𝑨𝒚),\mathrm{with}𝑨=\left[\begin{array}{cc}{a}_1& 0\\ 0& a_2\end{array}\right],}$

and fact that $t$ is a scalar product, deduce $s$ is also a scalar product.

1.2.5.Inner product in $E_m$

Problem

Show that for $a_1,a_2>0$, $s:{ℝ}^2\times {ℝ}^2\to ℝ$ defined by

${\displaystyle s(𝒙,𝒚)=a_1{x}_1{y}_1-a_2{x}_2{y}_2,}$

is not a scalar product in $E_2$.

Answer

From $s(𝒙,𝒙)=a_1{x}_1^2-a_2{x}_2^2$ notice that $s(𝒙,𝒙)$ can be negative, hence $s$ is not a scalar product.

1.2.6.Matrix-matrix products

Problem

Find examples of matrices for which $𝑨𝑩=𝑩𝑨$, and $𝑪𝑫\ne 𝑫𝑪$.

Answer

octave]

A=[2 1; 1 2]; B=[2 0; 0 2]; A*B-B*A

ans =

0 0

0 0

octave]

C=[2 1; -1 1]; D=[0 -2; 1 -1]; C*D-D*C

ans =

-1 -3

-2 1

octave]

Octave]

1.3.Linear functionals and mappings in analysis of EEG data

Electroencephalograms (EEGs) are recordings of the electric potential on cranium skin. Research on brain activity uses EEGs to determine specific activity patterns in the brain. For example, epileptic seizures have a distinctive EEG signature. EEG data can be loaded from the course data directory.

octave] 

load /home/student/courses/MATH547ML/data/eeg/eeg;

octave] 

data=EEG.data'; [m n]=size(data); disp([m n]);

octave] 

There are $n=32$ sensor recordings with $m=30504$ components each. A first common operation is scaling of the data.

octave] 

pdata=data./max(data)+meshgrid(0:n-1,0:m-1);

octave] 

hold on;
for j=1:n
  plot(pdata(:,j));
end;
hold off;

octave] 

cd /home/student/courses/MATH547ML/homework/hw01

octave] 

print -deps eeg.eps;

octave] 

Figure 2.

1.3.1.Functional relationships in the data

Over some time subinterval can the data from a sensor be expressed as a function of data from another sensor?

octave] 

cd /home/student/courses/MATH547ML/homework/hw01;

octave] 

[m,n]=size(data); 

octave] 

mt=5000:5005; plot(data(mt,1),data(mt,2),'-o'); print -deps hw01Fig02a.eps

octave] 

mt=6000:6005; plot(data(mt,1),data(mt,2),'-o'); print -deps hw01Fig02b.eps

octave] 

mt=7000:7005; plot(data(mt,1),data(mt,2),'-o'); print -deps hw01Fig02c.eps

octave] 

1.3.2.Data magnitude (norm)

Partition sensor data into time subintervals. Over each subinterval, evaluate the $p$-norm of centered data, for $p=1,2,3$, $p\to \infty$. Plot the $p$-norms over the entire time history.

octave] 

mud=mean(data);

octave] 

cdata=data-mud;

octave] 

m1=1000; m2=1100; cdata(m1:m2,:)

octave] 

function nrmd=normd(m1,m2,d)
  [m,n]=size(d);
  nrmd=zeros(4,n);
  for k=1:n
    nrmd(1,k)=norm(d(m1:m2,k),1);
    nrmd(2,k)=norm(d(m1:m2,k),2);
    nrmd(3,k)=norm(d(m1:m2,k),3);
    nrmd(4,k)=norm(d(m1:m2,k),Inf);
  end;
end;

octave] 

tnrm=zeros(100,4,n);
for j=1:100
  tnrm(j,:,:) = normd(100*j,100*(j+1),cdata);
end;

octave] 

it=1:100; plot(it,tnrm(:,1,1),'r',it,tnrm(:,2,1),'k',it,tnrm(:,3,1),'g',it,tnrm(:,4,1),'b')

octave] 

plot(it,tnrm(:,1,1),'r',it,tnrm(:,1,2),'k',it,tnrm(:,1,3),'g')

octave] 

figure(); plot(it,tnrm(:,2,1),'r',it,tnrm(:,2,2),'k',it,tnrm(:,2,3),'g')

octave] 

figure();

octave] 

jt=8900:9200; plot(jt,cdata(jt,1),'r',jt,cdata(jt,2),'k',jt,cdata(jt,3),'g')

octave] 

figure(1); pwd

octave] 

print -deps hw01fig03.eps

octave] 

Figure 4.

1.3.3.Orthogonality of sensor data (inner product)

Over some time interval, determine the angle between sensor data. Is sensor data orthogonal? Plot the angle between sensor data over the entire time history.

octave] 

function angd=angld(m1,m2,n1,d)
  [m,n]=size(d);
  angd=zeros(n1,n);
  for k=1:n
    angd(k)=d(m1:m2,n1)'*d(m1:m2,k)/norm(d(m1:m2,n1))/norm(d(m1:m2,k));
  end
end;

octave] 

tang=zeros(100,n);
for j=1:100
  tang(j,:) = angld(100*j,100*(j+1),1,cdata);
end;

octave] 

it=1:100; plot(it,tang(:,1),'r',it,tang(:,2),'k',it,tang(:,3),'g')

octave] 

figure(4)

octave] 

plot(tang)

octave] 

1.3.4.Periodic representation of data (linear mapping)

Use template from Lesson01 to construct a representation of the data through periodic functions.

octave] 

ns=5; A=[]; it=100:400; dt=0.1; t=it*dt; t=t'; A = [A ones(size(t))];

octave] 

for k=1:ns
  A = [A sin(k*t) cos(k*t)];
end

octave] 

bt=cdata(it,1);

octave] 

x=A\bt;

octave] 

b=A*x;

octave] 

plot(t,bt,'ok',t,b,'r');

octave] 

print -deps hw01fig04.eps

octave] 

Figure 5. Data from sensor 1 is not accurately captured by a trigonometric series.