While working on computational aspects in P01, homework will concentrate on analytical properties.

1Track 1

1. Let $\lambda ,\mu$ be distinct eigenvalues of $𝑨$ symmetric, i.e., $\lambda \ne \mu$, $𝑨={𝑨}^T$, $𝑨𝒙=\lambda 𝒙$, $𝑨𝒚=\mu 𝒚$. Show that $𝒙,𝒚$ are orthogonal.

   Solution. Compute ${(𝑨𝒙)}^T={𝒙}^T{𝑨}^T={𝒙}^T𝑨=\lambda {𝒙}^T\Rightarrow {𝒙}^T𝑨𝒚=\lambda {𝒙}^T𝒚$. Multiply $𝑨𝒚=\mu 𝒚$ with ${𝒙}^T$ to obtain ${𝒙}^T𝑨𝒚=\mu {𝒙}^T𝒚$. Subtracting gives

   ${\displaystyle 0=(\lambda -\mu ){𝒙}^T𝒚\Rightarrow {𝒙}^T𝒚=0}$

   since $\lambda \ne \mu$.

2. Consider

   ${\displaystyle 𝑨=\left[\begin{array}{lll}-i& -i& 0\\ -i& i& 0\\ 0& 0& 1\end{array}\right].}$

   1. Is $𝑨$ normal?

   2. Is $𝑨$ self-adjoint?

   3. Is $𝑨$ unitary?

   4. Find the eigenvalues and eigenvectors of $𝑨$.

   Solution.

   1. Observe block structure

      ${\displaystyle 𝑨=\left[\begin{array}{ll}𝑩& 0\\ 0& 1\end{array}\right],}$

      such that $𝑨$ is normal if $𝑩$ normal. Note that

      ${\displaystyle {𝑩}^{\ast }=\left[\begin{array}{ll}i& i\\ i& -i\end{array}\right]=-𝑩,}$

      such that $𝑩{𝑩}^{\ast }-{𝑩}^{\ast }𝑩=-{𝑩}^2+{𝑩}^2=𝟎$, hence $𝑩$ is normal.

      Note: Always look for any special properties of a matrix before attempting calculations.

   2. No, since ${𝑩}^{\ast }=-𝑩$.

   3. No, since $𝑩{𝑩}^{\ast }=-{𝑩}^2$.

   4. Block structure of $𝑨$ gives ${\lambda }_3=1$, ${𝒙}_3={𝒆}_3$, with remaining eigenvectors obtained by those of $𝑩$

      ${\displaystyle 𝑩𝒒=\mu 𝒒\Rightarrow \mu =\pm i\sqrt{2},𝑸=\left[\begin{array}{ll}1& -1\\ \sqrt{2}-1& \sqrt{2}+1\end{array}\right]\Rightarrow }$ ${\displaystyle 𝑿=\left[\begin{array}{lll}1& -1& 0\\ \sqrt{2}-1& \sqrt{2}+1& 0\\ 0& 0& 1\end{array}\right],𝚲=\mathrm{diag}(i\sqrt{2},-i\sqrt{2},1).}$

3. Find the eigenvalues and eigenvectors of the matrix $𝑹\in {ℝ}^{3\times 3}$ expressing rotation around the $z-$axis (unit vector ${𝒆}_3={\left[\begin{array}{lll}0& 0& 1\end{array}\right]}^T$).

   Solution. The rotation map $𝒇:{ℝ}^3\to {ℝ}^3$ is linear

   ${\displaystyle 𝒇(\alpha 𝒖+\beta 𝒗)=\alpha 𝒇\left(𝒖\right)+\beta 𝒇\left(𝒗\right)}$

   and the matrix $𝑹$ representing this rotation has columns given by the image of a basis set

   ${\displaystyle 𝑹=\left[\begin{array}{lll}𝒇\left({𝒆}_1\right)& 𝒇\left({𝒆}_2\right)& 𝒇\left({𝒆}_3\right)\end{array}\right]=\left[\begin{array}{lll}\cos \theta {𝒆}_1+\sin \theta {𝒆}_2& -\sin \theta {𝒆}_1+\cos \theta {𝒆}_2& {𝒆}_3\end{array}\right]=𝑰\left[\begin{array}{lll}\cos \theta & -\sin \theta & 0\\ \sin \theta & \cos \theta & 0\\ 0& 0& 1\end{array}\right].}$

   One eigenpair is

   ${\displaystyle {\lambda }_3=1,{𝒙}_3={𝒆}_3,}$

   since $𝑹{𝒆}_3={𝒆}_3$; a rotation does not change a vector along the axis of rotation. The characteristic polynomial ${𝒑}_3\left(\lambda \right)=(\lambda -1)({\lambda }^2-2\cos \theta +1)$ also has roots ${\lambda }_1=e^{-i\theta }$, ${\lambda }_2=e^{i\theta }$ with associated eigenvectors

   ${\displaystyle {𝒙}_1=\left[\begin{array}{l}1\\ i\\ 0\end{array}\right]={𝒆}_1+i{𝒆}_2,{𝒙}_2=\left[\begin{array}{l}i\\ 1\\ 0\end{array}\right]=i{𝒆}_1+{𝒆}_2.}$

   The above can be verified in an insightful manner by recalling that rotation does not change vector lengths (isometric mapping) hence $𝑹$ is orthogonal, and ${𝑹}^{-1}={𝑹}^T$, such that the eigenvalue relation $𝑹𝒙=\lambda 𝒙$ leads to

   ${\displaystyle 𝒙=\lambda {𝑹}^{-1}𝒙=\lambda {𝑹}^T𝒙.}$

   Verify that ${𝒙}_1$ is an eigenvector by computing,

   ${\displaystyle {\lambda }_1{𝑹}^T{𝒙}_1=e^{-i\theta }\left[\begin{array}{l}\cos \theta {𝒆}_1^T+\sin \theta {𝒆}_2^T\\ -\sin \theta {𝒆}_1^T+\cos \theta {𝒆}_2^T\\ {𝒆}_3^T\end{array}\right]({𝒆}_1+i{𝒆}_2)=e^{-i\theta }\left[\begin{array}{l}\cos \theta +i\sin \theta \\ -\sin \theta +i\cos \theta \\ 0\end{array}\right]=\left[\begin{array}{l}1\\ i\\ 0\end{array}\right]={𝒙}_1.}$

   A similar verification can be done for ${𝒙}_2$. Note how thinking of $𝑹$ as a collection of column vectors leads to a concise, elegant solution of the eigenvalue problem. Also, notice that for rotation within the $x_1{x}_2$ plane the eigenvectors are outside the real $x_1{x}_2$ plane and that all eigenvalues are of unit absolute value since the rotation is isometric.

4. Find the eigenvalues and eigenvectors of the matrix $𝑺\in {ℝ}^{3\times 3}$ expressing rotation around the axis with unit vector $𝒍=\frac{1}{\sqrt{3}}{\left[\begin{array}{lll}1& 1& 1\end{array}\right]}^T$.

   Solution. This is the same as the above problem, but in a different basis, one in which the axis of rotation is ${\left[\begin{array}{lll}1& 1& 1\end{array}\right]}^T$ instead of ${\left[\begin{array}{lll}0& 0& 1\end{array}\right]}^T$. As before, one eigenpair is ${\lambda }_3=1$, with ${𝒙}_3=𝒍$ along the rotation axis. Above, the other two eigenvectors were found using unit vectors perpendicular to the rotation axis. One can apply Gram-Schmidt to find $𝒋,𝒌$ orthonormal to $𝒍$ or simply observe that

   ${\displaystyle \left[\begin{array}{lll}\sqrt{2}𝒋& \sqrt{6}𝒌& \sqrt{3}𝒍\end{array}\right]=\left[\begin{array}{lll}1& 1& 1\\ -1& 1& 1\\ 0& -2& 1\end{array}\right]}$

   defines an orthogonal matrix $𝑩=\left[\begin{array}{lll}𝒋& 𝒌& 𝒍\end{array}\right]$. Rotating some vector $𝒙$ around the $𝒍$ axis is straightforward in the $𝑩$ basis, so set $𝒙=𝑰𝒙=𝑩𝒖\Rightarrow 𝒖={𝑩}^T𝒙$. Read this to state: the vector $𝒙$ has coordinates $𝒙$ in the $𝑰$ basis, but coordinates $𝒖$ in the $𝑩$ basis. In the $𝑩$ basis the rotation matrix has columns

   ${\displaystyle 𝑺=\left[\begin{array}{lll}𝒇\left(𝒋\right)& 𝒇\left(𝒌\right)& 𝒇\left(𝒍\right)\end{array}\right]=\left[\begin{array}{lll}\cos \theta 𝒋+\sin \theta 𝒌& -\sin \theta 𝒋+\cos \theta 𝒌& 𝒍\end{array}\right]=𝑩\left[\begin{array}{lll}\cos \theta & -\sin \theta & 0\\ \sin \theta & \cos \theta & 0\\ 0& 0& 1\end{array}\right]=𝑩𝑹.}$

   The result of the rotation is

   ${\displaystyle 𝒚=𝑺𝒖=𝑩𝑹{𝑩}^T𝒙,}$

   where $𝑺=𝑩𝑹{𝑩}^T$ is the rotation matrix. The eigendecomposition of $𝑹$ is known from above problem $𝑹=𝑸𝚲{𝑸}^{\ast }$ furnishing the eigendecomposition of $𝑺$

   ${\displaystyle 𝑺=𝑩𝑹{𝑩}^T=𝑩𝑹{𝑩}^{\ast }=𝑩𝑸𝚲{𝑸}^{\ast }{𝑩}^{\ast }=𝑼𝚲{𝑼}^{\ast }.}$

   The eigenvalues of $𝑺$ are the same of those of $𝑹$ ($e^{\pm i\theta }$,1) and the eigenvectors are

   ${\displaystyle 𝑼=𝑩𝑸.}$

5. Compute $\sin (𝑨t)$ for

   ${\displaystyle 𝑨=\left[\begin{array}{ll}3& -9\\ 2& -6\end{array}\right].}$

   Solution. The matrix is singular hence one eigenvalue is ${\lambda }_1=0$ with eigenvector

   ${\displaystyle {𝒙}_1=\left[\begin{array}{l}3\\ 1\end{array}\right].}$

   The other eigenpair is ${\lambda }_2=-3$,

   ${\displaystyle {𝒙}_2=\left[\begin{array}{l}3\\ 2\end{array}\right],}$

   and $𝑨$ with distinct eigenvalues is diagonalizable, $𝑨𝑿=𝑿𝚲\Rightarrow 𝑨=𝑿𝚲{𝑿}^{-1}$. Powers of $𝑨$ are given by

   ${\displaystyle {𝑨}^k=𝑿{𝚲}^k{𝑿}^{-1}.}$

   From the Euler formula $e^{i\theta }=\cos \theta +i\sin \theta$ find

   ${\displaystyle \sin \theta =\frac{e^{i\theta }-e^{-i\theta }}{2i},}$

   which extended to matrix arguments gives

   ${\displaystyle \sin (𝑨t)=\frac{1}{2i}[\exp (it𝑨)-\exp (-it𝑨)].}$

   From the power series ${𝒆}^t=1+t+t^2/2!+\cdots$ obtain

   ${\displaystyle \exp (it𝑨)=𝑰+it𝑨+{(it𝑨)}^2/2!+\cdots =𝑿\exp (it𝚲){𝑿}^{-1},}$

   leading to

   ${\displaystyle \sin (𝑨t)=𝑿\sin (t𝚲){𝑿}^{-1}=\left[\begin{array}{ll}3& 3\\ 1& 2\end{array}\right]\left[\begin{array}{ll}0& 0\\ 0& -\sin \left(3t\right)\end{array}\right]{\left[\begin{array}{ll}3& 3\\ 1& 2\end{array}\right]}^{-1}=\left[\begin{array}{ll}-2& 3\\ -4/3& 2\end{array}\right]\sin \left(3t\right).}$

6. Compute $\cos (𝑨t)$ for

   ${\displaystyle 𝑨=\left[\begin{array}{ll}5& -4\\ 2& -1\end{array}\right].}$

   Solution. As above, from eigendecomposition

   ${\displaystyle 𝑨=𝑿𝚲{𝑿}^{-1}=\left[\begin{array}{ll}2& 1\\ 1& 1\end{array}\right]\left[\begin{array}{ll}3& 0\\ 0& 1\end{array}\right]\left[\begin{array}{ll}1& -1\\ -1& 2\end{array}\right],}$

   obtain

   ${\displaystyle \cos (𝑨t)=𝑿\cos (𝚲t){𝑿}^{-1}=\left[\begin{array}{ll}2& 1\\ 1& 1\end{array}\right]\left[\begin{array}{ll}\cos \left(3t\right)& 0\\ 0& \cos \left(t\right)\end{array}\right]\left[\begin{array}{ll}1& -1\\ -1& 2\end{array}\right]}$

7. Compute the SVD of

   ${\displaystyle 𝑨=\left[\begin{array}{ll}1& -2\\ -3& 6\end{array}\right]}$

   by finding the eigenvalues and eigenvectors of $𝑨{𝑨}^T$, ${𝑨}^T𝑨$.

   Solution. With the SVD $𝑨=𝑼𝚺{𝑽}^T$, the matrix $𝑴=𝑨{𝑨}^T=𝑼{𝚺}^2{𝑼}^T$ has eigendecomposition

   ${\displaystyle 𝑴=\frac{1}{\sqrt{10}}\left[\begin{array}{ll}-1& 3\\ 3& 1\end{array}\right]\left[\begin{array}{ll}50& 0\\ 0& 0\end{array}\right]\frac{1}{\sqrt{10}}\left[\begin{array}{ll}-1& 3\\ 3& 1\end{array}\right],}$

   hence

   ${\displaystyle 𝑼=\frac{1}{\sqrt{10}}\left[\begin{array}{ll}-1& 3\\ 3& 1\end{array}\right],𝚺=\left[\begin{array}{ll}\sqrt{50}& 0\\ 0& 0\end{array}\right].}$

   From $𝑵={𝑨}^T𝑨=𝑽{𝚺}^2{𝑽}^T$ with eigendecomposition

   ${\displaystyle 𝑵=\frac{1}{\sqrt{5}}\left[\begin{array}{ll}-1& 2\\ 2& 1\end{array}\right]\left[\begin{array}{ll}50& 0\\ 0& 0\end{array}\right]\frac{1}{\sqrt{5}}\left[\begin{array}{ll}-1& 2\\ 2& 1\end{array}\right],}$

   obtain

   ${\displaystyle 𝑽=\frac{1}{\sqrt{5}}\left[\begin{array}{ll}-1& 2\\ 2& 1\end{array}\right].}$

8. Find the eigenvalues and eigenvectors of $𝑨\in {ℝ}^{m\times m}$ with elements $a_{ij}=1$ for all $1⩽i,j⩽m$. Hint: start with $m=1,2,3$ and generalize.

   Solution. Note that $\mathrm{rank}\left(𝑨\right)=1$ hence $𝑨$ has ${\lambda }_2=\cdots ={\lambda }_m=0$ an $m-1$ repeated zero root implying

   ${\displaystyle p_{𝑨}\left(\lambda \right)=\det (\lambda 𝑰-𝑨)=\left|\begin{array}{llll}\lambda -1& -1& \dots & -1\\ -1& \lambda -1& \dots & -1\\ ⋮& & & \\ -1& -1& & \lambda -1\end{array}\right|={\lambda }^{m-1}(\lambda -{\lambda }_1).}$

   Adding all rows gives a row of $\lambda -m$ such that ${\lambda }_1=m$ leads to a null determinant with associated eigenvector ${𝒙}_1={\left[\begin{array}{llll}1& 1& \dots & 1\end{array}\right]}^T$. The other eigenvectros are of the form ${𝒙}_j=$${\left[\begin{array}{llllll}1& 0& \dots & -1& \dots & 0\end{array}\right]}^T$ with the one in the $i^{\mathrm{th}}$ position.

2Track 2

1. Prove that $𝑨\in {ℂ}^{m\times m}$ is normal if and only if it has $m$ orthonormal eigenvectors.

   Solution. Apply Schur theorem $𝑨=𝑼𝑻{𝑼}^{\ast }$ such that $𝑨$ normal $𝑨{𝑨}^{\ast }={𝑨}^{\ast }𝑨$ implies $𝑻{𝑻}^{\ast }={𝑻}^{\ast }𝑻$, which implies that $𝑻$ is diagonal, as proven by induction

   ${\displaystyle 𝑻=\left[\begin{array}{ll}t& {𝒛}^{\ast }\\ 𝟎& 𝑺\end{array}\right],𝑻{𝑻}^{\ast }=\left[\begin{array}{ll}t& {𝒛}^{\ast }\\ 𝟎& 𝑺\end{array}\right]\left[\begin{array}{ll}\bar{t}& 𝟎\\ 𝒛& {𝑺}^{\ast }\end{array}\right]=\left[\begin{array}{ll}t\bar{t}+{𝒛}^{\ast }𝒛& {}^{}{𝒛}^{\ast }{𝑺}^{\ast }\\ 𝑺𝒛& 𝑺{𝑺}^{\ast }\end{array}\right],}$

   ${\displaystyle {𝑻}^{\ast }𝑻=\left[\begin{array}{ll}t& 𝟎\\ 𝒛& {𝑺}^{\ast }\end{array}\right]\left[\begin{array}{ll}\bar{t}& {𝒛}^{\ast }\\ 𝟎& 𝑺\end{array}\right]=\left[\begin{array}{ll}t\bar{t}& t{}^{}{𝒛}^{\ast }\\ 𝒛\bar{t}& 𝑺{𝑺}^{\ast }\end{array}\right],}$

   since $t\bar{t}+{𝒛}^{\ast }𝒛=t\bar{t}$ implies $𝒛=𝟎$.

2. Prove that $𝑨\in {ℝ}^{m\times m}$ symmetric has a repeated eigenvalue if and only if it commutes with a non-zero skew-symmetric matrix $𝑩$.

   Solution. $𝑨$ symmetric is normal and has an orthogonal eigendecomposition $𝑨=𝑸𝚲{𝑸}^T$, that states that in the $𝑸$ basis the effect of $𝑨$ is simple scaling of components by the eigenvalues. From $𝑩$ that commutes with $𝑨$, construct $𝑪={𝑸}^T𝑩𝑸=-{𝑪}^T$ (to work in $𝑸$ basis), and find

   ${\displaystyle 𝚲𝑪-𝑪𝚲={𝑸}^T(𝑨𝑩-𝑩𝑨)𝑸=𝟎\Rightarrow 𝚲𝑪=𝑪𝚲.}$

   A repeated eigenvalue is a statement about the components of $𝚲$. Equality of the $(i,j)$ components of $𝚲𝑪$ and $𝑪𝚲$ implies

   ${\displaystyle {\lambda }_i{c}_{ij}=c_{ij}{\lambda }_j.}$

   From above if there is at least one non-zero $c_{ij}$ element of $𝑪$ then ${\lambda }_i={\lambda }_j$. For the converse choose $c_{ij}=1=-c_{ji}$ at $i,j$ for which ${\lambda }_i={\lambda }_j$.

3. Prove that every positive definite matrix $𝑲\in {ℝ}^{m\times m}$ has a unique square root $𝑩$, $𝑩$ positive definite and ${𝑩}^2=𝑲$.

   Solution. $𝑲$ positive definite implies ${𝒒}^T𝑲𝒒>0$ for all $𝒒$ and $𝑲={𝑲}^T$. $𝑲$ symmetric admits an orthogonal eigendecomposition $𝑲=𝑸𝚲{𝑸}^T$ with eigenvalues ${\lambda }_j>0$. Define $𝑩=𝑸{𝚲}^{1/2}{𝑸}^T$. where diagonal elements of ${𝚲}^{1/2}$ are $\sqrt{{\lambda }_j}$.

4. Find all positive definite orthogonal matrices.

   Solution. In addition to above properties, $𝑲{𝑲}^T=𝑸{𝚲}^2{𝑸}^T=𝑰\Rightarrow {𝚲}^2=𝑰$, so possible elements of $𝚲$ are $\pm 1$. Imposing ${𝒆}_j^T{𝑸}^T𝑲𝑸{𝒆}_j>0$ then leads to $𝑲=𝑰$.

5. Find the eigenvalues and eigenvectors of a Householder reflection matrix.

   Solution. Write

   ${\displaystyle 𝑯=𝑰-2𝒒{𝒒}^T\in {ℝ}^{m\times m}}$

   with $𝒒$ the unit vector normal to the reflection hyperplane and note that $𝑯$ symmetric has an orthogonal eigendecomposition $𝑯=𝑼𝚲{𝑼}^T$. Since $𝑯$ is isometric eigenvalues $\lambda =\pm 1$. From

   ${\displaystyle 𝑯𝒒=𝒒-2𝒒\left({𝒒}^T𝒒\right)=-𝒒}$

   find eigenpair $(-1,𝒒)$. For any $𝒗\perp 𝒒$ obtain

   ${\displaystyle 𝑯𝒗=𝒗}$

   so $(1,𝒗)$ are the remaining $m-1$ eigenpairs.

6. Find the eigenvalues and eigenvectors of a Givens rotation matrix.

   Solution. The rotation matrix

   ${\displaystyle 𝑹(j,k,\theta )=𝑰+(\cos \theta -1)({𝒆}_j{𝒆}_j^{\ast }+{𝒆}_k{𝒆}_k^{\ast })-\sin \theta ({𝒆}_j{𝒆}_k^{\ast }-{𝒆}_k{𝒆}_j^{\ast })}$

   has $m-2$ eigenpairs $(\lambda ,{𝒆}_l)$ for $l\ne j$, $l\ne k$. The other eigenpairs are

   ${\displaystyle {\lambda }_j=e^{-i\theta },{𝒙}_j=i{𝒆}_j+{𝒆}_k;{\lambda }_k=e^{i\theta },{𝒙}_k={𝒆}_j+i{𝒆}_k.}$

   Verify

   ${\displaystyle 𝑹{𝒙}_j=i{𝒆}_j+{𝒆}_k+(\cos \theta -1)({𝒆}_j{𝒆}_j^{\ast }+{𝒆}_k{𝒆}_k^{\ast })(i{𝒆}_j+{𝒆}_k)-\sin \theta ({𝒆}_j{𝒆}_k^{\ast }-{𝒆}_k{𝒆}_j^{\ast })(i{𝒆}_j+{𝒆}_k)\Rightarrow }$

   ${\displaystyle 𝑹{𝒙}_j=i{𝒆}_j+{𝒆}_k+(\cos \theta -1)(i{𝒆}_j+{𝒆}_k)-\sin \theta ({𝒆}_j-i{𝒆}_k)=(i\cos \theta -\sin \theta ){𝒆}_j+(\cos \theta +i\sin \theta ){𝒆}_k=e^{-i\theta }{𝒙}_j.}$

7. Prove or state a counterexample: If all eigenvalues of $𝑨$ are zero then $𝑨=𝟎$.

   Solution. Counterexample

   ${\displaystyle 𝑨=\left[\begin{array}{ll}0& 1\\ 0& 0\end{array}\right],}$

   a Jordan block with 0 diagonal.

8. Prove: A hermitian matrix is unitarily diagonalizable and its eigenvalues are real.

   Solution. Apply Schur theorem $𝑨=𝑸𝑻{𝑸}^{\ast }={𝑨}^{\ast }={(𝑸𝑻{𝑸}^{\ast })}^{\ast }\Rightarrow 𝑸(𝑻-{𝑻}^{\ast }){𝑸}^{\ast }=𝟎\Rightarrow 𝑻={𝑻}^{\ast }$. Since $𝑻$ triangular is equal to its adjoint it must be diagonal with real elements $𝑻=𝚲$, $𝑨=𝑸𝚲{𝑸}^{\ast }$.