9.11

More on inverse matrices and elementary matrices

Observe the following examples: \(A \in \mathbb{R}^{3 \times 3}\)

1. \(\alpha, \beta, \gamma \in \mathbb{R}\), \(\begin{pmatrix} \alpha & 0 & 0 \\ 0 & \beta & 0 \\ 0 & 0 & \gamma \end{pmatrix} \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} = \begin{pmatrix} \alpha a_{11} & \alpha a_{12} & \alpha a_{13} \\ \beta a_{21} & \beta a_{22} & \beta a_{23} \\ \gamma a_{31} & \gamma a_{32} & \gamma a_{33} \end{pmatrix}\)

2. \(\begin{pmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix} A = \begin{pmatrix} a_{21} & a_{22} & a_{23} \\ a_{11} & a_{12} & a_{13} \\ a_{31} & a_{32} & a_{33} \end{pmatrix}\). This can change \(R_2 \to R_1\), \(R_1 \to R_2\)

   \(\begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ \alpha & 0 & 1 \end{pmatrix} \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} = \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ \alpha a_{11} + a_{31} & \alpha a_{12} + a_{32} & \alpha a_{13} + a_{33} \end{pmatrix}\)

   \(\alpha R_1 + R_3 \to R_3\)

More examples

1. \(A = \begin{pmatrix} 2 & -1 \\ 3 & -2 \end{pmatrix}\) \(A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}\) and \(ad - bc = -1\)

Determinant

The determinant of \(A \in \mathbb{R}^{2 \times 2}\) is \(\det(A) = ad - bc\), where \(A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}\)

Theorem

\(\det(A) \neq 0 \iff \exists A^{-1} \, | \, A^{-1} \cdot A = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\)

Proof

\(\Leftarrow\)) First, we prove \(det(A)det(B)=det(AB)\)

\(A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}\), \(B=\begin{pmatrix}e & f\\ g & h\end{pmatrix}\)

Then \(det(A)=ad-bc\), \(det(B)=eh-fg\) and \(det(A)det(B)=adeh+bcfg-adfg-bceh\)​

And \(AB=\begin{pmatrix}ae+bg & af+bh\\ ce+dg & cf+dh\end{pmatrix}\)

Then, \(det(AB)=\left(ae+bg\right)\left(cf+dh\right)-\left(af+bh)\left(ce+dg\right)\right.\)

\(=aecf+aedh+bgcf+bgdh-afce-afdg-bhce-bhdg\\=adeh+bcfg-adfg-bceh\)

Thus \(det(A)det(B)=det(AB)\)

If A is invertible, then \(AA^{-1} = I\), So \(1=\det I = \det(AA^{-1}) = \det A \det A^{-1}\)

Hence, \(\det A \neq 0\)

\(\Rightarrow\))

To find the inverse of \(A\)

\(\begin{pmatrix}2 & -1 & | & 1 & 0\\ 3 & -2 & | & 0 & 1\end{pmatrix}\sim\begin{pmatrix}1 & -\frac12 & | & \frac12 & 0\\ 3 & -2 & | & 0 & 1\end{pmatrix}\sim\begin{pmatrix}1 & -\frac12 & | & \frac12 & 0\\ 0 & -\frac12 & | & -\frac32 & 1\end{pmatrix}\sim\begin{pmatrix}1 & -\frac12 & | & \frac12 & 0\\ 0 & 1 & | & 3 & -2\end{pmatrix}\sim\begin{pmatrix}1 & 0 & | & 2 & -1\\ 0 & 1 & | & 3 & -2\end{pmatrix}\)

Thus, \(A^{-1} = \begin{pmatrix} 2 & -1 \\ 3 & -2 \end{pmatrix}\)

\(A A^{-1} = \begin{pmatrix} 2 & -1 \\ 3 & -2 \end{pmatrix} \begin{pmatrix} 2 & -1 \\ 3 & -2 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\)

Fun fact/question

Consider \(I = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\), the identity in \(\mathbb{R}\).

Find \(A \in \mathbb{R}^{2 \times 2} / A^2 = I\)

\(A = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\)

\(A = \begin{pmatrix} -1 & 0 \\ 0 & -1 \end{pmatrix}\)

\(A = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}\)

\(A^2 = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\)

Also, \(A = \begin{pmatrix} 2 & -1 \\ 3 & -2 \end{pmatrix}\) \(A^2 = \begin{pmatrix} 2 & -1 \\ 3 & -2 \end{pmatrix} \begin{pmatrix} 2 & -1 \\ 3 & -2 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\)

‍

‍

An inverse in \(\mathbb{R}^{3 \times 3}\): \(A = \begin{pmatrix} 2 & 2 & 1 \\ 1 & 1 & 0 \\ -1 & 2 & 1 \end{pmatrix}\) 思路是先消去第一列 第二列逐步来

1. \(\left(\begin{array}{ccc|ccc}2 & 2 & 1 & 1 & 0 & 0\\ 1 & 1 & 0 & 0 & 1 & 0\\ -1 & 2 & 1 & 0 & 0 & 1\end{array}\right)\)

2. \(R_1\leftrightarrow R_2\Rightarrow\left(\begin{array}{ccc|ccc}1 & 1 & 0 & 0 & 1 & 0\\ 2 & 2 & 1 & 1 & 0 & 0\\ -1 & 2 & 1 & 0 & 0 & 1\end{array}\right)\)

3. \(R_2-2R_1\rightarrow R_2;R_3+R_1\rightarrow R_3\Rightarrow\left(\begin{array}{ccc|ccc}1 & 1 & 0 & 0 & 1 & 0\\ 0 & 0 & 1 & 1 & -2 & 0\\ 0 & 3 & 1 & 0 & 1 & 1\end{array}\right)\)

4. \(R_3\leftrightarrow R_2\Rightarrow\left(\begin{array}{ccc|ccc}1 & 1 & 0 & 0 & 1 & 0\\ 0 & 3 & 1 & 0 & 1 & 1\\ 0 & 0 & 1 & 1 & -2 & 0\end{array}\right)\)

5. \(R_2-R_3\rightarrow R_2\Rightarrow\left(\begin{array}{ccc|ccc}1 & 1 & 0 & 0 & 1 & 0\\ 0 & 3 & 0 & -1 & 3 & 1\\ 0 & 0 & 1 & 1 & -2 & 0\end{array}\right)\)

6. \(\frac13R_2\rightarrow R_2\Rightarrow\left(\begin{array}{ccc|ccc}1 & 1 & 0 & 0 & 1 & 0\\ 0 & 1 & 0 & -\frac13 & 1 & \frac13\\ 0 & 0 & 1 & 1 & -2 & 0\end{array}\right)\)

7. \(\Rightarrow\left(\begin{array}{ccc|ccc}1 & 0 & 0 & \frac13 & 0 & -\frac13\\ 0 & 1 & 0 & -\frac13 & 1 & \frac13\\ 0 & 0 & 1 & 1 & -2 & 0\end{array}\right)\)

\[ \text{Check:} \quad \begin{pmatrix} 2 & 2 & 1 \\ 1 & 1 & 0 \\ -1 & 2 & 1 \end{pmatrix} \begin{pmatrix} \frac{1}{3} & 0 & -\frac{1}{3} \\ -\frac{1}{3} & 1 & \frac{1}{3} \\ 1 & -2 & 0 \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \]

‍

Matrices as functions

Consider \(A \in \mathbb{R}^{m \times n}\), define the function \(T_{A}:\mathbb{R}^{n}\to\mathbb{R}^{m}\)

Given \(v \in \mathbb{R}^n\), \(T_A(v) = A \cdot v \in \mathbb{R}^m\)

Examples: \(A = \begin{pmatrix} 2 & 1 & -1 \\ 1 & 0 & 1 \end{pmatrix} \in \mathbb{R}^{2 \times 3}\)

\(T_A: \mathbb{R}^3 \to \mathbb{R}^2\)

\(B = \begin{pmatrix} 2 & 1 \\ 3 & 4 \\ 0 & 1 \end{pmatrix} \in \mathbb{R}^{3 \times 2}\), \(T_B: \mathbb{R}^2 \to \mathbb{R}^3\)

\(C = \begin{pmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{pmatrix} \in \mathbb{R}^{2 \times 2}, \, \theta \in [0, 2\pi]\), \(T_C: \mathbb{R}^2 \to \mathbb{R}^2\)

‍

Special vectors

Consider \(A = \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} \in \mathbb{R}^{3 \times 3}\).

Define: \(e_1 = \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}, e_2 = \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix}, e_3 = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} \in \mathbb{R}^3\)

Canonical vectors.

\(Ae_1 = \begin{pmatrix} a_{11} \\ a_{21} \\ a_{31} \end{pmatrix}, \quad Ae_2 = \begin{pmatrix} a_{12} \\ a_{22} \\ a_{32} \end{pmatrix}, \quad Ae_3 = \begin{pmatrix} a_{13} \\ a_{23} \\ a_{33} \end{pmatrix}\)

First column of \(A\).

Now, observe the following:

\(v \in \mathbb{R}^3, v = \begin{pmatrix} v_1 \\ v_2 \\ v_3 \end{pmatrix}\) with \(v_1, v_2, v_3 \in \mathbb{R}\).

\(v\) is a linear combination of \(e_1, e_2, e_3\):

\(v = v_1 e_1 + v_2 e_2 + v_3 e_3\)

\(T_A(v) = Av = A(v_1 e_1 + v_2 e_2 + v_3 e_3) = v_1 Ae_1 + v_2 Ae_2 + v_3 Ae_3\)

\(T_A(v) = v_1 \begin{pmatrix} a_{11} \\ a_{21} \\ a_{31} \end{pmatrix} + v_2 \begin{pmatrix} a_{12} \\ a_{22} \\ a_{32} \end{pmatrix} + v_3 \begin{pmatrix} a_{13} \\ a_{23} \\ a_{33} \end{pmatrix}\)

\(T_A(v)\) is a linear combination of the columns of \(A\).

Example: \(A = \begin{pmatrix} 8 & 1 \\ 2 & 7 \end{pmatrix}\)

\(A \cdot \begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 8 \\ 2 \end{pmatrix}, \quad A \cdot \begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 1 \\ 7 \end{pmatrix}\)

\(v = \begin{pmatrix} 3 \\ 2 \end{pmatrix} = 3 \begin{pmatrix} 1 \\ 0 \end{pmatrix} + 2 \begin{pmatrix} 0 \\ 1 \end{pmatrix}\)

\(A \cdot v = 3 \begin{pmatrix} 8 \\ 2 \end{pmatrix} + 2 \begin{pmatrix} 1 \\ 7 \end{pmatrix} = \begin{pmatrix} 26 \\ 20 \end{pmatrix}\)

\(A \cdot \begin{pmatrix} 3 \\ 2 \end{pmatrix} = \begin{pmatrix} 8 & 1 \\ 2 & 7 \end{pmatrix} \cdot \begin{pmatrix} 3 \\ 2 \end{pmatrix} = \begin{pmatrix} 26 \\ 20 \end{pmatrix}\)

​

‍

The same idea works for \(A \in \mathbb{R}^{m \times n}\)

Consider the family \(\{ e_1, e_2, \dots, e_m \}, e_j \in \mathbb{R}^m\)

\(e_1 = \begin{pmatrix} 1 \\ 0 \\ \vdots \\ 0 \end{pmatrix}, \dots, e_j = \begin{pmatrix} 0 \\ \vdots \\ 1 \\ \vdots \\ 0 \end{pmatrix} \quad \text{(} j \text{ position)}, \quad e_n = \begin{pmatrix} 0 \\ \vdots \\ 1 \end{pmatrix}\)

\(A \cdot e_j = \begin{pmatrix} a_{1j} \\ a_{2j} \\ \vdots \\ a_{mj} \end{pmatrix}\)

Composition of transformations

\(A\in\mathbb{R}^{m\times n},\quad T_{A}:\mathbb{R}^{n}\to\mathbb{R}^{m},\quad T_{A}(v)=A\cdot v\in\mathbb{R}^{m}\)

\(B \in \mathbb{R}^{k \times m}, \quad T_B: \mathbb{R}^m \to \mathbb{R}^k, \quad T_B(w) = B \cdot w \in \mathbb{R}^k\)

We can compose \(T_{B}\circ T_{A}:\mathbb{R}^{n}\to\mathbb{R}^{k}\)

Is this composition \(T_B \circ T_A\) given by a matrix?

Yes! \((T_B \circ T_A)(v) = (B \cdot A)(v) = T_{BA}(v)\)

‍

Example:

\(A = \begin{pmatrix} 2 & 1 & 3 \\ 1 & 2 & 0 \end{pmatrix} \in \mathbb{R}^{2 \times 3}\)

\(T_A: \mathbb{R}^3 \to \mathbb{R}^2, \quad v \in \mathbb{R}^3, \quad v = \begin{pmatrix} v_1 \\ v_2 \\ v_3 \end{pmatrix}\)

\(T_A(v) = A \cdot v\)

\(T_A(v) = \begin{pmatrix} 2 & 1 & 3 \\ 1 & 2 & 0 \end{pmatrix} \cdot \begin{pmatrix} v_1 \\ v_2 \\ v_3 \end{pmatrix} = \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} \in \mathbb{R}^2\)

\(B \in \mathbb{R}^{4 \times 2}, \quad B = \begin{pmatrix} 1 & 2 \\ 0 & 1 \\ -1 & 2 \\ 2 & -1 \end{pmatrix}\)

\(T_B: \mathbb{R}^2 \to \mathbb{R}^4\)

\(T_{B}(T_{A}(v))=\begin{pmatrix}1 & 2\\ 0 & 1\\ -1 & 2\\ 2 & -1\end{pmatrix}\cdot T_{A}\left(v\right)=\begin{pmatrix}1 & 2\\ 0 & 1\\ -1 & 2\\ 2 & -1\end{pmatrix}\cdot\left\lbrack\begin{pmatrix}2 & 1 & 3\\ 1 & 2 & 0\end{pmatrix}\cdot\begin{pmatrix}v_1\\ v_2\\ v_3\end{pmatrix}\right\rbrack\)

Exercise: prove this

\(B\cdot\left(A\cdot v)=(B\cdot A\right)\cdot v\)

Hint

\(\left(B\cdot\left(A\cdot v\right)\right)_{i}=\left((B\cdot A\right)\cdot v)_{i},\quad(i=1,...,4 )\)

‍

​

‍

\(A\in\mathbb{R}^{3\times3},\quad T_{A}:\mathbb{R}^3\to\mathbb{R}^3\)

\(A = \begin{pmatrix} \cos\theta & -\sin\theta & 0 \\ \sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{pmatrix}\)

\(B = \begin{pmatrix} \cos\theta & 0 & -\sin\theta \\ 0 & 1 & 0 \\ \sin\theta & 0 & \cos\theta \end{pmatrix}\)

\(B \cdot \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} \cdots \\ y \\ \cdots \end{pmatrix}\)

‍

Inner product in \(\mathbb{R}^n\) (Dot product)

Let \(v, w \in \mathbb{R}^n\), \(v=\begin{pmatrix}v_1\\ \vdots\\ v_{n}\end{pmatrix}\), \(w=\begin{pmatrix}w_1\\ \vdots\\ w_{n}\end{pmatrix}\)

Define:

\(\langle v,w\rangle=\sum_{j=1}^{n}v_{j}w_{j}\), also denoted as \(\langle v, w \rangle = v \cdot w\)

Example:

\(v = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\), \(w = \begin{pmatrix} 2 \\ 3 \end{pmatrix}\), then \(\langle v, w \rangle = \langle \begin{pmatrix} 1 \\ 2 \end{pmatrix}, \begin{pmatrix} 2 \\ 3 \end{pmatrix} \rangle = 1 \cdot 2 + 2 \cdot 3 = 8\)

Note that if we identify \(\mathbb{R}^n\) with \(\mathbb{R}^{n \times 1}\),

\(v\in\mathbb{R}^{n}\), \(v=\begin{pmatrix}v_1\\ \vdots\\ v_{n}\end{pmatrix}\)​\(\Rightarrow v^T = (v_1, v_2, \dots, v_n)\)

Then: \(\langle v,w\rangle=v^{T}w=\begin{pmatrix}v_1 & \dots & v_{n}\end{pmatrix}\cdot\begin{pmatrix}w_1\\ \vdots\\ w_{n}\end{pmatrix}=\sum_{j=1}^{n}v_{j}w_{j}\)

‍

Properties of \(u, v, w \in \mathbb{R}^n\), \(\alpha, \beta \in \mathbb{R}\)

1. \(\langle v, w \rangle = \langle w, v \rangle\)

2. \(\langle \alpha v, w \rangle = \alpha \langle v, w \rangle = \langle v, \alpha w \rangle\)

3. \(\langle u + v, w \rangle = \langle u, w \rangle + \langle v, w \rangle\)

4. \(\langle v, v \rangle = \sum_{j=1}^{m} v_j^2 \geq 0\)

5. \(\langle v, v \rangle = 0 \Rightarrow v = \bar{0}\)

We can define a norm on \(\mathbb{R}^n\), \(v \in \mathbb{R}^n\):

\(\| v \| = \langle v, v \rangle^{1/2} = \left( \sum_{j=1}^{m} v_j^2 \right)^{1/2}\)

1. \(\|\lambda v\|=|\lambda|\cdot\|v\|,\,\lambda\in\mathbb{R}\)

2. \(\|v\|\geq0,\,\|v\|=0\Rightarrow v=\bar{0}\)

3. Triangular inequality: \(\| v + w \| \leq \| v \| + \| w \|\)

​

Requires a proof!

‍

‍

Example:

\(v = \begin{pmatrix} 1 \\ 2 \end{pmatrix}, \quad w = \begin{pmatrix} -2 \\ 1 \end{pmatrix}\)

​

\(\langle v, w \rangle = v^T w = \begin{pmatrix} 1 & 2 \end{pmatrix} \cdot \begin{pmatrix} -2 \\ 1 \end{pmatrix} = 0\)

Property: \(v, w \in \mathbb{R}^2\) (Also works in \(\mathbb{R}^n\)), \(\theta\) is the angle between \(v\) and \(w\), \(\theta \in [0, \pi]\) (The small angle)

\(\langle v, w \rangle = \| v \| \cdot \| w \| \cdot \cos\theta\)

​

This explains that \(\langle v, w \rangle = 0 \Rightarrow \theta = \frac{\pi}{2}\)

‍