5.15 Inner Product

Def: Let \(V\) be a vector space over \(\mathbb{R}\) (or \(\mathbb{C}\)).
An inner product on \(V\) is a function \(\Phi: V \times V \rightarrow \mathbb{F}\) s.t.

1. \(\Phi(u+v, w) = \Phi(u, w) + \Phi(v, w)\) and \(\Phi(\alpha v, w) = \alpha \Phi(v, w)\) since \(\Phi\) is linear in the first position

2. \(\Phi(u, v) = \overline{\Phi(v, u)}\)​

3. \(\Phi(u, u) > 0\) \(\forall u \neq 0\)​

Proposition

1. \(\Phi(u, v+w) = \Phi(u,v) + \Phi(u,w)\) and \(\Phi(u, \alpha v) = \bar{\alpha} \Phi(u,v)\).

2. If \(\Phi(u,v) = \Phi(u,w)\) \(\forall u \in V\), then \(v=w\).
   Proof
   ​\(\Phi(u,v) = \Phi(u,w) \Rightarrow \Phi(u,v) - \Phi(u,w) = 0\) \(\Rightarrow \Phi(u, v-w) = 0\). \(\forall u\).
   If \(u = v-w\), \(\Phi(v-w, v-w) = 0\).
   ​\(\therefore v=w\).

---

Exercise

1. \(V = \mathbb{C}^{2 \times 2}\) and \(\phi(A,B) = a_{11} \overline{b}_{11} + a_{12} \overline{b}_{12} + 5 a_{21} \overline{b}_{21} + a_{22} \overline{b}_{22}\), where \((A)_{ij} = a_{ij}\).

   Show that \(\phi\) is an inner product

   1. \(\phi(A+B,C) = (a_{11} + b_{11})\overline{c}_{11} + (a_{12} + b_{12})\overline{c}_{12} + 5(a_{21} + b_{21})\overline{c}_{21} + (a_{22} + b_{22})\overline{c}_{22}\)
      ​\(= a_{11}\overline{c}_{11}+ b_{11}\overline{c}_{11}+ a_{12}\overline{c}_{12}+ b_{12} \overline{c}_{12}+ 5 a_{21}\overline{c}_{21}+ 5 b_{21}\overline{c}_{21}\) \(+ a_{22}\overline{c}_{22} + b_{22}\overline{c}_{22}\)
      ​\(= (a_{11}\overline{c}_{11} + a_{12}\overline{c}_{12} + 5 a_{21}\overline{c}_{21} + a_{22}\overline{c}_{22}) + (b_{11}\overline{c}_{11} + b_{12}\overline{c}_{12} + 5 b_{21}\overline{c}_{21} + b_{22}\overline{c}_{22})\)
      ​\(= \phi(A,C) + \phi(B,C)\)

      \(\phi(\alpha A, B) = (\alpha a_{11})\overline{b}_{11} + (\alpha a_{12})\overline{b}_{12} + 5(\alpha a_{21})\overline{b}_{21} + (\alpha a_{22})\overline{b}_{22}\)
      ​\(= \alpha (a_{11}\overline{b}_{11} + a_{12}\overline{b}_{12} + 5 a_{21}\overline{b}_{21} + a_{22}\overline{b}_{22})\)
      ​\(= \alpha \phi(A,B)\)​ 2. \(\phi(B,A) = \overline{b_{11}\overline{a}_{11} + b_{12}\overline{a}_{12} + 5b_{21}\overline{a}_{21} + b_{22}\overline{a}_{22}}\) \(= \overline{b_{11}\overline{a}_{11}} + \overline{b_{12}\overline{a}_{12}} + \overline{5b_{21}\overline{a}_{21}} + \overline{b_{22}\overline{a}_{22}}\)​
      ​\(= a_{11}\overline{b}_{11} + a_{12}\overline{b}_{12} + 5 a_{21}\overline{b}_{21} + a_{22}\overline{b}_{22}\)
      ​\(= \phi(A,B)\) 3. \(\Phi(A,A) = a_{11}\bar{a}_{11} + a_{12}\bar{a}_{12} + 5a_{21}\bar{a}_{21} + a_{22}\bar{a}_{22}\)
      ​\(= |a_{11}|^2 + |a_{12}|^2 + 5|a_{21}|^2 + |a_{22}|^2 > 0\) if \(A \neq 0\).

2. \(V=C([0,1])\), \(\Phi(f,g) = \int_0^1 f(x)g(x) dx\). Prove \(\Phi\) is an inner product.

   \(C([0,1]) = \{f: [0,1] \to \mathbb{R}| f_{cont.}\}\) such that \((f+g)(x) = f(x)+g(x)\), \((\alpha \cdot f)(x) = \alpha f(x)\)​

   1. \(\Phi(f+g, h) = \int_0^1 (f+g)(x) \cdot h(x) \cdot x \, dx = \int_0^1 (f(x)+g(x))h(x)x \, dx\)
      ​\(= \int_0^1 (f(x)h(x)x + g(x)h(x)x) dx \stackrel{\text{lin. int}}{=} \int_0^1 f(x)h(x)x \, dx + \int_0^1 g(x)h(x)x \, dx = \Phi(f,h) + \Phi(g,h)\)​

      \(\Phi(\alpha f, g) = \int_0^1 (\alpha f)(x)g(x)x \, dx = \int_0^1 \alpha f(x) g(x) x \, dx\)
      ​\(= \int_0^1 \alpha (f(x)g(x)x) \, dx = \alpha \int_0^1 f(x)g(x)x \, dx\)
      ​\(= \alpha \Phi(f, g)\) 2. \(\overline{\Phi(g,f)}=\overline{\int_0^1g(x)f(x)x\,dx}=\int_{0}^{1}g(x)f(x)x\,dx\) \(= \Phi(f, g)\) 3. \(\Phi(f,f)>0\) if \(f \neq 0\).
      Let \(t \in (0,1)\) s.t. \(f(t) \neq 0\)
      Suppose that \(f(t)>0\), \(\exists \delta >0\) s.t. \(f(x)>0\) \(\forall x\in B(t,\delta)=(t-\delta,t+\delta)\cap[0,1]\) since the continuity of function

      Thus, \(\Phi(f,f)=\int_{0}^{1}f^{2}(x)xdx=\int_{t-\delta}^{t+\delta}f^{2}(x)xdx+\int_{[0,1]\setminus B(t,\delta)}f^{2}(x)xdx>0\) since the first term \(>0\) and second term \(\geq 0\)​

      If \(f(t)<0\), \(\exists \delta >0\) s.t. \(f(x)<0\) \(\forall x\in B(t,\delta)=(t-\delta,t+\delta)\cap[0,1]\)​

      \(\Phi(f,f)=\int_{0}^{1}f^{2}(x)x\,dx=\int_{t-\delta}^{t+\delta}f^{2}(x)x\,dx+\int _{[0,1]\setminus B(t,\delta)}f^{2}(x)x\,dx>0\) since the first term \(>0\) and second term \(\geq 0\)
      Thus \(\Phi(f,f) > 0 \text{ if } f \neq 0\)

---

Def: \((V, \Phi)\) is an inner product space over \(\mathbb{R}\) ( or \(\mathbb{C}\)) and \(B=\{v_1, v_2, ..., v_n\}\) is a basis for \(V\).
​\((\left\lbrack\Phi\right\rbrack_{B})_{ij}=\Phi(v_{i},v_{j})\) \(\forall\) \(1\le i,j \le n\).

Prop:

1. \(v, w \in V\), \(B\) a basis for \(V\). Then \(\Phi(v, w) = [v]_{B}[\Phi]_{B}\overline{[w]_{B}^{t}}\) where \([v]_B = (\alpha_1, \alpha_2, ..., \alpha_n)\), \(v = \alpha_1 v_1 + ... + \alpha_n v_n\)

2. \(B_1, B_2\) bases for \(V\), then \([\Phi]_{B_2} = C(B_2, B_1)^t [\Phi]_{B_1} C(B_2, B_1)\)

   Since \([v]_{B_1}^{t}=C(B_{2},B_{1})[v]_{B_2}^{t}\) and \([v]_{B_1}=[v]_{B_2}C(B_{2},B_{1})^{t}\)

   Proof 2: \(\Phi(v, w) = [v]_{B_1} [\Phi]_{B_1} [w]_{B_1}^t\)
   ​\(= [v]_{B_2}(C(B_{2}, B_{1}))^{t} [\Phi]_{B_1}\overline{(C(B_{2}, B_{1}) [w]_{B_2})^{t}}\)
   ​\(=[v]_{B_2}\underbrace{(C(B_2,B_1))^{t}[\Phi]_{B_1}\overline{C(B_2,B_1)}}_{[\Phi]_{B_2}}\, \overline{[w]_{B_2}}^{t}\)

Remark

1. \(A = [\Phi]_B\), then(this is iff because to prove \(A=[\Phi]_{B}\), NTP: \(A\) satisfy the three properties of inner product. The first property comes from the linear product of matrix naturally. The second property comes from the first property above. The third property comes from the second property above )

   1. \(\bar{A}^t = A (= A^*\) adjoint of A)
      ​\(([\Phi]_{B})_{ij}= \Phi(v_{i}, v_{j}) = \overline{\Phi(v_j, v_i)}= (\overline{[\Phi]_B} )_{ji}\).

   2. \(A\) is positive definite.\((A\in F^{n\times n},vAv^{t}>0\quad\forall v\in F^{n}\setminus\{0\})\).

      A is positive definite iff the eigenvalues of \(A\) are positive. Iff the determinant of each leading principal minors is positive
      ​\(A = \begin{bmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n} \\ a_{21} & a_{22} & a_{23} & \cdots & a_{2n} \\ a_{31} & a_{32} & a_{33} & \cdots & a_{3n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & a_{n3} & \cdots & a_{nn} \end{bmatrix}\)
      ​\(a_{11}>0\), \(\det \begin{vmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{vmatrix} > 0\), \(\det \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1i} \\ \vdots & \vdots & \ddots & \vdots \\ a_{i1} & a_{i2} & \cdots & a_{ii} \end{pmatrix} > 0\) \(\forall i\).

2. If \(A = \bar{A}^{t} \nRightarrow \exists \Phi\) an inner product s.t \([\Phi]_E = A\).
   Counter example: \(A = \begin{pmatrix} 1 & i \\ -i & 1 \end{pmatrix} = 1 - (-i^2) = 1 - (-(-1)) = 1 - (1) = 0\) since the contrapositive of 1.b

3. If A is positive definite \(\nRightarrow A=[\Phi]_{E}\).
   Counter example: \(A=\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\). \(A\) is positive definite, but \(A \neq \overline{A}^t\)

Def:\((V, \langle \cdot, \cdot \rangle)\). \(|| \cdot || : V \rightarrow \mathbb{R}_{\geq 0}\). the norm: \(||v|| = (\langle v,v \rangle)^{1/2}\)

Prop: \(\forall v \in V, ||v|| \geq 0\) and \(||v|| = 0\) iff \(v=0\).

\(||\alpha v||=|\alpha| ||v||\)

\(|\langle v,w\rangle| \le ||v|| ||w||\)

\(||v+w|| \le ||v|| + ||w||\)

Def:\(v\) is orthogonal to \(w\) if\(\langle v,w\rangle =0 \quad (v \perp w)\)

What is dot product, it is a specific inner product given by coordinates \(V=\mathbb{R}^n\) \(v=(x_1,...,x_n)\) \(w=(y_1,...,y_n)\) \(\langle v,w\rangle = v \cdot w = \sum_{i=1}^n x_i y_i\)

---

\(\mathcal{B}=\{v_1, v_2, ..., v_n\}\) is an orthogonal basis for \(v\) if \(v_i \perp v_j\), \(i \neq j\)

orthonormal basis if \(v_i \perp v_j\), \(i \neq j\), \(||v_i||=1\)

Remark: if \(B=\{v_{1}, ..., v_{n}\}\) is an orthogonal basis for \(V\), \([\Phi]_{B} = \begin{bmatrix} \langle v_{1}, v_{1} \rangle & 0 & ... & 0 \\ 0 & \langle v_{2}, v_{2} \rangle & ... & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & ... & \langle v_{n}, v_{n} \rangle \end{bmatrix} = \begin{bmatrix} ||v_{1}||^{2} & & & \\ & ||v_{2}||^{2} & & \\ & & \ddots & \\ & & & ||v_{n}||^{2} \end{bmatrix}\)

If \(B\) is orthonormal, \([\Phi]_{B} = I_{n\times n}\)

---

Ex 3: Let \(B = \{(1,1,0), (0,1,1), (0,1,0)\}\) be a basis for\(\mathbb{R}^3\). Define an inner product on \(\mathbb{R}^3\) s.t. \(B\) is an orthonormal basis for \(\mathbb{R}^3\)

\([\Phi]_B=I_{3\times 3}\)

\([\Phi]_{E} = C(E, B)^t [\Phi]_B \overline{C(E,B)}\) where \(C(B, E) = \begin{bmatrix} 1 & 0 & 0 \\ 1 & 1 & 1 \\ 0 & 1 & 0 \end{bmatrix}\)
\(= \begin{pmatrix} 1 & 0 & 0 \\ 1 & 1 & 1 \\ 0 & 1 & 0 \end{pmatrix}^{t} I \overline{\begin{pmatrix} 1 & 0 & 0 \\ 1 & 1 & 1 \\ 0 & 1 & 0 \end{pmatrix}}= \begin{bmatrix} 2 & -1 & 1 \\ -1 & 1 & -1 \\ 1 & -1 & 2 \end{bmatrix}\)

Then \(\Phi (x_1, x_2, x_3), (y_1, y_2, y_3) = 2x_1 y_1 - x_1 y_2 + x_1 y_3 - x_2 y_1 + x_2 y_2 - x_2 y_3 + x_3 y_1 - x_3 y_2 + 2x_3 y_3\) by formula

---

Def: \((V, \langle \cdot, \cdot \rangle)\) We define the distance \(d: V \times V \rightarrow \mathbb{F}\) where \((v, w) \mapsto d(v, w) = ||v - w|| = (\langle v-w, v-w \rangle)^{1/2}\)

Prop:

• \(d(v, w) \ge 0\)​
• \(d(v, w) = d(w, v)\)​
• \(d(v, w) \le d(v, u) + d(u, w)\)

Def: From \(\langle-,-\rangle\) we define the angle between \(v\) and \(w\) as \(\cos \theta = \frac{|\langle v, w\rangle|}{||v|| ||w||}\) since \(\frac{|\langle v, w\rangle|}{||v|| ||w||} \le 1\) since

Ex 4: Let \(V = C([0,1])\), \(\langle f, g\rangle = \int_{0}^{1} f(x) g(x) \, dx\). Find \(d(2x+7, x-2)\)
​\(d(2x+7, x-2) = ||(2x+7) - (x-2)|| = ||x+9|| = \left( \langle x+9, x+9 \rangle \right)^{1/2}\)
​\(d(2x+7, x-2) = \left( \int_{0}^{1} (x+9)^2 dx \right)^{1/2} = \left( \int_{0}^{1} x^2 + 2\cdot 9x + 9^2 dx \right)^{1/2}\)
​\(= \sqrt{\frac{271}{3}}\)

\([A]_B=(P_E^B)^{-1}[A]_EP_E^B\)

\([A]_{B}=C\left(E,B\right)^{-1}\left\lbrack A\right\rbrack_{E}C\left(E,B\right)\)​

‍

‍