5.22 Orthogonal

Prop: \((V, \langle -,-\rangle), \mathbb{F}=\mathbb{R}\) or \(\mathbb{C}\).
i) \(\beta = \{v_1, ..., v_n\}\) is an O.G. basis for \(V\). If \(v = \sum_{i=1}^n \alpha_i v_i\), then \(\langle v, v_i \rangle = \alpha_i ||v_i||^2 \Rightarrow \alpha_i = \frac{\langle v, v_i \rangle}{||v_i||^2}\).

\([v]_\beta = \left( \frac{\langle v, v_1 \rangle}{||v_1||^2}, ..., \frac{\langle v, v_n \rangle}{||v_n||^2} \right)\) and \(||v||^2 = \sum_{i=1}^n \frac{|\langle v, v_i \rangle|^2}{||v_i||^2}\).

ii) If \(\beta\) is O.N. Then \(\langle v, v_i \rangle = \alpha_i\). \([v]_\beta = (\langle v, v_1 \rangle, \langle v, v_2 \rangle, ..., \langle v, v_n \rangle); ||v||^2 = \sum_{i=1}^n |\langle v, v_i \rangle|^2\)

---

G-S. process
Let \(B = \{v_1, ..., v_n\}\) be a basis for V.

1. \(Z_1 = v_1\), \(Z_1' = v_1/||v_1||\)​

2. \(Z_{n+1} = v_{n+1} - \sum_{i=1}^r \frac{\langle v_{n+1}, z_i\rangle}{||z_i||^2} z_i\)
   ​\(Z_{n+1}' = \frac{z_{n+1}}{||z_{n+1}||}\)
   ​\(\therefore \{Z_1, Z_2, ..., Z_n\}\) is an O.G. basis for V. \(\{Z_1', ..., Z_n'\}\) is an O.N basis for V.

---

Ex: Consider \(R^{2 \times 2}\) with the i.p \(\langle A, B \rangle = tr(AB^t)\).
Let \(B = \{ \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}_{v_1}, \begin{pmatrix} -1 & 7 \\ 0 & 0 \end{pmatrix}_{v_2}, \begin{pmatrix} 1 & 18 \\ 1 & 1 \end{pmatrix}_{v_3}, \begin{pmatrix} 0 & 0 \\ 5 & -3 \end{pmatrix}_{v_4} \}\) be a basis for \(R^{2 \times 2}\).
Find an O.N. basis for \(R^{2 \times 2}\).

Solution

\(Z_1 = \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}\), \(||Z_1||^2 = 25\) since \(\langle Z_{1}, Z_{1} \rangle = tr(( \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}) ( \begin{pmatrix} 3 & 0 \\ 4 & 0 \end{pmatrix})) = tr( \begin{pmatrix} 25 & 0 \\ 0 & 0 \end{pmatrix})=25\)

\(Z_{2}= \begin{pmatrix} -1 & 7 \\ 0 & 0 \end{pmatrix}-\frac{\langle \begin{pmatrix} -1 & 7 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}\rangle}{||Z_{1}||^{2}} \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}\) since \(\langle \begin{pmatrix} -1 & 7 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix} \rangle = tr(\begin{pmatrix} -1 & 7 \\ 0 & 0 \end{pmatrix} \begin{pmatrix} 3 & 0 \\ 4 & 0 \end{pmatrix}) = tr(\begin{pmatrix} 25 & 0 \\ 0 & 0 \end{pmatrix}) = 25\), then \(=\begin{pmatrix}-1 & 7\\ 0 & 0\end{pmatrix}-\frac{25}{25}\begin{pmatrix}3 & 4\\ 0 & 0\end{pmatrix}=\begin{pmatrix}-1-3 & 7-4\\ 0 & 0\end{pmatrix}=\begin{pmatrix}-4 & 3\\ 0 & 0\end{pmatrix}\)

\(Z_2 = \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix}\), thus \(||Z_2||^2 = 25\)

\(Z_3 = \begin{pmatrix} 1 & 18 \\ 1 & 1 \end{pmatrix} - \frac{\langle \begin{pmatrix} 1 & 18 \\ 1 & 1 \end{pmatrix}, \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix} \rangle}{25} \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix} - \frac{\langle \begin{pmatrix} 1 & 18 \\ 1 & 1 \end{pmatrix}, \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix} \rangle}{25} \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix}\)

Since \(\langle \begin{pmatrix} 1 & 18 \\ 1 & 1 \end{pmatrix}, \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}\rangle=75\) and \(\langle \begin{pmatrix} 1 & 18 \\ 1 & 1 \end{pmatrix}, \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix}\rangle=50\), then \(\boxed{ Z_3 = \begin{pmatrix}0 & 0 \\ 1 & 1\end{pmatrix} }\) and \(\boxed{ \|Z_3\|^2 = 2 }\)

\(Z_4 = \begin{pmatrix} 0 & 0 \\ 5 & -3 \end{pmatrix} - \frac{\langle \begin{pmatrix} 0 & 0 \\ 5 & -3 \end{pmatrix}, \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix} \rangle}{25} \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix} - \frac{\langle \begin{pmatrix} 0 & 0 \\ 5 & -3 \end{pmatrix}, \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix} \rangle}{25} \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix}\)
​\(- \frac{\langle \begin{pmatrix} 0 & 0 \\ 5 & -3 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 1 & 1 \end{pmatrix} \rangle}{2} \begin{pmatrix} 0 & 0 \\ 1 & 1 \end{pmatrix}\) where \(\langle \begin{pmatrix} 0 & 0 \\ 5 & -3 \end{pmatrix}, \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}\rangle=0\), \(\langle \begin{pmatrix} 0 & 0 \\ 5 & -3 \end{pmatrix}, \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix}\rangle=0\), \(\langle \begin{pmatrix} 0 & 0 \\ 5 & -3 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 1 & 1 \end{pmatrix}\rangle=2\)

Thus \(Z_{4} = \begin{pmatrix} 0 & 0 \\ 4 & -4 \end{pmatrix}\) and \(||Z_{4}||^{2} = 32\)

\(B' = \left\{ \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 1 & 1 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 4 & -4 \end{pmatrix} \right\}\) is an O.G. basis for \(R^{2 \times 2}\).

\(B'' = \left\{ \frac{1}{5} \begin{pmatrix} 3 & 4 \\ 0 & 0 \end{pmatrix}, \frac{1}{5} \begin{pmatrix} -4 & 3 \\ 0 & 0 \end{pmatrix}, \frac{1}{\sqrt{2}} \begin{pmatrix} 0 & 0 \\ 1 & 1 \end{pmatrix}, \frac{1}{4\sqrt{2}} \begin{pmatrix} 0 & 0 \\ 4 & -4 \end{pmatrix} \right\}\) is an O.N. basis for \(R^{2 \times 2}\).

---

Ex: Let \(S = \text{span} \{1+x, x^2-1\}\) in \(R_3[x]\), where \(\langle p,q \rangle = \int_{-1}^{1} p(x) q(x) dx\).

Find \(S^\perp\). (Orthogonal complement of \(S\). \(S^\perp = \{h \in V \mid \langle p, h \rangle = 0 \forall p \in S\}).\)

If \(V\) is a f.d.i.p.s. and \(S \subseteq V\), then \(V = S \oplus S^\perp\) and \((S^\perp)^\perp = S\).

In this case \(S^\perp = \{h(x) \in R_3[x] \mid \langle h,p \rangle = 0 \forall p \in S\}\).

If \(p \in S\), \(p(x) = \alpha(1+x) + \beta(x^2-1)\).
Let \(h \in S^\perp\), then \(0 = \langle h(x), p(x) \rangle = \langle h(x), \alpha(1+x) + \beta(x^2-1) \rangle\) \(= \bar{\alpha} \langle h(x), 1+x \rangle + \bar{\beta} \langle h(x), x^2-1 \rangle\).
\(h \in S^\perp\) iff \(\langle h(x), 1+x \rangle = 0\) and \(\langle h(x), x^2-1 \rangle = 0\).
Since \(h \in \mathbb{R}_3[x] \Rightarrow h(x) = ax^3 + bx^2 + cx + d\), then \(\langle h(x), 1+x \rangle = \int_{-1}^{1} (ax^{3} + bx^{2} + cx + d)(1+x ) \, dx = \int_{-1}^{1} (ax^{4} + bx^{3} + cx^{2} + dx + ax^{3} + bx^{2} + cx + d ) \, dx\)
\(= \left[ \frac{ax^5}{5} + \frac{bx^4}{4} + \frac{cx^3}{3} + \frac{dx^2}{2} + \frac{ax^4}{4} + \frac{bx^3}{3} + \frac{cx^2}{2} + dx \right]_{-1}^1\)
\(= \left( \frac{a}{5} + \frac{b}{4} + \frac{c}{3} + \frac{d}{2} + \frac{a}{4} + \frac{b}{3} + \frac{c}{2} + d \right) - \left( -\frac{a}{5} + \frac{b}{4} - \frac{c}{3} + \frac{d}{2} + \frac{a}{4} - \frac{b}{3} + \frac{c}{2} - d \right)\)
\(= \frac{2a}{5} + \frac{2b}{3} + \frac{2c}{3} + 2d = 0\) (1).

\(\langle h(x), x^{2}-1 \rangle = \int_{-1}^{1}(ax^{3}+bx^{2}+Cx+d)(x^{2}-1) dx = \dots = \boxed{-\frac{4d}{3} - \frac{4}{15}b = 0}\textcircled{2}\)

\(\textcircled{2}\implies \boxed{b = -5d}\) \(\begin{array}{rl} \textcircled{1} \text{ and } \textcircled{2} \implies \end{array}\)​\(a = \frac{10}{3}d - \frac{5}{3}c\)

\(h(x) = \left(\frac{10}{3}d - \frac{5}{3}c\right)x^3 + (-5d)x^2 + cx + d\) \(= d\left(\frac{10}{3}x^3 - 5x^2 + 1\right) + c\left(-\frac{5}{3}x^3 + x\right)\).

\(\therefore S^\perp = \text{span}\left\{\frac{10}{3}x^3 - 5x^2 + 1, -\frac{5}{3}x^3 + x\right\}.\)

---

Remark: If \(V\) is not a f.d.v.s. \(F = \mathbb{R}\).
\(V = C[0, 1]\), \(\langle f, g \rangle = \int_0^1 f(x)g(x) dx\). \(S = \{ f \in C[0, 1] | f(0) = 0 \} \subseteq V\). \(S^\perp = \{ g \in C[0, 1] | \langle f, g \rangle = 0 \}\).

If \(g \ne 0 \in S^\perp\), then \(\langle f,g \rangle = 0\) and \(\exists x_1 \in [0,1]\) s.t. \(g(x) \ne 0\). (Assume that \(g(x_1) > 0\)).
Then \(\exists [a,b] \subseteq [0,1]\) s.t. \(g(x) > 0 \forall x \in [a,b]\).
Define \(f(x) = \begin{cases} (x-a)(b-x) & \text{if } x \in [a,b] \\ 0 & \text{if } x \in [0,1] \setminus [a,b] \end{cases}\)

Then \(f(x) > 0 \forall x \in (a, b)\).
​\(\langle f, g \rangle = \int_{0}^{1} f(x) g(x) dx = \int_{a}^{b} f(x) g(x) dx > 0\). This is a contradiction

Therefore, \(g(x) = 0 \forall x \in [0, 1]\).

\(\therefore S^{\perp} = \{0\}\)
\(\begin{cases}S \oplus S^{\perp} = S \oplus \{0\} = S \neq V & \begin{array}{c} cos~x \in V \\ cos~x \notin S \end{array} \\(S^{\perp})^{\perp} = V \neq S.\end{cases}\)
\((S^{\perp})^{\perp} = \{f \in C[0,1] | \langle f,g \rangle = 0 \quad \forall g \in S^{\perp} \}\)
\(=\{f \in C[0,1] | \langle f,0 \rangle = 0 \} = V \neq S.\)

---

\(\text{Def}: (V, \langle -,-\rangle ) \text{ a f.d. i.p.S. and } S \subseteq V\).

The orthogonal projection on \(S\) is the l.op \(P_{S}: V \rightarrow S\).

\(P_{S}(s)=s \quad \text{if } s \in S\)
\(P_{S}(t)=0 \quad \text{if } t \in S^{\perp} \quad V=S \oplus S^{\perp}\)

if \(\beta = \{v_1, \dots, v_r, v_{r+1}, \dots, v_n\}\) is an O.N. basis for V s.t.

\(\{v_1, \dots, v_r\}\) is a basis for S and \(\{v_{r+1}, \dots, v_n\}\) is a basis for \(S^{\perp}\). Then \(P_s: V \rightarrow V\) is the unique l.t. s.t.
\(P_{s}(v_{i}) = \begin{cases} v_{i} & 1 \le i \le r \\ 0 & r+1 \le i \le n \end{cases}\)
Remark: \(B = \{v_{1}, ..., v_{r}\}\) O.N. basis for S, then \(v \in v\). \(P_{s}(v) = \sum_{i=1}^{r} \langle v, v_{i} \rangle v_{i}\)

If B is O.G. basis for S, then \(P_{s}(v) = \sum_{i=1}^{r} \frac{\langle v, v_{i} \rangle}{||v_{i}||^{2}}v_{i}\)

Def: \(d(v, S) = \inf \{d(v, s) \mid s \in S\}\) \(= \inf \{\|v - s\| \mid s \in S\}\) \(= \|v - P_s(v)\|\) \(= \|P_{s \perp}(v)\|\) \(= \sqrt{\|v\|^2 - \|P_s(v)\|^2}\) since

\(v = P_s(v) + P_{s \perp}(v)\)
\(\|v\|^2 = \|P_s(v) + P_{s \perp}(v)\|^2\) \(= \|P_s(v)\|^2 + \|P_{s \perp}(v)\|^2\)

\(\langle s, t \rangle = 0\), ​
​\(\|s+t\|^2 = \langle s+t, s+t \rangle\) \(= \langle s, s \rangle + \langle s, t \rangle + \langle t, s \rangle + \langle t, t \rangle\) \(= \|s\|^2 + \|t\|^2\)

---

Example:
\(V = C[0, 1]\), \(\langle f, g \rangle = \int_0^1 f(x) g(x) dx\).
Find the orthogonal projection of \(e^x\) on \(\mathbb{R}_2[x]\) and \(d(e^x, \mathbb{R}_2[x])\). (Using the best app. of \(e^x\) by vectors in \(\mathbb{R}_2[x]\).)

Ans: \(W = \text{span}\{1, x, x^2, e^x\} \subseteq C[0,1]\) \(\therefore \text{dim } W = 4\), \(S = \mathbb{R}_2[x]\), then \(W=S \oplus S^\perp\).

\(P_s(e^x) = \sum_{i=1}^3 \frac{\langle e^x, v_i \rangle}{\|v_i\|^2} v_i\) where \(\{v_1, v_2, v_3\}\) is an O.G. basis for \(S\).

Using G-S, \(\beta = \{1, x-1/2, x^2-x+1/6\}\) (\(\mathcal{E}_x!\)) is an O.G. basis for \(\mathbb{R}_2[x]\).

Where \(\|1\|^2=1\), \(\|x-1/2\|^2 = \frac{1}{12}\) and \(\|x^2-x+1/6\|^2 = \frac{1}{180}\).

\(⟨e^{x}, 1⟩ = \int_{0}^{1} e^{x} dx = e-1\)
$⟨e^{x}, x-\frac{1}{2}⟩ = \int_{0}^{1} xe^{x} - \frac{1}{2}e^{x} dx = \frac{3-e}{2} $
\(⟨e^{x}, x^{2}-x+\frac{1}{6}⟩ = \int_{0}^{1} e^{x}(x^{2}-x+\frac{1}{6}) dx = \frac{7e-19}{6}\)

\(P_s(e^x) = (e-1) + \frac{\frac{3-e}{2}}{\frac{1}{12}} (x - 1/2) + \frac{\frac{7e-19}{6}}{\frac{1}{180}} (x^2 - x + 1/6)\)
\(P_s(e^x) = (e-1) + 6(3-e)(x - 1/2) + 30(7e-19) (x^2 - x + 1/6)\)

\(d(e^x,s) = \sqrt{||e^x||^2 - ||P_s(e^x)||^2}\)
\(= \sqrt{\frac{e^2-1}{2} - 249e^2 + 1350e - 1833}.\)