Skip to content

5.14 Orthogonal complement

Let \((V,\lang,\rang)\) be k-IPVS and \(S\) be a subset of \(V\)

Definition

The orthogonal complement to \(S\) is \(S^{\perp}=\{v\in V:\underbrace{v\perp w}_{\lang v,w\rang =0},\forall w\in S\}\)

Example

  1. \(S=\{0_V\}\), \(S^\perp =V\) since \(\forall v\in V,\lang v,0_V\rang =0\)​

  2. \(S=V\), \(S^\perp =\{0_V\}\). Suppose \(v\in S^\perp\), then \(\lang v,v\rang =0\), then \(||v||^2=0\), then \(v=0_V\)​

  3. \(V=\R^2\), \(\R\)-vector space. \(\lang,\rang\) is standard inner product

    \(S=\{v_0=(2,3)\}\), then \(S^{\perp}=\{v=(x_1,x_2)\in \R^2:v\perp w,\forall w\in S\}\)​

    Then \(\lang v,(2,3)\rang =0\), then \(2x_1+3x_2=0\). Then \(x_2=-\frac{2}{3}x_1\)​

    Thus \(S^{\perp}=\{(x_{1},-\frac{2}{3}x_{1}):x_{1}\in\mathbb{R}^{2}\}=\lang (1,-\frac{2}{3})\rang =\lang (3,-2)\rang\)​

    Let \(S'=\lang (2,3)\rang=\{(2z,3z):z\in \R\}\), then \((S')^{\perp}=\{(x_{1},x_{2}):\lang (x_{1},x_{2}),(2z,3z)\rang =0,\forall z\in \R \}\)​

    Then \(\langle(x_{1},x_{2}),(2z,3z)\rangle=z\lang(x_1,x_2),(2,3)\rang\), since \(z\in \R\), then \(\lang(x_1,x_2),(2,3)\rang=0\)​

    Then \((x_1,x_2)\in \lang(3,-2)\rang\)​

Remark

  1. \(\forall S\) be subset of \(V\), then \(S^\perp\) is a subspace

    Given \(S\) such that \(0_V\in S^\perp\) and If \(v_1,v_2\in S^\perp\), then given \(w\in S\), \(\lang v_{1}+v_{2},w\rang =\lang v_{1},w\rang+\lang v_{2},w\rang\)​

    Since \(v_1,v_2\in S^\perp\), then \(\lang v_{1},w\rang =0,\lang v_{2},w\rang =0\). Then \(\langle v_{1}+v_{2},w\rangle=0\)​

    If \(v_1\in S^\perp\), \(\lambda\in k\), then given \(w\in S\): \(\lang \lambda v_1,w\rang =\lambda \lang v_1,w\rang=\lambda \cdot 0=0\)​

  2. Given \(S=\{w_1,...,w_r\}\subseteq V\), then \(S^\perp=\lang S\rang ^\perp\)​

    We have \(S\subseteq \lang S\rang\), using 3) we get \(S^\perp\supseteq \lang S\rang^\perp\). Then we must prove that \(S^\perp\subseteq \lang S\rang^\perp\)​

    Given \(v\in S^\perp\), given \(w\in\lang S\rang\), we want to prove that \(\lang v,w\rang=0\)​

    \(\exists w_1,...,w_r\in S,\alpha_{1},...,\alpha_{r}\in k:w=\sum^{r}_{i=1}\alpha_{i}w_{i}\). so \(\langle v,w\rangle=\langle v,\sum_{i=1}^{r}\alpha_{i}w_{i}\rangle=\sum_{i=1}^{r} \bar{\alpha}_{i}\langle v,w_{i}\rangle\)​

    Since \(v\in S^\perp\) and \(w_i\in S\), then \(\lang v,w_i\rang =0\). Then \(\lang v,w\rang =0\)​

  3. Given subsets \(S,T\) of \(V\) such that \(S\subseteq T\), then \(S^\perp\supseteq T^\perp\)​

    Suppose \(v\in T^\perp\), then \(\lang v,w\rang =0\), \(\forall w\in T\)​

    Since \(S\subseteq T\), we get that \(\lang v,w\rang =0\), \(\forall w\in S\subseteq T\), then \(v\in S^\perp\)​

  4. \(v\in V, W\) is subspace of \(V\), \(w\) is the best approximation to \(v\) in \(W\)​\(\iff\)​\(v-w\in W^\perp\)​

    \(w\) is also called the \(\perp\) projection of \(v\) to \(W\)

  5. \(W\subseteq V\), \(w\in W\), \(\forall z\in W^\perp\), then \(\lang w,z\rang=0\), then \(w\in (W^{\perp})^\perp\), so \(W\subseteq (W^\perp)^\perp\)

Example

\(V=\R^2\), \(\R\)-v.s. with standard Inner Product

\(W=\lang (-3,1)\rang,v=(2,3)\), we want to find \(\perp\) projection of \(V\) to \(W\), it is \(w\in W:v-w\in W^\perp\)

Let \(w=\alpha(-3,1)\) for some \(\alpha\in \R\), then \(v-w=(2,3)-\alpha(-3,1)=(2+3\alpha,3-\alpha)\in W^\perp=\{(-3,1)\}^\perp\)​

Then \(0=\lang (2+3\alpha,3-\alpha),(-3,1)\rang=-3\cdot(2+3\alpha)+1\cdot(3-\alpha)\), then \(\alpha=-\frac{3}{10}\)

Then \(w=-\frac{3}{10}(-3,1)\)

Proposition

Let \((V,\lang,\rang)\) be k-IPVS and \(W\subseteq V\) be a finite dimension subspace. Let \(p:V\to W\) be the orthogonal projection to \(W\), that is \(p(v)=w\) such that \(v-w\in W^\perp\) for unique \(w\)

  1. \(p\) is a k-linear map

  2. The map \(Id_{V}-p:V\to V\) is the orthogonal projection to \(W^\perp\)

Proof

  1. Given \(v_1,v_2\in V\), we want to prove that \(p(v_1+v_2)=p(v_1)+p(v_2)\)​

    Let \(w_1=p(v_1),w_2=p(v_2),w_1,w_2\in W\) and \(v_1-w_1\in W^\perp,v_2-w_2\in W^\perp\)​

    Then we want to prove \((v_1+v_2)-(w_1+w_2)\in W^\perp\)​

    Given \(z\in W\), consider \(\lang(v_{1}+v_{2})-(w_{1}+w_{2}),z\rang=\lang\underbrace{(v_{1}-w_{1})}_{\in W^\perp} +\underbrace{(v_{2}-w_{2})}_{\in W^\perp},\underbrace{z}_{\in W}\rang\)

    Since \(W^\perp\) is subspace, then \((v_{1}-w_{1})+(v_{2}-w_{2})\in W^\perp\)​

    Given \(v\in V,\lambda \in k\). NTP: \(p(\lambda v)=\lambda \cdot p(v)\)

    Let \(w = P(\lambda v)\), \(w \in W\)​\(\implies \lambda v - w \in W^{\perp}\). NTP \(v - w \cdot \frac{1}{\lambda}\in W^{\perp}\)

    Given \(z \in W\), \(\langle v-\frac{w}{\lambda},z\rangle=\frac{1}{\lambda}\langle\lambda v-w,z\rangle =0\).

  2. Given \(v\in V\), we have to prove that \((Id_V-p)(v)\) is the best approximation of \(v\) to \(W^\perp\)​

    This is equivalent to \(v-(Id_V-p)(v)\in (W^\perp)^\perp\)​

    Another way to prove the statement is to prove that \(||v-(Id_V-p)(v)||\leq ||v-z||\), \(\forall z\in W^\perp\)​

    \(\iff\)​\(||p(v)||^{2}\leq||v-z||^{2}\), \(\forall z\in W^\perp\)​

    Given \(z\in W^\perp\), \(||v-z||^{2}=||\underbrace{v-p(v)-z}_{\in W^\perp}+\underbrace{p(v)}_{\in W}||^{2}=||p(v)||^2+||v-p(v)-z||^2+0\geq ||p(v)||^2\)​

Remark

\(p\) is well-defined because \(W\) is finite dimension

Theorem

Let \(W\) be a finite dimensional subspace of \(V\). Let \(p\) be the orthogonal projection to \(W\). Then

  1. \(p^2=p\)​

  2. \(\ker p=W^\perp\) and \(V=W\oplus W^\perp\)​

Proof

  1. Let \(v\in V\), \(p(v)\in W\), let \(p(p(v))\) be the best approximation of \(p(v)\) to \(W\).

    \(p^2(v)=p(p(v))\in W\) such that \(||p(v)-p(p(v))||\leq ||p(v)-z||,\forall z\in W\)

    Take \(z=p(v)\in W\), then \(||p(v)-z||=||p(v)-p(v)||=0\), then \(||p(v)-p(p(v))||=0\)​\(\iff\)​\(p(v)=p(p(v))=p^2(v)\)

  2. \(v\in W^{\perp},v-0_V\in W^{\perp}\iff v-p(v)\) is such that \(v-p(v)\in W^\perp\), then \(v-p(v)=v-0_{V}\) since \(p(v)\) and best approximation is unique\(\;\char"27FA \;p(v)=0_{V}\;\char"27FA \;v\in\ker p\)

    Thus \(\ker p=W^\perp\)

    Remark: We've strongly used unicity of the best approximation

    image​

    Suppose \(v\in W\cap W^\perp\), \(\lang \underbrace{v}_{\in W},\underbrace{v}_{\in W^\perp}\rang=0=||v||^2\), then \(v=0_V\), then \(W\cap W^{\perp} =\{0_V\}\)​

    Let's prove that \(V=W+W^\perp\). Given \(v\in V\), \(v=\underbrace{v-p(v)}_{\in W^\perp}+\underbrace{p(v)}_{\in W}\)​

Corollary

  1. If \(\dim V=n\) and \(W\) is subspace of \(V\)such that \(\dim W=r\), then \(\dim W^{\perp} = n-r\)

  2. If \(\dim V=n\) and \(W\) is subspace of \(V\)such that \(\dim W=r\).

    We know \(W\subseteq (W^{\perp})^{\perp}\), then \(\dim(W^{\perp})^{\perp} =n-\dim W^{\perp}=n-(n-r)=r\)​

    Then \(W=(W^{\perp})^{\perp}\)​

Given \(v_1,...,v_n\in V\) and \(i\) s.t. \(2\leq i\leq n+1\)\, consider the \(\perp\) projection \(p_i\) to \(\lang v_{1},...,v_{i-1}\rang^\perp\)

For \(i=1\), define \(p_{1}=Id_{V}\). Apply Theorem(Gram-Schmidt) to \(\{v_1,...,v_n\}\) and get \(\{w_1,...,w_n\}\)

Then Exercise: prove that \(w_{i}=p_{i}(v_{i}),\forall i,1\leq i\leq n\)

\(P_1 = Idv\), then \(P_1(v_1) = v_1 = w_1\)

let \(w_{2}=v_{2}-\frac{\langle v_{2},w_{1}\rangle}{||w_{1}||^{2}}w_{1}\)

\(P_{2}(v_{2})=w_{2}\Leftrightarrow v_{2}-w_{2}\in\langle v_{1}\rangle\)

\(v_{2}-w_{2}=\frac{\langle v_{2},w_{1}\rangle}{||w_{1}||^{2}}w_{1}=\frac{\langle v_{2},v_{1}\rangle}{||v_{1}||^{2}}v_{1}\in\langle v_{1}\rangle\)

\(v_{i+1}-w_{i+1}=\sum_{k=1}^{i}\frac{\langle v_{i+1},w_{k}\rangle}{||w_{k}||^{2}} w_{k}\in\langle v_{1},\dots,v_{i}\rangle\)

\(\text{since} \ v_{i} - w_{i} \in \lang v_{1}, \dots, v_{i-1}\rang\)

‍