5.7 Inner Product

Definition of Norm

Given a vector \(v\in V\), then Norm of \(v\) is \(||v||=\sqrt{\underbrace{\lang v,v\rang}_{\in \R_{\geq 0}}}\)

Example

\(V=\R^2\) with standard inner product such that \(\lang(x_1,y_1),(x_2,y_2)\rang =x_1x_2+y_1y_2\)

Given \(v\in \R^2\), \(v=(x_1,y_1)\), then \(\lang v,v\rang = x_1^2+y_1^2\), then \(||v||=\sqrt{x_1^2+y_1^2}\)​

Proposition

Let \((v,\lang ,\rang)\) be a k-IPVS, let \(v,w\in V,\alpha \in k\)

1. \(||\alpha \cdot v||=|\alpha|\cdot ||v||\)

2. \(||v||>0\) if \(v\neq 0_V\)

3. \(|\lang v,w\rang|\leq ||v||\cdot ||w||\) Cauchy-Schwartz Inequality

4. \(||v+w||\leq ||v||+||w||\) Triangular Inequality

Proof

1. \(||\alpha\cdot v||=\sqrt{\langle\alpha\cdot v,\alpha\cdot v\rangle}=\sqrt{\alpha\cdot\langle v,\alpha\cdot v\rangle}=\sqrt{\alpha\cdot\bar{\alpha}\langle v,v\rangle}=\sqrt{\alpha^{2}} \sqrt{\langle v,v\rangle}=\left|\alpha\right|\cdot ||v||\)

2. If \(v\neq 0_V\), then \(||v||=\sqrt{\lang v,v\rang}\). Then \(\lang v,v\rang >0\), then \(||v||>0\)​

3. Case 1: Suppose \(v=0_V\), then \(|\langle0_{V},w\rangle|=0\leq0\cdot||w||=||v||\cdot ||w||\)

   Case 2: Suppose \(v\neq 0_V\), then let \(z=w-\frac{\lang w,v\rang\cdot v}{||v||^{2}}\)

   Then \(\langle z,v\rangle=\langle w-\frac{\langle w,v\rangle\cdot v}{||v||^{2}},v\rangle =\langle w,v\rangle-\frac{\langle w,v\rangle}{||v||^{2}}\langle v,v\rangle=\langle w,v\rangle-\frac{\langle w,v\rangle}{||v||^{2}}||v||^{2}=0\)

   Let call \(\frac{\langle w,v\rangle}{||v||^{2}}=\lambda\), we know that \(0\leq ||z||^{2}=\lang z,z\rang=\lang z,w-\lambda v\rang=\lang z,w\rang -\bar\lambda \lang z,v\rang=\lang z,w\rang=\lang w-\lambda v,w\rang=\lang w,w\rang -\lambda \lang v,w\rang=||w||^{2}-\lambda \lang v,w\rang=||w||^{2}-\frac{\lang w,v\rang}{||v||^{2}} \lang v,w\rang =||w||^{2}-\frac{\overline{\lang v,w\rang}}{||v||^{2}}\lang v,w\rang =||w||^{2}-\frac{|\lang v,w\rang|^{2}}{||v||^{2}}\)

   Thus \(\frac{|\lang v,w\rang|^{2}}{||v||^{2}} \leq ||w||^{2}\Rightarrow|\lang v,w\rang|^{2}\leq ||w||^2||v||^2\Rightarrow |\lang v,w\rang |^2\leq ||w||\cdot ||v||\)

   ​

4. Consider \(||v+w||\leq ||v||+||w||\iff ||v+w||^{2}\leq (||v||+||w||)^{2}=||v||^{2}+||w||^{2} +2||v||\cdot||w||\)​

   Then \(||v+w||^{2}=\langle v+w,v+w\rangle=\langle v,v\rangle+\langle v,w\rangle+\langle w,v\rangle+\langle w,w\rangle=||v||^2+\langle v,w\rangle+\overline{\langle v,w\rangle} +||w||^2=||v||^2+2\text{Re}\lang v,w\rang +||w||^2\)​

   Thus we only need to prove \(\text{Re}\langle v,w\rangle\leq||v||\cdot||w||\)​

   Since \(\text{Re}\lang v,w\rang \leq|\lang v,w\rang|\leq ||v||\cdot ||w||\), then we finish this proof

Definition of Orthogonal

Let \(v,w\in V\), \((V,\lang ,\rang)\) be k-IPVS

• We say that \(v\) is Orthogonal(Perpendicular) to \(w\)​\(\iff\)​\(\lang v,w\rang =0\)
• We say that a subset \(S\) of \(V\) is Orthogonal\(\iff\)​$\forall v,w\in S,v\perp w $​
• We say that a subset \(S\) of \(V\) is Orthonormal\(\iff\)​\(S\) is orthogonal and \(\forall v\in S,||v||=1\)

Example​

\(v = (x_1, y_1) \ne 0_v\), \(w = (x_2, y_2) \ne 0_v\), \(0 = \langle v, w \rangle = x_1 x_2 + y_1 y_2\) \(\Leftrightarrow x_1 x_2 = -y_1 y_2\)

Let \(v = (1, 2)\), take \(w = (-2, 1)\) since \(\frac{1}{2} = \frac{x_1}{y_1} = -\frac{y_2}{x_2} \Leftrightarrow y_2 = -\frac{x_2}{2}\)

Remark

1. \(\forall v\in V,v\perp 0_V\) since \(\lang v,0_V\rang=0\)​

2. \(\forall v,w\in V\), \(v\) is orthogonal to \(w\)​\(\iff\)​\(w\) is orthogonal to \(v\), that is \(\lang v,w\rang =0\iff \lang w,v\rang =0\)​

3. \(\forall v,w,z\in V\), if \(v\perp w,v\perp z\), then \(v\perp(\alpha w+\beta z),\forall \alpha,\beta \in k\)​

---

Example

1. \(V = \mathbb{C}^n\) or \(\mathbb{R}^n\) with the standard inner product, then \(E = \{e_1, ..., e_n\}\) (the standard basis) is an orthonormal set.

2. \(V = \mathbb{R}^2\) with the std. inner product. \(\forall v=(x_{1},y_{1})\in\mathbb{R}^{2}\), then \(v \perp (-y_1, x_1)\)

Proposition

Let \((V,\lang ,\rang )\) be a k-IPVS, let \(S\) be an orthogonal set such that \(S\subseteq V\) and \(0_V\notin S\)

Then \(S\) is linearly independent

Proof

Given \(v_1,...,v_n\in S\), let \(V=\alpha_{1}v_{1}+...+\alpha_{n}v_{n}=\sum^{n}_{i=1}\alpha_{i}v_{i}\)

Given \(j,1\leq j\leq n\), then \(\langle v,v_{j}\rangle=\langle\sum_{i=1}^{n}\alpha_{i}v_{i},v_{j}\rangle=\sum_{i=1} ^{n}\alpha_{i}\langle v_{i},v_{j}\rangle\)

Since \(S\) is orthogonal, then it's equal to \(\alpha_j||v_j||^2\)

If \(v=0_V\), then \(\forall j,1\leq j\leq n\), \(0=\lang v,v_{j}\rang =\alpha_{j}\cdot ||v_{j}||^{2}\)

Since \(0_V\notin S\), then \(v_j\neq 0_V\), then \(\alpha _j=0\). Thus \(S\) is linearly independent

Corollary

1. If \(\dim V=n\) and \(S\) is \(\perp\), \(0_V\notin S\), then \(\#S\leq n\)​

2. Let \(\{v_1,...,v_n\}\) be an \(\perp\) set and \(v_i\neq 0_V,\forall i\), suppose \(v=\sum^n_{i=1}\alpha_{i}v_{i}\)​

   Then \(\lang v,v_j\rang =\alpha_j\cdot ||v_j||^2\), since \(||v_{j}||^{2}\neq 0\), then \(\alpha_j=\frac{\lang v,v_j\rang}{||v_j||^2}\)​

Remark corresponding to corollary 2

If \(B\) is an \(\perp\) basis of \(V\) and \(\dim V<\infty\), then given \(v\in V\), then \(v=\sum^{n}_{i=1}\frac{\lang v,v_{i}\rang}{||v_i||^2}v_i\)

‍