Skip to content

7.29 Hilbert Spaces and Tensor Products

Inner product space

Definition

An inner product \(\mathbb{F}\)-vector space \(A\) is a \(\mathbb{F}\)-vector space \(A\) together with a map \(\lang\cdot|\cdot\rang:A\times A\to \mathbb{F}\) where \(\mathbb{F}=\{\mathbb{C},\mathbb{R}\}\) and \(A=\{\mathbb{C}^n,\mathbb{F}^n\}\)(finite dimension) such that:

  1. Conjugate symmetry: \(\forall \psi,\phi\in A ,\lang \psi|\phi\rang = \overline{\lang \phi|\psi\rang}\)

  2. Linearity in the second argument: \(\forall \psi,\phi,\chi\in A,\forall c\in \mathbb{F}\)

    • \(\lang\psi|\phi+\chi\rang=\lang \psi|\phi\rang +\lang \psi|\chi\rang\)
    • \(\lang \psi|c\phi\rang=c\cdot\lang \psi|\phi\rang\)

  3. \(\lang \psi|\psi\rang \geq 0\) with equality iff \(\psi = 0\)

Example

  • Dot product
  • Integral Product: \(\int^b_ag(x)\bar f(x)=\lang f|g\rang\)
  • Hilbert Schmidt inner product: \(\forall M,N\in\mathbb{C}^{n\times n}\), \(\lang M|N\rang_{HS}:=\text{Tr}[M^*N]\) (Check).

These are all inner product.

Norm

\(||\psi||_{2}=\sqrt{\lang \psi|\psi\rang},\forall \psi\in A\quad\quad d(\psi,\phi)=||\psi-\phi||_{2}\)

Properties

  1. \(\forall \psi\in A,||\psi||\geq 0\)

  2. \(||c\psi||=|c|\cdot ||\psi||\)

  3. \(||\psi+\phi||\leq ||\psi||+||\phi||\)

A vector space equipped with a norm is called a normed space.

Homework 1: Let A be a normed space, and let \(\psi, \phi \in A\) with \(\|\phi\|=1\). Show that
​\(\left\|\frac{1}{\|\psi\|}\psi-\phi\right\| \leq 2\|\psi-\phi\|\)

Proof

\(\left\|\frac{1}{\|\psi\|}\psi-\phi\right\|= \left\lang \frac{1}{\|\psi\|}\psi-\phi \,\Big|\,\frac{1}{\|\psi\|}\psi-\phi \right\rang = \left\lang \frac{1}{\|\psi\|}\psi \,\Big|\,\frac{1}{\|\psi\|}\psi\right\rang + \lang \phi |\phi\rang -\left\lang \phi \,\Big|\,\frac{1}{\|\psi\|}\psi \right\rang-\left\lang \frac{1}{\|\psi\|}\psi \,\Big|\,\phi \right\rang\)(This is not inner product space. So I cannot do this)

Proof

\(\left\|\frac{1}{\|\psi\|}\psi-\phi\right\|=\left\|\frac{1}{\|\psi\|}\psi+\psi-\psi -\phi\right\|\leq\left\|\frac{1}{\|\psi\|}\psi-\psi\right\|+\left\|\psi-\phi\right \|\)
\(=\left|\frac{1}{\|\psi\|}-1\right|\left\|\psi\right\|+\left\|\psi-\phi\right\|=|1-||\psi|||+||\psi-\phi||=|\,||\phi||-||\psi||\,|+||\psi-\phi||\)
\(\leq ||\psi-\phi||+||\psi-\phi||=2||\psi-\phi||\)

Note that not all norms are derived from an inner product, and as a result, not every metric necessarily originates from either an inner product or a norm.

The p-Norm

\(\forall \psi \in A \cong\mathbb{C}^{n},\psi= \begin{pmatrix} a_{1} \\ a_{2} \\ \vdots \\ a_{n} \end{pmatrix}，||\psi||_p:=(|a_1|^p+|a_2|^p+...+|a_n|^p)^{1/p}\) where \(p\geq 1\)
This is induced by \(p=2\), \(\lang\psi|\psi\rang ^{1/2}=||\psi||_2\)

When \(p=\infty\), \(||\psi||_\infty=\max\limits_{x\in[n]}|a_x|\)(Comparison test), when \(p=1\), \(||\psi||_1=|a_1|+|a_2|+...+|a_n|\)

Exercise 2.3 (The \(p\)-Norms and the Hölder Inequality).

The normed space \(\ell^p(\mathbb{C}^n)\)(with \(p \in [1, \infty]\)) is the vector space \(\mathbb{C}^n\) equipped with the \(p\)-norm, \(\|\cdot\|_p\) defined on all \(\psi=(a_1, \dots, a_n)^T \in \mathbb{C}^n\) as \(\|\psi\|_{p}:=\left(|a_{1}|^{p}+\dots+|a_{n}|^{p}\right)^{1/p}\)
The Hölder inequality states that for any \(p, q \in [1, \infty]\) with \(\frac{1}{p} + \frac{1}{q} = 1\) and any \(\psi = (a_1, \dots, a_n)^T \in \mathbb{C}^n\)and \(\phi = (b_1, \dots, b_n)^T \in \mathbb{C}^n\), we have \(\sum_{x\in[n]}|a_{x}b_{x}|\leq\|\psi\|_{p}\|\phi\|_{q}\)

Use this to show that \(\|\cdot\|_p\) is a norm, and that for \(p \neq 2\) this norm is not induced from an inner product.

Proof

The first two properties are easy, let's see the third

NTP: \(||\psi+\phi|| \le ||\psi|| + ||\phi||\)
​\(||\psi+\phi||_{p} = (|a_{1}+b_{1}|^{p} + |a_{2}+b_{2}|^{p} + \dots + |a_{n}+b_{n} |^{p})^{\frac{1}{p}}\)
​\(||\psi+\phi||_{p}^{p} = \sum_{x \in [n]}|a_{x}+b_{x}|^{p} = \sum_{x \in [n]}|a_{x} +b_{x}||a_{x}+b_{x}|^{p-1}\le \sum_{x \in [n]}|a_{x}||a_{x}+b_{x}|^{p-1}+ |b_{x}| |a_{x}+b_{x}|^{p-1}\)
​\(\le ||\psi||_{p_1} ||\phi'||_{q_1} + ||\phi||_{q_2} ||\psi''||_{p_2}\) where \(\phi'=(...,(a_x+b_x)^{p-1},...)\) and \(\psi'=(...,(a_x+b_x)^{p-1},...)\)
Where \(\frac{1}{p_1} + \frac{1}{q_1} = 1\), \(p_1=p \Rightarrow \frac{1}{q_1} = 1-\frac{1}{p} = \frac{p-1}{p} \Rightarrow q_1=\frac{p}{p-1}\)
And \(\frac{1}{q_2} + \frac{1}{p_2} = 1\), \(q_2=p \Rightarrow \frac{1}{p_2} = 1-\frac{1}{p} \Rightarrow p_2=\frac{p}{p-1}\)
Since \(||\phi'||_{\frac{p}{p-1}} = \left( \sum_{n \in [n]} |(a_x+b_x)^{p-1}|^{\frac{p}{p-1}} \right)^{\frac{p-1}{p}} = \left( \sum_{n \in [n]} |a_x+b_x|^p \right)^{\frac{p-1}{p}} = (||\psi+\phi||_p^p)^{\frac{p-1}{p}} = (||\psi+\phi||_p)^{p-1}\) and \(\psi''=(...,a_{x}+b_{x},...)\)
We get \(||\psi+\phi||_{p}^{p}=||\psi||_{p}||\phi^{\prime}||_{\frac{p}{p-1}}+||\phi||_{p} ||\psi^{\prime}||_{\frac{p}{p-1}}=(\|\psi\|_{p} + \|\phi\|_{p}) \times (\|\psi''\|_{p})^{p-1} = (\|\psi\|_{p} + \|\phi\|_{p}) (\|\psi+\phi\|_{p})^{p-1}\)
Then \(||\psi+\phi||_p^p \leq (||\psi||_p+||\phi||_p) \cdot (||\psi+\phi||_p)^{p-1}\) \(\Rightarrow ||\psi+\phi||_p \leq (||\psi||_p+||\phi||_p)\)

Stochastic matrix

Let \(S = (s_{xy}) \in \text{STOC}(m,n)\) be an \(m \times n\) column stochastic matrix where \(\sum_{x=1}^m s_{xy}=1,\forall y\in [1,n]\) and \(s_{xy}\geq 0,\forall x,y\)

Then \(\forall S\in \R^{m\times n},\forall\psi\in\mathbb{C}^{n},\|S\psi\|_{1}\leq\|\psi\|_{1}\), that is \(\forall \vec v\in \mathbb{C^{n}}\), \(||S\vec v||_{1}\leq ||\vec v||_{1}\)

Proof

For \(S=(s_{xy}),||S\vec v||_{1}=\sum^{m}_{x=1}|(S\vec v)_{x}|=\sum^{m}_{x=1}\left|\sum ^{n}_{y=1}s_{xy}v_{y}\right|\leq \sum_{y}\underbrace{\sum_{x}s_{xy}}_{=1}|v_{y}|= \sum_{y}|v_{y}|=|| \vec v||_{1}\)

Example

An \(3\times 3\) column stochastic matrix: \(S=\begin{pmatrix} \frac{1}{4}&\frac{1}{3}&0\\ \frac{1}{2}&\frac{1}{3}&1\\ \frac{1}{4}&\frac{1}{3}&0 \end{pmatrix}\)

Homework 2: Let \(R \in \mathbb{R}_{+}^{m \times n}\) be a row stochastic matrix. Show that \(\|R \vec v\|_{\infty}\leq\|\vec v\|_{\infty}\quad \forall \vec v\in \mathbb{C}^{n}\)

Proof

\(\|R\vec{v}\|_{\infty}=\max_{x\in\left\lbrack m\right\rbrack}|\sum_{y=1}^{n}r_{xy} v_{y}|=\left|\sum_{y=1}^{n}r_{iy}v_{y}\right|\leq\sum_{y=1}^{n}r_{iy}\left|v_{y}\right |\) for some \(i\in[m]\)

Then \(||R\vec{v}||\leq\sum_{y=1}^{n}r_{iy}\max|v_{y}|=\sum_{y=1}^{n}r_{iy}|v_{k}|\) for some \(k\in [n]\)

Then \(\|R\vec{v}\|_{\infty}\leq|v_{k}|\leq\max_{j\in[n]}|v_{j}|=||\vec{v}||_{\infty}\) since the sum of entries in a row is \(1\)​

\(A\)-Hilbert space

Inner product space \(A\) is complete if for any Cauchy sequence, \(\{\psi_n\}_{n \in \mathbb{N}} \subseteq A\), there exists a state \(\psi \in A\) such that \(\lim_{n \to \infty} \|\psi_n - \psi\| = 0\).

Complete inner product spaces are called Hilbert spaces

Dirac Notations

Ket

\(|\psi\rang,|\phi\rang,|\chi\rang\in A\) and for example: For standard basis of \(\mathbb{C}^{2}:\) \(|0\rang =\begin{pmatrix} 0\\1 \end{pmatrix},|1\rang =\begin{pmatrix} 0\\1 \end{pmatrix}\)

For standard basis of \(\mathbb{C}^n:|1\rang,|2\rang,...,|n\rang\)

Bra

\(\lang 0|:= (1,0),\lang 1|:=(0,1)\).

Moreover, \(\lang \psi|=(\bar a_{1},\bar a_{2},...,\bar a_{n}),|\phi\rang := \begin{pmatrix} b_{1} \\ b_{2} \\ \vdots \\ b_{n} \end{pmatrix}\)

\(\lang \psi|\psi\rang = \sum_{x\in [n]}|a_x|^2\), \(|\psi\rang =\sum_{x\in[n]}a_x|x\rang\), \(\lang \psi |=\sum_{x\in [n]}\bar a_{x}\lang x|\)

\(\lang \phi|\psi\rang =\sum_{x\in[n]}\bar b_xa_x\)

Ket and Bra are dual, so \((|\phi\rang)^*=\lang \phi|\)

Tensor Products(KronecKer)

\(\vec v\in \mathbb{C}^n,\vec u\in \mathbb{C}^n\), then \(\vec v\otimes \vec u=\begin{pmatrix} v_1\\v_2\\\vdots\\v_n \end{pmatrix}\otimes \begin{pmatrix} u_1\\u_2\\\vdots\\ u_m \end{pmatrix}:=\begin{pmatrix} v_1\vec u\\ v_2\vec u\\ \vdots\\ v_n\vec u \end{pmatrix}\in \mathbb{C}^{nm}\) where this is a block matrix and \(\forall j\in [n],v_j\vec u=\begin{pmatrix} v_ju_1\\v_ju_2\\\vdots \\v_ju_m \end{pmatrix}\)

Moreover, \(M=(M_{xy})\in \mathbb{C}^{n\times m},M\otimes N:= \begin{pmatrix} m_{11}N & m_{12}N & \cdots & m_{1m}N \\ m_{21}N & m_{22}N & \cdots & m_{2m}N \\ \vdots & \vdots & \ddots & \vdots \\ m_{n1}N & m_{n2}N & \cdots & m_{nm}N \\ \end{pmatrix}\)

Properties

  1. \((\left|\psi_1\right\rangle + \left|\psi_2\right\rangle) \otimes \left|\phi\right\rangle = \left|\psi_1\right\rangle \otimes \left|\phi\right\rangle + \left|\psi_2\right\rangle \otimes \left|\phi\right\rangle\)

  2. \(\left|\psi\right\rangle \otimes (\left|\phi_1\right\rangle + \left|\phi_2\right\rangle) = \left|\psi\right\rangle \otimes \left|\phi_1\right\rangle + \left|\psi\right\rangle \otimes \left|\phi_2\right\rangle\)

  3. \(c (\left|\psi\right\rangle \otimes \left|\phi\right\rangle) = (c\left|\psi\right\rangle) \otimes \left|\phi\right\rangle = \left|\psi\right\rangle \otimes (c\left|\phi\right\rangle)\).

  4. \((A \otimes B)^{*} = A^{*} \otimes B^{*}\)

  5. \((M_A \otimes M_B)(|\psi\rangle \otimes |\phi\rangle) = (M_A|\psi\rangle) \otimes (M_B|\phi\rangle)\)

  6. \((\langle\psi_1| \otimes \langle\phi_1|) \cdot (|\psi_2\rangle \otimes |\phi_2\rangle) = \langle\psi_1|\psi_2\rangle \cdot \langle\phi_1|\phi_2\rangle\)

Example

\(\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}\otimes \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{pmatrix}= \begin{pmatrix} 1 \cdot\begin{pmatrix}1 & 0 \\0 & 1 \\1 & 1\end{pmatrix} & 2 \cdot\begin{pmatrix}1 & 0 \\0 & 1 \\1 & 1\end{pmatrix} \\ \\ 3 \cdot\begin{pmatrix}1 & 0 \\0 & 1 \\1 & 1\end{pmatrix} & 4 \cdot\begin{pmatrix}1 & 0 \\0 & 1 \\1 & 1\end{pmatrix} \end{pmatrix}= \begin{pmatrix} 1 & 0 & 2 & 0 \\ 0 & 1 & 0 & 2 \\ 1 & 1 & 2 & 2 \\ 3 & 0 & 4 & 0 \\ 0 & 3 & 0 & 4 \\ 3 & 3 & 4 & 4 \end{pmatrix}\)

Homework 3: Let \(M\) be \(m\times n\) matrix and \(N\) be \(n \times k\) matrix
Find all values of \(m,n,k\) for which \(M\otimes N=MN\)

Solution

\(M\otimes N= \begin{pmatrix} m_{11}N & m_{12}N & \cdots & m_{1n}N \\ m_{21}N & m_{22}N & \cdots & m_{2n}N \\ \vdots & \vdots & \ddots & \vdots \\ m_{m1}N & m_{m2}N & \cdots & m_{mn}N \end{pmatrix}\in \mathbb{C}^{mn\times nk}\)

\(MN= \begin{pmatrix} \sum_{j=1}^{n}m_{1j}n_{j1} & \sum_{j=1}^{n}m_{1j}n_{j2} & \cdots & \sum_{j=1}^{n}m_{1j}n_{jk} \\ \sum_{j=1}^{n}m_{2j}n_{j1} & \sum_{j=1}^{n}m_{2j}n_{j2} & \cdots & \sum_{j=1}^{n}m_{2j}n_{jk} \\ \vdots & \vdots & \ddots & \vdots \\ \sum_{j=1}^{n}m_{mj}n_{j1} & \sum_{j=1}^{n}m_{mj}n_{j2} & \cdots & \sum_{j=1}^{n}m_{mj}n_{jk} \end{pmatrix}\in\mathbb{C}^{m\times k}\)

Then \(mn=m,nk=k\), then \(n=1,k,m\in\mathbb{N}\)

Then \(M\otimes N= \begin{pmatrix} m_{11}n_{11} \\ m_{21}n_{11} \\ \vdots \\ m_{m1}n_{11} \end{pmatrix}\in\mathbb{C}^{m\times k}\)

\(MN= \begin{pmatrix} m_{11}n_{11} & m_{11}n_{12} & \cdots & m_{11}n_{1k} \\ m_{21}n_{11} & m_{21}n_{12} & \cdots & m_{21}n_{1k} \\ \vdots & \vdots & \ddots & \vdots \\ m_{m1}n_{11} & m_{m1}n_{12} & \cdots & m_{m1}n_{1k} \end{pmatrix}\in\mathbb{C}^{m\times k}\)

Clearly, they are always equal, then the condition is \(n=1,k,m\in\mathbb{N}\)

Exercise 2.10. Show that the Kronecker product is not commutative, and that there always exist permutation matrices \(P\) and \(Q\) in appropriate dimensions such that \(M \otimes N = P (N \otimes M) Q\).

Proof
​\(M\otimes N= \begin{pmatrix} M_{11}N & \dots & m_{1k}N \\ \vdots & \ddots & \vdots \\ M_{kl}N & \dots & M_{kl}N \end{pmatrix}= \begin{pmatrix} \begin{bmatrix}m_{11}n_{11}&\dots&m_{11}n_{1q}\\ \vdots&\ddots&\vdots\\ m_{11}n_{p1}&\dots&m_{11}n_{pq}\end{bmatrix} & \dots & \begin{bmatrix}m_{1k}n_{11}&\dots&m_{1k}n_{1q}\\ \vdots&\ddots&\vdots\\ m_{1k}n_{p1}&\dots&m_{1k}n_{pq}\end{bmatrix} \\ \vdots & \ddots & \vdots \\ \begin{bmatrix}m_{k1}n_{11}&\dots&m_{k1}n_{1q}\\ \vdots&\ddots&\vdots\\ m_{k1}n_{p1}&\dots&m_{k1}n_{pq}\end{bmatrix} & \dots & \begin{bmatrix}m_{kl}n_{11}&\dots&m_{kl}n_{1q}\\ \vdots&\ddots&\vdots\\ m_{kl}n_{p1}&\dots&m_{kl}n_{pq}\end{bmatrix} \end{pmatrix}\)

\(N \otimes M = \begin{pmatrix} n_{11}M & \dots & n_{1q}M \\ \vdots & \ddots & \vdots \\ n_{p1}M & \dots & n_{pq}M \end{pmatrix}\) \(= \begin{pmatrix} \begin{bmatrix}n_{11}m_{11}&\dots&n_{11}m_{1l}\\ \vdots&\ddots&\vdots\\ n_{11}m_{k1}&\dots&n_{11}m_{kl}\end{bmatrix} & \dots & \begin{bmatrix}n_{1q}m_{11}&\dots&n_{1q}m_{1l}\\ \vdots&\ddots&\vdots\\ n_{1q}m_{k1}&\dots&n_{1q}m_{kl}\end{bmatrix} \\ \vdots & \ddots & \vdots \\ \begin{bmatrix}n_{p1}m_{11}&\dots&n_{p1}m_{1l}\\ \vdots&\ddots&\vdots\\ n_{p1}m_{k1}&\dots&n_{p1}m_{kl}\end{bmatrix} & \dots & \begin{bmatrix}n_{pq}m_{11}&\dots&n_{pq}m_{1l}\\ \vdots&\ddots&\vdots\\ n_{pq}m_{k1}&\dots&n_{pq}m_{kl}\end{bmatrix} \end{pmatrix}\)

Assume \(M: m_1 \times n_1\), \(N: m_2 \times n_2\), then \(M \otimes N: m_1m_2 \times n_1n_2\), \(N \otimes M: m_1m_2 \times n_1n_2\)
\((M \otimes N)_{(i_1,i_2),(j_1,j_2)} = M_{(i_1,j_1)} \times N_{(i_2,j_2)}\) and \((N\otimes M)_{(i_2,i_1)(j_2,j_1)}=N_{(i_2,j_2)}\times M_{(i_1,j_1)}\) are same only position is not same

Notation of standard basis

\(|0\rang \otimes |0\rang = \begin{pmatrix} 1 \\ 0 \end{pmatrix}\otimes \begin{pmatrix} 1 \\ 0 \end{pmatrix}= \begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \end{pmatrix}=|0\rang |0\rang=|00\rang\)

\(|0\rang \otimes |1\rang = \begin{pmatrix} 1 \\ 0 \end{pmatrix}\otimes \begin{pmatrix} 0 \\ 1 \end{pmatrix}= \begin{pmatrix} 0 \\ 1 \\ 0 \\ 0 \end{pmatrix}=|0\rang |1\rang=|01\rang\)
\(|1\rang \otimes |0\rang = \begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \end{pmatrix}=|1\rang |0\rang=|10\rang\) and \(|1\rang \otimes |1\rang = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 1 \end{pmatrix}=|1\rang |1\rang=|11\rang\)

Here we denote \(|\psi\phi\rang =|\psi\rang|\phi\rang :=|\psi\rang \otimes |\phi\rang\)

Tensor Product of Hilbert spaces

(Space: \(\mathbb{C}^2\otimes \mathbb{C}^2\cong \mathbb{C}^4\))

\(A\cong \mathbb{C}^n=\text{span}_\mathbb{C}\{|x\rang\}^n_{x=1}\) where \(|x\rang\) is orthonormal basis

\(B\cong \mathbb{C}^{m}=\text{span}_{\mathbb{C}}\{|y\rang\}^{m}_{y=1}\) where \(|x\rang\) is orthonormal basis
Then \(A\otimes B:=\text{span}_{\mathbb{C}}\{|x\rang|y\rang\}_{x\in [n],y\in [m]}=\left\{\sum_{x,y}M_{xy}|xy\rang:M_{xy}\in \mathbb{C}\right\}\)

\(|\psi\rang\in A\otimes B:=AB\)

\(|\psi^{AB}\rangle=\sum_{x,y}m_{xy}|x\rangle|y\rangle,\,\,|\phi^{AB}\rangle=\sum_{x^{\prime},y^{\prime}} \mathcal{v}_{x^{\prime}y^{\prime}}|x^{\prime}y^{\prime}\rangle\)

Then \(\lang \phi^{AB}|\psi^{AB}\rang = \text{Tr}[M^*N]\) where \(M=(m_{xy}),N=(v_{x^{\prime}y^{\prime}})\)

Example

Let \(|0\rangle_A, |1\rangle_A\) be basis of \(A\). \(|0\rangle_B, |1\rangle_B\) be basis of \(B\).

Then \(\text{span}\{|x\rangle|y\rangle\}\) is basis of \(A \otimes B\).

Let \(|\psi\rangle = 2|00\rangle + 3i|01\rangle - 1|10\rangle + 5|11\rangle\), \(|\phi\rangle = 1|00\rangle + 4|01\rangle - 2i|10\rangle + 0|11\rangle\)

Then\(M = \begin{pmatrix} 2 & 3i \\ -1 & 5 \end{pmatrix}\), \(N = \begin{pmatrix} 1 & 4 \\ -2i & 0 \end{pmatrix}\)

  • Direct calculate:\(\langle\psi|\phi\rangle = 2+10i\)
  • Formula:\(\langle\psi|\phi\rangle = \text{Tr}(N^{*}M) = 2+10i\), why?

‍