8. Projection onto subspaces

In the Euklidean space \({\mathbb R}^2\) one can project orthogonally onto a line through the origin, i.e., onto a sub-space. The same geometric operation can be defined for closed subspaces of Hilbert spaces.

Theorem: Let \(S\) be a closed subspace of the Hilbert space \(V\). Let \(u \in V\). Then there exists a unique closest point \(u_0 \in S\) characterized by:

\[ \| u - u_0 \| \leq \| u - v \| \qquad \forall \, v \in S \]

There holds

\[ u - u_0 \, \bot \, S \]

Proof: Let \(d := \inf_{v \in S} \| u - v\|\), and let \((v_n)\) be a minimizing sequence such that \(\| u - v_n \| \rightarrow d\). We first check that there holds

\[ \| v_n - v_m \|^2 = 2 \, \| v_n - u \|^2 + 2 \, \| v_m - u \|^2 - 4 \, \| \tfrac{1}{2} (v_n+v_m) - u \|^2. \]

Since \(\tfrac{1}{2} (v_n+v_m) \in S\), there holds \(\| \tfrac{1}{2} (v_n+v_m) - u \| \geq d\). We prove that \((v_n)\) is a Cauchy sequence: Fix \(\varepsilon > 0\), choose \(N \in {\mathbb N}\) such that for \(n > N\) there holds \(\| u - v_n \|^2 \leq d^2 + \varepsilon^2\). Thus for all \(n,m > N\) there holds

\[ \| v_n - v_m \|^2 \leq 2 (d^2+\varepsilon^2) + 2 (d^2 + \varepsilon^2) - 4 d^2 = 4 \varepsilon^2. \]

Thus, \((v_n)\) is Cauchy and therefore converges to some \(u_0 \in V\). Since \(S\) is closed, \(u_0 \in S\). By continuity of the norm, \(\| u - u_0 \| = d\).

Fix some \(0 \neq w \in S\), and define \(\varphi(t) := \| u - \underbrace{u_0 - t w}_{\in S} \|^2\). \(\varphi(\cdot)\) is a convex function, it takes its unique minimum \(d\) at \(t=0\). Thus

\[ 0 = \frac{d \varphi(t)}{dt}|_{t=0} = \{ -2 (u-u_0, w) + 2 t (w,w) \} |_{t=0} = -2 (u-u_0,w) \]

We obtained \(u-u_0 \bot S\). If there were two minimizers \(u_0 \neq u_1\), then \(u_0-u_1 = (u_0-u) - (u_1-u) \bot S\) and \(u_0-u_1 \in S\), which implies \(u_0-u_1 = 0\), a contradiction. \(\Box\)

The theorem says that given an \(u \in V\), we can uniquely decompose it as

\[ u = u_0 + u_1, \qquad u_0 \in S, \quad u_1 \in S^\bot \]

This allows to define the operators \(P_S : V \rightarrow S\) and \(P_S^\bot : V \rightarrow S^\bot\) as

\[ P_S u := u_0 \qquad P_S^\bot u := (I - P_S) u = u_1 \]

Theorem: \(P_S\) and \(P_S^\bot\) are linear operators.

Definition: A linear operator \(P\) is called a projection if \(P^2 = P\). A projector is called orthogonal, if \((Pu,v) = (u,Pv)\).

Lemma: The operators \(P_S\) and \(P_S^\bot\) are both orthogonal projectors.

Proof: For \(u \in S\) there holds \(P_S u = u\). Since \(P_S u \in S\), there holds \(P_S^2 u = P_S u\). It is orthogonal since

\[ (P_Su,v) = (P_Su,v-P_Sv+P_Sv) = (\underbrace{P_S u}_{\in S}, \underbrace{v-P_Sv}_{\in S^\bot}) + (P_Su,P_Sv) = (P_Su,P_Sv). \]

With the same argument there holds \((u,P_Sv) = (P_Su,P_Sv)\). The co-projector \(P_S^\bot = I - P_S\) is a projector since

\[ (I-P_S)^2 = I - 2 P_S + P_S^2 = I - P_S. \]

It is orthogonal since \(((I-P_S) u,v) = (u,v)-(P_Su,v) = (u,v)-(u,P_Sv) = (u,(I-P_S)v)\) \(\Box\)