# Lecture 26

## Chapter VI Inner Product Spaces

### Inner Products and Norms 6A

---

Review

#### Dot products

#### Inner product

An inner product $\langle,\rangle:V\times V\to \mathbb{F}$

Positivity: $\langle v,v\rangle\geq 0$

Definiteness: $\langle v,v\rangle=0\iff v=0$

Additivity: $<u+v,w>=<u,w>+<v,w>$

Homogeneity: $<\lambda u, v>=\lambda<u,v>$

Conjugate symmetry: $<u,v>=\overline{<v,u>}$

#### Norm

$||v||=\sqrt{<v,v>}$

---

New materials

### Orthonormal basis 6B

#### Definition 6.22

A list of vectors is **orthonormal** if each vector has norm = 1, and is orthogonal to every other vectors in the list.

if a list $e_1,...,e_m\in V$ is orthonormal if $<e_j,e_k>=1\begin{cases}
    1 \textup{ if } j=k\\
    0 \textup{ if }j\neq k
\end{cases}$.

Example:

* Standard basis in $\mathbb{F}^n$ is orthonormal.  
* $(\frac{1}{\sqrt{3}},\frac{1}{\sqrt{3}},\frac{1}{\sqrt{3}}),(\frac{-1}{\sqrt{2}},\frac{1}{\sqrt{2}},0),(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{-2}{\sqrt{6}})$ in $\mathbb{F}^3$ is orthonormal.
* For $<p,q>=\int^1_{-1}pq$ on $\mathscr{P}_2(\mathbb{R})$. The standard basis $(1,x,x^2)$ is not orthonormal.

#### Theorem 6.24

Suppose $e_1,...,e_m$ is an orthonormal list, then $||a_1 e_1+...+a_m e_m||^2=|a_1|^2+...+|a_m|^2$

Proof:

Using induction of $m$.

$m=1$, clear ($||e_1||^2=1$)
$m>1$, $||a_1 e_1+...a_{m-1}e_{m-1}||^2=|a_1|^2+...+|a_{m-1}|^2$ and $<a_1 e_1+...+a_{m-1} e_{m-1},a_m e_m>=0$ by Pythagorean Theorem. $||(a_1 e_1+...a_{m-1}e_{m-1})+a_m e_m||^2=||a_1 e_1+...a_{m-1}e_{m-1}||^2+||a_m e_m||^2=|a_1|^2+...+|a_{m-1}|^2+|a_m|^2$

#### Theorem 6.25

Every orthonormal list is linearly independent.

Proof:

$||a_1 e_1+...+a_m e_m||^2=0$, then $|a_1|^2+...+|a_m|^2=0$, then $a_1=...=a_m=0$

#### Theorem 6.28

Every orthonormal list of length $dim\ V$ is a basis. 

#### Definition 6.27

An orthonormal basis is a basis that is an orthonormal list.

#### Theorem 6.26 Bessel's Inequality

Suppose $e_1,...,e_m$ is an orthonormal list $v\in V$

$$
|<v,e_1>|^2+...+|<v,e_m>|^2\leq ||v||^2
$$

Proof:

Let $v\in V$, then let $n=<v,e_1>e_1+...+<v,e_m>e_m$,

let $w=v-u$, Note that $<u,e_k>=<v,e_k>$, thus $<w,e_k>=0, <w,u>=0$, apply Pythagorean Theorem.

$$
||w+u||^2=||w||^2+||u||^2\\
||v||^2\geq ||u||^2
$$

#### Theorem 6.30

Suppose $e_1,...,e_n$ is an orthonormal basis, and $u,v\in V$, then 

(a) $v=<v,e_1>e_1+...+<v,e_n>e_n$  
(b) $||v||^2=|<v,e_1>|^2+...+|<v,e_n>|^2$  
(c) $<u,v>=<u,e_1>\overline{<v,e_1>}+...+<u,e_n>\overline{<v,e_n>}$

Proof:

(a) let $a_1,...,a_n\in \mathbb{F}$ such that $v=a_1 e_1+...+a_n e_n$.

$$
\begin{aligned}
<v,e_k>&=<a,e_1,e_k>+...+<a_k e_k,e_k>+...+<a_n e_n,e_n>\\
&=<a_k e_k,e_k>\\
&= a_k
\end{aligned}
$$

---

Note *6.30 (c)*  means up to change of basis, every inner product on a finite dimensional vector space "looks like" an euclidean inner products...

#### Theorem 6.32 Gram-Schmidt

Let $v_1,...,v_m$ be a linearly independent list. 

Define $f_k\in V$ by $f_1=v_1,f_k=v_k-\sum_{j=1}^{k-1}\frac{<v_k,f_j>}{||f_j||^2}f_j$

Define $e_k=\frac{f_k}{||f_k||}$, then $e_1,...,e_m$ is orthonormal $Span(v_1,...,v_m)=Span(f_1,...,f_m)$