update

pages/Math4121/Math4121_L31.md

@@ -0,0 +1,92 @@
+# Math4121 Lecture 31
+
+## Chapter 3: Lebesgue Integration
+
+### Non-measurable sets
+
+#### Definition: Vitali's construction
+
+Step 1. Define an equivalence relation on $\mathbb{R}$ as follows:
+
+Recall a relation is an equivalence relation if it is reflexive, symmetric, and transitive.
+
+1. Reflexive: $x\sim x$ for all $x\in\mathbb{R}$
+2. Symmetric: $x\sim y$ implies $y\sim x$ for all $x,y\in\mathbb{R}$
+3. Transitive: $x\sim y$ and $y\sim z$ implies $x\sim z$ for all $x,y,z\in\mathbb{R}$
+
+Say $x\sim y$ if $x-y\in\mathbb{Q}$.
+
+This is an equivalence relation, easy to show by the properties above.
+
+We denote the equivalence class of $x$ by $\mathbb{R}/\sim$, where $[x]=\{x+q:q\in\mathbb{Q}\}$.
+
+If $z\in [x]$, then so is the fractional part of $z$, i.e. $z-\lfloor z\rfloor\in [x]$. So in every equivalence class $[x]$ we can find an element in $[x]\cap (0,1)$. Take one such real number from every equivalence class, and call the set of all such numbers $\mathcal{N}$.
+
+Step 2. Show that $\mathcal{N}$ is not Lebesgue measurable.
+
+We defined the translation of $S$ as follows:
+
+Given a set $S\subseteq\mathbb{R}$ and a real number $a\in\mathbb{R}$, the translation of $S$ by $a$ is defined as
+
+$$
+S+a=\{x+a:x\in S\}
+$$
+
+Outer measure is translation invariant, i.e. $m_e(S+a)=m_e(S)$ for all $S\subseteq\mathbb{R}$ and $a\in\mathbb{R}$, which also holds for inner measure.
+
+Properties of $\mathcal{N}$:
+
+1. $(0,1)\subseteq\bigcup_{q\in \mathbb{Q}\cap (-1,1)} (\mathcal{N}+q)\subseteq (-1,2)$
+2. $\{\mathcal{N}+q:q\in\mathbb{Q}\cap (-1,1)\}$ is pairwise disjoint.
+
+Suppose $\mathcal{N}$ is measurable. Then by (1) 
+$$
+\begin{aligned}
+    1&\leq \sum_{q\in\mathbb{Q}\cap (-1,1)} (\mathcal{N}+q)\\
+    &=\sum_{q\in\mathbb{Q}\cap (-1,1)} m(\mathcal{N})
+\end{aligned}
+$$
+So $m(\mathcal{N})\neq 0$.
+
+By (2), we have
+
+$$
+\begin{aligned}
+3&\geq \sum_{q\in\mathbb{Q}\cap (-1,1)} m(\mathcal{N}+q)\\
+&=\sum_{q\in\mathbb{Q}\cap (-1,1)} m(\mathcal{N})\\
+&=m(\mathcal{N})\sum_{q\in\mathbb{Q}\cap (-1,1)} 1\\
+&=\infty
+\end{aligned}
+$$
+
+This is a contradiction. So $\mathcal{N}$ is not Lebesgue measurable.
+
+QED
+
+Appendix:
+
+(1) $I\subseteq\bigcup_{q\in\mathbb{Q}\cap (-1,1)} (\mathcal{N}+q)$
+
+Let $x\in I$. We need to find $q\in\mathbb{Q}\cap (-1,1)$ such that $x-q\in\mathcal{N}$. $\exists y\in\mathcal{N}$ such that $y\in (0,1)\cap [x]$. Then $x-y=q\in \mathbb{Q}$ and since $x,y\in I$, we have $q\in (-1,1)$.
+
+(2) $\{\mathcal{N}+q:q\in\mathbb{Q}\cap (-1,1)\}$ is pairwise disjoint.
+
+Suppose $\mathcal{N}+q_1=\mathcal{N}+q_2$ for some $q_1,q_2\in\mathbb{Q}\cap (-1,1)$. We want to show $q_1=q_2$.
+
+Take $x$ in the intersection, then  this means $y=x-q_1, z=x-q_2\in\mathcal{N}$.
+
+But $y\sim z$, this contradicts the fact that $\mathcal{N}$ contains only one element from each equivalence class. So $q_1=q_2$.
+
+#### Axiom of choice
+
+Given a set $S$, $\exists \psi:\mathscr{P}(S)\to S$ such that $\psi(T)\in T, \forall T\subseteq\mathscr{P}(S)$.
+
+For any set $S$, there exists a map that maps every non-empty subset of $S$ to an element of that subset.
+
+This leads to some weird results, e.g. Banach-Tarski paradox.
+
+_Godel showed that the axiom of choice is not contradictory to ZF set theory._ You have ZFC
+
+_Cohen showed that the negation of the axiom of choice is not contradictory to ZF set theory._ You have ZF
+
+You can choose the axiom or not.

pages/Math4121/Math4121_L32.md

@@ -0,0 +1,99 @@
+# Math4121 Lecture 32
+
+## Chapter 6: The Lebesgue Integral
+
+### Measurable Functions
+
+Definition: A function $f:\mathbb{R}\to\mathbb{R}$ is measurable on the interval $[a,b]$ if $\{x\in [a,b]:f(x) > c\}$ is measurable for all $c\in\mathbb{R}$, called the **super level set** of $f$.
+
+Denote $\{f> c\}$
+
+#### Proposition 6.1
+
+The following are equivalent:
+
+For all $c\in\mathbb{R}$,
+
+1. $\{x\in [a,b]:f(x) > c\}$ is measurable.
+2. $\{x\in [a,b]:f(x) < c\}$ is measurable.
+3. $\{x\in [a,b]:f(x) \geq c\}$ is measurable.
+4. $\{x\in [a,b]:f(x) \leq c\}$ is measurable.
+5. $\{x\in \mathbb{R}:c \leq f(x) < d\}$ is measurable for all $c,d\in\mathbb{R}$.
+
+Proof:
+
+Since the complement of a measurable set is measurable. (1) $\iff$ (4). and (2) $\iff$ (3).
+
+We only need to show (1) $\implies$ (2).
+
+Since $\{f>c\}=\bigcup_{n=1}^{\infty}\{f\geq c+\frac{1}{n}\}$.
+
+So (1) $\implies$ (2).
+
+Since (2) $\implies$ (1)-(4) $\implies$ (5).
+
+To see (5) $\implies$ (1), we have $\{f\geq c\}=\bigcup_{n=1}^{\infty}\{x\in\mathbb{R}:c \leq f(x) < c+n\}$
+
+QED
+
+#### Proposition 6.3
+
+Let $f,g$ be measurable on $[a,b]$ and $\alpha\in\mathbb{R}$. Then the following are measurable:
+
+1. $f+g$
+2. $fg$
+3. $\alpha f$
+4. $|f|^\alpha$
+
+Proof:
+
+If $\alpha=0$, then $\alpha f$ and $|f|^\alpha$ are constant functions, hence measurable.
+
+But for constant functions $h$, $\{h>c\}=\begin{cases}
+\emptyset & \text{if } c\geq h \\
+\mathbb{R} & \text{if } c < h
+\end{cases}$
+
+For $a\neq 0$, $\{x\in \mathbb{R}:\alpha f(x) > c\}=\{x\in \mathbb{R}:f(x) > \frac{c}{\alpha}\}$
+
+Similarly, $\{x\in \mathbb{R}:|f(x)|^\alpha > c\}=\{x\in \mathbb{R}:|f(x)| > c^{1/\alpha}\}=\{x\in \mathbb{R}:f(x) > c^{1/\alpha}\}\cup\{x\in \mathbb{R}:f(x) < -c^{1/\alpha}\}$
+
+We want to show $\{f+g>c\}=\bigcup_{q\in \mathbb{Q}}\{f>q\}\cap\{g>c-q\}$
+
+$\{f+g>c\}\supseteq \bigcup_{q\in \mathbb{Q}}\{f>q\}\cap\{g>c-q\}$
+
+if $x$ is in the RHS, then $\exists q\in \mathbb{Q}$ such that $f(x) > q$ and $g(x) > c-q$, therefore $f(x)+g(x) > c$.
+
+So $x$ is in the LHS.
+
+$\{f+g>c\}\subseteq \bigcup_{q\in \mathbb{Q}}\{f>q\}\cap\{g>c-q\}$
+
+Let $x\in \mathbb{R}$ such that $f(x)+g(x) > c$. Need to find $q\in \mathbb{Q}$ such that $f(x) > q$ and $g(x) > c-q$.
+
+Since $f(x)>c-g(x)$, by the density of $\mathbb{Q}$ in $\mathbb{R}$, $\exists q\in \mathbb{Q}$ such that $q > c-g(x)$.
+
+For $fg$, we have $fg=\frac{1}{4}((f+g)^2-(f-g)^2)$
+
+So $fg$ is measurable.
+
+QED
+
+### Limit of Measurable Functions
+
+#### Proposition 6.4
+
+Let $\{f_n\}$ be a sequence of measurable functions on $[a,b]$. Then,
+
+$$
+g(x)=\sup_{n\geq 1}f_n(x),\inf_{n\geq 1}f_n(x),\limsup_{n\to\infty}f_n(x),\liminf_{n\to\infty}f_n(x)
+$$
+
+are all measurable functions
+
+#### Corollary of Proposition 6.4
+
+If $\{f_n\}_{n=1}^{\infty}$ are measurable functions and $f(x)=\lim_{n\to\infty}f_n(x)$ exists for all $x\in[a,b]$, then $f$ is measurable. (pointwise limit of measurable functions is measurable)
+
+#### Definition of almost everywhere
+
+A property holds **almost everywhere** if it holds everywhere except for a set of measure zero.

pages/Swap/Math401/Math401_N3.md

@@ -126,6 +126,8 @@ def is_uniquely_decipherable(f):
     return True
 ```
 
+### Shannon's source coding theorem
+
 #### Definition 1.1.4
 
 An elementary information source is a pair $(A,\mu)$ where $A$ is an alphabet and $\mu$ is a probability distribution on $A$. $\mu$ is a function $\mu:A\to[0,1]$ such that $\sum_{a\in A}\mu(a)=1$.
@@ -142,6 +144,35 @@ $$
 L(\mu)=\min\{\overline{l}(\mu,f)|f:A\to S(B)\text{ is uniquely decipherable}\}
 $$
 
+
+#### Lemma: Jenson's inequality
+
+Let $f$ be a convex function on the interval $(a,b)$. Then for any $x_1,x_2,\cdots,x_n\in (a,b)$ and $\lambda_1,\lambda_2,\cdots,\lambda_n\in [0,1]$ such that $\sum_{i=1}^{n}\lambda_i=1$, we have
+
+$$f(\sum_{i=1}^{n}\lambda_ix_i)\leq \sum_{i=1}^{n}\lambda_if(x_i)$$
+
+Proof:
+
+If $f$ is a convex function, there are three properties that useful for the proof:
+
+1. $f''(x)\geq 0$ for all $x\in (a,b)$
+2. For any $x,y\in (a,b)$, $f(x)\geq f(y)+(x-y)f'(y)$ (Take tangent line at $y$)
+3. For any $x,y\in (a,b)$ and $0<\lambda<1$, we have $f(\lambda x+(1-\lambda)y)\leq \lambda f(x)+(1-\lambda)f(y)$ (Take line connecting $f(x)$ and $f(y)$)
+
+We use $f(x)\geq f(y)+(x-y)f'(y)$, we replace $y=\sum_{i=1}^{n}\lambda_ix_i$ and $x=x_j$, we have
+
+$$f(x_j)\geq f(\sum_{i=1}^{n}\lambda_ix_i)+(x_j-\sum_{i=1}^{n}\lambda_ix_i)f'(\sum_{i=1}^{n}\lambda_ix_i)$$
+
+We sum all the inequalities, we have
+
+$$
+\begin{aligned}
+\sum_{j=1}^{n}\lambda_j f(x_j)&\geq \sum_{j=1}^{n}\lambda_jf(\sum_{i=1}^{n}\lambda_ix_i)+\sum_{j=1}^{n}\lambda_j(x_j-\sum_{i=1}^{n}\lambda_ix_i)f'(\sum_{i=1}^{n}\lambda_ix_i)\\
+&\geq \sum_{j=1}^{n}\lambda_jf(\sum_{i=1}^{n}\lambda_ix_i)+0\\
+&=f(\sum_{j=1}^{n}\lambda_ix_j)
+\end{aligned}
+$$
+
 #### Theorem 1.1.5
 
 Shannon's source coding theorem
@@ -204,34 +235,6 @@ $$
 
 $\log \prod_{a\in A}\left(\frac{v(a)}{\mu(a)}\right)^{\mu(a)}=\sum_{a\in A}\mu(a)\log \frac{v(a)}{\mu(a)}$ is also called Kullback–Leibler divergence or relative entropy.
 
-> Jenson's inequality: Let $f$ be a convex function on the interval $(a,b)$. Then for any $x_1,x_2,\cdots,x_n\in (a,b)$ and $\lambda_1,\lambda_2,\cdots,\lambda_n\in [0,1]$ such that $\sum_{i=1}^{n}\lambda_i=1$, we have
->
-> $$f(\sum_{i=1}^{n}\lambda_ix_i)\leq \sum_{i=1}^{n}\lambda_if(x_i)$$
->
-> Proof:
->
-> If $f$ is a convex function, there are three properties that useful for the proof:
->
-> 1. $f''(x)\geq 0$ for all $x\in (a,b)$
-> 2. For any $x,y\in (a,b)$, $f(x)\geq f(y)+(x-y)f'(y)$ (Take tangent line at $y$)
-> 3. For any $x,y\in (a,b)$ and $0<\lambda<1$, we have $f(\lambda x+(1-\lambda)y)\leq \lambda f(x)+(1-\lambda)f(y)$ (Take line connecting $f(x)$ and $f(y)$)
->
-> We use $f(x)\geq f(y)+(x-y)f'(y)$, we replace $y=\sum_{i=1}^{n}\lambda_ix_i$ and $x=x_j$, we have
->
-> $$f(x_j)\geq f(\sum_{i=1}^{n}\lambda_ix_i)+(x_j-\sum_{i=1}^{n}\lambda_ix_i)f'(\sum_{i=1}^{n}\lambda_ix_i)$$
->
-> We sum all the inequalities, we have
->
-> $$
-\begin{aligned}
-\sum_{j=1}^{n}\lambda_j f(x_j)&\geq \sum_{j=1}^{n}\lambda_jf(\sum_{i=1}^{n}\lambda_ix_i)+\sum_{j=1}^{n}\lambda_j(x_j-\sum_{i=1}^{n}\lambda_ix_i)f'(\sum_{i=1}^{n}\lambda_ix_i)\\
-&\geq \sum_{j=1}^{n}\lambda_jf(\sum_{i=1}^{n}\lambda_ix_i)+0\\
-&=f(\sum_{j=1}^{n}\lambda_ix_j)
-\end{aligned}
-$$
-
-
-
 Since $\log$ is a concave function, by Jensen's inequality $f(\sum_{i=1}^{n}\lambda_ix_i)\leq \sum_{i=1}^{n}\lambda_if(x_i)$, we have
 
 $$