updates

content/Swap/Math4501/Math4501_L1.md

@@ -0,0 +1,60 @@
+# Math4501 Lecture 1
+
+In many practical problems (ODEs (ordinary differential equations), PdEs (partial differential equations), System of equations)
+
+closed-form analytical solutions are unknown.
+
+-> resort ot computational algorithms (approximation)
+
+For example,
+
+Deep learning classifiers
+
+**Root finding**
+
+$$
+f(x)=\sum_{i=1}^n a_i x^i
+$$
+
+for $n\geq 5$.
+
+find all roots $x\in \mathbb{R}$ of $f(x)=0$.
+
+**Investment**
+
+Invest a dollars every month return with the rate $r$.
+
+$g(r)=a\sum_{i=1}^n (1+r)^i=a\left[\frac{(1+r)^{n+1}-(1+r)}{r}\right]$
+
+Say want $g(r)=b$ for some $b$.
+
+$f(r)=a(1+n)^{n+1}-a(1+n)-br=0$
+
+use Newton's method to find $r$ such that $f(r)=0$.
+
+Since $f$ is non-linear, that is $f(x+y)\neq f(x)+f(y)$.
+
+Let
+
+$$
+f_1(x_1,\dots, x_m)=0\\
+\vdots\\
+f_m(x_1,\dots, x_m)=0
+$$
+
+be a system of $m$ equations $\vec{f} \mathbb{R}^m \to \mathbb{R}^m$. and $f_1(\vec{x})=\vec{0}$.
+
+If $\vec{f}$ is linear, note that
+
+$$
+\begin{aligned}
+\vec{f}(\vec{x})&=\vec{f}(\begin{bmatrix}x_1\\ \vdots\\ x_m\end{bmatrix})\\
+&=\vec{f}(x_1\begin{bmatrix}1\\ 0\\ \vdots\\ 0\end{bmatrix}+x_2\begin{bmatrix}0\\ 1\\ \vdots\\ 0\end{bmatrix}+\cdots+x_m\begin{bmatrix}0\\ 0\\ \vdots\\ 1\end{bmatrix})\\
+&=x_1\vec{f}(\begin{bmatrix}1\\ 0\\ \vdots\\ 0\end{bmatrix})+x_2\vec{f}(\begin{bmatrix}0\\ 1\\ \vdots\\ 0\end{bmatrix})+\cdots+x_m\vec{f}(\begin{bmatrix}0\\ 0\\ \vdots\\ 1\end{bmatrix})\\
+&=A\vec{x}
+\end{aligned}
+$$
+
+where $\vec{e}_i$ is the $i$-th standard basis vector.
+
+Gaussian elimination (LU factorization)

content/Swap/Math4501/Math4501_L2.md

@@ -0,0 +1,76 @@
+# Math4501 Lecture 2
+
+Solving non-linear equations
+
+Let $\vec{f}:\mathbb{R}^n\to\mathbb{R}^n$ we want to solve $\vec{f}(\vec{x})=\vec{0}$. ($m$ equations, $m$ variables)
+
+In case if $\vec{f}$ is linear, we can solve it by Gaussian elimination.
+
+Closely related to the problem: eigenvalue problem.
+
+related to root finding problem for polynomial.
+
+## Polynomial approximations
+
+Let $f:[0,1]\to\mathbb{R}$ be a continuous function.
+
+Find polynomial $p_n$ of degree $n$ such that $p_n(x_i)=f(x_i)$ for $i=0,1,\cdots,n$.
+
+Then, some key questions are involved:
+
+1. How to compute $c_0,c_1,\cdots,c_n$?
+2. If $f$ is continuously differentiable, does $p_n'$ approximate $f'$?
+3. If $f$ is integrable, does $\int_0^1 p_n(x)dx$ approximate $\int_0^1 f(x)dx$?
+
+Deeper questions:
+
+Is the approximation **efficient**?
+
+## Scalar problem
+
+Problem 1: Let $f:[a,b]\to\mathbb{R}$ be a continuous function. Find $\xi\in[a,b]$ such that $f(\xi)=0$.
+
+Problem 2: Let $f:[a,b]\to\mathbb{R}$ be a continuous function. Find $\xi\in[a,b]$ such that $f(\xi)=\xi$.
+
+P1, P2 are equivalent. $f(x)\coloneqq f(x)-x$ is a continuous function.
+
+[Intermediate value theorem](https://notenextra.trance-0.com/Math4121/Math4121_L3#definition-5121-intermediate-value)
+
+> Some advantage in solving P1 as P2
+
+### When does a solution exists
+
+Trivial case: $f(x)=0$ for some $x\in[a,b]$.
+
+Without loss of generality, assume $f(a)f(b)<0$, Then there exists $\xi\in(a,b)$ such that $f(\xi)=0$.
+
+Bisection algorithm:
+
+```python
+def bisection(f, a, b, tol=1e-6, max_iter=100):
+    # first we setup two sequences $a_n$ and $b_n$
+    # require:
+    # |a_n - b_n| \leq 2^{-n} (b-a)
+    for i in range(max_iter):
+        c = (a + b) / 2
+        if c < tol or f(c) == 0:
+            return c
+        elif f(a) * f(c) < 0:
+            b = c
+        else:
+            a = c
+    return None
+```
+
+Let $f(a_n)<0$ for all $n$ and $f(b_n)>0$ for all $n$.
+
+$\lim_{n\to\infty} f(a_n)\leq 0$ and $\lim_{n\to\infty} f(b_n)\geq 0$.
+
+If limit exists, then $\lim_{n\to\infty} f(a_n)=\lim_{n\to\infty} f(b_n)=0$.
+
+Such limit exists by the sequence $a_n$ and $b_n$ is Cauchy and we are in real number field.
+
+This can be used to solve P2:
+
+Recall that if we define $f(x)\coloneqq g(x)-x$, then $f(x)=0$ if and only if $f(a)f(b)<0$. That is $(g(a)-a)(g(b)-b)\leq 0$.
+

content/Swap/Math4501/Math4501_L3.md

@@ -0,0 +1,120 @@
+# Math4501 Lecture 3
+
+## Review from last lecture
+
+### Bisection method for finding root
+
+P1. Let $f$ be a continuous function on $[a,b]\to \mathbb{R}$, find $\xi \in [a,b]$ such that $f(\xi)=0$.
+
+P2. Let $g$ be a continuous function on $[a,b]\to \mathbb{R}$, find $\xi \in [a,b]$ such that $g(\xi)=\xi$.
+
+#### Theorem 1:
+
+solution to P1 exists if $f(a)f(b)<0$.
+
+#### Theorem 2:
+
+Fixed point theorem:
+
+If solution to P2 exists, then $g:[a,b]\to [a,b]$.
+
+#### Bijection method
+
+Obtain two sequence $(a_n)_{n=1}^\infty$ and $(b_n)_{n=1}^\infty$.
+
+Initially, we set $a_0=a$ and $b_0=b$.
+
+If $|a_n-b_n|<2^{-n}|a_0-b_0|$, then $a_n$ and $b_n$ are Cauchy sequence. So their limit exists.
+
+$\lim_{n\to\infty} a_n=\lim_{n\to\infty} b_n=\xi$.
+
+### Simple iteration method
+
+#### Definition of Simple Iteration
+
+Given $x_0\in [a,b]$, we define a sequence $(x_n)_{n=0}^\infty$ by
+
+$$
+x_{n+1}=g(x_n)
+$$
+
+If a simple iteration converges, the it converges to a fixed point of
+
+<details>
+
+<summary>Proof</summary>
+
+Let $c\coloneqq \lim_{n\to\infty} x_n=g(c)$.
+
+$g:[a,b]\to \mathbb{R}$ is continuous if and only if $g$ is continuous at $x_0\in [a,b]$.
+
+</details>
+
+#### Definition of Lipschitz continuous
+
+A function $g:[a,b]\to \mathbb{R}$ is Lipschitz continuous if for all $x,y\in [a,b]$, there exists a constant $L>0$ such that
+
+$$
+|g(x)-g(y)|\leq L|x-y|
+$$
+
+for some $L>0$.
+
+> [!NOTE]
+> 
+> Lipschitz continuous is a stronger condition than continuous.
+>
+> If a function is Lipschitz continuous, then it is continuous.
+>
+> However, the converse is not true.
+
+#### Definition of contraction mapping
+
+A function $g:[a,b]\to \mathbb{R}$ is a contraction mapping if it is Lipschitz continuous with $L<1$.
+
+#### Theorem of simple iteration
+
+Let $g:[a,b]\to [a,b]$ is a contraction mapping (Lipschitz continuous with $L<1$, with pointwise ontinuous $g$).
+
+Then
+
+- $g$ has a unique fixed point $\xi \in [a,b]$
+- Simple iteration $x_{n+1}=g(x_n)$ converges to $\xi$ for any $x_0\in [a,b]$.
+
+<details>
+
+<summary>Proof</summary>
+
+**Uniqueness**:
+
+Suppose $\xi_1$ and $\xi_2$ are two fixed points of $g$.
+
+Then $|x_1-x_2|=|g(\xi_1)-g(\xi_2)|\leq L|\xi_1-\xi_2|$.
+
+Thus, $(1-L)|\xi_1-\xi_2|\leq 0$, which implies $|\xi_1-\xi_2|=0$.
+
+A more general result:
+
+Brouwer's fixed point theorem
+
+**Convergence**:
+
+Let $\xi\in [a,b]$ be the unique fixed point of $g$.
+
+Then,
+
+$$
+\begin{aligned}
+|x_n-\xi|&=|g(x_{n-1})-g(\xi)|\\
+&\leq L|x_{n-1}-\xi|\\
+&=L|g(x_{n-2})-g(\xi)|\\
+&\leq L^2|x_{n-2}-\xi|\\
+&\vdots\\
+&\leq L^n|x_0-\xi|
+$$
+
+Thus, we can always find $N$ such that $L^N|x_0-\xi|<\epsilon$ for any $\epsilon>0$. Choose $N=\log(\frac{\epsilon}{|x_0-\xi|})/\log(L)$.
+
+Therefore, $x_n$ converges to $\xi$.
+
+</details>

content/Swap/Math4501/_meta.js

@@ -0,0 +1,9 @@
+export default {
+    index: "Course Description",
+    "---":{
+        type: 'separator'
+    },
+    Math4501_L1: "Numerical Applied Mathematics (Lecture 1)",
+    Math4501_L2: "Numerical Applied Mathematics (Lecture 2)",
+    Math4501_L3: "Numerical Applied Mathematics (Lecture 3)",
+}

content/Swap/Math4501/index.md

@@ -0,0 +1,3 @@
+# Math4501
+
+Numerical Applied Mathematics