update

pages/CSE559A/CSE559A_L22.md

@@ -30,13 +30,231 @@ In practice, we don't use (m,b) parameterization.
 
 Instead, we use polar parameterization:
 
+$$
+\rho = x \cos \theta + y \sin \theta
+$$
 
 Algorithm outline:
 
 - Initialize accumulator $H$ to all zeros
-- For each feature point $(x_i, y_i)$
-    - For $\theta=0$ to $180$
-    - $\rho=x_i \cos \theta + y_i \sin \theta$
+  - For each feature point $(x,y)$
+    - For $\theta = 0$ to $180$
+    - $\rho = x \cos \theta + y \sin \theta$
+    - $H(\theta, \rho) += 1$
+- Find the value(s) of $(\theta, \rho)$ where $H(\theta, \rho)$ is a local maximum (perform NMS on the accumulator array)
+  - The detected line in the image is given by $\rho = x \cos \theta + y \sin \theta$
+
+#### Effect of noise
+
+![Hough transform with noise](./images/hough_transform_noise.png)
 
 Noise makes the peak fuzzy.
 
+#### Effect of outliers
+
+![Hough transform with outliers](./images/hough_transform_outliers.png)
+
+Outliers can break the peak.
+
+#### Pros and Cons
+
+Pros:
+
+- Can deal with non-locality and occlusion
+- Can detect multiple instances of a model
+- Some robustness to noise: noise points unlikely to contribute consistently to any single bin
+- Leads to a surprisingly general strategy for shape localization (more on this next)
+
+Cons:
+
+- Complexity increases exponentially with the number of model parameters 
+  - In practice, not used beyond three or four dimensions
+- Non-target shapes can produce spurious peaks in parameter space
+- It's hard to pick a good grid size
+
+### Generalize Hough Transform
+
+Template representation: for each type of landmark point, store all possible displacement vectors towards the center
+
+Detecting the template:
+
+For each feature in a new image, look up that feature type in the model and vote for the possible center locations associated with that type in the model
+
+#### Implicit shape models
+
+Training:
+
+- Build codebook of patches around extracted interest points using clustering
+- Map the patch around each interest point to closest codebook entry
+- For each codebook entry, store all positions it was found, relative to object center
+
+Testing:
+
+- Given test image, extract patches, match to codebook entry 
+- Cast votes for possible positions of object center
+- Search for maxima in voting space
+- Extract weighted segmentation mask based on stored masks for the codebook occurrences
+
+## Image alignment
+
+### Affine transformation
+
+Simple fitting procedure: linear least squares
+Approximates viewpoint changes for roughly planar objects and roughly orthographic cameras
+Can be used to initialize fitting for more complex models
+
+Fitting an affine transformation:
+
+$$
+\begin{bmatrix}
+&&&\cdots\\
+x_i & y_i & 0&0&1&0\\
+0&0&x_i&y_i&0&1\\
+&&&\cdots\\
+\end{bmatrix}
+\begin{bmatrix}
+m_1\\
+m_2\\
+m_3\\
+m_4\\
+t_1\\
+t_2\\
+\end{bmatrix}
+=
+\begin{bmatrix}
+\cdots\\
+\end{bmatrix}
+$$
+
+Only need 3 points to solve for 6 parameters.
+
+### Homography
+
+Recall that
+
+$$
+x' = \frac{a x + b y + c}{g x + h y + i}, \quad y' = \frac{d x + e y + f}{g x + h y + i}
+$$
+
+Use 2D homogeneous coordinates:
+
+$(x,y) \rightarrow \begin{pmatrix}x \\ y \\ 1\end{pmatrix}$
+
+$\begin{pmatrix}x\\y\\w\end{pmatrix} \rightarrow (x/w,y/w)$
+
+Reminder: all homogeneous coordinate vectors that are (non-zero) scalar multiples of each other represent the same point
+
+
+Equation for homography in homogeneous coordinates:
+
+$$
+\begin{pmatrix}
+x' \\
+y' \\
+1
+\end{pmatrix}
+\cong
+\begin{pmatrix}
+h_{11} & h_{12} & h_{13} \\
+h_{21} & h_{22} & h_{23} \\
+h_{31} & h_{32} & h_{33}
+\end{pmatrix}
+\begin{pmatrix}
+x \\
+y \\
+1
+\end{pmatrix}
+$$
+
+Constraint from a match $(x_i,x_i^′)$: $x_i^′≅Hx_i$
+
+How can we get rid of the scale ambiguity? 
+
+Cross product trick:	$x_i^′ × Hx_i=0$
+
+The cross product is defined as:
+
+$$
+\begin{pmatrix}a\\b\\c\end{pmatrix} \times \begin{pmatrix}a'\\b'\\c'\end{pmatrix} = \begin{pmatrix}bc'-b'c\\ca'-c'a\\ab'-a'b\end{pmatrix}
+$$
+
+Let $h_1^T, h_2^T, h_3^T$ be the rows of $H$. Then
+
+$$
+x_i^′ × Hx_i=\begin{pmatrix}
+    x_i^′ \\
+    y_i^′ \\
+    1
+\end{pmatrix} \times \begin{pmatrix}
+    h_1^T x_i \\
+    h_2^T x_i \\
+    h_3^T x_i
+\end{pmatrix}
+=
+\begin{pmatrix}
+    y_i^′ h_3^T x_i−h_2^T x_i \\
+    h_1^T x_i−x_i^′ h_3^T x_i \\
+    x_i^′ h_2^T x_i−y_i^′ h_1^T x_i
+\end{pmatrix}
+$$
+
+Constraint from a match $(x_i,x_i^′)$:
+
+$$
+x_i^′ × Hx_i=\begin{pmatrix}
+    x_i^′ \\
+    y_i^′ \\
+    1
+\end{pmatrix} \times \begin{pmatrix}
+    h_1^T x_i \\
+    h_2^T x_i \\
+    h_3^T x_i
+\end{pmatrix}
+=
+\begin{pmatrix}
+    y_i^′ h_3^T x_i−h_2^T x_i \\
+    h_1^T x_i−x_i^′ h_3^T x_i \\
+    x_i^′ h_2^T x_i−y_i^′ h_1^T x_i
+\end{pmatrix}
+$$
+
+Rearranging the terms:
+
+$$
+\begin{bmatrix}
+    0^T &-x_i^T &y_i^′ x_i^T \\
+    x_i^T &0^T &-x_i^′ x_i^T \\
+    y_i^′ x_i^T &x_i^′ x_i^T &0^T
+\end{bmatrix}
+\begin{bmatrix}
+    h_1 \\
+    h_2 \\
+    h_3
+\end{bmatrix} = 0
+$$
+
+These equations aren't independent! So, we only need two.
+
+### Robust alignment
+
+#### Descriptor-based feature matching
+
+Extract features
+Compute putative matches
+Loop:
+
+- Hypothesize transformation $T$
+- Verify transformation (search for other matches consistent with $T$)
+
+#### RANSAC
+
+Even after filtering out ambiguous matches, the set of putative matches still contains a very high percentage of outliers
+
+RANSAC loop:
+
+- Randomly select a seed group of matches
+- Compute transformation from seed group
+- Find inliers to this transformation 
+- If the number of inliers is sufficiently large, re-compute least-squares estimate of transformation on all of the inliers
+
+At the end, keep the transformation with the largest number of inliers

pages/Math4121/Math4121_L33.md

@@ -0,0 +1,107 @@
+# Math4121 L33
+
+## Continue on Lebegue integration
+
+### Sequence of functions
+
+#### Proposition 6.4
+
+Let $f_n$ be a sequence of measurable functions, then $\sup_n f_n,\inf_n f_n, \limsup_n f_n, \liminf_n f_n$ are measurable.
+
+Proof:
+
+Consider the set $\{x\in \mathbb{R}, \sup_n f_n\leq c\}$.
+This is the set of $x$ such that $f_n(x)\leq c$ for all $n$.
+
+$\bigcap_{n=1}^{\infty} \{x\in \mathbb{R}, f_n(x)\leq c\} \subset \{x\in \mathbb{R}, \sup_n f_n(x)\leq c\}$, by the definition of least upper bound.
+
+Since the set on the right is intersection of measurable sets, it is measurable.
+
+Therefore, $\sup_n f_n$ is measurable.
+
+The proof for $\inf_n f_n, \limsup_n f_n, \liminf_n f_n$ are similar.
+
+Consider ${x\in \mathbb{R}, \inf_n f_n\leq c}=\bigcap_{n=1}^{\infty} \{x\in \mathbb{R}, f_n(x)\geq c\}$.
+
+$\limsup_n f_n(x)=\inf_n \sup_{k\geq n} f_k(x)$ is measurable by $\sup_{k\geq n} f_k(x)$ is measurable.
+
+$\liminf_n f_n(x)=\sup_n \inf_{k\geq n} f_k(x)$ is measurable by $\inf_{k\geq n} f_k(x)$ is measurable.
+
+QED
+
+#### Lemma of function of almost everywhere
+
+If $f$ is measurable function and $f(x)=g(x)$ for almost every $x$ (on a set which the complement has Lebesgue measure $0$), then $g$ is measurable.
+
+Proof:
+
+Let $c\in \mathbb{R}$, $F_1=\{x\in \mathbb{R}, f(x)>c\}$, $F_2=\{x\in \mathbb{R}, g(x)>c\}$.
+
+Recall the symmetric difference $F_1\triangle F_2=\{x\in \mathbb{R}, f(x)\neq g(x)\}$. By the definition of $g$, $F_1\triangle F_2$ has a measure $0$.
+
+In particular, all subsets of the $F_1\triangle F_2$ are measurable.
+
+Notice that $F_2=(F_1\setminus F_2)\cup (F_1\setminus (F_1\setminus F_2))$.
+
+Since $F_1\setminus F_2$ is measurable and $F_1$ is measurable, then $F_2$ is measurable.
+
+QED
+
+Example of measurable functions:
+
+- Continuous functions are measurable.
+
+$\{x:f(x)>c\}=\{x:f(x)\in (c,\infty)\}=f^{-1}(c,\infty)$ is open (by open mapping theorem, or the definition of continuity in topology).
+
+- Riemann integrable functions are measurable.
+
+Outer content of the discontinuity of the function is $0$.
+
+$\forall \sigma>0$, where $S_\sigma=\{x\in [a,b]: w(f,x)>\sigma\}$, $m(S_\sigma)=0$.
+
+$S=\bigcup_{n=1}^{\infty} S_{\frac{1}{n}}$ has a measure $0$. So $f$ is continuous outside a set of measure $0$.
+
+$m(S)\leq \sum_{n=1}^{\infty} m(S_{\frac{1}{n}})=0$. ~~So $f$ agrees with a continuous function outside a set of measure $0$. (almost everywhere)~~ (detailed proof in the textbook)
+
+#### Theorem 6.6
+
+Let $f_n$ be a sequence of measurable functions and $f$ is a function satisfying $\lim_{n\to\infty} f_n(x)=f(x)$ for almost every $x$ (holds for sets which the complement has Lebesgue measure $0$).
+
+Then $f(x)=\lim_{n\to\infty} f_n(x)$ is a measurable function.
+
+_Notice that $f(x)$ is defined "everywhere"_
+
+Proof:
+
+Apply the lemma of function of almost everywhere to the sequence $f_n$.
+
+QED
+
+#### Definition of simple function
+
+A measurable function $\phi:\mathbb{R}\to\mathbb{R}$ is called a simple function if it takes only finitely many values.
+
+$$
+\text{range}(\phi)=\{d(x):x\in \mathbb{R}\}\subset \mathbb{R}
+$$
+
+has finitely many values.
+
+Equivalently, $\exists \{a_1,a_2,\cdots,a_n\}\subset \mathbb{R}$ and disjoint measurable sets $S_1,S_2,\cdots,S_n$ such that
+
+$$
+\phi(x)=\sum_{i=1}^{n} a_i \chi_{S_i}(x)
+$$
+
+where $\chi_{S_i}$ is the indicator function of $S_i$.
+
+#### Theorem 6.7
+
+A function $f$ is measurable if and only if there exists a sequence of simple functions $\{\phi_n\}$ such that $\lim_{n\to\infty} \phi_n(x)=f(x)$ for almost every $x$.
+
+$f$ is a limit of almost everywhere convergent sequence of simple functions.
+
+(already proved backward direction)
+
+Continue on Monday.
+

pages/Math4121/Math4121_L34.md

@@ -0,0 +1,141 @@
+# Math4121 Lecture 34
+
+> Important:
+>
+> $\mathfrak{M}=\{S\subset \mathbb{R}: S \text{ satisfies the caratheodory condition}\}$, that is, for any $X$ of finite outer measure,
+>
+> $$m_e(X)=m_e(X\cap S)+m_e(X\cap S^c)$$
+> 
+> In particular, the measure of sets can be infinite, not necessarily bounded. (We want to make the real line measurable.)
+
+## Lebesgue Integral
+
+### Simple Function
+
+A function $\phi$ is called a simple function if
+
+$$
+\phi(x)=\sum_{i=1}^{n} a_i \chi_{S_i}(x)
+$$
+
+where $a_i\in \mathbb{R}$ and $\chi_{S_i}=\begin{cases}1, & x\in S_i \\ 0, & x\notin S_i\end{cases}$
+
+where $\{S_i\}_{i=1}^{n}$ are pairwise disjoint each having finite measure.
+
+**constant function** is not simple ($\mathbb{R}$ is not finite measurable sets.)
+
+#### Theorem 6.6
+
+A function $f$ is measurable on $[a,b]$ if and only if there exists a sequence of simple functions $\{\phi_n\}$ such that $\lim_{n\to\infty} \phi_n(x)=f(x)$ almost everywhere on $[a,b]$.
+
+Proof:
+
+Partition $[-n,n]$ into $n2^{n+1}$ pieces.
+
+(These are just horizontal strips from $-n$ to $n$ with width $\frac{1}{2^n}$.)
+
+$$
+E_{n,k}=\{x\in[-n,n]:\frac{k}{2^n}\leq f(x)<\frac{k+1}{2^n}\}
+$$
+
+for $-n2^n<k<n2^n$
+
+$$
+E_{n,n2^n}=\{x\in[-n,n]:f(x)\geq n\}
+$$
+
+$$
+E_{n,-n2^n}=\{x\in[-n,n]:f(x)<\frac{-n2^n+1}{2^n}\}
+$$
+
+$$
+\phi_n(x)=\frac{k}{2^n}\chi_{E_{n,k}}(x)
+$$
+
+is a simple function.
+
+We need to justify that $\phi_n(x)\to f(x)$ for all $x\in\mathbb{R}$.
+
+Let $x\in\mathbb{R}$. And choose $n_0$ large such that $x\in [-n_0,n_0]$ and $f(x)\in [-n_0,n_0]$.
+
+Then, for $n\geq n_0$,
+
+$$
+|\phi_n(x)-f(x)|<\frac{1}{2^n}\to 0
+$$
+
+as $n\to\infty$.
+
+QED
+
+### Integration
+
+Given a measurable set $E$ and a simple function $\phi$, we define
+
+$$
+\int_E \phi dm=\sum_{i=1}^{n} a_i m(E\cap S_i)
+$$
+
+#### Properties 6.10
+
+Let $\phi$ and $\psi$ be simple functions, $c\in \mathbb{R}$, and $E=E_1\cup E_2$ where $E_1\cap E_2=\emptyset$ and $E_1,E_2\in \mathfrak{M}$. Then,
+
+1. $\int_E c\phi dm=c\int_E \phi dm$ (linearity)
+2. $\int_E (\phi+\psi)dm=\int_E \phi dm+\int_E \psi dm$ (additivity of simple functions)
+3. if $\phi(x)\leq \psi(x)$ for all $x\in E$, then $\int_E \phi dm\leq \int_E \psi dm$ (monotonicity)
+4. $\int_E \phi(x)dm=\int_{E_1} \phi(x)dm+\int_{E_2} \phi(x)dm$ (additivity over **disjoint** measurable sets)
+
+Proof:
+
+Let $\phi(x)=\sum_{i=1}^{n} a_i \chi_{S_i}(x)$ and $\psi(x)=\sum_{j=1}^{m} b_j \chi_{T_j}(x)$.
+
+2.
+
+$$
+\phi+\psi=\sum_{i=1}^{n} a_i \chi_{S_i}+\sum_{j=1}^{m} b_j \chi_{T_j}
+$$
+
+Without loss of generality, we may assume that $x\in E$, $\bigcup_{i=1}^{n} S_i=\bigcup_{j=1}^{m} T_j=E$.
+
+So
+
+$$
+\phi+\psi=\sum_{i,j=1}^{n,m}(a_i+b_j) \chi_{S_i\cup T_j}
+$$
+
+is a simple function.
+
+$$
+\begin{aligned}
+\int_E (\phi+\psi)dm&=\sum_{i,j=1}^{n,m}(a_i+b_j) m(E\cap S_i\cup T_j) \\
+&=\sum_{i=1}^{n} a_i \sum_{j=1}^{m} m(E\cap S_i\cup T_j)+\sum_{j=1}^{m} b_j \sum_{i=1}^{n} m(E\cap S_i\cup T_j) \\
+&=\sum_{i=1}^{n} a_i m(E\cap S_i)+\sum_{j=1}^{m} b_j m(E\cap T_j) \\
+&=\int_E \phi dm+\int_E \psi dm
+\end{aligned}
+$$
+
+3.
+
+$$
+\phi(x)=\sum_{i=1}^{n} a_i\sum_{j=1}^{m} \chi_{S_i\cap T_j}(x)
+$$
+
+$$
+\psi(x)=\sum_{i=1}^{n} b_i\sum_{j=1}^{m} \chi_{S_i\cap T_j}(x)
+$$
+
+If $x\in S_i\cap T_j$, then $\phi(x)=a_i$ and $\psi(x)=b_j$, therefore $a_i\leq b_j$.
+
+So,
+
+$$
+\begin{aligned}
+\int_E \phi dm&=\sum_{i=1}^{n} \sum_{j=1}^{m} a_i m(E\cap S_i\cap T_j) \\
+&\leq \sum_{i=1}^{n} \sum_{j=1}^{m} b_i m(E\cap S_i\cap T_j) \\
+&=\int_E \psi dm
+\end{aligned}
+$$
+
+QED
+
+Back on Wednesday.