diff --git a/pages/CSE559A/CSE559A_L5.md b/pages/CSE559A/CSE559A_L5.md new file mode 100644 index 0000000..f925461 --- /dev/null +++ b/pages/CSE559A/CSE559A_L5.md @@ -0,0 +1,222 @@ +# Lecture 5 + +## Continue on linear interpolation + +- In linear interpolation, extreme values are at the boundary. +- In bicubic interpolation, extreme values may be inside. + +`scipy.interpolate.RegularGridInterpolator` + +### Image transformations + +Image warping is a process of applying transformation $T$ to an image. + +Parametric (global) warping: $T(x,y)=(x',y')$ + +Geometric transformation: $T(x,y)=(x',y')$ This applies to each pixel in the same way. (global) + +#### Translation + +$T(x,y)=(x+a,y+b)$ + +matrix form: + +$$ +\begin{pmatrix} +x'\\y' +\end{pmatrix} += +\begin{pmatrix} +1&0\\0&1 +\end{pmatrix} +\begin{pmatrix} +x\\y +\end{pmatrix} ++ +\begin{pmatrix} +a\\b +\end{pmatrix} +$$ + +#### Scaling + +$T(x,y)=(s_xx,s_yy)$ matrix form: + +$$ +\begin{pmatrix} +x'\\y' +\end{pmatrix} += +\begin{pmatrix} +s_x&0\\0&s_y +\end{pmatrix} +\begin{pmatrix} +x\\y +\end{pmatrix} +$$ + +#### Rotation + +$T(x,y)=(x\cos\theta-y\sin\theta,x\sin\theta+y\cos\theta)$ + +matrix form: + +$$ +\begin{pmatrix} +x'\\y' +\end{pmatrix} += +\begin{pmatrix} +\cos\theta&-\sin\theta\\\sin\theta&\cos\theta +\end{pmatrix} +\begin{pmatrix} +x\\y +\end{pmatrix} +$$ + +To undo the rotation, we need to rotate the image by $-\theta$. This is equivalent to apply $R^T$ to the image. + +#### Affine transformation + +$T(x,y)=(a_1x+a_2y+a_3,b_1x+b_2y+b_3)$ + +matrix form: + +$$ +\begin{pmatrix} +x'\\y' +\end{pmatrix} += +\begin{pmatrix} +a_1&a_2&a_3\\b_1&b_2&b_3 +\end{pmatrix} +\begin{pmatrix} +x\\y\\1 +\end{pmatrix} +$$ + +Taking all the transformations together. + +#### Projective homography + +$T(x,y)=(\frac{ax+by+c}{gx+hy+i},\frac{dx+ey+f}{gx+hy+i})$ + +$$ +\begin{pmatrix} +x'\\y'\\1 +\end{pmatrix} += +\begin{pmatrix} +a&b&c\\d&e&f\\g&h&i +\end{pmatrix} +\begin{pmatrix} +x\\y\\1 +\end{pmatrix} +$$ + +### Image warping + +#### Forward warping + +Send each pixel to its new position and do the matching. + +- May cause gaps where the pixel is not mapped to any pixel. + +#### Inverse warping + +Send each new position to its original position and do the matching. + +- Some mapping may not be invertible. + +#### Which one is better? + +- Inverse warping is better because it usually more efficient, doesn't have a problem with holes. +- However, it may not always be possible to find the inverse mapping. + +## Sampling and Aliasing + +### Naive sampling + +- Remove half of the rows and columns in the image. + +Example: + +When sampling a sine wave, the result may interpret as different wave. + +#### Nyquist-Shannon sampling theorem + +- A bandlimited signal can be uniquely determined by its samples if the sampling rate is greater than twice the maximum frequency of the signal. + +- If the sampling rate is less than twice the maximum frequency of the signal, the signal will be aliased. + +#### Anti-aliasing + +- Sample more frequently. (not always possible) +- Get rid of all frequencies that are greater than half of the new sampling frequency. + - Use a low-pass filter to get rid of all frequencies that are greater than half of the new sampling frequency. (eg, Gaussian filter) + +```python +import scipy.ndimage as ndimage +def down_sample(height, width, image): + # Apply Gaussian blur to the image + im_blur = ndimage.gaussian_filter(image, sigma=1) + # Down sample the image by taking every second pixel + return im_blur[::2, ::2] +``` + +## Nonlinear filtering + +### Median filter + +Replace the value of a pixel with the median value of its neighbors. + +- Good for removing salt and pepper noise. (black and white dot noise) + +### Morphological operations + +Binary image: image with only 0 and 1. + +Let $B$ be a structuring element and $A$ be the original image (binary image). + +- Erosion: $A\ominus B = \{p\mid B_p\subseteq A\}$, this is the set of all points that are completely covered by $B$. +- Dilation: $A\oplus B = \{p\mid B_p\cap A\neq\emptyset\}$, this is the set of all points that are at least partially covered by $B$. +- Opening: $A\circ B = (A\ominus B)\oplus B$, this is the set of all points that are at least partially covered by $B$ after erosion. +- Closing: $A\bullet B = (A\oplus B)\ominus B$, this is the set of all points that are completely covered by $B$ after dilation. + +Boundary extraction: use XOR operation on eroded image and original image. + +Connected component labeling: label the connected components in the image. _use prebuild function in scipy.ndimage_ + +## Light,Camera/Eyes, and Color + +### Principles of grouping and Gestalt Laws + +- Proximity: objects that are close to each other are more likely to be grouped together. +- Similarity: objects that are similar are more likely to be grouped together. +- Closure: objects that form a closed path are more likely to be grouped together. +- Continuity: objects that form a continuous path are more likely to be grouped together. + +### Light and surface interactions + +A photon's life choices: + +- Absorption +- Diffuse reflection (nice to model) (lambertian surface) +- Specular reflection (mirror-like) (perfect mirror) +- Transparency +- Refraction +- Fluorescence (returns different color) +- Subsurface scattering (candles) +- Photosphorescence +- Interreflection + +#### BRDF (Bidirectional Reflectance Distribution Function) + +$$ +\rho(\theta_i,\phi_i,\theta_o,\phi_o) +$$ + +- $\theta_i$ is the angle of incidence. +- $\phi_i$ is the azimuthal angle of incidence. +- $\theta_o$ is the angle of reflection. +- $\phi_o$ is the azimuthal angle of reflection. diff --git a/pages/CSE559A/CSE559A_L6.md b/pages/CSE559A/CSE559A_L6.md new file mode 100644 index 0000000..e69de29 diff --git a/pages/CSE559A/_meta.js b/pages/CSE559A/_meta.js index e6563d1..649aa52 100644 --- a/pages/CSE559A/_meta.js +++ b/pages/CSE559A/_meta.js @@ -7,4 +7,6 @@ export default { CSE559A_L2: "Computer Vision (Lecture 2)", CSE559A_L3: "Computer Vision (Lecture 3)", CSE559A_L4: "Computer Vision (Lecture 4)", + CSE559A_L5: "Computer Vision (Lecture 5)", + CSE559A_L6: "Computer Vision (Lecture 6)", }