upgrade structures and migrate to nextra v4

content/CSE559A/CSE559A_L6.md

@@ -0,0 +1,213 @@
+# CSE559A Lecture 6
+
+## Continue on Light, eye/camera, and color
+
+### BRDF (Bidirectional Reflectance Distribution Function)
+
+$$
+\rho(\theta_i,\phi_i,\theta_o,\phi_o)
+$$
+
+#### Diffuse Reflection
+
+- Dull, matte surface like chalk or latex paint
+
+- Most often used in computer vision
+- Brightness _does_ depend on direction of illumination
+
+Diffuse reflection governed by Lambert's law: $I_d = k_d N\cdot L I_i$
+
+- $N$: surface normal
+- $L$: light direction
+- $I_i$: incident light intensity
+- $k_d$: albedo
+
+$$
+\rho(\theta_i,\phi_i,\theta_o,\phi_o)=k_d \cos\theta_i
+$$
+
+#### Photometric Stereo
+
+Suppose there are three light sources, $L_1, L_2, L_3$, and we have the following measurements:
+
+$$
+I_1 = k_d N\cdot L_1
+$$
+
+$$
+I_2 = k_d N\cdot L_2
+$$
+
+$$
+I_3 = k_d N\cdot L_3
+$$
+
+We can solve for $N$ by taking the dot product of $N$ and each light direction and then solving the system of equations.
+
+Will not do this in the lecture.
+
+#### Specular Reflection
+
+- Mirror-like surface
+
+$$
+I_e=\begin{cases}
+I_i & \text{if } V=R \\
+0 & \text{if } V\neq R
+\end{cases}
+$$
+
+- $V$: view direction
+- $R$: reflection direction
+- $\theta_i$: angle between the incident light and the surface normal
+
+Near-perfect mirror have a high light around $R$.
+
+common model:
+
+$$
+I_e=k_s (V\cdot R)^{n_s}I_i
+$$
+
+- $k_s$: specular reflection coefficient
+- $n_s$: shininess (imperfection of the surface)
+- $I_i$: incident light intensity
+
+#### Phong illumination model
+
+- Phong approximation of surface reflectance
+  - Assume reflectance is modeled by three compoents
+    - Diffuse reflection
+    - Specular reflection
+    - Ambient reflection
+
+$$
+I_e=k_a I_a + I_i \left[k_d (N\cdot L) + k_s (V\cdot R)^{n_s}\right]
+$$
+
+- $k_a$: ambient reflection coefficient
+- $I_a$: ambient light intensity
+- $k_d$: diffuse reflection coefficient
+- $k_s$: specular reflection coefficient
+- $n_s$: shininess
+- $I_i$: incident light intensity
+
+Many other models.
+
+#### Measuring BRDF
+
+Use Gonioreflectometer.
+
+- Device for measuring the reflectance of a surface as a function of the incident and reflected angles.
+- Can be used to measure the BRDF of a surface.
+
+BRDF dataset:
+
+- MERL dataset
+- CURET dataset
+
+### Camera/Eye
+
+#### DSLR Camera
+
+- Pinhole camera model
+- Lens
+- Aperture (the pinhole)
+- Sensor
+- ...
+
+#### Digital Camera block diagram
+
+![Digital Camera block diagram](https://notenextra.trance-0.com/CSE559A/DigitalCameraBlockDiagram.png)
+
+Scanning protocols:
+
+- Global shutter: all pixels are exposed at the same time
+- Interlaced: odd and even lines are exposed at different times
+- Rolling shutter: each line is exposed as it is read out
+
+#### Eye
+
+- Pupil
+- Iris
+- Retina
+- Rods and cones
+- ...
+
+#### Eye Movements
+
+- Saccade
+  - Can be consciously controlled. Related to perceptual attention.
+  - 200ms to initiation, 20 to 200ms to carry out. Large amplitude.
+- Smooth pursuit
+  - Tracking an object
+  - Difficult w/o an object to track!
+- Microsaccade and Ocular microtremor (OMT)
+  - Involuntary. Smaller amplitude. Especially evident during prolonged
+    fixation.
+
+#### Contrast Sensitivity
+
+- Uniform contrast image content, with increasing frequency
+- Why not uniform across the top?
+- Low frequencies: harder to see because of slower intensity changes
+- Higher frequencies: harder to see because of ability of our visual system to resolve fine features
+
+### Color Perception
+
+Visible light spectrum: 380 to 780 nm
+
+- 400 to 500 nm: blue
+- 500 to 600 nm: green
+- 600 to 700 nm: red
+
+#### HSV model
+
+We use Gaussian functions to model the sensitivity of the human eye to different wavelengths.
+
+- Hue: color (the wavelength of the highest peak of the sensitivity curve)
+- Saturation: color purity (the variance of the sensitivity curve)
+- Value: color brightness (the highest peak of the sensitivity curve)
+
+#### Color Sensing in Camera (RGB)
+
+- 3-chip vs. 1-chip: quality vs. cost
+
+Bayer filter:
+
+- Why more green?
+  - Human eye is more sensitive to green light.
+
+#### Color spaces
+
+Images in python:
+
+As matrix.
+
+```python
+import matplotlib.pyplot as plt
+
+from mpl_toolkits.mplot3d import Axes3D
+from skimage import io
+
+def plot_rgb_3d(image_path):
+    image = io.imread(image_path)
+    r, g, b = image[:,:,0], image[:,:,1], image[:,:,2]
+    fig = plt.figure()
+    ax = fig.add_subplot(111, projection='3d')
+    ax.scatter(r.flatten(), g.flatten(), b.flatten(), c=image.reshape(-1, 3)/255.0, marker='.')
+    ax.set_xlabel('Red')
+    ax.set_ylabel('Green')
+    ax.set_zlabel('Blue')
+    plt.show()
+
+plot_rgb_3d('image.jpg')
+```
+
+Other color spaces:
+
+- YCbCr (fast to compute, usually used in TV)
+- HSV
+- L\*a\*b\* (CIELAB, perceptually uniform color space)
+
+Most information is in the intensity channel.