diff --git a/pages/CSE559A/CSE559A_L20.md b/pages/CSE559A/CSE559A_L20.md index 196e089..7781656 100644 --- a/pages/CSE559A/CSE559A_L20.md +++ b/pages/CSE559A/CSE559A_L20.md @@ -62,3 +62,84 @@ For matching: ## Local feature matching ### Matching + +Simplest approach: Pick the nearest neighbor. Threshold on absolute distance + +Problem: Lots of self similarity in many photos + +Solution: Nearest neighbor with low ratio test + +![Comparison of keypoint detectors](https://notenextra.trance-0.com/CSE559A/Comparison_of_keypoint_detectors.png) + +## Deep Learning for Correspondence Estimation + +![Deep learning for correspondence estimation](https://notenextra.trance-0.com/CSE559A/Deep_learning_for_correspondence_estimation.png) + +## Optical Flow + +### Field + +Motion field: the projection of the 3D scene motion into the image +Magnitude of vectors is determined by metric motion +Only caused by motion + +Optical flow: the apparent motion of brightness patterns in the image +Magnitude of vectors is measured in pixels +Can be caused by lightning + +### Brightness constancy constraint, aperture problem + +Machine Learning Approach + +- Collect examples of inputs and outputs +- Design a prediction model suitable for the task + - Invariances, Equivariances; Complexity; Input and Output shapes and semantics +- Specify loss functions and train model +- Limitations: Requires training the model; Requires a sufficiently complete training dataset; Must re-learn known facts; Higher computational complexity + +Optimization Approach + +- Define properties we expect to hold for a correct solution +- Translate properties into a cost function +- Derive an algorithm to solve for the cost function +- Limitations: Often requires making overly simple assumptions on properties; Some tasks can’t be easily defined + +Given frames at times $t-1$ and $t$, estimate the apparent motion field $u(x,y)$ and $v(x,y)$ between them +Brightness constancy constraint: projection of the same point looks the same in every frame + +$$ +I(x,y,t-1) = I(x+u(x,y),y+v(x,y),t) +$$ + +Additional assumptions: + +- Small motion: points do not move very far +- Spatial coherence: points move like their neighbors + +Trick for solving: + +Brightness constancy constraint: + +$$ +I(x,y,t-1) = I(x+u(x,y),y+v(x,y),t) +$$ + +Linearize the right-hand side using Taylor expansion: + +$$ +I(x,y,t-1) \approx I(x,y,t) + I_x u(x,y) + I_y v(x,y) +$$ + +$$ +I_x u(x,y) + I_y v(x,y) + I(x,y,t) - I(x,y,t-1) = 0 +$$ + +Hence, + +$$ +I_x u(x,y) + I_y v(x,y) + I_t = 0 +$$ + + + + diff --git a/public/CSE559A/Comparison_of_keypoint_detectors.png b/public/CSE559A/Comparison_of_keypoint_detectors.png new file mode 100644 index 0000000..cab7604 Binary files /dev/null and b/public/CSE559A/Comparison_of_keypoint_detectors.png differ diff --git a/public/CSE559A/Deep_learning_for_correspondence_estimation.png b/public/CSE559A/Deep_learning_for_correspondence_estimation.png new file mode 100644 index 0000000..f39361a Binary files /dev/null and b/public/CSE559A/Deep_learning_for_correspondence_estimation.png differ