# CSE559A Lecture 20

## Local feature descriptors

Detection: Identify the interest points

Description: Extract vector feature descriptor surrounding each interest point.

Matching: Determine correspondence between descriptors in two views

### Image representation

Histogram of oriented gradients (HOG)

- Quantization
  - Grids: fast but applicable only with few dimensions
  - Clustering: slower but can quantize data in higher dimensions
- Matching
  - Histogram intersection or Euclidean may be faster
  - Chi-squared often works better
  - Earth mover’s distance is good for when nearby bins represent similar values

#### SIFT vector formation

Computed on rotated and scaled version of window according to computed orientation & scale

- resample the window

Based on gradients weighted by a Gaussian of variance half the window (for smooth falloff)

4x4 array of gradient orientation histogram weighted by magnitude

8 orientations x 4x4 array = 128 dimensions

Motivation:  some sensitivity to spatial layout, but not too much.

For matching:

- Extraordinarily robust detection and description technique
- Can handle changes in viewpoint
  - Up to about 60 degree out-of-plane rotation
- Can handle significant changes in illumination
  - Sometimes even day vs. night
- Fast and efficient—can run in real time
- Lots of code available

#### SURF

- Fast approximation of SIFT idea
- Efficient computation by 2D box filters & integral images
  - 6 times faster than SIFT
- Equivalent quality for object identification

#### Shape context

![Shape context descriptor](https://notenextra.trance-0.com/CSE559A/Shape_context_descriptor.png)

#### Self-similarity Descriptor

![Self-similarity descriptor](https://notenextra.trance-0.com/CSE559A/Self-similarity_descriptor.png)

## Local feature matching

### Matching