Intro to Image Understanding (CSC420) Assignment 3

Intro to Image Understanding (CSC420)
Assignment 3
Max points: 15
1. [5 points] Interest point detection:
(a) [4 points] Implement a function to perform Harris corner detection. The
function should take as input an image, and return corners. You can make
the image grayscale.
(b) [1 point] Plot your result for the attached image building.jpg, and add it to
your pdf/doc file.
2. [10 points] For this exercise you will use the Scale-Invariant Feature Transform
(SIFT) for matching. You will extract SIFT features from two images and use
them to find feature correspondences and solve for the affine transformation between them. Please include code under each question. You are allowed to use
external code for SIFT keypoint and descriptor extraction (but not for matching).
(a) [2 points] Feature extraction: Compute SIFT features for reference.png,
test.png, and test2.png. You can use e.g. function sift() from existing packages to output, for each image, a list of feature descriptors and a list of their
corresponding frames. Please visualize the detected keypoints on the image.
By visualize we mean: plot the image, mark the center of each keypoint, and
draw either a circle or rectangle to indicate the scale of each keypoint. For
clarity, please plot only 100 keypoints. Please write the visualization function
yourself – a function that loops over the extracted keypoints, and displays
each keypoint on the image.
(b) [3 points] Matching: Given the extracted features on reference.png and
test.png, describe a simple matching algorithm to find the best feature matches
(correspondences) for the features in reference.png and features in image test.png.
How did you define “best” matches? Implement the algorithm in your favorite
programming language. Visualize the top (best) 3 correspondences. Please
describe what you chose as a criteria to evaluate best matches. Please do the
1
same using the test image test2.png. Show each image and visualize each correspondence by indicating the feature’s position and scale in the appropriate
image. Use a separate color for each correspondence.
(c) [3 points] Affine transformation: Use the top 3 correspondences from
part (b) to solve for the affine transformation between the features in the two
images.
(d) [2 points] Visualize the affine transformation. Do this visualization by taking
the four corners of the reference image, transforming them via the computed
affine transformation to the points in the second image, and plotting those
transformed points. Please also plot the edges between the points to indicate
the parallelogram. If you are unsure what the instruction is, please look at
Figure 12 of [Lowe, 2004].
2