Assignment 3 Estimation of the camera pose

Assignment 3
Problems
1. Programming: Estimation of the camera pose (rotation and translation of a calibrated camera) (80 points)
(a) Outlier rejection (20 points) Download input data from the course website. The file hw3_points3D.txt contains the coordinates of 60 scene points in 3D (each line of the file gives the ˜ Xi, ˜ Yi, and ˜ Zi inhomogeneous coordinates of a point). The file hw3_points2D.txt contains the coordinates of the 60 corresponding image points in 2D (each line of the file gives the ˜ xi and ˜ yi inhomogeneous coordinates of a point). The corresponding 3D scene and 2D image points contain both inlier and outlier correspondences. For the inlier correspondences, the scene points have been randomly generated and projected to image points under a camera projection matrix (i.e., xi = PXi), then noise has been added to the image point coordinates. The camera calibration matrix was calculated for a 1280×720 sensor and 45◦ horizontal field of view lens. The resulting camera calibration matrix is given by K =  1545.0966799187809 0 639.5 0 1545.0966799187809 359.5 0 0 1   For each image point x = (x,y,w) = (˜ x, ˜ y,1), calculate the point in normalized coordinates ˆ x = K−1x. Determine the set of inlier point correspondences using the M-estimator Sample Consensus (MSAC) algorithm, where the maximum number of attempts to find a consensus set is determined adaptively. For each trial, use the 3-point algorithm of Finsterwalder (as described in the paper by Haralick et al.) to estimate the camera pose (i.e., the rotation R and translation t from the world coordinate frame to the camera coordinate frame), resulting in up to 4 solutions, and calculate the error and cost for each solution. Note that the 3-point algorithm requires the 2D points in normalized coordinates, not in image coordinates. Calculate the projection error, which is the (squared) distance between projected points (the points in 3D projected under the normalized camera projection matrix ˆ P = [R|t]) and the measured points in normalized coordinates (hint: the error tolerance is simpler to calculate in image coordinates using P = K[R|t] than in normalized coordinates using ˆ P = [R|t]). In your report, describe any assumptions, including the probability p that at least one of the random samples does not contain any outliers (used to determine the number of attempts to find a consensus set), and the probability α that a given data point is an inlier and the variance σ2 of the measurement error (both used to determine the distance threshold; hint: this problem has codimension 2). Additionally, show the resulting number of inliers and the number of attempts to find the consensus set.
2
(b) Linear estimation (30 points) Estimate the normalized camera projection matrix ˆ Plinear = [Rlinear|tlinear] from the resulting set of inlier correspondences using the linear estimation method (based on the EPnP method) described in lecture. Include the numerical values of the resulting Rlinear and tlinear in your report with sufficient precision such that it can be evaluated (hint: use format longg in MATLAB prior to displaying your results). (c) Nonlinear estimation (30 points) Use Rlinear and tlinear as an initial estimate to an iterative estimation method, specifically the Levenberg-Marquardt algorithm, to determine the Maximum Likelihood estimate of the camera pose that minimizes the projection error under the normalized camera projection matrix ˆ P = [R|t]. You must parameterize the cam-era rotation using the angle-axis representation ω (where [ω]× = lnR) of a 3D rotation, which is a 3-vector. In your report, show the initial cost (i.e., the cost at iteration 0) and the cost at the end of each successive iteration. Show the numerical values for the final estimate of the camera rotation ωLM and RLM = e[ωLM]×, and the camera translation tLM in your report with sufficient precision such that it can be evaluated.