Stereo and Segmentation_Homework 3 Solution

Stereo  and  Segmentation    
For  Assignment  2,  you  will  implement  a  few  functions  in  a  stereo  pipeline  example   provided  by  mathworks.com     Provided  files:   In  your  Matlab  Command  Window,  type   edit DepthEstimationFromStereoVideoExample   What  You  Have  to  Do:       Task  1  (30  points):  Calculate  disparity  using  the  SSD  algorithm   Implement  the  SSD  matching  algorithm  for  a  range  of  window  sizes,  and  create  a   disparity  image  D(x,  y)  such  that  Left(x,  y)  =  Right(x  +  D(x,  y),  y)  when  matching   from  left  to  right.  Remember  to  reduce  the  search  only  over  epipolar  lines.  You   should  also  restrict  the  search  using  the  disparity  range  (see  line  56  in  the  original   script).     Write  a  function  to  replace  the  disparity  built-­‐in  function  on  line  54.  Your  function   needs  to  accept  as  an  input  value  the  size  of  the  window  search.  Call  your  function   three  times  for  the  following  window  sizes:  1,  3,  5.       Create  a  2x2  subplot  and  display  the  disparity  map  results  from  these  three  function   calls,  plus  the  one  obtained  using  Matlab’s  built-­‐in  disparity  function.     Note:  when  using  a  window  size  bigger  than  1,  it  is  common  practice  to  use  a   Gaussian  weighting  of  the  window,  so  that  the  center  has  greater  influence  than  the   periphery.  Convolve  your  window  matrices  with  a  Gaussian  filter  prior  to   computing  the  sum  of  squared  differences.       Task  2  (30  points):  Calculate  disparity  using  the  NCC  algorithm   Implement  the  normalized  cross  correlation  (NCC)  matching  algorithm  for  a  range   of  window  sizes,  and  create  a  disparity  image  D(x,  y)  such  that  Left(x,  y)  =  Right(x  +   D(x,  y),  y)  when  matching  from  left  to  right.  Similarly,  remember  to  reduce  the   search  only  over  epipolar  lines  and  to  restrict  the  search  using  the  disparity  range   (see  line  56  in  the  original  script).     Information  on  how  to  compute  NCC  for  image-­‐processing  applications  here:   https://en.wikipedia.org/wiki/Cross-­‐correlation#Normalized_cross-­‐correlation   The  inner  product  of  vectors  approach  mentioned  on  Wikipedia,  also  here,  slide  66:   http://www.gris.tu-­‐darmstadt.de/teaching/courses/ws0910/cv2_ws0910/slides/l3-­‐stereo-­‐v1_0.pdf  
Write  a  function  to  replace  the  disparity  built-­‐in  function  on  line  54.  Your  function   needs  to  accept  as  an  input  value  the  size  of  the  window  search.  Call  your  function   three  times  for  the  following  window  sizes:  3,  5  and  7.       Create  a  2  x  2  subplot  and  display  the  disparity  map  results  from  these  three   function  calls,  plus  the  one  obtained  using  Matlab’s  built-­‐in  disparity  function.     Task  3  (30  points):  Generate  outliers  map  -­‐  Refine  the  disparity  by  performing  a   left-­‐right  consistency  check.     The  disparity  map  dLR(x)  is  acquired  considering  as  reference  image  the  left  image   of  the  stereo  pair.  If  the  right  image  is  considered  as  reference,  then  the  disparity   map  dRL(x)  is  acquired.  The  disparity  maps  dLR(x)  and  dRL(x)  can  be  useful  in   detecting  problematic  areas,  especially  outliers  in  occluded  regions  and  depth   discontinuities.  One  strategy  for  detecting  outliers  is  the  Left–Right  consistency   check  introduced  by  1.  In  this  strategy,  the  outliers  are  disparity  values  that  are  not   consistent  between  the  two  maps  and  therefore,  they  do  not  satisfy  the  relation:     |  dLR(x)  −dRL(x−dLR(x))  |  ≤  TLR,  where  TLR  is  a  user-­‐defined  threshold.       Write  a  function  that  accepts  as  inputs  two  disparity  maps  (LR  and  RL)  and  a  value   for  TLR  and  returns  a  binary  image  where  the  outliers  have  the  value  1  and  the   inliers  have  the  value  0.       Call  this  function  twice,  passing  as  inputs  the  LR  and  RL  disparity  maps  from  SSD   and  NCC  matching  with  window  size  of  3.  Use  a  TLR  value  of  1.  With  this  value,  pixels   with  difference  equal  to  1  in  the  Left–Right  consistency  check  are  not  considered   outliers.  Plot  the  two  resulting  binary  images  side  by  side.     Task  4  (10  points):  Compute  depth  from  disparity   Now that we have a disparity image, computing depth at every pixel in the left image should be very straightforward. Use equation 11.1 from the Szeliski textbook.   Write  a  function  to  replace  the  reconstructScene  built-­‐in  function  on  line  64.  Your   function  should  have  the  same  input  arguments  as  the  built-­‐in  version  and  it  should   return  a  matrix  of  depth  values  for  each  pixel  location  from  the  left  image.