M4. 3D Vision

The goal of this module is to learn the principles of the 3D reconstruction of an object or a scene from multiple images or stereoscopic videos. For that, the basic concepts of the projective geometry and the 3-space are firstly introduced. The rest of the theoretical aspects and applications are built upon these basic tools. The mapping from the 3D world to the image plane will be studied, for that we will introduce different camera models, their parameters and how to estimate them (camera calibration and auto-calibration). The geometry that relates two pair of views will be analyzed. All these concepts will be applied to obtain a 3D reconstruction in the possible settings: calibrated or uncalibrated cameras. In particular, we will learn how to: estimate the depth of image points, extract the underlying 3D points given a set of point correspondences in the images, generate novel views, estimate the 3D object given a set of calibrated color images or binary images, and estimate a sparse set of 3D points given a set of uncalibrated images. The 3D representation in voxels and meshes will be studied. Finally, we will explain the reconstruction and modeling from Kinect data, as a particular model of sensors that provide and image of the scene together with its depths. The concepts and techniques learnt in this module are used in real applications ranging from augmented reality, object scanning, motion capture, new view synthesis, bullet-time effect, robotics, etc.
Project title: 
3D Recovery of urban scenes

The main goal of the project is the 3D reconstruction of buildings from a set of images taken from different points of view. We will work with two kind of images from different public datasets: frontal images of the façades and aerial images. The 3D reconstruction of urban scenes has many applications such as virtual navigation, urban planification and architectural design, traffic and noise simulation.
More specifically the student will learn how to rectify the perspective distortion from a single view, what is the geometry and the mathematical tools that relate points seen in two views and how to estimate them given a set of point correspondences between the images, calibrate a camera with a planar pattern, estimate the depth of points in the scene given two images, generate new views of the scene, and compute a 3D model either from a set of calibrated or uncalibrated cameras. These tools are not only specific to images of buildings and can be applied to generic images for different real applications ranging from augmented reality, object scanning, motion capture, new view synthesis, bullet-time effect, robotics, etc.

Module lectures: 
Academic Year 2017-2018        
week date time lecture lecturer university building room
1 Tue. Dec. 5th 16:00 - 18:00 Introduction and applications. 2D projective geometry Gloria Haro UPF Roc Boronat 52.323 
1 Thu. Dec. 7th   HOLIDAY        
               
2 Tue. Dec. 12th 16:00 -17:00

2D projective geometry. Planar transformations. 

Gloria Haro UPF Roc Boronat 52.323 
2 Tue. Dec. 12th 17:00 -18:00 Homography estimation. Affine and metric rectification. Coloma Ballester UPF Roc Boronat  52.323 
2 Thu. Dec. 14th 16:00 -18:00 3D projecte geometry and transformations. Camera models. Coloma Ballester UPF Roc Boronat  52.325 
2 Thu. Dec. 14th 18:00 - 19:00 Project introduction Gloria Haro UPF Roc Boronat  52.325  
               
3 Tue. Dec. 19th 16:00 - 18:00 Camera calibration. Pose estimation. Gloria Haro UPF Roc Boronat  52.325 
3 Thu. Dec. 21st 16:00 - 18:00 Epipolar geometry. Fundamental matrix. Essential matrix. Extraction of camera matrices Javier Ruiz UPF Roc Boronat  52.325 
3 Thu. Dec. 21st 18:00 - 19:00 Project follow-up Gloria Haro UPF Roc Boronat  52.325 
               
      Christmas Holidays (from 22nd December to 7th January)        
               
4 Tue. Jan. 9th 16:00 -18:00 Computation of the fundamental matrix. Image rectification. Javier Ruiz UPF Roc Boronat  52.325 
4 Thu. Jan. 11th 16:00 -18:00 Triangulation methods. Depth computation. New View Synthesis.  Gloria Haro  UPF Roc Boronat  52.421 
4 Thu. Jan. 11th 18:00 - 19:00 Project follow-up Gloria Haro UPF Roc Boronat  52.421  
               
5 Tue. Jan. 16th 16:00 -18:00 Multi-view stereo. Structure from motion. Antonio Agudo UPF  Roc Boronat 52.421  
5 Thu. Jan. 18th 16:00 -17:00 Auto-Calibration. Bundle adjustment.  Antonio Agudo  UPF  Roc Boronat  52.329 
5 Thu. Jan. 18th 17:00 -18:00 3D sensors (kinect) Josep Ramon Casas UPF  Roc Boronat  52.329 
5 Thu. Jan. 18th 18:00 - 19:00 Project follow-up Gloria Haro UPF  Roc Boronat  52.329 
               
6 Tue. Jan. 23rd 16:00 -18:00  HOMEWORK        
6 Thu. Jan. 25th 16:00 -18:00 Point cloud processing Josep Ramon Casas UPF Roc Boronat 52.221 
6 Thu. Jan. 25th 18:00 - 19:00 Project follow-up Gloria Haro UPF Roc Boronat  52.221 
               
7 Thu. Feb. 1st 16:00 -19:00 Project presentations Gloria Haro UPF Roc Boronat  52.329 
               
8 Tue. Feb. 6th   HOMEWORK        
8 Thu. Feb. 8th   HOMEWORK        
               
9 Thu. Feb. 15th 16:00 -18:00 EXAM Gloria Haro UPF Roc Boronat  52.119 
               

M4 Student Guide [here]