This project proposes to develop a methodology for video coding by using intermediate 3D representations. The coded video includes a representation of the scene’s 3D shape and texture, producing as output a sequence of synthetic views of the scene (which could be different from the original ones). This opens new perspectives to go one step forward towards general content-based video representations.
There are three main issues involved, which we propose to tackle:

1 – Image to image matching,
2 – Image to model matching and
3 – 3D model building and generation of images from the 3D model.

The main idea is to create a feedback loop of “a priori” knowledge provided by the existent 3D scene model into the matching process, in a global way. By formalizing image to image and image to model matching as an integer programming problem which is then relaxed to a concave programming problem (see [5]), authors believe the whole process of matching and 3D reconstruction can be integrated into a single recursive framework.