Computer vision can be defined as the processing of image data. Different information can be extracted, leading to a wide range of image based application including 3D reconstruction, object and face recognition, video surveillance, etc. In my research, we seek to recover geometric information of the scene from a series of images (structure, camera poses, etc). The camera, hand held or mounted on a system, is moved in an unknown scene. We aim to recover the camera motion and a sparse representation of the scene structure simultaneously.
In order to obtain the camera position, image features are detected and tracked in the sequence. An image feature is a visual landmark that corresponds to a physical feature of the scene; It can be, for example, the corner of a table. The relative movement of the features in the image as the camera moves allows to concurrently estimate the 3D position of the camera and of the features. In all vision based algorithms, it is explicitly assumed that corresponding features in images precisely correspond to the same physical point of the scene. Any departure from this assuption generates a bias in the positioning and in the reconstruction. The main objective of my work is to identify image features that are most likely to correspond to the exact same physical point of the scene. This is possible by analyzing 2D and 3D properties of the features as they are observed from multiple viewpoints.
Natural Feature Detection
Stability under viewpoint variation. In this part of my research, I am interested in developing an accurate keypoint detector that is stable under viewpoint change. The problem is illustrated in the image on the figure on the right where points are detected in the image and projected back into the first view of the sequence. The bottom-left image shows SIFT keypoint with color corresponding to the position of the camera when they where detected. As the camera moves, the detection is affected by the perspective deformation of the scene structure and for some image points, a drift is observed: the point do not correspond to the exact same physical point creating a line of point. The bottom-right image shows the desired behaviour where the features do not drift.
Camera Calibration
Camera calibration consists in recovering the relationship between pixel positions in the image and their actual position on the camera ccd sensor, with a metric scale (millimeter for example). Knowing the camera calibration is a prerequisite to perform the reconstruction of a scene in euclidean coordinates where lengths (distances), angles and parallelism of objects and structures are preserved (up to a scale factor).
Ellipses localization. To calibrate a camera, we identify correspondences between feature coordinates in the image and their corresponding coordinates in the scene. Generally, we acquire images of known target. In this context, circles offer a very high localization accuracy. This is important knowing that the overall accuracy of the calibration is directly related to the localization accuracy of features in the image. In 2000, Heikkila presented a calibration method exploiting a planar target printed with circular features [Heikkila2000]. Still, detecting and localizing ellipse features is difficult and requires multiple steps: gradient estimation, non-maxima supression, subpixel refinement and ellipse fitting. To simplifiy the detection task, we have developped a simple linear operator that estimates very accuratly the parameters of ellipse features directly from the gradient of an image region, without contour point extraction (see [CRV2007]).
Quality of input images. The presence of motion or out of focus blur in a calibration image reduce the sharpness of the feature boundary, decreasing the accuracy of localization and of the overall calibration. Image blur should be identified beforhand. This task requires the manual inspection of each image separately. As multiple images are generally used in calibration (15+), manual inspection is not appropriate. In [ICPR2004], we have proposed an indicator for predicting the quality of camera model parameters from a set of input images by assessing the blur. The indicator is based on the acutance and can indicate static or motion blur during image capture.
Structured Light System (SLS) calibration. We are currently developping a structured light system (SLS) combining a DLP projector with a camera to recover the 3D surface and appearance of objects. The geometric calibration of such system is crucial to obtain a precise surface reconstruction. The literature on SLS calibration is abundant but the methods presented could greatly benefit from an exhaustive and rigorous review. In this context, we presented a new geometric calibration method for SLS using a planar target with circular control points (see [3DPVT08]). The method solely exploit ellipse correspondences which result in a method strictly geometric unaffected by photometric bias. Furthermore, the method consistently ensures a uniform coverage of the working volume and automatically avoids interference between the projected and the printed patterns on the calibration target.