Vision+

In the context of the Vision + K project, a model-based tracking method is being developed that uses 3D models to calculate the 6D pose (orientation and position). A special focus is on the tracking of texture-less objects – such as cars or machines in industrial plants. The appli-cation fields of this tracking method range from robotics to augmented reality.

Detection of 3D object pose

We developed a markerless model-based track-ing solution (Fig. 1). Edges are the most in-formative local features on texture-less objects. Hence, the pose is calculated by establishing correspondences between projected edge points from the 3D object model and edge points from an input image

Fig. 1: Overview of the input and output data of the developed
edge-based tracking solution. (Copyright: AIT)

Model-based tracking

Together with our industry partner INR we developed within the Vision+ K-project a model-based edge tracking algorithm for texture–less objects. After the initialization, edge points are employed to estimate inter-frame motions. A 3D wireframe model is projected into the 2D image according to a pose hypothesis. Then a set of points is sampled along edges of the wireframe model with a fixed step size. As some of the sampled points are occluded by the object itself, a visibility test is necessary. A local moving edge tracker is used in order to provide tracking of points normal to the object contours. Afterwards we perform a refinement process based on Iteratively Reweighted Least Squares (IRLS) to improve the edge correspondences between sampled model edge points and image edge points. The system can optionally fuse the 2D/3D correspondences from the image analy-sis with the inertial measurements from the IMU (Inertial Measurement Unit) to improve the pose estimation. Some tracking results of the algo-rithm are presented in Fig. 2. Currently achieved results exhibit a promising quality. There is still, however, room for improving the robustness, e.g. with respect to varying illumination condi-tions. In the future, the algorithms need to run in real-time.

Impact and effects

Innovative computer vision algorithms enable the development of new applications in various markets. For example, model-based tracking is

fundamental for 3D visual tracking and servoing tasks in robotics, and augmented reality. The new tracking algorithm determines the pose with high precision, which is very suitable for many applications. For instance, the AR market spans numerous sectors, from training on installation maintenance to technical design of spaces such as shops, industrial installations and booths, from e-commerce to tourism, and from market-ing to education. The interest of various sectors of the industry underlines the potential and relevance of this development.

Fig. 2: Tracking results of the target object.
The CAD model is blended into the real scene. (Copy-right: AIT)