3D Tracking

Product: Autodesk Flame/Inferno 2007 Extension 1

Digital visual effects often involve computer generated elements inserted in live action shots. To be able to merge real and CG elements, the motion of the virtual camera has to match precisely the motion of the original one used for filming the shot. To create this virtual camera one has to use a "3d tracker" algorithm (also known as match moving) that analyzes the original clip and reconstructs the necessary camera animation.

First public demo of the tracker in Flame at NAB 2007, Las Vegas.

While most of the time 3D tracking in Flame is done by just "pushing a button", under the hood many - pretty complicated - algorithms are working together: features (interest points) are automatically detected on each frame, the motion of these points are tracked, wrong tracks are robustly filtered out and the camera parameters are estimated based on these point movements. The mathematical background of every step is very complex, so the development of the tool took a rather long time. (Well, compared to the visual effects projects I've been working on before) I started working on 3D tracking as I was finishing the university, and the tool was first released in Flame 2007, Extension 1.

Imagery courtesy of Absolute Post

Unfortunately I can't give any technical details about how our tracker works, but there any many innovative solutions to make the process fast and robust. The problem of 3D tracking, or "structure from motion" as called in the computer vision community, is still an actively researched area with many interesting and challenging problems. It was - and still is - a very nice experience working on it.