«

»

Sep 01

Using Voodoo to Estimate Handheld Camera Motion

I’m currently investigating fully automatic methods to determine the motion of a handheld cameras in unconstrained environments. Basically, if action recognition is going to work well in unconstrained videos, we need video representations that distinguish between motions that are “signal” (moving actors, moving objects) and motions that are irrelevant noise due to camera motion (panning, zooming, shake, etc.).

Today, I checked out one out-of-the-box solution: an open-source tool called Voodoo. Voodoo seems originally designed with graphics and special effects industries in mind, but . I found it quite easy to install and get up and running. Unfortunately, I found its performance on my test suite of two hand-held camera sequences to be disastrous. Perhaps I’m asking more than it was designed for. Read on for more…

Experimental Setup

I used Voodoo pretty much straight off-the-shelf, running in “Free Move” estimation mode and using a KLT tracker with all the default settings.  I’ll note that I did try switching to a SIFT tracker and got even worse results, so I only report results with the “best” default settings using KLT.

I attempted estimation on 3 image sequences: one from the set of demo sequences that installs with Voodoo, and two sequences from the Olympic Sports dataset.

Results

Dredger (ships with Voodoo):  Easy clip that slowly walks camera around side of an excavator(?).

The recovered camera motion (right side) accurately shows a “slide to the right”, so this looks good.

 

Download Video: MP4


 
 

High Jump : More difficult clip, but still just sideways motion (with minimal zoom)

The estimated motion here is all over the place.  Looks like the camera path is not well regularized. The first half and the second half by themselves look okay, but Voodoo seems unable to piece these together.

 

Download Video: MP4


 
 

Vault: Tough clip with huge zoom action

Again, motion path has little in common with the ground truth.  All kinds of weird stuff is happening, like the camera moving forward when it should be zooming out.

 

Download Video: MP4


 
 

Conclusion

Voodoo completely fails on two example sequences.  I suspect it is just not designed to handle clips where the first and last frame show very little of the same scene… but this is a real bummer, since this is exactly the real world scenario I’m investigating.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>