Getting Started with the Kinect

We got a Kinect in the lab this week and so I restructured my schedule a little bit in order to get my hands on it as quickly as possible.

I've got my laptop set up with the LibFreenect open source library for interfacing with the Kinect device, and begun to extract some data:

These images are screen grabs from one of the LibFreenect sample projects and show the extracted RGB image from the Kinect and a depth image that has been colorized (red is close to the camera dark blue is far away, black indicates a lack of depth information).

From this output, I've determined a couple of important details about the Kinect. The user has to be about two feet away from the Kinect or else the device will not be able to accurately determine their depth. This occurs because the Kinect uses an infrared painter to cast rays into the environment which the camera picks up, and objects that are too close to the Kinect get hit with too many rays that they get washed out. There is also a depth "shadow" that gets cast such that when one object is in front of another, the device cannot determine the depth of the background object around the left border of the foreground object.

I don't think that there should be any significant future problem based on these artifacts because I only need to track the hands with precision (and they should be in front of everything), and get a general positioning of the head (I don't need facial features or any specific data, just a generalized head position) so even if the player's hands are obscuring their face I shouldn't have trouble gathering the information I need.

It seems like working with the LibFreenect library moving forward should be fairly straight forward. The library provides a series of c-style function calls to the Kinect device, and I think that wrapping a class around these should allow me to integrate the library into my application in a suitably abstract fashion.

Next week I'll be researching the hand/head recognition algorithms/software that I want to use, and getting something resembling the above demo working using DirectX for rendering.