This project with LEAP was a blast. It came about before the official SDK for the Kinect was released so we had to get pretty creative.
The graphics were all done using the XNA framework along with the Farseer physics engine for all of the bouncing objects and falling particles that users interacted with (Writing particle modifiers for the Mercury Particle Engine is just plain fun!).
For the Kinect, we used a special build of the Primesense drivers along with the .net wrappers for OpenNI and EmguCV for OpenCV. We were located in an area that got direct sunlight for several hours a day, which rendered the Kinect useless during that period. To get around the issue we used a high resolution Microsoft LifeCam with auto focus and impressive brightness control.
LEAP is a full-service digital agency that makes brands easy-to-find and hard-to-ignore. LEAP integrates brands not
just across different platforms, but into people's lives.
I'm a senior developer at LEAP.
For those times that the Kinect had trouble getting data, we had the camera pointed directly at the ground and used an image matching algorithm to determine if people were standing in front of quiz options. For the more interactive screen, we fell back to a passive mode with timed events rather than 1:1 interactions.
In the couple of months that the display was up we got over 30,000 interactions (people stopping to look not just walking by) and close to 1,000 people stopped for around 4 minutes to take an entire quiz! "Digital out-of-home", as it's called in the industry, is an incredibly effective way to engage users.
From the tech side, I'm not super thrilled with this video (not just because I only make a 4 second cameo either). Most of the video was done very early on, and not enough of the fun interactions made it into it.
Addtionally, there is a small moment where the application just started and the butterfly animation stuttered as the app was building it's cache that was caught on film (Oh No! Our naughty bits!) That said, the project was a blast and we're always looking for new Digital out-of-Home Projects!
XNA + Farseer + Kinect + LifeCam + OpenNI + OpenCV