DaVinci Goes Touchless With XBox Kinect

Dec 02, 2010 by in Microsoft Kinect

The launch of Xbox Kinect has caused much excitement in the open source community. In the last few weeks, developers have managed to tap into the hardware with impressive results. We’ve seen applications ranging from gesture-based experiences to 3D imaging.

We’ve taken this exciting opportunity to port our popular DaVinci experience to the Kinect platform. Gestures are used to create objects and control the physics of the environment. Your hands appear in the interface which allows you to literally grab objects out of thin air and move them in the environment. Additional gestures allow you to affect the gravity, magnetism and “planetary attraction”.

To date, many of the experiments in gestural interface development have not taken into account the hands. Unfortunately, the result is an experience that isn’t precise – users have no context of where they are interacting in the virtual space and 1-to-1 manipulation of objects in a scene proves difficult. By using an clenched hand to signify “grabbing” an object and an open hand to signify “releasing” an object, we are able to create experiences with an higher level of precision which can mimic a touch based experience. In fact, we’ve created a Kinect plugin to enable our entire suite of touch-based experiences to work with gestures – more videos to come!

Gesture-based interaction is great when touch isn’t practical. For instance, on a large screen projected display as shown in the video above it is difficult or physically impossible to control the entire area using touch. Using a technology like Kinect, we can create a virtual canvas in mid-air in front of the user. Interactions within this virtual canvas space are projected into the experience as shown in the DaVinci example.

To be honest, we had a blast playing with this experience. It definitely fulfilled all of our Star Wars fantasies of controlling objects with your mind. We’ll be adding more features in the coming weeks including the Darth Vader death grip. Stay tuned!

“Control, control, you must learn control.” – Yoda

Share


    User Comments

    • Al Today

      Link this up with Powerpoint and it could help kill the painful presentation where people read dot points off screens. Force them to interact with their presentation.

    • Peter Parnes Today

      Very cool!! Is this something your are planning to make available for general use?

      /Peter

    • eric norlin Today

      Hey guys – i\’m organizing a conference in the HCI space in Feb in Orlando (Blur). I\’d love to chat about getting this involved…can you drop me a line? (enorlin AT mac.com).

    • Per Today

      Supercool
      Please release this application on PC or Xbox

    • John Today

      Why only 2d? What you showed could have been done with a webcam + opencv, but the depth camera makes other options practical.
      For example, ‘orbit’ as an AR overlay tracking your hand.

    • John Today

      Why only 2d? What you showed could have been done with a webcam + opencv, but the depth camera makes other options practical.
      For example, \\\\\\\\\\\\\\\’orbit\\\\\\\\\\\\\\\’ as an AR overlay tracking your hand.

    • Chris Rhoads Today

      I have a request, please port this over and release your demo on the Xbox 360. I can see my 6 year old son, and even my 13 year old daughter, playing with this for hours at a time. As a parent who like to spark my kids\\\\\\\’ creativity I would gladly pay up to 800 MS points for a product like this.

    • Lakisha Pergerson Today

      Nice info, I’ve linked it from my website.

    • PG Wagenaar Today

      Great music! How can give me the specs of this?! Please respond asap!

    33 Trackbacks / Pingbacks for this entry

    Leave a comment *required


    four + = 10