Presentation Video: External Link

Attraction

Navigation
Wall Projection

MotionMagix Interactive Floor Projection
Live Kinect Holography
Natal Xbox 360 Announcement
A virtual fitting room for shop
Interactive Air Hockey Floor Projection
Interactive floor piano via Kinect and Projections
Kinect Projector Dance
Navigation for Mobile Robot

The PZ9 floor of Uni konstanz is quite boring, longer path with no attraction rather than a TV and it stands at the corner too. Many people visit PZ9 floor for various reasons such as to meet professors, researchers, students, lab work and so on. In order to find a specific room, one has to move through the long corridor looking left and right with the hope of finding the room after a short distance.
However, this is quite boring and somewhat tiresome, especially when one has less knowledge of the location of the destination office.

A tree climbing approach was used for generating ideas for this project. We started with one main idea during the inspiration workshop and later created more ideas during the course of development. The main idea we started with was „Floor Navigation“ which we supplemented with the „Wall Projection“. The „Wall Projection“ idea involved two ideas: coffee counter - to count the number of coffee cups consumed and the step-to-wall display application - a slideshow about the news in the HCI department that would be triggered by a button stepped on at the entrance of the floor PZ9.

The idea for the floor navigation is for visitors to use a Kinect and a projector to interact with the floor and they shall get the projected direction to the selected offices respectively. Office selections shall be made with the foot of the visitors. In order to be guided to the selected office, a green arrow shall be displayed and projected from the entrance of the building to the office.
The arrow shall enable the visitors to easily navigate to their destinations without necessarily having to check each office number. This shall save time during office visits, especially for the first time visitors.

The idea is to have wall visualizations that can be interacted with and can be triggered by sensors, for instance: motion sensors, distance sensors, etc…

Getting the required resources (for example: hardware, software, etc…) and researching for information about the Kinect and projector visualizations.

Hardware:

• Item Microsoft Kinect
• 2/3 Projector (external monitor)
• Windows 7 (or higher) PC with i7 Processor

Software:

• .NET Framework 4.0
• Microsoft Kinect SDK v1.6
• UbiDisplays prototype toolkit

Programming language:

• Javascript
• Jquery

Overall idea

¹⁾

Kinect is a motion sensing input device, made by Microsoft. It was originally designed for Xbox 360 video game console. The primary sensor detected by IR camera. Laser beams are sprayed out by an IR projector, bounce back and are received by an IR camera, this way distances are detected. There have also multiple mice to be controlled by voice. Additionally, the Kinect has an RGB camera (a webcam, basically) with a maximum resolution of 640 x 480. People can interact or control it by computer. The kinect software development kit based applications to be developed are allowed by C++/CLI, C#, VB or .net.

²⁾

Projector displays a window that it received from a connected Computer. Here it will display a window to interact the floor.

Components connection
Connection among projector(s), kinect and a computer as bellow:

³⁾

Develop software for prototyping with the Kinect. A layout HTML page was designed to be used with the UbiDisplays software and the Kinect to allow users to interact with the page.
Due to the sensitivity of the Kinect, this goal could not be archived as earlier thought. Among the challenges faced with the use of the Kinect was:

• Daylight sensitivity: the aim was to use the system in the hallway which is obviously exposed to day light which would limit the use of the Kinect for this project, hence this caused a stiff obstacle.
• Kinect and Projector positioning: The Kinect range is about 3 meters, positioning it in the hallway would require either aerial view, front view, back view and side view. The aerial view would put the Kinect out of range, the front and back view would have interference from visitors' shadows and the side view would not project on the targeted surface area (floor).

Due to the unforeseen shortcoming of the Kinect, a new idea (NavBuddy application) was developed to take on the „heavy“ navigation duty that could not be accomplished by the Floor Navigation application using Kinect. Also Wall Projection work commenced.

Wall Projection
There are several ideas for this topic.
One of the ideas for Wall Projection is for visitors to interact with Phidgets Motion sensor and the visualization on the wall shall react accordingly to the motion of the visitors.

The project idea about depicts the HCI department by slide show. The welcome message and HCI department activities slideshow will be projected in the wall. The system is developed by VVVV and arduino. When a person come to hallway of HCI department (PZ9) display will be activated by foot step on the floor where the hidden button will be covered by small carpet and it will start showing welcome message and the pictures of HCI department.

VVVV patch and Arduino

The Ardunio has to be connected and the code has to be uploaded for the patch to work correctly. One button is connected as analog input port A2.

The picture shows above the circuit setup.

The Coffee counter is used for Wall Projections. It is integrated with touch functionality using Kinect. The display shows the picture of the person and a coffee cup. When a person touches the coffee cup, it shows the number of coffee cups taken by the person. When he takes the coffee, he touches the picture of the coffee cup, so that he gets the count of the coffee taken.

The Initial Display:

The initial wall display is a blank screen.If a person moves,then the kinect detects the motion and the display is shown.

The person touches the coffee cup and the number of coffee taken is shown for the corresponding person.

If the count of the coffee taken is more than ten,then a thumbs down smiley is shown indicating you have taken too much coffee.

This application is used for the same purpose as the floor navigation, the only difference here is instead of using a projected arrow to the selected office, a robot navigation vehicle is used. The vehicle accompanies the visitors to the selected office by physically navigating its way to the office.

1. Hardware: NXT Robot Kit, an Android Tablet
2. Software: LeJos Framework, Eclipse, Android Developer Tools plugin for Eclipse, Android SDK Tools integration for Eclipse
3. Programming Language: Java, Android

Challenge:
Inaccurate navigation for long distance.

Solution:
Use colour sensor (Not practical)
Use compass sensor (Practical but not available)

Coffee Counter Source Code
NavBuddy NXT Source Code
NavBuddy Android Source Code

Coffee Counter Demonstration Video

Create Interactive Projected Displays Anywhere