Intelligent Systems Lab Project: GAFBro
Participants
- Kevin Fischer
- Felix Hülsmann
- Janik Jaskolski
- Maximilian Koch
Supervisors
- Thomas Hermann
- Stefan Rüther
Motivation
- Users often face the problem that they want to show their photos which they have taken with their mobile camera on their big TV screen.
- They also want to be able to navigate through their photos by sitting on their couch and don't have to bother with other devices as their mobile.
- In normal room environments the users have to figure out how they can actually connect their mobile phone with their TV.
- To cycle through the photos the users have to use a remote control or a mouse/keybord which can be annoying to handle
- An Intelligent room could help to solve this problem by offering a PC program which allows the users to show their photos easily on the TV.
- Also an Intelligent room would be able to recognize the users hand gestures and could react accordingly.
Application Szenario
Martin and Corinna arrange to meet in Martins intelligent living room after Corinna was three weeks in Japan and took a lot of photos with her mobile. She can't wait to show these to Martin.They both sit on the couch in front of the TV and Corinna has already open a picture on her phone. Now she lifts her mobile over her head and swings it down in a wide arc like she would throw an imaginary fishing line. Her mobile vibrates and moments later her selected photo appears on the TV screen.
To show Martin the other photos she lays her mobile on the couch table and wipes with her hand in an extravagant gesture to the left whereupon the next photo appears on the screen.
Objectives
The project goals are- Transmission of pictures from an Android smartphone to a multimedia device
- Display the corresponding data on the multimedia device
- Gesture controllable user interface
Description
GAFBro provides browsing mobile photos on a typical media computer. For the usage you need an Android Smartphone, a computer, a wireless network and a Kincet camera. The software consists of four different applications. One program analysis the gestures (left or right wipe, mobile throw gesture or mouse movement) made by the user and sends its output either to the mouse controller software or sends it to the main program.The main program shows the photos on the screen and offers a server to which the smartphones connect so that there is a constant message exchange.
The last component is an Android App which lets the user choose which picture they want to send to the computer.
In order to avoid false positive gestures, the picture will only be send when the Android App and the Kinect recognizes the mobile throw gesture.
It's usage is made as easy as possible:
- Start the software on the computer
- Start the mobile app
- Enter the computer's IP
- Select a picture on the smartphone
- Put it on the computer's display using a throwing-gesture
- Browse the media via left- and right wipes.
GAFBro also supports using multiple smartphones at the same time.
Results
The system worked as planned. People testing the software noted:- Annoying calibration before being able of using gestures
- Successful detection of intuitive gestures
- MouseController is hard (exhausting) to use but works
Discussion and Conclusion
People liked using GAFBro although there were some problems concerning joy of use (MouseController, Calibration Pose, ...).The concept proved to be sustainable. The next step will be the improvement of the gesture detection (and the use of additional gestures), an improved mouse controller and a feature which provides direct manipulation of loaded images.
Outlook
GAFBro will be continued with a focus on direct manipulation of pictures using gestures.Gesture interaction becomes more and more popular and GAFBro could provide a new kind of interaction with media like pictures.