Activities

No upcoming events

Nao Sport

Participants

• Philip Kane
• Paul Lorenz
• Benjamin Siemoneit
• Christian Wienhues

Supervisors

• Dipl.-Inform. Ingo Luetkebohle
• M.Sc. Daniel Venjakob
• Dr.-Ing. Sebastian Wrede

Motivation

• A lot of people today, especially those working with computers, suffer from a lack of physical activity. This can lead to severe illness or even death. Possible malady's resulting from this are : Diabetes, High Blood Pressure, Depression, Osteoporose and in general Obesity.
• People generally enjoy a workout, however often times they dont engage in one by themselves.
• In normal environments there is a lack of motivation to engage in a physical activity. An intelligent room could provide it's inhabitants with opportunities to engage in such activity's on a regular base.
• A robot would very well fit into the setting of an intelligent room and while he might also have other uses, a robot could potentially be turned into a fitness trainer / motivator.
• The Nao robot is a perfect candidate to become a fitness trainer. He is used for robot football and has a humanlike body. This means he is able to do at least some excercises with his training partners.

Application Szenario

A team of students is working at the Isy Lab on a project for an intelligent room. They have worked for 5 hours straight, because the next presentation draws near and there ist still alot of work to be done. One of the students (let's call him P.) complains that his back is aching from all the sitting and typing and that he absolutly needs a break. Another student (let's call him C.) tells him that it would be good if he would actualy stretch his muscle's or even do a little workout to help fight the ache. P. does not know much about fitness and says that he would probably just end up hurting himself if he tried. C. however points over to the Nao robot they've got in the lab and starts setting it up. After a little persuasion P. agrees to try the Nao robot's fitness trainer mode. C. starts Nao and also the fitness trainer mode on his pc. Once everything is set up, C. lets Nao do some exercises with P.. About 10 minutes later P. is actualy feeling better from the workout and ready to get back to work.

Objectives

The project goals are :

• Proper tracking and classification of 3 basic excercises.
• Tracking and classification should also work with different persons and backgrounds.
• Execution of the 3 basic excercises on the Nao plattform.
• Give feedback to the user in order to motivate him.
• Being able to exchange Nao's speech, the excercises and the weight vector for classification quickly.

Description

Design Overview

A Sequence Diagram for Nao's mimicry behaviour is available here: Sequence Diagram:

Vision

• Motion tracking through Motion Energy Images (MEI)
• Image segmentation through difference images
• Motion segmentation through MEIs
• short update time (duration of one exercise)
• threshold of white pixels
• Moment extraction and calculation of Hu-Moments
• object recognition is invariant to rotation, scale and translation

Classification

• The open source ALGlib was used for Classification.
• This libary provides an implementation of the Fisher LDA, wich can be used to generate a weight vector out of a dataset.
• The weight vector and calculated means for all used movement classes are used for a simple distance calculation, using a Nearest Neighbour Algorithm.

NaoQi Movement

• .xar files contain behaviour sequences. They are stored on NAO and can directly be executed.
• increases modularity
• 3 behaviour sequences: biceps, shoulder exercise and squats

NaoQi Speech Synthesis

• NaoQi framework provides a text to speech module
• multiple languages and voices
• simple method call to generate speech output from strings

Data Storage

• Some data needs to be stored externally:
• Excercise data like id, excercise name, movement time
• Speech strings
• Classification parameters
• Data is read only on startup
• We've chosen XML for data storage
• Easy to use
• Small overhead
• TinyXML parser was used
• Lightweight, yet sufficient for our needs
• Easy to use
• Easy to integrate

Our XML scheme looks as follows:

<root>
        <behaviour>
                <entry id="0">
                        <location>...</location>
                        <sleeptime>...</sleeptime>
                        <name>...</name>
                        <utilize>...</utilize>
                        <initial>...</initial>
                        <final>...</final>
                        <reference>...</reference>
                </entry>
                <entry id="1">
                        ...
                </entry>
                ...
        </behaviour>
        <speech>
                <entry id="0">...<entry>
                ...
        </speech>
        <classification>
                <entry id="0">...</entry>
                ...
        </classification>
</root>

Sourcecode for our XMLReader module: xmleader.cpp

Camera Data

• CamReader module uses NAO's camera to provide the system with images
• Some camera parameters we used:
• Resolution: We use 320x240 (possible resolutions are 160x120, 320x240, 640x480) as a tradeoff between image quality and framerate.
• Color space: We get YUV422 Images from Nao and do the conversion to RGB in our CamReader module. Otherwise conversion would be done on NAO which reduces the framerate.
• We disabled automatic image corrections to avoid unwanted "movement"

Sourcecode for our CamReader module: camreader.cpp

Results

While it is only a proof of concept, our project clearly shows that using robots for fitness is possible.

• The tracking and classification of movements work for different people, even with different lightning conditions.
• Nao has only limited capabilities to engage in physical exercises, yet our prototype offers 3 distinguished movements.
• Nao can already do a single exercise autonomysly, it wouldnt be much effort to let him make an entire workout on his own.

Demonstration video The video shows a complete Workout with Nao and two different persons. This url Final Event links to a video made at the final event. The passwort is 'isy'. Thanks to Ingo for making the Video.

Discussion and Conclusion

• The project was overall successful, even though there still remain some rough edges, we suceeded in putting together a working prototype. We wouldn't be suprised to see a specialised fitness robot on the commercial market in the near future.
• We used very simple methods for the recognition and classifications of movements. Suprisingly enough these methods proved to be reliable. Certainly due to the very limited amount of movements we allow in our exercises.
• Working with Nao was an interesting experience for our team and we certainly look forward to work with him again.

Outlook

There are several things that could be done now.

• Changing the code so that Nao is able to do a full fitness workout autonomously.
• Adding the functionality to switch his voice and speech to be able to add several fitness trainer 'personalitys'.
• A study could be done, with the aim to show wether our prototype is able to motivate peole to fitness or not.