.. _speech_visualizer: Speech Visualizer ================= | The Dialog-Visualizer is a collection of tools that contains at the moment a SpeechVisualizer and a GrammarHelper. | Their aim is to assist the interaction with the dialog. | .. contents:: .. _speech_hypotheses_visualizer: Speech Hypotheses Visualizer ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Used to show results from speech recognition. Related resources ----------------- Component repository:`speech-visualization `_. System startup: - The speech recognition can be found in ``lsp-csra-supplementary.sh`` on the ``dialog`` tab. - The component script is called ``component_speech-hyp-visualizer`` The program can be invoked with the following parameter: -s, --scope Set the rsb-scope where you want to grab speech-results from. -p, --speaking-scope The scope which tells if robot is speaking. -i, --situation The scope on which situation recognition results are published. -a, --attention The Aspect / State to listen for attention information, can be a regular expression. -e, --addressee The Aspect / State to listen for addressee information, can be a regular expression. -d, --debug debug-mode, shows more information. Related projects: - :ref:`dialog_manager` - :ref:`speech_recognition` - :ref:`speech_recognition_tool` Interfaces ----------------- Currently there are two interaction islands, the kitchen and entrance. =============================================================== ======================================== Scope (Listener) Type =============================================================== ======================================== ``/speechhypotheses`` `SpeechHypotheses`_ , `DialogAct`_ ``/situation/result`` `ClassificationResultMap`_ ``/saytask/SAY`` `TaskState`_ ``/flobi/speakingstate`` String ``/speechhypotheses/visualization/debugmode`` String =============================================================== ======================================== .. _DialogAct: http://docs.cor-lab.de//rst-manual/trunk/html/generated/sandbox/package-rst-dialog.html#message-dialogact .. _SpeechHypotheses: http://docs.cor-lab.de//rst-manual/trunk/html/generated/sandbox/package-rst-dialog.html#rst.dialog.SpeechHypotheses .. _TaskState: http://docs.cor-lab.de//rst-manual/trunk/html/generated/stable/package-rst-communicationpatterns.html#rst.communicationpatterns.TaskState .. _ClassificationResultMap: http://docs.cor-lab.de//rst-manual/trunk/html/generated/sandbox/package-rst-classification.html#rst.classification.ClassificationResultMap Debug Mode ----------- Debugmode can be enabled and disabled via cmdline-properties or an rsb-event. The scope of the debugmode-listener is always ``speech-hypotheses-scope + /visualization/debugmode``. The rsb-event to handle debugmode must contain a string-message ``ON`` or ``OFF``, not case-sensitive. Enabling debugmode entails: * a textFrame for grammarTrees shows up. * grammarTrees (if available) are shown. * type of incoming dialogact is printed to terminal. Examples -------------- - start some speech-recognition-software, that sends its sprechhypothese or dialogacts via rsb. - start visualizer with scope to listen to ``./speechvisualizer -s /result_scope/`` If you give no scope, default will be "/citec/csra/home/hallway/entrance/speechhypotheses". - see what recognizer sends as result. A small bar at the bottom of the screen appears. The results from the speech-recognition will be posted there. The current hypothesis is shown in the first line, below is the history of last hypotheses. .. figure:: /images/software/speech-hyp-visualizer/speech-visualizer-text-final.png :figclass: align-center If the result is not final yet, the text will be greyed out. If it is final, the text is black. Filler-words will only be shown as long as they are not final. .. figure:: /images/software/speech-hyp-visualizer/speech-visualizer-text-not-final.png :figclass: align-center You can also let the visualizer show the speaking and attention / listening state of the robot-system. Therefore you can invoke it with the command -p, which tells the visualizer to show if the robot speaks on a certain scope. There is a small section at the right end which becomes colored if the system is speaking. .. figure:: /images/software/speech-hyp-visualizer/speech-visualizer-system-speaking.png :figclass: align-center Also, you can invoke it with the parameters -a and -d to to show the attention and addressee state of the system. When the backgroundcolour of the textbox is blue, the system has attention to something. .. figure:: /images/software/speech-hyp-visualizer/speech-visualizer-has-attention.png :figclass: align-center When the backgroundcolour of the textbox is green, the system has attention and an addressee. .. figure:: /images/software/speech-hyp-visualizer/speech-visualizer-has-addressee.png :figclass: align-center If you enable debug-mode, you will see a second frame above the recognition-bar. It will show grammar-trees from incoming `SpeechHypotheses`_ or `DialogAct`_. .. figure:: /images/software/speech-hyp-visualizer/speech-visualizer-grammar-tree-frame.png :figclass: align-center .. _grammar_helper: Grammar Helper ^^^^^^^^^^^^^^ Used to show possible speech commands for an interaction with the dialogsystem. Related resources ----------------- Component repository:`speech-visualization `_. System startup: The start script grammar-helper.sh can be found at ``/opt/speech-visualizer/bin/grammar-helper.sh``. The program can be invoked with the following parameter: -r, --scp-scope Set the rsb-scope where you want to call the :ref:`speech_command_proposer`. -d, --debug debug-mode, shows more information. -l, --dialog-act Set the scope where dialogacts should be published. Related projects: - :ref:`speech_visualizer` - :ref:`speech_command_proposer` Interfaces ---------- The property of ``-r`` or ``--scp-scope`` serves as prefix. =============================================================== ======================================== Scope (Listener) Type =============================================================== ======================================== ``prefix/grammarhelper/debugmode`` String ``prefix/grammarhelper/rulechange`` String =============================================================== ======================================== The scope for rulechange is used to inform the grammarhelper that active rules have changed. It is used by the :ref:`speech_command_proposer`. Debug Mode ---------- Debugmode can be enabled and disabled via cmdline-properties or an rsb-event. The scope of the debugmode-listener is always ``scp-scope + /grammarhelper/debugmode``. The rsb-event to handle debugmode must contain a string-message ``ON`` or ``OFF``, not case-sensitive. Enabling debugmode entails: * the grammar-helper frame will be switched to debugmode. * grammar control section lets you set and ask for the jsgf-grammar the :ref:`speech_command_proposer` uses. * two ``getRules`` buttons to grab rule-lists from the :ref:`speech_command_proposer`. Examples and Screenshots ------------------------- | The appearance of the grammar-helper differs for the debug mode and no-debug mode. | Below the two frames are shown for comparison. Only one mode can be active at a time. | .. figure:: /images/software/grammarhelper/overview.png :figclass: align-center Basic usage *********** | Start the :ref:`speech_command_proposer` . | For example ``./speech-command-proposer -s /ScopeToMyServer/ -g /path/to/my/grammar.gram`` | Start grammar-helper with scope to make remote-calls ``./grammar-helper -r /ScopeToMyServer/`` | A frame at the right of the screen appears. | After the tool has started, it will try to grab a list of active rules from the :ref:`speech_command_proposer`. | It will show an example-sentence for every rule in a list in the white list-box. | The rule is updated autonomous whenever the list of active rules or the grammar is changed. | The two checkboxes at the top-right enable their entitled action on click when checked. | So you can retrieve new examples or send a dialogAct for the clicked entry. | Example for debugmode ********************* | In debugmode you have more control about the grammar and what rules are shown. | To set a grammar to make suggestions from, simply type in the file-path in the top text-field and press setGrammar. | The label beneath the button will tell if setting the grammar worked fine. | | If you want to check which grammar the :ref:`speech_command_proposer` is using, simply click checkGrammar. | In this case, the text-field beneath the button will be filled with the grammar path. | .. figure:: /images/software/grammarhelper/grammar.png :figclass: align-center | If your are content with the used grammar file, you can retrieve its rules by hitting the getAllRules or getActiveRules button. | It will show them up in a list in the white box below. | Active rules are autonomous updated whenever you change the grammarfile. | Now you can click on an entry in the list to get an example sentence for this rule (shown in the white box at the bottom). | Also there is a checkbox above the rule-list which will allow you to send dialogActs for the clicked list-entry. | .. figure:: /images/software/grammarhelper/debug.png :figclass: align-center .. note:: Active rules are handled by the :ref:`speech_command_proposer`! .. note:: The scope for sending dialogActs is passed through to the speech-command-proposer. The scp will also send the dialogActs.