1.6.6. Speech Synthesis

The speech synthesis component based on MaryTTS is part of the incremental speech processing toolkit InproTK. At present we have one speech synthesis configuration for each interaction island (/citec/csra/home/kitchen/assistance , /citec/csra/home/hallway/entrance) and an additional location for developer feedback (/citec/csra/control). The default tts language is German for the interaction islands and English for the developer feedback.

1.6.6.2. Interfaces

A say action can be invoked by sending task states to the corresponding scope or by sending a simple string. In the latter case, the component gives no feedback about the current state of the say action.

Scope (Listener) Type
<location>/saytask Task State
<location>/SAY String
<location>/ttsi/SAY TextToSpeechInstruction

The informer gives feedback about the current state of the say task.

Scope (Informer) Type
<location>/saytask Task State

1.6.6.3. Examples

1.6.6.3.1. Change TTS voice

Change the voice of the speech synthesis is possible via arguments on startup .

Command Arguments Default
-Dinpro.tts.voice=<voice> bits3-hsmm, bits1-hsmm, cmu-slt-hsmm bits1-hsmm
-Dinpro.tts.language=<language> de, en-US de

1.6.6.3.2. Trigger a say task (in the hallway)

Trigger a task with TextToSpeechInstruction:

 // RSB Informer
 Informer<TextToSpeechInstructionType.TextToSpeechInstruction> informer =
     Factory.getInstance().createInformer("/citec/csra/home/hallway/entrance/ttsi/SAY");
 informer.activate();

 //create a text to speech instruction
 TextToSpeechInstructionType.TextToSpeechInstruction.Builder ttsiBuilder =
     TextToSpeechInstructionType.TextToSpeechInstruction.newBuilder();
 ttsiBuilder.setText("Dieser Text wird gesprochen!");
 ttsiBuilder.setPlaybackOption(TextToSpeechInstructionType.TextToSpeechInstruction.
     PlaybackOption.PLAY);

 //send the text to speech instruction
 informer.send(ttsiBuilder.build());

Trigger a task with TaskState, deprecated:

 // RSB Informer
 Informer<TaskStateType.TaskState> informer = Factory.getInstance().
     createInformer("/citec/csra/home/hallway/entrance/saytask");
 informer.activate();

 // create a task state
 TaskStateType.TaskState.Builder taskBuilder = TaskStateType.TaskState.newBuilder();
 taskBuilder.setOrigin(TaskStateType.TaskState.Origin.SUBMITTER);
 taskBuilder.setState(TaskStateType.TaskState.State.INITIATED);
 taskBuilder.setSerial(1);
 taskBuilder.setPayload(ByteString.copyFromUtf8("Dieser Text wird synthetisiert."));
 taskBuilder.setWireSchema(ByteString.copyFromUtf8("utf-8-string"));

 // send the task state
 informer.send(taskBuilder.build());

1.6.6.3.3. Configure own speech recognition

  1. Check out the configuration project:
git clone -b minimal https://projects.cit-ec.uni-bielefeld.de/git/lsp-csra.inprotk-conf.git
  1. Change configurations, e.g., the input scope in the iu-config.xml
 <component name="ttsilistener" type="de.unibi.agai.inproapp.module.RsbTtsInstructionListener">
     <property name="scope" value="/citec/csra/home/hallway/entrance/ttsi/SAY"/>
     <property name="synthesisAdaptable" value="aiSS"/>
     <propertylist name="hypChangeListeners">
         <item>aiSS</item>
     </propertylist>
 </component>