Search
Clear search
Close search
Google apps
Main menu

Voice-controlled Video component

Voice Controlled Video gives viewers a hands-free way to engage with video using their voice.

Install the component

This component only works with Google Web Designer. Don't have Google Web Designer? Download it here. If you have a previous version installed, it will automatically be updated to the latest version.

  1. Download the Voice-controlled Video component. (You don't need to unzip the file).
  2. In Google Web Designer, click the Image of add new component button in Google Web Designer button at the bottom of the Components panel.
  3. Select the .zip file you downloaded and click "Open".

Learn more about installing and removing custom components in Google Web Designer

Use the component

After installation, drag the component to the stage to start using it in your creative. To customize, first select the component on the stage, then open the Properties panel and scroll down to Component properties.

Before you set up the component:

  1. Add a video to your creative. Get your video's ID from the ID field in the Properties panel.
  2. Paste the ID into the Video element field to assign the cue point to your video.
The Voice-controlled Video component works with videos in both the YouTube and Video components in Google Web Designer.

Using video with mobile browsers

Mobile browsers disable video autoplay to keep users from incurring data charges for unrequested content. The best practice for mobile browsers is to turn off autoplay and to turn on video controls instead. This ensures that your video will work the way you intend it to.

Component properties

Image of the component properties panel for the voice controlled video component in Google Web Designer
Property Description
Name Sets the name of the voice controlled video component.
Video Element The id of the video element to control, for example:
gwd-youtube_1
Play commands

Set a list of voice commands that, when spoken, will play the video. Use commas to separate each command in the list.

For example:
To listen for someone to say "play" or "start", set the value to:
play, start

Pause commands

Set a list of voice commands that, when spoken, will pause the video. Use commas to separate each command in the list.

For example:
To listen for someone to say "pause" or "wait", set the value to:
pause, wait

Stop commands

Set a list of voice commands that, when spoken, will stop the video. Use commas to separate each command in the list.

For example:
To listen for someone to say "stop" or "quit", set the value to:
stop, quit

Custom commands

Set an array of voice command strings that, when spoken, will trigger custom video actions. Requires manual command handlers using the VoiceResult event.

For more, see Create a custom command.

Confidence threshold

Set the percentage (from 0 to 100) the component needs to be confident of the voice command match in order to trigger a video action.

Default value: 60

Recognition language

Set which language to attempt to match voice commands to. The accepted format is the BCP-47 language code.

Default value: en_US

Events

Events are how you connect a physical gesture from your user to a recorded action in your ad.

Event Name Description Event properties
voiceResult

Event dispatched when a voice recognition transcript is available above the set confidence threshold.

For more, see Handle voice results.

  • type (string): Command type (play, pause, stop, custom, invalid)
  • command (string): Specific command string recognized
  • confidence (number): Confidence level of the voice recognition
voiceError

Event dispatched when an error occurred (no audio input, no user permission, audio below confidence threshold).

For more, see Handle voice errors.

  • type (string): error type (invalidVideoType, unsupported, audioThreshold, audioError, timeout)
  • message (string): error description

Actions and methods

The Voice Controlled Video component does not automatically start listening for voice commands. To start listening, you can use your own JavaScript code to call the startListening method or use Web Designer's Events panel to add an action to a Tap Area or Image Button.

To start listening for commands:

  1. Add a button to the stage.
  2. Use the Events panel to add an event to the button.
  3. In the event Action panel, select Voice Controlled Video, then select the Start listening action.
  4. Choose the Voice Controlled Video component as the Receiver and click Save.

Follow the same steps above to add a button to stop listening for voice commands.

Learn more about how to add events to buttons in the Google Web Designer Help Center.

Alternatively, you can write your own code to call the public methods on the component instance to start or stop listening for voice commands.

Public methods

Method name Description
startListening Start listening for voice commands
stopListening Stop listening for voice commands

Create a custom command

In addition to the play, pause, and stop commands, you can create your own custom voice actions. To get started, create a condition inside the voiceResult event handler. If the commandType is custom then write your own code to find the value and match the transcript to one of the custom commands set in the Custom commands property.

Example custom command handler

In this example, "rewind" and "sound" represent custom commands set in the component's Custom commands property.

For more, see Component properties.

gwd.onVoiceResult = function(event) {
  if (event.detail.commandType == "custom" ) {
    var transcript = event.detail.transcript.toLowerCase();
    var transcriptMatch = ["rewind", "sound"];
    
    // Try to match the overheard command with the custom commands list set.
    for (var i = 0; i < transcriptMatch.length; i++) {
      if (transcript.indexOf(transcriptMatch[0]) != -1) {
        // Add code to run when the "rewind" command is heard.
        break;
      } else if (transcript.indexOf(transcriptMatch[1]) != -1) {
        // Add code to run when the "sound" command is heard.
        break;
      } else {
        // Add code to run if neither command matches.
        break;
      }
    }
  }
};

Handle voice results

When a voice command is heard that is equal to or greater than the confidence threshold, a voice recognition result event is triggered. The play, pause, and stop events will automatically work on the video set using the video element ID property.

You can also write your own code to handle voice results.

Example voice result handler
gwd.onVoiceResult = function(event) {
  console.log('type: ' + event.detail.commandType);
  console.log('command/ transcript: ' + event.detail.transcript);
  console.log('confidence: ' + event.detail.confidence);
  // Add code to take custom actions based on a voice command.
};

Handle voice errors

If no commands are spoken, permission to listen isn't given, or spoken commands are below the set confidence threshold, a voiceError event is dispatched. When this occurs, show the user a message explaining that something went wrong, and ask them to try again.

Example voice error handler
gwd.onVoiceError = function(event) {
  console.log('type: ' + event.detail.errorType);
  console.log('message: ' + event.detail.message);
  // Add code to display an error message and ask the user to try again.
};

 

Important Notes

  • Requires Web Speech API support, currently Chrome 38 (desktop and Android) or higher, Safari 8 and iOS Safari 8.1 or higher.
  • The voice detection detects possible command words even if they're not set.
    For example:
    If you set the Play command to "no". When you say the word "not", the component will detect this as similar enough to "no" and trigger this as valid command.
  • There is a hierarchy of commands. If you and a friend both try to control the video at the same time, the highest ranked command will be obeyed.
    • Play commands (1st)
    • Pause commands (2nd)
    • Stop commands (3rd)
    • Custom commands (4th)
  • Speech Recognition is an HTML5 API in development and is only supported in modern browsers and devices:
    • Windows 7 Pro+, Chrome 42+
    • Mac OS X 10.9.5+, Chrome 42+
    • Nexus 7+, 4.4.4+, (Chrome)

Preview

Components can't be previewed inside the Google Web Designer interface. To see the component in action, preview your ad in your preferred browser.

Was this article helpful?
How can we improve it?