Uh oh! We couldn’t find any match.

Please try other search keywords.

Bixby Developer Center

Guides

Hands-Free List Navigation

To fulfill requests, Bixby will often ask a user to make a selection from a list of options. If the user asks to make dinner reservations at a good Italian restaurant, Bixby might present several restaurants to select from; to make the reservation, Bixby will also need the user to choose the party size and the reservation date and time. Capsules can get selections from users by presenting views with lists on the screen of their phone or other device. Users can interact with those views by tapping on their selections.

But interacting with views through touch isn't always desired, or even possible. A user might be talking with Bixby while driving, or might need to always interact with their device in a hands-free or eyes-free mode. And, Bixby might be running on a device that doesn't have a visual interface. So Bixby needs a way to let users navigate through and select items from lists entirely by voice. We call this Hands-Free List Navigation.

Navigation Modes

Both result views and input views can have a navigation mode for hands-free list navigation. The navigation mode chosen tells Bixby how to present options in a list. Depending on the mode, Bixby will listen for specific navigation commands.

Note

There is always a list navigation mode, even when one isn't explicitly declared. If you don't include a navigation-mode block, Bixby will use the read-many mode, with a page-size set to the total number of items.

read-one

The read-one mode renders each item in the list and reads them aloud one at a time, asking the user if they want that one (an item-selection-confirmation prompt). The user has several navigation commands available.

  • An affirmative response will select that item
  • A negative or next response will go onto the next item
  • A previous response will return to the last item
  • A repeat response will repeat the current item
  • A cancel response aborts the list navigation
  • An ordinal selection response ("first", "second", "third", etc.) can be used to select specific items in the list
  • A select-none response skips an optional input, leaving its value unset (this option is not available if the input is required)

read-one-and-next

The read-one-and-next mode renders each item in the list and reads them aloud one at a time, asking the user if they wish to go on to the next item after each one with a next-item-question prompt. This is similar to read-one, but changes the meaning of "Yes" and "No":

  • a negative or cancel response aborts the list navigation
  • An affirmative or next response will go onto the next item
  • A previous response will return to the last item
  • A repeat response will repeat the current item
  • An ordinal selection response ("first", "second", "third", etc.) can be used to select specific items in the list
  • A select-none response skips an optional input, leaving its value unset (this option is not available if the input is required)

This is useful if the user doesn't need to get more information (e.g., a details view) on items within the list, but only needs the summaries read, letting them go through results more quickly. This mode is only allowed in a result-view, not an input-view.

read-many

The read-many mode reads each item in the list aloud in pages of more than one at a time, asking the user if they want a selection from that page or would like to go on to the next page (a page-selection-confirmation prompt).

  • An affirmative response prompts the user to select one of the items in that page by ordinal value ("first", "second", etc.), and use that as the selection
  • A negative or next response goes to the next page
  • A previous response returns to the last page
  • A repeat response repeats the current page
  • A cancel response aborts the list navigation
  • An ordinal selection response ("first", "second", "third", etc.) can be used to select specific items within the page
  • A select-none response skips an optional input, leaving its value unset (this option is not available if the input is required)

The distinction between read-one and read-many is that read-one presents items to the user one item at a time, asking them after each item whether it's the one they want to choose from the list, while read-many presents items grouped in pages, asking them after each page whether they wish to select any of the items on that page.

Diagram

read-many-and-next

The read-many-and-next mode reads each item in the list aloud in pages of more than one at a time, asking the user if they want to go onto the next set after each page is read (a next-page-confirmation prompt). This is similar to read-many, but changes the meaning of "Yes" and "No":

  • A negative or cancel response aborts the conversation
  • An affirmative or next response goes to the next page
  • A previous response returns to the last page
  • A repeat response repeats the current page
  • A cancel response aborts the conversation
  • An ordinal selection response ("first", "second", "third", etc.) can be used to select specific items in the list
  • A select-none response skips an optional input, leaving its value unset (this option is not available if the input is required)

This is useful if the user doesn't need to get more information (e.g., a details view) on items within the list, but only needs the summaries read, letting them go through results more quickly. This mode is only allowed in a result-view, not an input-view.

read-none

The read-none mode does not read any items from its list of options aloud, instead just prompting the user to make a choice. This should be used when the possible choices are implicit or obvious: for instance, when the user must choose a day of the week. You would not want to have Bixby read the name of each day of the week aloud as part of the prompt.

In addition to the selections, the user has a smaller set of implicit options:

  • A cancel response aborts the conversation
  • A select-none response skips an optional input, leaving its value unset (this option is not available if the input is required)
Note

There is no pre-defined behavior for navigation commands if the user offers an answer that does not match a navigation command (for instance, saying "find an Italian restaurant" in response to "what is your party size"). It's up to the capsule to handle such situations gracefully. Read the information about Prompts and to handle off-topic responses.

Navigation Commands

The responses available to users in the navigation modes are defined in navigation-commands blocks of your navigation-support file. These blocks contain keys such as next, previous, and repeat, with response keys that define vocabulary for those responses. Here's a simple example:

navigation-support {
match: _

navigation-commands {
next {
response (next)
response (more)
}
previous {
response (previous)
response (back)
}
item-selection-confirmation {
affirmative {
response (yes)
}
negative {
response (no)
}
}
ordinal-selection-patterns {
pattern ("the (first)[v:viv.core.SelectionOrdinal:1]")
}
}
}
  • The navigation-support block starts with a match pattern. This defines the concepts that these responses will apply to. These are general responses which can apply to any concept, so we use the wildcard match, but we could provide more specific language for concepts if it was appropriate to the capsule.
  • Confirmation commands like item-selection-confirmation or next-page-confirmation have child blocks for affirmative and negative.
  • You can define synonyms like next and more by providing more than one response in a command block.
  • You can support custom ordinal selection by defining an ordinal-selection-patterns block. This one uses the predefined viv.core.SelectionOrdinal primitive concept.

In practice, you might not have to define any navigation commands at all. The commands from this example are taken from a generalized set that's included in viv.core; your capsule only needs to define navigation-support if you need to cover more specific responses for your capsule's models.

Adding Hands-Free List Navigation to Views

The navigation-mode block can be added to either input or result views:

The navigation-mode block contains a child block for the specified navigation mode (e.g., read-many). This block, in turn, defines statements and questions used by Bixby in this conversation moment.

Here's a simple input view, from a capsule that allows users to book space resorts:

input-view {
match: SpaceResort (result)
message ("Which space resort would you like?")
render {
if (size(result) > 1) {
selection-of (result) {
has-details (true)
select-button-text("Book")
where-each (item) {
layout-macro (space-resort-summary) {
param (spaceResort) {
expression(item)
}
}
}
}
}
}
}

To add hands-free list navigation to this input view, we might choose the read-one mode, since there won't be more than a few results available after a resort search.

input-view {
match: SpaceResort (result)
message ("Which space resort would you like?")
render {
if (size(result) > 1) {
selection-of (result) {
navigation-mode {
read-one {
list-summary ("I found #{size(result)} resorts.")
underflow-statement (This is the first resort.)
item-selection-question (Do you want to book this resort?)
overflow-statement (Those are all the resorts that meet your search.)
overflow-question (What would you like to do?)
}
}
has-details (true)
select-button-text ("Book")
where-each (item) {
layout-macro (space-resort-summary) {
param (spaceResort) {
expression (item)
}
}
}
}
}
}
}

The read-one block defines statements and questions Bixby uses to guide the user through voice navigation.

  • list-summary is an optional key that can be used to provide a summary statement; in this case, it's how many resorts Bixby found in a search.
  • The underflow-statement is what Bixby says if the user asks for the previous item in the list when they're on the first item.
  • The overflow-statement is what Bixby says if the user asks for the next item in the list when they're on the last one.
  • The overflow-question is a prompt read after the overflow-statement. (There is no corresponding question for underflow.)
  • The item-selection-question is a yes-or-no question asking whether this is the item the user wants to select.

The modes share keys, but some keys are specific to certain modes. The read-many and read-many-and-next modes have extra keys relating to paging. If we wanted to use read-many for the space resort selection example, we could use something like this:

...
navigation-mode {
read-many {
page-size (3)
list-summary ("I found #{size(result)} resorts.")
underflow-statement (This is the first page.)
page-selection-question (Do you want one of these resorts?)
item-selection-question (Which resort would you like?)
overflow-statement (Those are all the resorts that meet your search.)
overflow-question (What would you like to do?)
}
}
...
  • page-size controls how many items are shown per page.
  • page-selection-question asks about the page as a whole, not an individual resort.
  • item-selection-question is asked when the user responds affirmatively to the page-selection-question. It expects an ordinal response (e.g., "the second one"), not yes or no.

For more information, read the discussion of each individual navigation mode in the reference section.