Bixby Developer Center

When Bixby Prompts Users

Bixby's planner takes user-supplied input values and your capsule's models to construct a program that reaches a specific goal, such as displaying tomorrow's weather conditions, or presenting a list of local Italian restaurants. However, a user utterance might not include all the information that Bixby needs to reach its goal. In this case, Bixby might interrupt execution to ask the user for more information. Bixby then has an intermediary step with a recovery goal: it uses the user's response from the prompt to create the type of information needed to continue execution and reach the final goal.

If the user asks "how much are flights to New York City next Friday," the utterance includes neither an arrival nor a departure airport. Bixby can use location services to find the closest major airport to the user and assume that it is the departure airport. It could also use previous preference learning to know that, for example, the user lives in Fremont, California, and prefers to fly out of Oakland rather than San Jose. But there are two airports in New York City, La Guardia and John F. Kennedy. Unless it knows a previously-expressed preference from the user, Bixby will need to prompt the user to choose which airport.

Bixby prompts users when a cardinality violation has occurred: a required input has no value available to it, or an input that can take only a single value has multiple values available to it (such as the airport example above). In general, Bixby tries to prompt users as infrequently as possible. An input could have a default-init that yields a value, it could have selection rules or selection learning that yield a value, or it could have learned a user's preference for a value. The user is prompted for a value when Bixby cannot resolve a cardinality violation on its own.

Types of Prompts

There are three common types of prompts Bixby might give a user:

• A value prompt occurs when a required input is missing a value, and Bixby needs that value to continue execution. In the dice capsule from the Quick Start Guide, if the user asks "Roll dice" and the RollDice action doesn't provide a default for NumDice, Bixby will prompt for the number of dice to roll.

  

• A selection prompt occurs when a max cardinality One input has multiple possible values and the user must select the correct one. This can happen, for example, if the user searches for hotels by city name, and the search action returns multiple possible cities with that name.

  

• A confirmation prompt occurs before Bixby executes a particular action. Generally, it's used before executing actions that have side effects, such as completing a purchase or sending a message.

Confirmation prompts are explicitly generated by your capsule by the confirm key. Value prompts and selection prompts are generated by Bixby when cardinality violations for inputs occur and cannot be resolved without user input:

• When a min (Required) input has no values available, the user will be prompted for that value.
• When a max (One) input has multiple values available, the user will be prompted to select a single value from the available values.

In addition, the min-preferred and max-preferred keys can be used to affect when Bixby prompts for inputs:

• An input with min (Optional) cardinality and min-preference (Required) indicates that an input does not require a value, but having a value is preferred. This will cause Bixby to prompt the user the first time the input is used in an action and has no value. If no value is provided, the action continues without it.
• An input with max (Many) and max-preferred (One) indicates that an input allows multiple values, but prefers just one. If only one value is available to the input, the user will not be prompted. If there are two or more values, Bixby will prompt the user to select one or more items from the available values.

Capsule Lock and Modality

When a user is at a prompt, utterances will be locked to the type being prompted for, and the user must answer the prompt to continue. You can add training examples specifically for prompts.

When at a prompt, the user is locked into a conversation with the current capsule for a period of time before it times out. This is known as capsule lock, and it ensures that subsequent utterances are always sent to the current capsule for a short amount of time. In hands-on mode, the capsule lock timeout is 15 seconds. In hands-free mode, the user will be re-prompted after 7 seconds. If the user doesn't respond within those 7 seconds, capsule lock ends.

Users know they are locked into a capsule on mobile when they see the message "You're now talking to %capsule-name%" in the action zone of the screen. For example, users are locked into a conversation with the Meme Generator capsule below:

Capsule lock is triggered under the following conditions:

• If there is a followup in the result view
• If the user needs to answer a prompt in an input or confirmation view
• If the user is in a list navigation using hands-free mode

Capsule lock ends when users do any of the following:

• Exit the UI by tapping the X
• Close the Conversation View
• Navigate away from Bixby (for example, by pressing the home button)
• Do not interact with the capsule even after a second prompt (hands-free mode only)
• Say an utterance that specifies a capsule through named dispatch, even if users are at a prompt

During capsule lock, it is possible for Bixby to detect a user's response as being "off-topic," which will cause the utterance to be re-interpreted in a non-modal way. Bixby then tries to address the off-topic utterance in the context of the current capsule.

The flag-as-off-topic effect can also be used to force Bixby to re-interpret the utterance. For instance, if a weather capsule prompts the user to select between multiple options ("Do you mean Portland, Oregon or Portland, Maine") and the user instead says "what's the weather", Bixby re-interprets the utterance and classifies it as a weather intent that is locked within the capsule. After the capsule lock expires, utterances are treated as "brand new" and will no longer be limited to the weather capsule.

If your capsule is on the Marketplace and you haven't used the result-view-capsule-lock flag to opt out of capsule lock, conversation drivers in result views only display if the device itself also supports capsule lock.

You can opt out of capsule lock by overriding the result-view-capsule-lock flag in your runtime version and setting it to false. If you need to opt out of capsule lock only during specific result moments, you can use the capsule-lock key in that particular result view.

If you do opt out of capsule lock, conversation drivers will still appear on the result view page, but they might not behave as expected. This is true even if the device does not support capsule lock.

Customizing Prompt UX

Your capsules should take advantage of Bixby's personalization and learning abilities to minimize the number of prompts when possible. For example, when the user asks for the weather but doesn't specify a location, it's reasonable to assume they want the weather for their current area; a ride-sharing service capsule could remember that the user always asks for the cheapest available ride, and so might present that as its default choice. However, in some cases—especially involving actions that might lead to purchases, such as booking a ride or a hotel—your capsule should prompt the user for confirmation.

When Bixby does prompt users, though, you should ensure the prompts are clear, easily understood, and presented in the most logical order.

Customizing Dialog

Bixby can generate default dialog for selection and value prompts, but you can refine its dialog by creating elicitation and selection dialog events.

An Elicitation Event generates dialog for a value prompt. The "I need a number of dice to continue" prompt message comes from the default template for a missing input with single value cardinality:

I need #{concept(this, 'Indefinite')} to continue.

You could provide your own dialog event for it to override the default:

dialog (Elicitation) {
  match: NumDice
  template("How many dice do you want to roll?")
}

A Selection Event generates dialog for a selection prompt. The default template for a selection is to present a concept fragment for the input's concept (see the dialog event reference for details). This template can also be overridden with a custom dialog event:

dialog (Selection) {
  match : viv.email.EmailAddress (this)
  // Which email address?
  template("Which #{concept(this)}?")
}

This dialogue uses the concept Expression Language formatting function to format viv.email.EmailAddress as a concept fragment.

Setting Prompt Order

If Bixby needs to prompt for more than one piece of information during execution, the order is determined by the order of the input blocks within the action's collect block. For example, this is part of the RollDice action from the Dice example capsule:

action (RollDice) {
  collect{
    input (numDice) {
      type (NumDiceConcept)
      min (Required)
      max (One)
    }

    input (numSides) {
      type (NumSidesConcept)
      min (Required)
      max (One)
    }
  } 
  output (RollResultConcept)
  type (Calculation)
}

View on GitHub

An utterance with no values, such as "Roll dice," will cause Bixby to prompt first for the number of dice and then the number of sides for the dice. An utterance missing only one value, such as "Roll 3 dice," will only cause one prompt.

Modifying Prompt Behavior

By default, Bixby will not prompt the user unless a cardinality violation has occurred. There are some circumstances, however, in which you might might want to always prompt the user. For an example, the sample basic shopping cart capsule creates a shirt store. The shirt's Size input should always be confirmed, even when the size is mentioned in the utterance. You can change prompting for a specific input through the prompt-behavior key. To always confirm the shirt size, set prompt-behavior on the size input to AlwaysSelection:

action (GetSize) {
  type (Constructor)
  collect {
    input (size) {
      type(Size)
      min(Required)
      max(One)
      plan-behavior (Never)
      prompt-behavior (AlwaysSelection)
      //Lists all available options
      default-init {
        intent {
          goal: Size
          value-set: Size { Size(ExtraSmall) Size(Small) Size(Medium) Size(Large) Size(ExtraLarge) }
        }
      }
    }
  }
  output (PromptedSize) {
    evaluate { $expr(size) }
  }
}

View on GitHub

This setting ensures the user will always be prompted to either supply a missing value or confirm it. Typically, this is the only behavioral mode you will use in input blocks other than default, which will only prompt the user to supply values when there is a cardinality violation (there are no values supplied, or there are two or more values supplied for a max (One) input).

Confirmations

As described in Implementing Delight Nuggets, Bixby offers three kinds of confirmation prompts:

• Persistence Confirmation: Asking the user if Bixby should remember a value and use it as a default in the future
• Action Confirmation: Confirming the user wants to take a specific action
• Input Adjustment: Confirming Bixby's understanding of the user's input values and allowing refinement

As with selection and value prompts, you can override Bixby's defaults by supplying new templates for dialog events.

A Storage Event generates dialog for a persistence confirmation prompt, asking the user if they wish to store a value.

dialog (Storage) {
  match: viv.payment.CreditCardPaymentSource (this)
  template ("Would you like us to save this #{concept (this)} for future use?")
}

A Confirmation Event generates dialog for an action confirmation prompt, asking the user if they wish to proceed with an action.

dialog (Confirmation) {
  match: viv.paymentService.MakePayment
  template ("Ready to send?")
}

Actions can be set to explicitly require confirmation through the use of the confirm key which calls an appropriate confirmation-view that you create. You can use confirmation layouts to create more complex confirmations or to override the default layouts. If all you need to do is create a simple confirmation, you can choose to render nothing.

For the more complex interactions, such as presenting a restaurant booking screen that lets the user verify and, if necessary, change the reservation date, time, and party size, you can render a layout within the confirmation view.

Permission Prompts

Bixby can also prompt a user to grant a capsule permission. Users can respond both by tapping "Yes" or "No", or by speaking a response. Permission prompts are not customizable.