Bixby Developer Center

Bixby uses natural language (NL) from the user as input. You can improve Bixby's ability to understand NL input by training Bixby to understand real-world examples of natural language in Bixby Developer Studio (Bixby Studio). For example, in the Quick Start Guide, you train the dice game to recognize "roll 2 6-sided dice". This phrase is an utterance. NL training is based on utterances that humans might type or say when interacting within Bixby. Utterances don't have to be grammatical and can include slang or colloquial language.

For any capsule you create, you'll need to add training examples in Bixby Studio for each language supported. If you are planning to support various devices, you need to add training for those devices too. Training examples can also specifically target supported Bixby devices (such as mobile) and specific regions that speak a supported language (for instance, en-US for the United States and en-GB for Great Britain). Bixby converts unstructured natural language into a structured intent, which Bixby uses to create a plan.

In this introduction, you'll learn how to plan and add training. You'll also learn how to search through training entries effectively, and use aligned natural language to test utterances. Finally, you'll learn how to add vocabulary that can assist in training.

For more information on effective training guidelines and best practices, see The Art and Science of Training.

Training

Bixby learns how to handle NL utterances by example. You provide training examples that consist of sample utterances annotated to connect words and phrases to your capsule's concepts and actions, aligning them to an intent. With good training examples, Bixby will do more than simply memorize the words and their meaning. It will automatically learn the most important aspects of those examples and apply that knowledge to new, unseen words and sentences to consistently apply the correct meaning to user requests. In short, natural language training teaches the platform about sentence meaning.

Training starts with one or more utterances that you want Bixby to understand. Follow these steps for each utterance:

1. Set a goal
2. Identify any values
3. Verify the plan

Let's look at this process in more detail.

Set a Goal

Goals are usually concepts. For example, if the user asks what the weather is, the goal would be viv.weather.Weather. In rare cases, you can use an action as a goal. This usually happens when training an action that can purchase a product or a service. For example, we might train to an action called viv.flightBooking.BookFlight, to purchase tickets for a flight. But most training examples set a concept as a goal. You can think of a concept-based goal as the subject or object of the utterance, while an action-based goal is more like a verb.

Identify Values

Many actions require inputs, such as names or numbers. You train Bixby to recognize that a particular word in an utterance identifies a typed value necessary to reach a goal. Here are some examples from one of Bixby's sample capsules for ordering shirts online:

In the utterance "Show Ralph Lauren shirts", the phrase "Ralph Lauren" is a value of type example.shirt.Brand.

In the utterance "Buy 2 Ralph Lauren shirts", "Ralph Lauren" is still a value of type example.shirt.Brand. But the word "2" is of type example.shirt.Quantity.

Utterances often include words, such as articles ("the" and "a"), that do not identify values. It's okay to leave those alone. Bixby's natural language engine automatically decides whether to ignore those words or use them to help identify variations on your trained utterances.

The values you tag in an utterance should be primitive concepts, not structure concepts. If you need a structure concept as an input for an action, you should create an action that takes the primitive concepts that are the properties of the structure concept as inputs, and outputs the structure concept. For example, a StreetAddress structure concept might have StreetNumber and Street properties:

structure (StreetAddress) {
  property (streetNumber) {
    type (StreetNumber)
    min (Required) max (One)
  }
  property (street) {
    type (Street)
    min (Required) max (One)
  }
}

To train Bixby to parse an address, tag the street number in a training example as streetNumber and the street name as street. Then, include a constructor action in your capsule that can combine those into a StreetAddress concept:

action (ConstructStreetAddress) {
 type (Constructor)
 collect {
   input (streetNumber) {
     type (StreetNumber)
     min (Optional) max (One)
   }
   input (street) {
     type (Street)
     min (Required) max (One)
   }
 }
 output (StreetAddress)
}

And a JavaScript function for the action:

export function constructStreetAddress(input) {
  const { streetNumber, street } = input
  return {
    streetNumber: streetNumber,
    street: street,
  }
}

You don't need to explicitly tell Bixby to use the constructor action; the planner uses the action in the execution graph when it needs to.

Verify the Plan

Once you have identified all the values in the utterance, verify that the plan looks like it should. Here's the plan for "buy 2 medium shirts":

This is a simple plan, and it should be easy to see how the inputs feed into actions that resolve the goal. Make sure the goal is correct, and the inputs have the right types. If you aren't sure, go ahead and run it. (You might need to compile the natural language model first; in the training tool, click the Compile NL Model button.) If anything in the plan looks odd, executing it often exposes the problem.

Creating Training Examples Using the Training Tool

To access the training tool, you must first create a training file. Create a new file (File > New File) and choose the Training file type. Choose the device, language, and region from drop-downs; Bixby Studio will place the file in the appropriate folder under resources.

Once you create the training file, you can access the training tool and related data in Bixby Studio in a target-specific subfolder of your capsule's resources folder. Here's an example from the Dice capsule from the Quick Start Guide:

To add new training examples to your capsule, type the utterance in the Adding New Training field at the top of the training tool and click Add. The training tool creates a new example under the Drafts tab:

Enter the goal concept or action in the "Goal" field. As you type, you will see a list of suggestions for completion:

After you enter the goal, annotate the values. Select the text of the word (or words) and enter the name of the matching model in the "Node" field. In this example, 3 should be annotated as a value for NumDice:

After you annotate the utterance with a goal and all necessary values, review the plan. (If it's hidden, click the downward-pointing arrow to open it.)

If the plan looks correct, click Save.

As you add more new utterances, Bixby will start to use your existing training entries to suggest training for the new utterances. You will see this under Training Suggestion. You can use this feature by reviewing the Training Suggestion's goal and values. If they are correct, click Accept to add these suggestions to the utterance.

Keep in mind that you won't always see suggestions, especially if you haven't provided many training examples yet. Over time, however, as Bixby learns about your domain, it provides more accurate suggestions.

The training tool also includes a search interface. Use this to find utterances that already have training. You can do this to inspect the training or make changes. For a simple search, select Show All, type a word into the search box, and click the search button. Here are some training examples from the Shoes sample capsule (which lets you search for and buy shoes) containing the word "dance":

Add Context with Roles

Sometimes Bixby needs a bit more context to discern between values that are similar. To add more context to certain values, you can use roles. For example, the following utterance includes two airport codes: "find flights from SFO to YUL leaving this December 22nd".

You can tell by the other words in the utterance that "SFO" is the departure airport and "YUL" is the arrival airport. Add a role to each value by clicking the label on each of the airport codes and clicking Add Role.

Now that you've added these roles to the training examples, the Bixby planner learns to distinguish that the second airport is using a type that is a role-of the parent type. You can learn more about role assignment.

Add Training Examples for Continuations

When a user asks a follow-up question or needs a refinement, this is a continuation. For example, when a user says something like "buy a medium shirt", they could follow it up with "change the size to large". This utterance only makes sense in the context of the previous utterance, so you must train utterances as continuations only when they don't make sense on their own.

In order to set this context for the continuation training example, first add the utterance, then enter its goal. This might be the same goal as the original utterance; in the case of the shirt capsule, it's a separate action: example.shirt.UpdateOrder. Choose Continuation of from the No specialization menu. Both the create order and update order actions have example.shirt.Order as their goals, which makes example.shirt.Order the specialization node.

Enter the same goal that you used in the original utterance, then annotate the rest of your training example as you normally would. In this example, we set the context of example.shirt.Size to be able to follow up after any query with that goal with "change the size to large":

Add Training Examples for Prompts

When there isn't enough information to reach a goal, Bixby prompts the user for further information.

For example, suppose (using the shirt ordering capsule) the user first says, "buy me a medium v-neck", then says "change the size". Bixby needs to know what to change the size to, and asks the user for this information.

As the developer, you can account for these prompts by adding examples of how the user would respond to a prompt.

To make a training example for a prompt, you need to add an example of a response to a prompt. Enter small in the Add Training field. This is a response to a prompt for example.shirt.Size, so enter example.shirt.Size as the goal. From the No specialization menu, choose At prompt for:

Enter example.shirt.Size as the Specialization Node. Now, you can annotate "small" as a example.shirt.Size to fill in the value.

Add Training Examples for Properties of a Goal

There are instances when a user utterance targets a property of a goal rather than the goal itself. This is property projection. For example, when a user asks "what's the total price of the order", the intent of the utterance is asking for the sum of the price of the shirts in the order, rather than asking to act on the order as a whole.

In cases like this, where the utterance asks for a specific part or "property" of the goal concept, you can add specific training entries to target properties like this.

To add a property to a target goal, specify the property with a hash (#) notation: in this case, example.shirt.Order#totalPrice. The structured concept Order has a property for the total (totalPrice). By making this the goal, you assert that the utterance's intent and goal is the total (totalPrice) in the current order (example.shirt.Order).

Add Training Examples on Full Utterances with Flags

When adding training, there are some utterances in which it is difficult to precisely annotate values.

Take, for example, the following utterance: "reserve a table for 3 people at Zona Rosa for dinner next Friday". We can annotate the number of people the reservation is for ("3"), the restaurant name ("Zona Rosa"), and the requested time ("dinner next Friday"). But, we need to add an extra criteria to this search query: we should filter out restaurants that are not currently accepting reservations.

In cases like this, when you cannot clarify intent by annotating specific words or adding additional vocabulary, you can use flags to give clues to Bixby. Flags offer a complementary way for you to inform training by adding training to the entire utterance rather than a specific word. With flags, you annotate the entire utterance and indicate the node and value that provides the "flag" to Bixby's machine learning.

To add flags:

1. After adding an utterance, select the gray spot located on the left side of the NL field. You will see tabs for Value and Route.

   

2. Within the Value tab, type a Node and Value. The Node should be the concept that serves as the signal sent for training, while the Value should be the value you want to assign to that concept.

3. Click Done.

   

For the utterance above, you would flag the utterance with the node ReservationIntent and a value of true.

While flags are supported with concepts of type enum, boolean, integer and decimal, enum and boolean will work best as they are enumerated types.

Add Training Examples with Pattern For

Patterns provide vocabulary for structured concepts. They are not real training examples, but are rather treated like vocabulary entries in a training-like format.

For example, consider DateTime training. The viv.time capsule uses patterns so that developers who import it only have to train one main type (viv.time.DateTimeExpression). The patterns themselves are all variations of the ways people could refer to some DateTime objects. Consider these training examples:

Example 1:

[g:viv.time.DateTime:pattern] (this)[v:viv.time.ExplicitOffsetFromNow:This] (friday)[v:viv.time.DayOfWeek:Friday] (around)[v:viv.time.DateTimePrecision:Approximately] (8)
[v:viv.time.Hour:8](pm)[v:viv.time.AmPm:Pm]

Example 2:

[g:viv.time.Date:pattern] (July)[v:viv.time.MonthName:July] (23rd)[v:viv.time.Day:23]

Here is the training entry in the editor for Example 2:

Note above that both goals (the left goal and the right Specialization Node... goal) are the same: viv.Time.Date.

All these different DateTime intent signatures should be interchangeable, for the most part, without having to train all of the possible combinations of DateTime inputs in your capsule.

Route Signals

A route signal tells the planner to go through a "waypoint" on the way to the goal, so it can factor in more information or prompt for more information.

For instance, take the utterance "Is it raining near me?" To train this, you could add a flag signaling that the weather condition is rain: set the node weather.WeatherCondition to the value rain. Then you would add a route of geo.CurrentLocation to let Bixby know to factor in the current user location.

Another example of using a route is a continuation that needs to prompt for more information. For example, in the restaurant reservation example, the user might give Bixby the utterance, "change the party size"; Bixby will need to prompt for the new size:

[g:ChangeReservation:continue:Reserve,r:UpdatePartySize] Change the party size

The PromptForPartySize route lets Bixby know that it needs to prompt the user.

Flags and route signals work with word annotation, but they do not replace it. You still need to annotate specific words in utterances as values. Additionally, when you can substitute specific words within a sentence (such as "cheap", "inexpensive", and "affordable"), normal training still offers the benefit of vocabulary.

Learned vs. Not Learned Training

When you add training examples in the training tool, they will usually be labeled as Not Learned. This label applies as long as Bixby hasn't generated NL models for those entries. The label can also mean that Bixby doesn't have enough information to actually learn from those training examples.

To generate new NL models when you add new examples or update old ones, click Compile NL Model. When the models are generated, Bixby changes the label of the learned utterances to Learned. If your training entry is similar enough to previous training examples, Bixby might label it as Learned even before you compile the model.

Changes in other training or vocabulary can cause an utterance to revert to a Not Learned state. If Bixby can't learn a specific example, consider adding similar training examples, reviewing existing training examples for conflicting utterances, or updating vocabulary.

Using Aligned NL

There are various situations where Bixby uses Aligned NL, a text format that allows you to directly map nodes such as values and goals to natural language utterances. When you annotate an utterance in the training tool, Bixby creates Aligned NL. Bixby also generates Aligned NL when processing a real query.

If you haven't compiled your training examples yet, Bixby's NL system doesn't know how to interpret it yet, but you can specify the Aligned NL directly in the Simulator:

Notice that in the simulator you'll see both the Aligned NL and the NL request. In the Debug Console, you can see the Aligned NL as well as the intent under the NL tab.

• NL:

    weather on saturday in san jose

• Aligned NL:

    [g:viv.twc.Weather] weather on (saturday)[v:viv.time.DateTimeExpression] in (san jose)[v:viv.geo.LocalityName]

• Intent:

  intent {
    goal {
      viv.twc.Weather
    }
    reference {
      viv.time.DateTimeExpression
    }
    value {
      viv.geo.LocalityName (san jose)
    }
  }

Here are a few examples of Aligned NL utterances:

[g:viv.twc.Weather] weather on (saturday)[v:viv.time.DateTimeExpression] in
(san jose)[v:viv.geo.LocalityName]

[g:flightBooking.TripOption] I need a flight from {[g:air.DepartureAirport]
(sjc)[v:air.AirportCodeIATA:SJC]} to (lax)[v:air.AirportCodeIATA:LAX]
(tonight)[v:time.DateTimeInterval]

[g:hotel.Hotel] find a (2)[v:hotel.ExactStarRating:2] star
(hotel)[v:hotel.HotelCategory:Hotel] in (san jose)[v:geo.LocalityName]

Note that words or groups of words follow the annotations expressing goals, routes, and values, which we collectively refer to as signals. You can group phrases in parentheses (like "san jose" above). The signals are in square brackets and include the signal type as a single-letter prefix:

• g for goal signal
• r for route signal
• v for value signal
• f for flagged signal

Training Limitations

Number of Training Entries

A capsule can have up to 2,000 training entries per target. A target is a specific combination of device, language, and region:

• en: All devices, English language, all regions
• bixby-mobile-en: Mobile devices, English language, all regions
• bixby-mobile-en-US: Mobile devices, English language, US region

The 2,000-entry limit is shared across parent targets. For example, the en target is a parent of all English-speaking device targets such as the bixby-mobile-en, bixby-mobile-en-US, and bixby-other-en-US; if en had 1,000 entries in it, then bixby-mobile-en would inherit those entries, and could have up to 1,000 more. If en has 1,000 entries in it and bixby-mobile-en had 500 more, then bixby-mobile-en-US would inherit from both of its parents, and could add up to 500 more.

No Named Entities

Bixby does not have platform-level support for recognizing named entities. That means it won't automatically be able to distinguish between people, places, movie titles, etc. It is up to you to distinguish these entities in a domain-specific way, through training examples and vocabulary.

Reserved Utterances

There are several utterances that have specific meanings in particular situations. For example, "Next" while navigating a list will read the next item or page of items. When possible, create complete training examples to help distinguish between these reserved meta-commands. For example, train "Cancel my reservation" instead of just "Cancel" for English because "Cancel" clears conversation context in Bixby.

For the full list of meta-commands, see the Reserved Utterances reference.

Automatic Speech Recognition and Text-to-Speech

Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) are useful Bixby capabilities for communicating with users. However, the natural language utterances you train and the vocabulary you add are only used in their specific capsule (although they might be used over time to help improve ASR and TTS performance).

Searching through Training Examples

Bixby offers a way to search through your trained utterances. This is especially useful when you have a large number of trained utterances for your capsule. Use search to get a filtered list of training entries based on your search query.

Any search phrases you enter are treated as required, and exact phrase matches rank higher in search results. So, if you search for "show me weather report san jose", all of those words must appear in the utterance.

Search is case-insensitive, so you can search for "Samsung" or "samsung", for example. You can exclude words from your search query using the hyphen (-) character.

The following table lists some other ways you can refine searches:

A synonym is simply a different name for a search operator. For example, you can either use node:viv.weather.WeatherCondition or match:weather.WeatherCondition.

You can combine the various methods to further customize searches. For example, to search for training with a goal of weather.Weather related to humidity tomorrow, you can use the following search:

goal:viv.weather.Weather "how humid" "tomorrow"

Use quotes for words that must appear exactly as you enter them.

Troubleshooting Training Examples

Bixby Studio informs you if your training examples have issues. However, if you have a large number of training entries, it can sometimes be difficult to quickly find a training entry causing an issue. This section discusses how to find any issues and what certain training entry statuses mean.

Finding Training Entry Issues

If you are submitting a capsule and one of your training entries is throwing an error, follow these steps to quickly find the error:

1. Open the submission page and select your failed entry.
2. On the log page, find the step where the error occurred. It will have an exclamation mark (!) next to it:
3. Click on the failure. A window pops up with more information on the error:
4. Copy the failure ID, as directed in the training window. In the above example, the ID is id:t-i20a4sycjvstauem5di5brg1.
5. Search for your training entry with the ID by pasting the ID into the
   This will bring you to your problematic plan.
6. Look at your plan and the reported training entry status that is shown in the upper left corner of the entry. For example, the above training entry has an illegal plan. The training has a ProductType value of "UberBLACK" that can't be used by the goal's action, so the planner in this case doesn't know what to do with the ProductType.
7. Fix the error accordingly and re-submit your capsule.

Training Entry Statuses

Bixby Studio reports the status of each training entry in the upper left corner. If you have any that are not listed as "Learned," then there are errors you might have to correct in your training entry or your models.

• Learned: The training entry is interpreted as expected when evaluated. Note that an entry cannot be considered Learned until the NL model is compiled. See Learned vs. Not Learned Training.

• Not Learned: Either the training entry has not been compiled, or running the training entry's text on the current interpreter results in a different interpretation than the one annotated. See Learned vs. Not Learned Training.

  Note

  Try to resolve all of the Not Learned issues before submitting your capsule. For more information, see General Recommendations and Best Practices for Training.

• Disabled: The training entry is not scanned for errors and not trained.

• Illegal Plan: The training entry has unsatisfied input constraints or unconsumed signals. This means that the intent of what you have trained does not match with the way the models have been designed to work.

• Unverified Plan: The training entry's current plan differs in a nontrivial way from the plan at the time this entry was saved. This can happen when model changes make the training entry's plan behave differently from the last time it was saved. You can resolve this by ensuring the plan for the entry is correct and recompiling the NL model, or by running Verify All Plans.

• Incomplete: These are generally semantic, such as the lack of a goal or referencing Node IDs that do not exist.

Verifying Plans

You can clear "Unverified Plan" statuses by selecting the Verify All Plans command in the Training Action menu (the drop-down under the "Compile NL Model" button). This command asserts that the plans for all complete, legal and enabled entries are correct. It will create a new plan from the Aligned NL for each entry, based on the current state of your capsule's models, vocabulary, and other data.

The training status of each entry might change after running this command. Disabled plans will not be verified.