This documentation applies to version 2 of the Training Editor, which opens when you select the top-level Training
entry in the sidebar. If you select an individual training file within the resources
folder, this will open the old Training Editor. The older editor is considered deprecated, and will be removed in a future release of Bixby Developer Studio.
If what you're looking at doesn't match the screenshots in this documentation, check to make sure you have the right training editor selected!
Bixby uses natural language (NL) from the user as input. You can improve Bixby's ability to understand NL input by training Bixby to understand real-world examples of natural language in Bixby Developer Studio (Bixby Studio). For example, in the Quick Start Guide, you train the dice game to recognize "roll 2 6-sided dice". This phrase is an utterance. NL training is based on utterances that humans might type or say when interacting within Bixby. Utterances don't have to be grammatical and can include slang or colloquial language.
For any capsule you create, you'll need to add training examples in Bixby Studio for each language supported. If you are planning to support various devices, you need to add training for those devices too. Training examples can also specifically target supported Bixby devices (such as mobile) and specific regions that speak a supported language (for instance, en-US
for the United States and en-GB
for Great Britain). Bixby converts unstructured natural language into a structured intent, which Bixby uses to create a plan.
In this introduction, you'll learn how to plan and add training. You'll also learn how to search through training entries effectively, and use aligned natural language to test utterances. Finally, you'll learn how to add vocabulary that can assist in training.
For more information on effective training guidelines and best practices, see Training Best Practices.
Training files are not meant to be edited outside of Bixby Studio, and doing so can introduce syntax errors. If Bixby Studio reports an error, try removing the problematic training entry and re-entering it through the training tool.
Video Tutorial: Training v2
To use the training editor, select Training in the Files view of Bixby Developer Studio's sidebar. The editor opens to the Training Set List.
This screen presents a list of Training Sets and Capsule Targets for your capsule. A training set consists of one or more training entries. A training entry is a specific utterance Bixby should understand, like "book a space resort", annotated with the information Bixby needs to be able to interpret that utterance.
A training entry in Bixby can have the following targets:
en
en-US
bixby-mobile-en
bixby-mobile-en-US
The list of training sets is derived from the targets
block in your capsule.bxb
file. For more information, read the Setting Up Your Capsule topic.
A target has a "training budget" of 2,000 training entries. This is computed as the sum of all entries that apply to that target. The training screen lists the budget available for each target as well as the parent targets that are part of that budget. For instance, if you had both bixby-mobile-en-US
and bixby-mobile-en-GB
targets, the en
language set would be shown as applying to both of them. Training entries for patterns do not count towards this budget.
In addition, this screen shows status counts for each target, and whether your capsule's natural language (NL) model needs to be recompiled. You can recompile targets individually by clicking the "Compile" button that appears when you hover over a capsule target (this button will be dimmed if compiling that target is unnecessary), and compile all targets by clicking "Compile All Targets".
"Training entries" can also be called "training examples," and you may see either name in Bixby's documentation.
Clicking on a target will open up the list of training entries for that target.
For each training entry, this page shows:
A target can have dozens, hundreds, or even thousands of training entries. You can search and filter the list in several ways:
en
entries in the bixby-mobile-en-US
list)From the Training Entry List, you can add a new training entry as well as recompile the natural language model.
After you've modeled your capsule's domain, defining the concepts, the actions, and the action implementations, the training is what pulls it all together. The user's utterance provides the natural language (NL) inputs to Bixby's program, and the goal associated with that utterance tells Bixby what the output of the program needs to be. A training example consists of a sample utterance annotated to connect values to your capsule's concepts and actions. Bixby uses the utterance and annotations to produce an intent.
With good training examples, Bixby will do more than simply memorize the words and their meaning. It will learn the most important aspects of those examples and apply that knowledge to new words and sentences. Natural language training teaches the platform about sentence meaning.
Add new training from the Training Entry List screen by clicking the Add button.
If the Training Entry Screen is set to show only entries for a specific language, locale, or device, the Add button will create an entry in that specific training set. If it the screen is set to All, clicking Add will show a dropdown to select which training set to create the entry in. For example, if you are in the bixby-mobile-en-US
target and showing all entries, you must select between en
, en-US
, bixby-mobile-en
, and bixby-mobile-en-US
. You should select the most widely applicable training set for the entry you wish to create. That is, if the utterance is in English and will not be either locale or device dependent, select en
.
For each training entry, you'll:
Let's look at each step in more detail.
Goals are usually concepts. If your capsule includes a Weather
concept that contains all the information for a weather report, then when the user asks "what's the weather", the goal is the Weather
concept.
Sometimes, though, goals might be actions. This usually happens when the user is purchasing a product or a service in a transaction. For example, a capsule that books airline flights would have training examples that use a BookFlight
action as a goal.
Concepts and actions used as goals in Natural Language training must always belong to the capsule that declares them. A concept or action declared in an imported library capsule cannot be used as a goal in the importing capsule. However, you can extend a concept in your importing capsule and use the child concept as a goal.
An utterance is a phrase or sentence, appropriate to the training example's language and locale, that directs Bixby toward the training example's goal. To reach a goal, Bixby often needs input values: the restaurant, date, and time of a dinner reservation; the location and day for a weather report. The user's utterances often contain some of the necessary values:
To help Bixby identify values in an utterance, you annotate the values, matching them to concepts in your capsule or in Bixby library capsules.
example.food.CuisineStyle
example.reservation.Size
viv.time.DateTimeExpression
viv.time.DateTimeExpression
example.spaceResorts.Planet
To annotate a value in an utterance, highlight the value in the utterance and click the v:value button to add a value signal. Then, fill in the appropriate information in the form that appears in Bixby Studio's right-hand sidebar.
You can also use keyboard shortcuts to annotate utterances:
Option/Alt+V
for valuesOption/Alt+R
for routesFor enumerated values, also set the Form field to a valid symbol that matches the annotated word or phrase; if the phrase is "extra large" and this matches a Size
of ExtraLarge
, the form should be ExtraLarge
.
Enumerated values must either match symbol names exactly, or be an exact match to vocabulary for that symbol.
The values you annotate in an utterance should match primitive concepts, not structure concepts. If you need a structure concept as an input for an action, create a Constructor
that takes the primitive concepts that match the properties of the structure concept as inputs, and outputs the structure concept. See the Constructor
documentation for an example.
You don't need to explicitly tell Bixby to use the constructor action; the planner uses the action in the execution graph when it needs to construct a concept.
If a goal is common to multiple targets, but has different parameter input values, do not try to handle the differentiation between targets using training or modeling. Instead, have your action implementation catch this as a dynamic error.
In addition to annotating utterances with values, you can also annotate them with routes. Read the Routes section for information on when you might use routes (and when you shouldn't).
What about sort signals? Earlier versions of Bixby supported annotating sorting utterances, like "sort by price" or "show me the nearest hotels", with sort signals. This has been deprecated. Instead, use sort-orderings
for including sorting functionality in concepts, or the sort
block for sorting output from actions.
Once you have identified all the values in the utterance, confirm that the plan looks like it should. Here's the plan for "Find hotels near Mars" from the Space Resorts capsule:
This is a simple plan, and you can see how the inputs feed into actions that resolve the goal. Make sure the goal is correct, and the inputs have the right types. If you aren't sure, go ahead and run it in the device simulator by clicking the Run on Simulator button. If anything in the plan looks odd, executing it often exposes the problem.
Some utterances require more information in your training example for Bixby to process them correctly. A user might say something that requires the context of the previous utterance, or Bixby might need to prompt the user for information needed to complete the initial request. These cases are handled by specializations.
After a user says something like "buy a medium shirt", they could follow it up with "change the size to large". When a user follows up their previous utterance with a related request or refinement, this is a continuation. A continuation is an utterance that only makes sense in the context of the previous utterance.
A continuation needs to specify two nodes:
To train "change the size to large" in the shirt store sample capsule:
CommitOrder
action.CommitOrder
action, so we specify CommitOrder
again in the "Continuation of" field.The program generated by the previous utterance doesn't have to reached its conclusion before the continuation is acted upon. In the Shirt capsule above, the user is presented with a confirmation view before the CommitOrder
action finishes executing, and that's the moment the user might change the order's details with a continuation utterance. In other capsules, the first utterance's execution might have fully completed before a continuation. For example, a weather capsule could support the utterance "give me the weather in London" followed by the continuation "what about Paris".
Continuations are a common use case for routes; in the example above, the Flags list for this training entry shows a route through the UpdateOrder
action, which modifies the existing saved order. Read about Routes for more information (and alternatives to using routes). In addition, "Large" has been given a Role of ChangedItem
; read about Adding Context with Roles for more information.
You should only train utterances as continuations when they don't make sense on their own. A continuation must have a direct, clear relation to a previous utterance.
For more examples, see the Continuation for Training sample capsule.
When there isn't enough information to reach a goal, Bixby prompts the user for further information. For example, suppose in the previous example, instead of "change the size to large" the user had only said "change the size". Bixby needs to know what to change the size to, and asks the user for this information. As the developer, you can account for these prompts by adding examples of how the user would respond to a prompt.
To make a training example for a prompt, add an example of a response to a prompt. Enter small
in the Add Training field. This is a response to a prompt for example.shirt.Size
, so enter Size
as the goal. From the No specialization drop-down menu, choose At
prompt for:
In this case, Size
is both the goal of this training example and the specialization node. The text small
in the utterance is annotated as the node Size
and the form Small
.
Patterns provide vocabulary for structured concepts. They are not real training examples, but are rather treated like vocabulary entries in a training-like format.
When you add training, you must ensure that all training entries that are patterns are learned. Otherwise, submission will fail!
Consider DateTime
training. The viv.time
DateTime library capsule uses patterns so that developers who import it only have to train one main type: viv.time.DateTimeExpression
. Inside viv.time
, training consists of patterns that map to the DateTimeExpression
concept. The patterns are all variations of ways people could refer to DateTime
objects. Patterns do not use machine learning, but are simpler templates whose pieces are matched explicitly.
Consider these training examples:
Example 1:
NL Utterance: "2:30 p.m. this Friday"
Aligned NL:
[g:DateTime:pattern] (2)[v:Hour]:(30)[v:Minute] (p.m.)[v:AmPm:Pm]
(this)[v:ExplicitOffsetFromNow:This] (friday)[v:DayOfWeek:Friday]
The pattern being matched here is:
[hour]:[minute] [AM or PM] this [day of week]
Example 2:
NL Utterance: "July 23rd"
Aligned NL:
[g:viv.time.Date:pattern] (July)[v:viv.time.MonthName:July] (23rd)[v:viv.time.Day:23]
This matches a simpler pattern:
[month] [day]
Here is the training entry in the editor for Example 2:
Note above that both goals (the left goal and the right Specialization Node... goal) are the same: viv.Time.Date
.
The pattern in Example 2 will not match an utterance of "the 23rd of July"; that will need to be matched by a different pattern. Multiple patterns can exist for the same goal.
Another feature of patterns that's distinct from normal training is that patterns can be reused in other training examples, and even in other patterns. The date patterns above could be included in a pattern for DateInterval
, allowing Bixby to understand ranges like "January 1st through January 7th".
All these different DateTime
intent signatures should be interchangeable, for the most part, without having to train all of the possible combinations of DateTime
inputs in your capsule.
Patterns are designed for matching language elements that are regular and structured; if an utterance exactly matches a pattern, Bixby can always understand it. Regular training entries allow for less regular speech patterns and variations to be understood through machine learning, where it's not practical to define every possible variation.
You might need to train a few varying lengths of phrases pertaining to the structure concept (in this example, DateTime
) to ensure that they get matched.
Remember that if your capsule needs to handle dates and times, you do not need to actually create patterns for them! Instead, use the DateTimeExpression
concept from the DateTime library.
You've already likely seen examples of Aligned NL, a plain text representation of utterances that include the annotations for goals, values, and routes. When you annotate an utterance in the training editor, Bixby creates aligned NL. Bixby also generates aligned NL when processing a real query.
You can use aligned NL in the Device Simulator to give Bixby queries even when you haven't compiled the current NL model. The training editor can switch between displaying and editing annotated NL and the aligned NL for utterances by toggling the Show Aligned NL checkbox:
Here are some other examples of aligned NL utterances:
[g:viv.twc.Weather] weather on (saturday)[v:viv.time.DateTimeExpression] in
(san jose)[v:viv.geo.LocalityName]
[g:flightBooking.TripOption] I need a flight from {[g:air.DepartureAirport]
(sjc)[v:air.AirportCodeIATA:SJC]} to (lax)[v:air.AirportCodeIATA:LAX]
(tonight)[v:time.DateTimeInterval]
Annotations are denoted in brackets. The first character indicates a signal, the kind of annotation, followed by a colon and the concept the annotation refers to. Words or phrases might follow the annotations, grouped in parentheses like "san jose" above. Subplans are enclosed in braces. There are four kinds of annotation types:
g
: goal signalv
: value signalr
: route signalf
: flag signalTo process a request, Aligned NL is translated by Bixby into an intent. You can see the intent for a request in the Debug Console. The intent for the "research space resorts on Mercury" request above is:
intent {
goal {
1.0.0-example.spaceResorts.SpaceResort#all
@context (Outer)
}
value {
1.0.0-example.spaceResorts.Planet (Mercury)
}
}
Sometimes, Bixby needs more context to fully understand values. For instance, the following utterance includes two airport codes: "find flights from SFO to YUL leaving this December 22nd". The codes are both the same AirportCode
concept, but one has a role of DepartureAirport
and one has a role of ArrivalAirport
.
To add roles to utterances, highlight the parts of the utterances that need roles and click the Role button. You can add the node for the role in the right-hand sidebar.
Now that you've added these roles to the training example, the Bixby planner can distinguish that the second airport is using a type that is a role-of
the parent type. You can learn more about role assignment.
Another place roles are used is when two or more values can be used together to specify an item. For instance, consider the utterance "Remove the XL shirt from my cart" in the shirt example capsule: "XL" is a Size
and "shirt" is a SearchTerm
; together, though, they have the role of RemovedItem
.
There are instances when a user utterance targets a property of a goal rather than the goal itself. This is called property projection. For example, when a user asks the space resorts capsule "Where is The Mercurial", the intent of the utterance is asking for the location of a specific space resort, not information about the space resort as a whole.
In cases like this, where the utterance asks for a specific part or "property" of the goal concept, you can add specific training entries to target properties like this.
To add a property to a target goal, specify the property with a hash (#
) notation: in this case, SpaceResort#planet
. The structured concept SpaceResort
has a property for the total (planet
). By making this the goal, you assert that the utterance's intent and goal is the planet (planet
) in the Space Resort model (SpaceResort
).
When adding training, there are some utterances in which it is difficult to precisely annotate values.
Take, for example, the following utterance: "reserve a table for 3 people for dinner next Friday". We can annotate the number of people the reservation is for ("3") and the requested time ("dinner next Friday"). But, we need to add an extra criteria to this search query: we should filter out restaurants that are not currently accepting reservations.
In cases like this, when you cannot clarify the intent by annotating specific words or adding additional vocabulary, you can use flags to give clues to Bixby. You can think of flags as a way to annotate the entire utterance rather than a specific word or phrase within it. Just like annotations, flags can provide values or routes.
In this example from the shirt capsule, "Remove the XL shirt from my cart" demonstrates several training concepts, including a route flag:
Size
SearchTerm
RemovedItem
UpdateOrder
, specifying an action to route through on the way to the goal of CommitOrder
To add a flag, click the Flag button, then select "Create Value Flag" or "Create Route Flag" from the dropdown. You will be prompted in the sidebar for the necessary information, such as the route's node, or the value's node and form.
While value flags are supported with concepts of type enum
, boolean
, integer
and decimal
, enum
and boolean
will work best as they are enumerated types.
If you are flagging training examples with a boolean, you must use all lower case. For example, do use v:ReservationIntent:true
and don't use something like v:ReservationIntent:True
.
A route tells the planner to go through a "waypoint" on the way to the goal, so it can factor in more information or prompt for more information.
For instance, take the utterance "Is it raining near me?" To train this, you could add a flag signaling that the weather condition is rain: set the node weather.WeatherCondition
to the value rain
. Then you would add a route of geo.CurrentLocation
to let Bixby know to factor in the current user location.
Another example of using a route is a continuation that needs to prompt for more information. For example, in the restaurant reservation example, the user might give Bixby the utterance, "change the party size"; Bixby will need to prompt for the new size:
[g:ChangeReservation:continue:Reserve,r:UpdatePartySize] Change the party size
The PromptForPartySize
route lets Bixby know that it needs to prompt the user.
Routes can be attached to individual values, but are more commonly associated with flags on a whole utterance. In either case, you still need to annotate specific words in utterances as values. Additionally, when you can substitute specific words within a sentence (such as "cheap", "inexpensive", and "affordable"), normal training still offers the benefit of vocabulary.
Routes can make your capsule's logic harder to follow, and in many cases you can design your capsule so they're unnecessary. The continuation example above could be implemented so that UpdatePartySize
is a valid goal, for instance, or party size
could itself be a value supplied to ChangeReservation
. Read Training Best Practices for more guidance.
You can select multiple training entries for certain batch operations:
The available batch actions when one or more entries are selected include:
The training editor offers several ways to search through your trained utterances.
weekend in december
it will match all entries with that exact phrase, but will not find "weekend in January".Samsung
, samsung
and SAMSUNG
are all equivalent.-
character: Show me the weather in san jose -bernadino
will match show me the weather in san jose
and show me the weather in san francisco
but not show me the weather in san bernadino
.Search Scope | Syntax | Synonyms | Example |
---|---|---|---|
Entry with training ID | id:<training-id> | training-id | id:t-687v0fxyy5580vipz1ab39gx |
Entries with exact NL | text:<nl-text> | txt , phrase | text:weather |
Entries with specific node | node:<nodeid> | match | node:viv.weather.WeatherCondition |
Entries with specific value | value:<nodeid> | v | value:viv.weather.WeatherCondition |
Entries with a specific flag | flag:<nodeid> | f | flag:viv.food.Diet |
Entries with a specific route | route:<nodeid> | rte , r | route:viv.geo.CurrentLocation |
Entries with a specific goal | goal:<nodeid> | g | goal:viv.weather.Weather |
Entries with a specific continuation | continuation:<nodeid> | continue , continuation-of | continuation:viv.weather.Weather |
Entries with a specific role | role:<nodeid> | - | role:viv.air.ArrivalAirport |
Entries with a specific prompt | prompt:<nodeid> | prompt-for | prompt:viv.air.DepartureDate |
Entries with a specific tag | tag:<text> | tagged | tag:needs-modeling |
Entries with at least one of a specific field type | has:<field-type> | is | has:role |
Entries with a specific learning status | status:<status> | status:learned |
Valid fields for the has:
scope are value
, route
, role
, continuation
, pattern
, prompt
, tag
, enabled
, disabled
, and flag
. This scope searches for entries that have any field of that type, rather than specific contents in that field.
Valid learning statuses are learned
, not-learned
, and not-compiled
.
You can combine search scopes to filter more narrowly. To search for training with a goal of weather.Weather
related to humidity tomorrow, you could use goal:weather.Weather humid tomorrow
.
Bixby Studio informs you if your training examples have issues. However, if you have a large number of training entries, it can sometimes be difficult to quickly find a training entry causing an issue. This section discusses how to find any issues and what certain training entry statuses mean.
Training entries usually exist in one of three states:
When you add training entries, they will usually start as Not Compiled. This status applies as long as Bixby hasn't generated NL models for those entries.
To generate new NL models when you add new examples or update old ones, click Compile NL Model. When the models are generated, Bixby changes the label of the learned utterances to Learned. If your training entry is similar enough to previous training examples, Bixby might label it as Learned even before you compile the model.
Changes in other training or vocabulary can cause an utterance to revert to a Not Learned or Not Compiled state. If Bixby can't learn a specific example, consider adding similar training examples, reviewing existing training entries for conflicting utterances, or updating vocabulary.
There are four other possible status values for training entries:
You can quickly find issues using the search sidebar on the Training Sets Screen. Check the Issues subheading for categories of warnings and errors.
If you are submitting a capsule and one of your training entries is throwing an error, you can search for that specific error using the id:
search scope.
Open the submission page and select your failed example.
Select "Capsule Interpreter Training".
On the log page, look for the entries causing the issue:
--------------------------------------------------------------------
(4) Entries in state NOT_LEARNED (ERROR: actively causing rejection)
--------------------------------------------------------------------
View all items in category (copy the search between the markers into the training tool)
>>>> BEGIN COPY <<<<
id:t-jcvrft523b1j3iwzti7wx2zq7,t-q1b5nsdjb2clwy8fkhre80ecg,t-wczwvczrezonkmp2djlaj9wcj,t-v7s9drvf7whsbfxup9n9rw2j8
>>>> END COPY <<<<
Copy the failure ID(s), as directed in the training window.
Search for your training example with the ID by pasting the ID into the search bar. This will find the entry blocking the submission.
How to fix your training example depends on how you've tagged the example and how your modeling is set up. There might be multiple ways to fix an issue. See Training Best Practices.
An illegal plan occurs when the graph that Bixby's planner has created for a specific training entry cannot be run. There are several possible reasons why a plan might be illegal:
plan-behavior (Always)
input.A "lossy" activity is one that has the potential of losing user-provided signals, such as a search action, fetch action, or transactions. You can often resolve the illegal plan error by adding a route, or making one of the actions a different non-lossy type such as Calculate
or Resolver
, though certain actions types should be used for specific situations. An illegal plan normally indicates a problem with your capsule's modeling; in general, you don't want to force Bixby to create long chains of actions in order to fulfill user requests.
You can clear "Unverified Plan" statuses by selecting the Verify All Plans command in the "more actions" menu (the dropdown menu accessed with the "•••" button at the right-hand side of the search bar). This command asserts that the plans for all complete, legal, and enabled entries are correct. It will create a new plan from the Aligned NL for each entry, based on the current state of your capsule's models, vocabulary, and other data.
The training status of each entry might change after running this command. Disabled plans will not be verified.
Number of Training Entries
A capsule can have up to 2,000 training entries per target. A target is a specific combination of device, language, and region:
en
: All devices, English language, all regionsbixby-mobile-en
: Mobile devices, English language, all regionsbixby-mobile-en-US
: Mobile devices, English language, US regionThe 2,000-entry limit is shared across parent targets. For example, the en
target is a parent of all English-speaking device targets such as the bixby-mobile-en
, bixby-mobile-en-US
, and bixby-other-en-US
; if en
had 1,000 entries in it, then bixby-mobile-en
would inherit those entries, and could have up to 1,000 more. If en
has 1,000 entries in it and bixby-mobile-en
had 500 more, then bixby-mobile-en-US
would inherit from both of its parents, and could add up to 500 more.
No Named Entities
Bixby does not have platform-level support for recognizing named entities. That means it won't automatically be able to distinguish between people, places, movie titles, etc. It is up to you to distinguish these entities in a domain-specific way, through training examples and vocabulary.
Reserved Utterances
There are several utterances that have specific meanings in particular situations. For example, "Next" while navigating a list will read the next item or page of items. When possible, create complete training examples to help distinguish between these reserved meta-commands. For example, train "Cancel my reservation" instead of just "Cancel" for English because "Cancel" clears conversation context in Bixby.
For the full list of meta-commands, see the Reserved Utterances topic reference.
Automatic Speech Recognition and Text-to-Speech
Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) are useful Bixby capabilities for communicating with users. However, the natural language utterances you train and the vocabulary you add are only used in their specific capsule (although they might be used over time to help improve ASR and TTS performance).