Bixby Developer Center

This tutorial shows you more advanced features of the Bixby platform. You'll create a capsule which allows Bixby to answer questions about world countries, using a free remote service (REST Countries) that can look up and return information about countries using a JSON API.

In this tutorial, you'll learn how to do the following:

• Fetch data from API calls in your actions
• Add continuations, flags, and routes to your Natural Language training
• Use complex macros and Expression Language constructions

Download and Open the Capsule

The Bixby Developers GitHub account has a repository with sample code you should download before starting this tutorial.

https://github.com/bixbydevelopers/capsule-sample-country-info/

After you clone this repository, you'll have two folders:

• capsule-sample-country-info-initial: This contains the initial set of files for the tutorial. It's already a working capsule, but with limited functionality. During the tutorial, you'll explore the existing code and add more features.
• capsule-sample-country-info-full: This contains the finished set of files for the tutorial.

While you can open the capsules directly in Bixby Developer Studio, you should make a copy of the capsule folders and use the copies while following this tutorial. Alternatively, you can create a new capsule in Bixby Studio and copy the files manually.

Start Bixby Developer Studio. Click Open Capsule in the sidebar if no capsules are open, or use File > Open Capsule… in the menu. Select the capsule-sample-country-info-initial folder (or a copy of it) in the dialog and click Open.

First Run

To run the capsule, open the Simulator with View > Open Simulator or click the button in the activity bar. Ensure the playground.countryinfoInitial capsule is selected. Click Compile to compile the Natural Language model, then enter the following utterance in the input box and click Run NL.

start

The capsule will prompt you for a country name.

Type a country name like "Canada" into the input box, or in the field in the Simulator's dialog box. You can also use voice input by holding down the Microphone icon.

The capsule will present you with information about your selected country.

You can also try utterances that ask for a country directly:

tell me about switzerland

Exploring the Capsule

As with the Quick Start Guide, let's start exploring the capsule by looking at the models, the concepts and actions that define the data models and functions.

Concepts

The models/concepts/ folder contains four concept models: two structure concepts and two primitive concepts.

• Country.model.bxb: A structure whose properties represent the information the capsule knows about a given country. Most of these properties are defined using simple types from the Core library, but the capsule defines two primitives by name so they can be used independently from the Country model.
• MultipleOptions.model.bxb: Another structure concept whose properties are countryName, countryCode, and countryOfficialName. This concept is used in rendering views to handle the case where there are multiple possible matches to a user's request for information. (For instance, if the user says, "tell me information about Korea", the capsule needs to prompt the user to disambiguate between North Korea and South Korea.)
• CountryCode.model.bxb: A primitive concept that stores the three-letter code for a country.
• CountryName.model.bxb: Another primitive concept, for storing the name of a country.

Actions

The CountryCode and CountryName concepts are used as inputs to CountryAction, an action model which outputs a Country concept. Let's explore the action in more detail.

action (CountryAction) {
  description (Handle request for country info)
  type (Search)
  collect {
    input-group (countryInput) {
      requires (OneOf)
      collect {
        input (countryName) {
          type (CountryName)
          min (Optional) max (One)
        }
        input (countryCode) {
          type (CountryCode)
          min (Optional) max (One)
        }
      }
    }
  }
  output (Country) {
   throws {
     error(CountryNotFound) {
       on-catch {
         replan {
           dialog ("Sorry I cannot find a country named #{value(countryName)}.")
           intent {
             goal: CountryAction
           }
         }
       }
     }
     error (APIFailure) {
       on-catch {
         halt {
           dialog("Sorry I'm unable to answer questions right now")
         }
       }
     }
   }
  }
}

View main on GitHub

The action type is Search, and the collect block specifies the inputs. The input-group keyword is used to specify that the action requires OneOf either a countryName or countryCode as input.

The output block is more complex. While it only takes one line to specify the output model as Country, the throws block handles two possible errors that can be thrown by the action's implementation: a CountryNotFound error and an APIFailure error.

• A CountryNotFound error indicates the user gave input that can't be matched to a country. The replan keyword is used to display the error message with dialog and specify a new intent for the plan with a goal of CountryAction. This forces Bixby to re-execute the CountryAction action, waiting for an input of a new country.
• An APIFailure error indicates a problem communicating with the API server, which is a more serious error. The capsule uses halt to bring the capsule's execution to an end.

JavaScript

The CountryAction model not only specifies the interface (inputs and outputs), but it handles some basic validation (ensuring that either a countryName or countryCode has been provided by the user) and even performs some error handling. As with the Dice capsule in the Quick Start Guide, though, the real functionality of the action is implemented in JavaScript functions.

Recall that an action model is linked to its JavaScript implementation by specifying an endpoint declaration. If the server the capsule communicated with was designed to receive the countryName and countryCode properties as a JSON object and to return a JSON object whose keys match the properties of the Country model, you wouldn't need any real implementation code at all! The endpoint file could provide the API's URL in a remote-endpoint declaration.

More commonly, though, you'll be communicating with an API that doesn't exactly match your capsule's models, so you'll need to write JavaScript code within your capsule that can do the translation work: it takes the input concepts as parameters, calls the API in the form it expects, then takes the data the API returns and uses it to construct the output model.

Open the resources/base/all.endpoints.bxb file:

endpoints {
  action-endpoints {
    action-endpoint (CountryAction) {
      accepted-inputs (countryName,countryCode)
      local-endpoint (GetCountryInfo.js)
    }
  }
}

View main on GitHub

This declares the action endpoint for CountryAction to be the local file GetCountryInfo.js. You can find that file in the code/ folder. It defines a default function that takes the JSON object {countryName, countryCode} as its parameter, and returns a JSON object that can be mapped to a Country model.

Unlike the Dice capsule, though, the Country Info capsule doesn't keep all its code in one file. It begins with several JavaScript import statements.

import console from 'console'
import {OVERRIDES} from "./lib/Data"
import {getCountryInfo} from "./lib/APICalls"
import {checkMultipleOptions, getCurrencies, getLanguages, getBorders} from "./lib/DataUtilities"

View 1782bf4 on GitHub

The first line imports Node's standard console library for debugging purposes. The next three lines import functions in files from the code/lib/ folder. This is a common way to structure code in larger applications, and you can use it when developing your capsules.

The DataUtilities.js file contains several utility functions that take the JSON object returned from REST Countries' API and map its values into a format that's easier for our actions to work with. The getCurrencies, getLanguages, and getBorders functions respectively return an array of currencies, languages, or countries that share borders (which is returned as an array of arrays). The checkMultipleOptions function maps a list of countries returned by the API as a nested array into an array of objects with keys that match the model's property names; you'll see this used shortly.

The APICalls.js file contains the functions that communicate with the remote server. Let's take a closer look at how the getCountryInfo function works.

function getCountryInfo(countryName, countryCode, debug) {
  let response
  if (debug) {
    return COUNTRYAPI
  } else {
    let url
    if (countryCode && !countryName) {
      url = encodeURI("https://restcountries.com/v3.1/alpha/".concat(countryCode))
    } else {
      url = encodeURI("https://restcountries.com/v3.1/name/".concat(countryName))
    }

    response = http.getUrl(url, { format: 'json', returnHeaders: true })

    if (response && response.status == 404) {
      if (debug) console.log("country not found error")
      throw fail.checkedError('Country not found', 'CountryNotFound', {})
    }
    if (!response || response.status != 200) {
      console.log("error: response = " + response)
      throw fail.checkedError('Bad API call', 'APIFailure', {})
    }
  }
  //return response - specify parsed to get parsed value from return object (needed with returnHeaders = true)
  return response.parsed
}

View 1782bf4 on GitHub

The function's parameters are the inputs to CountryAction, along with debug, a boolean that can be set in GetCountryInfo.js (see line 5 in that file) to turn on debugging. Here, it makes the function return a known value for testing purposes.

In normal operation, the function sets url to one of the API endpoints at REST Countries, the free service the capsule uses, to look up countries either by code or by name depending on which value was provided to the action (lines 13–18). Then we make the call to the API using Bixby's http JavaScript module (line 20). The getUrl method makes a synchronous call to the API, specifying that it would like the response returned as a JSON object (format: 'json') and that it would like the HTTP headers and status code to be returned with the response (returnHeaders: true).

Once the response is returned, we process it (lines 22–32) based on the response status code:

• An HTTP 404 status means the country wasn't found, so it throws a CountryNotFound error. This is handled by the output block in the CountryAction model.
• Any other HTTP status except 200 (OK) indicates some other error in retrieving the response. This is handled by throwing the APIFailure error.
• An HTTP 200 status returns the parsed property of the response object. This contains the JSON object returned from REST Countries, an array of one or more country objects.

With an understanding of how the API call is made and what's returned, let's go back and look at how the getCountryInfo function is called in the capsule.

  let response = getCountryInfo(countryName, countryCode, debug)

  if (debug) console.log ("response = " + JSON.stringify(response))

  let currencies = [], languages = [], borders = []

  // Check if multiple countries matched
  let multipleOptions =  checkMultipleOptions(response)

  if (!multipleOptions.length > 0) {

    // Get all the currencies the country uses
    currencies = getCurrencies(response)

    // Get all the languages the country uses
    languages = getLanguages(response) 

    // Get the borders of the country
    borders = getBorders (response, debug) 
  }

  return {
    "commonName": response[0]['name']['common'],
    "officialName": response[0]['name']['official'],
    "countryCode": response[0]['cca3'],
    "currencyNames": currencies,
    "capital": response[0]['capital'][0],
    "languages": languages,
    "borders": borders,
    "googleMapsURL": response[0]['maps']['googleMaps'],
    "population": response[0]['population'],
    "continents": response[0]['continents'],
    "flagURL": response[0]['flags']['png'] + "",
    "multipleOptions": multipleOptions,
  }
}

View 1782bf4 on GitHub

Empty arrays are initialized for currencies, languages, and borders of each country, and filled by calling the corresponding functions in the DataUtilities file. At the end of the function, a JSON object whose keys correspond to the properties of a Country model is returned.

Views

The capsule has two views in the resources/en/views folder:

• CountryNameInput.view.bxb is an input view that renders a simple form for entering a CountryName.

• CountryResult.view.bxb is a result view which renders either a layout with information about a single country or a list of possible matches.

Let's look at the render block in the result view in more detail.

  render {
    if (exists(country.multipleOptions)) {
      macro (MultipleOptions) {
        param (country) {
          expression(country)
        }
      }
    }
    else {
      layout {
        section {
          content {
            paragraph {
              value ("Name: #{value(country.commonName)}")
            }
            paragraph {
              value ("Official Name: #{value(country.officialName)}")
            }
            paragraph {
              value ("#{size(country.currencyNames)>1 ? 'Currencies':'Currency'}: #{value(country.currencyNames)}")
            }
            paragraph {
              value ("Capital: #{value(country.capital)}")
            }
            paragraph {
              value ("#{size(country.languages)>1 ? 'Languages':'Language'}: #{value(country.languages)}")
            }
            paragraph {
              value ("[Borders: #{value(country.borders)}]")
            }
            paragraph {
              value ("Population: #{integer(country.population)}")
            }
            paragraph {
              value ("Continents: #{value(country.continents)}")
            }
            image {
              url ("[#{value(country.flagURL)}]")
            }
            attribution-link {
              label(Show Google Map)
              url("[#{value(country.googleMapsURL)}]")
            }

          }
        }
      }
    }
  }

View 1782bf4 on GitHub

The render block contains an if-else conditional block that checks whether multipleOptions is set in the matched Country; if it isn't set, this means just one country has been returned. The layout block (lines 21–58) displays a series of paragraphs with information about the country.

If, however, there are multiple options, the if block executes this:

macro (MultipleOptions) {
  param (country) {
    expression (country)
  }
}

This is an example of invoking a macro. Macros let you reuse a section of layout or dialog in multiple places; in this case, a macro is being used to make the CountryResult view shorter and more readable.

Macros are defined in separate files that begin with a macro-def block, and their definitions can include named parameters that are passed values from the files that invoke them. In this case, the macro named MultipleOptions is being passed a parameter named country, whose value (expression) is the value of the country property in the view.

You can find the file MultipleOptions.macro.bxb in the resources/en/macro folder. When you look at that file, examine how the view is constructed.

• The list-of block loops over the countries in country.multipleOptions.
• A navigation-mode of read-many-and-next is set. This is used for hands-free list navigation.
• Bixby will read the names of countries four at a time (page-size (4)) and prompt the user to go on to the next page of results if there are more than four matching countries.
• The country information is presented in cell cards.

Training

Open the training tab. There are only a few training entries in the capsule. You only need a few examples that Bixby can apply more generally. Let's look at two of them.

Click on the entry for "tell me about Peru".

This shows us that "Peru" has been annotated with the value of CountryName, and that the goal for this training entry is CountryAction. If you check "Show Aligned NL", you can see those annotations in plain text.

[g:CountryAction] tell me about (Peru)[v:CountryName]

As a reminder, Bixby uses your models, action implementations, and training entries to plan an execution graph that starts with the user's utterance and ends at the desired goal. In this case, the generated program is simple: CountryAction has two inputs, one of which must be specified. The utterance includes a value for CountryName, which satisfies the input requirement.

Now let's look at the entry for "start".

This is an even simpler plan graph: no optional inputs are specified. To be able to reach the goal, Bixby must prompt for at least one of them. To see how to do this, find the entry for "South Korea" that's labeled "At prompt for CountryName". (There is another entry for South Korea that does not mention a prompt; that's not the one you want.)

This one has a few differences. The goal is CountryName, but it's been given a specialization of "At prompt for CountryName". In Aligned NL, this looks like this:

[g:CountryName:prompt] (South Korea)[v:CountryName]

This lets Bixby know that when it needs a CountryName to reach a goal (such as CountryAction), it can run this graph to prompt the user for that information.

Expanding the Capsule

The Country Info capsule already works quite well, but it could be made even better by using some more advanced features of Bixby's natural language system, particularly continuations and routes.

When you downloaded the code for this capsule, you downloaded both the initial version capsule and the expanded version. Open capsule-sample-country-info-full now.

Continuations allow users to follow up a previous utterance with a change. For instance, after saying "tell me about Mexico", the user might ask, "What is the capital?". Bixby needs information from the previous utterance to know that the user is asking about the capital of Mexico.

To see continuations in action, open the Simulator and select the playground.countryinfoFull capsule. Compile the Natural Language model, then run the following utterances in the input box.

What is the population of France

Bixby responds, "The population of France is 67,391,582." The Country Information capsule now can look up specific information about countries. Now try this utterance:

What is the capital

Bixby remembers you've asked it about France, and responds with "The capital of France is Paris". This is an example of continuations in action. Lastly, try this utterance:

What about Portugal

Bixby uses continuations to infer you're asking about Portugal's capital, too, and responds, "The capital of Portugal is Lisbon".

Let's step through the changes and see how they work.

Changes to Modeling

The biggest change to the capsule's underlying model is the addition of the information property to the Country model. If you open Country.model.bxb, you'll see it added at the end of the model definition:

  property (information) {
    type (Information)
    min (Optional) max (Many)
  }

View 1782bf4 on GitHub

The property's type is Information, a new model that defines an enum.

enum (Information) {
  description (Information you can request about a country)
  symbol (capital)
  symbol (currency)
  symbol (official)
  symbol (population)
  symbol (continent)
  symbol (language)
  symbol (map)
  symbol (flag)
}

View main on GitHub

The CountryCode, CountryName, and MultipleOptions models remain the same, but another new model has been added, ResetInformationFlag:

boolean (ResetInformationFlag) {
  description (Flag to reset Information)
  features {
    transient
  }
}

View main on GitHub

This defines a Boolean, a value that can only be true or false. ResetInformationFlag will be explicitly set to true by our capsule when an utterance is not a continuation, that is, when the user starts asking about a new country. However, the model is defined with a feature you may not have come across before: transient. This tells Bixby that it shouldn't store the value of this concept across execution context. In this case, Bixby won't keep the true value set after it finishes executing an utterance, effectively resetting it to false. This means that when we test the value of ResetInformationFlag, it can only be true because we explicitly set it to true on this specific utterance.

On the Action side of the capsule's models, we've made two changes to CountryAction to handle the information property and ResetInformationFlag. First, after the input-group, there are two input blocks:

    input (information) {
      type (Information)
      min (Optional) max (Many)
    }
    input (resetInformationFlag) {
      type (ResetInformationFlag)
      min (Optional) max (One)
    }
  }

View 1782bf4 on GitHub

Second, in the error block for CountryNotFound, we've added the value of the information property, if it exists, to the intent that's passed back to the planner so that part of the user's request isn't lost when CountryAction is re-executed.

We've also added a new action to manipulate ResetInformationFlag:

action (ResetInformationAction) {
  type(Constructor)
  description (Reset information for new query)
  collect {}
  output (ResetInformationFlag) 
}

View main on GitHub

Changes to JavaScript

There's very little JavaScript that needs to be added in GetCountryInfo.js. At the start of the function, we need to reset the information property if resetInformationFlag is true:

  if (resetInformationFlag) {
    information = null
  }

View 1782bf4 on GitHub

Second, the JSON structure returned at the function's end needs to include the information property.

  return {
    "commonName": response[0]['name']['common'],
    "officialName": response[0]['name']['official'],
    "countryCode": response[0]['cca3'],
    "currencyNames": currencies,
    "capital": response[0]['capital'][0],
    "languages": languages,
    "borders": borders,
    "googleMapsURL": response[0]['maps']['googleMaps'],
    "population": response[0]['population'],
    "continents": response[0]['continents'],
    "flagURL": response[0]['flags']['png'] + "",
    "multipleOptions": multipleOptions,
    "information": information
  }

View 1782bf4 on GitHub

There's also a JavaScript implementation for ResetInformationAction, which does nothing other than return true:

export default function () {  
  return true
}

View main on GitHub

We'll discuss how ResetInformation gets set and reset when we talk about changes to our training.

The all.endpoints.bxb file has also been changed to reflect new accepted-inputs for CountryAction and to define the new endpoint for ResetInformationAction:

endpoints {
  action-endpoints {
    action-endpoint (CountryAction) {
      accepted-inputs (countryName, information,countryCode,resetInformationFlag)
      local-endpoint (GetCountryInfo.js)
    }
    action-endpoint (ResetInformationAction) {
      accepted-inputs ()
      local-endpoint (ResetInformation.js)
    }
  }
}

View main on GitHub

Changes to Views

The changes to CountryResult.view.bxb are all around handling the information property. Instead of a single template for speech output giving information about the requested country, an if-else block is used to test whether the information property is set. If it is, a new template is used that gives the specific information requested followed by the rest of the data about the country. Otherwise, the original template is used.

      if (exists(country.information)) {
        // Tell user specific information they requested
        // The phrasing is broken up into two parts, the first part shows the first information requested e.g. "The capital of Egypt is Cairo"
        // the 2nd part is all the other information requested e.g " and the language used is Arabic" etc
        // the subtract EL removes the first part and passes on rest of the info requested to the macro
        template ("#{macro('FirstInfo', country, country.information[0])} [and #{list(macroEach('MoreInfo', subtract(country.information, country.information[0]),1,country), 'value')}]")
      } else {
        // Generic information about a country
        template () {
          speech ("#{value(country.commonName)} is located in #{value(country.continents)}. The capital is #{value(country.capital)} and the population is #{value(country.population)}. The official #{size(country.languages)>1 ? 'languages are':'language is'} #{value(country.languages)} and the #{size(country.currencyNames)>1 ? 'currencies are':'currency is'} #{value(country.currencyNames)}.")
        }
      }

View 1782bf4 on GitHub

How does the capsule handle all the possible information cases with just one one-line template? By using new macros! The FirstInfo macro has two parameters of type Country and Information. It uses a switch statement to return a different template based on the value of the Information parameter passed to it (currency, language, capital, and the rest of the enum values defined in the Information model). Since this is a long macro it isn't included here, but you should open it and examine how it's done yourself.

This macro is called from Expression Language:

#{macro('FirstInfo', country, country.information[0])}

The second macro, MoreInfo, is defined almost identically with the same parameters and switch statement, just with slightly different templates. However, it's called using a more complex Expression Language construct.

#{list(macroEach('MoreInfo', subtract(country.information, country.information[0]),1,country), 'value')}

Let's step through that.

• The list() function creates a conjunctive list (separate values separated by commas with an "and" at the end, like "apples, bananas, and oranges") by looping through values in a node and representing them as dialog fragments. A node is essentially a tree of values: a Country model whose values have been set can be passed to an EL function as a node. So can a property of a model, such as country.information. So can the output of another EL function that returns a node, which is what's happening here. The second parameter, value, tells list() to output value dialog fragments.
• The macroEach(macro, node, index, parameter) function is the node being passed to list. This takes a dialog macro and then loops through the values in a node, returning a new node with the same values after applying the macro to each one. Let's look at each parameter in turn.
  • 'MoreInfo' is the macro being invoked. Like FirstInfo, this uses a switch statement to return a template describing the specific information value being requested.
  • subtract(country.information, country.information[0]) takes the values in the information property and removes the first value (position 0). The resulting node is all the information about the requested country except the information that was given in the FirstInfo template. If the user asked, "What is the capital of Peru", this would be all the information except the capital.
  • 1 indicates that the second parameter of the MoreInfo macro, the information property, is the one whose value is being injected. That is, as macroEach() loops through the values in country.information, the second parameter of the MoreInfo macro is being set to each value as the macro is invoked. If MoreInfo only took one parameter, this wouldn't be necessary, but since it takes two, then macroEach() needs both the index and the unchanging parameter to pass to its macro.
  • country is the first parameter being passed to the MoreInfo macro on each loop. Every parameter except for the one being injected comes after the index value (1 in this case).

So, together, the EL functions above gather all the information about the currently selected country, remove the specific piece of information that's already been described with the FirstInfo template, and output them as a conjunctive list. It's a lot of functionality in a single expression!

The last change to the views is in MultipleOptions.macro.bxb, where there are just two changes to the cell-card for countries, in the on-click block. This defines the action for tapping or clicking on the card. The country.information value needs to be added here, and the CountryName value is also explicitly cleared. This tap is starting a new request (tapping the card for Peru has the same effect as the utterance "Tell me about Peru"), so it's important to ensure the country name isn't preserved from a previous request for continuations to work.

Changes to Training

Open the Training tab in the full Country Information capsule. You'll see many more entries, some of which are listed as continuations.

Before looking at one of those, though, let's examine two ways ResetInformationFlag is set to true when it needs to be. Open the "tell me about Peru" entry.

The plan graph starts the same, with the two optional entries that feed into the countryInput input group, but there are two more optional inputs now: ResetInformationFlag and Information. Notice that ResetInformationFlag is set to true.

If you look closely at the NL field for this entry, you'll see a blue bar to the left-hand side. If you hover over that, you'll see that this is an annotation, not of a word or phrase in the utterance but of the entire utterance. Bixby calls these kinds of annotations flags. Click the blue bar and then click the word "Value". The flag annotation window will pop up.

You can also use a route annotation on an utterance; this tells Bixby to incorporate another action into its execution. Open the "information about Italy" entry.

In this graph, you can see the ResetInformationAction being added in as a constructor. If you hover over the green bar to the left of the NL field, you'll see that this, too, is an annotation, in this case for a route. The ResetInformationAction action just sets ResetInformationFlag to true.

In this capsule's case, there's not much difference between using either a flag or a route, although the flag is a little simpler. More complicated cases, with more complicates secondary actions, might require routes.

Now, let's look at a continuation. Click on "what about South Korea".

This looks a lot like our other information request actions, but the specialization is set to "Continuation of CountryAction". And, since this isn't starting a new conversation, there's no flag. The Aligned NL for this utterance looks like this.

[g:CountryAction:continue] what about (south korea)[v:CountryName]

This continuation uses the new input value of "South Korea" for CountryName but keeps any other inputs, such as the value of Information, from the previous utterance.

At this point, you should understand enough about how training works to look at examples we haven't covered and understand them. Try looking at these specifically:

• "what is the capital of Japan": an utterance setting values for both CountryName and Information
• "what is the capital": a continuation that preserves the CountryName value from the previous utterance but uses a new input for Information
• "what is the capital, population, and language of Peru": an utterance setting multiple values for Information at once

Next Steps

By now, you should have a good grounding in how the platform works, and be ready to dive into more detail with the following documentation:

• The Developers' Guides describe Bixby's modeling and programming
• The Design Guides describe best practices for Bixby's user experience
• The Reference documentation goes into detailed descriptions about the Bixby language, libraries, API and more
• The Sample Capsules and Walkthroughs dive into specific features with working examples and detailed "how-to" explanations