On-Device Testing

While Bixby Home Studio (BHS) includes a Try it out feature that lets you test your action flow on a device, it also enables you to do on-device testing of metadata on compatible mobile devices.

On-device testing lets you say user utterances to a Bixby-enabled device to test the metadata of your devices. For example, you can load the metadata to your mobile phone for your lightbulb, and then say "Hi Bixby, turn on my light" to check if your lightbulb turns on. Currently, you can only test utterances through mobile devices.

To perform on-device testing, you'll need to:

Preparing for On-Device Testing

To enable on-device testing, you must first register your device with SmartThings. If you haven't completed this step, create a Samsung Account first. This is the same account you'll use to log into BHS. For more information about registering your device with SmartThings, see the Start Developing guide on the SmartThings Developers website.

Then you need to register the metadata through BHS. You can create new metadata or update existing metadata. For more information about registering metadata through BHS, see the Publishing the Bixby Device Graph (BHS Metadata Submission) guide

Enabling On-Device Testing

In order to perform on-device testing, you need to enable the Developer Options button in the Bixby app's settings. This is a hidden feature. To enable this button, first open the Bixby app and tap the More menu at the top right.

Tap More menu in Bixby app

A menu with multiple options will appear. Tap the Settings option.

Tap Settings on More menu

The Bixby settings page will appear. At the bottom, tap About Bixby.

Tap About Bixby

At the top, under "Bixby," you will find the version number. Tap this 5-7 times.

Tap Bixby Version Number

If successfully activated, you will see a notification saying "Developer options activated."

Developer options activated

Note

Currently, on-device testing is limited to the mobile device you are using.

You can also import push notifications to BHS for on-device testing.

Getting Execution Results from Push Notifications

You can import push notifications to Bixby Home Studio (BHS) for on-device testing. With this feature, you can get notifications in BHS when testing Bixby voice commands on mobile devices. This enables you to view and load execution results in BHS for debugging, which includes additional information to help you narrow down any issues, such as the execution path taken during on-device testing, parameters, etc.

To debug execution errors, you can click Load test execution result to show the graph execution result in BHS, which includes information such as parameters, node values, and execution paths from the on-device test.

Load execution results for debugging failed execution

Click "Copy execution result to Log" to see more information about the error in the Log Tab.

See details about failed execution in Log Tab

Enabling Push Notifications

  1. In BHS, open the console by clicking on the console icon in the bottom left corner.
  2. Click on Notifications to open the Notifications Tab.
  3. Click the Enable button underneath the Notifications Disabled message in the center of the Notifications Tab. Enabling push notifications in BHS
  4. A pop-up appears on the left that asks permission to show notifications. Click ALLOW. Pop-up Asking for Permission to Enable Push Notifications

You will see a Disable Notifications and a Clear Notifications icon appear in the bottom right corner.

Disable Notfications Icon and Clear Notifications Icon Highlighted in Right Corner

Note

If using a Mac, use the Chrome or Firefox browser to enable and view push notifications.

Setting Up Push Notifications on Mobile Devices

  1. On your mobile device, log in to SmartThings with same Samsung Account you used for BHS when enabling push notifications.
  2. Enable Developer Options and Bixby Home Platform Testing

Sending Push Notifications

  1. On your mobile device, give Bixby a command, such as "Hi, Bixby. Turn on the light."
  2. Afterwards, you should receive a notification of the execution result in BHS.

Receive notifications of execution results

Managing Execution Results

You can load or copy execution results in the Notifications Tab.

  1. Hover over a notification with your cursor, which opens a menu to the right.
  2. Make a selection:
    • To load and view the execution graph, click "Load test execution result."
    • To view the execution result in the Log Tab, click "Copy execution result to Log". Load or copy execution results

Also, you can see a notifications indicator in the status bar.

See indicator of unread notifications in status bar

Testing Metadata

Now that you've successfully enabled on-device testing, you can test your metadata. After activating developer options, go back to the Bixby settings screen and tap the now-visible Developer options.

Tap developer options

On the Developer options screen, tap Bixby Home platform testing. The On-device testing option only applies to developers who are testing Bixby capsules.

Tap BHP Testing

The Bixby Home platform testing screen will appear. To enable testing, turn on "Test mode" with the toggle button.

Enable Test Mode

At this point, you will have the option to select metadata (MNID, State, VID, Version) on your mobile device.

Metadata Options

When you tap a metadata category, such as MNID or Version, a new screen appears with multiple radio button options. Select the desired option and then tap Select.

Select Metadata Options

Once you've made your selections, you can start testing the metadata. To start your on-device testing, you can return to the Bixby home screen and test utterances for your metadata.

Testing Virtual Devices

If you don't have an actual device to test on, you can use the SmartThings simulator to create a virtual device. For more information, see the FAQ: Creating a virtual Device SmartThings community page.


Video Tutorial: Simulated Devices


Note

To go back to using Bixby regularly, you need to disable Bixby Home platform testing.