Bixby Developer Center

The bixby.audioPlayer library capsule allows your capsule to play audio streams on mobile devices from within the Bixby client. Bixby currently supports the following audio file formats: AAC/MP4, MP3, and HLS, with Bitrates ranging from 16 kbps to 384 kbps.

For an example of using bixby.audioPlayer, read about the example.audio sample capsule.

This reference lists out several of the models available in this library capsule. Some sections contain tables listing information about those concepts or actions.

Concepts

Types are always in the bixby.* namespace, and in the bixby.audioPlayer.* namespace unless another namespace is specified. All properties have a cardinality of min (Optional), max (One) unless otherwise specified.

AudioInfo

The AudioInfo structure concept is a wrapper for the inputs necessary to play audio on Bixby. To play an audio file or stream, your capsule must create an AudioInfo concept that will be passed to the PlayAudio action.

AudioInfo is a higher-level structure that holds metadata about how Bixby should present the audio to the user, as well as one or more AudioItem structures, which contain the actual data for audio streams/files.

By default, Bixby will wait for text-to-speech (dialog) to complete before playing audio. If doNotWaitForTTS is set to true, Bixby will immediately play audio provided without waiting for TTS to complete. In this case, you should suppress TTS for the action by providing empty dialog. If you do not, Bixby will talk over the audio, causing the audio to stop.

The displayName is the display name of the capsule.

If more than one audioItem is supplied, you can supply a value for startAudioItemIndex to indicate which audioItem should be played first.

Category

The Category enum in AudioInfo tells the notification bar on mobile devices which types of media controls to show during playback.

Category only supports one setting on mobile devices, MUSIC. It should always be set to this value.

RepeatMode

The RepeatMode enum in AudioInfo lets you choose between repeating playback of only the current AudioItem, all items (from start to end and then back to the beginning, like a playlist), or not repeating (playing each AudioItem in order and then stopping).

AudioItem

The AudioItem structure concept defines a single audio stream that Bixby will play. At least one AudioItem should be included in the AudioInfo concept passed to PlayAudio.

The title will be shown as the title for the AudioPlayer system notification while the audio file is playing. If provided, the subtitle might also be shown, depending on the playback device.

Stream

The format should be a MIME type for the kind of audio stream being played, such as audio/mp4.

The offsetInMilliseconds indicates where in the stream the client should start playback when playing this stream.

The token field is not currently used.

Playing Audio

The heart of bixby.AudioPlayer is the PlayAudio action. It takes an AudioInfo concept as input and starts the audio player. While PlayAudio outputs a Result concept, it is not intended to be consumed.

To use PlayAudio, your capsule should implement its own play action that calls PlayAudio by creating a computed-input. This example, from the example.audio sample capsule, takes an empty AudioInfo object and populates it by using BuildMeowAudioInfo.

// To play audio, there is a two step process
// First create the playlist (done via meowToPlay below)
// Second, pass the playlist to the audioPlayer (done via meow below)

action (PlayMeow) {
  type (Search)

  collect {
    input (meowToPlay) {
      description (Create the playlist to play)
      type (audioPlayer.AudioInfo)
      min (Required) max (One)
      default-init {
        intent {
          goal: BuildMeowAudioInfo
        } 
      }
      hidden     
    }

    computed-input (meow) {
      description (By passing in the AudioInfo object to the PlayAudio action, we ask the client to play our sound.)
      type (audioPlayer.Result)
      compute {
        intent {
          goal: audioPlayer.PlayAudio
          value: $expr(meowToPlay)
        }
      }
      hidden
    }
  }
  output (Result)
}

View master on GitHub

The Result model output by PlayMeow is, like audioPlayer.Result, an empty model. This is not required, however; your capsule could return any value here.

text (Result) {
  description (For confirmation on whether audio was played successfully or not)
}

View master on GitHub

The local-endpoint JavaScript file for PlayMeow can also return any value without affecting functionality, as the actual work of playing sound is being performed by audioPlayer.PlayAudio.

module.exports.function = function playMeow () {
  return 'Meow';
}

View master on GitHub