Sending external audio streams

You can send a custom external audio source or audio stream using the AudioStream interface. The AudioStream represents a stream of audio that performs various operations depending on the Audio Stream’s configuration provided.

📘

Audio Sample Rates

Web SDK supports the following Audio codecs and sample rates:

  • Linear16 - For sample rates of 8000, 16000, 24000, 44100, and 48000 Hertz.
  • FLAC - For sample rates 16000 Hertz and above.
  • MULAW - At a sample rate of 8000 Hertz.
  • Opus - For sample rates of 8000, 16000, 24000, and 48000 Hertz.

If you don't specify a sample rate for AudioStream, the default is 48000 Hertz.

📘

Authentication

Your Symbl API Credentials, that is, your App ID and App Secret are required for authentication. Learn how to get them in the Authentication section.

Use the following code to send custom audio stream externally:

🚧

View the Importing section for the various ways to import the Web SDK.

/* ES6 Import */
// import { Symbl, LINEAR16AudioStream } from "@symblai/symbl-web-sdk";
/* ES5 Import */
// const { Symbl, LINEAR16AudioStream } = require("@symblai/symbl-web-sdk");
/* Browser Import */
// const { Symbl, LINEAR16AudioStream } = window;

(async () => {

  try {

      // We recommend to remove appId and appSecret authentication for production applications.
      // See authentication section for more details
      const symbl = new Symbl({
          appId: '<your App ID>',
          appSecret: '<your App Secret>',
          // accessToken: '<your Access Toknen>'
      });

      // Boilerplate code for creating a new AudioContext and MediaStreamAudioSourceNode
      const stream = await navigator.mediaDevices.getUserMedia({
        audio: true,
        video: false,
      });
      const sampleRate = stream.getAudioTracks()[0].getSettings().sampleRate;
      const context = new AudioContext({ sampleRate });
      const sourceNode = context.createMediaStreamSource(stream);

      // Creating a new AudioStream
      const audioStream = new LINEAR16AudioStream(sourceNode);
      
      // Open a Symbl Streaming API WebSocket Connection.
      const connection = await symbl.createAndStartNewConnection({
        insightTypes: ["question", "action_item", "follow_up"],
        config: {
          encoding: "LINEAR16",
          sampleRateHertz: sampleRate
        }
      }, audioStream);

      // Retrieve real-time transcription from the conversation
      connection.on("speech_recognition", (speechData) => {
        const { punctuated } = speechData;
        const name = speechData.user ? speechData.user.name : "User";
        console.log(`${name}: `, punctuated.transcript);
      });
      
      // This is just a helper method meant for testing purposes.
      // Waits 60 seconds before continuing to the next API call.
      await Symbl.wait(60000);
      
      // Stops processing audio, but keeps the WebSocket connection open.
      await connection.stopProcessing();
      
      // Closes the WebSocket connection.
      connection.disconnect();
  } catch(e) {
      // Handle errors here.
  }
})();

Using AudioContext

The AudioStream creates a new instance of AudioContext to create the audio processing. The AudioContext interface represents an audio-processing graph built from audio modules linked together, each represented by an AudioNode. The AudioContext initializes the AudioStream to use an externally maintained or initialized AudioContext in the browser.