How to get real time transcription and insights on Streaming API

In this how to, you will learn how to get started with Symbl’s most accurate conversation analysis API, native Streaming API.

const accessToken = await getAccessToken();
const symblEndpoint = 'wss://api.symbl.ai/v1/realtime/insights/' +
                        UNIQUE_MEETING_ID +
                        '?accessToken=' + SYMBL_ACCESS_TOKEN;

Create the WebSocket endpoint by tagging on a unique meeting identifier and a query param, accessToken to the url:

const uniqueMeetingId = btoa(EMAIL_ADDRESS)
const accessToken = await getAccessToken();
const symblEndpoint = 'wss://api.symbl.ai/v1/realtime/insights/' +
                        UNIQUE_MEETING_ID +
                        '?accessToken=' + accessToken;

Now that we have constructed the endpoint, let's create a new WebSocket!

const ws = new WebSocket(symblEndpoint);

Before we connect the WebSocket to the endpoint we first want to subscribe to its event listeners so we don’t miss any messages.

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
    console.log(event);
};

// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
   console.error(err);
};

// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};

Now we are ready to start the WebSocket connection.

const ws = new WebSocket(
ws = send(JSON.stringify(
  {
    type: 'start_reqest',
    meetingTitle: 'Websockets How-to', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
    },
    speaker: {
      userId: 'example@symbl.ai',
      name: 'Example Sample',
    }
  });

To pass the audio stream to the Streaming API, we first need to grant access to peripherals. We can do this with the Navigator API by accessing mediaDevices and calling getUserMedia.

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });

Now that we have granted access to the microphone we can actually use the WebSocket to handle the the WebSocket the data streamed to Symbl’s cloud so transcripts and insights can be analyzed in real-time. We can do this by creating a new AudioContext and using the microphones stream we retrieved from the Promise resolution above to create a new source, and processor.

const handleSuccess = (stream) => {
    const AudioContext = window.AudioContext;
    const context = new AudioContext();
    const source = context.createMediaStreamSource(stream);
    const processor = context.createScriptProcessor(1024, 1, 1);
    const gainNode = context.createGain();
    source.connect(gainNode);
    this.gainNode.connect(processor);
    processor.connect(context.destination);
    processor.onaudioprocess = (e: any) => {
      // convert to 16-bit payload
      const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
      const targetBuffer = new Int16Array(inputData.length);
      for (let index = inputData.length; index > 0; index--) {
          targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
      }
      // Send to websocket
      if (ws.readyState === WebSocket.OPEN) {
        ws.send(targetBuffer.buffer);
      }
    };
  };
  handleSuccess(stream);