Events and callbacks

Both the connection and audio stream objects have a connection.on method which can be used to subscribe to events and perform callbacks.

Connection events

Listeners can subscribe to the following events on the Connection object:

Example

connection.on("topic", (topicData) => {
  topicData.forEach((topic) => {
    console.log("Topic: " + topic.phrases);
  });
});
 
connection.on("disconnected", () => {
  console.log("User has been disconnected")
});

Event	Description	Callback Data
`connected`	When the WebSocket connection is successfully established.	None
`disconnected`	When the WebSocket connection is disconnected.	None
`started_listening`	Started listening to input device.	None
`stopped_listening`	Stopped listening to input device.	None
`processing_started`	Audio data processing successfully started.	Processing Started Response Object
`processing_stopped`	Audio data processing stopped.	None
`conversation_created`	Conversation is created and an ID is generated.	Conversation Created Response Object
`conversation_completed`	Conversation is ended.	Conversation Completed Response Object
`session_modified`	When the sample rate of the session is modified.	Returns an object containing the new sample rate.
`speech_recognition`	When data is being transferred between the client and server.	Speech Recognition Object
`message`	When the message object is detected.	Message Response Object
`topic`	When topics are detected.	Topic Response Object
`tracker`	When Trackers are detected.	Tracker Response Object
`action_item`	When Action Items are detected.	Action Item Response Object
`follow_up`	When follow-ups are detected.	Follow Up Response Object
`question`	When questions are detected.	Question Response Object
`subscription_started`	Subscription is started and a `converstionId` is received.	Subscription Started Response Object

AudioStream events

Listeners can subscribe to the following events on the Connection object:

Example

const audioStream = new OPUSAudioStream();
audioStream.on("audio_source_disconnected", () => {
  // Do something.
});

// OR 

connection.audioStream.on("audio_source_disconnected", () => {
  // Do something.
});

Event	Description	Callback Data
`audio_source_connected`	The audio source is connected to Symbl.	Returns the sample rate of the new audio source.
`audio_source_disconnected`	The audio source is connected.	None
`audio_source_changed`	A new default device is detected.	None

Global events

Listeners can subscribe to the following global Symbl Events using the window object.

Example

window.addEventListener("error", (error) => {
  const thrownError = error.detail;

  // Do something.
})

Event	Description	Callback Data
`error`	Errors recorded when there are potential anti-patterns or non-recommended coding.	The Error Object that was thrown. Will be found in the `callbackData.detail`.

Callback data reference

Conversation Created response object

To retrieve the conversation ID as soon as the conversation is established, use this callback.

connection.on("conversation_created", (conversationData) => {
  // Handle conversationData here.
});

JSON response example

{
  "type": "conversation_created",
  "data": {
    "conversationId": "5093011103940608"
  }
}

Conversation Completed response object

To retrieve the conversation ID and summaryUrl as soon as the conversation is completed, use this callback.

connection.on("conversation_completed", (conversationData) => {
  // Handle conversationData here.
});

JSON response example

{
  "type": "conversation_completed",
  "conversationId": "5093011103940608",
  "summaryUrl": "https://meetinginsights.symbl.ai/meeting/#/eyJzZXNzaW9uSWQiOiI2NTA0OTI1MTg4MDYzMjMyIiwidmlkZW9VcmwiOiJodHRwczovL3N0b3JhZ2UuZ29vZ2xlYXBpcy5jb20vcmFtbWVyLXRyYW5zY3JpcHRpb24tYnVja2V0LzE5MzE0MjMwMjMubXA0In0="
}

Processing Started response object

To retrieve the conversation ID as soon as the audio processing begins, use this callback.

connection.on("processing_started", (recognitionData) => {
  // Handle conversationData here.
});

JSON response example

{
  "type": "recognition_started",
  "data": {
    "conversationId": "5093011103940608"
  }
}

Speech Recognition response object

To retrieve the real-time transcription results as soon as they are detected. You can use this callback to render a live transcription which is specific to the speaker of this audio stream.

connection.on("speech_recognition", (speechData) => {
  // Handle speechData here.
});

JSON response example

{
  "type": "recognition_result",
  "isFinal": true,
  "payload": {
    "raw": {
      "alternatives": [{
        "words": [{
          "word": "Hello",
          "startTime": {
            "seconds": "3",
            "nanos": "800000000"
          },
          "endTime": {
            "seconds": "4",
            "nanos": "200000000"
          }
        }, {
          "word": "world.",
          "startTime": {
            "seconds": "4",
            "nanos": "200000000"
          },
          "endTime": {
            "seconds": "4",
            "nanos": "800000000"
          }
        }],
        "transcript": "Hello world.",
        "confidence": 0.9128385782241821
      }]
    }
  },
  "punctuated": {
    "transcript": "Hello world."
  },
  "user": {
    "userId": "emailAddress",
    "name": "John Doe",
    "id": "23681108-355b-4fc3-9d94-ed47dd39fa56"
  }
}

Message response object

This callback function contains the finalized transcription data for this speaker and if used with multiple streams with other speakers this callback would also provide their messages.

The finalized messages mean that the automatic speech recognition has finalized the state of this part of transcription and has declared it final. Therefore, this transcription will be more accurate than the Speech Recognition Object.

connection.on("message", (data) => {
  // Handle data here.
});

JSON response example

[{
  "from": {
    "id": "0a7a36b1-047d-4d8c-8958-910317ed9edc",
    "name": "John Doe",
    "userId": "emailAddress"
  },
  "payload": {
    "content": "Hello world.",
    "contentType": "text/plain"
  },
  "id": "59c224c2-54c5-4762-9582-961bf250b478",
  "channel": {
    "id": "realtime-api"
  },
  "metadata": {
    "disablePunctuation": true,
    "timezoneOffset": 480,
    "originalContent": "Hello world.",
    "words": "[{\"word\":\"Hello\",\"startTime\":\"2021-02-04T20:34:59.029Z\",\"endTime\":\"2021-02-04T20:34:59.429Z\"},{\"word\":\"world.\",\"startTime\":\"2021-02-04T20:34:59.429Z\",\"endTime\":\"2021-02-04T20:35:00.029Z\"}]",
    "originalMessageId": "59c224c2-54c5-4762-9582-961bf250b478"
  },
  "dismissed": false,
  "duration": {
    "startTime": "2021-02-04T20:34:59.029Z",
    "endTime": "2021-02-04T20:35:00.029Z"
  }
}]

Action Item response object

This callback provides you with any of the detected action items in real-time as they are detected. As with the Message response object this also returns every speaker's action items in case of multiple streams.

connection.on("action_item", (data) => {
  // Handle data here.
});

JSON response example

[{
  "id": "94020eb9-b688-4d56-945c-a7e5282258cc",
  "confidence": 0.9909798145016999,
  "messageReference": {
    "id": "94020eb9-b688-4d56-945c-a7e5282258cc"
  },
  "hints": [{
    "key": "informationScore",
    "value": "0.9782608695652174"
  }, {
    "key": "confidenceScore",
    "value": "0.9999962500210938"
  }, {
    "key": "comprehensionScore",
    "value": "0.9983848333358765"
  }],
  "type": "action_item",
  "assignee": {
    "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0",
    "name": "John Doe",
    "userId": "emailAddress"
  },
  "dueBy": {
    "value": "2021-02-05T00:00:00-07:00"
  },
  "tags": [{
    "type": "date",
    "text": "today",
    "beginOffset": 39,
    "value": {
      "value": {
        "datetime": "2021-02-05"
      }
    }
  }, {
    "type": "person",
    "text": "John Doe",
    "beginOffset": 8,
    "value": {
      "value": {
        "name": "John Doe",
        "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0",
        "assignee": true,
        "userId": "emailAddress"
      }
    }
  }],
  "dismissed": false,
  "payload": {
    "content": "Perhaps John Doe can submit the report today.",
    "contentType": "text/plain"
  },
  "from": {
    "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0",
    "name": "John Doe",
    "userId": "emailAddress"
  }
}]

Question response object

This callback provides you with any of the detected questions in real-time as they are detected. As with the Message Response Object this also returns every speaker's questions in case of multiple streams.

connection.on("question", (data) => {
  // Handle data here.
});

JSON response example

[
  {
    "id": "e0e44c21-c965-47b0-92d9-878ac22302ae",
    "confidence": 0.9834683553122807,
    "hints": [
      {
        "key": "confidenceScore",
        "value": "0.9957259328650095"
      },
      {
        "key": "comprehensionScore",
        "value": "0.971210777759552"
      }
    ],
    "type": "question",
    "assignee": {
      "id": "29c192e0-6fbc-4b94-9cb8-040783654003",
      "name": "Jane Doe",
      "userId": "[email protected]"
    },
    "tags": [],
    "dismissed": false,
    "payload": {
      "content": "How may I help you today?",
      "contentType": "text/plain"
    },
    "from": {
      "id": "29c192e0-6fbc-4b94-9cb8-040783654003",
      "name": "Jane Doe",
      "userId": "[email protected]"
    },
    "entities": null,
    "messageReference": {
      "id": "79a57ed7-d043-4a82-85fc-ae7844d8d2bb"
    }
  }
]

Follow-up response object

This callback provides you with any of the detected follow-ups in real-time as they are detected. As with the Message response object, this also returns every speaker's follow-ups in case of multiple streams.

connection.on("follow_up", (data) => {
  // Handle data here.
});

JSON response example

[
  {
    "id": "05bfb176-c2d3-42fd-a7e7-bbc80596a3e9",
    "confidence": 1,
    "hints": [
      {
        "key": "addressedTo",
        "value": "[\"first_person_singular\",\"second_person_singular\"]"
      },
      {
        "key": "informationScore",
        "value": "0.7361413043478261"
      },
      {
        "key": "confidenceScore",
        "value": "1.0"
      }
    ],
    "type": "follow_up",
    "assignee": {
      "id": "29c192e0-6fbc-4b94-9cb8-040783654002",
      "name": "Adam Voliva",
      "userId": "[email protected]"
    },
    "tags": [
      {
        "type": "person",
        "text": "Adam Voliva",
        "beginOffset": 0,
        "value": {
          "value": {
            "name": "Adam Voliva",
            "id": "29c192e0-6fbc-4b94-9cb8-040783654002",
            "assignee": true,
            "userId": "[email protected]"
          }
        }
      }
    ],
    "dismissed": false,
    "payload": {
      "content": "Adam Voliva can send it internet service technician to your home.",
      "contentType": "text/plain"
    },
    "from": {
      "id": "29c192e0-6fbc-4b94-9cb8-040783654002",
      "name": "Adam Voliva",
      "userId": "[email protected]"
    },
    "entities": null,
    "messageReference": {
      "id": "05bfb176-c2d3-42fd-a7e7-bbc80596a3e9"
    }
  }
]

Topic response object

This callback provides you with any of the detected topics in real-time as they are detected. As with the Message response object this also returns every topic in case of multiple streams.

connection.on("topic", (data) => {
  // Handle data here.
});

JSON response example

[{
  "id": "e69a5556-6729-11eb-ab14-2aee2deabb1b",
  "messageReferences": [{
    "id": "0df44422-0248-47e9-8814-e87f63404f2c",
    "relation": "text instance"
  }],
  "phrases": "auto insurance",
  "rootWords": [{
    "text": "auto"
  }],
  "score": 0.9,
  "type": "topic"
}]

Tracker response object

This callback provides you with any of the detected trackers in real-time as they are detected. As with the Message response object this also returns every tracker in case of multiple streams.

connection.on("tracker", (data) => {
  // Handle data here.
});

JSON response example

{
  "type": "tracker_response",
  "isFinal": true,
  "trackers": [
    {
      "name": "Goodness",
      "matches": [
        {
          "type": "vocabulary",
          "value": "This is awesome",
          "messageRefs": [
            {
              "id": "fa93aa64-0e8d-4697-bb52-e2916ca63192",
              "text": "This is awesome.",
              "offset": 0
            }
          ],
          "insightRefs": []
        },
        {
          "type": "vocabulary",
          "value": "Hello world",
          "messageRefs": [
            {
              "id": "8e720656-fed7-4b11-b359-3931c53bbcec",
              "text": "Hello world.",
              "offset": 0
            }
          ],
          "insightRefs": []
        }
      ]
    },
    {
      "name": "Goodness",
      "matches": [
        {
          "type": "vocabulary",
          "value": "I like it",
          "messageRefs": [
            {
              "id": "193dc144-2b55-4214-b211-ab83bd3e4a2e",
              "text": "I love it.",
              "offset": -1
            }
          ],
          "insightRefs": []
        }
      ]
    }
  ],
  "sequenceNumber": 1
}

Subscription started response object

When a subscription to a real-time conversation or stream is successfully initiated, a subscription_started event is sent. This event signifies that the WebSocket connection is now actively listening for real-time data related to the specified conversation.

connection.on("subscription_started", (data) => {
  // Handle data here.
});

JSON response example

Subscription started event received: {
  type: 'message',
  message: {
    type: 'subscription_started',
    data: { conversationId: '4776748637749248' }
  }
}
Subscription started for Conversation ID: 4776748637749248
Response type: message. Object:  {
  type: 'message',
  message: {
    type: 'subscription_started',
    data: { conversationId: '4776748637749248' }
  }
}
Live transcript: Hi
Response type: message. Object:  {
  type: 'message',
  message: {
    type: 'recognition_result',
    isFinal: false,
    payload: { raw: [Object] },
    punctuated: { transcript: 'Hi' },
    user: {
      email: '[email protected]',
      userId: '[email protected]',
      name: 'Test',
      id: '07ef3734-a817-4623-8ed1-b17a0500472c'
    }
  },
  timeOffset: 14152
}