Events and Callbacks

Both the connection and audio stream objects have a connection.on method which can be used to subscribe to events and perform callbacks.

Connection Events

Listeners can subscribe to the following events on the Connection object:

Example

connection.on("topic", (topicData) => {
  topicData.forEach((topic) => {
    console.log("Topic: " + topic.phrases);
  });
});
 
connection.on("disconnected", () => {
  console.log("User has been disconnected")
});
EventDescriptionCallback Data
connectedWhen the WebSocket connection is successfully established.None
disconnectedWhen the WebSocket connection is disconnected.None
started_listeningStarted listening to input device.None
stopped_listeningStopped listening to input device.None
processing_startedAudio data processing successfully started.Processing Started Response Object
processing_stoppedAudio data processing stopped.None
conversation_createdConversation is created and an ID is generated.Conversation Created Response Object
conversation_completedConversation is ended.Conversation Completed Response Object
session_modifiedWhen the sample rate of the session is modified.Returns an object containing the new sample rate.
speech_recognitionWhen data is being transferred between the client and server.Speech Recognition Object
messageWhen the message object is detected.Message Response Object
topicWhen topics are detected.Topic Response Object
trackerWhen Trackers are detected.Tracker Response Object
action_itemWhen Action Items are detected.Action Item Response Object
follow_upWhen follow-ups are detected.Follow Up Response Object
questionWhen questions are detected.Question Response Object

AudioStream Events

Listeners can subscribe to the following events on the Connection object:

Example

const audioStream = new OPUSAudioStream();
audioStream.on("audio_source_disconnected", () => {
  // Do something.
});

// OR 

connection.audioStream.on("audio_source_disconnected", () => {
  // Do something.
});
EventDescriptionCallback Data
audio_source_connectedThe audio source is connected to Symbl.Returns the sample rate of the new audio source.
audio_source_disconnectedThe audio source is connected.None
audio_source_changedA new default device is detected.None

Global Events

Listeners can subscribe to the following global Symbl Events using the window object.

Example

window.addEventListener("error", (error) => {
  const thrownError = error.detail;

  // Do something.
})
EventDescriptionCallback Data
errorErrors recorded when there are potential anti-patterns or non-recommended coding.The Error Object that was thrown. Will be found in the callbackData.detail.

Callback Data Reference

Conversation Created Response Object

To retrieve the conversation ID as soon as the conversation is established, use this callback.

connection.on("conversation_created", (conversationData) => {
  // Handle conversationData here.
});

JSON Response Example

{
  "type": "conversation_created",
  "data": {
    "conversationId": "5093011103940608"
  }
}

Conversation Completed Response Object

To retrieve the conversation ID and summaryUrl as soon as the conversation is completed, use this callback.

connection.on("conversation_completed", (conversationData) => {
  // Handle conversationData here.
});

JSON Response Example

{
  "type": "conversation_completed",
  "conversationId": "5093011103940608",
  "summaryUrl": "https://meetinginsights.symbl.ai/meeting/#/eyJzZXNzaW9uSWQiOiI2NTA0OTI1MTg4MDYzMjMyIiwidmlkZW9VcmwiOiJodHRwczovL3N0b3JhZ2UuZ29vZ2xlYXBpcy5jb20vcmFtbWVyLXRyYW5zY3JpcHRpb24tYnVja2V0LzE5MzE0MjMwMjMubXA0In0="
}

Processing Started Response Object

To retrieve the conversation ID as soon as the audio processing begins, use this callback.

connection.on("processing_started", (recognitionData) => {
  // Handle conversationData here.
});

JSON Response Example

{
  "type": "recognition_started",
  "data": {
    "conversationId": "5093011103940608"
  }
}

Speech Recognition Object

To retrieve the real-time transcription results as soon as they are detected. You can use this callback to render a live transcription which is specific to the speaker of this audio stream.

connection.on("speech_recognition", (speechData) => {
  // Handle speechData here.
});

JSON Response Example

{
  "type": "recognition_result",
  "isFinal": true,
  "payload": {
    "raw": {
      "alternatives": [{
        "words": [{
          "word": "Hello",
          "startTime": {
            "seconds": "3",
            "nanos": "800000000"
          },
          "endTime": {
            "seconds": "4",
            "nanos": "200000000"
          }
        }, {
          "word": "world.",
          "startTime": {
            "seconds": "4",
            "nanos": "200000000"
          },
          "endTime": {
            "seconds": "4",
            "nanos": "800000000"
          }
        }],
        "transcript": "Hello world.",
        "confidence": 0.9128385782241821
      }]
    }
  },
  "punctuated": {
    "transcript": "Hello world."
  },
  "user": {
    "userId": "emailAddress",
    "name": "John Doe",
    "id": "23681108-355b-4fc3-9d94-ed47dd39fa56"
  }
}

Message Response Object

This callback function contains the finalized transcription data for this speaker and if used with multiple streams with other speakers this callback would also provide their messages.

The finalized messages mean that the automatic speech recognition has finalized the state of this part of transcription and has declared it final. Therefore, this transcription will be more accurate than the Speech Recognition Object.

connection.on("message", (data) => {
  // Handle data here.
});
JSON Response Example
[{
  "from": {
    "id": "0a7a36b1-047d-4d8c-8958-910317ed9edc",
    "name": "John Doe",
    "userId": "emailAddress"
  },
  "payload": {
    "content": "Hello world.",
    "contentType": "text/plain"
  },
  "id": "59c224c2-54c5-4762-9582-961bf250b478",
  "channel": {
    "id": "realtime-api"
  },
  "metadata": {
    "disablePunctuation": true,
    "timezoneOffset": 480,
    "originalContent": "Hello world.",
    "words": "[{\"word\":\"Hello\",\"startTime\":\"2021-02-04T20:34:59.029Z\",\"endTime\":\"2021-02-04T20:34:59.429Z\"},{\"word\":\"world.\",\"startTime\":\"2021-02-04T20:34:59.429Z\",\"endTime\":\"2021-02-04T20:35:00.029Z\"}]",
    "originalMessageId": "59c224c2-54c5-4762-9582-961bf250b478"
  },
  "dismissed": false,
  "duration": {
    "startTime": "2021-02-04T20:34:59.029Z",
    "endTime": "2021-02-04T20:35:00.029Z"
  }
}]

Action Item Response Object

This callback provides you with any of the detected action items in real-time as they are detected. As with the Message Response Object this also returns every speaker's action items in case of multiple streams.

connection.on("action_item", (data) => {
  // Handle data here.
});
JSON Response Example
[{
  "id": "94020eb9-b688-4d56-945c-a7e5282258cc",
  "confidence": 0.9909798145016999,
  "messageReference": {
    "id": "94020eb9-b688-4d56-945c-a7e5282258cc"
  },
  "hints": [{
    "key": "informationScore",
    "value": "0.9782608695652174"
  }, {
    "key": "confidenceScore",
    "value": "0.9999962500210938"
  }, {
    "key": "comprehensionScore",
    "value": "0.9983848333358765"
  }],
  "type": "action_item",
  "assignee": {
    "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0",
    "name": "John Doe",
    "userId": "emailAddress"
  },
  "dueBy": {
    "value": "2021-02-05T00:00:00-07:00"
  },
  "tags": [{
    "type": "date",
    "text": "today",
    "beginOffset": 39,
    "value": {
      "value": {
        "datetime": "2021-02-05"
      }
    }
  }, {
    "type": "person",
    "text": "John Doe",
    "beginOffset": 8,
    "value": {
      "value": {
        "name": "John Doe",
        "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0",
        "assignee": true,
        "userId": "emailAddress"
      }
    }
  }],
  "dismissed": false,
  "payload": {
    "content": "Perhaps John Doe can submit the report today.",
    "contentType": "text/plain"
  },
  "from": {
    "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0",
    "name": "John Doe",
    "userId": "emailAddress"
  }
}]

Question Response Object

This callback provides you with any of the detected questions in real-time as they are detected. As with the Message Response Object this also returns every speaker's questions in case of multiple streams.

connection.on("question", (data) => {
  // Handle data here.
});
JSON Response Example
[
  {
    "id": "e0e44c21-c965-47b0-92d9-878ac22302ae",
    "confidence": 0.9834683553122807,
    "hints": [
      {
        "key": "confidenceScore",
        "value": "0.9957259328650095"
      },
      {
        "key": "comprehensionScore",
        "value": "0.971210777759552"
      }
    ],
    "type": "question",
    "assignee": {
      "id": "29c192e0-6fbc-4b94-9cb8-040783654003",
      "name": "Jane Doe",
      "userId": "[email protected]"
    },
    "tags": [],
    "dismissed": false,
    "payload": {
      "content": "How may I help you today?",
      "contentType": "text/plain"
    },
    "from": {
      "id": "29c192e0-6fbc-4b94-9cb8-040783654003",
      "name": "Jane Doe",
      "userId": "[email protected]"
    },
    "entities": null,
    "messageReference": {
      "id": "79a57ed7-d043-4a82-85fc-ae7844d8d2bb"
    }
  }
]

Follow Up Response Object

This callback provides you with any of the detected follow ups in real-time as they are detected. As with the Message Response Object this also returns every speaker's follow ups in case of multiple streams.

connection.on("follow_up", (data) => {
  // Handle data here.
});
JSON Response Example
[
  {
    "id": "05bfb176-c2d3-42fd-a7e7-bbc80596a3e9",
    "confidence": 1,
    "hints": [
      {
        "key": "addressedTo",
        "value": "[\"first_person_singular\",\"second_person_singular\"]"
      },
      {
        "key": "informationScore",
        "value": "0.7361413043478261"
      },
      {
        "key": "confidenceScore",
        "value": "1.0"
      }
    ],
    "type": "follow_up",
    "assignee": {
      "id": "29c192e0-6fbc-4b94-9cb8-040783654002",
      "name": "Adam Voliva",
      "userId": "[email protected]"
    },
    "tags": [
      {
        "type": "person",
        "text": "Adam Voliva",
        "beginOffset": 0,
        "value": {
          "value": {
            "name": "Adam Voliva",
            "id": "29c192e0-6fbc-4b94-9cb8-040783654002",
            "assignee": true,
            "userId": "[email protected]"
          }
        }
      }
    ],
    "dismissed": false,
    "payload": {
      "content": "Adam Voliva can send it internet service technician to your home.",
      "contentType": "text/plain"
    },
    "from": {
      "id": "29c192e0-6fbc-4b94-9cb8-040783654002",
      "name": "Adam Voliva",
      "userId": "[email protected]"
    },
    "entities": null,
    "messageReference": {
      "id": "05bfb176-c2d3-42fd-a7e7-bbc80596a3e9"
    }
  }
]

Topic Response Object

This callback provides you with any of the detected topics in real-time as they are detected. As with the Message Response Object this also returns every topic in case of multiple streams.

connection.on("topic", (data) => {
  // Handle data here.
});
JSON Response Example
[{
  "id": "e69a5556-6729-11eb-ab14-2aee2deabb1b",
  "messageReferences": [{
    "id": "0df44422-0248-47e9-8814-e87f63404f2c",
    "relation": "text instance"
  }],
  "phrases": "auto insurance",
  "rootWords": [{
    "text": "auto"
  }],
  "score": 0.9,
  "type": "topic"
}]

Tracker Response Object

This callback provides you with any of the detected trackers in real-time as they are detected. As with the Message Response Object this also returns every tracker in case of multiple streams.

connection.on("tracker", (data) => {
  // Handle data here.
});
JSON Response Example
{
  "type": "tracker_response",
  "isFinal": true,
  "trackers": [
    {
      "name": "Goodness",
      "matches": [
        {
          "type": "vocabulary",
          "value": "This is awesome",
          "messageRefs": [
            {
              "id": "fa93aa64-0e8d-4697-bb52-e2916ca63192",
              "text": "This is awesome.",
              "offset": 0
            }
          ],
          "insightRefs": []
        },
        {
          "type": "vocabulary",
          "value": "Hello world",
          "messageRefs": [
            {
              "id": "8e720656-fed7-4b11-b359-3931c53bbcec",
              "text": "Hello world.",
              "offset": 0
            }
          ],
          "insightRefs": []
        }
      ]
    },
    {
      "name": "Goodness",
      "matches": [
        {
          "type": "vocabulary",
          "value": "I like it",
          "messageRefs": [
            {
              "id": "193dc144-2b55-4214-b211-ab83bd3e4a2e",
              "text": "I love it.",
              "offset": -1
            }
          ],
          "insightRefs": []
        }
      ]
    }
  ],
  "sequenceNumber": 1
}

Did this page help you?