Skip to main content

Detect Key Phrases

In Beta Phase

This feature is in the Beta phase. If you have any questions, ideas or suggestions please reach us at devrelations@symbl.ai.

This example goes over how you can use the Symbl Streaming API to detect key prases using the Trackers API. This example uses both the Symbl's Javascript SDK, which is meant to be run using Node.js, and native Javascript which can be run in the browser.

info

If you wish to learn more about Trackers you can view our Trackers Concept Page.

Track Phrases#

To detect words or phrases you pass them in the configuration as such:

{
"trackers": [
{
"name": "COVID-19",
"vocabulary": [
"social distancing",
"cover your face with mask",
"vaccination"
]
}
]
}

If these phrases are found in the conversation the API will trigger a callback which would contain the tracker response.

Snippet#

Below is an example that shows how to pass Trackers in the config object for the startRealtimeRequest of the Symbl’s Javascript SDK. This example also shows how to consume the results of the detected Trackers in real-time:

const {sdk} = require('symbl-node');
const uuid = require('uuid').v4;
(async () => {
try {
// Initialize the SDK. You can find the appId and appSecret at https://platform.symbl.ai.
await sdk.init({
appId: appId,
appSecret: appSecret,
basePath: 'https://api.symbl.ai',
})
// Need unique Id
const id = uuid()
const connection = await sdk.startRealtimeRequest({
id,
insightTypes: ['action_item', 'question'],
trackers: [
{
name: "COVID-19",
vocabulary: [
"social distancing",
"cover your face with mask",
"vaccination"
]
}
],
config: {
meetingTitle: "My Awesome Meeting",
confidenceThreshold: 0.7,
languageCode: "en-US",
sampleRateHertz: 48000,
trackers: {
"interimResults": true
}
},
speaker: {
// Optional, if not specified, will simply not send an email in the end.
userId: "john@example.com", // Update with valid email
name: "John",
},
handlers: {
onTrackerResponse: (data) => {
// When a tracker is detected in real-time
console.log('onTrackerResponse', JSON.stringify(data, null, 2));
if (!!data) {
data.forEach((tracker) => {
console.log(`Detected Tracker Name: ${tracker.name}`);
console.log(`Detected Matches`);
tracker.matches.forEach((match) => {
console.log(`Tracker Value: ${match.value}`);
console.log(`Messages detected against this Tracker`);
match.messageRefs.forEach((messageRef) => {
console.log(`Message ID: ${messageRef.id}`);
console.log(`Message text for which the match was detected: ${messageRef.text}`);
console.log(`\n`);
});
console.log(`\n\n`);
console.log(`Insights detected against this Tracker`);
match.messageRefs.forEach((insightRef) => {
console.log(`Insight ID: ${insightRef.id}`);
console.log(`Insight text for which the match was detected: ${insightRef.text}`);
console.log(`Insight Type: ${insightRef.type}`);
console.log(`\n`);
});
console.log(`\n\n`);
});
});
}
},
},
});
} catch (e) {
console.error(e);
}
})();

Connect Mic#

After you connect, you want to connect to your device's microphone. This code is inserted after the connection and before the closing of the try...catch

const micInputStream = micInstance.getAudioStream()
/** Raw audio stream */
micInputStream.on('data', (data) => {
// Push audio from Microphone to websocket connection
connection.sendAudio(data)
})
micInputStream.on('error', function (err) {
console.log('Error in Input Stream: ' + err)
})
micInputStream.on('startComplete', function () {
console.log('Started listening to Microphone.')
})
micInputStream.on('silence', function () {
console.log('Got SIGNAL silence')
})
micInstance.start()
setTimeout(async () => {
// Stop listening to microphone
micInstance.stop()
console.log('Stopped listening to Microphone.')
try {
// Stop connection
await connection.stop()
console.log('Connection Stopped.')
} catch (e) {
console.error('Error while stopping the connection.', e)
}
}, 60 * 1000) // Stop connection after 1 minute i.e. 60 secs

Testing#

Create a javascript file named app.js and copy this code into the file. Fill in the placeholder values with the proper values. Use npm to install the required libraries: npm install symbl-node uuid. Now in the terminal run

$ node app.js

If successful you should receive a response in the console.

Handlers Reference#

  • handlers: This object has the callback functions for different events

    • onTrackerResponse: This callback provides you with any of the detected trackers in real-time as they are detected. As with the onMessageCallback this would also return every tracker in case of multiple streams.

    onTrackerResponse JSON Response Example#

    [
    {
    "id": "4527907378937856",
    "name": "My Awesome Tracker",
    "matches": [
    {
    "messageRefs": [
    {
    "id": "4670860273123328",
    "text": "Wearing mask is a good safety measure.",
    "offset": -1
    }
    ],
    "type": "vocabulary",
    "value": "wear mask",
    "insightRefs": []
    }
    ]
    }
    ...
    ]