This document provides information about how to build and use the Symbl Conversation Intelligence extension with the Agora SDK for Android applications.
Symbl is an AI-powered Conversation Intelligence (CI) platform for developers to build and extend applications capable of understanding natural human conversations at scale.
Our comprehensive suite of products and APIs enable developers to easily build and deploy intelligent speech-to-text functionality, extract contextual insights, generate domain-specific insights and intelligence, and access advanced conversation analytics.
With Symbl, you can analyze every call & interaction in real time as well as asynchronously to automatically surface the most important interactions that will help you identify compliance gaps, coaching opportunities, and missed opportunities to drive growth and revenue.
Make your events accessible and inclusive: Enable live captioning in all your events along with facilitating translation and localization in different languages based on your geographical reach. Show the current topic of discussion and important questions asked for the audience to keep face with how the conversation flows.
Real-time coaching: You can take your existing contact centre experience to the next level by automatically providing real-time suggestions that a customer care agent can use during a call, enabling the agent to focus on the immediate engagement with the customer and resolve customer needs quickly thanks to task automation. This results in the ability to serve more customers with higher satisfaction during a shift.
Automated Call Summarization: After a call, Symbl automates call disposition along with key follow-ups and questions, that shaves off a few seconds from the average call handling time and gives your agent time to focus on customer experience.
Enable effective search, catch-up and further content recommendations: Automatically index the event enabling the search by keywords, phrases, enable late attendees to catch up to the event by showing a historic transcript with topics and last questions asked. Based on the engagement of watched events, enable future content recommendations.
Real time compliance and automation: Easily monitor and score interactions by adding agent and customer specific analytics to your existing dashboards. Dive into deeper analysis and metadata on call statistics, agent productivity, and call quality, all of which lead to a better understanding of the priorities for the business. Also take actions based on intents, agent characteristics, answering machine detection and custom rules.
Symbl analyzes the conversation data and lets you:
- Generate Conversation Intelligence like Sentiment Analysis, Action Items, Topics, Trackers, Summary, and much more in your applications.
- Generate Speech-to-text capabilities like Transcriptions, Speaker Separation, and Speaker Diarization.
- Leverage multiple channels such as Video, Audio, Text, and Streaming.
To learn more about what is Symbl and how it works, go to our Introduction page.
This extension allows you to embed real-time conversation intelligence into your mobile application providing contextual analysis of conversation data along with speech recognition without any upfront data training requirements.
Before you use this extension, make sure you activate it in the Agora Marketplace. After successfully activating this extension, you can start sending audio data through your mobile application to Symbl. As a result of your application configuration and the audio data provided, you will receive the analyzed results.
This extension provides the following out-of-the-box conversational intelligence features:
- Speech-to-Text capabilities converting speech from a live audio stream into text for each speaker in the meeting. This feature is also known as transcription.
- Conversation Insights:
- Action Item: An action item is a specific outcome recognized in the conversation that requires one or more people in the conversation to act on it in the future.
- Follow-Ups: This is a category of Action Items with a connotation to follow up a request or a task (e.g. sending an email or making a phone call or booking an appointment or setting up a meeting).
- Questions: Explicit question or request for information that comes up during the conversation, whether answered or not, is recognized as a question.
- Topics: The most relevant topics of the discussion from the conversation that is generated based on the combination of the overall scope of the discussion.
- Sentiment Analysis: Sentiment analysis over each Topic that is being discussed in the conversation.
- Trackers: Trackers (custom business intents) allow you to track the occurrence of certain keywords or phrases in a conversation so you can identify emerging trends and gauge the nature of interactions. You can define keywords or phrases in a Tracker and Symbl will return messages that contain the same or contextually relevant phrases.
- Speaker Analytics: Provides you with functionality such as finding the speaker ratio, talk time, silence, pace, and overlap in a conversation.
Android API Level 29 or higher.
This section describes how to integrate the Symbl Conversation Intelligence extension into your Agora based mobile application. The code samples are written in Java.
Following are some of the prerequisites of integrating Symbl-Agora extension:
- Agora Credentials: Make sure you have the Agora credentials and required information (App ID, Channel Name, Temp Token and Meeting ID). This information can be retrieved from the Agora Console.
Learn more about the Agora required credentials in the Setup Authentication page.
- Symbl API Credentials: You will also need the Symbl API credentials which are available in the Symbl Conversation Intelligence extension page on a specific project.
To get your credentials, click View under Credentials column and you will be able to see the Symbl credentials (App ID and App Secret).
This section walks you through the steps necessary to set up the Symbl Conversation Intelligence extension in your mobile application.
- Download the following files:
- Add the
.aarfile as a dependency to your application.
- Create a new folder called
jniLibsunder your application (
- Extract the
agora-sdk/v1.0/arm64-v8a.zipand copy the folder
jniLibsfolder you created in the previous step.
- Extract the
agora-sdk/v1.0/armeabi-v7a.zipand copy the folder
jniLibsfolder you created in the previous step.
- Extract the
agora-sdk/sdk-v1.0/libs.zipand copy the file
agora-rtc-sdk.jarunder the folder
- Add the following information into your
- Implement the interface io.agora.rtc2.IMediaExtensionObserver
- Add the following method to set all the necessary information to initialize the Symbl configuration. You can find description for the parameters used in the table below:
The following table lists the parameters and their descriptions used in the sample code above:
|Used to identify the user in the real-time meeting.|
|The name of the user attending the real-time meeting.|
|Unique identifier for the meeting.|
|The URL for the Symbl platform.|
|The Symbl App ID.|
|The Symbl App Secret.|
|The language code. Currently, en-US (English US) is the only language supported.|
|The URL for secure token generation.|
|The audio encoding in which the audio will be sent over the WebSocket connection.|
|The rate of the incoming audio stream.|
|The minimum confidence score that you can set for an API to consider it as valid insight. The minimum confidence score should be in the range <=0.5 to <=1.0 (greater than or equal to 0.5 and less than or equal to 1.0.). Default value is 0.5.|
You would have to call the method above during the initialization of the Agora Engine in your application.
On your onEvent() method, you should receive all the transcription information and the analyzed conversation events where you can parse the JSON responses.
For a basic Speech-to-Text (recognition result) type of response, you should expect the following payload:
For a Topic type of response, you should expect the following payload:
For a Tracker type of response, you should expect the following payload:
Given below is a complete sample of
MainActivity.java class which includes the Agora RTC and Symbl settings.
Find comprehensive information about our REST APIs in the API Reference section.