Skip to main content

How to implement speaker separation with Async Audio or Video Files

Symbl's Async API allows you to process stored recordings of audio or video from files or URLs or even textual content from a conversation. In this guide, we will walk you through how to implement Speaker Separation with audio or video files. Speaker Separation, in short, is the ability to detect and separate unique speakers in a single stream of audio & video without the need for separate speaker events.

Speaker Diarization Language Support

Currently, Speaker Diarization is available for English and Spanish languages only.

Contents​

Enabling the Diarization​

To enable Speaker Separation in the Async Audio or Video API is as simple as adding these query parameters to the URL:

Parameter NameDescription
enableSpeakerDiarizationBoolean
Will enable the speaker separation for the audio or video data under consideration.
diarizationSpeakerCountInteger
Sets the number of unique speakers in the audio or video data under consideration.

This snippet shows a cURL command for consuming the Async Video URL-based API which takes in the URL for a publicly available URL of a Video File:

info

The example below uses the Async Video URL API, but Speaker Separation can be achieved with other Async Audio/Video APIs in the same way with the enableSpeakerDiarization and diarizationSpeakerCount parameters.

For accuracy, NUMBER_OF_UNIQUE_SPEAKERS should match the number of unique speakers in the Audio/Video data.

caution

You must wait for the job to complete processing before you proceed with getting the Conversation Intelligence. If you immediately make a GET request to Conversation API, it is possible that you'll receive incomplete insights. Therefore, ensure that you wait for the job to complete.

Authentication​

Before using this API, you must generate your authentication token (AUTH_TOKEN). To learn how to get the authentication token, see the Authentication page.

Code Example​

curl --location --request POST "https://api.symbl.ai/v1/process/video/
url?enableSpeakerDiarization=true&diarizationSpeakerCount=$NUMBER_OF_UNIQUE_SPEAKERS"
--header 'Content-Type: application/json'
--header "Authorization: Bearer $AUTH_TOKEN"
--data-raw '{
    "url": "https://storage.googleapis.com/demo-conversations/interview-prep.mp4"
}'

JSON Response Example​

{
    "conversationId": "4601416062599168",
    "jobId": "e33d764c-c663-488f-8581-d7182ad0d7a0"
}

Getting the Speaker Separated Results​

Now that you have a conversationId from the above response you can invoke the messages call in the Conversation API which returns the speaker-separated results.

Code Example​

View the API Reference for information on how to get speech-to-text messages from the conversation

👉 GET Messages

JSON Response Example​

{
    "messages": [
        {
            "id": "4591723946704896",
            "text": "You're hired two words, everybody loves to hear.",
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "name": "Speaker 2"
            },
            "startTime": "2020-08-04T07:18:17.573Z",
            "endTime": "2020-08-04T07:18:21.573Z",
            "conversationId": "5105430690791424"
        },
        {
            "id": "6328236401229824",
            "text": "But before we hear these words comes the interview today's video is part one in a series.",
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "name": "Speaker 2"
            },
            "startTime": "2020-08-04T07:18:21.973Z",
            "endTime": "2020-08-04T07:18:30.473Z",
            "conversationId": "5105430690791424"
        },
    ]
}

The above snippet shows the speaker in the from object with a unique ID. These are the uniquely identified members of this conversation.

info

Reminder: The speaker number in the above snippet is arbitrary and the number doesn’t necessarily reflect the order in which someone spoke.

Identifying Unique Speakers​

You can then invoke the members call in the Conversation API, which will return the uniquely identified speakers for the conversation when Speaker Diarization is enabled.

Code Example​

View the API Reference for information on how to get member information.

👉 GET Member Information

JSON Response Example​

{
    "members": [
        {
            "id": "9d6d34d9-5019-4694-9c9a-8ba7bfc8cfab",
            "name": "Speaker 1"
        },
        {
            "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
            "name": "Speaker 2"
        }
    ]
}

The name assigned to a uniquely identified speaker/member from a separated audio/video will follow the format Speaker <number> where <number> is arbitrary and does not necessarily reflect in what order someone spoke.

The id can be used to identify a speaker/member for that specific conversation and can be used to update the details for the specific member demonstrated below in the Updating Detected Members section.

Updating the Detected Members​

The detected members (unique speakers) would have names like Speaker 1 as the automatic speaker recognition wouldn’t have any context to who this speaker is (name or other details of the speaker). Therefore, it is important to update the details of the detected speakers after the Job is marked as complete.

GET members​

The members call in the Conversation API returns the uniquely identified speakers as shown in the Identifying Unique Speakers section above when the Speaker Separation is enabled.

Let’s consider the same set of members that can be retrieved by calling the GET members call in the Conversation API.

👉 GET Member Information

JSON Response Example​

{
    "members": [
        {
            "id": "9d6d34d9-5019-4694-9c9a-8ba7bfc8cfab",
            "name": "Speaker 1"
        },
        {
            "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
            "name": "Speaker 2"
        }
    ]
}

PUT members​

We can now use the PUT members call to update the details of a specific member as shown below. This call would update the Speaker 2 as shown in the above section with the values in the cURL’s request-body:

👉 PUT Members Information

$ curl --location --request PUT "https://api.symbl.ai/v1/conversations/$CONVERSATION_ID/members/2f69f1c8-bf0a-48ef-b47f-95ae5a4de325"
       --header 'Content-Type: application/json'
       --header "Authorization: Bearer $AUTH_TOKEN"
       --data-raw '{
            "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
            "email": "john@example.com",
            "name": "John Doe"
        }'

  • The CONVERSATION_ID needs to be replaced with the actual Conversation ID (conversationId)

  • The AUTH_TOKEN needs to be replaced with the Bearer token generated during our authentication process.

The URL has the id of the member we want to append to PUT /members with the request body containing the updated name of this member.

There is also the option to include the email of the member. The email will be used as an identifier for tracking those specific members uniquely in that conversation. (Refer to the Updating the Detected Members section below for more details)

After the above call is successful, we will receive the following response:

{
    "message": "Member with id: 2f69f1c8-bf0a-48ef-b47f-95ae5a4de325 for conversationId: <CONVERSATION_ID> updated successfully! The update should be reflected in all messages and insights along with this conversation"
}

The message is self-explanatory and tells us that all the references to the member with the id of 2f69f1c8-bf0a-48ef-b47f-95ae5a4de325 in the conversation should now reflect the new values we updated this member with. That includes insights, messages and the conversation’s members as well.

So if we call the GET /members API now, we would see the following result:

{
    "members": [
        {
            "id": "9d6d34d9-5019-4694-9c9a-8ba7bfc8cfab",
            "name": "Speaker 1"
        },
        {
            "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
            "email": "john@example.com",
            "name": "John Doe"
        }
    ]
}

And similarly, with the GET /messages API call, we would see the updates reflected below as well:

{
    "messages": [
        {
            "id": "4591723946704896",
            "text": "You're hired two words, everybody loves to hear.",
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "email": "john@example.com",
                "name": "John Doe"
            },
            "startTime": "2020-08-04T07:18:17.573Z",
            "endTime": "2020-08-04T07:18:21.573Z",
            "conversationId": "5105430690791424"
        },
        {
            "id": "6328236401229824",
            "text": "But before we hear these words comes the interview today's video is part one in a series.",
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "email": "john@example.com",
                "name": "John Doe"
            },
            "startTime": "2020-08-04T07:18:21.973Z",
            "endTime": "2020-08-04T07:18:30.473Z",
            "conversationId": "5105430690791424"
        },

    ]
}

Curious about the GET /insights API? It would reflect these updates as well!

{
    "insights": [
        {
            "id": "5501181057040384",
            "text": "We need to go over three more common interview questions.",
            "type": "action_item",
            "score": 1,
            "messageIds": [
                "5710067261243392"
            ],
            "entities": [],
            "phrases": [
                {
                    "type": "action_phrase",
                    "text": "go over three more common interview questions"
                }
            ],
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "email": "john@example.com",
                "name": "John Doe"
            },
            "definitive": true,
            "assignee": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "name": "Speaker 2"
            }
        },
        {
            "id": "5519156904460288",
            "text": "How did you hear about this position?",
            "type": "question",
            "score": 0.999988666660899,
            "messageIds": [
                "4616389407014912"
            ],
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "email": "john@example.com",
                "name": "John Doe"
            }
        },

    ]
}