Apply speaker separation to audio and video recordings

Symbl.ai Async API enables you to process a previously recorded audio or video conversation from URLs or files. You can also process the text file of a conversation.

This page describes how to apply speaker separation to a conversation stored in audio or video files.

Speaker Separation, often called Diarization, is the ability to detect and separate unique speakers in a single stream of audio or video without the need for separate speaker events.

Speaker Diarization is available for English language only.

Authentication

Before using this API, you must generate your authentication token (AUTH_TOKEN) as described in Authenticate.

Enable diarization

To enable Speaker Separation in the Async Audio or Video API, you must pass both of these request parameters:

Parameter	Type	Description
`enableSpeakerDiarization`	Boolean	Enable speaker separation for the audio or video data under consideration.
`diarizationSpeakerCount`	Integer	Sets the number of unique speakers in the audio or video data under consideration.

The example in this document uses the Submit Video request, but Speaker Separation can be achieved with other Async Audio/Video requests in the same way using both the enableSpeakerDiarization and diarizationSpeakerCount request parameters.

For then most accurate results, NUMBER_OF_UNIQUE_SPEAKERS should match the number of unique speakers in the Audio/Video data.

Note that you must wait for the job to finish processing before generating Conversation Intelligence. If you make a request from the Conversations API while the job is processing, you might receive incomplete insights. You can find out if a job is still processing by passing the jobId in the Get job status request.

The following sample code shows how to process a conversation using the Async Video URL-based API using a publicly available URL of a Video File:

Sample request

curl --location --request POST "https://api.symbl.ai/v1/process/video/
url?enableSpeakerDiarization=true&diarizationSpeakerCount=$NUMBER_OF_UNIQUE_SPEAKERS"
--header 'Content-Type: application/json'
--header "Authorization: Bearer $AUTH_TOKEN"
--data-raw '{
    "url": "https://storage.googleapis.com/demo-conversations/interview-prep.mp4"
}'

const authToken = AUTH_TOKEN;
const numberOfUniqueSpeakers = NUMBER_OF_UNIQUE_SPEAKERS;

const payload = {
  "url": "https://storage.googleapis.com/demo-conversations/interview-prep.mp4"
}

const responses = {
  400: 'Bad Request! Please refer docs for correct input fields.',
  401: 'Unauthorized. Please generate a new access token.',
  404: 'The conversation and/or it\'s metadata you asked could not be found, please check the input provided',
  429: 'Maximum number of concurrent jobs reached. Please wait for some requests to complete.',
  500: 'Something went wrong! Please contact [email protected]'
}

const fetchData = {
  method: "POST",
  headers: {
    'Authorization': `Bearer ${authToken}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(payload),
}

fetch(`https://api.symbl.ai/v1/process/video/url?enableSpeakerDiarization=true&diarizationSpeakerCount=${numberOfUniqueSpeakers}`, fetchData).then(response => {
  if (response.ok) {
    return response.json();
  } else {
    throw new Error(responses[response.status]);
  }
}).then(response => {
  console.log('response', response);
}).catch(error => {
  console.error(error);
});

import json
import requests

url = "https://api.symbl.ai/v1/process/video/url?enableSpeakerDiarization=true&diarizationSpeakerCount=" + NUMBER_OF_UNIQUE_SPEAKERS

payload = {
    "url": "https://storage.googleapis.com/demo-conversations/interview-prep.mp4"
}

# set your access token here. See https://docs.symbl.ai/docs/developer-tools/authentication
access_token = 'your_access_token'

headers = {
    'Authorization': 'Bearer ' + access_token,
    'Content-Type': 'application/json'
}

# webhookUrl = <Optional, string| your_webhook_url| Webhook url on which job updates to be sent. (This should be post API)>" e.g https://yourdomain.com/jobs/callback
# if webhookUrl is not None:
#   url += "?webhookUrl" + webhookUrl  

responses = {
    400: 'Bad Request! Please refer docs for correct input fields.',
    401: 'Unauthorized. Please generate a new access token.',
    404: 'The conversation and/or it\'s metadata you asked could not be found, please check the input provided',
    429: 'Maximum number of concurrent jobs reached. Please wait for some requests to complete.',
    500: 'Something went wrong! Please contact [email protected]'
}

response = requests.request("POST", url, headers=headers, data=json.dumps(payload), params=json.dumps(params))

if response.status_code == 201:
    # Successful API execution
    print("conversationId => " + response.json()['conversationId'])  # ID to be used with Conversation API.
    print("jobId => " + response.json()['jobId'])  # ID to be used with Job API.
elif response.status_code in responses.keys():
    print(responses[response.status_code])  # Expected error occurred
else:
    print("Unexpected error occurred. Please contact [email protected]" + ", Debug Message => " + str(response.text))

exit()

Sample response

{
    "conversationId": "4601416062599168",
    "jobId": "e33d764c-c663-488f-8581-d7182ad0d7a0"
}

Get speaker-separated results

Now that you have a conversationId from the previous response, you can use the Get messages request to get speaker-separated results.

Get messages request

curl --request GET \
     --url https://api.symbl.ai/v1/conversations/<CONVERSATION_ID>/messages \
     --header 'Accept: application/json' \
     --header 'Authorization: Bearer <AUTH_TOKEN>'

const options = {
  method: 'GET',
  headers: {
    Accept: 'application/json',
    Authorization: 'Bearer <AUTH_TOKEN>'
  }
};

fetch('https://api.symbl.ai/v1/conversations/<CONVERSATION_ID>/messages', options)
  .then(response => response.json())
  .then(response => console.log(response))
  .catch(err => console.error(err));

import requests

url = "https://api.symbl.ai/v1/conversations/<CONVERSATION_ID>/messages"

headers = {
    "Accept": "application/json",
    "Authorization": "Bearer <AUTH_TOKEN>"
}

response = requests.get(url, headers=headers)

print(response.text)

Get messages response

{
    "messages": [
        {
            "id": "4591723946704896",
            "text": "You're hired two words, everybody loves to hear.",
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "name": "Speaker 2"
            },
            "startTime": "2020-08-04T07:18:17.573Z",
            "endTime": "2020-08-04T07:18:21.573Z",
            "conversationId": "5105430690791424"
        },
        {
            "id": "6328236401229824",
            "text": "But before we hear these words comes the interview today's video is part one in a series.",
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "name": "Speaker 2"
            },
            "startTime": "2020-08-04T07:18:21.973Z",
            "endTime": "2020-08-04T07:18:30.473Z",
            "conversationId": "5105430690791424"
        },
    ]
}

The previous sample shows the speaker in the from object with a unique ID. These are the uniquely identified members of this conversation.

The speaker number in this sample is arbitrary, and the number doesn’t necessarily reflect the order in which someone spoke.

Identify unique speakers

You can then use the Get messages request to get the uniquely identified speakers for the conversation when Speaker Diarization is enabled.

Get messages request

curl --request GET \
     --url https://api.symbl.ai/v1/conversations/<CONVERSATION_ID>/members \
     --header 'Accept: application/json' \
     --header 'Authorization: Bearer <AUTH_TOKEN>'

const options = {
  method: 'GET',
  headers: {
    Accept: 'application/json',
    Authorization: 'Bearer <AUTH_TOKEN>'
  }
};

fetch('https://api.symbl.ai/v1/conversations/<CONVERSATION_ID>/members', options)
  .then(response => response.json())
  .then(response => console.log(response))
  .catch(err => console.error(err));

import requests

url = "https://api.symbl.ai/v1/conversations/<CONVERSATION_ID>/members"

headers = {
    "Accept": "application/json",
    "Authorization": "Bearer <AUTH_TOKEN"
}

response = requests.get(url, headers=headers)

print(response.text)

Get messages response

{
    "members": [
        {
            "id": "9d6d34d9-5019-4694-9c9a-8ba7bfc8cfab",
            "name": "Speaker 1"
        },
        {
            "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
            "name": "Speaker 2"
        }
    ]
}

The name assigned to a uniquely identified speaker/member from a separated audio/video follows the format Speaker <number> where <number> is arbitrary and does not necessarily reflect in which order people spoke.

Use the id to identify a speaker/member for that specific conversation and to update the details for the specific member, as demonstrated in the next section.

Updating detected members

The detected members (unique speakers) have names like Speaker 1. The automatic speaker recognition has no context to identify the speaker by name or other details. You can update the details of the detected speakers after the job is complete.

Get members

The Get messages request returns the uniquely identified speakers as shown in the preceding Identifying Unique Speakers section, when the Speaker Separation is enabled.

Consider the same set of members that can be retrieved using the Get members request.

JSON Response Example

{
    "members": [
        {
            "id": "9d6d34d9-5019-4694-9c9a-8ba7bfc8cfab",
            "name": "Speaker 1"
        },
        {
            "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
            "name": "Speaker 2"
        }
    ]
}

Update members

Now you can use the Update members request to update the details of a specific member. This example updates Speaker 2 with the values in the request:

$ curl --location --request PUT "https://api.symbl.ai/v1/conversations/$CONVERSATION_ID/members/2f69f1c8-bf0a-48ef-b47f-95ae5a4de325"
       --header 'Content-Type: application/json'
       --header "Authorization: Bearer $AUTH_TOKEN"
       --data-raw '{
            "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
            "email": "[email protected]",
            "name": "John Doe"
        }'

const authToken = AUTH_TOKEN;
const conversationId = 'your_conversation_id'  // Generated using Submit text end point
const memberId = 'your_member_id'  // MemberId of members fetched using fetchMember API
const url = `https://api.symbl.ai/v1/conversations/${conversationId}/members/${memberId}`;

payload = {
    'id': "UUID_to_be_updated",  // Should be a valid UUID e.g. f170371e-d9db-4d55-9d49-a111a89cf078
    'email': "email_id_to_be_updated",  // Should be a valid emailId e.g. [email protected]
    'name': "name_to_be_updated"  // Should be a valid string e.g. John
}

const responses = {
  400: 'Bad Request! Please refer docs for correct input fields.',
  401: 'Unauthorized. Please generate a new access token.',
  404: 'The conversation and/or it\'s metadata you asked could not be found, please check the input provided',
  429: 'Maximum number of concurrent jobs reached. Please wait for some requests to complete.',
  500: 'Something went wrong! Please contact [email protected]'
}

const fetchData = {
  method: "PUT",
  headers: {
    'Authorization': `Bearer ${authToken}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(payload),
}

fetch(url, fetchData).then(response => {
  if (response.ok) {
    return response.json();
  } else {
    throw new Error(responses[response.status]);
  }
}).then(response => {
  console.log('response', response);
}).catch(error => {
  console.error(error);
});

import json
import requests

baseUrl = "https://api.symbl.ai/v1/conversations/{conversationId}/members/{memberId}"
conversationId = 'your_conversation_id'  # Generated using Submit text end point
memberId = 'your_member_id'  # MemberId of members fetched using fetchMember API

url = baseUrl.format(conversationId=conversationId, memberId=memberId)

# set your access token here. See https://docs.symbl.ai/docs/developer-tools/authentication
access_token = 'your_access_token'

headers = {
    'Authorization': 'Bearer ' + access_token,
    'Content-Type': 'application/json'
}

payload = {
    'id': "UUID_to_be_updated",  # Should be a valid UUID e.g. f170371e-d9db-4d55-9d49-a111a89cf078
    'email': "email_id_to_be_updated",  # Should be a valid emailId e.g. [email protected]
    'name': "name_to_be_updated"  # Should be a valid string e.g. John
}

responses = {
    401: 'Unauthorized. Please generate a new access token.',
    404: 'The conversation and/or it\'s metadata you asked could not be found, please check the input provided',
    500: 'Something went wrong! Please contact [email protected]'
}

response = requests.request("PUT", url, headers=headers, data=json.dumps(payload))

if response.status_code == 200:
    # Successful API execution
    print(response.json()['message'])  # message containing status of response
elif response.status_code in responses.keys():
    print(responses[response.status_code])  # Expected error occurred
else:
    print("Unexpected error occurred. Please contact [email protected]" + ", Debug Message => " + str(response.text))

exit()

Replace CONVERSATION_ID with the actual Conversation ID (conversationId).
Replace AUTH_TOKEN with the Bearer token generated during Authentication.

The URL has the id of the member we want to append to PUT /members with the request body containing the updated name of this member.

You can also include the member's email address. If you include an email value, it can be used to identify the member in that conversation.

A successful Update members request generates the following response:

{
    "message": "Member with id: 2f69f1c8-bf0a-48ef-b47f-95ae5a4de325 for conversationId: <CONVERSATION_ID> updated successfully! The update should be reflected in all messages and insights along with this conversation"
}

The message is self-explanatory and tells us that all the references to the member with the id of 2f69f1c8-bf0a-48ef-b47f-95ae5a4de325 in the conversation now reflects the new values. That includes insights, messages of the conversation’s members.

So if we call Get members now, we see the following result:

{
    "members": [
        {
            "id": "9d6d34d9-5019-4694-9c9a-8ba7bfc8cfab",
            "name": "Speaker 1"
        },
        {
            "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
            "email": "[email protected]",
            "name": "John Doe"
        }
    ]
}

Similarly, when you use the Get messages request, you can see the updated member information.

{
    "messages": [
        {
            "id": "4591723946704896",
            "text": "You're hired two words, everybody loves to hear.",
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "email": "[email protected]",
                "name": "John Doe"
            },
            "startTime": "2020-08-04T07:18:17.573Z",
            "endTime": "2020-08-04T07:18:21.573Z",
            "conversationId": "5105430690791424"
        },
        {
            "id": "6328236401229824",
            "text": "But before we hear these words comes the interview today's video is part one in a series.",
            "from": {
                "id": "2f69f1c8-bf0a-48ef-b47f-95ae5a4de325",
                "email": "[email protected]",
                "name": "John Doe"
            },
            "startTime": "2020-08-04T07:18:21.973Z",
            "endTime": "2020-08-04T07:18:30.473Z",
            "conversationId": "5105430690791424"
        },

    ]
}