Twilio - Symbl.ai Video React App

This document describes how to implement the Symbl.ai and Twilio Video React App.

For the source code, see Symbl.ai Twilio Video React | github.com.

Symbl.ai Conversation Intelligence empowers developers to implement:

Real-time analysis of free-flowing discussions to automatically surface highly relevant summary discussion topics, contextual insights, suggestive action items, follow-ups, decisions, and questions.
Voice APIs that make it easy to add AI-powered Conversational Intelligence to either Telephony API or Streaming API (WebSocket) interfaces.
Conversation APIs that provide a REST interface for managing and processing your conversation data.
Summary UI with a fully customizable and editable reference experience that indexes a searchable transcript and shows generated actionable insights, topics, timestamps, and speaker information.

Enable Symbl for Twilio Video Conference

This is a multi-party video conferencing application that demonstrates Symbl.ai's Streaming API. This application is inspired by Twilio's video app and is built using twilio-video.js and Create React App.

Prerequisites

JS ES6+
Node.js v10+*
NPM v6+
Twilio account, sign up at Try Twilio Free

Features

Live Closed Captioning
Real-time Transcription
Video conferencing with real-time video and audio
Enable/Disable camera
Mute/unmute mic
Screen sharing
Dominant Speaker indicator
Network Quality Indicator

Browser Support

This application is only supported on Google Chrome.

Setup and Deploy

The first step is to sign up at Platform.Symbl.ai.

Get your Symbl credentials:

Your App Id that you can get from Platform.Symbl.ai.
Your App Secret that you can get from Platform.Symbl.ai.

This application offers two options for authorizing your Symbl.ai account, in the application, or via the included token server. Your Twilio account will be authorized via the token server.

The default behavior is for your Symbl account to authorize in-app. A dialog box is automatically shown if you're opening the app for the first time. In the config.js file you will find enableInAppCredentials set to true. For this option, you are not required to update the .env file with Symbl credentials.

If you are planning to use the included token server to generate your Symbl.ai token, you may disable the in-app App ID / App Secret configuration. You can disable it by setting enableInAppCredentials to false in the config.js

Store your Symbl credentials in the .env file.

SYMBL_APP_ID=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
SYMBL_APP_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Store your Twilio credentials in the .env file:

In your Twilio console, click on Settings and note of your Account SID.
Navigate to Settings > API Keys to generate a new Key SID and Secret.

TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TWILIO_API_KEY_SID=SKxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TWILIO_API_KEY_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

The local token server is managed by server.js.

Run the app locally with

$ npm start

This will start the local token server and run the app in the development mode. Open [<http://localhost:3000>](http://localhost:3000`) to see the application in the browser.

The page will reload if you make changes to the source code in src/.

You will also see any linting errors in the console. Start the token server locally with:

$ npm run server

The token server runs on port 8081 and exposes two GET endpoints. One to generate the access token for Symbl.ai and one for to generate the access token for Twilio.

The Symbl.ai token endpoint expects a GET request at /symbl-token route with no parameters.

The response will be a JSON format file with accessToken and expiresIn values with Symbl access token and expiry of token.

Try it out with this sample cURL command:

curl 'localhost:8081/symbl-token'

The Twilio token endpoint expects a GET request at /twilio-token route with the following query parameters:

identity: string,  // the user's identity
roomName: string   // the room name

The response will be a token that can be used to connect to a room.

Try it out with this sample cURL command:

curl 'localhost:8081/twilio-token?identity=TestName&roomName=TestRoom'

Multiple Participants in a Room

If you want to see how the application behaves with multiple participants, open localhost:3000 in multiple tabs in your browser and connect to the same room using different user names.

In addition, if you want to invite other participants to a room, each participant needs to have their own installation of this application and use the same room name and Account SID. The API Key and Secret can be different.

Dependencies

  "dependencies": {
    "@material-ui/core": "^4.11.0",
    "@material-ui/icons": "^4.9.1",
    "@material-ui/styles": "^4.10.0",
    "@primer/octicons-react": "^10.0.0",
    "@testing-library/jest-dom": "^4.2.4",
    "@testing-library/react": "^9.5.0",
    "@testing-library/user-event": "^7.2.1",
    "clsx": "^1.1.1",
    "concurrently": "^5.1.0",
    "d3-timer": "^1.0.10",
    "dotenv": "^8.2.0",
    "express": "^4.17.1",
    "fscreen": "^1.0.2",
    "is-plain-object": "^4.1.1",
    "lodash-es": "^4.17.15",
    "lodash.throttle": "^4.1.1",
    "moment": "^2.27.0",
    "node-fetch": "^2.6.0",
    "react": "^16.13.1",
    "react-copy-to-clipboard": "^5.0.2",
    "react-dom": "^16.13.1",
    "react-router-dom": "^5.2.0",
    "react-scripts": "3.4.1",
    "twilio": "^3.48.1",
    "twilio-video": "^2.7.1"
  }

Conclusion

This application lets you join a demo Twilio video conference with Symbl.ai transcripts displayed on the screen in real time.

Community

If you have any questions or comments, contact us at [email protected], through our Developer Slack, or Developer Community.

Source code available at Symbl.ai Twilio Video React | github.com.