Embedding Model

The Nebula embedding model creates vector embeddings from conversations, documents, or text data. It is optimized for calculating vector representations of conversation text (transcripts, emails, chats, etc.) while supporting the non-conversation text (documents, articles, etc.) with high performance.

See the Embedding API reference for details about the API.

What is embedding?

An embedding is a vector representation of text used to compare and identify text with similar characteristics. The vector consists of an array of floating-point numbers. These vectors can be used to calculate the relationship or similarity between two or more texts. You can identify the similarity or relationship by calculating the distance between the vectors. Cosine similarity is a recommended way of calculating the distance between the vectors.

You can use embeddings to perform several tasks that require similarity matching between two or more instances of text. A few of the popular tasks include the following:

• Search: Rank search results by relevance to a query string
• Clustering: Create clusters that group conversations and documents by similarity
• Recommendations: Identify recommendations based on related content
• Zero-shot classification: Classify content into similar groups

Other use cases can use embeddings to identify similar or dissimilar content, such as finding anomalies in data, analyzing data for diversity, and visualizing data.

Furthermore, you can provide the output of these embedding-based tasks to an LLM such as Nebula in order to perform additional generative tasks on historic or large amounts of data. Examples of these additional tasks include question-answering, knowledge extraction, summarization, topic modeling, report generation, and recommendations across historical conversations.

Get embeddings

To get an embedding for your text, call the Embedding API on the text, and the response will include the embedding.

Make sure you have access to an API key to use this API.

Request

curl --location 'https://api-nebula.symbl.ai/v1/model/embed' \
--header 'ApiKey: <api_key>' \
--header 'Content-Type: application/json' \
--data '{
    "text": "Dan: Definitely, John. The first feature we'\''re introducing ...."
}'

import requests
import json

url = "https://api-nebula.symbl.ai/v1/model/embed"

payload = json.dumps({
  "text": """Dan: Definitely, John. The first feature we're introducing ...."""
})
headers = {
  'ApiKey': '<api_key>',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Response

{
    "model": "nebula-text-embedding",
    "embedding": [
        0.01977849006652832,
        0.006099482532590628,
        0.021700561046600342,
        -0.008543084375560284,
        0.054180026054382324,
        ...
        -0.006577858701348305,
        -0.029168985784053802,
    ]
}

Calculate similarity

To calculate the similarity between the embeddings obtained from the API, you can use a function that calculates the distance between the embeddings. We recommend using cosine similarity. Here's a simple Python example that shows how to calculate the cosine similarity between two embeddings using the cosine_similarity function from the scikit-learn library.

from sklearn.metrics.pairwise import cosine_similarity
import numpy as np

# embedding_1 and embedding_2 are embedding lists from API response
embedding_1 = [0.01977849006652832, 0.006099482532590628, ...]
embedding_2 = [-0.008543084375560284, 0.054180026054382324, ...]

embedding_1 = np.array(embedding_1)
embedding_2 = np.array(embedding_2)

similarity = cosine_similarity([embedding_1, embedding_2])
print(similarity)