twelvelabs-video-intelligence（十二实验室视频智能）

TwelveLabs

Search inside videos, generate summaries, chapters, highlights, and analyze video content using the TwelveLabs Python SDK (v1.2.1).

Setup

1. Get your API key: https://api.twelvelabs.io
2. Install the SDK:

   python3 -m pip install twelvelabs

1. Set environment variable:

   export TWELVE_LABS_API_KEY="your-api-key"

Important: index before you search or analyze

Videos must be indexed before they can be searched or analyzed. Always index a video and wait for status Ready before running search or analysis commands.

Initialize the client

All snippets below assume this setup:

import os
from twelvelabs import TwelveLabs

client = TwelveLabs(api_key=os.environ["TWELVE_LABS_API_KEY"])

List indexes

for index in client.indexes.list():
    print(f"{index.id}: {index.index_name}")

Create an index

from twelvelabs.indexes import IndexesCreateRequestModelsItem

index = client.indexes.create(
    index_name="my-index",
    models=[
        IndexesCreateRequestModelsItem(model_name="marengo3.0", model_options=["visual", "audio"]),
        IndexesCreateRequestModelsItem(model_name="pegasus1.2", model_options=["visual", "audio"]),
    ],
    addons=["thumbnail"],
)
print(f"Created index: {index.id}")

Use both models: Marengo 3.0 for search/embeddings, Pegasus 1.2 for analysis.

Upload an asset

From URL:

asset = client.assets.create(method="url", url="https://example.com/video.mp4")
print(f"Asset: {asset.id}")

From local file:

with open("/path/to/video.mp4", "rb") as f:
    asset = client.assets.create(method="direct", file=f)
print(f"Asset: {asset.id}")

Assets are reusable — upload once, index into multiple indexes.

Index an asset

indexed = client.indexes.indexed_assets.create(
    index_id="<index-id>",
    asset_id=asset.id,
)
print(f"Indexed asset: {indexed.id}")

Check indexing status

result = client.indexes.indexed_assets.retrieve(
    index_id="<index-id>",
    indexed_asset_id="<indexed-asset-id>",
)
print(f"Status: {result.status}")

Statuses: pending → queued → indexing → ready (or failed).

Poll until ready

import time

while True:
    result = client.indexes.indexed_assets.retrieve(
        index_id="<index-id>",
        indexed_asset_id=indexed.id,
    )
    if result.status == "ready":
        print(f"Ready. Indexed asset ID: {indexed.id}")
        break
    if result.status == "failed":
        raise RuntimeError("Indexing failed")
    print(f"Status: {result.status} — waiting...")
    time.sleep(5)

List indexed assets

for video in client.indexes.indexed_assets.list("<index-id>"):
    duration = video.system_metadata.duration if hasattr(video, "system_metadata") and video.system_metadata else ""
    filename = video.system_metadata.filename if hasattr(video, "system_metadata") and video.system_metadata else ""
    print(f"{video.id}  {filename}  {duration}s")

Search

results = client.search.query(
    index_id="<index-id>",
    query_text="person giving a presentation",
    search_options=["visual", "audio"],
)

for result in results:
    print(f"{result.start:.1f}s - {result.end:.1f}s  rank={result.rank}  video={result.video_id}")

Results include timestamps (start/end in seconds) and relevance rank (1 = most relevant).

Search options

results = client.search.query(
    index_id="<index-id>",
    query_text="your query",
    search_options=["visual", "audio"],  # or ["transcription"]
    threshold="medium",                   # high, medium, low (default), none
    group_by="video",                     # clip (default) or video
    sort_option="clip_count",             # score (default) or clip_count
    page_limit=20,                        # max 50
)

Transcription search

results = client.search.query(
    index_id="<index-id>",
    query_text="what the speaker said about revenue",
    search_options=["transcription"],
    transcription_options=["semantic"],  # semantic, lexical, or both (default)
)

Search with filtering

import json

# Filter by duration range
results = client.search.query(
    index_id="<index-id>",
    query_text="your query",
    search_options=["visual"],
    filter=json.dumps({"duration": {"gte": 60, "lte": 300}}),
)

Filter fields: id (array of video IDs), duration, width, height, size, filename.

Analyze a video

Use a prompt to generate summaries, chapters, highlights, titles, topics, hashtags, action items, or any open-ended analysis:

response = client.analyze(
    video_id="<video-id>",
    prompt="Generate a summary with chapters and key takeaways",
)
print(response.data)

Example prompts:

• "Summarize this video in 3 paragraphs"
• "List the chapters with timestamps"
• "Extract action items from this meeting"
• "Generate a title, 5 topics, and 10 hashtags"
• "Create a table of contents"

Streaming analysis

stream = client.analyze_stream(
    video_id="<video-id>",
    prompt="Describe this video",
)
for chunk in stream:
    if chunk.event_type == "text_generation":
        print(chunk.text, end="")

Composed search (text + image)

Combine text and image for refined results. Marengo 3.0 required:

with open("image.jpg", "rb") as f:
    results = client.search.query(
        index_id="<index-id>",
        query_text="red color",
        query_media_type="image",
        query_media_file=f,
        search_options=["visual"],
    )

Entity search

Find specific people or objects using reference images. Marengo 3.0 required:

# Create collection + entity with reference images
collection = client.entity_collections.create(name="my-team")
asset = client.assets.create(method="url", url="https://example.com/person.jpg")
entity = client.entity_collections.entities.create(
    entity_collection_id=collection.id,
    name="Person Name",
    asset_ids=[asset.id],
)

# Search for entity
results = client.search.query(
    index_id="<index-id>",
    query_text=f"<@{entity.id}> is walking",
    search_options=["visual", "audio"],
)

Embeddings

Retrieve embeddings from an indexed video

video = client.indexes.videos.retrieve(
    index_id="<index-id>",
    video_id="<video-id>",
    embedding_option=["visual", "audio"],
)

for segment in video.embedding.video_embedding.segments:
    print(f"{segment.start_offset_sec}s - {segment.end_offset_sec}s  scope={segment.embedding_scope}  option={segment.embedding_option}  dims={len(segment.float_)}")

Create new embeddings (Embed v2 API)

from twelvelabs.types import TextInputRequest, ImageInputRequest, MediaSource

# Text embedding (512 dimensions)
res = client.embed.v_2.create(
    input_type="text",
    model_name="marengo3.0",
    text=TextInputRequest(input_text="your text here"),
)
embedding = res.data[0].embedding

# Image embedding from URL
res = client.embed.v_2.create(
    input_type="image",
    model_name="marengo3.0",
    image=ImageInputRequest(media_source=MediaSource(url="https://example.com/image.jpg")),
)
embedding = res.data[0].embedding

Sample video for testing

Big Buck Bunny — 10 seconds, 1MB, free to use:

https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/720/Big_Buck_Bunny_720_10s_1MB.mp4

Limitations

• Indexing is async — videos take time to process. Poll client.indexes.indexed_assets.retrieve() until status is ready.
• Composed search requires Marengo 3.0 — text + image search only works with this model.
• Entity search requires Marengo 3.0 — and additional setup (collections, assets, entities).
• Query limit — search queries support up to 500 tokens (Marengo 3.0) or 77 tokens (older models).
• Page limit — max 50 results per search page.
• Index names must be unique — use timestamps or UUIDs to avoid conflicts.
• client.summarize() and client.gist() are deprecated — use client.analyze(prompt="...") instead.

Resources

• TwelveLabs Documentation
• API Reference
• Python SDK v1.2.1 (latest)
• Get API Key