NVIDIA/cosmos

Use HuggingFace Diffusers for Cosmos 3 Generator research, training, and model development. This path loads the full Cosmos 3 checkpoint, including the reasoner path, diffusion generation path, and media tokenizers.

uv venv --python 3.13 --seed --managed-python
source .venv/bin/activate
uv pip install --torch-backend=auto \
  "diffusers @ git+https://github.com/huggingface/diffusers.git" \
  accelerate \
  av \
  cosmos_guardrail \
  huggingface_hub \
  imageio \
  imageio-ffmpeg \
  torch \
  torchvision \
  transformers

--torch-backend=auto lets uv detect your NVIDIA driver and install a matching CUDA build of torch/torchvision. Without it, uv pulls the newest CUDA wheel (currently cu130), which fails on pre-CUDA-13 drivers with The NVIDIA driver on your system is too old and torch.cuda.is_available() returns False. Pin an explicit backend instead if you prefer, e.g. --torch-backend=cu128 for a CUDA 12.8 driver.

A text-to-video run takes a while: the first run downloads Cosmos3-Nano, and diffusion is compute-heavy, running through every inference step before producing output. Long step times are expected, not a hang.

import torch
from diffusers import Cosmos3OmniPipeline
from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler
from diffusers.utils import export_to_video

pipe = Cosmos3OmniPipeline.from_pretrained(
    "nvidia/Cosmos3-Nano",
    torch_dtype=torch.bfloat16,
    device_map="cuda",
)
pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config, flow_shift=10.0)

result = pipe(
    prompt="A mobile robot navigates a warehouse aisle and stops at a shelf.",
    negative_prompt="",
    image=None,
    num_frames=189,
    height=720,
    width=1280,
    fps=24,
    num_inference_steps=35,
    guidance_scale=6.0,
    enable_sound=False,
    add_resolution_template=False,
    add_duration_template=False,
    generator=torch.Generator(device="cuda").manual_seed(1234),
)

export_to_video(result.video, "cosmos3_t2v.mp4", fps=24, macro_block_size=1)

Diffusers modes:

See the Cosmos 3 Diffusers documentation for runnable examples of each mode.

Use vLLM-Omni for Generator production inference behind an OpenAI-compatible API. This integration loads the full Cosmos 3 checkpoint, including the Qwen3-VL-based reasoner path and the diffusion generation path. For understanding-only tasks that return text, use Reasoner with vLLM instead, which loads only the reasoner.

Compatibility status: Cosmos 3 Generator support is available in vllm-project/vllm-omni main for text-to-image, text-to-video, image-to-video, video-to-video, transfer-control video-to-video, video-with-sound, and action generation. For current setup and per-modality usage, see the maintained recipes: Cosmos3-Nano and Cosmos3-Super.

Start the server from the vllm/vllm-omni:cosmos3 Docker image. Mount any directory that contains local media or action files you want the server to read. The command below runs from /workspace, so repo-local paths such as cookbooks/... resolve inside the container.

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -v "$(pwd):/workspace" \
  -p 8000:8000 \
  --ipc=host \
  -w /workspace \
  vllm/vllm-omni:cosmos3 \
  vllm serve nvidia/Cosmos3-Nano \
  --omni \
  --model-class-name Cosmos3OmniDiffusersPipeline \
  --allowed-local-media-path / \
  --port 8000 \
  --init-timeout 1800

Cosmos3 checkpoints can exceed the default server init timeout; use --init-timeout 1800 on every vllm serve command in this section.

vLLM-Omni prints Application startup complete. when the API is ready.

For nvidia/Cosmos3-Super (the larger 64B model), split weights across GPUs and optionally offload layers to reduce peak memory: --tensor-parallel-size splits model weights across multiple GPUs, and --enable-layerwise-offload offloads transformer blocks between CPU and GPU with a latency tradeoff and extra CPU RAM use. For example, on four GPUs, add --tensor-parallel-size 4 --enable-layerwise-offload --init-timeout 1800 to the vllm serve command.

Additional parallelism options:

When combining parallelism options, ensure the server has enough GPUs for the product of the enabled degrees (tensor_parallel_size × cfg_parallel_size × ulysses_degree).

To install vLLM-Omni from main instead of using the Docker image, create a venv and install, choosing the CUDA build that matches your driver. This path uses the same request formats as the Docker image; see the Cosmos3-Nano and Cosmos3-Super recipes for per-modality usage:

uv venv --python 3.13 --seed --managed-python
source .venv/bin/activate
# CUDA 13 driver:
uv pip install --torch-backend=cu130 \
  "vllm-omni @ git+https://github.com/vllm-project/vllm-omni.git@main"
# CUDA 12.8 driver:
# uv pip install --torch-backend=cu128 \
#   "vllm-omni @ git+https://github.com/vllm-project/vllm-omni.git@main"

Then run vllm serve nvidia/Cosmos3-Nano --omni --model-class-name Cosmos3OmniDiffusersPipeline --allowed-local-media-path / --port 8000 --init-timeout 1800 directly, without the docker run ... vllm/vllm-omni:cosmos3 wrapper.

Vision endpoints:

Action modes use Cosmos 3 as a world model: they condition on an embodiment (domain_name) and exchange video and action sequences. Policy and inverse dynamics return a predicted action chunk, so send those through the asynchronous POST /v1/videos job and read the action data from the completed result; forward dynamics returns only video and can use synchronous POST /v1/videos/sync.

Pass embodiment settings through extra_params: action_mode, domain_name (for example bridge_orig_lerobot, av, or camera_pose), raw_action_dim, and action_chunk_size. Forward dynamics also takes an action_path pointing at an action file the server can read, so start the server with --allowed-local-media-path covering that file (for Docker, mount the file and pass the container-visible path). For the full set of robot, autonomous-vehicle, and camera-pose variants, see the Cosmos 3 vLLM-Omni recipes.

Example video request:

curl -sS -X POST http://localhost:8000/v1/videos/sync \
  --form-string "prompt=A small warehouse robot moves a blue box across a clean floor." \
  --form-string "negative_prompt=blurry, distorted, low quality" \
  --form-string "size=1280x720" \
  --form-string "num_frames=189" \
  --form-string "fps=24" \
  --form-string "num_inference_steps=35" \
  --form-string "guidance_scale=6.0" \
  --form-string "flow_shift=10.0" \
  --form-string "seed=0" \
  --form-string 'extra_params={"use_resolution_template":false,"use_duration_template":false,"guardrails":true}' \
  -o cosmos3_t2v_output.mp4

Example video-to-video request:

curl -sS -X POST http://localhost:8000/v1/videos/sync \
  -H "Accept: video/mp4" \
  --form-string "prompt=Continue the same driving scene with smooth natural motion." \
  --form-string "negative_prompt=blurry, distorted, low quality, jittery, deformed" \
  --form-string "size=832x480" \
  --form-string "num_frames=61" \
  --form-string "fps=10" \
  --form-string "num_inference_steps=35" \
  --form-string "guidance_scale=6.0" \
  --form-string "flow_shift=10.0" \
  --form-string "seed=2222" \
  --form-string 'extra_params={"use_resolution_template":false,"use_duration_template":false,"guardrails":true,"condition_frame_indexes_vision":[0,1],"condition_video_keep":"first"}' \
  -F "input_reference=@cookbooks/cosmos3/generator/action/assets/videos/av_0.mp4;type=video/mp4" \
  -o cosmos3_v2v_output.mp4

Example transfer-control request:

curl -sS -X POST http://localhost:8000/v1/videos/sync \
  -H "Accept: video/mp4" \
  --form-string "prompt=Generate a realistic scene following the provided depth control video." \
  --form-string "negative_prompt=blurry, distorted, low quality" \
  --form-string "size=1280x720" \
  --form-string "num_frames=121" \
  --form-string "fps=30" \
  --form-string "num_inference_steps=50" \
  --form-string "guidance_scale=3.0" \
  --form-string "flow_shift=10.0" \
  --form-string "seed=2026" \
  --form-string 'extra_params={"use_resolution_template":false,"use_duration_template":false,"guardrails":true,"depth":{"control_path":"cookbooks/cosmos3/generator/transfer/assets/depth/control_depth.mp4"},"resolution":"720","control_guidance":1.5,"num_video_frames_per_chunk":121,"max_frames":121}' \
  -o cosmos3_transfer_depth.mp4

Use --form-string for text fields (prompt, negative_prompt, extra_params) rather than -F: with -F, curl treats ; as a content-type separator and silently truncates any value that contains one.

Common request fields (the image endpoint follows the Image Generation API, and the video endpoints follow the Videos API):

Disabling guardrails: Cosmos 3 ships safety guardrails that screen prompts and blur faces in generated output. Disable them per request by adding guardrails: false to extra_params:

curl -sS -X POST http://localhost:8000/v1/videos/sync \
  --form-string "prompt=A small warehouse robot moves a blue box across a clean floor." \
  --form-string 'extra_params={"guardrails":false,"use_resolution_template":false,"use_duration_template":false}' \
  -o cosmos3_t2v.mp4

To disable guardrails server-wide so the guardrail models are never loaded (per-request overrides then cannot turn them back on), pass a deploy config — a future release replaces this with a dedicated --cosmos3-no-guardrails flag:

# no_guardrails.yaml
async_chunk: false
stages:
  - stage_id: 0
    max_num_seqs: 1
    enforce_eager: true
    trust_remote_code: true
    model_class_name: Cosmos3OmniDiffusersPipeline
    model_config:
      guardrails: false
      offload_guardrail_models: false

vllm serve nvidia/Cosmos3-Nano --omni \
  --model-class-name Cosmos3OmniDiffusersPipeline \
  --deploy-config no_guardrails.yaml \
  --port 8000 \
  --init-timeout 1800

References:

• Cosmos 3 vLLM-Omni recipes
• vLLM-Omni Videos API
• vLLM-Omni Image Generation API

Use the Cosmos3-Generator NIM for turnkey Generator deployment through an NGC container. This NIM serves Text2Video and Image2Video only. It does not expose text-to-image, video-to-video, sound/audio generation, action modes, or transfer controls; use Generator with vLLM-Omni or Cosmos Framework for those broader Generator workflows.

The Generator NIM API differs from vLLM-Omni: send JSON requests to POST /v1/infer, and decode the JSON response field b64_video to get the MP4 bytes. The NIM infers the mode automatically from request fields:

Authenticate to NGC and launch the default Nano server:

export NGC_API_KEY=<your_key>
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

export LOCAL_NIM_CACHE="${LOCAL_NIM_CACHE:-$HOME/.cache/nim}"
mkdir -p "$LOCAL_NIM_CACHE"
chmod -R 777 "$LOCAL_NIM_CACHE" 2>/dev/null || true

docker run --runtime=nvidia --gpus all \
  --shm-size=32GB \
  --ulimit nofile=65536:65536 \
  -e NGC_API_KEY="$NGC_API_KEY" \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -p 8000:8000 \
  nvcr.io/nim/nvidia/cosmos3-generator:1.0.0

For the larger model, add -e NIM_MODEL_SIZE=super. The main launch-time knobs are NIM_MODEL_SIZE=nano|super (default nano), NIM_PRECISION=bf16|fp8|nvfp4 (default fp8; nvfp4 requires Blackwell), NIM_PERF_PROFILE=latency|throughput (default latency), and advanced NIM_TAGS_SELECTOR profile filters.

Wait for readiness:

curl -fsS http://127.0.0.1:8000/v1/health/ready

Send a Text2Video request and decode the MP4:

curl -sS -X POST http://127.0.0.1:8000/v1/infer \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "prompt": "A humanoid robot walks through a futuristic warehouse, inspecting shelves of mechanical components.",
    "seed": 42,
    "guidance_scale": 6.0,
    "steps": 35,
    "resolution": "256",
    "num_output_frames": 25,
    "fps": 24.0
  }' | jq -r '.b64_video' | base64 -d > cosmos3_generator_nim_t2v.mp4

For Image2Video, provide image as raw base64, a data:image/...;base64,... URI, or a public URL when URL inputs are enabled.

Request constraints include: guidance_scale in [1.0, 7.0], steps in [1, 100], num_output_frames on the 4k+1 cadence (25, 29, 33, ...) with per-tier caps (256 <= 397, 480 <= 297, 720 <= 197), and resolution keys 256, 480, 720 plus optional suffixes _16_9, _1_1, _9_16, _4_3, and _3_4.

See the Generator NIM cookbook for an end-to-end notebook that launches the container, polls readiness, inspects service metadata, runs T2V and I2V, decodes b64_video, and previews the generated MP4 files.

Use SGLang Diffusion for native Cosmos 3 visual generation behind OpenAI-compatible image and video APIs. Cosmos 3 also includes video-with-sound and action/policy models; this SGLang section focuses on the currently supported text-to-image, text-to-video, and image-to-video generator serving paths.

Supported checkpoints:

Install SGLang from the main branch with diffusion extras:

git clone --branch main https://github.com/sgl-project/sglang.git
cd sglang
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -e "python[diffusion]"
pip install "cosmos-guardrail==0.3.1"

Version note: Cosmos 3 support in SGLang Diffusion currently requires the SGLang main branch. Switch to a stable SGLang release once Cosmos 3 support is included there.

Start a Nano server:

sglang serve --model-path nvidia/Cosmos3-Nano

For a video-specialized checkpoint, use Cosmos3-Super-Image2Video with multiple GPUs:

sglang serve \
  --model-path nvidia/Cosmos3-Super-Image2Video \
  --num-gpus 4

This is the performance-mode setup. If it runs out of memory, switch to SGLang Diffusion's memory preset:

sglang serve \
  --model-path nvidia/Cosmos3-Super-Image2Video \
  --num-gpus 4 \
  --performance-mode memory

Vision endpoints:

Action modes use Cosmos 3 as a world model: they condition on an embodiment (domain_name) and exchange video and action sequences. Policy and inverse dynamics return a predicted action chunk, and read the action data from the completed result; forward dynamics returns only video.

Pass embodiment settings through extra_params: action_mode, domain_name (for example bridge_orig_lerobot, av, or camera_pose), raw_action_dim, and optionally action_view_point. SGLang derives the action chunk length from num_frames - 1, so set num_frames to action_chunk_size + 1. For forward dynamics, pass the action trajectory directly in extra_params["action"] as a JSON array of shape [action_chunk_size, raw_action_dim]. SGLang does not use action_path for HTTP requests, so no --allowed-local-media-path setup is needed for action files.

Text-to-video example:

# Submit an async video generation job and capture its ID.
job_id=$(curl -sS -X POST http://localhost:30000/v1/videos \
  --form-string "prompt=A small warehouse robot moves a blue box across a clean floor." \
  --form-string "negative_prompt=blurry, distorted, low quality" \
  --form-string "size=1280x720" \
  --form-string "num_frames=81" \
  --form-string "fps=24" \
  --form-string "num_inference_steps=35" \
  --form-string "guidance_scale=4.0" \
  --form-string "flow_shift=10.0" \
  --form-string "seed=42" \
  --form-string 'extra_params={"guardrails":true,"use_resolution_template":false,"use_duration_template":false}' \
  | jq -r .id)

# Poll until the job completes. Cosmos 3 video generation can take several minutes.
status=""
until [ "$status" = "completed" ]; do
  status=$(curl -sS "http://localhost:30000/v1/videos/${job_id}" | jq -r .status)
  [ "$status" = "failed" ] && exit 1
  sleep 5
done

# Download the completed MP4.
curl -sS -L "http://localhost:30000/v1/videos/${job_id}/content" \
  -o cosmos3_t2v_output.mp4

Text-to-image example:

curl -sS -X POST http://localhost:30000/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A warehouse robot folds a blue cloth on a clean workbench.",
    "size": "1280x720",
    "n": 1,
    "num_inference_steps": 35,
    "guidance_scale": 6.0,
    "flow_shift": 10.0,
    "seed": 0,
    "extra_args": {
      "use_resolution_template": false,
      "guardrails": true
    }
  }'

Video-to-video-with-sound example:

job_id=$(curl -sS --fail-with-body -X POST "http://localhost:30000/v1/videos" \
  -H "Accept: application/json" \
  --form-string 'prompt=A small warehouse robot moves a blue box across a clean floor.' \
  --form-string 'negative_prompt=blurry, distorted, low quality' \
  --form-string 'size=1280x720' \
  --form-string 'num_frames=61' \
  --form-string 'fps=24' \
  --form-string 'num_inference_steps=30' \
  --form-string 'guidance_scale=4.0' \
  --form-string 'flow_shift=10.0' \
  --form-string 'seed=1234' \
  --form-string 'generate_sound=true' \
  --form-string 'extra_params={"use_resolution_template":false,"use_duration_template":false,"guardrails":true}' \
  -F 'video_reference=@/path/to/video.mp4;type=video/mp4' \
  | jq -r .id)

# Poll until the job completes. Cosmos 3 video generation can take several minutes.
status=""
until [ "$status" = "completed" ]; do
  status=$(curl -sS "http://localhost:30000/v1/videos/${job_id}" | jq -r .status)
  [ "$status" = "failed" ] && exit 1
  sleep 5
done

# Download the completed MP4.
curl -sS -L "http://localhost:30000/v1/videos/${job_id}/content" \
  -o cosmos3_v2vs_output.mp4

SGLang accepts Cosmos 3 request options including max_sequence_length, flow_shift, extra_params.guardrails, extra_params.use_resolution_template, and extra_params.use_duration_template. Guardrails are enabled by default when cosmos-guardrail is installed; set SGLANG_DISABLE_COSMOS3_GUARDRAILS=1 before starting the server to skip loading the guardrail models.

For complete serving instructions and request examples, see the Cosmos3 SGLang cookbook.

Use Hugging Face Transformers for Python-first Reasoner inference. This path loads only the Reasoner tower from the unified nvidia/Cosmos3-Nano or nvidia/Cosmos3-Super checkpoint and returns text from text, image, or video inputs. It does not load the Generator diffusion, audio, or action heads; use Generator with Diffusers, Generator with vLLM-Omni, or Generator with NIM for supported non-text outputs.

Cosmos3 support first appears in the Transformers v5.11.0 release tag. Install Transformers 5.11.0 or newer:

uv venv --python 3.13 --seed --managed-python
source .venv/bin/activate
uv pip install --torch-backend=auto \
  accelerate \
  av \
  pillow \
  "safetensors>=0.8.0" \
  torch \
  "torchvision==0.25.0" \
  "transformers>=5.11.0"

--torch-backend=auto lets uv pick a CUDA build that matches your driver. Pin an explicit backend instead if needed, for example --torch-backend=cu128 for a CUDA 12.8 driver.

Run an image reasoning request:

from pathlib import Path

import torch
from transformers import AutoProcessor, Cosmos3OmniForConditionalGeneration

model_id = "nvidia/Cosmos3-Nano"
image_path = Path("cookbooks/cosmos3/reasoner/assets/robot_153.jpg").resolve()

processor = AutoProcessor.from_pretrained(model_id)
model = Cosmos3OmniForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "path": str(image_path)},
            {"type": "text", "text": "Caption the image in detail."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, torch.bfloat16)

generated_ids = model.generate(**inputs, do_sample=False, max_new_tokens=512)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)
print(output[0])

For video reasoning, use a video content block and pass a frame sampling rate to apply_chat_template:

messages = [
    {
        "role": "user",
        "content": [
            {"type": "video", "path": "cookbooks/cosmos3/reasoner/assets/video_caption.mp4"},
            {"type": "text", "text": "Describe the notable events in this video."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    fps=2,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, torch.bfloat16)

Then reuse the model.generate and batch_decode block from the image example.

For nvidia/Cosmos3-Super, change model_id to nvidia/Cosmos3-Super. device_map="auto" can shard the model across multiple GPUs when Accelerate is installed. For an OpenAI-compatible server, use Reasoner with vLLM or Reasoner with NIM.

Use vLLM for Reasoner production inference behind an OpenAI-compatible chat-completions API. This path loads only the reasoner; for generation tasks that return images or video, use Generator with vLLM-Omni, or Generator with NIM for turnkey T2V/I2V video generation only.

uv venv --python 3.13 --seed --managed-python
source .venv/bin/activate
uv pip install --torch-backend=auto "vllm>=0.23.0"

Or use vllm/vllm-openai:v0.23.0 docker image.

vllm serve nvidia/Cosmos3-Nano \
  --async-scheduling \
  --allowed-local-media-path / \
  --port 8000

For notebook launch commands (Cosmos3-Super on four GPUs, media-path defaults, and full flag sets), see cookbooks/cosmos3/README.md — Start the server.

If your vLLM build reports that DeepGEMM is unavailable, disable it before starting the server:

export VLLM_USE_DEEP_GEMM=0

Configuration notes:

Use the Cosmos 3 Reasoner NIM for the fastest path to a production-grade, OpenAI-compatible Reasoner endpoint. NIM ships a prebuilt, optimized container so you skip the vLLM dependency and CUDA-pairing setup above; it serves text outputs from text, image, and video inputs.

You can try it interactively in your browser on the cosmos3-nano-reasoner build page — that playground is powered by this same NIM. See the Cosmos Reason 3 NIM API reference for the full request reference.

The container serves two sizes, selected with NIM_MODEL_SIZE:

Launch the NIM container (Nano shown; set -e NIM_MODEL_SIZE=super for Super). NGC_API_KEY must be set in your environment first — generate one from NGC and log Docker in to nvcr.io once (docker login nvcr.io, username $oauthtoken, password = your key).

export CONTAINER_NAME="nvidia-cosmos3-reasoner"
export IMG_NAME="nvcr.io/nim/nvidia/cosmos3-reasoner:1.7.0"
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"

docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus all \
  --shm-size=32GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MODEL_SIZE=nano \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -u $(id -u) \
  -p 8000:8000 \
  $IMG_NAME

The OpenAI-compatible API is then available at http://127.0.0.1:8000/v1. Query it with curl:

IMAGE_DATA_URI="data:image/jpeg;base64,$(base64 -w 0 cookbooks/cosmos3/reasoner/assets/robot_153.jpg)"

curl -X POST 'http://127.0.0.1:8000/v1/chat/completions' \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  --data-binary @- <<JSON
{
    "model": "nvidia/cosmos3-nano-reasoner",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "$IMAGE_DATA_URI"}},
        {"type": "text", "text": "Describe what is happening in this image in one sentence."}
      ]}
    ],
    "max_tokens": 256,
    "stream": false
  }
JSON

Or with the OpenAI Python client:

from openai import OpenAI

# The container exposes the OpenAI-compatible API locally; the api_key is unused.
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="not-used")

response = client.chat.completions.create(
    model="nvidia/cosmos3-nano-reasoner",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {
            "role": "user",
            "content": [
                {"type": "video_url", "video_url": {"url": "https://download.samplelib.com/mp4/sample-5s.mp4"}},
                {"type": "text", "text": "List the notable events with approximate timestamps."},
            ],
        },
    ],
    max_tokens=256,
    stream=False,
    extra_body={"media_io_kwargs": {"video": {"fps": 4.0}}},
)
print(response.choices[0].message.content)

Inputs and request notes:

References:

• Cosmos Reason 3 NIM API reference
• cosmos3-reasoner NGC container
• cosmos3-nano-reasoner build page