krahets/sam-3d-body

SAM 3D

SAM 3D Body is one part of SAM 3D, a pair of models for object and human mesh reconstruction. If you’re looking for SAM 3D Objects, click here.

SAM 3D Body: Robust Full-Body Human Mesh Recovery

Xitong Yang*, Devansh Kukreja*, Don Pinkus*, Anushka Sagar, Taosha Fan, Jinhyung Park⚬, Soyong Shin⚬, Jinkun Cao, Jiawei Liu, Nicolas Ugrinovic, Matt Feiszli†, Jitendra Malik†, Piotr Dollar†, Kris Kitani†

Meta Superintelligence Labs

*Core Contributor, ⚬Intern, †Project Lead

SAM 3D Body (3DB) is a promptable model for single-image full-body 3D human mesh recovery (HMR). Our method demonstrates state-of-the-art performance, with strong generalization and consistent accuracy in diverse in-the-wild conditions. 3DB estimates the human pose of the body, feet, and hands based on the Momentum Human Rig (MHR), a new parametric mesh representation that decouples skeletal structure and surface shape for improved accuracy and interpretability.

3DB employs an encoder-decoder architecture and supports auxiliary prompts, including 2D keypoints and masks, enabling user-guided inference similar to the SAM family of models. Our model is trained on high-quality annotations from a multi-stage annotation pipeline using differentiable optimization, multi-view geometry, dense keypoint detection, and a data engine to collect and annotated data covering both common and rare poses across a wide range of viewpoints.

Qualitative Results

Our SAM 3D Body demonstrates superior reconstruction quality with more accurate pose estimation, better shape recovery, and improved handling of occlusions and challenging viewpoints compared to existing approaches.

Latest updates

11/19/2025 -- Checkpoints Launched, Dataset Released, Web Demo and Paper are out!

Installation

See INSTALL.md for instructions for python environment setup and model checkpoint access.

Getting Started

3DB can reconstruct 3D full-body human mesh from a single image, optionally with keypoint/mask prompts and/or hand refinement from the hand decoder.

For a quick start, run our demo script for model inference and visualization with models from Hugging Face (please make sure to follow INSTALL.md to request access to our checkpoints.).

# Download assets from HuggingFace
hf download facebook/sam-3d-body-dinov3 --local-dir checkpoints/sam-3d-body-dinov3

# Run demo script
python demo.py \
    --image_folder <path_to_images> \
    --output_folder <path_to_output> \
    --checkpoint_path ./checkpoints/sam-3d-body-dinov3/model.ckpt \
    --mhr_path ./checkpoints/sam-3d-body-dinov3/assets/mhr_model.pt

You can also try the following lines of code with models loaded directly from Hugging Face

import cv2
import numpy as np
from notebook.utils import setup_sam_3d_body
from tools.vis_utils import visualize_sample_together

# Set up the estimator
estimator = setup_sam_3d_body(hf_repo_id="facebook/sam-3d-body-dinov3")

# Load and process image
img_bgr = cv2.imread("path/to/image.jpg")
outputs = estimator.process_one_image(cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB))

# Visualize and save results
rend_img = visualize_sample_together(img_bgr, outputs, estimator.faces)
cv2.imwrite("output.jpg", rend_img.astype(np.uint8))

For a complete demo with visualization, see notebook/demo_human.ipynb.

Model Description

SAM 3D Body checkpoints

The table below shows the performance of SAM 3D Body checkpoints released on 11/19/2025.

SAM 3D Body Dataset

The SAM 3D Body data is released on Hugging Face. Please follow the instructions to download and process the data.

SAM 3D Objects

SAM 3D Objects is a foundation model that reconstructs full 3D shape geometry, texture, and layout from a single image.

As a way to combine the strengths of both SAM 3D Objects and SAM 3D Body, we provide an example notebook that demonstrates how to combine the results of both models such that they are aligned in the same frame of reference. Check it out here.

License

The SAM 3D Body model checkpoints and code are licensed under SAM License.

Contributing

See contributing and the code of conduct.

Contributors

The SAM 3D Body project was made possible with the help of many contributors: Vivian Lee, George Orlin, Nikhila Ravi, Andrew Westbury, Jyun-Ting Song, Zejia Weng, Xizi Zhang, Yuting Ye, Federica Bogo, Ronald Mallet, Ahmed Osman, Rawal Khirodkar, Javier Romero, Carsten Stoll, Jean-Charles Bazin, Sofien Bouaziz, Yuan Dong, Su Zhaoen, Fabian Prada, Alexander Richard, Michael Zollhoefer, Roman Rädle, Sasha Mitts, Michelle Chan, Yael Yungster, Azita Shokrpour, Helen Klein, Mallika Malhotra, Ida Cheng, Eva Galper.

Citing SAM 3D Body

If you use SAM 3D Body or the SAM 3D Body dataset in your research, please use the following BibTeX entry.

@article{yang2025sam3dbody,
  title={SAM 3D Body: Robust Full-Body Human Mesh Recovery},
  author={Yang, Xitong and Kukreja, Devansh and Pinkus, Don and Sagar, Anushka and Fan, Taosha and Park, Jinhyung and Shin, Soyong and Cao, Jinkun and Liu, Jiawei and Ugrinovic, Nicolas and Feiszli, Matt and Malik, Jitendra and Dollar, Piotr and Kitani, Kris},
  journal={arXiv preprint; identifier to be added},
  year={2025}
}