An open-source entry in TopGit's GitHub warehouse: LiheYoung/Depth-Anything, 8.3k stars, Python. [CVPR 2024] Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. Foundation Model for Monocular Depth Estimation
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
This work presents Depth Anything, a highly practical solution for robust monocular depth estimation by training on a combination of 1.5M labeled images and 62M+ unlabeled images.
Try our latest Depth Anything V2 models!
News
2024-06-14: Depth Anything V2 is released.
2024-02-27: Depth Anything is accepted by CVPR 2024.
2024-02-05: Depth Anything Gallery is released. Thank all the users!
2024-02-02: Depth Anything serves as the default depth processor for InstantID and InvokeAI.
2024-01-25: Support video depth visualization. An online demo for video is also available.
2024-01-23: The new ControlNet based on Depth Anything is integrated into ControlNet WebUI and ComfyUI's ControlNet.
2024-01-23: Depth Anything ONNX and TensorRT versions are supported.
2024-01-22: Paper, project page, code, models, and demo (HuggingFace, OpenXLab) are released.
Features of Depth Anything
If you need other features, please first check existing community supports.
Relative depth estimation:
Our foundation models listed here can provide relative depth estimation for any given image robustly. Please refer here for details.
Metric depth estimation
We fine-tune our Depth Anything model with metric depth information from NYUv2 or KITTI. It offers strong capabilities of both in-domain and zero-shot metric depth estimation. Please refer here for details.
Better depth-conditioned ControlNet
We re-train a better depth-conditioned ControlNet based on Depth Anything. It offers more precise synthesis than the previous MiDaS-based ControlNet. Please refer here for details. You can also use our new ControlNet based on Depth Anything in ControlNet WebUI or ComfyUI's ControlNet.
Downstream high-level scene understanding
The Depth Anything encoder can be fine-tuned to downstream high-level perception tasks, e.g., semantic segmentation, 86.2 mIoU on Cityscapes and 59.4 mIoU on ADE20K. Please refer here for details.
Performance
Here we compare our Depth Anything with the previously best MiDaS v3.1 BEiTL-512 model.
Please note that the latest MiDaS is also trained on KITTI and NYUv2, while we do not.
Method
Params
KITTI
NYUv2
Sintel
DDAD
ETH3D
DIODE
AbsRel
$\delta_1$
AbsRel
$\delta_1$
AbsRel
$\delta_1$
AbsRel
$\delta_1$
AbsRel
$\delta_1$
AbsRel
$\delta_1$
MiDaS
345.0M
0.127
0.850
0.048
0.980
0.587
0.699
0.251
0.766
0.139
0.867
0.075
0.942
Ours-S
24.8M
0.080
0.936
0.053
0.972
0.464
0.739
0.247
0.768
0.127
0.885
0.076
0.939
Ours-B
97.5M
0.080
0.939
0.046
0.979
0.432
0.756
0.232
0.786
0.126
0.884
0.069
0.946
Ours-L
335.3M
0.076
0.947
0.043
0.981
0.458
0.760
0.230
0.789
0.127
0.882
0.066
0.952
We highlight the best and second best results in bold and italic respectively (better results: AbsRel $\downarrow$ , $\delta_1 \uparrow$).
Pre-trained models
We provide three models of varying scales for robust relative depth estimation:
Model
Params
Inference Time on V100 (ms)
A100
RTX4090 (TensorRT)
Depth-Anything-Small
24.8M
12
8
3
Depth-Anything-Base
97.5M
13
9
6
Depth-Anything-Large
335.3M
20
13
12
Note that the V100 and A100 inference time (without TensorRT) is computed by excluding the pre-processing and post-processing stages, whereas the last column RTX4090 (with TensorRT) is computed by including these two stages (please refer to Depth-Anything-TensorRT).
You can easily load our pre-trained models by:
from depth_anything.dpt import DepthAnything
encoder = 'vits' # can also be 'vitb' or 'vitl'
depth_anything = DepthAnything.from_pretrained('LiheYoung/depth_anything_{:}14'.format(encoder))
Depth Anything is also supported in transformers. You can use it for depth prediction within 3 lines of code (credit to @niels).
No network connection, cannot load these models?
Click here for solutions
First, manually download the three checkpoints: depth-anything-large, depth-anything-base, and depth-anything-small.
Second, upload the folder containing the checkpoints to your remote server.
Note that in this locally loading manner, you also do not have to install the huggingface_hub package. In this way, please feel free to delete this line and the PyTorchModelHubMixin in this line.
Usage
Installation
git clone https://github.com/LiheYoung/Depth-Anything
cd Depth-Anything
pip install -r requirements.txt
--img-path: you can either 1) point it to an image directory storing all interested images, 2) point it to a single image, or 3) point it to a text file storing all image paths.
--pred-only is set to save the predicted depth map only. Without it, by default, we visualize both image and its depth map side by side.
--grayscale is set to save the grayscale depth map. Without it, by default, we apply a color palette to the depth map.
Depth Anything in TouchDesigner: https://github.com/olegchomp/TDDepthAnything
LearnOpenCV research article on Depth Anything: https://learnopencv.com/depth-anything
Learn more about the DPT architecture we used: https://github.com/heyoeyo/muggled_dpt
Depth Anything in NVIDIA Jetson Orin: https://github.com/ZhuYaoHui1998/jetson-examples/blob/main/reComputer/scripts/depth-anything
If you have your amazing projects supporting or improving (e.g., speed) Depth Anything, please feel free to drop an issue. We will add them here.
Acknowledgement
We would like to express our deepest gratitude to AK(@_akhaliq) and the awesome HuggingFace team (@niels, @hysts, and @yuvraj) for helping improve the online demo and build the HF models.
Besides, we thank the MagicEdit team for providing some video examples for video depth estimation, and Tiancheng Shen for evaluating the depth maps with MagicEdit.
Citation
If you find this project useful, please consider citing:
@inproceedings{depthanything,
title={Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data},
author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
booktitle={CVPR},
year={2024}
}
How active is development on LiheYoung/Depth-Anything?
The most recent commit recorded on LiheYoung/Depth-Anything was 2.3 years ago, based on the GitHub push timestamp. The repository has 621 forks — one of the better signals of community interest.
How many stars does LiheYoung/Depth-Anything have?
LiheYoung/Depth-Anything has 8.3k GitHub stars — refresh the page for the live number, or check github.com/LiheYoung/Depth-Anything. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
Is LiheYoung/Depth-Anything open source?
Yes — LiheYoung/Depth-Anything ships under the Apache-2.0 license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/LiheYoung/Depth-Anything.
What is LiheYoung/Depth-Anything?
LiheYoung/Depth-Anything (LiheYoung/Depth-Anything) is a Python project on GitHub. From the project's own README: [CVPR 2024] Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. Foundation Model for Monocular Depth Estimation
What language is LiheYoung/Depth-Anything written in?
LiheYoung/Depth-Anything is written primarily in Python. GitHub's language field is based on the largest share of bytes in the default branch.
What license does LiheYoung/Depth-Anything use?
LiheYoung/Depth-Anything is released under the Apache-2.0 license. Always verify the LICENSE file directly on GitHub for the authoritative terms — license strings can be edited out of sync with a project's actual stance.
Where do I read more about LiheYoung/Depth-Anything?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/LiheYoung/Depth-Anything is the definitive source.
Read full README in the tab above.
Is Depth-Anything worth your time?
ChatGPT, Claude and Perplexity can all read this page. Ask one of them what it makes of Depth-Anything.