Là một dự án mã nguồn mở, facebookresearch/detr đã đạt 15.4k sao trên GitHub, ngôn ngữ Python. End-to-End Object Detection with Transformers
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
DE⫶TR: End-to-End Object Detection with Transformers
PyTorch training code and pretrained models for DETR (DEtection TRansformer).
We replace the full complex hand-crafted object detection pipeline with a Transformer, and match Faster R-CNN with a ResNet-50, obtaining 42 AP on COCO using half the computation power (FLOPs) and the same number of parameters. Inference in 50 lines of PyTorch.
What it is. Unlike traditional computer vision techniques, DETR approaches object detection as a direct set prediction problem. It consists of a set-based global loss, which forces unique predictions via bipartite matching, and a Transformer encoder-decoder architecture.
Given a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. Due to this parallel nature, DETR is very fast and efficient.
About the code. We believe that object detection should not be more difficult than classification,
and should not require complex libraries for training and inference.
DETR is very simple to implement and experiment with, and we provide a
standalone Colab Notebook
showing how to do inference with DETR in only a few lines of PyTorch code.
Training code follows this idea - it is not a library,
but simply a main.py importing model and criterion
definitions with standard training loops.
Additionnally, we provide a Detectron2 wrapper in the d2/ folder. See the readme there for more information.
For details see End-to-End Object Detection with Transformers by Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.
See our blog post to learn more about end to end object detection with transformers.
Model Zoo
We provide baseline DETR and DETR-DC5 models, and plan to include more in future.
AP is computed on COCO 2017 val5k, and inference time is over the first 100 val5k COCO images,
with torchscript transformer.
name
backbone
schedule
inf_time
box AP
url
size
0
DETR
R50
500
0.036
42.0
model | logs
159Mb
1
DETR-DC5
R50
500
0.083
43.3
model | logs
159Mb
2
DETR
R101
500
0.050
43.5
model | logs
232Mb
3
DETR-DC5
R101
500
0.097
44.9
model | logs
232Mb
COCO val5k evaluation results can be found in this gist.
The models are also available via torch hub,
to load DETR R50 with pretrained weights simply do:
model = torch.hub.load('facebookresearch/detr:main', 'detr_resnet50', pretrained=True)
COCO panoptic val5k models:
name
backbone
box AP
segm AP
PQ
url
size
0
DETR
R50
38.8
31.1
43.4
download
165Mb
1
DETR-DC5
R50
40.2
31.9
44.6
download
165Mb
2
DETR
R101
40.1
33
45.1
download
237Mb
Checkout our panoptic colab
to see how to use and visualize DETR's panoptic segmentation prediction.
Notebooks
We provide a few notebooks in colab to help you get a grasp on DETR:
DETR's hands on Colab Notebook: Shows how to load a model from hub, generate predictions, then visualize the attention of the model (similar to the figures of the paper)
Standalone Colab Notebook: In this notebook, we demonstrate how to implement a simplified version of DETR from the grounds up in 50 lines of Python, then visualize the predictions. It is a good starting point if you want to gain better understanding the architecture and poke around before diving in the codebase.
Panoptic Colab Notebook: Demonstrates how to use DETR for panoptic segmentation and plot the predictions.
Usage - Object detection
There are no extra compiled components in DETR and package dependencies are minimal,
so the code is very simple to use. We provide instructions how to install dependencies via conda.
First, clone the repository locally:
Download and extract COCO 2017 train and val images with annotations from
http://cocodataset.org.
We expect the directory structure to be the following:
A single epoch takes 28 minutes, so 300 epoch training
takes around 6 days on a single machine with 8 V100 cards.
To ease reproduction of our results we provide
results and training logs
for 150 epoch schedule (3 days on a single machine), achieving 39.5/60.3 AP/AP50.
We train DETR with AdamW setting learning rate in the transformer to 1e-4 and 1e-5 in the backbone.
Horizontal flips, scales and crops are used for augmentation.
Images are rescaled to have min size 800 and max size 1333.
The transformer is trained with dropout of 0.1, and the whole model is trained with grad clip of 0.1.
Evaluation
To evaluate DETR R50 on COCO val5k with a single GPU run:
We provide results for all DETR detection models in this
gist.
Note that numbers vary depending on batch size (number of images) per GPU.
Non-DC5 models were trained with batch size 2, and DC5 with 1,
so DC5 models show a significant drop in AP if evaluated with more
than 1 image per GPU.
Multinode training
Distributed training is available via Slurm and submitit:
pip install submitit
Train baseline DETR-6-6 model on 4 nodes for 300 epochs:
We show that it is relatively straightforward to extend DETR to predict segmentation masks. We mainly demonstrate strong panoptic segmentation results.
Data preparation
For panoptic segmentation, you need the panoptic annotations additionally to the coco dataset (see above for the coco dataset). You need to download and extract the annotations.
We expect the directory structure to be the following:
We recommend training segmentation in two stages: first train DETR to detect all the boxes, and then train the segmentation head.
For panoptic segmentation, DETR must learn to detect boxes for both stuff and things classes. You can train it on a single node with 8 gpus for 300 epochs with:
For instance segmentation, you can simply train a normal box model (or used a pre-trained one we provide).
Once you have a box model checkpoint, you need to freeze it, and train the segmentation head in isolation.
For panoptic segmentation you can train on a single node with 8 gpus for 25 epochs:
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/facebookresearch/detr là nguồn chính thức.
facebookresearch/detr có bao nhiêu sao?
facebookresearch/detr có 15.4k sao GitHub — tải lại trang để xem số mới nhất, hoặc xem trực tiếp github.com/facebookresearch/detr. TopGit phản chiếu số sao của GitHub nhưng không cam kết đến từng phút.
facebookresearch/detr có phải mã nguồn mở không?
Có — facebookresearch/detr phát hành theo license Apache-2.0, nghĩa là mã nguồn mở để đọc, fork và (tùy license) tái sử dụng. Mã: github.com/facebookresearch/detr.
facebookresearch/detr còn đang phát triển không?
Commit gần nhất trên facebookresearch/detr là 2.7 năm trước (theo timestamp GitHub). Repo có 2.7k fork — một chỉ báo về mức độ quan tâm của cộng đồng.
facebookresearch/detr dùng license gì?
facebookresearch/detr phát hành theo license Apache-2.0. Nên mở file LICENSE trên GitHub để xác nhận — license metadata đôi khi lệch với thực tế dự án.
facebookresearch/detr là gì?
facebookresearch/detr (facebookresearch/detr) là dự án Python trên GitHub. Theo mô tả gốc: End-to-End Object Detection with Transformers
facebookresearch/detr viết bằng ngôn ngữ gì?
facebookresearch/detr chủ yếu viết bằng Python. Trường "language" của GitHub dựa trên phần lớn byte ở nhánh mặc định.
Đọc đầy đủ README ở tab phía trên.
detr có đáng để bạn bỏ thời gian?
ChatGPT, Claude và Perplexity đều đọc được trang này. Hỏi thử xem họ nghĩ gì về detr.