4.9k sao GitHub và vẫn tăng — liuzhuang13/DenseNet là dự án Lua mà TopGit đang theo dõi trên nền tảng. Densely Connected Convolutional Networks, In CVPR 2017 (Best Paper Award).
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
This repository contains the code for DenseNet introduced in the following paper
Densely Connected Convolutional Networks (CVPR 2017, Best Paper Award)
Gao Huang*, Zhuang Liu*, Laurens van der Maaten and Kilian Weinberger (* Authors contributed equally).
Now with much more memory efficient implementation! Please check the technical report and code for more infomation.
The code is built on fb.resnet.torch.
Citation
If you find DenseNet useful in your research, please consider citing:
@inproceedings{DenseNet2017,
title={Densely connected convolutional networks},
author={Huang, Gao and Liu, Zhuang and van der Maaten, Laurens and Weinberger, Kilian Q },
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2017}
}
Other Implementations
Our [Caffe], Our memory-efficient [Caffe], Our memory-efficient [PyTorch],
[PyTorch] by Andreas Veit, [PyTorch] by Brandon Amos, [PyTorch] by Federico Baldassarre,
[MXNet] by Nicatio,
[MXNet] by Xiong Lin,
[MXNet] by miraclewkf,
[Tensorflow] by Yixuan Li,
[Tensorflow] by Laurent Mazare,
[Tensorflow] by Illarion Khlestov,
[Lasagne] by Jan Schlüter,
[Keras] by tdeboissiere,
[Keras] by Roberto de Moura Estevão Filho,
[Keras] by Somshubra Majumdar,
[Chainer] by Toshinori Hanya,
[Chainer] by Yasunori Kudo,
[Torch 3D-DenseNet] by Barry Kui,
[Keras] by Christopher Masch,
[Tensorflow2] by Gaston Rios and Ulises Jeremias Cornejo Fandos.
Note that we only listed some early implementations here. If you would like to add yours, please submit a pull request.
Some Following up Projects
Multi-Scale Dense Convolutional Networks for Efficient Prediction
DSOD: Learning Deeply Supervised Object Detectors from Scratch
CondenseNet: An Efficient DenseNet using Learned Group Convolutions
Fully Convolutional DenseNets for Semantic Segmentation
Pelee: A Real-Time Object Detection System on Mobile Devices
Contents
Introduction
Usage
Results on CIFAR
Results on ImageNet and Pretrained Models
Updates
Introduction
DenseNet is a network architecture where each layer is directly connected to every other layer in a feed-forward fashion (within each dense block). For each layer, the feature maps of all preceding layers are treated as separate inputs whereas its own feature maps are passed on as inputs to all subsequent layers. This connectivity pattern yields state-of-the-art accuracies on CIFAR10/100 (with or without data augmentation) and SVHN. On the large scale ILSVRC 2012 (ImageNet) dataset, DenseNet achieves a similar accuracy as ResNet, but using less than half the amount of parameters and roughly half the number of FLOPs.
Figure 1: A dense block with 5 layers and growth rate 4.
Figure 2: A deep DenseNet with three dense blocks.
Usage
Install Torch and required dependencies like cuDNN. See the instructions here for a step-by-step guide.
Clone this repo: git clone https://github.com/liuzhuang13/DenseNet.git
As an example, the following command trains a DenseNet-BC with depth L=100 and growth rate k=12 on CIFAR-10:
Please refer to fb.resnet.torch for data preparation.
DenseNet and DenseNet-BC
By default, the code runs with the DenseNet-BC architecture, which has 1x1 convolutional bottleneck layers, and compresses the number of channels at each transition layer by 0.5. To run with the original DenseNet, simply use the options -bottleneck false and -reduction 1
Memory efficient implementation (newly added feature on June 6, 2017)
There is an option -optMemory which is very useful for reducing GPU memory footprint when training a DenseNet. By default, the value is set to 2, which activates the shareGradInput function (with small modifications from here). There are two extreme memory efficient modes (-optMemory 3 or -optMemory 4) which use a customized densely connected layer. With -optMemory 4, the largest 190-layer DenseNet-BC on CIFAR can be trained on a single NVIDIA TitanX GPU (uses 8.3G of 12G) instead of fully using four GPUs with the standard (recursive concatenation) implementation .
More details about the memory efficient implementation are discussed here.
Results on CIFAR
The table below shows the results of DenseNets on CIFAR datasets. The "+" mark at the end denotes for standard data augmentation (random crop after zero-padding, and horizontal flip). For a DenseNet model, L denotes its depth and k denotes its growth rate. On CIFAR-10 and CIFAR-100 without data augmentation, a Dropout layer with drop rate 0.2 is introduced after each convolutional layer except the very first one.
Model
Parameters
CIFAR-10
CIFAR-10+
CIFAR-100
CIFAR-100+
DenseNet (L=40, k=12)
1.0M
7.00
5.24
27.55
24.42
DenseNet (L=100, k=12)
7.0M
5.77
4.10
23.79
20.20
DenseNet (L=100, k=24)
27.2M
5.83
3.74
23.42
19.25
DenseNet-BC (L=100, k=12)
0.8M
5.92
4.51
24.15
22.27
DenseNet-BC (L=250, k=24)
15.3M
5.19
3.62
19.64
17.60
DenseNet-BC (L=190, k=40)
25.6M
-
3.46
-
17.18
Results on ImageNet and Pretrained Models
Torch
Note: the pre-trained models in Torch are deprecated and no longer maintained. Please use PyTorch's pre-trained DenseNet models instead.
Models in the original paper
The Torch models are trained under the same setting as in fb.resnet.torch. The error rates shown are 224x224 1-crop test errors.
Network
Top-1 error
Torch Model
DenseNet-121 (k=32)
25.0
[Download (64.5MB)]
DenseNet-169 (k=32)
23.6
[Download (114.4MB)]
DenseNet-201 (k=32)
22.5
[Download (161.8MB)]
DenseNet-161 (k=48)
22.2
[Download (230.8MB)]
Models in the tech report
More accurate models trained with the memory efficient implementation in the technical report.
Network
Top-1 error
Torch Model
DenseNet-264 (k=32)
22.1
[Download (256MB)]
DenseNet-232 (k=48)
21.2
[Download (426MB)]
DenseNet-cosine-264 (k=32)
21.6
[Download (256MB)]
DenseNet-cosine-264 (k=48)
20.4
[Download (557MB)]
Caffe
https://github.com/shicai/DenseNet-Caffe.
PyTorch
PyTorch documentation on models. We would like to thank @gpleiss for this nice work in PyTorch.
Keras, Tensorflow and Theano
https://github.com/flyyufelix/DenseNet-Keras.
MXNet
https://github.com/miraclewkf/DenseNet.
Wide-DenseNet for better Time/Accuracy and Memory/Accuracy Tradeoff
If you use DenseNet as a model in your learning task, to reduce the memory and time consumption, we recommend use a wide and shallow DenseNet, following the strategy of wide residual networks. To obtain a wide DenseNet we set the depth to be smaller (e.g., L=40) and the growthRate to be larger (e.g., k=48).
We test a set of Wide-DenseNet-BCs and compared the memory and time with the DenseNet-BC (L=100, k=12) shown above. We obtained the statistics using a single TITAN X card, with batch size 64, and without any memory optimization.
Model
Parameters
CIFAR-10+
CIFAR-100+
Time per Iteration
Memory
DenseNet-BC (L=100, k=12)
0.8M
4.51
22.27
0.156s
5452MB
Wide-DenseNet-BC (L=40, k=36)
1.5M
4.58
22.30
0.130s
4008MB
Wide-DenseNet-BC (L=40, k=48)
2.7M
3.99
20.29
0.165s
5245MB
Wide-DenseNet-BC (L=40, k=60)
4.3M
4.01
19.99
0.223s
6508MB
Obersevations:
Wide-DenseNet-BC (L=40, k=36) uses less memory/time while achieves about the same accuracy as DenseNet-BC (L=100, k=12).
Wide-DenseNet-BC (L=40, k=48) uses about the same memory/time as DenseNet-BC (L=100, k=12), while is much more accurate.
Thus, for practical use, we suggest picking one model from those Wide-DenseNet-BCs.
Updates
08/23/2017:
Add supporting code, so one can simply git clone and run.
06/06/2017:
Support ultra memory efficient training of DenseNet with customized densely connected layer.
Support memory efficient training of DenseNet with standard densely connected layer (recursive concatenation) by fixing the shareGradInput function.
05/17/2017:
Add Wide-DenseNet.
Add keras, tf, theano link for pretrained models.
04/20/2017:
Add usage of models in PyTorch.
03/29/2017:
Add the code for imagenet training.
12/03/2016:
Add Imagenet results and pretrained models.
Add DenseNet-BC structures.
Contact
liuzhuangthu at gmail.com
Any discussions, suggestions and questions are welcome!
liuzhuang13/DenseNet thuộc nhóm AI Tools trên TopGit, cùng 1 topic GitHub. Trang Trending và Topics liệt kê các repo cùng số sao và cùng ngôn ngữ để so sánh.
Đọc thêm về liuzhuang13/DenseNet ở đâu?
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/liuzhuang13/DenseNet là nguồn chính thức.
liuzhuang13/DenseNet có phải mã nguồn mở không?
Có — liuzhuang13/DenseNet phát hành theo license BSD-3-Clause, nghĩa là mã nguồn mở để đọc, fork và (tùy license) tái sử dụng. Mã: github.com/liuzhuang13/DenseNet.
liuzhuang13/DenseNet là gì?
liuzhuang13/DenseNet (liuzhuang13/DenseNet) là dự án Lua trên GitHub. Theo mô tả gốc: Densely Connected Convolutional Networks, In CVPR 2017 (Best Paper Award).
liuzhuang13/DenseNet so với các dự án AI Tools khác thế nào?
liuzhuang13/DenseNet được TopGit xếp vào nhóm AI Tools, với 4.9k sao GitHub và viết bằng Lua. Xem trang chủ đề AI Tools trên TopGit để so sánh với các dự án tương tự theo số sao và mức độ hoạt động.
Vì sao liuzhuang13/DenseNet được xếp vào nhóm AI Tools?
TopGit xếp liuzhuang13/DenseNet vào nhóm AI Tools dựa trên GitHub topics và mô tả của repo (gắn thẻ: "deep-learning"). Việc phân loại dựa trên metadata thật của repo, không phải đoán theo cảm tính biên tập.
Đọc đầy đủ README ở tab phía trên.
Vẫn đang phân vân về DenseNet?
Một cú bấm sẽ gửi câu hỏi kèm trang này cho AI — xem AI nói gì về DenseNet.