victoresque/pytorch-template là dự án mã nguồn mở trên GitHub với 5.1k sao, viết chủ yếu bằng Python. PyTorch deep learning projects made easy.
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
tensorboard >= 1.14 (see Tensorboard Visualization)
Features
Clear folder structure which is suitable for many deep learning projects.
.json config file support for convenient parameter tuning.
Customizable command line options for more convenient parameter tuning.
Checkpoint saving and resuming.
Abstract base classes for faster development:
BaseTrainer handles checkpoint saving/resuming, training process logging, and more.
BaseDataLoader handles batch generation, data shuffling, and validation data splitting.
BaseModel provides basic model summary.
Folder Structure
pytorch-template/
│
├── train.py - main script to start training
├── test.py - evaluation of trained model
│
├── config.json - holds configuration for training
├── parse_config.py - class to handle config file and cli options
│
├── new_project.py - initialize new project with template files
│
├── base/ - abstract base classes
│ ├── base_data_loader.py
│ ├── base_model.py
│ └── base_trainer.py
│
├── data_loader/ - anything about data loading goes here
│ └── data_loaders.py
│
├── data/ - default directory for storing input data
│
├── model/ - models, losses, and metrics
│ ├── model.py
│ ├── metric.py
│ └── loss.py
│
├── saved/
│ ├── models/ - trained models are saved here
│ └── log/ - default logdir for tensorboard and logging output
│
├── trainer/ - trainers
│ └── trainer.py
│
├── logger/ - module for tensorboard visualization and logging
│ ├── visualization.py
│ ├── logger.py
│ └── logger_config.json
│
└── utils/ - small utility functions
├── util.py
└── ...
Usage
The code in this repo is an MNIST example of the template.
Try python train.py -c config.json to run code.
Config file format
Config files are in .json format:
{
"name": "Mnist_LeNet", // training session name
"n_gpu": 1, // number of GPUs to use for training.
"arch": {
"type": "MnistModel", // name of model architecture to train
"args": {
}
},
"data_loader": {
"type": "MnistDataLoader", // selecting data loader
"args":{
"data_dir": "data/", // dataset path
"batch_size": 64, // batch size
"shuffle": true, // shuffle training data before splitting
"validation_split": 0.1 // size of validation dataset. float(portion) or int(number of samples)
"num_workers": 2, // number of cpu processes to be used for data loading
}
},
"optimizer": {
"type": "Adam",
"args":{
"lr": 0.001, // learning rate
"weight_decay": 0, // (optional) weight decay
"amsgrad": true
}
},
"loss": "nll_loss", // loss
"metrics": [
"accuracy", "top_k_acc" // list of metrics to evaluate
],
"lr_scheduler": {
"type": "StepLR", // learning rate scheduler
"args":{
"step_size": 50,
"gamma": 0.1
}
},
"trainer": {
"epochs": 100, // number of training epochs
"save_dir": "saved/", // checkpoints are saved in save_dir/models/name
"save_freq": 1, // save checkpoints every save_freq epochs
"verbosity": 2, // 0: quiet, 1: per epoch, 2: full
"monitor": "min val_loss" // mode and metric for model performance monitoring. set 'off' to disable.
"early_stop": 10 // number of epochs to wait before early stop. set 0 to disable.
"tensorboard": true, // enable tensorboard visualization
}
}
Add addional configurations if you need.
Using config files
Modify the configurations in .json config files, then run:
python train.py --config config.json
Resuming from checkpoints
You can resume from a previously saved checkpoint by:
python train.py --resume path/to/checkpoint
Using Multiple GPU
You can enable multi-GPU training by setting n_gpu argument of the config file to larger number.
If configured to use smaller number of gpu than available, first n devices will be used by default.
Specify indices of available GPUs by cuda environmental variable.
Use the new_project.py script to make your new project directory with template files.
python new_project.py ../NewProject then a new project folder named 'NewProject' will be made.
This script will filter out unneccessary files like cache, git files or readme file.
Custom CLI options
Changing values of config file is a clean, safe and easy way of tuning hyperparameters. However, sometimes
it is better to have command line options if some values need to be changed too often or quickly.
This template uses the configurations stored in the json file by default, but by registering custom options as follows
you can change some of them using CLI flags.
# simple class-like object having 3 attributes, `flags`, `type`, `target`.
CustomArgs = collections.namedtuple('CustomArgs', 'flags type target')
options = [
CustomArgs(['--lr', '--learning_rate'], type=float, target=('optimizer', 'args', 'lr')),
CustomArgs(['--bs', '--batch_size'], type=int, target=('data_loader', 'args', 'batch_size'))
# options added here can be modified by command line flags.
]
target argument should be sequence of keys, which are used to access that option in the config dict. In this example, target
for the learning rate option is ('optimizer', 'args', 'lr') because config['optimizer']['args']['lr'] points to the learning rate.
python train.py -c config.json --bs 256 runs training with options given in config.json except for the batch size
which is increased to 256 by command line options.
Data Loader
Writing your own data loader
Inherit BaseDataLoader
BaseDataLoader is a subclass of torch.utils.data.DataLoader, you can use either of them.
BaseDataLoader handles:
Generating next batch
Data shuffling
Generating validation data loader by calling
BaseDataLoader.split_validation()
DataLoader Usage
BaseDataLoader is an iterator, to iterate through batches:
for batch_idx, (x_batch, y_batch) in data_loader:
pass
Example
Please refer to data_loader/data_loaders.py for an MNIST data loading example.
Trainer
Writing your own trainer
Inherit BaseTrainer
BaseTrainer handles:
Training process logging
Checkpoint saving
Checkpoint resuming
Reconfigurable performance monitoring for saving current best model, and early stop training.
If config monitor is set to max val_accuracy, which means then the trainer will save a checkpoint model_best.pth when validation accuracy of epoch replaces current maximum.
If config early_stop is set, training will be automatically terminated when model performance does not improve for given number of epochs. This feature can be turned off by passing 0 to the early_stop option, or just deleting the line of config.
Implementing abstract methods
You need to implement _train_epoch() for your training process, if you need validation then you can implement _valid_epoch() as in trainer/trainer.py
Example
Please refer to trainer/trainer.py for MNIST training.
Iteration-based training
Trainer.__init__ takes an optional argument, len_epoch which controls number of batches(steps) in each epoch.
Model
Writing your own model
Inherit BaseModel
BaseModel handles:
Inherited from torch.nn.Module
__str__: Modify native print function to prints the number of trainable parameters.
Implementing abstract methods
Implement the foward pass method forward()
Example
Please refer to model/model.py for a LeNet example.
Loss
Custom loss functions can be implemented in 'model/loss.py'. Use them by changing the name given in "loss" in config file, to corresponding name.
Metrics
Metric functions are located in 'model/metric.py'.
You can monitor multiple metrics by providing a list in the configuration file, e.g.:
"metrics": ["accuracy", "top_k_acc"],
Additional logging
If you have additional information to be logged, in _train_epoch() of your trainer class, merge them with log as shown below before returning:
You can test trained model by running test.py passing path to the trained checkpoint by --resume argument.
Validation data
To split validation data from a data loader, call BaseDataLoader.split_validation(), then it will return a data loader for validation of size specified in your config file.
The validation_split can be a ratio of validation set per total data(0.0 <= float < 1.0), or the number of samples (0 <= int < n_total_samples).
Note: the split_validation() method will modify the original data loader
Note: split_validation() will return None if "validation_split" is set to 0
Checkpoints
You can specify the name of the training session in config files:
"name": "MNIST_LeNet",
The checkpoints will be saved in save_dir/name/timestamp/checkpoint_epoch_n, with timestamp in mmdd_HHMMSS format.
A copy of config file will be saved in the same folder.
This template supports Tensorboard visualization by using either torch.utils.tensorboard or TensorboardX.
Install
If you are using pytorch 1.1 or higher, install tensorboard by 'pip install tensorboard>=1.14.0'.
Otherwise, you should install tensorboardx. Follow installation guide in TensorboardX.
Run training
Make sure that tensorboard option in the config file is turned on.
"tensorboard" : true
Open Tensorboard server
Type tensorboard --logdir saved/log/ at the project root, then server will open at http://localhost:6006
By default, values of loss and metrics specified in config file, input images, and histogram of model parameters will be logged.
If you need more visualizations, use add_scalar('tag', data), add_image('tag', image), etc in the trainer._train_epoch method.
add_something() methods in this template are basically wrappers for those of tensorboardX.SummaryWriter and torch.utils.tensorboard.SummaryWriter modules.
Note: You don't have to specify current steps, since WriterTensorboard class defined at logger/visualization.py will track current steps.
Contribution
Feel free to contribute any kind of function or enhancement, here the coding style follows PEP8
Code should pass the Flake8 check before committing.
TODOs
Multiple optimizers
Support more tensorboard functions
Using fixed random seed
Support pytorch native tensorboard
tensorboardX logger support
Configurable logging layout, checkpoint naming
Iteration-based training (instead of epoch-based)
Adding command line option for fine-tuning
License
This project is licensed under the MIT License. See LICENSE for more details
Acknowledgements
This project is inspired by the project Tensorflow-Project-Template by Mahmoud Gemy
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/victoresque/pytorch-template là nguồn chính thức.
victoresque/pytorch-template có bao nhiêu sao?
victoresque/pytorch-template có 5.1k sao GitHub — tải lại trang để xem số mới nhất, hoặc xem trực tiếp github.com/victoresque/pytorch-template. TopGit phản chiếu số sao của GitHub nhưng không cam kết đến từng phút.
victoresque/pytorch-template có phải mã nguồn mở không?
Có — victoresque/pytorch-template phát hành theo license MIT, nghĩa là mã nguồn mở để đọc, fork và (tùy license) tái sử dụng. Mã: github.com/victoresque/pytorch-template.
victoresque/pytorch-template còn đang phát triển không?
Commit gần nhất trên victoresque/pytorch-template là 2.5 năm trước (theo timestamp GitHub). Repo có 1.1k fork — một chỉ báo về mức độ quan tâm của cộng đồng.
victoresque/pytorch-template dùng license gì?
victoresque/pytorch-template phát hành theo license MIT. Nên mở file LICENSE trên GitHub để xác nhận — license metadata đôi khi lệch với thực tế dự án.
victoresque/pytorch-template là gì?
victoresque/pytorch-template (victoresque/pytorch-template) là dự án Python trên GitHub. Theo mô tả gốc: PyTorch deep learning projects made easy.
victoresque/pytorch-template viết bằng ngôn ngữ gì?
victoresque/pytorch-template chủ yếu viết bằng Python. Trường "language" của GitHub dựa trên phần lớn byte ở nhánh mặc định.
Đọc đầy đủ README ở tab phía trên.
Vẫn đang phân vân về pytorch-template?
Một cú bấm sẽ gửi câu hỏi kèm trang này cho AI — xem AI nói gì về pytorch-template.