Kaggle-NDSB
Code for National Data Science Bowl at Kaggle. Ranked 10th/1049.
Summary
Ensemble Deep CNNs trained with real-time data augmentation.
|
Preprocessing
|
centering, convert to a square image with padding, convert to a negative.
|
|
Data augmentation
|
real-time data agumentation (apply the random transformation each minibatchs).
transformation method includes translation, scaling, rotation, perspective cropping and contrast scaling.
|
|
Neural Network Architecture
|
Three CNN architectures for different rescaling inputs.
cnn_96x96,
cnn_72x72,
cnn_48x48
|
|
Normalization
|
Global Contrast Normalization (GCN)
|
|
Optimization method
|
minibatch-SGD with Nesterov momentum.
|
|
Results
|
| Model | Public LB score |
|---|
| cnn_48x48 single model | 0.6718 | | cnn_72x72 single model | 0.6487 | | cnn_96x96 single model | 0.6561 | | cnn_48x48 average of 8 models | 0.6507 | | cnn_72x72 average of 8 models | 0.6279 | | cnn_96x96 average of 8 models | 0.6311 | | ensemble (cnn_48x48(x8) * 0.2292 + cnn_72x72(x8) * 0.3494 + cnn_96x96(x8) * 0.4212 + 9.828e-6) | 0.6160 |
|
Developer Environment
- Ubuntu 14.04
- 16GB RAM
- GPU & CUDA (I used EC2 g2.2xlarge instance)
- Torch7
- NVIDIA CuDNN
- cudnn.torch
Installation
Install CUDA, Torch7, NVIDIA CuDNN, cudnn.torch.
Checking CUDA environment
th cuda_test.lua
Please check your Torch7/CUDA environment when this code fails.
Convert dataset
Place the data files into a subfolder ./data.
ls ./data
test train train.txt test.txt classess.txt
-
th convert_data.lua
Training, Validation, Make submission
training & validate single cnn_48x48 model.
th train.lua -model 48 -seed 101
ls -la models/cnn*.t7
make submission file.
th predict.lua -model 48 -seed 101
ls -la models/submission*.txt
when use cnn_72x72 model.
th train.lua -model 72 -seed 101
th predict.lua -model 72 -seed 101
when use cnn_96x96 model.
th train.lua -model 96 -seed 101
th predict.lua -model 96 -seed 101
Ensemble
This task is very heavy. I used x20 g2.xlarge instances for this task and it's takes 4 days.
(helper tool can be found at ./appendix folder.)
th train.lua -model 48 -seed 101
th train.lua -model 48 -seed 102
th train.lua -model 48 -seed 103
th train.lua -model 48 -seed 104
th train.lua -model 48 -seed 105
th train.lua -model 48 -seed 106
th train.lua -model 48 -seed 107
th train.lua -model 48 -seed 108
th train.lua -model 72 -seed 101
th train.lua -model 72 -seed 102
th train.lua -model 72 -seed 103
th train.lua -model 72 -seed 104
th train.lua -model 72 -seed 105
th train.lua -model 72 -seed 106
th train.lua -model 72 -seed 107
th train.lua -model 72 -seed 108
th train.lua -model 96 -seed 101
th train.lua -model 96 -seed 102
th train.lua -model 96 -seed 103
th train.lua -model 96 -seed 104
th train.lua -model 96 -seed 105
th train.lua -model 96 -seed 106
th train.lua -model 96 -seed 107
th train.lua -model 96 -seed 108
th predict.lua -model 48 -seed 101
th predict.lua -model 48 -seed 102
th predict.lua -model 48 -seed 103
th predict.lua -model 48 -seed 104
th predict.lua -model 48 -seed 105
th predict.lua -model 48 -seed 106
th predict.lua -model 48 -seed 107
th predict.lua -model 48 -seed 108
th predict.lua -model 72 -seed 101
th predict.lua -model 72 -seed 102
th predict.lua -model 72 -seed 103
th predict.lua -model 72 -seed 104
th predict.lua -model 72 -seed 105
th predict.lua -model 72 -seed 106
th predict.lua -model 72 -seed 107
th predict.lua -model 72 -seed 108
th predict.lua -model 96 -seed 101
th predict.lua -model 96 -seed 102
th predict.lua -model 96 -seed 103
th predict.lua -model 96 -seed 104
th predict.lua -model 96 -seed 105
th predict.lua -model 96 -seed 106
th predict.lua -model 96 -seed 107
th predict.lua -model 96 -seed 108
th ensemble.lua > submission.txt
