DLR-RM/AugmentedAutoencoder được TopGit xếp vào nhóm dự án mã nguồn mở, với 365 sao trên GitHub, viết chủ yếu bằng Python. Official Code: Implicit 3D Orientation Learning for 6D Object Detection from RGB Images
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
Implicit 3D Orientation Learning for 6D Object Detection from RGB Images
Martin Sundermeyer, Zoltan-Csaba Marton, Maximilian Durner, Manuel Brucker, Rudolph Triebel
Best Paper Award, ECCV 2018.
paper, supplement, oral
Citation
If you find Augmented Autoencoders useful for your research, please consider citing:
@InProceedings{Sundermeyer_2018_ECCV,
author = {Sundermeyer, Martin and Marton, Zoltan-Csaba and Durner, Maximilian and Brucker, Manuel and Triebel, Rudolph},
title = {Implicit 3D Orientation Learning for 6D Object Detection from RGB Images},
booktitle = {The European Conference on Computer Vision (ECCV)},
month = {September},
year = {2018}
}
Multi-path Learning for Object Pose Estimation Across Domains
Martin Sundermeyer, Maximilian Durner, En Yen Puang, Zoltan-Csaba Marton, Narunas Vaskevicius, Kai O. Arras, Rudolph Triebel
CVPR 2020
The code of this work can be found here
Overview
We propose a real-time RGB-based pipeline for object detection and 6D pose estimation. Our novel 3D orientation estimation is based on a variant of the Denoising Autoencoder that is trained on simulated views of a 3D model using Domain Randomization. This so-called Augmented Autoencoder has several advantages over existing methods: It does not require real, pose-annotated training data, generalizes to various test sensors and inherently handles object and view symmetries.
1.) Train the Augmented Autoencoder(s) using only a 3D model to predict 3D Object Orientations from RGB image crops
2.) For full RGB-based 6D pose estimation, also train a 2D Object Detector (e.g. https://github.com/fizyr/keras-retinanet)
3.) Optionally, use our standard depth-based ICP to refine the 6D Pose
Requirements: Hardware
For Training
Nvidia GPU with >4GB memory (or adjust the batch size)
RAM >8GB
Duration depending on Configuration and Hardware: ~3h per Object
Please note that we use the GLFW context as default which does not support headless rendering. To allow for both, onscreen rendering & headless rendering on a remote server, set the context to EGL:
export PYOPENGL_PLATFORM='egl'
In order to make the EGL context work, you might need to change PyOpenGL like here
Support for Tensorflow 2.6 / Python 3
The code now also supports TF 2.6 with python 3. Instead of the pip installs above, you can also use the provided conda environment.
conda env create -f aae_py37_tf26.yml
In the activated environment proceed with the preparatory steps.
Preparatory Steps
1. Pip installation
pip install .
2. Set Workspace path, consider to put this into your bash profile, will always be required
export AE_WORKSPACE_PATH=/path/to/autoencoder_ws
3. Create Workspace, Init Workspace (if installed locally, make sure .local/bin/ is in your PATH)
mkdir $AE_WORKSPACE_PATH
cd $AE_WORKSPACE_PATH
ae_init_workspace
Train an Augmented Autoencoder
1. Create the training config file. Insert the paths to your 3D model and background images.
Here are AAE models trained the BOP datasets with codebooks of all 108 objects:
Download
Extract it to $AE_WORKSPACE_PATH/experiments
Also get precomputed MaskRCNN predictions for all BOP datasets:
Download
Open the bop20 evaluation configs, e.g. auto_pose/ae/cfg_m3vision/m3_config_lmo.cfg, and point the path_to_masks parameter to the downloaded maskrcnn predictions.
You can visualize (-vis option) and reproduce BOP results by running:
python auto_pose/m3_interface/compute_bop_results_m3.py auto_pose/ae/cfg_m3vision/m3_config_lmo.cfg
--eval_name test
--dataset_name=lmo
--datasets_path=/path/to/bop/datasets
--result_folder /folder/to/results
-vis
Note: You will need the bop_toolkit. I created a package bop_toolkit_lib from it, but you can also just add the required files to sys.path()
Original paper evaluation with T-LESS v1
For the evaluation you will also need
https://github.com/thodan/sixd_toolkit + our extensions, see sixd_toolkit_extension/help.txt
[Paths]
# Path to the model file. All formats supported by assimp should work. Tested with ply files.
MODEL_PATH: /path/to/my_3d_model.ply
# Path to some background image folder. Should contain a * as a placeholder for the image name.
BACKGROUND_IMAGES_GLOB: /path/to/VOCdevkit/VOC2012/JPEGImages/*.jpg
[Dataset]
#cad or reconst (with texture)
MODEL: reconst
# Height of the AE input layer
H: 128
# Width of the AE input layer
W: 128
# Channels of the AE input layer (default BGR)
C: 3
# Distance from Camera to the object in mm for synthetic training images
RADIUS: 700
# Dimensions of the renderered image, it will be cropped and rescaled to H, W later.
RENDER_DIMS: (720, 540)
# Camera matrix used for rendering and optionally for estimating depth from RGB
K: [1075.65, 0, 720/2, 0, 1073.90, 540/2, 0, 0, 1]
# Vertex scale. Vertices need to be scaled to mm
VERTEX_SCALE: 1
# Antialiasing factor used for rendering
ANTIALIASING: 8
# Padding rendered object images and potentially bounding box detections
PAD_FACTOR: 1.2
# Near plane
CLIP_NEAR: 10
# Far plane
CLIP_FAR: 10000
# Number of training images rendered uniformly at random from SO(3)
NOOF_TRAINING_IMGS: 10000
# Number of background images that simulate clutter
NOOF_BG_IMGS: 10000
[Augmentation]
# Using real object masks for occlusion (not really necessary)
REALISTIC_OCCLUSION: False
# Maximum relative translational offset of input views, sampled uniformly
MAX_REL_OFFSET: 0.20
# Random augmentations at random strengths from imgaug library
CODE: Sequential([
#Sometimes(0.5, PerspectiveTransform(0.05)),
#Sometimes(0.5, CropAndPad(percent=(-0.05, 0.1))),
Sometimes(0.5, Affine(scale=(1.0, 1.2))),
Sometimes(0.5, CoarseDropout( p=0.2, size_percent=0.05) ),
Sometimes(0.5, GaussianBlur(1.2*np.random.rand())),
Sometimes(0.5, Add((-25, 25), per_channel=0.3)),
Sometimes(0.3, Invert(0.2, per_channel=True)),
Sometimes(0.5, Multiply((0.6, 1.4), per_channel=0.5)),
Sometimes(0.5, Multiply((0.6, 1.4))),
Sometimes(0.5, ContrastNormalization((0.5, 2.2), per_channel=0.3))
], random_order=False)
[Embedding]
# for every rotation save rendered bounding box diagonal for projective distance estimation
EMBED_BB: True
# minimum number of equidistant views rendered from a view-sphere
MIN_N_VIEWS: 2562
# for each view generate a number of in-plane rotations to cover full SO(3)
NUM_CYCLO: 36
[Network]
# additionally reconstruct segmentation mask, helps when AAE decodes pure blackness
AUXILIARY_MASK: False
# Variational Autoencoder, factor in front of KL-Divergence loss
VARIATIONAL: 0
# Reconstruction error metric
LOSS: L2
# Only evaluate 1/BOOTSTRAP_RATIO of the pixels with highest errors, produces sharper edges
BOOTSTRAP_RATIO: 4
# regularize norm of latent variables
NORM_REGULARIZE: 0
# size of the latent space
LATENT_SPACE_SIZE: 128
# number of filters in every Conv layer (decoder mirrored)
NUM_FILTER: [128, 256, 512, 512]
# stride for encoder layers, nearest neighbor upsampling for decoder layers
STRIDES: [2, 2, 2, 2]
# filter size encoder
KERNEL_SIZE_ENCODER: 5
# filter size decoder
KERNEL_SIZE_DECODER: 5
[Training]
OPTIMIZER: Adam
NUM_ITER: 30000
BATCH_SIZE: 64
LEARNING_RATE: 1e-4
SAVE_INTERVAL: 5000
[Queue]
# number of threads for producing augmented training data (online)
NUM_THREADS: 10
# preprocessing queue size in number of batches
QUEUE_SIZE: 50
DLR-RM/AugmentedAutoencoder có 365 sao GitHub — tải lại trang để xem số mới nhất, hoặc xem trực tiếp github.com/DLR-RM/AugmentedAutoencoder. TopGit phản chiếu số sao của GitHub nhưng không cam kết đến từng phút.
DLR-RM/AugmentedAutoencoder có phải mã nguồn mở không?
Có — DLR-RM/AugmentedAutoencoder phát hành theo license MIT, nghĩa là mã nguồn mở để đọc, fork và (tùy license) tái sử dụng. Mã: github.com/DLR-RM/AugmentedAutoencoder.
DLR-RM/AugmentedAutoencoder còn đang phát triển không?
Commit gần nhất trên DLR-RM/AugmentedAutoencoder là 4.3 năm trước (theo timestamp GitHub). Repo có 97 fork — một chỉ báo về mức độ quan tâm của cộng đồng.
DLR-RM/AugmentedAutoencoder là gì?
DLR-RM/AugmentedAutoencoder (DLR-RM/AugmentedAutoencoder) là dự án Python trên GitHub. Theo mô tả gốc: Official Code: Implicit 3D Orientation Learning for 6D Object Detection from RGB Images
Đọc thêm về DLR-RM/AugmentedAutoencoder ở đâu?
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/DLR-RM/AugmentedAutoencoder là nguồn chính thức.
Đọc đầy đủ README ở tab phía trên.
AugmentedAutoencoder có đáng để bạn bỏ thời gian?
ChatGPT, Claude và Perplexity đều đọc được trang này. Hỏi thử xem họ nghĩ gì về AugmentedAutoencoder.