Được TopGit lập chỉ mục từ metadata GitHub: cvg/Hierarchical-Localization có 4.3k sao, viết chủ yếu bằng Python. Visual localization made easy with hloc
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
This is hloc, a modular toolbox for state-of-the-art 6-DoF visual localization. It implements Hierarchical Localization, leveraging image retrieval and feature matching, and is fast, accurate, and scalable. This codebase combines and makes easily accessible years of research on image matching and Structure-from-Motion.
With hloc, you can:
Reproduce state-of-the-art results on multiple indoor and outdoor visual localization benchmarks
Run Structure-from-Motion with SuperPoint+SuperGlue to localize with your own datasets
Evaluate your own local features or image retrieval for visual localization
Implement new localization pipelines and debug them easily 🔥
Hierachical Localization uses both image retrieval and feature matching
Quick start ➡️
Build 3D maps with Structure-from-Motion and localize any Internet image right from your browser! You can now run hloc and COLMAP in Google Colab with GPU for free. The notebook demo.ipynb shows how to run SfM and localization in just a few steps. Try it with your own data and let us know!
Installation
hloc requires Python >=3.7 and PyTorch >=1.1. Installing the package locally pulls the other dependencies:
All dependencies are listed in requirements.txt. Starting with hloc-v1.3, installing COLMAP is not required anymore. This repository includes external local features as git submodules – don't forget to pull submodules with git submodule update --init --recursive.
The toolbox is composed of scripts, which roughly perform the following steps:
Extract local features, like SuperPoint or DISK, for all database and query images
Build a reference 3D SfM model
Find covisible database images, with retrieval or a prior SfM model
Match these database pairs with SuperGlue or the faster LightGlue
Triangulate a new SfM model with COLMAP
Find database images relevant to each query, using retrieval
Match the query images
Run the localization
Visualize and debug
The localization can then be evaluated on visuallocalization.net for the supported datasets. When 3D Lidar scans are available, such as for the indoor dataset InLoc, step 2. can be skipped.
Strcture of the toolbox:
hloc/*.py : top-level scripts
hloc/extractors/ : interfaces for feature extractors
hloc/matchers/ : interfaces for feature matchers
hloc/pipelines/ : entire pipelines for multiple datasets
hloc can be imported as an external package with import hloc or called from the command line with:
python -m hloc.name_of_script --arg1 --arg2
Tasks
We provide step-by-step guides to localize with Aachen, InLoc, and to generate reference poses for your own data using SfM. Just download the datasets and you're reading to go!
Aachen – outdoor localization
Have a look at pipeline_Aachen.ipynb for a step-by-step guide on localizing with Aachen. Play with the visualization, try new local features or matcher, and have fun! Don't like notebooks? You can also run all scripts from the command line.
InLoc – indoor localization
The notebook pipeline_InLoc.ipynb shows the steps for localizing with InLoc. It's much simpler since a 3D SfM model is not needed.
SfM reconstruction from scratch
We show in pipeline_SfM.ipynb how to run 3D reconstruction for an unordered set of images. This generates reference poses, and a nice sparse 3D model suitable for localization with the same pipeline as Aachen.
Results
Supported local feature extractors: SuperPoint, DISK, D2-Net, SIFT, and R2D2.
Supported feature matchers: SuperGlue, its faster follow-up LightGlue, and nearest neighbor search with ratio test, distance test, and/or mutual check. hloc also supports dense matching with LoFTR.
Supported image retrieval: NetVLAD, AP-GeM/DIR, OpenIBL, and MegaLoc.
Using NetVLAD for retrieval, we obtain the following best results:
Methods
Aachen day
Aachen night
Retrieval
SuperPoint + SuperGlue
89.6 / 95.4 / 98.8
86.7 / 93.9 / 100
NetVLAD top 50
SuperPoint + NN
85.4 / 93.3 / 97.2
75.5 / 86.7 / 92.9
NetVLAD top 30
D2Net (SS) + NN
84.6 / 91.4 / 97.1
83.7 / 90.8 / 100
NetVLAD top 30
Methods
InLoc DUC1
InLoc DUC2
Retrieval
SuperPoint + SuperGlue
46.5 / 65.7 / 78.3
52.7 / 72.5 / 79.4
NetVLAD top 40
SuperPoint + SuperGlue (temporal)
49.0 / 68.7 / 80.8
53.4 / 77.1 / 82.4
NetVLAD top 40
SuperPoint + NN
39.9 / 55.6 / 67.2
37.4 / 57.3 / 70.2
NetVLAD top 20
D2Net (SS) + NN
39.9 / 57.6 / 67.2
36.6 / 53.4 / 61.8
NetVLAD top 20
Check out visuallocalization.net/benchmark for more details and additional baselines.
Supported datasets
We provide in hloc/pipelines/ scripts to run the reconstruction and the localization on the following datasets: Aachen Day-Night (v1.0 and v1.1), InLoc, Extended CMU Seasons, RobotCar Seasons, 4Seasons, Cambridge Landmarks, and 7-Scenes. For example, after downloading the dataset with the instructions given here, we can run the Aachen Day-Night pipeline with SuperPoint+SuperGlue using the command:
If you report any of the above results in a publication, or use any of the tools provided here, please consider citing both Hierarchical Localization and SuperGlue papers:
@inproceedings{sarlin2019coarse,
title = {From Coarse to Fine: Robust Hierarchical Localization at Large Scale},
author = {Paul-Edouard Sarlin and
Cesar Cadena and
Roland Siegwart and
Marcin Dymczyk},
booktitle = {CVPR},
year = {2019}
}
@inproceedings{sarlin2020superglue,
title = {{SuperGlue}: Learning Feature Matching with Graph Neural Networks},
author = {Paul-Edouard Sarlin and
Daniel DeTone and
Tomasz Malisiewicz and
Andrew Rabinovich},
booktitle = {CVPR},
year = {2020},
}
Going further
Debugging and Visualization
[Click to expand]
Each localization run generates a pickle log file. For each query, it contains the selected database images, their matches, and information from the pose solver, such as RANSAC inliers. It can thus be parsed to gather statistics and analyze failure modes or difficult scenarios.
We also provide some visualization tools in hloc/visualization.py to visualize some attributes of the 3D SfM model, such as visibility of the keypoints, their track length, or estimated sparse depth (like below).
Using your own local features or matcher
[Click to expand]
If your code is based on PyTorch: simply add a new interface in hloc/extractors/ or hloc/matchers/. It needs to inherit from hloc.utils.base_model.BaseModel, take as input a data dictionary, and output a prediction dictionary. Have a look at hloc/extractors/superpoint.py for an example. You can additionally define a standard configuration in hloc/extract_features.py or hloc/match_features.py - it can then be called directly from the command line.
If your code is based on TensorFlow: you will need to either modify hloc/extract_features.py and hloc/match_features.py, or export yourself the features and matches to HDF5 files, described below.
In a feature file, each key corresponds to the relative path of an image w.r.t. the dataset root (e.g. db/1.jpg for Aachen), and has one dataset per prediction (e.g. keypoints and descriptors, with shape Nx2 and DxN).
In a match file, each key corresponds to the string path0.replace('/', '-')+'_'+path1.replace('/', '-') and has a dataset matches0 with shape N. It indicates, for each keypoint in the first image, the index of the matching keypoint in the second image, or -1 if the keypoint is unmatched.
Using your own image retrieval
[Click to expand]
hloc also provides an interface for image retrieval via hloc/extract_features.py. As previously, simply add a new interface to hloc/extractors/. Alternatively, you will need to export the global descriptors into an HDF5 file, in which each key corresponds to the relative path of an image w.r.t. the dataset root, and contains a dataset global_descriptor with size D. You can then export the images pairs with hloc/pairs_from_retrieval.py.
Reconstruction with known camera parameters
[Click to expand]
If the calibration of the camera is known, for example from an external calibration system, you can tell hloc to use these parameters instead of estimating them from EXIF. The name of the camera models and their parameters are defined by COLMAP. Python API:
Use the new pycolmap Reconstruction objects and pipeline API
Do not require an installation of COLMAP anymore - pycolmap is enough
Faster model reading and writing
Fine-grained control over camera sharing via the camera_mode parameter
Localization with unknown or inaccurate focal length
Modular localization API with control over all estimator parameters
3D visualizations or camera frustums and points with plotly
Package-specific logging in the hloc namespace
Store the extracted features by default as fp16 instead of fp32
Optionally fix a long-standing bug in SuperPoint descriptor sampling
Add script to compute exhaustive pairs for reconstruction or localization
Require pycolmap>=0.1.0 and Python>=3.7
v1.2 (December 2021)
Bug fixes and usability improvements.
Support PIL backend for image resizing.
Add __version__ attribute to check against future releases.
v1.1 (July 2021)
Breaking: improved structure of the SfM folders (triangulation and reconstruction), see #76
Support for image retrieval (NetVLAD, DIR) and more local features (SIFT, R2D2)
Support for more datasets: Aachen v1.1, Extended CMU Seasons, RobotCar Seasons, Cambridge Landmarks, 7-Scenes
Simplified pipeline and API
Spatial matcher
v1.0 (July 2020)
Initial public version.
Contributions welcome!
External contributions are very much welcome. Please follow the PEP8 style guidelines using a linter like flake8. This is a non-exhaustive list of features that might be valuable additions:
support for GPS (extraction from EXIF + guided retrieval)
covisibility clustering for InLoc
visualization of the raw predictions (features and matches)
other local features or image retrieval
Created and maintained by Paul-Edouard Sarlin with the help of many contributors.
cvg/Hierarchical-Localization thuộc nhóm AI Tools trên TopGit, cùng 7 topic GitHub. Trang Trending và Topics liệt kê các repo cùng số sao và cùng ngôn ngữ để so sánh.
cvg/Hierarchical-Localization có bao nhiêu sao?
cvg/Hierarchical-Localization có 4.3k sao GitHub — tải lại trang để xem số mới nhất, hoặc xem trực tiếp github.com/cvg/Hierarchical-Localization. TopGit phản chiếu số sao của GitHub nhưng không cam kết đến từng phút.
cvg/Hierarchical-Localization có phải mã nguồn mở không?
Có — cvg/Hierarchical-Localization phát hành theo license Apache-2.0, nghĩa là mã nguồn mở để đọc, fork và (tùy license) tái sử dụng. Mã: github.com/cvg/Hierarchical-Localization.
cvg/Hierarchical-Localization có website riêng không?
TopGit chưa ghi nhận URL trang chủ cho cvg/Hierarchical-Localization. Phần README ở tab phía trên thường có link demo, hoặc xem mô tả GitHub của repo.
cvg/Hierarchical-Localization là gì?
cvg/Hierarchical-Localization (cvg/Hierarchical-Localization) là dự án Python trên GitHub. Theo mô tả gốc: Visual localization made easy with hloc
cvg/Hierarchical-Localization so với các dự án AI Tools khác thế nào?
cvg/Hierarchical-Localization được TopGit xếp vào nhóm AI Tools, với 4.3k sao GitHub và viết bằng Python. Xem trang chủ đề AI Tools trên TopGit để so sánh với các dự án tương tự theo số sao và mức độ hoạt động.
Đọc thêm về cvg/Hierarchical-Localization ở đâu?
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/cvg/Hierarchical-Localization là nguồn chính thức.
Đọc đầy đủ README ở tab phía trên.
Muốn nghe thêm một ý kiến về Hierarchical-Localization?
Hỏi một AI đọc được trang này — một cú bấm là có ngay nhận định về Hierarchical-Localization.