As an AI tool, allegro/allRank has picked up 1.0k stars on GitHub (Python). allRank is a framework for training learning-to-rank neural models based on PyTorch.
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
allRank is a PyTorch-based framework for training neural Learning-to-Rank (LTR) models, featuring implementations of:
common pointwise, pairwise and listwise loss functions
fully connected and Transformer-like scoring functions
commonly used evaluation metrics like Normalized Discounted Cumulative Gain (NDCG) and Mean Reciprocal Rank (MRR)
click-models for experiments on simulated click-through data
Motivation
allRank provides an easy and flexible way to experiment with various LTR neural network models and loss functions.
It is easy to add a custom loss, and to configure the model and the training procedure.
We hope that allRank will facilitate both research in neural LTR and its industrial applications.
Features
Implemented loss functions:
ListNet (for binary and graded relevance)
ListMLE
RankNet
Ordinal loss
LambdaRank
LambdaLoss
ApproxNDCG
RMSE
NeuralNDCG (introduced in https://arxiv.org/pdf/2102.07831)
Getting started guide
To help you get started, we provide a run_example.sh script which generates dummy ranking data in libsvm format and trains
a Transformer model on the data using provided example config.json config file. Once you run the script, the dummy data can be found in dummy_data directory
and the results of the experiment in test_run directory. To run the example, Docker is required.
Getting the right architecture version (GPU vs CPU-only)
Since torch binaries are different for GPU and CPU and GPU version doesn't work on CPU - one must select & build appropriate docker image version.
To do so pass gpu or cpu as arch_version build-arg in
All the hyperparameters of the training procedure: i.e. model defintion, data location, loss and metrics used, training hyperparametrs etc. are controlled
by the config.json file. We provide a template file config_template.json where supported attributes, their meaning and possible values are explained.
Note that following MSLR-WEB30K convention, your libsvm file with training data should be named train.txt. You can specify the name of the validation dataset
(eg. valid or test) in the config. Results will be saved under the path <job_dir>/results/<run_id>
Google Cloud Storage is supported in allRank as a place for data and job results.
Implementing custom loss functions
To experiment with your own custom loss, you need to implement a function that takes two tensors (model prediction and ground truth) as input
and put it in the losses package, making sure it is exposed on a package level.
To use it in training, simply pass the name (and args, if your loss method has some hyperparameters) of your function in the correct place in the config file:
The model will be used to rank all slates from the dataset specified in config. Next - a click model configured in config will be applied and the resulting click-through dataset will be written under <job_dir>/results/<run_id> in a libSVM format.
The path to the results directory may then be used as an input for another allRank model training.
Continuous integration
You should run scripts/ci.sh to verify that code passes style guidelines and unit tests.
Research
This framework was developed to support the research project Context-Aware Learning to Rank with Self-Attention. If you use allRank in your research, please cite:
@article{Pobrotyn2020ContextAwareLT,
title={Context-Aware Learning to Rank with Self-Attention},
author={Przemyslaw Pobrotyn and Tomasz Bartczak and Mikolaj Synowiec and Radoslaw Bialobrzeski and Jaroslaw Bojar},
journal={ArXiv},
year={2020},
volume={abs/2005.10084}
}
Additionally, if you use the NeuralNDCG loss function, please cite the corresponding work, NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting:
@article{Pobrotyn2021NeuralNDCG,
title={NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting},
author={Przemyslaw Pobrotyn and Radoslaw Bialobrzeski},
journal={ArXiv},
year={2021},
volume={abs/2102.07831}
}
No homepage URL was recorded for allegro/allRank in TopGit's last sync. The README tab above frequently contains screenshots and demo links, or check the repository description on GitHub.
How does allegro/allRank compare to other AI Tools projects?
allegro/allRank is tracked by TopGit in the AI Tools category, with 1.0k GitHub stars and written in Python. Browse the AI Tools topic page on TopGit to compare it against similar projects by stars and activity.
How many stars does allegro/allRank have?
allegro/allRank has 1.0k GitHub stars — refresh the page for the live number, or check github.com/allegro/allRank. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
Is allegro/allRank open source?
Yes — allegro/allRank ships under the Apache-2.0 license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/allegro/allRank.
What else is in the AI Tools space?
allegro/allRank is tracked by TopGit under the AI Tools category, alongside 10 GitHub-tagged topics. Trending and Topics pages list peer repositories of comparable stars and language.
What is allegro/allRank?
allegro/allRank (allegro/allRank) is a Python project on GitHub. From the project's own README: allRank is a framework for training learning-to-rank neural models based on PyTorch.
Where do I read more about allegro/allRank?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/allegro/allRank is the definitive source.
Read full README in the tab above.
Still deciding about allRank?
One click hands the question to an AI along with this page — see what it says about allRank.