ThilinaRajapakse/simpletransformers sits at 4.3k stars on GitHub, written primarily in Python. Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
This library is based on the Transformers library by HuggingFace. Simple Transformers lets you quickly train and evaluate Transformer models. Only 3 lines of code are needed to initialize, train, and evaluate a model.
Supported Tasks:
Information Retrieval (Dense Retrieval)
(Large) Language Models (Training, Fine-tuning, and Generation)
Encoder Model Training and Fine-tuning
Sequence Classification
Token Classification (NER)
Question Answering
Language Generation
T5 Model
Seq2Seq Tasks
Multi-Modal Classification
Conversational AI
Citation
If you use Simple Transformers in your work, please cite:
@inproceedings{Rajapakse2024SimpleTransformers,
author = {Rajapakse, Thilina C. and Yates, Andrew and de Rijke, Maarten},
title = {Simple Transformers: Open-source for All},
booktitle = {Proceedings of the 2024 Annual International ACM SIGIR
Conference on Research and Development in Information
Retrieval in the Asia Pacific Region},
series = {SIGIR-AP 2024},
pages = {209--215},
year = {2024},
doi = {10.1145/3673791.3698412},
url = {https://doi.org/10.1145/3673791.3698412},
location = {Tokyo, Japan}
}
Table of contents
Simple Transformers
Table of contents
Setup
With Conda
Optional
Usage
A quick example
Experiment Tracking with Weights and Biases
Current Pretrained Models
Contributors ✨
How to Contribute
How to Update Docs
Acknowledgements
Setup
With Conda
Install Anaconda or Miniconda Package Manager from here
Create a new virtual environment and install packages.
$ conda create -n st python pandas tqdm
$ conda activate st
Install Weights and Biases (wandb) for tracking and visualizing training in a web browser.
$ pip install wandb
Usage
All documentation is now live at thilinarajapakse.github.io/simpletransformers
Simple Transformer models are built with a particular Natural Language Processing (NLP) task in mind. Each such model comes equipped with features and functionality designed to best fit the task that they are intended to perform. The high-level process of using Simple Transformers models follows the same pattern.
Initialize a task-specific model
Train the model with train_model()
Evaluate the model with eval_model()
Make predictions on (unlabelled) data with predict()
However, there are necessary differences between the different models to ensure that they are well suited for their intended task. The key differences will typically be the differences in input/output data formats and any task specific features/configuration options. These can all be found in the documentation section for each task.
The currently implemented task-specific Simple Transformer models, along with their task, are given below.
Task
Model
Binary and multi-class text classification
ClassificationModel
Conversational AI (chatbot training)
ConvAIModel
Language generation
LanguageGenerationModel
Language model training/fine-tuning
LanguageModelingModel
Multi-label text classification
MultiLabelClassificationModel
Multi-modal classification (text and image data combined)
MultiModalClassificationModel
Named entity recognition
NERModel
Question answering
QuestionAnsweringModel
Regression
ClassificationModel
Sentence-pair classification
ClassificationModel
Text Representation Generation
RepresentationModel
Document Retrieval
RetrievalModel
Please refer to the relevant section in the docs for more information on how to use these models.
Example scripts can be found in the examples directory.
See the Changelog for up-to-date changes to the project.
A quick example
from simpletransformers.classification import ClassificationModel, ClassificationArgs
import pandas as pd
import logging
logging.basicConfig(level=logging.INFO)
transformers_logger = logging.getLogger("transformers")
transformers_logger.setLevel(logging.WARNING)
# Preparing train data
train_data = [
["Aragorn was the heir of Isildur", 1],
["Frodo was the heir of Isildur", 0],
]
train_df = pd.DataFrame(train_data)
train_df.columns = ["text", "labels"]
# Preparing eval data
eval_data = [
["Theoden was the king of Rohan", 1],
["Merry was the king of Rohan", 0],
]
eval_df = pd.DataFrame(eval_data)
eval_df.columns = ["text", "labels"]
# Optional model configuration
model_args = ClassificationArgs(num_train_epochs=1)
# Create a ClassificationModel
model = ClassificationModel(
"roberta", "roberta-base", args=model_args
)
# Train the model
model.train_model(train_df)
# Evaluate the model
result, model_outputs, wrong_predictions = model.eval_model(eval_df)
# Make predictions with the model
predictions, raw_outputs = model.predict(["Sam was a Wizard"])
Experiment Tracking with Weights and Biases
Weights and Biases makes it incredibly easy to keep track of all your experiments. Check it out on Colab here:
Current Pretrained Models
For a list of pretrained models, see Hugging Face docs.
The model_types available for each task can be found under their respective section. Any pretrained model of that type
found in the Hugging Face docs should work. To use any of them set the correct model_type and model_name in the args
dictionary.
Contributors ✨
Thanks goes to these wonderful people (emoji key):
hawktang 💻
Mabu Manaileng 💻
Ali Hamdi Ali Fadel 💻
Tovly Deutsch 💻
hlo-world 💻
huntertl 💻
Yann Defretin 💻 📖 💬 🤔
Manuel 📖 💻
Gilles Jacobs 📖
shasha79 💻
Mercedes Garcia 💻
Hammad Hassan Tarar 💻 📖
Todd Cook 💻
Knut O. Hellan 💻 📖
nagenshukla 💻
flaviussn 💻 📖
Marc Torrellas 🚧
Adrien Renaud 💻
jacky18008 💻
Matteo Senese 💻
sarthakTUM 📖 💻
djstrong 💻
Hyeongchan Kim 📖
Pradhy729 💻 🚧
Iknoor Singh 📖
Gabriel Altay 💻
flozi00 📖 💻 🚧
alexysdussier 💻
Jean-Louis Queguiner 📖
aced125 💻
Laksh1997 💻
Changlin_NLP 💻
jpotoniec 💻
fcggamou 💻 📖
guy-mor 🐛 💻
Cahya Wirawan 💻
BjarkePedersen 💻
tekkkon 💻
Amit Garg 💻
caprone 🐛
Ather Fawaz 💻
Santiago Castro 📖
taranais 💻
Pablo N. Marino 💻 📖
Anton Kiselev 💻 📖
Alex 💻
Karthik Ganesan 💻
Zhylko Dima 💻
Jonatan Kłosko 💻
sarapapi 💻 💬
Abdul 💻
James Milliman 📖
Suraj Parmar 📖
KwanHong Lee 💬
Erik Fäßler 💻
Thomas Søvik 💬
Gagandeep Singh 💻 📖
Andrea Esuli 💻
DM2493 💻
Nick Doiron 💻
Abhinav Gupta 💻
Martin H. Normark 📖
Mossad Helali 💻
calebchiam 💻
Daniele Sartiano 💻
tuner007 📖
xia jiang 💻
Hendrik Buschmeier 📖
Mana Borwornpadungkitti 📖
rayline 💻
Mehdi Heidari 💻
William Roe 💻
Álvaro Abella Bascarán 💻
Brett Fazio 📖
Viet-Tien 💻
Bisola Olasehinde 💻 📖
William Chen 📖
Reza Ebrahimi 📖
gabriben 📖
Prashanth Kurella 💻
dopc 💻
Tanish Tyagi 📖 💻
kongyurui 💻
Andrew Lensen 💻
jinschoi 💻
Le Nguyen Khang 💻
Jordi Mas 📖
mxa 💻
MichelBartels 💻
Luke Tudge 📖
Saint 💻
deltaxrg 💻 📖
Fortune Adekogbe 💻
This project follows the all-contributors specification. Contributions of any kind welcome!
If you should be on this list but you aren't, or you are on the list but don't want to be, please don't hesitate to contact me!
How to Contribute
How to Update Docs
The latest version of the docs is hosted on Github Pages, if you want to help document Simple Transformers
below are the steps to edit the docs.
Docs are built using Jekyll library, refer to their webpage for a detailed explanation of how it works.
Install Jekyll: Run the command gem install bundler jekyll
Visualizing the docs on your local computer:
In your terminal cd into the docs directory of this repo, eg: cd simpletransformers/docs
From the docs directory run this command to serve the Jekyll docs locally: bundle exec jekyll serve
Browse to http://localhost:4000 or whatever url you see in the console to visualize the docs.
Edit and visualize changes:
All the section pages of our docs can be found under docs/_docs directory, you can edit any file you want by following the markdown format and visualize the changes after refreshing the browser tab.
Acknowledgements
None of this would have been possible without the hard work by the HuggingFace team in developing the Transformers library.
<div>Icon for the Social Media Preview made by <a href="https://www.flaticon.com/authors/freepik" title="Freepik">Freepik</a> from <a href="https://www.flaticon.com/" title="Flaticon">www.flaticon.com</a></div>
How active is development on ThilinaRajapakse/simpletransformers?
The most recent commit recorded on ThilinaRajapakse/simpletransformers was 5 months ago, based on the GitHub push timestamp. The repository has 709 forks — one of the better signals of community interest.
How does ThilinaRajapakse/simpletransformers compare to other AI Tools projects?
ThilinaRajapakse/simpletransformers is tracked by TopGit in the AI Tools category, with 4.3k GitHub stars and written in Python. Browse the AI Tools topic page on TopGit to compare it against similar projects by stars and activity.
How many stars does ThilinaRajapakse/simpletransformers have?
ThilinaRajapakse/simpletransformers has 4.3k GitHub stars — refresh the page for the live number, or check github.com/ThilinaRajapakse/simpletransformers. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
Is ThilinaRajapakse/simpletransformers open source?
Yes — ThilinaRajapakse/simpletransformers ships under the Apache-2.0 license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/ThilinaRajapakse/simpletransformers.
What is ThilinaRajapakse/simpletransformers?
ThilinaRajapakse/simpletransformers (ThilinaRajapakse/simpletransformers) is a Python project on GitHub. From the project's own README: Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI
Where do I read more about ThilinaRajapakse/simpletransformers?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/ThilinaRajapakse/simpletransformers is the definitive source.
Read full README in the tab above.
Is simpletransformers worth your time?
ChatGPT, Claude and Perplexity can all read this page. Ask one of them what it makes of simpletransformers.