NLP-progress: a GitHub tracker for NLP benchmarks
/NLP-progress
NLP-progress is a GitHub-hosted markdown reference that tracks SOTA results across dozens of NLP tasks, from named entity recognition to question answering. It earns a bookmark because every entry links back to the paper and, where one exists, the code — not because it tries to be a live leaderboard. The catch is currency: a table only moves when someone opens a pull request, so it can lag behind whatever just got published.
What is NLP-progress
NLP-progress is a curated GitHub repository, maintained by Sebastian Ruder and contributors, that catalogs benchmark datasets and SOTA results for common NLP tasks. Each task — language modeling, machine translation, named entity recognition, sentiment analysis, and dozens more — gets its own markdown file with a sorted results table, so you can scan one file and see the current SOTA model plus the paper behind it.
What the repository covers
- ✓Dozens of English task pages, from part-of-speech tagging and dependency parsing to natural language inference, summarization, and paraphrase generation
- ✓Sections for languages beyond English, including Vietnamese, Chinese, Hindi, Spanish, French, Russian, German, Arabic, Korean, Persian, Turkish, Bengali, Nepali, and Portuguese
- ✓Each results table lists the model, its score, the paper or source, and a Code column marked Official or Link where an implementation exists
- ✓A 'Missing elements' page that tracks gaps the maintainers already know about
- ✓A structured/ export path that turns the markdown tables into machine-readable JSON
- ✓Jekyll build instructions for running the nlpprogress.com companion site locally
Who uses NLP-progress and why
- •A researcher scoping a new paper skims the relevant task file for the current SOTA and its citation before designing an experiment
- •Someone new to a subfield reads the linked papers task by task instead of hunting across separate leaderboard sites
- •A practitioner picks a baseline for a task like named entity recognition or sentiment analysis by checking which rows have a filled-in Code column
- •A contributor whose paper just landed opens a pull request to add their result to the matching table
Strengths
- ✓Every entry links to the original paper, so you're never trusting an unsourced number
- ✓The Code column shows at a glance whether an official implementation exists before you go looking for one
- ✓One markdown file per task keeps related results together instead of scattered across separate leaderboard sites
- ✓MIT-licensed, so forking a task file into your own literature review needs no permission
What NLP-progress does not cover
- △Coverage depends entirely on pull requests, so a table can go stale between edits — the README itself tells contributors to defer to a public leaderboard instead when one is actively maintained
- △The README's own wish list names tasks still missing, including bilingual dictionary induction, discourse parsing, and semi-supervised learning
- △Non-English coverage is uneven: languages like French, Portuguese, Korean, Nepali, and Turkish list only one or two task pages, against dozens for English
- △There's no code to run here. It's a reading reference, not a benchmark harness or evaluation library
Other ways to track NLP benchmarks
Frequently asked questions
NLP-progress covers dozens of NLP tasks in its English table of contents, spanning core problems like part-of-speech tagging and dependency parsing alongside newer ones such as natural language inference and paraphrase generation. The maintainers don't publish a running total; task pages get added over time as pull requests come in, one markdown file per task.
NLP-progress does include sections beyond English, covering Vietnamese, Chinese, Hindi, Spanish, French, Russian, German, Arabic, Korean, Persian, Turkish, Bengali, Nepali, and Portuguese. Coverage is much thinner outside English, though — most of these sections list only a handful of task pages.
NLP-progress relies on community pull requests to keep results current, so how fresh a table is varies task by task rather than following a fixed schedule. The README tells contributors that when a task already has an actively maintained public leaderboard, they should point readers there instead of duplicating it in NLP-progress.
Contributing to NLP-progress means opening a GitHub pull request: edit the task's markdown file, add a row to the results table sorted with the best score on top, and mark the Code column Official or Link if an implementation exists. A brand-new task needs its own file, a description, the evaluation metric, and an annotated example.
NLP-progress is released under the MIT license, covering the repository's content alongside its code. That makes it straightforward to fork a task file or reuse a results table in your own notes or literature review without asking anyone for permission first.
NLP-progress is a static, community-edited markdown reference, not a live-scoring leaderboard — results change only when someone submits a pull request, and there's no automated evaluation or submission pipeline behind it. The README explicitly defers to an official leaderboard for tasks that already have one.
Who should try it — and who should skip
Try NLP-progress if you're starting research on a specific NLP task and want the paper trail behind a result, not just a raw leaderboard number. Skip it if you need real-time rankings — the README already defers to an actively maintained public leaderboard when one exists, so NLP-progress does its best work on tasks that don't have one.
Related repositories
Curious whether NLP-progress is right for you?
Let ChatGPT, Claude, or Perplexity look into it — click below and see what AI actually says about NLP-progress.