On GitHub, stas00/ml-engineering has picked up 19.7k stars, AI Tools, Python. Machine Learning Engineering Open Book
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
This is an open collection of methodologies, tools and step by step instructions to help with successful training and fine-tuning of large language models and multi-modal models and their inference.
This is a technical material suitable for LLM/VLM training engineers and operators. That is the content here contains lots of scripts and copy-n-paste commands to enable you to quickly address your needs.
This repo is an ongoing brain dump of my experiences training Large Language Models (LLM) (and VLMs); a lot of the know-how I acquired while training the open-source BLOOM-176B model in 2022 and IDEFICS-80B multi-modal model in 2023, and RAG models at Contextual.AI in 2024.
I've been compiling this information mostly for myself so that I could quickly find solutions I have already researched in the past and which have worked, but as usual I'm happy to share these notes with the wider ML community.
Table of Contents
Part 1. Insights
The AI Battlefield Engineering - what you need to know in order to succeed.
How to Choose a Cloud Provider - these questions will empower you to have a successful compute cloud experience.
When Is It Worth Upgrading GPUs? - a practical framework for deciding whether a GPU generation upgrade is worth its cost, worked through on a real H200 → B200 benchmark.
Part 2. Hardware
Compute - accelerators, CPUs, CPU memory.
Storage - local, distributed and shared file systems.
Network - intra- and inter-node networking.
Part 3. Orchestration
Orchestration Systems - managing containers and resources
SLURM - Simple Linux Utility for Resource Management
Part 4. Training
Training - model training-related guides
Part 5. Inference
Inference - model inference insights
Part 6. Development
Debugging and Troubleshooting - how to debug easy and difficult issues
And more debugging
Testing - numerous tips and tools to make test writing enjoyable
Part 7. Miscellaneous
Resources - LLM/VLM chronicles
Updates
I announce any significant updates on my twitter channel https://twitter.com/StasBekman.
Ebook versions of the book
You can download various ebook formats of this book:
PDF
EPUB
I will try to rebuild these once in a few weeks or so, but if you want the latest ebook versions, the instructions for building are here.
Thanks to HuggingFace for giving me permission to host my book's ebook formats at the HF hub.
Skills for AI agents
I maintain two SKILL.md files that you can use to teach your AI agent:
Machine Learning Engineering — train and operate large-scale ML models.
Evaluate a GPU cluster — trial-node / acceptance loop: compute, network, and storage, then a markdown report.
See also the companion skills: The Art of Debugging and Stas' Python Cookbook.
Courses
Lessons Learned from Training LLMs - provides a very different way of reading my open books, by going over the terse learned insights and allowing you to quickly dive deeper when you need to.
Lectures/Talks
Building resilient ML Engineering skills given on 2026-01-10 for the GPU Mode community. Only had time to discuss performance reality of accelerators, network and storage and how each of them can be crucial to the ensemble's performance. Thanks to Mark Saroufim for organizing and providing an awesome support during the talk.
Discussions
If you want to discuss something related to ML engineering this repo has the community discussions available - so please don't hesitate to share your experience or start a new discussion about something you're passionate about.
Key comparison tables
High end accelerators:
Theoretical accelerator TFLOPS
Accelerator memory size and speed
Networks:
Theoretical inter-node speed
Theoretical intra-node speed
Shortcuts
Things that you are likely to need to find quickly and often.
Tools:
torch-dist-bench.py - a much easier way to benchmark network throughput than nccl-tests (used to be called all_reduce_bench.py).
torch-distributed-gpu-test.py - a tool to quickly test your inter-node connectivity
mamf-finder.py - what is the actual TFLOPS measurement you can get from your accelerator.
Guides:
debugging pytorch applications - quick copy-n-paste solutions to resolve hanging or breaking pytorch applications
slurm for users - a slurm cheatsheet and tricks
make tiny models/datasets/tokenizers
LLM/VLM chronicles collection
Gratitude
None of this would have been possible without me being entrusted with doing the specific LLM/VLM trainings I have learned the initial know-how from. This is a privilege that only a few enjoy due to the prohibitively expensive cost of renting huge ML compute clusters. So hopefully the rest of the ML community will vicariously learn from these notes.
Special thanks go to Thom Wolf who proposed that I lead the BLOOM-176B training back when I didn't know anything about large scale training. This was the project that catapulted me into the intense learning process. And, of course, HuggingFace for giving me the opportunity to work full time on BLOOM-176B and later on IDEFICS-80B trainings.
Recently, I continued expanding my knowledge and experience while training models and building scalable training/inference systems at Contextual.AI and I'm grateful for that opportunity to Aman and Douwe.
I'd also like to thank the numerous contributors who have been making this text awesome and error-free.
Contributing
If you found a bug, typo or would like to propose an improvement please don't hesitate to open an Issue or contribute a PR.
Related Open Books
The Art of Debugging Open Book — methodologies and recipes for debugging Unix, Python and PyTorch programs.
Stas' Python Cookbook — everyday Python and standard-library recipes.
License
The content of this site is distributed under Attribution-ShareAlike 4.0 International.
Citation
@misc{bekman2024mlengineering,
author = {Bekman, Stas},
title = {Machine Learning Engineering Open Book},
year = {2023-2026},
publisher = {Stasosphere Online Inc.},
journal = {GitHub repository},
url = {https://github.com/stas00/ml-engineering}
}
My repositories map
✔ Books:
Machine Learning Engineering |
The Art of Debugging |
Stas' Python Cookbook
✔ Applications:
ipyexperiments
✔ Tools and Cheatsheets:
bash |
conda |
git |
jupyter-notebook |
make |
python |
tensorboard |
unix
How active is development on stas00/ml-engineering?
The most recent commit recorded on stas00/ml-engineering was 15 days ago, based on the GitHub push timestamp. The repository has 1.3k forks — one of the better signals of community interest.
How many stars does stas00/ml-engineering have?
stas00/ml-engineering has 19.7k GitHub stars — refresh the page for the live number, or check github.com/stas00/ml-engineering. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
Is stas00/ml-engineering open source?
Yes — stas00/ml-engineering ships under the CC-BY-SA-4.0 license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/stas00/ml-engineering.
What else is in the AI Tools space?
stas00/ml-engineering is tracked by TopGit under the AI Tools category, alongside 16 GitHub-tagged topics. Trending and Topics pages list peer repositories of comparable stars and language.
What topics is stas00/ml-engineering associated with?
GitHub's repository topics for stas00/ml-engineering: "ai", "debugging", "gpus", "inference", "large-language-models", "llm", "machine-learning", "machine-learning-engineering", "mlops", "network", "pytorch", "scalability", "slurm", "storage", "training", "transformers". TopGit's editorial category is AI Tools.
Where can I see stas00/ml-engineering in action?
The project maintains a homepage at https://stasosphere.com/machine-learning/. The README tab on this page also usually contains screenshots and a quickstart.
Where do I read more about stas00/ml-engineering?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/stas00/ml-engineering is the definitive source.
Read full README in the tab above.
Still deciding about ml-engineering?
One click hands the question to an AI along with this page — see what it says about ml-engineering.