Machine Learning Foundations
This repo is home to the code that accompanies Jon Krohn's Machine Learning Foundations curriculum, which provides a comprehensive overview of all of the subjects — across mathematics, statistics, and computer science — that underlie contemporary machine learning approaches, including deep learning and other artificial intelligence techniques.
There are eight subjects in the curriculum, organized into four subject areas. See the "Machine Learning House" section below for detail on why these are the essential foundational subject areas:
- Linear Algebra
- 1: Intro to Linear Algebra
- 2: Linear Algebra II: Matrix Operations
- Calculus
- 3: Calculus I: Limits & Derivatives
- 4: Calculus II: Partial Derivatives & Integrals
- Probability and Statistics
- 5: Probability & Information Theory
- 6: Intro to Statistics
- Computer Science
- 7: Algorithms & Data Structures
- 8: Optimization
Later subjects build upon content from earlier subjects, so the recommended approach is to progress through the eight subjects in the order provided. That said, you're welcome to pick and choose individual subjects based on your interest or existing familiarity with the material. In particular, each of the four subject areas are fairly independent so could be approached separately.
Where and When
The eight ML Foundations subjects were initially offered by Jon Krohn as live online trainings in the O'Reilly learning platform from May-Sep 2020 (and were offered a second time from Jul-Dec 2021; see here for individual lecture dates).
To suit your preferred mode of learning, the content is now available via several channels:
- YouTube
- Linear Algebra complete playlist here and detailed blog post here
- Calculus complete playlist here
- Probability playlist is in active development (sign up for my email newsletter at jonkrohn.com to be notified of new video releases)
- In time, all of the subjects of my ML Foundations curriculum will be freely available on YouTube.
- O'Reilly (many employers and educational institutions provide free access to this platform; if you don't have access, you can get a 30-day free trial via my special SDSPOD23 code)
- Linear Algebra videos published in Dec 2020 (free hour-long lesson)
- Calculus videos published in Jan 2021 (free hour-long lesson)
- Probability and Stats videos published in May 2021 (free hour-long lesson)
- Computer Science videos published in Jun 2021 (free hour-long lesson)
- (For convenience, this publisher compiled all 28 hours of the above four video series into a single playlist here.)
- Udemy: All the Linear Algebra and Calculus content has been live in a Mathematical Foundations of ML course since Sep 2021 (free overview video here). While this course stands alone as a complete introduction to the math subjects, Subjects 5-8 will eventually be added as free bonus material.
- Open Data Science Conference: The entire series was taught live online from Dec 2020 to Jun 2021. On-demand recordings of all these trainings are now available in the Ai+ Platform.
- Book: A book deal with Pearson is in place; eventually I'll have bandwidth to work on the manuscript and pre-release chapter drafts will be available via oreilly.com.
(Note that while YouTube contains 100% of the taught content, the paid options — e.g., Udemy, O'Reilly, and ODSC — contain comprehensive solution walk-throughs for exercises that are not available on YouTube. Some of the paid options also include exclusive, platform-specific features such as interactive testing, "cheat sheets" and the awarding of a certificate for successful course completion.)
Push Notifications
To stay informed of future live training sessions, new video releases, and book chapter releases, consider signing up for Jon Krohn's email newsletter via his homepage.
Notebooks
All code is provided within Jupyter notebooks in this directory. These notebooks are intended for use within the (free) Colab cloud environment and that is the only environment currently actively supported.
That said, if you are familiar with running Jupyter notebooks locally, you're welcome to do so (note that the library versions in this repo's Dockerfile are not necessarily current, but may provide a reasonable starting point for running Jupyter within a Docker container).
The Machine Learning House
To be an outstanding data scientist or ML engineer, it doesn't suffice to only know how to use ML algorithms via the abstract interfaces that the most popular libraries (e.g., scikit-learn, Keras) provide. To train innovative models or deploy them to run performantly in production, an in-depth appreciation of machine learning theory (pictured as the central, purple floor of the "Machine Learning House") may be helpful or essential. And, to cultivate such in-depth appreciation of ML, one must possess a working understanding of the foundational subjects.
When the foundations of the "Machine Learning House" are firm, it also makes it much easier to make the jump from general ML principles (purple floor) to specialized ML domains (the top floor, shown in gray) such as deep learning, natural language processing, machine vision, and reinforcement learning. This is because, the more specialized the application, the more likely its details for implementation are available only in academic papers or graduate-level textbooks, either of which typically assume an understanding of the foundational subjects.
The content in this series may be particularly relevant for you if:
- You use high-level software libraries to train or deploy machine learning algorithms, and would now like to understand the fundamentals underlying the abstractions, enabling you to expand your capabilities
- You’re a data scientist who would like to reinforce your understanding of the subjects at the core of your professional discipline
- You’re a software developer who would like to develop a firm foundation for the deployment of machine learning algorithms into production systems
- You’re a data analyst or A.I. enthusiast who would like to become a data scientist or data/ML engineer, and so you’re keen to deeply understand the field you’re entering from the ground up (very wise of you!)
- You're simply keen to understand the essentials of linear algebra, calculus, probability, stats, algorithms and/or data structures
The foundational subjects have largely been unchanged in recent decades and are likely to remain so for the coming decades, yet they're critical across all machine learning and data science approaches. Thus, the foundations provide a solid, career-long bedrock.
Pedagogical Approach
The purpose of this series it to provide you with a practical, functional understanding of the content covered. Context will be given for each topic, highlighting its relevance to machine learning.
As with other materials created by Jon Krohn (such as the book Deep Learning Illustrated and his 18-hour video series Deep Learning with TensorFlow, Keras, and PyTorch), the content in the series is brought to life through the combination of:
- Vivid full-color illustrations
- Paper-and-pencil comprehension exercises with fully-worked solutions
- Hundreds of straightforward examples of Python code within hands-on Jupyter notebooks (with a particular focus on the PyTorch and TensorFlow libraries)
- Practical ML applications
- Resources for digging even deeper into topics that pique your curiosity
Prerequisites
Programming: All code demos will be in Python so experience with it or another object-oriented programming language would be helpful for following along with the code examples. A good (and free!) resource for getting started with Python is Al Sweigart's Automate the Boring Stuff.
Mathematics: Familiarity with secondary school-level mathematics will make the class easier to follow along with. If you are comfortable dealing with quantitative information – such as understanding charts and rearranging simple equations — then you should be well-prepared to follow along with all of the mathematics. If you discover you have some math gaps as you work through this ML Foundations curriculum, I recommend the free, comprehensive Khan Academy to fill those gaps in.
Oboe
Finally, here's an illustration of Oboe, the Machine Learning Foundations mascot, created by the wonderful artist Aglaé Bassens:
/ML-foundations