trekhleb/machine-learning-octave

Machine Learning in MatLab/Octave

For Python/Jupyter version of this repository please check homemade-machine-learning project.

This repository contains MatLab/Octave examples of popular machine learning algorithms with code examples and mathematics behind them being explained.

The purpose of this repository was not to implement machine learning algorithms using 3^rd party libraries or Octave/MatLab "one-liners" but rather to practice and to better understand the mathematics behind each algorithm. In most cases the explanations are based on this great machine learning course.

Supervised Learning

In supervised learning we have a set of training data as an input and a set of labels or "correct answers" for each training set as an output. Then we're training our model (machine learning algorithm parameters) to map the input to the output correctly (to do correct prediction). The ultimate purpose is to find such model parameters that will successfully continue correct input→output mapping (predictions) even for new input examples.

Regression

In regression problems we do real value predictions. Basically we try to draw a line/plane/n-dimensional plane along the training examples.

Usage examples: stock price forecast, sales analysis, dependency of any number, etc.

🤖 Linear Regression - example: house prices prediction.

Classification

In classification problems we split input examples by certain characteristic.

Usage examples: spam-filters, language detection, finding similar documents, handwritten letters recognition, etc.

🤖 Logistic Regression - examples: microchip fitness detection, handwritten digits recognitions using one-vs-all approach.

Unsupervised Learning

Unsupervised learning is a branch of machine learning that learns from test data that has not been labeled, classified or categorized. Instead of responding to feedback, unsupervised learning identifies commonalities in the data and reacts based on the presence or absence of such commonalities in each new piece of data.

Clustering

In clustering problems we split the training examples by unknown characteristics. The algorithm itself decides what characteristic to use for splitting.

Usage examples: market segmentation, social networks analysis, organize computing clusters, astronomical data analysis, image compression, etc.

🤖 K-means algorithm - example: split data into three clusters.

Anomaly Detection

Anomaly detection (also outlier detection) is the identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data.

Usage examples: intrusion detection, fraud detection, system health monitoring, removing anomalous data from the dataset etc.

🤖 Anomaly Detection using Gaussian distribution - example: detect overloaded server.

Neural Network (NN)

The neural network itself isn't an algorithm, but rather a framework for many different machine learning algorithms to work together and process complex data inputs.

Usage examples: as a substitute of all other algorithms in general, image recognition, voice recognition, image processing (applying specific style), language translation, etc.

🤖 Neural Network: Multilayer Perceptron (MLP) - example: handwritten digits recognition.

Machine Learning Map

The source of the following machine learning topics map is this wonderful blog post

How to Use This Repository

Install Octave or MatLab

This repository contains *.m scripts that are intended to be run in Octave or MatLab. Thus in order to launch demos you need either Octave or MatLab to be installed on you local machine. In case of MatLab you may also use its web-version.

Run Demos

In order to run the demo of your choice you should move to the chosen folder (i.e. neural-network):

cd neural-network

Launch Octave console:

octave

Launch demo script from Octave console:

demo

To see all demo variables you may launch:

whos

To exit the demo you may launch:

exit

Also be aware that demo scripts opens additional window with charts and other graphical information that is related to the running algorithm. You may find screenshots of the window that each demo will render for you on the dedicated README files for each machine learning algorithm.