tyiannak/pyAudioAnalysis

A Python library for audio feature extraction, classification, segmentation and applications

Developed at MagCIL, the Multimodal Analysis Group of the Institute of Informatics and Telecommunications, NCSR "Demokritos", Athens, Greece.

This is general info. Click here for the complete wiki and here for a more generic intro to audio data handling

General

pyAudioAnalysis is a Python library covering a wide range of audio analysis tasks. Through pyAudioAnalysis you can:

• Extract audio features and representations (e.g. mfccs, spectrogram, chromagram)
• Train, parameter tune and evaluate classifiers of audio segments
• Classify unknown sounds
• Detect audio events and exclude silence periods from long recordings
• Perform supervised segmentation (joint segmentation - classification)
• Perform unsupervised segmentation (e.g. speaker diarization) and extract audio thumbnails
• Train and use audio regression models (example application: emotion recognition)
• Apply dimensionality reduction to visualize audio data and content similarities

Installation

• Clone the source of this library: git clone https://github.com/tyiannak/pyAudioAnalysis.git
• Install dependencies: pip install -r ./requirements.txt
• Install using pip: pip install -e .

An audio classification example

More examples and detailed tutorials can be found at the wiki

pyAudioAnalysis provides easy-to-call wrappers to execute audio analysis tasks. Eg, this code first trains an audio segment classifier, given a set of WAV files stored in folders (each folder representing a different class) and then the trained classifier is used to classify an unknown audio WAV file

from pyAudioAnalysis import audioTrainTest as aT
aT.extract_features_and_train(["classifierData/music","classifierData/speech"], 1.0, 1.0, aT.shortTermWindow, aT.shortTermStep, "svm", "svmSMtemp", False)
aT.file_classification("data/doremi.wav", "svmSMtemp","svm")

Result: (0.0, array([ 0.90156761, 0.09843239]), ['music', 'speech'])

In addition, command-line support is provided for all functionalities. E.g. the following command extracts the spectrogram of an audio signal stored in a WAV file: python audioAnalysis.py fileSpectrogram -i data/doremi.wav

Further reading

Apart from this README file, to bettern understand how to use this library one should read the following:

• Audio Handling Basics: Process Audio Files In Command-Line or Python, if you want to learn how to handle audio files from command line, and some basic programming on audio signal processing. Start with that if you don't know anything about audio.
• Intro to Audio Analysis: Recognizing Sounds Using Machine Learning This goes a bit deeper than the previous article, by providing a complete intro to theory and practice of audio feature extraction, classification and segmentation (includes many Python examples).
• The library's wiki
• How to Use Machine Learning to Color Your Lighting Based on Music Mood. An interesting use-case of using this lib to train a real-time music mood estimator.
• A more general and theoretic description of the adopted methods (along with several experiments on particular use-cases) is presented in this publication. Please use the following citation when citing pyAudioAnalysis in your research work:

@article{giannakopoulos2015pyaudioanalysis,
  title={pyAudioAnalysis: An Open-Source Python Library for Audio Signal Analysis},
  author={Giannakopoulos, Theodoros},
  journal={PloS one},
  volume={10},
  number={12},
  year={2015},
  publisher={Public Library of Science}
}

For Matlab-related audio analysis material check this book.

Author

Theodoros Giannakopoulos, Director of Research in Multimodal Machine Learning at MagCIL, the Multimodal Analysis Group of the Institute of Informatics and Telecommunications, of the National Center for Scientific Research "Demokritos"

About MagCIL

This repository comes from MagCIL, the Multimodal Analysis Group of the Institute of Informatics and Telecommunications at the National Centre for Scientific Research "Demokritos" in Athens, Greece. We work on multimodal deep learning for audio, speech, music and video, from fundamental research to deployed systems.

• Group website: https://magcil.github.io
• Publications: https://magcil.github.io/pub.html
• All our open-source code and data: https://magcil.github.io/code.html
• Work with us, in industry projects or research consortia: https://magcil.github.io/collaborate.html

Other tools from the group:

• deep_audio_features — training and using CNNs for audio classification (PyTorch)
• deepaudio-x — a PyTorch framework for audio classification
• deepaudio-lab — a no-code platform for training, evaluating and deploying audio classifiers
• amvoc — analysis of mouse vocal communication
• paura — real-time audio recording and analysis

The full list is at https://github.com/magcil.