TarsosDSP

TarsosDSP

TarsosDSP is a Java library for audio processing. Its aim is to provide an easy-to-use interface to practical music processing algorithms implemented, as simply as possible, in pure Java and without any other external dependencies. The library tries to hit the sweet spot between being capable enough to get real tasks done but compact and simple enough to serve as a demonstration of how DSP algorithms work.

TarsosDSP features an implementation of a percussion onset detector and a number of pitch detection algorithms: YIN, the Mcleod Pitch method and a “Dynamic Wavelet Algorithm Pitch Tracking” algorithm. Also included is a Goertzel DTMF (Dual tone multi frequency) decoding algorithm, a time stretch algorithm (WSOLA), resampling, filters, simple synthesis, some audio effects, and a pitch shifting algorithm.

To show the capabilities of the library, TarsosDSP example applications are available. Head over to the TarsosDSP release directory for freshly baked binaries and code smell free (that is the goal anyway), oven-fresh sources.

Quickly Getting Started with TarsosDSP

The source code of TarsosDSP is compatible with Java 11.

Add this to your build.gradle file:

repositories {
    maven {
        name = "TarsosDSP repository"
        url = "https://mvn.0110.be/releases"
    }
}
dependencies {
    implementation 'be.tarsos.dsp:core:2.5'
    implementation 'be.tarsos.dsp:jvm:2.5'
}

Add this to your pom.xml file:

<repository>
  <id>be.0110.repo-releases</id>
  <name>0110.be repository</name>
  <url>https://mvn.0110.be/releases</url>
</repository>
<dependency>
  <groupId>be.tarsos.dsp</groupId>
  <artifactId>core</artifactId>
  <version>2.5</version>
</dependency>
<dependency>
  <groupId>be.tarsos.dsp</groupId>
  <artifactId>jvm</artifactId>
  <version>2.5</version>
</dependency>

Citing TarsosDSP

Some information about TarsosDSP can be found in the paper TarsosDSP, a Real-Time Audio Processing Framework in Java, by Joren Six, Olmo Cornelis, and Marc Leman, in Proceedings of the 53rd AES Conference (AES 53rd), 2014. If you use TarsosDSP in academic research, please cite this paper.

@inproceedings{six2014tarsosdsp,
  author      = {Joren Six and Olmo Cornelis and Marc Leman},
  title       = {{TarsosDSP, a Real-Time Audio Processing Framework in Java}},
  booktitle   = {{Proceedings of the 53rd AES Conference (AES 53rd)}},
  year        =  2014
}

Building TarsosDSP with Gradle

To build TarsosDSP from source the gradlew script should get you started. The following commands fetch the source and builds the library:

git clone --depth 1 https://[email protected]/JorenSix/TarsosDSP.git
cd TarsosDSP
./gradlew build

Source Code Organization & Developing

The library is separated into three module folders: the main core functionality is found in core/src/main/java, TarsosDSP example applications are found in examples/src/main/java, and JVM audio I/O in jvm/src/main/java.

TarsosDSP Examples

To see TarsosDSP in action there are a number of examples. These examples have either a graphical user interface or a command line interface. To see a window in which you can start all GUI examples the jar file needs to be started without arguments:

gradle shadowJar
java -jar examples/build/libs/examples-all.jar

To use the command line examples, execute the jar with the example name as first argument. For example java -jar examples/build/libs/examples-all.jar feature_extractor pitch audio.mp3 starts the feature_extractor application to extract pitch from an audio file.

To get a list of all command line examples: java -jar examples/build/libs/examples-all.jar list

Credits

TarsosDSP was developed at University College Ghent, School of Arts between 2009 and 2013, from late 2013 the project is supported by University Ghent, IPEM.

TarsosDSP borrows algorithms from various other libraries and research papers. Below, a complete list of credits can be found.

• The onset detector implementation is based on a VAMP plugin example by Chris Cannam at Queen Mary University, London. The method is described in Drum Source Separation using Percussive Feature Detection and Spectral Modulation by Dan Barry, Derry Fitzgerald, Eugene Coyle and Bob Lawlor, ISSC 2005.
• For the implementation of the YIN pitch tracking algorithm. Both the the YIN paper and the GPL'd aubio implementation were used as a reference. Matthias Mauch (of Queen Mary University, London) kindly provided the FastYin implementation which uses an FFT to calculate the difference function, it makes the algorithm up to 3 times faster.
• The Average Magnitude Difference (AMDF) pitch estimation algorithm is implemented by Eder Souza and adapted for TarsosDSP by myself.
• For the MPM pitch tracking algorithm, the paper titled A Smarter Way To Find Pitch by Philip McLeod and Geoff Wyvill was used.
• The Dynamic Wavelet pitch estimation algorithm is described in Real-Time Time-Domain Pitch Tracking Using Wavelets by Eric Larson and Ross Maddox. The implementation within TarsosDSP is based on the implementation in the Dynamic Wavelet Algorithm Pitch Tracking library by Antoine Schmitt, which is released under the MIT open source license, a license compatible with the GPL.
• The audio time stretching algorithm is described in An Overlap-Add Technique Based on Waveform Similarity (WSOLA) For High Quality Time-Scale Modifications of Speech by Werner Verhelst and Marc Roelands. As a reference implementation the WSOLA implementation by Olli Parviainen in the LGPL SoundTouch - an open-source audio processing library was used.
• The FFT implementation used within TarsosDSP is by Piotr Wendykier and is included in his GPL'd JTransforms library. JTransforms is the first, open source, multithreaded FFT library written in pure Java.
• The sample rate conversion feature is implemented by Laszlo systems in the GPL'd libresample4j library. libresample4j is a Java port of Dominic Mazzoni's libresample 0.1.3, which is in turn based on Julius Smith's Resample 1.7 library
• Various FFT window functions are done by Damien Di Fede and Corban Brook for the GPL'd Minim project.
• Beat induction based on onsets and saliences is done using code from Simon Dixon's BeatRoot system. The software is licensed under the GPL. The algorithm is documented in the 2001 JNMR paper Automatic Extraction of Tempo and Beat From Expressive Performances and in the 2007 JNMR article Evaluation of the Audio Beat Tracking System BeatRoot
• A complex domain onset detection function is implemented using Aubio as an inspiration. Aubio, by Paul Brossier, contains very clean object-oriented C code, the cleanest C code I have ever seen. The algorithm is described in Complex Domain Onset Detection for Musical Signals by Christopher Duxbury, Mike E. Davies, and Mark B. Sandler, in Proceedings of the Digital Audio Effects Conference, DAFx-03, pages 90-93, London, UK, 2003
• An implementation of the Constant-Q transform by Karl Helgason for the GPL'd RasmusDSP project has been adapted for use in TarsosDSP. More information about the Constant-Q transform can be found in the following papers Calculation of a Constant Q Spectral Transform by Judith C. Brown, An Efficient Algorithm for the Calculation of a Constant Q Transform, by Judith C. Brown and Miller S. Puckette, and The Constant Q Transform by Benjamin Blankertz
• The Haar Wavelet Transform implemented in TarsosDSP is based on the pseudocode found in Wavelets Made Easy by Yves Nievergelt.
• The lifting scheme wavelet package is developed by Ian Kaplan and is based on the concepts explained in Ripples in Mathematics: The Discrete Wavelet Transform by A. Jensen and A. la Cour-Harbo, Springer. It is only slightly modified for easy use in TarsosDSP
• The frequency domain pitch shifter is developed by Stephan M. Bernsee and is based on the concepts explained in Pitch Shifting Using The Fourier Transform. For the moment it is a rather direct translation of the C implementation. It was released under a 'Wide Open License'. As the name of the license suggests, it is a license without much restrictions.
• The granulator is developed for the beads project. It was released under a GPL-license.

Changelog

Version 1.0 (2012-04-24)

First release which includes several pitch trackers and a time stretching algorithm, amongst other things. Downloads and javadoc API can be found at the TarsosDSP release directory

Version 1.1 (2012-06-4)

Changed how the audio dispatcher stops. Added StopAudioProcessor. Added FastYin implementation by Matthias Mauch Added AMDF pitch estimator by Eder Souza

Version 1.2 (2012-08-21)

Modified the interface of PitchDetector to return a more elaborate result structure with pitch, probability and a boolean "is pitched". Added an implementation of an envelope follower or envelope detector.

Version 1.3 (2012-09-19)

TarsosDSP can do audio synthesis now. The first simple unit generators are included in the library. It has a new audio feature extraction feature, implemented in the FeatureExtractor example. Added ASCII-art to the source code (this is the main TarsosDSP 1.3 feature).

Version 1.4 (2012-10-31)

Included a resample feature, implemented by libresample4j. Together with the WSOLA implementation, it can be used for pitch shifting (similar to Phase Vocoding). A pitch shifting example (both with a CLI and a UI) is added in the 1.4 version of the TarsosDSP library as well.

Version 1.5 (2013-04-30)

Converted TarsosDSP to maven. This is known as the Malaryta-release. The "Malaryta" release is provided to you by RikkiMongoose (idea, documents, git things) and Ultar (converting to maven, refactoring). Malaryta is the capital of Malaryta Raion, Brest Region in the Republic of Belarus. Both developers spent their childhood in Brest, and think that the title Malaryta is as strange as Ubuntu or Whis