~ The Power of the Pentatonic Scale

The following video shows Bobby McFerrin demonstrating the power of the pentatonic scale. It is a fascinating demonstration of how quickly a (western) audience of the World Science Festival 2009 adapts to an unusual tone scale:

With Tarsos the scale used in the example can be found. This is the result of a quick analysis: it becomes clear that this, in fact, a pentatonic scale with an unequal octave division. A perfect fifth is present between 255 and 753 cents:

The pentatonic scale

Tarsos analysing a scale

The pentatonic scale

featured, Computational musicology, en Tarsos

scale_bobby.png, audio_pitch_class_histogram.tex, en audio_pitch_class_histogram.pdf

~ Tarsos at 'WASPAA 2011'

During the the demo session of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics a demonstration of Tarsos was given. During the demo, the 18th of Octobre 2011 feedback was gathered.

During the conference I met interesting people and their work:

Carnatic Music Analysis: Shadja, Swara Identification and Raga Verification in Alapana using Stochastic Models
Ranjani HG, Arthi S, Sreenivas TV

Simulation of the Violin Section Sound based on the analysis of orchestra performance
Jukka Pätynen, Sakari Tervo, Tapio Lokki

Another interesting paper is Informed Source Separation: Source Coding Meets Source Separation. A demo of this can be found here.

featured, Presentation, en Tarsos

~ Tarsos at 'ISMIR 2011'

A paper about Tarsos was submitted for review at the 12th International Society for Music Information Retrieval Conference which will be held in Miami. The paper Tarsos – a Platform to Explore Pitch Scales in Non-Western and Western Music was reviewed and accepted, it will be published in this year’s proceedings of the ISMIR conference. It can be read below as well.

An oral presentation about Tarsos is going to take place Tuesday, the 25 of October during the afternoon, as can be seen on the ISMIR preliminary program schedule.

If you want to cite our work, please use the following data:

  
1
2
3
4
5
6
7
8
9
10

  
@inproceedings{six2011tarsos,
  author     = {Joren Six and Olmo Cornelis},
  title      = {Tarsos - a Platform to Explore Pitch Scales 
                in Non-Western and Western Music},
  booktitle  = {Proceedings of the 12th International 
                Society for Music Information Retrieval Conference,
                ISMIR 2011},
  year       = {2011},
  publisher  = {International Society for Music Information Retrieval}
}

Computational ethnomusicology, Research papers, Music Information Retrieval, Code, featured, en Tarsos

tarsos_ismir_paper.bibtex.txt en tarsos_ismir_2011.pdf

~ Tarsos at 'IPEM Open House'

The 25th of May 2011 Tarsos was present at the IPEM open house.

IPEM (Institute for Psychoacoustics and Electronic Music) is the research center of the Department of Musicology, which is part of the Department of Art, Music and Theater Studies of Ghent University. IPEM provides a scientific basis for the cultural and creative sector, especially for music and performance arts, and does pioneering research work on the relationship between music body movement and new technologies. The institute consists of an interdisciplinary team but also welcomes visiting researchers from all over the world. One of its aims is also to actively try and validate research results during public events and by means of user studies.

There are close relations between the Royal Conservatory Ghent, where we are located, and IPEM. There is more information about the IPEM open house available. Also available is the program of the IPEM open house 2011

Tarsos was presented using a poster, a flyer and a live demo. The poster about Tarsos and the flyer about Tarsos are both downloadable.

Computational ethnomusicology, Computational musicology, Presentation, featured, en Tarsos

tarsos_flyer.pdf, tarsos_poster.pdf, en ipem_logo.gif

~ Tarsos at 'First International Workhop of Folk Music Analysis'

Tarsos will be presented at the First International Workhop of Folk Music Analysis: Symbolic and Signal Processing:

“The First International Workhop of Folk Music Analysis: Symbolic and Signal Processing, will take place in Athens, Greece, on the 19th and 20th of May, 2011. … The purpose of the event is to gather reseachers who work in the area of computational folk music analysis, using symbolic or singal processing methods, to present their work, discuss and exchange views on the topic.”

The submitted abstract about Tarsos can be downloaded. A presentation about Tarsos is also available.

featured, Presentation, Computational musicology, Computational musicology, en Tarsos

Tarsos_FMA_2011.pdf en 2011-05-20-tarsos_at_athens.pdf

~ TarsosDSP: a small JAVA audio processing library

TarsosDSP is a collection of classes to do simple audio processing. It features an implementation of a percussion onset detector and two pitch detection algorithms: Yin and the Mcleod Pitch method.

Its aim is to provide a simple interface to some audio (signal) processing algorithms implemented in JAVA.

To make some of the possibilities clear I coded some examples.

• TarsosDSP UtterAsterisk a game that shows real-time pitch detection with YIN.
• TarsosDSP Sound Detector is simply to show how to react when (loud) sound is available.
• TarsosDSP Percussion Detector is capable of detecting percussion onsets using the method described here. It plays a sound from freesound.org when percussion is detected.

The source code of TarsosDSP is available on github.

Presentation at Newline

Saturday the 25th of March TarsosDSP was presented at Newline, a small conference organized by whitespace. Here you can download the slides I used to present TarsosDSP, I also created an introductory text on sound and Java.

Percussion detection

UtterAsterisk

Sound Detector

Code, Java, featured, TarsosDSP, en Tarsos

PitchDetector.jar, PercussionDetector.jar, tarsosDSP_presentation.pdf, sound_and_java.pdf, UtterAsterisk.jar, en SoundDetector.jar

~ Tarsos at 'Lectures on Computational Ethnomusicology'

This monday the 28th of February Tarsos will be presented at “Lectures on Computational Ethnomusicology” which is held at Izmir, Turkey. The presentation of Tarsos is available here.

Next to the interesting programme it is a great opportunity to meet Baris Bozkurt who has been working on similar research but applied to Makam music.

On wednesday the second of March there is a small seminar at Electrical and Electronics Eng. Dept. of İzmir Yüksek Teknoloji Enstitüsü where Tarsos will be presented also.

Computational ethnomusicology, Presentation, featured, en Tarsos

2011-02-28_tarsos_at_izmir.pdf, 2011-02-28-tarsos_at_izmir.odp, en programme_lectures_on_comp_ethno_izmir.pdf

~ Tarsos Live - Real Time Tone Scale Analysis

A new version of Tarsos was uploaded today and it contains an exciting (at least my kind of exciting) new feature. It is capable of real-time pitch analysis and tone scale construction. A video should make its use clear:

The immediate feedback is practical for educational purposes: it makes rather vague things like quarter tones or (uncommon) pitch intervals in general quite tangible. It could be used by singers or string players to explore microtonality or to improve their technique. Another use case is ethnomusicologic field-work: if you would want to research Kora tuning (an African harp) Tarsos could be a practical tool for real-time analysis.

Thanks to Olmo Cornelis and Wannes Gonnissen for playing the thumb piano and Saz respectively.

Computational ethnomusicology, featured, en Tarsos

~ Digital Music Research Network Workshop - Queen Mary University London

Monday the 20th and Tuesday the 21th of December I attended two workshops at The Queen Mary University of London: The Machine Listening Workshop and Digital Music Research Network One-day Workshop 2010

At the workshop I had an interesting meeting with Dan Tidhar. He researches harpsichord temperament estimation at QMUL. Together they created the Tempest web service where you can upload harpsichord audio and let the system guess the temperament. The process is described in the paper “High precision frequency estimation for harpsichord tuning classification”. Although Tarsos was not officially part of the programme I hijacked the poster sessions to show a live demo of Tarsos with Dan’s dataset.

Another interesting talk was about 2032, a tunable synthesizer with definable Harmonics. It elaborates on the ideas of Sethares about tone scales .

Computational ethnomusicology, featured, Computational musicology, en Research papers

~ Groovy Tarsos Scripting

There is more to Tarsos then meets te eye. The graphical user interface only exposes some functionality; the API exposes all of Tarsos’ capabilities.

Tarsos is programmed in Java so the API is accessible trough Java and other programming languages targeting the JVM like JRuby, Scala and Groovy. The following examples use the Groovy programming language because I find it the most aesthetically pleasing with regards to interoperability and it gets the job done without getting in your way.

To run the examples a copy of the Tarsos JAR-file needs to be added to the Classpath and the Groovy runtime must be installed correctly. I’ll leave this as an exercise for the reader: godspeed to you, brave soul. Quick protip: placing a copy of the jar in the extensions directory seems to work best, e.g. see important java directories on mac OS X.

The first example extracts pitch class histograms from a bunch of files and saves them as EPS-files. It iterates a directory recursively and handles each file that matches a given regular expression. In this example the regular expression matches all WAV-files. Batch processing is one of those things scripting is ideal for, doing the same thing with the user interface would be tedious or even mind-numbingly boring, not groovy at all indeed.

  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

  
import be.hogent.tarsos.*
import be.hogent.tarsos.util.*
import be.hogent.tarsos.util.histogram.ToneScaleHistogram
import be.hogent.tarsos.sampled.pitch.Annotation
import be.hogent.tarsos.sampled.pitch.PitchDetectionMode

dir = "/home/joren/audio"

FileUtils.glob(dir,".*.wav",true).each { file ->
        audioFile = new AudioFile(file)
        pitchDetector = PitchDetectionMode.TARSOS_YIN.getPitchDetector(audioFile)
        pitchDetector.executePitchDetection()
        //get some annotations
        annotations = pitchDetector.getAnnotations()
        //create an ambitus and tone scale histogram
        ambitusHistogram = Annotation.ambitusHistogram(annotations)
        toneScaleHisto = ambitusHistogram.toneScaleHistogram()
        //plot a smoothed version of the histogram
        p = new SimplePlot()
        p.addData 0, toneScaleHisto.gaussianSmooth(0.2)
        p.save FileUtils.basename( file) + ".eps"
}

The second example uses functionality that is currently only available trough the API. It takes a MIDI-file and synthesizes it to a wave file using an arbitrary scale. In this case 10-TET. The heavy-work is done by the Gervill synthesizer. The resulting file is available for download, micro—macro?—tonal Bach is great: BWV 1013 in 10-TET. The result of an analysis with Tarsos on the synthesized audio clearly shows an interval of 120 cents with some deviations.

  
1
2
3
4
5
6
7
8
9
10
11
12
13

  
import java.io.File
import be.hogent.tarsos.midi.MidiToWavRenderer
import be.hogent.tarsos.util.ScalaFile

midiFile = new File("BWV_1013.mid")
outFile = new File("out.wav")

tuning = [0,120,240,360,480,600,720,840,960,1080] as double []

MidiToWavRenderer renderer
renderer = new MidiToWavRenderer()
renderer.setTuning(tuning)
renderer.createWavFile(midiFile, outFile)

BWV 1013 10-TET by jorensix

An extended version of this second example script could be used to generate a dataset with audio and corresponding tone scale information on the fly. The dataset could then be used as a baseline.

The API is not yet well documented and is still in flux or more correctly: superflux. Note to self: I will provide documentation and a number of useful examples when the dust settles down. I’m not even sure if I will stick with Groovy. Scala has a nice Lispy feel to it and seems more developed. Groovy has a less steep learning curve, especially if you have some experience with Ruby. JRuby is also nice but the interoperability with legacy Java looks like an ugly hack.

EPS export as PNG

10-TET in Tarsos

Code, Java, featured, en Tarsos

medium.png, 120.png, en BWV_1013_10-TET.mp3

~ Tarsos Presented at the "Perspectives for Computational Musicology" Symposium

Yesterday Tarsos was publicly presented at the symposium Perspectives for Computational Musicology in Amsterdam. The first public presentation of Tarsos, excluding this website. The symposium was organized by the Meertens Institute on the occasion of Peter van Kranenburg’s PhD defense.

The presentation included a live demo of a daily build of Tarsos (a Friday evening build) which worked, surprisingly, without hiccups. The presentation was done by Olmo Cornelis. This was the small introduction:

Tarsos – a Platform for Pitch Analysis of Ethnic Music
Ethnic music is a vulnerable cultural heritage that has received only recently more attention within the Music Information Retrieval community. However, access to ethnic music remains problematic, as this music does not always correspond to the Western concepts of music and metadata that underlie the currently available content-based methods. During this lecture, we like to present our current research on pitch analysis of African music. TARSOS, a platform for analysis, will be presented as a powerful tool that can describe and compare scales with great detail.

To give Tarsos a try ou can start Tarsos using JAVA WebStart or download the executable Tarsos JAR-file. A JAVA 1.5 runtime is required.

Computational ethnomusicology, Music Information Retrieval, featured, en Presentation

~ Tarsos User Interface Prototype

Tarsos now has an easy to use drag and drop User Interface. It can be used to extract tone scale information from audio files.

Start tarsos using JAVA WebStart.

Drag and drop works for scala tone scale files and different kinds of audio files. Audiofiles are transcoded automagically using an embedded ffmpeg binary which is platform dependend. It works on linux and windows, on other platforms only WAV files are supported.

Some of the current features:

• Scala file extraction from audio
• Real time pitch tracking
• Real time pitch class histogram visualization
• Alignment of pitch intervals with histogram using mouse dragging
• MIDI Tuning messages based on scala files.
• Histogram smoothing functions
• … a lot more is available under the hood.

Analysis

Browser

Computational ethnomusicology, Java, featured, en Tarsos

~ Rendering MIDI Using Arbitrary Tone Scales

Tarsos can be used to render MIDI files to audio (WAV) files using arbitrary tone scales. This functionallity can be used to (automatically) verify tone scale extraction from audio files. Since I could not find a dataset with audio and corresponding tone scales creating one using MIDI seemed a good idea.

MIDI files can be found in spades, tone scales on the other hand are harder to find. Luckily there is one massive source, the Scala Tone Scale Archive: A large collection of over 3700 tone scales.

Using Scala tone scale files and a midi files a Tone Scale – Audio dataset can be generated. The quality of the audio depends on the (software) synthesizer and the SoundFont used. Tarsos currently uses the Gervill synthesizer. Gervill is a pure Java software synthesizer with support for 24bit SoundFonts and the MIDI tuning standard.

How To Render MIDI Using Arbitrary Tone Scales with Tarsos

A recent version of the JRE needs to be installed on your system if you want to use Tarsos. Tarsos itself can be downloaded in the form of the Tarsos JAR Package.

Currently Tarsos has a Command Line Interface. An example with the files you can find attached:

  


  
java -jar tarsos.jar --midi BWV_1007.mid --scala 120.scl --out bach.wav

The result of this command should yield an audio file that sounds like the cello suites of bach in a nonsensical tone scale with steps of 120 cents. Executing tone scale extraction on the generated audo yields the expected result. In the pich class histogram every 120 cents a peak can be found.

To summarize: by rendering audio with MIDI and Scala tone scale files a dataset with tone scale – audio information can be generated and tone scale extraction algorithms can be tested on the fly.

This method also has some limitations. Because audio is rendered there is no (background) noise, no fluctuations in pitch and timbre,… all of which are present in recorded audio. So testing testing tone scale extraction algorithms on recorded audio remains advised.

120 Cents difference

Computational ethnomusicology, Music Information Retrieval, featured, en Tarsos

bach_BWV_1007_120.mp3, 120.scl.txt, BWV_1007.mid, en tarsos.jar

~ Tone Scale Matching With Tarsos

Tarsos can be used to search for music that uses a certain tone scale or tone interval(s). Tone scales can be defined by a Scala tone scale file or an exemplifying audio file. This text explains how you can use Tarsos for this task.

Search Using Scala Tone Scale Files

Scala files are text files with information about a tone scale. It is used to share and exchange tone scales. The file format originates from the Scala program :

Scala is a powerful software tool for experimentation with musical tunings, such as just intonation scales, equal and historical temperaments, microtonal and macrotonal scales, and non-Western scales. It supports scale creation, editing, comparison, analysis, …

The Scala file format is popular because there is a library with more than 3000 tone scales available on the Scala website.

Tarsos also understands Scala files. It is able to create a pitch class histogram using a gaussian mixture model. A technique described in A. C. Gedik, B.Bozkurt, 2010, "Pitch Frequency Histogram Based Music Information Retrieval for Turkish Music ", Signal Processing, vol.10, pp.1049-1063. (doi:10.106/j.sigpro.2009.06.017).

An example should make things clear. Lets search for an interval of 300 cents or exactly three semitones. A scala file with this interval is easy to define:

  
1
2
3
4
5
6
7

  
! example.scl
! An example of a tone interval of 300 cents
Tone interval of 300 cents
2
!
900
1200.0

The next step is to create a histogram with an interval of 300 cents. In the block diagram this step is called “Peak histogram creation”. The Similarity calculation step expects a list of histograms to compare with the newly defined histogram. Feeding the similarity calculation with the western12ET tone scale and a pentatonic Indonesian Slendro tone scale shows that a 300 cents interval is used in the western tone scale but is not available in the Slendro tone scale.

This example only uses scala files, creating histograms is actually not needed: calculating intervals can be done using the scala file itself. This changes when audio files are compared with each other or with scala files.

Search Using Audio Files

When audio files are fed to the algorithm additional steps need to be taken.

1. First of all pitch detection is executed on the audio file. Currently two pitch extractors are implemented in pure Java, it is also possible to use an external pitch extractor such as aubio
2. Using pitch annotations a Pitch Histogram is created.
3. Peak detection on the Pitch Histogram results in a number of peaks, these should represent the distinct pitch classes used in the musical piece.
4. With the pitch classes a clean peak histogram is created during the Peak Histogram construction phase.
5. Finally the Peak histogram is matched with other histograms.

The last two steps are the same for audio files or scala files.

Using real audio files can cause dirty histograms. Determining how many distinct pitch classes are used is no trivial task, even for an expert (human) listener. Tarsos should provide a semi-automatic way of peak extraction: a best guess by an algorithm that can easily be corrected by a user. For the moment Tarsos does not allow manual intervention.

Tarsos

To use tarsos you need a recent java runtime (1.6) and the following command line arguments:

  
1
2

  
java -jar tarsos.jar rank --detector TARSOS_MPM 
--needle audio.wav --haystack scala.scl other_audio.wav other_scala_file.scl

Data flow audio

Data flow scala

300 cents interval

12ET and 300 cents

Slendro and 300 cents

Realistic Tone scale

Computational ethnomusicology, Music Information Retrieval, featured, en Tarsos

data_flow_scala.png, data_flow_audio.png, en tarsos.jar

~ Static Code Analysis For Java Using Eclipse

This post is about the tools I use to keep the source code of Tarsos reasonably clean, consistent and readable. Static code analysis can be of great help if you want to maintain strict coding standards and follow language idioms. Some of the patterns they can detect for you:

• Dead code – unused variables, parameters, methods
• Suboptimal code – wasteful resource usage
• Overcomplicated expressions – unnecessary if statements, for loops that could be while loops
• Duplicate code – copied/pasted code is a code smell.
• Formatting inconsistencies, e.g. variable modifier order

And even more subtle, but equally important:

• Resource management: is a resource handled (closed) correctly on all possible code paths?
• Abstraction level: is it needed to expose the concrete type of an object or could an (abstract) supertype or even an interface be used instead?
• …

In a previous life I used .NET and the static code analysis tools FxCop & StyleCop. FxCop operates on bytecode (or intermediate language in .NET parlance) level, StyleCop analyses the source code itself. Tarsos uses JAVA so I looked for JAVA alternatives and found a few.

• PMD & Checkstyle both operate on source code level.
• FindBugs operates on bytecode level.

On freesoftwaremagazine.com there is an article series on JAVA static code analysis software. It covers PMD and FixBugs and integration in Eclipse. It does not cover Checkstyle. Checkstyle is essentialy the same as PMD but it is better integrated in eclipse: it checks code on save and uses the standard ‘Problems’ interface, PMD does not.

To fix problems Eclipse save actions can save you some time. IBM has an article on how to keep your code clean using Eclipse.

Continuous testing is also a really nice thing to have: detecting unexpected behavior while refactoring/programming can prevent unnecessary bug hunts. A video about immediate feedback using continuous testing makes this clear.

Another tip is a more philosophical one: making your code and code revisions publicly available makes you think twice before implementing (and subsequently publishing) a quick and dirty hack. Tarsos is available on github.

References

• FxCop & StyleCop
• Checkstyle
• Find bugs
• PDM
• Infinitest
• Github

PMD

Checkstyle

Java, Code, en featured

~ YIN Pitch Tracker in JAVA

To make Tarsos more portable I wrote a pitch tracker in pure JAVA using the YIN algorithm based on the implementation in C of aubio. The implementation also uses some code written by Karl Helgasson and Teun de Lange.

It can be used to perform real time pitch detection or to analyse files. To use it as a real time pitch detector just start the JAR-file by double clicking. To analyse a file execute one of the following. The first results in a list of annotations (text), the second shows the annotations graphically.

  
1
2

  
java -jar pitch_detector_yin.jar  flute.novib.mf.C5B5.wav
java -jar pitch_detector_yin.jar  --file flute.novib.mf.C5B5.wav

The provided flute sample is from The Musical Samples library of the University of Iowa and converted to mono wav. The source code of the pitch tracker can be found below.

Screenshot

Research papers, featured, Music Information Retrieval, Code, Java, en Tarsos

flute.novib.mf.C5B5.wav, pitch_detector_yin.jar, en pitch_detector_yin_src.zip

~ Tarsos on GitHub

The JAVA software program we are developing is called Tarsos and can now be found on GitHub. GitHub is a web-based hosting service for projects that use the Git version control system.

Currently Tarsos is a collection of Java classes to create, compare and process pitch-frequency data using histograms. In it’s current state it is not usable for end-users.

Credits

Tarsos is developed at University College Ghent, Faculty of Music and uses a number of open source libraries:

• Gervill: a software sound synthesizer, supports the MIDI Tuning Standard. API.
• Jave: a wrapper for ffmpeg.
• Apache Commons Math: a library of lightweight, self-contained mathematics and statistics components API.
• JASS: a unit generator based audio synthesis programming environment. API.
• Java-getopt: a port of the GNU getopt family of functions. API.
• Ptplot a 2D plotting library. API.

Computational ethnomusicology, featured, Code, Java, en Tarsos

Partners

• University College Ghent, Faculty of Music
• Institute of Psychoacoustics and Electronic Music

featured