Graphic Score Sonification Installation for electronic music

Graphic Score Sonification- INSTALLATION DESCRIPTION

Graphic Score - Flyer with QR codes links to sonification videos
Graphic Score – Flyer with QR codes links to sonification videos

if (find == color) makesound

Graphic Score Installation exposed @ IOMO Galery
Installation exposed @ IOMO Galery

Graphic score sonification Installation, shape and color tracking application and their translation into electronic analog sound parameters. Complete system for studying the synesthesia of shapes and colors with sound.

Sonification of Graphic Score - Display
Display of Raspberry PI HDMI output

Built around a Raspberry PI3 (1) linux mini computer, in C ++ using the openFrameworks platform, the application scans the graphic scores, filters them by color and then sonificate the graphic shapes.

mini computer linux Raspberry PI3
mini computer linux Raspberry PI3

The installation exposed in this exhibition can analyze graphic compositions in the palette of up to 16 colors, and for 4 electronic instruments simultaneously, the sonication being performed with an analog synthesizer Elektron Analogue 4 MK2 (2) amplified by two active studio monitors (3). Visualization of the color tracking and decoding process can be done on a monitor or projector (4).

Graphic Score Installation - Main Function Diagram
Components of the installation / Prototype

Graphic Score Sonification HOW IT WORKS ?

The concept of sonification goes on the idea of ​​a paper strip, tape roll, which runs in one direction, and the graphic reading is done in a narrow area (8, 16 or 32 pixel wide) guided by CURENT PLAY CURSOR named tracking play buffer. Using the CV tracking library, the app works like a color tracking process in the play buffer. So you can extract some data from every blobs (shapes finded by CV tracker) like Y position in the score, Area, Mass, Perimeter …

how it works - Play buffer & Colors tracking
Play buffer & Colors tracking

Graphic scores are loaded one by one into the system, RPI3 mini computer (Raspberry PI3B +), and played in real time. The graphic scores are composed for 4 electronic instruments distributed on 4 tracks TRACK 01-04.

The application filters the shapes according to the 16 colors in the color palette placed at the beginning of the graphic score. Each audio track is reserved for 4 colors, 4 x 4 = 16 colors per score.

Colors - Sound Maping
Sound Mapping / Color Blobs finded data converted in sound parameters

Shape decoding is translated from blobs tracking data (the image above) into MIDI data (Musical Instrument Digital Interface) signals.

The process of matching colors / sound parameters is called Sound Mapping. From each color you can get several values ​​that correspond to the synthetic sound parameters (Note Trigger, Velocity, Pitch, Panning, Filter, LFO, Delay and Reverb).

Graphic Score Sonification Installation 1st Public Exposure

IOMO Gallery - Event Poster

The installation, under the name if (find == color) makesound, is exhibited at the IOMO gallery, between 08.05-30.05.2021 Address: Băiculești 29, 013193, Bucharest

IOMO Contemporary Art Gallery

More details about what is a Graphic Score

Flyer with QR codes links to sonification videos
Flyer with QR codes links to sonification videos
Graphic Score Sonification - Flyers details
Graphic Score Sonification – Flyers details
Graphic Score Sonification - Flyers details QR code
Graphic Score Sonification – Flyers details QR code