Graphic Score Sonification- INSTALLATION DESCRIPTION
if (find == color) makesound
Graphic score sonification Installation, shape and color tracking application and their translation into electronic analog sound parameters. Complete system for studying the synesthesia of shapes and colors with sound.
Built around a Raspberry PI3 (1) linux mini computer, in C ++ using the openFrameworks platform, the application scans the graphic scores, filters them by color and then sonificate the graphic shapes.
The installation exposed in this exhibition can analyze graphic compositions in the palette of up to 16 colors, and for 4 electronic instruments simultaneously, the sonication being performed with an analog synthesizer Elektron Analogue 4 MK2 (2) amplified by two active studio monitors (3). Visualization of the color tracking and decoding process can be done on a monitor or projector (4).
Graphic Score Sonification HOW IT WORKS ?
The concept of sonification goes on the idea of a paper strip, tape roll, which runs in one direction, and the graphic reading is done in a narrow area (8, 16 or 32 pixel wide) guided by CURENT PLAY CURSOR named tracking play buffer. Using the CV tracking library, the app works like a color tracking process in the play buffer. So you can extract some data from every blobs (shapes finded by CV tracker) like Y position in the score, Area, Mass, Perimeter …
Graphic scores are loaded one by one into the system, RPI3 mini computer (Raspberry PI3B +), and played in real time. The graphic scores are composed for 4 electronic instruments distributed on 4 tracks TRACK 01-04.
The application filters the shapes according to the 16 colors in the color palette placed at the beginning of the graphic score. Each audio track is reserved for 4 colors, 4 x 4 = 16 colors per score.
Shape decoding is translated from blobs tracking data (the image above) into MIDI data (Musical Instrument Digital Interface) signals.
The process of matching colors / sound parameters is called Sound Mapping. From each color you can get several values that correspond to the synthetic sound parameters (Note Trigger, Velocity, Pitch, Panning, Filter, LFO, Delay and Reverb).
Graphic Score Sonification Installation 1st Public Exposure
The installation, under the name if (find == color) makesound, is exhibited at the IOMO gallery, between 08.05-30.05.2021 Address: Băiculești 29, 013193, Bucharest
More details about what is a Graphic Score