Location: House of Music Hungary, Creative Sound Space
Format: Interactive multitouch musical applications
Year: 2022
Background
We have been asked to create some visual music intruments for the Creative Sound Space in the House of Music, Budapest. These pieces are exhibited in a space fullfilled with custom, non-conventional mechanic musical instruments, all of them introducing special relationships with human perception, creativity and music. The exhibition was curated by Daniel Vaczi, the physical constructions of instruments and installations are realized by Medence Csoport and the software and electronic pieces were developed by XORXOR, Bálint Bolcsó, Viktor Vicsek, Péter Márton among others.
Visual music refers to the creation of a visual analogue to musical form by adapting musical structures for visual composition. It also refers to methods or devices which can translate sounds or music into a related visual presentation. An expanded definition may include the translation of music to painting; this was the original definition of the term, as coined by Roger Fry in 1912 to describe the work of Wassily Kandinsky. There are a variety of definitions of visual music, particularly as the field continues to expand. In some recent writing, usually in the fine art world, visual music is often confused with or defined as synaesthesia, though historically this has never been a definition of visual music. Visual music has also been defined as a form of intermedia.
We ended up with three different apps, where each represents a different form of musical composition in which audio content is created using visuality. Visible shapes, colors, layouts, movements shape the melodies and sounds. We can communicat with the system through touches, and with the help of movements and gestures, we can discover the methods of more complex compositional procedures in a playful, comprehensible form.
BeatBox
Players can place various visual elements in an endless sound space. By spatially positioning the elements, sound compositions can be created. A slice of the playing space is always visible, this crop can be shifted, the camera can zoom in on it and show the continuously evolving visual instrument from a distance.
Contour
The player creates music with different keys assigned to the sounds. Each press adds a new sound to the system. This creates a network where different sounds are heard automatically, one after the other, with different probabilities. Weighted random melodies depend on the order of the sounds that are played on the keyboard. Thus, the net can be reshaped and taught while listening, live. This random weighted probability decision making method is known as a Markov Chain, and is being used in many systems from language and text generation through climate and weather modeling or music composition, just to name just a few. The network is represented by a physical system in which the positions of the elements interact with different forces (Force Directed Layout).
Memo
The player learns about the internal structure of different musical content by letting the system cut the audio content into small slices, remembering their original temporal positions, and rearranging the sound slices in space based on their similarity to each other. Thus, spectrally similar sounds get close to each other and different ones get far apart. Their similarity is based on their spectral components. These comparative methods are used in a diverse set of fields from speech recognition and scientific audio analysis to the recommendation systems of Spotify or Shazaam. The resulting map shows the musical (spectral) structure of the original pieces as unique fingerprints. Audio slices can be played and discovered through multitouch gestures.
The exhibiton is open permanently at the House of Music
Monday - Sunday: 10 AM - 6 PM