Location: Balassi Hungarian Institute (Istanbul, TUR)
Format: Algorithmic Installation 
Year: 2019

Origins

Lately we’ve been asked to propose an artwork as a contribution for an exhibition on the 100th anniversary of the Bauhaus. Endre Lehel von Paks, the curator is an expert art historian in light related art, mixed, new media and the like, so he was asking us if we can contribute with something that is related to light, space, sound, and anything in-between. Since we have made projects together with Budapest Music Center that are focusing on the philosophy and aesthetics of the Bauhaus, we were very interested to take part in the process. One of our recent pieces that's called Moment’s Notice was also dealing with this artistic approach, especially focusing on the imagery and visual concepts of László Moholy Nagy. However this piece was a kind of visualization for live performance, made by musicians, so we had to think over the whole process and decided to turn its pieces into a complete audiovisual installation, including sound, movements and the visual appearance. 

Description

The term floating point refers to the fact that a number's radix point can "float"; that is, it can be placed anywhere relative to the significant digits of the number. This position is indicated as the exponent component, and thus the floating-point representation can be thought of as a kind of scientific notation. Instead of numbers and math, the piece investigates what if we are dealing with similar, proportional transitions in music, sound and spatial imagery? How do we interpolate over from one state to the other? What are the key characteristics of such movements?

The 3d visual space can be observed by the audience, they can interrupt the automation at any time by moving into the installation space. Participants are free to explore the sonic and visual components, generated by the underlying automations. Floating point merits from a diverse set of inspirations, from ancient methods of space filling techniques through the traditions of the interdisciplinary Bauhaus philosophy and visual language made by avant garde artists, to the extremities of fluxus and graphical music notation. Practically its overall existence is based on a tension between a set of some predefined rules that are contradicting an open ended flow, an incomplete causality of improvisational processes by many independent components. The musical composition is based on some machine learning algorithms (see below). The system iterates over the different possibilities of a musical connection between different melodies. This sort of melody interpolation creates a very vivid and emotionally complex sonic environment. The whole process leads to a strange, creative engagement between the raises of the artificial system and the cognitive condition of the human audience.   

Process 

early sketches

As usual, part of our creative process was sketching, trying out new ideas to see whether they work or not. This type of iterative design led to a lot of different results both in visuals and sounds. The starting point of the color palettes was the colors used by Moholy-Nagy within his paintings. Then, we were playing around with the shapes, forms and configurations that can be found in his (and other bauhaus related constructivist) works. We did this process before, in the work with live performers, but we also investigated some new algorithms and methods to bring in the scope. Also, speaking of audio, we were able to leave all the machine listening process behind (since we are generating the audio part, too). So we were able to focus more into the fields of algorithmic music creation. We were using a Kinect depth camera for user interaction. Also, we developed a custom software for the visual system in openFrameworks. The processes, changes temporal shifts are all automated. The sonic environment is made with Magenta and PureData. 

Visual components

installation view

The visual structure of Floating Point consists of different scenes that can be turned into each other smoothly. These scenes are representing different kinds of abstract, algorithmic ways to organize lines, growth, behaviours and filling of space. Thus, it can be thought of as an architectural system instead of a digital painting machine. The depth of space is important: certain events are affecting the positions, perspectives and conditions of the visual elements. All components are interconnected with the sounds, where tiny fragments, onset events or more abstract musical components are shaping the parameters of the visible domain. 

Sound

The sounds are created in a way where there is no repetition, looping, or larger fragment that have happened before. The basic components are percussion, bass and some harmonic structure synth. On one hand, their timbre is continuously shifted and modulated over time. On the other hand, melody creation is something where machine improvisation comes into the scene. We are using special neural networks (Variational AutoEncoders, VAE) provided by the open source Magenta project in order to create the melody progression. We were generating some basic, short melodies as a starting point. Then, we let the network to interpolate between those melodies. The network is pretrained on several monophonic melody progressions within different scales, and it is trying to find the most accurate answer for a certain point in the interpolation process. It raises questions about how humans can cooperate with artificial creative systems: in this case, a human composer provides some constraints, and the machine tries to improvise, based on its knowledge, that comes from its dataset.

Conclusion

Thanks to the lovely people at Balassi Institute Istanbul, we were able to install the piece and show it to a new, diverse audience. It was really nice to see that finally the piece is balancing on a state that is in between high-end artistic phenomena and more open, everyday entertaining setup. If one starts to experience a thing like this, questions arise, such as: is it these audible sounds, visible moving shapes that form a whole as an art piece? Or, is it about the whole experience where playfulness, mood, set and setting, mental position of the audience also take part?