Location: Unfolding Intelligence (MIT Center for Art, Science and Technology, Online), Trafó House of Contemporary Arts Budapest (Online)  
Format: Interactive Sonic Instrument, Lecture Performance 
Year: 2021

Description

Songlines is a musical game, with a tribute to visual music notation, improvisation, and mnemonic devices. A songline, also called dreaming track, is one of the paths across the land (or sometimes the sky) within the animist belief system of The First Nations People of ‘Australia’ , which mark the route followed by localised "creator-beings" during the Dreaming. The paths of the songlines are recorded in traditional song cycles, stories, dance, and art, and are often the basis of ceremonies. They are a vital part of Aboriginal culture, connecting people to their land. This piece evolved as part of a research on human creativity and machine intelligence, investigating autonomous agents, game theory, altruistic probabilities. It has been selected to MIT's Unfolding Intelligence Symposium's Generative Unfoldings online exhibition. Curated by Nick Montfort, poet and MIT professor of digital media, the Generative Unfoldings exhibition consists of fourteen generative software artworks, running live in the browser. They were commissioned by the MIT Center for Art, Science & Technology (CAST) and co-presented by the MIT Trope Tank in conjunction with the 2021 CAST Symposium, “Unfolding Intelligence.” 

Sonic Instrument

When playing with the interface, there are two agents on the stage whose behaviour is based on reinforcement learning, they learn from their environment. They are seeking food provided by your touch, while trying to keep distance from the auto generated dark objects. What they see and percieve from their surrounding is the sonic instrument itself. The unfolding paths and labyrinths resemble a minimalistic algorithmic representation of their presence in their world, constructing songs based on their moving lines.

“A phase change in the designing of agents, highlights the paradoxical nature of code: Code is text that performs: It straddles the representational and performative idioms.” (Simon Penny)

At the end of each musical composition, a small statement can be read about the cultural references of songlines. What is interesting in these thought fragments is that they begin with a sentence written by a human then a transformer-based artificial language model (based on the GPT-2 model) intakes this partial sentence and predicts subsequent text from that input. This method creates a dream-like contemplation based on co-authorship, where written human words and thoughts are amplified by autonomous, non-human procedures.

Technical Notes

The reinforcement learning implementation is based on A. Karpathy's ReinforceJS library, the gpt-2 based pre-generated text fragments are made with DeepAI's text generation API. Sound generation is made with Tone.js, visualization and UI is built with p5.js. The source code is availalable at the exhibition repository (GNU/GPLv3).

Lecture Performance

“ The choices we are faced with today are especially important because digital technology so dramatically increases the ‘space of the possible’ that it includes the potential for machines that possess knowledge and will eventually want to make choices of their own.” (Albert Wenger)

The artwork and the accompanying research resulted in an online lecture performance, where influences, key concepts and examples have been introduced from the fields of mnemonic devices and cognition, algorithmic thinking, artificial intelligence, networks, fungi, labyrinths and spatial navigation. It was presented as a talk and a Q&A at Trafó House of Contemporary Arts, curated by Andrea Kovács. The video documentation will be available in 2021 fall.