Location: Researcher’s Night, Goethe Institut (Budapest, H)
Format: Twisted language models for permutational glitch robotics 
Year: 2021

Description
As an ongoing collaboration with Goethe Institut Budapest, we had a chance to work and play with a humanoid robot and do some artistic research on how robots, artificial intelligence and automation affects our culture and society. Since we are really interested in the concept of the so called mind-body problem, spiritual practice, emergent behaviour, coding and artificial language, we are examining imagination and higher consciousness within the context of automated processes and the nature of algorithmic decisions. 

The installation examines the boundaries between human and non-human cognition. In our human culture, we reinforce our lives, our decisions, our image of reality in ourselves again and again through various rituals. Learning, playing, repeating, imitating help you navigate everyday challenges and navigate more and more complex systems from time to time. Artificial intelligence, software and robots, as well as emergent systems found in nature (animal swarms, plant ecosystems) or abstract memes and unimaginable deities flowing through consciousness, can be considered as cognitive beings living with us at different levels. Do our human actions, our rituals, differentiate us from these? Just as our repetitive, repetitive workflows are being taken over by robots these days, can we outsource the processes that help us experience reality? Can an artificial entity pray for us? Can imagination be automated, or can contemplation to attain higher consciousness? Nao lives outside on the street like yogis, gurus, wanderers, beggars, ascetics. An oracle, a palmist, a court fool and a speculative algorithmic calculator in one. He advises the visitor, outlines permutation possibilities, tells the future, begs and prays, his movements and words act as a source of joy, happiness, astringent dystopias and hope, he himself is the tireless, endless, 0/24 hour magical realist personal coach.

Computer vision, generative tarot cards
Visitors could communicate with the robot through touch and some custom made graphic symbols. Those symbols were printed on cards and the robot could recognize them upon showing them. The selection of the cards mimicked the act of divination and tarot rituals, where an expert can read the meaning of special constellation of the cards. The graphics are generated characters, or avatars, based on simple rules of weighted randomness and permutation. Each of them are very similar, but non of them identical. The cards are anchors and reminders: they are gifts that can be taken away after the experience. For recognition, we were using Nao's built in ALVisionRecognition system.   
 

Generative avatars, prepared for the printed cards, used for interacting with the robot
 

Language and texts
As a basis for the language, we started investigating methods to generate human-like texts. Since we had some experiments with basic n-gram permutations based on markov chains, we decided to use a more recent method, where we can search from a set of predefined text, finetuning the system based on its corpus, and let the generated outcome unfold. Finally we used the GPT-Neo for the raw text generation. GPT Neo is an implementation of model & data parallel GPT3-like models using the mesh-tensorflow library. Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive language model that uses deep learning to produce human-like text. It is the third-generation language prediction model in the GPT-n series (and the successor to GPT-2) created by OpenAI, a San Francisco-based artificial intelligence research laboratory. The texts we chose are all freely reusable and available on publicly available datasets, originating at the Internet Archive and Project Gutenberg. We were selecting texts from i-Ching, tea divination ceremonies, morning and evening prayers, healthy life cookbook resources in order to fine tune GPT-Neo for weird psychedelic improvisational sessions.

For speech synthesis, we were using a regular text to speech software, constantly altering its speed, pitch and pronunciation with randomly injected parameters and extra characters, which gave a glitchy, androgyn character for the robot. Timing, interaction subroutines, strange out of context sounds gave a very intimate tone for the whole interaction cycle. 

Some written lines and glitched audio synthesis examples of Nao Bot Divination Impro (En):

"Learn to the Mountain apparent to the needy. Small accomplishments are possible. Don't fight this malady until you've already won it. Misfortune! Keep this melon growing in the shade. It leads to marriage, It leads to marriage. Hexagram Three: The Mountain towers overhead, yet remains this side of Heaven: The Superior Person waits for wisdom and clarity before exacting Justice, then lets no protest sway him. He rests beneath a dead tree, trying to make sense of this dark."

"Always keep the records of the writer--there are vast numbers of persons of intellect, the world over, who place reliance upon omens. That there is some good ground for belief in some omens seems indisputable. Whether this has arisen as the result of experience, by the following of some particular event close upon the heels of signs observed, or whether it has been an intuitive science, in which provision has been used to afford an interpretation, is not quiteclear, and be tokens adversities, but the presence of the hammer there denotes triumph over these, a sign confirmed by the hat on the side."

"Always keep me from anger and bringeth life eternal. May I die, and grant, that I may keep the example of my faith and a Christian love, or err in sundry other things and are not seduced by the inconurity of another living thing. Keep me from arrogant pomp, or idleness and lounging, from the snares and nets of the devil. Grant, that I may serve Thee with a pure soul and an undefiled body in true faith, as did pious and chaste Joseph in Egypt."

Since the intention was to let Nao Bot Guru speak Hungarian, at the end of the generation pipeline, we filtered through the resulting text over the Google Translate API, which gave an extra layer of strangeness to the texts. Sometimes finetuning, translating between languages back and forth felt like applying semantic compressors and filters on musical or sonic content. 

Some written lines and glitched audio synthesis examples of Nao Bot Divination Impro (Hu):

"Tanulj a hegyi szülőnek a rászorulóknak. Kicsi eredmények lehetségesek. Ne harcolj ezzel a betegséggel, amíg már nem nyerted meg. Szerencsétlenség! Tartsd ezt a divatot az árnyékban, Házassághoz vezet, Házassághoz vezet. Háromszög: A hegytornyok a feje fölött, de továbbra is a mennyország ezen oldala: A felettes személy bölcsességre és világosságra vár az igazságosság előtt, majd nem hagyja, hogy tiltakozás uralkodjon rajta. Egy döglött fa alatt nyugszik, és megpróbálja megérteni ezt a sötétséget."

"Mindig őrizd meg az író nyilvántartását-hatalmas az értelmiségi személyek száma világszerte, akik az előjelekre támaszkodnak. Az, hogy van valami jó alap arra, hogy bizonyos előjelekben hinni lehessen, vitathatatlannak tűnik. Nem egészen világos, hogy ez a tapasztalatok eredményeképpen, a jelek észleléséig közel álló esemény nyomán következett -e be, vagy intuitív tudomány volt, amelyben a rendelkezéseket értelmezésre használták, és felidézi a csapásokat de a kalapács jelenléte diadalot jelent ezek felett, ezt a kalap megerősíti az oldalán. "

"Mindig tarts meg harag és örök életet hoz. Hadd haljak meg, és adjam meg, hogy megtartsam hitem és keresztény szeretetem példáját, vagy tévedhessek más dolgokban, és ne csábítsak el egy másik élőlény bizonytalanságától. Tarts meg az arrogáns pompától, a tétlenkedéstől és a heverészéstől az ördög csapdáitól és hálóitól. Add, hogy tiszta hittel és tisztátalan testtel szolgálhassak téged igaz hitben, akárcsak a jámbor és tisztaságos József Egyiptomban, és kegyelmesen vegyen be engem."

Performance & Installation Views (Goethe Institut, Researcher's Night)