Workshops 2013/14 winter [Interaction, Process, Design]
Interaction Modalities, Poetic Interfaces (Bergen Academy of Art & Design)
A collection ollection of resources, code snippets, material and references from the workshops at Bergen Academy of Art & Design, Norway, 2013. These workshops are focusing on the communication of ideas through different interaction modalities, creating models through making (prototypes), learning from mistakes, diving in the unexpected.
The first session (Interaction Modalities) is giving an overall introduction to the term Interaction: the natural feedback between several elements that are affecting each other in a conversation. Interaction involves a diverse set of fields such as perception, cognitive aspects, psychology, formal systems, logic, language (auditory, visual), the concept of data, memory, intuition, error etc. Lecture slides can be accessed here:
The second session (Poetic Interfaces) is about building prototypes, mockups and art pieces where glitch, errors and probability can take part in the compositional process. This can remind one to creative writing where sudden and unwanted events can shape the resulting process, thus a real conversation is induced between the work and te people who are creating and observing it.
Procedural Drawing (Sandarbh Artist Residency, Partapur, India)
A followup of recent procedural drawing workshops, where participants are practicing computational thinking without the use of machines. This event was made together with Artus Contemporary Arts Studio. The language of the workshop is pseudocode. There is no specific type, no predefined syntax, we do not rely on any computers, function libraries, we use only pens, lines, papers, logic, repetition, rules, process. We are following the path that many philosophers, engineers, inventors, mystics have been followed before us: think and draw. The method is relying heavily on fluxus practice and the exercises of the Conditional Design team. The following images are a selection of some executed, completed examples.
InBetween States [Sound, Process, Notation]
“Varying combination of sounds and “non-sounds” (silence) through time, sounding forms (Eduard Hanslick), time length pieces (John Cage). These definitions of music are referring to both a concrete and a non physical presence that are holding the vision of a special space-time constellation. The process of investigating the distribution of sounds is leading us to the magical state of the musical thought. Waiting, along the observation procedure are constructing an extension of the listeners traditional relationship with music.
What states might a musical thought have? How does interpretation is taking part in the process? How does representation affected? What are the bases of the musical experience? The exhibition is seeking to find these “in-between” states through the works of the New Music Studio , and students of Faculty of Music and Visual Arts University of Pécs and - of course - the presence of the visitors.” (from the exhibition catalogue)
Exhibition Space (2B Gallery)
New Music Studio is a group of musicians formed in the seventies in Budapest. They were focusing on some kind of methodology and philosophy of the contemporary music scene that was different from the canonized mainstream. They brought “cageian” influences and the legacy of fluxus to the regions of eastern european society. The main founders of the group ( Laszlo Sary, Laszlo Vidovszky,Zoltan Jeney, Peter Eötvös, Zoltán Kocsis, András Wilheim et al) amongst other creative minds produced several interesting pieces that were questioning traditional musical interpretation methods, notation, performance and the articulation of musical expressions. Their legacy is hard to interpret hence the large amount of their pieces. These neither fit in the traditional performative space, also difficult to adjust to gallery spaces.
The exhibition is a truly great reflection to this situation: installations, homage works and cross references are introduced to the audience in order to shed light on most of these problematic aspects. The tribute works of the next generation are also deeply experimental and fresh pieces that are using post digital concepts the aesthetics of raw, ubiquitous, programmable electronics where people can focus on content and aural space instead of regular computers and the like. The presented spatial sound installations find their place perfectly in the gallery space. The collected documents and visual landmarks of the exhibition are giving hints on very interesting topics without the need for completeness which is a good starting point to dive into this world deeper. The exhibition is curated by Reka Majsai.
Hommage á Autoconcert (Balázs Kovács & students from University of Pécs)
“Hommage á Autoconcert” by Balazs Kovacs (xrc) [link] & students from the University of Pécs is a sound installation that is constructed of several found objects and instruments. It is a tribute to the so called “Autoconcert” (1972) by Laszlo Vidovszky. The original piece has a diverse set of found objects that are falling from above over different time intervals during the performance. The present installation is dealing with the sound events in a reproducible way: the objects are not falling, but get hit by small sticks from time to time. These events are constructing a balanced aural space where some tiny noise, amplified strings and colliding cymbals can be heard as some acoustic background layer. The sticks are moved by little servo motors, the logic is controlled by a few arduino boards .
Repeating and overlapping of three musical processes (Gábor Lázár)
“Repeating and overlapping of three musical processes” by Gábor Lázár is also a sound installation made with custom micro controllers and three speakers set in plastic glasses as acoustic resonators. The processes are based on ever-changing time manipulations between musical trigger events. Dividing time within these in-between temporal events cause ever-increasing pitch shifting in the auditory field. The overall sound architecture forms a very saturated and diverse experience.
Visual Notations (New Music Studio)
“Visual Notations” by the New Music Studio members themselves are in fact several notation pieces crafted in a beautiful way. Varying nonstandard notation techniques are introduced here: using the graphical alignment of stellar constellations (Flowers of heaven, Sary), spiral like, self-eating dragon (Lap of honour, Vidovszky), ancient hindu symbols (Yantra, Jeney) are all taking part in the semantics and development in these sounding images.
Three pieces by Sáry (reinterpretation & software by Agoston Nagy)
There is also an installation I’ve been involved with: interpreting three pieces by Laszlo Sary. The triptych is letting the observer see and hear the inner sound structures at the same time. The upper region of the screen is displaying the original notation, this can be referred to the piece as a spatial constellation of the sounds to be played. Each work has different notations and different instructions on how to play the notes. The bottom region is displaying a live spectrogram of the actual sound. This is showing a temporal distribution of the events which has different characteristics with each piece.
Musical Spectra of the three different pieces by Sáry
“Sunflower” (1989, based on Snail Play, 1975) has a spiral-like spatial distribution in the sheet, these patterns are clearly visible in the musical spectra: infinite ladders are running up and down as the musicians are playing on three percussive marimbas. “Sounds for” (1972) is based on some predefined notes that can be accessed by the performer in any order they prefer. Larger, distant, sudden triggers are defining a more strict, cityscape-like spectra. The final piece, “Full Moon” (1986) is based on a permutation procedure [link]: the individual notes are played back simultaneously by four (or more) musicians at the same time. Their individual routes are defining the final constellation of sounds. The visible spectra is like a continuous flow, where soft pitch intonations are in focus against the previous two, rhythm oriented works. The setup is made with free & open source tools, using OpenFrameworks on three raspberry pi boards.
Images of the exhibition opening
It is hard to describe all the aspects of the exhibited works and the legacy of the New Music Studio, as their works had many local influences in the East European region at their time. The states of “in-between-ness” however are clearly recognizable on several layers of the whole event: the political and cultural influences of the original artists, the continuous phase shift between the musical and the visual domain, the tribute works where the creators are halfway in-between authors and interpreters. The exhibition is all together a really important summary of a niche cultural landmark in the Hungarian and international contemporary music scene.
Thanks for Gabor Toth for the photos.
Accidental Calculus [Nature, Code, Design]
What are the similarities between ancient arabic systems for automation, origami folding, poetic code, adaptive design, legacy of John Cage or western music tradition including gameflow-like dice music pieces from Johann Philipp Kirnberger or Wolfgang Amadeus Mozart? All of these examples are representing a kind of relation between the logic of the mind and some behavioral aspect of our external environment. On one hand, these activities and artists are bringing forward some aesthetic decisions where creative minds let independent elements into the flow of the creative process to take part in the creation of the piece. Hereby the designer is defining frameworks and boundaries for semi-autonom processes which will lead to the evaluation of the final piece. On the other hand these fundamental processes are not only aesthetic decisions, but ways of thinking about our environment and about ourselves. These are borrowing inspiration from scientific, philosophic and mystic traditions that can be applied to many different fields and phenomena.
When dealing with formalized systems we always have to follow the rules of some procedure. Frameworks are usually constructed of some basic set of rules which might lead to complex structures while iterating and repeating these instructions. These simple set of repetitive rules (or algorithms) are applied to achieve different goals in different contexts and situations (think of any language, for example). Many people like to think that randomness adds value and complexity to their systems because we can not predict its behavior so it will surprise the observer with ever changing, never repeating outputs and combinations. According to a recent interview with Michelle Whitelaw (can’t find the direct link but it has appeared in Neural), it is true in terms of variability, but it is not about adding qualitative values for a system. Random events are really useful for simulating chunks of data for unpredictable behaviors: when we need to simulate unpredictable input values for a system for further investigation. Let’s observe three very basic distribution model.
fig. 1 - Random probabilities. Left to right: a ) no random factor, linear steps. b ) random walk with small amount of randomness. c ) random walk with increased randomness. d ) full range random steps
If we know the parameters of our goal exactly, we can get there by very strict and precise instructions easily. Say, we have a point on a plane, we can get there if we add a simple vector from our starting point to the direction of the given location. Simple & solved. If we know only one parameter of our point we can converge into the desired location by adding some random probabilities to our system. This will be less effective but we will achieve the goal sooner or later. Does it mean if we don’t know any of the coordinates of our point shall we use totally random distribution? Well, not really, because a totally random distribution allows repetition (the following states are not depending on their previous ones) so maybe we will never reach the goal. If we really would like to reach our goal, we still need to use some semi random algorithm where we combine order with chaotic behavior. This can be implemented several ways. We can use some kind of basic random walk algorithm which is not very effective, or we can use very sophisticated path finding algorithms that are way more useful. Or, we can use a straight, ordered algorithm with no random probability like a television scan line when its cathode ray is moving from point to point over each line, sequentially. Or, we can generate random numbers and avoid previously generated ones so finally we will stumble in the position of our point. There are many-many ways to solve a problem. What we chose might depend on the context of the situation (how much time do we have, how precise do we need to be, etc).
Specific distribution models are easily observable in our regular daily routines, also. The temporal distribution of our activities are more or less predictable if we find some patterns and repetitive occurrences between successive events. Hungarian scientist Laszlo Barabasi-Albert found an interesting model that can describe key aspects of our behaviors. He calls these successive events “bursts”. He and his colleagues were observing how frequently people respond to mails, talk to each other or even how often move between two locations. They have found that there are surprisingly big gaps and silence between two active events. It’s obvious that we do not speak for a few hours, then we meet someone and talk a lot. We do not read mails for a few days than we respond all to them within a few hours. Our basic communication structure is working like this. Similarly, but less obviously moving between physical locations are showing bursts and calm series, too. It is not a constant average value how much we travel. If you go to vacation, you move a lot, then for several months you are just moving locally between the places of your regular life. This concept can be extended even to our regular routines, relationships and consumer behaviors.
fig. 2 - Brownian motion (left), Levy Flight (right)
This distribution model is a well known behavior in other natural processes and has the name Levy Flight. According to the blog Seeing Complexity:
“Lévy flights are described by a random walk, which is more or less any mathematical expression of a trajectory with successive random steps, distributed with a non-exponentially bounded probability distribution with theoretically infinite variance. So what does that mean? It is a description of motion. Take as a contrast Brownian motion, which is modeled after random movement of particles in a fluid.A Lévy flight is instead a cluster of movements, some short and some long. It was identified by the late Benoit Mandelbrot, as an outgrowth of chaos theory.”
Why are these facts important from a designer point of view? Think of procedures. Our regular routines are constructed subconsciously from trivial procedures. Our current states are more or less predictable. And if we deal with this potential we might build tools that are easier to understand, that are adapting themselves to a personalized usage. An object that is capable to measure its environment and change its internal structure based on these principles can be adapted more seamlessly and easily to someone’s internal representations. This might lead to some adaptive, invisible interfaces where there is no UI or Graphic Design anymore, but something else. Something, which deals with rhythm, temporality, routines and changing interests, fluid contexts and self regulating algorithms. One might call it contextual design, behavior design or anything that is brought forward by some sort of dynamic nature. Observing rhythms around (and inside) us, temporal sequences, music, dance, human relationships is a key element for asking the right questions within these unexplored territories. Software has meaningful dynamic inputs and deeply reconfigurable internal representations so all type of software can adapt its internal structure according to its environment. Software must be open not only in the regular meaning which refers to its source but also in terms of its behavior. Think of responsive design, reactive architecture, or any open ended process from the art world.
As Bret Victor, a great interventionist and interface designer points out: interaction is good when dealing with manipulation software (editors, games, etc). But once we are dealing with information software where we are more likely to learn instead of create, interaction is turning into interruption. An interface that represents information should work with as less interaction as possible, in order to let the perceptual and cognitive flow go seamlessly. These software might use environmental conditions, reconfigure their behavior when needed. This restructuring is based on some data from the users history, actual physical position, current time and other environmental parameters, but also includes social integration, similar interests and rankings from relevant people etc. The interface must be in real conversation with the context of the user. This type of meaningful relationship is originally described by Gordon Pask who adapted human conversational elements when he was dealing with artificial, responsive input & output systems that took form within (and out of) the fields of cybernetics.
fig. 3 - One sequence of the 64 hexagrams of the I Ching, or the Book of Changes. This ancient system is a method of divination based on a binary system and chance operations.
What are the strategies that we can observe in procedural thinking and algorithmic art? Janett Zweig introduces three basic concepts that she found while studying mystical systems, procedural art and computer practices. In her work “Ars Combinatoria” (pdf link) she makes difference between permutation, combination and variation. These three are formalized concepts for manipulating an existing set of entities (numbers, musical sequences, words etc). Permutation is when we have our set and rearranging its elements in different ways without adding or repeating any new element. Combination is to rearrange and select specific elements, reducing its original set. Making variations means to rearrange the elements of the system with the allowance of repetition, redundancy and multiplication. Her interest in these methods are going far more beyond practical definitions: she is investigating the difference between spiritually based or purely process based methods. She is asking when someone is making a mystical experience via permuting the letters of the alphabet (as with the gates of Sefer Yetzirah or the ancient representations of the Tree of Life) or permuting abstract binary symbols (such as with the I Ching system or modern computer programs), is it a creative transformation, or more like a meditative activity?
According to Zweig these procedural systems went through a qualitative shift from a purely mystical state to a more formal, process based methodology during our history. Starting with great “Mystical Universalist” systems, thinkers and philosophers introduced “Symbolic logic” and logical games which led to different “Semantic Interventions”. These are forming the basis for the concept of “Play” and playfulness that can be found in a wide spectrum of our activities including board games, contemporary music, educational systems.
fig. 4 - Spiral experiments: Generating Fermat (or Archimedean) spirals with modified divergence angles.
What happens if we are using a purely ordered algorithm (such as a formula for a spiral) with no random probabilities? Can we predict the result? It turns out that it is far away from the truth, at least in terms of visual representation. As you see on this visual experiment, each shape shows different characteristics compared to each other. A spiral shape is always present because of the clearly defined, ordered algorithm (r*r) = (a*a)*θ. However as soon as you start tweaking one single parameter a tiny bit (in this case, the parameter of the divergence angle of the system which is variable”a” in the code below) completely different spatial distributions arise. An initial value is repeated and iterated over itself within the loop, thus it has a multiplicated effect which brings forward emerging shapes of visual complexity. A simple implementation in Processing language:
float a; // divergence angle
float c; // scaling factor
int N; // number of dots
float r, phi; // polar coordinates
int j, i; // row, column coordinates
for (int n=0; n<N; n++)
{
phi = n*a;
r = c*sqrt(1.0*n);
j = int( r*cos(phi) );
i = int( r*sin(phi) );
ellipse(j,i,10,10);
}
Addition: an ellipse() is a function to draw ellipses at specified positions with specified dimensions (these are the four parameters of the function). Also, you have to initialize the variables (a,c,N) to run the code.
Language is definitely a permutation of letters from the alphabet. However the meaning and the core concepts of our consciousness are not contained in the discrete elements or parameters and not even between the configurations of these elements which are defined by syntactical rules and grammatical formulas. If we are bringing forward the world and our environment from concepts that are inherited from language, it is more than just communication tool. It is the incubator of analogies that are the core working mechanisms for human thinking as Douglas Hofstadter points it out in his recent book named Surfaces and Essences. It is the landscape around and inside us (be it visual, textual or mental language), it is the transfer channel of memes, it creates the internal representation of the world around and inside us. Some might say: the music is not in the piano.
A Procedural Drawing Course [Code, Pen, Paper]
Procedural properties are ancient problem solving techniques in communication, art, crafts and all type of human activities. We can see its incarnations in everyday life, from beautiful handmade indian rangolis to formalised design practices, mathematics, geometry, music, architecture and even pure code language. This workshop is dealing with Procedural Drawing. Why procedural?
Rangoli, South India
Media and technology has an overwhelming relationship with our cultural society. Stream of events, speed of information sharing growing rapidly, it is hard to catch up with the latest directions of specific areas of tools, art and technology. New social contexts, cultural interactions arise day by day. These days designers should focus and reflect on the here and the now and have an overview on the current methodologies, design practices. We can always learn and adapt new, short term, overhyped technologies, but it is much harder to grasp the underlying phenomenons, ancient design decisions, the human factor. This workshop is about to catch these properties under the hoods.
The language of the workshop is pseudocode. There is no specific type, no predefined syntax, we do not rely on any computers, function libraries, we use only pens, lines, papers, logic, repetition, rules, process. We are following the path that many philosophers, engineers, inventors, mystics have been followed before us: think and draw.
3d L-system made by Sebastien Parody
A very beautiful procedural example is the system made by Aristid Lindenmayer who was trying to describe complex biological and natural growth with a simple set of rules. This system is named after him. He also wrote a really inspiring book on “The Algorithmic Beauty of Plants” (pdf link). He wanted a way to test his theory about the growth pattern of a particular type of algae. His theory stated that the cells of this algae could be in one of two states: growth or reproduction. An algae in the growth state eventually grew into the reproduction state. An algae in the reproduction state eventually divided into two cells, one of which was in the growth state and the other in the reproduction state. Lindenmayer’s grammar system proved a fantastic method for proving his theory. What Lindenmayer could not have predicted is the incredible usefulness of his system in many other areas, both in biology and in mathematics.
Basic Diagram of the Lindenmayer System
A Lindenmayer grammar is fully defined by an initial axiom and a set of one or more transformation rules. The initial axiom consists of a “string” of characters (e.g., alphabetic letters, punctuation, etc.). Each transformation rule gives a set of characters to search for in an axiom, and a set of characters to replace the original characters with. Applying all of the transformation rules to the initial axiom produces a new axiom. The rules can then be applied to this second axiom to produce a third axiom. Applying the rules to the third axiom produces the fourth, and so on. Each application of the transformation rules is called an iteration of the grammar.
Pen Drawing from the Course: Marcell Torok gave instructions to synchronize your length of line with the time you are breathing in and breathing out
Another interesting aspect of procedural, algorithmic thinking is something that we can call the “stage-analogy”. As Rohid Gupta (Fadereu) refers to the analogies between stage, performance & programming: “As much as our machines, we have become the processors of information, but we do not read text and instructions line by line as computers do. We scan in less than a blink and get an overall visual sensory pattern - like the notes in a musical chord, or the Fourier Transform of a rich spectral signal from atomic emission, we break it down in a subconscious way.” Actors, performers are acting based on predefined set of rules, be it improvisational or fully, “strictly” followed algorithms.
Pen Drawing from the Course: Attila Somos was experimenting on how to formalize & define recursion
Visual arts started to adapt these concepts with the rising of the Dada movement in first decades of the twentieth century: artists, performers were giving instructions, pseudo codes for the interpreting (compiling) their piece. Like musical notations, which are originally interfaces for representing different events that are distributed in time, drawings, visual arts also reached a state where the liveness or the actual “happening” of the concept is separated from the thoughts aka the software. Fluxus in the sixties, conceptual artists, Sol Lewitt, Miklos Erdely with his creative drawing workshops and conceptual works also continued the tradition.
This tradition is widely used and democratized today, since we are working with software separated from hardware, we can find this concept embed very deep in our everyday life. You can see all the works (rules are in hungarian) from the course here. The workshop was held at MOME Media Design, 2013.
Sand Experiments
The last few weeks I was out of the circles & routines of my regular life, did not use my ordinary tools, I was visiting and contemplating different reality concepts during my journey in the far east. Self confidence, clean associations, purity arise without the regular routines that are wired so deep in me. To avoid regular routines: I travelled far and was doing something else.
Some days I was playing with sand. I saw some crabs digging holes into the sand to avoid attacks on the beach. They left really interesting patterns on the sand. I started cutting holes into a cardboard, pouring sand onto the surface of it, then observing the naturally organized patterns that are exactly the same that we find in the -these days so overhyped- algorithms of voronoi tessellations and diagrams.
After all it is really interesting that some basic computational calculations can be done with some natural forces (gravity and density in this case). Take time for example: sandglass is one of the simplest exact visualizations of time that we can find in nature.
The other example is a visible process instead of a “compiled” result. The sand is dancing here. It is in continuous movement, an ever-cycling entity, acts like a liquid, morphing shapes of mountains and oceans become visible between the interaction of the grains. It is a sort of low-tech reaction diffusion system. These methods are also used so widely in the world of computer graphics as natural simulations. The system here consists of sand, oscillating air pressure and gravity: a speaker is placed under the surface that is resonating at 60Hz.