Category: Lectures

Assignment: A/V Performance



Addictive TV

Check all their works

Eclectic Method

Takeshi Murata

(Video artist, not real time)

Cory Arcangel

(video generated from midi data, I think)

Prepare a short (~2 min) audio/video performance by remixing found or original footage live.
You can work in groups of two: one person controlling the sound, the other controlling the visuals.

– use videos with similar visual properties i.e. belonging to the same era or genre
– don’t use sources that are already art/artsy or already remixed
– if you are mashing up songs together you should make sure their beats and key match, or you edit them to make them match (it’s not easy)
– the remix doesn’t have to be fast and noisy, you can try a droney, slow flow

Art in Real Time

Max/MSP is a visual language originally created in the mid-80 as a tool for electronic music composition. Due to its modular nature, through the years it has been expanded into a flexible multimedia environment focused on real time Audio/Video manipulation.
Some things you can do with MAX/MSP/Jitter:

Pop(ish) Electronic Music
The English IDM duo Authechre creates music using Max/MSP (the music video is unrelated)

Alternative music instruments

Robotic Instruments

Computer-enhanced Performances

Information sonification

Hybrid bio mechanical installations


Real time sound analysis

(Made in Pure Data a Max sibling)

Real time video manipulation / VJ

Networked Objects


It all started with automata like the Digesting Duck


Or the praying monk, built in South Germany or Spain around 1560.

Hear the story in this episode of Radiolab.

Or the Ultimate machine, theorized by Claude Shannon, the father of information theory:

Modern automata of the Cabaret Mechanical Theater

Existentialist mechanical sculptures by Arthur Ganson

Sprawling modular mechanisms by SOA alumn Greg Witt

Wind powered “beach animals” by Theo Jansen

Riotous robotic performances by the Survival Research Lab

Light Emitting Art

Jim Campbell

Jenny Holzer
Selected works

Light Painting

Some examples

Light Painting with Roombas

Pulse Room by Rafael Lozano-Hemmer


Automatic Glassware by Jesse Stiles

Atom Performance

Particles by Daito MANABE & Motoi ISHIBASHI

Leo Villareal

LED throwies by Graffiti Research Lab

The Aqua Teen Hunger Force case or: the issue with Electronics in public spaces

The bomb

The Star Simpson case a.k.a. the MIT “fake bomb” girl

Electronics for Artists

Let’s cover some basic principles with the Arduino comic


How it works:

How to power it with the Arduino:



The one on the right is the best.

Color code:


Light Emitting Diode. They are polarized!

Did you know LEDs have been around for 50 years and CHANGED EVERYTHING?

Hello World

Our first circuit:

Schematic equivalent to:

More specifically:

That can be physically implemented in this way:


Before you start wiring be aware that if you make mistakes you can potentially damage your equipment.
This usually happens when you create short circuits. S/C are circuits with no resistance, like when you connect negative and positive terminals of a battery.
Ohm’s law:
I = V/R
what if R = 0? Overcurrent, heat and troubles.


1. Always using RED wire for the power and BLACK wire to the ground. Never ever use these wires to connect data ports.

2. Don’t modify your circuit while it’s powered. Disconnect the red, change things, and re-connect.

3. Connect all ground wires before connecting anything else.

4. Don’t wire together power rails.

5. I/O Pins should never be directly connected to the Ground or to power

6. I/O Pins should never be connected to each other

7. Don’t connect cathode and anode of an LED. For example by putting them in the same row.

8. Wires are isolated but the leads of resistors and LEDs aren’t! Make sure they don’t touch each other accidentally.

9. Don’t power the circuit with batteries or anything that is not the Arduino power/ground pins unless you know what you’re doing.

10. Double check your circuit before powering, debugging by trial and error like in programming is not the best solution.

Complexity and Emergence

Game of life by John Conway (1970)

Cellular Automata

Inspired by Go, originally implemented as analog system.

Two steps (generations) of the game of life

Try it in motion online or offline.
Reminds of fire burning forests, bacteria, molds, etc…
(SimCity is based on cellular automata as well)


Flock algorithm (Boids) implementation in processing

Birds, fish, bugs, buffalos…

1) Don’t get too crowded.
2) Don’t get too isolated.
3) Swim with neighbors.
(An optional 4th rule is that you should do what it takes to flee predators)


In order to understand recursion you must first understand recursion.
Ok, imagine a function that calls itself.

Soulwire recursion toy (click)


Self similar patterns based on mathematical sets. Formulas are relatively simple to define but generate a great level of complexity.

Cauliflower, self-similar patterns are also found in nature

Mandelbrot Fractal Set

Some resources about fractals here.

Artificial Life

Levitated – Walking things

Levitated – Legged Creatures

Cindermedusae by Marcin Ignac

Evolutionary algorithms

Darwin Hill (arbitrary selection)

Karl Sims – Evolved Virtual Creatures, Evolution Simulation, 1994


  • Generate the initial population of individuals randomly – first Generation
  • Evaluate the fitness of each individual in that population
  • Repeat on this generation until termination (time limit, sufficient fitness achieved, etc.):
  • Select the best-fit individuals for reproduction – parents
  • Breed new individuals through crossover and mutation operations to give birth to offspring
  • Evaluate the individual fitness of new individuals
  • Replace least-fit population with new individuals

Communion by Matt Pyke (Universal Everything) and – Marcus Wendt and Vera-Maria Glahn. Video here.

The installation is in a rectangular room with 6 separate projection on the walls creating almost a 360º environment. The final piece includes an array or evolving creatures going through stages of development – evolution from simple to complex with human like properties with generative behaviours

Want more of this? Read Out of control by Kelvin Kelly. Not extremely new, a little too Californian Ideology (pre dot com boom) but still good.

Patterns, patterns, patterns…

Experimental Typography

20th Century Avant-gardes

Futurism: Filippo Tommaso Marinetti Words in Liberty – 1913 Filippo Tommaso Marinetti Words in Liberty – 191- DADA: Ilia Zdanevich – 1923 Kurt Schwitters – German DADA. Edited a Magazine called Merz edited with El Lissitzky Russian Suprematism: El Lissitzky
Propaganda poster “Beat the Whites with the Red Wedge“ 1919
El Lissitzky – For the Voice a “book construction” based on the poems of Vladimir Mayakovsky De Stijl / neoplasticism: Piet Zwart Dutch ’20-’30.

Bauhaus (1919-1933) hugely influential school combining craft and fine art Herbert Bayer’s 1925 experimental universal typeface combined upper and lowercase characters into a single character set.

Flash Forward…

Brutally skipping a century of commercial graphic design, we should at least mention the graffiti/street art movement that has been pushing the boundaries of writing since the late 70s (influencing art, typography and design in the process). Here’s a project that study and celebrates this form:

Computational design is affecting typography as well.
Here are some examples that use tools like Geomerative in combination with complex systems simulations:

The project Growing Data uses virtual plant growth to visualize air quality in various large cities.

Genotyp allows you to create new fonts by selective breeding

Potential Literature

Given enough time, a hypothetical monkey typing at random would, as part of its output, almost surely produce all of Shakespeare’s plays.
– the Infinite Monkey Theorem

Not to be taken literally…

In an experiment conducted in a Zoo in England, zookeepers left a computer keyboard in the cage of six macaques for a month. The monkeys produced only a five page document, consisting mostly of the letter S, until the alpha male bashed the keyboard with a stone and all the other monkeys urinated and defecated on it.
– Dario Maestripieri, Primatologist

Writers and artists have been fascinated with randomness and with the combinatorial properties of text since DADA.

To make a Dadaist poem
Take a newspaper.
Take a pair of scissors.
Choose an article as long as you are planning to make your poem.
Cut out the article.
Then cut out each of the words that make up this article and put them in a bag.
Shake it gently.
Then take out the scraps one after the other in the order in which they left the bag.
Copy conscientiously.
The poem will be like you.
And here you are a writer, infinitely original and endowed with a sensibility that is charming though beyond the understanding of the vulgar.
– Tristan Tzara, 1920

Writer William Burroughs in the ’50s applied this technique, dubbed cut-up technique, to his own writing and recordings. (And David Bowie, and Kurt Cobain, and Thom Yorke…)

Not only text:

In John Cage’s Music of Changes (1951) the composer selected duration, tempo, and dynamics by using the I-Ching, an ancient Chinese book which prescribes methods for arriving at random numbers.

Raymond Queneau – Hundred Thousand Billion Poems (1961)
Web port here

Queneau was the founder of Oulipo – Ouvroir de littérature potentielle (workshop of potential literature).

Principle of variability in new media

A new media object is not something fixed once and for all but can exist in different, potentially infinite, versions. This is another consequence of numerical
coding of media and modular structure of a media object
Lev Manovich – The language of New Media

Manovich also noted how new media favor Database logic over sequential logic of linear media. The logic of a collection of items (data) recombined by algorithm is always present in generative literature and poetry.

Taroko Gorge by Nick Momford. Derivative works using different databases and algorithms.

Generation by interaction:

Generation by phonetics & genetic algorithms:

Generation by structural analysis of existing texts:

Gnoetry synthesizes language randomly based on its analysis of existing texts. Any machine-readable text or texts, in any language, can serve as the basis of the Gnoetic process. Gnoetry generates sentences that mimic the local statistical properties of the source texts. This language is filtered subject to additional constraints (syllable counts, rhyming, etc.) to produce a poem.

Word generation under constraint:
Adhesion text
You can create pretty interesting literature with it.

Text generation as parody:
Postmodernism generator

Automatic CS Paper Generator

Art critique generator

Bad literature

And so on…

And let’s not forget the most common and utilitarian use of text generation: Spam
Spam Poetry Institute