Category: Lectures

Final Project: Beyond the screen

Develop a project in which you combine some of the concepts and tools you learned in this class with your existing practice.
No assigned theme or format. It has to have a digital/computational component but also go beyond screen and computer-only based work, incorporating object making, performance, social practice, in either the process or the end product.

Here are some inspirations and strategies (some of them are technologically very sophisticated):

Hand drawn assets for games and apps

Printed computer generated images

Sculptural Projections

From digital to physical

Internet Performances

Performances with custom tools

Public space interventions

Networked objects

Comments Off on Final Project: Beyond the screen Posted in

Artificial companions

The next assignment involves the creation of a virtual pet. The goal is not to create a sophisticated simulation but rather provide an illusion of life-like behavior.

The Digesting Duck, was an automaton in the form of a duck, created by Jacques de Vaucanson in 1739. It appeared to have the ability to eat kernels of grain, and to metabolize and defecate them (it didn’t, the food was collected and the feces was already there).

A playthrough with today’s kids.

Somewhat controversial at the time, addictive, distracting, traumatizing. Introduced the pervasive always-on gaming you also find in Farmville and Neco Neko Atsume.

Supporters (including PETA) considered digital pets as training and test of committments in preparation for real pets.

Read In praise of Tamagotchi

Popularized the virtual pet genre, both electronic and software based: Neopets, Nintendogs, horse raising games etc.

Including some weird ones

The Tamagotchi effect is the psychological phenomenon of owners becoming emotionally attached to a robot or other digital object.

Furbidden Knowledge – From Radiolab (Play from 9:45)

The Tamagotchi effect is related to the Eliza effect, the tendency to unconsciously assume that computer behaviors are analogous to human behaviors, to attribute to the computer’s response more understanding than it actually has.

Created at the MIT by Joseph Weizenbaum between 1964 and 1966.

A web port of Eliza

Unichapel (Robotic Confession Booth) from THX 1138 by George Lucas (1971)

The attribution of human emotion to robots is one of the central themes in HBO’s Westworld or Spike Jonze’s HER, complicated by the fact the robots are acquiring self awareness.

The Eliza effect has been used strategically to “fake” artificial intelligence and make interactions with computers more intuitive and less threatening.

AI digital assistants are popping up everywhere Siri, Amazon Alexa, Cortana…
And by the way Why are they usually female?

Comments Off on Artificial companions Posted in

Visualizing Time

Data visualization



One Human Heartbeat by Jen Lowe One Human Heartbeat by Jen Lowe
Read her talk transcript here


Obelisks and sun dials 1500 BC from ancient Egyptian and Babylonian astronomy Hour glasses were used in churches to time the length of the sermon; long sermons were especially popular in the 17th century, and could easily stretch to three hours or more. Water Clepsydra 1819


Human Clock

Human Calendar

Standard Time by Mark Formanek

A 24-hour long performance.

The Clock by Christian Marclay

A 24-hour film made up of six or seven thousand clips, each of which either says or displays the (actual) time of day or makes a cogent reference to time.


Time zones clock

Pong Clock

John Maeda’s 12 ‘o Clocks

Classic Polar Clock

Dot Clock

Colour clock Hex to colors

Last Clock

Comments Off on Visualizing Time Posted in

Visual Music – Synesthesia

There have been many attempts to blur the boundaries of sound and visual arts. Instruments for the eyes have been conceptualized since the obscure ocular harpsicord in 1725.

In the early 20th century many artists claim to be affected by synesthesia and incorporate their affliction into their art making.

Color Organ invented by composer Alexander Scriabin in 1915 for his Prometheus: Poem of Fire.

Wassily Kandinsky.-Composition IV (1911) Wassily Kandinsky.-Composition IV (1911)

That’s the one that got accidentally turned upside down, and Kandinsky got the epiphany of non-figuration.

composition VIII - 1923 composition VIII (1923)

Pure painting like music, an “internal necessity” – leap into abstraction.

“Color is the keyboard, the eyes are the harmonies, the soul is the piano with many strings. The artist is the hand that plays, touching one key or another, to cause vibrations in the soul.”


He worked with other composers, making librettos and stage designs.

A student's graphic analysis of music according to Kandinsky’s theories on graphic representation of music made during coursework at the Bauhaus. Bauhaus Archive, Berlin, 1930  A student’s graphic analysis of music according to Kandinsky’s theories on graphic representation of music made during coursework at the Bauhaus. Bauhaus Archive, Berlin, 1930

Experimental filmmakers embrace the idea of visual music. Music as a pathway to abstraction.

Oskar Fischinger – An Optical Poem (1938)

Norman McLaren – Dots (1940)
“Made by painting directly onto clear frames of film, as pictured in the frame enlargements below. Interestingly, the music was created in the same way, painting directly into the area on the film strip usually reserved for the soundtrack.”

Norman McLaren – Synchromy 1971
“To produce the film’s musical soundtrack, McLaren photographed rectangular cards with lines on them. He arranged these shapes in sequences on the analog optical sound track to produce notes and chords.”

John Whitney – Permutations (1968)
Early computer animation, he applied musical principles such as the harmonic progression to the generation of images.

The interest in enhancing musical experiences with visuals re-emerged in the 60s and 70s with psychedelia and the “expanded cinema”

Liquid light shows – overhead projector + live manipulation of fluids. Late 60s – today.

Exploding Plastic Inevitable events by Andy Warhol, 1966 – 1967

Atari video music, 1976. The first (analog) electronic music visualizer.

Jeff Minter, Virtual Light Machine 1990, Atari Jaguar
Real-time sound analysis

Live visuals or VJing

Hexstatic+Coldcut 1997 in collaboration with Greenpeace.
This is a 1997 music video but gives an idea of the visuals/sound relationship.

Ryoji Ikeda, musician and installation artist. a/v performance

Sound apps and sound games, some of which I mentioned before

Rez – Tetsuya Mizuguchi – 2001
Inspired by a techno street parade

Everyday Shooter – Jonathan Mak 2007

Thumper – Drool – 2016

Polyfauna – Universal Everything + Radiohead


Comments Off on Visual Music – Synesthesia Posted in

Repetition, variation, generativity

Masjid-i-Jami, Isfahan Masjid-i-Jami, Isfahan
Alhambra, Granada Alhambra, Granada
More images here
Although decorative patterns have been employed by Greeks and Romans, Islamic artists achieved an unprecedented degree of mastery due to discoveries in Mathematics and aniconic precepts. (geometric, calligraphic, floreal)

But flash-forwarding to machine-influenced or machine-derived ideas of repetition.

Ghanaian kente cloth Ghanaian kente cloth
And of course patterns emerge in textiles throughout cultures.
Rich symbology embedded in the traditional Kente cloth.

In August 62 I started doing silkscreens. I wanted something stronger that gave more of an assembly line effect. With silkscreening you pick a photograph, blow it up, transfer it in glue onto silk, and then roll ink across it so the ink goes through the silk but not through the glue. That way you get the same image, slightly different each time. It was all so simple quick and chancy. I was thrilled with it.
– Andy Warhol

Sol LeWitt, Successive Rows of Horizontal, Straight Lines from Top to Bottom, and Vertical, Straight Lines from Left to Right, 1972

In Op-Art (60s)
Abstract subgenre, short lived but influential all across interior design, fashion and advertising. Op-artists were exploring the limits of perception, creating visually tense images and optical illusions.

Victor Vasarely – Orion C – 1962 (not quite Op-art yet) Victor Vasarely, Boglar II Victor Vasarely – Titan Bridget Riley – Movement in Squares Balm by Bridget Riley Bridget Riley – Cataract 1967 Bridget Riley looking mod in 1964 Edna Andrade – Black Dragons 1969 Edna Andrade finale 1979 Edna Andrade

Repetition + variation in textile design:

Anni Albers Second Movement – 1978

In minimalist music:

Steve Reich – Clapping music 1972

In early computer art:

Manfred Mohr Cubic Limit 72-77

In «Cubic Limit,» Mohr introduces the cube into his work as a fixed system with which signs are generated. In the first part of this work phase (1972–75), an alphabet of signs is created from the twelve lines of a cube. In some works, statistics and rotation are used in the algorithm to generate signs. In others, combinatorial, logical and additive operators generate the global and local structures of the images.

10 PRINT CHR$(205.5+RND(1)); : GOTO 10
From the Commodore 64 manual

In generative art

Every Icon (1997) John F. Simon, Jr.
Can you reverse engineer Every Icon?

Generative artist LIA

Jared Tarbell – click to see it in real time Jared Tarbell – Quarter Round


Casey Reas

Same principle applied to rotational symmetry:

Marius Watz – Illuminations B Marius Watz – Illuminations B



Generative art by Mario Klingermann

Organic/lifelike generation…

Moth Generator

In level design
Modular generative systems are often used in games to generate levels procedurally:

Forget me not


and also beyond the 2D grid


Fuji by Robert Hodgin (flight404)

Fuji, test render from flight404 on Vimeo.

No Man’s Sky

The problem with the hype, and the media understanding of procedural generation:
No Man’s Sky Is Like 18 Quintillion Bowls of Oatmeal

No Man’s Sky and the trickiness of advertising a procedurally generated game

Comments Off on Repetition, variation, generativity Posted in

Drawing Machines

Ivan Sutherland MIT 1963 (Action starts at 4:20)

Computational art before computers?

Tinguely, swiss kinetic sculptor known for the self destructing machine.
In the 50s produced a body of work known as metamatics. A parody of American action painting.

In the same period matematician and artist Ben Laposky (American from Iowa),
made the first computer generated images, photographing the output of an oscilloscope.

Desmond Paul Henry (British) used bombsight analogue computers which were employed in World War II bombers to calculate the accurate release of bombs onto their target.

These artists seemed to be fascinated by the apparent randomness (unpredictability) of these machines and let them “do their thing”.
On the opposite side of the spectrum there is Sol LeWitt who didn’t use computers but conceived many of his works as a series of instructions. As if they were meant to be created by machines (art gallery interns).

This is the result of the sequence of instructions. What is the status of authorship and ownership when the works are just instructions that anybody can execute?

Computer art

Georg Nees 1965-1968.
In the mid-60 computer begin to become relatively more accessible and the first printers (plotters) become available. Georg Nees – Schotter 1968.
Nees had to write his own graphics libraries. His works often deals with order vs disorder. He also made the world’s first computer-generated sculpture in 1968 using a computer aided milling machines. Vera Molnar Interruptions-1968-69.
The computer art movement was international. Vera Molnar was a French Hungarian artist.

Manfred Mohr – Random Walk 1969 Frieder Nake, Nr. 2 (also known as Hommage to Paul Klee) 1965 Herbert W. Franke’s Serie Mondrian (1980), a software created for the Texas Instruments TI 99/4 home computer. Serie Mondrian produced Mondrian-style images according to user defined parameters.

Mark Wilson – STL D30 (1986)
In the 80’ computer generated abstraction grows in complexity. And can be printed in full colors. Mark Wilson – Small Three Skew (1983)

Many of these artists are still around but in the ‘90s computer art morphed into new media art and interactivity and social engagement became the main focus for artists working with technology.

With the democratization of robotics, drawing machines started to get out of the gallery (see graffiti writer) and became more autonomous.

Or self-aware?

Mechanical Parts by Matthias Dörfelt a robot that draws randomly generated “connectors” aka robot genitals.
Mechanical Parts by Matthias Dörfelt

Mechanical Parts by Matthias Dörfelt 2

Cory Arcangel gradient works are a playful post-modern legacy of computer art.

Cory Arcangel
Photoshop CS: 110 by 72 inches, 300 DPI, RGB, square pixels, default gradient “Spectrum”, mousedown y=1098 x=1749.9, mouse up y=0 x=4160 Cory Arcangel
Photoshop CS: 110 x 72 inches, 300 DPI, RGB, square pixels, default gradient “Spectrum,” mousedown y=27450 x=6700, mouseup y=4800 x=13400, 2010
The preset gradient as pop culture reference that many recognize. Cory Arcangel
Photoshop CS: 110 by 72 inches, 300 DPI, RGB, square pixels, default gradient “Spectrum”, mousedown y=1098 x=1749.9, mouse up y=0 x=4160
Low end sloppiness vs high end packaging

Computer assisted and generate drawing in the age of Artificial Intelligence (Neural Networks)

Comments Off on Drawing Machines Posted in

Branching narrative from Borges to Twine

Linear story

Branching story

The problem with branching stories

The most common solution

Or another solution…

From You chose wrong

Your first assignment is a branching story.
The most popular artifacts of this kind are the Gamebooks or Choose Your Own Adventure. Very popular teenage literature in the 80s and 90s.

CYOA diagram – Michael Niggel

PDF here

Or you can check this impressive formal analysis of Choose your own adventure books:

Visualizations, playable books and essay here.

Before computers

We have to go back in time to find more “noble” precursors of non-linear and interactive storytelling.

Apollinaire – il pleut 1916 F.T. Marinetti and the Futurists “Words in freedom” (1909 – 1916)

And the DADA movement

To make a Dadaist poem
Take a newspaper.
Take a pair of scissors.
Choose an article as long as you are planning to make your poem.
Cut out the article.
Then cut out each of the words that make up this article and put them in a bag.
Shake it gently.
Then take out the scraps one after the other in the order in which they left the bag.
Copy conscientiously.
The poem will be like you.
And here you are a writer, infinitely original and endowed with a sensibility that is charming though beyond the understanding of the vulgar.
– Tristan Tzara, 1920

Writer William Burroughs in the ’50s applied this technique to his own writing.
(And David Bowie, and Kurt Cobain, and Thom Yorke…)

Jorge Luis Borges

The Garden of Forking Paths (1941)
In the short story a character named Ts’ui Pên tells everybody that he wanted to write a book and build a labyrinth. Nobody ever found the labyrinth, only a very confusing and contradictory book. We then discover that the book *is* the labyrinth. In the fictional book, every chapter is followed by “every” possible continuation.

"In all fiction, when a man is faced with alternatives he chooses one at the expense of the others. In the almost unfathomable work of of Ts'ui Pên, he chooses - simultaneously - all of them. He thus creates various futures, various times which start others that will in turn branch out and bifurcate in other times. This is the cause of the contradictions in the novel"
The Garden of Forking Paths - Jorge Luis Borges

Agusto Boal’s Forum Theatre (1960)
In this process, the actors or audience members could stop a performance, often a short scene in which a character was being oppressed in some way (for example, a chauvinist man mistreating a woman or a factory owner mistreating an employee). The audience could propose any solution, so long as they conveyed it on stage, working, acting, and directing not from the comfort of their seat.

Starting from the Mid-20th century even an inherently linear form like cinema starts to experiment with non linear storytelling and interlocked and contradictory plots.

Kinoautomat by Radúz Činčera 1967 – the first interactive film

“The film is a black comedy, opening with a flash-forward to a scene in which Petr Novák (Miroslav Horníček)’s apartment is in flames. No matter what choices are made, the end result is the burning building, making the film —as Činčera intended— a satire of democracy”

Raymond Queneau

Founder of Oulipo – Ouvroir de littérature potentielle (workshop of potential literature).

Hundred Thousand Billion Poems (1961)

web port here

Also by Queneau: Story as You Like It (1984)


Computer Lib / Dream Machines – Theodor H. Nelson, 1974. Self produced later republished by Microsoft.

Like many early geeks, Ted Nelson saw computers and networks as empowering tools and advocated for the democratization of these technology (You can and you must understand computers now!).Personal computer = personal liberation.

Ted Nelson coined the term Hypertext in the 60s.

“A system of non-sequential writing that would allow the reader to aggregate meaning in snippets, in the order of his or her choosing, rather than according to a pre-established structure fixed by the author.”

Years before the world wide web was implemented. A visionary application called Hypercard (1987-) already tried to make hypertext creation accessible to anybody


The concept of hypertext is now familiar to anybody thanks to the World Wide Web (invented in 1990 and popularized in 1995 with the invention of modern browsers) but in the 80s it was a quite exotic medium, especially for non-utilitarian uses.
Still some fiction writers started to experiment with the hypertext as literary form.

Afternoon, a story by Michael Joyce (1987) used a tool very similar to what we are using today Twine.

A cool example is “253” by Geoff Ryman here originally published in 1996.

Interactive fiction

Adventure games popularized another way to control and navigate interactive texts: the parser.

The most vital legacy of hypertext literature and parser-center storytelling is Interactive Fiction.
IF uses more sophisticated structures than the simple branching and a parser for the interaction, usually employed to navigate spaces, interact with characters and objects in a game-like fashion.

IF is a kind of universe on its own which deserves its own course. A great introduction and a collection of resources can be found on Emily Short’s website.

In the last years a movement of game makers revived the hypertext fiction tradition adopting a tool called Twine. These authors were attracted by the accessibility and flexibility of hypertext creating a variety of experimental and personal works.

Try some of these in class

Comments Off on Branching narrative from Borges to Twine Posted in

Strange Web

The web didn’t use to be this boring and standardized

one terabyte
One terabyte of kilobyte age
Geocities archive curated by Olga Lialina

net.artists exploring the possibility of the web and resisting the stylistic omologation of an increasingly commercialized web

Jodi (late 90s)


Mouchette (Martine Neddam 1996)

In the last ~5 years a new generation of internet artists expressed a similar rebellious attitude regarding the narrow visual standards of social media

petra cort hands1
Petra Cortright

Ying Miao
Ying Miao

Screen Shot 2016-08-30 at 1.37.34 PM
Emilie Gervais

Screen Shot 2016-08-30 at 1.38.29 PM
davidmarinos on newhive

camerons world
Cameron’s World – Cameron Askin

Yung Jake

Screen Shot 2016-08-30 at 1.42.27 PM
Hyper Geography – Joe Hamilton

Screen Shot 2016-08-30 at 1.43.42 PM


Screen Shot 2016-08-30 at 1.57.11 PM
alexandra gorczynsk on newhive

Comments Off on Strange Web Posted in

What is new media art

Let’s start with a cheesy question:

This second part is crucial. New media artists use the same technologies used by engineers, military, advertisers etc. But they do it in a different way.

Art made in Photoshop or with a digital production tool is not automatically new media art.

Oron Catts and Ionat Zurr at the Tissue Culture & Art Project.
The artists based the jacket on a biodegradable polymer, coated it with 3T3 mouse cells to form connective tissue and topped it up with human bone cells in order to create a stronger skin layer. Although the technologies can be new, the artistic concerns are not. The conceptual and aesthetic roots of New Media extend to the Dada movement in the ’20. There are many common strategies: use of multiple physical media, appropriation and subversion, irony and absurdity, everything can be used for art (Fluxus will further elaborate on this). Net.flag is an interactive work hosted on a museum’s website. It contains images of all the national flags in the world. Each flag is broken into its respective components, which are then categorized according to several broadly descriptive terms; users may pick and choose from a menu of these components to form a new flag, which they may then save to the site. In addition, the project features a browsable history, through which users may look over past creations which have been saved and titled.
Internet as a new post national space. Pop art is another important influence. Many works of New Media art refer to and are engaged with commercial visual culture.

Like concept art New media art is often more focused more on ideas rather than on objects.

And now an embarrassing question:

(if you are a painter, sculptor etc…)

The new media field tends to include a lot of things that don’t belong to the traditional gallery formats. by One year of GPS self-surveillance. The artists even put their own telephone under control for the entire month. All Internet users had real-time access to any phone conversation trough the website. And their hard drives and email correspondence as well. Data-nudism. Anticipating the post-privacy era.

(In)Security Camera from ben chang on Vimeo.

Contestational Robotics – Institute for Applied Autonomy from Rich Pell on Vimeo.

Sometimes certain ideas are too visionary to have a place in the market.

Myron Krueger – Videoplace, Responsive Environment, 1972-1990s

Hand from Above from Chris O'Shea on Vimeo.

Traditional visual artist may find materials and concepts that can inform their practice.

Emily Gobeille & Theo Watson aka Design I/O
“Jungle” and “City” are the first two in a series of experimental posters for children that combine science, nature, algorithm and design, to feed children’s imagination and curiosity.

Because our contemporary visual culture is deeply informed by the internet and the digital world

Oliver Laric's Icon - an artist associated with the "post-internet art" trend Oliver Laric’s Icon – an artist associated with the “post-internet art” trend
Comments Off on What is new media art Posted in