Category: Assignments

Assignment: A/V Performance



Addictive TV

Check all their works

Eclectic Method

Takeshi Murata

(Video artist, not real time)

Cory Arcangel

(video generated from midi data, I think)

Prepare a short (~2 min) audio/video performance by remixing found or original footage live.
You can work in groups of two: one person controlling the sound, the other controlling the visuals.

– use videos with similar visual properties i.e. belonging to the same era or genre
– don’t use sources that are already art/artsy or already remixed
– if you are mashing up songs together you should make sure their beats and key match, or you edit them to make them match (it’s not easy)
– the remix doesn’t have to be fast and noisy, you can try a droney, slow flow

Assignment: Networked Object

Make an artifact that creates a meaningful relation between the Internet (Twitter) and the “physical” world.
It can be a tweeting sensor or an object that responds to the flows of information on Twitter. Or there can be a two way communication. Ideas are more important than technical complexity.

*The point is to connect the physical world with the digital world: Twitter to Processing or Processing to Twitter is not enough.

*If you are reading data from accounts that you are not controlling, first make sure that the whole system works and then try to run it with the actual data.

*Don’t think about Twitter as just a personal microblogging platform but as a scale model of the Internet, a model that is extremely easy to parse and analyze. Pretty much every institution, group or media has a Twitter account, the possibilities are endless.

*If you are not familiar with Twitter find out how #hashtags and @replies and mentions work.

*Recap of the interactions you can easily implement:
– Tweeting from Processing
– Looking for tweets from Processing by account name or basic parsing (somebody tweets a specific word).
– Read sensors and control actuators via Processing/Arduino
– Capture images from the webcam
– Post images on Flickr from processing
– Everything you already know about Processing
– External Processing libraries from the internet

Due dates
Ideas by Tuesday 27
Projects by Thursday 29

Assignment: Alternative Interface

Make a processing sketch (screen based work) that employs a non-standard Arduino-powered interface.

– Think about how these sensors can be integrated into an object (es. flex sensor to detect the opening of a box) and within a specific context (es. pressure sensor to detect somebody sitting in front of the screen).

– Think about a screen or sound output that is meaningful and related to the interaction. Abstract visuals are fine but go beyond randomish flickering shapes this time.

– More is not necessarily better but consider using more than one sensor.

Advanced: make a simple game (single player or multiplayer) built around the interface. Don’t spend too much time making graphics please.

Document interaction and screen output with a video and post the link here. Phone cameras are fine but screen recordings may work better.
Quicktime for Mac can record what happens on the screen and can be used in combination with a webcam feed from photo booth.

Assignment: robot(?) / interactive artwork

You have three options for this assignment on Actuators:

1. Make a “social robot”: not necessarily a proper robot (we don’t have the equipment for that) but a machine that produce a life-like behavior that exploit the Eliza Effect.
Remember the Furby guy:
– expressive character (eyes, face)
– response to the environment
– unpredictable/evolving behavior the illusion of life often comes from a non-direct mapping of the input. If the input is used as a switch and the behavior is purely deterministic the object will appear as smart as a blender. If you implement some kind of processing and memory you are more likely to produce the Eliza effect.

2. Make a stand-alone interactive artwork that responds to the environment.

For both the options the only requirement is an input-process-output cycle. The thing has to receive an input from a sensor and produce a meaningful output through motor and or servo and/or LEDs or piezo speaker.

*Let me know by the end of the class if you want to use my distance sensor or my 3axis accelerometer.

*Modifying existing toys is another way to go. Electronic toys may offer more possibilities but hacking miniaturized and undocumented printed circuits is hard.

*You can combine microphone input via Processing (first assignment) with actuators.

*For example this is how you play sound files in Processing through Minim

import ddf.minim.*;

Minim minim;
AudioPlayer song;

void setup()
  size(100, 100);

  minim = new Minim(this);

  // this loads mysong.wav from the data folder
  song = minim.loadFile("mysong.wav");;

void draw()

void stop()


“Advanced” students could try to use processing with other libraries.
For example with openCv you can track a colored object or a face fairly easily (and if you have an external webcam you can try to mount it on the servo an make it follow the shape)

Assignment: Automaton

We know how to use actuators (servos and DC motor), now let’s try to actuate something interesting.

Using the Automino kit as a starting point, make an automaton / kinetic sculpture / mechanical thingy with the following characteristics:

– produces a complex and meaningful movement(s)
– employs at least one laser cut piece of your own devise
– integrates non-laser-cut elements. Tip: try non-rigid, non structural materials (jelly, feathers, balloons, beetles, leaves, taxidermy…)
– You can work in groups of two
– If you don’t have a kit just use other materials
– I recommend using only the servos (if you are working in groups you can control 4 of them!)
– If you think your project is too ambitious, then it’s probably too ambitious.
– I mean technically ambitious not conceptually ambitious, there’s no limit to the latter
– Favor original, poetic and beautiful projects rather than technically complex ones.
– Can machines be expressive? Can you make me forget there’s a buzzing servo behind it?

Don’t make it interactive or responsive to the environment, that’s the next assignment. Focus on the “actuation”.

Assignment: Hallouino

Arduino controlled Silly String shooter

Use the 9V battery to make a standalone (not connected to the computer) Halloween prop. It can be part of your costume or a modification of a commercially available decoration. It should be spoooky!
Shoot a video of the object in action, post it on youtube and link it on the blog.

– It must have at least one input: buttons, potentiometer, sensors

– It must have some kind of output: LEDs and/or servo.

– It shoul work without a computer attached. Program the Arduino, unplug the usb an plug the 9V cable. You can press the tiny reset button to restart the program.

– You can use alligator clips or wire extensions to separate Arduino + board from sensors and actuators.

– You can check the examples in the manual, like the DC motor…

– LEDs themselves are not that exciting, but you can use all sorts of object to diffuse their light ping-pong balls, balloons, pumpkins…

Code Example:
Light sensor controlled twitchy servo. Servo twitches randomly when it gets dark.

#include <Servo.h> 
Servo servo1;
int pos1 = 0;
int period = 500;
int sensorValue = 0;
unsigned long lastDidItTime = 0;

void setup() { 
  servo1.attach(3);  // attaches the servo on pin 3 to the servo object 

void loop() {
  sensorValue = analogRead(A0);
  unsigned long presentTime = millis();
  if(sensorValue < 90){  // if it's dark...
    unsigned long elapsedAmount = presentTime - lastDidItTime;
    if (elapsedAmount > period){  // do something spooky
      pos1 = random (0, 180); 
      lastDidItTime = presentTime;


Assignment: Particle System

This is a suggested starting point for the assignment. It’s a slightly different particle system than the one we programmed in class. The “things” move according to a angular direction instead of speedX and speedY.

int NUM_THINGS = 10;

// creating an (empty) global array with a
// capacity of 10 elements of type Ball
Thing[] things = new Thing[NUM_THINGS];

void setup() 
  size(500, 500); 

  for (int i = 0; i < NUM_THINGS; i++) 
    things[i] = new Thing();   // populating the array

    things[i].x = random(0, width);
    things[i].y = random(0, height);
    things[i].direction = random(0, 360);

void draw() 
  //bg is not conceptually part of the class
  background( 0, 0, 0);

  //call the method update everytime I draw
  for (int i = 0; i < NUM_THINGS; i++)

class Thing
  //parameters of my thing
  float myDiameter = 10;
  float x;
  float y;
  color col = #00FFFF;
  float speed = 1;
  float direction = 0;
  //Constructor function - same name as the class
  like the setup in the main sketch the constructor function 
  is executed once as soon as the object is created

  //method(s) for my thing
  void update()
    convert direction into radians
     break it down to the sinus and cosinus
     which are the vertical and horizontal component 
     of the angle then multiply for speed 
     (because in trigonometry you always work with a radius of 1)
    float dX = cos(radians(direction))*speed;
    float dY = sin(radians(direction))*speed;

    //the result are the steps in the vertical and horizontal direction
    x = x + dX;
    y = y + dY;


    //managing borders
    if (x > width)
      x = 0;

    if (x < 0)
      x = width;

    if (y > height)
      y = 0;

    if (y < 0)
      y = height;

    //draw the thing  
    ellipse(x, y, myDiameter, myDiameter);

    //example of interaction among things//
    //every object scans the other objects using the global array
    for (int i = 0; i < NUM_THINGS; i++)
      //temporary variable to store the other
      Thing other = things[i];

      //it's not a good idea to make theinteract with itself
      //this points to the specific instance
      if (this != other)
        //after I determine it's another thing I check the distance
        if (dist(x, y, other.x, other.y)<100)
          //and do something
          line(x, y, other.x, other.y);


Start from this code and produce a “more interesting” “flock of things” using interaction and/or random (to change behavior every so often for example) and by interrelating the behavior of the various things with each other (uncomment the distance part).

Advanced: create one or more additional classes and produce an artificial ecosystem in wich certain objects interact with others.

Assignment: Breaking Breakout

Create a digital artwork by modifying the code of a breakout clone.
It doesn’t have to playable or winnable, but it has to work and to be interesting to look at for at least 10 seconds.

Download source from here

-Don’t try to make another, completely different game starting from that code. Try to maintain some kind of breakout “feel”.

-Avoid importing external graphic resources. Keep it abstract.

-Try to start by tweaking the code, not with a specific idea.

Some works from the previous classes.

Data visualization

Lately it has been popular in tech related companies to create contribution maps that collectively show the contributions of each employ to various projects and aspects of projects over time. It’s an interesting concept but its purpose is hard to define. it can’t really be used for determining how much someone contributed because objectively measuring the value of each contribution would be quite difficult to include in the map.

Still it’s fun to watch the little workers running around and building this task map.