13 December 2013

Starry Night

This is a music visualizer that simulates the starry night sky.

A music-loving friend of mine once told me he missed seeing the stars at night after coming to Pittsburgh. The idea for this project came as an idea for a present for that friend. I liked the idea of a portable, personal set of stars that could be charmed to life by playing music. The stars react to new notes being played, and the aurora appears at certain volume of music and duration of continuous music. (This may not seem very obvious in the video at the beginning because I wasn’t playing the notes hard enough. Also pardon my rustiness on piano – I haven’t really played in 2 years.)

The end product uses an Arduino Mega 2560, with an Electret Mic Amplifier for sound input, and loads of LEDs for display. Frequency analysis utilizes code from Adafruit’s Piccolo (https://github.com/adafruit/piccolo), which uses Elm-Chan’s FFT (Fast Fourier Transformation) library.

The creation of this project was a long and arduous process for me. My initial idea was to have a box filled with blue origami stars (https://fc04.deviantart.net/fs25/f/2008/072/f/e/Straw_Stars_by_Miraka.jpg), with white LEDs hidden inside white origami stars scattered around in the box. However, I quickly ran out of material for making the blue origami stars, and so replaced it with black cardstock and tissue paper. The end result of the stars adhere to my original idea in terms of visuals and functionality. The end product still has the white LEDs hidden inside white origami stars, and you just can’t tell clearly because they are now covered by black tissue paper. The white origami stars make the light of the white LEDs spread a little bit, and if you look carefully, the spread is in the shape of 5-pointed stars. I also wanted more white LED stars, but was limited by the number of PWM pins on the board (and later, space for the wires).

I also wanted to actually learn how to use the FFT library to implement more accurate frequency measurement, for picking out very roughly which notes are being played. It turned out that this is actually quite difficult due to harmonics, and it was hard to understand how to use the library partly due to poor documentation, so I ended up working with code from Adafruit for frequency analysis. A lot of testing was done to get it more suited for piano music. After getting the stars to work the way I wanted them to, I reflected on I could make it appear more interesting/visually appealing. The easy answer was “colors”, so I tried to implement something that appears similar to auroras. The source of the auroras are a number of LEDs. The ideal way to do this would be to use a LED strip (like this one https://www.adafruit.com/products/306), but since this was late into the project, I didn’t have time to get one.

Physically putting this together was also very hard and time-consuming. I had a lot of trouble getting the connections for all the LEDs to work. I had to basically tear my project apart several times because the conductive copper tape wasn’t effective for LEDs, or wires broke, or solder wasn’t strong enough, etc. In the end my breadboard had almost every single slot filled. Then more things fell apart as I was trying to get everything to fit inside a small box. I didn’t realize all those wires would take up so much space.

Weird, but useful tidbits I’ve learned about Arduino:
– variables with types that don’t match won’t raise an error while compiling, but would cause weird things when run
– error in uploading program to Mega board can sometimes be fixed by unplugging a few pins

In the end, I was fairly satisfied with the final product. The stars worked almost as well as I hoped they would. I just wish I was able to show off the craftsmanship that went into this project more. If I get up enough energy, I’d replace the RGB LEDs with an RGB strip. It would be difficult though, because I’d literally have to tear apart my project again, both physically and coding-wise. I enjoy watching it as someone else is playing the piano. Too bad I can’t really watch it while playing at the same time, since I have to watch the keyboard, haha.

[I just realized I accidentally named this the same as that famous van Gogh piece. Ugh. Need better naming skills.]

Code, if you’re interested. It’s messy and long and uncommented:

/* Starry Night -
a music visualizer that simulates the starry night sky.

Parts of the code are written by Adafruit Industries.  Distributed under the BSD license. 
See https://github.com/adafruit/piccolo for original source.

Additional code written by Jun Huo.
*/

#include 
#include 
#include  $#include #ifdef __AVR_ATmega32U4__
 #define ADC_CHANNEL 7
#else
 #define ADC_CHANNEL 0
#endif

int16_t       capture[FFT_N];    // Audio capture buffer
complex_t     bfly_buff[FFT_N];  // FFT "butterfly" buffer
uint16_t      spectrum[FFT_N/2]; // Spectrum output buffer
volatile byte samplePos = 0;     // Buffer position counter

byte
  peak[8],      // Peak level of each column; used for falling dots
  dotCount = 0, // Frame counter for delaying dot-falling speed
  colCount = 0, // Frame counter for storing past column data
  sparkCount = 0;
  
int
  col[8][10],   // Column levels for the prior 10 frames
  minLvlAvg[8], // For dynamic adjustment of low & high ends of graph,
  maxLvlAvg[8], // pseudo rolling averages for the prior few frames.
  colDiv[8];    // Used when filtering FFT output to 8 columns


PROGMEM uint8_t
  // This is low-level noise that's subtracted from each FFT output column:
  noise[64]={ 8,6,6,5,3,4,4,4,3,4,4,3,2,3,3,4,
              2,1,2,1,3,2,3,2,1,2,3,1,2,3,4,4,
              3,2,2,2,2,2,2,1,3,2,2,2,2,2,2,2,
              2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,4 },
  // These are scaling quotients for each FFT output column, sort of a
  // graphic EQ in reverse.  Most music is pretty heavy at the bass end.
  eq[64]={
    255, 175,218,225,220,198,147, 99, 68, 47, 33, 22, 14,  8,  4,  2,
      0,   0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
      0,   0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
      0,   0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0 },
  
  //New filter, tuned for general piano music
  //RED
  col0data[] = {  3,  1, 
     130, 190,  20}, 
  //RED/RED/GREEN
  col1data[] = {  5,  2,
           20, 160, 140, 20,  5},
  //RED/GREEN
  col2data[] = {  7,  4,
                     40, 170, 160,  80,  20,  10,
       5,   1},
  //GREEN
  col3data[] = {  9,  5,
                           5,  20,  90, 170, 120,  
      60,  20,  10,   2,   1},
  //GREEN/BLUE
  col4data[] = { 13,  7,
                                     4,  10,  60, 120, 
     160, 170, 160, 110,  60,  20,  10,   5,   3,   1},
  //BLUE
  col5data[] = { 19, 10,
       3,   8,  20,  50, 110, 150, 170, 180, 170, 140,
      90,  60,  40,  20,  10,   5,   2,   1,   1},
  //BLUE/RED
  col6data[] = { 23, 14,
                           1,   3,   8,  15,  35,  60,
     100, 120, 150, 170, 180, 185, 160, 130, 100,  50,
      20,  10,   5,   2,   1,   1,   1},
  //BLUE/RED/RED
  col7data[] = { 35, 18,
                                               1,   2,
       3,   5,  10,  20,  30,  60,  90, 120, 120, 160,
     180, 185, 165, 140, 120, 100,  80,  60,  45,  35,
      25,  20,  15,  10,   7,   5,   3,   2,   2,   1,
       1,   1,   1},
  // And then this points to the start of the data for each of the columns:
  *colData[] = {
    col0data, col1data, col2data, col3data,
    col4data, col5data, col6data, col7data };
    
    
int nPins = 8;
int ledPins[] = {6,7,8,9,10,11,12,13};
float brightness[] = {0,0,0,0,0,0,0,0};
float fadeFactor = 2.0;

boolean newSpark = false;

int rgb1[] = {44,45,46};
int rgb2[] = {3,4,5};
float color1[] = {0,0,0};
float color2[] = {0,0,0};

int vol[60];

int auroraWait = 0;
int waitThreshold = 0;
int auroraThreshold = 0;
int auroraCount = 0;

boolean playAurora = false;

void setup() {
  Serial.begin(9600);
  
  uint8_t i, j, nBins, binNum, *data;

  memset(peak, 0, sizeof(peak));
  memset(col , 0, sizeof(col));
  memset(brightness,0,sizeof(brightness));
  memset(vol, 0, sizeof(vol));

  for(i=0; i<8; i++) {
    minLvlAvg[i] = 0;
    maxLvlAvg[i] = 512;
    data         = (uint8_t *)pgm_read_word(&colData[i]);
    nBins        = pgm_read_byte(&data[0]) + 2;
    binNum       = pgm_read_byte(&data[1]);
    for(colDiv[i]=0, j=2; j complex #s
  samplePos = 0;                   // Reset sample counter
  ADCSRA |= _BV(ADIE);             // Resume sampling interrupt
  fft_execute(bfly_buff);          // Process complex data
  fft_output(bfly_buff, spectrum); // Complex -> spectrum

  // Remove noise and apply EQ levels
  for(x=0; x > 8);
  }

  int oldPeakSum = 0;
  int newPeakSum = 0;
  int highestPeakIndex = -1;
  int highestPeak = -1;
  // Downsample spectrum output to 8 columns:
  for(x=0; x<8; x++) {
    int oldPeak = peak[x];
    oldPeakSum+=oldPeak;
    
    data   = (uint8_t *)pgm_read_word(&colData[x]);
    nBins  = pgm_read_byte(&data[0]) + 2;
    binNum = pgm_read_byte(&data[1]);
    for(sum=0, i=2; i maxLvl) maxLvl = col[x][i];
    }
    
    if((maxLvl - minLvl) < 8) maxLvl = minLvl + 8;
    minLvlAvg[x] = (minLvlAvg[x] * 7 + minLvl) >> 3; // Dampen min/max levels
    maxLvlAvg[x] = (maxLvlAvg[x] * 7 + maxLvl) >> 3; // (fake rolling average)

    // Second fixed-point scale based on dynamic min/max levels:
    level = 10L * (col[x][colCount] - minLvlAvg[x]) /
      (long)(maxLvlAvg[x] - minLvlAvg[x]);

    // Clip output and convert to byte:
    if(level < 0L)      c = 0;
    else if(level >  8) c = 8; // Allow dot to go a couple pixels off top
    else                c = (uint8_t)level;

    if(c > peak[x]) peak[x] = c; // Keep dot on top

    y = 8 - peak[x];
    
    int newPeak = peak[x];
    newPeakSum+=newPeak;
    
    if (newPeak>0 && newPeak>highestPeak) {
      highestPeakIndex = x;
      highestPeak = newPeak;
    }
  }
  
  if (oldPeakSum< (newPeakSum-2) && newSpark==false) {
    newSpark = true;
  }
  
  int currVolSum = 0;
  for (int v=59; v>0; v--) {
    vol[v] = vol[v-1];
    currVolSum+=vol[v];
  }
  vol[0] = abs(512-int(ADC));
  currVolSum+=vol[0];
  
  int currVolAvg = currVolSum/60;
  
//  int currVolume = abs(512-int(ADC));
  Serial.println(currVolAvg);
  
  int newColor0 = 0, newColor1 = 0, newColor2 = 0;
  if (currVolAvg>64) {if (highestPeakIndex==0) {
    newColor0 = 80;
    newColor1 = 20;
    newColor2 = 20;
  } else if (highestPeakIndex==1) {
    newColor0 = 80;
    newColor1 = 60;
    newColor2 = 20;
  } else if (highestPeakIndex==2) {
    newColor0 = 70;
    newColor1 = 70;
    newColor2 = 20;
  } else if (highestPeakIndex==3) {
    newColor0 = 30;
    newColor1 = 100;
    newColor2 = 30;
  } else if (highestPeakIndex==4) {
    newColor0 = 20;
    newColor1 = 90;
    newColor2 = 90;
  } else if (highestPeakIndex==5) {
    newColor0 = 20;
    newColor1 = 60;
    newColor2 = 90;
  } else if (highestPeakIndex==6) {
    newColor0 = 20;
    newColor1 = 20;
    newColor2 = 100;
  } else {
    newColor0 = 60;
    newColor1 = 20;
    newColor2 = 90;
  }
}
  
  if (newSpark==true) {
    int led = random(nPins);
    float b = 255.0;
    brightness[led] = b; 
    newSpark = false;  
  }
  
  int starsLight = 0.0;
  for (int s=0; s 0.0) auroraWait++;
  else auroraWait-=(waitThreshold/20);
  
  if (starsLight>0.0 && auroraWait>=waitThreshold){
    playAurora = true;
  } else {
    playAurora = false;
  }

  // Every third frame, make the peak pixels drop by 1:
  if(++dotCount >= 3) {
    dotCount = 0;
    for(x=0; x<8; x++) {
      if(peak[x] > 0) peak[x]--;
    }
  }
  
  if (++sparkCount>=50) {
    sparkCount = 0;
    if (newSpark==true) newSpark = false;
  }
  
  if (++auroraCount>=30) {
    auroraCount = 0;
    if (playAurora && newColor0+newColor1+newColor2>0) {
      color2[0] = color1[0];
      color2[1] = color1[1];
      color2[2] = color1[2];
      color1[0] = float(newColor0);
      color1[1] = float(newColor1);
      color1[2] = float(newColor2);
    }
  }
  
  analogWrite(rgb1[0],(color1[0]));
  analogWrite(rgb1[1],(color1[1]));
  analogWrite(rgb1[2],(color1[2]));
  if (color1[0]+color1[1]+color1[2]>0.0) {
    analogWrite(rgb2[0],color2[0]);
    analogWrite(rgb2[1],color2[1]);
    analogWrite(rgb2[2],color2[2]);
  }
  
  for (int p=0; p = 10) colCount = 0;
  
  for (int q=0; q<3; q++) {
    float newFade = float(fadeFactor)/5.0;
    float br1 = color1[q];
    br1 = (br1-newFade< =0) ?0: (br1-=newFade);
    color1[q] = br1;
    float br2 = color2[q];
    br2 = (br2-newFade<=0) ?0: (br2-=newFade);
    color2[q] = br2;
  }
}

ISR(ADC_vect) { // Audio-sampling interrupt
  static const int16_t noiseThreshold = 4;
  int16_t              sample         = ADC; // 0-1023

  capture[samplePos] =
    ((sample > (512-noiseThreshold)) &&
     (sample < (512+noiseThreshold))) ? 0 :
    sample - 512; // Sign-convert for FFT; -512 to +511

  if(++samplePos >= FFT_N) ADCSRA &= ~_BV(ADIE); // Buffer full, interrupt off
}$