Category: 07-Telematic

The concept for this project was a multiplayer game of jumprope within the browser. The idea was that each player has their own role (jumper, swinger).

In its original concept, the goal was to have the experience be a one-to-two person connection, where two people swing the rope while the third jumps. Due to the limitations of trying to get the physics to work across browsers, the scope has been limited to one-to-one person connections in synchrony. The complementary roles of jumper and swinger would need to be fleshed out if I was to continue working on this sketch. There is a certain power dynamic between being the one swinging the rope and the one jumping. Participants are anonymous, but anonymity does not have a significant impact on the experience. Location is also less important, though due to timing, being in the same room probably makes the experience easier to manage. The project is ultimately trying to explore a novel interaction over wireless connections in browser. Unfortunately, the project is unfinished currently as the logic turned out to be difficult to crack. In order to solve the physics issues, I have the rope swinger using real physics, while the jumper just has points mirroring the swinger. If I were to take this further, I would implement a reward/interaction system for the jump interaction.

A Simple Music-Based Chatroom

https://glitch.com/edit/#!/harmonize?path=public/sketch.js:124:17

I edited the project after making the .gif, to make it easier to visualize what sound each client was making.

My main idea behind this project was about how to make a music-creating service that could be edited by multiple people at once, like a Google Doc. I first thought of using clicks on specific spaces to record a note, but I later realized it would be more stimulative if I mapped certain keys on the keyboard to piano keys. As such, the keys from a to k map to a corresponding note from a3 to a4, and w, r,t,u,and i map to the sharps/flats.

As per Golan's suggestion and general user confusion over the column location for each note, I've changed it so that each client's notes only occur within a specified column, and changing the note played only changes the row location. Now, each user's "cursor" will be constrained to within a specific column. Also, the text on the left-most column tells you which key to press to play that row's specific note.

My network model is effectively many-to-one(-to-many). I'm taking multiple clients' music data and note location, and sharing it with all clients on the same canvas. The project is synchronous, so people will be "communicating" in real-time. Participants are unknown, as there is no recorded form of identification, so communication across the server is anonymous.

The biggest problem for my project was getting the oscillator values to be emitted from/to multiple clients. I couldn't directly emit an oscillator as an object with socket.emit, so with some help, I found that I needed to just emit the parameters for the functions each client's oscillator used, and create a new oscillator to play that frequency upon calling socket.on. This'll definitely lead to memory leaks, but it was the best hack to solve this problem. Another earlier problem I had was that since music used time at a specific rate and because draw() updated 60 times per second, I needed to decrease that number so that the note changes would actually be visible on each client's screen. This was an easy fix, and I used frameRate(5) to decrease it down to a visually manageable number.

A problem that I found unsolvable was that glitch kept saying that all p5.js objects or references were undefined, even though I defined p5 in my html file. I later found out that this was because though there was a definition of p5.js on the browser side (thus the code compiled and the p5 elements were visible), the glitch server did not contain a reference to p5, so it would throw an error.

Chaotic Garden Glitch

Enter Chaotic Garden 

This was inspired by Ken Goldberg & Joseph Santarromana's TeleGarden. I really liked the idea of maintaing a garden together, and the idea of community.

Users collaborate simultaneously, but plant their own seeds independent of each other. ( The user can only see their own plants). When they water their plants, they are watering everyone's plants (whoever is online). Users are anonymous, only shown by a cursor. This makes taking care of your garden somewhat chaotic. If someone is watering their plants, then it seems like water is appearing out of nowhere and watering your plants as well. Each user's watering action affects all the other users' gardens

Initially, I tried to make a virtual shared musical garden. The y position of the plants determine music note, like notes on a music sheet. (In addition, the plants would die not watered within 10 minutes). I was not able to implement the shared plants and music in time. The only thing shared is the watering action.

(first ideas of motion tracking and hand drawings)

A chatting application where you can only post sentences that have been written down before (and exist in the google books database).

https://glitch.com/edit/#!/quotechat?path=public/sketch.js:92:15

This piece is, to my dismay, a very unfinished experiment that doesn't really create interesting results. I was not able to really explore the realm of possibility to create a more engaging interaction.

Moood is a collaborative listening(ish) experience that connects multiple users to each other and Spotify, using sockets.io, node.js, p5.js, and Spotify API's.

In this "group chatroom" of sorts, users remotely input information (song tracks) asynchronously, and have equal abilities in leveraging the power (color, or mood) of the room.

This first iteration, as it currently stands, is a bare minimum collaborative Spotify platform. Users type a song track (must be available on Spotify), which would then be sent to a server, to be communicated to Spotify. Spotify would then analyze the track based on six audio features: 1. Valence (a measure describing the musical positiveness), 2. Danceability (a measure describing how suitable a track is for dancing based on a combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity), 3. Energy (a measure representing a perceptual measure of intensity and activity), 4. Acousticness (a confidence measure of whether the track is acoustic) 5. Instrumentalness (a prediction on whether a track contains no vocals), and 6. Liveness (a detection of the presence of an audience in the recording) These six audio features are then mapped onto a color scale, and are the aspects in which dictate the color gradient being represented on the screen, which will then be broadcasted to all clients, including the sender.

At this stage, users would be able to share current songs they are listening to and dictate the way in which the "mood" of the room is represented, by changing the color in which room would be. Color is an extremely expressionistic and emotional visual cue, which has the ability to tie in beautifully with the aspect of music.

Our initial idea is a lot more ambitious, however, we ran into several (an understatement lol), issues. The original play was to create a web player environment that would consist of the 3 RGB colors, and CMY color overlaps, with white in the middle. Users would be able to click onto different colors, and the combination / toggle of the colors would trigger different songs to be played based on our mapping of colors to the Spotify API endpoints used above (in our current iteration). Users would then be able to dictate the visual mood of the room, as well as audio mood of the room, by mixing colors and playing different songs. First, there was the issue of the being able to create user authorization; there are several different types of it, some not being compatible with certain codes, and others having certain time limits. Next, there was the issue of being able to handle playback on Spotify Web API, versus Spotify Playback SDK, versus using Spotify Connect. SDK did not allow for collaboration with node.js, but the other two ended up creating issues in overlapping sockets, listening ports, and so on. We were also unable to manipulate / figure out how to pull apart certain songs from select playlists, but that was an issue that we could only have dip into due to the other issues that were more pressing. Because there is not only server and clients being communicated across here, and instead the entire addition of another party (Spotify), there was often conflicting interests in where that code intersected.

That being said, because we have managed to overcome the main hill of having all these parties communicate to each other, we would want to further work on this project to incorporate music (duh). It is quite sad that it is a project revolving around Spotify and music as a social experience, without the actual audio part.

by Sabrina Zhai and Vicky Zhou 

Unfortunately for this project, I was unable to make it work completely with the server in Glitch. For this post, I'm sharing my trials and showing things that did end up working.

Shown above is an example of the convex hull algorithm I implemented in the p5 editor with some mouse interaction. I was interested in creating a generative gem or some kind of abstract shape with the cursors of each client visiting the page, but as these things sometimes go, I could not get it to work in Glitch. I liked the idea of a shape growing and shrinking as a group of people collaborate on its formation, all the while leaving history of each move made with the trail of points each cursor would leave.

The sketch in the p5 editor can be found here, and the sketch I ended up on in Glitch can be found here.