vikz-arsculpture

Untitled Duck AR by Meijie and Vicky is an augmented reality duck that appears on your tongue when you open your mouth, and is triggered to yell with you when you stick out your tongue.

https://www.youtube.com/watch?v=xt1FgOcHXko&feature=youtu.be

Process 

We initially started off by using a bunch of our limited developer builds (heads up for future builds: there is a limit of 10 per week, per free developer account lol) by testing the numerous different types of build templates that we could use to implement AR over our mouth, most particularly image target, face feature tracker, and face feature detector.

We actually got to successfully have an image tracker work for Meijie's open mouth, however, it was a very finicky system because she would have to force her mouth to be in the same exact shape, and very similar lighting, in order for it to register. We plugged in an apple prefab, and thought it was quite humorous as it almost was like being a big stuffed with an apple.

With this, we initially wanted to explore having an animation of some sort take place in the mouth. However, that proved difficult due to the lack of accuracy with small differences in depth, and also the amount of lighting that would need to be taken into consideration. Also, because the image target had some issues with detecting the mouth, we decided to migrate to the face mesh and facial feature detector.

We combined both the face mesh and feature detector, to trigger a duck to appear on the tongue when the mouth is open.

12.4.19 Updated Prototype

Having the duck appear (within grass and more refined detail) when mouth is first opened, and then having a raw duck (yum yum!) appear the second time mouth is opened.

vikz-arwo-justaline

For the Justaline project Arden and I played around with different possibilities and ideas. Ultimately, we really enjoyed the depth effect we could make in the app by drawing multiple "doorways" to move through. We also took advantage of the potential to create transformative effects by having shaped frames change into other shapes while passing through. This augmented reality creates an anticipation for the viewer, kind of like a tunnel or a rabbit hole, as they travel through the floating framed shapes. Here we chose to transform a triangle into a circle:

viks-UnityScripting

For my Unity Scripting tutorial, I decided to start at the basic fundamentals and dive into the Course with Code. I went through the first tutorial of the series, Start Your 3D Engine, in which a simple car game "world" and assets are being created and imported, and then I went through the second tutorial, in which we create and apply the first C# script.

Part 1.1

Part 1.2 with scripts

vikz-SituatedEye

We were really inspired by Google's drawing machine learning, and the ability to play around with the different types of applications that machine learning has in with drawing. In order to most quickly and accurately iterate over and over again, we started our explorations by playing around with the whiteboard. We started off playing around with the program to see if machine learning was able to detect the difference between written text and drawings. From this, we were also thinking of maybe incorporating mathematical graphs and/or equations as a possible third scope; an example that lives between text and drawing.

From our experiments, we saw that computer could usually detect between drawings and text, presumptuously mostly dependent on the text. The diversity of drawings was differed widely, as we literally began to draw just about everything that first came to mind, whereas text was definitely more limited in terms of aesthetic, and was visually more uniform. However, we came upon an interesting discovery when drawing stars, but in a linear form. Despite being drawings, it was detected as text, because of its linear nature. This propelled us into thinking about the possible implications for using machine learning to detect the differences between languages.

The stars that sparked off our stars.

Our final exploration dealt with exploring the detecting the difference between western and eastern languages; western being more letter-based, and eastern being more pictorial-based characters.

Western Languages

Eastern Languages

Training our model with white background, western text, and eastern text.

We decided to map the result out visually through three colors:

  • White indicates that there is no hand written text being shown. (We fed a series of white background images to train this part)
  • Blue indicates that it is more-so western text. (We fed a series of handwritten phrases and words in English, Spanish, French, and Latin to train this part)
  • Red indicates that is more-so eastern text. (We fed a series of handwritten phrases and words in Chinese, Japanese, and Korean to train this part)

From our results, we've discovered a couple things.

The program is relatively good at detecting a "blank background" though a couple times, when our paper was shifted, the program recognized it as "western".

But most importantly, the program was very accurate in detecting western text, but significantly less so with eastern text.

This observation has led us to a couple hypotheses:

  • Our data is lacking. We took about 100 photos for western and eastern each, but this may have been not enough for the machine learning to generative a conclusive enough database.
  • The photos that we took could also have been of not high enough quality.
  • In our sample data, we only wrote horizontally for western text, where as eastern had both horizontal and vertical.

Future thoughts...

To test the machine learning program to see if could simply tell the difference between western and eastern languages, we could do away with the "varied handwriting" completely and use very strict criteria (for handwriting style) in writing our sample text. When we tested the learned program, we could continue to write in that same style between the eastern and western texts. This could help isolate our variables to test out our above hypothesis.

vikz-ML

A. Pix2Pix

edges2cats

For edges2cats, I found it most fun and interesting when I would try to create things completely unlike cats -- for example, a teacup, and see what kind of "cat" would be created. I found it odd, however, that when I did strive to make a cat, it didn't seem to register it all that well (ie. my double headed cats)

facades 

Facades was also interesting to play with in that it allowed me to create this unique entities that, in my opinion, are barely recognizable as buildings and architecture.

B. GANPaint Studio

GANPaint studio, in my experience, was a bit more frustrating to use. I had some difficulty in painting over certain areas and altering the landscape. The "dome feature" was the only one that I could paint over in random areas and have be the most notably.

C. ArtBreeder

General 

Portrait 

The most intriguing part of ArtBreeder for my was by far toggling between the different percentage of genes in each photo. Even merely scaling one gene on one side of the spectrum to the other side of the spectrum produced crazy diverse results. I found it interesting how I managed to create something that looks of a bread / boiled crab / cheesey tomato dish without using a single element of anything of that sort.

D. Infinite Patterns by Alexander Mordvintsev

Original Image

 

Tiling Image

E. GPT-2

How to make a chicken pot pie in under 5 minutes:  This recipe makes over 8 chicken pot pie, or 6 for a larger meal.  Make sure your soup is very soupy and thick before pouring your soup into a pie shell.  This chicken pot pie is also great with a crust or rice pudding.  

3 lbs bone in chicken, skin on for quick cooking for lunch/dinner  

3 cups chicken stock (or broth)

1/2 tsp salt

1/2 tsp pepper

1/2 tsp garlic powder

1/2 cup onions, chopped

12 eggs, whisked and peeled

1 egg white, whisked

a splash of lemon juice

1 Tbsp olive oil

1 1/2 cups baby carrots, peeled and chopped

1 tsp vanilla extract

1 cup chicken stock (or broth) 

a dash of red wine vinegar

2 tsp salt

1 egg, beaten with about 1/3 cup of the stock (or broth) in a small bowl (preferably more than your own cooking water)

2 Tbsp extra virgin olive oil

It all started when I first met her at Lion King'. Her face was red all over and she was so upset. It was the same for me. It wasn't good that she looked like she was crying, I didn't want this kind of sight in my eyes. Well, it's not like I care that much anymore, so let's do this.

"Please wait."

"Please don't come."

After that I said it again when I said it to the other party. They didn't come, so I had to leave the hall and walk past it.

We arrived at a store with an attendant.

"Here, take out the Cake'."

The person came out. I tried to ask if he wanted to go out, but it seems that the person said that that would be for me. So, I took out the cake.

I'm pretty sure there are others who didn't get a chance to see the cake.

"Here, take out the Cake'."

F. Google AI Experiments

I played around with a lot of the AI Experiments listed on the site, but the one that I found had the most satisfying experience with the highest "accuracy" was definitely the "Quick, Draw!" activity. Many times, I was shocked at how little I had drawn before the neural net had already detected what I was trying to accomplish.

vikz-Telematic

 

Moood is a collaborative listening(ish) experience that connects multiple users to each other and Spotify, using sockets.io, node.js, p5.js, and Spotify API's.

In this "group chatroom" of sorts, users remotely input information (song tracks) asynchronously, and have equal abilities in leveraging the power (color, or mood) of the room.

This first iteration, as it currently stands, is a bare minimum collaborative Spotify platform. Users type a song track (must be available on Spotify), which would then be sent to a server, to be communicated to Spotify. Spotify would then analyze the track based on six audio features: 1. Valence (a measure describing the musical positiveness), 2. Danceability (a measure describing how suitable a track is for dancing based on a combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity), 3. Energy (a measure representing a perceptual measure of intensity and activity), 4. Acousticness (a confidence measure of whether the track is acoustic) 5. Instrumentalness (a prediction on whether a track contains no vocals), and 6. Liveness (a detection of the presence of an audience in the recording) These six audio features are then mapped onto a color scale, and are the aspects in which dictate the color gradient being represented on the screen, which will then be broadcasted to all clients, including the sender.

At this stage, users would be able to share current songs they are listening to and dictate the way in which the "mood" of the room is represented, by changing the color in which room would be. Color is an extremely expressionistic and emotional visual cue, which has the ability to tie in beautifully with the aspect of music.

Our initial idea is a lot more ambitious, however, we ran into several (an understatement lol), issues. The original play was to create a web player environment that would consist of the 3 RGB colors, and CMY color overlaps, with white in the middle. Users would be able to click onto different colors, and the combination / toggle of the colors would trigger different songs to be played based on our mapping of colors to the Spotify API endpoints used above (in our current iteration). Users would then be able to dictate the visual mood of the room, as well as audio mood of the room, by mixing colors and playing different songs. First, there was the issue of the being able to create user authorization; there are several different types of it, some not being compatible with certain codes, and others having certain time limits. Next, there was the issue of being able to handle playback on Spotify Web API, versus Spotify Playback SDK, versus using Spotify Connect. SDK did not allow for collaboration with node.js, but the other two ended up creating issues in overlapping sockets, listening ports, and so on. We were also unable to manipulate / figure out how to pull apart certain songs from select playlists, but that was an issue that we could only have dip into due to the other issues that were more pressing. Because there is not only server and clients being communicated across here, and instead the entire addition of another party (Spotify), there was often conflicting interests in where that code intersected.

That being said, because we have managed to overcome the main hill of having all these parties communicate to each other, we would want to further work on this project to incorporate music (duh). It is quite sad that it is a project revolving around Spotify and music as a social experience, without the actual audio part.

by Sabrina Zhai and Vicky Zhou 

vikz-CriticalInterface

All the topics within the Critical Interface Manifesto was stimulating and of valuable insight. After reading through them, I decided to focus on tenant #13: In the design of the interface, not only skills but also emotions and affects are deployed. How are emotions produced and circulated in interfaces? A couple of proposals within that tenant were "Have breakfast first, then connect. It is not necessary turn on your smartphone in the bed. Or: connect first and have no breakfast.", "Laugh out loud every time you put a smiley.", and "The means of emotion predictions are today's means of production."

The propositions under this claim I found to be particularly interesting, because of how it taps into the most complex of all complex systems -- emotions. A good amount of the propositions listed were tips typically associated in self-improvement books, such as choosing to power down all devices and go for a walk or run, or choosing to leave a mobile device at home for an entire day (which, is an ongoing trend within the YouTube community). However, some of the propositions were definitely ones that would create odd discomfort, such as choosing to either have breakfast first, and then connect on social media, or choosing to connect, and then banning oneself from having any breakfast at all. One of the ones that stuck out most with me was "the means of emotions predictions are today's means of production.", and what I took away from that, was the idea that emotions heavily play into the productivity and how we conduct each day. As a creative, I particularly feel that my self-worth is correlated with my ability and quality of the things I produce, and so I feel as though this sentence can be applied vice versa as well, in that today's means of production, many times, end up heavily influencing my means of emotions. This is a trait that, I hope to get rid of over time, and better understand to separate my work from my own self worth.