Javascript, Arduino, Photoshop, Illustrator, Leap Motion Sensor

Team Project with Amanda Lee.


PICSELF is a mixed reality drawing photo booth using hand motion sensors. Users can see themselves on the screen while decoration background by drawing in the air with raw hands. When they finish doodling i the background they can capture the moment while playing around with green props that will mix reality with digital drawings. 


Ideation & Research


I was inspired by a sticker photo booth in Korea. People using the photo booth choose background images, take selfies, decorate on top of the picture and print them out. This was very popular in overall Asia by the time I was a teenager(about 10 years ago). I wanted to recreate ad share this old good memories and experiences with people using ew technologies. 


The green screen will make users' images blend into digital drawings generated by their hand motions. The user will stand in front of a green screen which will be converted into the blank digital canvas so that they ca decorate their own background with their raw hands and physical buttons. Their hands will be brushes, erasers or stickers.



As shown above, we used Leap motion sensor to catch hand motion. The range was relatively small but very accurate. I was able to get hands' data in Javascript language, so we started to play around with it using the web. However, even though the hand's movement data was accurate, it wasn't easy to draw precisely due to the absence of tangible interaction feedback. So we slightly changed the concept from "drawing" to "decorating" using stickers and shapes.

IMB_7sZgqG 2.GIF

Build & Test (multiple times)

After doing few experiments with the motion sensor ad green screen, we started to build the actual project we were imagining. We used Javascript and Arduino mainly. Also we did lots of user tests and iterations to make the most enjoyable machine. 

FullSizeRender 7.jpg
Screen Shot 2016-12-11 at 1.28.22 AM.png

After few iteration, we started building a wooden box to keep all electronic devices and to display buttons. This box gives users sense of the range of drawing area and a deeper experience by providing both tangible (buttons) ad no-tangible actions(drawing in the air) at the same time. Also, we prepared a variety of green props that will blend with users' drawings. 



We showed out project in ITP Winter Show 2016 and NYC Media Lab 2017. Lots of people enjoyed PICSELF. I wanted all users could take their pictures with them like a souvenir, thus created a simple web archive site directly connected to the project. Users can download pictures taken from PICSELF in the web page whenever they want.