SNAP LENS ACADEMY (SLA)
This summer (2024), I was one of 15 scholars selected for Snapchat's 9 week Augmented Reality training program. I didn’t really know what I was getting myself into. I had a lot of different skills that didn’t seem to connect, and yet they all were applicable to AR, something I had never even considered before.
Lens Studio
The crazy thing is that Lens Studio actually made all of this so accessible, broke almost everything down so that it was easy to understand, and made it relatively easy to create and publish something from start to finish. I was able to create something in a program that I had never touched before, and it was something I was proud of. I still remember how excited I was about seeing my artwork in a real life space through my phone.
AR MARKERS
One of our lessons in core curriculum covered Marker Tracking. This essentially meant that you could create a marker in the real world that someone could scan to activate an AR experience.
This lesson was definitely my favorite.
I had been wanting to learn this for a while because as a 2D illustrator and animator, i always like playing with the idea of bringing a drawing to life.
Lion Marker
I wanted to make it feel like the drawing popped out of the page and into the real world!
Little Painters
The characters are relaxing and then feel pressured to start working when the user scans them. I wanted to try a continuous loop for the GIF so it felt more natural
final project
Gong Gong’s House
An AR Immersive Experience
PROJECT OVERVIEW
“Gong Gong’s House” is an augmented reality experience built in Snapchat’s Lens Studio. We really wanted to create a lens that showcased immersive education, leveraging AR for interactive learning and cultural education.
FINAL showcase / education
For the final project, the scholars were split up into groups of three and given a topic to base their lens around. Darren and I were paired up and were given the topic of Education.
With just the two of us, we were able to quickly go back and forth on ideas and decisions and be extra collaborative. Our skills complemented each other really well; Darren had more experience with 3D and scripting, and I was very comfortable with 2D work. We both knew our way around Lens Studio at this point too.
NARROWING DOWN THE TOPIC
Education was a really broad topic to be given. Initially Darren and I were all over the place with ideas, but we ended up pivoting to focus on something much more personal.
Both Darren and I happen to come from Taiwanese and Chinese backgrounds. And with that, family plays a big role. We wanted to bring up education in the context of our culture.
Mahjong
For me in particular, my Gong Gong’s memory has been slowly getting worse. It’s been really difficult to watch. But! Even though there’s a lot he can’t remember, when you put mahjong in front of him he remembers everything about it. And he’s good.
These were some initial ideas for the inside of the house.
Visual Development - Props and The House
Beyond mahjong, our goal was to teach people about the small moments that capture parts of our culture, the importance of family, and explore the topic of memory. This meant that the space itself was very important. I started by trying to capture things that are staples in my grandparent’s home. These items are all familiar to Darren, as well as many Asian Americans.
After feedback, we simplified the house to resemble more of a cube so the user could walk around more easily.
DarRen + Blender
From these initial sketches, Darren went in and started modeling the whole house over the course of a day.
Working off of Darren’s 3D models, I started to draw out some visuals to get a feel for the coloring and textures of the furniture in the house.
UV MAPS AND TEXTURING
After finally deciding on the layout of the house and the final positioning of all of the furniture, we grouped all of the UV maps of all of the furniture into one image so that I could go in and add textures. This was a technique my mentor suggested to bring down the size of the lens.
During our planning, one of the things we really wanted to keep in mind was how to make this a uniquely AR experience. We wanted to separate this lens from just a normal video game where you could tap the screen. And so we decided to implement…
Hand Interaction
We began to test out what we could do with hand interactions so that the user could actually reach out into the space.
We tried many different things. Initially we started with a grab function, but that feature would only stay in orthographic view and I couldn’t quite figure it out. We shifted to using physics bodies but that came with some problems too. The user could move the tile but it didn’t feel like they were collecting it.
In the end we finally added particles so that when the user reaches out it feels like they actually touch the tile and collect it. Darren modified the 2D animated gifs I made into flip book animations in Lens Studio to make the particles.
Immersive Scavenger Hunt
Because the environment is such an integral part of the story behind the lens, we were posed with the question of how we would get the user to actually explore the space. One of the ideas we went with was turning the lens into a scavenger hunt, which would force the user to move around.
LENS TECH STACK
2D Assets, 3D Modeling, 2D Textures, Custom VFX, Particle Systems, Behavior Scripts, Hand Tracking & Interaction, Physics Bodies and Colliders, 2D Animation, 3D Animation, World Tracking, Spatial Triggers, and Custom Scripts
The End
I learned so much in such a short amount of time at the Snap Lens Academy. I’m so grateful for the experience, and really lucky to have been able to build a project with Darren that we were both very proud of. Looking back, I realized that we incorporated so many things that I had never even known about before, and it was really all thanks to the teachers at SLA and how easy it is to pick up Snap’s Lens Studio.
While this might not be the kind of content that is popular on Snapchat (it’s not a cute dog filter or a funny face), I think that there’s still a whole world of AR to explore by utilizing Lens Studio to create immersive environments that capture meaningful experiences, moments, and stories.