CardBoard Universe: AR Project

For my final project I was interested in diving into the world of augmented reality (AR). I have worked a small amount with VR before but never with AR directly. I was excited to learn how to make things in AR and figure out all the pros and cons that come with using AR.

I began by just playing around in AR using Adobe Aero. I imported a lot of models I already had made or downloaded and played with those. I quickly found myself very interested in AI and putting random things into AI and seeing how well it can handle making these models. As well as filling in the gaps for areas of the model that are unseen in the pictures. I started with a lot of random pictures of plants from my camera roll and putting those into AI and messing around with them until I got 3D models that resembled the plants.

After generating a lot of models of random plants I started to try to make some AR scenes involving lots of plants so it would look like the user is in a jungle of massive flowers. I unfortunately realized quite quickly that my laptop did not have the hardware to support these massive scenes as Adobe Aero began crashing frequently after only adding a small fraction of the pants I was hoping to add. I also came to learn that the more stuff I added to the AR, the more time it takes to load on a users phone.

I realized that making giant complex scenes would be too intense for both my laptop and users phones. I decided to move away from this concept and focus more on bringing real life objects into AR via AI tools like Meshy.

From here I took random pictures of myself and friends, then would test what AI like Meshy were good and not good at generating. I noticed Meshy was good at generating the general shape of the models but still struggles a good amount when it comes to smaller details such as the face as well as anything written on any clothing. There were also a few perspective issues depending on the photos used. The basketball one is a good example, the model when viewed form the front, you can tell the basketball is not aligned at all how it would be in real life.

After making these models I found myself very fascinated by how AI interprets certain models especially the parts that are generated by the AI that can't be seen in the photos. I uploaded one of me and 2 of my friends but our legs were cut out of the picture. The AI decided to generate one of my friends with 5 legs randomly and the other with a 3rd. I also made a model of my friend Maximus who was wearing a shirt with a cat faces on it. Meshy took the face as being a real cat so in his model he has what looks like the back of a cat coming out of him which I found to be a funny little accident.

Because AI allows for 3D models to be made in the span of 5-30 minutes rather then multiple hours, I decided to add an interactive element to my exhibit and told people I can make 3D models of them or anything they had a picture of if they wanted. My family where the only people to take me up on this offer at the show.

I made a model of model of my younger brother who said it was "weird and a little scary" to see a model of himself. He was disappointed that after several redos we still couldn't get the face on his model to look more like his own.

My younger brother also gave me a picture of our dog Burrito to make into a 3D model as he loved the cat model and wanted one of a dog too. There were more accurate models to choose from on Meshy however the model I chose was generated with an obscenely big head that me and my younger brother found to be incredibly hilarious. I noticed a lot of random accidents like this with AI that end up adding an element of humor more often then not just because of how weird and random it can be to see a cat model coming out the back of someone or a dog with an unusually big head for no reason.

The last model I generated during the show was of my mom. The AI did a terrible job at generating her face multiple times in a row and my mom may now have a slight hatred towards AI for doing her dirty in the models. Despite the result my mom still found the whole process to be very cool that we were able to generate many models similar to her live during a show in a small amount of time, she thought it did a good job of being able to get the shape of the pose she was in and differentiate the layers of clothing. However through around 10 different attempts the Meshy AI still could not get a face that resembled hers.

I forgot to take a video or any pictures while at the show. Here is a picture of the setup re-setup similarly at my house after the show.

This project was a cool new experience for me. I definitely had some learning curves. Trying to figure out a way to display an exhibit that was entirely online was challenging. I opted to print out big QR codes in hopes of grabbing attention and getting people to scan them. However if I was to redo this project I would probably try to come up with a better way of drawing peoples attention to the codes and getting them to scan them as I felt a lot of people would walk past without even knowing it was setup.

Another issue I found quickly was the size I made some of my models. The space I was testing them in was very open and made it easy to place the models and look at them. In the more crowded exhibit area it was harder to place the models. As well as view them because of there size and placement, often leaving some of my users struggling with their phones to find a place to set the model or be able to view the entire model.

I hand't thoguht about the internet connection prior to the project either. Adobe Aero creates a QR code that when scanned brings up the AR project. However it pulls this project form the adobe cloud and therefore needs an internet connection to open the project. The bigger projects also would take significantly longer to load then smaller ones, giving me another incentive to change the models sizes to smaller ones if I was to do a future exhibit involving AR.

Outside of the Learning curves this project was fun to explore a new form of digital art and play around with Adobe Aero and AI like Meshy. I wish I had more time prior to the exhibit to create more detailed scenes in AR. If I was to make a future project in AR I would play around with making a scene that has interactivity involved. I saw on Adobe Aero you can add interactivity to models, I didn't have enough time to add it to this project unfortunately. I definitely think it would be the next step in taking an exhibit similar to this one to the next level.