Georgia Tech OIT Summer Job - Omniverse and Photogrammetry

Summer 2023

This was my first full-time job with coworkers I met at Georgia Tech’s ImmerseGT hackathon, Austin Graves and Connor Wright, so it was a great learning experience. I worked under Didier Contis, the Executive Director of the Office of Information Technology at Georgia Tech. There were multiple main focuses of the job, with each of us mostly working on individual projects specific to our specialties, as Connor is Literature, Media, and Communication (LMC), and Austin is Mechanical Engineering (ME). For the first half of the summer, I mainly focused on many of the features in Nvidia Omniverse and its potential use cases. I experimented with its PhysX and RTX ray tracing capabilities to make fun sun study renders of different buildings on the Georgia Tech campus, such as Klaus Atrium and the Kendeda Building, as well as a model created by my good friend, roommate, and architecture major Brian Roberts. As for XR capabilities, we unfortunately found that the platform is very inaccessible, basically requiring an RTX 3090 GPU, as well as lacking valuable features for VR experiences besides real-time ray tracing. This was somewhat generally the case for Omniverse’s other flatscreen use cases.

In the second half of the summer, I focused more on digital twin applications in VR. We were lucky enough to obtain a white 3D model of the Kendeda Building from Georgia Tech’s IMAGINE Lab, so I put that in a standalone Meta Interaction SDK app so you could teleport around it and explore it in VR with hand tracking or controllers. This specific project was fun because I learned about light baking and was also challenged by performance limitations for the first time. The model provided was 9 million polygons, so I decimated much of the high poly count furniture in Blender to cut the overall poly count in half. I also baked the lighting and experimented with SSW (synchronous space warp) to render the app at half the frame rate, but was unsuccessful. 

To continue digital twin experiments, my colleagues and I met at Georgia Tech on a Friday and spent a couple of hours capturing the entire Flowers Invention Studio with a Matterport Pro2 camera. This allowed us to create a flatscreen digital tour of the space with many info markers, as well as the ability to pay to export the entire space as an optimized 3D model. This was the best software I had tried for large-scale space capture because it has just the right amount of polygon optimization while still being good quality. I was very excited to work with this because it was the largest photogrammetry model I’ve had the opportunity to work with so far. I blocked out optimized collision for it and uploaded it to VRChat as a public world anyone can tour. I also helped Austin set up a PCVR app version and taught him a little bit of GitHub and Blender.

Flowers Invention Studio VRChat World

Klaus Atrium VRChat World

There were many other small projects that were unsuccessful or didn’t lead anywhere, as well as work on an exciting NDA platform. I am very grateful for the opportunity provided by Didier Contis to have my first full-time job be one that allowed a relatively stress-free environment for open experimentation, and am therefore going to continue the job in the fall part-time as well as onboard new students.