Klaus Photogrammetry Project

March 2023

As an exec member of the GTXR club, this project was a kind of “commission” to be made before ImmerseGT. As it was based in Klaus Atrium, the idea was it could be used as inspiration by participants or in their projects if they were interested in doing something with a digital twin of the space they were in. It was inspired by the digital twin of MIT’s Media Lab that was made for Reality Hack for the same purpose. Unfortunately, I don’t think either digital twin was utilized by participants in a project, but it was a really fun challenge anyway.

Actually capturing the atrium was the most difficult part of the project because of its very large scale and abundance of reflective surfaces, which computers usually have a hard time understanding for photogrammetry processing and image alignment. I think I went back to the atrium to capture it with different experimental methods about 5 times, and had some difficulties there too because I had to find times when the lighting conditions were appropriate and there was nobody in the atrium. Another problem with the atrium was it was filled with tables and chairs, which made for a lot of complicated geometry, and their matte white textureless surfaces added additional difficulty for the software to track. The big breakthrough that made this project possible was using Polycam with the LiDAR sensor on my iPad to accurately scan the entire first floor, except for a little bit of alignment drift in such a large space. LiDAR only works up to 6 feet away from you, but cuts through the reflective problems with traditional image capture with its depth information. In Blender, I combined this with a model of floors two and up, the walls, and the ceiling. This model was my only decent result from Reality Capture, but it had to be smoothed out a lot.

I also took advantage of this opportunity to incorporate the newly released teleportation and snap-turning hand tracking gestures in Meta’s Interaction SDK. It took less than a day to reverse engineer the sample project by putting my model in it, adding a skybox, and setting the areas you are allowed to teleport to on the floor. I did this because I wanted it to be easy and intuitive for anybody to navigate the project, whether they wanted to use controllers or not. 

This was the first project I’ve ever published to SideQuest, and I’m proud of that. You can sideload the APK if you're interested in trying it out for yourself:

https://sidequestvr.com/app/15296/georgia-tech-klaus-atrium-photogrammetry-project