REAL2015: Reality Capture Brings Enlightenment from the Cave to the Cloud

Professor Sarah Kenderdine, University of New South Wales, discusses the Pure Land immersive 3D exhibit, a new kind of museum exhibit.

Autodesk Project Memento, shown here with the model of a Garuda (a Southeast Asian mythical humanoid bird) , reconstructed from scans of a statue.

Autodesk Recap offers an easy way to work with acquired data from reality capture devices.

I was standing in the Herbst Pavilion at San Francisco’s Fort Mason, with the Golden Gate Bridge looming in the background. I was also standing in the middle of Cave 220 in Mogao Grottoes at Dunhuang, China, just a few feet away from the ancient Buddhist murals painted on the wall. The surreal experience of being in two places at once was made possible by a pair of virtual reality (VR) goggles loaded with 3 million polygons, one of the exhibits at REAL2015 Conference (March 25-27, 2015).

A representative from the City University of Hong Kong (CUHK) helped me get into gear. He helped me strap on a backpack-mounted workstation loaded with the 3D data and adjusted my head-mounted goggles. He explained that I needed to literally carry the data (inside the workstation’s hard drive) with me during my virtual tour because the rich 3D data was too heavy to be streamed from the cloud, at least in the present incarnation of the setup. Once he flipped the switch, the dim-lit interior of Cave 220 appeared before me as a panoramic view, stitched together in 3D surfaces with texture maps.

“You can’t touch the walls, obviously,” he said, “but feel free to crouch down, look up, or walk up to the murals.”

The VR booth was a demonstration of how some forward-thinking museums are incorporating reality capture and display devices to offer a new kind of exhibit to visitors. The Pure Land exhibit, as the one for Cave 220 is called, first debut in 2011 at Gallery 360, City University of Hong Kong. Since then, it has also made its way to the Smithsonian Institute.

REAL2015 is where Autodesk showcased many of its reality capture products, with Autodesk Recap  and Project Memento leading the pact. Recap is “a family of products and services that allows you to easily integrate reality capture data into your design process,” according to Autodesk. Project Memento is described as “an end-to-end solution for converting any captured reality input (photos or scans) into high definition 3D meshes that can be cleaned up, fixed, and optimized for the Web, mobile or 3D printing / fabbing.”

Tatjana Dzambazova, project lead for Autodesk Memento, said, “Reality capture is now becoming much easier because scanners are becoming much more affordable, so people don’t have to hire service bureaus to do that. They can invest in a scanner and capture the environments.”

Reality capture is traditionally the territory of high-end laser scanners, usually used by civil engineers and surveyors to capture environments as point-cloud data. Autodesk’s software-based approach, which debut as a technology preview at Autodesk Labs under the name Photofly, relies on photogrammetry, using digital photos as input to calculate and construct 3D data. The technology is available today as Autodesk 123D Catch. Recap uses both photogrammetry and point-cloud data.

“Recap enables you to register the scans simply, without knowing anything about scanners. Usually it’s a painful process,” said Tatjana. Since reality capture is rarely accomplished in a single scan, the technology relies on recognizable markers and targets placed on objects to stitch together results from separate scan sessions into a cohesive 3D environment. Recap uses software-driven pixel analysis to identify matching regions, simplifying the registration or data-synching process.

Currently, Recap’s interface lets you measure and interrogate the 3D scene built with acquired data. In the future, Tatjana said, “the software would become smart enough to automatically detect and recognize objects like doors and windows.”

Recap is also the gateway to other Autodesk 3D design programs, like AutoCAD, Autodesk Revit (for architecture), and Autodesk Inventor (for mechanical design).

“The point is to let you work directly with the captured reality data,” said Tatjana, “so you don’t have to remodel what you recognize to be a pipe in point cloud as a cylinder again.”

For more on the Pure Land exhibit, watch the clip below: