Home / General / AI / NVIDIA GTC 2017: Welcome to the Holodeck

NVIDIA GTC 2017: Welcome to the Holodeck

Last Wednesday, donning his trademark leather jacket, NVIDIA CEO Jensen Huang took the stage at the San Jose McEnery Convention Center to deliver his keynote to the estimated 7,000 attendees at the annual NVIDIA GTC Conference.

“GTC is where the future is invented; GTC is where we create what others think of as science fiction.” Huang said. To prove his statement is not a hyperbole, he proceeded to demonstrate an iconic sci-fi concept, the Holodeck.

In Roddenberry’s Star Trek, the Holodeck is a reality simulator built with tangible, solid holograms, a feature of the Federation starships. NVIDIA’s Holodeck, on the other hand, is built with pixels and bytes, a virtual reality (VR) environment with photorealistic avatars and interactive physics, powered by the company’s VR-ready GPUs.

Though it lacks some of the features dreamed up by the creators of the cult TV show, NVIDIA’s version stands as a tantalizing early prototype, made possible by a convergence of the technologies that have become NVIDIA’s core strength.

“We play at the intersection of virtual reality and artificial intelligence,” Huang said. “Nothing exemplifies that intersection like the Holodeck.”

NVIDIA CEO Jensen Huang revealing an early unit of Tesla V100, based on the next generation Volta architecture (photo courtesy of NVIDIA).

A Trek into the Holodeck

NVIDIA’s Holodeck is a project, not a product, the company’s executives were careful to point out. The setup was created using NVIDIA’s GameWorks, VRWorks and DesignWorks SDKs (software developer kits).

“The Holodeck is not only a place to go, but a place to share. It has to obey the laws of physics. Otherwise, it wouldn’t feel like a place,” Huang noted.

To make this point, Huang invited carmaker Christian von Koenigsegg and several other participants into the Holodeck, to preview the Koenigsegg Regera, a luxury vehicle powered by a twin-turbine V8 combustion engine.

In many VR setups currently available, when you reach for an object (such as a steering wheel), your hand does not feel its weight. Similarly, when you encounter a barrier (such as a wall), you can still walk through it. NVIDIA’s vision is to add a layer of realism by introducing haptic feedback and realistic physics, usually accomplished with gloves in addition to the headset. The shape of the Holodeck is still evolving, and NVIDIA is looking to the early adopters for ideas.

“The goal is to continue to increase the number of simultaneous users who can participate [in the Holodeck’s VR sessions],” said Jason Paul, NVIDIA’s GM of VR, in the post-keynote Q&A. “That’s the reason we want to bring it out in September, to get the enthusiasts to produce social content. There’ll be a massive social media event … First, we want to show people the power of collaboration using good audio, physics, and massive models … We expect there to be Mods [modified version created by users].”

NVIDIA CEO Huang demonstrated the Holodeck project, a highlight of his keynote at this year’s GTC (photo courtesy of NVIDIA).

Affordable Realism

In some cases, VR-driven training blurs the line between science fiction and real science. Speaking on the panel titled “Beyond Games: How Unreal Engine is Putting the Reality into Virtual Reality,” Matthew Noyes, the software lead at the NASA Johnson Space Center’s Hybrid Reality Lab, discussed the use of VR and 3D-printed replicas for astronaut training.

“We don’t want to just teach the astronauts how to use the tools, but we want them to develop the muscle memory of actually using the tool,” said Noyes.

In consumer-class VR setups, the users tend to use joysticks or sensor-equipped handles that symbolically represent a real object, be it a sword, a laser gun, or a screwdriver. Though sufficient for entertainment, the same approach may be counterproductive in real training, as the experience is markedly differently from how the real device or tool feels in the user’s hand.

In NASA’s hybrid reality training setup, the virtual environment (for example, a realistically rendered, physically accurate interior of a spacecraft) is delivered to the trainee in an HTC Vive headset.

“A maintenance drill used on the Hubble station for repair costs about a million dollars to manufacture. But a 3D-printed facsimile can be created with about $20’s worth of plastic materials,” Noyes offered a comparison. “The 3D-printed tool is hollow inside, so we can [add artificial weight] to make it weigh as much as the real thing.”

The real drill weighs about 10 LBs on earth. But, to accurately represent how the tool would feel in space, the 3D-printed replica is made to weigh only 2 LBs. Thus, the trainees can use the inexpensive 3D-printed equipment to develop muscle memory for crucial repair and maintenance tasks abroad the space station.

The Volta Leap

This year’s GTC is the launch pad for NVIDIA’s next generation GPU, the Tesla Volta V100, representing a significant improvement in GPU architecture to the predecessor Pascal line.

“This is radical limit,” Huang said. “I mean, it is at the limit of photolithography. You can’t make a chip any bigger than this, because if you do, the transistors will fall on the ground. Every single transistor possible to make by today’s physics is crammed into this processor … The fact that this is manufacturable is just in incredible feat.”

The Tesla V100 comes with 5,120 CUDA cores, stacking up 21 billion transistors, capable of 7.5 FP64 TFLOPS or 15 FP32 TFLOPS.

“Volta is the next generation, with a brand new instruction set, a full compiler and optimizer for inferencing called TensorRT,” Huang pointed out. Inferencing is an important feature for deep learning or machine learning, part of AI projects.

Personal AI Trainer

Last year, NVIDIA announced the release of DGX 1, a specialized hardware product aimed at AI researchers and AI pioneers. This year, Huang introduced the DGX Station, a compact version of the same hardware, meant for personal use.

Pumping out 480 Tensor TFLOPS, the personal version is equipped with four Tesla V100 GPUs, supporting up to three display ports. “Putting that much power next to an engineer means we need to keep it cool, so we use liquid cooling. It’s whisper-quiet; you can’t hear it at all,” explains Huang. “You can order it now. We’ll deliver it in Q3.”

At $69,000, the DGX Station is not exactly impulse buy. But for AI researchers who benefit from GPU-accelerated number crunching to perfect their machine-learned algorithms, the DGX Station may be well worth the speed increase it offers.

With deep learning, machine learning, and AI workflows, no matter the hardware, some algorithms will likely demand more. Therefore, the option to burst into the cloud for faster results is a critical component of such projects. To capture this market, NVIDIA is releasing the HGX-1, described as “hyperscale GPU accelerator for AI cloud computing.”

“This particular box is meant for the public cloud, so that the versatility of the service can expand the reach of the customer base, whether you use it for deep learning, graphics, or CUDA HPC computation,” said Huang.


All these are but a prelude to the next revelation — the NVIDIA GPU Cloud (NGC), to be fully tested and maintained by NVIDIA.

Huang said, “You register, download a stack of software that’s fully optimized and fully containerized into your machine — you start doing deep learning basically in a few minutes. No configuration, no worrying about different versions.”

The web application of NGC running from the browser allows you to quickly create a deep learning job, attach a dataset to analyze, then submit it to an on-premise cluster (such as DGX-1) or to the resources of a cloud service provider.

The NGC includes datasets, pretained models, and containers. Beta release is scheduled for July.

A Deal with Toyota

In the last fear years, as automakers began developing autonomous vehicles, they became the biggest AI software and hardware consumers. NVIDIA’s GPUs, for examples, can be deployed to speed up the development of image-processing algorithms — a critical part of the self-driving cars’ pathfinding and navigation functions.

“Toyota has selected NVIDIA DRIVE PX for their autonomous vehicles,” Huang announced.

DRIVE PX is NVIDIA’s road map for the autonomous car industry, a platform that anticipates vehicles to evolve from human-driven, machine-assisted, and, ultimately, self-driven.

“We dedicated ourselves to go solve the self-driving car problem, create the software stack, then open it up,” Huang said. “We have now more than 200 developers around the world using DRIVE PX, from startups and shell companies to trucking companies.”

A Contribution to the Open Source World

Furthermore, NVIDIA Xavier is set to be the processor inside Toyota’s autonomous car, according to Huang. Xavier, a system on a chip (SoC), has integrated GPU architecture and a 8-core CPU. It’s is particularly suited for self-driving cars because its architecture includes DLAs, or deep-learning accelerators, Huan pointed out.

“We thought, Why don’t we democratize deep learning, accelerate adoption, and lower the barrier of entry? So today, we’re going to open source the Xavier DLA,” Huang announced.

NVIDIA’s core business is selling hardware; therefore, if the DLA adoption increases, the company’s hardware business that caters to AI developers stand to benefit from it. In that sense, the company’s decision to put its Xavier DLA into the hands of the open source community is not only a gesture of goodwill but also a sound business strategy.

The company’s expansion into AI has an impact on its hardware partners. Last week, BOXX, a hardware maker offering products with NVIDIA GPUs, acquired Cirrascale Corporation, a vendor who provides GPU-driven hardware and cloud solutions for AI workflows. After the acquisition, BOXX plans to manage Cirrascale’s hardware business, giving the company a bigger footprint in the AI market.

Isaac’s Brain for Your Robot

One of the advantages of the virtual world is, you can accelerate time. A real robot training to play hockey, for example, is still limited by the time constraint — how long it takes to run the robot until it develops the necessary skills to correctly shoot and score. But if the robot training were to take place in the virtual world, there’s no reason 20, 100, or 200 robots cannot be training simultaneously, while each is learning from the behavior of the best performer in the herd.

That is the idea behind NVIDIA’s robot simulator program ISAAC. The name is a tribute to Sir Isaac Newton (famous physicist) and Isaac Asimov (sci-fi writer and AI champion). Isaac can work with virtual sensors, can be connected to the reinforcement-based learning system OpenAI Gym.

“We would simulate [decision-making activities] in this environment, run it on NVIDIA GPUs, and inside that computer is a virtual brain,” said Huang. “When we’re done with it, we take that virtual brain and put this brain into a real robot. So this robot wakes up, almost as if it was born to know this world. Then the last little bit of domain adaptation can be done in the physical world.”

NVIDIA’s ISAAC is a virtual robot training environment to speed up AI-driven, reinforcement-based machine learning (image courtesy of NVIDIA).

Maintaining Computing Momentum

“In the course of the last 30 years, we’ve improved microprocessor improvement by nearly a million times,” Huang noted. “Then, in the last few years, it started to slow down. Our ability to harvest parallelism out of sequential instructions started to diminish. And the number of transistors we had to add in order to squeeze out the little tiny bit of extra performance was simply too costly … We’re now up against the laws of semiconductor physics.”

A better approach to maintain the computing momentum, he proposes, is to offload the parallel-processing workload from the CPU to the GPU, the domain of NVIDIA. “The algorithms of the artists, scientists, engineers, explorers, discoverers, inventors, the da Vincis of our time, the Einsteins of our times — their software includes some parallel processing aspects. If we could figure out a way to offload those [workloads] from the sequential processing microprocessor, we could achieve a speedup,” he notes.

Huang has good reasons to believe people are heeding his message. By NVIDIA’s own tallies, CUDA, the GPU computing and programing framework from NVIDIA, has been downloaded more than a million times; the number of GPU application developers have expanded 11 folds in the last five years, standing at 511,000 at the present; and the GTC attendees have tripled in the last five years.


If you enjoyed this post, make sure you subscribe to my RSS feed!

About Kenneth

Kenneth Wong has been a regular contributor to the CAD industry press since 2000, first an an editor, later as a columnist and freelance writer for various publications. During his nine-year tenure, he has closely followed the migration from 2D to 3D, the growth of PLM (product lifecycle management), and the impact of globalization on manufacturing. His writings have appeared in Cadalyst, Computer Graphics World, and Manufacturing Business Technology, among others.


Leave a Reply

Your email address will not be published. Required fields are marked *