Nvidia Reveal New AI Algorithm that can Turn 2D Images into 3D

While there are plenty of current applications for turning 3D objects into a 2D perspective, there is very little that can successfully reverse engineer that process. Put simply, if you want an object to be 3D, you have to render it in 3D. Once the hard work of that is done, turning it into a 2D image is a pretty straight forward process. Well, probably not that straight forward, but you get the idea.

Nvidia, however, has just announced that a brand new AI being developed has successfully been able to recreate various 3D images from a singular static 2D picture.

Using pictures of birds, the AI was able to successfully replicate images from various angles. More than that, however, it was also able, to an impressive degree, to recreate the various textures.

What Does Nvidia Have to Say?

Now, the science behind this is amazingly complicated. We’re talking about mathematic symbols and equations of which I have literally zero understanding. I’ll, therefore, allow Nvidia themselves to tell you about it.

“In traditional computer graphics, a pipeline renders a 3D model to a 2D screen. But there’s information to be gained from doing the opposite. A model that could infer a 3D object from a 2D image would be able to perform better object tracking, for example.

NVIDIA researchers wanted to build an architecture that could do this while integrating seamlessly with machine learning techniques. The result, DIB-R, produces high-fidelity rendering by using an encoder-decoder architecture. A type of neural network that transforms input into a feature map or vector that is used to predict specific information. Such as shape, color, texture and lighting of an image.

It’s especially useful when it comes to fields like robotics. For an autonomous robot to interact safely and efficiently with its environment, it must be able to sense and understand its surroundings. DIB-R could potentially improve those depth perception capabilities.”

What Do We Think?

The main purpose of this new DIB-R (differentiable interpolation-based renderer) technology means that a process that could formerly take AI algorithms weeks to be ‘trained’ to do can now essentially learn ‘depth-perception’ on any object within milliseconds.

The bottom line is that while this might sound a little dry, it could offer some amazing potential. Some kudos should definitely go out to Nvidia!

For more information, you can check out the official Nvidia blog via the link here!

What do you think? Do you think this is impressive? What applications can you see it having? – Let us know in the comments!