Stereo Style Transfer AI

See the world as a 3D painting ...learn more

Project status: Under Development

Virtual Reality, Artificial Intelligence

Intel Technologies
Intel Opt ML/DL Framework

Overview / Usage

What if Virtual Reality glasses could transform your environment into a three-dimensional work of art in realtime? What if every detail of the world would be shown in the style of a painting from Van Gogh? One of the many interesting developments in the field of Deep Learning is the so called "Style Transfer". It describes a possibility to create a patchwork (or pastiche) from two images. While one of these images defines the the artistic style of the result picture, the other one is used for extracting the image content. The Hardware Hacking Team from TNG Technology Consulting managed to build an application using OpenCV and Tensorflow to realize such goggles. When you see the world through these glasses, your environment will be displayed in the styles of famous painters like Claude Monet or Pablo Picasso.

Methodology / Approach

The world is recorded by two cameras which are integrated into VR glasses. The recording laptop is then sending the images from those cameras to a powerful workstation computer. There the images are distortion corrected using OpenCV and then stylized using a convolutional neural net and Tensorflow. The stylized content is streamed back to the recording laptop, where it is shown in the Windows Mixed Reality VR goggles. This can all be done in real time with around 100ms delay.

Training of the neural net is done up-front in the cloud using a style image. The neural net is then trained to stylize arbitrary images.

Technologies Used

Windows Mixed Reality,
Tensorflow,
OpenCV,
Gstreamer,
OpenGL

Collaborators

There are no people to show.

Comments (0)