Overview / Usage

Linux users often still need access to other operating systems for applications that are not supported on Linux and usually resort to either Dual Booting their system or using a Virtual Machine.

Dual booting is a big inconvenience as it means closing down all running application, remote links, etc simply to run a single tool or utility. It also prevents easy interoperability between the target application and applications in use in the Linux environment. While a Virtual Machine can solve most of these issues, when the application needed requires GPU acceleration the options are limited.

Our solution is to install a secondary GPU into the PC which is then given to the VM for it's use through VFIO Pass-through, however the GPU is not designed for this and does not have any knowledge of being inside a Virtual Machine, as such will only output it's image to a physically attached monitor. Looking Glass address this shortcoming while also providing many more features for productivity, gaming and streaming making the Virtual Machine behave as if it's a native application inside of Linux with 100% software compatibility and near bare-metal CPU & GPU performance.

Looking Glass is a highly optimised application that makes use of shared memory to capture, copy and display the output of a pass-through GPU on the Linux desktop lossless and without compression. It also incorporates keyboard and mouse interaction with the virtual machine as well as audio support. Clipboard synchronisation is also implemented making it possible to simply copy & paste between the guest and the host. Special care has been taken to the mouse handling to ensure a seamless experience when interacting with the guest virtual machine via Looking Glass, making the guest feel like a native application instead of a clunky VM.

With the technology we have developed, under certain conditions it is possible to achieve negative latency. Looking Glass can often get the frame from the guest on screen faster then the physical GPU can display it on a physical monitor. As such this makes Looking Glass suited for tasks such as competitive gaming.

Using the memory sharing technology we invented (LGMP) it's possible to share the output with multiple applications with zero additional overheads. This is perfect for streaming/recording using our OBS plugin, eliminating the need for a capture and encode PC setup that streamers often employ.

Methodology / Approach

Looking Glass is written in C using C99 standards and includes four main components.

• Host Application
  Runs inside the virtual machine and uses several technologies depending on what is available to capture the output of the guest and copy it into shared memory for usage. Also synchronises information such as the guest cursor position, shape, and guest state, such as when applications want attention inside the guest.
• Client Application
  Runs either on the host or inside another virtual machine with access to the shared memory segment that the host application is writing to. This application is an enhanced remote viewer for the virtual machine allowing interaction along with audio support and clipboard synchronisation. It also supports post processing filters to enhance the virtual machine's output through technologies such as AMD FSR and CAS. Options are also provided for people who suffer with colourblindness granting them access to applications that do not have any support for such disabilities.
• KVMFR Kernel Module
  Runs in either the host or inside another virtual machine. This is a helper module that provides access to the IVSHMEM virtual device shared memory as either a user-space memory mapped block of RAM, or a Linux DMA-BUF for GPUs that support DMA memory transfers into textures.
• OBS Module
  A plugin for Open Broadcast Studio that is able to take the feed in shared memory directly into OBS. This can be done simultaneously while the Looking Glass client is running thanks to the shared memory management technology we have invented for this project. Doing this incurs no additional overheads.

Technologies Used

• GCC, MinGW, Git, CMake
• QEMU, KVM, VFIO, IVSHMEM
• DMA-BUF
• OpenGL, OpenGL ES
• DirectX Desktop Duplication
• NVidia Frame Buffer Capture (NvFBC)
• Intel VT-d, AMD SRV
• Intel GVT-g
• SR-IOV
• Pipewire, PulseAudio, X11, Wayland, XSHM
• SPICE (https://github.com/gnif/PureSpice)
• AMD FSR, AMD CAS

Repository

https://github.com/gnif/LookingGlass

Other links

• Looking Glass Discord
• Looking Glass