Remote UIs with WebGL and WebAssembly

A frequently requested feature by Qt customers is the possibility to access, view and use a Qt-made UI remotely.

However, in contrast to web applications, Qt applications do not offer remote access by nature as communication with the backend usually happens via direct functions call and not over socket-based protocols like HTTP or WebSockets.

But the good thing is, with right system architecture with strong decoupling of frontend and backend and using the functionality of the Qt framework, it is possible to achieve that!

If you want the embedded performance of a Qt application and together with zero installation remote access for your solution, you might consider the following bits of advice and technologies.

Remote access via WebGL Streaming or VNC

When having a headless device or an embedded device with a simple QML-made UI that only needs to be accessed by a small number of users remotely via web browser, WebGL streaming is the right thing for you. In WebGL streaming, the GL commands to render the UI are serialized and sent from the web server to the web browser. The web browser will interpret and render the commands. Here's how Bosch did it:

[embed]https://youtu.be/Ik_nXZv86lQ[/embed]

On a headless device, you can simply start your application with these command line arguments: -platform webgl.

This enables the WebGL streaming platform plugin. WebGL data is accessible as the app runs locally.

For widget-based applications, you might consider the VNC based functionality, but this requires more bandwidth since the UI is rendered into pixel buffers which are sent over the network.

The drawback of both of the platform plugin approach of VNC and WebGL is, that within one process, you can start your application only either remotely or locally.

If your device has a touchscreen and you still want to have remote access, you need to run at least two processes: One for local and one for remote UI.

The data between both processes is shared via the Qt RemoteObjects library.

Use cases examples are remote training or remote maintenance for technicians in an industrial scenario, where from the browser you can show and control remotely mouse pointer on an embedded HMI device.

Have a look at the previous blog posts:

http://www.qt.io/blog/2017/02/22/qt-quick-webgl-streaming/

http://www.qt.io/blog/2017/07/07/qt-webgl-streaming-merged/

http://www.qt.io/blog/2017/11/14/qt-webgl-cinematic-experience/

WebAssembly

WebGL streaming and VNC are rather suited for a limited number of users accessing the UI at the same time.

For example, you might have an application that needs to be accessed by a large number of users simultaneously and does not require to be installed. This could be the case when the application is running as Software-as-a-Service (SaaS) in the cloud. Fortunately, there is another technology that might suit your needs: Qt for WebAssembly.

While WebAssembly itself is not a Remote UI technology like WebGL or VNC, it is an open bytecode format for the web and is standardized by W3C.

Here's a short video of our sensortag demo running on WebAssembly:

[embed]https://youtu.be/OJ1ieyuIQzE[/embed]

With Qt for WebAssembly, we are able to cross-compile Qt applications into the WebAssembly bytecode. The generated WASM files can be served from any web server and run in any modern web browser. For remote access and distributed applications, a separate data channel needs to be opened to the device. Here it needs to considered that a WebAssembly application runs in a sandbox. Thus, the only way to communicate with the web server is via HTTP requests or web sockets.

However, this means that in terms of web server communication, Qt applications then behave exactly like web applications!
Of course, you could still compile and deploy the application directly into another platform-specific format with Qt and use Qt Remote Objects for client-server communication. But only with Qt for WebAssembly, the zero-installation feature and sandboxing come for free ;)

Exemplary use case scenarios are SaaS applications deployed in the cloud and running in the browser, multi-terminal UIs, UIs for gateways or headless devices without installation.

Have a look at our previous blog posts on Qt for WebAssembly:
https://www.qt.io/blog/2018/04/23/beta-qt-webassembly-technology-preview/
https://www.qt.io/blog/2018/05/22/qt-for-webassembly/

That's it for today! Please check back soon for our final installment of our automation mini-blog-series, where we will look at cloud integration. In the meantime, have a look at our website for more information, or at Lars' blog post for an overview of our blog series on Qt for Automation 2.0.