SmartHome

Research

A UI application for each device (HVAC, fridge, printer, dishwasher), a companion app running on mobile and a HUB application for centralized control will be the core digital elements of our domestic environment. 

Such a variety of home appliances presents Qt technology with a wide range of use cases and development tracks, both in terms of system requirements and concrete user-centric solutions. The latter should focus on the actual user's needs, including the possibility to customize the UI and to add extra functionality. 

Design vision

Real life and customer needs inspires our creation. Our demo encompasses: 

• 4 types of devices to control 
• 5 rooms to interact with
• 4 predefined daytime modes for an easy customization
• remote control via HUB and companion apps

Our design choices drew on glassmorphism and 3D graphics, elements that Qt advanced graphics capabilities and platform-agnostic technology renders uniformly and seamlessly across all applications and screen types. 

User flow

Understanding the customers' needs and their "journey" when accessing an application till the point their need is fulfilled is an essential task for application design. For each type of device, such understanding helps identify key features and functionality. At the same time, alternative user experiences are vetted for usability and smart look.

After doing this work we moved to Figma for the creation of wireframes, interactions and navigation flow. 

Project structure 

Creating a visual UI application requires a disciplined workflow. In Figma, we created the visual assets: from style libraries (defining e.g. colors, fonts, materials), to UI components (like buttons, icons, etc.). These will be used across the different UI projects to ensure both a uniform look and feel and the smallest development effort. Create once, deploy everywhere is our ethos.

Once a Figma project is properly setup, importing it to Qt Design  Studio for its conversion into a real application is a straightforward operation. 

Interaction

In Figma, each screen is composed into an interactive mockup enabling basic navigation through the intended functionality. This helps to ensure that the application logic, transitions and animations work as expected. 

The interaction is intended to model usability and at the same time to detect opportunities and breakpoints.

At this stage, we are only working on a Figma prototype that will become a real application only when imported into Qt Design Studio. 

Qt Design Bridges

Each component and screen in Figma was imported to Qt Design Studio via Qt Bridge for Figma. 

The import seamlessly preserves all the visual properties of the components (size, color, stroke, font, position, etc.) and those of their variants (e.g. the variants related to different states or themes).

Once imported, the assets were reviewed in Qt Design Studio for possible finetuning on position, gradient, etc.

Building interaction

After importing the graphics assets, we implemented the application's interaction and logic in Qt Design Studio. This defines the actual application behavior based on what we prototyped in Figma during the UX design.

Qt Design Studio offers visual editing tools enabling a no-code approach to UI creation. Application states, state transitions, data binding and all the application logic can be created without the need for the designer to write a single line of code.

Instead, Qt Design Studio converts automatically the UI application into QML code that can be accessed by developers for backend work, profiling, debugging and quality assurance.    

Creating reusable components

Application-specific components include visible elements, but also logic operations and functionalities that run in the backend. In Qt, cross-platform reusability applies to both visual and logical components alike.

An example is the 3D area: a component created to view and interact with a 3D object with its own specific behavior.

Another is the area for the user's interaction with the UI that defines the operational meaning of specific gestures, like tapping or swiping.

Qt allows turning these elements into reusable components that can be easily placed in other projects and other devices.

Bringing the UI to life

UI applications interact with the world and with other applications by sending and receiving data. Changes in the UI reflect changes in the state of things.

To modify the appearance and behavior of a component dynamically, elements of the UI are linked and synchronized with real-world data, such as date and time, temperature, state of other devices, etc.

This fundamental operation is called property binding and consists in binding the visual properties of a UI element to external data. Qt Design Studio has a powerful and reliable data-binding system that can be implemented visually or by accessing the QML code.

Qt Design Viewer

Complex software projects involve large teams and many stakeholders. They all need to see how things are progressing, how the design is implemented and how the functionality is executed. They all have a say.

What's better than sharing fully functional projects with all stakeholders over a browser? And we don't mean only the visual appearance, but the real application! And be able to interact in real-time with all its functionality.

No need to package a repository, to install libraries, to run commands or compile code. With Qt Design Viewer you can just share a link with your co-workers and ask for their feedback in a matter of seconds.

Optimizing 3D assets for real-time rendering

High performance on embedded systems is achieved through 3D model optimization. This consists in a series of operations:

• Clean the  model geometry
• Remove hidden meshes and unused data
• Use PBR materials for high-impact components
• Use single-color materials on less visible areas 
• Reduce the number of textures
• Ensure textures are appropriately sized based on visual impact.

With the exception of the model geometry optimization, all other operations can be performed in Qt Design Studio.

Importing 3D assets

As usual, we created 3D assets in a 3D content authoring tool like Blender or 3ds Max and then imported the scene to Qt Design Studio. Qt supports all major 3D file formats like .blend, .dae, .fbx, .glb, .gltf, .obj, .uia and .uip. 

In the SmartHome project, we used the glTF 2.0 format as it allows us to easily import the entire 3D scene description with lighting, materials and textures in place.

Physically based rendering

Materials and lights bring the 3D objects and scenes to life. Qt Design Studio offers full support to physically based rendering (PBR) for seamless compatibility with standard material models. Adherence to the PBR standard substantially simplifies the task of importing 3D assets in that it ensures that the model will look in Qt Design Studio exactly the same as in the content creation tool, without the need to create custom shaders for different surface types.

PBR also enables access to thousands of 3D models available online that can be used and customized according to the UI application's needs.

Digital twins

A 3D digital twin consists of different parts and our interaction should trigger movement and changes in their appearance.  Mobile 3D parts, such as the door of a fridge, the cartridge of a printer, the light in a room should change when we interact with the UI. 

In Qt Design Studio, such changes are implemented via states and transitions between them. Each action on the UI triggers a new state and the transition function to reach it displaying the change in a fluid, non-disrupting way. Qt Design Studio offers a wide choice of easing functions for transitions (linear, ease in, quadratic, etc.) and the possibility to customize them at pleasure.

Cinematic camera movements

Camera movements bring a cinematic touch to the scene. The camera movement shapes the audience's perception of the action, controls how the narrative unfolds and influences the scene's stylistic tone.

As in the case of interaction with 3D objects, camera movements can be easily created in Qt Design Studio by defining an initial and a final position of the camera and by using a transition style to move the camera between them. 

Connectivity

Connectivity enables data synchronization and exchange of notifications across devices.

We used Qt framework's MQTT and Bluetooth modules to provide seamless communication across systems. 

The various appliances use Bluetooth and WIFI technology to communicate with each other.

Plug-and-play

The system architecture is designed to automatically detect new appliances in the proximity of the HUB and easily integrate them into the SmartHome infrastructure.

Once the new device is connected, a new dedicated widget appears in the HUB and is ready for use. The widget inherits all the functionality specific to the new device and enables remote interaction with all the specific appliance's functions.  

Cross-platform by design

Qt platform-agnostic setup allows running Qt applications on any hardware and software configuration. Create once, deploy everywhere is our core principle. The advantages are many-fold:

• Save time & cost, by reusing same assets on different UIs
• Ensure consistent visuals, aligned to your brand
• Reduce hardware and software dependency—essential in times of chip shortage
• Scale your apps across the full range of hardware, from MCUs, to MPUs, desktop, mobile, etc.

Performance optimization

Converting a proof of concept into a consumer application that runs smoothly on the target hardware to ensure a compelling user experience requires thorough work toward the identification of bottlenecks in performance, lags, memory leaks, etc.

Qt profiling tools help identify possible issues in resource usage, paving the way for performance optimization.

In our SmartHome project, for instance,

• we resorted to SVG images, over the heavier PNG format
• we re-used the same graphics assets and logical components whenever possible—as opposed to re-creating them when needed

The HUB

The HUB is the central control unit of our SmartHome.

Running on a tablet or other high-end hardware, the HUB enables seamless interaction with each connected device via widgets mirroring the interface of the target appliances. The widgets can be rearranged and resized within the main HUB view for better responding to the user's needs. 

All target devices communicate with the HUB via Bluetooth or WIFI.

Companion app

The companion app takes all the HUB functionality, and more, to a smartphone. It enables remote interaction with each device even from outside the house, as well as receiving notifications or information related to the appliances' status, shopping list, scheduled tasks, etc.

While currently interaction with the devices uses touch, voice commands can be enabled by integrating an AI assistant.

Communication relies on Bluetooth or WIFI technology.

High- & mid-end devices

In our SmartHome, the fridge uses high-end micro-processing units to power rich 3D graphics and advanced functionality.

In addition to the standard functions, the smart fridge's display tracks grocery storage and creates a shopping list based on the supply needs.

The heating, ventilation and air conditioning system (HVAC) uses a mid-range device. This is enough to control the temperature of each room and offers various pre-set configurations to automatically adjust the climate. 

The HVAC and fridge interface with the HUB and companion app via Bluetooth and WIFI.

Low-end devices

Printer and dishwasher use low-end touch-screen devices and Qt for MCUs for its lightweight performance. These devices support WIFI connectivity and, except for real-time 3D graphics, are built out of the same assets used in the other devices to ensure a consistent look-and-feel across the line.

On top of standard functions, like printing and scanning, the printer allows users to check the ink level, change the settings and more.

The dishwasher has a selection of predefined programs and the possibility to regulate its operation based on load, schedule and duration.