BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group,...

13
BIT uses QML for Medical Diagnostic device UIs We switched entirely to SVGs Why and how it worked Presented by: Stanley Morris Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML for our touchscreen interfaces and C++ for control logic. 9 months ago we switched to using SVG images entirely in our QML UIs. I will discuss why and how we switched and why it works for us.

Transcript of BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group,...

Page 1: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• BIT uses QML for Medical Diagnostic device UIs

• We switched entirely to SVGs

• Why and how it worked

Presented by:

Stanley Morris

Senior UI Engineer

BIT Group, Inc. USA

Irvine, CA

BIT Group manufactures Medical Diagnostic devices. We use

QML for our touchscreen interfaces and C++ for control logic.

9 months ago we switched to using SVG images entirely in

our QML UIs. I will discuss why and how we switched and

why it works for us.

Page 2: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Primary utility of SVG is its scalability

• Product lifecycle for medical devices can number in

decades

• Replacing an end-of-life LCD panel can be difficult

• Simple dimensions framework is solution

• SVG images scale to target resolution

Our applications run in embedded systems that may have a

long product life cycle. I am working on a product that is the

second generation of a product that has been on the market

for over 20 years.

Sourcing LCD panels becomes a problem when a production

panel is end-of-life’d. I realized that this is a serious problem

and developed a simple framework to solve this problem.

SVG images are natural part of this solution.

Page 3: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Performance hit comes from CPU rendering

• Complexity increases render time

• Unsupported SVG features are not showstoppers

• Ability to scale to any resolution outweighs cons

The performance hit comes when the SVG is rendered to a

target resolution. Unlike a bitmap which is essentially fixed in

resolution, an SVG can be scaled but each element in the

SVG has to be converted into bitmap data. The more

complex the SVG, the longer it takes.

Page 4: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Small, simple SVGs animate well

• OpenGL virtually eliminates draw time compared to Qt4

• Tested performance to find limits

We never encountered performance issues in our UIs. A lot of this

has to do with timing; we switched to SVG images at the same time

that we switched to Qt5. Although the SVG is rendered using the

CPU, Qt5 uses GPU hardware when displaying images and provides

more bandwidth for the CPU to render SVG images.

SVG complexity increases the time needed to render the SVG. I

wrote a utility that re-rendered SVGs for each frame of animation,

automatically testing different sizes to find the point at which 60 fps

could no longer be maintained.

On our embedded platform, I can animate 11 SVG images

simultaneously as they are resized around 200x200 pixels. That is

actually a lot of work being done. The results suggest a linear nature;

that if my SVGs were twice as complex I could only animate while re-

rendering 5 images at a time.

Page 5: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Complex SVG images cannot be animated

• Next slide shows scales converted to bitmap

• Set complex SVG size once and do not resize; resizing causes

re-rendering

The simple image I used in the previous test was no problem for an

ARM i.MX 6 processor. I tried a far more complex image and found

that there was no way to animate it acceptably.

It is possibly to improve the dragon SVG image’s efficiency by

converting the layer with scales to a bitmap layer. The rest of the

image is left as scalable vector paths. As long as we do not zoom

into the dragon image, it looks acceptable.

If it is necessary to animate re-rendering a complex SVG image, set

the sourceSize to the final size and animate width and height. If

sourceSize does not change the SVG is not re-rendered and

animates as smoothly as any regular bitmap image.

Page 6: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Example of complex elements converted to bitmap

• Demo Inkscape conversion of a blur to bitmap• Create 32x32 px “check” icon SVG image with drop shadow

• Show that drop shadow looks bad in QML

• Use Inkscape’s Edit/Make a bitmap copy option on drop shadow shape and

delete the original shape

• Show that now looks good in QML even stretched to 600x600 pixels

For complex SVG images, strategically convert elements to bitmap

layers to reduce the number of shapes. Scaling bitmaps is efficient

and if there is lack of detail in the bitmap, like a drop shadow, the

SVG may be scaled up acceptably.

QML SVGs do not support SVG drop shadows. Convert drop

shadows to bitmap layers. This works because drop shadows have

no detail; they scale up 10-20x acceptably.

In the dragon image with bitmap layer scales, the scales look fine

until zoomed in. Render time is reduced by 40% with a 2000x2000

bitmap layer (88% for 360x360) compared to vector image scales but

is still too complex to animate rendering.

6

Page 7: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• sourceSize property controls rendering

• Binding sourceSize to width/height renders to correct

dimensions

• Rendering occurs during “Completion” phase of QML

element initialization, after properties have all been initialized

• Even though both width and height are specified, rendering

only happens once when QML component is loaded

The sourceSize property controls the rendering of the SVG. If

bound to the width and height as in this example, it will always

look excellent.

By monitoring the statusChanged signal, we can see when the

SVG is rendered by tracking when the status changes to

Image.Ready. If the sourceSize is bound to the width and height

as shown, it only renders once. If either the width or height

changes, it will re-render.

Page 8: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Specifying pixels breaks scalability

• Only use logical units like points or millimeters

• Time needed to convert one screen

Because we are creating a UI that adapts to changing display

requirements, we cannot use pixels to specify dimensions. If use

pixel values and later we change the display resolution, our

application will not scale correctly.

We had a situation where the display hardware was changed for us

unexpectedly. Almost all of the UI had been converted to SVG

images except the test sample tray. It took 3 hours to complete the

conversion once our graphics designer provided the SVG assets.

Other than that one page, the entire application scaled flawlessly and

once converted to use point units with SVGs, the revised page

scaled flawlessly too.

Page 9: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Pixel conversion component

• QML Singleton Object

• QML Singleton is tricky with 3 required parts: pragma, qmldir &

explicit import

• The one element of the interface that remains the same

dimension is the human finger. We target a 9mm touch area.

To specify dimensions, we use a QML singleton component. There

are three pieces needed for the singleton to work. The pragma

Singleton directive, the qmldir file identifying the singleton object and

the explicit import to load the qmldir file if in the same directory.

This example favors the imperial measurement system. The

corresponding metric version is:

property double mmDotPitch: .25

property double dpi: 25.4/mmDotPitch

property double mm: 1/mmDotPitch

Page 10: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Mix of points and inch dimensions

• Use font.pixelSize instead of font.pointSize

This shows a mix of dimension specifications and contents

generated. On my laptop, which has a 133 DPI display, I can

use a ruler and see exactly 4 x 4.5 inches of window area.

It is interesting that the font.pointSize is hardly ever correct.

We never use it anymore, instead setting font.pixelSize to a

point size using the framework. This produces correct point

sized text and can be verified using a point size chart.

Page 11: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• Work with actual screen size

• Text scaling capabilities

• Small app window targets 16DPI projector screen

The ability to change the DPI on-the-fly allows verification

that the UI is not using fixed pixel measurements and that it

scales appropriately.

This example uses a dropdown combobox to switch between

a variety of screen resolutions. At 13 DPI, the application

should correspond on the Dev Days presentation room

projector screen to the size that the customer will work with

and holding a ruler up to the screen verifies this. Obviously,

13 DPI has poor detail but the point is that this framework

lets us see the final target size even if we do not have the

final display device.

Page 12: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

• With proper use, SVGs can have good performance

• Only one image needed instead of multiple for known

target resolutions

• Adapts to any resolution, especially important for the

unknown resolutions that you may encounter

Page 13: BIT uses QML for Medical Diagnostic device UIs Presented ...€¦ · Senior UI Engineer BIT Group, Inc. USA Irvine, CA BIT Group manufactures Medical Diagnostic devices. We use QML

13