Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day,...

65
Hands-on Workshop: Industrial HMI Using i.MX25 and QNX® Smart Energy Reference Software for Home and Building Control Sheridan Ether Manager, Application Engineering Team QNX Software Systems Month, Day, Year

Transcript of Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day,...

Page 1: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Hands-on Workshop:

Industrial HMI Using i.MX25 and QNX® Smart Energy

Reference Software for Home and Building Control

Sheridan Ether

Manager, Application Engineering Team

QNX Software Systems

Month, Day, Year

Page 2: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

2 QNX. All content copyright QNX Software Systems.

Agenda

QNX Overview

I.MX25 Family Overview

QNX Aviage HMI Architecture Overview

Adobe Flash as a next-gen embedded HMI tool

Integrating the HMI to real-time components

ActionScript extensions

Persistent Publish-Subscribe

Hands-on

Smart Metering Reference Application

Actionscript code walkthrough and interfacing to native OS services

Exploring PPS objects and behaviours

OS control of peripheral devices

Page 3: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

3 All content copyright QNX Software Systems.

QNX Core Product Offerings

ServicesRuntime Product Portfolio

Aviage Middleware Portfolio

Development and productivity tools

QNX Neutrino RTOS (Micro Kernel)

QNX Photon microGUI

Instrumented kernel

File systems

Multicore

Adaptive partitioning

High availability

Connectivity and networking

Integrated Java

QNX Momentics IDE

Multicore visualization

Performance optimization

Memory and system analysis

Performance and footprint optimization

Photon application builder

Target information & remote debug

Libraries and GNU tools

Support

Custom engineering

Professional Services

Training

QNX Aviage multimedia solution

QNX Aviage HMI suite (including Flash Lite 3)

QNX Aviage acoustic processing

Voyager web browser

Page 4: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

4 All content copyright QNX Software Systems.

Advanced Runtime Module

QNX Runtime Product Portfolio

Multimedia

Suite

QNX Neutrino

RTOS

QNX Neutrino

Extensions

QNX Aviage

Middleware

Instant

Device

Activation

Extended

Networking

(security)

Multi-coreAdaptive

Partitioning

Web

Browser

Java

Acoustic

Processing

HMI

Suite

QNX Neutrino RTOS Runtime

Core

File Systems

Peripheral

Hardware

Core

Networking

Core

OS

TDP/QNETHigh

Availability

2D/3D

GraphicsPhoton

Page 5: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

5 All content copyright QNX Software Systems.

QNX Standards and Certifications

Page 6: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

QNX Microkernel: Designed for Reliability, Designed for Embedded

Graphics

Networking

Drivers

Kernel

File System

Monolithic KernelWinCE 3.9 million lines of code

Linux: 5.76 million lines of code

XP: 40 million lines of code

Microkernel

Memory Protection

Message

passing

Micro

kernel

File System

Networking

Driver

Driver

Graphics

QNX 0.1 million lines of code

Designed for reliable,

embedded devices

Page 7: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

i.MX25 Family Overview

Sujata Neidig

i.MX Product Manager

Page 8: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Freescale i.MX25 Focus Markets

Low-Power

High Integration

Advanced Performance

Platform Software

Portable Consumer

► Portable Media Player

► MID/Netbook

► Smartphone

► E-book

► Personal Navigation

Automotive Infotainment

► Audio

► Connectivity and Telematics

► Video and Navigation

Home Consumer

► Media Phone/Terminal

► iPod accessories

► Remote controls

► Digital Photo Frame

► Appliances

Industrial

► Point of Sale

► Security and Surveillance

► Industrial HMI

► Barcode Scanners

► Printers

► Medical

► Meteringi.MX25 Target Markets

Page 9: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

i.MX25x Series

Key Features and Advantages

400MHz ARM926EJ-S™

16KB L1 I-Cache, 16KB L1 D-Cache

128KB on-chip SRAM for low power LCD refresh

External memory interface supports DDR2, mDDR, or SDRAM up to 133MHz, 16-bit data bus

Supports off-chip NAND or NOR Flash

10/100 Ethernet MAC with RMII support

USB 2.0 OTG 480Mbps with high-speed PHY

USB 2.0 Host 480Mbps with full-speed PHY or ULPI

Two CAN interfaces

Two Smartcard interfaces

SDIO interface for external Wi-Fi module

VGA (640x480) LCD controller

Resistive touchscreen controller

CMOS sensor interface

P-ATA for external CD connection or HDD

CE-ATA and SD/MMC+ for external storage

Enhanced serial audio interface

3 general purpose 12-bit ADC channels

UART’s, CSPI’s, I2C, I2S

Enhanced security features, including tamper detection for voltage, frequency and temperature

High-Assurance Boot (HAB)

3.3V I/O reduces external component count

Available Parts

i.MX251, i.MX255, i.MX253, i.MX257, i.MX258

Package and Temperature

0.8mm, 400-pin MAPBGA

-20C to +70C, -40C to +85C temperature options

AEC-Q100 Grade3 Qualification

Audio

ESAI

SSI/I2S x 2

Security

SCC RNGB

HAB SRTC

RTICv3 Dry-Ice

Standard

System

Timer x 4

PWM x 4

Watch Dog

SDMA

Advanced Connectivity

10/100 Ethernet HS OTG + Phy

CAN x 2 HS Host + Phy

SDIO x 2 Smartcard x 2

Ext Memory I/F

NAND

NOR

DDR2

mDDR

SDRAM

User I/F

LCD Controller

Touchscreen

Controller

CMOS Sensor I/F

8x8 Keypad

SLCD Controller

Ext Storage

MMC+/SD x 2

P-ATA

CE-ATA x 2

Int Memory

128KB SRAM

32KB ROM

Standard

Connectivity

CSPI x 3

UART x 5

I2C x 3

GPIO x 4

12-bit ADC x 3

1-Wire

i.MX25x

ARM926EJ-S

400MHz

16K I

Cache

16K D

Cache

System Debug

ETM SJTAG

Not available

on all variants

Common IP with

i.MX35x

Page 10: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

i.MX25 Family: 3-Digit Part Numbering

Target Markets Automotive Automotive Industrial & Consumer

Industrial & Consumer Industrial

Example Applications

• Audio Connectivity• Secure Black Box• Smart Toll Road

Systems

• Infotainment• Secure Black Box• Smart Toll Road

Systems

• HMI• Printers• Medical• Factory Automation

• “Smart Touch” HMI• Factory Automation• Barcode Scanners

• Point-of-sale• Biometrics• Smart Meters• Secure Devices

Key Differences

• No LCD Controller• No Touchscreen

Controller• No CE-ATA/P-ATA

Full-featured automotive device

with security

• No Camera Interface

• No Touchscreen Controller

• No Smartcard• No CAN

Standard Part• No Security

Full-featured device with security

10Ku Re-sale (2009) Automotive – $8.44 Automotive – $9.37

Consumer – $6.96 Consumer – $7.78Industrial – $9.01

Industrial – $7.21 Industrial – $8.19

Qualification Tier Automotive AECQ100-G3

Automotive AECQ100-G3 Industrial, Consumer Industrial, Consumer Industrial

Package 17x17 MAPBGA 0.8mm

17x17 MAPBGA 0.8mm

17x17 MAPBGA 0.8mm

17x17 MAPBGA 0.8mm

17x17 MAPBGA 0.8mm

Temperature -40C to +85C -40C to +85C -20C to +70C-40C to +85C

-20C to +70C-40C to +85C -40C to +85C

Page 11: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Freescale i.MX25 PDK

For more information please visit www.freescale.com/imx25pdk

Page 12: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

i.MX25 System Cost Advantage Examples

12

Memory interface supports low cost DDR2

Multiple boot options, including boot from NAND and SPI flash

2 USB ports – no need for external hub

Ethernet 10/100 MAC and USB PHYs are integrated – eliminates external components

Integrated display and touchscreen controller’s

3.3V I/O – direct connectivity to peripherals

0.8mm pitch package and optimized pinout reduces PCB layer count

Page 13: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

QNX HMI Technology Overview

Page 14: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

The “iPhone Factor”

The iPhone has been accepted as an easy to use interface device for

many functions, and is driving consumer expectations in other areas,

including Home Automation

Page 15: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Why Design your HMI with Flash Lite?

Industry Standard Tools

Create Rich Graphics and Animations

Designed for Embedded Devices

Large Community of Flash Developers

Adobe FlashCS3/CS4

Adobe Illustrator

Adobe Photoshop

Page 16: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

16 All content copyright QNX Software Systems.16

QNX HMI Technology

Flash

Adobe Flash

OS interfaces

QNX HMI Technology Architecture

Vertical Stacks

& Demos

Core Applications

WebKit

engine

HTML

App

storeBrowser

Media

player

Widget

Examples

Graphics Engine

OpenVG

Screen Management

Native and EGL interface

HW Accelerated Screen TransitionsHW & Virtual Layer Management

Operating SystemQNX RTOS

Video

Render

Video

OpenGL

Application

3D native

OpenVG

Application

2D native

Composition Manager

HW Acceleration

Application framework (ActionScript 3 based)

Flash Services and widgets

Scroll List Buttons Sliders Soft keyboard NavBar ...Transitions Themes

Native OS Services

Media Browser SQL Remote ...Graphics LayersPPS

Application launcher

Loading Settings State ...Resize

Input

Touch, keyboard, mouse

QNX CARIndustrial

SMA Demo

Industrial

White Goods DemoMedical Demo

OpenGL ES 1.1 & 2.0

Docs

Core Documentation

Generic Demo

Page 17: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

1717 All content copyright QNX Software Systems.

QNX HMI Technology Components: Flash

Flash Lite 4

Hardware accelerated OpenVG (on supported platforms)

ActionScript3 support

Debugging capabilities

X-Ray

AS3 compatible native extensions

Webkit Browser

Multimedia – video, audio playback

Publish Subscribe – data distribution services

SQLite – database access

etc

Native video decode (for supported formats / platforms)

Complete HMI framework provided by QNX

Apps, widgets, services, lifecycle management

internet radio

Page 18: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

18 All content copyright QNX Software Systems.

QNX HMI Technology Components: Webkit and Multimedia

Webkit

HTML5 support

Includes new additions to HTML language for video, audio, geolocation, etc

Uses Skia standard for 2D drawing

Current Webkit version 525; updating to version 533

Improved components for user interaction

High performance panning and zooming

HTML rendering engine; browser app provided in Flash

Multimedia

Based on Aviage 1.2.1

Providing audio/video playback without needing full framework

New hardware encode/decode support for new targets

Streaming or file-based video playback

Page 19: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Combine and overlay graphics from different render engines

Flash acts as a “Window Manager”

Based on standards

Action Script, OpenGL, OpenVG, OpenKode, HTML, JavaScript, POSIX

QNX HMI Technology Components: HMI Composition

Composition

Flash

HTML

OpenVG

OpenGLVideo

Page 20: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Integration of Flash HMI with the Rest of the System

Page 21: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Designing Next Generation HMI’s with

Adobe Flash

Page 22: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

22 All content copyright QNX Software Systems.

Adobe Flash in Embedded

1 billion mobile devices in 2009

400+ device models

Page 23: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

23 All content copyright QNX Software Systems.

Why Adobe Flash?

Flash is the de facto HMI standard

Flash player installed on*:

98% of US Web users

99.3% of all Internet desktop users

Over one million graphics designers worldwide

Flash realizes the Java promise of Write Once, Run Anywhere

Class library dependency problems do not exist as in Java

Content developed for web or PC can run on embedded without change

Embedded FlashLite player uses less memory and provides faster rendering

Flash is ideal for HMI creation

Graphical environment by nature

Used by industrial designers to design look & feel

Ability to create “executable specifications”

*Numbers are estimates from Adobe, and depend on demographical & statistical info

Page 24: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

24 All content copyright QNX Software Systems.

Flash Lite in Embedded

Adobe Flash Lite 3

Highly optimized implementation of the Flash runtime

Specifically for embedded market

Requirements in line with embedded

32-bit data bus, 200MHz

380KB core player DLL size (without video codecs)

RAM requirements:

For user interface: 4–6MB

For standalone content: 2–4MB (excludes video)

For web browsing: 32MB

Advantages

Web/desktop Flash content to mobile and back again, with minimal effort

Vector graphics adjusts to different screen sizes without loss of quality

Vector graphics creates very small content files, compared to bitmaps

Page 25: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

25 All content copyright QNX Software Systems.

From Design to Deployment, no Detours

Flash HMI Development

1) Designer creates HMI in Adobe Flash on PC

2) .swf file from design process runs directly on the target system

3) Done.

Standard HMI Development1) Designer creates HMI in Adobe Flash on PC

2) .swf file from design process run by software engineers on PC as mock-up of system

3) Engineers attempt to duplicate behavior by recreating screens and graphics

a) Design all state logic from scratch

b) Use C/C++ code or another toolkit

c) Graphical assets not directly reusable

4) Engineers debug and get C/C++ code running on target

5) Design difficulties encountered in HMI are sent back to designers for rework with real hardware constraints

6) Engineers make adjustments to software & graphics for new design

7) Pass running system by design for verification

8) Errors in HMI duplication reworked and recoded

9) Repeat steps 4 – 8 as necessary

10) Done.

Page 26: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Integrating the HMI to Real-time Components

ActionScript Extensions

Page 27: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

The Problem

How do you make the Flash player extensible?

How do you interface the Flash runtime environment with native OS services?

How do you handle computationally expensive operations?

Page 28: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

28 All content copyright QNX Software Systems.

Flash to Native Gateway

QNX Aviage HMI Suite can implement ActionScript classes in C or C++

Interface defined in ActionScript

Implementation created in native code (C/C++)

Shared object (.so) containing C/C++ extensions loaded at player startup

Calls into C/C++ execute by…

…blocking Flash VM until finished OR

…spawning threads and using notification for callbacks

/**

* ActionScript access to system level functions

*/

intrinsic class com.qnx.extensions.example.SysUtil

{

public function SysUtil(); // constructor

function getCPUUtilization():Array; // returns [ core1%, core2%, ...]

function getMemoryUsage():Object; // returns {current:<#>, total:<#>}

function getDiskUtilization():Object; // returns {reads:<%>, writes: <%>}

function getNetworkUtilization():Object; // returns {tx:<%>, rx:<%>}

}

#include <flqnx.h>

typedef struct {

const char *class_name;

int method_count;

const char **method_names;

flqnx_as_method_type_t *method_types;

flqnx_as_method_handler_t method_handler;

int flags;

FLQNX_CLASS_CONTEXT_T context_data;

} flqnx_as_class_t;

Page 29: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

29 All content copyright QNX Software Systems.

Considerations Going from ActionScript to Native

What should move from ActionScript into native code (C/C++)?

Time consuming or computationally intensive operations

C/C++ is faster

Lengthy or blocking operations

Flash VM is single-threaded

Items requiring deterministic behavior

Flash can’t guarantee response time

Interfaces to existing native software blocks

No need to re-write code

Visible components: Reports/charts, browser, video playback, maps, 3D visualization, etc.

Libraries: cryptography, multimedia, acoustic processing, parsing, sockets, protocols

Page 30: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Integrating the HMI to Real-time Components

Persistent Publish-Subscribe

Page 31: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

The Problem

How do you easily inform multiple applications that common data has been updated?

How do you create modular software that allows components to be easily tested independently?

How do you integrate applications written in different languages?

Page 32: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

32 All content copyright QNX Software Systems.

Persistent Publish Subscribe (PPS)

Subscriber and Publisher

Can be unknown from each other at compile time

Can be dynamically substituted at runtime

Can have one-to-one, one-to-many, many-to-one relationships

Do not use direct calling API; available to all languages, scripts, shells

Can be in different languages, virtual machines, sandboxes

Are asynchronous from each other

Publisher

Can be configured how to persist data

Can support on-demand publishing (pull rather than push)

Persists attribute data over reboots

Subscriber

Are notified when data changes

Do not poll (block until delta update received)

Can receive notification on changes within a publishing tree

Heat sensor

pll

EtherCAN

/fs/pps/heat, ethercan, pll

Temperature

MonitorMotor

Page 33: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

33 All content copyright QNX Software Systems.

PPS Benefits

Attribute Feature Benefit

Applications loosely coupled to services

Object-based data publication

Asynchronous data push/pull

No polling required; unblocking notification interface

POSIX file namespace

Less brittle connection

Modifications to client or server won’t break interface

Implementation isolation (client doesn’t need to know who publisher is)

Language independent interface

Multi-view publishing Multiple objects can give differing views on same data source

Can have more than one publisher update different attributes of the same object

Easier for client’s most appropriate use

Can publish derived data

Can extend system and retain logical coherence

Client can get delta updates

Persistent state save Save to media of choice (NAND flash, NOR flash, Harddisk, USB, etc.)

Save on shutdown and system controlled points

Save attributes across system reboot

Retain object state for intermittent clients

Page 34: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Hands-On:

Exploring Smart Energy Reference on the i.MX25

Page 35: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Hands-On Agenda

Use case overview

Smart energy reference design

Examining the components

HMI

Drivers

HMI to Service Integration

Methods Available

Example Deep-dive

Summary

Page 36: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Use Case OverviewSmart Energy Reference Design

Page 37: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Smart Energy Reference Design – Menu Tree

Home Screen Weather Info

Lighting Control Video Surveillance

Page 38: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Smart Energy Reference Design – More Detail

Energy Usage Detailed Monthly Energy Usage

Energy Management Editing Energy Rules

Page 39: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Hardware Configuration

Network

Powerline Communications

InsteonServer

DimmerControllers

FreescaleIP Camera

Zigbee Gateway

Utility Simulator

Zigbee SensorsTemperature & Humidity

Smart Meter

Load Switches

Weather, Flickr, Pandora, etc.

Page 40: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Smart Energy – Software Architecture

Flash Lite 3 Player

Flash(Action Script)

Drivers & Services(C/C++)

PPSExtQDBExt

SQLite

ASE

energy

usage

database

power

monitor/

scheduler

insteon

manager

media

player

hvac

wrapper

io-media

envsensor

motorhvac

TCP/IP

MPEG4

TCP/IP

PPS Components

Resource Managers

Insteon

server

Page 41: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Examining the Components HMI

Page 42: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Adobe CS Development Environment

Both scripted (code) and non-scripted (timeline) behavior

Many pre-scripted actions (effects, transitions, tweens, etc.)

Graphical resources are primarily vector graphics (scalable)

Can include videos (.flv)

Very well suited for Internet or connected development

remote connection, datasets

web services

wsdl, xml, urls, etc.

Page 43: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Programmatic

Create graphical assets in CS3/4/5

AS3 resides in separate source files

Use Flash Develop to bind “resource-only” swfs with AS3 code for final swf

Two Methods for Flash HMI Development Workflow

Graphical/Timeline

Create graphics and ActionScript mixed together in traditional “Flash” timeline

Final .swf output directly from CS3/4/5

Page 44: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Flash HMI Source Layout

MVC

Page 45: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Flash HMI Source Layout

Framework Binding & Resources

Page 46: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Example ActionScript

Page 47: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Examining the ComponentsDrivers

Page 48: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

HVAC System

hvac

DescriptionController process for each zone

InputsResMgr: EnvSensor for temp, humidity, light

PPS: Receive temp settings

OutputsResMgr: Controls motor (furnace, a/c & fan)

PPS: Publish temp & humidity data

hvac-sep-wrapper

DescriptionSingle control for access to all zones

InputsPPS: set zone temp

Output: PPS: current zone temp

motor

DescriptionMotor proxy (dummy)

InputsResMgr: Accept motor controls

EnvSensor

Description

Zigbee sensor gateway

Inputs

TCP/IP: read sensors through gateway

Outputs

ResMgr: sensor data

hvac

wrapper

envsensor

motorhvac

TCP/IP

Page 49: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Power Control

insteon-mgr

Description

Controls electrical on/off for lighting, water heater, pool pump

Inputs

PPS: Receive control commands

Outputs

Send on/off to devices through server

power monitor

Description

Periodic recording of power consumption

Handle scheduled power events

Receive power grid messages & pricing

Inputs

PPS: functions for open, play, stop, etc.

Outputs

SQL: Record power usage data

SQLite

power

monitorinsteon-mgr

TCP/IP

Insteon

server

Page 50: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Video Camera System

pps-media-player

Description: Plays video camera stream

Inputs

PPS: Turn on/off video source

Outputs

None

io-media

Description

Creates graph for streaming read and MPEG4 video decoding

Inputs

C API: functions for open, play, stop, etc.

TCP/IP: Camera stream from URL

Outputs

Video data

pps-media-

player

io-media

MPEG4

Page 51: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

HMI to Service IntegrationMethods available

Page 52: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

PPS Objects at the Command-Line

# ls /fs/pps

.notify hvac mediaplayer system

doorBell_back insteon network

doorBell_front launcher power_monitor

# cd power_monitor

# pwd

/fs/pps/power_monitor

# ls

constants poolpump_schedule

hvac_schedule utility

info waterheater_schedule

# cat utility

@utility

grid_state::low

kwh_price::0.08

simulation_speed::0

user_message::Potential Critical Peak Level - Aug 24, 2011

#echo “khw_price::0.09”>>/fs/pps/power_monitor/utility

Heat sensor

pll

EtherCAN

/fs/pps/heat, ethercan, pll

Temperature

MonitorMotor

Page 53: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

PPS Subscriber in C

int setLightDimmer(const char *floor, int light_num, int percent)

{

const char *level_str=“Level::”; // Name of dimmer level attribute

char buffer[128];

int fd;

int currentlevel = -1;

// dimmer_up1, dimmer_main2, etc.

sprintf(“/fs/pps/insteon/dimmer_%s%d”, floor, light_num);

fd = open(buffer, O_RDONLY);

if (fd < 0) return -1; // Bad floor or light number

// Better to get notified through /fs/pps/insteon/.notify or by getting

// delta changes through /fs/pps/insteon/dimmer_*?delta

while (read(fd, &buffer, sizeof(buffer)) > 0 && currentlevel==-1) {

if (strncmp(buffer, level, strlen(level_str))==0)

currentlevel = atoi(buffer+strlen(level_str));

}

sprintf(buffer, “%s%d\n”, level_str, (255*percent)/100);

write(fd, buffer);

close(fd);

return (currentlevel*100)/255;

}

Page 54: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

HMI to Service IntegrationExample Deep-dive

Page 55: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Getting Temperature

PPSWrapper

envsensor

hvac

HVACZoneData 1

2

3

5

6

PPSExt

4pps_extension

Page 56: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

1) HVACZoneData.as

com.qnx.smartenergypanel.system.hvac.HVACZoneData

public function HVACZoneData(i_id:String, listen:Boolean)

{

. . .

_pps = new PPSWrapper("/fs/pps/hvac/zone" + i_id +

"_state", PPSConstants.O_RDWR, listen);

. . .

}

public function getCurrentTemp():Number

{

_pps.open();

return Number(_pps.getData().currentTemp);

}

Page 57: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

2 & 3) PPSWrapper.as / PPSExt.as

com.qnx.smartenergypanel.system.pps.PPSWrapper

public function getData():Object

{

return ppsext.data;

}

private function handleChange(event:Object):Void

{

// Logger.trace("PPS Change Event for:"+ppspath);

var ev:PPSEvent = new PPSEvent(PPSEvent.CHANGE, this);

dispatchEvent(ev);

}

com.qnx.extensions.pps.PPSExt

OnChange method invoked by PPSEvent sets ppsext.data

Page 58: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

4) asextension/pps/ase-pps.so

pps_extension.c

static int _pps_read(struct pps_data *pdata) {

...

while ( ! done ) {

...

len = read(pdata->fd, info->data,

sizeof(info->data) - 1);

...

flash_as_ext_if->notify(pdata->player, _pps_notify_data, (void*)info);

...

}

...

}

Page 59: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

5) hvac

hvac-resmgr.c

void update_pps(int updateMode) {

...

rv = snprintf(pps_ptr, size, "currentTemp::%.2f\n",

hvacattrs.state.sensor_data.temperature);

...

write(hvacattrs.pps_state_fd, pps_buffer, strlen(pps_buffer));

flushall();

}

//Polled by timer 3 times a second, but could be event driven by hardware

int pulse_handler (message_context_t *ctp, int code, unsigned flags,

void *handle)

{

. . .

status = devctl(hvacattrs.sensor_fd, DCMD_SENSOR_STATE_GET,

&sensor_state, sizeof (sensor_state), NULL);

. . .

update_pps(0);

}

Page 60: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

6) Envsensor

sensor.c

void update_from_network()

{

server_host = gethostbyname(server_hostname); /* connection setup */

. . .

fd = socket(AF_INET, SOCK_STREAM, 0); /* create and send request */

bytes = snprintf(buf, sizeof(buf), "sample %s %s\n", sensor_type,

sensor_address);

write(fd, buf, bytes);

read(fd, buf, sizeof(buf)); /* read response */

. . . /* parse response */

}

int io_devctl(resmgr_context_t *ctp, io_devctl_t *msg, RESMGR_OCB_T *ocb)

{

. . .

case DCMD_SENSOR_TEMP_GET:

*(unsigned*)dptr = ocb->attr->data.temperature;

nbytes = sizeof(ocb->attr->data.temperature);

break;

. . .

msg -> o.ret_val = 0;

msg -> o.nbytes = nbytes;

return (_RESMGR_PTR (ctp, &msg -> o, sizeof (msg -> o) + nbytes));

}

Page 61: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Summary

Use case overview

Smart energy reference design

Examining the components

HMI

Drivers

HMI to Service Integration

Methods Available

Direct call

PPS

Example Deep-dive

Getting Temperature

Questions?

Page 62: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

62 All content copyright QNX Software Systems.

Questions?

Sheridan Ethier

[email protected]

Page 63: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Backup Slides

Page 64: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

Hands On

Instructor will present demo on VMWare, students work on i.MX25 targets

Instructor and students will explore the modular software architecture and discuss the advantages this design

Instructor will walk through Flash Action Script code down to native C-code drivers

Students will be able to modify data objects at the OS level to follow the reaction at the HMI and vice-versa

Page 65: Hands-on Workshop: Industrial Human-Machine Interfacing ... · QNX Software Systems. Month, Day, Year. 2. QNX. All content copyright QNX Software Systems. Agenda. QNX Overview. I.MX25

65 All content copyright QNX Software Systems.

Reference Design Software Architecture

Flash Lite 3 Player

PPS SQL

Power

Scheduler

SQL

SQLite

Energy Usage

Database

PP

S

Flash Player Extensions

Insteon

Lighting

PPS

HVAC

Controlle

r

PPS

MPG4Mediaplaye

r

PPS

Load

Switches

PPS

Smart

Meter

PPS

FlashAction Script

C/C++Drivers &

Applications

Zigbee

Sensors

PPS