This project has received funding fromthe European Union’s Horizon 20 20research and innovation programmeunder grant agreement No 688403

www.tulipp.eu

TULIPP

Place :

Date :

PEGPUM Workshop

https://www.hipeac.net/events/activities/7435/pegpum/

Project Overview

HiPEAC’17, Stockholm

24th of January 2017

Philippe Millet, Thales, Coordinator

8 Partners

• Thales : Coordinator and Medical use case

• Sundance : Hardware

• Hipperos : Operating system

• Synective Labs : ADAS use case

• Efficient Innovation : Management

• Fraunhofer IOSB : UAV use case

• Ruhr Universität Bochum : FPGA tools

• NTNU : Performance tools

Outline

• Project objectives

• Main outcomes

• Advisory Board and Ecosystem

WP presentation

WP7: Management, Coordination

LABEL : Marketing, Ecosystem and Pre-normalisation

WP6: IP protection, Dissemination, Communication, Advisory Boardand Exploitation preparation

WP1: Reference platform definition(Interfaces & implementation Rules)

Instantiations

WP2: Hardware

WP4: Programming Toolchain

WP3: Runtime, API, Libraries & OS

feedback WP5 : Usecases description and Integration and platformvalidation

Starter Kit

Handbook Overview

Guidelines

Guidelines

Advice: Exploit both vectorization and multithreading for high performance on multicore

processors with vector units such as the ARM Cortex A9. On these architectures, utilizing

all hardware execution resources is key to achieve high performance [2] [4, 5].

Recommended implementation method: Use OpenMP. OpenMP is a widely supported

parallel programming API that enables programmers to express vectorization and

multithreading operations concisely using compiler directives. Programmers need not

worry about specifying scheduling and synchronization operations in code. These are

handled transparently by the OpenMP runtime system. See the official OpenMP

examples[6] to understand in more detail about exploiting vectorization and

multithreading simultaneously.

Reference Platform

Towards Ubiquitous Low-Power Image Processing Platforms

Component tools

Operating System

Processor

Toolchain

Reference Platform

Memory

IO

Processor

• The use case is taken from the many applications that now enter our cars, doing emergency braking, lane keeping, etc., with direction towards autonomous cars

• The chosen image processing algorithm for the use case is pedestrian detection that typically is used for emergency braking and driver assistance systems.

• Pedestrian detection is today mostly made by either using Viola/Jones classifiers or Deep Learning

• Viola Jones classifying has been chosen due to its more challenging memory access patterns

ADAS use case

Pedestrian detection

Safety application

Car integration

The Use Case

ADAS use case

Medical imaging use case

• Real-Time X-Ray imaging for surgery

• Reduce radiation dose by 75%

• Add noise removal processing with critical real-time constraints

Medical imaging use case

TDLP

RAW IMAGE

THALES Processing Unit

CI / ICS UI

GigE-Vision + Msg

THALES Flat panel detectorCustomer system

UI

GigE-Vision + Msg

CI / ICS

Nano Processing UnitInside the detectorBased on SoC (credit card size board)

Customer systemTHALES Flat panel detector

Before Tulipp

After Tulipp

Unmanned Aerial Vehicle (UAV) use case

• Performs real-time stereo depth estimation to do obstacle / collision avoidance (for an UAV), i.e. to detect obstacles in direction of flight

• Based on dual cameras

Unmanned Aerial Vehicle (UAV) use case

• Uses state-of-the art stereo algorithms (image correlation)

• Produces a distance image, i.e. where the image data shows the distance to each object

TULIPP Hardware Node

• Now – Zynq SoC SoM • Soon– Zynq MPSoC SoM

40mm x 50 mm – Z7030 40mm x 50 mm - ZU5EV

TULIPP System Node

Low-Power Image Processing RTOS

Needs OS - high reliability, - low power, - hard real-time - high performance

This kind of RTOS is not (yet) available today, as current GPOS (e.g. Linux) or RTOS lack one or more of the required features and performance.

Specific Image processing Needs- supporting the hardware accelerators - the libraries needed for image processing.

STHEM: The TULIPP Tool-chain

Status:• Xilinx SDSoC has been extended to

support the current platform• Support for HIPPEROS OS is underway

Insights:• Significant effort has been invested

into the development of vendor tools• STHEM fills the productivity gaps

between existing tools

Support uTilities for Heterogeneous Embedded image processing (STHEM) •Supports development for all platform

components

•Map source files of the application to the appropriate tool chain

•Retrieve OS configuration from the developer

Development and Mapping

•Boot OS with selected configuration (if needed due to changed configuration)

•Update files (binaries, bitfiles, etc.)

• Initialise the reconfigurable logic (if needed)

•Start the application with the requested instrumentation

Runner

•Analyses performance results and presents findings to the developer

Analyser

TULIPP in the news

20

Advisory Board and EcoSystem

Advisory

Board(WP6)

ReferencePlatform

(WP1)

Tulipp Guide, implementation and demosAsk for review / advise

Roles in the project:Provide information about standardsGive feedback on the approachEarly adopters

Sys temEco

AB: first step tothe ecosystem

Advisory Board Members

22

Advisory Board and EcoSystem

• Pledge• Interconnectivity

• Faster time-to-market

• Open standards