HUAWEI TECHNOLOGIES CO., LTD.

www.huawei.com

VALID’2016

ModeratorXinli Gu, Huawei Technologies, USA

Panelists

Jos van Rooyen, Indentify Test Services, the Nederland

Martin Zinner, Technische Universitat Dresden, Germany

Yingzhen Qu, Huawei Technologies, USA

Simon Genser, Kompetenzzentrum das Virtuelle Fahrzeug, Austria

Volker Gollucke, OFFIS Institute for Information Technology, Germany

Panel: Systems & Data – Challenges in Simulation and

Verification/Validation for Complex Systems & Big-Data

7. December 2011

HUAWEI TECHNOLOGIES Co., Ltd.

www.huawei.com

HUAWEI Confidential

Security Level: confidential

Thank You

www.huawei.com

© VIRTUAL VEHICLE

VIRTUAL VEHICLE Research Center is funded within the COMET – Competence Centers for Excellent Technologies – programme by the Austrian Federal

Ministry for Transport, Innovation and Technology (BMVIT), the Federal Ministry of Science, Research and Economy (BMWFW), the Austrian Research

Promotion Agency (FFG), the province of Styria and the Styrian Business Promotion Agency (SFG). The COMET programme is administrated by FFG.

Derivative Estimation via DELC-Method with Noisy

Data

Simon Genser

VIRTUAL VEHICLE Research Center

© VIRTUAL VEHICLE

My Person

10.10.2017 Derivative Estimation by DELC with Noisy Data 2

• Simon Genser

• Completed Master in Industrial Mathematics at the TU Graz, Austria.

• numerical methods (FEM, BEM, FDM,…)

• functional analysis

• partial differential equations

• Employed at the Virtual Vehicle Competence Center in Graz, Austria.

• Junior Researcher

• working on my PhD thesis about coupling of control systems

• Fields of Interest:

• numerical derivative estimation

• coupling of control systems

© VIRTUAL VEHICLE

Example: Sensitivity Analysis of a Blackbox System

10.10.2017 Derivation of a New Method for Derivative Estimation by Linear Combinations 3

Interested in Output-Input Sensitivity:

So we need:

Computation:

© VIRTUAL VEHICLE

Derivative Estimation by Linear Combinations (DELC) Method

10.10.2017 Derivation of a New Method for Derivative Estimation by Linear Combinations 4

Computation formula:

First solve:

© VIRTUAL VEHICLE

General Statement of the Problem

10.10.2017 Derivative Estimation by DELC with Noisy Data 5

One is interested in only by samples .

Therefore use the DELC method

For noiseless signals it works

but for noisy signals it fails.

© VIRTUAL VEHICLE

DELC with noisy signal

10.10.2017 Derivative Estimation by DELC with Noisy Data 6

© VIRTUAL VEHICLE

Why is the DELC Method not applicable for noisy signals?

10.10.2017 Derivative Estimation by DELC with Noisy Data 7

1. You can see the DELC Method as interpolating the samples

with a polynomial and derive this polynomial to get the derivative .

2. Affine Arithmetic Theory:

Noisy samples are represented as

with as the amplitude of the white noise.

The uncertainty in the DELC computation is

© VIRTUAL VEHICLE

Affine Arithmetic and DELC with noisy signal

10.10.2017 Derivative Estimation by DELC with Noisy Data 8

© VIRTUAL VEHICLE

How to improve it?

10.10.2017 Derivative Estimation by DELC with Noisy Data 9

1. Smooth the signal first and then use DELC on the smoothed signal

1. Smooth with Least Square Methods

2. ….

2. Some adjustment to the DELC to get rid of the interpolating behaviour

1. Try something like approximation instead of interpolation

2. Other ideas=?

© VIRTUAL VEHICLE

Simon Genser

VIRTUAL VEHICLE Research Center

simon.genser@v2c2.at

Security Level:

www.huawei.com

Intelligent Data Center

with Machine Learning

Yingzhen Qu

Senior Research Scientist, Future Networks

Huawei Technologies, US Research Center

October 2017

HUAWEI TECHNOLOGIES CO., LTD. Page 2

A Generalized Machine Learning Loop

Examples RegressionNeural Networks

outputinput PredictionA

pply

Reward(Data, action-A)Penalize(data, action-B)

input

HUAWEI TECHNOLOGIES CO., LTD. Page 3

An Example: ECMP Based Link Utilization Problem in a Switch

Massive Scale DCs use fixed spine-leaf topology

ECMP distributes traffic across multiple paths

ECMP uses Hash computation to balance similar flows over multiple links

However, the flows are not evenly balanced

› Low-bandwidth (Mice) flows: Majority of flows are short-lived and latency sensitive.

» Example: Web, chat applications

› High-bandwidth (Elephant) flows consume majority bandwidth and are long-lived.

» Example Storage-intensive big-data, data-replication and backup applications

Problem

› Variance in the amount of bandwidth used between long-lived vs short-lived flows does not ensure that traffic

is balanced across all the links.

› Increase in Mean-time-to completion for mice flows

› Reduced data-rate for elephant flows due to congestion control

HUAWEI TECHNOLOGIES CO., LTD. Page 4

Facebook Data Center Fabric

Source: https://code.facebook.com/posts/360346274145943/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/

HUAWEI TECHNOLOGIES CO., LTD. Page 5

Machine Learning for ECMP Link Utilization in a Switch

ECMP linksElephant flows

Mice flowssame hash forcertain flows

outputinput PredictionMiceflow?

Y

ECMP Link select

No

• 70% times pickECMP prescribedlink

• 30% Pick LeastCongested Link

Live input

Classify flow Tuple actions

Classify flow size > 2Mbps as elephant flows

RLA – Reinforcement Learning Algorithm

HUAWEI TECHNOLOGIES CO., LTD. Page 6

Intelligence Driven Networking – DC Scenarios with Global Scope

• Extend to wider scoped learning - Global models acrossmultiple switches

• Different Learning models for different scenariostogether

Src: Facebook data center

Learn to optimally place services, VMs in DC

Optimize End to End paths

© Identify 2009-2017 13-11-2017

IT mediation

Jos van Rooyen

© Identify 2009-2017

• Employed at Identify as partner / principal consultant

• >25 years in software testing & quality management

• Co-author TestGrip ,TestFrame, Project de Baas, QualitySupervision, Textbook; “Aan de slag met software testen”

• Test expert online magazine Computable

• Publication areas; Test process Improvement, BI-testing, Testautomation, Test Education, Risk Based Testing, QualitySupervision

• Review committee software conference Valid2017

• Founder of the “Houten Groep”

• Member NESMA working party; Metrics in Contracts

• Member of several working parties Dutch Testing Society:

• Model Based Testing

• Test Education universities of applied science

22

Who am I?

© Identify 2009-2017

Contracts by suppliers are very promising

A lot of functionality for a low price

I see a lot of projects failing just at this point:

• Wrong functionality or less functionality delivered

• Misinterpretation of the requirements

• Delivery of the system too late

• Budget problems. Running out of money

• Poor quality so the demanding organization has todo a lot of rework

• Demand / supply doesn’t work together

33

Interpretation of the problem

© Identify 2009-2017

Based on IT mediation:

Determination of the suitability of the new information system in theexisting organization, with respect to the business process, the people, ITand the organization:

• Quality

• Maintainability

• Metrics

• Process

• Vendor management

• Contract management

• Legal

44

Possible solutions to solve the problem

© Identify 2009-2017

• Based on metrics:

• Number of defects discovered

• Defect Removal Efficiency

• Compliance to the requested functional quality

• Quality in use

• Coverage degree requirements

55

Possible solutions to solve the problem(2)

© Identify 2009-2017

• Do you recognize the described situation?

• What are your ideas to manage these kind of situations?

• Are the described solutions indeed good solutions?

66

Questions to the audience

© Identify 2009-2017 Ketenregie

Jos van RooyenE-mail: jos.van.rooyen@identify.nlMobile: +31-654906282LinkedIn: Jos van Rooyen

Questions?

Data Analysis for a Reference Waterway

Dr.-Ing. Volker Gollücke

Observe, , Test

Reference Waterway

The Reference Waterway covers the Elbeand Kiel Canal Approach, Germany. Itcovers a basic maritime surveillanceinfrastructure with three Sensorboxes(including AIS, Radar, cameras) and broadband communication.

• Setting up a databasewith travel pattern andnear collisions

• Gathering Live Radar,AIS, Wind and VideoData

• Connected to the eMirInfrastructure with LTE

Observing and learning from the real world

13.11.2017

All sensors can be gathered foran overview of the situation onthe ElbeeMir Infrastructure allows dataexchange between differentsources and users

Example: Maneuverpoints

13.11.2017

application of the knowledge acquired

13.11.2017

• Research Vessel• Not only AIS data but also Video Streams• Online prediction of behavior of target vessels

Challenges Little’s Law Operating Curve Management Conclusion Literature

Manufacturing AnalyticsMethods to analyze and improve the performance of a fab

Martin Zinner

October, 10 2017 @ ICSEA 2017 (Athens)

1 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Agenda

Challenges

Little’s Law

Operating Curve Management

Conclusion

Literature

2 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Agenda

Challenges

Little’s Law

Operating Curve Management

Conclusion

Literature

3 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Challenges in a fab

I Expensive tools should not lack of material to process.I Tools must be maintained having downtimes (planned and

unplanned).

Technical Objectives:1) Reduce inventory and implicitly reduce cycle time.2) Increase throughput.

4 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Agenda

Challenges

Little’s Law

Operating Curve Management

Conclusion

Literature

5 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Little’s Law

I Average cycle time (“flow time” / “sojourn time”)(avg)CT : Time interval that a unit spent in the system

I Average work in process - inventory(avg)WIP: Average number of items in the system

I ThroughputTH: Average number of items arriving per unit time

The average departure rate exists and equals the average arrival rate:

(avg)CT =(avg)WIP

TH.

6 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Little’s Law: Valability Conditions

Valability Conditions:I In steady state and non preemptive systems (properties are

unchanged in time, no interrupts and later resumes)I In stochastic systems if the probabilities that some events

occur in the system are constantI Valid as limit on time (holds eventually)

Properties:I Average departure rate exists, equals the average arrival rate.I It can be used on all level of the production system (tools,

single production step, group of production steps, wholeproduction line for a product, whole production line).

I It is one of the central theorems of the operational research.

7 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Agenda

Challenges

Little’s Law

Operating Curve Management

Conclusion

Literature

8 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Flow Factor

Flow factor FF is a reference value, controls the production.

I Cycle time CT : Time to process unit including waiting timeI Raw process time RPT : Minimal time to process a unit

FF :=CT

RPT.

FF can be calculated by . . .a) . . . applying Little’s Law to estimate the cycle time,b) . . . using the effective values of the cycle time.

9 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Capacity, Utilization, Variability [3]

I Capacity Capa: Maximal throughputI Utilization U: Throughput / CapacityI Variability α: Can be calculated using a single (FF ,U) tuple

FF = α · UU − 1

+ 1.

Utilization vs. flow factor Throughput vs. cycle time10 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Agenda

Challenges

Little’s Law

Operating Curve Management

Conclusion

Literature

11 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Conclusion

Requested optimization objectives:1) Reduce inventory and implicitly reduce the cycle time.2) Increase throughput.

Conclusion:The two objectives are contradictory!They cannot be optimized at the same time!

Challenge:Seeking a good balance between both objectives.

12 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Agenda

Challenges

Little’s Law

Operating Curve Management

Conclusion

Literature

13 / 14

Challenges Little’s Law Operating Curve Management Conclusion Literature

Literature

1) Book, one of the best:Wallace J. Hopp, Marc L. Spearman: “Factory Physics:Foundations of Manufacturing Management”, 2nd edition,2001, McGraw-Hill, Higher Education

2) PhD thesis, easy to follow:Kirsten Hilsenbeck: “Optimierungsmodelle in derHalbleiterproduktionstechnik”, 2005, Doctoral dissertation,Technische Universität München,https://mediatum.ub.tum.de/doc/601620/601620.pdf

3) Slides, very accurate and comprehensive:Walter Hansch, Tina Kubot: ‘Factory Dynamics”, Chapter 7,Lectures at Universität der Bunderwehr Munich,http://www.unibw.de/rz/dokumente/fakultaeten/

getFILE?fid=6186607&tid=fakultaeten

14 / 14

Introduction Practical Example Summary Bibliography

Real-Time Data Analytics

Martin Zinner

October, 10 2017 @ ICSEA 2017 (Athens)

1 / 11

Introduction Practical Example Summary Bibliography

Agenda

Introduction

Practical Example

Summary

Bibliography

2 / 11

Introduction Practical Example Summary Bibliography

Agenda

Introduction

Practical Example

Summary

Bibliography

3 / 11

Introduction Practical Example Summary Bibliography

Research project “Cool Silicon”

1 out of 5 leading-edge clusterswithin hightech strategy of the German Federal Government

I Project term:2009 – 2014

I Budget:135 million Euro

Members of Cool Silicon (Source: cool-silicon.de)

Aim: Substantially increase energy efficiency of ICs andinformation technology

4 / 11

Introduction Practical Example Summary Bibliography

Scope of a IT-subproject in “Cool Silicon”

Aims:1) Performance improvement2) Reducing energy consumption3) New functionality for Business Intelligence

Methods:I Investigation and testing of appropriate infrastructure and

system architectureI Definition and testing of appropriate modeling / algorithmsI Creation of prerequisites for efficient real-time aggregations

and analysis (including ad hoc reporting)I Creating prerequisites for efficient parallelizationI Reduction of complexity of nightly batch runs for data

aggregation5 / 11

Introduction Practical Example Summary Bibliography

Agenda

Introduction

Practical Example

Summary

Bibliography

6 / 11

Introduction Practical Example Summary Bibliography

Illustration of principles of real-time data analytics

Practical example: Computation of standard deviation sN for Nobservations / items x1, x2, x3, . . . , xN

I Classical calculation: (Cannot be used for real-timecomputation)

sN =( 1

N

N∑i=1

(xi − x̄)2)1/2

with x̄ =1N

N∑i=1

xi .

To calculate sN red sum has to be recomputed from scratch.

I Real-time calculation:

sN =1N

(∣∣N N∑i=1

x2i − (

N∑i=1

xi)2∣∣)1/2

.

Obtained by multiplication and regrouping.If new item is added, then both sums can be updated.sN will be calculated only on demand. 7 / 11

Introduction Practical Example Summary Bibliography

Agenda

Introduction

Practical Example

Summary

Bibliography

8 / 11

Introduction Practical Example Summary Bibliography

Classical vs. real-time aggregation

Classical aggregation (batch mode):Calculation is started only when all the data involved becomesavailable (e.g., end of the day, etc.).

I Large amounts of data have to be calculated.I Potentially complicated performance improvement algorithms

are necessary.I Race against time.

Real-time aggregation:Data are aggregated as soon as data are known to the system.

I Small amounts of data are computed quasi continuously (e.g.,in memory spread over the whole day).

I Smaller hardware is sufficient.I Energy efficient.

9 / 11

Introduction Practical Example Summary Bibliography

Agenda

Introduction

Practical Example

Summary

Bibliography

10 / 11

Introduction Practical Example Summary Bibliography

Bibliography

Patent US 2017/0032016 A1Zinner et al.: “Real-time Information Systems and MethodologyBased on Continuous Homomorphic Processing in LinearInformation Spaces”http://www.google.ch/patents/US20170032016

11 / 11