BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is...

23
INFOTECH OULU Annual Report 2016 1 BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG) Professor Juha Röning and Dr. Heli Koskimäki, Biomimetics and Intelligent Systems Research Unit, Faculty of Information Technology and Electrical Engineering, and Professor Seppo Vainio, Oulu Center for Cell- Matrix Research, Faculty of Biochemistry and Molecular Medicine, University of Oulu juha.roning(at)oulu.fi, heli.koskimaki(at)oulu.fi, seppo.vainio(at)oulu.fi http://www.oulu.fi/bisg Background and Mission Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci- ence and biology. In BISG, our basis are intelligent systems and our research areas include data mining, machine learning, robotics, and information security. More precise research topics vary from data mining algorithm development and optimization of industrial manufacturing processes all the way to environmental monitoring with mobile robots. Bringing expertise from ICT and Biotech together, we will reach the skills to make use of the mechanisms common in information processing and the biological data processing system and extrapolate this to intelli- gent solution making in ICT. One important goal of this program is to be able to physically link living cells via identified signaling systems to establish learning complex that involves Bio and ICT in a unified bifunc- tional interactive machine. The group consists of four sub-groups: Data Analysis and Inference Group, Organ BioEngineering Biology, Robotics and Secure Programming We have conducted basic research in intelligent sys- tems and tissue engineering for over ten years as indi- vidual groups. Now we have joint our efforts. Our team consists of 2 professors, 10 post-doctoral researchers and 15 doctoral students. The annual external funding of the group is more than two million Euros, in addi- tion to our basic university funding. In the reported year, there have been 23 completed doctoral degree from the group. From the research of the group, 11 spin-off companies have been established so far: Codenomicon, Clarified Networks, Hearth Signal, Nose Laboratory, Nelilab, Atomia, Indalgo Probot, Aquamarine Robots, Radai and IndoorAtlas. We co-operate with many international and domestic partners. In applied research, we are active in European projects. In addition, several joint projects are funded by the Finnish Funding Agency for Technology and Innovation (Tekes) and industry. We were a research partner in the SIMP and CyberTrust SHOKs. Prof. Juha Röning was selected as ACO (Academic coordi- nator) of the Cyber Trust program. We are active in the scientific community. For exam- ple, Prof. Juha Röning is acting as visiting professor of Tianjin University of Technology and as the Robot Science Adviser of Tianjin Science and Technology Center for Juveniles. He served as a member of the Board of Directors in euRobotics and as a member of the SAFECode International Board of Advisors. He chaired the euRathlon / TRADR Summer School 2016 in Oulu, Finland, 22nd to 26th of August. It was a five- day course to provide participants with a full overview and hands-on experience with multi-domain real robot- ic systems. He also chaired with prof. Othmane the First International Workshop on Agile Development of Secure Software (ASSD’16) in Salzburg 1st of Sep- tember. With robotics group, he participated NORDRUM project where aim was to collect radiation data from the environment using an unmanned aerial vehicle (UAV). The testing area was located in Nor- way, Hauerseter Leir military campsite (5th to 7th of September). During the reporting year, the group organized the 9th International Crisis Management Workshop and Winter School (CrIM’16) and NordSec conference which brought together both Finnish and international infor- mation security experts. The group also organized Summer School 2016 of Data mining, big data and open data together with Exactus DP and Aurora DP 15th to 19th of August. Representing Finland as a Partner for Peace (PfP) na- tion, BISG / Prof. Röning participated the Specialists’ Meeting on Intelligence and Autonomy in Robotics, held in Wachtberg, Bonn, Germany on 25 27 October 2016. Celentano and Röning are co-editors, together with collaborating partners, of an IEEE Access special sec- tion on Recent Advances in Socially-aware Mobile Networking. Prof. Seppo Vainio has been the chair in the Minisym- posium ”Omics in Biomedicine” (2016). He is also part of a European nanotechnology ”HyNanoDend” net- work. Scientific Progress Intelligent Systems Incorporating Security Within the Biomimetics and Intelligent Systems Group, the Oulu University Secure Programming Group (OUSPG) has continued research on security and safety in intelligent systems. Security and safety challenges in

Transcript of BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is...

Page 1: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 1

BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)

Professor Juha Röning and Dr. Heli Koskimäki, Biomimetics and Intelligent Systems Research Unit, Faculty

of Information Technology and Electrical Engineering, and Professor Seppo Vainio, Oulu Center for Cell-

Matrix Research, Faculty of Biochemistry and Molecular Medicine, University of Oulu

juha.roning(at)oulu.fi, heli.koskimaki(at)oulu.fi, seppo.vainio(at)oulu.fi

http://www.oulu.fi/bisg

Background and Mission

Biomimetics and Intelligent Systems Group (BISG) is

a fusion of expertise from the fields of computer sci-

ence and biology. In BISG, our basis are intelligent

systems and our research areas include data mining,

machine learning, robotics, and information security.

More precise research topics vary from data mining

algorithm development and optimization of industrial

manufacturing processes all the way to environmental

monitoring with mobile robots.

Bringing expertise from ICT and Biotech together, we

will reach the skills to make use of the mechanisms

common in information processing and the biological

data processing system and extrapolate this to intelli-

gent solution making in ICT. One important goal of

this program is to be able to physically link living cells

via identified signaling systems to establish learning

complex that involves Bio and ICT in a unified bifunc-

tional interactive machine.

The group consists of four sub-groups: Data Analysis

and Inference Group, Organ BioEngineering Biology,

Robotics and Secure Programming

We have conducted basic research in intelligent sys-

tems and tissue engineering for over ten years as indi-

vidual groups. Now we have joint our efforts. Our team

consists of 2 professors, 10 post-doctoral researchers

and 15 doctoral students. The annual external funding

of the group is more than two million Euros, in addi-

tion to our basic university funding. In the reported

year, there have been 23 completed doctoral degree

from the group. From the research of the group, 11

spin-off companies have been established so far:

Codenomicon, Clarified Networks, Hearth Signal,

Nose Laboratory, Nelilab, Atomia, Indalgo Probot,

Aquamarine Robots, Radai and IndoorAtlas.

We co-operate with many international and domestic

partners. In applied research, we are active in European

projects. In addition, several joint projects are funded

by the Finnish Funding Agency for Technology and

Innovation (Tekes) and industry. We were a research

partner in the SIMP and CyberTrust SHOKs. Prof.

Juha Röning was selected as ACO (Academic coordi-

nator) of the Cyber Trust program.

We are active in the scientific community. For exam-

ple, Prof. Juha Röning is acting as visiting professor of

Tianjin University of Technology and as the Robot

Science Adviser of Tianjin Science and Technology

Center for Juveniles. He served as a member of the

Board of Directors in euRobotics and as a member of

the SAFECode International Board of Advisors. He

chaired the euRathlon / TRADR Summer School 2016

in Oulu, Finland, 22nd to 26th of August. It was a five-

day course to provide participants with a full overview

and hands-on experience with multi-domain real robot-

ic systems. He also chaired with prof. Othmane the

First International Workshop on Agile Development of

Secure Software (ASSD’16) in Salzburg 1st of Sep-

tember. With robotics group, he participated

NORDRUM project where aim was to collect radiation

data from the environment using an unmanned aerial

vehicle (UAV). The testing area was located in Nor-

way, Hauerseter Leir military campsite (5th to 7th of

September).

During the reporting year, the group organized the 9th

International Crisis Management Workshop and Winter

School (CrIM’16) and NordSec conference which

brought together both Finnish and international infor-

mation security experts. The group also organized

Summer School 2016 of Data mining, big data and

open data together with Exactus DP and Aurora DP

15th to 19th of August.

Representing Finland as a Partner for Peace (PfP) na-

tion, BISG / Prof. Röning participated the Specialists’

Meeting on Intelligence and Autonomy in Robotics,

held in Wachtberg, Bonn, Germany on 25 – 27 October

2016.

Celentano and Röning are co-editors, together with

collaborating partners, of an IEEE Access special sec-

tion on Recent Advances in Socially-aware Mobile

Networking.

Prof. Seppo Vainio has been the chair in the Minisym-

posium ”Omics in Biomedicine” (2016). He is also part

of a European nanotechnology ”HyNanoDend” net-

work.

Scientific Progress

Intelligent Systems Incorporating Security

Within the Biomimetics and Intelligent Systems Group,

the Oulu University Secure Programming Group

(OUSPG) has continued research on security and safety

in intelligent systems. Security and safety challenges in

Page 2: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 2

intelligent systems are threefold: increasing complexity

leads to unforeseeable failure modes, quality is not the

priority and awareness is lacking. We have approached

the challenges from these three directions in our re-

search.

Complexity - Model Inference and Pattern Recogni-

tion: we work under the premises of unmanageable

growth in software and system complexity and emer-

gent behaviour (unanticipated, not designed) having a

major role in any modern non-trivial system. We have

worked on natural science approaches to understanding

artificial information processing systems. We have

developed and applied model inference and pattern

recognition to both content and causality of signalling

between different parts of systems.

Quality - Building Security In: software quality prob-

lems, wide impact vulnerabilities, phishing, botnets,

and criminal enterprise have proven that software and

system security is not just an add-on, despite the past

focus of the security industry. Instead, security, trust,

dependability and privacy have to be considered over

the whole life-cycle of the system and software devel-

opment, from requirements all the way to operations

and maintenance. This is furthermore emphasized by

the fact that large intelligent systems are emergent and

do not follow a traditional development life-cycle.

Building security in not only makes us safer and se-

cure, but also improves overall system quality and

development efficiency. Security and safety are trans-

formed from inhibitors to enablers. We have developed

and applied black-box testing methods to set quantita-

tive robustness criteria. International recognition of the

Secure Development Life Cycle has provided us with a

way to map our research on different security issues.

Awareness - Vulnerability Life Cycle: Intelligent sys-

tems are born with security flaws and vulnerabilities,

new ones are introduced, old ones are eliminated. Any

deployment of system components comes in genera-

tions that have different sets of vulnerabilities. Tech-

nical, social, political and economic factors all affect

this process. We have developed and applied processes

for handling the vulnerability life-cycle. This work has

been adopted in critical infrastructure protection.

Awareness of vulnerabilities and the processes to han-

dle them all increase the survivability of emergent

intelligent systems for developers, users and society.

These research goals are reached through a number of

research activities.

Secure Software Development Lifecycle as a part of

the Cyber Trust project - we approach all three goals

by researching practical ways of building security into

Secure Platforms, Cloud Computing services and Criti-

cal Infrastructure, from the design phase to actual oper-

ational use (Figure 1).

Figure 1. Dependencies of a single cloud based web service visualized by technology and location.

Situational Awareness in Information and Cyber Secu-

rity aims to understand critical environments and accu-

rately predict and respond to potential problems that

might occur. Networked systems and networks have

vulnerabilities that present significant risks to both

individual organizations and critical infrastructure. By

anticipating what might happen to these systems, lead-

ers can develop effective countermeasures to protect

their assets (Figure 2).

Figure 2. Port scanning visualized in an industrial auto-mation network.

Coverage based robustness testing: Modern web

browsers are feature rich software applications availa-

ble for different platforms ranging from home comput-

ers to mobile phones and modern TVs. Because of this

variety, the security testing of web browsers is a di-

verse field of research. Previously, we have found a

number of bugs in browsers, but previous methods

were seeing diminishing returns. By utilizing code

coverage we were able to improve on existing state of

the art. This work introduces a cross-platform testing

harness for robustness testing, called CovFuzz. In the

design of CovFuzz, test case generators and instrumen-

tation are separated from the core into separate mod-

ules. This allows the user to implement feature specific

test case generators and platform specific instrumenta-

tions, and to execute those in different combinations.

Identification of a protocol gene: This research, PRO-

TOS-GENOME, approaches the problems of com-

plexity and quality by developing tools and techniques

for reverse-engineering, and identification of protocols

based on using protocol genes - the basic building

Page 3: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 3

blocks of protocols. The approach is to use techniques

developed for bioinformatics and artificial intelligence.

Samples of protocols and file formats are used to infer

structure from the data. This structural information can

then be used to effectively create large numbers of test

cases for this protocol.

OUSPG Open: This activity brought together over 60

people from 23 academic and commercial organiza-

tions by organizing regular events throughout the

summer. Five novel security tools were produced as

collaborative projects. For example, TryTLS, a tool for

the software and library developers, vulnerability re-

searchers, and end-users, who want to use TLS safely,

resulted in a number of issues being found in common-

ly used programming libraries.

Privacy and Security and Online Social Networks:

Exploiting Social Structure for Cooperative Mobile

Networking (SOCRATE), a two -year (2015-2016)

Tekes funded project under the Wireless Innovation

between Finland and U.S. programme WiFiUS

[http://wifius.org/], is a collaboration between the Uni-

versity of Oulu (co-PI Dr Ulrico Celentano), VTT,

Aalto University, Arizona State University, and Uni-

versity of Nevada, supported by NSF funding on the

US side. BISG contribution focused on privacy and

security issues in online social networks data mining

and the related architecture aspects and on privacy and

security issues.

The final goal of SOCRATE project is to exploit for

optimised radio network operation the knowledge

about the social structure of network users. Clearly,

this potentially exposes users to disclosure of sensitive

information. The disclosure of personal information, if

not anonymised, exposes also to additional threats such

as identity theft or even physical security or denial of

service or sabotage. Similar privacy and security ques-

tions are found in other applications, such as those

enabled by the Internet of Things (IoT) paradigm. In

this extension (Figure 3, top), we may use the term

user to mean a person or an entity possibly related to a

person, and the word social is used to refer to a person

or equivalently in more abstract terms to the contextual

relationship of devices.

IoT devices are increasingly permeating the human

environment. Connected sensors and/or actuators are

found for example in smart environments, cars and

wearables, in both industrial and nonindustrial scenari-

os. Whereas they serve a range of applications and

services, virtually any IoT device has access to sensing

data about humans or it acts on the environment hu-

mans live within and in particular on appliances and

services humans rely upon. Clearly, information securi-

ty, hence privacy, and physical security, hence safety,

are therefore unavoidable themes in such people-

centric IoT. More generally, security enforcement is a

fundamental enabler of the success of IoT and people-

centric IoT in particular.

IoT DEVICE IoT DEVICE

RADIO FEATURES, LOCATION, MOBILITY

IoT DEVICE

ATTRIBUTES

ATTRIBUTES

NETWORK

ATTRIBUTESATTRIBUTES

WIRELESS NETWORK

MANAGEMENTIoT SERVICE

CROSS-EXPLOITATION

IoT SERVICE

SOCIAL FEATURES

ATTRIBUTES

ATTACKERATTACKER

INT

I/F

EXT

I/F

ENH

AN

CEM

ENTSI/F

CONTEXT AND INFERENCE

ATTRIBUTES

WIR

ELES

S N

ETW

OR

KA

TTR

PEOPLE-CENTRIC SERVICE

OPTIONAL EXTERNAL SERVICE

DEPLOYED NETWORK

INTERFACES

Figure 3. A people-centric ICT scenario, above, and the conceptual framework supporting it, below. From Celen-tano et al. (manuscript).

BISG research in this area follows data minimisation

principles, where 1) occasions for collection of sensi-

tive data, 2) the extent of data collection, and 3) the

time duration of data storage are minimised. The

above, and the second point in particular, have direct

impact on architectural choices, see Figure 3.

At the top of the figure are depicted on the left the

people-centric IoT service and on the right the wireless

network supporting it. The objective is to infer obser-

vations of features at one domain and exploit them at

the other domain, through the enhancement protocols

shown in yellow at the bottom of Figure 3. By data

minimisation principles, attributes are stored at various

parts of the system to mitigate the threats of possible

attackers. Separated interfaces towards individual

sources guarantee flexibility of the design. Attackers

may target either side of the system (Figure 3, top).

Different scenarios enabled by the knowledge of social

structure and users’ features are compared in Höyhtyä

et al. (2016). Relationship among users and users’

preferences can be used to identify alternative data

distribution strategies, for example relying on clusters.

These strategies are then associated with corresponding

transmit power requirements. Options include changes

in the topology (direct data transfer from base station

BS to user equipment UE; or from BS to cluster-head

CH and then from CH to UEs; or having chunks of data

sent from BS to UEs, and then sharing them among

UEs). Considering the availability of various radio

access techniques (RAT), the above options can be

combined with changes in the RAT for communication

between CH and UEs and among UEs (LTE; WiFi).

The most power-efficient scheme depends also on the

Page 4: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 4

required data rate, and in turn the data rate impacts on

transmission time and hence on energy efficiency.

Related to the above research and in the framework of

SOCRATE project co-operation, Celentano and

Röning are co-editors, together with collaborating

partners, of an IEEE Access special section on Recent

Advances in Socially-aware Mobile Networking.

Intelligent Systems Incorporating Machine Learning and Data Mining

Data mining methods for steel industry applications:

BISG is a member of the Centre for Advanced Steels

Research - CASR, which is one of the interdisciplinary

umbrella organizations of the University of Oulu. Year

2016 was the third year in participation to a large na-

tional research programme System Integrated Metals

Processing – SIMP.

One of the main goals in SIMP programme has been

the development of an innovative supervisor system to

assist the process development personnel and the oper-

ators of a steel production line over the whole produc-

tion chain, and to help discover new alternative solu-

tions for improving both the products and the manufac-

turing process. The quality monitoring tool (QMT) is

based on statistical models that predict different quality

properties and rejection risks in several process steps,

and it provides also model visualization (Figure 4). The

tool has been developed in co-operation with VTT, In

year 2016, QMT was delivered for online use at Ou-

tokumpu, Tornio and for offline use at SSAB, Raahe.

The development work continues, and the functionality

of the tool can be improved with the feedback of the

test period.

Figure 4. The tool for quality monitoring and visualiza-tion during the steel making process.

One of the models in QMT predicts the rougness of the

stainless steel surface with hot rolling parameters. In

previous case, the roughness was visually evaluated to

9 classes by experts in Germany. The goal was to pre-

dict the quality after hot rolling, before the product was

shipped to Germany for further processing. In 2016, a

new data was gathered with roughness measured with a

device. Although, the roughness type was in this case

different, the research revealed that also the finishing

step has a significant effect on the surface quality. The

new models will be implemented into QMT next year.

In the year 2016 one goal of the SIMP project was

reached, as the selection of combination parameters for

slab design were updated and improved with statistical

models. The aim of the combination is to ensure the

sufficiency of the material to produce the ordered

product with desired dimensions. The variance model-

ling increased the knowledge of process deviation and

the factors behind it. As a result, the new selection

procedure is expected to increase yield, reduce the risk

of rejection, energy consumption and emissions, which

in its turn improves the cost-effectiveness of the steel

mill. This study showed that using specific statistical

modelling methods and classification, the knowledge

behind the steel process can be pointed out and utilized

in manufacturing. Powerful data mining methods ena-

ble the effective use of process data in slab design. This

research is brought out vital problems in production

line and there has been a lot of development work done

also at SSAB, Raahe due to this study. The results of

this research were published in journal of Ironmaking

& Steelmaking on August 2016.

In the steel plate production process it is important to

minimize the wastage piece produced when cutting a

mother steel plate to the size ordered by a customer.

The uneven shapes at the plate end sides and lateral

sides cause yield loss, amounting to about 5% to 6% of

a total tonnage of slab used. To minimize the loss, aim

is to produce plates with concave side edges because

wastage from concave side edges is smaller than from

convex. We developed a method for automatic recogni-

tion of steel plate edge shape with classification and

regression models. First, we defined the curvature of a

time series describing the steel plate side edge, and

used this information to build statistical distribution

model to visualize what kind of curve shapes the stud-

ied data set includes and how the amount of curvature

is distributed in the manufacturing process. This infor-

mation can then be used to optimize manufacturing

parameters to manufacture more plates with desired

shape. Data for the study was collected from the steel

plate mill at SSAB, Raahe.

A tool for finding clusters of inclusions in SEM (Scan-

ning Electron Microscope) specimens of steel samples

was developed. The inclusion clusters may have a

significant effect on mechanical properties of the steel;

especially novel ultra-high strength steels require very

high steel purity. This tool enables quick and efficient

inspection of the specimens. The summaries of the

clusters are produced, as well as, visualizations of the

whole test area or interesting parts of it, The visual

presentation of the chemical composition of the clus-

ters helps to understand the birth mechanism of the

formation, and thus, to find a way to prevent it in steel

products. An example of a test sample from finished

product can be seen in Figure 5. One cluster has been

selected for a closer inspection, and it can be seen that

Page 5: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 5

in this case there are only MnS particles in quite large

inclusion formation. The shape of the cluster is unusu-

al, but the direction of the test sample may explain it.

The research was carried out in the co-operation with

SSAB, Raahe.

Figure 5. An example of a clustered SEM specimen with zoomed in visualizations and the chemical compo-sition of the selected cluster,

SIMP programme will continue for another 6 months,

and we will present our research results annually on

SIMP and DIMECC seminars around Finland as well

as at international publishing venues.

Uncertainty of classification models. Many real-world

data sets contain missing data values. These might be

the result of e.g. malfunctioning sensors or some meas-

urements being too expensive to measure from every

sample etc. Having missing values when classifying a

sample means that there is an increase in uncertainty in

the final classification result. Knowing how uncertain

the result can sometimes be as important information

as the classification result itself. In an Infotech doctoral

program project, we are quantifying that uncertainty so

that interpreting the classification results becomes

easier.

Classification algorithms have traditionally been de-

veloped using complete data sets and most require

values for all variables to be present to work. Many

real world data sets are, however, cursed with missing

data. To tackle this problem, we developed an algo-

rithm that uses multiple imputation to handle the miss-

ing values. The algorithm can be used with any classi-

fier that supports estimation of class posterior probabil-

ities. The developed algorithm performs as well or

even better as a benchmark algorithm (see Figure 6)

and it does not require the classifier to support handling

of missing values.

The uncertainty does not, however, behave consistently

across different data sets. In a follow-up work we ad-

dressed this issue and the results of this work were

presented in a conference, see Figure 7.

Figure 6. Modelling the uncertainty as a function of classification rate.

Figure 7. Prediction of test sample uncertainty based on the uncertainty model from Figure 4.

Page 6: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 6

Figure 8. Calibration plots from raw prediction scores, two commonly used calibration algorithms and our novel algorithm. Unpublished results.

To generalize these results, the classification confi-

dence algorithm was modified to a calibration algo-

rithm. In practice this means that the prediction scores

of a classification algorithm are attempted to get more

closely to resemble true posterior probabilities. Still

unpublished example of the performance this calibra-

tion algorithm compared to two state-of-the-art can be

seen in Figure 8.

Data mining methods for data of wearable sensors:

BISG has a long experience is studying data from

wearable sensors. Earlier the study has concentrated on

human activity recognition based on accelerometer data

from a wrist-worn device or mobile phone. This year

BISG extended the application area from human activi-

ty recognition to early diagnosis of diseases. In addi-

tion, we have used variety of sensors including elec-

tromyogram, thermometer, electrodermal activity sen-

sor and photoplethysmography sensor.

Human activity recognition: The activity recognition

approaches can be used for entertainment, to give peo-

ple information about their own behavior, and to moni-

tor and supervise people through their actions. Thus, it

is a natural consequence of that fact that the amount of

wearable sensors based studies has increased as well,

and new applications of activity recognition are being

invented in the process. In 2016 BISG concentrated on

studying human activity recognition in two scenarios:

adaptive models and comparing electromyogram data

to accelerometer data.

Usually human activity recognition is based on user-

independent models. However, as people are different

these models do not work equally well with every sub-

jects. Therefore, in order to obtain high recognition

rates with all users, models need to adapt to each user’s

personal movements. In our study, it was shown that

personal models can be trained without a separate data

gathering session if wearable device has several types

of sensors. On the other hand, it was shown that per-

sonal models always do not provide as high recognition

accuracies as reported in the literature. The reason for

this is that models are not general enough and therefore

they cannot react to changing conditions. In fact, in

order to build reliable user-dependent recognition

model, a lot of personal data needs to be collected. This

requires an extensive, separate data collection session

for each user. If the aim is to build a commercial appli-

cation for the masses, this is far from ideal situation.

However, BISG introduced an noise injection based

method to expand the area covered by training data,

and in this way, make the models trained using it more

general and less vulnerable to changing conditions.

This is shown to improve the recognition rates, espe-

cially if the amount of the training data is small. An

another approach to tackle the same problem a solution

combining the human independent and personal mod-

els more effectively using self-organizing maps based

distance as a selection criteria was introduced. By us-

ing the approach, the selection can be done in real time

and within wearable device itself. The results show that

the approach clearly outperforms posterior probability

based approach in preserving the high recognition

accuracy regardless of which model is used.

BISG also studied the possibility to use electromyo-

gram data to recognize human activities. The actual

research problem tackled is one of the major drawbacks

in activity recognition, namely to add completely new

activities in real life to the recognition models. In this

study, it was shown that in gym settings electromyo-

gram signals clearly outperforms the accelerometer

data in recognition of completely new sets of gym

movements from streaming data even though the sen-

sors would not be positioned directly to the muscles

trained. However, it was noted than when the task is to

recognize previously known activities, accelerometer

data outperforms electromyogram signals.

Early diagnosis: The progress in the sensor develop-

ment including improved memory and battery proper-

ties has made possible to measure human physiology

24/7, and more importantly with such accurate readings

that have previously been possible only in laboratory

settings. BISG is aiming to use a single, easy and com-

fortable device to measure 24/7 data from persons bio-

signals and based on these recognize upcoming sei-

zures. For this purpose BISG got a six month funding

from TEKES from Challenge Finland program.

Though the funding period was short, promising results

were obtained. The first use case was based on mi-

graine. The migraine is a chronic, incapacitating neu-

rovascular disorder, characterized by attacks of severe

headache and autonomic nervous system dysfunction.

Moreover, migraine headache is usually associated

with nausea, vomiting, or sensitivity to light, sound, or

movement and when untreated, typically lasts 4 to 72

hours. Typical migraine attack consist of five phases:

prodrome (e.g., food craving), aura (e.g., visual, senso-

Page 7: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 7

ry, or motorsymptoms preceding the headache), head-

ache (usually unilateral, pulsating), resolution (pain

wanes), and recovery. The illustration of these phases

are shown as Figure 9. The migraine condition starts in

over 40% of the cases before 18 years of age, thus

making it also a childhood disease. The medication of

migraine can be divided into two categories: preventive

(daily dose) and acute medication (when symptoms

start). The preventive medication is an expensive solu-

tion and especially with children, it is avoided as long

as possible. On the other hand, the problem with acute

medication is that some people do not have early symp-

toms, some tend to easily dismiss the early symptoms,

sometimes even on purpose. Thus the early diagnosis

of migraine attack would be a valuable addition to

treatment of the disease. The research concept was

aimed to early detection of migraine attach based on

human bio-signals collected with wearable sensors is

presented. In our approach, the idea is to use a single,

easy and comfortable device to measure 24/7 data.

Results from this project will be published in 2017.

Moreover, BISG is actively seeking new funding pos-

sibilities and instruments to continue in studying the

early diagnosis also in 2017.

Figure 9. Phases of typical migraine attack.

Developing software for data mining in public health:

The multidisciplinary MOPO project combined tradi-

tional health promotion, modern technology and meas-

urement of physical activity. Altogether about 6000

conscription aged men (five call-up age classes) were

invited to participate in the study, where the partici-

pants’ physical condition, well-being, health, relation-

ship towards physical activity, information behaviour

and use of media and technology were investigated

during the years 2009–2014 using questionnaires,

measurements and interviews. In addition to this, a

novel wellness coaching service for preventing mar-

ginalization and promoting physical activity and health

in young men was developed in the project. The ser-

vice, which took the form of a gamified Web portal

optimized for mobile devices (Figure 10), was devel-

oped by BISG personnel and incorporated functionality

for collecting, storing and analysing physical activity

data and presenting the results of the analysis to the

user as personalized feedback.

The Web portal went through a number of iterations

that were evaluated in a series of intervention studies,

culminating in an intervention where access to the

portal and a wrist-worn activity monitor was given to

approximately 250 conscription-aged men (Figure 11).

The intervention started in autumn 2013 and lasted 6

months. Following this final intervention, the lessons

learned over the course of the MOPO study were gen-

eralised into a set of design principles intended to be

applied by medical researchers and software developers

implementing digital interventions for health behaviour

change. A paper on the proposed design principles was

published in 2016. Also in 2016, BISG researchers

contributed to an analysis of MOPO data comparing

self-reported versus measured physical activity and

sedentary behaviour.

The operators of the MOPO study were the Oulu Dea-

coness Institute’s Department of Sports and Exercise

Medicine, the University of Oulu, the City of Oulu, the

Virpiniemi Sports Institute, the Finnish Defence Forces

and several wellness technology companies in North-

ern Finland. The project website can be found at

www.tuunaamopo.fi.

Figure 10. The Web portal offers tailored information

about topics such as physical activity, fitness, health,

and nutrition.

Page 8: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 8

Figure 11. In autumn 2013, 250 conscription aged men were recruited from call-ups to test the wellness coach-ing service developed in the MOPO project.

Foundations of knowledge discovery and data mining:

Knowledge discovery in data (KDD) was defined in

1996 by Fayyad et al. as “the nontrivial process of

identifying valid, novel, potentially useful, and ulti-

mately understandable patterns in data”. Although this

definition still has its merits, it represents a rather nar-

row interpretation of the concept of knowledge that

may prove a hindrance to the development of more

advanced KDD tools. Meanwhile, the seminal process

model proposed by Fayyad et al., which depicts the

KDD process as a sequence of five major steps, is still

embedded in most KDD process models, including the

standard model CRISP-DM. This established model,

while essentially correct, represents a limited perspec-

tive on the KDD process that is likely to prove inade-

quate in the long run.

In its research on the foundations of KDD and data

mining, BISG has sought to expand this traditional

view of the nature of KDD. The resulting model, like

the established one, accounts for the data transfor-

mations required in order to get from raw data to

knowledge, but also for the actors of the process and

the interactions among them that need to take place for

the process to move forward. Furthermore, the model

explicitly considers the contributions of non-expert

actors, as well as the possibility of technology taking

on a more autonomous role in the process, which is

likely to be realized in the near future as KDD software

grows more intelligent and becomes capable of han-

dling tasks that currently require a human actor. Hav-

ing a model that provides a more complete account of

the KDD process is essential in unlocking the full po-

tential of KDD technology, which in turn is crucial in

making sense of the deluge of digital data that seems to

have become a permanent feature of high-technology

societies. Figure 12 illustrates the process actors and

how different interactions among them lead to different

types of KDD processes.

Intelligent Systems Incorporating Robot-ics and Cybernetics

euRathlon Summer School

The ERL Emergency/TRADR summer school 2016

was organized from the 22nd to 26th of August by the

robotics group members and the staff from TRADR

(Long-Term Human-Robot Teaming for Disaster Re-

sponse). The summer school was attended by 55 stu-

dents, mostly doctoral, originating from 17 different

countries (Figure 13). Also, 6 invited lecturers from

TRADR held lectures during the summer school. Ac-

cording to the satisfaction survey, the participants were

very pleased with the summer school as all of those

who answered the survey would recommend it to oth-

ers.

Figure 12. The actors of the KDD process can be illustrated as the vertices of a triangle, with technology in the cen-ter, being both an actor in its own right and a mediator of interactions among human actors (a). The process can take on a number of different forms, characterized by which of the actors are present and how they interact: the standard KDD process (b), KDD using personal data (c), KDD using volunteer computing (d), and KDD driven by a non-expert actor (e). A good example of the latter is the so-called Quantified Self movement.

Page 9: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 9

Figure 13. Attendees and organizers in the ERL Emer-gency/TRADR summer school 2016 held at the Univer-sity of Oulu.

This year, the ERL Emergency summer school focused

on developing algorithm for controlling land robots

with a strong focus on SLAM and multi-source persis-

tent data integration.

In total, the summer school lasted for four and a half

days consisting roughly 35% of lectures and 65% of

practical exercises in which the students developed

control and SLAM algorithms. These practical sessions

were held indoors in the University of Oulu’s facilities

and outdoors in a nearby (less than 1km of walking)

botanical garden where electricity, shelter and internet

access were also provided. The hands-on practices

culminated to a challenge scenario that each team per-

formed on the last day at the botanical garden.

The students were provided with two of TRADR’s

UGVs and one UAV (Figure 14) provided by Ascend-

ing Technologies. The students could modify and de-

velop software for the two UGVs but the UAV was

flown only by the trained representative of Ascending

Technologies. Before the summer school, these UGVs

and UAV were also used to gather preliminary data for

software development during the exercises. The raw

data was used to form initial maps that were given to

the students to work on for testing and performing

simulations. This was done to reserve the students’

time for more meaningful tasks as the generation of

maps from raw data takes many hours of processing on

a desktop computer.

During the registration process the students were asked

to provide a brief description of their programming

experience. This info was used on the first day to form

eight balanced teams of six or seven persons. The bal-

ancing was done mainly in regard of C, Python and

ROS experience as at least one person in each group

had to have at least basic understanding of these to

ensure that the practices would proceed in a timely

fashion.

Figure 14. TOP; One of the two identical unmanned ground robots (UGV) provided by TRADR. BOTTOM; The UAV, AscTec Falcon 8, provided by TRADR part-ner Ascending Technologies.

On the first day the student teams were presented with

a challenge scenario they would perform and compete

on the fifth, and last, day of the summer school. The

scenario consisted of a simulated toxin leak at the bo-

tanical garden and the teams’ task would be to find and

localize the toxic materials using the UGV, UAV and

the pre-recorded maps of the area.

To fulfil this task, the teams needed to fulfil the sub-

tasks:

Map the area using one UGV and one UAV.

Update and refine the map based on new data.

Develop strategies to safely navigate the UGVs in

the danger zone.

Navigate the UGV to designated points of interest

with the highest degree of autonomy possible.

Detect objects automatically if possible.

Have the UGV perform automatic collision avoid-

ance if possible.

The practical sessions focused on developing ways to

fulfil these tasks and mapping the scenario area (see

Figures 15-18).

Page 10: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 10

Figure 15. The used waypoint planner and simulator.

Figure 16. Outdoor testing at the botanical garden.

The challenge scenario was run on the final day of the

summer school. The toxic leaks were simulated with

bright balloons and rough estimates of their locations

were given to the teams. Compared to previous days,

the scenario environment was somewhat changed by

added obstacles (chairs, tables, etc.). Each team had 30

minutes of time to complete the mission, during which

they had full access to the UGV. The teams also had a

limited 5 minutes access to the UAV, flown by the

trained operator, to get a rough overview of the envi-

ronment.

Figure 17. Scenario briefing and composed map of the area.

The students were also given a demonstration of the

Aquamarine Robots Dolphin marine robot.

Figure 18. Aquamarine Robots Dolphin robot shown during the summer school.

Page 11: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 11

Robotics Research

In 2016 BISG had a wide range of research in the area

robotics, including industrial safety, aerial data gather-

ing, battery life management and control of complex

wheeled land robots.

ReBorn

The EU funded ReBorn project has ended. The project

had participants from 17 industrial and academic insti-

tutions from 10 different countries. During the project,

the sufficiency of current standards related to robot

development and reusability in industrial environments

was investigated by a paper review and by surveys sent

to the project partners. The current standards (Figure

19) for designing user safe robots were deemed suffi-

cient for fulfilling the requirements to implement safe

robots for traditional industrial applications. However,

in some applications shortcomings in the currently

available standards were found.

Figure 19. Main standards applicable in implementing user safety in industrial robotics.

One of the commonly mentioned issues was that there

is a lack of standardized commonly applicable perfor-

mance descriptions for the existing line of robots. This

especially hinders the flexible use of heterogeneous

and modular robotics. Also, reuse and repurposing old

robots for new applications is more difficult without a

common form of performance descriptions and capa-

bilities of the robots. It was also found out there cur-

rently are no dedicated ISO/EN/DIN standards specifi-

cally related to safety and design assisting in imple-

mentation of reconfigurable manufacturing cells. Other

area that was mentioned in the survey responses from

the project partners was the lack of LCC (Life Cycle

Cost) standards, similar to what are already in use in

the building construction industry.

One area of interest in the project was the requirements

for implementing CWS (Collaborative Work Spaces).

In this field, some standards already exists (Figure 20)

that can be utilized to implement the minimum safety

features required to avoid serious injuries. However,

from the applications side, the standards are currently

vague on how the software should be implemented and

how the user should be taken in to account as an agent

acting in the control loop when performing tasks co-

operatively with a robot. The actions of the human in

the loop needs quite a lot of prediction and behavior

observation to implement co-operation efficiently and

safely in flexible manufacturing units. This is an area

that is currently under a lot of research and appropriate

standards should also be developed on how the user

monitoring and behavior prediction should be per-

formed on hardware and algorithm level.

Cloud computing is an area that could be better utilized

in industrial environments as a channel for data pro-

cessing, learning and teaching of industrial robots in

the future. Cloud computing could also be used for

openly collecting and sharing data about robot reliabil-

ity for evaluation of the reusability, safety and costs of

running specific types of robots in certain tasks.

Figure 20. Standards applicable for collaborative work spaces.

NORDUM Exercise

NORDUM (Intercomparison of Nordic unmanned

aerial monitoring platforms) exercise was organized in

the Hauerseter Leir military campsite, Gardermoen,

Norway. The NKS-B activity NORDUM is the first

joint Nordic exercise for unmanned systems. All in all,

five teams participated in this event coming from dif-

ferent universities and radiation safety related institu-

tions located in Norway, Sweden and Finland.

In the NORDUM exercise, the objective was to locate

and identify potential radioactive materials from the

arranged scenario areas. The scenario areas varied from

cluttered areas containing large shipping containers and

various metal structures to open field and forest scenar-

ios. For the measurements, a stand-alone sensor pack-

age was constructed containing a RTK (Real Time

Kinematic) capable GPS (u-blox C94-M8P-3), meas-

urement computer (Raspberry Pi 3 Model B), a 433

MHz (3DR) radio, a 3.7V Li-ion battery and a gamma

radiation spectrometer (Kromek GR1-A).

In the arranged scenarios, the teams needed to localize

hidden radiation sources and visualize their location

utilizing GPS. In the scenarios, the constructed stand-

alone sensor package performed well in most scenari-

os, although some radio link related issues were en-

Page 12: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 12

countered especially near large metal structures. The

utilized Kromek GR1-A was sensitive enough that

radiation sources could also be identified from local

spectrum histograms when enough flybys near the

radiation source was made.

The stand-alone sensor package was carried with a DJI

Inspire 1 T600 quadcopter, hanging 1.5 meters below

the vehicle. This made possible manoeuvring the sen-

sor very close to the objects being measured. The

quadcopter had a flight time of 10 minutes with a 5.7

Ah 22.2V Li-ion flight battery and the sensor package.

The 4k resolution camera was utilized during flight to

observe the sensor position and to manoeuver it to

wanted positions. Although the conditions were rather

windy, flying with the sensor package was manageable.

The quadcopter with the sensor package is shown in

Figure 21. The measurement results from one of the

three scenarios is shown in Figure 22 and Figure 23.

Figure 21. The DJI Inspire 1 carrying the constructed stand-alone sensor package containing a GPS, a radio transceiver and a gamma radiation detector. On the right is a still image captured by the onboard 4k resolu-tion camera.

Figure 22. Gamma radiation measurements made with the stand-alone sensor package carried by the quad-copter in one of the testing scenarios. The brightness of the green color indicates the intensity of the detected gamma radiation activity.

Figure 23. Local histogram collected from area en-closed by the red circle 3 in the scenario image. The detected spike corresponds with Cs-137 (Cesium with a theoretical gamma radiation energy emissions of 661.64 keV).

Robots

The Mörri robot has been equipped with more easily

maintainable and more powerful electronics in its rein-

carnation. The main drive electronics are now mostly

off-the-shelf components controlled with an Ardupilot

APM2 based controller that is connected to an onboard

computer handling the overall robot control and com-

munications with a remote control station. The Mörri

platform is also used in testing the test batch of intelli-

gent battery modules that have been constructed. The

functional diagram of the new drive system is shown in

Figure 24 and the Mörri mobile platform in Figure 25.

Figure 24. The overview of the renewed fundamental electrical system required to drive the Mörri robot.

Figure 25. Mörri with Microsoft’s Kinect 2 sensor driving on a field and on snow with tracks put on.

In anticipation of performing joint missions simultane-

ously with multiple UAVs and UGVs, also custom

quadcopter platforms are being constructed. The basic

platform, shown below in Figure 26, is low-cost and is

constructed from off-the-shelf components for better

maintainability. The quadcopter platform is built

around the open-source ArduPilot PX4 flight control-

ler, allowing more freedom for customization and test-

ing our own implementations required for autonomous

operation, which is not as easy to do with most prebuilt

and significantly more expensive quadrotors. Com-

bined with LIDAR (Light Detection And Ranging), the

copters will be used for SLAM and environment classi-

fication efforts in joint missions with UGVs, such as

Mörri. Because both Mörri and the quadcopter utilize

ArduPilot based controllers, the development of both

platforms is simpler due to having very similar proto-

cols for using the controller responsible for inertial

measurements and platform control.

Page 13: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 13

Figure 26. A semi-ready customizable low-cost quad-copter platform.

Intelligent battery modules

Intelligent battery modules (Figure 27) have been de-

veloped in collaboration with Probot Ltd. and the test

batch is being tested with our robot platforms. The

initial tests of the test batch have showed that the de-

signed battery electronics are functioning as was in-

tended. The battery module has an integrated heater for

winter operation and a charger module allowing energy

transfer from one module to another in any parallel

connected energy bus. With a developed charge control

module, the bus can also be potentially used to recover

energy from multiple power sources, such as solar

panels.

Figure 27. Assembled intelligent battery module for general use in modular robotics.

Control of Complex Wheeled Robots

Pseudo-omnidirectional robots with individually steer-

able wheels offer a good balance between payload,

robustness and mobility. However, the non-holonomic

nature of the regular wheels and the often redundantly

actuated structure of these robots make their control a

complex issue. This complexity of control is further

exacerbated when the wheels are not rigidly connected

to the robot body but are instead connected via actuated

chains which allow the wheels move relative to the

body. BISG has developed control algorithms for such

Articulated Wheeled Vehicles (AMW). The control

algorithms are mathematically simple closed-form

analytical functions and are thus computationally light

but are currently limited to planar cases. The computa-

tional load is only linearly dependant on the number of

wheels making the developed control algorithm suita-

ble for multi-wheel configurations and/or low-powered

embedded MCUs. The control algorithms synchronize

the rolling and steering velocities of complex planar

robots (plausible simulated example in Figure 28) with

freely located wheels forming fixed or variable foot-

prints. The rolling and steering velocities remain syn-

chronized even with very complex motions of the robot

(Figure 29). With the developed control algorithms, the

traversable path, robot’s heading on different points of

the path and the path velocity can be controlled sepa-

rately, thus offering great freedom on how to control

the robot on a given practical task. The control algo-

rithms do not in practice suffer from representation

singularities which are a common problem in wheeled

control. The control algorithms also compensate for the

proximity of mechanical singularities by adjusting the

robot’s path velocity according to the maximum capa-

bilities of its wheels’ steering and rolling actuators. In

fact the developed control algorithms are time optimal

in a sense that at any given moment the robot is either

traversing with maximum allowed path velocity or at

least one of its steering or rolling actuators is turning at

its maximum velocity (Figure 30), i.e. the robot

traverses the given path in the given way with the giv-

en velocity restrictions as fast as it possibly can.

Figure 28. Example of complex wheeled planar robot.

Figure 29. Simulation run of Figure 26’s robot traversing a given yellow path while keeping its front directed at all times to a point of interest (green larger dot). Note the smooth convergence of the robot (black line) and the target path.

Page 14: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 14

Figure 30. (Top) wheel rolling speeds, (Middle) wheel

steering speeds and (Bottom) robot path velocity for

the first 30 seconds of a simulation run.

In summary, the developed control algorithms can be

used in a wide range of robot configurations and sce-

narios with low computational cost. The control algo-

rithms are currently limited to planar surfaces and can

cause sudden and large changes in velocity and the

control algorithms are being extended to work also

with uneven surfaces and limited motor torques.

In year 2016 the control algorithms have been en-

hanced to allow the wheels to have non-zero lateral and

longitudinal offsets, making the algorithm suitable for

practically any configuration of a wheeled planar robot.

In addition, a path tracking algorithm was developed.

The algorithm is very simple yet provides smooth and

robust path convergence in simulated environments

(Figure 31).

Figure 31. Smooth path convergence in cluttered envi-ronment.

Two ERDF project started; Labrobot and OuluZone+

projects. Labrobot focuses on Food industry, and

OuluZone+ for autonomous vehicles in harsh condi-

tions.

Labrobot-project focuses on boosting regional Food

industry by technology transfer demonstrations, build-

ing up test facility and network of stakeholders. By

surveying challenges in factories, combined with

knowledge of robotics, big data, machine vision and

biotechnologies; new kind of solutions are searched for

base of new business possibilities. This project is done

in cooperation with Center of Machine Vision and

signal processing, Biocenter Oulu and Luke. Project is

partly funded by City of Oulu, Yaskawa, Probot,

Maustaja, Antel, Kinnusen mylly, mekitech, and SR-

Intruments.

In the OuluZone+ project the focus is on automatic

road building machines and smaller mobile robots

(UGV and UAV) for supporting operation on the field.

In the project are studied how the capabilities of auton-

omous cars could be formally verfied, and tested from

perspectives of operting in all weather conditions and

all situations. Project is partly funded by City of Oulu,

OSEKK, Ouluzone Operointi Oy and industrial part-

ners.

The Evolutionary Active Materials

The Evolutionary Active Materials (EAM) project,

which is funded by the Academy of Finland, is a joint

effort between the Computer Science and Engineering

laboratory (CSE) and the Microelectronics and Materi-

als Physics laboratories. The aim of the EAM project is

to develop novel, evolutionary computation (EC) based

design methods for active and versatile materials and

structures. The first components are being developed

through a novel holistic design process utilizing con-

stantly increasing computation power, the development

of multi-physics simulators, and EC techniques, such

as genetic algorithms (GA).

During 2016, the height and the top diameter of Cym-

bal type piezoelectric actuator were optimized by ge-

Page 15: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 15

netic algorithm and FEM modelling. From the opti-

mized results, maps of electromechanical capabilities

of different structures were generated. The blocking

force of the actuator was maximized for different val-

ues of displacement by optimizing the height of the cap

and the length flat region of the end cap profile. By

using values obtained from a genetic algorithm optimi-

zation process, a function was formulated for design

parameters. Using the function, a map of displacement,

the steel thickness and the height of the end cap the

optimized length of flat region was constructed (Figure

32). A similar map with the length of the flat region for

the optimized height of end cap was created. The re-

sults will be published at 2017.

Figure 32. The top diameter of the steel cap as a func-tion of steel thickness and displacement for Cymbal.

New type of actuator called Mikbal (Figure 33) was

invented, optimized with genetic algorithm and

realized. Mikbal was developed from Cymbal by

adding additional steel structures around the steel cap

to increase displacement and save the amount of used

piezoelectric material. The best displacement to

amount of used piezo material ratio was achieved with

25 mm piezo material diameter in the case of 40 mm

steel structures, and lower height and top diameter of

the cap increased the displacement. The results will be

published during 2017.

Figure 33. The von Mises stresses in Mikbal actuator under 500 V voltage.

Also optimization of the end cap structure of the Cym-

bal type energy harvester was done with genetic algo-

rithm and FEM modeling software Comsol Multiphys-

ics. The aim was to improve harvested power levels

from human walking (Figure 34). The power produced

by the energy harvester was increased by allowing the

algorithm to modify thickness in certain regions as

grooves in the end cap. By evolution of the structure,

power produced by the harvester increased by 38 %

compared to traditional linear type Cymbal harvester

which was also optimized by the algorithm. Increase in

power was obtained by change of mode in mechanics

of the harvester by grooves.

Figure 34. Cymbal type energy harvester in a shoe and an optimised profile for the harvester. In the profile piezoceramic disc is depicted in yellow and steel cap in grey. The grooves shown in the left side of the profile have been found by the genetic algorithm.

New grooved Cymbal energy harvester (Figure 35)

gave promising results in physical measurements pro-

ducing same power with less force than uniform shape.

The model was invented based on results given by

genetic algorithm optimization process with spline

shapes. Grooved Cymbal is easy to produce compared

to spline shape. Depth and place of grooves were opti-

mized by genetic algorithm. The parameters of the

algorithm itself were optimized also with GA, called

metaGA. Results of the metaGA will be published

during 2017.

Figure 35. Grooved cymbals.

Page 16: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 16

Intelligent Systems Incorporating Bio-IT solutions

We have taken part in the Ruby/Diamond HILLA pro-

ject. This was based on collaboration between the poly-

techniques, VTT and BISG. This and a previous Tekes

project lead to establishment of four strategies that

should offer openings in the aims to establish minimal-

ly invasive of non-invasive wellness and health param-

eter monitoring technologies. Via a collaborative net-

work, we acquired novel nanomaterials offering ways

to couple electronics to biomonitoring behaviour of

live cells.

Developing novel real-time biosensors for glucose

monitoring. For developing “second generation biosen-

sors”, we have taken use of our skills to purify and

culture the skin derived progenitor cells that are re-

sponsible in skin renewal and regeneration. We ob-

tained for the project a Tekes strategic opening fund-

ing. With this support, we have advanced the work to

develop of a novel biosensor strategy (Figure 36).

Figure 36. Novel biosensor strategy. Donor skin renew-

ing cells are set to culture and a specific responsive component is engineered to target a tag to the 3´end of the coding sequence in the genome. Such a cell is then implanted to the donor to serve as a measure for a given physiological parameter. These serve to offer novel ways to biomonitor in real time physiologically relevant factors with and external electronic reader that is coupled wirelessly to the cloud to data analysis of multiple sensors at the end.

By now, we have been able to conduct the proof of

principle set up in the sensor construction. These indi-

cate that the skin is indeed responsive to the changes in

certain serum constituents. The data also indicated that

the cells with in the skin can also be engineered and be

converted genetically to serve as biosensors, thus to

report changes in the physiological parameters such as

glucose. We have screened in selected biological phe-

nomena with the proteomics and transcriptomics the

respective mediators in the glucose response in the

skin. We also generated experimental diabetic models

to identity diabetes associated and insulin independent

responders. The approach has turned a successful one.

First of all the skin appears responsive for physiologi-

cal levels of glucose. Due to this reason we also were

able to identify candidate factors whose genes and

encoded are currently being engineered to convert the

respective protein into an isoform whose activity can

read with an external electronic device.

We have also tested the capacity to culture of FACS

purified cells of the skin and if such cells can be trans-

planted with a fluorescent tagged vital sensor cells to

the donor so that the cells indeed become incorporated.

We assayed the stability of the sensor cells as trans-

plants. The data suggest that a syngeneic host suggest-

ing that the aimed biosensor strategy is feasible accepts

the skin progenitor graft.

In collaboration with VTT we have also developed the

electronic unit, a tunable spectral camera. This has the

capacity to measure the changes in the skin basal pro-

genitor cell integrated sensor. We have filed a patent of

these biotechnological avenues with VTT.

Developing an ex vivo supernatural personal mobile

biosensor device. To advance the goal to develop novel

wearable sensory devises we started to assemble first

via a HILLA funded project a micro fluidistic set up

that will be converted to a bio recognition tool. During

the research period, several micro fluidistic prints were

planned, made and tested. Out of these a configuration

was obtained that collected successfully, the skin asso-

ciated fluids as depicted by the presence of color dye in

the fluidistic chamber (Figure 37). A patent search of

the strategy has been conducted.

Figure 37. A micro filudistic print design is able to col-

lect the skin-associated fluids as depicted by the accu-mulation of a blue indicator dye in the chamber.

During 2016, we developed capacity to the micro fluid-

istic set up to monitor specific biomolecules present in

the skin fluids. This work lead to an opening via identi-

fication of novel types of biological nanomaterial’s

from the skin. These components are generated nor-

mally by the cells, they cargo wealth of physiologically

relevant biomolecules and they can cross the biological

barriers. Given the numerous amounts, small size of

the nano scale components, the opening has stimulated

a need to establish both bio and databanks. This is

Page 17: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 17

currently being conducted with via deep sequencing

and proteomics to diagnose the samples that are de-

rived from cohorts.

During 2016, wealth of medical technical developmen-

tal lines with VTT and companies have been initiated

and also a new Tekes project grant filed. We obtained a

new Academy of Finland funded grant from the Bio

Future 2025 program to advance the nanobioelectronic

analysis strategies, one of the Infotech Oulu research

program targets.

To advance the biosensor openings we have started to

develop at the same time more complex diagnostic

platforms as the fluidic champers. To be able to read

the fluorescence that is revealed by specific antibodies

bound to the diagnostic components reagents against

these factors are being developed during 2017 with our

collaborator. Our partners in the HILLA project were

able to develop a mobile phone based micro fluidistic

reader capacity. Together with the developed biochips,

such printable materials are likely to set the stage for

the point of care diagnostics in the field of personalized

medicine during 2017.

Screening of electromagnetic and opto/chemo/electro

genetic responses in organs generated from stem cells. The genetic engineering offers opportunities to devel-

oped technologies where the cellular in or output sig-

nals can also be regulated by certain wavelengths in the

electromagnetic spectrum. Alternatively the cellular

actions can be genetically constructed so that a signal

will be transmitted to a biosensor that will convert it to

a form readable by an electric device. To advance these

tasks we have initiated with private funding screens

that aim to identify cellular channels that are regulated

by specific spectral frequencies such as the RF ones.

Such diagnostics use a paradigm shift where the cellu-

lar responses to given stimuli will be screened primari-

ly via vital “biosensors” with live cellular tags. Thus

the approach in the bioelectronics analytics have be-

come possible via the crisp Cas9 genome-editing tech-

nologies where libraries of gene edited diagnostic cells

can be generated.

During 2016, we developed novel tissue engineering

technologies that do enable introduction of specific

gene expression constructs to individual cells of the

model organ such as the mammalian kidney. Here the

organ primordia is dissociated to single cells, the genet-

ic construct encoding the protein of interest such as the

opto, chemo or radiogenetic responsive component is

transduced to such a cell with a reporter for the read

out screens. There after the organ is let to self-assemble

and placed for a long-term culture (Figure 38).

Figure. 38. An organ primordia can be dissociated to single cells, the constitute cells transduced with a ge-netic construct to acquire opto-, chemo- and radio ge-netic guidance capacity to the morphogenetic cells ex vivo.

With the developed model systems we have taken use

of the image analysis technologies to visualize how the

morphogenetically active cells behave in three dimen-

sion in the 4D conditions that offer a whole organ pri-

mordia to be cultured ex vivo. To achieve this we

applied defined pressure to the assembled organ pri-

mordia in ex vivo setting depicted in Figure 39.

Figure 39. The 3D kidney organ primordia that is rela-tively thick being composed of multiple cell layers de-velops also under a mild pressurize in ex vivo. Here the mechanical pressure converts the 3D development more towards a 2D configuration. The developed setup will offer ways to identify pressure sensors in the cells and also to develop novel organ pressure monitoring tools. The power of this novel “organoid” culture set up is that it enables for the first time is complex organs image analysis and follow up of the behavior of the individual constituent cell while the complex 3D anatom-ical structure of the organ become laid down. It is im-portant that the quality of the data good enough to offer segmentation and “computer vision” analysis. With such “Fixed Z-Dimension” (FZD) culture we are in a process of illustrating the fine details how biological shapes, namely the organ structure in 3D becomes constructed from the cellular building blocks. These data serves also as the digital 3D landscape for developing 3D bio printing when advancing a European Union FET

Page 18: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 18

FLAGSHIP representing a regenerative medicine and nanotechnology initiative.

We found that under a defined pressure the organ flat-

tens towards two dimension (2D) but yet morphogene-

sis progressed (Figure 40). This novel set up has made

it possible follow the fate of individual cells is the cells

are constricting a detailed manner while the natural

form.

Figure 40. Operetta confocal workstation coupled to a robotic set up and an incubator was assembled. A) A

holder for plates and transported by the robotic arm (B) and the cells with in will be transported to an incubator (C). The whole set up is inside a hood (D) and the ro-botic arm transports the plates to the Operetta confocal semi-high throughout microscope fluorescent reader. The data is analyzed by wealth of machine vision/image analysis programs present with in the assembled bio robotic set up. The bio robotic core facility will be used to screen with a library of live indicators cellular re-sponse to specific frequencies in the electromagnetic spectra.

To target the detailed dynamics by which the form is

assembled in a model organ we took use of the genet-

ically engineered Wnt4CreGFP knock in mouse model.

This was crossed to the floxed Rosa26 Yellow Fluores-

cent Protein (YFP) transgenic mice. In this genetic

crossing the stem cells that generate whole of the neph-

ron will become labeled with the YFP.

With the fixed Z-dimension culture we have captured

3D movies from the developing kidney with the confo-

cal microscope in a time-lapse setting. We are in a

process of analyzing the detailed cell behavior via the

machine learning/computer based image analysis with

Prof. Janne Heikkilä. With Dr. Jari Juuti we aim to

construct a specific device that allows detailed measure

of the pressure forced encountered by the tissue under-

going morphogenesis. These novel capabilities now

allow analysis in great detail the mode by which the

spatial and temporal organization of the cells go on to

construct natural form that is open at present in any

developing organ system. We will use models to identi-

fy the pressure sensors from the cells with the OMICS

technologies.

Developing high throughput robotic aided platforms to

screen complex cellular responses to magnetic/electric

fields via signaling pathway reporters. To advance the

strategies to measure in a high through put manner the

cellular responses to stimuli we have assembled a bio

robotic workstation. Here an Operetta confocal micro-

scope was obtained and this was coupled to a hood that

contains an automated plate-cargo arm, a rack for the

plates with a bar code reader and incubator for long

term exposure of the cells to compounds such as drugs

or specific electromagnetic spectral radiation (Figure

40). The Operetta confocal microscope has machine

learning/image analysis capacity for wealth of meas-

urements to be conducted from the cells.

To take use of the set up a yeast cell library was ob-

tained and three replica clones from it was generated

and stored for later use. The library is composed of cell

where each of the 3´end of each of the yeast gene was

targeted by a green fluorescent protein (GFP) tag. The

next goal is to obtain capacity to start to use the set up

to define the oscillating properties of the cellular genes

and to use it as live measures for screening responses

to stimuli such as those mediated by the opsins for the

visible light frequencies. Such genome wide screens

vital bio indicator based scan be expected to lead to

identification of novel biosensor pathways for certain

spectral frequencies. When the strategy will be subject-

ed to patient derived gene edited human induced plu-

ripotent (iPS) cells and those whose fate has been engi-

neered to defined directions this technology should

offer avenues for the era personalized medicine diag-

nostic developmental aims.

Intelligent Systems with cohort data sets: Cohort data

set is a special data set from the medical domain, which

has not been studied with a machine learning approach

before. The data set, Northern Finland Birth Cohort

1966 (NFBC 1966), is a unique data set with over 14

000 original variables in various yet heterogeneous

formats (numerical, ordinal, categorical, images, text

etc.) from a population of over 12 000 mothers and

their children without any complete data points. The

amount of variables rises to millions if genetics and

epigenetics are considered (p >> n).

There are two extremely important aspects of modeling

this type of data: confidence of the predictions made

with the model and model interpretability. Steps to-

wards instance level confidence estimates have been

made in our previous work (see above) and we will

continue to pursue this goal, along with keeping model

interpretability in focus also, when we start digging

into this fascinating data set. Our goal is to use a ma-

chine learning approach to make novel discoveries

from the data that traditional data analysis approach

has not yet uncovered.

Elders are an increasingly large fraction of the popula-

tion in developed countries. From one hand people

Page 19: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 19

expect an independent life also in presence of more or

less important diseases. On the other hand the treat-

ments to care those diseases, often together with co-

morbidities, imply larger costs. To respond to both

these goals, the disease progress should be kept as low

as possible (see Figure 41), which means early disease

detection, deinstitutionalisation and personalised medi-

cine, striving to allow a better quality of life, a more

cost-efficient healthcare system and a more inclusive

access to healthcare both in developing countries and

in remote areas in developed countries.

Novel Bio-ICT technologies are needed to achieve

these targets and BISG is active in this area in many

fronts summarised below.

By tracking health status of large groups and including

in the analysis a wealth of metrics and parameters,

large amounts of data are generated. On the other hand,

by downscaling biology-based technologies down to

the nanoscale including sensing biological parameters

directly from living cells, potential security threats are

correspondingly moving into human bodies, but prom-

ising tools are offered for personalised medicine and

treatments, including tight biological interaction, pros-

theses and their control (Celentano and Röning 2015).

BISG is strong in all these areas (data analysis, security

and robotics) and it is therefore pushing itself among

the world leaders in this growingly important area.

0

Healthy state

1

Degeneration starts,

no noticeable impact

on everyday life

2

Mild impacts on

everyday life

3

Disturbs appear

evident or important

4

Severe degeneration

D

Death (complications,

accidents, suicide)

C

Daily care needed

B

Assistance needed

A

Normal life (almost)

preserved

D

Medical

Doctor

H

Hospital

Active

H

osp

ita

lise

d

Cost-

eff

ective

E

xp

ensiv

e

Figure 41 Progress of a disease (left), outcome (right) and access to healthcare (Celentano and Röning 2015).

Towards a Holistic Self-awareness in Humans and AI

Artificial entities like robots and unmanned or autono-

mous vehicles are more and more present in the human

environment. Social interaction among all the players

in such a heterogeneous scenario (Figure 42) calls for a

number of research issues to be addressed and its study

offers interesting potentialities.

Figure 42. Interwork among heterogeneous agents and within them. From Celentano & Röning (2016b).

Self agency. Self-awareness in humans plays a role in a

number of brain functions and disturbances. On the

other hand, self-awareness improves the efficiency in

robotic systems (Celentano and Röning 2016a).

Awareness of the self is achieved through analysis of

observations, or measurements, of various entities

involved. This interwork in a heterogeneous multi-

agent system (Figure 42) may occur with different

topologies: sensing the actuation of other entities, as in

Figure 43a; acquiring information shared by others, as

in Figure 43b; exploiting different functions for

self/nonself discrimination. In short, through perrcep-

tion, action, and sharing information (Celentano &

Röning 2016a).

Figure 43. a) Left: An entity (bottom) sensing the actua-tion of two entities (top). b) Right: Entities (bottom) acquiring instructions shared by another entity (top). From Celentano & Röning (2016a).

Embodied agent. As psychologist James Gibson ob-

served, there is an interdependency of perception and

action (“perceive to act, act to perceive”). We study the

social intelligent entity as embodied (Celentano &

Röning 2016b), where are brought to evidence not only

the interaction among entities but also the interwork

within them (cf. Figure 42 and Figure 44).

Considering that an instantiation of the agent may

possess only part of its functions, the same generic

model can be used at different scales, applied to enti-

ties, at brain, body and world domains (the latter possi-

bly including other entities or agents), benefitting

modularity and scalability (Celentano & Röning

2016b).

Page 20: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 20

Figure 44 The embodied agent and its environment. From Celentano & Röning (2016b).

Structured information representation and instruction

logic. Interwork among modular agents include as seen

perception and action but also the exchange of infor-

mation or commands, both referred to as instructions as

in Celentano & Röning (2016a). These communica-

tions may be subject to noise, as it is the case in an

operating room or in air traffic control (Figure 45, top).

Whereas machines are subject to environmental noise

only, humans suffer both environmental noise and

cognitive noise (Celentano & Röning 2016c), affecting

different steps of the information communication pro-

cess (Figure 45, bottom).

decodingencoding

interpretation mapping transfer mapping interpretation

cognitive noise

environmental noise

Figure 45. Top: Interaction among heterogeneous agents in a noisy environment. Bottom: Information communication between remote source and destination entities in noisy conditions. From Celentano & Röning (2016c).

For reliable information exchange among heterogene-

ous agents is needed a formal representation of the

exchanged instructions, usable by both humans and

machines (Figure 46).

Unambiguous

Language

Formal

Representation

Human

Robot

ImplementationMappingInterpretation

Atomi

ROSMSDL

higher level lower level

Human

Robot

Elements

Definitions

BML

Figure 46 Interaction through specified processes (lan-guages and representation). From Celentano & Röning (2016c).

Using the instruction logic in Celentano & Röning

(2016c), the example situation in which mobile m0 at

x0 orders mobile m3 to be in x1 at t1 to search a book b

and bring it immediately to m0 can be represented by

«report_to,m3,m0,x0,t,1»

«move_to,m3,-,x1,t<t1,1»

«search,m3,b,-,t,1»

«bring,m3,b,x0,0,1».

Exploitation of Results

BISG continued co-operation with the SpAtial, Motor

& Bodily Awareness (SAMBA) research group

[http://dippsicologia.campusnet.unito.it/do/gruppi.pl/Sh

ow?_id=hhuv] at the Department of Psychology of the

University of Turin, Italy. Several initiatives for EU

projects including Horizon 2020 are ongoing and these

efforts will be continued.

Outside Europe, BISG is currently co-operating with

the University of Nevada, Arizona State University and

Carnegie Mellon University.

The results of our research were applied to real-world

problems in many projects, often in collaboration with

industrial and other partners. Efficient exploitation of

results is one of the core objectives of the national

Digile and FIMEC ICT SHOK projects like SIMP, IoT

and Cyber Trust; in these projects we work in close

collaboration with companies throughout the projects.

During the reporting year, the group continued utilizing

outdoor robotic systems. Development and utilization

of Mörri, a multipurpose, high performance robot plat-

form continued. More focus was put on perception in

natural conditions, representation of detections,

knowledge, and an environment model of the operating

environment. The software architecture further devel-

oped the earlier work on Property Service Architecture,

and the Marker concept as general purpose representa-

tion was further developed.

Page 21: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 21

Future Goals

The partnership in the SIMP programme that belongs

to the SHOK concept of Tekes enables us to continue

our steel research into new areas. The new goals are in

quality prediction at different process stages and for

more challenging properties. As a result more ad-

vanced expert systems can be developed to aid the

operators with different roles in steel making.

We will continue to strengthen our long term research

and researcher training. We will also continuously seek

opportunities for the exploitation of our research results

by collaborating with partners from industry and other

research institutions on national and international re-

search programs and projects. The University of Oulu

is a founding member of euRobotics. Juha Röning is a

member of the Board of Directors of euRobotics.

We will strengthen our international research co-

operation. With the University of Tianjin in China, we

have a joint project in which methods and a system will

be developed for vision-based navigation of Autono-

mous Ground Vehicles, which utilize an omni-

directional camera system as the vision sensor. The aim

is to provide a robust platform that can be utilized in

both indoor and outdoor AGV (Autonomous Ground

Vehicles) applications. This co-operation will continue.

In the USA, we will continue to co-operate with the

Human-Computer Interaction Institute in Carnegie

Mellon University with Assistant Professor Anind K.

Dey. The research is on human modelling in the area of

human-machine interaction. We continue and strength-

en US-Finland co-operation through an NSF grants.

The co-operation within SOCRATE co-operation with

the University of Nevada can be exploited complemen-

tary expertise in the area of multi-layer security. Two

new BISG project proposals for co-operative projects

under the WiFiUS programme are currently under

review at the Academy of Finland and NSF.

Shorter research visits to European partners in EU-

funded projects are also planned. The cooperation with

Prof. Raffaella Ricci and her colleagues, focuses on

bridging neuroscience and artificial intelligence. This

research aims at cross-fertilising the two scientific

domains, continuing and strengthening the research

paths currently active at respective sides.

In 2017, the aim is to utilize more widely the know-

how from sensor technology and data mining. New

application areas will be studied, including rehabilita-

tion, exercise motivation and energy efficiency in

households, and the benefits of our expertise will be

highlighted to actors in the areas.

In human-environment interaction and sensor net-

works, our research will continue. Our main goals are

to develop analysis methods for sensor network data

and to develop applications utilizing physical user

interfaces. Research on novel software architectures,

reasoning and knowledge representations will continue

as well. Field trials in realistic settings, and close col-

laboration with research groups (national and interna-

tional) and companies will be emphasized.

Personnel

professors 2

senior research fellows

postdoctoral researchers 10

doctoral students 15

other research staff 5

total 32

person years for research 25

External Funding

Source EUR

Academy of Finland 156 000

Tekes 735 000

domestic private 176 000

international 200 000

total 1 267 000

Doctoral Theses

Latvakoski, Juhani (2016) Small world for dynamic

wireless cyber-physical systems. VTT Science 142.

Selected Publications

Alasalmi T., Koskimäki H., Suutala J. and Röning J. (2016).

Instance level classification confidence estimation. Advances

in Intelligent Systems and Computing. The 13th International

Conference on Distributed Computing and Artificial Intelli-

gence 2016, Springer.

Celentano U (2016) Panel: European Project Space on Intel-

ligent Technologies for Innovation and Sustainability. Invit-

ed. 8th International Conference on Agents and Artificial

Intelligence (ICAART). 24–26 Feb 2016, Rome, Italy.

Celentano U, Röning J (2016a) Multi-robot systems, ma-

chine-machine and human-machine interaction, and their

modelling. 8th International Conference on Agents and Arti-

ficial Intelligence (ICAART), vol. 1, pp. 118–125. 24–26

Feb, Rome, Italy.

Celentano U, Röning J (2016b) Modular agents for heteroge-

neous human-robot systems. ERL Emergency & TRADR

Workshop on Heterogeneity in Robotic Systems, Oulu, Fin-

land, 22–26 Aug.

Celentano U, Röning J (2016c) Structured information repre-

sentation and exchange in heterogeneous multi-agent systems

in mission-critical scenarios. Information Systems Technolo-

gy Panel: Specialists Meeting on Intelligence & Autonomy

(Ro-botics). Bonn, Germany, 25–27 Oct.

Celentano U, Röning J, Yang L, Zhang J, Ermolova N, Tirk-

konen O, Chen T, Höyhtyä M (manuscript) Information and

physical security in people-centric IoT.

Höyhtyä M, Mämmelä A, Celentano U, Röning J (2016)

Power-efficiency in social-aware D2D communications.

Proc. European Wireless Conference (EW 2016). Oulu,

Finland, 18–20 May 2016.

Page 22: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 22

Koskimäki H and Siirtola P (2016) Recognizing Unseen Gym

Activities from Streaming Data - Accelerometer vs. Electro-

myogram Advances in Intelligent Systems and Computing,

International Conference on Distributed Computing and

Artificial Intelligence

Koskimäki H and Siirtola P (2016) Adaptive Model Fusion

for Wearable Sensors Based Human Activity Recognition

International Conference on Information Fusion, ISIF, 07,

1709-1713.

Koskimäki H and Siirtola P (2016) Model Update in Weara-

ble Sensors Based Human Activity Recognition IEEE Sym-

posium on Computational Intelligence and Data Mining,

accepted.

Tuovinen L (2016) A conceptual model of actors and interac-

tions for the knowledge discovery process. In Proc. 8th Inter-

national Joint Conference on Knowledge Discovery,

Knowledge Engineering and Knowledge Management –

Volume 1: KDIR, 240–248.

Tuovinen L, Ahola R, Kangas M, Korpelainen R, Siirtola P,

Luoto T, Pyky R, Röning J & Jämsä T (2016) Software de-

sign principles for digital behavior change interventions:

Lessons learned from the MOPO study. In Proc. 9th Interna-

tional Conference on Biomedical Engineering Systems and

Technologies – Volume 5: HEALTHINF, 175–182.

Niemelä Maisa, Ahola Riikka, Pyky Riitta, Jauho Anna-

Maiju, Tuovinen Lauri, Siirtola Pekka, Tornberg Jaakko,

Mäntysaari Matti, Keinänen-Kiukaanniemi Sirkka, Röning

Juha, Jämsä Timo, Korpelainen Raija (2016) Nuorten miesten

fyysinen aktiivisuus ja istuminen itsearvioituna ja mitattuna

Liikunta & Tiede, 53(2-3):73-79.

Pietikäinen P, Kettunen A & Röning, J (2016) Steps Towards

Fuzz Testing in Agile Test Automation. International Journal

of Secure Software Engineering, Volume 7 Issue 1, January

2016, pp. 38-52.

Siirtola P & Röning J (2016) Reducing Uncertainty in User-

independent Activity Recognition - a Sensor Fusion-based

Approach International Conference on Pattern Recognition

Applications and Methods, Rome, Italy 24-26 February 2016,

611--619.

Siirtola P, Koskimäki H & Röning J (2016) From User-

independent to Personal Human Activity Recognition Models

Exploiting the Sensors of a Smartphone 24th European Sym-

posium on Artificial Neural Networks, Computational Intelli-

gence and Machine Learning, ESANN 2016., Bruges, Bel-

gium 27-29 April 2016, 471--476.

Siirtola P, Koskimäki H & Röning J (2016) Personal models

for eHealth - improving user-dependent human activity

recognition models using noise injection IEEE Symposium

on Computational Intelligence and Data Mining, December,

accepted.

Siirtola P.; Tamminen S.; Ferreira E.; Tiensuu H.; Prokkola

E.; Röning J. (2016) Automatic Recognition of Steel Plate

Side Edge Shape Using Classification and Regression Models

The 9th Eurosim Congress on Modelling and Simulation,

September.

Tiensuu H, Tamminen S, Pikkuaho A & Röning J (2016)

Improving the yield of steel plates by updating the slab de-

sign with statistical models. Ironmaking and Steelmaking:

Processes, Products and Applications, accepted, August

2016.

Tuovinen L, Ahola R, Kangas M, Korpelainen R, Siirtola P,

Luoto T, Pyky R, Röning J, Jämsä T (2016) Software Design

Principles for Digital Behavior Change Interventions: Les-

sons Learned from the MOPO Study Proceedings of the 9th

International Joint Conference on Biomedical Engineering

Systems and Technologies - Volume 5: HEALTHINF, 175-

182.

Drelon C, Berthon A, Sahut-Barnola I, Mathieu M, Dumontet

T, Rodriguez S, Batisse-Lignier M, Tabbal H, Tauveron I,

Lefrançois-Martinez AM, Pointud JC, Gomez-Sanchez CE,

Vainio S, Shan J, Sacco S, Schedl A, Stratakis CA, Martinez

A, Val P. PKA inhibits WNT signalling in adrenal cortex

zonation and prevents malignant tumour development. Nat

Commun. 2016 Sep 14;7:12751. doi:10.1038/ncomms12751.

PubMed PMID: 27624192; PubMed Central PMCID:

PMC5027289.

Nagy II, Xu Q, Naillat F, Ali N, Miinalainen I, Samoylenko

A, Vainio SJ. Impairment of Wnt11 function leads to kidney

tubular abnormalities and secondary glomerular cystogene-

sis. BMC Dev Biol. 2016 Aug 31;16(1):30. doi:

10.1186/s12861-016-0131-z. PubMed PMID: 27582005;

PubMed Central PMCID:PMC5007805.

Rak-Raszewska A, Vainio S. Nephrogenesis in organoids to

develop novel drugs and progenitor cell based therapies. Eur

J Pharmacol. 2016 Nov 5;790:3-11.

doi:10.1016/j.ejphar.2016.07.011. PubMed PMID: 27395798.

Xu Q, Krause M, Samoylenko A, Vainio S. Wnt Signaling in

Renal Cell Carcinoma. Cancers (Basel). 2016 Jun 17;8(6).

pii: E57. doi: 10.3390/cancers8060057. Review. PubMed

PMID: 27322325; PubMed Central PMCID: PMC4931622.

Vidal V, Sacco S, Rocha AS, da Silva F, Panzolini C,

Dumontet T, Doan TM, Shan J, Rak-Raszewska A, Bird T,

Vainio S, Martinez A, Schedl A. The adrenal capsule is a

signaling center controlling cell renewal and zonation

through Rspo3. Genes Dev. 2016 Jun 15;30(12):1389-94.

doi: 10.1101/gad.277756.116. PubMed PMID: 27313319;

PubMed Central PMCID: PMC4926862.

Halt KJ, Pärssinen HE, Junttila SM, Saarela U, Sims-Lucas

S, Koivunen P, Myllyharju J, Quaggin S, Skovorodkin IN,

Vainio SJ. CD146(+) cells are essential for kidney vascula-

ture development. Kidney Int. 2016 Aug;90(2):311-24. doi:

10.1016/j.kint.2016.02.021. PubMed PMID: 27165833.

Pietilä I, Prunskaite-Hyyryläinen R, Kaisto S, Tika E, van

Eerde AM, Salo AM, Garma L, Miinalainen I, Feitz WF,

Bongers EM, Juffer A, Knoers NV, Renkema KY, Myllyhar-

ju J, Vainio SJ. Wnt5a Deficiency Leads to Anomalies in

Ureteric Tree Development, Tubular Epithelial Cell Organi-

zation and Basement Membrane Integrity Pointing to a Role

in Kidney Collecting Duct Patterning. PLoS One. 2016 Jan

21;11(1):e0147171. doi: 10.1371/journal.pone.0147171.

PubMed PMID: 26794322; PubMed Central PMCID:

PMC4721645.

Prunskaite-Hyyryläinen R, Skovorodkin I, Xu Q, Miinalainen

I, Shan J, Vainio SJ. Wnt4 coordinates directional cell migra-

tion and extension of the Müllerian duct essential for onto-

genesis of the female reproductive tract. Hum Mol Genet.

2016 Mar 15;25(6):1059-73. doi: 10.1093/hmg/ddv621.

PubMed PMID: 26721931; PubMed Central PMCID:

PMC4764189.

Krause M, Samoylenko A, Vainio SJ. Exosomes as renal

inductive signals in health and disease, and their application

as diagnostic markers and therapeutic agents. Front Cell Dev

Biol. 2015 Oct 20;3:65. doi: 10.3389/fcell.2015.00065. Re-

view. PubMed PMID: 26539435; PubMed Central PMCID:

PMC4611857.

Daniel E, Onwukwe GU, Wierenga RK, Quaggin SE, Vainio

SJ, Krause M. ATGme: Open-source web application for rare

Page 23: BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci-ence and biology. In BISG,

INFOTECH OULU Annual Report 2016 23

codon identification and custom DNA sequence optimization.

BMC Bioinformatics. 2015 Sep 21;16:303. doi:

10.1186/s12859-015-0743-5. PubMed PMID: 26391121;

PubMed Central PMCID: PMC4578782.

Bibikova O, Popov A, Bykov A, Prilepskii A, Kinnunen M,

Kordas K, Bogatyrev V, Khlebtsov N, Vainio S, Tuchin V.

Optical properties of plasmon-resonant bare and silica-coated

nanostars used for cell imaging. J Biomed Opt. 2015

Jul;20(7):76017. doi: 10.1117/1.JBO.20.7.076017. PubMed

PMID: 26230637.

Rak-Raszewska A, Hauser PV, Vainio S. Organ In Vitro

Culture: What Have We Learned about Early Kidney Devel-

opment? Stem Cells Int. 2015;2015:959807. doi:

10.1155/2015/959807. Review. PubMed PMID: 26078765;

PubMed Central PMCID: PMC4452498.

Berry RL, Ozdemir DD, Aronow B, Lindström NO, Dudna-

kova T, Thornburn A, Perry P, Baldock R, Armit C, Joshi A,

Jeanpierre C, Shan J, Vainio S, Baily J, Brownstein D, Da-

vies J, Hastie ND, Hohenstein P. Deducing the stage of origin

of Wilms' tumours from a developmental series of Wt1-

mutant mice. Dis Model Mech. 2015 Aug 1;8(8):903-17. doi:

10.1242/dmm.018523. PubMed PMID: 26035382; PubMed

Central PMCID: PMC4527280.

Ali N, Hosseini M, Vainio S, Taïeb A, Cario-André M, Rez-

vani HR. Skin equivalents: skin from reconstructions as

models to study skin development and diseases. Br J Derma-

tol. 2015 Aug;173(2):391-403. doi: 10.1111/bjd.13886. Re-

view. PubMed PMID: 25939812.

Krause M, Rak-Raszewska A, Pietilä I, Quaggin SE, Vainio

S. Signaling during Kidney Development. Cells. 2015 Apr

10;4(2):112-32. doi: 10.3390/cells4020112. Review. PubMed

PMID: 25867084; PubMed Central PMCID: PMC4493451.

Naillat F, Yan W, Karjalainen R, Liakhovitskaia A,

Samoylenko A, Xu Q, Sun Z, Shen B, Medvinsky A, Quag-

gin S, Vainio SJ. Identification of the genes regulated by

Wnt-4, a critical signal for commitment of the ovary. Exp

Cell Res. 2015 Mar 15;332(2):163-78. doi:

10.1016/j.yexcr.2015.01.010. PubMed PMID: 25645944.

Rajaram RD, Buric D, Caikovski M, Ayyanan A, Rougemont

J, Shan J, Vainio SJ, Yalcin-Ozuysal O, Brisken C. Proges-

terone and Wnt4 control mammary stem cells via myoepithe-

lial crosstalk. EMBO J. 2015 Mar 4;34(5):641-52. doi:

10.15252/embj.201490434. PubMed PMID: 25603931;

PubMed Central PMCID: PMC4365033.

Junttila S, Saarela U, Halt K, Manninen A, Pärssinen H,

Lecca MR, Brändli AW, Sims-Lucas S, Skovorodkin I, Vain-

io SJ. Functional genetic targeting of embryonic kidney

progenitor cells ex vivo. J Am Soc Nephrol. 2015

May;26(5):1126-37. doi: 10.1681/ASN.2013060584. Pub-

Med PMID: 25201883; PubMed Central PMCID:

PMC4413750.

Naillat F, Veikkolainen V, Miinalainen I, Sipilä P, Poutanen

M, Elenius K, Vainio SJ. ErbB4, a receptor tyrosine kinase,

coordinates organization of the seminiferous tubules in the

developing testis. Mol Endocrinol. 2014 Sep;28(9):1534-46.

doi: 10.1210/me.2013-1244. PubMed PMID: 25058600.

Rymer C, Paredes J, Halt K, Schaefer C, Wiersch J, Zhang G,

Potoka D, Vainio S, Gittes GK, Bates CM, Sims-Lucas S.

Renal blood flow and oxygenation drive nephron progenitor

differentiation. Am J Physiol Renal Physiol. 2014 Aug

1;307(3):F337-45. doi: 10.1152/ajprenal.00208.2014. Pub-

Med PMID: 24920757; PubMed Central PMCID:

PMC4121567.

Maezawa Y, Onay T, Scott RP, Keir LS, Dimke H, Li C,

Eremina V, Maezawa Y, Jeansson M, Shan J, Binnie M,

Lewin M, Ghosh A, Miner JH, Vainio SJ, Quaggin SE. Loss

of the podocyte-expressed transcription factor Tcf21/Pod1

results in podocyte differentiation defects and FSGS. J Am

Soc Nephrol. 2014 Nov;25(11):2459-70. doi:

10.1681/ASN.2013121307. PubMed PMID: 24904088;

PubMed Central PMCID: PMC4214535.

Pietilä I, Vainio SJ. Kidney development: an overview.

Nephron Exp Nephrol. 2014;126(2):40. doi:

10.1159/000360659. Review. PubMed PMID: 24854638.

Halt K, Vainio S. Coordination of kidney organogenesis by

Wnt signaling. Pediatr Nephrol. 2014 Apr;29(4):737-44. doi:

10.1007/s00467-013-2733-z. Review. PubMed PMID:

24445433; PubMed Central PMCID: PMC3928513.

Prunskaite-Hyyryläinen R, Shan J, Railo A, Heinonen KM,

Miinalainen I, Yan W, Shen B, Perreault C, Vainio SJ. Wnt4,

a pleiotropic signal for controlling cell polarity, basement

membrane integrity, and antimüllerian hormone expression

during oocyte maturation in the female follicle. FASEB J.

2014 Apr;28(4):1568-81. doi: 10.1096/fj.13-233247. PubMed

PMID: 24371124.

Cheddad A, Nord C, Hörnblad A, Prunskaite-Hyyryläinen R,

Eriksson M, Georgsson F, Vainio SJ, Ahlgren U. Improving

signal detection in emission optical projection tomography

via single source multi-exposure image fusion. Opt Express.

2013 Jul 15;21(14):16584-604. doi: 10.1364/OE.21.016584.

PubMed PMID: 23938510.