Alexander Vodyaho & Nataly Zhukova — Implementation of Agile Concepts in Recommender Systems for...

Implementation of Agile Concepts in Recommender Systems for Data

Processing and Analyses

Alexander Vodyaho,Nataly Zhukova

St. Petersburg Electrotechnical University “LETI”

E-mail: [email protected]

AIST-2015, April, 9-11, Yekaterinburg

САНКТ-ПЕТЕРБУРГСКИЙ ГОСУДАРСТВЕННЫЙ ЭЛЕКТРОТЕХНИЧЕСКИЙ УНИВЕРСИТЕТ «ЛЭТИ»

Data Processing Applications Current State

Constantly increasing number and complexity of the solved problems in the sphere of data processing

Algorithms are developed by various groups of researchers all over the world using various tools and without taking into account any standards

Constantly increasing requirements to accuracy, reliability and operativeness of data processing

Algorithms are open source and can be modified

Lack of time, financial and human resources for applications development

Specialized features of processed data require comprehensive testing of the developed algorithms


SPbETU «LETI» www.eltech.ru

Outline

Introduction

Problem statement

Agile concepts for IS and DPAS

Description of the developed ontologies and their relations

Description of telemetric information processing application

Case study

Conclusion

1

2

3

4

5

6

7



Recommender Systems



Problem statement

Requirements to the DPA RS

have to make suggestions for selecting and configuring algorithms using available information and knowledge about the subject domain of data processing and applied subject domains

have to consider information about solved problems, characteristics of analyzed objects, features of input data, parameters of the data processing systems

Required means and tools of data processing systems

means and tools for building knowledge-based descriptions of the algorithms and defining relations with the objects of the subject domain of data processing;

tools for interpreting logical rules that define conditions of algorithms efficient application and other conditions;

tools for building sequences of algorithms;

procedures for estimating the formed results;

storages of the detailed information about results of algorithms application ;

modeling tools for estimating alternatives;

Jena



Information and Knowledge of the Subject Domain of Measurements Processing

User

Human User

Subject domain expert

Common User

Machine User

Data processing Application

External Knowledge

Base Application

Providing information and knowledge

Creation and assessment of information and

knowledge

Using result of data processing

New data processing and analyses

Retrieving new knowledge from historical data

Solving complicated task of data processing

Solving complicated specialized task

Extending available knowledge and

information

Building, Improving and estimating ontology

Application of knowledge for solving

task of data processing

Exchanging information and knowledge in standard formats

Receiving information and knowledge



Features of the subject domain of measurements processing

• Structured binary streams, time series or separate measurements

Initial measurements

• Huge volume, bad quality, heterogeneity, distribution in time and space, non stationary behavior of time series, multiple complicated relations

Measurements features

• Formalized knowledge about measurements

Requirements to the results

• Results of data processing are used for solving tasks at the level of objects, situations and for decision making support

Consumers



Basic agile concepts

Analyses

Design

ImplementationTesting

Evaluation



Agile Сoncepts for DPAS

DPA RS

DPAS

DesignDevelopment

(methodological aspect)

Development (implementation

aspect)Support

First level of agile features support (industrial level)

Second level of agile features support

(research-oriented level level)

Ready methodological

solutions

Ready technological

solutions

Information Systems (IT

sphere)

Scientific prototypes

Base level of agile features support

(IT level)

Third level of agile features support

(research level)

Ready technological

solutions

Execution

Suggestions for new algorithms

Ready implementation of algorithms

Ready technological

solutions

Ready technological

solutions

Life cycle Life cycle

System Agility support



Common and Agile Features of DPA RS

Agile features and D

PA

RS

technologies lead to new problem

sA

gile features and DP

A R

S technologies lead to new

problems

Features are based on technologies Features are based on technologies

Technologies for DPA RSTechnologies for DPA RS

Technologies for RS Technologies for DPA RS

Content-based approach

Collaborative filtering

Hybrids

Knowledge-based approach

Logical inference

Experience-based approach

Exploration analyses

DPA RS features DPA RS features

RS features Agile features

Capability to process huge amounts of data

Capability to make suggestions

Ranging capabilities

Easy integration of new methods and algorithms

Easy development of new methods and algorithms

Easy extension of data processing and analyses systems business logic

DPA RS problems DPA RS problems

Easy integration of new methods and algorithms

Easy development of new methods and algorithms

Easy extension of data processing and analyses systems business logic

Low cost of design, development and support

Short time of design, development and support

Convenient working space



DPA RS Information Model

Model of the applied subject domain of data processing

and analyses

Model of the applied subject domain of data processing

and analyses

DPAS dynamic information model

Information model of the environmentInformation model of the environment

Subject domain modelsSubject domain models

DPAS information modelDPAS information model

RS DPAS information modelRS DPAS information model

DPAS satic information modelDPAS satic information model

Model of the applied subject domain

Model of the applied subject domain

DPAS RS dynamic information model DPAS RS static information modelDPAS RS static information model

<<inherits>> <<inherits>>

<<inherits>> <<inherits>> <<inherits>>

<<uses>>

<<uses>>

<<inherits>>

<<uses>>



DPA RS Architecture

RS GUI tools GUI Processes management toolOntologies and knowledge bases editors

RS tasks manager

RS content manager

Ontologies

Service of mathematical and modeling libraries

Service for external connections management

and support

Data, information and knowledge visualization

service

Processes management and execution service

Network

Nework

Knowledge bases

Data, information and knowledge sever

Data bases File storage

Data, information and knowledge

Backend

Administrative serviceData processing and

analyses service

Data processing and analyses tools

Frontend

Inference machine

RS ontologies

RS knowledge bases

RS data, information and knowledge manager

DPAS componentsDPAS componentsDPA RS componentsDPA RS components

Research-oriented services



Case study

The system has the aim to analyze and control structure and contents of the binary streams received from space objects.

Example of the binary steams structure



General procedure for the binary streams processing and analyses

Apply methods based on calculation of the frequency distribution of the streamsApply methods based on calculation of the frequency distribution of the streams

Binary streams

Compare the descriptions with the descriptions of the earlier received streamsCompare the descriptions with the descriptions of the earlier received streams

Preliminary streams descriptions

Similar streams are found?

Restore the length of the cards and words, subcommutation and supercommutation of the parameters using methods of correlation analyses

Restore the length of the cards and words, subcommutation and supercommutation of the parameters using methods of correlation analyses

The structure needs improvements?

Build and analyze the graphs that represent the structure of the streamsBuild and analyze the graphs that represent the structure of the streams

Streams descriptions





GUI of program complexes for processing stream structure


Experimental results

Parameters of the stream

98 80 91,3 90,1

92 40 90,2 92,4

98 78 93 89,5

80 20 72,3 71,7

wL Sub SupfL

%60|| Sub

%60|| Sup

%5DP

%35SVP

%40FVP

%10FPP

%10OP

%5DP

%35SVP

%10FVP

%40FPP

%10OP



Contacts

Contact information:

Nataly Zhukova

5, Prof. Popova str, St. Petersburg, 197376

E-mail:[email protected]:+7 812 346-46-41

Fax:+7 812 346-46-41



Alexander Vodyaho & Nataly Zhukova — Implementation of Agile Concepts in Recommender Systems for...

Science

Transcript of Alexander Vodyaho & Nataly Zhukova — Implementation of Agile Concepts in Recommender Systems for...