The Software Matrix An Architecture for Software Salvage

107
The Software Matrix An Architecture for Software Salvage Riddhiman Ghosh Advisor: Dr. James Fawcett Master of Science Thesis Dept. of Electrical Engineering and Computer Science, Syracuse University December 15, 2004

description

The Software Matrix An Architecture for Software Salvage. Riddhiman Ghosh Advisor: Dr. James Fawcett. Master of Science Thesis Dept. of Electrical Engineering and Computer Science, Syracuse University December 15, 2004. Software Salvage. - PowerPoint PPT Presentation

Transcript of The Software Matrix An Architecture for Software Salvage

Page 1: The Software Matrix An Architecture for Software Salvage

The Software MatrixAn Architecture for Software Salvage

Riddhiman GhoshAdvisor: Dr. James Fawcett

Master of Science ThesisDept. of Electrical Engineering and Computer Science, Syracuse UniversityDecember 15, 2004

Page 2: The Software Matrix An Architecture for Software Salvage

2

Software Salvage

Salvage: lifting of a significant block of existing systems and inserting them into a newly developed system.

Page 3: The Software Matrix An Architecture for Software Salvage

3

Overview Our work centers on significantly managing the problem of

software salvage. We propose an architecture for system construction to

simplify salvage and use message passing and mediator structures.

Message passing has been used before to communicate between processes and machines.

Our contribution is to show that Mediated Queued Message Passing at the module level is an effective way to support software salvage

Adding the mediator structure makes assembling a new application with salvaged parts much easier

Our work shows we gain a lot of flexibility at an acceptable performance cost.

Page 4: The Software Matrix An Architecture for Software Salvage

4

Introduction Approximately 100 billion lines of source code

at work in the world today. Large fractions of code in systems are

functionally equivalent. However systems are often constructed without

leveraging existing code bases – less than 15% of new code serves an original purpose.

“Reinvention of the wheel” in the software industry.

Page 5: The Software Matrix An Architecture for Software Salvage

5

Introduction Software construction is needlessly error-

prone and expensive.

– We’re maintaining multiple copies of essentially the same software.

– Time and cost of developing, testing and documenting a piece of software is multiplied by the number of equivalent copies in existence.

Page 6: The Software Matrix An Architecture for Software Salvage

6

Software Reuse Study of reuse – recycling of software

assets – has been an important branch in the software engineering discipline.

Effective reuse promises– Reduced development and maintenance costs– Gains in development schedule and quicker

time-to-market– Increased robustness and quality

Not a new idea.

Page 7: The Software Matrix An Architecture for Software Salvage

7

Software Reuse Systematic Reuse: “an institutionalized

organizational approach to product development in which software assets are intentionally created or acquired to be reusable”

However, few organizations practice systematic reuse, in spite of a recognition of its potential benefits.

Reuse is hard!

Page 8: The Software Matrix An Architecture for Software Salvage

8

Gap between “theory” and “practice” in Reuse

“ Not all the “theoretical oriented” and “sophisticated” solutions presented by researcher[s] have ...thrilled the practitioners... – Zand, M. ...the reuse community has worked on complex technologies and methods with high ceremony, yet most of the software community seems to be looking for simpler solutions... High ceremony methods require an organization with high process maturity to achieve success. – Griss, M. ”

Page 9: The Software Matrix An Architecture for Software Salvage

9

Software Salvage What we seen in the industry is not

necessarily software reuse, but rather “software salvage”

Salvage: lifting of a significant block of existing systems and inserting them into a newly developed system.

Radar Systems Department, General Electric Company (Syracuse, NY) routinely attempted salvage in the building of a new radar.

Page 10: The Software Matrix An Architecture for Software Salvage

10

Reuse vs. Salvage We make a distinction between the terms

software reuse and software salvage. Reuse connotes immutability

– the individual pieces meant for reuse cannot be modified; they’re designed and implemented to be adaptable but with no intent to change even a single character of source code.

Salvage makes no guarantee of immutability– very often source code of individual pieces being

salvaged is modified.

Page 11: The Software Matrix An Architecture for Software Salvage

11

Problem

Effective salvage is difficult to accomplish– The large pieces we want often have many

dependencies on parts we don’t want

– Requires expensive changes

– Think of salvage as analogous to transplanting the heart from one living organism to another! Similar problems, when we pull out a part from a software system due to the connectedness inherent in typical software.

Page 12: The Software Matrix An Architecture for Software Salvage

12

Problem

It is difficult to salvage existing parts of systems and build new systems from them, with ease.

Page 13: The Software Matrix An Architecture for Software Salvage

13

Goal Enable leveraging of major parts of

systems

Eliminate or reduce changes required by salvage so that salvage moves closer to the reuse model (immutability), without requiring high-process organization.

Simplify software salvage to make it a useful paradigm.

Page 14: The Software Matrix An Architecture for Software Salvage

14

Prior Approaches Two major (and fairly recent) approaches

towards encouraging reuse have been

– object-oriented reuse

– component-oriented reuse.

Page 15: The Software Matrix An Architecture for Software Salvage

15

Object-Oriented Reuse Reuse has been one of the classic

motivations of object-orientation

Object oriented theory – mandated discipline in writing code through

best-practice guidelines such as separating interface from implementation

– Encouraged reuse through inheritance, composition, parameterization

Page 16: The Software Matrix An Architecture for Software Salvage

16

Object-Oriented Reuse However OO technologies didn’t quite

engender the reuse revolution that was hoped for.– Only OO libraries widely used are user-interface

frameworks (such as MFC) and libraries for data structures.

– OO techniques have made compiler libraries an effective means of reuse but business organizations have had a harder time getting leverage from their pre-existing software assets only through the creation and consumption of OO libraries.

Page 17: The Software Matrix An Architecture for Software Salvage

17

Object-Oriented Reuse The definition of an object is purely “technical”

– Defined as an encapsulation of state and behavior– No direct mapping to the physical unit that is

actually deployed, versioned and potentially reused. Objects don’t fit the “salvage” model well

– In terms of granularity, they are of an inappropriate size to be mixed and matched.

– They are the size of grains of sand, while what salvagers are typically looking for in building systems, are bricks.

Page 18: The Software Matrix An Architecture for Software Salvage

18

Component-Oriented Systems The logical next step was the notion of

components– a coherent package of software

implementation that can be independently deployed and composed with other components.

– Defining feature: system can be made better by only updating components

– of coarser granularity than objects– independent and deployable implies

executable or loadable code

Page 19: The Software Matrix An Architecture for Software Salvage

19

Component-Oriented Systems There are several component technology

offerings– COM, CORBA, JavaBeans/EJB, .NET Components

Adopting a component-oriented approach (rather than only concentrating on programming language as in OO theory) is beneficial.

But the existence of these component technologies has not solved the problems that plague software salvage, discussed earlier

Packaging techniques alone are not enough

Page 20: The Software Matrix An Architecture for Software Salvage

20

Our Approach We are of the view that components are

useful only if there is a framework that actively supports and promotes their reuse.

.NET and J2EE have framework support, however it is focused on generic industry problems (network communication, web publishing, application security, etc.)

We seek to support vertical, perhaps proprietary, applications, where building a huge framework is impractical from a return on investment point of view.

Page 21: The Software Matrix An Architecture for Software Salvage

21

Our Approach Make salvage easier by

– viewing applications as compositions of different pieces

– having a framework of collaborating pieces from which applications can be composed dynamically

Try to achieve benefits of the “Software-IC” model— a plug-and-play approach to software construction.

Page 22: The Software Matrix An Architecture for Software Salvage

22

Our Approach

We are limiting the hard problem of general reuse to a smaller domain—of salvage within an organization, to be used in the construction of modest-sized systems (about 200,000 lines of code).

To address this domain would be to provide solutions of value to the small and medium-sized software shops that have been resistant to adopt systematic reuse.

Page 23: The Software Matrix An Architecture for Software Salvage

23

Software Matrix The Software Matrix is a framework that actively

supports and promotes the salvage of components.

We focus on the reuse of major blocks of code rather than low-level functionality

By employing message-passing and mediator structures and by supporting the discovery of needed types, we’ve built a pluggable architecture that can gracefully adapt to salvage operations.

Page 24: The Software Matrix An Architecture for Software Salvage

24

Software Matrix

In particular, the Matrix is a runtime infrastructure that acts as a substrate into which individual pieces of an application – different blocks of code – can plug.

Page 25: The Software Matrix An Architecture for Software Salvage

25

Software Matrix

From these “plugged-in” individual pieces the Matrix dynamically composes applications.

Page 26: The Software Matrix An Architecture for Software Salvage

26

Software Matrix This infrastructure was named the Software

Matrix in order to connote a very structured pattern of building software.

System elements are embedded in this substrate – a matrix /collection – through which applications are dynamically composed.

The image of endless banks of incubators of humans in a matrix, as from “The Matrix” motion picture, isn’t entirely unintentional, if somewhat flippant.

Page 27: The Software Matrix An Architecture for Software Salvage

27

Cells

• The individual blocks of code that plug-in to the Matrix are called “Cells” and are the building blocks out of which applications are built.

Page 28: The Software Matrix An Architecture for Software Salvage

28

Cells

Sub-routine

Object

Component

Cell • Cells represent the

unit of composition and reuse in our system.

Page 29: The Software Matrix An Architecture for Software Salvage

29

Cells

An application is built through the collaboration of Cells

Cells communicate with each other strictly through messages (we use XML encoded messages)

Page 30: The Software Matrix An Architecture for Software Salvage

30

Cells

One of the problems of salvaging: – extracting parts of monolithic (or very

tightly-coupled) applications.

We are insisting on loose coupling– Enforce a separation of concern between

Cells by inserting a message bus between them

Page 31: The Software Matrix An Architecture for Software Salvage

31

Message Passing Advantages of Message Passing Interface (MPI) over

Procedure Call Interface (PCI)– MPI is a universal interface. All entities using MPI use

something equivalent to GetMessage() and PostMessage(msg). So any Cell providing MPI is guaranteed to be plug-compatible with another Cell accepting MPI

– Messages are self describing command and state carriers. If a Cell does not understand all elements sent to it, its free to ignore what it does not understand. PCI has no such support.

– Messages can be easily intercepted and inspected. Therefore MP systems are easier to debug

– Message Passing makes it easier to use the Mediator pattern, since the Mediator does not need to know a lot of interfaces, just the small MPI.

Page 32: The Software Matrix An Architecture for Software Salvage

32

Mediator Pattern The use of the Mediator pattern is significant in the Software

Matrix About the Mediator pattern Gamma et al. say:

Thus, proliferating interconnections between different partitions of a system reduces the potential for salvage. The Matrix acts as a mediator – the different Cells only know about the mediator.

“Mediator promotes loose coupling by keeping objects from referring to each other explicitly, and it lets you vary their interaction independently. …lots of interconnections make it less likely that an object can work without the support of others”.

Page 33: The Software Matrix An Architecture for Software Salvage

33

A closer look at Cells…

Message Queue

ID

Capability List

Functionality

Every Cell contains:– a message queue:

holds request and response messages during collaboration with other cells.

Page 34: The Software Matrix An Architecture for Software Salvage

34

A closer look at Cells…

Message Queue

ID

Capability List

Functionality

Every Cell contains:– a capability list:

used to advertise the capabilities of a cell to other cells, via the Matrix. (E.g. SU.Math.Convolution)The capability list is used by the Matrix in order to discover the right cells for system construction.

Page 35: The Software Matrix An Architecture for Software Salvage

35

A closer look at Cells…

Message Queue

ID

Capability List

Functionality

Every Cell contains:– a globally unique

identifier (GUID):Cells can be uniquely identified using a GUID. This is used by the Matrix for several operations such as discovery and registration.

Page 36: The Software Matrix An Architecture for Software Salvage

36

A closer look at Cells…

Message Queue

ID

Capability List

Functionality

Every Cell contains:– functionality:

Cells also contain the functionality that allows them to be considered as software assets with potential for reuse.

Page 37: The Software Matrix An Architecture for Software Salvage

37

A closer look at Cells…

All cells subscribe to a common protocol (ICell) that specifies how to:– register and un-register

with the Matrix– advertise capabilities– send and accept messages– collaborate with other cells

(could be synchronous, asynchronous or one-way)

ICell

MyCell

Page 38: The Software Matrix An Architecture for Software Salvage

38

A closer look at Cells…

Every cell also has an entry-point (start), and is given a chance to execute once it is plugged-in.

This entry-point (empty/non-empty) decides whether a cell will be only a passive ‘server’, or itself actively seek collaboration from other cells.

Page 39: The Software Matrix An Architecture for Software Salvage

39

Example Sample application: needs to read data samples

from an input file, perform a signal processing operation on the data (e.g., filtering), plot the results of the operations on the system display, and log the results to a file.

The major pieces of the application would be responsible for: – file operations– signal processing– graphical plotting

Page 40: The Software Matrix An Architecture for Software Salvage

40

Example These would be written as cells and plugged-in to

the Matrix. The Matrix would then assume the responsibility of

constructing the application from these individual cells.

In order to perform its task a cell may need services of another cell. But cells do not explicitly bind to other cells – they only specify what message type they need handled, and the Matrix discovers cells capable of handling that message.

If no suitable cell found, a “not supported” message is generated.

Page 41: The Software Matrix An Architecture for Software Salvage

41

Example

Page 42: The Software Matrix An Architecture for Software Salvage

42

Example

Matrix Cell 1 Cell 2 Cell 3 Cell 4 Cell 5

start

send

Cell 1 SU.FileOperations.Read

discoverrequest

response

SU.FileOperations.Readrequest

sendSU.DSP.PeakingFilter request

discover

Cell 1 SU.DSP.PeakingFilter

request

response

sendSU.Math.Convolution request

discover

Cell 2 SU.Math.Convolution

request

response

sendSU.Graphics.PlotBarChart request

request

discover

Cell 1 SU.Graphics.PlotBarChart request

responsesend

SU.FileOperations.Write request

discover

Cell 1 SU.FileOperations.Write

request

response

Sequence of events in the construction of the sample application

Page 43: The Software Matrix An Architecture for Software Salvage

43

Example We see here:

Dynamic composition – we are building a system from pieces that exist on the Matrix at runtime. Matrix takes care of compositional aspects (as opposed to only computational aspects) of software.

Matrix automatically connects the right pieces at run-time without having to bind to anything explicitly at compile-time.

Software is now amenable to salvage operations. The very same Cells could be used to build other applications.

Page 44: The Software Matrix An Architecture for Software Salvage

44

How do cells “plug-in”? Cells are implemented as “plug-in” modules,

and are realized using .NET components.

The Matrix uses the ‘reflection’ and ‘late-binding’ (Fusion) features of the .NET framework to discover and register plug-ins.

Given an .NET assembly, the Matrix will reflect over the contained types to determine which of them implement the ICell interface in order to recognize valid plug-ins.

Page 45: The Software Matrix An Architecture for Software Salvage

45

How do cells “plug-in”?Matrix

Monitor plug-in directory, reflect over code packages

Cell A Cell B Cell C

Register Cell

Register Cell

GUID

GUID

GUID

. . . . .

Register Cell

Throughout its lifetime, the Matrix monitors the plug-in directory, in order to discover new cells

Valid cells are ‘registered’ and available for system construction.

Page 46: The Software Matrix An Architecture for Software Salvage

46

Steps to take advantage of the Matrix

In order to enable salvage, pieces of an application (at the time of writing it, or existing pieces) are wrapped in a Cell.

Create a wrapper that inherits from ICell (e.g. we wish to create FileManager Cell responsible for common file operations )

FileManager : ICell{ ...}

Page 47: The Software Matrix An Architecture for Software Salvage

47

Steps to take advantage of the Matrix

The Capability List of this

FileManager Cell is then populated to indicate the types of messages it is capable of

handling. CapabilityList.Add(“SU.FileManager.Files.Read”);

CapabilityList.Add(“SU.FileManager.Files.Write”);CapabilityList.Add(“SU.FileManager.Files.Search”);CapabilityList.Add(“SU.FileManager.Files.Compression”);

Page 48: The Software Matrix An Architecture for Software Salvage

48

Steps to take advantage of the Matrix

Override the “process” method to

add appropriate message processing – Basically you specify what is to be

done in response to a particular message type – delegate calls to your implementation.

Page 49: The Software Matrix An Architecture for Software Salvage

49

Steps to take advantage of the Matrix

• Compile, and copy the resulting

binary into the “plug-in” directory of the Software Matrix. The Matrix will automatically detect and register the cell, and it will be available for composition.

Page 50: The Software Matrix An Architecture for Software Salvage

50

Steps to take advantage of the Matrix

• If a cell wishes to use other cells (or is a program executive for instance), it will probably say:...Result = syncSend("SU.FileManager.Files.Search",Params);...

• Here it is trying to locate a cell that can handle the named message type.

Page 51: The Software Matrix An Architecture for Software Salvage

51

Features Fine-grained message-passing

– We are using message passing at a much finer level of granularity than is normally seen

– The Matrix requires messages be the only mode of collaboration between different parts of an application

– Decreases degree of coupling – critical to the success of salvage operations.

Page 52: The Software Matrix An Architecture for Software Salvage

52

Features Dynamic Composition

– The Matrix takes care of the compositional aspects of software by automatically discovering and connecting the right pieces and building an application at runtime

– By adding a few more cells into the Matrix if needed, we can build new applications by reusing existing cells.

Page 53: The Software Matrix An Architecture for Software Salvage

53

Features Support for System Evolution

– Appropriate cell to serve a particular message type is selected at runtime

– Evolving requirements can be accommodated easily by modifying only those cells that represent the affected part of the system.

– Easy to field-replace cells – only copy the new cell over the old cell in the plug-in directory.

– Effective way to support program maintainability, bug fixes/upgrades.

Page 54: The Software Matrix An Architecture for Software Salvage

54

Features Simplicity

– The Matrix is a supporting infrastructure that is lightweight (bare infrastructure is roughly only 2000 lines of code)

– Simple to use. The infrastructure comes with documented full source (if needed) and sample applications.

– The underlying technology used is the .NET component model – less complex as compared to other models such as COM.

Page 55: The Software Matrix An Architecture for Software Salvage

55

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm

High-level partitioning of the Software Matrix

Page 56: The Software Matrix An Architecture for Software Salvage

56

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)

Page 57: The Software Matrix An Architecture for Software Salvage

57

Assessment The Software Matrix was used in the construction of

“real world” tools:– File Synchronizer– NetView

An assessment of our technique was made based on qualitative and quantitative factors such as:– ease of salvage– cognitive distance– lines of source code– performance

Built using the “Matrix” way and traditional way.

Page 58: The Software Matrix An Architecture for Software Salvage

58

File Synchronizer Application

File Synchronizer is a useful tool that allows for remote directory synchronization

Page 59: The Software Matrix An Architecture for Software Salvage

59

File Synchronizer Application What does it do?

– Allows contents of a directory of one machine to match that of a specified directory on another machine over the network/Internet.

– Always have the latest versions of a specified set of files -- Newer versions update older versions

– Files present on one side and not on the other are copied

– Older files will not overwrite newer files without explicit user permission

File Synchronizer has a peer-to-peer architecture– instances running on different machines are exactly

identical and can serve in both client and server roles.

Page 60: The Software Matrix An Architecture for Software Salvage

60

Building File Synchronizer The Matrix is concerned with building blocks of

applications rather than low-level pieces of functionality

For this application 3 major blocks were identified– file upload and download block– file synchronization policy block– user-interface

Each of these 3 blocks was built into an independent Cell with appropriate functionality.

FileTransfer Synchronizer UI

Page 61: The Software Matrix An Architecture for Software Salvage

61

Building File Synchronizer Capability lists of the different Cells were

decided uponSU.FileTransfer.GetLocalDirsFilesSU.FileTransfer.RunServerSU.FileTransfer.Connect SU.FileTransfer.GetRemoteDirsFilesSU.FileTransfer.UploadLocalFile SU.FileTransfer.DownloadRemoteFile

SU.Synchronizer.Policy.Compare

Null

FileTransfer Cell UI Cell

Synchronizer Cell

Page 62: The Software Matrix An Architecture for Software Salvage

62

Building File Synchronizer The Cells were

built by incorporating wizard generated “boiler-plate” code for a generic Cell, and Cell–specific implementation code.

Cell

Wizard generated code

Provide Cell-specific implementation

Page 63: The Software Matrix An Architecture for Software Salvage

63

Building File Synchronizer The resulting

binaries upon compilation were dropped into the Matrix plug-in directory

Plug-ins directory

Page 64: The Software Matrix An Architecture for Software Salvage

64

Building File Synchronizer

The Matrix discovers and composes the right pieces to form the File Synchronizer application.

Matrix Admin Console

File Synchronizer Application

Page 65: The Software Matrix An Architecture for Software Salvage

65

Evaluation: Cognitive Distance Cognitive Distance

– amount of intellectual effort needed to understand and adopt a new technique/methodology

Cognitive distance of the Software Matrix approach is moderate– steps required are neither to numerous nor too

complicated for the average developer with a fair understanding of the platform/language being used.

Page 66: The Software Matrix An Architecture for Software Salvage

66

Evaluation: SLOC Source Lines of Code (SLOC)

– The File Synchronizer application was built both the “traditional” way, and the Matrix way.

– In terms of SLOC we notice an increase in the Matrix version of File Synchronizer

– However most of the increase constitutes “boiler-plate” code required by the infrastructure, which the developer does not write

– Effective increase in SLOC is marginal not very large (1729 vs. 1526)

Page 67: The Software Matrix An Architecture for Software Salvage

67

Source File TotalSLOC

ChangeConfirmation.cs 261FileTransfer.cs 368TreeViewExplorer.cs 75UI.cs 822Total 1526

Source File Total SLOC

SLOC less wizard code

FileTransferPeer.cs 452 257FileTransfer.cs 168 168TreeViewExplorer.cs 79 79UI.cs 985 790ChangeConfirmation.cs 234 234Node.cs 80 80Synchronizer.cs 316 121Total 2314 1729

Traditional File Synchronizer

Matrix File Synchronizer

Evaluation: SLOC

Page 68: The Software Matrix An Architecture for Software Salvage

68

Evaluation: Performance In employing the Software Matrix supporting

infrastructures we anticipate a performance impact on applications– The “traditional” and “Matrix” versions of the File

Synchronizer were compared to see how they fared in terms of performance

No perceptible difference while running the 2 versions from a user’s perspective

To quantify the performance difference common tasks in the Synchronizer were timed using OS high-resolution timers.

Page 69: The Software Matrix An Architecture for Software Salvage

69

File Size Application µs µs µs µs µs

Avg(ms)

512 B Traditional File Synchronizer 23749 23186 24089 25085 24511 24

Software Matrix File Synchronizer 33510 31345 29040 29036 28933 30

1 KB Traditional File Synchronizer 30546 25439 26900 24625 26008 27

Software Matrix File Synchronizer 27400 31043 28744 30657 31091 30

2 KB Traditional File Synchronizer 27383 25942 25083 28363 27874 27

Software Matrix File Synchronizer 32156 31245 42399 32025 29613 33

4 KB Traditional File Synchronizer 28971 33569 26922 26085 27051 29

Software Matrix File Synchronizer 33407 29847 34121 36220 33198 33

8 KB Traditional File Synchronizer 28050 28037 29482 29644 28997 29

Software Matrix File Synchronizer 37271 37373 36055 35443 36722 37

16 KB Traditional File Synchronizer 27364 27792 31073 31185 28414 29

Software Matrix File Synchronizer 36086 36154 40061 40438 38444 38

32 KB Traditional File Synchronizer 30263 34870 32966 33374 35854 33

Software Matrix File Synchronizer 42243 40392 40052 40197 40192 41

64 KB Traditional File Synchronizer 33401 31481 35427 33863 30715 33

Software Matrix File Synchronizer 44499 41803 44969 43757 42634 44

Timing remote file transfer

Page 70: The Software Matrix An Architecture for Software Salvage

70

Timing remote file transferFile Size Application µs µs µs µs µs

Avg(ms)

128 KB Traditional File Synchronizer 36313 34334 38538 35821 38726 37

Software Matrix File Synchronizer 50476 48506 52670 52060 47213 50

256 KB Traditional File Synchronizer 41855 40845 42391 45852 49748 44

Software Matrix File Synchronizer 61082 61340 66317 68880 52989 62

512 KB Traditional File Synchronizer 63793 66424 63992 63604 67101 65

Software Matrix File Synchronizer 76199 74739 74177 74044 75412 75

1 MB Traditional File Synchronizer 90762 90185 101821 116057 116019 103

Software Matrix File Synchronizer 110680 113260 128165 106651 111242 114

2 MB Traditional File Synchronizer 212977 219673 209435 207322 202039 210

Software Matrix File Synchronizer 226795 231194 205266 251471 202615 223

4 MB Traditional File Synchronizer 407272 390905 383886 390538 398694 394

Software Matrix File Synchronizer 391203 415990 407826 411143 419071 409

8 MB Traditional File Synchronizer 857204 817551 853566 806387 866713 840

Software Matrix File Synchronizer 810735 1170234 838999 851975 902344 915

Page 71: The Software Matrix An Architecture for Software Salvage

71

Evaluation: Performance The timing values of the Matrix File Synchronizer

is only marginally higher in most cases– On an average 20% higher– Cells with low processing requirements may suffer

higher overhead

Performance impact of using the Software Matrix should usually not be prohibitive.

Tradeoff here between increased productivity and ease of salvage, vs. slightly better performance using traditional approach.

Page 72: The Software Matrix An Architecture for Software Salvage

72

Evaluation: Ease of salvage We pull out approximately one-third of the File

Synchronizer application, without any “surgery” and use it through a minimalist interface.

This interface will exercise the upload/download functionality of File Synchronizer.

We simply– copy the Matrix infrastructure to a suitable location– drop the upload/download Cell into the plug-in directory– write the Cell to send messages the extracted part, and

drop it into plug-in directory

Page 73: The Software Matrix An Architecture for Software Salvage

73

Evaluation: Ease of salvage

With a mere 25 lines of developer code we were able to extract a major chunk of an existing application, in this case 30%, with ease – a successful salvage operation.

Page 74: The Software Matrix An Architecture for Software Salvage

74

NetView Application NetView is a

simple conferencing tool that allows users to drive presentations, slide shows or similar content from their computer, on other remote machines over the Internet/network.

Machine A

Machine B

Page 75: The Software Matrix An Architecture for Software Salvage

75

NetView Application• Built by

salvaging from File Synchronizer.

Software Matrix plug-ins directory

Plugging-in existing Cell

Page 76: The Software Matrix An Architecture for Software Salvage

76

NetView Application

NetVIEW Application

Matrix Admin Console

Application composed by the Software Matrix

Page 77: The Software Matrix An Architecture for Software Salvage

77

Comparison with related work

There exists a combination of middleware and component technologies such as COM and CORBA that have reusability as one of their goals.– COM, CORBA are considered highly complex,

over-specified and require heavy-weight supporting infrastructures.

– The Matrix was meant to address the specific problem of salvage and is lightweight (approx. 2000 SLOC) and simple.

Page 78: The Software Matrix An Architecture for Software Salvage

78

Comparison with related work

– Most middleware technologies use the procedure-call model of collaboration. Having components bind to exact function signatures is not flexible enough – leads to the tightly-coupled systems that make salvage operations difficult.

– The Matrix uses message-passing as the mode of collaboration between components. This leads to looser coupling of system elements resulting in potentially easier salvage in the future.

Page 79: The Software Matrix An Architecture for Software Salvage

79

Comparison with related work

– Message Oriented Middleware (MOM) and to a certain extent Web Services also use message-passing (most web services use XML-RPC, though they support a message-passing abstraction)

– The Matrix differs from them in the granularity and scale of message-passing, since we are focused on local compositions of cells (components) to build applications, rather than accessing remote functionality over the network.

Page 80: The Software Matrix An Architecture for Software Salvage

80

Future Work Capabilities of

application building blocks are identified by message types

Useful to have catalogs that list message type handlers available in a particular domain/organization

Would aid developers in salvaging from existing software assets.

Message Catalogs

Versioning Support Schemes

Remote Cell Collaboration

Applications of the Matrix

Usability Studies

Page 81: The Software Matrix An Architecture for Software Salvage

81

Future Work

Message Catalogs

Versioning Support Schemes

Remote Cell Collaboration

Applications of the Matrix

Usability Studies

Two Cells handling exactly the same message type are considered to provide equivalent functionality

Associating versioning information with message types will add more flexibility for the graceful evolution of systems.

Leverage inherent .NET assembly versioning features

Page 82: The Software Matrix An Architecture for Software Salvage

82

Future Work

Message Catalogs

Versioning Support Schemes

Remote Cell Collaboration

Applications of the Matrix

Usability Studies

Matrix aids in composition of systems from locally available Cells

Could be extended – system composition includes Cells present on a remote machine

Page 83: The Software Matrix An Architecture for Software Salvage

83

Future Work

Message Catalogs

Versioning Support Schemes

Remote Cell Collaboration

Applications of the Matrix

Usability Studies

Investigate other scenarios apart from salvage where the Matrix infrastructure can be used

E.g. in regression testing of systems, since the Matrix can act as an interceptor.

Page 84: The Software Matrix An Architecture for Software Salvage

84

Future Work

Message Catalogs

Versioning Support Schemes

Remote Cell Collaboration

Applications of the Matrix

Usability Studies

Unfamiliarity with a new technique is a stumbling block in the way of its adoption

Would be very instructive to conduct a usability study amongst developers to obtain feedback on how the Matrix infrastructure services could be more intuitive/simple.

Page 85: The Software Matrix An Architecture for Software Salvage

85

Conclusion The Software Matrix is a runtime substrate with a plug-in

architecture that enables simpler salvage We’ve applied message-passing at a much finer granularity

than has been done before. We’ve used mediator structures to enforce loose coupling of system elements

We’ve shown that mediated queued message-passing at the module level is an effective way to support software salvage

Our work shows that we gain a lot of flexibility at an acceptable performance cost

In the Software Matrix we’ve used a combination of techniques to solve a problem that has not been solved before – we are significantly managing the problems associated with software salvage

Simplifying software salvage is a worthy goal and will have a positive impact on productivity in the software industry.

Page 86: The Software Matrix An Architecture for Software Salvage

86

End of Presentation

Page 87: The Software Matrix An Architecture for Software Salvage

87

Backup Slides

Page 88: The Software Matrix An Architecture for Software Salvage

88

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm• Serves as the executive module – co-ordinates and uses services of all helper modules

• Provides administrative console interface for status and errors

Page 89: The Software Matrix An Architecture for Software Salvage

89

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm

• Identifies valid Cells through code reflection

• Loads valid Cells into the Matrix

Page 90: The Software Matrix An Architecture for Software Salvage

90

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm• Registers Cells with the Matrix so they are reachable

• Associates Cells with a GUID.

Page 91: The Software Matrix An Architecture for Software Salvage

91

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm• Represent the unit of composition and reuse

• Encapsulate functionality that makes them candidates for potential reuse

Page 92: The Software Matrix An Architecture for Software Salvage

92

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm

• Performs discovery to locates Cells of a particular capability

Page 93: The Software Matrix An Architecture for Software Salvage

93

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm• Responsible for generating (sender side) and interpreting (receiver side) the well-formed XML messages being passed in the Matrix

Page 94: The Software Matrix An Architecture for Software Salvage

94

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm• Serializes data such as arguments in requests and return values in responses so that they can be persisted in messages

• Also performs reverse process of de-serialization

Page 95: The Software Matrix An Architecture for Software Salvage

95

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm• Converts binary serialized data into the Base-64 encoding so that it can be easily encapsulated within XML tags

• Also performs decoding

Page 96: The Software Matrix An Architecture for Software Salvage

96

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm

• Used to maintain blocking queues used for messaging

• Used for temporal decoupling of client and server Cells

Page 97: The Software Matrix An Architecture for Software Salvage

97

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm

• Collaboration between Cells can be synchronous, asynchronous or one-way

• Implements efficient synchronous collaboration

Page 98: The Software Matrix An Architecture for Software Salvage

98

System Design

Loader

Serializer/Deserializer

Discovery

Matrix Executive

XML Messaging

Sync-AsyncWait

Registration Cells

MessageQueues

Base-64 Codec

Comm

• Relays messages (request/response) between message queues of collaborating Cells.

Page 99: The Software Matrix An Architecture for Software Salvage

99

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)Exec

• Executive of the Software Matrix.

• Manages administrative interface

Page 100: The Software Matrix An Architecture for Software Salvage

100

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)Loader

• At periodic intervals Loader.monitor checks the system “plug-in” directory

• Uses services in System.Reflection and System.Type of the .NET FCL to identify and load Cells

Page 101: The Software Matrix An Architecture for Software Salvage

101

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)Matrix

• Represents substrate where Cells plug-in to.

• Maintains collection of all active Cells.

− Matrix.register− Matrix.unregister− Matrix.discover− Matrix.send

Page 102: The Software Matrix An Architecture for Software Salvage

102

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)BlockingQueu

e• Implements

efficient blocking queues used by Cells.

• Uses FCL AutoResetEvent

BlockingQueue.enQ BlockingQueue.deQ

Page 103: The Software Matrix An Architecture for Software Salvage

103

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)ICell

• Defines the protocol to be followed by all Cells

• Actual Cells are concrete classes deriving from ICell, and are provided default implementation.

ICell.registerICell.unregisterICell.acceptICell.getCloneICell.syncSendICell.startICell.queryCapabilityICell.process

Page 104: The Software Matrix An Architecture for Software Salvage

104

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)ExecObject

ReturnObject

• Assist in the packaging of arguments, return values, exceptions, etc. in requests and responses between cooperating Cells.

Page 105: The Software Matrix An Architecture for Software Salvage

105

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)Messaging

• Provides services for the generation of well-formed XML messages in the correct format.

− Messaging. buildRequest− Messaging. buildResponse− Messaging. extract

Page 106: The Software Matrix An Architecture for Software Salvage

106

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)Clone

• Provides clone of a particular Cell enabling stateful collaboration between Cells.

Page 107: The Software Matrix An Architecture for Software Salvage

107

Class Relationships

. . .

Matrix

ICellBlockingQueue

Messaging

ExecObject ReturnObject Clone

SyncWait

Cell A Cell B Cell C

Loader

Exec(executive)SyncWait

• Allows for synchronous collaboration between Cells.

• Collaboration could also be asynchronous or one-way.