Creating an RAD Authoratative Data Environment

1

Data Sharing Architecture

Art Nicewick

2

3/6/2009 12:46:08 PM

3

Origins of data sharing architecture Initiative (November 2007)

I need a database, to share education programs and I have a MAC

Greg, can Art help Kim?

Art, help Kim?,

Hmm, Filemaker is not SQL Compliant, and MSAccess does not run on the MAC,

We need something better.

O.K., What should we do?

4

3 Areas of Data Integration• Collect

– Bring data in to

structured format

• Share – Define\Reference

Authoritative Data

• Disseminate – Make data

available to users

TMSRaisersEdge

FMS

IRIS

COEIRIMS

Vista

AI

Empl

GalleryData

$

$ GalleryData

Te

xt

12

34

5

Bar graph

DataCollection

Data Sharing Dessemination

5

Questions this presentation tries to answer?

TMSRaisersEdge

FMS

IRIS

COEIRIMS

Vista

AI

Empl

GalleryData

$

DataCollection

• What problem is being addressed by a web database

• When should Filemaker or MSAccess be used to address this problem

• When should Metastorm be used to address this Problem

• Why was PHPRunner selected for the web database front end

• How is the PHPRunner architecture to fullfull the requirement that there will be little to none application code in the implementation (allowing allowing it to be easily ported to a new application (DCLPA used as example).

6

[ Missing Gap]Need to have customized department systems

that reference authoritative data

Types of Gallery Data sharingWord Docs

ExcelOne user at a time, not shared

MSAccessFilemaker

Isolated non-integrated database (Stand-alone) shared between a handful of users, Highly customized

Raisers EdgeTMS

Many users shared, expensive, little-customization

InternetIntranet

Enterprise-wide and beyond.

Systems integrated to authoritative source, shared with many users across departments, customizable

DepartmentalAuthoritative Data Systems

Workflow (IRIS)

7

Database ?

• The gallery has two definitions of a database. – MSAccess\Filemaker (Single Platform)

– SQL Server\Postgress (Multiple Platform)

8

Pluses and Minus of COTs

• Cots are very inexpensive solution.. However: – They frequently do not have everything you need, and

you cannot easily modify\Extend them– They will not follow any naming standards based on

your organization – The governance of the code\logic\ and schema is

outside of the clients control (Although the Actual data is in their control)

– They cannot “Alone” be used to collect the Gallery’s Data needs – They need to be extended!

9

Documentation (UDB)

Project to Define when and how to define different application architectures (Nov, 2007)

Project Folder • \\nga\group\t\TDP\Projects\UDB - User Database Deployment

Documents – Proposed End User Data Architecture Implementations (UDB) 20071129a.doc– Filemaker and SQL Compliant Database(UDB) 20080109c.doc– User Database Prototype and Alpha Five(UDB).doc– End User Database Deployments Summary (UDB) 20071128a.doc

10

Fate of UDB project

There was concern at the time, that the UDB approach would cause “End Users” to be involved in the development process and that TDP would be unable to control their development demands.

The contrarian position was that TDP would still be the primary developers for any enterprise component of the architecture (e.g. Data Model) and the proposed solution would eliminate the current practice of users “independently” developing on systems that could not be shared, secured or extended to the enterprise (e.g. Filemaker, MSAccess, …)

These considerations were never resolved and the project was cancelled in February of 2008.

However, the issues still exist, and the proposed architecture is still worth reviewing.

11

What is a Database? (Two Views)

010010101001010101010101010101010101010101010100010010010101

Presentation “Front End”

Application“Business Rules”

Database (Storage)

Technologist \ Programmer understanding of the term

Business Analysts \ End User understanding of the term

What is a Database?

Integrated Development

Environment (IDE)“Tools”

IDE + Presentation + Business Rules + Data Storage

TechnologistView

BusinessView

An all-in-one System where I Can save data

(e.g. MSAccess)

Single ProductA part of a

much larger System, used to Store and share

Data (e.g. MSSQL)

Multi-Product

12

Examples of “Single” verses “Multi” software applications

Application Front End \ Presentation

Business Rules\ Application

Database Storage Development Environment

Single product Systems

Gallery Archives DB File Maker

Request Management System File Maker

Purchase Card MS Access

Excel Excel

Multi-Product Systems

Asset Inventory MS Access MSSQL MS Access

Employee Database MS Access MSSQL MS Access

Portfolio \Extensis (Service) Web \ Portfolio MSSQL Vendor

Inside.Nga.Gov Web Mason\Perl\CGI PostgresSQL Eclipse

IRIS Web Metastorm MSSQL Metastorm

FMS Java Applet J2EE Oracle OracleForms

TMS Visual Basic Visual Basic MSSQL Visual Basic

Raisers Edge Visual Basic Visual Basic MSSQL Visual Basic

Paper Conservation Web PHP MSSQL PHPRunner

13

Where is the Data

• Single version of Truth

14

Worst Case

Exhibition Code: 23 = XX

Exhibition Code: 23 = YY

Exhibition Code: 23 = ZZ

Exhibition Code: 21 = ZZExhibition Code: 21 = AA

Exhibition Code: 21 = AA




Exhibition Code: 21 = AA

• Data Everywhere, No single version of truth • How is correct?

16

Goal • Single version of truth • Defined Source of Authoritative data


18

Not one answer !

• Different users

• Different needs

19

Five Strategies

File Maker Application

1

Temp Storage or Local Copy Storage Only

No Data Storage

X

PH

PC

onversion

`

PHPApplciation

File Maker Application

2

Access

Read Only

OPTION 1

OPTION 3

OPTION 4

File Maker

Access

OPTION 2Single UserNon-Authoritative Data

Within Single DepartmentNon-Authoratative

Within Single DepartmentAuthoritative Data

Cross DepartmentsAuthoritative Data

Dept A Dept B

Dept C

Option 5.

Custom (Thin Client)

Option 1.Single User

Option 2.SharedDepartmentData(Fat Client)

Option 3.SharedEnterpriseData (Fat Client)

Option 4. Shared Enterprise Data(Thin Client)

20

Attributes Not Displayed in Slides

• Conditions for selecting each deployment

• User responsibility in each deployment

• Number of users estimates for each deployment

• TDP Role in each deployment

• . . .

Proposed End User Data Architecture Implementations (UDB) 20071129a.doc

21

Option 1 – Single User • A single user works alone, disconnected from the rest of the world. With there own version of

the data • This is another example of a “Single User” application. Filemaker, Excel and Access also fit into

this category. • A minor improvement, is to allow this “single user” the ability to linked to existing ‘authoritative

source’. This is mostly a “Reporting: feature. For example, Greg and Alan Newman use this method for reporting

Authorization

Optional

One TimeLoad

•Susan’s Pull from IA to Excel•TMS Crystal Reports •Raisers Edge Queries

22

Option 2 – Multi-User Departmental Application for Departmental-only Data

• This is a multiple user application. The data in this situation is not ‘authoritative’ to the Gallery and therefore is considered a ‘copy’ of some other data stored as a Gallery resource. This Architecture assume simple business rules, and little to no workflow.

User /Developer

UserUser

AFM AFM AFM

Independent

•Gallery Filemaker Archives •AFM Access database

24

Option 3 – Multi-User Departmental Application for Gallery-Wide Data

User

User

AFM

XDO

TMS

IRIS

DAC

RE

IRIMS

Web

Authoritative Source

Other..

Authorization

DataConsistencyVerfication

End UserDeveloper

This architecture attempts to blend the “ease of use” of a “Single product System” with the centralized control of the “Enterprise Database”. This Architecture assumes simple business rules, and little to no workflow.

•Asset Inventory•Scott Steven Employee Database

25

Option 4 - Multi-User Gallery-Wide Application for Gallery-Wide Data

TDP

User

XDO

TMS

IRIS

DAC

RE

IRIMS


Other..

DPUB

WEB

End UserDeveloper

DataModel

ConvertTo

PHP

Deploy toServer /

Generated from

Data Model

ConsistencyChecking

Authorization User

User

User

User

User

DM

PHPRunner

Overrides

This architecture extends the previous architecture to limit the impact of “Client Maintenance” by using a thin client Web Based Application..

•DCLPA•VDMS•Patch Reporting

26

Option 5 - Strategic Applications

TDP

User

XDO

TMS

IRIS

DAC

RE

IRIMS


Other..

DPUB

WEB

Developer

ConsistencyChecking

Authorization User

User

User

User

User

TMS\WEB

TRAIN

Other..

iHeat

Web Services , Native AJAX,

Complex application Rules

ComplexCode •IRIS

•Art Extract

27

Metastorm

• Not around when UDB study was developed (Dec 2007)

• Today, I would define it as a hybrid of Complex Business Rules and Generated application

28

SQL Server Database

Web Browser

Architecture

PHPRunner Application Server (Web Server)

`

29

What kind of tool do we need to do this ?

Required:- Must be easy to build – Low Maintenance - Must use database standards - Must have Security - Must have adequate performance

Desired:- We did not need to install it (e.g. Web based)- Was reasonably priced- Will be easy to maintain in the future (Standards based)- We should be able to easily replace it !

30

A few options

• MSAccess

• Filemaker

• PHPRunner

• MetaStorm

• Custom Web Develop? (Mason,Perl,CGI)

• Other?

31

MSAccess (November 2007)


Will you change to a PC, then we’ll give you Microsoft Access

Hmm, We really cannot ignore the requirement for MACs. For that matter,

expense to install is so high, a web solution would save the gallery the most money.. The

Web should be a requirement.

In that case, “Nevermind”

33

Filemaker ! Required:- Must be easy to build – Low Maintenance - Must use database standards - Must have Security - Must have adequate performance

Desired:- We did not need to install it (e.g. Web based)- Was reasonably priced- Will be easy to maintain in the future (Standards based)

How about File Maker to a SQL Database … Isn't that a feature a feature of Filemaker server?

Isn't that called ESS (External Data Sources)?

35

Problems With Filemaker• “Filemaker External SQL Sources (ESS) In Depth”

(Filemaker publication) – “Value lists cannot be based on data in an ESS table”

– “in a FileMaker Pro context, date-only or time-only data entry will not be valid” – ESS data in FileMaker Pro has the potential to be slightly out of date – Binary data is not supported by ESS at present – Scrolling operations … can be problematic in large record sets. . . . and will

perform fairly slowl – Sort not performed in database – The ESS feature set is primarily designed to allow FileMaker Pro solutions to

integrate data from SQL-based solutions. ESS is not primarily intended as a means to scale solutions beyond the bounds of a purely FileMaker Pro based solution.

– See: Filemaker and SQL Compliant Database(UDB) 20080109c.doc

“Value lists cannot be based on data in an ESS table”

Means

No field list validation (Dropdown List) from

non-Filemaker authoritative sources!

Or in simple terms

“Can’t validate against to authoritative data”

37

Filemaker (My Last Straw)From: John Blakeley [mailto:[email protected]] Sent: Tue 1/8/08 2:27 PM To: Nicewick, Arthur Subject: RE: your post on Filemaker about ODBC conversion errors Hi Arthur Thanks and a happy New Year to you! We gave up on the idea of pulling data from Filemaker using it as a linked server. In the end we scheduled a script to run that exported data on an hourly basis. SQL would then import it. We had to use MS scheduled tasks to open a FM file that would autostart an export script as FM server schedule cannot run scripts that aren't web compatible. Nothing is ever simple in Filemaker! One day...Cheers John Blakeley John Blakeley Mobile: + 64 21 948037 Email: [email protected] Skype: john.blakeley Bayview North Shore New Zealand

“We gave up on the idea of pulling data from Filemaker using it as a linked server”

38

Not Filemaker – Then what?

39

Prototyped

Typical “Types” of Assets , with issue numbers mapped

“Lookup” provides means to search for Issuance

40

Rated the bestEasy to Learn \ Use

Can be extended

Can integrate with Cots Packages

“Open” Architecture

Java Server Faces

\ Netbeans

Ruby on Rails\ Active Scaffold

Alpha Five

Cold Fusion

.NetPHPRunner

41

PHPRunner

Required:- Must be easy to build – Low Maintenance - Must use database standards - Must have Security - Must have adequate performance

Desired:- We did not need to install it (e.g. Web based)- Was reasonably priced- Will be easy to maintain in the future (Standards based)

Very little coding

Yes

Pretty good, but we need to make it better

Looks OK, Need testing

Yes

Yes

Standard Industry Language (PHP)

Framework not widespread as we would like

43

Framework

PHPRunner Code Generation

CodeGenerator

Access Routine

CodesOverrides ,

Screen Customizations

Templates

CodesOverrides

RelationshipsFramework

Tables

Framework

Relationship

Web Forms

Security / Navigation

DataDefinitions

Custom

Code Changes

Won’t break

by upgrades

Code Changes

Won’t break

by upgrades

Code change

“Might” break by

upgrades.

But the system

will try to

prevent it..

Upgrades willRevert changes

DB Rules

48

Charts

49

Reports

50

Searchable (Advanced and Simple)

Reports, Charts, Tables, …

51

End User Reports Creation(Web Based)

• Very Simple

• Version 1 is very limited

• Tightly integrated with Security

• Version 2 expected in the summer

52

How about Metastorm

• Workflow Focus • Excellent integration with in-box process • Ideal for “Approval” Processing • Ideal for “Request” processing • Somewhat RAD• No focus Data model • No focus of Data Business Rules • “Probably” ported from non-relational

53

Web Database vs Workflow

• PHPRunner (Web Database) has the user create the a normalized database (Modeling Business Rules) , then “Semi-Automatically” creates the User Interface

• Metastorm (Workflow) has the user create a workflow , and then “Semi-Automatically” creates a “deNormalized” (Technical) database behind it

54

Metastorm vs Web Database

• PHPRunner is “Only” code development, The tools is run on “Apache\PHP”. Therefore there is no license issues for users (unlimited users)

– PHPRunner toolkit $300 per developer

– Unlimited users – No Additional cost • Metastorm provides both a “Toolkit” and a

“Runtime” environment. The runtime environment cost

– User License $149.88 per named user (not concurrent) – Work Workgroup Server, restricted to a maximum of 250 named users$17,984.89 per

server – The Gallery has purchased 8 developer licenses at ~$2500 each (Need to verify)– Note: I current do not know many users exist in metastorm, We may be required to

purchase more licenses. However, once purchased, a single user license will work with unlimited Workflow task (However, we may need addition servers)

55

Replacing Filemaker and MSAccess

• PHPRunner is a Database driven tool and is therefore dependent on a normalized data model. Filemaker and MSAccess are also database driven tools that are also dependent on normalized data models. Therefore, PHPRunner should be ideal for database application migration of systems like “Gallery Archives”.

• Metastorm is “Workflow” focused and does not allow the user to define the database. For pure data storage, it is not a good application. However, For workflow, it is very good.

56

PHPRunner

Normalized Data Model - Clean Data Type- Logical Business Modeled Data Model - Data enforced Business Rules - Easy to report

Header(e.g.

Name,

Address)

Child1(e.g. Kids

Names)

Child2(Roles)

Child3(e.g. Cars)

Child1(e.g. Kids

Names)

AdminReference

Tables

Simple CRUD User Interface - Basic Menu - No “Workflow” or “Inbox” - Light integration with Email - Decent report generation

Auto Generated

57

Metastorm

KeysAnd

Pointers

Header(e.g.

Name,

Address)

Child1(e.g. Kids

Names)

Child2(Roles)

Child3(e.g. Cars)

Auto Generated

AdminReference

Tables ignore

TempScreenData

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore ignore

ignore

ignoreignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore ignore

ignore

ignore

ignore

ignore

ignore

Complex Workflow - Great “Inbox”- “Approvals” out of box - Logical Diagramming of Workflow - Tight integration with Email - “Bad” report generation

DeNormalized Data Model - Data not always Types (Integers are character) - Single Logical Table storing parents and children - “Temp” Screen data stored in Database - Fields defined but not used - Admin screen cannot be linked to data model

58

Metastorm and Workflow

• RAD\Visual development of Workflow is what Metatstorm is good at!

• A good standard for Workflow application in the Gallery

59

Data Model Differences

Header(e.g.

Name,

Address)

Child1(e.g. Kids

Names)

Child2(Roles)

Child3(e.g. Cars)

Child1(e.g. Kids

Names)

Parent

AdminReference

Tables

KeysAnd

Pointers

Header(e.g.

Name,

Address)

Child1(e.g. Kids

Names)

Child2(Roles)

Child3(e.g. Cars)

ignore

TempScreenData

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore ignore

ignore

ignoreignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore ignore

ignore

ignore

ignore

ignore

ignore

61

Classifications

Classification_id int

Name nvarchar(50)

LastUpdateUserName nvarchar(50)

LastUpdateTime datetime

Column Name Data Type Allow Nulls

ConservatorsConservator_id

Gallery_id

Comment

isInEmployeeDirectory

LocalVersionOfName

LastUpdateUserName

LastUpdateTime

ExhibitionActivities *ExhibitionActivity_id

SortOrderInReport

Name

RequestReason_id

isOnWorklog

Exhibitions

Exhibition_id int

code nvarchar(50)

Name nvarchar(100)

isMaintainedInTMS bit

Comment nvarchar(MAX)

RequestReason_id int

DropDown_Display_Or... int


ReportTypeGroupsReportTypeGroup_id

Name

isTechnicalAnalysis

isDetailedExam

isTreatmentProposal

isObjectReport

isOther

LastUpdateUserName

LastUpdateTime

ReportTypes

ReportType_id int

ReportTypeGroup_id int

Name nvarchar(100)



isValidWithBulk bit

isValidWithQuick bit

isValidWithMajor bit

isValidWithMinor bit

DropDown_Display_Or... int


RequestApprovingRolesRequestApprovingRole_id

Request_id

ApprovingRole_id

ApprovingUser_id

Approval_id

Comment

LastUpdateUserName

LastUpdateTime

RequestBulkClassificationCountsRequestBulkClassificationCount_id

Request_id

CountForClassification

Classification_id

isNGA

Comment

RequestReasons *

RequestReason_id

DropDown_Display_Order

Name

Column Name

RequestReportDACsRequestReport_DAC_id

RequestReport_id

DAC_Record_id

Comment

LastUpdateUserName

LastUpdateTime

RequestReportStatusesRequestReportStatus_id

Name


ReportType_id

LastUpdateUserName

LastUpdateTime

RequestReportWorkOrdersRequestReport_WorkOrder_id

RequestReport_id

WorkOrder_Line

WorkOrderType_id

LastUpdateUserName

LastUpdateTime

Requests *

Request_id int

EntryShortcutSelect_... int


RequestType_id int

RequestStatus_id int

Requestor_id int

Comment nvarchar(300)

QuickNotes nvarchar(MAX)

RequestDate datetime



Exhibition_id int

ExhibitionActivity_id int

ExhibitionOtherActivit... int

EntryShortcutAjax_M... int

Loan_id int

LoanActivity_id int


RequestStatuses

RequestStatus_id int

Name nvarchar(100)

DropDown_Display_Order int



SetToThisWhenFirstReportIs... bit


RequestTMSObjectsRequestTMSObject_id

Request_id

TMSObject_id

EntryShortcutAjax_TMSObje...

RequestTypeApprovingRolesRequestTypeApprovingRole_id

RequestType_id

ApprovingRole_id

Comment

LastUpdateUserName

LastUpdateTime

WorkOrderTypesWorkOrderType_id

Name

LastUpdateUserName

LastUpdateTime

RequestTypes *

RequestType_id

Name

isMajor

isMinor

isQuick

isLoan

isBulk

isObjectRequired

isMultiObjectsAllowed

isApprovalRequired


LastUpdateUserName

Column Name

RequestReports *

RequestReport_id int

Request_id int

ReportType_id int

StartedDate datetime

EndedDate datetime

Conservator_id int

Description nvarchar(MAX)

User_id int

RequestReportStatus_id int

AnalogImageLocation_id int

IncludeInFinalObjectR... bit

Report nvarchar(MAX)



RequestTMSObject_id int

Attachment nvarchar(MAX)

Image nvarchar(MAX)

RequestNonTMSObjec... int


RequestReportTechnicalInfosRequestReport_id

TechnicalNotes

BetaRadiograph

XRayFluorescenceSpectroscopy

InfraredSpectroscopy

MicroFadometer

FourierTransformInfraredSpectroscopy

FluorescenceSpectroscopy

ColorSpectroscopy

None

Other

OtherDescription

Loans

Loan_id int

Name nvarchar(50)

DropDown_Display_Order int






LoanActivities

LoanActivity_id int

Name nvarchar(50)






RequestTypeReasons *RequestTypeReasons_id

RequestType_id

RequestReason_id

RequestReportAttachments *RequestReportAttachment_id

RequestReport_id

FileName

Comment

RequestNonTMSObjects *RequestNonTMSObject_id

Name

NGAContactGallery_id

Request_id

RequestReports

RequestReport_id

Request_id

ReportType_id

StartedDate

EndedDate

Conservator_id

Description

User_id

RequestReportStatus_id

AnalogImageLocation_id

IncludeInFinalObjectReport

Report

LastUpdateUserName

LastUpdateTime

RequestTMSObject_id

Attachment

Image

RequestNonTMSObject_id

Column Name

RequestReportDACs

RequestReport_DAC_id

RequestReport_id

DAC_Record_id

Comment

LastUpdateUserName

LastUpdateTime

Column NameRequestReportAttachments

RequestReportAttachment_id

RequestReport_id

FileName

Comment

Column Name

RequestNonTMSObjects


Name


Request_id

Column NameRequestBulkClassificationCounts

RequestBulkClassificationCount_id

Request_id


Classification_id

isNGA

Comment

Column Name

RequestApprovingRoles

RequestApprovingRole_id

Request_id

ApprovingRole_id

ApprovingUser_id

Approval_id

Comment

LastUpdateUserName

LastUpdateTime

Column Name

RequestReportWorkOrders

RequestReport_WorkOrder_id

RequestReport_id

WorkOrder_Line

WorkOrderType_id

LastUpdateUserName

LastUpdateTime

Column Name

RequestReportTechnicalInfos

RequestReport_id

TechnicalNotes

BetaRadiograph



MicroFadometer



ColorSpectroscopy

None

Other

OtherDescription

Column Name

RequestTMSObjects

RequestTMSObject_id

Request_id

TMSObject_id

EntryShortcutAjax_TMSObject_id

Column Name

Requests

Request_id

EntryShortcutSelect_MajorTMSObj...

RequestReason_id

RequestType_id

RequestStatus_id

Requestor_id

Comment

QuickNotes

RequestDate

LastUpdateUserName

LastUpdateTime

Exhibition_id

ExhibitionActivity_id

ExhibitionOtherActivity_id

EntryShortcutAjax_MajorTMSObje...

Loan_id

LoanActivity_id

Column Name

PHPRunner (DCLPA) Data Model

62

RequestReports

RequestReport_id

Request_id

ReportType_id

StartedDate

EndedDate

Conservator_id

Description

User_id




Report

LastUpdateUserName

LastUpdateTime

RequestTMSObject_id

Attachment

Image


Column Name

RequestReportDACs


RequestReport_id

DAC_Record_id

Comment

LastUpdateUserName

LastUpdateTime



RequestReport_id

FileName

Comment

Column Name



Name


Request_id



Request_id


Classification_id

isNGA

Comment

Column Name



Request_id

ApprovingRole_id

ApprovingUser_id

Approval_id

Comment

LastUpdateUserName

LastUpdateTime

Column Name



RequestReport_id

WorkOrder_Line

WorkOrderType_id

LastUpdateUserName

LastUpdateTime

Column Name


RequestReport_id

TechnicalNotes

BetaRadiograph



MicroFadometer



ColorSpectroscopy

None

Other

OtherDescription

Column Name

RequestTMSObjects

RequestTMSObject_id

Request_id

TMSObject_id


Column Name

Requests

Request_id


RequestReason_id

RequestType_id

RequestStatus_id

Requestor_id

Comment

QuickNotes

RequestDate

LastUpdateUserName

LastUpdateTime

Exhibition_id




Loan_id

LoanActivity_id

Column Name

PHPRunner (DCLPA)

Key tables Data Model

63

PHPRunner Actual Data

Normalized means the new child tables can be added or deleted, with little impact On the rest of the database, also unlimited child tables can be added

RequestReport

RequestReport

Workorders

Requests

64

PHPRunner

• Focus on “Data Model” in database and not in “Application Logic” – Example

• DCLPA – Lines of PHP code [~75 lines]• DCLPA – Custom HTML [~86 lines]• Tables 40 tables • Database Stored Procedure Code [~250 lines]

Designed to be easily replaceable with new “user Interface” tool (if desired)

65

IRIS Data Model

eEvent (

eFolderID nvarchar (31) NOT NULL , eEventID int NOT NULL , eVersion int NULL , ePriority smallint NULL , eEventTime datetime NULL , eEntryTime datetime NULL , eDeadline datetime NULL , eMapName nvarchar (31) NULL , eFromStage nvarchar (31) NULL , eActionName nvarchar (31) NULL , eToStage nvarchar (31) NULL , eUserName nvarchar (100) NULL , eFlagName nvarchar (63) NULL , eFlagFolder nvarchar (31) NULL , eAlertMessage nvarchar (250) NULL , eNotes ntext NULL

eFolder (

eFolderID nvarchar (31) NOT NULL , eParent nvarchar (31) NULL , eFolderName nvarchar (31) NULL , eStageName nvarchar (31) NULL , eMapName nvarchar (31) NULL , eServerName nvarchar (31) NULL , eCategory nvarchar (31) NULL , eOriginator nvarchar (100) NULL , eSubject nvarchar (250) NULL , eArchived smallint NULL , ePriority smallint NULL , eActionCount int NULL , eCreationTime datetime NULL , eEntryTime datetime NULL , eUpdated datetime NULL , eDeadline datetime NULL , eActionTime datetime NULL , eActionName nvarchar (31) NULL , eActionUser nvarchar (100) NULL , eFragmentStack ntext NULL

Deliverables (

Id int IDENTITY (1, 1) NOT NULL , Name nvarchar (250) NOT NULL , Cost int NOT NULL

Line_Items (

EFOLDERID nvarchar (31) NOT NULL , AO_AccNum nvarchar (250) NULL , AO_Artist nvarchar (250) NULL , AO_ArtTreatment smallint NULL , AO_Complete_Total money NULL , AO_DateNeeded datetime NULL , AO_Deliverables nvarchar (250) NULL , AO_Description ntext NULL , AO_DexNum nvarchar (250) NULL , AO_Dimensions nvarchar (250) NULL , AO_ExistingAssets ntext NULL , AO_Height int NULL , AO_Id nvarchar (250) NULL , AO_Info ntext NULL , AO_Lecture smallint NULL , AO_LightAngle nvarchar (250) NULL , AO_lineItemNum nvarchar (250) NULL , AO_LocalPrinting smallint NULL , AO_NewAssets ntext NULL , AO_NonNGAObj smallint NULL , AO_ObjectClass nvarchar (250) NULL , AO_PhotographicCost money NULL , AO_Prepress smallint NULL , AO_ProductionJournal ntext NULL , AO_Quantity int NULL , AO_SpecialDeliveryInstructions ntext NULL , AO_Spectrum nvarchar (250) NULL , AO_Status nvarchar (250) NULL , AO_Title nvarchar (250) NULL , AO_Total money NULL , AO_TreatmentPhase nvarchar (250) NULL , AO_VisualReference smallint NULL , AO_Web smallint NULL , AO_Width int NULL , Assignee1 nvarchar (250) NULL , Assignee2 nvarchar (250) NULL , checkDeliverables smallint NULL , DeliverablesAmount int NULL , destinationFolderId nvarchar (250) NULL , destinationFolderName nvarchar (250) NULL , Difference_Total money NULL , Duty1 nvarchar (250) NULL , Duty2 nvarchar (250) NULL , EP_AdditionalDetails ntext NULL , EP_ArtTreatment smallint NULL , EP_Cell nvarchar (250) NULL , EP_Complete_Total money NULL , EP_Contact nvarchar (250) NULL , EP_DateNeeded datetime NULL , EP_Dress nvarchar (250) NULL , EP_Email nvarchar (250) NULL , EP_EventEnd datetime NULL , EP_EventName nvarchar (250) NULL , EP_EventStart datetime NULL , EP_EventType nvarchar (250) NULL , EP_ExistingAssets ntext NULL , EP_Fax nvarchar (250) NULL , EP_Info ntext NULL , EP_Lecture smallint NULL , EP_lineItemNum nvarchar (250) NULL , EP_LocalPrinting smallint NULL , EP_Location nvarchar (250) NULL , EP_NewAssets ntext NULL , EP_PhotographerEnd datetime NULL , EP_PhotographerStart datetime NULL , EP_PhotographicCost money NULL , EP_Prepress smallint NULL , EP_ProductionJournal ntext NULL , EP_SpecialDeliveryInstructions ntext NULL , EP_Total money NULL , EP_VisualReference smallint NULL , EP_Web smallint NULL , Line_Item_Total money NULL , memAssignments ntext NULL , msg nvarchar (250) NULL , NAO_ArtTreatment smallint NULL , NAO_Complete_Total money NULL , NAO_DateNeeded datetime NULL , NAO_Deliverables nvarchar (250) NULL , NAO_Description ntext NULL , NAO_Dimensions nvarchar (250) NULL , NAO_ExistingAssets ntext NULL , NAO_Height int NULL , NAO_ID nvarchar (250) NULL , NAO_Info ntext NULL , NAO_Lecture smallint NULL , NAO_lineItemNum nvarchar (250) NULL , NAO_LocalPrinting smallint NULL , NAO_NewAssets ntext NULL , NAO_ObjectType nvarchar (250) NULL , NAO_PhotographicCost money NULL , NAO_Prepress smallint NULL , NAO_ProductionJournal ntext NULL , NAO_Quantity int NULL , NAO_Source nvarchar (250) NULL , NAO_SpecialDeliveryInstructions ntext NULL , NAO_Total money NULL , NAO_VisualReference smallint NULL , NAO_Web smallint NULL , NAO_Width int NULL , nextStage nvarchar (250) NULL , O_ArtTreatment smallint NULL , O_Complete_Total money NULL , O_DateNeeded datetime NULL , O_Deliverables nvarchar (250) NULL , O_Description ntext NULL , O_ExistingAssets ntext NULL , O_Height int NULL , O_Info ntext NULL , O_Lecture smallint NULL , O_lineItemNum nvarchar (250) NULL , O_LocalPrinting smallint NULL , O_NewAssets ntext NULL , O_PhotographicCost money NULL , O_Prepress smallint NULL , O_ProductionJournal ntext NULL , O_Quantity int NULL , O_SpecialDeliveryInstructions ntext NULL , O_Total money NULL , O_VisualReference smallint NULL , O_Web smallint NULL , O_Width int NULL , old_parent nvarchar (250) NULL , old_parent_foldername nvarchar (250) NULL , Old_Total money NULL , prnt nvarchar (250) NULL , prntFolderName nvarchar (250) NULL , ProductionStartDate datetime NULL , Requestor nvarchar (250) NULL , result nvarchar (250) NULL , resultCount int NULL , Staff1 nvarchar (250) NULL , Staff2 nvarchar (250) NULL , Status nvarchar (250) NULL , Status1 nvarchar (250) NULL , Status2 nvarchar (250) NULL , Studio1 nvarchar (250) NULL , Studio2 nvarchar (250) NULL , tmpPrnt nvarchar (250) NULL , txtUrgent nvarchar (250) NULL , type nvarchar (250) NULL , urgent smallint NULL , WO_initiator nvarchar (250) NULL , EP_OfficePhone nvarchar (250) NULL , WorkOrderApproved nvarchar (250) NULL , AO_Medium nvarchar (250) NULL , newPhotography smallint NULL , AO_Other smallint NULL , AO_Unknown smallint NULL , AO_AssetName nvarchar (250) NULL , AO_EAStatus nvarchar (250) NULL , AO_FileName nvarchar (250) NULL , NAO_AssetName nvarchar (250) NULL , NAO_EAStatus nvarchar (250) NULL , NAO_FileName nvarchar (250) NULL , NAO_Other smallint NULL , NAO_Unknown smallint NULL , O_Other smallint NULL , O_Unknown smallint NULL , Deliverables nvarchar (250) NULL , Quantity int NULL

Projects (

Id int IDENTITY (1, 1) NOT NULL , Name varchar (400) NOT NULL

Requestors (

Id int IDENTITY (1, 1) NOT NULL , Name varchar (250) NOT NULL , Office varchar (150) NULL , Phone varchar (50) NULL , eName varchar (150) NULL

Work_Order (

EFOLDERID nvarchar (31) NOT NULL , Account nvarchar (250) NULL , AO_AccNum nvarchar (250) NULL , AO_Artist nvarchar (250) NULL , AO_ArtTreatment smallint NULL , AO_Complete_Total money NULL , AO_DateNeeded datetime NULL , AO_Deliverables nvarchar (250) NULL , AO_Description ntext NULL , AO_DexNum nvarchar (250) NULL , AO_Dimensions nvarchar (250) NULL , AO_Duty1 nvarchar (250) NULL , AO_Duty2 nvarchar (250) NULL , AO_ExistingAssets nvarchar (250) NULL , AO_Height int NULL , AO_Id nvarchar (250) NULL , AO_Lecture smallint NULL , AO_LightAngle nvarchar (250) NULL , AO_lineItemNum nvarchar (250) NULL , AO_LocalPrinting smallint NULL , AO_NewAssets nvarchar (250) NULL , AO_NonNGAObj smallint NULL , AO_ObjectClass nvarchar (250) NULL , AO_PhotographicCost money NULL , AO_Prepress smallint NULL , AO_ProductionJournal ntext NULL , AO_Quantity int NULL , AO_SpecialDeliveryInstructions ntext NULL , AO_Spectrum nvarchar (250) NULL , AO_Staff1 nvarchar (250) NULL , AO_Staff2 nvarchar (250) NULL , AO_Status nvarchar (250) NULL , AO_Status1 nvarchar (250) NULL , AO_Status2 nvarchar (250) NULL , AO_Studio1 nvarchar (250) NULL , AO_Studio2 nvarchar (250) NULL , AO_Title nvarchar (250) NULL , AO_Total money NULL , AO_TreatmentPhase nvarchar (250) NULL , AO_VisualReference smallint NULL , AO_Web smallint NULL , AO_Width int NULL , Assignee1 nvarchar (250) NULL , Assignee2 nvarchar (250) NULL , BriefDescription ntext NULL , checkDeliverables smallint NULL , Code nvarchar (250) NULL , Comments ntext NULL , CostAdjustments float NULL , DateBilled datetime NULL , DateDelivered datetime NULL , DateNeeded datetime NULL , DateSubmitted datetime NULL , DeliverablesAmount int NULL , DeliveryMedia nvarchar (250) NULL , DivisionCenter nvarchar (250) NULL , Duty1 nvarchar (250) NULL , Duty2 nvarchar (250) NULL , EmailAddress nvarchar (250) NULL , EmailMessage ntext NULL , EmailSubject nvarchar (250) NULL , EP_AdditionalDetails ntext NULL , EP_ArtTreatment smallint NULL , EP_Cell nvarchar (250) NULL , EP_Complete_Total money NULL , EP_Contact nvarchar (250) NULL , EP_DateNeeded datetime NULL , EP_Dress nvarchar (250) NULL , EP_Email nvarchar (250) NULL , EP_EventEnd datetime NULL , EP_EventName nvarchar (250) NULL , EP_EventStart datetime NULL , EP_EventType nvarchar (250) NULL , EP_ExistingAssets nvarchar (250) NULL , EP_Fax nvarchar (250) NULL , EP_Lecture smallint NULL , EP_lineItemNum nvarchar (250) NULL , EP_LocalPrinting smallint NULL , EP_Location nvarchar (250) NULL , EP_NewAssets nvarchar (250) NULL , EP_PhotographerEnd datetime NULL , EP_PhotographerStart datetime NULL , EP_PhotographicCost money NULL , EP_Prepress smallint NULL , EP_ProductionJournal ntext NULL , EP_SpecialDeliveryInstructions ntext NULL , EP_Total money NULL , EP_VisualReference smallint NULL , EP_Web smallint NULL , FundSource nvarchar (250) NULL , initStatus nvarchar (250) NULL , Instructions ntext NULL , LineItems nvarchar (250) NULL , mediaPrice money NULL , NAO_ArtTreatment smallint NULL , NAO_Complete_Total money NULL , NAO_DateNeeded datetime NULL , NAO_Deliverables nvarchar (250) NULL , NAO_Description ntext NULL , NAO_Dimensions nvarchar (250) NULL , NAO_ExistingAssets nvarchar (250) NULL , NAO_Height int NULL , NAO_ID nvarchar (250) NULL , NAO_Lecture smallint NULL , NAO_lineItemNum nvarchar (250) NULL , NAO_LocalPrinting smallint NULL , NAO_NewAssets nvarchar (250) NULL , NAO_ObjectType nvarchar (250) NULL , NAO_PhotographicCost money NULL , NAO_Prepress smallint NULL , NAO_ProductionJournal ntext NULL , NAO_Quantity int NULL , NAO_Source nvarchar (250) NULL , NAO_SpecialDeliveryInstructions ntext NULL , NAO_Total money NULL , NAO_VisualReference smallint NULL , NAO_Web smallint NULL , NAO_Width int NULL , nextStage nvarchar (250) NULL , O_ArtTreatment smallint NULL , O_Complete_Total money NULL , O_DateNeeded datetime NULL , O_Deliverables nvarchar (250) NULL , O_Description ntext NULL , O_ExistingAssets nvarchar (250) NULL , O_Height int NULL , O_Lecture smallint NULL , O_lineItemNum nvarchar (250) NULL , O_LocalPrinting smallint NULL , O_NewAssets nvarchar (250) NULL , O_PhotographicCost money NULL , O_Prepress smallint NULL , O_ProductionJournal ntext NULL , O_Quantity int NULL , O_SpecialDeliveryInstructions ntext NULL , O_Total money NULL , O_VisualReference smallint NULL , O_Web smallint NULL , O_Width int NULL , OrderCloseDate datetime NULL , OrderStatus nvarchar (250) NULL , OrderTotal float NULL , p_Total money NULL , prntFolderName nvarchar (250) NULL , ProductionJournal ntext NULL , ProjID nvarchar (250) NULL , ProjNum nvarchar (250) NULL , PurchasingCategories nvarchar (250) NULL , RecipientName nvarchar (250) NULL , RecipientOffice nvarchar (250) NULL , RecipientPhone nvarchar (250) NULL , RequestorName nvarchar (250) NULL , RequestorOffice nvarchar (250) NULL , RequestorPhone nvarchar (250) NULL , Result nvarchar (250) NULL , Staff1 nvarchar (250) NULL , Staff2 nvarchar (250) NULL , Status nvarchar (250) NULL , Status1 nvarchar (250) NULL , Status2 nvarchar (250) NULL , Studio1 nvarchar (250) NULL , Studio2 nvarchar (250) NULL , Temp1 nvarchar (250) NULL , tempUrgent smallint NULL , test nvarchar (250) NULL , Type nvarchar (250) NULL , urgent smallint NULL , UserName nvarchar (250) NULL , UserType nvarchar (250) NULL , VenueCode nvarchar (250) NULL , VisualServiceContractNum nvarchar (250) NULL , WO_initiator nvarchar (250) NULL , WOTotal float NULL , OfficePhone nvarchar (250) NULL , EP_OfficePhone nvarchar (250) NULL , WorkOrderApproved nvarchar (250) NULL , AO_Medium nvarchar (250) NULL , AO_Other smallint NULL , AO_Unknown smallint NULL , FiscalYear nvarchar (250) NULL , NAO_Other smallint NULL , NAO_Unknown smallint NULL , O_Other smallint NULL , O_Unknown smallint NULL , PayerDepartment nvarchar (250) NULL , USSGL nvarchar (250) NULL , DateDeliveryFinalized datetime NULL , DateSentPicked datetime NULL , countChildren int NULL

The Metastorm Data Model is Generated to Support the Workflow Diagrams.

It is not a logical Representation of Data Business Rules

Workflow Diagramming is Where Metastorm Excels

66

IRIS Data Model

eEvent (


eFolder (


Deliverables (


Line_Items (


Projects (


Requestors (


Work_Order (

EFOLDERID nvarchar (31) NOT NULL , Account nvarchar (250) NULL , AO_AccNum nvarchar (250) NULL , AO_Artist nvarchar (250) NULL , AO_ArtTreatment smallint NULL , AO_Complete_Total money NULL , AO_DateNeeded datetime NULL , AO_Deliverables nvarchar (250) NULL , AO_Description ntext NULL , AO_DexNum nvarchar (250) NULL , AO_Dimensions nvarchar (250) NULL , AO_Duty1 nvarchar (250) NULL , AO_Duty2 nvarchar (250) NULL , AO_ExistingAssets nvarchar (250) NULL , AO_Height int NULL , AO_Id nvarchar (250) NULL , AO_Lecture smallint NULL , AO_LightAngle nvarchar (250) NULL , AO_lineItemNum nvarchar (250) NULL , AO_LocalPrinting smallint NULL , AO_NewAssets nvarchar (250) NULL , AO_NonNGAObj smallint NULL , AO_ObjectClass nvarchar (250) NULL , AO_PhotographicCost money NULL , AO_Prepress smallint NULL , AO_ProductionJournal ntext NULL , AO_Quantity int NULL , AO_SpecialDeliveryInstructions ntext NULL , AO_Spectrum nvarchar (250) NULL , AO_Staff1 nvarchar (250) NULL , AO_Staff2 nvarchar (250) NULL , AO_Status nvarchar (250) NULL , AO_Status1 nvarchar (250) NULL , AO_Status2 nvarchar (250) NULL , AO_Studio1 nvarchar (250) NULL , AO_Studio2 nvarchar (250) NULL , AO_Title nvarchar (250) NULL , AO_Total money NULL , AO_TreatmentPhase nvarchar (250) NULL , AO_VisualReference smallint NULL , AO_Web smallint NULL , AO_Width int NULL , Assignee1 nvarchar (250) NULL , Assignee2 nvarchar (250) NULL , BriefDescription ntext NULL , checkDeliverables smallint NULL , Code nvarchar (250) NULL , Comments ntext NULL , CostAdjustments float NULL , DateBilled datetime NULL , DateDelivered datetime NULL , DateNeeded datetime NULL , DateSubmitted datetime NULL , DeliverablesAmount int NULL , DeliveryMedia nvarchar (250) NULL , DivisionCenter nvarchar (250) NULL , Duty1 nvarchar (250) NULL , Duty2 nvarchar (250) NULL , EmailAddress nvarchar (250) NULL , EmailMessage ntext NULL , EmailSubject nvarchar (250) NULL , EP_AdditionalDetails ntext NULL , EP_ArtTreatment smallint NULL , EP_Cell nvarchar (250) NULL , EP_Complete_Total money NULL , EP_Contact nvarchar (250) NULL , EP_DateNeeded datetime NULL , EP_Dress nvarchar (250) NULL , EP_Email nvarchar (250) NULL , EP_EventEnd datetime NULL , EP_EventName nvarchar (250) NULL , EP_EventStart datetime NULL , EP_EventType nvarchar (250) NULL , EP_ExistingAssets nvarchar (250) NULL , EP_Fax nvarchar (250) NULL , EP_Lecture smallint NULL , EP_lineItemNum nvarchar (250) NULL , EP_LocalPrinting smallint NULL , EP_Location nvarchar (250) NULL , EP_NewAssets nvarchar (250) NULL , EP_PhotographerEnd datetime NULL , EP_PhotographerStart datetime NULL , EP_PhotographicCost money NULL , EP_Prepress smallint NULL , EP_ProductionJournal ntext NULL , EP_SpecialDeliveryInstructions ntext NULL , EP_Total money NULL , EP_VisualReference smallint NULL , EP_Web smallint NULL , FundSource nvarchar (250) NULL , initStatus nvarchar (250) NULL , Instructions ntext NULL , LineItems nvarchar (250) NULL , mediaPrice money NULL , NAO_ArtTreatment smallint NULL , NAO_Complete_Total money NULL , NAO_DateNeeded datetime NULL , NAO_Deliverables nvarchar (250) NULL , NAO_Description ntext NULL , NAO_Dimensions nvarchar (250) NULL , NAO_ExistingAssets nvarchar (250) NULL , NAO_Height int NULL , NAO_ID nvarchar (250) NULL , NAO_Lecture smallint NULL , NAO_lineItemNum nvarchar (250) NULL , NAO_LocalPrinting smallint NULL , NAO_NewAssets nvarchar (250) NULL , NAO_ObjectType nvarchar (250) NULL , NAO_PhotographicCost money NULL , NAO_Prepress smallint NULL , NAO_ProductionJournal ntext NULL , NAO_Quantity int NULL , NAO_Source nvarchar (250) NULL , NAO_SpecialDeliveryInstructions ntext NULL , NAO_Total money NULL , NAO_VisualReference smallint NULL , NAO_Web smallint NULL , NAO_Width int NULL , nextStage nvarchar (250) NULL , O_ArtTreatment smallint NULL , O_Complete_Total money NULL , O_DateNeeded datetime NULL , O_Deliverables nvarchar (250) NULL , O_Description ntext NULL , O_ExistingAssets nvarchar (250) NULL , O_Height int NULL , O_Lecture smallint NULL , O_lineItemNum nvarchar (250) NULL , O_LocalPrinting smallint NULL , O_NewAssets nvarchar (250) NULL , O_PhotographicCost money NULL , O_Prepress smallint NULL , O_ProductionJournal ntext NULL , O_Quantity int NULL , O_SpecialDeliveryInstructions ntext NULL , O_Total money NULL , O_VisualReference smallint NULL , O_Web smallint NULL , O_Width int NULL , OrderCloseDate datetime NULL , OrderStatus nvarchar (250) NULL , OrderTotal float NULL , p_Total money NULL , prntFolderName nvarchar (250) NULL , ProductionJournal ntext NULL , ProjID nvarchar (250) NULL , ProjNum nvarchar (250) NULL , PurchasingCategories nvarchar (250) NULL , RecipientName nvarchar (250) NULL , RecipientOffice nvarchar (250) NULL , RecipientPhone nvarchar (250) NULL , RequestorName nvarchar (250) NULL , RequestorOffice nvarchar (250) NULL , RequestorPhone nvarchar (250) NULL , Result nvarchar (250) NULL , Staff1 nvarchar (250) NULL , Staff2 nvarchar (250) NULL , Status nvarchar (250) NULL , Status1 nvarchar (250) NULL , Status2 nvarchar (250) NULL , Studio1 nvarchar (250) NULL , Studio2 nvarchar (250) NULL , Temp1 nvarchar (250) NULL , tempUrgent smallint NULL , test nvarchar (250) NULL , Type nvarchar (250) NULL , urgent smallint NULL , UserName nvarchar (250) NULL , UserType nvarchar (250) NULL , VenueCode nvarchar (250) NULL , VisualServiceContractNum nvarchar (250) NULL , WO_initiator nvarchar (250) NULL , WOTotal float NULL , OfficePhone nvarchar (250) NULL , EP_OfficePhone nvarchar (250) NULL , WorkOrderApproved nvarchar (250) NULL , AO_Medium nvarchar (250) NULL , AO_Other smallint NULL , AO_Unknown smallint NULL , FiscalYear nvarchar (250) NULL , NAO_Other smallint NULL , NAO_Unknown smallint NULL , O_Other smallint NULL , O_Unknown smallint NULL , PayerDepartment nvarchar (250) NULL , USSGL nvarchar (250) NULL , DateDeliveryFinalized datetime NULL , DateSentPicked datetime NULL , countChildren int NULL

Database Enforce integrityNot used (Possible?)

67

IRIS Data Model

Events

Where type =

“Event”

eEvent (


eFolder (


Deliverables (


Line_Items (


Projects (


Requestors (


Work_Order (

EFOLDERID nvarchar (31) NOT NULL , Account nvarchar (250) NULL , Assignee1 nvarchar (250) NULL , Assignee2 nvarchar (250) NULL , BriefDescription ntext NULL , checkDeliverables smallint NULL , Code nvarchar (250) NULL , Comments ntext NULL , CostAdjustments float NULL , DateBilled datetime NULL , DateDelivered datetime NULL , DateNeeded datetime NULL , DateSubmitted datetime NULL , DeliverablesAmount int NULL , DeliveryMedia nvarchar (250) NULL , DivisionCenter nvarchar (250) NULL , Duty1 nvarchar (250) NULL , Duty2 nvarchar (250) NULL , EmailAddress nvarchar (250) NULL , EmailMessage ntext NULL , EmailSubject nvarchar (250) NULL , FundSource nvarchar (250) NULL , initStatus nvarchar (250) NULL , Instructions ntext NULL , LineItems nvarchar (250) NULL , mediaPrice money NULL , nextStage nvarchar (250) NULL , OrderCloseDate datetime NULL , OrderStatus nvarchar (250) NULL , OrderTotal float NULL , p_Total money NULL , prntFolderName nvarchar (250) NULL , ProductionJournal ntext NULL , ProjID nvarchar (250) NULL , ProjNum nvarchar (250) NULL , PurchasingCategories nvarchar (250) NULL , RecipientName nvarchar (250) NULL , RecipientOffice nvarchar (250) NULL , RecipientPhone nvarchar (250) NULL , RequestorName nvarchar (250) NULL , RequestorOffice nvarchar (250) NULL , RequestorPhone nvarchar (250) NULL , Result nvarchar (250) NULL , Staff1 nvarchar (250) NULL , Staff2 nvarchar (250) NULL , Status nvarchar (250) NULL , Status1 nvarchar (250) NULL , Status2 nvarchar (250) NULL , Studio1 nvarchar (250) NULL , Studio2 nvarchar (250) NULL , Temp1 nvarchar (250) NULL , tempUrgent smallint NULL , test nvarchar (250) NULL , Type nvarchar (250) NULL , urgent smallint NULL , UserName nvarchar (250) NULL , UserType nvarchar (250) NULL , VenueCode nvarchar (250) NULL , VisualServiceContractNum nvarchar (250) NULL , WO_initiator nvarchar (250) NULL , WOTotal float NULL , OfficePhone nvarchar (250) NULL , WorkOrderApproved nvarchar (250) NULL , FiscalYear nvarchar (250) NULL , PayerDepartment nvarchar (250) NULL , USSGL nvarchar (250) NULL , DateDeliveryFinalized datetime NULL , DateSentPicked datetime NULL , countChildren int NULL

Art Object

Where type =

“Art”

Non Art Object

Where type =

“Non Art”

O = Other

Where type =

“Other”

Art Object

Info

Event info

Event info

NonArt

Other

IRISFolder ID

ArtObject Event

NonArt Other

Other ignore

TempScreenData

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore ignore

Art Object

Info

Event info ignore

ignoreignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore

ignore ignore

ignore

ignore

ignore

ignore

ignore

68

Why Normalize

• 1. To free the collection from undesirable inserts, update and deletion dependencies;

• 2. To reduce the need for restructuring as new types of data are introduced

• 3. To be more informative to users; • 4. To make simplify query statistics

E.F. Codd, "Further Normalization of the Data Base Relational Model"

69

Normalization (Date & Codd)

First Normal Form: Create separate tables for each group of related data and identify each row with a unique column or set of columns (the primary key). I.e. Use Codes not Values

Second Normal Form: Remove subsets of data that apply to multiple rows of a table and place them in separate tables and create relationships between these new tables and their predecessors through the use of foreign keys. I.e. Store one “logical” table per “Table”

Third Normal Form:Remove columns that are not dependent upon the primary key. I.e. Do not store data that is not related to key

70

Metastorm Actual Table (Imaging_Services_LI)

Logical Header Table info (2NF)

Logical Art Table Info (2NF)

Work OrderLines

LogicalNon ArtData (2NF)

Miscellaneous Data (Not Shown in this sample)

Values Not Codes(1NF issue)

Logical Event Line

71

Metastorm

Note: These are problems with the Metastorm architecture, The Metatstorm developer needs to work within these confines.

Key Take Away:

MetaStorm is very good for workflow, BUT, not it is not architected well for data management

72

Normalizing IRIS• Complex set of views created to make

IRIS “Look” as if it was Normalized • Stored in Separate Database • Cannot be used for update • Performance issues make these not

feasible with complex joins

Deliverables

Projects

Requestors

Workflow events

For header

Workflow events

for lines

Lines with

common info

Lines for other

request

Lines for nonart request

Lines for Event

request

Lines for Art

request

Headers for

Imaging Request

73

Documentation (DIR)

• Project Folder – \\nga\group\T\TDP\Projects\DIR- DIV Images Request System

• Documents – Converting Metastorm Schema in a Reportable ERD model (DIR)-

20070919a 1 1.doc

74

SQL Server Database

Web Browser

Architecture


`

76

Screen Customization

Very hard to say “no” since, it so easy … However,I really demand a very good reason, and prefer

To avoid this . . .

Lots of this, can cause serious maintenance issues

Override eventsOverride events can be added in define locations

This little block of code says 1. Prompt user message to load approvals 2. If selected, Call database routine to do it

The goal is to keep the maximum amount of code in the database, in case we want to a new frond end .Making it “upgrade” or “migration” proof

77

Business Modeling through Data Model

• Define Relationships Between Data

Parent

Eye Color: Bwn

Child

PrimaryHome

ChildChild

78

SQL Server Database

Web Browser

Architecture


`

79

DCLPA (July 2008)The current Filemaker database on

the MAC is having lots of problems, I need help

Lets help them

This sounds like an opportunity to test out the

ideas I have to fix the Gallery's Data Problems. Lets do it

O.K. Sounds Feasible

This will help Conservation figure out how we do things… Were on board!

Greg and Art, Can you Help us

80

DCLPA (July 2008)


Will you change to a PC, then we’ll give you Microsoft Access

Hmm, We really cannot ignore the requirement for MACs. For that matter,

expense to install is so high, a web solution would save the gallery the most money.. The

Web should be a requirement.

In that case, “Nevermind”

81

DCL-PA Database

• Existing Filemaker Database deemed inappropriate

• TDP brought in to DCL-PA to address storage of filemaker application issues

• TDP was also simultaneously undertaking an Data Quality (DQ) initiative to connect system to authoritative sources.

• DCL-PA was in a good position to leverage both initiatives

• DCL-PA was adopted as DQ Candidate

83

Data Model Iterations

Type of Work

Type of Work Name

Major?

Type_of_work_ind

Event Group

Conservators

Request

Request Events

Request_id

Event_id

Started_on

Completed_on

Conservator_id

Description

BT_Image_Workorder_line_id

DT_Image_Workorder_line_id

AT_Image_Workorder_line_id

Extensis_Image_Pointer

Request_id

Request_event_status_id

Request_status_id

Object_id

Description

Conservator_id

Gallery_id (Badge number)

Name

Analog Image Locations

Name

Analog_Image_location_id

Events

Name

Event_id

Event_Group_id

Event_Group_id

NameRequestEvent_id


TMS (External)

Object_id

IRIS (External)

WorkOrder_Line_ID

Request Event Statuses

Name


Event_Group_id

Employees (External)

Gallery_id

Request Status Request_status_id

Name

Request_date

Requestor_id

Requestors Requestor_id


NameType_of_work_ind

Type_of_work_ind

CommentDCL Paper Portfolio Images


Type of Work

Event Group

Conservators

Request

Request Events

Request_id

Event_id

Started_on_Datetime

Completed_on_Datetime

Conservator_id

Description

BT_Image_Workorder_line_id

DT_Image_Workorder_line_id

AT_Image_Workorder_line_id

Comment

Request_id


Request_status_id

Object_id

Short Description

Conservator_id


Name

Analog Image Locations Analog_Image_location_id

Events

Event Name

Event_id

Event_Group_id

Event_Group_id

Event Group NameRequestEvent_id


TMS (External)

Object_id

IRIS (External)

WorkOrder_Line_ID

Request Event Statuses

Status Name


Event_Group_id

Employees (External)

Gallery_id

Request Status

Request_status_id

Name

Request_datetime

Requestor_id

Requestors Requestor_id


Request Status Name

ClassificationAccession #ArtistTitleDate/PeriodMediumPaper/Support(s)Mark(s)Inscription(s)DimensionsDonorCollectionLoan Dates

Requestor NamePhone Number

Requestor Name

ClassificationAccession #ArtistTitleDate/PeriodMediumPaper/Support(s)Mark(s)Inscription(s)DimensionsDonorCollectionLoan Dates

Long Description

Event Name

Analog Location Name

Status Name

Name

Analog Location NameGallery Name

Gallery Name

Gallery Name

Event Group Name

Type_of_work_ind

Type of Work Name

Major?

Type_of_work_ind Type of Work Name

Type_of_work_ind

Location Code – translated to general code

Location Code – translated to general code

DCL Paper Portfolio Images Extensis_Image_Pointer


AnalogImageLocationsAnalogImageLocation_id

Name

LastUpdateUserName

LastUpdateTime

RequestReportsRequestReport_id

Request_id

ReportType_id

StartedDate

EndedDate

Conservator_id

Description

User_id




Report

LastUpdateUserName

LastUpdateTime RequestReportStatusesRequestReportStatus_id

Name


ReportType_id

LastUpdateUserName

RequestStatusesRequestStatus_id

Name


LastUpdateUserName

LastUpdateTime

UsersUser_id

Username

Password

email

LastUpdateUserN...

LastUpdateTime

WorkOrderTypesWorkOrderType_id

Name

LastUpdateUserName

LastUpdateTime

RequestReportWorkOrdersRequestReport_WorkOrder_id

RequestReport_id

WorkOrder_Line

WorkOrderType_id

LastUpdateUserName

LastUpdateTime

RequestsRequest_id

TMSObject_id

RequestReason_id

RequestType_id

RequestStatus_id

Requestor_id

Comments

QuickNotes

RequestDate

LastUpdateUserName

LastUpdateTime

Exhibition_id


RequestorsRequestor_id

Gallery_id

Comments


LocalVersionOfName

Classification_id

LastUpdateUserName

LastUpdateTime

test_id

---> TMS

---> ASD. Employee Table

---> IRIS Line

----> ASD Employee

ReportTypesReportType_id

ReportTypeGroup_id

Name

LastUpdateUserName

LastUpdateTime

ReportTypeGroupsReportTypeGrou...

Name

isTechnicalAnalysis

isDetailedExam

ConservatorsConservator_id

Gallery_id

Comments


LocalVersionOfName

LastUpdateUserName

LastUpdateTime

RequestReportDACsRequestReport_DAC_id

RequestReport_id

DAC_Record_id

Comments

LastUpdateUserName

LastUpdateTime

XRFStatusesXRFStatus_id

Name

LastUpdateUserName

LastUpdateTime

RequestTypesRequestType_id

Name

isMajor

isMinor

LastUpdateUserName

LastUpdateTime

isQuick

isBulk

ApprovingUsersApprovingUser_id

User_id

Name

LastUpdateUserName

LastUpdateTimeApprovals

Approval_id

Name

isApproved

LastUpdateUserName

LastUpdateTime

ApprovingRolesApprovingRole_id

Name

LastUpdateUserName

LastUpdateTime

ApprovingUserRolesApprovingUserRole_id

ApprovingUser_id

ApprovingRole_id

Comments

LastUpdateUserName

LastUpdateTime

RequestApprovingRolesRequestApprovingRole_id

Request_id

ApprovingRole_id

ApprovingUser_id

Approval_id

Comments

LastUpdateUserName

LastUpdateTime

RequestTypeApprovingRolesRequestTypeApprovingRole_id

RequestType_id

ApprovingRole_id

Comments

LastUpdateUserName

LastUpdateTime

---> Portfolio (DCL-PA)

Refined the data model as we to envision how use cases would work with it !

84

RequestReports

RequestReport_id

Request_id

ReportType_id

StartedDate

EndedDate

Conservator_id

Description

User_id




Report

LastUpdateUserName

LastUpdateTime

RequestTMSObject_id

Attachment

Image


Column Name

RequestReportDACs


RequestReport_id

DAC_Record_id

Comment

LastUpdateUserName

LastUpdateTime



RequestReport_id

FileName

Comment

Column Name



Name


Request_id



Request_id


Classification_id

isNGA

Comment

Column Name



Request_id

ApprovingRole_id

ApprovingUser_id

Approval_id

Comment

LastUpdateUserName

LastUpdateTime

Column Name



RequestReport_id

WorkOrder_Line

WorkOrderType_id

LastUpdateUserName

LastUpdateTime

Column Name


RequestReport_id

TechnicalNotes

BetaRadiograph



MicroFadometer



ColorSpectroscopy

None

Other

OtherDescription

Column Name

RequestTMSObjects

RequestTMSObject_id

Request_id

TMSObject_id


Column Name

Requests

Request_id


RequestReason_id

RequestType_id

RequestStatus_id

Requestor_id

Comment

QuickNotes

RequestDate

LastUpdateUserName

LastUpdateTime

Exhibition_id




Loan_id

LoanActivity_id

Column NameSimplified Data Model

85

Request

Non TMS Object Request

Reports

Tallies (Bulk Counts)

Approvals

Report related Report

Tech Info

Report related IRIS

Work Orders

Report related Portfolio

Images (DAC)

Report related attachements

TMS Object

Simplified Data ModelRequest to fix paper

Request id: 1Date: 1/1/90

3 XX Y ApprovedX Approved

Deputy Approved10 Papers

10 Prints

Examination

Tech Notes For exam

Work Order 10Work Order 10

Image ID: 16Image ID: 16

Excel SpreadsheetExcel

Spreadsheet

NonNGA Item 3NonNGA Item 5

NGA Item 2NGA Item 1

Treatment

Tech Notes For exam Work Order 10Work Order 23

Image ID: 16Image ID: 14 PDF

86

Request


Reports


Approvals


Tech Info

Report related IRIS

Work Orders


Images (DAC)


TMS Object

Request

Request Reports



Tech Info

Report related IRIS

Work Orders


Images (DAC)


TMS Object /

NonTMS Object

Recent Changes

87

SQL Server Database

Web Browser

Architecture


`

88

Rapid application developmentFrom Wikipedia

• Rapid application development is a software development methodology, which involves iterative development and the construction of prototypes. It is a merger of various structured techniques, especially the data driven Information Engineering with prototyping techniques to accelerate software systems development.

• RAD calls for the interactive use of structured techniques and prototyping to define user's requirements and design the final system. Using structured techniques the developer first builds preliminary data models and business process models of the business requirements. Prototyping then helps the analyst and users to verify those requirements and to formally refine the data and process models. The cycle of models, then prototypes, then models, then prototypes and so forth on, ultimately results in a combined business requirements and technical design statement to be used for constructing new systems.

The speed and agility of the RAD tools, allows the communication of the design process to also take place In the “Prototype” instead of only the “Design Document”.

“If a picture is worth a thousand words, a prototype is worth a thousand Pictures” - Anonymous TEXT TEXT+DIAGRAMS (MODELS, UML,DFD,…)TEXT+DIAGRAMS + PROTOTYPES

89

The Big Question ?

• Will the “Prototype” ever be good enough to become a real “applications”?

Then this Prototype model become “real” powerful

• Historically “No”

• What’s new?

• Dynamic Languages PHP, Ruby, JavaScript and relational database are now making it possible (and inexpensive)

90

RAD\Prototype Development

Verify With User

Collect user Requirements

DevelopInitial

Design

Develop/Update

Prototype

Collect\Document changes

User Approved System

91

PHPRunner (step 1)Map relationships between tables

92

All the web pages in the application

93Web Pages related to structure

Of request

Data Model

Request


Reports


Approvals


Tech Info

Report related IRIS

Work Orders


Images (DAC)


TMS Object

Where did “Work Log” come from?Why doesn't “Request” point to “Request Classification count?”

User Interface Model

94

User Interface Model

95

User Interface Model“Request” and “Work Logs” both store data in the same place (Request Table), they as are only different in which tables are Linked to them, what appears on there screens and processing rules.

RequestAKA “Treatment Reports” : Reports Allowed : Object Required : Bulk counts not allowed : Have linked Object Report : Approvals (Major Only)

Work Logs : Do not contains reports : Allow for Tallies : Object Optional : No Approvals : Simple and Freeform

Request

96

Links to Authoritative Sources

Portfolio

IRIS

EmployeeDatabase

TMS Public Data Extract

TMS

Nightly Rebuild

97

Data Sharing “vwRmt”

Create view [dbo].[vwRmtEmployee] as (SELECT [ID#] . . .

,[RestrictAlpha] ,[PayPlan] ,[Sponsor] ,[Super#]from NGA-LANDESK.Employee.dbo.[T-Employees]where [LastName] is not null

98

Documentation

Project Folder •\\nga\group\T\TDP\Projects\PCD - Paper Conservation Database

Keys Documents

•General Design - Draft (PCD) 20090327a.doc•DCL-PA Database 2.ppt

99

Future Features

• Tabs between children in table

• More links to Reports

• WebCast for training

• Single Signon Integration

• Web Reporting

• Audit Trail

• Mail and Public folders

100

Tricks

• Triggers to allow multiple selection features • Cascading Drop Down to Emulate Field existence • DB Constraints formatted as error messages• JavaScript as column values • Other . . .

101

Toolkit Problems

• Multiple Fields to perform different types of selects

• Only 2-Level Hierarchy supported

• UI customizations might be lost during regeneration

102

IssuesMajor IssuesMajor Issues • Maintenance

– Bugs, Enhancements, Upgrades, Reports– What if the toolkit can’t do it ! (Limitations/Tricks)– Generated code issues (re-customizations)– Dependence on toolkit (Double edge sword)

Other IssuesOther Issues • Performance

– Network, Query, Concurrency, Scalability

• Misc– Security, Upload File Placement, – Editing Capabilities (i.e.Spellcheck), – Search (Attachments), Audit Trail, Delete Logic

103

Other Issues

• Scope• Production Planning • Testing and QA• Data Conversion • Procedures and documentation• Web Reports • Iterations and upgrades • Integration with existing processes

104

End of Main Presentation

• Questions

Creating an RAD Authoratative Data Environment

Technology

Transcript of Creating an RAD Authoratative Data Environment