Web Trns•port TechnologyWeb Trns•port Technology

Trns•port User Group Conference

Bob DeHoff, Info Tech, Inc

October 9, 2008

TopicsTopics

• Technical TRT• Deployment• Installation and Migration• Database and Data Interfaces• Customization• Anticipated Future Infrastructure• Resources• Q&A

Technical TRT (formed September 2007)Technical TRT (formed September 2007)

Member Agency Role

Don Grayson Mississippi Chair, TTF member

Todd Bergland Minnesota Vice Chair, TTF member

Chris Calvert Florida Member

Igor Crljenica Michigan Member

Thane Mitchell Texas Member

Lee Burbach Nebraska Member

Brian Goluska Michigan Member

Sarah Collins Louisiana Member

DeploymentDeployment

Th ickC lie n t s

B a tchS e rv e rs

D a ta ba s eS e rv e r

Th inC lie n t s

B a tchS e rv e rs

D a ta ba s eS e rv e r

Te rm in a lS e rv e rs

Th inC lie n t s

W e b /A pplica t io n

S e rv e rs

D a ta ba s eS e rv e r

C /S T rns •po rt

C /S T rns •po rtus ing

T e rm ina lS e rve r

w e b T rns •po rt

Business Processing CapacityBusiness Processing Capacity

Business processing is distributed differently in web Trns•port than it is in Client/Server Trns•port…

Th ickC lie n t s

B a tchS e rv e rs

D a ta ba s eS e rv e r

Th inC lie n t s

B a tchS e rv e rs

D a ta ba s eS e rv e r

Te rm in a lS e rv e rs

Th inC lie n t s

W e b /A pplica t io n

S e rv e rs

D a ta ba s eS e rv e r

C /S T rns •po rt

C /S T rns •po rtus ing

T e rm ina lS e rve r

w e b T rns •po rt

C/S Trns•port Processing DistributionC/S Trns•port Processing Distribution

C/S distributes processing across both user desktops and batch servers

Th ickC lie n t s

B a tchS e rv e rs

D a ta ba s eS e rv e r

Th inC lie n t s

B a tchS e rv e rs

D a ta ba s eS e rv e r

Te rm in a lS e rv e rs

Th inC lie n t s

W e b /A pplica t io n

S e rv e rs

D a ta ba s eS e rv e r

C /S T rns •po rt

C /S T rns •po rtus ing

T e rm ina lS e rve r

w e b T rns •po rt

C/S Trns•port with Terminal ServerC/S Trns•port with Terminal Server

Terminal servers shift most of the load off the desktop clients

Th ickC lie n t s

B a tchS e rv e rs

D a ta ba s eS e rv e r

Th inC lie n t s

B a tchS e rv e rs

D a ta ba s eS e rv e r

Te rm in a lS e rv e rs

Th inC lie n t s

W e b /A pplica t io n

S e rv e rs

D a ta ba s eS e rv e r

C /S T rns •po rt

C /S T rns •po rtus ing

T e rm ina lS e rve r

w e b T rns •po rt

Web Trns•port Processing DistributionWeb Trns•port Processing Distribution

All processing performed by web/application servers.

For good performance, more capacity is needed for these servers than what you have for your C/S batch servers.

Deployment – Minimum ConfigurationDeployment – Minimum ConfigurationClients

Web/Application Server

Hosts 3 Logical LayersPresentationBusinessData

User session lives on server

Database Server

Scale Up Using an IIS Web GardenScale Up Using an IIS Web Garden

Multi-CPU server configured as an IIS “Web Garden”

Local ASP.NET State Server sharesuser session across Worker Processesin Web Garden

Scale Up More With an IIS Web FarmScale Up More With an IIS Web Farm

L o a d B a la nc e rMultiple servers configured as a “Web Farm” using a Load Balancer suchas Windows Network Load Balancing

ASP.NET State Server shares session across servers in the farm. Can be hosted onone of the web servers or a separate server

The configuration shown here is the one used for web Trns•port performance testing

Where’s “Batch”?Where’s “Batch”?

• Long running processes run as asynchronous processes on the Web/Application Servers– Examples: Bid import, Export to SiteManager– Does not use Distributed Processing Services

• User notified of process completion• User can view and manage process history• User can schedule processes such as reports to

run on a periodic basis– Load-balanced across farm when scheduled– Scheduler uses Windows Task Manager API

Release Test PlatformsRelease Test Platforms

ApplicationComponent Language and Tools Platform Components

Clients HTMLCSSJavaScript

Windows XP SP2Internet Explorer 6/7, Firefox 1PDF Viewer

Web/ApplicationServers

.NET Framework 2 SP1ASP.NETC#ActiveReports

Windows Server 2003 R1 SP2IIS 6Native database drivers*

Database Server DBMS Windows Server 2003 R1 SP2**Oracle 10g (10.2.0.3)DB2 8.2

* ODBC drivers required for Data Migration Utility only

** Oracle 10g also supported on Sun Solaris and HPUX database servers

Installation and MigrationInstallation and Migration

InstallationInstallation

• Step-by-step installation guide– Includes web server and garden configuration guidelines– Web farm configuration is dependent on the load

balancing solution your agency chooses

• Software installation package– Application software and online documentation– Database scripts

• Skills needed– IIS and Windows server administration– Database administration

Installation ValidationInstallation Validation

• Automated testing package (optional install)• Suite of thousands of automated tests

– Typically takes a few hours to run

• Use with a dedicated database instance– First thing tests do is empty all database tables!– Data Migration Utility tests clear PES/LAS database too!

• Tests must be run against base configuration– Tests depend on base configuration, so customizations

can cause tests to fail

• To test product updates, maintain a separate application instance of the base configuration

Migration from Client/Server PES/LASMigration from Client/Server PES/LAS

• Data Migration Utility– Command-line program run outside of web Trns•port– Direct database-to-database pull/push

• Migrates business data, including generic fields• Migrates user list, but not passwords or privileges• Validates data against web Trns•port rules

– PES/LAS referential integrity issues are most common issues pre-release and beta agencies have experienced

• Migration only supported for latest PES/LAS release (5.9c)• Customizations such as RTF, custom processes, and

PowerBuilder customizations not migrated

Database and Data InterfacesDatabase and Data Interfaces

Data PartitioningData Partitioning

• At the physical level, separate sets of Preconstruction and Construction tables– Example: Proposal table and Contract table

• But only one set of reference data• All in one unified database instance• Contracts transition from Preconstruction to

Construction when ready to progress• Where needed, Construction tables include both

original and current information– Example: Original and current item quantity

Data PartitioningData Partitioning

Preconstruction Tree Construction Tree

Reference Tree

Trns•port Database

Data Partitioning – Why?Data Partitioning – Why?• Performance

– Most functions require either only Preconstruction data or only Construction data, not both

– Extra unneeded data would be fetched often if Preconstruction and Construction data stored in same table

– For the few functions that need data from both sides,use views that pull from both

• Lower system complexity– Example: Contracts can start in Preconstruction or directly in

Construction - rules are more complex if everything is combined• Ad hoc access

– Sensitive engineer’s estimate information separated from construction information

– Ad hoc queries don’t have to deal with separating preconstruction and construction information

Accessing “PROPITEM” DataAccessing “PROPITEM” Data

• Contained in web Trns•port Proposal, Proposal Section, and Proposal Item tables

• Use workflow phase rule to tell system when proposal structure is locked in– “FixProposalItem” rule– Typically at phase where bids are received

• Prior to locking proposal structure…– User function to fill proposal’s data into tables– Analogous to LAS Initialize Letting Proposals function

• Once FixProposalItem rule is triggered…– Data is permanently stored– Can be accessed and re-accessed from then on

Record Creation and Update StampsRecord Creation and Update Stamps

• Similar to SiteManager• Record stamp columns in each business table

– Who created it and when– Who last updated it and when

• Filled in whenever row is created or updated• Provides some traceability

– Doesn’t address deletes– Doesn’t provide transaction-by-transaction history

Database Transaction LoggingDatabase Transaction Logging

• Enabled through agency option• Option setting applies to all servers in farm• Transactions are logged to a database table to

provide a unified, farm-wide log• Option to only log for one specific user

– Use this to focus on something specific

• Logging adds overhead, so leaving logging on all the time is not recommended

Data InterfacesData Interfaces

• Preconstruction interfaces to C/S and standalone Trns•port– 2-way: CES, Estimator, SAPW, SiteXchange– 1-way: CAS, SiteManager, BAMS/DSS– Plus LAS Data Views for BAMS/DSS

• CRLMS interfaces to C/S Trns•port (all 1-way)– PES/LAS, CAS, SiteManager

• Interfaces to external systems– Bidding systems (e.g., Bid Express)

• Export to Expedite, import bids

– aecXML standard (e.g., Bentley Quantity Manager)• Export reference data, import project

CustomizationCustomization

Data CustomizationData Customization

• Add custom fields to any table– As many as you need– Of any of the standard data types

• Customize data validations– Further constrain base field validations– Define validations for your custom fields– Set of validation rules provided

• Set default value for any field

User Interface & Report CustomizationsUser Interface & Report Customizations

• User interface– Change entity and field names

• “County” “Parish”, “Project Id” “Job Id”

– Hide unused fields– Make fields read-only– Change field order on pages

• Reports– Interactive report design tool– Change report format– Can add queries against the database– Can add business logic

Custom Reports & ProcessesCustom Reports & Processes

• Custom reports– Interactive report design tool– Create custom reports using ActiveReports– Define who can run them– Optionally drive off same datasets as base reports– Define runtime parameters for user to fill in

• Custom processes– Make agency programs accessible to users

• Anything that can be run from a command line

– Define who can run them– Optionally send output back to user’s browser

Custom InterfacesCustom Interfaces

• Data Loader– Load data from a flat file– Predefined columnar file format– Input file can include data from multiple tables– Validates data against web Trns•port rules– Similar to C/S batch load, but input format automatically

includes generic fields you define

• Custom Exports– Use ActiveReports to create flat files– Columnar, XML, and Excel formats– Optionally drive off same datasets as base reports

Anticipated Future InfrastructureAnticipated Future Infrastructure

Anticipated Future InfrastructureAnticipated Future Infrastructure

• Roughly 50% of Preconstruction project was devoted to developing foundation that will be used in the other business areas

• Anticipated future infrastructure aspects…– Access to Trns•port by external stakeholders– Meeting the needs of field personnel (standalone)– Custom data collection (templates)– Decision support (Analytical Store)– Section 508 & WCAG accessibility standards compliance

ResourcesResources

• Web Trns•port home page on Cloverleaf– http://www.cloverleaf.net/webtrnsport– Hardware/Software requirements– Platform plan– Web Trns•port versus C/S summary of changes– Mapping from C/S to web Trns•port database– etc.

• Web Trns•port Online Help– See System Management section

Q & A