InteragencyCollaboration 1-27FINAL

70
Collaborative Application Development Among New York State Agencies Strategy for collaborative, cost-effective development of computer applications supporting government business processes January 11, 2008 State of New York Department of Health Department of Labor Department of Taxation and Finance Office of Children and Family Services Office of Temporary and Disability Assistance

description

 

Transcript of InteragencyCollaboration 1-27FINAL

Page 1: InteragencyCollaboration 1-27FINAL

Collaborative Application DevelopmentAmong New York State Agencies

Strategy for collaborative, cost-effective development of computer applications supporting government business processes

January 11, 2008

State of New York Department of Health Department of Labor Department of Taxation and Finance Office of Children and Family Services Office of Temporary and Disability Assistance

Page 2: InteragencyCollaboration 1-27FINAL

Table of Contents

...................................................................................................................................................................Introduction 3

..................................................................................................................................................State OCIO Principles 5

..................................................................................................................State Enterprise Architecture Principles 10

....................................................................................................Approaches Specific to Application Development 12

.........................................................................................................................................Open New York (OpenNY) 19

..............................................................................................................OpenNY Governance and Quality Control 21

....................................................................................................................................Service Oriented Architecture 28

..................................................................................................Service Oriented Architecture – the Business Case 34

................................................................................................................Service Oriented Architecture Formalism 37

..........................................................................Service Oriented Architecture Governance and Quality Control 43

....................................................................................................................................................................Conclusion 47

..........................................................................................................Appendix A - Open Standards Organizations 49

...........................................................................................Appendix B – NYS Enterprise Architecture Principles 53

......................................................................................Appendix C – Manifesto for Agile Software Development 69

Collaborative Application Development Among New York State Agencies

2 / 70

Page 3: InteragencyCollaboration 1-27FINAL

A. Introduction

In his 2007 State of the State message, Governor Eliot Spitzer called for changes that “...transform a government that is structurally oriented to resist change into one that is oriented to embrace it.” He went on to say that “we must work together to implement structural reform at every level of government to make it more flexible and adaptive to change.” For technology, change in government policies and procedures translates into responsiveness and agility: a requirement that technology-driven processes are aligned with the business of government. As the business of government changes to better support the needs of the people, so must the information technologies which support those business processes change to better align with governmental processes. A government of innovations—a government which embraces an innovation economy—requires technology which innovates. In that spirit, we offer an approach that provides for new and effective ways to support existing procedures as well as methods for collaborating on cost-effective solutions for the future, while still addressing the diversity of need in New York State communities.

Each of the New York State agencies which comprise the study group for this white paper have a number of governmental business partners, organizations with which we conduct the public’s business. In addition to each other, these partners include local governments, other state governments, inter-state authorities, federal agencies, private not-for-profit corporations, employers, agents of employers, commercial interests, and of course other New York State agencies not in the study group. Moreover, our agencies have a wide variety of individual clients and customers, including individual taxpayers, unemployment insurance claimants, Medicare claimants, and citizens who live beneath the poverty line. We cannot serve all of these organizations and individuals through a single, one-size-fits-all, monolithic application.1 Human-to-computer interfaces with governmental business processes (whether through an on-line tax filing mechanism, an interactive telephone claims system, caseworker-assisted data entry, or a business-to-business bulk data transfer) must be shaped to speak to individual communities of need. And of course the business processes that are internal to government agencies, and that are used to serve these communities, vary widely with governmental program designs: a “case management system” for mental

Collaborative Application Development Among New York State Agencies

3 / 70

1 There are many different cateories of computer software. The distinctions are important in the context of this paper especially because different categories of software use development methodologies unique to them. Application software is a computer program (or suite of programs) that addresses some task that ordinary humans need to have done. It contrasts with system software (such as an operating system like Windows or UNIX), which provides underlying support for applications, but does not provide a direct service to users. Business application software refers to applications which support the flow of office work through an organization. Engineering application software is software supporting systems like the Space Shuttle. Office automation applications, like Microsoft Word™ or email systems, provide generalized utility to users.

Page 4: InteragencyCollaboration 1-27FINAL

health patients is nothing like a legal “case management system” such as that used at the Attorney General’s Office.

Diversity in application architectures translates to choice and accommodation in how services are delivered through automated solutions. But over-diversification can sometimes become wasteful. Waste arises in a number of ways in information systems. One way is when individual technology units go off in different directions, adopting their own standards and reducing opportunities for aggregate acquisitions, code-sharing, and reuse of solutions. This is an especially large problem in New York State where governmental business processes grew up without any overarching strategy. But another way that waste arises is when aggressive consolidation—often as a reaction to too much diversity—ignores the complexity of government programs and individual business needs in favor of homogenization and standards for the sake of standards. Approaches to code consolidation can often be more costly than accommodating some diversity.2 3

Happily, newer approaches—supported by some innovative technologies—can provide the needed flexibility in solution development, without waste.

Collaborative Application Development Among New York State Agencies

4 / 70

2 Natis, Yefim 2005 “Applied SOA: Conquering IT Complexity Through Software Architecture,” Gartner Research paper no. G00128127, May 25.

3 Strassmann, Paul A. 1995 The Politics of Information Management: Policy Guidelinesp. 57 ff.

Page 5: InteragencyCollaboration 1-27FINAL

B. State OCIO Principles

At the behest of Melodie Mayberry-Stewart, the New York State CIO (State CIO), the agencies which are partnering in this study presented a series of “guiding principles” that can shape how State agencies work together going forward. These principles were articulated by the State CIO at the September 2007 Government Technology Conference in Albany and were presented on her behalf by William Travis, CIO at the Office of Children and Family Services, before a meeting of the Governor’s Economic Security Subcabinet in early October 2007. They were also presented by the State CIO herself at a meeting in Connecticut of state welfare agencies.

In this section of the report we provide further clarification of these principles, defining terms with which non-technology managers might not be familiar. Throughout this document we provide additional clarification of these concepts.

Provide for Interoperability using Open Standards and seamless data sharing through common enterprise platforms. Interoperability refers to the ability of applications to exchange data, although they might have been

written in different computer languages, operate on different makes and models of computer, or use data internally that is organized in a unique way.4

Open Standards are those protocols and methods which are defined by such standards-setting bodies as OASIS, NIST, ISO, W3C, IEEE.5 By favoring Open Standards over closed systems, the State avoids lock-in, a situation in which an organization’s information assets become dependent upon a particular company’s proprietary approach. Such a situation leads to the inability of organizations to control costs since they have made themselves subject to the whim of companies whose technology cannot be easily replaced, except at great cost. Through use of standards that are openly defined in the industry all vendors have the opportunity to participate in contract opportunities. Moreover, products which adhere to Open Standards can be used together as components in a larger system, regardless of manufacturer. This latter quality is called Interoperability.

Collaborative Application Development Among New York State Agencies

5 / 70

4 The ability to exchange data, although it might be represented differently in each application, is the hallmark of interoperability. Using the XML markup language, computer applications can interoperate, even though one might represent the names of clients as one 65-character text item, while another uses separate items of varying length to represent first name, middle name, last name, etc. Nonetheless, the disparity between data organizations adds effort. Later in this document, we propose a collaborative body—along the lines of international standards bodies—to arrive at standard data definitions and structures.

5 See Appendix A for a description of these organizations.

Page 6: InteragencyCollaboration 1-27FINAL

Seamless data sharing is the ability to send data between two applications without any special intervention. The classic counter example is transfer of data using a method where a printed copy of reports are generated out of one computer system, so that a clerk can manually enter data off the report into another computer system. In contrast, seamless sharing occurs through totally automatic transfers.

Modern data transfers occur through a mechanism called “messaging.” A message is package of data that is sent over a network that includes information about security, routing, and priority. Such messages are typically encoded in a mark-up language called XML. In effect, XML is today’s lingua franca of data exchange.

Common enterprise platforms means that the many computers (IBM, Sun, Unisys, Dell, Apple) and computer operating systems (AIX, Unix, Linux, Z/OS, Windows, OS X) which are components of New York State’s hardware infrastructure are to be considered a shared resource. Sharing can occur in a number of ways. One way is through use of the same computer by two agencies. Another form of sharing occurs when a computer operated by one agency provides a service for another agency.6

Deploy an “Open New York” community approach to facilitate peer review and enhance quality control.

Open New York is our branding of the Open Source model of collaborative development. While formal, joint development projects and contracts for individual development for sharing have their place from time to time, Open Source development does not require those formalities, yet can lead to cost-saving and very nimble responses to program requirements.7 8

Leverage prior IT investments with software reuse when feasible to achieve greater cost efficiencies.

A fundamental principle of modern application development, leveraging prior investments, is often described with the phrase, “standing on the shoulders of those who came before us,” or more simply as “reuse.” With the development of the Object Oriented model of programming, reuse was effected,

Collaborative Application Development Among New York State Agencies

6 / 70

6 Common platforms does not mean, however, that all applications must use the same make and model computer. Most technology managers would like that to be the case because it is easier to administer one kind of thing, and there is the presumption (not always borne out in empirical studies) that platform consolidation reduces cost. The reality, however, is that different computer platforms can have different capability and better service specific business needs. That, combined with the fact that shifting costs, competitive bidding requirements, and need to maintain vast libraries of legacy (i.e. historical) applications, draws us all to the conclusion that we have to be good at managing heterogeneity.

7 Riehle, Dirk n.d. “The Economic Motivation of Open Source Software: Stakeholder Perspectives,” Computing Practices. SAP Research.

8 Gallagher, Scott, “Patterns of Open Innovation in Open Source Software,” in Henry Chesbrough, Wim Vanhaverbeke, and Joel West, eds., Open Innovation; Researching a New Paradigm. Oxford Press, Cambridge, p. 82 ff.

Page 7: InteragencyCollaboration 1-27FINAL

at least in principle, through the sharing of class objects: not so dissimilar from code libraries associated with earlier structured languages, like C. Today, however, reuse has a better chance of widespread realization through sharing of Services within a Service Oriented Architecture. Credite Suisse reports that 75 to 80 percent of required Services for any new application are already available in its repository of Services.9 A Service Oriented approach to reuse is superior to earlier attempts because it is not necessary for all participants in the sharing to use identical programming languages and platforms (something which is frequently undesirable).10 As is described in more detail later, Service Oriented Architecture allows programs to interoperate, regardless of the development platforms on which they were built or in which they execute. And by making possible for the Services themselves (separately compiled, externally orchestratable programs) to be reconfigured into applications on the fly, the strictures of sharing programming code gives way to sharing what a business process is actually after.

Implement agile systems development approaches to improve speed to market.

Speed to Market. Such phrases as “speed to market,” “time to market,” and “delivery interval” speak to the requirement that IT deliver useful products (computer applications) to governmental programs on short order. Changes to business processes occur regularly and persistently throughout the history of any program. Regulations or procedural changes or outreach strategies in a program like Food Stamps can occur many, many times throughout a fiscal year. If IT cannot respond to such changes rapidly, with effective solutions, opportunity is lost and waste rises.

Agile systems development.11 A modern method of development, called, “Agile, Evolutionary Development,” contrasts with an historical method called “Waterfall.” Waterfall method requires long periods for comprehensive requirements gathering, followed in sequence by a design phase, then a programming phase, then a testing phase. (There can be more phases than this in some approaches.) Waterfall works well with static systems, such as work processing programs like Word, or large engineering projects like the software which supports the Space Shuttle. It has been found largely ineffective in producing business applications. Agile development, on the other hand, divides application development work into short-term intervals (as short as two weeks) at the end of which some deliverable results. Further “iterations” of this result in adding more components. Along the

Collaborative Application Development Among New York State Agencies

7 / 70

9 Krafzig, Dirk, Karl Banke, and Dirk Slama. 2005 Enterprise SOA; Service-Oriented Architecture Best Practices. Prentice Hall, New York, p. 356.

10 It is generally acknowledged today that languages and platforms should be chosen to meet the need of the business objective. Domain specific languages (DSL) have been developed to address specific business problem areas, making reuse of lines of program code difficult if not impossible.

11 Larman, Craig 2004 Agile & Iterative Development; A Manager’s Guide. Addison Wesley, Boston.

Page 8: InteragencyCollaboration 1-27FINAL

way, component sub-assemblies are placed into production so that business units get early use, and so that changes to business plans can be accommodated immediately, rather than five years into a project.

Establish strong enterprise governance to ensure alignment of technology plans with business goals.

Governance is a detoxified term of art for authority and control. Governance can proceed from many quarters and via many methods. Where Information Technology is responsive to business needs, it flows first from the requirements that are articulated by policy makers and governmental program managers. The mechanics of governance assures that there is faithful adherence to contracts and service level agreements between executive leaders and technology managers on the one hand, and among technology managers themselves.

Alignment is a quality of IT organizations as well as IT products such that IT activities conform to the needs of program areas, as opposed to either expecting programs to conform to computer products, or allowing IT to steer a course that is of interest to engineers and technical managers regardless of whether those interests support business requirements.

Seek innovative collaborations to leverage State enterprise IT resources and assets.

Innovation is the result of taking risks to move outside a generally accepted standard with the result of obtaining new efficiencies. If the solution is worth the risk, we declare a success and modify the standard to accommodate the approach as an accepted pattern12. If the solution carried high tolls in cost, user dissatisfaction, and inefficiency, we have a failure and strengthen change controls to prevent experimentation along paths proven to end poorly.

CodicilsThere are some supplementary considerations that further amplify these principles:

COTS. One codicil is a caution against significant modifications to Commercial Off-the-Shelf (COTS) products. Use of COTS products is unavoidable; they provide pre-built components that can hasten the delivery of key functionality in application suites. Fortunately, many COTS products are based on Open Standards and easily interoperate. But all COTS applications also have a proprietary bent. When

Collaborative Application Development Among New York State Agencies

8 / 70

12 Pattern is a term of art in Information Technology. A pattern can be thought of as a formula for handling a certain set of recurring problems. Unlike a standard, which covers wide ranges of behavior and which are authoritative if adopted, patterns are specific to parochial problems and only suggest a possible solution. Think of a litigator who might have several pre-written paragraphs to insert into a brief if the case has certain characteristics. The paragraphs don’t work in every situation, and they frequently have to be tweaked for the particulars of a specific case. But without patterns, work would take longer and require that, for each instance of a particular problem, a solution would have to be re-discovered.

Page 9: InteragencyCollaboration 1-27FINAL

commercial applications are modified and extended, the risk is that they grow tentacles that lock them in to the enterprise architecture in deleterious ways. As COTS applications are modified to align as closely as possible with business needs, and when those business needs are far from the central functionality provided by the COTS application, we end up with a situation in which we no longer have the opportunity to substitute one COTS solution for another, as when monopolistic costs becomes an overarching problem.

Data Standards, Data Directories, and Ontologies. The notion of Interoperability is focused on the ability of disparate applications to share data. To assure inter-operable applications can occur, New York’s data models must use a common enterprise data model. Turning again to Open Standards, organizations which set international data exchange standards (now largely an exercise in defining the schemas that comprise XML “documents”) have methods for arriving at such frameworks.

Care is needed in expressing what is meant by data standardization, since there is a history in the industry of misusing standardization. As a State enterprise, we need to be wary of rigid data definitions, specifically in terms of standard sizes for data fields across agencies. Past attempts to reconcile one standard across every agency, and their external partners, stalled development and ultimately failed. A more practical alternative is a publish-and-subscribe paradigm (which is like the service registry idea today and what DTF did in OTC), where agencies can publish their data exchanges or services and participating agencies subscribe to them. This means the subscriber knows what they are getting (individual name as first, middle initial, last or as a string) and the subscriber can react - “let the buyer beware.” They don’t have to use the service, but if they want to then they can react. Otherwise the service publisher may have to change the service to meet some standard thus limiting productivity. Each consumer can do what he wants with the information returned.

Collaboration. Additionally, New York State governmental entities should work toward operating an integrated enterprise team. To that end, procurements should be guided by a single, integrated governance structure which is aligned with the enterprise strategy and informed by individual and collective agency requirements.

Collaborative Application Development Among New York State Agencies

9 / 70

Page 10: InteragencyCollaboration 1-27FINAL

C. State Enterprise Architecture Principles

In 2004, the OCIO approved and published “Enterprise Architecture.” The principles in this document were subsequently tied to the Annual Technology Plan, an on-line database of IT projects maintained by the OCIO which individual agencies update with projects which they individually authorize, and for which they seek funding through the State budget process. In recording their projects, State agencies are obliged to explicitly show adherence to these principles.

Appendix B contains the entire text of the New York State Government Enterprise Architecture principles. These principles are consistent with both the new OCIO principles above, as well as the specific objectives listed later in this document for Service Oriented Architecture and Interoperability, and agile development method. Though these principles arise from separate discussions and are aimed at addressing somewhat different problem spaces, they are stunning in their consistency. Here we cite those principles which deal especially with application development.

1. Maximize cost-effectiveness through shared use.

2. Reduce complexity.

3. Maximize flexibility.

4. Facilitate information sharing.

5. Facilitate data warehousing and data centric end-user solutions

6. Promote enterprise-wide data standardization

7. Facilitate common solutions for business processes shared by agencies.

8. Facilitate code re-use and smooth integration of business processes.

9. Business process re-engineering will be considered when defining requirements.

10. Achieve a standard look-and-feel to GUI design.

11. Define standard office automation suites

12. Employ thin client solutions where appropriate.

13. Promote interoperability and integration across all applications

14. Favor applications which are highly partitioned, modular in design, with components that are maximally decoupled and which use standards-based messaging protocols internally and externally.

Collaborative Application Development Among New York State Agencies

10 / 70

Page 11: InteragencyCollaboration 1-27FINAL

15. Achieve a uniform look-and-feel

16. Establish a finite set of reproducible configurations.

17. Assure scalability.

18. NYeNET will be the statewide backbone for enterprise applications.

19. Enterprise systems will adhere to all applicable security, confidentiality, and privacy strictures.

20. A single method of user authentication will be used across all applications.

21. Application-level security and data retention requirements will be maintained.

22. There will be a formal governance process for planning and managing the enterprise.

23. There will be a formal Enterprise Architecture review process.

24. Disaster recovery and business continuity plans will be in place.

25. New technological advances will be continuously reviewed.

26. Relevant business area experts will participate in application development.

27. All acquisition decisions will be made with a view of their impact on the overall enterprise.

28. A Total Cost of Ownership metric will be used in examining the cost of information technology.

29. Applications will be delivered in phases (“chunked up”) so as to prove interim deliverables for users.

30. Industry standard software engineering practices will be used in application development.

31. To the fullest extend possible, acquired applications and infrastructure will be commercially viable, proven solutions.

32. Products and solutions should use open standards to facilitate inter-operability.

Collaborative Application Development Among New York State Agencies

11 / 70

Page 12: InteragencyCollaboration 1-27FINAL

D. Approaches Specific to Application Development

The sets of principles describes above speak to the management of the entire computing enterprise. To examine these principles in light of the application development efforts that most concern us, we drill down further to explore specific approaches to application design. Software development is a complex undertaking that requires significant expertise and skill. It also requires active participation by all those involved in the process.13

Portability Versus Interoperability

Throughout the 1990’s, a discussion raged especially between Sun Java adherents on the one side and promoters of the Windows-only variant of Java, J#, on the other. Sun argued that the model for software development going forward should be based on Portability: code—written in Java—should be able to run anywhere, regardless of the platform. Microsoft countered that the better paradigm is Interoperability: software can be written for any platform, as long as it can make remote calls to programs that are resident on other platforms. To demonstrate its position, Microsoft introduced the Simple Object Application Protocol (SOAP), an open exchange pattern which can be used to send corresponding messages between applications regardless of the platform or environment in which they reside.14 Both stands have been cooperating winners. While Java has become the leading programming language and run-time environment in the new millennium, SOAP (which now only stands for SOAP) has become the quintessential messaging service for Web Services that reside within a Service Oriented Architecture (SOA).

Interoperability is about the federation of resources. Portability is about the consolidation of resources. While consolidation—the reduction of products, services, and methods to a limited number of brands, models, and instances—is sometimes an effective way to achieve cost-savings, it can also limit innovation and increase risk through the cost of consolidation itself. (Re-training a workforce to use only one code base or language or hardware platform, for example, tosses away one’s knowledge investment in the hopes of distant efficiency brought about through homogenization.) Federation, on the other hand, acknowledges that one size cannot fit all, but seeks to strike a balance between central standards and unbridled choice. Interoperability permits that balance to take place.

Collaborative Application Development Among New York State Agencies

12 / 70

13 Ambler, Scott W. 2002 Agile modeling; Effective Practices for eXtreme Programming and the Unified Process. John Wiley and sons, New York, p. 126 ff.

14 Seely, Scott and Kent Sharkey 2001 SOAP: Cross Platform Web Services Development Using XML. Pearson Education, New York.

Page 13: InteragencyCollaboration 1-27FINAL

Portability and the Benefits of Open Development

There are instances where the same code fragment, compiled program, or program suite (application) can be profitably used on separate platforms. For example, to achieve a certain level of performance on a distributed network, it is sometimes useful to run an instance of the same program at two locations. In other cases, it might be useful to share a consolidated code base with other organizations simply as a cost-saving measure. Why write a program twice when you can re-use what already exists?

Despite this fair weather principle, it is often difficult to develop code that exactly meets the need of two organizations. A municipality using a common code base might need to extend that code to meet local requirements, while still wanting to benefit from continuing enhancements made to the core. Organizations working closely together on common code might have to include far more functionality in the shared code to address the separate needs of both organizations.

Use of shared code must be elective, because only individual organizations can judge whether code it might share fits its local business needs. However, collaborative methodology exist which provide choice while still providing a high degree of sharing and collaboration. The Open Source development method provides such an approach.

Interoperability and the Benefits of SOA

In his celebrated book on Service Oriented design, Thomas Erl asserts that it is important to understand why both vendors and users within the IT industry are taking pains to adopt Service Oriented Architecture. He lists the following strategic goals and benefits that result from Service Oriented Architecture:15

• Increased intrinsic interoperability - Efficiencies are gained today through the sharing of data by organizational units and the applications which support them. Applications which are not interoperable must be integrated at significant cost. Service Orientation makes interoperability a native (intrinsic) part of applications so that consideration of integration disappears as a special requirement.

Collaborative Application Development Among New York State Agencies

13 / 70

15 Erl, Thomas 2007 SOA; Principles of Service Design. Prentice-Hall, New York, p. 55 ff.

Page 14: InteragencyCollaboration 1-27FINAL

• Increased federation - Federation unified information resources while allowing their individual autonomy and self-governance. SOA fosters creation of standardized and externally orchestratable (composable) services each of which encapsulates a piece of the enterprise.16

• Increased vendor diversification options - Diversification allows selection of best-of-breed products and technology innovations. SOA allows a diverse portfolio to be used together. This can be done cost-effectively if they are interoperable, thus avoiding integration costs (if integration is at all possible).

• Increased business and technology domain alignment - Interoperability allows for re-use of services in different orchestrations (applications). This also allows for rapid response to changes in business rules and regulations because by changing a single service, all those which depend upon it automatically benefit from that change. As a result, applications maintain their alignment in a more responsive fashion.

• Increased Return On Investment (ROI) - A key metric in assessing how cost effective a software development undertaking might be is ROI.

• Reduced IT burden

• Increased organizational agility

The diagram on the following page shows how Labor interoperates with other governmental and private agencies. We show that while Labor might develop Services for internal consumption (i.e. for assembly into composite applications that it uses within its own agency), these same services are available to other agencies for re-use in the composite applications serving their business ends. A banking service developed for the Trade Act Assistance program in Labor can be consumed by other agencies which have programs that need banking services, without the need for each to write their own private interface to banking institutions.

Collaborative Application Development Among New York State Agencies

14 / 70

16 Self-governance means that individual development communities (e.g. IT units in various state agencies) may work in a quasi-independent fashion, but within the overall governance framework of the enterprise. During early discussions of the OASIS standards committee on the SOA Reference Model, a question was raised about whether it was the responsibility of the orchestrator to have enough reach to compose services, or whether services must be written so that they are composable. [See: [email protected], Wed 25 May 2005 20:29:20 -0400.] The conclusion reached was that services must be written (according to a standard) to be composable. That assures, for example, that one agency can build an application using services developed in another agency. In effect, services must conform to an overall community standard to be composable.

Page 15: InteragencyCollaboration 1-27FINAL

Figure 1: Labor-centric view of how the Services and Portals it contributes are used to achieve interoperability.

Collaborative Application Development Among New York State Agencies

15 / 70

Page 16: InteragencyCollaboration 1-27FINAL

Waterfall Development Style versus an Agile, Evolutionary Style

Old views of software engineering persist. Here’s what we thought. Software is architected in pretty much the same way houses are architected. First you go out and collect requirements, finding out what the customer wants to have built—maybe a compact house for heat efficiency, but with a large, comfortable master bedroom and two smaller bedrooms for the kids, with a second bathroom shared by the two. A living room with a view would be nice, and a “rec room” that the family can tear apart without worrying about visitors. With the requirements in hand, the architect (and his engineers) goes about creating a design.

In practice, the functional choices that the buyer has to make are quite restricted. There are only a few kinds of rooms that can be built. And rooms are such that a bedroom can generally be turned into a study or solarium without much ado once the kids leave. What’s left after those few choices are aesthetic issues, what software developers see as the look-and-feel of an application.

Business processes are not like residential dwellings. Not only are processes complex, with many unique elements, they are a moving target. Homes, too, sometime fail to address the need of growing families. Homes that no longer fit a family’s needs can be enlarged or remodeled to accommodate changing requirements. But there comes a point where it is better to trade up (sell the old house for a new one). In software development terms, trading up is re-writing an application, throwing out the old and replacing it with a new, improved version. That is a costly approach for an IT project because business requirements change vigorously, and the length of time to design a new solution is quickly outpaced by shifting requirements. By the time the new application is built, it needs to be reconstructed.

The sequential approach to software development—where analysis must be complete before design begins, and design must be complete before implementation can start—is called the Waterfall Method. The Waterfall Method is an engineering approach, akin to what an architect does is drawing plans for a house, to be handed off later to a construction company. Three generations of application developers have learned this method as the canonical approach to programming. Waterfall does, in fact, work quite well where static products—like homes—are being engineered. Development of the code associated with a space shuttle, or an office automation product like Microsoft Word™, yield well to Waterfall. But for most business processes, agile and evolutionary development is a better path.

Agile software development evolved in the mid 1990s as part of a reaction against “heavyweight” methods. Agile methods are a family of development processes, not a single approach to software development. In 2001, 17 prominent figures in the field of agile development (then called “light-weight methodologies”) met in Utah to discuss ways of creating software in a lighter, faster, more people-centric

Collaborative Application Development Among New York State Agencies

16 / 70

Page 17: InteragencyCollaboration 1-27FINAL

way. They created the Agile Manifesto, widely regarded as the canonical definition of agile development, and accompanying agile principles.

Some of the principles behind the Agile Manifesto are:

• Customer satisfaction by rapid, continuous delivery of useful software

• Working software is delivered frequently (weeks rather than months)

• Working software is the principal measure of progress

• Even late changes in requirements are welcomed

• Close, daily cooperation between business people and developers

• Face-to-face conversation is the best form of communication

• Projects are built around motivated individuals, who should be trusted

• Continuous attention to technical excellence and good design

• Simplicity

• Self-organizing teams

• Regular adaptation to changing circumstances

The publishing of the manifesto spawned a movement in the software industry known as agile software development.17

Portability and Collaboration

Although Sun Microsystems’ Portability stand has been eclipsed in recent times by SOA and Interoperability, Portability remains an important pattern for software development. Looking at the Labor Department’s partners in other state governments, it is unlikely that a service operated by the State of New York would be consumed by an Unemployment Insurance Taxation application in the State of California. Bandwidth issues, at least at this point in time, would easily confound that effort. But

Collaborative Application Development Among New York State Agencies

17 / 70

17 Appendix C contains the full Agile Manifesto.

Page 18: InteragencyCollaboration 1-27FINAL

California might well want to replicate software developed by New York, and execute on its own enterprise platform.

When software exchange is made within a collaborative environment, there is an added benefit—noted by Sun—of mutual development. The Open Source movement is best known environment for such exchanges, though there is no requirement that such exchanges only occur within a restricted programming environment such as J2EE. On the contrary, as long as the base requirements for shared software are defined, there are no restrictions on collaboration. This is handy because it means that source code is available to anyone for possible rework for another platform. Hence, many Open Source projects have multi-platform instances of its products.

To facilitate both the exchange of software and collaborative development, Labor has established the LaborForge. LaborForge is an Open Source repository for the exchange of code and discussion with other state workforce agencies. Located at http://laborforge.org, the repository is open to other states, other New York State agencies, and the global development community at large.

Collaborative Application Development Among New York State Agencies

18 / 70

Page 19: InteragencyCollaboration 1-27FINAL

E. Open New York (OpenNY)

Collaboration comes in many forms. One that has untold success is the collaborative methodology associated with the Open Source software movement. Many discussions of Open Source focus on use of products that result from Open Source collaboration, especially the Linux operating system and the Apache web server. But Open Source is a larger concept. Open Source development is a method of free and open work on projects that groups and individuals find of interest. Open New York gives a face to New York State government’s emerging efforts to share in the development of software components. Components can be as small as code fragments (such as an implementation of an algorithm to calculate least squares that might be reused in any number of statistical programs), or as large as whole applications (like a statewide budgeting system).

Open Source development style is largely attributed to Linus Torvald, the individual behind the free Linux operating system effort. Torvald’s method calls for early and frequent release of deliverables, delegation of as much as possible to other partners, and “openness to the point of promiscuity.”18

Unilateral Open Source Arrangements

The Office for Disability and Temporary Assistance has begun to share the code base for a Food Stamp application with the State of Wisconsin. This is an example of a unilateral, Open Source arrangement. Wisconsin, the primary developer of the application, has thrown access to the source code open to the State of New York for use as New York sees fit. However, there is a condition. New York and Wisconsin agree to collaborate on future development of the application on an as-needed basis. This means that while the states are not bound to a joint schedule, and while there is no dependency of work done by one state on the other, each agrees to make any future development available to the other party.

Government Open Source Repositories

Open Source arrangements can be wider than one between two parties. Indeed, thousands of collaborative repositories exist to bring developers together to build joint solutions to common business problems. The well-known SourceForge repository (at http://sourceforge.net) boasts over 165,000 projects and over 1 ¾ million participants.

Although such open repositories have had wide success, there has been only small interest in shared development of applications for government.

Collaborative Application Development Among New York State Agencies

19 / 70

18 Raymond, Eric S. 2001 The Cathedral & the Bazaar. O’Reilly and Associates, Inc., Sebastopol, California. p. 21.

Page 20: InteragencyCollaboration 1-27FINAL

The New York State Labor Department has established one such repository at http://LaborForge.org. The LaborForge site is the center of a cooperative agreement with workforce agencies in other states. The repository is promoted by the U.S. Labor Department’s Interstate Development Group at the University of Maryland and supported by federal grants.

Open Source License Agreements

Open Source collaboration methodology is a natural approach for within State government. But wider audiences of interest—in other governments as well as among private interests—have potential for yielding better products at a faster pace. Hence, methods are needed for encouraging wide participation beyond New York State agencies.

One barrier to open source collaboration which government needs to address is redistribution authorization or, simply, the open source license. Products and materials acquired with public funds do not enter the public domain. Government retains ownership of the snow plows it purchases for the general good; citizens have no public domain right to borrow a plow for their private driveways. The same is true of the products which government develops. Generally, government produces documents and provides services. The availability of government services are controlled by rules and regulations, while the availability of documents fall under freedom of information and records management laws. But government also manufactures property with marketable value, and computer software is arguably the largest portable asset it creates. The federal government promotes free and open distribution of such products through regulations which seek to stimulate economic development for American business. In contrast, state and local governments are largely silent on what distributions might be available. At best, all property is taken together as assets which are to be protected under public finance and public officers laws.

The State’s copyright interest in what it produces is clear. But to effectively participate in an Open Source community beyond the State’s own walls, New York must establish so-called copy-left provisions that grants to any collaborator a license to re-use, modify, and reproduce any government work or its derivatives. Copy-left license provisions provide these usage rights to users in exchange for a requirement that any derivative work which that user might make will also be distributed under similar terms.19 Such licenses are called “viral” in that acceptance of (contact with) the initial license causes the propagation of the same terms in subsequent licenses.

Collaborative Application Development Among New York State Agencies

20 / 70

19 Dixon, Rod 2004. Open Source Software Law. Artech House Publishers, Norwood, Massachusetts. p. 22 ff.

Page 21: InteragencyCollaboration 1-27FINAL

F. OpenNY Governance and Quality Control

Open source software allows users access to a body of source codes which they can technically modify to fix defects or to add new functionalities to meet business needs. In an effort to gain broad acceptance and maintain long-term sustainability, we must coordinate these changes through a formal governing body which should at least provide

• Open access to everybody with the right skills to contribute

• Meritocracy

• Standards

• Transparent review and approval processes

• Quality control

These are critical success factors for an open source community that leverages the power of collective intelligence of its members and provides a high-level of confidence for the use of its software.

For instance, Apache HTTP server is the most commonly used web server in the world, as of December 2007, it had about 50% of the market share in a population of about 155 millions sites, according to the statistics collected by Netcraft. Since its inception in 1996, Apache has been developed and maintained by an open community of developers under the auspices of the Apache Software Foundation which consists of a formal governance structure with well-defined roles and responsibilities.

For OpenNY, we recommend a very similar, but simplified, approach adopted from OpenSolaris to establish the governance body. In what follows, we describe its structure, roles and responsibilities.

OpenNY Governance Structure

As a function of the enterprise, the OpenNY Community is structured as an organization of agency participants in which Members are given the right to vote on Community-wide decisions, the most significant of which is to elect an OpenNY Governing Board (OGB) to be responsible for overall day-to-day operations and representation of the organization to third parties. The OGB, in turn, delegates the organization and decision-making for specific OpenNY activities, such as new features development and “bug” fixes, through the creation of Community Groups. Each Community Group consists of participants and contributors, a subset of whom become long-term Core Contributors and are given the responsibility

Collaborative Application Development Among New York State Agencies

21 / 70

Page 22: InteragencyCollaboration 1-27FINAL

for governance within the Community Group. Finally, the set of all individuals that have been named by one or more Community Groups as Core Contributors are the Members who are given the right to vote on Community-wide decisions. A schematic of the structure is depicted below.

OpenNY Governance Structure

OpenNY is a free and openly available resource, yet imposes controls aimed at assuring seriousness of purpose of those who join the community ranks. Although anyone who has access to NYeNET may view the development discussions of the OpenNY Community and use the products of the OpenNY community in accordance with each product's associated licenses, participation in development or decision-making within the OpenNY Community is restricted to persons registered with the OpenNY Community. Registration shall be available to any person who has not been removed by previous action of the Community or forbidden by applicable law or regulation.

Roles

Various terms are used to describe the people who are involved in the OpenNY Community efforts, based on their recognized contributions, length of commitment, and current activity. The OpenNY Community recognizes three levels of involvement by registered persons:

1. Participant. Any registered person who participates in the OpenNY Community, either through general discussion areas or within one or more Community Group efforts, shall be termed an OpenNY Participant.

2. Contributor. A participant who has been acknowledged by one or more Community Groups as having substantively contributed toward accomplishing the tasks of that Community Group, or

Figure 2: Participating groups in an Open Source governance structure.

Collaborative Application Development Among New York State Agencies

22 / 70

Page 23: InteragencyCollaboration 1-27FINAL

by the OGB for at-large contributions, shall be termed an OpenNY Contributor. Such designation is permanent and persists regardless of the person's current level of activity or status within the Community. A Contributor may request that their status not be published or published only in the form of a pseudonym that is unique within the Community.

3. Core Contributor. A Contributor who is an active and sustained contributor to any Community Group and accepts designation as such by said Group shall be termed a Core Contributor for said Group and granted the status of Member for the OpenNY Community as a whole.

Disputes

Disputes among participants arising from their participation in any Community Group effort shall be resolved by said Group according to its normal decision-making procedures. If such a dispute falls outside the scope of any existing Community Group, or extends to the OpenNY Community at-large, then any of the participants may appeal to the OGB, or to one or more committees designated by the OGB, to serve as an arbitrator in order to resolve the dispute.

Suspension and Expulsion of Participants

The OGB is responsible for ensuring that participation in the OpenNY Community is constructive in nature and not harmful to other participants. When abuse of the Community is reported to the OGB, the OGB shall review a participant's actions according to the following procedure:

1. The OGB shall notify the participant via electronic mail that his or her participation in the OpenNY Community is under review and that the review will be completed within thirty (30) days. The review shall occur in a closed session and shall not be made public unless an action is made to suspend the participant.

2. The OGB may order a temporary removal of the participant's write access to the OpenNY Community infrastructure until the review is complete.

3. Upon conclusion of the review, the OGB may, upon affirmative vote by two-thirds (2/3) of the OGB in office, suspend the participant's write access to the OpenNY Community infrastructure for a period of up to six (6) months and, if said participant has previously been subjected to such a suspension, recommend expulsion of the participant from the OpenNY Community .

4. A suspended Member may attend and vote at a meeting of the Members, in person or by proxy, subject to the normal rules of conduct for that meeting.

Collaborative Application Development Among New York State Agencies

23 / 70

Page 24: InteragencyCollaboration 1-27FINAL

A Participant under suspension may be expelled from the OpenNY Community by an affirmative vote of a two-thirds majority of the Members of record. The effect of such a removal shall be the immediate expiration of any grants of Core Contributor status to that Participant and removal of the Participant's write access to the OpenNY Community infrastructure until such time as a subsequent act of the Members revokes the expulsion.

Membership

The OpenNY Community is a membership-based organization in which active long-term contributors are given the authority to participate in decisions through voting and consensus. These Members are responsible for ratifying Constitution, electing an OGB to oversee the day-to-day operations of the OpenNY Community, and providing guidance to the OGB.

The initial set of Members for the OpenNY Community shall be those natural persons designated by the existing OpenNY projects as Core Contributors to their projects. The initial set is intended to be a representative sample of the Community's existing core contributors and need not include every person that might fit that level of involvement. The initial set shall be admitted upon the affirmative vote of the OGB at the initial meeting of the OGB. Thereafter, persons shall be admitted as Members of the OpenNY Community upon accepting the designation of Core Contributor for one or more OpenNY Groups and receipt by the Secretary of their willingness to accept the status of Member.

Each grant of Core Contributor status, including the grants that form the initial set of Members, shall have a duration of two (2) years and is renewable by the granting Community Group. A Member may be recognized as a Core Contributor by multiple Community Groups, thereby qualifying for Member status multiple times, and each grant is independent from any others. Each such person counts as only one (1) Member regardless of the number of grants upon which their Member status is based. A Member's membership shall be terminated upon the expiration of all prior grants of Core Contributor status or upon his or her earlier resignation, removal, or death.

A Member who is expelled from the OpenNY Community shall also be removed from the Membership and shall not be eligible for subsequent grants of Member status until such time as a subsequent act of the Members revokes the expulsion.

Community Group Voting Procedures

Purpose. Most of a Community Group's work is done by individual Contributors within a climate of ongoing discussion, informal group consensus, and a general desire to meet the community's shared

Collaborative Application Development Among New York State Agencies

24 / 70

Page 25: InteragencyCollaboration 1-27FINAL

objectives. However, occasional conflict is an inevitable part of working together to solve complex problems. The Community Group voting procedures are designed to support both friendly collaboration and individual innovation, while at the same time providing adequate oversight for the OpenNY Community as a whole by enabling equal participation in formal decisions regardless of a person's affiliation or location. The Community Group voting procedures determine how future conflicts will be resolved so that volunteers on a project need not fear differences of opinion: contrary ideas can be voiced without derailing progress. The objective is to avoid unnecessary conflict over changes, while at the same time encouraging opinions that work towards producing quality products in a timely manner.

Location. Decisions by a Community Group are made by vote or consensus on a public meeting place of the Community Group that is most applicable to the action being discussed. For example, Community Group decisions that are specific to a single project are made through discussion and voting/consensus on the asynchronous meeting place (e.g., mailing list) for that project, whereas general Community Group decisions are made on the Community Group's general discussion place. Some discussions, such as investigation of security issues prior to their publication and nominations for Core Contributor status, are expected to take place in private prior to any public vote by the Community Group.

Votes. Each Core Contributor of a Community Group has the right to one binding vote on each proposed action of the Community Group. Other Participants may express their opinions through non-binding votes. A Core Contributor may change their own vote on a given action until such time as the action is no longer applicable (e.g., decisions regarding a given product release are final once the release is made public). Votes are expressed in the range [-1,1] with the following meanings:

+1 Yes, "Agree," or "the action should be performed." A +1 vote indicates that the voter is in favor of implementation of the proposed action.

±0

Abstain, "no opinion," or "I am happy to let the other group members decide this action." A vote in the interval between 0 and 1 (such as +0.5) can describe varying degrees of support for the proposal but still counts as an abstention. A vote in the interval between 0 and -1 (such as -0.9) can describe varying degrees of disagreement with the proposal but still counts as an abstention.

-1No, "Disagree," or "the action should not be performed." On actions where consensus is required, this vote counts as a veto if it includes an explanation of why the veto is appropriate and at least one other Core Contributor agrees that the explanation is valid ( regardless of their own vote).

Collaborative Application Development Among New York State Agencies

25 / 70

Page 26: InteragencyCollaboration 1-27FINAL

Voting Systems. Decisions of a Community Group are determined by several related voting systems, chosen per action type based on the degree of consensus or quorum needed to make the decision:

Voting System Approved by Action Types

Consensus At least three (3) binding +1 votes and no -1 votes (i.e., unanimous with a minimal quorum of three votes).

Putbacks (when subjected to vote). Initiate new project. Terminate project. Addition of Core Contributor. Removal of Core Contributor (subject must abstain).

Majority

At least three (3) binding +1 votes and more +1 votes than -1 votes (i.e., majority vote with a minimal quorum of three +1 votes).

Design choice. Release plan. Product release. Facilitator nomination. Documentation (when subjected to vote).

Assumed

Approval is assumed unless a -1 vote is received, after which the action is reverted or the issue is decided by Majority or Consensus vote, depending on the type of action.

Putbacks. Documentation.

Voting Period. Actions requiring a vote shall have a voting period of no less than seventy-two (72) hours. Actions are actionable before the voting period expires if all Core Contributors have voted and the issue would be approved by those votes.

Appeal. Once a decision is made, an action can only be revisited (brought to another vote) if at least one voter on the prevailing side of that decision declares a desire to change their vote and proposes a new vote on the action. There is no mechanism for escalation of decisions legitimately made according to these Community Group voting procedures. If it is believed that these procedures were not followed and, after being informed, the Community Group refuses to revisit the decision, then a review of the Community Group's procedures may be requested of the OGB. Upon review, the OGB may act to terminate the Community Group, partition the Community Group into multiple Community Groups, or provide advice to the Community Group; the OGB shall not make decisions for the Community Group.

OpenNY Quality Control

Reliability and auditability are some of the benefits for open source. As the source code is openly published and widely distributed, defects can be detected and fixed more quickly than a commercial product can. The National Alliance for Medical Image Computing sponsored by National Institute of Health is an excellent example of how an open source community can develop mission critical tools for common good. It facilitates parallel development and testing of software projects in multiple sites in

Collaborative Application Development Among New York State Agencies

26 / 70

Page 27: InteragencyCollaboration 1-27FINAL

multiple configurations (hardware, operating systems, compilers, etc.). Results from a build/test sequence are transmitted to a central server using standard internet protocols. The server produces concise dashboards, summarizing the current state of a software system. The dashboards link to detailed reports on inter- and intra-configuration results. Testing results are tracked over time, allowing developers to trace the history of development. The Open Source Visual Toolkit (VTK) quality dashboard is a good example of such a quality dashboard and a snap shot of it is shown in the figure below:20

A new build is conducted nightly and tested against a set of benchmark cases. Defects are detected immediately and posted with sufficient information so that community members can take the appropriate action in a timely fashion. It has been reported some defects were fixed within several hours. We recommend OpenNY to follow a similar approach.

Figure 3: Visual Toolkit quality dashboard

Collaborative Application Development Among New York State Agencies

27 / 70

20 On on-line example of the VTK can be found at http://public.kitware.com/dashboard.php?name=vtk

Page 28: InteragencyCollaboration 1-27FINAL

G. Service Oriented Architecture

A second, modern approach to collaboration arises from emerging technologies that permit composite applications to be built from components developed and operated by different organizations. Service Oriented Architecture is the technology that does this and Services are the “bite sized” pieces of software that can be mixed and matched to build new and innovative works at reduced cost.

A Service is a small program that receives requests from other programs, and performs some desirable task on their behalf. Consider, for example, a Direct Deposit Banking Service. Such a Service might receive requests from an Unemployment Insurance Payment program, a Medicaid Payment program, a STAR Tax Rebate Payment program, or a Welfare Payment program. Each of these programsServices themselvesmight send a

message to the Banking Service containing the name of the recipient, the recipient’s bank and account information, a payment amount, the New York State fund from which to withdraw monies for payment, and perhaps other data. The Banking Service arranges to make the payment to the payee through direct deposit, and returns a success (or failure) message to the Service that invoked it. If this operation were done in a non-Service Oriented way, each of the programs (Unemployment Insurance, Medicaid, etc.) would have to write its own unique extension (in COBOL or Java, or C, or whatever programming language it might use). But in a SOA world, each of the programs can merely consume the one Banking Service and it doesn’t matter to anyone what programming language might

have been used to implement that Service.

Services make requests and receive responses from other Services by exchanging messages. A message is a standards-defined data package that travels over a network (an internal network, private extranets, or even the Internet), very much like electronic mail messages are distributed. Unlike email, however,

Collaborative Application Development Among New York State Agencies

28 / 70

Figure 4: Applications maintained by two agencies share a common Service.

Page 29: InteragencyCollaboration 1-27FINAL

messages between Services are formed using a protocol named SOAP. All SOAP messages must be written in the XML mark-up language.21

Although messages are carried over the Internet, they are routed and processed within an organization’s own internal network by one or more computers which comprise an Enterprise Service Bus. The diagram below depicts an Enterprise Service Bus and the broad categories of Services whose messages pass through it.

Figure 5: An Enterprise Service Bus routes messages among true Services as well as other, non-Service Oriented components.

It might seem curious that we would need an Enterprise Service Bus just to send messages between applications. You would not in the scenario depicted earlier, where a large monolithic program is consuming an outside Direct Deposit Service. But that example is really showing a legacy application or a COTS application (which are not Service Oriented) making a request into a Service Oriented environment. Real Service Oriented applications are composites, entirely make up of Services. The structure of such composite applications looks something like that shown in the following diagram.

Collaborative Application Development Among New York State Agencies

29 / 70

21 In reality, XML is not much of a programming language at all. XML is more of a way to structure data so that any program can read it.

Page 30: InteragencyCollaboration 1-27FINAL

Whole applications can be created simply by having one Service call another service that calls another Service. Some Services interact with databases. Some do calculations. Some process rules. And along the way, Services make contact with real live humans through Graphical User Interfaces (portals).

It can get quite difficult to figure out how an application works in a simple Service Oriented Architecture. But sophisticated SOA applications do not rely on individual Services themselves to know how they should behave in a greater application. Instead, applications can be created by orchestrating the sequence of Services through use of a computer called a Business Process Server that lives on the Enterprise Service Bus. That mouthful of technologies becomes clearer after examining the following diagram.

Figure 6: Two, true Service Oriented applications have both program-specific Services as well as shared Services.

Collaborative Application Development Among New York State Agencies

30 / 70

Page 31: InteragencyCollaboration 1-27FINAL

Here we show how an application is a composite of Services. Some Services are specific to the Unemployment Insurance program and would not be used anywhere else. Others are specific to the Food Stamps Program and have no value outside of that domain. Many services, however, are program agnostic. They can be used by any composite application that needs the service they provide.

Services can be written so that each Service knows what other Service it should contact, what information it should send, and what data it should receive back, if any. But there is also another way to build applications from Services, and that is by independently orchestrating them. A Business Process Server, mentioned above, is a computer that executes another kind of XML script which tells the process server which Services to invoke and in what sequence. The diagram above would look the same even with use of a process server. A process server, however, provides an additional layer of abstraction so that agency workflow processes can be composed and re-composed without resorting to change to code in even the small, Services.

Legacy applications (whether they are newer, object-oriented works or older, mainframe works) can participate in a Service Oriented Architecture, even though they are not themselves Service Oriented. This is also true of commercial enterprise (COTS) applications. Consider, for example, the OCIO’s Annual Technology Plan (ATP) application and various commercial project management applications. The legacy ATP application is a well-engineered, web-based, client-server application. It is not Service Oriented, but there are object libraries in this java application that could be leveraged to allow the exchange of data in a Service Oriented fashion with other applications.22 The OCIO uses the Mercury project management system. Other agencies use other mainstream products, such as Primavera and Microsoft Project. None of these commercial project management systems are Service Oriented. They can be made to join a Service Oriented Architecture, however, by using SOA wrappers that map native I/O streams to a messaging interface through an Enterprise Service Bus. Linking the COTS applications to the ATP application in this manner would allow agencies to update their projects once, in their local systems, and have them automatically provide the data needed by the ATP. In this way, the general statewide ATP and the more detailed agency project systems can maintain perfect synchronization, while eliminating double work.

Note also that SOA development can be started even before every other partner is ready. Under the New York State Elderly Pharmaceutical Insurance Coverage (EPIC) program, the Department of Health must

Collaborative Application Development Among New York State Agencies

31 / 70

22 The OCIO has announced that it would like integration between the ATP application and the separate Intent to Purchase (ITP) database application. If both were Service enabled, they could exchange messages without rewriting the applications into a consolidated, monolithic application. If both were truly Service Oriented, linking the two applications would be only a few weeks work.

Page 32: InteragencyCollaboration 1-27FINAL

verify the income of participants. Tax and Finance provides the Department of Heath through an income look up service. Health cannot directly consume Tax’s service to get this information because it does not have the infrastructure. So presently, Health sends Tax a file of records to look up (5 million). Tax’s internal code reads the EPIC file and then calls its service that does the look-up and adds the data to the file. In the future Health can call the service directly either in batch at night or in-line during the filling out of the Epic application. On Tax’s side of the conversation, it will only require enabling the Service for external invocation by Health.

Similarly, Labor has built a Service for capturing debarment data from the Workers’ Compensation Board (WCB). Because the Board has not yet established a Service Oriented framework, the connection between Labor and the Board is a traditional, point-to-point integration strategy. Internally within Labor, however, an application uses this Service to query a joint Labor-WCB database (maintained at Labor) in response to public inquiries on Labor’s public web site. The Service is used again internally at Labor to gather data on debarments made by Labor’s Worker protection programs. In time, the Service will be directly consumed by WCB as part of it’s own debarment application. Moreover, the Service is also ready for consumption by the Office of the Attorney General as part of its Project Sunlight initiative, and by the Comptroller as part of its Vendor Responsibility initiative. (Both applications involve collecting information for public exposure in different formats and for varying purposes.)

Such data exchanges can be accomplished in a one-by-one effort to create common XML documents (i.e. data sets expressed in the XML markup language) between individual applications. A more efficient way to do this, however, is to use generally agreed upon data standards. Many such standards already exist. The OASIS standards organization, in particular, has been instrumental in developing XML document standards in a wide number of subject areas. Where there are no such standards available outside New York State government—and this will be the larger situation, agencies will want to establish a central data standards board. Such a board would logically come under the direction of the OCIO.23

Note, however, that data standards in an open world provide flexibility that past efforts lacked. We need to be wary of rigid data definitions, specifically in terms of standard sizes for data fields across agencies. Past attempts to reconcile one standard across every agency, and their external partners, stalled development and ultimately failed. A more practical alternative is to use the Service Oriented publish and subscribe paradigm where agencies can publish their data exchanges or Services and participating agencies may subscribe to them. This means the subscriber knows what they are getting (e.g., individual name as first, middle initial, last or as a single text string) and the subscriber can respond

Collaborative Application Development Among New York State Agencies

32 / 70

23 See the OCIO Enterprise Architecture principles, especially item 3.2.3, in Appendix B for a parallel view.

Page 33: InteragencyCollaboration 1-27FINAL

accordingly. Agencies don’t have to use the service, but if they want to then they are free to consume the Service under the published terms of use.

Another strong feature of SOA is that it can be more secure, especially from a disclosure perspective. Today, agencies send files around or they give partner agencies access to only parts of their applications, like Tax and Finance’s provision to Labor. This allows for general searches by name, for example, but there is really no way today to verify whether any particular name search is warranted. Future implementation as a Service allows access to the application based on explicit parameters. Moreover, the Tax application which Labor uses now is a poor fit for users at Labor. But by consuming Tax’s Service, instead of Tax’s application, Labor would put Tax’s data on a screen designed by Labor to meet its business requirements and its users’ needs. What this means is the Labor application (not the user) can determine if there is a business need, thus eliminating random walks through the Tax data by Labor employees.

Service Oriented Architecture also provides opportunity to further the effort to apply performance measurement to State programs. With communications between Services traveling across an Enterprise Service Bus, opportunities become available for developing metrics which measure work flow. Because business services are aligned perfectly with business processes, measurement of Service work flow provides business metrics that can be used for making quality improvements to governmental programs.

Collaborative Application Development Among New York State Agencies

33 / 70

Page 34: InteragencyCollaboration 1-27FINAL

H. Service Oriented Architecture – the Business Case

Business cases are intended to present the benefits, either financial or services provided, derived from an initiative. Both types of benefits are difficult to articulate in a public service based organization, rather than for a profit-driven organization whose benefits are measured by the impact on the bottom line. Nonetheless, the following section provides a number of areas for evaluation to develop sources of Business benefits that can be attributed to accepting an SOA approach.

We know that the private sector’s Return on Investment experience with SOA is stunning. Consider that Helvetia Patria, a leading European insurance company based in Switzerland, saw increased profitability and business agility according to an independent study by Thoughtware Worldwide, LLC. The company also reduced e-business IT operational costs by 59 percent while it experienced an internal rate of return of 26 percent.24 25

In a study conducted by IBM Corporation, all 30 companies investigated that adopted SOA showed major returns. A cellular telecommunications company that was studied created an entirely new service for locating cell phones out of its legacy systems, re-orchestrated through SOA. This capability is estimated to open up a billion dollar market by 2009 for this company.26 In another instance, a large agricultural machinery manufacturer needed to boost its ability to finance sales in its showroom. It tapped SOA not only to improve and expedite current lending practices, but to provide a new lending product to keep pace with competitive alternatives. It was able to double loan application volumes and increase the loan decision rate from 15 percent to 55 percent.27

Ultimately SOA’s success is driven by the realization of benefits delivered to the business in the form of cost and/or service-based improvements. A Service Oriented Architecture provides a number of tools to stimulate these improvements. Although for the purposes of representing a Business Case it is simpler to consider SOA as an enabler for the business improvement opportunities.

Collaborative Application Development Among New York State Agencies

34 / 70

24 Thoughtware Worldwide customer case study on Helvetia Patria, “Quantifying the Value of the eBusiness Center: HP Cost Savings Solutions for the Insurance Industry,” July 8, 2005.

25 Thoughtware Worldwide customer case study Helvetia Patria, “Helvetia Patria turns to HP to create industry leading e-Insurance Solutions – realizes 201% ROI,” January 2006.

26 Driscoll, Clement. “U.S.Mobile Resource Management Systems Market Shows Strong Growth in Subscribers and Revenues.” Location Intelligence. January 3, 2006.

27 IBM Global Business Services, 2006. “Service Oriented Architecture; a practical guide to measuring return on investment.” Somers, New York.

Page 35: InteragencyCollaboration 1-27FINAL

Business improvements will be recognized in two areas: IT resources and Program resources. Every department will have a different degree of improvement depending on the current state of their IT and Program processes and staff. Accepting that each department may realize differing levels of benefit, all agencies should consider the following sources of improvement to characterize the possible benefits that could be realized.

The IT resource improvements will ultimately be realized through two sources: 1) the reuse of services reducing the amount of software that needs to built and maintained; and 2) the improved agility of the IT staff to provide new composite applications or modify existing services. Shorter durations required to create or modify software translates to fewer resources and cost.

The program resource improvements will be realized through two sources as well: 1) improved resource efficiencies; and 2) improved resource effectiveness. Both of these are availed through the improved recognition of Business Process enhancements through Business Process Management practices.

Ultimately the business will decide whether efficiencies will be translated as a cost reduction or a repurposing of staff to improve the service delivered by an agency. Both of which are not easily translated to a dollar figure across an enterprise of business services provided.

Standards Based Integration

To summarize points made in Section D, SOA dictates the use of standards-based interfaces between services. Considering that the services will be either providing access to information or business services, the use of a standard to support the interface will ease the use of services across state agencies or across separate programs within an agency.

Standards will ease the effort to leverage other agency or program information and or business services. The use of new or improved data should enhance a business process either through sharper clarity within a business interaction or quicker delivery of services. Consider the business scenario where individual’s data in the control of the Department of Motor Vehicles could be easily accessed to determine whether the person interacting with the application is whom they report to be.

The use of a business service would either reduce internal staff requirements or simply make a customer’s interaction with the state easier via a single interface point or clearer interaction creating a simpler, shorter customer-to-government interaction. Consider the business scenario where an individual is registering with Governor’s Office of Regulatory Reform’s on-line license and registration system. By Labor (and other corresponding state agencies) providing access to their UI Employer Registration business service

Collaborative Application Development Among New York State Agencies

35 / 70

Page 36: InteragencyCollaboration 1-27FINAL

from each agency, one registration form into the Governor’s Office of Regulatory Reform system could simplify the individuals interaction with the state and standardize the core data representing the business across the various state agencies.

Business Process Management

SOA utilizing Business Process Management tools (for example, IBM’s Business Process Manager) will improve any institution’s grasp on the Business Processes that support their business and ultimately should improve the institution’s Business Intelligence. Through the management and measurement of key business metrics now available via the Business Process Management tools, it is proven that those things measured will improve over time.

As stated earlier, measured improvements in a business process are subject to both the current state of the business process and the ability to institute change in the business process. As with any improvements in the business processes as a result of standardized interfaces, a better understanding of a business process resulting in better decisions regarding the business processes are expected to provide benefit either in the efficiency of the program resources required or an improvement in the services provided to the consumers of the business service.

Improved Agility

As described in Section 2, SOA enables both the “Speed to Market” and “Agile Systems Development” factors. Any savings within the IT resources in delivering new or enhanced application functionality is a direct savings in the cost of enhancement. As with the program improvements, savings with the IT resources in the delivery of change will translate into further improvements in the delivery of automation support.

Collaborative Application Development Among New York State Agencies

36 / 70

Page 37: InteragencyCollaboration 1-27FINAL

I. Service Oriented Architecture Formalism

SOA is a conceptual business architecture where business functionality, or application logic, is made available to users, or “consumers”, as shared, reusable services on an IT network.28 Organizations will have two broad classifications of services: (1) business services and (2) technical services. Business services reflect business concepts and events, and thus are excellent organizing principles for an SOA. They could also be called business process services because they are associated with executions of business functions of an organization or business process.29 Technical services are those services that are horizontal in nature or are reusable by all business processes, business units, or process domains. Technical services include security services, logging services, audit services, transformation services and similar ‘IT’ services that would be leveraged by and across all lines of business. In some organizations, these technical services would be described as enterprise common services.30

Characteristics of Services

Services, in order to meet the needs of the organization, must meet certain criteria to provide the most value to the organization.

Coarse grained

Well-defined services contracts

Loosely coupled

Discoverable

Durable

Composable

Business aligned

Reusable

Collaborative Application Development Among New York State Agencies

37 / 70

28 Marks, Eric A. and Bell, Michael, Michael 2006 Service-Oriented Architecture: A Planning and Implementation

Guide for Business and Technology, Bell, Michael John Wiley & Sons, Inc., Hoboken New Jersey. p 1.

29 Ibid. Page 37.

30 Ibid. Page 38.

15 Ibid. Page 38.

Page 38: InteragencyCollaboration 1-27FINAL

Interoperable

Stateless

Dynamically Bound31

Coarse Grained Services

Services should be coarse-grained entities. Service granularity depends upon how much functionality a service encapsulates and exposes. Coarse-grained means that services should represent business functions, processes, or transactions and encapsulate other fine-grained components or services within them. A service that is too big or too coarse-grained will suffer from performance issues and will not be reusable. A service that is too fine-grained will be too narrow in scope to meet requirements of multiple business processes.32

Well-Defined Service Contracts

Services must have well-defined contracts that separate the functionality of the service from its specific technical implementation. The service contract informs consumers what the service does as well as how to consume or use the service. Services contracts present the service functionality to the outside world in a standardized, interoperable fashion while hiding the specific internal technical details of the services.33

Loosely Coupled

In the services design aspect of the term ‘loosely coupled’ means designing services that specific implementations of services can be replaced, modified, and evolved over time without disrupting the current service consumers and the overall activities of an SOA.34 “Coupling refers to the number of dependencies between modules. There are two types of coupling; loose and tight. Loosely coupled modules have a few well-known dependencies. Tightly coupled modules have many unknown dependencies. Every software architecture strives to achieve loose coupling between modules. Service

Collaborative Application Development Among New York State Agencies

38 / 70

31 Thomas Manes, Anne 2006 Service-Oriented Architecture: Developing the Enterprise Roadmap, Version 2.0. Burton Group, Midvale, Utah p 41.

32 Marks, Eric A. and Bell, Michael 2006 Service-Oriented Architecture: A Planning and Implementation Guide for Business and Technology, John Wiley & Sons, Inc., Hoboken New Jersey. p 39-40.

33 Ibid. Page 40.

34 Ibid. Page 40-41.

Page 39: InteragencyCollaboration 1-27FINAL

oriented architecture promotes loose coupling between service consumers and service providers and the idea of few well-known dependencies between consumers and providers.35

Discoverable

Services should be discoverable. This means that their contracts are published and visible to an intended audience – the consumers. Discoverable services implies that the service contracts – WSDL documents in the case of Web Services - are published to a location where they can be discovered, whether that location is a service registry, metadata repository, subdirectory, or some known location. Once the service contracts are published, they must be advertised to potential consumers.36

Stateless

One of the major characteristics of a service is that it is Stateless. It should not have to know the process state that invokes it (that for the process or composite application). If the service knows state it get vastly more complicated because it has tests to see what has to be done. This vastly reduces reusability because before you can use the service you have to figure out the state to call it in, and you may even have to create a new state that has to be coded in the service.

Dynamically Bound

A service consumer that needs a service discovers what service to use based on a set of criteria at runtime. The service consumer asks a registry for a service that fulfills its need. The Registry returns all entries that support this. The entries also contain information about the service, including transaction fees. The consumer selects the service (provider) from the list based on the lowest transaction fee. Using a pointer from the registry entry, the consumer then binds to the provider of the service. The description of the service consists of all the arguments necessary to execute the service. The consumer formats a request message with the data, based on the description provided by the pointer. The consumer then binds the message to a transport type that the service expects and sends the service the request message over the transport. The service provider executes the [service] and returns a message whose format is also specified by the service description. The only dependency between the provider and the consumer is the

Collaborative Application Development Among New York State Agencies

39 / 70

35 Thomas Manes, Anne 2006 Service-Oriented Architecture: Developing the Enterprise Roadmap, Version 2.0. Burton Group, Midvale, Utah p 49.

36 Marks, Eric A. and Bell, Michael 2006 Service-Oriented Architecture: A Planning and Implementation Guide for Business and Technology, John Wiley & Sons, Inc., Hoboken New Jersey. p 41.

Page 40: InteragencyCollaboration 1-27FINAL

contract, which the …registry provides. The dependency is a runtime dependency, and not a compile-time dependency. All the information the consumer needs about the service is obtained and used at runtime.37

Durable

Services should be durable yet elastic. Durable services are those that map to lasting business or process themes.38

Composable

Services should be Composable. Composable services are designed to be incorporated into other services as composite services as necessary. In addition, composable services can be assembled into orchestrated process flows. In this sense, composable services are stateless and anatomic in nature. They stand on their own yet rely on other services or infrastructure for state and context.39 The goal for service design is to identify the smallest unit of software that can be reused in different contexts.40 A service’s composability is related to its modular structure. Modular structure enables services to be assembled into applications the developer had no notion of when designing the services. Using preexisting, tested services greatly enhances a system’s quality and improves its return on investment because of ease of reuse.41

Business Aligned

Services should be business aligned. Service identification and analysis should begin with business imperatives and business requirements. Services, or business services, represent business concepts and match business needs as determined through business strategy and planning and the associated business discovery processes that should precede an SOA initiative.42

Collaborative Application Development Among New York State Agencies

40 / 70

37 Thomas Manes, Anne 2006 Service-Oriented Architecture: Developing the Enterprise Roadmap, Version 2.0. Burton Group, Midvale, Utah p 41.

38 Ibid. Page 42.

39 Ibid. Page 42.

40 Thomas Manes, Anne 2006 Service-Oriented Architecture: Developing the Enterprise Roadmap, Version 2.0. Burton Group, Midvale, Utah p 42.

41 Ibid. Page 59.

42 Marks, Eric A. and Bell, Michael 2006 Service-Oriented Architecture: A Planning and Implementation Guide for Business and Technology, John Wiley & Sons, Inc., Hoboken New Jersey. p 43.

Page 41: InteragencyCollaboration 1-27FINAL

Reusable

Services should be reusable. This is a function of proper services identification, analysis, and design. Implement services that have clear and defined reuse across and within business processes and that have multiple consumption patterns in your current and planned business processes.43

Interoperable

Services should be interoperable. Interoperability is the ability of systems using different platforms and languages to communicate with each other. Each Service provides an interface that can be invoked through a connector. An interoperable connector consists of a protocol and data format that each of the potential clients of the service understands. Interoperability is achieved by supporting protocol and data formats of the service’s current and potential clients.44

Location Transparency

Location Transparency is a key characteristic of service-oriented architecture. Consumers of a service do not know a services location until they locate it in the registry. The lookup and dynamic binding to a service at runtime allows the service implementation to move from location to location without the client’s knowledge. The ability to move services improves service availability and performance.45

Service Development Process

The development life-cycle of services includes the following practices:

Identification of Services

The initial step involved in the development of services is to first identify candidate business services, which can later be analyzed for their utility and feasibility, leading to selection of a set of final business services to be designed and realized for use.

Candidate Service Analysis

Analysis of potential services is a top-down process of logical operations and transformation mechanisms to study and inspect enterprise core entities, ascertain business ideas and concepts, and lead to the

Collaborative Application Development Among New York State Agencies

41 / 70

43 Ibid. Page 43.

44 Thomas Manes, Anne 2006 Service-Oriented Architecture: Developing the Enterprise Roadmap, Version 2.0. Burton Group, Midvale, Utah p 48.

45 Ibid. Page 59.

Page 42: InteragencyCollaboration 1-27FINAL

formation of final business services. Services analysis on candidate business services can address consolidation, decomposition, reuse, simplification, and refactoring of legacy assets as dictated by agency imperatives. These activities present opportunities to transition to shared reusable services in a loosely coupled architecture.

This service identification and preliminary analysis is a primary responsibility of the UISIM Development Board.

Services Design

The service design phase enables the implementation of physical services through a series of design activities. The service design process transforms final business services into physical solution services. Inputs into service design are a set of target services that meet organizational needs and have been prioritized and selected from the list of final candidate services. Service design begins with a final business services granularity map that facilitates the visualization of encapsulated business process, their desired business value, and their potential as physical solution services. The relative granularity of services helps determine their viability and appropriateness as part of the SOA strategy. The service design phase is not focused on the underlying technology or implementation details of service components. Source code is irrelevant to physical service structure design because service design activities tackle conceptual, logical, and physical compositions of services, examine their encapsulated business processes, and facilitate establishing reliable and reusable physical services.46

Realisation of Services

From service analysis and design, the next step is to generate physical solution services that correlate to the services designs. Solution services are physical implementations of service abstractions. They consist of components, which are comprised of business processes. Solution services may also contain other services. They should provide access and exposure to external consumers, yet they have a distinct internal architecture that comprises business and technological ingredients.47

Collaborative Application Development Among New York State Agencies

42 / 70

46 Marks, Eric A. and Bell, Michael 2006 Service-Oriented Architecture: A Planning and Implementation Guide for Business and Technology, John Wiley & Sons, Inc., Hoboken New Jersey. p 123-124.

47 Ibid. Page 137.

Page 43: InteragencyCollaboration 1-27FINAL

J. Service Oriented Architecture Governance and Quality Control

As a function of the enterprise, the NYSOA Community is structured as an organization of agency participants in which Members are given the right to vote on Community-wide decisions, the most significant of which is to elect an NYSOA Governing Board (SOAGB) to be responsible for overall day-to-day operations and representation of the organization to third parties. The SOAGB, in turn, negotiates a contract with a specific agency of group of agencies which agree to a specific Service Level Agreement with respect to the Services which they propose to build for public consumption.48 Each Community Group consists of participants and contributors, a subset of whom become long-term Core Contributors and are given the responsibility for governance within the Community Group. Finally, the set of all individuals that have been named by one or more Community Groups as Core Contributors are the Members who are given the right to vote on Community-wide decisions. In this way, the SOAGB is similar to the ONGB. However, the approaches then diverge.

The central notion of a Service Oriented Architecture is that a Service is always designed and refined to be reused by as many consumers as possible. Services, however, must be governed to become reusable. All foreseeable consumers must be able to express their requirements which are subsequently prioritized and phased, while a service owner is assigned and a funding model is defined. This contrasts sharply with an Open Source approach in which governance is through elective participation. Open Source products are utilized within an agency; Service Oriented products are consumed more widely.

SOA governance facilitates the creation of reusable, enterprise class services by ensuring the timely resolution of issues and conflicts due to the necessary tradeoffs that are made when shared requirements are defined. In particular, the Service Governance organization is chartered to define clear service ownership boundaries and specify a fair funding model. SOA governance monitors the deployment and reuse of public services across the enterprise. A high degree of service reuse, a steady flow of enterprise class service deployments, and the decommissioning services without depriving agencies of an essential component are the indicators of successful governance.

SOA Roles

The role of SOA governance is largely passive with Service candidates identified by specific projects that are reported by agency CIOs. Nonetheless, the SOAGB should be empowered to define and find funding

Collaborative Application Development Among New York State Agencies

43 / 70

48 Public, in this sense, means outside of the agency, either across the enterprise or on the world wide web. Of course, authority rights could shape access to the service more finely, but the point here is that if a service is to be exposed outside of an agency, a formal contract (memorandum of understanding) must be in place so that other agencies may consume the service without fear that it will go unmaintained.

Page 44: InteragencyCollaboration 1-27FINAL

methods for enterprise-level services which are independent of the budget and resource restrictions of the project that will initially consume the service candidate because reusability usually comes with a larger scope which translates into a higher price tag.

The roles that participate in SOA are shown in the following three tables. These roles change over time, as the use and management of SOA in the enterprise matures. The following roles are sufficient to ensure the effective delivery of enterprise class services.49

Role Description

Business Service Owner (Agency Program Managers)

• Direct and control the service implementation, evolution and retirement • Owns the functional scope of the service, the Service Level Agreements • Manages effectively the capabilities of the service to meet governance requests

and ensure appropriate levels of reuse • Report service activity to the governance organization

Technical Service Owner (Agency CIOs)

• Execute the service implementation, evolution and retirement • Owns the Operations Level Agreement and manages the service to meet its

objectives in terms of availability, performance, security • Monitors the service to identify potential issues with SLA and OLA • Report service activity to the business owner

SOA Platform Architect

• Advise and discuss SOA technical standards with IT and SOA governance organization

• Make sure that service implementations are compliant

Service Developer• Assist the domain architect and platform architect in their governance related

activities • Implement governance policies and recommendations

SOAGB Process

There are five types of activities performed by the SOA governance organization:

• Service Candidate Management

• Service Consumer Management

• SOA Governance Policy Changes

During the Service Candidate Management process, an agency project team (through its agency CIO) may identify a service and create a service proposal. This proposal is then approved, approved with

Collaborative Application Development Among New York State Agencies

44 / 70

49 Tolkov, Stephan 2007 “Roles in SOA Governance, ” InfoQ, July 18 at http://www.infoq.com/articles/tolkov--soa-roles.

Page 45: InteragencyCollaboration 1-27FINAL

modifications or rejected (as an enterprise service) when this service candidate is not potentially reusable by other parts of the enterprise.

Once the Service candidate is accepted, the ownership and funding model are defined and the Service Level Agreement is specified with the help of the service owner and potential consumers. Central distribution—and hence central governance—is important here because the potential consumers can be any government agency within New York State. Once the service has been developed and tested, its metadata is published to a registry and repository.

Service Consumer Management activities are mostly performed by a service librarian who helps potential service consumers identify the target service and acquire a copy of its metadata. Librarians could be within each agency that provides public Services, or managed centrally. In either case, coordination of these activities assures that Services are consumed according to standard.

Service Candidate Proposals

The service candidate proposal contains a description of the service interface as well as all the functional and non functional requirements associated to the service that will be used in the Service Level Agreement. A complete proposal would contain a service definition that includes proposed operations, policies and other Services that might need to interact with the proposed Service before the new Service can be realized. It is useful to also describe the business processes, business capabilities, and business rules that the Service will support.

Monitoring Service Reusability

Since Service Oriented Architecture gains are the result of reusability, the SOAGB needs to assure that opportunities for re-use are present. It does this through consumer management, but it also achieves this end by monitoring actual use of Services so as to recommend modifications that can increase use across the enterprise. There are three important factors to consider when helping promote the specification of reusable services.

• A service interface must be complete with respect to its current and potential consumers. A good metric to track is the number of interface and implementation changes as new consumers come on board, for both the ones that are forward-compatible and the ones that are not.

Collaborative Application Development Among New York State Agencies

45 / 70

Page 46: InteragencyCollaboration 1-27FINAL

• Some SLA might work perfectly for one consumer, but fail to serve another. And Service Level Agreements, though well-intentioned, can be difficult to achieve in the real world. The SOAGB should keep track of the incidents and monitor the number of changes to Service Level Agreements that resulted from these incidents as well as the number of changes to the service implementation with a view toward improving future implementations and agreements.

• The SOAGB should seek to identify all potential consumers of a service candidate and involve each in the process of ratifying the service interface proposal. A good metric to track there is the number of unexpected customers found after a service was designed. This metric should be interpreted carefully since it could mean that the service was well designed and attracted a lot of consumers, or it could mean that not enough time was spent to identify the right consumers which resulted in a lot of subsequent changes.

Collaborative Application Development Among New York State Agencies

46 / 70

Page 47: InteragencyCollaboration 1-27FINAL

K. Conclusion

In modern computer literature, there are ample principles that guide us toward cost-effective, collaborative application development. New York State has recited these principles in formal standards and papers such as this. It is now time to give action to these principles and move further along the road toward building the systems and services which reduce the overall cost of state government, increase appropriate revenues, improve the quality of our governmental programs, and help us to more closely adhere to the laws and regulations under which we operate. To that end, the agencies submitting this study paper have embarked upon a road to collaborative development and joint governance which we believe will be a model for the entire New York State enterprise.

Collaborative Application Development Among New York State Agencies

47 / 70

Page 48: InteragencyCollaboration 1-27FINAL

Collaborative Application Development Among New York State Agencies

48 / 70

Page 49: InteragencyCollaboration 1-27FINAL

Appendix A - Open Standards OrganizationsStandards organizations are established to develop, coordinate, promulgate, revise, and interpret, canonical approaches that address the interests of a wide base of users outside the standards development organization. Although standard-making can be a tedious and lengthy process, formal standard setting is essential to developing new technologies. Since 1865, the telecommunications industry has depended on the International Telecommunications Union (ITU) to establish the telecommunications standards that have been adopted worldwide. The ITU has created numerous telecommunications standards including telegraph specifications, allocation of telephone numbers, interference protection, and protocols for a variety of communications technologies. The standards that are created through standards organizations lead to improved product quality, ensured interoperability of competitors’ products, and provide a technological baseline for future research and product development. Formal standard setting through standards organizations has numerous benefits for consumers including increased innovation, multiple market participants, reduced production costs, and the efficiency effects of product interchangeability.

Among some of the most well-known standards bodies are the following:

OASIS - The Organization for the Advancement of Structured Information Standards is a global consortium that drives the development, convergence and adoption of e-business and web service standards. OASIS was first formed as “SGML Open” in 1993 as a trade association of SGML tool vendors to cooperatively promote the adoption of SGML through mainly educational activities, though some amount of technical activity was also pursued including an update of the CALS Table Model specification and specifications for fragment interchange and entity management.

In 1998, with the movement of the high tech industry to XML, SGML Open changed its emphasis from SGML to XML, and changed its name to OASIS Open to be inclusive of XML and any future structured information standards. The focus of the consortium's activities also moved from promoting adoption (as XML was getting lots of attention on its own) to developing technical specifications. In July 2000 a new technical committee process was approved. With the adoption of the process the manner in which technical committees were created, operated, and progressed their work was regularized. At the adoption of the process there were five technical committees; by 2004 there were nearly 70.

During 1999 OASIS was approached by UN/CEFACT, the committee of the United Nations dealing with standards for business, to jointly develop a new set of specifications for electronic business. The joint initiative, called "ebXML" and which first met in November 1999, was chartered for a three year period. At the final meeting under the original charter, in Vienna, UN/CEFACT and OASIS agreed to divide the

Collaborative Application Development Among New York State Agencies

49 / 70

Page 50: InteragencyCollaboration 1-27FINAL

remaining work between the two organizations and to coordinate the completion of the work through a coordinating committee. In 2004 OASIS submitted its completed ebXML specifications to ISO TC154 where they were approved as ISO 15000.

Members of the consortium decide how and what work is undertaken through an open, democratic process.

NIST - The National Institute of Standards and Technology was known between 1901–1988 as the National Bureau of Standards (NBS). NIST is a non-regulatory agency of the United States Department of Commerce. The institute's mission is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve quality of life.

As part of this mission, NIST scientists and engineers continually refine the science of measurement, making possible the ultra precise engineering and manufacturing required for today’s most advanced technologies. They also are directly involved in standards development and testing done by the private sector and government agencies. U.S. technological innovation and progress depend on NIST’s unique skills and capabilities, especially in four key areas: biotechnology, nanotechnology, information technology and advanced manufacturing.

ISO - The International Organization for Standardization (Organisation internationale de normalisation), widely known as ISO, is an international standard-setting body composed of representatives from various national standards organizations. Founded on 23 February 1947, the organization promulgates world-wide industrial and commercial standards. It is headquartered in Geneva, Switzerland.

While ISO defines itself as a non-governmental organization, its ability to set standards that often become law, either through treaties or national standards, makes it more powerful than most non-governmental organizations. In practice, ISO acts as a consortium with strong links to governments.

W3C - The World Wide Web Consortium is the main international standards organization for the World Wide Web. It is arranged as a consortium where member organizations maintain full-time staff for the purpose of working together in the development of standards for the W3. As of March 2007, the W3C had 441 members. It is always open for new organizations to join.

The World Wide Web Consortium was founded by Tim Berners-Lee after he left the European Organization for Nuclear Research (CERN) in October, 1991. It was founded at the Massachusetts

Collaborative Application Development Among New York State Agencies

50 / 70

Page 51: InteragencyCollaboration 1-27FINAL

Institute of Technology Laboratory for Computer Science (MIT/LCS) with support from the Defense Advanced Research Projects Agency (DARPA)—which had pioneered the Internet—and the European Commission.

W3C was created to ensure compatibility and agreement among industry members in the adoption of new standards. Prior to its creation, incompatible versions of HTML were offered by different vendors, increasing the potential for inconsistency between web pages. The consortium was created to get all those vendors to agree on a set of core principles and components which would be supported by everyone.

It was originally intended that CERN host the European branch of W3C; however, CERN wished to focus on particle physics, not information technology. In April 1995 the Institut national de recherche en informatique et en automatique (INRIA) became the European host of W3C, with Keio University becoming the Japanese branch in September 1996. Starting in 1997, W3C created regional offices around the world; as of October 2007 it has sixteen World Offices covering Australia, the Benelux countries (the Netherlands, Luxemburg, and Belgium), China, Finland, Germany and Austria, Greece, Hungary, India, Ireland, Israel, Italy, Japan, South Korea, Korea, Morocco, South Africa, Spain, Sweden, and the United Kingdom.

In January 2003, the European host was transferred from INRIA to the European Research Consortium for Informatics and Mathematics (ERCIM), an organization that represents European national computer science laboratories.

IEEE - The Institute of Electrical and Electronics Engineers is an international non-profit, professional organization for the advancement of technology related to electricity. It has the most members of any technical professional organization in the world, with more than 360,000 members in around 175 countries. The IEEE is incorporated in the State of New York, United States. It was formed in 1963 by the merger of the Institute of Radio Engineers (IRE, founded 1912) and the American Institute of Electrical Engineers (AIEE, founded 1884).

The major interests of the AIEE were wire communications (telegraph and telephony) and light and power systems. The IRE concerned mostly radio engineering, and was formed from two smaller organizations, the Society of Wireless and Telegraph Engineers and the Wireless Institute. With the rise of electronics in the 1930s, electronics engineers usually became members of the IRE, but the applications of electron tube technology became so extensive that the technical boundaries differentiating the IRE and the AIEE became difficult to distinguish. After World War II, the two organizations became increasingly

Collaborative Application Development Among New York State Agencies

51 / 70

Page 52: InteragencyCollaboration 1-27FINAL

competitive, and in 1961, the leadership of both the IRE and the AIEE resolved to consolidate the two organizations. The two organizations formally merged as the IEEE on January 1, 1963.

IEEE is one of the leading standards-making organizations in the world. IEEE performs its standards making and maintaining functions through the IEEE Standards Association (IEEE-SA). IEEE standards affect a wide range of industries including: power and energy, biomedical and healthcare, Information Technology (IT), telecommunications, transportation, nanotechnology, information assurance, and many more. In 2005, IEEE had close to 900 active standards, with 500 standards under development. One of the more notable IEEE standards is the IEEE 802 LAN/MAN group of standards which includes the IEEE 802.3 Ethernet standard and the IEEE 802.11 Wireless Networking standard.

Collaborative Application Development Among New York State Agencies

52 / 70

Page 53: InteragencyCollaboration 1-27FINAL

Appendix B – NYS Enterprise Architecture Principles

[The following is an excerpt from the document issued by the New York State Office of the Chief Information Officer in March, 2004.]

Policy Title: Principles Governing The New York State Information Technology Enterprise Architecture

Issued By: New York State CIO Council, Technology Committee Issue Date: March 5, 2004

Principle 1: The Enterprise Architecture will maximize the cost-effectiveness of its information technology efforts.

The Enterprise will strive to reduce procurement, development, integration, and support costs associated with duplicative architectures and obsolete or unused technologies by providing a common architecture that is flexible, reusable and cost effective across the Enterprise. Migration of existing Enterprise applications to the common architecture will be gradual, with consideration given to cost and available budgets.

Future Enterprise applications and technologies will leverage New York State’s existing Enterprise technology assets (e.g., NYENET, State Data Center) where available and applicable. Opportunities for additional cost savings and operational efficiencies through further centralization of IT resources and functions will be explored. For instance, it may be cost-effective to centralize the administration of servers used for Enterprise Applications. Decisions concerning further centralization of resources should be based on the results of formal analyses and empirical data on total cost of ownership (see section 4.2.4 below) and impact on the agencies including in the Enterprise.

Collaborative Application Development Among New York State Agencies

53 / 70

Page 54: InteragencyCollaboration 1-27FINAL

Principle 2: The Enterprise Architecture will reduce the complexity of New York State’s information technology environment.

Especially complex technology solutions - for example, “spaghetti code” or wildly diverse hardware platforms - make change especially difficult, if for no other reason than the fear of breaking the whole complex mess by changing some small inter-related part of it. As complexity is reduced, the ability of the Enterprise to adapt and change is increased. Reduced complexity should also reduce the cost of products and their support through the leverage of Enterprise-wide buying power.

As a practical matter, this principle implies the need to impose and maintain the discipline to reduce the number of platforms, configurations, and products in the Enterprise, thereby reducing training and support requirements.

Principle 3: The Enterprise Architecture will be multi-tiered to maximize flexibility, adaptability and stability.

In a multi-tiered model, the overall Enterprise Architecture consists of several distinct but highly interrelated layers, each of which can be conceptualized as having its own distinct architecture. The Technology Committee has adopted the following taxonomy for the major domains of the Enterprise Architecture:

• Information Architecture;

• Business Process Architecture;

• End-User Architecture;

• Infrastructure Architecture (enabling hardware, software, network, storage), and;

• Security Architecture (protection and authentication).

We discuss each of these domains separately below.

3.2 Information Architecture

The information architecture deals with the modeling and use of the information assets of the Enterprise. “Information assets” includes both the information needed by the Enterprise to carry out its business

Collaborative Application Development Among New York State Agencies

54 / 70

Page 55: InteragencyCollaboration 1-27FINAL

functions, and also the information generated by the Enterprise. An overall goal for the information architecture is to maximize the utility of these information assets.

Principle 4: The Enterprise Architecture will facilitate information sharing across the Enterprise to enhance and accelerate decision-making.

Information itself is an Enterprise asset. Government decisions and services are better if they are based on the appropriate use of information. Thus, the EA needs to facilitate the use of information by decision makers and program staff. It needs to protect information from loss and corruption. Data elements need to have compatible definitions and consistent coding across the Enterprise. The capacity must exist to share data appropriately across the Enterprise, however, always within the boundaries established by law (e.g. HIPAA privacy protections).

A traditional Enterprise asset management strategy is well understood and implemented for the various physical inventories found in an Enterprise. Where traditional asset management falls short is in recognizing that information is as much an Enterprise asset as any of the physical equipment. By addressing information as a valuable Enterprise asset at the outset, decision-making, technology investment, and Enterprise-wide access to information can be accommodated.

Following this principle will require identification and authentication of information assets as well as a unified information asset management system. The information assets will need to be structured for easy access and management without compromises in security, privacy, and confidentiality.

Systems must be designed, acquired, developed, or enhanced such that data can be shared and integrated across the Enterprise and with our partners to the extent permitted by law. This will require the establishment of XML standards for common types of data records to facilitate information exchange.

Principle 5: The Enterprise Architecture will support data warehousing and other information-centric end-user computing.

The Information Architecture must support both online transaction processing (OLTP) and data warehousing applications. However, these two classes of applications require very different data models and make very different demands on database systems. On-line transaction processing focuses on quick updates of the data. Often, the speed of these updates can be dramatically slowed down by the processing

Collaborative Application Development Among New York State Agencies

55 / 70

Page 56: InteragencyCollaboration 1-27FINAL

generated by user queries. For this reason, it is best to separate the data warehouse from the OLTP. In so doing, we also provide for a more secure environment for both OLTP and data warehousing.

To successfully support data warehousing, the Enterprise must ensure that IT staff understand the different database design and performance requirements of OLTP and data warehouse applications, and that there are sufficient numbers of staff skilled in key data warehousing methodologies such as dimensional modeling, in which data models are partially de-normalized to facilitate easy end-user querying.

The Enterprise Architecture must include Enterprise-wide approaches to data warehousing, including methods for data warehousing that span agency boundaries as permitted by law in order to facilitate appropriate information sharing and integration, Enterprise-wide views of information assets, and elimination of unnecessary redundancies in data warehouse content, data storage, and end-user information product development. Recent advances in federated data warehousing, which allow for the integration through middleware of logically and physically disconnected data warehouse resources, should be investigated for their potential usefulness in this area in order to meet the operational demands of the Enterprise while at the same time ensuring privacy and confidentiality as required by law.

In addition to traditional methods for querying the data warehouse, publish-subscribe methods of access can increase the utility of data warehousing applications. Publish-subscribe methods include such features as automatic end-user notification of changes and updates to data warehouse content in areas of particular interest that they are authorized to access, and automatic receipt of particular data warehouse reports that they are authorized to receive. These methods can avoid the need to have users make continual requests of the database in order to determine if anything has changed. Thus, publish-subscribe methods can help reduce the overall volume of end-user queries, and thereby mitigate potential performance degradation from increasing numbers of end-users. However, such methods must be implemented in ways that ensure there are no inadvertent unauthorized disclosures of protected information.

Principle 6: The Enterprise Architecture will promote Enterprise-wide data standardization, reuse, interoperability, and information management across applications and agencies.

This principle implies the need for a central authority that maintains the standards to be used for such common data fields as name, address, etc. and also builds the appropriate meta-data that can enable the sharing of data. Industry data standards (e.g. HIPAA transaction standards in the health care arena) should

Collaborative Application Development Among New York State Agencies

56 / 70

Page 57: InteragencyCollaboration 1-27FINAL

be adopted whenever feasible to facilitate business transactions between local, state and national government entities and between government and the private-sector. A formal process for the development of Enterprise data standards should be established as part of overall Enterprise Architecture governance.

The business process architecture deals with the goodness-of-fit fit between information systems and the business processes they are meant to facilitate. This includes business process analysis and, where appropriate and feasible, business process re-engineering. Goals include common solutions for business process needs shared by multiple entities within the Enterprise, development of business logic models and components that can be reused across multiple applications, and increasing the efficiency of Enterprise business processes.

Principle 7: The Enterprise Architecture will facilitate common solutions for business processes and needs shared by multiple agencies.

Where there are common business processes across multiple agencies in the Enterprise, the Enterprise Architecture will facilitate the development of common solutions. This is in contrast to the traditional process of merely converting a set of physical requirements, without further analysis, into a delivered (silo) application. The key is the designer’s understanding of the process in concept, not how the users label it. To use a simple example, a software component that allows for the scheduling of a generic resource at a particular date and time could be part of many applications, as diverse as scheduling people or machines or whatever the user has in mind.

In this view, the requirements for applications are built from the bottom-up, driven by business users, as before. However, the solutions are architected, in part, from a set of modular, but Enterprise-level, components which can be used by any agency within the Enterprise. This should help accelerate software development, while keeping costs down.

Clearly, an inventory of these common solutions and components, as well as methodologies for maintenance of these components, will need to be established if this principle is to be achieved in practice. Appropriate mechanisms will need to be determined as part of overall EA governance.

Collaborative Application Development Among New York State Agencies

57 / 70

Page 58: InteragencyCollaboration 1-27FINAL

Principle 8: The EA will facilitate Enterprise-wide business logic code reuse, smooth integration of Enterprise business processes, and maximum utility for end-users.

In so doing, the business process logic must be designed, acquired, developed, or enhanced such that processes can be shared and integrated across the Enterprise and with our partners.

As with other tiers, the implementation of business rules will employ reusable components across the Enterprise.

Principle 9: Business process re-engineering will be considered when defining requirements for new Enterprise applications.

Technology is a critical tool to empower public servants to continuously improve government services. However, the return on technology investment will be limited if all it does is to automate inefficient and ineffective business practices. We envision business process improvements that result from the optimization of both human and technology resources.

Business Process Reengineering (BPR) utilizing Information Technology as an automation tool will help reshape the way the Enterprise conducts business. Adopting a BPR model will reduce the total cost of ownership, increase efficiency, improve customer services, and reduce development times across the Enterprise.

All Enterprise systems development must begin with BPR as an essential predicate to systems design. To the extent that such systems affect multiple entities within the Enterprise, a common BPR model will be adopted to ensure all relevant business operations are reviewed and streamlined before development and design progresses”.

The End-User Architecture deals with the interface and interaction between Enterprise technology and the end-user. Goals include user-interface consistency across applications, in keeping with system requirements, and user-interface designs that maximize usability.

Collaborative Application Development Among New York State Agencies

58 / 70

Page 59: InteragencyCollaboration 1-27FINAL

Principle 10: Enterprise applications will be designed with a standard “look and feel” to facilitate ease of use, software design simplification, and branding.

As with other layers of the architecture, the presentation layer will make use of standardized, reusable components. Standardized presentation layer components will provide Enterprise Applications with a common, consistent end-user interface. The benefits of this approach in the presentation layer are similar to those in other architecture layers: it allows for the upgrade, exchange, and reuse of products with minimal retooling or disruption to the overall environment. It encourages deeper functionality and reduces training time, since users are more likely to have experience with the component from other applications.

Principle 11: The Enterprise Architecture will define standards for basic information technology tools and services (e.g., email, voicemail, internet access) that employees should have available to them, consistent with available resources and job functions.

Employees across the Enterprise who perform relevant job functions should be given access to training tools and/or facilities to all basic services/applications, to the extent permitted by available resources. This will allow for greater volume discounts to be realized by the entire Enterprise, as well has having a larger pool of similarly trained users. It will also allow for an easier integration of new employees into agencies.

Principle 12: The Enterprise Architecture will employ server-based thin client solutions requiring only network access and a web browser for end-user access wherever such solutions are technically appropriate.

Through the use of server-based applications, thin client technologies (especially web-based clients), portals, and gateways, organizations can reduce the cost and complexity of all IT functions, making it easier to implement, deploy, manage, and monitor Enterprise applications and information resources. Server-based architecture provides the ability to rollout new applications and upgrades to the entire organization simultaneously.

The infrastructure architecture addresses the underlying enabling hardware, software and network that support Enterprise applications. It also addresses the communications between architecture layers and between systems.

Collaborative Application Development Among New York State Agencies

59 / 70

Page 60: InteragencyCollaboration 1-27FINAL

Principle 13: The EA will promote interoperability and integration across Enterprise applications.

The more applications are interoperable, the more likely it is that the Enterprise can respond flexibly and inexpensively to new user requirements. By its nature, interoperability also enhances integration across the Enterprise, thus enabling the Enterprise to provide better, more coordinated services to the public.

Principle 14: The EA will favor solutions that are highly partitioned, modular in design, that are comprised of components that are maximally decoupled, and that use standards-based messaging protocols for communication between external and internal systems.

An essential part of any strategy to reduce complexity and enhance flexibility in an Enterprise Architecture is to break down the traditional monolithic systems and to reduce as much as possible the extent of the coupling of different components. In this way, any internal change or improvement in a component will not require massive changes in software to be found elsewhere in the Enterprise.

The use of message-based communications is an especially useful tactic in the integration of legacy, silo systems into the Enterprise. This allows for the creation of a thin layer that wraps the legacy application, providing a more modern means for other software components to communicate with that application. Then the legacy system can be replaced in an orderly fashion, without that change causing further impact on the Enterprise.

Such an approach makes possible the use of efficient, user-friendly techniques. In particular, we would encourage the use of publish-subscribe, rather than continual request-reply, modes of data acquisition by the user (see section 3.2.2 above).

This kind of modular implementation will allow for the upgrade, exchange, and reuse of products with minimal retooling or disruption to the overall environment. Modularity will reduce the complexity and upgrade time of IT assets while providing the Enterprise with geographical independence, skill-leveraging, and improved functionality.

The implications of this approach include:

• A culture of reuse and sharing of code and components must be further nurtured within

the Enterprise.

Collaborative Application Development Among New York State Agencies

60 / 70

Page 61: InteragencyCollaboration 1-27FINAL

• Enterprise-wide component management will become a core competency.

• Design reviews will become an integral part of IT decision-making.

• Modular components will be shared across organizational boundaries, to the maximum

extent permissible.

Principle 15: The EA will establish uniform standards for Enterprise technology.

Standardization will facilitate consistency and uniformity across applications. It will simplify software design, reduce application development time, facilitate learning, improve system maintenance and support, and promote information-sharing among organizations within the Enterprise, and thus reduce total cost of ownership. By fostering the development of the same “look and feel” across the Enterprise, standardization will enhance end-user friendliness and satisfaction, and contribute to the identity of the Enterprise.

As part of this effort, existing IT platforms must be identified and documented and compared to Enterprise-wide configuration standards, which in turn must be established. A review process must be developed for setting standards, reviewing and revising them periodically, and granting exceptions where appropriate.

Principle 16: The EA will define a small number of standardized, easily-reproducible system configurations for deployment across the Enterprise.

Establishing configurations that are easily reproduced will cut down on costs associated with support and maintenance as well as simplifying training and knowledge transfer. This will also mean that any proposed changes must function correctly and consistently throughout the entire organization. Consistent configurations will help provide a similar look and feel to end-users, as well as being able to support users of all ability levels, from novice to expert.

This principle also makes possible the end-to-end systems management that is a necessary part of reliable delivery of technology services to our users. This will require a change in some decision-making standards. For example, we will deploy applications on uniformly configured servers. Proprietary software used in standardized system configurations will be maintained at vendor-supported version

Collaborative Application Development Among New York State Agencies

61 / 70

Page 62: InteragencyCollaboration 1-27FINAL

levels. For these reasons, there may be an initial increase in capital investment.

In the short run, this means that we will strive to replace multiple, non-standard configurations with a smaller number of consistent configurations. In the long run, this means that we plan for the retirement and replacement of obsolete platform components and configurations.

Principle 17: Enterprise applications and infrastructure will be scalable in size, capacity, and functionality.

Scalable components will better adapt to the changing needs of the Enterprise and will be less likely to impede its growth. Scalability, in effect, will serve to improve Enterprise-wide integration, maximize resource utilization, and minimize duplication and application redundancy.

Principle 18: The NYeNET will be used as a statewide network backbone for Enterprise applications and services.

This will foster greater collaboration and sharing of data between different agencies and entities.

Among the implications of this principle:

• Need to implement a robust, unified directory services capability.

• May require higher speed networks and higher bandwidth networks.

• Will require the interconnection of distributed LANs.

• Need to create connections from legacy systems to client/server and Internet applications.

Principle 19: Enterprise systems will adhere to all applicable security, confidentiality, and privacy policies and statutes.

This helps to safeguard confidential and proprietary information, as well protects its integrity. It enhances public trust and the proper stewardship over public information. In addition to the technical requirements of security, education on issues of privacy and confidentiality must become a routine part of normal business processes.

Collaborative Application Development Among New York State Agencies

62 / 70

Page 63: InteragencyCollaboration 1-27FINAL

Principle 20: The EA will use a single method of user authentication to control access to Enterprise applications and services.

The security framework will eliminate multiple IDs/passwords, substituting a single login (user authentication via a common LDAP directory service). User authentication will be augmented to include dual-factor methods (e.g. RSA tokens, smart cards, biometrics, and certificates) for sensitive applications where such extra measures are necessary to achieve the required level of security.

Principle 21: The EA will support variable, application-specific security and data retention requirements.

Security and data retention requirements will vary with the type of information and the Enterprise Architecture must support these various requirements. The types of information a system builds, edits and/or displays will dictate its security requirements. No matter what type of data, systems should be designed with security as an integral part.

An application should make use of rights assigned by the user login to the environment rather than building access controls within itself. Because of the continued existence of legacy systems, the security framework should be designed so that can be retrofitted to these older applications.

EA management includes the decision-making structures and processes needed to govern the overall Enterprise Architecture and the development and procurement of Enterprise systems. This section describes the governing structure to achieve the Enterprise Architecture and also the core day-to-day processes that must be managed in keeping with EA principles.

Principle 22: The EA will be planned and managed through a formal governance process.

Architecture support and review structures shall be used to ensure that the integrity of the architecture is maintained as systems and infrastructure are acquired, developed and enhanced.

A structured review process will be needed to ensure that information systems comply with the IT Architecture and related standards, and also to determine the appropriateness of any proposed exceptions to EA standards. Processes incorporating these EA principles must be developed for all application procurement, development, design, and management activities - building upon the current “Intent to Purchase” reviews, but not limited to that process.

The compliance process must allow for the introduction of new technology and standards, and also for the

Collaborative Application Development Among New York State Agencies

63 / 70

Page 64: InteragencyCollaboration 1-27FINAL

vetting and approval where appropriate of exceptions to standards. Conceptual Architecture and Technical Domain principles should be used as evaluation criteria for purchasing as well as developing software.

Principle 23: The EA will include an Enterprise-level architecture review process, which will include representatives of agencies that vary in size, responsibility, and location, to oversee the alignment of Enterprise systems with the EA.

This review process will evaluate the impact of new and continuing systems activities against the Enterprise Architecture principles. For example, the review process will assess the extent to which a proposed solution leverages and possibly enhances existing, already available, components.

Findings and recommendations from the review process will be provided to the NYS CIO, who is ultimately responsible for the determination of whether a project significantly conforms to the EA.

Principle 24: Appropriate disaster recovery and business continuity plans will be implemented to ensure the stability and integrity of Enterprise applications and information assets.

Systems must be categorized according to their level of importance for Enterprise business continuity. For those that are Enterprise-critical, fault tolerance and recovery protocols must be incorporated into the planning and design of the system. Similarly, work site recovery plans must be developed and implemented. These should prioritize Enterprise-critical systems.

Principle 25: The EA will be continuously reviewed to assess the potential impact, positive and negative, of advances in technology and industry trends.

The integrity of the architecture will be maintained, but the implementation reviewed as technology evolves. Compliance to the EA principles must allow for the introduction of new technology and standards. An adoption process, incorporating conceptual and technical concerns, should be developed and followed for the introduction of any new technology standard. Caution should be used when acquiring any newly developed products or technologies, especially when we do not have control over the source code or design.

Collaborative Application Development Among New York State Agencies

64 / 70

Page 65: InteragencyCollaboration 1-27FINAL

A necessary corollary to this principle is that there will be a periodic review of technologies that have become obsolete and need to be phased out.

Principle 26: Relevant business area experts will be included in application development/acquisition teams, and end-users will be consulted throughout the application lifecycle.

Success within most agency-based initiatives has always required collaboration between the program management, program specialists and IT staff. The business area or program experts are those stakeholders that are best equipped to provide feedback throughout the development process. Their involvement in any development team is thus essential. Clearly, participation in development teams requires both a time and energy investment in communication between the team members. This requires methods of communication and patterns of work that facilitate ongoing collaboration between end-user communities and IT staff.

Joint teams of technology and business experts present a unique challenge when those teams arise from different organizations. Determination of an overall project management process that incorporates the needs and priorities of each involved organization is especially important to the success of multi-agency initiatives.

The general increase in the use of previously built components described elsewhere in this document would make it easier for end-users to envision the system they will be using and to suggest improvements. This stands in contrast with the elaborate paper requirements process that preceded a long period of development for monolithic systems in the past.

Principle 27: All decisions concerning acquiring, developing, and enhancing systems will include an analysis of the relative impact of the decision on the Enterprise.

Applications and information are valuable State assets and must be protected. Enterprise business impact analysis will yield valuable information to assist IT decision makers in planning, acquiring, designing, developing, enhancing, and recovering systems, thereby ensuring the alignment of technology decisions with the mission of the Enterprise while protecting the integrity of applications and information.

Collaborative Application Development Among New York State Agencies

65 / 70

Page 66: InteragencyCollaboration 1-27FINAL

Principle 28: The Enterprise will adopt a total cost of ownership (TCO) model for information technology.

Consideration of the total costs of ownership—including those in affected agencies—associated with a system over its entire life span will result in significantly more cost effective system choices and will enable improved accuracy in the planning and budget decision-making across the Enterprise. Incomplete consideration of TCO can lead to idiosyncratic and “silo” solutions that became ineffective in a short time period.

The Enterprise must agree upon a TCO model. The model should include the costs of acquisition, development, support, enhancements, training, operations, disaster recovery and retirement against the costs of flexibility, scalability, ease of use, and reduction of integration complexity.

However, the TCO alone must not be used as the sole criterion for technology decisions. The TCO model should be used to inform the decision process, in conjunction with other criteria, including fit with business and legal requirements, and impact on overall Enterprise Architecture.

Principle 29: The EA will promote solutions for large projects that include intermediate deliverables for end-users.

Governmental IT initiatives are frequently large endeavors, with timetables often measured in years. “Big Bang” approaches to project delivery often fail because problems are not identified until it is too late in the life cycle to correct. Creative use of intermediate deliverables will reduce the incidence of project failure, improve communications between the customer and the supplier, and build momentum that can only enhance project success.

Identifying meaningful intermediate deliverables, particularly when deploying projects that interface with legacy systems, is a challenging task. Ensuring that this process does not slow down the process of deployment is an activity that must be monitored closely by both customer and the project team.

A well planned and implemented architecture will make it easier to provide frequent deliverables because of its use of pre-existing components and foundation in common patterns across the Enterprise.

Collaborative Application Development Among New York State Agencies

66 / 70

Page 67: InteragencyCollaboration 1-27FINAL

Principle 30: Enterprise applications will be developed using software engineering practices that are consistent with accepted industry standards.

The principles outlined in this document will require consistent commitment across the Enterprise. Adhering to industry-accepted software engineering practices is a necessary part of this discipline. It also reduces training costs and improves quality assurance. In the longer run, it can provide the basis for benchmarks and other measurements.

It is essential that appropriate engineering practices be described and promulgated as part of the EA, along with training of staff in these practices as required and consistent with available resources. Similarly, all third party developers will be expected to follow compatible industry-standard practices.

Principle 31: Enterprise applications and infrastructure will use commercially viable, industry-proven, widely-used technology to the maximum extent possible.

Use of industry-proven, widely-used technology allows for easier access to affordable skills and a large base of proven software solutions. It can reduce risk, and helps ensure robust product support. Wherever practical, the Enterprise should strive to implement commercial-off-the-shelf technology as a first preference over completely custom applications.

Principle 32: The EA will favor products and solutions that use open standards to facilitate interoperability between applications, systems and organizations.

Open standards are technology specifications that are publicly available and affirmed by an industry-recognized standards body. The use of open standards that allow for interoperability between applications and vendor-specific products is essential for the Enterprise Architecture to be successful. Requiring that products selected for Enterprise systems support open standards will also help ensure the flexibility and adaptability of the EA.

Collaborative Application Development Among New York State Agencies

67 / 70

Page 68: InteragencyCollaboration 1-27FINAL

Collaborative Application Development Among New York State Agencies

68 / 70

Page 69: InteragencyCollaboration 1-27FINAL

Appendix C – Manifesto for Agile Software Development

Agile Manifesto

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

Principles behind the Agile Manifesto

To achieve these ends, we follow these principles:

• Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

• Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

• Business people and developers must work together daily throughout the project.

• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

• Working software is the primary measure of progress.

• Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

• Continuous attention to technical excellence and good design enhances agility.

Collaborative Application Development Among New York State Agencies

69 / 70

Page 70: InteragencyCollaboration 1-27FINAL

• Simplicity—the art of maximizing the amount of work not done—is essential.

• The best architectures, requirements, and designs emerge from self-organizing teams. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Collaborative Application Development Among New York State Agencies

70 / 70