UNit-vii MIS

8/7/2019 UNit-vii MIS

1/44

Acquisition of SW and HW (software and hardware)

Acquisition process improvement has been defined as the analysis of current softwareacquisition processes for deficiencies and implementing new/modified processes tocorrect those deficiencies. The software acquisition process improvement, defining it as:

A documented process for software acquisition planning, requirements

development/management, project management/oversight and risk management

Efforts to develop appropriate metrics for performance measurement and

continual process improvement

A process to ensure that key program personnel have an appropriate level of

experience/training in software acquisition

A process to ensure that each military department and defense agency select,

implement and adhere to established processes and requirements relating to software

acquisition

One of the better descriptions of the characteristics of a software acquisition process thatis successfully being used by leading software acquirers/developers is contained in theStronger Management Practices. Three fundamental management strategies are: Evolutionary Environment

Disciplined Development Processes

Collecting/Analyzing Meaningful Metrics

These strategies limit software development efforts to what can be managed and result indecisions based on sufficient systems engineering knowledge to adequately assess risks.In addition, business decisions are consciously made to invest time/resources in achievinghigh software process maturity levels. Strong upper-level management support is astrong enabler for ensuring that these combined strategies are successful.

Essential conditions in acquisition of SW AND HW

Organizations that have a strong, consistent evolutionary environment and practices forsetting product requirements, maintaining a disciplined development process and usingmetrics to oversee development progress achieve favorable cost, schedule and qualityoutcomes.

1


2/44

Acquisition

The conceptualization, initiation, design, development, test, contracting, production,deployment, Logistics Support (LS), modification, and disposal of software systems,

Acquisition environment

Internal and external factors that impact on, and help shape, every software acquisitionprogram. These factors include political forces, policies, regulations, reactions tounanticipated requirements and emergencies.

Acquisition life-cycle

The life of a software acquisition program consists of phases, each preceded by amilestone or other decision point, during which a system goes through Research,

Development, Test and Evaluation (RDT&E) and production.Acquisition program

A directed, funded effort over the software acquisition life-cycle that provides a new,improved, or continuing materiel, weapon or information system or service capability inresponse to an approved need.

Acquisition process

The steps involved, and practices employed, in the planning, contracting,implementation, acceptance and follow-on activities of a software acquisition. Theacquirer will bear most of the responsibility for the results from the planning, contracting,and follow-on phases of the acquisition, while the developer is most accountable for theresults coming out of the implementation and acceptance phases of the acquisition.

Development process

The process of working out and extending the theoretical, practical, and usefulapplications of a basic design, idea, or scientific discovery, characterized by the design,coding, modification or improvement of software.

2


3/44

The SW&HW Acquisition Process

It is necessary to first understand what a software acquisition process is before steps canbe taken to improve In following figure a generic nine-step process for softwareacquisition, summarized in Figure 1.

3


4/44

4


5/44

Figure 1: IEEE Std 1062, 1998 Edition S\W AND HW Acquisition Process StepsThere are three steps that comprise the planning phase of a software acquisition. Duringthis phase, the acquisition strategy should be planned based on a review of the acquirers

objectives. A formal Acquisition Strategy document is developed to guide theacquisition.

Implementing the acquisition strategy into the organizations process is the next step.Appropriate contracting practices need to be included in the process. The final step of theplanning process encompasses the determination of software requirements. In addition todefining the software being acquired, this step includes the preparation of the quality andmaintenance plans for accepting the suppliers/developers software. The release of aRequest for Proposal (RFP) is the completion milestone for the planning phase of thesoftware acquisition life-cycle.

The contracting phase of the software acquisition is also comprised of three steps. Thefirst step is the identification of potential suppliers. Suppliers should be selected whowill provide documentation for their software, demonstrate its performance, and submitformal proposals. Failure to perform any of these actions may serve as a basis forrejecting a potential supplier.

Past performance data from previous contracts should be reviewed for all potentialsuppliers. The next step is to actually prepare the contract requirements. The quality ofwork (acceptable performance and acceptance criteria) should be described, and contractprovisions that tie payments to deliverables should be prepared. Legal counsel shouldalso be sought to review and approve contractual language/requirements. Finally,proposals will be evaluated and a qualified supplier(s) will be selected. If appropriate, analternate supplier should be negotiated with, primarily as a risk reduction measure.

Supplier selection completes the contracting phase of the software acquisition life-cycle,which started with the release of the RFP and ends with the contract signing.Once the contract is signed, software development can begin and the productimplementation phase of the software acquisition life-cycle is initiated. During thisphase, the acquirer must manage supplier performance. The suppliers progress ismonitored to ensure all milestones are met. The acquirer also must approve allappropriate work products. Note that, during this phase, the acquirer has theresponsibility to provide all acquirer deliverables to the supplier when required. Theproduct implementation phase of the software acquisition is completed when the softwareproduct is delivered to the acquirer (pre-approval).

The next phase of the software acquisition life-cycle is product acceptance. Adequatetesting needs to be performed, and a process for certifying that all discrepancies havebeen corrected and all acceptance criteria have been satisfied must be established.Acceptance of the software product by the acquirer based on mutually agreed-to pre-defined criteria signals the end of the product acceptance phase.

5


6/44

The follow-on phase of software acquisition covers the period between softwareacceptance and software retirement, when it is no longer in use. A follow-up of thesoftware acquisition contract should typically be performed to evaluate contractingpractices, record lessons learned, and evaluate user satisfaction with the software. During

this final phase of the software acquisition life-cycle, the acquirer should record andretain supplier performance data to use for the acquisition of future software products.More recently, the Software Engineering Institute (SEI) has released the SoftwareAcquisition Capability Maturity Model to represent the important phases and elements ofthe software acquisition processes.

Key areas necessary in SW &HW acquisition

The areas necessary to be considered while planning for acquisition of SW &HW orhardware includes many sub-components as mentioned in table -

Table 1: Key Process Areas

LEVEL FOCUS KEY PROCESS AREA5Optimizing

Continuous ProcessImprovement

Acquisition Innovation Management Continuous Process Improvement

4Quantitative

Quantitative Management Quantitative Acquisition Management Quantitative Process Management

3Defined

Process Standardization Training Program Acquisition Risk Management

Contract Performance Management Project Performance Management User Requirements Process Definition and Maintenance

2Repeatable

Basic ProjectManagement

Transition to Support (Acceptance/Follow-On) Evaluation (Implementation/Acceptance) Contract Tracking and Oversight(Implementation) Project Management (Implementation) Requirements Development and Management(Planning/Contracting)

Solicitation (Planning/Contracting) Software Acquisition Planning (Planning)

1Initial

Competent People and Heroics

6


7/44

What is Software Acquisition Process Improvement?

Acquisition process improvement has been defined as the analysis of current softwareacquisition processes for deficiencies and implementing new/modified processes to

correct those deficiencies. The most comprehensive definition of software acquisitionprocess improvement, defining it as: A documented process for software acquisition planning, requirementsdevelopment/management, project management/oversight and risk management

Efforts to develop appropriate metrics for performance measurement andcontinual process improvement

A process to ensure that key program personnel have an appropriate level ofexperience/training in software acquisition

A process to ensure that each military department and defense agency select,implement and adhere to established processes and requirements relating to softwareacquisition

SW AND HW acquisition/development life-cycle:

Acquisition planning Requirements development/management Configuration management

Risk management Project management/oversight Test & evaluation Integrated team management Solicitation/source selection Performance measurement

Factors supporting software acquisition process improvement

System/software objectives and constraints are adequately defined /validated

System/software acquisition strategies are appropriate /compatible

Success-critical stakeholders have committed adequate softwareCapability / resources to perform their software-related tasks

Software product/process plans are feasible/ compatible

7


8/44

Feasibility Rationale provides convincing evidence that software progress issatisfactory

Specified improvements include

(1) Documenting agreements between the software developer and acquirer that containbaseline requirements based on systems-engineering knowledge,

(2) Gated reviews during the software development process

(3) obtaining meaningful metrics from the developer for managing the program,

(4) attaining knowledge about the technology that is planned early in the developmentprocess,

(5) ensuring an appropriate balance between requirements and available resources,

(6) ensuring that the software design is mature before it is released, and

(7) having all processes under control prior to release. In general, the more thatacquisition managers know about software development processes and metrics, the betterequipped they are to improve the processes for software acquisition.(8) provide applicable improvement program administration and compliance guidance

and ensure that secretaries of the departments and selected agencies comply with thatguidance and

(9) assist departments/agencies with their respective improvement programs by ensuring

the use of applicable source-selection criteria and access to information regardingsoftware acquisition/development best practices in both the public and private sectors.

Three fundamental management strategies for SW AND HW acquisition are

Evolutionary Environment

Disciplined Development Processes

Collecting/Analyzing Meaningful Metrics]

These strategies limit software development efforts to what can be managed, and theyresult in informed decisions based on sufficient systems engineering knowledge such thatrisk can be adequately assessed. In addition, business decisions are consciously made toinvest sufficient time and resources in order to achieve high software process maturitylevels (high quality, on-time, within budget). Committed upper-level managementsupport is an effective enabler for ensuring that these combined strategies are successful.

8


9/44

Suggestions for strengthening acquisition plans included specific

criteria are

Requirements baselines based on systems engineering are documented/agreed toby both the acquirer/developer before a program is initiated

Cost/benefits analyses are required when new requirements are proposed

Software acquirers/developers make efforts to continually improve practices overtime Gated reviews and deliverables are integrated into development processes

Requiring software developers to collect/analyze metrics (including earned value)to obtain knowledge of progress and to manage risk

Evolutionary Environment:

An evolutionary software acquisition environment is characterized by incrementalimprovements to software performance, as opposed to succumbing to programmaticpressures to set unrealistic expectations, in order to improve software processes on acontinual basis. An evolutionary environment limits software development to what ispossible to manage by adhering to well-understood, well-defined, manageablerequirements while simultaneously encouraging continuous process improvement. Anevolutionary product development is one of the fundamental elements of a manageableacquisition environment. The concept of continuous improvement is a critical part of theevolutionary environment, and must be supported by both the environment and theorganizational culture (including senior management) in order to be successful. Ad hoc

processes make it impossible to understand exactly when and how defects in the processoccur.

Disciplined Development Processes:Developers must follow disciplined, structured management and development processesin order for the software acquisition process to improve. Each phase of the softwareacquisition should end in a management review/gate to ensure that the project is ontrack. To pass these gated reviews, developers must demonstrate that acquirersexpectations and quality standards are met before proceeding to the next phase. Peerreviews to check each others work and remove defects, beginning at the earliest stages

of software development, are necessary to prevent costly rework and schedule delaysresulting from downstream defect detection. Disciplined software development processesthat are characterized by their reliance on configuration management, peer reviews andquality assurance significantly increase the probability of a successful acquisition, andhave been proven to assist leading organizations in identifying opportunities for softwareprocess improvement. As a result, areas of risk can be identified early and actions can betaken to control them.

9


10/44

Figure 4 highlights a disciplined knowledge-based software development process that canserve as a basis for software acquisition process improvement, describing the informationneeded, the relevant activities and the deliverables for each phase of the process.

Figure 4: Highlights of the Knowledge-Based Software Development Process [GAO,2004]Requirements need to be managed and controlled before design work begins and alllower-level design elements should be adequately defined before the start of coding. It iscritical that the software acquirer and developer work closely together and have open andhonest discussions. Requirements management should be viewed as one of the mostcritical development tasks for ensuring a successful acquisition.

Requirements should be sufficiently documented and validated prior to preliminarydesign. In addition, acquirer/developer stakeholders must use sound systems engineeringtechniques to establish the software requirements, and aggressively manage and controlrequirements changes.

To enhance software coding and testing quality, requirements should be well-written andachievable. By this phase of the acquisition, designs should be sufficiently detailed.Other processes that are critical to achieving high quality software include peer reviews,

10
https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]


11/44

documented coding standards, frequent unit testing, access to a reuse library, and the useof program languages that enable code documentation to facilitate future understanding.Test plans should not be developed until after requirements are considered stable, andthere should be a minimum of one test to verify/validate each requirement.

One of the most widely-known and best-structured software acquisition and developmentprocesses is the SEIs CMMI series of processes. A very recent addition to this suite oftools is the CMMI Acquisition Module (CMMI-AM.

The focus of the CMMI is on what should be done, not how it should be done, so itallows a great deal of latitude in terms of implementation of the CMMI Key ProcessAreas. As with the other CMMI documents, a primary emphasis of the AcquisitionModule is on Integrated Product and Process Development (IPPD) concepts. Theseinclude the effective use of cross-functional or multidisciplinary teams; leadership

commitment (particularly of senior management); appropriate allocation/delegation ofdecision-making authority; definition of organizational structures that reward teamperformance; the design of downstream processes (transition to Operations and Support O&S) during the acquisition; a strong focus on customer needs throughout the softwarelife-cycle; timely and effective collaboration between all relevant stakeholders; thecontinuous and proactive identification and management of risk; and a focus on themeasurement and improvement of processes to develop/deliver a software product.

A graphical representation of the CMMI-AM process is shown in Figure 5. Followingthat figure is Table 2, which includes checklist-type questions that can be used to self-assess coverage of each defined CMMI-AM process area. The reader is referred to theCMMI-AM document to get a more in-depth discussion of each of the process areasdefined in Figure 5 and Table 2.

11


12/44

Figure 5: Graphical Representation of the CMMI-AM ProcessTable 2: CMMI-AM Acquisition Process Areas [CMMI-AM, 2004]Process Area QuestionsOrganizational Environment

for Integration

Is infrastructure provided to establish fully functional teams fromamong all stakeholders?

Does the acquisition project have two focuses on integration technicaland organizational? Are technical capabilities/requirements examined/understood as theyrelate to overall project goals/objectives? Do organizational project entities operate with a single vision andcooperative purpose? Do project members understand the interrelationships of all projectaspects and how they relate to specific goals/objectives?

12
https://goldpractice.thedacs.com/practices/api/#_[CMMI-AM,_2004]%23_[CMMI-AM,_2004]https://goldpractice.thedacs.com/practices/api/#_[CMMI-AM,_2004]%23_[CMMI-AM,_2004]


13/44

IntegratedAcquisitionProjectManagement

Are project management processes consistent with/tailored from theorganizations standard processes? If a standard set of processes does not exist, has the project defined itsown processes to meet project needs? Is there an integrated management process that incorporates/involves all

stakeholders? Is the project management process defined in an overall ProjectManagement Plan, or equivalent document? Are formal interfaces used among stakeholders, in the form ofMemorandums of Understanding (MOUs), Memorandums of Agreement(MOAs), contractual commitments, associate contractor agreements, andsimilar documents, depending on the nature of the interfaces and involvedstakeholders? Does the projects defined process include all life-cycle processes(including IPPD) that are applied by the project? Do the projects defined processes include (1) selecting the team

structure, (2) allocating personnel resources, (3) implementing cross-integrated team communication, and (4) conducting issue-resolutionprocesses? Does development of a comprehensive project plan or the nature of theprojects defined process require an iterative effort due to a complex,multilayered, integrated team structure?

AcquisitionProjectPlanning

Does planning start by setting the acquisition project strategy, thenplanning the acquisition process in increasing levels of detail? Does the acquisition project review potential suppliers planningprocesses for adequacy and compliance? Are potential suppliers plans reviewed for consistency with system

acquisition plans? If the acquisition project involves replacement of an existing system,has operation/disposal of the existing system been considered as part of theacquisition planning for the new system? Are transition activities included in the acquisition project plan? For integrated teams: Does acquisition project data include that developed/used solely by aparticular team, as well as data applicable across integrated teamboundaries? Does acquisition project resource planning consider staffing of theintegrated teams?

Is stakeholder involvement planned down to the integrated team level? Is special attention paid to resource commitments? Do integrated team plans get buy-in from team members, interfacingteams, the acquisition project, and the acquisition process owners whosestandard processes are selected for tailoring?

Solicitation &ContractMonitoring

Does the project comply with applicable federal, departmental andservice acquisition regulations and policies? Does the solicitation address issues appropriate to the system domain or

13


14/44

acquisition environment (e.g., supplier process evaluations, operationalsafety suitability/effectiveness, safety, certifications, architecturalevaluations and interoperability)? Are representatives responsible for functional disciplines within theacquisition project or stakeholder organizations consulted for appropriate

inclusion of those disciplines into the solicitation and contract monitoringprocess? When integrated teams are formed, is team membership negotiated withsuppliers and incorporated into an agreement? Do teaming agreements identify integrated decision-making, reportingrequirements (business and technical) and trade studies requiring supplierinvolvement? Are supplier efforts orchestrated to support the acquisition projectsIPPD efforts?

RequirementsManagement

Does requirements management include the direct management ofacquirer-controlled requirements and oversight of supplier requirements

management? Are requirements managed/maintained such that changes are notimplemented without assessing the impact on the project? Does the requirements management process define approvedrequirements providers and an approved path by which requirements areprovided to the supplier? Are suppliers prevented from receiving requirements changes fromunauthorized sources that are outside the flow of the acquirers establishedrequirements management process? Are commitments to requirements by acquisition project participantsdemonstrated by having coordinated/approved documents that define

requirements? Is each change to a controlled requirement assessed for impact onacquisition project performance, cost and schedule baselines, and projectrisk? Are existing cost, schedule and performance baselines changed, asrequired, to accommodate a requirements change?

RequirementsDevelopment

Are customer requirements grouped/coordinated into a set ofrequirements that will define the scope/direction of the acquisition? Do customer requirements and additional acquirer requirements becomethe basis for the processes used by the suppliers organization? Do requirements flow from the stakeholders/customer to the system

level, to multiple subsystem levels and, eventually, to software componentlevels? Is the responsibility for requirements development/flowdown splitbetween the acquirer and developer, and who is responsible for whichlevels? Is there an iterative process of requirements allocation, high-leveldesign and requirements definition for the next lower level? Does the acquirer exercise responsibility for defining/baselining the

14


15/44

requirements levels under their control? Does the acquirer exercise responsibility for monitoring the suppliersdefinition of lower-level requirements, and reviewing/ensuring thatrequired work products are being developed.? Does the acquirer exercise responsibility for monitoring the lower-level

requirements development process to ensure that requirements produced ateach level result in a system that satisfies customer/operationalrequirements? In addition to functional/performance requirements, are interfacerequirements also covered? Do requirements also include non-functional requirements such as datarights, delivery dates, milestone exit criteria, and attributes such asevolvability, maintainability and reusability? Is the same requirements process used for acquiring services instead ofproducts?

Integrated

Teaming

Does integrated teaming consider the overall scope of/requirement for

participation of stakeholders from users; acquisition executives; acquisitionorganizations; developers (primes, subcontractors, suppliers, vendors); testorganizations; and other support organizations? Should the team adopt a common process for team operation or rely oneach team member to use his/her own organizations processes? Are various team member processes compatible at the interface pointsof the processes? Do life-cycle support considerations drive the selection of processesacross the team members (tools, support data commonality/compatibility)? Are integrated information infrastructures needed tofacilitate/coordinate within/external to the team, especially for

geographically dispersed members/stakeholders?Validation Is validation performed early and continuously throughout theacquisition life-cycle? Are validation processes used to demonstrate that acquisition processwork products fulfill the acquisition strategy, and that the processes willsuccessfully acquire products/services? Are validation processes used to ensure that received products/servicesfulfill their intended use? Is the test community treated as a major stakeholder in the validationprocess, beginning with up-front planning through final system validation? Has the acquisition project defined the degree to which validation is

required, both early in the acquisition project definition and whenproducts/systems are received? Do acquisition project plans identify adequate resources to executevalidation activities?

Verification Is verification performed early and continuously throughout theacquisition life-cycle? Does the acquisition project ensure that selected work products meetproject requirements?

15


16/44

Does the acquisition project ensure that the evolving acquired productssatisfy contractual requirements?

Transition toOperations &Support

Does acquisition planning for transition include establishing supportstrategies through organic support infrastructures, contractor logisticssupport, or other sources?

Are the roles/responsibilities of the acquirer, supporter and user definedin the life-cycle support of the system? Has the acquirer determined whether it will execute the transitionfunction directly, or as a result of the acquisition itself? Are responsibilities for capability enhancements during the supportphase defined, taking into account the magnitude/complexity of theenvisioned change? Is the organization responsible for support (i.e., level 1 maintenance)and enhancements (i.e., level 2 maintenance) explicitly identified? Has the acquisition project assigned responsibility for implementationof process improvement practices?

Is the acquisition project working with operational units to plan forproduct transition into operational use and eventual disposal of the product(or technical/functional obsolescence of the software)?

ProjectMonitoring &Control

Are monitoring/control functions implemented in the acquisition projectas acquisition planning is performed and the acquisition strategy isdefined? Does the acquisition project have internal processes, plans and workproducts that should be monitored for progress/satisfactory completion? Do the monitored work products include specifications, plans, Requestfor Proposal components, etc.? Do monitored items include staffing levels/qualifications, system

performance objectives/thresholds, infrastructure readiness (tools,networks, etc.) and other project planning activities/products? Is acquisition project risk actively identified and mitigated? Is corrective action applied when execution does not match acquisitionproject planning (e.g., internal staffing, project plan completion dates,draft/final solicitation & contract award milestone slips)? Are corrective actions required to resolve variances from acquisitionproject plans tracked to closure? After supplier selection/contract award, is monitoring and control stillapplied to the internal acquisition project After supplier selection/contract award, does monitoring and control

also cover supplier performance against the suppliers project plan?AcquisitionProcess &ProductQualityAssurance

Have the acquisition project products and processes (e.g., solicitationpackages, etc.) been evaluated? Has the solicitation package been developed per the standard/formatagreed to by the project team? Does the solicitation package conform to all applicable policies andlaws? Does the acquisition projects risk management process conform to that

16


17/44

called out for in the risk management plan?RiskManagement

Is the acquisition strategy influenced by risk identification andestimation of risk probability/impact? Does the acquirer assess/manage overall acquisition project risks overthe duration of the project?

Does the acquirer assess/manage risks associated with supplierperformance? For acquisitions using integrated teams, are risks associated with loss ofinter- or intra-team coordination considered?

ConfigurationManagement

Are acquisition work products (e.g., solicitation packages) placed underinternal CM control? Are interim/final primary/subordinate supplier work products monitoredto ensure that project goals are met? Are provisions made for conducting CM of supplierproducts/documentation? Are methods established/maintained to ensure that acquisition project

data is complete/consistent? Has the acquisition project determined which work products should beCM-controlled?

DecisionAnalysis &Resolution

Is there a repeatable, criteria-based decision-making process used formaking critical decisions that define/guide the acquisition process itself? Is there a repeatable, criteria-based decision-making process used formaking critical decisions with the selected supplier? Does the process for decision making document the acquisition projectdecision rationale? Can criteria for critical decisions be revisited when changes/technologyinsertion decisions impacting essential requirements are considered?

Measurement& Analysis Does the acquisition project have the information it needs fordetermining status of its activities throughout the acquisition life-cycle, thesuppliers activities per contractual requirements and the status of theevolving products acquired? In acquisition projects requiring multiple products or teamingrelationships, is additional information needed/available to ensureprogrammatic, technical and operational interoperability objectives areidentified/measured/achieved?

The use of checklists such as the one above helps maintain discipline and structure in thesoftware acquisition process, and can cover any phase of a software acquisition life-cycle-Table 3: IEEE Std 1062, 1998 Edition Software Acquisition Checklists [IEEE, 1998]

17
https://goldpractice.thedacs.com/practices/api/#_[IEEE,_1998]%23_[IEEE,_1998]https://goldpractice.thedacs.com/practices/api/#_[IEEE,_1998]%23_[IEEE,_1998]


18/44

No.

Category Subcategories

1 Organizational Strategy2 Define the Software3 Supplier Evaluation Financial soundness

Experience/capability Development/controlprocesses Technical assistance Quality practices

Maintenance service

Product usage Product warranty Costs Contracts

4 Supplier/AcquirerObligations

5 Quality/Maintenance Plans Contents of a qualityplan

Contents of amaintenance plan

6 User Survey Operation Reliability

Maintenance service Performance Flexibility

Installation Costs

Security Documentation Miscellaneous

7 Supplier PerformanceStandards

Performance criteria Evaluation/test

Correction ofdiscrepancies Acceptance criteria

8 Contract Payments9 Monitor Supplier Progress10 Software Evaluation Functionality

Performance Reliability Availability Ease of modification

Serviceability Ease of installation Ease of use Adequacy ofdocumentation Cost to acquire/use

11 Software Test12 Software Acceptance Rate actions for

certification Rate remedies ifsupplier fails

Finally, the use of checklists can benefit specific monitoring and oversight activities ofsoftware acquisition such as risk management. Gallagher[Gallagher, 1997] defines asoftware acquisition risk management Key Process Area (KPA) that can be put into ahigh-level checklist form (see Table 5) in order to assess whether Goals, Activities,Commitments, etc., associated with risk management have been covered in the softwareacquisition process.Table 5. Software Acquisition Risk Management KPA [Gallagher, 1997]

18
https://goldpractice.thedacs.com/practices/api/#_[Gallagher,_1997]%23_[Gallagher,_1997]https://goldpractice.thedacs.com/practices/api/#_[Gallagher,_1997]%23_[Gallagher,_1997]https://goldpractice.thedacs.com/practices/api/#_[Gallagher,_1997]%23_[Gallagher,_1997]https://goldpractice.thedacs.com/practices/api/#_[Gallagher,_1997]%23_[Gallagher,_1997]


19/44

Category Goal Description Check Goals 1 SA risk management is an integral part of the projects

defined SA process ______ 2 The project identifies/deals with risk in a positive manner,

such that identification is recognized/rewarded, and results

in effective risk handling ______ Activities 1 SA risk management activities are integrated into SA

planning ______ 2 The Software Acquisition Risk Management Plan is

developed in accordance with the projects defined SAprocess ______

3 The project team performs its SA risk managementactivities in accordance with documented plans ______

4 Risk management is conducted as an integral part of thesolicitation, project performance management, andcontract performance management processes ______

5 SA risk handling actions are tracked and controlled untilthe risks are mitigated ______

Commitments 1 The acquisition organization has a written policy for themanagement of SA risk ______

2 Responsibility for SA risk management activities isdesignated ______

Abilities 1 A group that is responsible for coordinating SA riskmanagement activities exists ______

2 Adequate resources are provided for SA risk managementactivities ______

3 Individuals performing SA risk management activitieshave experience or receive required training ______

Measurement 1 Measurements are made/used to determine status of theacquisition risk management activities and resultantproducts ______

Verification 1 Acquisition risk management activities are reviewed byacquisition organization management on a periodic basis ______

2 Acquisition risk management activities are reviewed bythe project manager on both a periodic and event-drivenbasis ______

Collecting/Analyzing Meaningful Metrics:Meaningful metrics are collected and analyzed by both software acquirers and developersin order to track progress, confirm knowledge, manage risk, improve processes andensure that all stakeholders are well-informed. Meaningful and suitably tailored metricsshould cover cost, schedule, software size, requirements, tests performed/completed,number of defects detected/corrected, and quality attributes. Acquirers can apply some ofthese same metrics to assess whether the developer will be able to deliver software withincost, schedule, performance and quality constraints. The use of earned value analysis

19


20/44

tools to track cost/schedule and help mitigate risk is an important driver in determiningmeaningful metrics, supporting the ability of a program to maintain consistent,predictable practices and acceptable process outcomes. Sharing of metrics betweendevelopers and acquirers is an important aspect of a successful software acquisitionprocess.

Leading software developers have been described as relentless in their efforts to collectmetrics to improve project processes and results[GAO, 2004]. They typically alsoestablish goals and track metrics for company-wide initiatives, such as cost reductionefforts and customer satisfaction. These leaders have continually emphasized the criticalnature of measuring processes, collecting metrics and using them to improve performancein their workforce through training. In addition, a good part of their success can beattributed to their use of an Earned Value Management System (EVMS) and a projecttime-tracking system (to record time spent on cost of quality and cost of poorquality). The cost of poor quality is a direct reflection of the relative level ofeffectiveness of an organizations software acquisition and/or development processes.Typical metrics used by leading software developers are indicated in Table 6.

Software and hardware evaluation criteria

1)

Table 6: Metrics Used by Leading Software Developers [GAO, 2004]MajorMetric

Examples Usefulness

Cost Cost per phase Effort per phase Planned vs. actual cost

Cost performance index

Products of an earned valuemanagement system, indicatingprogress towards completing software

development against the plan Poor cost performance indexindicates problems meeting cost goals May need to consider project scopereduction to meet release dates, orcanceling the program

Schedule Planned vs. actual delivery dates Schedule estimation accuracy Percentage of project on time Schedule performance index

Products of an earned valuemanagement system, indicatingachieved schedule progress againstthe plan Used over the development life-

cycle to gauge progress towarddeveloping key products or meetingcritical milestones Close attention to scheduledeviations identifies the ability tomeet project goals, and if/whenadditional resources are needed

Size Amount of new, modified and Used by management to compare

20
https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]


21/44

reused code Size estimation accuracy

amount of code produced vs.estimated Changes to size estimates indicatepotential cost/schedule problems

Requirements Total requirements/features

committed for delivery Percentage of requirementscompleted Number of requirements changes byphase

Used to assess progress towards

meeting acquirers performancedemands Large number of changes, or latechanges, can impact cost/schedulecommitments Large number of changes, or latechanges, can result in software withhigher defect levels

Tests Number if tests planned, completed,passed

Percent of plan tests completed Percent of planned tests passed

Determine extent to which plannedsoftware tests have been successfully

accomplished Deviation from planned number oftests may indicate inadequate testingand subsequent quality problems,resulting in costly rework in laterproject phases

Defects Number of defects per phase Phase defect originated vs. phasefound Cost to fix defect

Severity of defect Total unresolved defects

Track software problems to (1) thephase where they were found, (2) thephase where they should have beenfound and (3) determine the cost to

fix Defects found after the phase inwhich they were created indicateperformance problems that mayincrease cost/schedule due to reworkof software and correction ofdevelopment processesToo few defects found may indicateinadequate test coverage during thetest phase, or insufficient formaltechnical review of documents in the

design phaseQuality Cost of quality efforts Cost of poor quality (rework) Number of quality goalsmissed/achieved Customer satisfaction survey results

Provides information on potentialreliability of delivered software Indicates the amount ofmoney/time invested in developmentto assure a given quality level Defects found/fixed in the phasethat they occur indicates good process

21


22/44

quality Defects found/fixed in downstreamphases are costly/time-consuming,and indicate weaknesses in thedevelopment process that will require

corrective action

2)Table 8: Recommended COTS-Based Software System AcquisitionBest PracticesPractice CommentsBest Practices for Establishing a Program BaselinePerform Software Architecture TradeStudies

Perform with system architecture trades Include major COTS and legacy components Supports Government software architecture

baseline selection Include user in all trades

Determine Realistic, IndependentBaseline Software Estimates

Size, effort, cost and schedule COTS, reuse and newly-developed Include tasks not reflected in cost models Include COTS refresh through both developmentand sustainment

Include Software in SystemsPerformance Requirements

Prioritized requirements COTS software support requirements Specialty engineering (reliability, maintainability& availability (RMA), security, safety)

Key Performance Parameters (KPP) Open system architecture

Best Practices for Obtaining Contractual InsightRequire Key Software Technical andManagement Deliverables

Highest risk reduction potential is in plans;requirements and architecture; test plans,procedures and reports; metrics reports; delivery,installation and O&M documentation Use electronic delivery

Require Timely Electronic Access toAll Software Products

COTS evaluation trade studies Intermediate and final products (requirements,architecture and design; implementation (including

code); integration and verification testing)Require Software-Level Technicaland Management Reviews

In addition to system reviews Include COTS software experts in reviews

Best Practices for Obtaining Contractual CommitmentMandate Compliance with RobustCommercial Standards

E.g., EIA/IEEE J-STD-016 (commercializedversion of MIL-STD-498) Tailor standard for COTS-based software system

22


23/44

developmentRequire Contractor Commitment toSoftware Development Plan (SDP)

Require SDP to include processes for COTSsoftware Require Integrated Management Plan (IMP) tohave adequate system engineering and sustainment

for COTS Include commitment to SDP in IMPBest Practices for Providing Tools for Contract ManagementIncentivize Software Quality (NotJust Cost/Schedule)

Use award and incentive fee plans Reward adherence to defined software processesand software process improvement Reward timely/acceptable response toGovernment comments Reward low rework rates Reward meeting performance requirements (e.g.,RMA) post delivery/launch

Reward architecture development that supportsCOTS-based software system evolutionMandate Periodic Team SoftwareCapability Appraisals

Relate results and improvement actions directlyto award fees Explicitly include COTS processes in appraisals

Best Practices for Selecting a Capable Software Contractor TeamEvaluate SoftwareCapability/Processes of OfferorTeams

Individual team member evaluation isinsufficient Evaluate software capability as a separatesubfactor under the Mission Capability factor Weight according to software risk

Evaluate Teams Proposed Softwareand Related Processes

Corporate and past project process evaluation isinsufficient Include COTS software, systems engineering andlogistics processes

Evaluate Realism of Cost andSchedule Bids

Be suspicious of productivity extremes, COTS,reuse, low lines of code, and short integrationtimes Ensure that all COTS software tasks are includedin the cost and schedule bid Ensure bid contains sufficient cost and schedulemargin

Evaluate Software and HardwareArchitecture with System Design

Best Practices for Performing Technical Product ReviewFocus Technical Review Resourceson Areas of Highest Risk

IPTs, Technical Interchange Meetings (TIMs),working groups, per reviews, etc. Software-level technical reviews High risk/critical software products (includingCOTS software)

23


24/44

Key software technical deliverablesInclude Users/Operators in AllTechnical Review Activities

Ensure users/operators understand the evolvingdesign, including COTS software capabilities andimpacts on O&M

Monitor Software Integration and

Verification Adequacy

Begin at the build level

Focus on areas of highest risk Focus on early performance analysis results andmeeting KPPs Ensure that COTS software performance ismeasured Ensure requirements allocated to COTS softwareare verified

Best Practices for Performing Software Process ReviewReview Effectiveness of TeamsDefined Software and RelatedProcesses

Identify process deficiencies (especially acrossteam boundaries and with COTS products) Assist with process improvement

Individual Level 2 and 3 CMMI/CMMcompliance may not be sufficient

Perform Periodic Team SoftwareCapability Appraisals

During contract performance Support for significant program or award feemilestones Explicitly include COTS processes

Review Teams Adherence toDefined Software and RelatedProcesses

Identify adherence deficiencies Assist in deficiency correction

Best Practices for Managing the ContractUse Incentive/Award Fees

Aggressively

Motivate good software and related practices

Focus on quality an d architectureApply Proactive QuantitativeManagement

Ensure a comprehensive software/systemsmetrics program balanced across informationcategories (include leading quality indicators, e.g.,rework) Perform cross-metric analysis Earned value alone is insufficient

Ensure Adherence to Software-Inclusive Requirements

Especially RMA, safety and security Especially COTS software supportability

Perform Periodic IndependentAssessments

Support for significant program or award feemilestones Act aggressively on findings

Best Practices That Span the Life-cycleSoftware Acquisition RiskManagement

Continuous software acquisition riskmanagement across all acquisition organizationlevels Program-level risk management and contractordevelopment risk management are necessary, but

24


25/44

not sufficient Establish management reserves consistent withsoftware risks (including/especially COTSsoftware risks)

Software Systems Acquisition Integrate software acquisition with the system

acquisition process (from mission needsidentification through system retirement, especiallyduring pre-contract activities)

Figure 7: Use Quality Function Deployment to Translate Successful Acquisition

3) Phase Containment Matrix:A Phase Containment Matrix is a well-known tool for identifying and reducing softwaredefects across the various phases of a software life-cycle. It is constructed by identifying

the phases during which a defect is inserted as the column headings, and the phaseswhere the defect was detected as the row headings (see Figure 8).

Figure 8: Phase Containment Matrix for Tracking Software Development QualityWithin each cell, the number of defects that are detected at that phase in the software life-cycle are entered. There is a basic, yet critical, assumption that root-cause defect analysishas been performed in order to accurately determine in which phase of the life-cycle thedefect was actually inserted.In-phase defects are often not counted as rework, as these defects are assumed to be anatural part of the process. For example, the rework required to correct the 25 defects

25


26/44

that were introduced during the requirements phase, but were also detected andeliminated in the requirements phase, would not be counted, even though there is stillpotential cost and schedule impact associated with the rework. Ignoring the cost of in-phase rework deludes both the customer and the organization, and deprives theorganization of an opportunity to improve its in-phase processes. Analyzing the cost of

performing rework on 25 in-phase requirements defects might provide significantjustification for improving an acquisition or development process to reduce the numberof introduced defects.

Out-of-phase defects are generally classified as rework and, just as generally, are going tocost more to fix than in-phase defects. The larger the span (in project phases) betweenthe insertion point and the detection point, the more expensive it is going to be, on a perdefect basis, to rework and fix it. This more expensive factor can be an order ofmagnitude of cost between each phase.The concept of a Phase Containment Matrix could, with some adaptation, also be appliedto improve software acquisition processes by defining an appropriate set of processes

within which software acquisition defects can be detected. Figure 9 provides a simpleexample of how this might be accomplished.

26


27/44

ANTICIPATED BENEFITS OF IMPLEMENTATION: of SW and HWSuccessful implementation of software acquisition process improvement can result in:

Realistic, incremental and continuous improvements to software acquisitions Aprocess of evolutionary software acquisition avoids the temptation to succumb tounrealistic expectations in an attempt to force revolutionary changes in softwaredevelopment. Adherence to well-understood, well-defined, manageable requirements Anevolutionary software acquisition process ensures that software development is limited tothings that are possible to manage. Only requirements that are well-defined and well-understood can be adequately managed. Following of disciplined, structured software management and developmentprocesses that provide predictable results Disciplined, structured software managementand development process phases that end in a management review/gate ensure that the

acquisition will remain on track, and that project course corrections can be identified andimplemented effectively. Well-informed, well-disciplined and knowledgeable software acquisition teams The suitably tailored collection and use of process metrics ensures that the softwareacquisition is adequately tracked; that project knowledge is confirmed and shared amongstakeholders; that acquisition risk is effectively managed; and that software acquirers arefirmly integrated into the continuous improvement process.

27


28/44

Increased probability of software acquisition successes and best value awards Heavy reliance on configuration management, risk identification/management, peerreviews and quality assurance, coupled with earned value management techniques, helpsensure a successful and improving acquisition process.

DETAILED CHARACTERISTICSKey Characteristics of the Acquisition Process Improvement Gold PracticeCharacteristic CommentsManageableDevelopmentEnvironment isDefined andImplemented

Business decisions must be made to invest suitabletime/resources in achieving high levels of software process maturity Honest relationships between the acquirer and developer must beestablished There must be a mutual understanding of software requirementsbetween the acquirer and developer in order to optimize

setting/managing requirements There must be a match between available resources andrequirements, supported by systems engineering analysis and trade-off analyses that consider the performance, cost and scheduleimpacts of major changes to software requirements Technical and administrative knowledge must be obtainedearly in the software development process Sufficient requirements knowledge must be demanded by PMsand other stakeholders at key process points In order to ensure that software requirements are appropriatelyset/managed, it is necessary to document that software requirements

are achievable based on systems engineering knowledgeDisciplinedDevelopmentProcesses,Supported by GatedReviews

Processes that meet program needs must be established andadhered to Acquirers should develop a list of tailored systems engineeringdeliverables that include software based on the results of requiredengineering activities at the appropriate stages/phases of systemdevelopment A sufficient amount of software development time should bedevoted to requirements-setting activities Well-written and achievable requirements must be developedand managed/controlled before actual software design begins

Lower-level software design elements should be defined beforeany coding begins The number and timing of requirements changes should beaggressively managed Test plans that are developed should be based on a stable set ofrequirements Ensure that the software design is mature before approvingrelease/production

28


29/44

All production processes must be under control beforeproduction begins Provide for gated reviews of systems engineering baselines onan event-driven basis

Established and

Leveraged Metricsare Defined andUtilized

The collection and reporting of metrics related to software

acquisition and development should be required internally, and fromcontractors, in order to ensure adequate oversight knowledge ofsoftware-intensive acquisitions There should be relentless pursuit of tailored metrics that arederived from effective processes Require software contractors to collect/report cost, schedule,size, requirements, test, defect and quality metrics on a per-monthbasis, if appropriate, and before program milestones Ensure that contractors are using an earned value system thatreports work at a suitably detailed level Software acquisition should be enforced with suitable controls

and performance incentives in DoD acquisition policy, softwareacquisition improvement plans and software development contractsUse of AppropriateTools andTechniques toSupport AcquisitionActivities

Formal processes such as SA-CMM and CMMI-AM provide aframework for implementing a structured software acquisitionimprovement process Tools and techniques to be considered include documenttemplates, documentation standards, checklists, QFD, AHP, PhaseContainment Matrices, earned value tracking, knowledge-sharingsystems, lessons learned repositories, past performance databases,centralized acquisition resources, software tools that support riskmanagement, configuration management, source selection, decision-

making, etc.Use of IntegratedProcess Teams(IPTs) That InvolveAll Stakeholders(including EndUsers)

Integrated Process Teams conduct process analyses and identifybottlenecks and non-value-added process structure and flow, with aconstant focus on measurement/improvement of software acquisitionprocesses. IPTs keep the focus on the customers needs and requirements Effective IPTs contribute to collaborative requirementsdevelopment, an important attribute of the Air Force AgileAcquisition initiative Teams should be cross-functional and multi-disciplinary, andthere must be leadership/senior management commitment to support

the team (with appropriate allocation/delegation of decision-making) Organizational structures should be defined such that teamperformance is suitably rewarded

RELATIONSHIPS TO OTHER PRACTICES:

29


30/44

The Figure below represents a high-level process architecture for the subject practice,depicting relationships among this practice and the nature of the influences on thepractice (describing how other practices might relate to this practice). These relationshipstatements are based on definitions of specific best practices found in the literature andthe notion that the successful implementation of practices may influence (or be

influenced by) the ability to successfully implement other practices. A brief descriptionof these influences is included in the table below.

Process Architecture for the "Acquisition Process Improvement" Gold Practice

30


31/44

Other general criteria for evaluation of SW and HWLen Bass

Feb 2, 2010 2010 Carnegie Mellon University Externally visible properties Relationships among elements

Multiple different structures

Uses Guide to construction Artifact for analysisQuality attributes are properties of a system such as Functionality Reliability Usability Efficient 2010 Carnegie Mellon University Maintainability

Portability

Acquisition perspective

Given the importance of software architecture to software development,there are three portions of the life cycle where architectural knowledgeis important Requirements Proposal evaluation Architecture evaluation of system during developmentRequirements information must include quality attribute requirements

Business/mission goals must be available to proposers/designers

During Development

Acquirers should evaluate architecture during development To ensure designed system satisfies mission goals To ensure conformance To enable adaptation to changes in mission

Evaluating integrity and quality f SW and HW

Requirements analysis

Risk assessment

Development analysis

Code review

31


32/44

Independent testing

Contract verification

Algorithm analysis

Software metrics

32


33/44

Essentials In system Implementation

Adaptive tailoring to the scope of the target program

Integrated working closely with development teams

Self-improving

Essentials in System Implementation

Depending on the unique circumstances of the implementation process, the status of data

compilation, and the organizational climate, increased productivity is normally reached

between the second and fifth year of implementation. This is identified by the threshold

point. Again, this is dependent on a variety of factors including:

the skills and experience of the staff involved;

the priority and commitment by the organization;

33


34/44

the implementation strategy; and

the status of data compilation.

The Implementation Plan

Implementation can be seen as a six phase process. They are:

Creating an awareness GIS needs to besoldwithin an organization. Theeducation of staff is very important. Depending on the way in which GIStechnology is being introduced to the organization the process for creating anawareness may differ. Technical workshops are often appropriate when a top-down approach exists, while management workshops are often more relevantwhen a bottoms-up approach exists. Education of the new technology should

focus on identifying existing problems within an organization. These often helpjustify a GIS acquisition. They include :

spatial information is poorly maintained or out ofdate;

spatial data is not recorded or stored in a standardway;

spatial data may not be defined in a consistentmanner, e.g. different classifications for timberinformation;

data is not shared between departments within anorganization;

data retrieval and manipulation capabilities areinadequate to meet existing needs; and

new demands are made on the organization thatcannot be met with existing information systems.

Identifying System Requirements

The definition of system requirements is usually done in a user needs analysis. A userneeds analysis identifies users of a system and all information products required by those

users. Often a prioritization of the information products and the data requirements of

those products is also undertaken. A proper user needs analysis is crucial to the

successful evaluation of GIS software alternatives.

34


35/44

After user needs have been identified and prioritized they must be translated into

functional requirements. Ideally, the functional requirements definition will result in a set

of processing functions, system capabilities, and hardware requirements, e.g. data

storage, performance. Experienced GIS consultants often play a major role in this phase.

System Evaluations

Evaluating alternative hardware and software solutions is normally conducted in several

stages. Initially a number of candidate systems are identified. Information to support this

process is acquired through demonstrations, vendor literature, etc. A short listing of

candidates normally occurs based on a low level assessment. This followed by a high

level assessment based on the functional requirements identified in the previous phase.

This often results in a rating matrix or template. The assessment should take into account

production priorities and their appropriate functional translation. After systems have beenevaluated based on functional requirements a short list is prepared for those vendors

deemed suitable. A standard benchmark, as discussed earlier, is then used to determine

the system of choice.

Justifying the System Acquisition

The proper justification of the chosen system requires consideration of several factors.

Typically a cost-benefit analysis is undertaken to analyze the expected costs and benefitsof acquiring a system. To proceed further with acquisition the GIS should provide

considerable benefits over expected costs. It is important that the identification of

intangible benefits also be considered.

The justification process should also include an evaluation of other requirements. These

include data base development requirements, e.g. existing data versus new data needs and

associated costs; technological needs, e.g. maintenance, training, and organizational

requirements, e.g. new staff, reclassification of existing job descriptions for those staff

who will use the GIS.

System Acquisition and Start Up

After the system, e.g. hardware, software, and data, is acquired the start up phase begins.

This phase should includepilot projects. Pilot projects are a valuable means of assessing

progress and identifying problems early, before significant resources have been wasted.

35


36/44


37/44

Smith-son, 1988). Further Hirschheim & Smithson means that this can have majornegative conse-quences in terms of decreased user satisfaction but also broaderorganizational consequences in terms of system value.

We agree with the criticism of Hirschheim & Smithson, but when analysing goal-based

evaluation in an ideal typical way there is no imperative relation be-tween a focus ontechnical and economical aspects and goal-based evaluation. Of course, the stated goalscan be of a human or organisational character. However, the traditional way ofunderstanding goal-based evaluation is often related to harder measurable goals.

Further, there is no imperative relation between a goal-based approach, and a quantitativeprocess. A judgement of, if the goals have been fulfilled can be evaluated with aqualitative process. As we see it, the differences between a quantitative and qualitativestrategy is that the quantitative strategy aims to decide if the goals are fulfilled and whichgoals that are ful-filled. The fulfilment of the goals will be expressed in quantitative

numbers.

There are also goals of more social or human character. The fulfilment of this types ofgoals is preferably expressed in qualitative terms. The qualitative process has also,besides the if- and which questions, a better possibility to describe how the goals arefulfilled. This means that the qualitative approach aims at achieving richer descriptions.

The goals that are used for evalua-tion are derived from an organisational context. Thatmeans that they are situationally appli-cable, which means that they act like specificbusiness goals.

The basic strategy of this approach is to measure if predefined goals are fulfilled or not;to what extent and in what ways. The approach is deductive. What is measured dependson the

37


38/44

character of the goals and a quantitative approach as well as qualitative approach couldbe used. In this paper we adopt the concept of goal-based evaluation from Patton (1990)in order to identify this approach.

2.2 Goal-free evaluation

The second identified approach is a more interpretative approach (e.g. Remenyi, 1999;Wal-sham, 1993). The interpretative perspective views IT-systems as social systems thathave in-formation technology embedded into it (Goldkuhl & Lyytinen, 1982). The aim ofinterpretive evaluation is to gain a deeper understanding of the nature of what is to beevaluated and to generate motivation and commitment (Hirschheim & Smithson, 1988).

The involvement of a wide range of stakeholder groups is essential to this approach ofevaluation. This can also be a practical obstacle where time or resources for theevaluation are short. Patton (1990) uses the term goal-free evaluation. Goal-freeevaluation is defined as gathering data on a broad array of actual effects and evaluating

the importance of these effects in meeting demonstrated needs (Patton, 1990, Scriven,1972). The evaluator makes a deliberate attempt to avoid all rhetoric related to programgoals; no discussion about goals is held with staff; no program brochures or proposals areread; only the programs outcomes and measurable effects are studied. The aim of goal-free evaluation is to (Patton, 1990):

1) avoid the risk of narrowly studying stated program objectives and thereby missingimportant unanticipated outcomes

2) remove the negative connotations attached to discovery of unanticipated effect:The hole language of side-effected or secondary effect or even unanticipatedeffect tended to be a put-down of what might well be a crucial achievement,especially in terms of new priorities.

3) eliminate the perceptual biases introduced into an evaluation by knowledge ofgoals; and

4) maintain evaluator objectivity and independence through goal-free conditions

2.3 Criteria-based evaluation

The third identified approach is a criteria-based approach. There are lot of criteria-basedap-proaches around such as checklists, heuristics, principles or quality ideals. In the areaof Human-Computer Interaction you can find different checklists or heuristics (e.g.Nielsen, 1994; Nielsen, 1993, Shneiderman, 1998). What is typical for these approachesis that the IT-systems interface and/or the interaction between users and IT-systems actsas a basis for the evaluation together with a set of predefined criteria. More actionoriented quality ideals and principles for evaluation can be found in Cronholm &

38


39/44

Goldkuhl (2002) and in gerfalk et al (2002). The basis for these action oriented ideals isto understand if and how the IT-system support the actions performed in the business (seediscussion of IT-systems in section 3.1)

The criteria used are grounded in and derived from one or more specific perspectives ortheo-ries. For example, the criteria in Nielsens (1994) checklist are derived fromcognitive and computer science. The action oriented ideals are mainly derived fromlanguage action theory but also inspired by usability issues. Using criteria means to setfocus on certain qualities that according to the perspective is important to evaluate. At thesame time the attention accord-

39


40/44

ing to the criteria also de-emphasize other qualities. The criteria chosen governs theevalua-tors attention and thereby the kind of knowledge the evaluator achieves.

Another difference in comparison to goal-based evaluation is that the criteria thatare used are not derived from a specific organisational context. That means that they aremore general ap-plicable (see section 2.1). Ideal typically, the basic strategy of criteria-

based evaluation is pure deductive. The word criteria is often used in relation topreordinate designs, and the use of this term has a hard scientific feel which supportsthe tendency to prioritize technical and quantitative data

40


41/44

Different Units topics

a) MIS Approach

MIS is generally used by medium and larger scale organizations. However, smallorganizations are yet to understand its application. There is dire need to build upcomputer culture by properly disseminating information about computer applications andits benefits.

Implementation of MIS can be achieved by using any of the methods such as direct,parallel, modular or phase in.

Direct Approach

Direct installation of the new system with immediate discontinuance of the old existingsystem is reffered as cold turnkey approach. This approach becomes useful when thesefactors are considered.

1. The new system does no replace the existing system.2. Old system is regarded absolutely of no value3. New system is compact and simple.4. The design of the new system is inexpensive with more advantages and less riskinvolved.

Parallel Approach

The selected new system is installed and operated with current system. This method isexpensive because of duplicating facilities and personal to maintain both the systems. Inthis approach a target date must be fixed when the operations of old system cease andnew one will operate on its own.

Modular Approach

This is generally recognized as Pilot approach, means the implementation of a systemin the Organization on a piece-meal basis.

This has few advantages / merits

1. The risk of systems failure is localized2. The major problem can be easily identified and corrected before furtherimplementation.3. Operating personal can be trained before system is installed in a location.

41


42/44

Phase-in-Implementation

This approach is similar to modular method but it differs because of segmentation ofsystem, however, not the organization. It has advantages that the rate of changes in agiven Organization can be totally minimized and the data processing resource can be

acquired gradually over a period of time. System exhibits certain disadvantages such aslimited applicability, more costs incurred to develop interface with old system and afeeling in the Organization that system is never completed.

b) Implementation Procedures

Planning the Implementation

After designing the MIS it is essential that the organization should plan carefully forimplementation. The planning stage should invariably include the following:

1. Identification of tasks of Implementation

Planning the implementation activities, acquisition of facilities, procedures development,generating files and forms, testing the system and evaluating and maintenance of thesystem.

2. Relationship establishment among the activity

Network diagram must be prepared to correlate concurrent and sequential activities.

3. Establishing of MIS

For monitoring the progress of implementation and for proper control of activities,efficient information system should be developed.

4. Acquisition of Facilities

For installation of new system or to replace current system the manager should prepare aproposal for approval from the management by considering space requirement movementof personal and location for utility outlets and controls.

5. Procedure Development

This is an important stop for implementation of the system including various activitiessuch as evaluation selection of hardware, purchase or development of software, testing

42


43/44

and implementation strategies.

6. Generating Files and Forms

The MIS manager should generate files and formats for storing actual date. This requires

checklist data, format date storage forms and other remarks in data base.

7. Testing of the System

Test should be performed in accordance with the specifications at the implementationstage consisting of component test sub system test and total system test. Evaluation and maintenance of system

The performance should e evaluated in order to find out cost effectiveness and efficacy ofthe system with minimum errors due to designs environmental changes or services.

c) Software Maintenances

The proper maintenance is the enigma of the system development and it holds softwareindustry captive lying up programming resources. There are some problems inmaintenance such as regarding it as non rewarding non availability of technicians andtools no cognizance of users about maintenance problem and cost lack of standardprocedures and guidelines. Most programmers feel maintenance as low level drudgery. Ifproper attentions is paid over a period of time eventually less maintenance is required.

Types of Maintenance

The maintenance of system are classified into corrective/adaptive/perfective. Correctivemaintenance means repairing process or performance failures. Adaptive maintenancemeans changing the programming function whereas perfective maintenance deals withenhancing the performance or modifying the program.

Primary Activities of a Maintenance Procedure

Documentation is major part of maintenance in system development. Maintenance staffreceives requests from the authorized user. Programming library should be maintained.

Reduction in Maintenance Costs

Several organizations having MIS generally go in for reducing maintenance costs and itconsists of three major phases.

1. Maintenance management audit through questionnaires and interviews.2. Software system audit.3. Software modification.

43


44/44

Evaluation Methods

Evaluation of the MIS in an organization is integral part of the control processes. Thereare several evaluation approaches such as quality assurance review compliance of auditsbudget performance review computer personal productivity assessment computer

performance evaluation service level monitoring user audit survey post installationreview and cost benefit analysis.

Evaluation performance measurement can be classified into two classes as effectivenessand efficiency. The relationship between effectiveness and efficiency is that the format isa measure of goodness of out put and the latter is a measure of the resources required toachieve the output.

UNit-vii MIS

Documents

Transcript of UNit-vii MIS