New “MIERUKA Visualization - IPA · 2019. 10. 25. · 2 Appendix MIERUKA (Visualization) Tools...

“MIERUKA (Visualization)” of IT Projects

Downstream Process

AuthorEditor

Software Engineering CenterInformation-Technology Promotion Agency, Japan (IPA)

1

1. Management Forms

Management forms are widely used as the most basic tool for understanding the project status. Organizations use various forms that differ in name and data items, such as review records, problem description forms, bug report forms, and work progress management charts.

The data items on these forms and the items on the management charts used for report management can be used to uncover underlying problems in the project, and for successful project management.

2. Check Sheets

Several dozen check items derived from the experiences of skilled project managers were compiled into check sheets for qualitatively grasping whether a project is facing issues. There are two kinds of check sheets; “Self-Check Sheets” for self-diagnosis by project managers, and “Interview Sheets” for interview diagnosis by expert teams (i.e. PMO). Refer to page 6 for a sample of a self-check sheet, and page 20 for a sample of an interview sheet.

3. Case Study of Problems and Solutions (Summary of Problem Projects)

Development experts were interviewed to compile a Problem Projects Database of experienced issues, their causes, and their solutions. If the Project managers can respond appropriately to various problems that occur during a project by learning the specifics of failures and their solutions. Such information are particularly useful for less experienced project managers. The interviews were conducted with experts who have either 10 or more years of experience as project managers, or have 10 or more years of experience troubleshooting in the field of quality assurance. For details of the problem projects database, refer to page 60.

4. Measurement Items List

For a quantitative MIERUKA (visualization) of the project status, the required measurement items

MIERUKA (Visualization) Tools and References

Appen

dix

2

MIERUKA (Visualization) Tools and ReferencesApp

endix

must be determined. In determining the measurement items, it is necessary to clearly define the objectives of the measurements, and provide a comprehensive and structured measurement items list to ensure no measurement is missing. A systematic “Measurement Items List” has been compiled based on expert knowledge and available reference materials. For details, refer to page 174 (J).

5. Analytical Indicators of EPM Tool

Measurement Items List contains a number of measurement items, some of which can be automatically measured and graphed using project monitoring tools such as EPM (Empirical Project Monitor) developed under the EASE (Empirical Approach to Software Engineering) project. Refer to page 192 (J) for a list of items that can be measured and analyzed using EPM.

6. Case Classification Table

Past failed projects were analyzed by focusing on the “point of occurrence” and “state” of problems, which were then organized into a “Case Classification Table.” The table is structured as a classification matrix with the problem’s “point of occurrence” on the vertical axis and the “state” on the horizontal axis.

There are four kinds of case classification tables. “Case Classification Table (General Map)” shown on page 198 (J) lists all case patterns possible as a combination of the “point of occurrence” on the vertical axis and the “state” on the horizontal axis. “Case Classification Table (Interview Map),” “Case Classification Table (Measurement Items Map),” and “Case Classification Table (Case Study Map)” have the same structure for the vertical and horizontal axes, but are each associated with the data items for interview sheet, measurement items list, and problem projects database, respectively.

Items in the “Case Classification Table (Interview Map)” are linked to the check item numbers on the Interview Sheet (see page 202 (J)). Items in the “Case Classification Table (Measurement Items Map)” are associated to the measurement item numbers in the Measurement Items List (see page 206 (J)). Items in the “Case Classification Table (Case Study Map)” are associated to the case numbers (see page 208 (J)).

7. Knowledge Areas

In this document, six knowledge areas have been defined in addition to the nine PMBOK knowledge areas, for a total of 15 knowledge areas. The six added areas are “Customer,” “Organization,” “Basic Conduct/Action,” and “Motivation” from the humanware perspective, “Technology” which is a management area specific to IT projects and hence not included in PMBOK, and “Issue Management”

3

“MIERUKA (Visualization)” of IT ProjectsDownstream Process

from the perspective of monitor/control process in the downstream process of a project.The six additional knowledge areas are described in Figure A.

Figure A: Definition of the additional knowledge areas

Additional Knowledge Area Third Layer Definition

Customer

A person or an organization who is a project stakeholder with the authority of making the final decision on system specifications and budget. In the case of a contract-based IT project, the customer and other stakeholders will often work together towards building a consensus for providing the system which is the final deliverable.

Organization

An organization engaged in the system development. Some IT projects adopt a multi-level contractor model or a multi-vendor model in which multiple development companies are involved. In reality, the development organizational structure may impose various constraints on “Human Resources” and “Procurement.”

Basic Conduct/Action Commonplace knowledge and behaviors expected of a system developer. These include system development management skills.

Motivation

Enthusiasm and motivation of personnel engaged in the system development. It also includes a wide range of career development elements such as mentality, working environment, and personal development goals of the personnel engaged in the system development.

TechnologyManagement items related to system building technology. It covers aspects of system development management that are specific to the industry and not covered by PMBOKTM.

Issue Management

Management items related to issue management in IT projects. It corresponds to the management of Monitoring and Controlling Process in PMBOKTM. Since this document is focused on the downstream process, this knowledge area has been added to the expansion area as an area of particular importance. It deals with how the issue management should be performed in the system development site.

4

8. SEC Activities and “Project MIERUKA Committee of IPA / SEC”

The function of IPA / SEC’s “Project MIERUKA committee” is to provide overall assistance in the project development process amidst the series of SEC enterprise activities (See Figure B). The committee was established in 2005. In the first year, 2005, the committee developed a MIERUKA (visualization) methodology which focuses on the downstream process. In the following years, the committee developed MIERUKA methodologies for the upstream and midstream processes.

Figure B: SEC enterprise activities

Development Process, Process Improvement

Quantitative Data Analysis, Estimation Model / Methodology

MIERUKA (Visualization) of the Project

Systematization Strategy

Systematization Plan

Requirements Definition

System Design

Software Design

Are the requirements satisfied?

Has the investment effect improved?

Are the specifications met?Design /

Development Technology

EstimationRequirements

Engineering

Development Process Sharing in the "Super-Upstream"

Indi

vidu

al P

roce

sses

Ove

rall

Supp

ort

Business Evaluation

Management Evaluation

System Test

Operation Test

Software Test

Programming

MIERUKA (Visualization) Tools and ReferencesApp

endix

5


6

2. Check Sheet (Self-Check Sheet)App

endix

No. Knowledge Area Check Item Assessment Criteria Management Tips Measures No.

S1 Integration Has the project plan been created and reviewed?

The project plan shall have been created within a specified period from the initiation project, and shall have been reviewed.

Set a time limit if one has not been specified. If the project is significantly diverted from the plan, the plan is essentially non-existent. Even if time is limited, a project plan must be created or updated, and agreed upon by the project stakeholders.

S1

S2 Integration Are the delivery items, required work, and deliverables clearly identified as the target of configuration management?

A clear image of the deliverables must be shared with the customer.

If you have not clarified the image of the deliverables with the customer, you might make critical errors in judgment when assigning tasks or estimating the scale for later stages.

If the target of configuration management is not clear, identify them clearly and develop configuration management rules.

S2

S3 Integration Have all work items been scheduled? Each work item shall have been listed on the schedule. Clarification of the critical paths will allow for intensive monitoring of time-critical tasks.

The following are milestones in project operation:- Test process- Operation training- Migration- Production release- Attendance plan- ClosingAnd the following are customer-driven events:- Power outage of the building- Important meetings- Integration tests with other vendorsMilestones and events such as these should be grasped through interviews to clarify critical paths.

S3

S4 Integration Are unexpected tasks occurring frequently? Unplanned tasks not included in the schedule must not exist. Frequent changes in the plan will result in a confusion in the project. If there is confusion in the project, the project plan will require substantial review and deliberation.

- The task items must be controlled in considering work efficiency.- All stakeholders including the customer, partners, and the project manager should participate in an interview about unplanned tasks to prepare a task items list.

- Based on the list, they need to discuss the task priorities and sequence, sort out the tasks, and deliberate the overall efficiency.

S4

S5 Integration Are important and mission-critical functions presented and managed?

The priorities of important functions and their points of notice shall have been stated.

Since mission-critical functions must be implemented without failure, they must be carefully managed, performed, and monitored continuously from the stage of planning and design.

- Clearly identify critical paths.- Based on the clarification, intensively manage, test, and review poor quality programs.

- In particular, require a daily progress report for strict control of the critical paths.- Involve skilled engineers.

S5

S6 Integration Have the overall strategy and role division for tests sufficiently considered?

A document describing the overall strategy for tests (what to test in which test) and the role division in each test shall have been formulated and reviewed with stakeholders.

If the examination is not sufficient, the overall test policy (what to test in which test) should be organized based on past case studies, and the role assignment among the stakeholders should be determined.

- If schedules and quality targets are not planned respectively for the unit tests, integration tests, and qualification tests, prepare and plan them based on past cases.

- Clearly identify the test framework.- In particular, it is important to assign highly capable personnel (such as key personnel or highly capable engineers) for the integration tests and subsequent tests to avoid delay.

S6

7



S1 Integration Has the project plan been created and reviewed?

The project plan shall have been created within a specified period from the initiation project, and shall have been reviewed.

Set a time limit if one has not been specified. If the project is significantly diverted from the plan, the plan is essentially non-existent. Even if time is limited, a project plan must be created or updated, and agreed upon by the project stakeholders.

S1

S2 Integration Are the delivery items, required work, and deliverables clearly identified as the target of configuration management?

A clear image of the deliverables must be shared with the customer.

If you have not clarified the image of the deliverables with the customer, you might make critical errors in judgment when assigning tasks or estimating the scale for later stages.

If the target of configuration management is not clear, identify them clearly and develop configuration management rules.

S2

S3 Integration Have all work items been scheduled? Each work item shall have been listed on the schedule. Clarification of the critical paths will allow for intensive monitoring of time-critical tasks.

The following are milestones in project operation:- Test process- Operation training- Migration- Production release- Attendance plan- ClosingAnd the following are customer-driven events:- Power outage of the building- Important meetings- Integration tests with other vendorsMilestones and events such as these should be grasped through interviews to clarify critical paths.

S3

S4 Integration Are unexpected tasks occurring frequently? Unplanned tasks not included in the schedule must not exist. Frequent changes in the plan will result in a confusion in the project. If there is confusion in the project, the project plan will require substantial review and deliberation.

- The task items must be controlled in considering work efficiency.- All stakeholders including the customer, partners, and the project manager should participate in an interview about unplanned tasks to prepare a task items list.

- Based on the list, they need to discuss the task priorities and sequence, sort out the tasks, and deliberate the overall efficiency.

S4

S5 Integration Are important and mission-critical functions presented and managed?

The priorities of important functions and their points of notice shall have been stated.

Since mission-critical functions must be implemented without failure, they must be carefully managed, performed, and monitored continuously from the stage of planning and design.

- Clearly identify critical paths.- Based on the clarification, intensively manage, test, and review poor quality programs.

- In particular, require a daily progress report for strict control of the critical paths.- Involve skilled engineers.

S5

S6 Integration Have the overall strategy and role division for tests sufficiently considered?

A document describing the overall strategy for tests (what to test in which test) and the role division in each test shall have been formulated and reviewed with stakeholders.

If the examination is not sufficient, the overall test policy (what to test in which test) should be organized based on past case studies, and the role assignment among the stakeholders should be determined.

- If schedules and quality targets are not planned respectively for the unit tests, integration tests, and qualification tests, prepare and plan them based on past cases.

- Clearly identify the test framework.- In particular, it is important to assign highly capable personnel (such as key personnel or highly capable engineers) for the integration tests and subsequent tests to avoid delay.

S6

8


endix


S7 Integration Is the work progress checked periodically? For example, the progress of the following items shall be checked:- The number of changes in a plan- The number and type of reviews performed- Difference between planned and resulting schedules- Planned and actual number of coding steps- Code inspection coverage rate- The number of risks (defined, monitored, decreased)

The work progress should be verified by checking the actual status, and not by relying solely on the partner’s reports.

- Investigate the reason for regular meetings not being held. - Understanding the necessity - Reserving the time - Members from more than one company - The issue of venue- Arranging a regular progress meeting. - Determine the frequency depending on the duration and status of the project - Involve persons in charge, not stakeholders - Organize a hierarchical structure- Provide an alternate measure if a regular meeting is not possible. - Move the time (if needs be, to a later time) or shorten the meeting if the

participants are busy - Utilize TV conference or telephone conference to resolve geographical issues

S7

S8 Integration Has configuration management been conducted?

Changes to the configuration management target shall be recorded and reported in accordance with the documented procedures and rules of configuration management.

Confirm whether configuration management is implemented so that the actual management activities are recorded and reported in accordance with the procedures and rules.

- If there are configuration management rules, strictly abide by the rules.- If no configuration management is in place, investigate the reason. - Is the necessity of configuration management understood? - Imagine what may happen If configuration management is not in place. - Understand the substantial meaning of configuration management, not its literal

meaning.- Designate a responsible person (preferably someone who is strict).- Deliberate the activities and tools for configuration management.- Strictly comply with the original documents and rules.

S8

S9 Scope Have the functions and performance of the system been verified that they meet the customer’s requirements?

The developer shall fully understand the customer’s requirements. A significant deviation from the customer’s requirements is a problem.

- If there is a requirements definition, compare the current status with the definition, and if there are areas where there are significant deviations, report them and share these information with the customer.

- If some requirements are not described in the requirements definition, promptly identify them. If they are critical requirements, consider their implementation from the perspectives of function and cost.

- If requirement change requests are frequently raised for a specific function, suspend the development process and discuss the specifications again to finalize it.

S9

S10 Scope Is the relationship between requirements definition and specification change request being clarified?

Correspondence between the customer’s needs (requirements) and the items in the requirements definition shall be identified clearly.

Manage the correspondence between requirement definitions and specification changes to ensure their consistency with customer requirements.

- Clarify the correspondence between the requirements definition and the specification changes.

- Prepare a list of correspondence between the requirements definition and the specifications, and manage the requirements definition changes and specification changes which influence each other.

S10

S11 Scope Are specification changes managed as a list and their change history recorded?

Specification changes shall be managed as a list such as a specification change management sheet.

Confirm the following:- The location (storage location) of the specification change management sheet

- Personnel responsible for making entries in the specification change management table and its storage

- Consistency between the specifiation change history and the change management sheet

- Prepare a specification change list which shows the items corresponding to the requirements definition.

S11

9



S7 Integration Is the work progress checked periodically? For example, the progress of the following items shall be checked:- The number of changes in a plan- The number and type of reviews performed- Difference between planned and resulting schedules- Planned and actual number of coding steps- Code inspection coverage rate- The number of risks (defined, monitored, decreased)

The work progress should be verified by checking the actual status, and not by relying solely on the partner’s reports.

- Investigate the reason for regular meetings not being held. - Understanding the necessity - Reserving the time - Members from more than one company - The issue of venue- Arranging a regular progress meeting. - Determine the frequency depending on the duration and status of the project - Involve persons in charge, not stakeholders - Organize a hierarchical structure- Provide an alternate measure if a regular meeting is not possible. - Move the time (if needs be, to a later time) or shorten the meeting if the

participants are busy - Utilize TV conference or telephone conference to resolve geographical issues

S7

S8 Integration Has configuration management been conducted?

Changes to the configuration management target shall be recorded and reported in accordance with the documented procedures and rules of configuration management.

Confirm whether configuration management is implemented so that the actual management activities are recorded and reported in accordance with the procedures and rules.

- If there are configuration management rules, strictly abide by the rules.- If no configuration management is in place, investigate the reason. - Is the necessity of configuration management understood? - Imagine what may happen If configuration management is not in place. - Understand the substantial meaning of configuration management, not its literal

meaning.- Designate a responsible person (preferably someone who is strict).- Deliberate the activities and tools for configuration management.- Strictly comply with the original documents and rules.

S8

S9 Scope Have the functions and performance of the system been verified that they meet the customer’s requirements?

The developer shall fully understand the customer’s requirements. A significant deviation from the customer’s requirements is a problem.

- If there is a requirements definition, compare the current status with the definition, and if there are areas where there are significant deviations, report them and share these information with the customer.

- If some requirements are not described in the requirements definition, promptly identify them. If they are critical requirements, consider their implementation from the perspectives of function and cost.

- If requirement change requests are frequently raised for a specific function, suspend the development process and discuss the specifications again to finalize it.

S9

S10 Scope Is the relationship between requirements definition and specification change request being clarified?

Correspondence between the customer’s needs (requirements) and the items in the requirements definition shall be identified clearly.

Manage the correspondence between requirement definitions and specification changes to ensure their consistency with customer requirements.

- Clarify the correspondence between the requirements definition and the specification changes.

- Prepare a list of correspondence between the requirements definition and the specifications, and manage the requirements definition changes and specification changes which influence each other.

S10

S11 Scope Are specification changes managed as a list and their change history recorded?

Specification changes shall be managed as a list such as a specification change management sheet.

Confirm the following:- The location (storage location) of the specification change management sheet

- Personnel responsible for making entries in the specification change management table and its storage

- Consistency between the specifiation change history and the change management sheet

- Prepare a specification change list which shows the items corresponding to the requirements definition.

S11

10


endix


S12 Scope Has the feasibility of the requirements defined in the specifications been verified?

The feasibility shall have been verified from both technical and performance standpoints.

It is important to consider feasibility at the planning or design stage. In order to determine that a requirement is feasible, you must recognize the risks in cost, function, delivery etc. and identify countermeasures against them.

- If feasibility has not been considered yet, consider it at an early stage.- If most of the implementation has been completed in the downstream process, the feasibility may already be fully clarified. However, a list of required items showing their feasibility should still be prepared to grasp the overall picture.

- If a function with low feasibility is already under testing, pay particular attention to check that it is free of quality issues.

S12

S13 Time Has the previous process been completed and approved within the project (by the project manager)?

If some issues were not resolved in the previous process, paths to resolving the issues shall be clarified before they are passed on to the next process.

Poor quality in the upstream process may bring about larger adverse effects in downstream processes.

- Check the quality and if there are any quality problems, execute measures to improve quality.

S13

S14 Time Has the previous process completed and passed the verification by the quality assurance department?

If approval has been provided with conditions or if improvements to be considered have been pointed out, they shall be subject to issue management.

Confirm the following:- Have the criteria for ending the process been defined?

- Have all tasks been completed in each process?

If any pending tasks exist, they must be clarified and solved in the next process.


S14

S15 Time Has the test plan been reviewed by the stakeholders including the implementation manager (project manager or project leader) ?

The validity of the plan shall have been examined. Some tests may require customer approval. - Ensure that the project leader and the partners share the same understanding of the test plan. If any discrepancy is found, extract the problems and review the test plan.

S15

S16 Time Has critical path been clearly identified? Path in the plan relating to core project functions or deliverables shall have been identified and managed.

Tasks on a critical path that have direct consequences on delays in the schedule must be consciously checked more intensively than others.

- Get key personnel together to extract critical path. Then, assign extra resources for tasks related to critical path.

S16

S17 Time Are the schedules for supplied materials from other stakeholders being kept?

Status of schedules for deliverables shall be continuously watched, and measures shall have been created in advance depending on the impact if a schedule is not kept.

Managing the roles of other stakeholders and their schedules for deliverables separately will help prevent project delays due to ambiguous role division.


S17

S18 Cost Is planned versus actual outcome managed appropriately?

A balance management plan shall have been created in advance (understanding weekly / monthly actual performance).

- Term (start / end of process)- Heaping personnel- WBSConduct planning and performance management on the above items.

- Collect and organize performance information, and review future plans.- If it is difficult to collect performance information for all the work, collect and evaluate performance information at least for critical path.

S18

S19 Cost Are additional costs incurred? Required business / knowledge areas and the skills of personnel shall have been compared, and assignment of consultants or specialists shall have been considered for business / knowledge areas where skills are insufficient.

If a problem occurs in the system infrastructure, assigning specialists is effective. Furthermore, if there is insufficient knowledge relating to special tasks, it is necessary to consider commissioning personnel from an outside specialty company.

- Plan to allocate the most capable specialists. S19

11



S12 Scope Has the feasibility of the requirements defined in the specifications been verified?

The feasibility shall have been verified from both technical and performance standpoints.

It is important to consider feasibility at the planning or design stage. In order to determine that a requirement is feasible, you must recognize the risks in cost, function, delivery etc. and identify countermeasures against them.

- If feasibility has not been considered yet, consider it at an early stage.- If most of the implementation has been completed in the downstream process, the feasibility may already be fully clarified. However, a list of required items showing their feasibility should still be prepared to grasp the overall picture.

- If a function with low feasibility is already under testing, pay particular attention to check that it is free of quality issues.

S12

S13 Time Has the previous process been completed and approved within the project (by the project manager)?

If some issues were not resolved in the previous process, paths to resolving the issues shall be clarified before they are passed on to the next process.

Poor quality in the upstream process may bring about larger adverse effects in downstream processes.


S13

S14 Time Has the previous process completed and passed the verification by the quality assurance department?

If approval has been provided with conditions or if improvements to be considered have been pointed out, they shall be subject to issue management.

Confirm the following:- Have the criteria for ending the process been defined?

- Have all tasks been completed in each process?

If any pending tasks exist, they must be clarified and solved in the next process.


S14

S15 Time Has the test plan been reviewed by the stakeholders including the implementation manager (project manager or project leader) ?

The validity of the plan shall have been examined. Some tests may require customer approval. - Ensure that the project leader and the partners share the same understanding of the test plan. If any discrepancy is found, extract the problems and review the test plan.

S15

S16 Time Has critical path been clearly identified? Path in the plan relating to core project functions or deliverables shall have been identified and managed.

Tasks on a critical path that have direct consequences on delays in the schedule must be consciously checked more intensively than others.

- Get key personnel together to extract critical path. Then, assign extra resources for tasks related to critical path.

S16

S17 Time Are the schedules for supplied materials from other stakeholders being kept?

Status of schedules for deliverables shall be continuously watched, and measures shall have been created in advance depending on the impact if a schedule is not kept.

Managing the roles of other stakeholders and their schedules for deliverables separately will help prevent project delays due to ambiguous role division.


S17

S18 Cost Is planned versus actual outcome managed appropriately?

A balance management plan shall have been created in advance (understanding weekly / monthly actual performance).

- Term (start / end of process)- Heaping personnel- WBSConduct planning and performance management on the above items.

- Collect and organize performance information, and review future plans.- If it is difficult to collect performance information for all the work, collect and evaluate performance information at least for critical path.

S18

S19 Cost Are additional costs incurred? Required business / knowledge areas and the skills of personnel shall have been compared, and assignment of consultants or specialists shall have been considered for business / knowledge areas where skills are insufficient.

If a problem occurs in the system infrastructure, assigning specialists is effective. Furthermore, if there is insufficient knowledge relating to special tasks, it is necessary to consider commissioning personnel from an outside specialty company.

- Plan to allocate the most capable specialists. S19

12


endix


S20 Quality Have mechanisms for entering daily quantification numbers including number of test items completed, number of defects detected, and number of defects resolved been implemented, and managed using bug curves?

A framework for managing the test progress and results shall have been implemented.

- The following report forms shall be in place (Check that they actually exist):

- Completed bug report forms. - Bug curve to the current date (preferable). - Test execution procedures. - Test execution and results management

report forms.- The data aggregation mechanism and the responsible personnel must be clearly identified.

(Using dedicated tools / using tools such as Excel / manual aggregation)

- A meeting shall be held every morning (after the integration tests).

- Prepare a test results management tool (Excel spreadsheet, DB, etc.) as soon as possible if one is not already in place.

- If the test process has already been started and tests are being conducted discretely, integrate the test results management process to collect / control the results from the completed tests.

S20

S21 Human Resources

Have the stakeholders been determined? Proper stakeholder analysis shall have been performed and contact points for negotiations shall have been clearly identified.

Clearly identify the key personnel of the project. For this purpose, a stakeholder diagram is necessary. In order to take actions in the most effective way, it is important to clearly identify not only the names of the stakeholders, but also the budget owner and decision maker of the specifications.

- Perform a stakeholder analysis to clearly identify key personnel. It is important to identify not only the pro forma key personnel but also the de facto key personnel.

S21

S22 Human Resources

Has allocation of personnel (in terms of quantity) been arranged for?

Allocation of personnel shall have been planned so that personnel can be increased in a timely manner (for example, when the project moves to the next process).

Each person must be assigned to a task with clear start and end dates.If the number of necessary personnel changes due to reasons such as specification changes in the previous process, it will be necessary to arrange for additional personnel without waiting for the process to complete.

- Perform personnel transition analysis instead of heaping personnel, and clarify excesses or deficiencies of personnel with required skills for each WBS.

- It is important for project managers to focus on staff allocation for the next process before the end of the previous process.

- Perform personnel allocation early and slightly overstaff.

S22

S23 Human Resources

Have key personnel with required business knowledge been acquired?

Personnel with experience in a particular business operation shall have been acquired.

In searching for a person with “experience in a particular business operation,” it is necessary to clearly define the required level of experience and knowledge, instead of vague criteria.

- Specifically define the required level of experience and knowledge.- Then, check whether a key person satisfying the requirements is already acquired.- If such resource is available within the company, negotiate for participation in the project, at least by means of consultation or review, if further involvement is difficult.

- If no such in-house resource is available: - Obtain the resource from a partner. - Appoint or employ personnel with good understanding. - Withdraw from the project (considering tradeoff between failure cost and

penalties / loss of credibility).- Personnel lacking leadership or who are shy in communicating are not eligible as key personnel even if they have the required experience and knowledge.

S23

13



S20 Quality Have mechanisms for entering daily quantification numbers including number of test items completed, number of defects detected, and number of defects resolved been implemented, and managed using bug curves?

A framework for managing the test progress and results shall have been implemented.

- The following report forms shall be in place (Check that they actually exist):

- Completed bug report forms. - Bug curve to the current date (preferable). - Test execution procedures. - Test execution and results management

report forms.- The data aggregation mechanism and the responsible personnel must be clearly identified.

(Using dedicated tools / using tools such as Excel / manual aggregation)

- A meeting shall be held every morning (after the integration tests).

- Prepare a test results management tool (Excel spreadsheet, DB, etc.) as soon as possible if one is not already in place.

- If the test process has already been started and tests are being conducted discretely, integrate the test results management process to collect / control the results from the completed tests.

S20

S21 Human Resources

Have the stakeholders been determined? Proper stakeholder analysis shall have been performed and contact points for negotiations shall have been clearly identified.

Clearly identify the key personnel of the project. For this purpose, a stakeholder diagram is necessary. In order to take actions in the most effective way, it is important to clearly identify not only the names of the stakeholders, but also the budget owner and decision maker of the specifications.

- Perform a stakeholder analysis to clearly identify key personnel. It is important to identify not only the pro forma key personnel but also the de facto key personnel.

S21

S22 Human Resources

Has allocation of personnel (in terms of quantity) been arranged for?

Allocation of personnel shall have been planned so that personnel can be increased in a timely manner (for example, when the project moves to the next process).

Each person must be assigned to a task with clear start and end dates.If the number of necessary personnel changes due to reasons such as specification changes in the previous process, it will be necessary to arrange for additional personnel without waiting for the process to complete.

- Perform personnel transition analysis instead of heaping personnel, and clarify excesses or deficiencies of personnel with required skills for each WBS.

- It is important for project managers to focus on staff allocation for the next process before the end of the previous process.

- Perform personnel allocation early and slightly overstaff.

S22

S23 Human Resources

Have key personnel with required business knowledge been acquired?

Personnel with experience in a particular business operation shall have been acquired.

In searching for a person with “experience in a particular business operation,” it is necessary to clearly define the required level of experience and knowledge, instead of vague criteria.

- Specifically define the required level of experience and knowledge.- Then, check whether a key person satisfying the requirements is already acquired.- If such resource is available within the company, negotiate for participation in the project, at least by means of consultation or review, if further involvement is difficult.


penalties / loss of credibility).- Personnel lacking leadership or who are shy in communicating are not eligible as key personnel even if they have the required experience and knowledge.

S23

14


endix


S24 Human Resources

Have key personnel with the required technical skills been acquired?

If a package software is utilized in the project, personnel who can perform a fit and gap analysis or has experience with the software (database, middleware, tools) shall be assigned as key personnel.

The core technology of the target project must be understood (the required technical skills vary from one project to another).The experience and skill level pertaining the technology of the key personnel must be verified. (Note that the appraisal will depend on the technical level of the interviewer. An interviewer with low technical skill will tend to be more generous in marking.)

- Specifically define the required level of technical skills.- Then, check if a key person with such technical skills is already acquired. - If such resource is available within the company, negotiate for participation in the project, at least by means of consultation or review, if further involvement is difficult.


penalties / loss of credibility).- A person lacking leadership or who is shy in communicating is not eligible as a key person even if the person has the required experience and knowledge.

S24

S25 Communication Have meeting structures (for each customer level, between teams, and with partners) been defined and carried out?

Required meeting structures shall have been established through negotiation with stakeholders.

Informal communication paths shall also be established. It is important that the meeting structure is functioning effectively, and not remain as a mere desk plan.

- Establish the required meeting structures depending on the project status (such as the status of customer and partner companies) and call meetings periodically.

S25

S26 Communication Are reports being made to the customer as required?

A meeting structure with the customer shall be organized and actually practiced.

Progress reports on troubleshooting and investigation requests must be prepared and provided as required.

- Establish a meeting structure for reporting challenges, risks, and project status. S26

S27 Communication Is the project plan visible to everybody and available to all members of the project?

The location of the project plan shall have been informed to all members.

It is important for each member to clearly understand the contents of the project plan.

- Keep the project plan up to date, and ensure that the members look through items that are especially important.

- In particular, do not forget to explain to members new to the project.

S27

S28 Risks Have the risk items been clearly identified based on a risk analysis?

Risks shall be managed, for example, by creating a risk management table.

When identifying risks, it is effective to hold a discussion with stakeholders.

- Meet with team members to identify risks, and determine the probability of these risk and their impacts.

S28

S29 Risks Is the timing for performing risk countermeasures clear?

It shall be explicitly specified on the risk management table. Risk countermeasures include avoidance, transfer, and mitigation.

- Examine and determine the risk countermeasures for each identified risk. Furthermore, designate a person responsible for risk management and have him / her monitor the timing for executing the risk countermeasures.

S29

S30 Risks Is a contingency plan (supplemental budget and schedule) provided?

It shall be explicitly specified on the risk management table. In the case of a contingency, supplemental cost and schedule allocation, personnel replacement or addition will be required.

- Request the customer and senior management for contingency support (supplemental budget and schedule).

S30

S31 Procurement Are the work and deliverables of the partner companies being monitored on a periodically?

It shall be checked periodically. When checking, it is important to check the actual content of reports or deliverables.

- Manage deliverables and work reports (in particular, documents such as specifications) so that they are kept current by seeking the partner company’s cooperation.

- Especially for sources and system environment, organize a configuration management team to manage them exhaustively.

S31

S32 Customer Are the customer’s project goals clear? The project plan must contain not only the development goals, but also the customer’s goals.

In order to ensure smooth negotiations with the customer, it is important to understand what the customer wants to achieve through the system development.

- If the customer’s goals are not clear, conduct an interview with the customer and document the findings.

S32

S33 Technology Are the architecture requirements (workload, performance, and reliability) and the architecture design results valid?

The architecture requirements and the architecture design results shall have been reviewed.

The customer shall take responsibility in determining the architecture requirements, based on which the architecture is to be designed. Engage experts to review the validity of the requirements and the design results.

- If the programming process or the test process is started before the architecture is examined, it may result in a problem with the specifications. In particular, reconfirm that there is no architecture problem on the critical paths.

S33

15



S24 Human Resources

Have key personnel with the required technical skills been acquired?

If a package software is utilized in the project, personnel who can perform a fit and gap analysis or has experience with the software (database, middleware, tools) shall be assigned as key personnel.

The core technology of the target project must be understood (the required technical skills vary from one project to another).The experience and skill level pertaining the technology of the key personnel must be verified. (Note that the appraisal will depend on the technical level of the interviewer. An interviewer with low technical skill will tend to be more generous in marking.)

- Specifically define the required level of technical skills.- Then, check if a key person with such technical skills is already acquired. - If such resource is available within the company, negotiate for participation in the project, at least by means of consultation or review, if further involvement is difficult.


penalties / loss of credibility).- A person lacking leadership or who is shy in communicating is not eligible as a key person even if the person has the required experience and knowledge.

S24

S25 Communication Have meeting structures (for each customer level, between teams, and with partners) been defined and carried out?

Required meeting structures shall have been established through negotiation with stakeholders.

Informal communication paths shall also be established. It is important that the meeting structure is functioning effectively, and not remain as a mere desk plan.

- Establish the required meeting structures depending on the project status (such as the status of customer and partner companies) and call meetings periodically.

S25

S26 Communication Are reports being made to the customer as required?

A meeting structure with the customer shall be organized and actually practiced.

Progress reports on troubleshooting and investigation requests must be prepared and provided as required.

- Establish a meeting structure for reporting challenges, risks, and project status. S26

S27 Communication Is the project plan visible to everybody and available to all members of the project?

The location of the project plan shall have been informed to all members.

It is important for each member to clearly understand the contents of the project plan.

- Keep the project plan up to date, and ensure that the members look through items that are especially important.

- In particular, do not forget to explain to members new to the project.

S27

S28 Risks Have the risk items been clearly identified based on a risk analysis?

Risks shall be managed, for example, by creating a risk management table.

When identifying risks, it is effective to hold a discussion with stakeholders.

- Meet with team members to identify risks, and determine the probability of these risk and their impacts.

S28

S29 Risks Is the timing for performing risk countermeasures clear?

It shall be explicitly specified on the risk management table. Risk countermeasures include avoidance, transfer, and mitigation.

- Examine and determine the risk countermeasures for each identified risk. Furthermore, designate a person responsible for risk management and have him / her monitor the timing for executing the risk countermeasures.

S29

S30 Risks Is a contingency plan (supplemental budget and schedule) provided?

It shall be explicitly specified on the risk management table. In the case of a contingency, supplemental cost and schedule allocation, personnel replacement or addition will be required.

- Request the customer and senior management for contingency support (supplemental budget and schedule).

S30

S31 Procurement Are the work and deliverables of the partner companies being monitored on a periodically?

It shall be checked periodically. When checking, it is important to check the actual content of reports or deliverables.

- Manage deliverables and work reports (in particular, documents such as specifications) so that they are kept current by seeking the partner company’s cooperation.

- Especially for sources and system environment, organize a configuration management team to manage them exhaustively.

S31

S32 Customer Are the customer’s project goals clear? The project plan must contain not only the development goals, but also the customer’s goals.

In order to ensure smooth negotiations with the customer, it is important to understand what the customer wants to achieve through the system development.

- If the customer’s goals are not clear, conduct an interview with the customer and document the findings.

S32

S33 Technology Are the architecture requirements (workload, performance, and reliability) and the architecture design results valid?

The architecture requirements and the architecture design results shall have been reviewed.

The customer shall take responsibility in determining the architecture requirements, based on which the architecture is to be designed. Engage experts to review the validity of the requirements and the design results.

- If the programming process or the test process is started before the architecture is examined, it may result in a problem with the specifications. In particular, reconfirm that there is no architecture problem on the critical paths.

S33

16


endix


S34 Organization Are the missions of the project members clearly defined and are the members aware of them?

The project manager and the project members shall share each other’s individual mission.

- Clearly define each member’s responsibility assignment.

- It is important to determine who must do what, and by when.

- If this is not clarified, the project manager may end up making comments such as “I asked you to do it” and “that wasn’t what I wanted.”

- The project manager must present the scope of responsibility and the schedule (who does what, and by when) as a solid plan, not just by verbal explanations.

- Identify the required skills and provide appropriate education including external training.

- Reserve replacement personnel from partners.

S34

S35 Basic Conduct/Action

Are basic conducts/actions (practices and manners) enforced?

All members shall be aware of basic discipline and manners such as “take minutes of a meeting” or “document design details.”

Direct all members to observe prescribed rules so that the mood of the entire team will not allow for cutting corners. Good communication is not the same as letting things pass. It is important to undertake work with a sense of urgency.

- Continue providing educational guidance regarding basic conducts/actions on a routine basis.

S35


Are you aware of the situation of your team members?

Both formal and informal communication with subordinates shall be maintained.

If your team members do not report anything, they may have problems which they do not want to report, or there may be other communication problems such as the existence of a mood which discourages them from discussing a problem. It is important to communicate with the members as required, both in and out of meetings.

Understand the situation of team members from various perspectives such as by directly meeting with them or meeting with third parties who have relationships with them.

S36

S37 Motivation Do members feel cynical about the project plan?

Members must not have doubts about the schedule or feasibility of the project.

The plan should be developed by building on the necessary work. The baseline of the schedule and cost must be provided for the project members to meet with reasonable efforts.

- If there are members who are not satisfied, meet with them to determine which points they cannot accept, and after identifying the problems, share these information with all members.

- If a cynical mood remains, the motivation of the entire team may decline.

S37

S38 Motivation Are the members exhausted? Ensure that the project members are not showing signs of fatigue in their facial expressions, conversations, and behaviors.

If, for example, a member is working in a haze, making more mistakes, or looks tired, appropriate actions must be taken.

- Based on the understanding of the working status and work volume, ensure at least a day off every week, and review the work allocation to transfer some of the load to other member. This will also help to foster the other members as well as enforce the solidarity of the team.

S38

S39 Motivation Is decline in motivation of the project manager or project members observed?

Ensure that the project members are not gloomy in their facial expressions and comments.

If a problem exists, it is necessary to investigate the cause of motivation impairment by checking the project members’ work behavior and communication amongst the members.

- Conduct interviews with the project members to determine the cause of motivation impairment.

- Teams experiencing problems have a greater tendency to lose motivation. Provide particular support for such teams so that they can experience success.

- This will also contribute to raising the motivation of other teams.

S39

17



S34 Organization Are the missions of the project members clearly defined and are the members aware of them?

The project manager and the project members shall share each other’s individual mission.

- Clearly define each member’s responsibility assignment.

- It is important to determine who must do what, and by when.

- If this is not clarified, the project manager may end up making comments such as “I asked you to do it” and “that wasn’t what I wanted.”

- The project manager must present the scope of responsibility and the schedule (who does what, and by when) as a solid plan, not just by verbal explanations.

- Identify the required skills and provide appropriate education including external training.

- Reserve replacement personnel from partners.

S34


Are basic conducts/actions (practices and manners) enforced?

All members shall be aware of basic discipline and manners such as “take minutes of a meeting” or “document design details.”

Direct all members to observe prescribed rules so that the mood of the entire team will not allow for cutting corners. Good communication is not the same as letting things pass. It is important to undertake work with a sense of urgency.

- Continue providing educational guidance regarding basic conducts/actions on a routine basis.

S35


Are you aware of the situation of your team members?

Both formal and informal communication with subordinates shall be maintained.

If your team members do not report anything, they may have problems which they do not want to report, or there may be other communication problems such as the existence of a mood which discourages them from discussing a problem. It is important to communicate with the members as required, both in and out of meetings.

Understand the situation of team members from various perspectives such as by directly meeting with them or meeting with third parties who have relationships with them.

S36

S37 Motivation Do members feel cynical about the project plan?

Members must not have doubts about the schedule or feasibility of the project.

The plan should be developed by building on the necessary work. The baseline of the schedule and cost must be provided for the project members to meet with reasonable efforts.

- If there are members who are not satisfied, meet with them to determine which points they cannot accept, and after identifying the problems, share these information with all members.

- If a cynical mood remains, the motivation of the entire team may decline.

S37

S38 Motivation Are the members exhausted? Ensure that the project members are not showing signs of fatigue in their facial expressions, conversations, and behaviors.

If, for example, a member is working in a haze, making more mistakes, or looks tired, appropriate actions must be taken.

- Based on the understanding of the working status and work volume, ensure at least a day off every week, and review the work allocation to transfer some of the load to other member. This will also help to foster the other members as well as enforce the solidarity of the team.

S38

S39 Motivation Is decline in motivation of the project manager or project members observed?

Ensure that the project members are not gloomy in their facial expressions and comments.

If a problem exists, it is necessary to investigate the cause of motivation impairment by checking the project members’ work behavior and communication amongst the members.

- Conduct interviews with the project members to determine the cause of motivation impairment.

- Teams experiencing problems have a greater tendency to lose motivation. Provide particular support for such teams so that they can experience success.

- This will also contribute to raising the motivation of other teams.

S39

18


endix


S40 Issue Management

Are issues managed with the issues management table?

Issues (known problems that must be resolved) shall be managed at all times using a management table or a similar document which contains the date when the issue arose, the details of the issue, the priority level, the status, and the scheduled date for implementing countermeasures.

The actual issue management table must be verified:- Are the contents in the table really issues?- Is it clear who the person responsible for the issue is?

- Is the process until resolution described (including the change in target date)?

- Are there any inconsistencies between the target resolution date and the completion date?

- Are there any unresolved issues that have passed their target dates?

- Is there a disproportionate emphasis on certain issues?

- Meet with key personnel on both the customer and developer sides to extract issues and organize and manage them in a table.

S40

19



S40 Issue Management

Are issues managed with the issues management table?

Issues (known problems that must be resolved) shall be managed at all times using a management table or a similar document which contains the date when the issue arose, the details of the issue, the priority level, the status, and the scheduled date for implementing countermeasures.

The actual issue management table must be verified:- Are the contents in the table really issues?- Is it clear who the person responsible for the issue is?

- Is the process until resolution described (including the change in target date)?

- Are there any inconsistencies between the target resolution date and the completion date?

- Are there any unresolved issues that have passed their target dates?

- Is there a disproportionate emphasis on certain issues?

- Meet with key personnel on both the customer and developer sides to extract issues and organize and manage them in a table.

S40

20

No. Knowledge Area Check Item Assessment Criteria Individual Interview Instructions Evidence /

Checking Method Measures No.

H1 Integration Has the plan been approved and declared? In explaining the plan, it shall have been made clear whether the plan has been properly approved by the customer, relevant departments, and upper-level departments. For example, this shall be checked from the following perspectives:

- A kickoff meeting has been held.

- Developers and senior management were involved in the kickoff meeting.

- The project plan has been explained in detail to new project members.

Such aspects should generally not be addressed in the downstream process, but it is worthwhile.

- Minutes of the kickoff meeting

- Record of explanations

- Find out what the essential project plan, test plan, and migration plan are

- Have the project plan, test plan, and migration plan confirmed and approved by the stakeholders.

H1

H2 Integration Has an appropriate test plan been developed for mission critical functions of high importance?

A test plan for mission critical functions shall have been explicitly described in the test plan document.

- Since function requirements with high mission criticality cannot fail, they must be carefully managed, implemented, and monitored continuously from the stage of planning and design.

- In particular, if testing is insufficient and problems surface upon actual delivery or when operation starts, that will cause serious problems. As such, thorough test plans will be necessary.

- Mission critical functions must be managed separately from the standard Gantt chart, and their progress must also be managed separately.

- Progress should be managed with means such as a birds-eye view.

- A standard function may become mission critical as a result of specification changes.

- Project plan document

- Test plan document

- Critical functions list

- Develop the plans as soon as possible

- The test plan in particular should be prepared by an highly-skilled engineer with extensive business knowledge.

- Since qualification tests will be relating to actual operation, it is desirable to request customer cooperation concerning the content of the tests.

H2

H3 Integration Are the quantitative assessment results (cost and delivery date) of the basic design (external design and internal design) free of problems?

For example, this shall be checked from the following perspectives:

- The original budget, schedule, and estimated size are consistent.

- There is no large difference between the original plan and the actual situation.

- It is important that they are reviewed by experts who have knowledge of the business or technology.

- If a problem is found, the process may be under delay.

- If the estimation is incorrect, budget overrun and process delay will occur in the test phase.

- When the scale is not clear, the amount of tests cannot be estimated.

- Has the difference between the plan and the actual situation been analyzed at the right timing when the most accurate estimation can be obtained?

- Has a negotiation been held with the customer based on the results of the above analysis?

- Has the arrangement of resource been considered?

- If negotiation with the customer was unsuccessful, has a remedial action been taken?

- If the customer was unwilling to compromise, how was the risk managed?

- Review records

- Risk management table

- Confirm the validity if a quantitative assessment has not been performed on the basic design.

- If there are problems in cost and delivery date as a result of quantitative evaluation, make adjustments within the company and with the customer about cost, functions, and delivery dates.

H3

2. Check Sheet (Interview Sheet)App

endix

21




H1 Integration Has the plan been approved and declared? In explaining the plan, it shall have been made clear whether the plan has been properly approved by the customer, relevant departments, and upper-level departments. For example, this shall be checked from the following perspectives:

- A kickoff meeting has been held.

- Developers and senior management were involved in the kickoff meeting.

- The project plan has been explained in detail to new project members.

Such aspects should generally not be addressed in the downstream process, but it is worthwhile.

- Minutes of the kickoff meeting

- Record of explanations

- Find out what the essential project plan, test plan, and migration plan are

- Have the project plan, test plan, and migration plan confirmed and approved by the stakeholders.

H1

H2 Integration Has an appropriate test plan been developed for mission critical functions of high importance?

A test plan for mission critical functions shall have been explicitly described in the test plan document.

- Since function requirements with high mission criticality cannot fail, they must be carefully managed, implemented, and monitored continuously from the stage of planning and design.

- In particular, if testing is insufficient and problems surface upon actual delivery or when operation starts, that will cause serious problems. As such, thorough test plans will be necessary.

- Mission critical functions must be managed separately from the standard Gantt chart, and their progress must also be managed separately.

- Progress should be managed with means such as a birds-eye view.

- A standard function may become mission critical as a result of specification changes.



- Critical functions list

- Develop the plans as soon as possible

- The test plan in particular should be prepared by an highly-skilled engineer with extensive business knowledge.

- Since qualification tests will be relating to actual operation, it is desirable to request customer cooperation concerning the content of the tests.

H2

H3 Integration Are the quantitative assessment results (cost and delivery date) of the basic design (external design and internal design) free of problems?


- The original budget, schedule, and estimated size are consistent.

- There is no large difference between the original plan and the actual situation.

- It is important that they are reviewed by experts who have knowledge of the business or technology.

- If a problem is found, the process may be under delay.

- If the estimation is incorrect, budget overrun and process delay will occur in the test phase.

- When the scale is not clear, the amount of tests cannot be estimated.

- Has the difference between the plan and the actual situation been analyzed at the right timing when the most accurate estimation can be obtained?

- Has a negotiation been held with the customer based on the results of the above analysis?

- Has the arrangement of resource been considered?

- If negotiation with the customer was unsuccessful, has a remedial action been taken?

- If the customer was unwilling to compromise, how was the risk managed?

- Review records


- Confirm the validity if a quantitative assessment has not been performed on the basic design.

- If there are problems in cost and delivery date as a result of quantitative evaluation, make adjustments within the company and with the customer about cost, functions, and delivery dates.

H3

22


endix



H4 Integration Has the forecast or the plan of the environment (work site, supporting system, and tools) been prepared appropriately?

Has a test environment utilization plan been developed?

These plans must be explicitly described in the project plan document.

- These plans are generally examined during the planning phase to make necessary forecasts and take appropriate measures.

- Is the environment preparation plan identified in the WBS?

- Package utilization, combination of open source software, and other hidden issues may suddenly surface during the integration phase.

- Have differences in matters that can only be verified under the production environment been identified?

- For example, a middleware unsupported by the OS may have been selected.

- Have any new architectures been adopted?

Use of new components (unfamiliar technology or software combination) and packages require particular attention. The actual specifications of the package may differ from the catalog specifications.



- Test environment utilization plan

- Related WBS

- In planning, check against future work plans to ensure that there are no problems with the work site, supporting system, tools, and other environmental factors.

H4

H5 Integration Have the stakeholders agreed on the project schedule, scope of responsibilities, and WBS?

The stakeholders must have accepted and reached a consensus.


- A list of functions and modules for application development and the personnel in charge have been clarified.

- The personnel in charge of the work items other than development (procurement of computers, reservation of work site, document management, change management, etc.) have been clarified.

- There is prospect for the migration of the master data and the test data.

- Agreement has been reached on whether the customer is responsible for test data preparation.

- It is important to compare the project WBS with the customer WBS

- This should be done in collaboration with the customer.

- The main purpose of this task is to clarify the work allocated to each party.

- The vendor will be responsible for any work unallocated beyond this phase.


- Responsibility assignment matrix

- WBS

- Hold a meeting with the stakeholders to reach an agreement on the schedule and the scope of responsibilities in order to ensure proper and complete notification.

H5

H6 Integration Have the configuration management targets (documents and source codes) and the start date for the management been clarified?

The deliverables must be managed in an appropriate manner.


- The documents are managed.

- The source codes are managed.

- The history is managed in an appropriate manner, for example, by logging the date when the first version was approved.

- The number of specification generations that must be traced back for updates is determined for program modifications at the design level.

- In a multi-vendor project, a delay in one vendor’s work may prevent the smooth implementation of the project as a whole.

- The test plan must be changed if integration cannot be achieved.

- If a vendor has submitted a false report, configuration management will not be performed properly.

- Additional requirements will place extra burden on configuration management.

- Configuration management rules shall be defined (for example, the actions to be taken when a defect is found).

- Tracking of bug report forms shall be possible

- Tracking and management of problems shall be possible

- Issues shall be managed based on priorities

- Configuration management plan document

- Configuration management tools

- Operation rule

- If the configuration targets are unclear or not managed, make a list of the required configuration management targets. Collect these latest versions of these items, authorize to use them as the basis of configuration management, and start the management procedure.

H6

23




H4 Integration Has the forecast or the plan of the environment (work site, supporting system, and tools) been prepared appropriately?

Has a test environment utilization plan been developed?

These plans must be explicitly described in the project plan document.

- These plans are generally examined during the planning phase to make necessary forecasts and take appropriate measures.

- Is the environment preparation plan identified in the WBS?

- Package utilization, combination of open source software, and other hidden issues may suddenly surface during the integration phase.

- Have differences in matters that can only be verified under the production environment been identified?

- For example, a middleware unsupported by the OS may have been selected.

- Have any new architectures been adopted?

Use of new components (unfamiliar technology or software combination) and packages require particular attention. The actual specifications of the package may differ from the catalog specifications.



- Test environment utilization plan

- Related WBS

- In planning, check against future work plans to ensure that there are no problems with the work site, supporting system, tools, and other environmental factors.

H4

H5 Integration Have the stakeholders agreed on the project schedule, scope of responsibilities, and WBS?

The stakeholders must have accepted and reached a consensus.


- A list of functions and modules for application development and the personnel in charge have been clarified.

- The personnel in charge of the work items other than development (procurement of computers, reservation of work site, document management, change management, etc.) have been clarified.

- There is prospect for the migration of the master data and the test data.

- Agreement has been reached on whether the customer is responsible for test data preparation.

- It is important to compare the project WBS with the customer WBS

- This should be done in collaboration with the customer.

- The main purpose of this task is to clarify the work allocated to each party.

- The vendor will be responsible for any work unallocated beyond this phase.



- WBS

- Hold a meeting with the stakeholders to reach an agreement on the schedule and the scope of responsibilities in order to ensure proper and complete notification.

H5

H6 Integration Have the configuration management targets (documents and source codes) and the start date for the management been clarified?

The deliverables must be managed in an appropriate manner.


- The documents are managed.

- The source codes are managed.

- The history is managed in an appropriate manner, for example, by logging the date when the first version was approved.

- The number of specification generations that must be traced back for updates is determined for program modifications at the design level.

- In a multi-vendor project, a delay in one vendor’s work may prevent the smooth implementation of the project as a whole.

- The test plan must be changed if integration cannot be achieved.

- If a vendor has submitted a false report, configuration management will not be performed properly.

- Additional requirements will place extra burden on configuration management.

- Configuration management rules shall be defined (for example, the actions to be taken when a defect is found).

- Tracking of bug report forms shall be possible

- Tracking and management of problems shall be possible

- Issues shall be managed based on priorities

- Configuration management plan document

- Configuration management tools

- Operation rule

- If the configuration targets are unclear or not managed, make a list of the required configuration management targets. Collect these latest versions of these items, authorize to use them as the basis of configuration management, and start the management procedure.

H6

24


endix



H7 Integration Is the member responsible of configuration management clearly identified?

Furthermore, is configuration management actually practiced?

The member responsible for configuration management shall be actually conducting configuration management.

- The presence of a member who takes full responsibility in keeping track of configuration management is more important than merely having a member appointed as the “configuration manager.”

- It is desirable to have audits performed periodically.

- Use open questions such as “how do you perform xxx?” rather than “Are you doing xxx?”

- In conducting configuration management, an operation management scheme shall be organized which involves the steering committee and the customer, if one does not already exist.


- Organization chart

- Activity records

- Appoint a configuration manager. H7

H8 Integration Are configuration management rules and procedures clear?

Configuration management rules and procedures shall have been documented.

If various versions are mixed, or the release module management procedure have not been documented, problems such as incorrect releases and regression may occur, resulting in a serious rollback of the project.

- Rules and procedures

- Release management flow

- Ensure the implementation of release management.

- If incorrect releases and regressions occur frequently, review the implementation procedure to reduce mistakes, and separately organize a configuration management team to implement strict management of the system environment and source codes.

H8

H9 Integration Are the maintenance and operation plans and the operation procedures and rules clearly defined and reviewed?

The organizational framework and the maintenance and operation plan shall have been checked to ensure that there is no problem.

- There are cases when problems arise upon closing the project.

- Have they been finalized before integration tests?

- If not finalized, check the following items:

- Is it still possible to carry out remedial actions?

- Can the problems be corrected locally?

- Should maintenance work be included in the recovery plan?

- Have considerations been given to the operation cycles (daily, weekly, monthly)?

- Has operation training been planned?

- Maintenance and operation plan document

- Review records

- Plan the maintenance and operation framework and reach an agreement with the stakeholders.

- If there are no operation procedure and rules, develop them based on past cases.

H9

H10 Integration Is there a clear production migration plan?

(The migration plan includes migration of human resources, systems, and data)

Responsibility assignments between the customer and other stakeholders shall have been clarified. The migration plan and associated tasks shall have also been agreed upon by the customer.

When migrating customer data, needless to say migration of files, there are many cases where there are problems in the quality of customer data.

It is important to verify the reliability of the data owned by the customer, and to develop the migration plan based on the assessment.


- Migration plan document

- Prepare a production migration plan, and have it reviewed by the stakeholders inclusive of experts and the customers.

H10

25




H7 Integration Is the member responsible of configuration management clearly identified?

Furthermore, is configuration management actually practiced?

The member responsible for configuration management shall be actually conducting configuration management.

- The presence of a member who takes full responsibility in keeping track of configuration management is more important than merely having a member appointed as the “configuration manager.”

- It is desirable to have audits performed periodically.

- Use open questions such as “how do you perform xxx?” rather than “Are you doing xxx?”

- In conducting configuration management, an operation management scheme shall be organized which involves the steering committee and the customer, if one does not already exist.



- Activity records

- Appoint a configuration manager. H7

H8 Integration Are configuration management rules and procedures clear?

Configuration management rules and procedures shall have been documented.

If various versions are mixed, or the release module management procedure have not been documented, problems such as incorrect releases and regression may occur, resulting in a serious rollback of the project.

- Rules and procedures

- Release management flow

- Ensure the implementation of release management.

- If incorrect releases and regressions occur frequently, review the implementation procedure to reduce mistakes, and separately organize a configuration management team to implement strict management of the system environment and source codes.

H8

H9 Integration Are the maintenance and operation plans and the operation procedures and rules clearly defined and reviewed?

The organizational framework and the maintenance and operation plan shall have been checked to ensure that there is no problem.

- There are cases when problems arise upon closing the project.

- Have they been finalized before integration tests?

- If not finalized, check the following items:

- Is it still possible to carry out remedial actions?

- Can the problems be corrected locally?

- Should maintenance work be included in the recovery plan?

- Have considerations been given to the operation cycles (daily, weekly, monthly)?

- Has operation training been planned?

- Maintenance and operation plan document

- Review records

- Plan the maintenance and operation framework and reach an agreement with the stakeholders.

- If there are no operation procedure and rules, develop them based on past cases.

H9

H10 Integration Is there a clear production migration plan?

(The migration plan includes migration of human resources, systems, and data)

Responsibility assignments between the customer and other stakeholders shall have been clarified. The migration plan and associated tasks shall have also been agreed upon by the customer.

When migrating customer data, needless to say migration of files, there are many cases where there are problems in the quality of customer data.

It is important to verify the reliability of the data owned by the customer, and to develop the migration plan based on the assessment.



- Prepare a production migration plan, and have it reviewed by the stakeholders inclusive of experts and the customers.

H10

26


endix



H11 Scope Are changes to the scope of functionality and/or work that may have an impact on the direction of the system appropriately managed?


- The procedures for handling changes shall have been clearly defined.

- The contact points and coordinators shall have been clarified.

- The procedures shall be fully functional.

- Make sure beforehand that the stakeholders strictly comply with the change management procedures.

- Check the thresholds and criteria.

- The “2:4:3 formula” (an empirical rule in Japan on scope expansion. It is said that the relative scope volume initially indicated is 2, then expands to 4 during project progress, and finally settles at 3.) indicates an overall perspective of the scope.

- This check item verifies preciseness of the project manager.

- Changes often occur during a project. These changes must be controlled based on the relationship between delivery date and cost, and must not be approved just by the course of the situation.

- The key point is whether the change influences the direction of the system development.

- Once the scope has been determined, the WBS will influence the schedule and the work period.

- It is important that the scope does not change just by the course of the situation.

- Change management table

- Change management rules

- Clarify the change management procedures if there are no change management rules in place.

- Conduct interviews with the stakeholders to check whether there are any problems in the implementation of the change management, and resolve problems, if any.

H11

H12 Scope Is there any unexpected expansion of scope (scope of functionality and work)?

If yes, is the expansion within the acceptable range?

The expansion of the scope must be kept track quantitatively.

The expansion shall be assessed from the perspective of cost and schedule whether it is within the acceptable range.

- Countermeasures (reduction of functions, increase of cost, and change of scope) in response to scope expansion shall have been agreed in advance with the customer.

- If a scope expansion occurs, identify its point of occurrence and its effect on the test volume and critical functions.


- Minutes of meetings

- If the expansion is beyond the acceptable range, clarify the scale of the expansion and then hold a discussion with the customer.

H12

H13 Scope Is understanding sufficient when the project is handed over from the person in charge of the previous process?

Handover materials shall have been defined. Areas of insufficient understanding shall be clarified.

- Ensure that the handover materials are complete, and understand their contents.

- Understand in depth the output from the previous process.

- If their is insufficient understanding, measures shall be taken, such as reviewing the handover materials and having a person in charge of the upstream process explain the details.

- Handover may not have been completed appropriately when the consulting company, who was in charge of the previous process, provided the deliverables to the SI vendor. This may cause the project to fail.

- Using the V-shape model, check the number of people who can determine whether the test results are acceptable.

- Handover materials - Ask for the consulting company for further assistance if there is insufficient understanding.

H13

27




H11 Scope Are changes to the scope of functionality and/or work that may have an impact on the direction of the system appropriately managed?


- The procedures for handling changes shall have been clearly defined.

- The contact points and coordinators shall have been clarified.

- The procedures shall be fully functional.

- Make sure beforehand that the stakeholders strictly comply with the change management procedures.

- Check the thresholds and criteria.

- The “2:4:3 formula” (an empirical rule in Japan on scope expansion. It is said that the relative scope volume initially indicated is 2, then expands to 4 during project progress, and finally settles at 3.) indicates an overall perspective of the scope.

- This check item verifies preciseness of the project manager.

- Changes often occur during a project. These changes must be controlled based on the relationship between delivery date and cost, and must not be approved just by the course of the situation.

- The key point is whether the change influences the direction of the system development.

- Once the scope has been determined, the WBS will influence the schedule and the work period.

- It is important that the scope does not change just by the course of the situation.


- Change management rules

- Clarify the change management procedures if there are no change management rules in place.

- Conduct interviews with the stakeholders to check whether there are any problems in the implementation of the change management, and resolve problems, if any.

H11

H12 Scope Is there any unexpected expansion of scope (scope of functionality and work)?

If yes, is the expansion within the acceptable range?

The expansion of the scope must be kept track quantitatively.

The expansion shall be assessed from the perspective of cost and schedule whether it is within the acceptable range.

- Countermeasures (reduction of functions, increase of cost, and change of scope) in response to scope expansion shall have been agreed in advance with the customer.

- If a scope expansion occurs, identify its point of occurrence and its effect on the test volume and critical functions.



- If the expansion is beyond the acceptable range, clarify the scale of the expansion and then hold a discussion with the customer.

H12

H13 Scope Is understanding sufficient when the project is handed over from the person in charge of the previous process?

Handover materials shall have been defined. Areas of insufficient understanding shall be clarified.

- Ensure that the handover materials are complete, and understand their contents.

- Understand in depth the output from the previous process.

- If their is insufficient understanding, measures shall be taken, such as reviewing the handover materials and having a person in charge of the upstream process explain the details.

- Handover may not have been completed appropriately when the consulting company, who was in charge of the previous process, provided the deliverables to the SI vendor. This may cause the project to fail.

- Using the V-shape model, check the number of people who can determine whether the test results are acceptable.

- Handover materials - Ask for the consulting company for further assistance if there is insufficient understanding.

H13

28


endix



H14 Scope If connecting with a system developed by another company (including the customer), has the responsibility assignment been clarified?

For connections with systems developed by another company, the responsibility assignment chart as well as the test and migration schedule shall have been prepared and agreed upon with the relevant parties.


- The schedule for the connection test shall have been agreed upon with the other companies.

- The test procedures shall have been agreed upon.

- The party responsible for preparing the test data shall have been agreed upon.

- The party responsible for test result aggregation shall have been agreed upon.

- Have the WBS to be used (own or another’s) and the task assignment been clarified?

- Have the rules for processing bug reports been clarified?

- A person responsible for the tests shall be determined.

- The 5W1H shall be indicated clearly.

- Clearly define the purpose and preparatory steps for the tests.




- Test and migration schedule

- Clearly define the responsibility assignment and the system connection schedule, and hold a meeting with the customer and the other companies.

H14

H15 Scope Is there an agreement with the customer regarding the (contractual) handling of specification changes, and are measures such as separate payment being practiced?

An agreement in writing shall have been concluded with the customer.

- For example, the approval route, payment method, and risk impact shall have been agreed upon.

- Contract issues shall have been addressed. Industry conventions should not be brought forth in later processes (frequent in logistics and financial industries).

- The intention of this check item is to stress that incomplete contracts may lead to problems.

- Matters concerning the scope should be clearly indicated in the contract The scope must be clarified not only for the prime contractors, but also for secondary contractors. If the scope is not clear for secondary and subsequent levels of contractors, the prime contractor will be unable to provide proper service.


- Approval route

- Payment method

- If the contract terms regarding the handling of specification changes have not been agreed upon, suspend the development process and work to reach a consensus as soon as possible.

H15

H16 Time Are tasks to be carried out clearly defined? It shall be clear what must be done by when. Has a list of tasks been prepared? If a list has been prepared, is it clear what is to be done in those tasks?

- Schedule chart

- WBS

- Estimates

- Responsibility assignment matrix (organization chart)

- Gather key personnel to make a list of tasks to be tests.

- For each listed task, identify the work details and the deliverables, estimate of workload, and clarify the work schedule.

- Assign persons in charge.

H16

29




H14 Scope If connecting with a system developed by another company (including the customer), has the responsibility assignment been clarified?

For connections with systems developed by another company, the responsibility assignment chart as well as the test and migration schedule shall have been prepared and agreed upon with the relevant parties.


- The schedule for the connection test shall have been agreed upon with the other companies.

- The test procedures shall have been agreed upon.

- The party responsible for preparing the test data shall have been agreed upon.

- The party responsible for test result aggregation shall have been agreed upon.

- Have the WBS to be used (own or another’s) and the task assignment been clarified?

- Have the rules for processing bug reports been clarified?

- A person responsible for the tests shall be determined.

- The 5W1H shall be indicated clearly.

- Clearly define the purpose and preparatory steps for the tests.




- Test and migration schedule

- Clearly define the responsibility assignment and the system connection schedule, and hold a meeting with the customer and the other companies.

H14

H15 Scope Is there an agreement with the customer regarding the (contractual) handling of specification changes, and are measures such as separate payment being practiced?

An agreement in writing shall have been concluded with the customer.

- For example, the approval route, payment method, and risk impact shall have been agreed upon.

- Contract issues shall have been addressed. Industry conventions should not be brought forth in later processes (frequent in logistics and financial industries).

- The intention of this check item is to stress that incomplete contracts may lead to problems.

- Matters concerning the scope should be clearly indicated in the contract The scope must be clarified not only for the prime contractors, but also for secondary contractors. If the scope is not clear for secondary and subsequent levels of contractors, the prime contractor will be unable to provide proper service.


- Approval route

- Payment method

- If the contract terms regarding the handling of specification changes have not been agreed upon, suspend the development process and work to reach a consensus as soon as possible.

H15

H16 Time Are tasks to be carried out clearly defined? It shall be clear what must be done by when. Has a list of tasks been prepared? If a list has been prepared, is it clear what is to be done in those tasks?

- Schedule chart

- WBS

- Estimates

- Responsibility assignment matrix (organization chart)

- Gather key personnel to make a list of tasks to be tests.

- For each listed task, identify the work details and the deliverables, estimate of workload, and clarify the work schedule.

- Assign persons in charge.

H16

30


endix



H17 Time Are the workload, overlaps, and openings of the assigned personnel being checked?

Both the project manager and the personnel in charge must do the checking.

- If a check is not conducted, the manager will eventually not pay sufficient attention to the workload of each member, resulting in inconsistency with the plan.

- What is used to measure the progress is important.

- Is the progress checked properly?

Even if materials have been prepared using time and effort, is it referred by the members?

- Is the actual material checked?

- Is it not assumed as completed without reviewing the actual materials?

- How is the completion of a check proved?

Check that evidence and completion criteria (process completion criteria) are established by the actual material.

- Be sure to rule out all possibilities of a 90% syndrome (i.e. never reaches 100%).

- Schedule chart (progress management)

- WBS

- Estimates


- Hold progress meetings to check the workload status on a regular basis.

H17

H18 Time Is the process progress consistent? For example, this shall be checked from the following perspectives:

- There should be no inconsistency in the step sequence on the work item schedule charts (small step chart or Gantt chart).

- The qualification test must not be started until both basic specifications and qualification test plan have been completed.

- Is there a clear distinction between critical and non-critical paths?

- Have changes been handled appropriately depending on the situation, and not only in the early stages?

- Are not the corrections limited to local corrections?

- Schedule chart

- WBS

- Estimates


- Check the overall Gantt chart and each task item. If any inconsistency is found, suspend the work, conduct a review, and then perform tasks that must be completed first.

H18

H19 Time Is there adequate consistency between the master schedule and resource histogram?

A resource histogram shall have been created and checked for consistency when preparing the master schedule.

The difference between the original plan and the actual situation should be examined. In reference to the V-shape model structure of development, testing personnel should be included in the resource histogram. More specifically, personnel in charge of detail design should be assigned to the integration test.

- Master schedule

- Resource histogram

- Review the personnel adjustment plan.

H19

H20 Time Is there adequate consistency between the master schedule and the team schedules?

Consistency must be maintained by first preparing the master schedule, then the team schedules, and then finally revising the master schedule.

- Has the project been under control in the event there had been a change in the schedule submitted by one of the many vendors?

Had consistency been maintained when integrating the individual schedules?

- How was the impact on the master schedule when changes occurred to each company’s schedule?

If individual schedules are not checked against the master schedule, it indicates that scheduling is entirely left entirely to the individual companies.

- This check item is to verify that scheduling is not entirely left entirely to the individual companies.

- A paper plan schedule is pointless. It is important that the schedule is based on reasonable grounds.

- A work that requires three months by two persons does not necessarily mean that it could be completed in two months by three persons.

- Master schedule

- Team schedules

- Ensure consistency between the master schedule and team schedules by first preparing the master schedule, then the team schedules, and then finally revising the master schedule.

H20

31




H17 Time Are the workload, overlaps, and openings of the assigned personnel being checked?

Both the project manager and the personnel in charge must do the checking.

- If a check is not conducted, the manager will eventually not pay sufficient attention to the workload of each member, resulting in inconsistency with the plan.

- What is used to measure the progress is important.

- Is the progress checked properly?

Even if materials have been prepared using time and effort, is it referred by the members?

- Is the actual material checked?

- Is it not assumed as completed without reviewing the actual materials?

- How is the completion of a check proved?

Check that evidence and completion criteria (process completion criteria) are established by the actual material.

- Be sure to rule out all possibilities of a 90% syndrome (i.e. never reaches 100%).

- Schedule chart (progress management)

- WBS

- Estimates


- Hold progress meetings to check the workload status on a regular basis.

H17

H18 Time Is the process progress consistent? For example, this shall be checked from the following perspectives:

- There should be no inconsistency in the step sequence on the work item schedule charts (small step chart or Gantt chart).

- The qualification test must not be started until both basic specifications and qualification test plan have been completed.

- Is there a clear distinction between critical and non-critical paths?

- Have changes been handled appropriately depending on the situation, and not only in the early stages?

- Are not the corrections limited to local corrections?

- Schedule chart

- WBS

- Estimates


- Check the overall Gantt chart and each task item. If any inconsistency is found, suspend the work, conduct a review, and then perform tasks that must be completed first.

H18

H19 Time Is there adequate consistency between the master schedule and resource histogram?

A resource histogram shall have been created and checked for consistency when preparing the master schedule.

The difference between the original plan and the actual situation should be examined. In reference to the V-shape model structure of development, testing personnel should be included in the resource histogram. More specifically, personnel in charge of detail design should be assigned to the integration test.

- Master schedule

- Resource histogram

- Review the personnel adjustment plan.

H19

H20 Time Is there adequate consistency between the master schedule and the team schedules?

Consistency must be maintained by first preparing the master schedule, then the team schedules, and then finally revising the master schedule.

- Has the project been under control in the event there had been a change in the schedule submitted by one of the many vendors?

Had consistency been maintained when integrating the individual schedules?

- How was the impact on the master schedule when changes occurred to each company’s schedule?

If individual schedules are not checked against the master schedule, it indicates that scheduling is entirely left entirely to the individual companies.

- This check item is to verify that scheduling is not entirely left entirely to the individual companies.

- A paper plan schedule is pointless. It is important that the schedule is based on reasonable grounds.

- A work that requires three months by two persons does not necessarily mean that it could be completed in two months by three persons.

- Master schedule

- Team schedules

- Ensure consistency between the master schedule and team schedules by first preparing the master schedule, then the team schedules, and then finally revising the master schedule.

H20

32


endix



H21 Time Is the schedule prepared based on reasonable grounds?

Correspondence between the work volume and the organizational framework shall have been made clear.

- How far are the reasonings thought through?

- It depends on how much pressure is placed on the project. The pressure from the customer, SI vendors, and supervisors will accumulate.

- When secondary and subsequent contractors have signed the project contract, how had their processes been judged as reasonable?

- Are capabilities and skills considered?

- When the SI vendor places an order of the work, what are the grounds for the decision?

- Do the prime contractors confirm the grounds of the secondary subcontractors?

- Grounds on which the schedule chart was prepared

- Investigate why a schedule was prepared without reasonable grounds.

- Groundless pressure from the customer, prime contractors, and supervisors.

- Knowledge/experience of project manager was insufficient.

- The project manager is totally under the influence of the team members.

- The project manager lacks initiative.

- Forcing a schedule will only result in re-scheduling.

[Re-scheduling Tips]

- Consider required resource, capability of the individuals, and degree of concentration (avoid overload).

- Consider resource increase and/or extension of work period against cost.

- Make the schedule risk-tolerant.

- Reserve time for intermediate checks, reviews, and corrections.

- If the delivery date is fixed from the outset, it is highly possible that it may become a risk factor. Each phase has a minimum requisite period that cannot be shortened further. Make the schedule so that those requisites are met.

H21

H22 Time Is the progress line on the Gantt chart based on clear grounds?

The progress rate must be based on reasonable grounds, not on sense or impression.

- Check the logical consistency between previous, current, and next steps. Check the progress line from the perspective of planned and actual progress management.

- Are there any problems with the granularity of the number of test cases?

- Have scenarios and business cases been prepared?

- Are there any problems with the grounds for the progress and the plan?

- Test case completion ratio

- Bug reports

- Schedule chart

- WBS

- Formalize and document the progress assessment method, and have progress report comply with those rules.

H22

33




H21 Time Is the schedule prepared based on reasonable grounds?

Correspondence between the work volume and the organizational framework shall have been made clear.

- How far are the reasonings thought through?

- It depends on how much pressure is placed on the project. The pressure from the customer, SI vendors, and supervisors will accumulate.

- When secondary and subsequent contractors have signed the project contract, how had their processes been judged as reasonable?

- Are capabilities and skills considered?

- When the SI vendor places an order of the work, what are the grounds for the decision?

- Do the prime contractors confirm the grounds of the secondary subcontractors?

- Grounds on which the schedule chart was prepared

- Investigate why a schedule was prepared without reasonable grounds.

- Groundless pressure from the customer, prime contractors, and supervisors.

- Knowledge/experience of project manager was insufficient.

- The project manager is totally under the influence of the team members.

- The project manager lacks initiative.

- Forcing a schedule will only result in re-scheduling.

[Re-scheduling Tips]

- Consider required resource, capability of the individuals, and degree of concentration (avoid overload).

- Consider resource increase and/or extension of work period against cost.

- Make the schedule risk-tolerant.

- Reserve time for intermediate checks, reviews, and corrections.

- If the delivery date is fixed from the outset, it is highly possible that it may become a risk factor. Each phase has a minimum requisite period that cannot be shortened further. Make the schedule so that those requisites are met.

H21

H22 Time Is the progress line on the Gantt chart based on clear grounds?

The progress rate must be based on reasonable grounds, not on sense or impression.

- Check the logical consistency between previous, current, and next steps. Check the progress line from the perspective of planned and actual progress management.

- Are there any problems with the granularity of the number of test cases?

- Have scenarios and business cases been prepared?

- Are there any problems with the grounds for the progress and the plan?

- Test case completion ratio

- Bug reports

- Schedule chart

- WBS

- Formalize and document the progress assessment method, and have progress report comply with those rules.

H22

34


endix



H23 Time Have critical paths been clearly identified? Task items on the critical paths shall have been identified. - A delay in a critical path tends to cause serious problems with the project.

- How are critical paths identified? A PERT diagram could help if it is always correct. However, since the actually required time varies between tasks, in some aspects, it is left to chance.

- A PERT diagram is often prepared during the planning stage, but is rarely maintained.

- Have items that may become a critical path been identified?

- Have potential bottlenecks in the project been identified?

- Have critical paths of past projects been checked?

- Is critical path within the test phase being managed?

- Critical paths may change everyday, making their management difficult.

- It is sometimes better to identify the bottlenecks instead. Examples of a bottleneck include excessive workload on a specific member, a delay in a specific task, and variance in quality. A member who tries to carry too much burden may become a bottleneck.

- Assign skilled personnel to areas that are critical. Otherwise, it may lead to failure cases where personnel expected to serve as key personnel were actually not.

- Have potentially critical areas been identified? Are there any changes to the critical areas?

- Schedule chart

- WBS

- If there is a delay on the critical path, conduct a meeting with the key personnel to identify the problems as soon as possible, and take appropriate measures.

- Make use of methodologies such as crashing and fast tracking.

H23

H24 Cost How far can cost constraints be adjusted? Measures such as reduction of functions, increase of the contract amount, and change of scope shall have been agreed upon in advance with the customer from the perspective of cost constraints.

- For example, how much is the upper limit of the payment amount? Is a reduction of functions acceptable?

- Are these specified as special provisions on a separate contract attached to the basic contract?

- In general, basic contracts are determined depending on each company. If these provisions are not properly prepared, they may adversely affect both cost and schedule. It is difficult to prepare perfect provisions, but ordering parties are becoming increasingly aware of their importance.

- Check the project plan for items related to quality goals.

- If the quality goals are not described in the plan, that in itself represents a high risk.

- Appendix to the contract

- Agree with the customer on reducing functions in order to meet cost constraints.

H24

H25 Cost Are estimates been made from multiple viewpoints?

Estimates shall have been validated by using multiple estimation methods or by having them checked by multiple experienced personnel.

By making multiple estimates, you may recognize important requirements, feasibility, or risks to be considered that are not recognizable through a simple estimate.

- Quotation - If an estimate is based on a single method or on experience, it may diverge considerably from the actual scale or required efforts.

- If there is a large difference between the initially forecasted scale and the actual scale, the subsequent plans for the project should be revised.

H25

35




H23 Time Have critical paths been clearly identified? Task items on the critical paths shall have been identified. - A delay in a critical path tends to cause serious problems with the project.

- How are critical paths identified? A PERT diagram could help if it is always correct. However, since the actually required time varies between tasks, in some aspects, it is left to chance.

- A PERT diagram is often prepared during the planning stage, but is rarely maintained.

- Have items that may become a critical path been identified?

- Have potential bottlenecks in the project been identified?

- Have critical paths of past projects been checked?

- Is critical path within the test phase being managed?

- Critical paths may change everyday, making their management difficult.

- It is sometimes better to identify the bottlenecks instead. Examples of a bottleneck include excessive workload on a specific member, a delay in a specific task, and variance in quality. A member who tries to carry too much burden may become a bottleneck.

- Assign skilled personnel to areas that are critical. Otherwise, it may lead to failure cases where personnel expected to serve as key personnel were actually not.

- Have potentially critical areas been identified? Are there any changes to the critical areas?

- Schedule chart

- WBS

- If there is a delay on the critical path, conduct a meeting with the key personnel to identify the problems as soon as possible, and take appropriate measures.

- Make use of methodologies such as crashing and fast tracking.

H23

H24 Cost How far can cost constraints be adjusted? Measures such as reduction of functions, increase of the contract amount, and change of scope shall have been agreed upon in advance with the customer from the perspective of cost constraints.

- For example, how much is the upper limit of the payment amount? Is a reduction of functions acceptable?

- Are these specified as special provisions on a separate contract attached to the basic contract?

- In general, basic contracts are determined depending on each company. If these provisions are not properly prepared, they may adversely affect both cost and schedule. It is difficult to prepare perfect provisions, but ordering parties are becoming increasingly aware of their importance.

- Check the project plan for items related to quality goals.

- If the quality goals are not described in the plan, that in itself represents a high risk.

- Appendix to the contract

- Agree with the customer on reducing functions in order to meet cost constraints.

H24

H25 Cost Are estimates been made from multiple viewpoints?

Estimates shall have been validated by using multiple estimation methods or by having them checked by multiple experienced personnel.

By making multiple estimates, you may recognize important requirements, feasibility, or risks to be considered that are not recognizable through a simple estimate.

- Quotation - If an estimate is based on a single method or on experience, it may diverge considerably from the actual scale or required efforts.

- If there is a large difference between the initially forecasted scale and the actual scale, the subsequent plans for the project should be revised.

H25

36


endix



H26 Quality Have specific methods for the sequential step-by-step testing of components, sub-programs, and systems been planned and applied?

The schedule shall have been verified that it is free of any inconsistency (for example, by conducting the integration test before hardware installation).

A delay in the testing of one component may cause a delay in the testing of the entire system. Hence, it is important to develop an overall test plan in advance.

Make sure that the test plan is specific and free of problems.

- Overall test plan document


- Test procedures

- If there is a problem with the test plan, identify the problem and risks, and examine solutions and risk countermeasures.

H26

H27 Quality Has the validity of the test cases, test data, and test tools been verified?


- There is no bias in the test cases and test data.

- The test coverage is sufficient.

- The test cases should not be limited to normal operation cases.

- Provided test data is appropriate for the tests conducted.

- Check the extraction criteria for the test cases and ensure that they are properly reviewed.

- Check that the coverage of the test tools is understood.

- Check that tests are planned and conducted based on scenarios.

- Are the scenarios valid?

In particular, be sure to include abnormal operation patterns.

- Test cases table

- Test data

- Test tools

- Test review records

- Correct biased test cases, if any.

- Prepare additional tests and tools.

H27

H28 Quality Are the results of code inspection (code review) managed in accordance with the predefined methods? Are quality and productivity monitored to see whether they meet their goals?


- Indicators for code inspection, such as the number of detected problems, shall have been defined.

- Problem detection criteria for code inspection shall have been clearly defined.

A method must be provided for checking that code inspection results are satisfactory. Peer review is one example of an effective method.

- Review records - If code inspection results are not managed, prepare a sheet or form to record and manage them, and assess quality based on indicators such as the number of detected problems.

H28

H29 Quality Are the tests conducted with considerations for boundaries such as logic coverage, data, and dates?

For example, the following test items must be included:

- Measurement of coverage ratio

- Tests based on boundary values

- Test for leap years

Check the utilization of tools. - Test results report - Measure the coverage ratio using a coverage measurement tool.

- If unit tests are conducted manually, implement an automation mechanism together with the coverage measurement tool.

H29

H30 Quality Is the test plan clearly defined for each test process? Are the tests conducted as planned?

Tests shall not be conducted without a plan.


- Test is conducted based on an overall test plan.

- Test results are reviewed in accordance with the plan.

- Rollbacks as a result of the tests are not occurring frequently (the same tests are not conducted repeatedly due to release errors or implementation errors).

- When preparing the test plan, check that the scale of the system has not changed from the original plan. Otherwise, the scale of the test will be estimated incorrectly.

- The test strategy must be clarified. Starting with minute details is not effective. Functions most frequently used by the customer should be brought to stable quality first.

- When a library is replaced, check whether a retest is necessary. Incorrect judgment may result in unexpected degradation.

- Test results report



- If the test plan is not clear, reorganize the test plan.

- If the test plan is clear but test is not conducted as planned, investigate the cause. In the case of a personnel issue, provide instructions to ensure compliance.

- If the cause is external (for example, the priority was changed due to a request from another company or the customer), ensure that the change is reported.

- For plan changes due to external factors, confirm the reason for the change.

- Reexamine future test plans as necessary.

H30

H31 Quality Are defects being managed using bug report forms?

The bug report form management method must be fully understood by the project members.

Check whether the bug report forms are filled appropriately, and provide thorough instructions to personnel as necessary.

- Bug report forms - If bug report forms are not being used, instruct and ensure personnel to use bug reports in accordance with predefined procedures.

H31

37




H26 Quality Have specific methods for the sequential step-by-step testing of components, sub-programs, and systems been planned and applied?

The schedule shall have been verified that it is free of any inconsistency (for example, by conducting the integration test before hardware installation).

A delay in the testing of one component may cause a delay in the testing of the entire system. Hence, it is important to develop an overall test plan in advance.

Make sure that the test plan is specific and free of problems.

- Overall test plan document


- Test procedures

- If there is a problem with the test plan, identify the problem and risks, and examine solutions and risk countermeasures.

H26

H27 Quality Has the validity of the test cases, test data, and test tools been verified?


- There is no bias in the test cases and test data.

- The test coverage is sufficient.

- The test cases should not be limited to normal operation cases.

- Provided test data is appropriate for the tests conducted.

- Check the extraction criteria for the test cases and ensure that they are properly reviewed.

- Check that the coverage of the test tools is understood.

- Check that tests are planned and conducted based on scenarios.

- Are the scenarios valid?

In particular, be sure to include abnormal operation patterns.

- Test cases table

- Test data

- Test tools

- Test review records

- Correct biased test cases, if any.

- Prepare additional tests and tools.

H27

H28 Quality Are the results of code inspection (code review) managed in accordance with the predefined methods? Are quality and productivity monitored to see whether they meet their goals?


- Indicators for code inspection, such as the number of detected problems, shall have been defined.

- Problem detection criteria for code inspection shall have been clearly defined.

A method must be provided for checking that code inspection results are satisfactory. Peer review is one example of an effective method.

- Review records - If code inspection results are not managed, prepare a sheet or form to record and manage them, and assess quality based on indicators such as the number of detected problems.

H28

H29 Quality Are the tests conducted with considerations for boundaries such as logic coverage, data, and dates?

For example, the following test items must be included:

- Measurement of coverage ratio

- Tests based on boundary values

- Test for leap years

Check the utilization of tools. - Test results report - Measure the coverage ratio using a coverage measurement tool.

- If unit tests are conducted manually, implement an automation mechanism together with the coverage measurement tool.

H29

H30 Quality Is the test plan clearly defined for each test process? Are the tests conducted as planned?

Tests shall not be conducted without a plan.


- Test is conducted based on an overall test plan.

- Test results are reviewed in accordance with the plan.

- Rollbacks as a result of the tests are not occurring frequently (the same tests are not conducted repeatedly due to release errors or implementation errors).

- When preparing the test plan, check that the scale of the system has not changed from the original plan. Otherwise, the scale of the test will be estimated incorrectly.

- The test strategy must be clarified. Starting with minute details is not effective. Functions most frequently used by the customer should be brought to stable quality first.

- When a library is replaced, check whether a retest is necessary. Incorrect judgment may result in unexpected degradation.




- If the test plan is not clear, reorganize the test plan.

- If the test plan is clear but test is not conducted as planned, investigate the cause. In the case of a personnel issue, provide instructions to ensure compliance.

- If the cause is external (for example, the priority was changed due to a request from another company or the customer), ensure that the change is reported.

- For plan changes due to external factors, confirm the reason for the change.

- Reexamine future test plans as necessary.

H30

H31 Quality Are defects being managed using bug report forms?

The bug report form management method must be fully understood by the project members.

Check whether the bug report forms are filled appropriately, and provide thorough instructions to personnel as necessary.

- Bug report forms - If bug report forms are not being used, instruct and ensure personnel to use bug reports in accordance with predefined procedures.

H31

38


endix



H32 Quality Have test verification procedures and methods been planned? Are they being used for test verification?

Test verification procedures and methods shall have been fully understood by project members.

It is important that a verification plan has been developed in advance for program code reviews and test results verifications, and that verifications are performed in accordance with the plan, not in an ad-hoc manner.

- Procedures manual


- If the procedures and methods are not being followed, instruct the personnel to comply.

H32

H33 Quality Are the test (integration test/qualification test) plans and their review results free of problems?

Findings from the reviews shall have been addressed. The test cases should be reviewed based on quality assessment indicators and target values that have been predefined.



- If the results of a test plan review is unsatisfactory, list the test cases that should be added.

- If test cases requiring expert knowledge (for example, of the business or the system infrastructure) have not been identified, seek assistance of an expert.

H33

H34 Quality Are test item completion records verified to confirm whether they were really tested?

Check that test performance data such as execution date and results are recorded.

The method for confirming the test results must be clearly defined.

If a test result identifies a defect, a bug report form should be issued with the correspondence between the test item and the bug report clearly indicated.

- Test results report - If test performance data is not being recorded, make sure records are maintained in future tests.

- If there is no test performance record for the critical paths, ensure quality by adjustments such as re-testing.

H34

H35 Quality Has a plan been prepared for tests to be conducted under the production environment along with a list of items to be checked under the production environment?

Items that can only be tested under the production environment shall have been clearly defined.

If a test cannot be executed under the production environment, an alternative method must be considered.



- If a plan has not been prepared for testing under the production environment, prepare one.

- If a test cannot be executed under the production environment, consider an alternative method.

H35

H36 Quality Is a test plan clearly defined for confirming that performance requirements are met?

A performance test shall have been conducted ahead of time so that performance problems do not occur during the operation test or the actual operation.

While some performance tests can only be conducted under the production environment, the planning itself of performance tests will help to identify issues, examine countermeasures against overflow, and implement solutions.



- If a plan has not been prepared for testing performance, prepare one.

- If the target performance values are not clearly set, discuss with the customer to clarify and agree on a goal.

H36

H37 Quality Does the test plan clearly describe the testing of failure recovery and system operation?

Countermeasures for failures under actual operation shall have been clearly identified in advance.

Verify in advance whether the procedures and actions are feasible under actual operation.


- Operation design document

- Normal operation guide

- Failure operation guide

- If a plan has not been prepared for failure recovery and operation tests, prepare one.

H37

39




H32 Quality Have test verification procedures and methods been planned? Are they being used for test verification?

Test verification procedures and methods shall have been fully understood by project members.

It is important that a verification plan has been developed in advance for program code reviews and test results verifications, and that verifications are performed in accordance with the plan, not in an ad-hoc manner.

- Procedures manual


- If the procedures and methods are not being followed, instruct the personnel to comply.

H32

H33 Quality Are the test (integration test/qualification test) plans and their review results free of problems?

Findings from the reviews shall have been addressed. The test cases should be reviewed based on quality assessment indicators and target values that have been predefined.



- If the results of a test plan review is unsatisfactory, list the test cases that should be added.

- If test cases requiring expert knowledge (for example, of the business or the system infrastructure) have not been identified, seek assistance of an expert.

H33

H34 Quality Are test item completion records verified to confirm whether they were really tested?

Check that test performance data such as execution date and results are recorded.

The method for confirming the test results must be clearly defined.

If a test result identifies a defect, a bug report form should be issued with the correspondence between the test item and the bug report clearly indicated.

- Test results report - If test performance data is not being recorded, make sure records are maintained in future tests.

- If there is no test performance record for the critical paths, ensure quality by adjustments such as re-testing.

H34

H35 Quality Has a plan been prepared for tests to be conducted under the production environment along with a list of items to be checked under the production environment?

Items that can only be tested under the production environment shall have been clearly defined.

If a test cannot be executed under the production environment, an alternative method must be considered.



- If a plan has not been prepared for testing under the production environment, prepare one.

- If a test cannot be executed under the production environment, consider an alternative method.

H35

H36 Quality Is a test plan clearly defined for confirming that performance requirements are met?

A performance test shall have been conducted ahead of time so that performance problems do not occur during the operation test or the actual operation.

While some performance tests can only be conducted under the production environment, the planning itself of performance tests will help to identify issues, examine countermeasures against overflow, and implement solutions.



- If a plan has not been prepared for testing performance, prepare one.

- If the target performance values are not clearly set, discuss with the customer to clarify and agree on a goal.

H36

H37 Quality Does the test plan clearly describe the testing of failure recovery and system operation?

Countermeasures for failures under actual operation shall have been clearly identified in advance.

Verify in advance whether the procedures and actions are feasible under actual operation.


- Operation design document

- Normal operation guide

- Failure operation guide

- If a plan has not been prepared for failure recovery and operation tests, prepare one.

H37

40


endix



H38 Quality Are standards for documentation and coding clearly defined and practiced?

Furthermore, does the project leader, from time to time, make the developers aware of complying with such standard?

In-house standards or frameworks shall have been complied with, if any.

It is important that all stakeholders recognize, understand, and use the standards.

- Code review results

- Comparison between the document templates and the actual documents

- Comparison between the source codes and coding conventions

- Have the leader understand the importance of the coding standards, and make the leader seek compliance throughout the team.

- If there is no referential standard, develop one based on the standards for past projects.

H38

H39 Quality Are program change forms issued and reviewed in accordance with the program change form creation process defined during the planning phase?

Programs must not be modified based on assumptions or sole judgment.

In order to make changes to a program, the reason for the change and how the change may affect other parts must be clarified. Therefore, all changes to the programs must be managed using program change forms.

- Program change forms

- Promote the usage of program change forms.

- Before working on a source code, check whether a corresponding program change form has been issued in an appropriate manner.

H39

H40 Human Resources

Are personnel explicitly assigned to each task?

The project manager and all personnel in charge of tasks must share a common understanding of who does what.

- If a specific member is not assigned to a task, a problem will arise at the last moment in determining who should undertake the task.

- WBS tasks includes tasks that can be handled in-house and tasks that require collaboration with other companies.

- Many of the failed projects involve tasks that require collaboration.

- the assignment of responsibilities must be clear.


- Schedule chart

- WBS


- If some members are overloaded while others are idle, review the allocation of tasks or try to reallocate the workload from the busy members to the others.

H40

H41 Human Resources

When collaboration with other departments is required, do the stakeholders have a clear understanding of the scope of work, sharing of roles, and the personnel in charge?

The personnel in charge, for example of reporting progress and organizing meeting structures, shall have been clearly identified.

Relationship between the departments, also within the company, should be clearly defined.

- Progress reports - Identify the position of each stakeholder so as to clarify who is responsible for coordination.

H41

H42 Human Resources

Is there a specific person in charge of responding sincerely (making corrections if required) to suggestions and opinions provided by the quality assurance department and stakeholders not directly involved with the project?

Measures for addressing issues and problems shall have been planned.

Alternatively, a discussion shall have been held to solve the problems, or the stakeholders shall have accepted the strategy for resolving the issues and failures.

- A department that tries to cover up their mistake is considered harmful.

- Each company’s quality assurance department has its own way of taking responsibility. The project manager is responsible for assuring quality. However, if the project is large and the project manager is overloaded, dedicated quality assurance personnel should be assigned.

- According to PMBOK, quality management is a responsibility of the management.

- It is important to perceive that quality is equal to profit.

- Issue management table

- Investigate the reason for corrective actions not being taken, and apply countermeasures based on the significance and impact of the findings.

- For example, the reason may be an excessive workload required by the corrective action, or the lack of skills or resource.

- Examine and implement improvement measures based on the significance of the findings.

H42

H43 Communication Are the requirements specifications of the customer as well as key personnel of the customer sufficiently considered, and are reviews of the requirements specification performed as necessary?

Understanding of the customer’s requirements specifications shall have been checked to verify there are no problems in the understanding.

The requirements specifications provided by the customer should be confirmed in a face-to-face meeting. One shall not assume that he /she has understood the requirements specifications just through a written request to review the specifications.

- Review records


- Requirements management table

- Hold a meeting with the customer regarding customer requirements.

H43

41




H38 Quality Are standards for documentation and coding clearly defined and practiced?

Furthermore, does the project leader, from time to time, make the developers aware of complying with such standard?

In-house standards or frameworks shall have been complied with, if any.

It is important that all stakeholders recognize, understand, and use the standards.

- Code review results

- Comparison between the document templates and the actual documents

- Comparison between the source codes and coding conventions

- Have the leader understand the importance of the coding standards, and make the leader seek compliance throughout the team.

- If there is no referential standard, develop one based on the standards for past projects.

H38

H39 Quality Are program change forms issued and reviewed in accordance with the program change form creation process defined during the planning phase?

Programs must not be modified based on assumptions or sole judgment.

In order to make changes to a program, the reason for the change and how the change may affect other parts must be clarified. Therefore, all changes to the programs must be managed using program change forms.

- Program change forms

- Promote the usage of program change forms.

- Before working on a source code, check whether a corresponding program change form has been issued in an appropriate manner.

H39

H40 Human Resources

Are personnel explicitly assigned to each task?

The project manager and all personnel in charge of tasks must share a common understanding of who does what.

- If a specific member is not assigned to a task, a problem will arise at the last moment in determining who should undertake the task.

- WBS tasks includes tasks that can be handled in-house and tasks that require collaboration with other companies.

- Many of the failed projects involve tasks that require collaboration.

- the assignment of responsibilities must be clear.


- Schedule chart

- WBS


- If some members are overloaded while others are idle, review the allocation of tasks or try to reallocate the workload from the busy members to the others.

H40

H41 Human Resources

When collaboration with other departments is required, do the stakeholders have a clear understanding of the scope of work, sharing of roles, and the personnel in charge?

The personnel in charge, for example of reporting progress and organizing meeting structures, shall have been clearly identified.

Relationship between the departments, also within the company, should be clearly defined.

- Progress reports - Identify the position of each stakeholder so as to clarify who is responsible for coordination.

H41

H42 Human Resources

Is there a specific person in charge of responding sincerely (making corrections if required) to suggestions and opinions provided by the quality assurance department and stakeholders not directly involved with the project?

Measures for addressing issues and problems shall have been planned.

Alternatively, a discussion shall have been held to solve the problems, or the stakeholders shall have accepted the strategy for resolving the issues and failures.

- A department that tries to cover up their mistake is considered harmful.

- Each company’s quality assurance department has its own way of taking responsibility. The project manager is responsible for assuring quality. However, if the project is large and the project manager is overloaded, dedicated quality assurance personnel should be assigned.

- According to PMBOK, quality management is a responsibility of the management.

- It is important to perceive that quality is equal to profit.


- Investigate the reason for corrective actions not being taken, and apply countermeasures based on the significance and impact of the findings.

- For example, the reason may be an excessive workload required by the corrective action, or the lack of skills or resource.

- Examine and implement improvement measures based on the significance of the findings.

H42

H43 Communication Are the requirements specifications of the customer as well as key personnel of the customer sufficiently considered, and are reviews of the requirements specification performed as necessary?

Understanding of the customer’s requirements specifications shall have been checked to verify there are no problems in the understanding.

The requirements specifications provided by the customer should be confirmed in a face-to-face meeting. One shall not assume that he /she has understood the requirements specifications just through a written request to review the specifications.

- Review records


- Requirements management table

- Hold a meeting with the customer regarding customer requirements.

H43

42


endix



H44 Communication Aren’t there any ambiguities in the understanding by the members with regard to the work content and completion criteria?


- The conditions for unit test completion shall have been clearly defined.

- Testing shall not be carried out under impractical plans.

If there are ambiguities, an overall understanding of project progress cannot be obtained because the progress of each task cannot be ascertained or the understanding of each member is different. Quality problems may also result.

- Check the completion criteria for the work

- If understanding is ambiguous, clarify the conditions for which completion can be declared and inform workers.

- Also, establish a method to quantitatively measure the progress of work, and ensure that progress is reported quantitatively.

H44

H45 Communication Are there any problems in communication with the development teams and/or partner companies?

An appropriate communication plan shall have been established and shall have been followed.

Confirm that meeting structures have been defined and implemented.

- Check meeting frequency from the minutes of the meetings

- Interview partner companies

- When it is determined that communication is not smooth, establish an appropriate meeting structure.

- When there is a problem in affinity between the personnel in charge, as one-on-one communication may often produce problems, regularize meeting structures which include third-parties (such as project managers).

- If a third-level subcontractor (sub-sub contractor) communicates with the first-level subcontractor only via the second-level subcontractor (ordering party), preventing smooth communication, flatten the hierarchical structure and establish a meeting structure involving key personnel from each company.

H45

H46 Communication Are there review records, such as minutes, with the customer, and have they been properly agreed upon (approved)?

When various reviews are performed with the customer, meeting minutes must be prepared and agreed upon with the customer.

- Has a review plan with the customer been created?

- Has a review operation procedure been prepared?

- Are checking methods defined in documents such as the review operation procedure?

- The review records must be kept in the form of meeting minutes or e-mail.

- Minutes of meetings - If review minutes are not being created, create them.

- Make it a custom to conduct reviews of minutes in meetings, such as report meetings.

H46

H47 Communication Are project risks which should be shared among stakeholders such as the customer actually being shared?

The risk management table shall be reviewed among stakeholders.

It is necessary to clarify the assignment of responsibilities with the customer, and then discuss how to respond to delays in delivery. Otherwise, a “he said she said” scenario may result, and the project manager may end up undertaking all work responsibilities.



- In order to raise awareness with regard to risks, review the risk management table in meetings with stakeholders.

H47

H48 Communication When important matters are pointed out, are these matters informed to everyone to prevent them from recurrence?

For example, a mechanism and method to share information such as code inspection results and failure information shall have been defined.

It is important to share information since similar errors and mistakes may be found in other areas.


- Review records

- Record of keeping everyone informed (such as e-mail)

- Require everyone to inform of important suggestions pointed out such as code inspection results and failure information.

H48

43




H44 Communication Aren’t there any ambiguities in the understanding by the members with regard to the work content and completion criteria?


- The conditions for unit test completion shall have been clearly defined.

- Testing shall not be carried out under impractical plans.

If there are ambiguities, an overall understanding of project progress cannot be obtained because the progress of each task cannot be ascertained or the understanding of each member is different. Quality problems may also result.

- Check the completion criteria for the work

- If understanding is ambiguous, clarify the conditions for which completion can be declared and inform workers.

- Also, establish a method to quantitatively measure the progress of work, and ensure that progress is reported quantitatively.

H44

H45 Communication Are there any problems in communication with the development teams and/or partner companies?

An appropriate communication plan shall have been established and shall have been followed.

Confirm that meeting structures have been defined and implemented.

- Check meeting frequency from the minutes of the meetings

- Interview partner companies

- When it is determined that communication is not smooth, establish an appropriate meeting structure.

- When there is a problem in affinity between the personnel in charge, as one-on-one communication may often produce problems, regularize meeting structures which include third-parties (such as project managers).

- If a third-level subcontractor (sub-sub contractor) communicates with the first-level subcontractor only via the second-level subcontractor (ordering party), preventing smooth communication, flatten the hierarchical structure and establish a meeting structure involving key personnel from each company.

H45

H46 Communication Are there review records, such as minutes, with the customer, and have they been properly agreed upon (approved)?

When various reviews are performed with the customer, meeting minutes must be prepared and agreed upon with the customer.

- Has a review plan with the customer been created?

- Has a review operation procedure been prepared?

- Are checking methods defined in documents such as the review operation procedure?

- The review records must be kept in the form of meeting minutes or e-mail.

- Minutes of meetings - If review minutes are not being created, create them.

- Make it a custom to conduct reviews of minutes in meetings, such as report meetings.

H46

H47 Communication Are project risks which should be shared among stakeholders such as the customer actually being shared?

The risk management table shall be reviewed among stakeholders.

It is necessary to clarify the assignment of responsibilities with the customer, and then discuss how to respond to delays in delivery. Otherwise, a “he said she said” scenario may result, and the project manager may end up undertaking all work responsibilities.



- In order to raise awareness with regard to risks, review the risk management table in meetings with stakeholders.

H47

H48 Communication When important matters are pointed out, are these matters informed to everyone to prevent them from recurrence?

For example, a mechanism and method to share information such as code inspection results and failure information shall have been defined.

It is important to share information since similar errors and mistakes may be found in other areas.


- Review records


- Require everyone to inform of important suggestions pointed out such as code inspection results and failure information.

H48

44


endix



H49 Communication Are the performance test results approved by the customer?

The review results shall be described in meeting minutes and approved by the customer.

The performance test results should be reported to the customer to have them judge whether there are any problems.

- Minutes of meetings - If the customer does not agree with the results, seek points of compromise with the customer while promptly examining how to improve performance.

- In particular, if a performance problem relates to a critical path, monitor the problem periodically and control the improvement activities.

H49

H50 Communication Has a clear consensus on milestones been reached with the customer?

Details of the milestones shall have been recorded in the minutes of meetings and approved.

Milestones and events shall be recorded in the minutes of meetings, as well as summarized in Gantt charts so that stakeholders on both the customer and developer side can share a common understanding.


- Review records


- Reach a consensus on the details of milestones with the customer.

H50

H51 Communication Are there any gaps in understanding with the customer regarding schedule or tasks?

Customer agreement shall have been obtained. A review must be planned for explaining the schedule and work details to the customer.

- Minutes of meetings - Check and reach a consensus on the validity of schedule within the company, then review it with the customer.

- Reconsider the schedule when there are any problems.

H51

H52 Communication Are failure and correction information shared between the developers and test personnel?

Correction information shall have been made available also to test personnel by means such as failure information management tools. Test personnel shall know where to obtain the latest specifications.

If the information is not shared, work efficiency will decline because personnel may retest a program before a problem is corrected, or fail to test a program which has been corrected and requires retesting.

- Release announcement


- Establish rules regarding release announcements etc., and inform stakeholders of them.

H52

H53 Communication Are interactions with the customer concerning “specifications,” “costs,” and “delivery dates” done using documents?

Communication concerning specifications, cost, and delivery date must always be conducted in writing.

Oral promises tend to grow into issues of what was actually said or not.

- Report (documents)


Important matters in the communication with the customer shall be exchanged as documents.

H53

45




H49 Communication Are the performance test results approved by the customer?

The review results shall be described in meeting minutes and approved by the customer.

The performance test results should be reported to the customer to have them judge whether there are any problems.

- Minutes of meetings - If the customer does not agree with the results, seek points of compromise with the customer while promptly examining how to improve performance.

- In particular, if a performance problem relates to a critical path, monitor the problem periodically and control the improvement activities.

H49

H50 Communication Has a clear consensus on milestones been reached with the customer?

Details of the milestones shall have been recorded in the minutes of meetings and approved.

Milestones and events shall be recorded in the minutes of meetings, as well as summarized in Gantt charts so that stakeholders on both the customer and developer side can share a common understanding.


- Review records


- Reach a consensus on the details of milestones with the customer.

H50

H51 Communication Are there any gaps in understanding with the customer regarding schedule or tasks?

Customer agreement shall have been obtained. A review must be planned for explaining the schedule and work details to the customer.

- Minutes of meetings - Check and reach a consensus on the validity of schedule within the company, then review it with the customer.

- Reconsider the schedule when there are any problems.

H51

H52 Communication Are failure and correction information shared between the developers and test personnel?

Correction information shall have been made available also to test personnel by means such as failure information management tools. Test personnel shall know where to obtain the latest specifications.

If the information is not shared, work efficiency will decline because personnel may retest a program before a problem is corrected, or fail to test a program which has been corrected and requires retesting.

- Release announcement


- Establish rules regarding release announcements etc., and inform stakeholders of them.

H52

H53 Communication Are interactions with the customer concerning “specifications,” “costs,” and “delivery dates” done using documents?

Communication concerning specifications, cost, and delivery date must always be conducted in writing.

Oral promises tend to grow into issues of what was actually said or not.

- Report (documents)


Important matters in the communication with the customer shall be exchanged as documents.

H53

46


endix



H54 Risks What is the project size? Which of the following is the project size?

- More than 300 person-months

- 100 to 300 person-months

- less than 100 person-months

- Examine the system to see if it can be divided into subsystems which are loosely coupled (to minimize online connection) at a size of about 100 to 200 person-months.

- Examine the project to see if it can be divided into several phases.

- Perform an estimation using exponential models such as COCOMO or PUTNUM.

- Is there any difference between the original project size and the current size?

- Has the project size expanded?

- Even if there were no addition of functions, the estimation itself may not have been correct.

- Has a measure been implemented to address the difference in size?

- If the estimation was wrong, how should it be corrected?

- Has a specific approach to scale back the expanding requirements been finalized in coordination with the customer?

- Does the project size allow for the delivery date to be met?

- Materials showing subsystem division (basic design, etc.)


- It may be possible to divide the project into phases at the planning stage, but it will be more difficult to do so in the latter half of the project.

- In the case of a large project, implement a measure to reinforce the project management team as needed.

H54

H55 Risks Have risk analysis and prioritization according to level of impact, and risk countermeasures been sufficiently considered to mitigate threats?

For each risk, objective analysis shall have been performed, and measures considered.

- Risks do not necessarily have to be avoided and can be accepted as necessary.

- have there been any Risks that have become exposed?

- If Risks have been exposed, have countermeasures been taken?

- Countermeasures may include addition of personnel, adjustment of the delivery date, and reduction of functions.

- Is there any room to adjust the delivery date on function basis?

- Has it been possible to get the customer involved in the risk solution?


- If risk management is not performed, create a risk management table and consider measures to avoid risks.

H55

H56 Risks Are the details of risks clearly described? The risk management table must list the details of the risks. - For example, avoid descriptions that can only be understood by those who have attended meetings with the customer.

- Normally, it is too late if risk management is implemented in the downstream process.

- Identify the risk itself.

- Is the trigger of the risk clearly identified?

- Are countermeasures clearly identified?

- Is it possible to find dominant items after the crisis occurs?

- Most risks can be traced to human problems.


- Check whether there are any ambiguous or misleading descriptions in the risk management table. If there are, identify the personnel who entered the description and ascertain their meaning, then revise them into clear and understandable descriptions.

- Then, perform a review again, and share this information among stakeholders.

H56

47




H54 Risks What is the project size? Which of the following is the project size?

- More than 300 person-months

- 100 to 300 person-months

- less than 100 person-months

- Examine the system to see if it can be divided into subsystems which are loosely coupled (to minimize online connection) at a size of about 100 to 200 person-months.

- Examine the project to see if it can be divided into several phases.

- Perform an estimation using exponential models such as COCOMO or PUTNUM.

- Is there any difference between the original project size and the current size?

- Has the project size expanded?

- Even if there were no addition of functions, the estimation itself may not have been correct.

- Has a measure been implemented to address the difference in size?

- If the estimation was wrong, how should it be corrected?

- Has a specific approach to scale back the expanding requirements been finalized in coordination with the customer?

- Does the project size allow for the delivery date to be met?

- Materials showing subsystem division (basic design, etc.)


- It may be possible to divide the project into phases at the planning stage, but it will be more difficult to do so in the latter half of the project.

- In the case of a large project, implement a measure to reinforce the project management team as needed.

H54

H55 Risks Have risk analysis and prioritization according to level of impact, and risk countermeasures been sufficiently considered to mitigate threats?

For each risk, objective analysis shall have been performed, and measures considered.

- Risks do not necessarily have to be avoided and can be accepted as necessary.

- have there been any Risks that have become exposed?

- If Risks have been exposed, have countermeasures been taken?

- Countermeasures may include addition of personnel, adjustment of the delivery date, and reduction of functions.

- Is there any room to adjust the delivery date on function basis?

- Has it been possible to get the customer involved in the risk solution?


- If risk management is not performed, create a risk management table and consider measures to avoid risks.

H55

H56 Risks Are the details of risks clearly described? The risk management table must list the details of the risks. - For example, avoid descriptions that can only be understood by those who have attended meetings with the customer.

- Normally, it is too late if risk management is implemented in the downstream process.

- Identify the risk itself.

- Is the trigger of the risk clearly identified?

- Are countermeasures clearly identified?

- Is it possible to find dominant items after the crisis occurs?

- Most risks can be traced to human problems.


- Check whether there are any ambiguous or misleading descriptions in the risk management table. If there are, identify the personnel who entered the description and ascertain their meaning, then revise them into clear and understandable descriptions.

- Then, perform a review again, and share this information among stakeholders.

H56

48


endix



H57 Procurement Is an intermediate follow-up plan for the progress and quality of partner companies clear?

Regularization of progress meetings and clarification of quality criteria must have been planned and agreed upon among stakeholders.

- Is there a mentality that considers lump-sum contracts just require delivery of deliverables?

- Is there recognition that poor quality of deliverables will become a major problem?

- It is necessary to confirm the quality policy, quality control system, and specific quality management procedures implemented by the partner company.

- Have intermediate (periodical) checks been promised and performed?

- Progress management table

- Contract agreement (may include provisions with regard to intermediate deliverables to be provided)

- It is OK if there are no problems relating to the follow-up of partner companies. However, plan and perform follow-up if no follow-up has been provided.

H57

H58 Procurement Have the personnel responsible for project implementation (such as the project manager or project leader) considered, in association with experts, whether the scale and difficulty of work to be ordered to partner companies is appropriate?

The partner company must be examined to confirm that it possesses an appropriate number of personnel, level of technology and skills, past performance, and business know-how.

- Are the competence of partner companies known?

- If there was no choice but to order the job to an inappropriate partner company, check the actual work conditions in the workplace in addition to reviewing the situation at a regular weekly meeting.

- Do not accept reports without questioning them.

- Clarify the commitments of the contract when ordering the job.

- For example, if the number of deficiencies exceeds anticipated levels, implement quality improvement measures or make the partner company liable for the defects.

- Materials such as a project plan document and procurement plan document

- Review records of such plan documents

- Results of interviews with partner company managers

- If jobs had been ordered without adequate examination, such actions must be corrected.

- If such an examination has not been held, it is recommended to confirm with the partner company which accepted the order whether the job is of an appropriate size and difficulty.

H58

H59 Procurement Are the basis for orders to partner companies clear?

The reason why the job was entrusted to the partner must be clarified from the perspectives of business knowledge and development skills.

For example, if it is the first time that a partner company was entrusted the work, clarify the reasons for selecting this partner company.

- Procurement plan document

- Statement for partner company selection

- Clarify the grounds for placing the order, and take these results into account in risk management.

H59

H60 Procurement Are additional jobs requested to the partner company without making commitments to additional orders?

Check whether additional jobs are verbally requested to the partner without making commitments to additional orders.

Check whether there are personnel from the partner company who are working on a job before a contract has been concluded.

If there are no commitments, the partner company may be confused as to whether it can start working on the job, which may later develop into a contractual problem, which may cause a delay in the start of the job.

- Contract document such as a purchase order

- WBS

- Unofficial order notice

- Make a commitment to the additional job order, or request for appropriate activities by explaining the priority level of each job.

H60

H61 Customer Are the content and the conditions of the preliminary release clarified?

A “preliminary release” is a release made under a request etc. by the customer with constraints and prerequisites before the production release. The content and various conditions must be clearly specified.

The following must be agreed with the customer: “Purpose,” “constraints and prerequisites,” “functions,” “schedule,” “the correspondence between the formal specifications and the formal programs,” and so on.


- Preliminary release specification documents

- Make it clear to the customer that the release is a prototype.

- Since the purpose of a prototype is to test something, it would be meaningless if there is a gap between the test objective and the functions actually provided by the prototype.

- List the functions provided by the prototype and review the list with the customer to check for any problems.

H61

49




H57 Procurement Is an intermediate follow-up plan for the progress and quality of partner companies clear?

Regularization of progress meetings and clarification of quality criteria must have been planned and agreed upon among stakeholders.

- Is there a mentality that considers lump-sum contracts just require delivery of deliverables?

- Is there recognition that poor quality of deliverables will become a major problem?

- It is necessary to confirm the quality policy, quality control system, and specific quality management procedures implemented by the partner company.

- Have intermediate (periodical) checks been promised and performed?

- Progress management table

- Contract agreement (may include provisions with regard to intermediate deliverables to be provided)

- It is OK if there are no problems relating to the follow-up of partner companies. However, plan and perform follow-up if no follow-up has been provided.

H57

H58 Procurement Have the personnel responsible for project implementation (such as the project manager or project leader) considered, in association with experts, whether the scale and difficulty of work to be ordered to partner companies is appropriate?

The partner company must be examined to confirm that it possesses an appropriate number of personnel, level of technology and skills, past performance, and business know-how.

- Are the competence of partner companies known?

- If there was no choice but to order the job to an inappropriate partner company, check the actual work conditions in the workplace in addition to reviewing the situation at a regular weekly meeting.

- Do not accept reports without questioning them.

- Clarify the commitments of the contract when ordering the job.

- For example, if the number of deficiencies exceeds anticipated levels, implement quality improvement measures or make the partner company liable for the defects.

- Materials such as a project plan document and procurement plan document

- Review records of such plan documents

- Results of interviews with partner company managers

- If jobs had been ordered without adequate examination, such actions must be corrected.

- If such an examination has not been held, it is recommended to confirm with the partner company which accepted the order whether the job is of an appropriate size and difficulty.

H58

H59 Procurement Are the basis for orders to partner companies clear?

The reason why the job was entrusted to the partner must be clarified from the perspectives of business knowledge and development skills.

For example, if it is the first time that a partner company was entrusted the work, clarify the reasons for selecting this partner company.

- Procurement plan document

- Statement for partner company selection

- Clarify the grounds for placing the order, and take these results into account in risk management.

H59

H60 Procurement Are additional jobs requested to the partner company without making commitments to additional orders?

Check whether additional jobs are verbally requested to the partner without making commitments to additional orders.

Check whether there are personnel from the partner company who are working on a job before a contract has been concluded.

If there are no commitments, the partner company may be confused as to whether it can start working on the job, which may later develop into a contractual problem, which may cause a delay in the start of the job.

- Contract document such as a purchase order

- WBS

- Unofficial order notice

- Make a commitment to the additional job order, or request for appropriate activities by explaining the priority level of each job.

H60

H61 Customer Are the content and the conditions of the preliminary release clarified?

A “preliminary release” is a release made under a request etc. by the customer with constraints and prerequisites before the production release. The content and various conditions must be clearly specified.

The following must be agreed with the customer: “Purpose,” “constraints and prerequisites,” “functions,” “schedule,” “the correspondence between the formal specifications and the formal programs,” and so on.


- Preliminary release specification documents

- Make it clear to the customer that the release is a prototype.

- Since the purpose of a prototype is to test something, it would be meaningless if there is a gap between the test objective and the functions actually provided by the prototype.

- List the functions provided by the prototype and review the list with the customer to check for any problems.

H61

50


endix



H62 Customer Does the customer have sufficient ability to carry the project forward?

The customer’s ability to carry the project forward refers to the customer’s decisiveness and decision-making speed, which must be at a sufficient level.

Are activities, the speed of decision-making, and task management implemented to reach a consensus among project members?

- Interviews

- Steering committee


- Ask the customer to prepare an occasion to check for project consensus within the customer’s company.

- Also, document the customer’s tasks and request the key personnel of the team conducting the actual work to have the tasks performed thoroughly.

- Suggest a customer project manager supporting contract to be executed by an organizational framework separate from development.

H62

H63 Customer Are key personnel on the customer side identified?

For example, the following shall be understood:

- The relationship between the system department and user department on the customer side is understood.

- The department having the authority to determine specifications, budget, and commitment to management is identified.

- The personnel having decision-making authority and influence are identified.

As part of risk management, list the names of individuals to consult with if a problem occurs.

The relationship between key personnel in the responsible department must be identified because key personnel may not necessarily have decision-making authority.

- Interviews



- Confirm the customer’s organizational framework and ask the key personnel concerned with the project to attend the regular meetings.

H63

H64 Customer Does the RFP provide adequate information? For example, it must clearly provide information such as the scope of development, contract conditions, education and training, acceptance validation, and schedule.

If the RFP was created by a customer company, it must clearly describe matters such as the definitions of the project and output as well as hot to obtain requirements.

If the RFP was created by a third-party such as a consulting company, the customer should thoroughly understand its content.

- RFP - Request the customer to provide any missing information.

- List the missing content and ask for permission to conduct an investigation.

H64

H65 Customer If there is a need to assure current system functions, are current documents sufficiently maintained?

If an existing function must be guaranteed, the specifications for this function must be provided by the customer as a written document.

The scope of requests to “guarantee existing functions” may often be ambiguously defined.

- Specifications for existing functions

- If a document on existing functions is not available, explain to the customer that these functions cannot be guaranteed and obtain the customer’s consent.

- Even when a document on existing functions is available, if some of the functions cannot be provided due to a migration problem to a new system, obtain the customer’s agreement.

- Reach an agreement with the customer on the completion criteria for the validation of existing functions.

H65

H66 Customer If business restructuring is included, has feasibility been considered during planning?

The positioning and expected role of the system in business restructuring must be understood.

It is important that feasibility of the details of business restructuring has been verified and a consensus has been reached with the customer.



- If not considered yet, even if already in the latter half of the project, consider the feasibility of business restructuring and manage it as a risk if there may be a problem.

- Since field-level adjustments may be required, the customer shall share understanding with the customer.

H66

51




H62 Customer Does the customer have sufficient ability to carry the project forward?

The customer’s ability to carry the project forward refers to the customer’s decisiveness and decision-making speed, which must be at a sufficient level.

Are activities, the speed of decision-making, and task management implemented to reach a consensus among project members?

- Interviews



- Ask the customer to prepare an occasion to check for project consensus within the customer’s company.

- Also, document the customer’s tasks and request the key personnel of the team conducting the actual work to have the tasks performed thoroughly.

- Suggest a customer project manager supporting contract to be executed by an organizational framework separate from development.

H62

H63 Customer Are key personnel on the customer side identified?

For example, the following shall be understood:

- The relationship between the system department and user department on the customer side is understood.

- The department having the authority to determine specifications, budget, and commitment to management is identified.

- The personnel having decision-making authority and influence are identified.

As part of risk management, list the names of individuals to consult with if a problem occurs.

The relationship between key personnel in the responsible department must be identified because key personnel may not necessarily have decision-making authority.

- Interviews



- Confirm the customer’s organizational framework and ask the key personnel concerned with the project to attend the regular meetings.

H63

H64 Customer Does the RFP provide adequate information? For example, it must clearly provide information such as the scope of development, contract conditions, education and training, acceptance validation, and schedule.

If the RFP was created by a customer company, it must clearly describe matters such as the definitions of the project and output as well as hot to obtain requirements.

If the RFP was created by a third-party such as a consulting company, the customer should thoroughly understand its content.

- RFP - Request the customer to provide any missing information.

- List the missing content and ask for permission to conduct an investigation.

H64

H65 Customer If there is a need to assure current system functions, are current documents sufficiently maintained?

If an existing function must be guaranteed, the specifications for this function must be provided by the customer as a written document.

The scope of requests to “guarantee existing functions” may often be ambiguously defined.

- Specifications for existing functions

- If a document on existing functions is not available, explain to the customer that these functions cannot be guaranteed and obtain the customer’s consent.

- Even when a document on existing functions is available, if some of the functions cannot be provided due to a migration problem to a new system, obtain the customer’s agreement.

- Reach an agreement with the customer on the completion criteria for the validation of existing functions.

H65

H66 Customer If business restructuring is included, has feasibility been considered during planning?

The positioning and expected role of the system in business restructuring must be understood.

It is important that feasibility of the details of business restructuring has been verified and a consensus has been reached with the customer.



- If not considered yet, even if already in the latter half of the project, consider the feasibility of business restructuring and manage it as a risk if there may be a problem.

- Since field-level adjustments may be required, the customer shall share understanding with the customer.

H66

52


endix



H67 Technology Has the use of commercially-available products and other products in every system development process been reviewed from a technical perspective with involvement by experts?

Findings from the reviews shall have been addressed. Some package products are poor in quality and cannot be modified easily as their contents are provided as a black box. For this reason, poor product quality leads to performance problems and insufficient functionality.


- If a review has not been completed, there may be a problem with the externally procured product.

- Review the product again from the perspective of risk analysis, even it is in the latter half of the project.

- Establish a channel with the product vendor.

- Alternative measures shall also be considered, including product replacement.

- Reach an agreement with the customer on the cost for such measures.

H67

H68 Technology Are appropriate measures taken to utilize new or unfamiliar technologies?

Schedule shall be laid out to perform evaluation based on product materials, assess internal and external achievements, and evaluate the product on actual machines before starting the development.

- Attention must be paid to both overestimation and underestimation. Check whether objective judgment is made.

- It must be noted that it is possible that the evaluators themselves do not understand the new technologies.

- Are there evaluation and support organizations within the company?


- If there are problems in handling new technologies or unexperienced technologies within the development team, take measures such as assigning experts.

- Reflect in the schedule the fact that they are new technologies or unexperienced technologies.

H68

H69 Technology Has the method of migration / switchover (for example, parallel operation of the existing system, migration from another system) been extensively examined?

The feasibility of the migration method shall have been checked based on an expert’s review.

It is important to consider, as much as possible, the simplest of migration / switchover method.

Clearly identify the assignment of roles with the customer, and consider using appropriate contract formats as necessary.




- If the migration method is difficult to apply, identify the risks, discuss them with the customer, and consider a phased migration plan.

- Consider a recovery method to handle unanticipated situations and create a plan to mitigate risks of failure in migration.

H69

H70 Technology Has the architecture design been sufficiently examined, such as to ensure response and eliminate no-response faults?

For example, measures against thread exhaustion when there is massive access, or duplexing with clusters to prepare for failures must be considered.

Are non-functional requirements such as ensuring actual response and eliminating non-response faults examined from both software and hardware perspectives, without placing too much emphasis on the design of application functions?

- Basic design document

- If architecture design has not been performed, check to see if there are any problems in the architecture.

- Take measures such as asking for help to an outside vendor or consultant with knowledge and experience in the operation of system infrastructure technology.

H70

53




H67 Technology Has the use of commercially-available products and other products in every system development process been reviewed from a technical perspective with involvement by experts?

Findings from the reviews shall have been addressed. Some package products are poor in quality and cannot be modified easily as their contents are provided as a black box. For this reason, poor product quality leads to performance problems and insufficient functionality.


- If a review has not been completed, there may be a problem with the externally procured product.

- Review the product again from the perspective of risk analysis, even it is in the latter half of the project.

- Establish a channel with the product vendor.

- Alternative measures shall also be considered, including product replacement.

- Reach an agreement with the customer on the cost for such measures.

H67

H68 Technology Are appropriate measures taken to utilize new or unfamiliar technologies?

Schedule shall be laid out to perform evaluation based on product materials, assess internal and external achievements, and evaluate the product on actual machines before starting the development.

- Attention must be paid to both overestimation and underestimation. Check whether objective judgment is made.

- It must be noted that it is possible that the evaluators themselves do not understand the new technologies.

- Are there evaluation and support organizations within the company?


- If there are problems in handling new technologies or unexperienced technologies within the development team, take measures such as assigning experts.

- Reflect in the schedule the fact that they are new technologies or unexperienced technologies.

H68

H69 Technology Has the method of migration / switchover (for example, parallel operation of the existing system, migration from another system) been extensively examined?

The feasibility of the migration method shall have been checked based on an expert’s review.

It is important to consider, as much as possible, the simplest of migration / switchover method.

Clearly identify the assignment of roles with the customer, and consider using appropriate contract formats as necessary.




- If the migration method is difficult to apply, identify the risks, discuss them with the customer, and consider a phased migration plan.

- Consider a recovery method to handle unanticipated situations and create a plan to mitigate risks of failure in migration.

H69

H70 Technology Has the architecture design been sufficiently examined, such as to ensure response and eliminate no-response faults?

For example, measures against thread exhaustion when there is massive access, or duplexing with clusters to prepare for failures must be considered.

Are non-functional requirements such as ensuring actual response and eliminating non-response faults examined from both software and hardware perspectives, without placing too much emphasis on the design of application functions?


- If architecture design has not been performed, check to see if there are any problems in the architecture.

- Take measures such as asking for help to an outside vendor or consultant with knowledge and experience in the operation of system infrastructure technology.

H70

54


endix



H71 Technology Are assumptions on traffic volume and patterns made appropriately?

Are the measures to be taken clear?

Estimate the load requirements of the system as a whole and during peaks such as the number of clients, amount of traffic in applications, and traffic used by middleware.

- Discuss the worst case traffic volume with the customer.

- Consider blocking off in the event of excess incoming volume data.

- Clarify the scope of responsibilities.

- Requirements definition document


- Collect information about current operation from the customer to derive expected amount of traffic and its pattern.

- In a totally new project where there is no operational experience, it is difficult to determine the amount of traffic and its pattern. So prepare a prototype to measure it or measure the traffic by performing a preliminary release before going into production.

- If the above is also difficult, define the amount of traffic and its pattern after making the assumptions clear, and reach a consensus with the customer.

H71

H72 Organization Does the project manager have appropriate experience to handle the size and characteristics of the project, or are organizational support measures (including the personnel responsible for the project) available if he/she lacks the required experience?

Both the skills of the project manager and the actual size / complexity of the project must be adequately examined.

- Is the organization a supportive-type PMO instead of a management-type PMO?

- Is there a culture of providing support before a project collapses?

- Is there a supporting organization on the management side?

- The project may start out well, but go wrong later on.


- Give proper project manager training to project managers.

- Assign coaches.

- Review the project plan with experts.

- Request cooperation from a PMO.

H72

H73 Organization Are the organization’s responsibilities / authorities clarified and does the organization have sufficient functions (management office, etc.)?

The organization’s responsibilities, authorities, meeting structures, and goals shall be defined for each process.

Management-level meeting structures shall be prepared to be able to cope with unexpected problems.

Efforts made to improve sense of participation in meeting structures and to facilitate follow-up on measures by clarifying the scope of responsibility of each department.


- Meeting structures

- Review organization chart and clearly declare the scope of responsibilities in documents such as the project plan.

H73

H74 Organization Are partner companies constituting a hierarchical structure, making scope of responsibilities ambiguous?

Aren't there problems as a result of this multilevel relationship (such as scope of responsibility and communication)?

Role assignment and responsibilities must be clear. The responsible party may often become unclear in a hierarchical organization.


- Role assignment matrix

- Interviews

- Establish a meeting structure and gather key personnel from each company.

- At the meeting structure, confirm and review the scope of responsibilities of each company and have regular communication.

H74

H75 Basic Conduct/Action

Is system backup performed when modifying the system?


- Backup the configuration files before making modifications.

- Backup original modules before replacing them with new modules.

- Backup of the entire system shall be taken.

- Backup of data shall be taken.

A mechanism to quickly restore the system environment must be implemented, such as replacement of modules or configuration files.

- Release procedure documents etc.

Clearly define procedures such as prohibiting overwriting files and performing database backup before modifying the database.

H75


Regarding the meaning of the terms used in the project, are efforts made to minimize perception gaps between the customer and project members by means such as creating and managing a list of terms?

A glossary must be created and shared among stakeholders. Misunderstanding of used terms may result in design error or work error.

- Glossary - Create a glossary and have it reviewed by the customer and project members.

H76

55




H71 Technology Are assumptions on traffic volume and patterns made appropriately?

Are the measures to be taken clear?

Estimate the load requirements of the system as a whole and during peaks such as the number of clients, amount of traffic in applications, and traffic used by middleware.

- Discuss the worst case traffic volume with the customer.

- Consider blocking off in the event of excess incoming volume data.

- Clarify the scope of responsibilities.

- Requirements definition document


- Collect information about current operation from the customer to derive expected amount of traffic and its pattern.

- In a totally new project where there is no operational experience, it is difficult to determine the amount of traffic and its pattern. So prepare a prototype to measure it or measure the traffic by performing a preliminary release before going into production.

- If the above is also difficult, define the amount of traffic and its pattern after making the assumptions clear, and reach a consensus with the customer.

H71

H72 Organization Does the project manager have appropriate experience to handle the size and characteristics of the project, or are organizational support measures (including the personnel responsible for the project) available if he/she lacks the required experience?

Both the skills of the project manager and the actual size / complexity of the project must be adequately examined.

- Is the organization a supportive-type PMO instead of a management-type PMO?

- Is there a culture of providing support before a project collapses?

- Is there a supporting organization on the management side?

- The project may start out well, but go wrong later on.


- Give proper project manager training to project managers.

- Assign coaches.

- Review the project plan with experts.

- Request cooperation from a PMO.

H72

H73 Organization Are the organization’s responsibilities / authorities clarified and does the organization have sufficient functions (management office, etc.)?

The organization’s responsibilities, authorities, meeting structures, and goals shall be defined for each process.

Management-level meeting structures shall be prepared to be able to cope with unexpected problems.

Efforts made to improve sense of participation in meeting structures and to facilitate follow-up on measures by clarifying the scope of responsibility of each department.


- Meeting structures

- Review organization chart and clearly declare the scope of responsibilities in documents such as the project plan.

H73

H74 Organization Are partner companies constituting a hierarchical structure, making scope of responsibilities ambiguous?

Aren't there problems as a result of this multilevel relationship (such as scope of responsibility and communication)?

Role assignment and responsibilities must be clear. The responsible party may often become unclear in a hierarchical organization.


- Role assignment matrix

- Interviews

- Establish a meeting structure and gather key personnel from each company.

- At the meeting structure, confirm and review the scope of responsibilities of each company and have regular communication.

H74


Is system backup performed when modifying the system?


- Backup the configuration files before making modifications.

- Backup original modules before replacing them with new modules.

- Backup of the entire system shall be taken.

- Backup of data shall be taken.

A mechanism to quickly restore the system environment must be implemented, such as replacement of modules or configuration files.

- Release procedure documents etc.

Clearly define procedures such as prohibiting overwriting files and performing database backup before modifying the database.

H75


Regarding the meaning of the terms used in the project, are efforts made to minimize perception gaps between the customer and project members by means such as creating and managing a list of terms?

A glossary must be created and shared among stakeholders. Misunderstanding of used terms may result in design error or work error.

- Glossary - Create a glossary and have it reviewed by the customer and project members.

H76

56


endix




Are documents (such as meeting minutes and bug report forms) written using appropriate language?

The content of the meeting minutes, bug report forms, etc. must be verified.

Using appropriate wording is a prerequisite for understanding project status.

Especially considering the personnel who read the meeting minutes or bug report forms, it is important to describe them using correct wording.


- Bug report forms

- If there are any problems in the meeting minutes or bug report forms, provide personnel with educational direction.

- In addition, publicize and share examples of the correction of bad cases to promote awareness.

H77

H78 Motivation Are the labor conditions (hours of labor, work environment) of employees bad?

Decisions shall be made based on information such as work hours, and checks shall be performed to see whether there are people remaining in the office.

If there is a problem, perform root cause analysis and take appropriate actions (such as adding more resource or reexamining resource allocation).

- Work hours management table

- Work site inspection

- In order to improve labor conditions, implement measures such as increasing the number of personnel, rethinking of personnel deployment, and improving the work environment.

H78

H79 Motivation Is there a lot of personnel turnover?

(due to poor physical or mental shape or for personal reasons)

Trends in personnel absence and turnover shall be checked. If there are a lot of turnover, it may be due to distrust or a sense of crisis with the project.

- Organization chart - If the reason for high rate of turnover is distrust or a sense of crisis toward the project, announcing milestones (end points) for the project may help.

- Building a sense of unity or fellowship in the project may help decrease turnover.

- Assign personnel to support the project manager for enhancing the organization.

H79

H80 Motivation Are not the members lacking commitment to fulfill their responsibilities within their capacity?

Are they not indifferent to issues outside their scope of responsibility?

The motivation of all project members must be observed, not only direct subordinates.

Are project members working with a sense of responsibility judging from their everyday behavior such as their opinions, the number of opinions voiced, and the quality of deliverables?

- Minutes of meetings such as a progress meeting

- Reprts (documents)

- Interviews of personnel in charge

- Require each member to personally report the progress of their tasks.

- Provide countermeasures or specific instructions on reported issues.

H80

H81 Motivation How many project members are expressing distrust towards the project?

Appropriate communication shall be established with project members.

A strong sense of distrust will negatively impact the morale of the entire project.

- E-mail

- Interviews


H81

H82 Motivation Is each project member encouraged to set personal growth goals in this project?

- The project manager shall have discussed personal growth targets with each project members.

Whether project members have set growth targets or not will significantly affect their motivation.

- Personnel development targets

- Define development targets for each member in the project and reach a consensus with each member.

H82

H83 Issue Management

When a managed issue had not been resolved by the due date, are any corrective actions performed?

The issues must be monitored with the issue management table. If an issue has not been resolved long after the due date, a significant problem may exist.

The reasons for delay may include reasons such as nobody is appointed to the issue, the member in charge is not aware of the issue, or the issue is too large for him/her to identify a specific solution.


- Trace the reasons why corrective actions were not taken.

- In particular, if the issue is too large and it is not possible to judge who should take charge, key personnel will need to be gathered to discuss measures.

- if the issue is large, escalate the matter and implement measures.

H83

57





Are documents (such as meeting minutes and bug report forms) written using appropriate language?

The content of the meeting minutes, bug report forms, etc. must be verified.

Using appropriate wording is a prerequisite for understanding project status.

Especially considering the personnel who read the meeting minutes or bug report forms, it is important to describe them using correct wording.


- Bug report forms

- If there are any problems in the meeting minutes or bug report forms, provide personnel with educational direction.

- In addition, publicize and share examples of the correction of bad cases to promote awareness.

H77

H78 Motivation Are the labor conditions (hours of labor, work environment) of employees bad?

Decisions shall be made based on information such as work hours, and checks shall be performed to see whether there are people remaining in the office.

If there is a problem, perform root cause analysis and take appropriate actions (such as adding more resource or reexamining resource allocation).

- Work hours management table

- Work site inspection

- In order to improve labor conditions, implement measures such as increasing the number of personnel, rethinking of personnel deployment, and improving the work environment.

H78

H79 Motivation Is there a lot of personnel turnover?

(due to poor physical or mental shape or for personal reasons)

Trends in personnel absence and turnover shall be checked. If there are a lot of turnover, it may be due to distrust or a sense of crisis with the project.

- Organization chart - If the reason for high rate of turnover is distrust or a sense of crisis toward the project, announcing milestones (end points) for the project may help.

- Building a sense of unity or fellowship in the project may help decrease turnover.


H79

H80 Motivation Are not the members lacking commitment to fulfill their responsibilities within their capacity?

Are they not indifferent to issues outside their scope of responsibility?

The motivation of all project members must be observed, not only direct subordinates.

Are project members working with a sense of responsibility judging from their everyday behavior such as their opinions, the number of opinions voiced, and the quality of deliverables?

- Minutes of meetings such as a progress meeting

- Reprts (documents)

- Interviews of personnel in charge

- Require each member to personally report the progress of their tasks.

- Provide countermeasures or specific instructions on reported issues.

H80

H81 Motivation How many project members are expressing distrust towards the project?

Appropriate communication shall be established with project members.

A strong sense of distrust will negatively impact the morale of the entire project.

- E-mail

- Interviews


H81

H82 Motivation Is each project member encouraged to set personal growth goals in this project?

- The project manager shall have discussed personal growth targets with each project members.

Whether project members have set growth targets or not will significantly affect their motivation.

- Personnel development targets

- Define development targets for each member in the project and reach a consensus with each member.

H82


When a managed issue had not been resolved by the due date, are any corrective actions performed?

The issues must be monitored with the issue management table. If an issue has not been resolved long after the due date, a significant problem may exist.

The reasons for delay may include reasons such as nobody is appointed to the issue, the member in charge is not aware of the issue, or the issue is too large for him/her to identify a specific solution.


- Trace the reasons why corrective actions were not taken.

- In particular, if the issue is too large and it is not possible to judge who should take charge, key personnel will need to be gathered to discuss measures.

- if the issue is large, escalate the matter and implement measures.

H83

58


endix




Has an agreement been reached with the customer on the managed issues?

Perform issue reviews with the customer, prepare meeting minutes and obtain consent on the minutes.

- In issue management, agreement with the customer is important. For example, if the customer considers a matter as an issue and the developer does not, it may result in a significant problem in the future.

- A gap in perception regarding the level of importance and urgency will also cause problems.


- Review records


- Check the issue management table and review it with the customer.

H84


Has a solution to an issue been selected from among multiple solutions in order of priority based on clear decision criteria (advantages / disadvantages)?


- One must think of multiple solutions and select the better solution through logical comparison.

- A solution must not be just a preconception.

- Decisions based only on experience or intuition will lead to significant miscalculations.

- Decisions require a basis and criteria.

- It is important that options that provide basis for a judgment be thoroughly checked while always being conscious of making better decisions.


- Make a judgment after clarifying the basis for the decision.

- Record the basis for the decision and the order of priority in the issue management table.

H85

59





Has an agreement been reached with the customer on the managed issues?

Perform issue reviews with the customer, prepare meeting minutes and obtain consent on the minutes.

- In issue management, agreement with the customer is important. For example, if the customer considers a matter as an issue and the developer does not, it may result in a significant problem in the future.

- A gap in perception regarding the level of importance and urgency will also cause problems.


- Review records


- Check the issue management table and review it with the customer.

H84


Has a solution to an issue been selected from among multiple solutions in order of priority based on clear decision criteria (advantages / disadvantages)?


- One must think of multiple solutions and select the better solution through logical comparison.

- A solution must not be just a preconception.

- Decisions based only on experience or intuition will lead to significant miscalculations.

- Decisions require a basis and criteria.

- It is important that options that provide basis for a judgment be thoroughly checked while always being conscious of making better decisions.


- Make a judgment after clarifying the basis for the decision.

- Record the basis for the decision and the order of priority in the issue management table.

H85

60

3. Project Trouble Events and Countermeasures (Summary of Problem Projects)App

endix

No. Title of Event Knowledge Area

Event

Cause Countermeasures Temporary Measures

Reasons and how the project ended with issues

Measures that were taken during the project in resolving the issues

Measures that were taken at that point

Reoccurrence Prevention Measures (Lessons Learned)

Measures that should have been taken in upstream / midstream process

1. Key Systems Engineer (SE) Left the ProjectIntegration

The key application design SE, who was a member of the partner company, left the project just before the integration test due to being hospitalized. The detailed design and source code that the SE was in charge were nowhere to be found, including the SE’s PC or the server.


As the application design leader, the SE has regularly filed the team’s progress reports, no project progress issue was encountered up to and including the unit test process, and questions on project progress were appropriately answered. As such, deliverables were not verified, trusting the SE. Furthermore, there were arguments with the order ing par ty on whether the contracts of the ent i re project ( the secondary and te r t i a ry con t rac to r agreements) were considered to be a lump-sum contract or a time and material contract. A report was received from the team in an attempt to verify which, but it was ambiguous.

• Even in the case of time and material contracted work, it was considered that managemen t respons ib i l i t y ove r personnel lied with the ordered party. A support group was organized to finish the work that the relevant leader was responsible for within a short term.

• A full-time employee of the ordered party was assigned as leader, and document management was shifted to a centralized document management structure using a document server. The integration test of this part was rescheduled to the latter half of the project.

• Add personnel• Configuration management

(Documentation)• Reschedule


• Establish configuration management rules

• Clarify work division

61


2. Delay in Requirements Specifications FreezeIntegration

A three-month project was planned with a month each for establishing the requirements specifications, developing and testing, but the project, including system design review, was actually shortened to two months since meetings for establishing the requirements definition during the first month could not be held with the ordering party.


An order for application development was r e c e i v e d w i t h d e v e l o p m e n t t o b e completed in three months. Requirements specifications were to be finalized during interviews with the company that would be selling the application, but the order placement from the sales company was delayed. Specification decision meetings with the sales company did not start until the second month of the project, and there was no cho ice bu t t o sho r ten the development phase by a month.

• As i t was a small system, a set of prototypes was prepared in a month based on the expected operation of this system.

• In the second month, requirements specifications were finalized for each part at the customer’s site. Addition, changes and deletion requests were responded w h i l e d e v e l o p i n g a n d f i n a l i z i n g requirements specifications.

• Reschedule (Release period)• Negotiate function reduction


• Conduct feasibility study (verification of feasibility)

• Customer involvement based on contracts (Work with customers until requirement specifications are finalized.)

• Subdivide the contract into split contracts

• Conclude a time and material contract

3. Progress Delays of Specific Functions and Quality DeteriorationIntegration

An important business rule was not documented for a functional change project, and the source code could not be changed, resulting in quality deterioration. The original lead systems engineer had been transferred to another division.


Since the source code was spaghetti code (uns t ruc tu red ) , ma in tenance was impossible. Even rewriting would not have been possible since the business rules were not documented.

• Pulled the original lead system engineer back for a month to do the coding and testing with a new member.

• Business rules were documented on each occasion (source code was not rewritten).

• The relevant functions were postponed to the second release.

• Rescheduling• Split releases• Get hold of personnel who is

familiar with the application• Promote skill transfer through group

work• Inspect the extent of impact on the

program


• Ensure thorough documentation• Thorough reviews• Define and practice coding

standards

62

4. Failure in One-Time Delivery during Acceptance Inspection PeriodIntegration

Customer requested preliminary delivery which was not mentioned in the contract, to be due three months in advance for advance verification and training. However, delays in progress resulted in not being able to fulfill one-time delivery for all application components.


The promise with the customer was not shared among all project members, and the project manager had solely judged that acceptance inspection and training could be accommodated by providing the systems as they were completed on an application-by-application basis.

• Explained the project progress to the customer, and requested them to perform acceptance inspection and training based on the development status.

• Submitted a written apology to senior managements of customer, and had an experienced project manager to negotiate with the customer.

• The deve lopmen t o rgan i za t i ona l framework was reinforced, and a quality control team was established to handle delays and quality problems.

• Split releases• Replace project manager• Practice problem-solving

management• Quality measurement


• Communicate with customer• Clarify task assignments• Share information within the

project group

5. Boundary between the Package and Customized Source Code Was UnclearIntegration

Source code was to be disclosed to customer on delivery but the scope became unclear.


A development order was received for the customization of a package product. While the source code of the package was to be kept undisclosed, customized parts were to be disclosed to the customer under the agreement. However, the disclosure scope became unclear after the core function was being customized due to request from customer.

Defined that the modified sections are to be considered as customized code, and made adjustments with the customer to disclose those code on a program-by-program basis. The core source was prevented from being disclosed.

• Adjust with customer (contract details)


• Advance confirmation of contract (regarding code disclosure on changes in core sections)

• Clarify program structure and scope of rights (copyright and ownership of code)


endix

63


6. Frequent Specification Changes during Test ProcessScope

Frequent design changes, such as screen design, application specifications and output forms, occurred during the qualification test process, resulting in delivery delay.


In the development of a new web-based product, it was difficult to accurately and adequately assess the technical aspects, performance and restrictions assessments. There were excessive review items for user requirements in the application specifications causing delay in specification freeze, and specification changes had to be made during test process.

• Product characteristics were identified after creating a prototype and deviation from the predefined plan was determined to come up with countermeasures.

• With regard to customer requirements, systematization scope with consideration on cost, delivery date and quality was identified and consented with customer.

• Fixed the period for specification freeze.

• Differentiate bugs against specification changes

• Prioritize response actions• Clarify the specification freeze

period


• Split contracts• Clarify specification freeze period• Perform a thorough preliminary

survey (such as restrictions and functions)

7. Insufficient System Operation Comprehension and Inadequate Specification Explanation from Development Side

Scope

After the service was launched, garbled characters were encountered in the webmail client, and the customer pointed out that it was unusable.


The supported email header structures and encoding methods are different depending on the email client. Based on the prerequisites of the functions to be developed, the requirements specification was defined to limited types of email header structures and encoding methods. However, since the customer receives emails from various business partners, the email client that was developed with provided specifications, was unusable. Even though the customer had signed off the specification, the customer in fact did not understand the significance.

• As the customer had signed off the specification, the countermeasure of this issue was implemented as a specification change. Implementation period was set during the parallel operation with the old system.

• The developer verified the implementation and operation under the expected user requirements of the customer by sending and receiving emails, which include machine dependent characters, via all versions of any obtainable free email software.

• Differentiate bugs against specification changes

• Parallel operation with old system• Identify problems via testing in

various software


• Thorough investigation of existing system

• Thorough investigation of existing system operation

• Sufficient implementation of acceptance tests

64

8. Customer’s Insufficient Functional Requirements Explanation Capability and Development Side’s Failure in Understanding Customer’s Requirements

Scope

A lot of complaints were drawn after the system was presented to customer for operation test.


T h e i m p l e m e n t e d f u n c t i o n s w e r e described in the specification but those not implemented were not described. Since the customer was expecting that the specification would be based on the various functions of the old system, a significant difference in understanding arose.

• As the customer had signed off the specification, the countermeasure of this issue was implemented as a specification change.

• Of the old functions, those that were considered useful were listed based on results from interviews with the users, and from the list, those that gained consensus from the customer in terms of cost and development schedule were implemented.

• Negotiate on specification change• Prioritize the additional functions

to be implemented


• Clarify role allocation (individuals involved in finalizing the specification)

9. Design Leader, Who Was the Center of Application Design, Collapsed and LeftScope

The designer leader, who was familiar with the application and played a central role in the project, collapsed due to overwork.


All issues were routed to the design leader. The design leader was not so good at distributing work to subordinates.

• When a design leader, who had been playing a central role in application design, leaves a project, measures taken would depend on whether the design leader is a type who would take all the work to oneself resulting in a black box effect or a type who would share work and m e m b e r s h a v e c e r t a i n l e v e l o f understanding of the work.

• In the former case, project advancement is impossible without the design leader, and hence, schedule and organizational f r a m e w o r k m u s t b e r e e x a m i n e d fundamentally.

• In the latter case, assign the assistant leader as the leader, and reorganize the structure by promoting al l exist ing members and bringing in personnel.

• Reschedule• Add personnel


• Manage deliverables and issues for each process

• Organize application study sessions

• Have developers attend meetings with users

• Strict work management• Refine work assignments


endix

65


10. Significant Increase in Work Amount (Effort) and Process DelaysScope

In comparison to the estimate, there was significant increase in the amount of work (effort) required, resulting process delays


The original estimate, which roughly matches the actual performance, did not meet the customer ’s budget, and the customer requested a reduction. The ordered party failed to make objections to the customer’s statement that “it can be easily done” because of insuff icient knowledge in the application field in question and also due to a sense of crisis that there will be no future unless they could receive this order.

• Do not easi ly accept work amount reduction requests that lack sufficient grounds.

• Investigate any similar projects and put the knowledge into practice.

• Add personnel (for process recovery)

• Negotiate pricing with customer


• Investigate similar past projects and evaluate its estimate and its relevance

11. Deteriorating in Profit Due to Significant Increase in Work Amount (Effort)Scope

Within the order scope, a significant increase of work amount (effort) occurred in a first-experience field causing major deterioration in profit.


It took much time in establishing an organizational framework for the first-experience field after order receipt, which resulted in insufficient time for deliberating work details. Since the estimate was made under such a condition, and the contract was concluded as a lump-sum contract , negot ia t ion on addi t ional expenses could not be established even though the work scope expanded.

• Ensure sufficient time is available in defining the order scope when receiving orders on first-experience work.

• Avoid lump-sum contracts as much as possible when the specifications and work scope are unclear, and enter into agreements, such as time and material contracts, that can offset the risk.

• Reschedule• Renegotiate budget with customer• Add personnel


• Perform thorough risk management of first-experience field

• Stick to time and material contracts until work scope is clear

• Avoid lump-sum contracts when there are unclear factors

• For any first-experience works, conclude time and material contracts

• Contracts shall be concluded considering the significance of the system

• Share concerns on the risks with the customer and conclude contracts in accordance

66

12. Significant Increase in Work Amount (Effort) and Process DelaysScope

As a result of being forced to accept customer requests on delivery date, addition and changes of functions, not being able to object, an increase in work amount (effort) occurred and processes were delayed.


Due to underestimation when requirements specifications were unclear, the number of project managers, management personnel and team leaders were insuff ic ient. Hence, specification change management was not thoroughly implemented.

• Assigned a sub-project manager and added management personnel and team leaders.

• Necessary rules with customer, such as spec i f i ca t i on au tho r i za t i on ru le , speci f icat ion addi t ion and change schemes and account settlement rule.

• Prepared standards and rules, such as f a i l u r e m a n a g e m e n t , r e l e a s e m a n a g e m e n t a n d c o n f i g u r a t i o n management, that required the project team to follow, and ensured compliance.

• Add project manager support personnel

• Establish procedures and rules


• Define procedures and rules in advance

• Consensus with customer regarding rules and procedures

• Consensus with customer regarding specification finalization

13. Failed to Meet the Delivery DateScope

The first-time experience with the business was more complex than expected. In addition, there were only a few experts on the relevant business. As such, it was discovered that the agreed delivery date could not be met.


During the development, it was discovered that departments were operated in accordance to department-specific rules, which led to substantial expansion of development scale exceeding the initial estimate. Due to insufficient number of expe r t s o f t he re levan t bus iness , proposals of sharing, integration and consolidation of operations could not be offered and customer requests could not be objected.

Since the business was more complex and diverse than expected, a minimal set of applications necessary for business operation was extracted for development, and work prioritization was established for the development of the rest.

• Request cooperation of customer• Get hold of personnel with

experience• Add personnel• Split releases• Reschedule


• Ensure sufficient risk management for first-time experience

• Request experts of the business from the respective departments to participate in the specification meetings

• Split contracts• Establish business cooperation

structure with customer


endix

67


14. System Was Provided While Specifications Remained VagueScope

It was a first-time experience with the business and the system was developed based on the detailed design authorized by the customer. However, many bugs were encountered after service launch.


There were differences among departments regarding application specifications, and the variations were wider than initially expec ted . The cus tomer, who had authorized the detailed design document, realized the issue, but the person in charge on the customer side was replaced (during the development). And since the development side could not allocate more personnel, they intentionally did not clarify the doubts whether it was permissible to develop as defined in the detailed design to avoid any expansion in specifications.

• Made the customers aware of the differences among the appl icat ion specifications as per department. Instead of finding out all the specifications, it was decided to find defects during operations. Upon discovering a defect, specifications were finalized and reflected into the system.

• With regard to defects, a team to support the ongoing operation and another team to finalize specifications and fix the sys tem was es tab l i shed in o rder eventually correct the defects that will be found during the course of production operation.

• Uncover defects during usage of the system

• Maintain documentation• Establish organizational

framework to fix defects


• Perform sufficient preliminary survey to clarify specifications

15. Project Manager Was Too Busy (Heavily Loaded)Scope

Keeping the health of the project manager was an issue. (There was no replacement in case the project manager fell ill and collapsed.)


The project manager’s skill and knowledge were outstanding, and ended up over loaded.

Personnel were assigned under the project manager to handle minor general works and actively fill-in for the project manager’s duties in order to lower the burden on the project manager.

• Allocate project manager supporting personnel


• Perform study sessions• Clarify work division• Clarify role allocation


endix

68

16. Problems Caused by Unclear Development Responsibility of a Jointly Awarded Project

Scope

After being jointly-awarded a publicly-offered project, confusions occurred due to unclear work division of responsibilities with the prime contractor.


A joint effort was established giving priority to receiving the order but role and budget allocation had not been finalized which led to confusion after the order was placed.

Even if giving priority to receiving orders, role allocation upon receiving the order shall be clearly documented and agreed u p o n . B u d g e t a n d r e s p o n s i b i l i t y allocations shall be clarified.

• Negotiate (roles and expenses)


• Clarify role allocation• Clarify budget and responsibility

allocation

17. Due to Insufficient Requirements Definition, Dropped Indispensable Functions Required for Release Was Found during the Test Process

Scope

Due to insufficient requirements definition, it was discovered during the test process that indispensable functions that were required for release had been left out, making it difficult to meet the scheduled release date.


The project was judged as similar to a project experienced in the past but it t u r n e d o u t t o b e n o t . S i n c e t h e requirements definition was made without suff ic ient understanding of current functions, many requests were raised during the later phase causing increased amount of work.

• The deve lopmen t o rgan i za t i ona l framework was reinforced. Reinforcement was applied not only to the own company but was also to partner companies to add members.

• The scope was narrowed to release part of the functions

• Reinforced organizational framework and added of personnel

• Performed split releases• Reconfirmed specifications with

customer


• Documentation• Evaluate at Process breakpoints• Review by customer• Thorough preliminary survey

(whether the project is in fact similar to past experienced projects)

69


18. Test Procedures Were Unclear and Test Plan Cannot Be MadeTime

The system entered the qualification test phase but all the members, whom only have experiences in developing small-scale systems, did not know how to proceed with the qualification test for a large-scale development project.


Insufficient experience and knowledge in developing a large-scale systems.

• I f test procedures are unclear, use those of a similar project within the company.

• Bring in personnel with experiences of executing large-scale qualification tests and create a plan.

• Get hold of experts• Obtain rules and procedures

(judgment by personnel with experience is required)


• Define organizational process (method of assigning a project manager, get hold of personnel, and establishment of organizational framework)

19. Integration and Qualification Test Plans Are Insufficient Even with Only a Month and a Half to Service Launch

Time

With only a month and half to service launch, the business operation manager (supervisor of the project manager) raised an alarm that the project might undergo crisis. While already in the unit test phase, plans for integration and qualification tests are still not yet established.


Insufficient project management and personnel from the prime contractor. The development responsibility scope of the subcontractors (3 companies) was to complete unit test in house, and the prime contractor was responsible to coordinate the later phase from integration test (inter-company test) onwards.

• A l l t h e m e m b e r s , i n c l u d i n g t h e subcontractors, were gathered in one place.

• Reinforcement of the prime contractor’s organizational framework. Key persons from the subcontractors were added as members of the prime contractor to draw up the test plans.

• Meetings attended by project manager and operation managers were held every morning.

• Release was split into first and second releases (delayed releasing of some functions).

• Gather the members and foster a sense of unity

• Establish a PMO (Project Management Office)

• Prepare a test plan and test environment

• Reschedule• Reinforce the project

organizational framework


• Use a plan-oriented project promotion approach

• Prepare a project management framework

• Evaluate at process breakpoints

70

20. Profitability Deterioration Due to Significant Increase in Amount of Work (Effort)

Time

A significant increase in the amount of work (effort) with regard to migration operation caused a major drop in profitability.


An order was accepted under a lump-sum contract as requested by the customer despite not being able to clarifying the workload of the migration operation. There was a major increase in the amount of work than expected. In receiving the order, prerequisites were stated, but they were not clear. Hence, the customer did not accept additional charges.

Avoid lump-sum contracts as much as possible when the specifications or work scopes are unclear, and conclude a contract that can offset the risk, such as a time and material contract.

• Negotiated cost with customer• Pricing was arranged through top

management talks


• Conclude contracts considering the risks

• Inform of the high risk to customer, and do not accept the project if the customer fails to understand

21. Schedule Delays in Common Library and Quality IssuesTime

A common library was developed in parallel with the application, but there was a delay in the development of the common library resulting in significant impact on the development schedule of the application.


The deve lopment schedu le o f the common library, which was one of the critical paths, was poorly managed and functional requirements definitions were insuff ic ient ( the responsib i l i t ies o f functional division between the application and common library were ambiguous).

• Prepared a manual for the common library (addition of functional requirements d e f i n i t i o n s ) a n d r e i n f o r c e d t h e development organizational framework.

• Uni t tests of the appl icat ion were performed using stubs.

• Add personnel• Reexamine the design• Devise ways to perform tests


• Get hold of highly skilled personnel

• Manage progress with critical path in mind

• Define process management standards


endix

71


22. Insufficient Assumptions When EstimatingCost

System engineering work at the customer site such as acting as observer or providing operational support had occurred in number and significant increased in cost exceeded projections.


The work site and work details were not clarified during contracting phase. Hence, transportation expenses were not included in the estimate but could not be declared as being out of the support scope. Since the scope of SE work that would be requested was unclear, even if the work exceeded the expected cost, the cost could not be declared as being out of the estimation scope.

By coordinat ing wi th the customer, clarified the work scope within the contract to convert the organizational structure that continuously receives requests, into a structure that clarifies task completion goals.

• Customer coordination• Clarify work scope


• Clarify contracted work scope• Refer to estimation examples of

other projects

23. System Fails to Operate Due to Problems with a Vendor Supplied ProductQuality

A vendor supplied product was deployed on a broad scale, but it failed to sustain actual operation due to repeated memory leaks.


It was believed that technical assessment was completed after a prototype was created, but when it was put into actual use during development, memory usage gradually increased and eventually an o v e r f l o w o c c u r r e d . I n s u f f i c i e n t confirmation with massive data entries was the cause

The countermeasure in this situation is to commit to resolving the issue or to give up ear l y and focus on look ing fo r an alternative function. It is essential to give up at the earliest if there are no chances or if one has no confidence in fixing the issue. Investigate the package, consider how to respond, and coordinate with the customer.

• Involve the vendor• Reselect a package• Recreate from scratch• Reschedule• Consider operational workaround

(such as rebooting the system daily)


• Conclude a maintenance contract (Clarify of the scope of warranty)

• Perform thorough preliminary survey (investigate of actual performance, function coverage, preliminary tests)

• Clarify responsibility scope if the package is designated by the customer

• Research and prepare proven packages beforehand

• Get hold of personnel with experience with the package

• Investigate the package vendor (stability)

72

24. Quality Issues in SystemQuality

Being overwhelmed with the schedule, a low quality system is released, resulting in getting preoccupied with responding to system failures, and not being able to escape from this negative spiral.


Schedule and cost were being prioritized instead of establishing quality (document creat ion and review) or performing t h o r o u g h t e s t s . T h e d e v e l o p e r ’ s understanding of the required quality of the system was poor.

• Added management personnel centering on quality management.

• Prepared release schedule and system failure response rules, and ensured that they were being practiced.

• Perform quality analysis (identification of weaknesses)

• Add personnel


• Clarify quality requirements• Thoroughly implement tests• Clarify release quality standard• Share understanding of quality

with customer

25. Quality Issues in SystemQuality

Multiple unit test level bugs were encountered during integration tests. The integration tests were suspended and postponed resulting in a delay in process.


During the development of additional funct ions, an exist ing program was updated to improve its maintainability. Integration tests were started without performing sufficient tests on the updated sections.

• Redo unit tests on the program.• Perform source code review by third

parties.

• Redo tests• Perform source code review


• Clarify function addition rules• Consensus with customer shall be

gained regarding the risk of adding function


endix

73


26. Time Consuming in Massive PrintingQuality

An application package that was developed as a client-server system was rebuilt into a web-based application. The printing processing system of the print server was built in-house, but the time consumed of the massive printing during the qualification test was longer than expected.


A processing delay was encountered due to insufficient common resources in the middleware during data passing process between the web application server and the print server.

The common resource configurations of the middleware were reexamined and a new massive printing processing system was bui l t to handle operat ions with massive printings involved.

• Reexamine similar programs• Reexamine processing system


• Perform preliminary survey (technology survey)

• Perform preliminary load tests• Clarify and consider performance

requirements

27. Processing Speed Suddenly Decreased during User Application Operation Training

Quality

During user application operation training, processing speed became significantly slow while processing the same application processes.


In addition to the processing concentrating on the authentication server and database server, there was a defect in the exclusive control of shared resources as well as massive redundant database access due incorrect usage of SQL syntax.

• The exclusive control system of the shared resources was reexamined, and the application programs with concentrated t ransac t ions were p r io r i t i zed fo r modification.

• The database access log was analyzed, and SQL s ta tements were f i xed , especially those that were performing unnecessary processing in the application program.

• Bring in specialists (reexamine the exclusive control system)


• Perform preliminary survey (technology survey)

• Conduct design review with experts

• Create a list of don’ts

74

28. Leaving System Development Entirely to a Partner Company (100% Subcontracting) Was a Failure

Quality

System development was consigned to a partner company which was believed to have business knowledge and experience in system development projects, but progress and quality issues were encountered.


T h e p a r t n e r c o m p a n y h a d p o o r management ability. Furthermore, when trusting the partner and consigning it the project in its entirety, the estimate from the partner should be verified and quality check should be performed.

• Reinforce the organizational framework of the partner company.

• Consensus with customer regarding scope shall be built if requirements from the customer are still to come.

• Foster a sense of project unity by supporting the project manager and work of the partner company.

• Reinforce organizational framework of partner company


• Reexamine work division


• Preliminary survey and evaluation of the partner company

• Thorough project management of the partner company

• Establish consensus with the partner companies regarding communication planning (requiring reporting on progress)

29. Bug Fixes of Package Software Requires Much TimeQuality

The package functions that were used for the first time had quality issues, and fixing those bugs required much time.


There were quality issues in package functions that were used for the first time. Since it was an overseas package vendor, it required time to fix the bugs. Some bug fixes caused function degradation, and hence, a thorough acceptance test was required.

• Thorough quality verification of package functions used for the first should be conducted.

• C rea te tes t schedu les under the assumption that bugs exist.

• Consider workarounds• Investigate alternative products• Customization via add-ons


• Create test schedules• Confirm package support

organizational framework• Obtain a memorandum of

agreement on support• Perform preliminary survey

(functions / performance of first-time use package)

• Share concerns on risk with customer


endix

75


30. Design Mistakes in Massive Data Processing SystemQuality

A test environment and plan had been formulated for qualification tests, but the application development leader indicated doubts concerning performance issues.


Upon rece i v ing repo r t s abou t t he concentration of business on the current s y s t e m a n d i m p r o v e m e n t i n t h e processing abilities, concerns with regard to the processing system of the new system were raised from the application development leader. The processing s y s t e m d e v e l o p m e n t l e a d e r h a d establ ished per formance design in accordance with the workload and had confirmed it in the test environment, but the leader was not famil iar with the transactions that were associated with business operations which were not included in the assumptions.

• Reexamine the processing system.• Approval from customer was given by

applying operation restrictions and performing tests in the production e n v i r o n m e n t i n o r d e r t o a v o i d concentration of business processing.

• Reexamine similar areas• Operation restrictions• Reexamine processing system


• Conduct Preliminary survey (operational features)

31. Implementation Performed under Ambiguous Specifications Resulted in Many Requirements Inconsistencies during Test Process

Quality

Integration test was started upon delivery from the development contractor, but functional requirements were not met.


Requirements (business rules) were not documented sufficiently and were only in the head of the project manager (of the prime contractor). Review with regard to del iverables f rom the development subcontractor was not sufficient.

• Described test procedures in the form of a decision table.

• Requested the development subcontractor to rewrite the program and redo tests in accordance with the test procedures.

• Split the release into first and second releases.

• Document business rule• Document test procedures• Retest• Perform code reviews• Reschedule


• Document specifications

76

32. Issues in Printing External CharactersQuality

On a system that prints documents, after the service was launched, reports were printed with missing characters and blanks in place.


Academic-use characters were registered as external characters, and configuration for these characters were also required in production environment in order to print them f rom the deve loped sys tem. However, the development side did not know of the scope and necessity external character registration, and as such, the character codes were sent to the printer as-is, causing the issue to arise.

After configuring external characters, all the printed results were visually confirmed to ensure no blanks were encountered.

• Perform test to confirm all characters are printed

• Clarify customer’s requirements specifications


• Perform preliminary survey (investigation of existing system)

33. Business Halted during OperationQuality

There was no issue encountered in the test environment, but the system terminated abnormally after the service was launched.


Parameter modification was required when migrating to production environment since the test environment and production environment were different. Operation verification in production environment was not possible, and as such, differences in parameter f i les were ext racted for confirmation. However, while i t was confirmed that “correct differences” were obtained, a check to confirm that “there may be parameters that required changes in areas that showed no difference” was not performed which allowed parameters from the test environment to remain.

Output the parameter lists for the entire the source code, and had two experts, whom unders tood the d i f f e rences between the production environment and the test environment, visually confirm the parameters.

• Reexamine existing configurations• Have experts participate• Assign a configuration manager


• Review by experts• Review test plans• Reexamine release procedures


endix

77


34. Unclear Chain of Order and CommandHuman Resources

The chain of order and command within the project was ambiguous, making it difficult for the project manager to coordinate resources.


This project was formed with a dedicated team A, and team B which was also participating in another project. The project manager’s authority over team B was ambiguous, and as such, the project manager was unable to coordinate resources over team B.

The responsibility allocation and chain of order and command for moving the project forward were clarified.

• Create and reexamine the birds-eye view diagram of the organizational framework

• Clarify work allocation• Clarify responsibility allocation• Coordinate work among teams

(judgment of priorities, consensus establishment with the project manager and superiors)


• Clarify responsibility and work allocations during project planning phase

• Clarify workload• Clarify project scope

35. Confusion Due to Two Persons Acting in the Role of Project Manager Personnel within Project Organizational Structure

Human Resources

Under a project manager with an overall perspective who had been involved since sales activities, a successful development leader leading the development team joined the project. Because of the successful experiences the project manager possessed, the project eventually had two project managers.


There was insuff ic ient control over development within the company.

Pay careful attention to internal control wi th in a company when start ing up development teams. Monitoring and act ions by senior management and executives are important.

• Clarify responsibility allocations


• Clarify responsibility and work allocations during project planning phase


endix

78

36 Problems Caused by an Inexperienced Prime Contractor Originally from a Different Industry

Human Resources

The project was received as a subcontract from a prime contractor who was originally from a different industry. The contractor did not have established software engineering standards, and development work had to be performed in tandem with the education of the contractor.


The prime contractor was originally from a di fferent industry, and whi le having difficulty in acquiring the skills required of a prime contractor, it carried on working as the prime contractor despite having insufficient skills.

• Clarify the division of functions between the prime contractor and subcontractor, and work to improve skill levels through long term corporate cooperation.

• Educate the prime contractor about software engineering standards, and obtain consensus regarding basic procedures.

• Clarify work division• Share development and software

engineering standards


• Establish consensus regarding development and software engineering standards

37 Unable to Handle Technical DBMS ProblemsHuman Resources

Application development started with development personnel that had little DBMS experience. DB operation response was too slow, making the system unusable.


Insufficient IT architecture personnel. Normally, once the technologies to be used are decided, engineers with sufficient skill are assigned and a technical group is formed. In this case, the project was moved forward without doing so.

• Even if late in the development process, this problem can only be resolved by bringing in specialist engineers.

• Do not leave matters entirely to engineers. Instead form a technical group with a team containing several application development personnel.

• Get hold of adequate engineers• Educate development personnel• Redesign• Reconstruct


• Have experts participate• Review skills of personnel• Educate personnel

79


38 Quality Issues Occurred during Integration TestHuman Resources

Integration test of a web application began on the server, but many bugs were encountered due to errors in program design.


Business appl icat ion speci f icat ions increased during the detailed design process, so many members were added during the program design process. This was the first experience these members had with the business application field in ques t ion , as we l l as the p ro jec t ’s development environment. There were also repeated progress delays, and insufficient advance training in object orientation program design and new processing methods. No reviews by experts were conducted to make up for the lack of training, and program design was left to the members that joined the project midway.

• Discontinue the integration test, and add personnel experienced in the business application field in question.

• Implement reviews to discover mistakes and share them with all members, who then review their own program design to overhaul all programs.

• Reschedule• Add experienced personnel• Conduct source code review• Educate the added members


• Conduct training• Establish coding conventions

39 Defects in Requirements Detailing and Basic Design Organizational Structure after Order Receipt

Human Resources

The members only had experience in post-design development, and lacked sufficient requirements definition design capabilities.Delays started to occur from design process.


Only orders for post-design development had been accepted, resulting in members not cultivating requirements definition design skil ls. The corporate posture became one wai t ing for completed designs.

As a short term measure, search for and add personnel with the needed skills.

• Add experienced personnel


• Develop Designers• Hire upstream design engineers


endix

80

40 Major Effort Increases and Process DelaysHuman Resources

There were insufficient human resources for the project, but the project manager made overly optimistic in deciding that the problems of insufficient personnel and project turn-around time would somehow work out in the end, and forged ahead with the project.


H u m a n r e s o u r c e s w i t h t h e s k i l l s necessary for the project were taken by other projects. Personnel brought in to make up for this lacked sufficient skills, and had scheduling conflicts, resulting in the concentration of workload on a small number of key personnel. Cross-function coordination, design flaws checking, and appropriate response to requirement expansions could not be enacted in a timely fashion.

• Add project sub-managers.• Add management personnel.• Add development personnel.• Rev iew rema in ing tasks ( func t ion

development, failure handling, specification change hand l ing , e tc . ) , c rea te an implementation schedule for those tasks, assign personnel to those tasks, and coordinate with the customer.

• Manage progress daily.

• Escalate to organization manager• Add personnel• Review remaining tasks• Reschedule• Manage progress daily• Coordinate with customer (overall

optimization across projects)


• Get hold of sufficient resources• Risk management• Thorough project management

41 Business Application Leader Left ProjectHuman Resources

The business application leader, who was thoroughly versed in the business application field but about whom doubts remained concerning management, left the project just before the qualification test.


Directions were given to make the person most versed in the business application field the business application leader in the management team in preparation for the qualification test. There was an expert of the business application field, but that person was from a partner company, and there were doubts regarding the person’s management capabilities. The project manager proposed assigning a different person, but this proposal was rejected as it went against the decided policy.

• A d d s o m e o n e e x p e r i e n c e d i n management to the team.

• Through interviews with the remaining members, select a member who can take c h a r g e o f b u s i n e s s a p p l i c a t i o n specifications to make up for the loss of the expert in the business application field.

• Request cooperation of customer and educate human resource while performing business application specification reviews.

• Add experienced personnel• Add business application field

experts• Develop business application

leader• Request cooperation from

customer


• Assign project manager with management ability

81


42 Frequent Changes in Partner Company Leader and Core PersonnelHuman Resources

Personnel of the partner company changed upon project startup.


While the economy was successful, the partner company prioritized accepting orders despite insufficient personnel, assigning available personnel ad-hoc to get by for the moment. The full-fledged organizational structure was only able to be established after proceeding far into the development process.

• Restrain from personnel-dispatching type contracts.

• Use a document-centered development approach.

• Establish full-fledged organizational structure

• Reschedule


• Create an organizational structure in line with the work to be performed

43 The Partner Company’s Skill Level Was So Low to the Point of Being Useless

Human Resources

Design was performed using a partner company, but they were unable to produce viable designs. The level of the design work was low.


Insufficient partner company personnel skill and experience.

It depends on specific project situations, but normally the partner company would have to be changed. However, if the company has potentials, education may make the partner company a beneficial partner. Gathering sufficiently skilled personnel is important, not only for partner companies, but the reality is that many workers are not skil led, and training unskilled workers to make them skilled workers is sometimes the best policy.

• Train partner company personnel• Conduct design review• Add experts of the business application field


• Investigate partner company capabilities

• Create guidelines for partner companies


endix

82

44 The Relationship between the Project Manager, the Leader, and the Worker Deteriorated, Resulting in Both Communication and Work to Stall

Communication

The worker involved in the next generation system construction had been involved in the maintenance of the old system, and as such, had extensive business application knowledge.However, the project manager and leader were brought in from a different project. There were differences in opinion regarding how to do things, and the communication situation deteriorated.


There was a mismatch regarding how to do things, how to move the project forward, and how to communicate. The problem was noticed too late.

• This problem can only be handled by the responsible person of the project.

• The responsible person of the project should determine whether the problem resides with the project manager or the worker and take corrective actions on the problem.

• Interview related parties (individual meetings)

• Establish an occasion for discussions (organization of a personnel camp, etc.)

• Hold a meeting for the worker to provide a briefing

• Share information regarding each member’s work situation

• Share information regarding issues


• Indicate work policies• Establish meeting structures• Hold a kick-off meeting (perform

again if the project manager changes)

• Understand the difference between member assignment by project manager and team building

45 Unclear Role AssignmentsCommunication

Reworking became necessary due to insufficient communication between the application development team and operation team. Workload increased significantly, exhausting members.


The pro jec t was the f i r s t t ime the a p p l i c a t i o n d e v e l o p m e n t a n d t h e operation teams worked on a project together, and their respective roles were unclear. Communication between the two was also poor. The application development team was particularly late in conveying information to the operation team, and repeated specification changes greatly increased the operation design workload.

• Clarify the roles and responsibility scope of each party, as well as how to coordinate.

• Hold regular meetings, attended by leaders and key personnel from both teams, to share information.

• Hold meetings involving all members

• Pursue team building• Rule establishment


• Establish communication rules• Share information on work schedule

83


46 Frequent Non-Written Requests and Notifications within a Complex Development Organizational Structure Resulted in Confusion

Communication

The structure of the development organization was complex, but work standards were not unified, and notifications were frequently provided orally, resulting in massive confusion.


Communicat ion wi th in the complex organization was not documented, and t h e r e w a s n o c o m m u n i c a t i o n management.

• Always communicate in written form (requests, responses, notifications, etc.).

• Assign an serial ID number to each document (one item, one page), and manage the documents based on the serial ID numbers.

• Assign a document manager to manage these documents. Perform document con t ro l a t p rogress management meetings.

• Create a problems list (information aggregation)

• Establish communication rules• Create mailing lists• Document messages to members


• Establish communication rules• Establish problem item

management rules

47 Poor Coordination with Back-Office Led to Isolation of Development Group at Customer Site

Communication

In a subcontracted project, the work environment of the development group at the prime contractor site worsened, exhausting the personnel.


The subcontractor’s back-office had little understanding of the situation at the development si te, leaving the labor situation unmanaged.

• Bring back-off ice personnel to the development site so that they could understand the situation, and make them take any possible measures for the company.

• Arrange for hotels, distribute taxi vouchers and arrange for food supplements

• Assign back-office support personnel, improve communication environment, take budgetary measures, etc.

• Establish a system for tracking the current situation of the site at any time.

• Improve work environment• Dispatch PMO• Place back-office support

personnel


• Plan work environment in advance


endix

84

48 Quality Issues Prevent Qualification Tests to ProgressCommunication

Quality problems were discovered during qualification tests by a team that was fastest up to and including integration tests.


Qualification test was started by a team that had made the largest progress up to and including integration tests. The business application leader of the team knew the specif ication changes and additions that had been made after the detailed design process, and planned to proceed with the qualification tests while limiting the test scope, making specification changes and addi t ions in para l le l . However, this was not sufficiently conveyed to the customer, and the qualification tests were scheduled based on customer convenience. Specification changes and additions which had not yet been handled were all considered as quality issues.

• Delay start of the qualification, and implement specification changes and additions first.

• The business application leader moved to another work site when the qualification tests started, so a new sub-leader with business application knowledge was se lec ted and spec i f ica t ions were organized.

• The sub-leader communicated regularly with the manager, carrying out business application specification confirmation work.

• Reschedule (test start timing)• Develop business application

leader


• Share progress status with customer

• Establish consensus with customer regarding test plans

49 The Customer Complained about Differences between the Specifications and the Actual System

Communication

The system screen, which was designed based on the basic design document, was presented to the customer for confirmation. The customer complained about minute differences, resulting in pixel-level corrections.


The screen specifications were based on a prototype, but the customer thought that they represented the completed system. Coordination with the customer failed, as it was not explained to the customer that there would be minor differences.

All screens were corrected to perfectly match the specifications.

• Perform corrections in response to customer demands


• Confirm with customers which screens and report forms must be rigidly followed

85


50 The Customer Side Project Manager ChangedRisk

Design work was complete and development was about to start when the customer side project manager suddenly left the company, negatively impacting project progress.


There is always a possibility that a customer side project manager would change regardless of situation. The project was negatively impacted because measures were not quickly enacted.

• Confirm to what degree the project manager was providing status and progress reports within the company.

• Request the assignment of a new project manager.

• As soon as the new project manager arrives, hold focused meetings to share information such as project overview, progress status, budget and actual costs, organization, and risks.

• New project managers tend to trust the f i r s t p e r s o n t h a t p r o v i d e s t h e m information, so it is important from the information provision side to be fast and accurate.

• Request assignment of new project manager

• Convey project management policies to the new project manager

• Hold a kick-off meeting• Establish an occasion for

discussions (organization of a personnel camp, etc.)

• Hold a meeting for the worker to provide a briefing

• Share information regarding each member’s work situation

• Share information regarding issues


• Risk management (document any information, get

hold of customer side sub-leader)• Add provisions to the contract

concerning project manager replacement

• Arrange for advance notification of replacement of key personnel or project managers


endix

86

51 Upon Completing the Design Phase, the Partner Company Refused to Begin the Next Development Phase

Risk

On approaching the completion of the design, the partner company entrusted with basic design announced that they would complete the basic design but would withdraw from subsequent phases. They were quite adamant.


The partner company was worried about project management, the implementation of the design contents, possible inability to add personnel during the next process, and the like.

• The initial action is to negotiate to retain them. If that does not prove fruitful, have them commit to cooperating in the handing over to the next partner company.

• Then decide on handover measures, including costs, with the partner company (normally, an agreement is reached with the partner company that handover costs will not be paid for).

• Take retaining measures• Handover work commitment and

contract


• Improve management• Share information on issues with

partner company• Place orders in line with partner

company capabilities

52 Qualification Tests Conducted Without Business Application ExpertsRisk

The person in charge was replaced shortly before qualification tests in the production environment. A new person was hurriedly assigned, but there were unexpected costs involved in coordinating with the customer and creating test scenarios.


An expert member left the project before qual i f icat ion tests star ted, and the replacement’s level of business application knowledge and customer support were not up to the predecessor’s level.

The project manager and an expert who handled related business applications were called in and provided temporary assistance. The initial plan was for the work to be done by one person. By increasing the number of people involved, costs rose, but impact on the customer was mostly resolved.

• Cultivate the next key personnel• Reschedule• Check for documentation• Bring in experts


• Allocate multiple key personnel• Implement documentation (not just

specifications, but also clarify judgment criteria, business application know-how, and the like)

• Pay careful consideration during contracting phase (number of people versed in the business application field)

87


53 Work Completed before ContractProcurement

A short-term, small-scale system development was initiated and completed before the contract had been concluded.


Development work started when the c u s t o m e r i n d i c a t e d a p p r o v a l o f development plans and the estimate. The customer’s unofficial notice was provided wi th in the month, but off ic ia l order placement did not occur during the development period.

Had the customer establish an opportunity for negotiations including the superior of the person in charge, and informed them that unless formal order is placed quickly, system delivery would not be possible. The next month, the customer formally placed their order.

• Strongly request formal order placement


• Begin development only after an order has been received

54 Operation Cancelled due to Quality Problems in Vendor Supplied Business Package Software

Procurement

A package software was customized for system usage, but quality and performance problems became evident, and operation was cancelled shortly after being launched.


Determinations regarding what would be provided by package specifications and what wou ld be deve loped th rough customization were left to the package vendor. As a result, not enough time was available for customization development, a n d t h e d e l i v e r y d a t e a r r i v e d . Fur thermore, the pro ject manager, troubled by specification adjustments with the customer and package vendor, left the project. The replacement project manager was performing double duty, and not having the time to fully grasp the situation of the project or its problems, decided to go live using the package.

• As a result of situation judgment by the package vendor and business application experts, system operation was delayed by half a year in order to improve its quality and performance.

• The division of development duties with the package vendor was reexamined, and a new development organizational structure was established.

• Conduct survey of current situation

• Reschedule• Negotiate with customer• Reexamine role assignment and

development organizational structure


• Conduct preliminary survey• Perform load test• Clarify responsibility assignment

with package vendor• Have progress managed by a third

party


endix

88

55 The Browser Selected by the Sales SE (Systems Engineer) During Sales Was Not Suited for Actual Use

Procurement

During sales, the sales SE decided on the use of a certain browser and proposed it to the customer. It was built into the actual system, but during the test it was discovered that it could not stand up to actual use.


The sales SE was not in a position to confirm product viability, and there was no structure within the company which could d o s o . T h e s y s t e m d e v e l o p m e n t department merely performed system construction based on the conditions handed to them after order placement.

• After order placement, the development department shall review the system architecture, reevaluate it in terms of viability, and redesigning it.

• Do not accept conditions inherited from sales activities unconditionally.

• Include those costs in estimates.

• Procure a usable replacement product


• Conduct preliminary survey

56 The Company’s Middleware Product Did Not Support the Latest Version of RDB Software Decided on by the Sales SE During Sales

Procurement

During sales, the sales SE decided to use the latest version of an RDB product and proposed it to the customer. Once the order was placed, the project moved into the system design phase. However, the company’s middleware product which worked together with the RDB did not support the latest RDB version, and it was discovered that supporting it would cost both time and money.


The sales SE did not have an accurate u n d e r s t a n d i n g o f t h e c o m p a n y ’ s middleware product development plans. The Sales SE did not have an accurate grasp of the costs and time involved in supporting the latest RDB version.

• Establish an internal structure which enables sales SE to obtain accurate information regarding company product development.

• Create a system for internally evaluating system proposals for the sales activity, and for pooling necessary costs.

• Coordinate with customer regarding procured products


• Conduct preliminary survey (package, internal product compatibility, etc.)

• Educate sales personnel regarding in-house packages

• Have persons in charge of the system to review contents of proposals

89


57 Black Box Effect due to Use of Package Products Creates ProblemsProcurement

A package software with a proven track record was used, but the quality was extraordinarily low.The company that supplied the package software insisted that there were no problems, but that was far from the actual case.


Quality confirmation of the internals of the package software could not be performed, and the project team was forced to rely on the word of the company supplying the package.

Request quality management organization and support structure improvements of the c o m p a n y s u p p l y i n g t h e p a c k a g e . Requests for test support are also an option.

• Involve package vendors• Avoid via add-ons• Procure replacement products


• Conclude a warrantee contract with the package vendor

58 Repeated Failures and Degradation Encountered during the Integration TestBasic Conduct/Action (Extended)

The same bugs were repeatedly encountered during the integration test despite applying bug fixes when discovered, and there were many interface mistakes.


Revision history and program generation (version) management was not performed strictly when conducting bug fixes on multiple programs. Programs which had not been fixed were used, or the person in charge merely replaced objects, making it impossible to understand the configuration of the business application program for the integration test.

Assigned a dedicated integration test environment operator, and improved version history, generation, and configuration management rules. Improved system such that test environment was provided when author ized by the test environment manager.

• Formulate and implement configuration management rules

• Assign configuration manager


• Formulate configuration management rules

• Assign configuration manager• Determine configuration

management tools


endix

90

59 The Company’s Project Manager Fell Ill and CollapsedMotivation (Extended)

The company’s project manager fell ill and collapsed.


The project manager was always battling the stress of long hours of overtime, heavy responsib i l i t ies , and menta l exhaust ion, resu l t ing in h is heal th condition deteriorating.

Project manager replacement can breed uncertainty throughout the project, so the issue must be handled rapidly. A temporary project manager could be chosen from one of the leaders, but if not handled well, this project manager could also leave the product, producing a domino effect. It is best to temporari ly have the project director also act as the project manager.

• Place a new project manager (higher ranking personnel, internal)


• Higher ranking personnel should care for project managers

• Organize company labor management

60 The Customer’s Person in Charge Lacks Capability and Experience to the Point of Being Useless

Customer (Extended)

Adequate communication with the customer side person in charge was impossible, and no matter what was said to the person, handling was always insufficient.The project itself stalled because of this.


The customer side person in charge had insufficient communication skills. The person understood neither the system nor business.

When information important for system development cannot be obtained from the customer side person in charge, create a list of all risks that may result from this, and discuss them with the responsible person on the customer side. Explain the r isks to the project object ively and sincerely, and request an improvement.

• Create a risk list• Explain the risks to the customer• Request change of customer

contact point through a third party


• Plan communication rules

91


61 Underestimated Test EffortCost

Since package software was to be used, it was believed that actual development effort would be low, and estimates were based on total effort required for the functions.


Package sof tware use can reduce development effort, but it requires test efforts for work including validation and in te r face tes t ing . These were no t considered in the estimate.

• Add experts, determine what tests are miss ing , and es t imate add i t iona l resources, such as t ime required, personnel, and test environments.

• Recreate a viable test plan, and obtain approval of the change from stakeholders.

• If project objectives cannot be met, coordinate with stakeholders, including review of quality, cost, and delivery date.

• List up remaining tasks• Reschedule• Negotiate cost with customer• Add personnel


• Confirm the validity of the estimate• Investigate the package

62 Multiple Command Structure for Development TeamCommunication

There were two lines of instruction to the development team (subcontractor), resulting in confusion at the frontline.This resulted in work delays and degradation in work quality.


T h e p r i m e c o n t r a c t o r e n t r u s t e d development to a subcontractor, which had a further subcontracting development vendor. The prime contractor issued i n s t r u c t i o n s d i r e c t l y t o t h e s u b -subcontractor development vendor, resulting in confusion. The instructions issued by the prime contractor were not b roken down in to f i e ld - leve l work procedures.

Create an occasion for the contractor, subcontractor, and sub-subcontractor to reconfirm as consensus that directions are not to be issued without following the chain of command, nor are directions which do not follow the chain of command to be followed.

• Clarify and ensure following of chain of command


• Plan communication rules• Create stakeholder birds-eye view,

and obtain consensus


endix

92

63 Failure to Understand Status of Each Team Resulted in Inability to Determine Whether Qualification Test Process Should Be Started

Human Resources

There were doubts about whether it was alright to start the qualification test after the integration test had been completed.


Overall project control was weak, with the project organization heavily development oriented, resulting in key personnel being assigned as subsystem development leaders. As such, there were no people in the test team who were capable of evaluating whether the qualification test should start.

• Make each team submit a “Quali ty Eva lua t i on Repor t ” ( summary o f integration test results). This can be used to judge when each team (functions, subsystems) can progress to the qualification test.

• Based on those results, if necessary, revise integration test structure (test order, etc.) This is due to the possibility that the integration test may not be possible as described in the original test plan.

• Have the leader of the team moving on to the qualification test assume the function of overall control of the test.

• Understand the quality situation• Formulate the qualification test

plan• Establish organizational structure

for the qualification test


• Evaluate process breakpoint• Formulate quality evaluation

criteria• Ensure reporting of quality and

perform reviews• Formulate qualification test quality

management rules• Establish organizational structure

plan

93


64 Key Personnel ExhaustedHuman Resources

Within a project in critical status, tasks to drive the project concentrated on the shoulders of a key personnel. The key personnel became exhausted, which hindered work efficiency improvement, making progress of the project even slower. This caused a vicious cycle, as it placed further burden on that key personnel.


While concentration of load on the key personnel could not be avoided, other members have not developed. If the key personnel were to collapse, so would the project.

• Assign a person to organize the key personne l ’s t asks , and cu l t i va te personnel who could fill-in for the key personnel’s “key” functions.

• T h e p r i m a r y d u t i e s o f t h e t a s k management personnel are “traff ic control” of high volumes of tasks, and management of priorities and deadlines. The key personnel can then dedicate him/herself to performing given tasks in the order specified.

• The primary duties of the key personnel substitute persons are the learning of the “key” elements of the key personnel duties, and the acquisition of the ability to, at the least, collect information necessary to perform initial isolation of problems and to provide resolutions thereof. The key personnel can then concentrate on providing answers based on the data organized to some extent.

• Distribute the load of key personnel


• Manage work of key personnel• Divide labor to prevent excessive

burden on key personnel• Develop successors


endix

94

65 A Deep Hierarchy of Partner Companies Makes Responsibility Scopes VagueHuman Resources

In an extremely large scale project, the quality of programs and systems delivered by partner companies were low.The responsibility scope across partner companies was vague, making it difficult to see what course of action should be implemented to resolve the problems.


Under the prime contractor, there were a l s o a s u b c o n t r a c t o r a n d a s u b -subcontractor, making the responsibility scopes of each individual company unclear. As a result, quality dropped, and the d i f f i cu l t y invo lved in s i tua t ion assessment became evident. The overall management team was insuff ic ient considering the scale of the project.

• Frequently, partner companies in the intermediate layer of the hierarchy fail to function adequately. In this case, an e x c e p t i o n c a n b e m a d e i n t h e establishment of the chain of command that is neither hierarchical or company oriented.

• If the partner companies are gathered in one location, the measure above can be implemented easily.

• If the partner company in the intermediate layer of the hierarchy is not aware of the sub-subcontractor’s development status, authority can essentially be stripped off from the intermediate level company (stress that the company would retain responsibility, and the fact that they were incapable of proper management).

• If the partner companies are physically far a p a r t f r o m e a c h - o t h e r, i m p r o v e communication with the upper level partner company. The last measure would be emergency meetings.

• Gather workers together• Organize instruction and

command structure• Establish overall management

team


• Ensure thorough management by partner company (mid-level)

66 Management Actually Performed by Partner Company Development GroupHuman Resources

The prime contractor placed an order with a subcontractor, but as the subcontractor’s products made up the core of the system, actual project management was performed by the subcontractor. This placed a significant burden on the subcontractor, delaying actual development work.


The subcontractor performed both project management and development work, overloading them.

Division of work need to be clarified at the contract stage. The number of cases where manufacturers leave system development entirely to subcontractor partner companies is increasing. This situation, where the manufacturer is supposed to perform project management, bu t i n f ac t does no th i ng , pa r t ne r companies are charged with all duties and deve lopment respons ib i l i t i es , and customers go after partner companies when there are problems, undermine system development.

• Clarify work division• Reallocate work


• Create a work assignment table• Establish a meeting structure

95


67 Poor Qualification Testing ProgressHuman Resources

The upstream process went alright, but the project began to lag in the downstream process qualification test phase.


Qualif ication test was performed by engineers with relatively low skill levels.

• Replace personnel.• Use test tools and standardized test

methods to homogenize test operations.• Review test specifications. Create an

environment where even relatively unskilled personnel can perform tests by following specifications.

• Add testing personnel• Formulate rules• Standardize tools• Education• Assign roles and work in

accordance with skill levels


• Establish test plans• Request participation of customer

in the qualification test• Use highly skilled personnel in the

qualification test


endix

96

68 Inefficient Test WorkHuman Resources

Half of the time spent using the test equipment was occupied by unproductive retests and fixes caused by mistakes in operation and error ridden test data.


• The test equipment system configuration information was incorrect. The quality of the test environment, such as the test database and data used for test, was low. There were also quality problems in test procedures and test operations, such as data acquisition and analysis procedures when failures occurred, restoration p r o c e d u r e s , a n d c o n f i g u r a t i o n management.

• Increase number of test environments. Make it such that configuration information changes and database / test data replacement are not necessary each time a test is performed. Alternatively, decrease frequency of configuration information changes in order to minimize the impact of quality defects on test operations.

• Install an overall management supervisor to handle multiple test environments, and manage the following:

- Creation of test environment usage plans / schedule management

- Creation of test environment switching plans / schedule management

- Tes t env i ronment con f igu ra t ion management, including configuration information

- Creation of test environment switching p r o c e d u r e d o c u m e n t a t i o n a n d enforcement on all parties

• Install a fault management supervisor, and manage the following:

- Creation of failure handling flows and p r ocedu r e documen ta t i on , and enforcement on all parties (flow for entire process, including failure registration, check-out, check-in, program revision and test, test environment, program release, and reporting) Clarification of responsibilities of testers, program revision personnel, library managers, and fault managers.

- Creation of revised program release schedules, and follow-up of results

• C r e a t e t e s t o p e r a t i o n f l o w a n d procedures, and enforcement on all parties.

• Explain the process delays incurred by the above, as well as measures to make up for this through the overall process, to the customer, and receive their approval.

• Prepare test environments• Appoint test environment overall

responsible person• Appoint fault management

responsible person• Create test operation flow and

procedure documentation, and enforce them

• Explain recovery measures to customer


• Establish test plans and perform review with experts (test pros, expert in the business application field, customer)

97


69 Service Launch Decision Cannot Be MadeHuman Resources

Progress was delayed significantly, making it impossible to determine service launch.


The headquarters information system department left specification determination whol ly in the hands of the branch’s information system department, so the headqua r te r s i n f o rma t i on sys tem department had no understanding of specification or progress conditions.

• Interview project manager and key personnel, understand actual project progress status, and identify of problems.

• Forecast service launch timing.• Forecast achievement levels resulting

from project reconstruction and restart.• Roughly in parallel with above, identify

absolute minimum service items required by customer, and reschedule.

• Reset service items and service launch timing, and reestablish the meeting structure consisting of the customer, project manager, and members.

• Understand present conditions• Add project manager support

personnel• Reschedule• Reestablish organizational

structure• Establish consensus with

customer regarding functions to be released


• Formulate project plan• Establish system for project

auditing

70 No Overall Gantt ChartHuman Resources

There were Gantt charts for individual development teams, but none to provide an overall birds-eye view.This made it impossible to assess project progress of the system as a whole.


I n s u f f i c i e n t p r o j e c t m a n a g e m e n t capabilities.

• Replace project manager or assign project manager support personnel.

• Establish system integration team.• Reconfirm the t iming the customer

requires the service.• Formulate and implement system

i n t e g r a t i o n m i g r a t i o n s c h e d u l e , organizational structure, and work items (WBS).


• Replace project manager• Adjust required functions with

customer• Reschedule• Reestablish organizational

structure


• Formulate project plan• Establish system for project

auditing• Conduct process breakpoint

auditing


endix

98

71 Lack of Project Unity between Customer and DevelopmentHuman Resources

There was a lack of project unity between the customer and development sides of the project.


Project organizat ional structure on development side is insufficient.

• Establish a project promotion meeting. The meeting structure shall include key personnel and experts from the customer side and development side and shall provide directions and advise with regard to the project through deliberation of critical issues, etc.

• R e o r g a n i z e d e v e l o p m e n t s i d e organizational structure. Request the placement of members with business application knowledge and management support by PMO members.

• Revise schedule, such releasing functions in stages. Include customer side key personnel in schedule revision, both in order to gain consensus and to foster a uni f ied mindset regard ing pro ject handling.

• Request customer side key personnel participation in progress meetings, and promote information sharing.

• Foster a sense of unity by establishing a meeting structure



• Pursue team building• Plan communication rules• Establish means for unofficial

communication

72 No Project Experience with Open SystemsScope

Due to lack of experience with open system construction, system quality suffered, and project delays occurred.


O p e n s y s t e m d e s i g n , e s p e c i a l l y distributed processing system design incorporat ing mult ip le vendors and products, requires advanced technical and management ability levels, as it involves many product vendors. However, the general management department lacked experience and personnel, and was unable to coordinate the project.

• Move personnel with experience in open systems, and open system specialists, into the project immediately.

• Seek advice from people experienced in integrated use of open system product lines.

• If there are insufficient suitable human resources within the company, seek for them outside the company.

• Get hold of personnel with experience in each product that will be used. Establish r e l a t i o n s h i p s w i t h t h e s u p p o r t depa r tmen ts o f p roduc t vendors (developers, distributors). Alternatively, sw i t ch the deve lopmen t t eam to personnel experienced in open systems.

• Add experts• Create a support structure

involving product vendors


• Improved risk management• Pursue team building

99


73 Development Started without Finalizing Requirements SpecificationsScope

Requirements specifications were not finalized, and there was no requirements specification documentation, when development started, resulting in the collapse of the project.


P r o j e c t m a n a g e m e n t a b i l i t y w a s insufficient. In recent small-scale projects, it has become common to start development w i t h o u t f i n a l i z i n g r e q u i r e m e n t s specifications. Requirements specification documentation is often created after the fact.

• In principle, development should not start without f irst creating requirements specifications. Stop development, and establish a requirements definit ion f ina l i za t ion task . Use the des ign documentation created up to that point in order to create requirement specifications in a minimal amount of time.

• Use prototyping to quickly create primary function screens, forms, etc., and check actual requirements specifications using the prototype.

• If the development scale is small (7 people or less), switch to the XP (Extreme Programming) approach.

• Temporarily stop development• Finalize and document

requirements specifications• Create a prototype


• Establish development process (include in development plan documents)

• Establish consensus with customer regarding development process

74 The Entire Project Is Plagued by a Feeling of ExhaustionTime

The entire project is plagued by a feeling of exhaustion, and a precipitous drop in motivation is evident just by observing the workplace.


There were many causes for the exhaustion and reduced motivation, but principal among them were the fact that there was no end in sight for the project, with day after day spent in testing.

• Communicate with team members to determine the causes of their exhaustion and lack of motivation.

• Reconfirm project objectives and status, establish objective attainment oriented work prioritization and optimization, and revise plans to be achievable.

• Reestablish shared understanding by members of critical paths and milestones (reconfirm goals).

• Reconfirm project objectives• Communicate with team members• Prioritize work• Revise plan• Establish goals


• Establish thorough labor management

• Pursue team building


endix

100

75 Quality of Externally Procured Package Product Is Low, Making Them Unusable

Procurement

The quality of externally procured package product was low.


Before procur ing external package products, the service level they are target ing should be conf irmed. The problem in this case was caused by using a package product that was not intended for use in mission critical situations.

• When using package products, always enter into a support contract with the company supporting the package product, creating a structure with no possibility of neglecting the obligation.

• When using products from overseas compan ies , pe r f o rm p re l im ina ry investigation of product usage limitations and application limitations.

• Ensure that the customer sufficiently understands that customer business process changes may be necessary when using package products.

• Enter into a support contract• Investigate package• Share information with customer


• Preliminary survey (service level)• Confirm contractual terms with

package vendor

101


76 Low Program QualityQuality

The program quality was low, with massive bugs detected during the test.


Insuff ic ient programming ski l ls and insufficient reviewing.

• Perform at least one source code review f o r each p rog rammer to i den t i f y insufficiently skilled programmers. Identify the problems in the reviewed programs, and decide on corrective policies and measures. If the load incurred by reviews is high, use third party reviewers. Use static source code analysis tools for style checking.

• Co r rec t a l l p rog rams c rea ted by insuff iciently ski l led programmers. Correction work should be performed by a skilled programmer within the same team.

• Remove the insu f f i c ien t l y sk i l led programmer from programming duties, and use a skilled programmer within the same team to perform failure handling program corrections. If possible, use the programmer who has been removed from programming duties to perform pre-release correct ion conf irmation of corrected programs, and to perform a regression test.

• Perform an integration test again on programs corrected after source code review, correct any bugs, and increase quality.

• Explain the process delays incurred by the above, as well as measures to make up for this through the overall process, to the customer, and receive their approval.

• Implement code review and peer review

• Correct program and retest• Identify insufficiently skilled

programmer


• Check programmer skills• Encourage reviews• Enrich description methodology of

documents• Formulate and enforce coding

conventions


endix

102

77 Inadequate Performance DesignQuality

There was no architecture design to ensure responsiveness and eliminate program non-responsiveness, nor were traffic volumes or traffic patterns understood. As a result, performance problems became evident during the qualification test.


Many engineers took a simple stance that performance problems could be resolved by adding hardware. However, performance problems should have been considered during the design phase.

• Understand performance problem status• In many cases, involve performance

tuning specialists.• Establish a performance support team.• Con f i rm , w i th the cus tomer, any

d i f f e r e n c e s b e t w e e n t h e i n i t i a l performance model and traffic conditions and the current performance model and traffic conditions.

• Rees tab l i sh sys tem per fo rmance requirements.

• Ident i fy and resolve per formance bottlenecks.

• Conduct per fo rmance tun ing and performance confirmation tests.

• Add performance problem specialists

• Confirm performance requirements

• Identify and analyze bottlenecks• Consider and implement

corrective measures• Establish consensus with

customer regarding performance requirements


• Review of architecture design by experts

• Confirm performance requirements with customer

New “MIERUKA Visualization - IPA · 2019. 10. 25. · 2 Appendix MIERUKA (Visualization) Tools...

Documents

Transcript of New “MIERUKA Visualization - IPA · 2019. 10. 25. · 2 Appendix MIERUKA (Visualization) Tools...