Post on 12-Jun-2020
Software Certification
Autores
Antonio José Segovia Henares
Carlos Augusto Sánchez Martelo
Henry Leonardo Avendaño Delgado
Manuel Antonio Sierra Rodríguez
Carlos Andrés Collazos Morales
Domingo Alirio Montaño Arias
Breed Yeet Alfonso Corredor
José Daniel Rodríguez Munca
Edición
Editorial Universidad Manuela Beltrán
Autores
Antonio José Segovia Henares
Magíster en Seguridad de la
Información, Ingeniería Técnica
Informática Sistemas por la
Universitat Oberta de
Catalunya, Perito Informático,
Especializado En Hacking
Ético, AENOR S16, IS 05, IS
02, IS 01
Carlos Augusto Sanchez
Martelo
Dr. (c) en Pensamiento
Complejo, Maestría en Diseño,
Gestión y Dirección de
Proyectos, Ingeniero de
sistemas, Certificado
Internacionalmente en ITIL Foundation v3,
Procesos en Desarrollo de Software y TIC
Henry Leonardo Avendaño
Delgado
Dr. (c) en Educación línea de
investigación Tecnologías de
la Información y
Comunicación para la
inclusión, Magister en
Educación, Especialista en Gerencia de
Telecomunicaciones, Ingeniero Electrónico.
Manuel Antonio Sierra
Rodríguez
Dr. (c) en Proyectos en la línea
de investigación en
Tecnologías de la Información
y Comunicación, Magíster en
Software Libre, Especialista en
Seguridad en Redes, Ingeniero de Sistemas,
Consultor en Seguridad de la Información y
Comunicaciones.
Domingo Alirio Montaño Arias
Dr. En Genética, Magister en
Biología, Biólogo, Investigador
Asociado, Universidad Manuela
Beltrán, BSc, MSc, PhD
Intereses de investigación en
Neurociencias, Genética y TIC
Aplicadas a la Educación. Miembro comité
editorial revista Journal of Science Educations.
Miembro fundador de la Sociedad Iberoamericana
de Biología Evolutiva.
Carlos Andrés Collazos
Morales
Postdoctorado en Ciencia y
Tecnología Avanzada, Dr. en
Ciencias, Magister en
Ingeniería Electrónica y
Computadores, Físico.
Breed Yeet Alfonso Corredor
Dr. (c) en Proyectos, Magister
en Educación, Especialista en
Desarrollo e Implementación
de Herramientas Telemáticas,
Ingeniera Electrónica,
Directora Académica y
Calidad, Consultora Nacional e Internacional
Académica de Educación Superior.
José Daniel Rodríguez Munca
Magister en Ciencias de la
Educación, Master en
Estrategias y Tecnologías
para el Desarrollo,
Especialista en docencia
mediada por las TIC e
Ingeniero Electrónico.
Daniela Suarez Porras
Corrección de estilo (Editor secundario)
Diagramación: Beatriz del Pilar Mejía
Diseño de portada: Beatriz del Pilar Mejía
Publicado en Diciembre de 2018
Formato digital PDF (Portable Document Format)
Editorial Universidad Manuela Beltrán
Avenida Circunvalar Nº 60-00
Bogotá – Colombia
Tel. (57-1) 5460600
Antonio José Segovia Henares, Carlos Augusto Sánchez Martelo,
Henry Leonardo Avendaño Delgado, Manuel Antonio Sierra
Rodríguez, Carlos Andrés Collazos Morales, Domingo Alirio
Montaño Arias, Breed Yeet Alfonso Corredor, José Daniel
Rodríguez Munca
Software Certification, Bogotá, UMB
© Antonio José Segovia Henares, Carlos Augusto Sánchez Martelo,
Henry Leonardo Avendaño Delgado, Manuel Antonio Sierra
Rodríguez, Carlos Andrés Collazos Morales, Domingo Alirio
Montaño Arias, Breed Yeet Alfonso Corredor, José Daniel
Rodríguez Munca
© Universidad Manuela Beltrán
Bogotá, Colombia
http:// www.umb.edu.co
Queda prohibida la reproducción total o parcial de este libro por
cualquier proceso gráfico o fónico, particularmente por fotocopia,
Ley 23 de 1982
Software Certification. / Antonio José Segovia Henares… (y otros 7) - Bogotá:
Universidad Manuela Beltrán, 2018.
108 p.: ilustraciones, gráficas, tablas; [versión electrónica]
Incluye bibliografía
ISBN: 978-958-5467-21-7
1. Desarrollo de software 2. Patrones de diseño de software 3. Ingeniería de
Software. . i. Sánchez Martelo, Carlos Augusto. ii. Avendaño Delgado, Henry
Leonardo. iii. Sierra Rodríguez, Manuel Antonio. iv. Collazos Morales, Carlos
Andrés. v. Montaño Arias, Domingo Alirio. vi. Alfonso Corredor, Breed Yeet. vii.
Rodríguez Munca, José Daniel.
005.068 cd 23 ed.
CO- BoFUM
Catalogación en la Publicación – Universidad Manuela Beltrán
Autoridades Administrativas
Gerente
Juan Carlos Beltrán Gómez
Secretario General
Juan Carlos Tafur Herrera
Autoridades Académicas
Rectora
Alejandra Acosta Henríquez
Vicerrectoría de Investigaciones
Fredy Alberto Sanz Ramírez
Vicerrectoría Académica
Claudia Milena Combita López
Vicerrectoría de Calidad
Hugo Malaver Guzman
ISBN: 978-958-5467-21-7
13
TABLE OF CONTENTS
Software Certification
Capítulo 1: Quality of the Process ........................................................................................... 19
1. Quality of the Process ........................................................................................................ 19
1.1. Introducción Quality of the process ............................................................................ 19
1.2. Conceptual Framework ................................................................................................... 19
1.2.1. Introduction to the Maturity Model ....................................................................................... 19
1.2.2. Approach About SPICE ........................................................................................................... 19
1.2.3. ISO/IEC 15504 structure .......................................................................................................... 19
1.2.4. Software life cycle processes – ISO/IEC 12207 ................................................................ 20
1.2.5. SPICE certification ................................................................................................................... 20
1.3. Examples (not to exceed 2 pages) ............................................................................... 20
1.4. Exercises (not to exceed 2 pages) ............................................................................... 20
1.5. Conclusions (not to exceed 1 page) ............................................................................ 20
1.6. Maturity Model ................................................................................................................... 21
1.6.1. Introduction to the Maturity Model....................................................................................... 21
1.6.2. Approach About SPICE ........................................................................................................... 26
1.6.3. Structure of ISO/IEC 15504 .................................................................................................... 31
1.6.4. Software Life Cycle Processes – ISO 12207 ...................................................................... 40
1.6.5. SPICE Certification ................................................................................................................... 50
Capítulo 2: Quality of the Product and Service .................................................................... 59
2.1. Introducción ....................................................................................................................... 59
2.2. Conceptual Framework ................................................................................................... 59
2.2.1. Introduction ................................................................................................................................ 59
2.2.2. Quality of the Software Product ........................................................................................... 59
2.2.3. Main Standards About the Quality of the Software Product ......................................... 60
2.2.4. Standard ISO/IEC 25000 (SQuaRE) ...................................................................................... 60
2.2.5. Tools for the Assessment of the Quality of the Software Product .............................. 60
2.2.6. Example of an Environment for the Evaluation of the Software Product .................. 60
2.2.7. Introduction to ISO/IEC 20000 ............................................................................................... 60
2.2.8. Structure of the ISO/IEC 20000 ............................................................................................. 61
2.2.9. Global management of ISO/IEC 20000 ................................................................................ 61
2.2.10. Model and key areas .............................................................................................................. 61
14
2.2.11. Exercises .................................................................................................................................. 61
2.2.12. Conclusions ............................................................................................................................. 62
2.3. Quality of the Product ..................................................................................................... 62
2.3.1- Introduction ............................................................................................................................... 62
2.3.2. Quality of the Software Product ........................................................................................... 63
2.3.3. Main Standards About the Quality of the Software Product ......................................... 65
2.3.4. Standard ISO/IEC 25000 (SQuaRE) ...................................................................................... 77
2.3.5. Tools for the Assessment of the Quality of the Software Product .............................. 83
2.3.6. Example of an Environment for the Evaluation of the Software Product .................. 84
2.3.7. Quality of the Service (ISO 20000) ....................................................................................... 87
2.3.8. Introduction to ISO/IEC 20000 ............................................................................................... 87
2.3.9. Structure of the ISO/IEC 20000 ............................................................................................. 88
2.3.10. Global Management of ISO/IEC 20000 .............................................................................. 91
2.3.11. The PDCA model and key areas ......................................................................................... 97
15
Foreword
Anyone with basic knowledge about programming can develop an application,
and this implies that you can find all kinds of solutions in the market.
The problem is that not everyone applies quality controls when developing
software. Imagine that we want to manufacture a new type of car, by copying
designs from other cars, and without establishing some minimum controls of
security. We can probably sell many units; but when any disaster occurs, problems
may arise.
Therefore, it is important to define a quality model, in terms of software
engineering, with the aim of establishing some controls that ensure the quality of
the developments. As far as it is the main objective of this text, we will focus on
quality models for software processes, products, and services.
We can consider a process for the measurement of the quality, but it is not only
the unique parameter, we can also consider directly the product, and also the
service that the company offers (generally related to the development and
maintenance of software).
So basically during this module, we will see new standards that can help us for
the establishment of quality in our product, and in our service.
17
Capítulo I
Qu
alit
y o
f th
e P
roce
ss
Introduction to the Maturity Model
Approach About SPICE
ISO/IEC 15504 Structure
Software Life Cycle Processes – ISO 12207
SPICE Certification
Quality of the Process
19
CAPÍTULO 1: QUALITY OF THE PROCESS
In this module, we will see the software quality for processes, so we will cover the
SPICE model, which is composed mainly by the ISO/IEC 15504 (evaluation) and
ISO/IEC 12207 (processes) standards.
1. Quality of the Process
1.1. Introducción Quality of the process
In terms of quality, you cannot improve a product or a service if you do not
improve the processes involved.
There are many specific processes in the software area, that can be used to
improve the product or the service, so we will see different models and their
processes.
1.2. Conceptual Framework
1.2.1. Introduction to the Maturity Model
The maturity model allows us to know the maturity of an organization, based on
its processes, in relation to the software quality. However, there is also a model of
capacity, which measures the capacity of each of the software processes within the
organization.
1.2.2. Approach About SPICE
The SPICE model allows to evaluate the maturity and capacity of an organization.
It is similar to CMMI, but it has a big difference. SPICE is based in an ISO
standard, and it has a certification scheme. SPICE defines mainly 6 levels for the
evaluations.
1.2.3. ISO/IEC 15504 structure
ISO/IEC 15504 is composed by 10 different parts, which can be used to
understand the standard better, and to know how to implement it.
20
1.2.4. Software life cycle processes – ISO/IEC 12207
ISO/IEC 12207 defines a group of software processes, which can be used to
improve the quality. SPICE can be based on these processes to perform the
evaluation.
1.2.5. SPICE certification
If you have already implemented SPICE, the next step is to certificate it.
Depending on the processes implemented, you will get a level according to the
SPICE model.
1.3. Examples (not to exceed 2 pages)
AENOR model:
http://www.aenor.es/AENOR/certificacion/calidad/calidad_software_15504.asp#.
WCWsltxhrAo
1.4. Exercises (not to exceed 2 pages)
1.- There are many companies developing software, but do all companies have
the same model for quality system?
2.- If two companies use the same model, do these companies have the same
level for quality?
3.- How can we prove that we have a standard –related to the software quality-
implemented in the organization?
1.5. Conclusions (not to exceed 1 page)
By improving software processes, software quality can be also improved, so our
customers will be more satisfied with our service.
21
1.6. Maturity Model
1.6.1. Introduction to the Maturity Model.
In the software engineering industry, errors are typical. Probably, your computer
or any application you use to surf the internet has shown you an error message
more than once, not allowing you to work with it.
These errors were much more common in the past, because few people worked
on the quality of the product.
In fact, in most of the systems developed in the past, quality focused on detecting
errors in a test phase, once the software was developed.
Nowadays, systems have improved and error detection of errors is not the only
important aspect, but also the definition of process as well as to use them for the
development of the software, in order to improve the quality of the developed
product.
Therefore, it is important to test the product once it has been developed; but it is
also very important to establish a series of processes, to be used for the
development of the product, to ensure the quality of the software.
Figure 1 – Software quality
Software quality
Software
Software process
Error detection
22
The concept of software process includes all the activities that are carried out, the
methods that are defined, and the practices that can be used for the software
development and maintenance.
A process can be basically compared to a black box, where a series of activities
can be carried out, taking a series of entries as the basis, and providing a series of
results as the output.
Figure 2 - Process
Therefore, the activities will provide results on the basis of some parameters.
These activities may consist of the following actions:
Establishment of a methodology and a systematic common work.
Setting of the different roles (developers, analysts, head of project, etc.)
Assignment of responsibilities (for example: programming developers,
analysts for the review of the source code, project manager for the
coordination of the project implementation, etc.)
Definition of the technologies to be used (for example: C++ and Java for
the logic of business, PHP for the web layer, etc.)
Inputs Activities Outputs
23
The main objective of these processes is the development of a final quality
product, which is obviously much better in order to satisfy the needs of current
customers, or to be able to get the attention of future clients.
You can also see this process approach as an opportunity for the positioning on
the market: not all organizations that develop software are really worried about the
quality of the product to develop. Bearing in mind the processes’ approach, we can
give greater quality guarantees.
On the other hand, to ensure that the processes improve the quality of the
product, they have to meet certain requirements:
They have to agree with the results that were expected, or established.
They must be based on a proper definition of the process.
They have to be constantly improved, depending on the organization
objectives.
In addition to gaining a competitive advantage, the processes approach can
provide cost savings, since it may allow an organization to establish an order,
which will involve a significant optimization of the tasks that are carried out within
the organization. This will lead to a time reduction and has a clear effect on the
costs (it is not the same to run a project in 12 months, than in 6 months).
An organization that does not define its processes or does not define them
properly, does not measure, control or improve them, is considered as an
immature organization. Obviously, if the company meets all these aspects, it is
considered a mature organization.
Therefore, the best way of ensuring the quality of a software product is to define
processes by establishing a method to measure control and improve them.
24
Figure 3 – Steps of the process software
In this way, a mature organization, with software-defined, measured, controlled,
and improved processes, will develop software according to the required
specifications, agree with the time limits that were set forth in the project plan,
achieve customer satisfaction, get the staff of the project team united, coordinated
and functioning all in one line.
Another advantage of mature organizations, is that the knowledge is usually kept
in the organization (through defined processes), while immature organizations are
made up by people who can possibly change their company, and therefore bring
their knowledge elsewhere. So they may leave a disclosure of knowledge within
the organization (a company may not depend on one or more people, it must have
its own work strategies, etc.).
Other of the advantages provided by this model, is that different levels of maturity
Defining the process
Measuring the process
Controlling the process
Improving the process
25
can be established, in order to know the degree of maturity of an organization in
relation to its processes.
On the other hand, you can also measure the capacity of each process, i.e., the
degree of capacity that each individual process has.
So, two main models of evaluation can be found:
Assessment of capacity model: Focused on each individual process.
Evaluation of maturity model: Focused on all the processes within the
organization
The evaluation of the maturity model is generally more common, since it gives a
global picture of the organization. However, these models are not incompatible,
i.e., they are often used in a parallel way; thus, the level of maturity of the
organization, and the level of capacity for each of their processes can be known.
Figure 4 – Capacity and maturity models
Therefore, it can be said that the quality of a product will be determined by the
quality of the processes used for its development and maintenance.
• Individual processesCapacity
evaluation
• All processesMaturity
evaluation
26
1.6.2. Approach About SPICE
From now on, it is clear that within the software engineering field, the process
model is a necessary step if we want to offer quality products, as well as to have a
powerful tool to be able to be competitive in the market.
But, how can an organization prove that it has its own processes, and uses them
to develop quality software? The answer is, by getting a certification.
The CMMI-DEV certification offers further insights about the maturity of the
organization, i.e., it uses a model of evaluation of the maturity to determine the
maturity of processes, and it is well known in the software engineering industry.
However, this certification is also known to be especially expensive, and in many
cases, it can also be considered complex, especially for small organizations (that
abound in this sector).
It can be interesting to consider other options, for example SPICE (Software
Process Improvement and Capability Determination), which is very similar to
CMMI-DEV.
SPICE is based on the standard ISO/IEC 15504, and it is used to certify the
maturity of processes in software development, which is basically the same thing
that makes CMMI-DEV.
The difference is that the SPICE certification tends to be cheaper, and also easier
to evaluate. In addition, being based on an ISO standard, makes it easily
integrated with other ISO standards, as for example ISO 27001 (information
security), ISO 22301 (business continuity), ISO 20000 (IT services), ISO 9001
(quality), etc.
27
Another important difference is the certification scheme. In the case of CMMI-
DEV, assessments (known as "appraisal"), are made via the SCAMPI method
(Standard CMMI Appraisal Method for Process Improvement), which identifies
strengths and weaknesses in the processes, and can reveal risks of development.
SPICE follows the certification scheme of management systems: ISO standards,
so that annual audits are performed (there is an initial audit, consisting of a stage 1
and a stage 2, and in the following years, surveillance audits are performed each
year, and a recertification audit is performed every 3 years).
Therefore, since SPICE it is easy to understand and implement, and it is cheaper,
we will give more information about this standard throughout this module.
The different levels that the SPICE model established (you already know that it is
based on ISO/IEC 15504) for capacity are the following:
Figure 5 – Capacity levels
• The process is improved continuouslyLevel 5: Optimized
• The process is managed using quantitative techniquesLevel 4: Predictable
• An adapted process is used, based on a standard processLevel 3: Established
• The process is managedLevel 2: Realizado
• The process is performed and there are clear evidencesLevel 1: Performed
• The process is not completeLevel 0: Incomplete
28
On the other hand, the levels of maturity that ISO / IEC 15504 defines are the
following:
Figure 6 – Maturity levels
The majority of organizations certified in the SPICE model, are found level 2 and
3, being the level 2 the first certifiable level. So, levels 0 and 1 are not certifiable
(especially level 0, which indicates that the processes are not implemented).
On the other hand, the ISO/IEC 15504 also relies on another standard, the
ISO/IEC 12207, which defines a common framework for software life cycle
processes. We will talk more in detail about this standard later.
•The processes are improved continuouslyLevel 5: Optimized
•The processes are managed using quantitative techniques
Level 4: Predictable
•The organization uses adapted processes, based on standards
Level 3: Established
•The processes are managedLevel 2: Managed
•The organization implements the processes and achieves the objectivesLevel 1: Basic
•The organization has not implemented the processesLevel 0: Immature
29
Therefore, an organization of software development based on the SPICE model,
can implement the ISO/IEC 12207 processes.
There is a specific SPICE model focused in the automotive industry, which is
known as Automotive SPICE, and it is composed by specific processes of this
industry.
CMMI-DEV defines the following levels of maturity:
Figure 7 – Capacity levels of CMMI-DEV
As you can observe, it is very similar to the SPICE model. Therefore, it is very
simple to map their levels.
• The processes are improved continuouslyLevel 5:
Optimized
• The processes are measured and controlledLevel 4:
Managed
• The processes are characterized by the organization, and it acts proactively
Level 3: Defined
• The projects are characterized by the processes, and the organization often acts reactively
Level 2: Repeatable
• The processes are unpredictable, poorly controlled, and the organization acts reactivelyLevel 1: Initial
30
Finally, as it was mentioned in a previous section, the SPICE certification scheme
is similar to the ISO management systems certification scheme (ISO 27001, ISO
20000, ISO 9001, ISO 14001, ISO 22301, etc). Therefore, a certification audit of
SPICE is composed by the following types of audit:
Initial audit: Composed by stage 1 (the audit is planned and the
documentation is reviewed), and stage 2 (the audit is performed to verify
that the organization has implemented the system, in this case the SPICE
model).
Surveillance audit: Similar to stage 2 of the initial audit, i.e. the audit is
conducted to check that the organization maintains the system.
Recertification audit: The certificate issued by a certification body usually
has a duration of 3 years, after this period, it is necessary to renew it, and
to repeat the certification process again (without stage 1 and stage 2, that
is, there will be surveillance audits each year, until a new recertification
audit is done.
Figure 8 – Main types of certification audits
Initial auditSurveillance
audit
Recertification audit
31
After each audit, the lead auditor is responsible for issuing a final report with the
results of this audit. This report may contain findings that the company needs to
treat, developing a corrective actions plan.
Once the organization presents the corrective actions plan, if the lead auditor
validates it, he gives a decision for the certification to the certification body, and if
everything is all right, the certification body gives the certificate to the company (it
is valid for 3 years).
The organization must ensure, during the validity time of the certificate, to keep
the system implemented and maintained, and may lose certification if during any
surveillance or recertification audit, it has not worked on the weaknesses identified
during an audit.
There is another type of audit, an unusual one, which is called extraordinary
audit. It is used in those cases where very important findings are detected and the
certification body must verify in situ, after a normal audit, if the organization has
treated and solved these very important findings.
These audits are performed by qualified auditors within certification bodies. They
also have to conduct audits according to another standard: the ISO/IEC 17021. In
other words, this standard defines requirements for the certification bodies and the
activities carried out during the certification audit.
1.6.3. Structure of ISO/IEC 15504
The ISO/IEC 15504, consists of 10 parts.
a.- ISO/IEC 15504-1:2004 Information technology. Process assessment. Part
1: Concepts and vocabulary
32
This first part of the standard provides information about basic concepts related to
the evaluation of processes. On the other hand, it also provides information about
the terminology used in the standard for the evaluation of processes.
Therefore, this standard provides information about basic concepts and
vocabulary that is used in all the standards for the evaluation of the processes.
b.- ISO/IEC 15504-2:2003 Information technology. Process assessment. Part
2: Performing an assessment
It sets the minimum requirements to carry out the evaluation of processes, so it
tends to be used by certification bodies to perform certification audits.
For the evaluation of the process, this part of the standard is based on a model
composed by 2 dimensions:
Dimension of the process: It relies on an external reference model
(usually ISO/IEC 12207, which we will see later), for the definition of a set
of processes.
Dimension of capacity: It defines 6 levels of capacity (which we saw in a
previous chapter), to establish a framework for the measurement of
processes.
Therefore, this allows us to have a series of processes of an external model
(usually ISO/IEC 12207 processes, that are specific to software), and a mechanism
for measuring, composed by 6 levels of capacity.
33
Level 5
Level 4
Level 3
Level 2
Level 1
Level 0
Process A Process B Process C Process D
Figure 9 – 2 dimensions model
On the other hand, to be able to measure the capacity of each process, it is
necessary to use what is known as process attributes.
These process attributes are grouped according to the different levels of capacity;
each attribute measures a particular aspect of the ability of each process.
In this way, we can see the following:
Capacity level ID of the process attribute
Process attribute
Level 0 Incomplete process
N/A N/A
Level 1 Performed process
PA 1.1 Realization of the process
34
Level 2 Managed process
PA 2.1 Management of the realization
PA 2.2 Work product management
Level 3 Established process
PA 3.1 Definition of the process
PA 3.2 Deployment of the process
Level 4 Predictable process
PA 4.1 Measurement of the process
PA 4.2 Control of the process
Level 5 Optimizing
PA 5.1 Innovation of the process
PA 5.2 Continual optimization
Table 1 – Capacity levels and process attributes
It is also appropriate to highlight that an external model can be used for the
definition of processes, but this model has to meet the following requirements:
It must contain documentation of the community that is interested in the
development of the model, and the actions performed to reach a
consensus within this community (this requirement is met by ISO/IEC
12207, given that it has been developed and maintained, by ISO.org).
The processes defined within the model, must have a description and a
unique process ID. In addition, the descriptions of the processes must
describe the general objectives that are trying to achieve with the
processes at a high level, as well as the results that can show that the
process is achieved (it is also complied by ISO/IEC 12207).
c.- ISO/IEC 15504-3:2004 Information technology. Process assessment. Part
3: Guidance on performing an assessment
It provides guidance for an evaluation using ISO/IEC 15504-2 as the reference,
providing an overview of the evaluation of processes.
In addition, this part of the standard also offers a guide to establish the framework
for the process capability measurement, oriented in the selection and use of
assessment tools. It helps to define competencies for evaluators, and also helps
with the verification of the conformity of the assessment.
35
On the other hand, this part of the standard also provides an example of
evaluation according to the ISO/IEC 15504-2 requirements, i.e., the part 2 of the
ISO/IEC 15504.
d.- ISO/IEC 15504-4:2004 Information technology. Process assessment. Part
4: Guidance on use for process improvement and process capability
determination
This part of the standard also provides guidance, but in this case the guide helps
to improve the processes, and also helps to determine the capability of a process.
If we make an analysis of a process based on their results (this is also known as
a result of the process or outcomes), and cross them with the business objectives
of a unit or organization area, we can identify the strengths, weaknesses and risks
that are associated to the processes (it can also help us to determine the ability of
the processes).
Therefore, this analysis can basically help us to determine the ability of the
processes, and the effectiveness of processes in achieving the business’
objectives, which can be used to identify and make improvements on the
processes.
e.- ISO/IEC 15504-5:2012 Information technology. Process assessment. Part
5: An exemplar software life cycle process assessment model
This part of the standard basically shows an example about how to perform an
assessment according to the requirements of part 2 of the ISO/IEC 15504, i.e. in
accordance with the ISO/IEC 15504-2.
The evaluation also relies on the model of the ISO/IEC 12207 processes, and it
establishes a series of indicators for process capability, which can be used to know
whether has achieved the established capacity.
36
f.- ISO/IEC 15504-6:2013 Information technology. Process assessment. Part
6: An exemplar system life cycle process assessment model
This part of the standard is similar to part 5, so this also gives an example about
how to perform an assessment according to the requirements of the ISO/IEC
15504-2; but in this case, it is only as a process reference model the standard
ISO/IEC 15288.
The main difference between ISO/IEC 12207 and ISO/IEC 15288 is that the first
one is focused on software life cycle processes, while the second one focuses on
the system life cycle processes.
Therefore, the main difference between ISO/IEC 15504-6 and ISO/IEC 15504-5,
is that the first one is focused on a model of evaluation of the system life cycle
processes, while the ISO/IEC 15504-5 focuses on a model of evaluation of the
software life cycle processes.
g.- ISO/IEC TR 15504-7:2008 Information technology. Process assessment.
Part 7: Assessment of organizational maturity
This is one of the most important parts of the standard (together with parts 1 and
2), and it mainly establishes part of the standard the minimum requirements for an
assessment of the maturity of an organization. This assessment of maturity relies
mainly on the evaluation of the ability of the processes, so it has an important link
with the ISO/IEC 15504-2.
In this way, an array like the one shown below can be used in order to determine
the level of maturity of an organization:
37
ML3
ML4
ML5
ML2
ML1
1A 1B 1C 2D 2E 2F 3D 3E 3F 4D 4E 4F 5D 5E 5F
Table 2 – Maturity levels
In the vertical line, you can see different levels of capacity (level 1, level 2, level 3,
level 4 and level 5), while in the horizontal line, you can see different categories of
the processes.
Representing each value as follows:
xA: minimum processes to the level of maturity n
xB: additional processes required for the level of maturity n
xC: optional processes for the maturity level of n
Where "n" indicates the level of corresponding maturity.
The crossing of the level of capacity with the corresponding process group,
determines the level of maturity of the organization.
In this way, for example, if the organization wants a level of maturity 3, the
organization should take all processes 1A, 1B, 1C, 2D, 2E, 2F, 3D, 3E and 3F to
reach the level of capacity 3.
Level 5
Level 4
Level 3
Level 2
Level 1
38
On the other hand, as you can see in the chart, to reach the level of maturity 4, it
is not necessary that all processes reach a level of ability 4 (for example, for the
level 4, are not necessary: 1B, 3E, 4D, and 4F).
With regards to the level of maturity 5, you can also see in the chart, that the
processes that are required to reach a level of capacity 5 are 1A, 2E, 3D and 4E,
although the latter is optional.
Therefore, the matrix above can be useful to determine the level of maturity of an
organization based on the levels of each process.
h.- ISO/IEC TS 15504-8:2012 Information technology. Process assessment.
Part 8: An exemplar process assessment model for IT service management
This part of the standard also provides an example of the evaluation of
processes; in this case, it is focused on the ISO/IEC 20000 processes (we will see
this standard in the next module, but it basically establishes the requirements to
define an IT service management system).
For the evaluation of the ISO/IEC 20000 processes, the standard is based on
ISO/IEC 15504-2.
i.- ISO/IEC TS 15504-9:2011 Information technology. Process assessment.
Part 9: Target process profiles
This part of the standard establishes a number of requirements to be able to
determine the ability to achieve the organization's specific needs, using process
profiles.
j.- ISO/IEC TS 15504-10:2011 Information technology. Process assessment.
Part 10: Safety extension
39
This part of the standard adds a security component as an extension to the
previous parts.
Figure 9 – Parts of ISO/IEC 15504
•Conceps and vocabularyISO/IEC 15504-1
•Performing an assessmentISO/IEC 15504-2
•Guidance on performing an assessmentISO/IEC 15504-3
•Guidance on the use for process improvement and process capability determinationISO/IEC 15504-4
•An exemplar software life cycle process assessment modelISO/IEC 15504-5
•An exemplar system life cycle process assessment modelISO/IEC 15504-6
•Assessment of organizational maturityISO/IEC 15504-7
•An exemplar process assessment model for IT service managementISO/IEC 15504-8
•Target process profilesISO/IEC 15504-9
• Safety extensionISO/IEC 15504-10
40
1.6.4. Software Life Cycle Processes – ISO 12207
So far, we have mainly focused on processes, but we need to know what
processes can be used together with ISO/IEC 15004 for the evaluation.
As you know, in this context, the ISO/IEC 12207 standard provides a model of the
software life cycle processes.
This standard consists of the following groups of processes:
Agreement processes
Organizational project-enabling processes
Project processes
Technical processes
Software implementation processes
Software support processes
Software reuse processes
Each of these groups, defines specific processes, which are described below.
Figure 10 – Agreement processes
Acquisition process
Supply process
Agreement processes
41
Acquisition process: Its main aim is to obtain the product or service in
accordance with the requirements established by the customer. So,
basically this process essentially identifies the needs of the client.
Supply process: Its main purpose is to provide the customer with a
product or service that is appropriate to their needs and requirements.
Figure 11 – Organizational project-enabling processes
Life cycle model management process: This process mainly provides a
series of procedures of the software life cycle, processes and policies, which are
consistent with the objectives of the organization, and that are defined, adapted,
improved and maintained to support the individual needs of a project, within the
context of the organization.
Life cycle model management process
Infrastructure management process
Project portfolio management process
Human resource management process
Quality management process
Organizational project-enabling processes
42
Infrastructure management process: This process helps to define,
provide and maintain the organization facilities, tools and resources of
communication and information technologies that are necessary for the activities
that plays the organization during the software life cycle.
Project portfolio management process: Its main objective is to define and
maintain a series of projects that may be necessary and sufficient for the strategic
objectives of the organization. Therefore, this process helps to make a proper
investment of resources.
Human resource management process: Its main aim is to provide the
human resources that are necessary for the organization. The process is also the
competence of human resources, which must be in accordance with the needs of
the business and helps to have qualified, specialized and experienced staff in the
software life cycle processes in order to help the organization reach its goals.
Quality management process: The main objective of this process is to
ensure the quality of products and services, which have to fulfil the objectives of
the customer as well as their needs.
Figure 12 – Project processes
Project planning process
Project assessment and control process
Decision management process
Risk management process
Configuration management process
Information management process
Measurement process
Project processes
43
Project planning process: The main objective of this process is to define
and communicate project plans. Therefore, the process describes the
scope of management and technical activities of the project, it also
identifies the outputs, tasks and deliverables of the project, and it also
establishes plans to perform the tasks in the project, including the scope,
and the resources needed to perform the tasks in the project.
Project assessment and control process: The aim of this process is to
monitor the status of the project, and ensure that it is performed according
to the established plans. During a review of the project, it can be
necessary to re-plan the project.
Decision management process: This process is intended to determine
the more commensurate decision with regards to the needs of the
organization, when there are different alternatives. Therefore, it intends to
analyze different alternatives, and the final decision is recorded along with
their reasons, for possible future decisions that are similar.
Risk management process: The main objective of this process is to
manage risks during the product, software, service, or system life cycle.
Configuration management process: This process’ main objective is to
establish and maintain the integrity of the outputs of a process or a project,
and make them available to the interested parties.
Information management process: The main objective of this process is
to manage the information. For doing so, it is necessary to provide
relevant, complete, valid, and confidential information in time, to the parties
concerned during the system life cycle (and where appropriate, also after
the life cycle). The information that can manage this process is the
information related to the project, including technical information of the
organization, agreements, and user information.
Measurement process: This process aims to obtain, analyze and report
data concerning products and processes implemented in the organization,
44
to support effective management of the processes, and objectively
demonstrate the quality of the products.
Figure 13 – Technical processes
Stakeholder requirements definition process: This process has as main
objective the definition of requirements for a system. The process also
transforms the needs of stakeholders in a set of requirements.
Stakeholder requirements definition process
System requirements analysis process
System architectural design process
Implementation process
System integration process
System qualification testing process
Software installation process
Software acceptance support process
Software opeation process
Software maintenance process
Software disposal process
Technical processes
45
System requirements analysis process: The aim of this process is to
transform the stakeholder requirements into a set of technical
requirements of the system.
System architectural design process: The main objective of this process
is to establish the system requirements that must be set for each element
of the system.
Implementation process: This process aims to implement an element of
the system.
System integration process: The main objective of this process is to
integrate the different elements of the system, thus producing a complete
system which satisfies the system design and requirements, as well as the
needs of the client. On the other hand, the elements of the system can be
software, hardware, manuals, and if necessary, other systems.
System qualification testing process: The aim of this process is to
ensure that each requirement of the system has been implemented and is
ready to be delivered.
Software installation process: The main objective of this process is to
install the software according to the requirements.
Software acceptance support process: The aim of this process is to get
that customer accepts that the software complies with the requirements.
Software operation process: The main objective of this process is to
operate the software product in the agreed environment, and also provide
help to customers on the operation of the software product they have
acquired.
Software maintenance process: The aim of this process is to provide
maintenance of the software, in a cost-effective way, to deliver the
software in appropriate conditions.
Software disposal process: The main objective of this process is to
remove or delete an element of a system. The removal must be done with
responsibility, and in accordance with applicable law, agreements,
46
restrictions that may exist in the organization and the stakeholders’
requirements.
Figure 14 – Software implementation processes
Software implementation process
Software requirements analysis process
Software architectural design process
Software detailed design process
Software construction process
Software integration process
Software qualification testing process
Software implementation
processes
47
Software implementation process: The main objective of this process is
to implement specific elements of the system. The result of this process is
a software element that meets the design requirements, and the
requirements of the stakeholders.
Software requirements analysis process: The main objective of this
process is to define the requirements of the elements.
Software architectural design process: The aim of this process is to
provide a design for the software, which will be implemented in accordance
with the established requirements and can be verified.
Software detailed design process: The main objective of this process is
to provide a design that implements the requirements. On the other hand,
the architecture of the software has to be sufficiently detailed in order to
allow the software codification and the corresponding tests.
Software construction process: The aim of this process is to build
software units that adequately reflect the design of the software,
Software integration process: The main objective of this process is to
integrate the units of software and the software components. This
integration is required to meet the functional and non-functional
requirements.
Software qualification testing process: The aim of this process is to
confirm that the integrated software product meets the requirements.
48
Figure 15 – Software support processes
Software documentation management process
Software configuration management process
Software quality assurance process
Software verification process
Software validation process
Software review process
Software audit process
Software problem resolution process
Software support
processes
49
Software documentation management process: The main objective of
this process is to develop and maintain the records produced by a process.
Software configuration management process: The main objective of
this process is to define and maintain the integrity of a process or project
software items, and keep them available to the interested parties.
Software quality assurance process: The main objective of this process
is to ensure that products and processes comply with the defined plans.
Software verification process: The main objective of this process is to
verify and confirm that each product, or each service of process or project,
adequately complies with the requirements.
Software validation process: The main objective of this process is to
validate and confirm the requirements for the specific use of a software
product.
Software review process: The main objective of this process is to keep
stakeholders informed on the progress of the objectives established, and
what can be done to ensure that the product meets the needs of the
stakeholders. So the software revisions are made during the life of the
project.
Software audit process: The aim of this process is to determine the
compliance of the requirements, plans and agreements. So, it is necessary
to perform audits.
Software problem resolution process: The main objective of this
process is to ensure that all of the problems detected, are identified,
analyzed, managed, controlled and solved.
50
Figure 16 – Software reuse processes
Domain engineering process: The main objective of this process is to
develop and maintain models, architectures, and resources for the domain.
Reuse asset management process: The aim of this process is to
manage the life of resources that are reusable.
Reuse program management process: The main objective of this
process is to plan, establish, manage, control, and review the reusable
programs of the organization.
1.6.5. SPICE Certification
Taking into account the information in the previous sections, if we wish to certify
an organization in SPICE, we need to work with the following standards:
Domain engineering process
Reuse asset management process
Reuse program management process
Software reuse
processes
51
ISO/IEC 15504-2: It defines the requirements for an evaluation of process
capability
ISO/IEC 15504-7: It defines the requirements for an assessment of the
maturity of an organization
ISO/IEC 12207: It defines a set of processes, structured by groups, for the
software life cycle.
ISO/IEC 17021: It defines the requirements for a certification body that
performs certification audits. So, this is standard needs to be implemented
only by the certification body.
Obviously, the other parts of the standard ISO/IEC 15504 can be also used, as a
support to implement and certify SPICE, but the essential parts are the ones which
have been summarized here.
Figure 17 – Standards for the SPICE certification
It is important to mention that there are different SPICE certification models,
which can have a different selection of processes for each maturity level.
SPICE certification
ISO 17021
ISO 12207
ISO 15504-2 + ISO
15504-7
52
In other words, to achieve each level of maturity, it is necessary to implement
certain processes of ISO/IEC 12207, with a certain level of ability. But what
processes should be taken into account?
For the AENOR certification body, the processes that must be implemented for
the maturity level of 1, 2 and 3 are the following:
Figure 18 – Processes of the AENOR model, maturity level 1
Maturity level 1
Supply process
Life cycle model management
process
Software configuration management
process
53
Figure 19 – Processes of the AENOR model, maturity level 2
Maturity level 2
Stakeholder requirements
definition process
System requirements
analysis process
Project planning process
Project assessment and control process
Configuration management
process
Measurement process
Software quality assurance
process
54
Figure 20 – Processes of the AENOR model, maturity level 3
Maturity level 3
Software requirements analysis process
Software architectural design process
System architectural design process
Infrastructure management process
Human resource management process
Risk management process
Decision management process
Software integration process
System integration process
Software veritication process
Software validation process
55
With this model, an organization can be certified SPICE from level 2 to 5, and to
obtain a level you it must have the previous ones, i.e., to achieve level 3, it is
necessary to implement all processes of the maturity level 3, in addition to those of
level 2, and level 1.
57
Capítulo II
Qu
alit
y o
f th
e P
rod
uct
an
d
Serv
ice
Quality of the Product
Quality of the Software product
Quality of the Service (ISO 20000)
Introduction to ISO/IEC 20000
Comunicaciones por Línea Eléctrica
Quality of the Product and Service
59
CAPÍTULO 2: QUALITY OF THE PRODUCT
AND SERVICE
2.1. Introducción
We have seen in the module 1 that the processes are key to obtain quality, but if
we also focus on the product and the service, considering some new parameters
related to the product, and new processes related to the service, we can obtain
better results.
2.2. Conceptual Framework
2.2.1. Introduction
The major interest of any company is to have satisfied customers, and for that, it
is fundamental to offer quality. So, if the company follows an adequate way
focusing on processes, the product, and the service, will be easier to obtain the
satisfaction of their clients.
Anyway, the more important here is the processes and the product, because can
be specific, although the service can be also interesting, but it is generic, and there
are various quality standards related to the service.
2.2.2. Quality of the Software Product
For the quality of the software product, we can follow various models, although
most of them have the same origin.
60
2.2.3. Main Standards About the Quality of the Software Product
The most important standards related to the quality of the software product are
ISO 9126 and ISO 25000, and they have a similar structure, based on
characteristics and attributes.
2.2.4. Standard ISO/IEC 25000 (SQuaRE)
One of the most important standards related to the quality of the software product
is ISO/IEC 25000, which is also known as SQuaRE, although really this standard is
composed of various groups of standards.
2.2.5. Tools for the Assessment of the Quality of the Software
Product
There are various tools that we can use for the assessment of the quality of the
software product, some of them are free.
2.2.6. Example of an Environment for the Evaluation of the
Software Product
It Is important to know how to evaluate the quality of a software product,
considering the characteristics and attributes defined by ISO/IEC 25000, so we will
see an easy example.
2.2.7. Introduction to ISO/IEC 20000
ISO/IEC is an international standard related to the quality of the IT service and
can help us to obtain the satisfaction of our clients. It is the best standard for
software companies because is specifically related to IT.
61
2.2.8. Structure of the ISO/IEC 20000
ISO/IEC 20000 is composed of various parts, defining each one specifics aspects
about the quality of the service.
2.2.9. Global management of ISO/IEC 20000
For the evaluation of the service, ISO/IEC 20000 uses processes related to the IT
service, and these processes are structured in various groups.
2.2.10. Model and key areas
For the management of the IT service, it is important to include the PDCA model,
to keep a continual improvement.
1.3. Examples (not to exceed 2 pages)
Como estandarizar la evaluación de la calidad del producto software - 1:
http://www.javiergarzas.com/2012/10/iso-9126-iso-25000-1.html
Cómo estandarizar la evaluación de la calidad del producto software – 2 :
http://www.javiergarzas.com/2012/10/iso-9126-iso-25000-2.html
2.2.11. Exercises
1.- The quality of a product is very subjective, so if you buy a software, how can
you evaluate if it has quality?
2.- We can use some standards related to the software product, for the evaluation
of the quality, but we can use an automatic way?
62
3.- If we think that we can also improve the service, what are the elements related
to the IT service that we need to consider?
2.2.12. Conclusions
The satisfaction of the client is very important, but all clients are different, but we
can establish a generic way, based on international standards, to give them quality,
from the point of view of the processes, the product, and the service.
2.3. Quality of the Product
2.3.1- Introduction
In software engineering is generally accepted that companies improve their
processes software, to improve the final product.
To do this, companies often use CMMI (one of the more widespread, popular and
used worldwide standard), or its equivalent, SPICE, which defines a model of
evaluation of maturity of processes.
As you recall, SPICE saw it in the previous module and based on a series of
levels of capacity and maturity, we could identify the maturity of an organization
(ISO/IEC 15504-7), and we could also identify the ability of each process (ISO/IEC
15504-2).
Now we will see a different approach, but it will also help us to develop software
with quality. Therefore, we will now focus on the quality of the product, which can
offer many guarantees to develop the software with quality, which will obviously
result in customer satisfaction.
63
Figure 1 – How to obtain the customer satisfaction
Therefore, the final idea is not to decide between use the model of processes or
focus only on the quality of the product, the best is to use both perspectives, to get
the development of a software with quality, and obtain the satisfaction of the
customers. So, now we will see the approach of the quality of the product.
2.3.2. Quality of the Software Product
It is clear that everyone prefers a product with quality that a product without
quality. Imagine that we buy a bicycle that is manufactured with quality controls,
and another in which quality controls have not been established.
Probably, the bicycle that has been produced with quality parameters will give us
fewer problems, because a company can give us a guarantee of quality. And if we
have problems, a company can give us a support.
But with the bicycle that has been produced without minimum quality parameters,
generally, nobody can ensure that the bicycle can run adequately.
Final client satisfied
Quality standards
Product
Processes
64
Products that have been developed with quality parameters, generally have a
higher cost, and it is logical, because ensure that quality takes time, and generally
additional resources are necessary.
We can also find a product that has been developed with quality parameters, and
which also fails or does not work well, but in principle the probability of failure is
low, and usually, we will have a support from the company.
In software occurs something similar, because we also have a product, but in this
case, the product is software.
But then, with a software product, how can we determine that it has a good
quality? Also, think that the software product never fails, but is very difficult to use
or understand.
Therefore, it is necessary to define some criteria or quality parameters and unify
them, so that they can be used to evaluate the quality of any software product.
In 1977 Mr. McCall created a list composed by several parameters that can affect
the quality of the software product, and which can be used to evaluate or measure
its quality.
This list was focused solely on the quality of the software product, i.e. not
addressed anything relating to processes, methodologies, etc.
These parameters of quality of the product addressed:
The operation of the software products
The revision of the software products
The transition from software products
65
Figure 2 –McCall model
Subsequently, specifically 1 year later, Boehm developed and published a model
similar to the of McCall, that also considered the "performance" and the "utility" of
the software product, to determine its quality.
Several years more afternoon, on 1991, he began to develop the first standards
for the assurance of the quality of the software product, and these standards were
used as a reference for the current standards.
2.3.3. Main Standards About the Quality of the Software Product
The first standard that was developed, related to the quality of the software
product, based on the models of McCall and Boehm, was the ISO 9126
With the publication of this first ISO standard, companies began to use these
models as a fundamental tool for the development of software product applying
quality parameters.
Operation
Revision
Transition
66
However, the ISO 9126 today really is not used, since it was updated by the ISO
25000, which actually consists of a set of standards, as we will see in the next
chapter.
Figure 3 – Standards related to the quality of the software product
These standards define a quality model mainly composed of 3 components:
Internal: Refers to the quality of the code of the software (development)
External: This refers to the quality of the execution of the software
(production)
Use: Refers to the use of the software (use from the point of view of the
user)
ISO 9126
ISO 25000
Standards related to the quality of the
software product
67
Figure 4 – Quality model of ISO 9126 and ISO 25000
On the other hand, this model also provides other 10 parameters, which are
known as "characteristics", that in turn, each feature, is also composed of multiple
attributes.
These 10 characteristics related to the quality of the software product are the
following:
Functional suitability: Capacity of the software product to provide
functions which meet stated needs, when the product is used under the
specified conditions.
Performance efficiency: Performance relative to a number of resources
that are used, under certain conditions.
Compatibility: Capacity of 1, or various systems or components, to
exchange information and/or perform its functions when they share the
same environment (software or hardware).
Usability: Ability that the software product can be understood, understood,
used, and that appeal is for the user when it is used under certain
conditions.
Internal
External
Use
68
Reliability: Ability of a system or a component, that can perform their
functions required, when it is used under certain conditions and for a
period of time.
Security: Ability to protect information, in such a way that persons, or also
the systems, cannot access to information which does not have access.
Maintainability: Capability of the software product to be modified in a way
to effectively and efficiently, by evolving needs, corrective or perfective.
Portability: Capacity of the software product, or a component, can be
transferred, in an effective and efficient way, from a hardware, software,
operational or use environment, to another.
Figure 5 – 10 characteristics about the quality of the software product
Each of these characteristics, in addition, has a number of attributes, we are also
going to see below in each case:
Functional suitability
Performance efficiency
Compatibility Usability
Reliability Security Maintainability Portability
69
Figure 6 – Attributes of the functional suitability
Functional completeness: Degree in which a set of capabilities covers all
the tasks and objectives of user specified.
Functional correctness: Capacity of the product, or a system, to provide
correct results, and the level of precision required.
Functional appropriateness: Capability of the software product, to
provide a set of functions appropriate for the tasks and objectives specified
by the user.
Fun
ctio
nal
su
itab
ility
Functional completeness
Functional correctness
Functional appropriateness
70
Figure 7 – Attributes of the performance efficiency
Time behavior: Response time, and processing, and ratios of throughput
of a system when it performs its functions under certain conditions, in
relation to an established benchmark.
Resource utilization: Quantities and types of resources used when the
software performs its function under certain conditions.
Capacity: Degree in which the limits of a parameter of a product or
system, comply with the requirements.
Perf
orm
ance
eff
icie
ncy
Time behaviour
Resource utilization
Capacity
71
Figure 8 – Attributes of the compatibility
Co-existence: Product capacity to coexist with other software, in a
common environment, sharing resources.
Interoperability: Ability of two or more systems or components to
exchange information and to use the information exchanged.
Co
mp
atib
ility
Co-existence
Interoperability
72
Figure 9 – Attributes of the usability
Appropriateness recognizability: Capacity of the product that allows the
user to understand whether the software is suitable for their particular
needs.
Learnability: Product capability that allows the user to learn its application.
Operability: Product capability that allows the user to use it easily.
User error protection: Capacity of the system to protect users to commit
errors.
User interface aesthetics: Capacity that the user interface can satisfy the
interaction of the user.
Accessibility: Product capability that allows that it can be used by users
with certain characteristics and disabilities.
Usa
bili
ty
Appropriateness recognizability
Learnability
Operability
User error protection
User interface aesthetics
Accessibility
73
Figure 10 – Attributes of the reliability
Maturity: Capacity of the system to meet the needs of reliability under
normal conditions.
Availability: Capacity of the system, or component, to be available and
accessible when required.
Fault tolerance: Ability of the system, or component, to operate according
to provisions against possible failures of hardware or software.
Recoverability: Capacity of the software product to recover any lost data,
and restore the system in case of interruption or failure.
Rel
iab
ility
Maturity
Availability
Fault tolerance
Recoverability
74
Figure 11 – Attributes of security
Confidentiality: Ability to protect information against unauthorized access.
Integrity: Capacity of the system, or component, to prevent unauthorized
modification.
Non-repudiation: Capacity to demonstrate that actions or events have
taken place, so that these actions or events may not subsequently be
repudiated.
Authenticity: Ability to trace clearly the actions of an entity.
Accountability: Ability to demonstrate the identity of a resource, or a
subject.
Secu
rity
Confidentiality
Integrity
Non-repudation
Authenticity
Accountability
75
Figure 12 – Attributes of the maintainability
Modularity: Ability of a system that allows a change in a component to
have a minimum impact on other components.
Reusability: Capacity of an asset that can permit that it can be used in
more than one software system, or in the construction of other assets.
Analyzability: Ease with which evaluate the impact that can have a
change, in the rest of the software, diagnosing deficiencies, or the causes
of the failure of the software, or by identifying the parts that must be
modified.
Modifiability: Product capability that allows that this be modified
effectively and efficiently, without introducing defects, or without degrading
performance.
Testability: Ease with which you can establish criteria of evidence for a
system or component, and ease with which tests may be done to
determine if these criteria are met.
Mai
nta
inab
ility
Modularity
Reusability
Analyzability
Modifiability
Testability
76
Figure 13 – Attributes of the portability
Adaptability: Capacity of the product that allows to be adapted effectively,
and efficiently, to different hardware environments, software, operational or
use.
Installability: Ease with which a product can install or uninstall without
problems in a given environment.
Replaceability: Capacity of a product so that this can be used instead of
other product software determined, with the same purpose, and in the
same environment.
Port
abili
ty Adaptability
Installability
Replaceability
77
2.3.4. Standard ISO/IEC 25000 (SQuaRE)
Arrived at this point, already know that the more important standard international,
with recognition world, related to the quality of the software product is the ISO / IEC
25000, that also is often know as SQuaRE, by its initial "Software Quality
Requirements and Evaluation". Therefore, we will see a little more detail the
structure of this standard.
ISO/IEC 25000 is actually composed of a series of standards, which can be
grouped into 5 groups:
Division of the quality management: Composed by the group of
standards ISO/IEC 2500n
Division for quality model: Composed by the group of standards ISO/IEC
2501n
Division for the quality measurement: Composed by the group of
standards ISO/IEC 2502n
Division for the requirements of the quality: Composed by the group of
standards ISO/IEC 2503n
Division for the quality evaluation: Composed by the group of standards
ISO/IEC 2504n
Figure 14 – Main groups of ISO/IEC 25000 standards
ISO/IEC 2501n ISO/IEC 2502n
ISO/IEC 2503n ISO/IEC 2504n
ISO/IEC 2500n
78
Then we will see the standards that make up each of these groups.
Figure 15 – Standards of ISO/IEC 2500n
The series of standards ISO/IEC 2500n basically defines models, terms and
common definitions that may be used or referenced by the standards of ISO/IEC
25000 series. On the other hand, ISO/IEC 2500n is composed of the following
standards:
ISO/IEC 25000 – Guide to SQuaRE: Defines the architecture of the
SQuaRE, the terminology of the family of standards of this series, including
a summary of each party, as well as reference models.
ISO/IEC 25001 – Planning and Management: Defines a set of
requirements and guidelines, to manage the assessment and specification
of the software product requirements.
ISO/IEC 2500n
ISO/IEC 25000
ISO/IEC 25001
79
Figure 16 – Standards of ISO/IEC 2501n
The series of standards ISO/IEC 2501n includes detailed models of quality,
including features for the internal, external quality and use of the software product.
On the other hand, ISO/IEC 2501n is composed of the following standards:
ISO/IEC 25010 – System and software quality models: Establishes a
model of quality for the software product, and the quality in use. On the
other hand, it also defines a number of features and quality attributes to
evaluate the software product.
ISO/IEC 25012 – Data Quality model: Establishes a global model for the
quality of the data, which may be applicable to the data that is stored in a
structured way, and furthermore is a part of an information system.
ISO/IEC 2501n
ISO/IEC 25000
ISO/IEC 25001
80
Figure 17 – Standards of ISO/IEC 2502n
The series of standards ISO/IEC 2502n includes a reference model for the
measurement of the quality of the product, including definitions of quality measures
(internal, external and in use), as well as practical guidelines for its implementation.
On the other hand, ISO/IEC 2502n is composed of the following standards:
ISO/IEC 25020 – Measurement reference model and guide: Establishes
a common model of elements for the measurement of the quality of the
software product, including also a guide to select or develop, and
implement the measures proposed by other ISO (for example, ISO/IEC
9126).
ISO/IEC 25021 – Quality measure elements: It is a set of metric base,
that can be used throughout the software life cycle.
ISO/IEC 25022 – Measurement of quality in use: Establishes specific
metrics to measure the quality of use of the product.
ISO/IEC 2502n
ISO/IEC 25020
ISO/IEC 25021
ISO/IEC 25022
ISO/IEC 25023
ISO/IEC 25024
81
ISO/IEC 25023 – Measurement of system and software product
quality: Establishes specific metrics for the measurement of the quality of
products and software systems.
ISO/IEC 25024 – Measurement of data quality: Establishes specific
metrics for the measurement of the quality of the data.
Figure 18 – Standards of ISO/IEC 2503n
The series of standards ISO/IEC 2503n allows you to specify quality requirements
which may be used in the process of definition of the software product quality
requirements. On the other hand, ISO/IEC 2503n is composed by the following
standard:
ISO/IEC 25030 – Quality requirements: Provides a series of
recommendations for the establishment of quality requirements for the
software product.
ISO/IEC 2503n
ISO/IEC 25030
82
Figure 19 – Standards of ISO/IEC 2504n
The series of standards ISO/IEC 2504n provides a set of requirements,
recommendations, and guidelines to perform the process of evaluation of the
software product. On the other hand, ISO/IEC 2504n is composed of the following
standards:
ISO/IEC 25040 – Evaluation reference model and guide: Defines a
generic reference model for evaluation.
ISO/IEC 25041 – Evaluation guide for developers, acquirers and
independent evaluators: Sets a number of requirements and
recommendations for the implementation of the evaluation of the software
product, from the point of view of the developers, from the point of view of
customers (entity that buys the software), and from the point of view of the
independent evaluators.
ISO/IEC 25042 – Evaluation modules: Sets the content of the
documentation that will be used to describe the evaluation module.
ISO/IEC 25045 – Evaluation module for recoverability: Provides a
module for the assessment of the recoverability.
ISO/IEC 2504n
ISO/IEC 25040
ISO/IEC 25041
ISO/IEC 25042
ISO/IEC 25045
83
Moreover, also there is a set of standards that are used as an extension, which
contains aspects more specific, and can be used as a complement to the
standards seen until now.
These complementary standards, are defined between the number ISO/IEC
25050 - ISO/IEC 25099, the most noteworthy being the following:
ISO/IEC 25051 – Requirements for quality of Commercial Off-The-
Shelf (COTS) software product and instructions for testing:
Establishes requirements for the product software COTS.
ISO/IEC 25062 – Common industry format for usability test reports:
Provides basic information on how to report the results of a test of usability
in a specific context of use.
2.3.5. Tools for the Assessment of the Quality of the Software
Product
Some tools that you can use to ensure the quality of the software product are the
following:
ChecKing QA: Allows to control elements of the software development
process, and also allows you to control the elements that can be analyzed,
as for example the source code, documentation, scripts, etc.
Kiuwan: Let's analyze code and measure the technical quality of the
software. It has a scorecard based on the ISO/IEC 9126, it runs in the
cloud, in SaaS mode.
PMD: Allows to analyze statically source code, identifying problems, such
as for example the repetition of code, or the use of nested ifs, etc. Mainly
be used for Java.
Check Style: Also allows analysis of static source code, checking if there
are style rules. Mainly it is also used for Java.
84
SONAR: Allows you to collect, analyze, and visualize source code metrics.
Mainly used for Java, although it provides support for other languages.
Google CodePro Analytix: Allows you to evaluate code, defining metrics,
analyze dependencies, generate unit testing, etc.
Simian: Allows detecting duplicated code.
2.3.6. Example of an Environment for the Evaluation of the
Software Product
Now that you know which are the parameters that can be used to measure the
quality of a software product, we can see a small example, very simple, about how
to assess a software product.
Imagine that we have developed a software, and we want to assess their quality
based on the characteristics and the attributes that we have seen in a previous
section.
If we want to evaluate each of the characteristics and attributes, we can use a table such as the following:
Characteristic Attribute Complies Not
complies Not
required
Functional suitability
Functional completeness
Functional correctness
Functional appropriateness
Performance efficiency
Time behavior
Resource utilization
Capacity
Compatibility Co-existence
Interoperability
Usability
Appropriateness
Learnability
Operability
User error protection
User interface aesthetics
Accessibility
Reliability Maturity
Availability
85
Fault tolerance
Recoverability
Security
Confidentiality
Integrity
Non-repudiation
Authenticity
Accountability
Maintainability
Modularity
Reusability
Analyzability
Modifiability
Testability
Portability
Adaptability
Installability
Replaceability
Table 1 – Evaluation of characteristics and attributes
So, we will assess for each attribute of each characteristic, the software that you
want to evaluate.
So, after completing the table with data:
Characteristic Attribute Complies Not
complies Not
required
Functional suitability
Functional completeness 1
Functional correctness 1
Functional appropriateness 1
Performance efficiency
Time behavior 1
Resource utilization 1
Capacity 1
Compatibility Co-existence 1
Interoperability 1
Usability
Appropriateness 1
Learnability 1
Operability 1
User error protection 1
User interface aesthetics 1
Accessibility 1
Reliability
Maturity 1
Availability 1
Fault tolerance 1
Recoverability 1
Security Confidentiality 1
86
Integrity 1
Non-repudiation 1
Authenticity 1
Accountability 1
Maintainability
Modularity 1
Reusability 1
Analyzability 1
Modifiability 1
Testability 1
Portability
Adaptability 1
Installability 1
Replaceability 1
Table 2 – Evaluation of characteristics and attributes with data
If we now make an analysis of the results obtained, classified by features, we have the following:
Characteristic Complies Not complies Not required Total
Functional suitability
3 3
Performance efficiency
3 3
Compatibility 1 1 2
Usability 5 1 6
Reliability 2 1 1 4
Security 2 2 1 5
Maintainability 1 3 1 5
Portability 1 1 1 3
Total 18 9 4 31
Table 3 – Results of the evaluation
We see, therefore, that we have evaluated a total of 31 attributes, of which 18
comply with their duties (58% of the total), 9 do not meet (29% of the total), and 4
are not required (13% of the total).
With these results, and the definition of acceptance criteria:
Range Value Acceptance
0% - 30% Does not complies requirements
No
31% - 50% Minimally complies requirements
No
87
51% - 70% Complies properly Yes
71% - 100% Exceeds the requirements
Yes
Table 4 – Acceptance criteria
Given that the degree of compliance is 58%, and according to the defined
acceptance criteria table, is within the margin of 51% - 70%, we can conclude that
software adequately meets a minimum quality, so that we can accept it as well.
However, it should be noted that these tables are indicative, and can be developed
with other values.
2.3.7. Quality of the Service (ISO 20000)
We have talked about the improvement of processes, and improving the quality of
the software product, but we need also see the quality from the point of view of the
service.
I.e., if we consider that we need to provide a service to a client, which can be the
development and maintenance of a software product, we can also improve the
quality.
There are several international standards, but we are going to focus on ISO/IEC
20000 because it is one of the international standards with greater recognition,
related to the management of IT services.
2.3.8. Introduction to ISO/IEC 20000
The ISO/IEC 20000 is very similar to ITIL standard, although ITIL mainly defines
a set of best practices for IT service management and ISO 20000 in addition
clearly defines a model of continuous improvement, based on the PDCA cycle.
88
On the other hand, the companies can be certified in ISO/IEC 20000, as they can
be also certified ISO 27001, ISO 15504, or ISO 9001, etc. But they cannot be
certified in ITIL, because it is not an ISO standard (you can certify individuals in
ITIL, but not companies, because ITIL does not have a scheme of certification for
companies).
ISO 20000, like other management system, has many common points with other
ISOs and shares many elements with the ISO 27001 (in fact, one of the processes
included in ISO 20000, is the process of information security management).
This standard aims to improve or ensure the quality of the service so is intended
primarily for companies that provide technology services, such as for example
companies which develop software and offer a service of maintenance of this
software.
ISO 20000 was developed in 2005 and was subsequently revised in 2011, which
is now the current version of this standard. For the development of the first version,
was used as a basis the BS 15000, which was a British standard developed by BSI
(British Standards Institution).
2.3.9. Structure of the ISO/IEC 20000
This standard, similar to the ISO 15504 we saw in module 1, consists of several
parts, which are detailed below:
ISO 20000-1 – Service management system requirements: This is
actually the standard that is certifiable since it defines requirements for the
establishment of a service management system.
ISO 20000-2 – Guidance on the application of service management
systems: Offers a guide to the implementation of a service management
system, based on the requirements of ISO/IEC 20000-1.
89
ISO 20000-3 – Service providers: Provides a guide for the definition of
the scope and applicability of the standard.
ISO 20000-4 – Process assessment model: Offers a process reference
model.
ISO 20000-5 – Exemplar implementation plan for ISO/IEC 20000-1:
Provides an example of the implementation of the requirements of the
ISO/IEC 20000-1.
ISO 20000-9 – Guidance on the application of ISO/IEC 20000-1 to
cloud services: Offers a guide for the implementation of the ISO 20000-1
in service providers that offer cloud services.
ISO 20000-10 – Concepts and terminology: Offers a list of concepts and
terminology of the ISO/IEC 20000.
ISO 20000-11 – Guidance on the relationship between ISO/IEC 20000-
1:2011 and service management frameworks: ITIL@: Provides a guide
to the relationship between the ISO/IEC 20000-1 and ITIL.
90
Figure 20 – Parts of ISO/IEC 20000
• Service management system requirementsISO 20000-1
• Guidance on the application of service management systemsISO 20000-2
• Service providersISO 20000-3
• Process assessment modelISO 20000-4
• Exemplar implementation plan for ISO/IEC 20000-1ISO 20000-5
• Guidance on the application of ISO/IEC 20000-1 to cloud servicesISO 20000-9
• Concepts and terminologyISO 20000-10
• Guidance on the relationship between ISO/IEC 20000-1:2011 and service management frameworks: ITIL@
ISO 20000-11
91
2.3.10. Global Management of ISO/IEC 20000
The most important part of ISO/IEC 20000 is the part 1 (requirements definition),
and the part 2 (best practices for implementing the processes).
Therefore, to be able to manage an IT service, we need the requirements of
ISO/IEC 20000-1 (requirements for the establishment of a PDCA and processes),
and best practices of the ISO/IEC 20000-2 for the implementation of the
processes.
These processes that are based on the ISO/IEC 20000, are mainly grouped into 4
groups:
Service delivery processes
Control processes
Resolution processes
Relationship processes
Each of which contains a series of processes, which we can see below:
92
Figure 21 – Service delivery processes
Capacity management: This process tries to manage that the service is
provided with the agreed capacity. Therefore, in addition to constantly
monitor the service, you perform a trend analysis, i.e. measure the current
capacity, and estimate how it will evolve in the future.
Service continuity and availability management: This process allows
mechanisms so that, in the event that an unforeseen event occurs, the
service is not interrupted, or if it is interrupted, there is a plan to retrieve it.
With respect to availability, it is important that the service is available in the
Service delivery processes
Capacity management
Service continuity and availability management
Service level management
Service reporting
Information security management
Budgeting and accounting for services
93
conditions in which has been established by contract (usually in service
level agreements).
Service level management Allows you to define service levels, from
which the service will be provided. These levels must agree formally
between the customer and the company that provides the service.
Service reporting: The idea of this process is to keep informed, on a
regular basis, the customer about the status of the service by notifying
potential incidents, complaints, etc.
Information security management: The main objective of this process is
to manage all matters related to the information security, considering the
risk management, the information security policy, security information
incidents, etc.
Budgeting and accounting for services: The main objective of this
process is to bring control over budgets and basic accounting related to
the service. I.e. basically aims to take control of the expenses incurred by
the service, and the budgets of possible improvements that may be
needed (this process does not address the economic benefits of the
service).
94
Figure 22 – Control processes
Configuration management: This process sets out a number of elements
of the service configuration, which we can consider as configuration items
to provide the service. For the registration of these configuration items, it is
necessary to define a CMDB, which must ensure its integrity through
periodic revisions. Before launching a new service, it is usually set a base
line with the current configuration of the configuration items of the service.
Change management: This process manages any change that is
produced in the service. To do this, this process should consist of a stage
of request for change, another stage of review, another of approval, and
finally another stage of implementation of the change. During the change,
something goes wrong, we can have a procedure of roll-back.
Release and deployment management: The Mission of this process is to
manage everything concerning to the delivery of the service and their
Control processes
Configuration management
Change management
Release and deployment
management
95
subsequent deployment. For example, if we need to deliver a software
product, first we need to deliver an executable, and after we need to install
it in the production environment of the customer. In this case, can exist a
delivery of emergency, that the client requires with maximum priority.
Figure 23 – Resolution processes
Incident and service request management: With this process, we can
identify possible issues related to the service. In this case, it is also
important to have a knowledge base of the incidents registered and
treated, because if we have an incident that has been treated in the past,
the information registered can help us. It is also important to define
different levels of incidents, and you can assign to each level resolution
times (example: If the incidence is High, resolution time must be less than
24 hours, but if it is Low, the resolution time can be more than 48 hours,
etc.)
Resolution processes
Incident and service request management
Problem management
96
Problem management: This process is similar to the previous one, with
the difference that when an incidence is repeated and is unable to close
permanently, is considered a problem, and in that case, we need to search
the origin of the problem. Here is also important to have a knowledge
base.
Figure 24 – Relationship processes
Business relationship management: This process is related to the
relationship that exists between the company that and its clients. So, it is
necessary that the company establish a channel for possible claims, and
also need to measure the satisfaction of customers with respect to the
service provided.
Supplier management: With this process, we can evaluate suppliers that
we need to offer the service, i.e. if we provide a software maintenance
Relationship processes
Business relationship
management
Supplier management
97
service, we can need to outsource a service cloud, so, we will need to
establish quality requirements, and select a provider based on these
requirements.
2.3.11. The PDCA model and key areas
In addition to the processes you have seen until now, ISO 20000 is based on a
model of continual improvement, like any other ISO management system, for
example, ISO 9001, ISO 27001, ISO 22301, etc.
This model of continual improvement is based on 4 stages or key areas:
Figure 25 – Continual improvement model
This model is also commonly known as the Deming cycle, and in the case of the
management of an IT service, you can start with planning (Plan) of the service
(project plan, definition of activities/resources, development of documentation,
resources), subsequently you need to implement everything that has been planned
Plan Do
CheckAct
98
in the previous stage (Do), then checked that everything that has been
implemented is according to what we planned (Check), and finally if any
inconsistency is detected during the review, you need to perform corrective actions
for the treatment and resolution (Act).
101
GLOSARIO
- AENOR: Certification body with various headquarters in the world, including Latin
America.
- Audit: External review of the system.
- Benchmark: Bank of tests that generally is used to test software.
- BSI: British Standard Institution. It is an international entity that publishes
standards, and also gives certification services.
- Capacity: Ability of a process to meet some requirements.
- CMMI: Software model for quality evaluation.
- Developers: People that develops software.
- Environment: The place where an application is executed, for example an
internal network, or a production section.
- ISO: International organization that develops and publishes ISO standards.
- IT service: Service that is related to the Information Technologies.
- Knowledge base: Database with specific information about some topics.
- Maturity: Ability to offer quality in software developments.
- Outcomes: Results of processes.
- PDCA: Continual improvement model, composed by: Planning, Doing, Checking,
and Acting.
- Process: A set of activities that have several inputs and give several outputs.
102
- Software life cycle: The development of software is composed by different
steps, generally: Requirements analysis, Design of the software, Implementation,
Documentation and testing, and System Operation and maintenance.
- Software product: Result of the development of the software and the product
that can be sold to the customers.
- Source code: Lines of code the software is composed by. This source code can
be written in Java, C++, PHP, or any other programming language.
- SQuaRE: Software Quality Requirements and evaluation. It is a model composed
by various standards for the evaluation of the software product.
103
BIBLIOGRAFÍA
Satish Kini. 2016. So what the heck is ISO 15504 (SPICE) Capability assessment?
United Stated. Servicemanagers.org. Recovered from:
https://servicemanagers.org/iso-15504-spice-capability-assessment/
ISO/IEC 15504. Foundation. Recovered from:
https://en.wikipedia.org/wiki/ISO/IEC_15504
QUALITAT & INFORMATIK. 2016. What is ISO/IEC TR 15504?
Softwareresearch.net. Recovered from:
http://www.softwareresearch.net/fileadmin/src/docs/teaching/SS03/PM/1T3e
wa.pdf
Francisco J. Pino Correa, Mario Piattini Velthuls, Carlos Manuel Fernandez
Sánchez. 2015. Modelo de madurez de ingeniería del software. Editorial de
AENOR ediciones. España
Gonzalez (2014) Diseño de redes telemáticas. RA-MA. España
Xunta de Galicia.(2014) I Jornada sobre Calidad del Producto Software e ISO
25000. Santiago de Compostela: Colexio Profesional de Enxeñaría en
Informática de Galicia
Jarvier Garzas (2012) Cómo estandarizar la evaluación de la calidad del producto
software – 1. http://www.javiergarzas.com/2012/10/iso-9126-iso-25000-
1.html
Jarvier Garzas (2012) Cómo estandarizar la evaluación de la calidad del producto
software – 2. http://www.javiergarzas.com/2012/10/iso-9126-iso-25000-
2.html
104
ISO 25000 : http://iso25000.com/index.php/en/iso-25000-standards/iso-25010
ISO 20000 : https://www.iso.org/obp/ui/#iso:std:iso-iec:20000:-1:ed-2:v1:en
ISO 9126 : https://en.wikipedia.org/wiki/ISO/IEC_9126
León, M. Certificación en Pruebas Software:
http://mti.cucea.udg.mx/sites/default/files/Documento%20de%20titulacion%
20Miguel%20Angel%20Leon.pdf