Eduardo Santana de Almeida cin.ufpe.br/~esa2 [email protected]
Towards an Effective Software Component Certification Process Advisor Silvio Lemos Meira...
-
date post
21-Dec-2015 -
Category
Documents
-
view
221 -
download
2
Transcript of Towards an Effective Software Component Certification Process Advisor Silvio Lemos Meira...
Towards an Effective Software Component Certification
Process
AdvisorSilvio Lemos Meira
StudentAlexandre Alvaro
Agenda
• Introduction– Reuse– RiSE {Reuse in Software Engineering}– CBD {Component-Based Development}– Component– Quality– Component Certification
• Component Certification History• Future
Introduction {Reuse}
• [Frakes, 1995]•“Software reuse is the use of existing software knowledge or artifacts to build new software artifacts.”
•“Everything that was done, should not be done again“
Introduction {Reuse}
• Advantages• Increase
• Quality• Productivity
• Decrease • Time to market • Manutenability
• Reduce life-cycle of development
• Software Reuse…– Not simple– Systematic Software Reuse
Introduction {RiSE}
• [Almeida et al., 2004]• The IEEE International Conference on Information Reuse and Integration.
http://www.cin.ufpe.br/~rise
Introduction {CBD}
• Software reuse idea’s is not new• [McIlroy, 1969]
“Mass Produced Software Components”
• “To (re)use”, instead of “to develop”• To keep a set of reused components
•Repository systems•[Mili, 1998]
• 50 proposed solutions to this problem
Introduction {Component}
“ A software component is a unit of composition with contractually specified
interfaces and explicit context dependencies only. A software component
can be independently deployed and is subject to third-party composition”.
“ A software component is a unit of composition with contractually specified
interfaces and explicit context dependencies only. A software component
can be independently deployed and is subject to third-party composition”.
[Szyperski, 2002]
• The exactly concept of component in CBD is not yet a consensus...
Introduction {Component}
• [Bass et al., 2000]• CMU/SEI’s report
• Inhibitors:• Lack of available components;• Lack of stable standards for component technology;• Lack of certified components;• Lack of an engineering method to consistently
produce quality systems from components.
Introduction {Component}
• Besides CMU/SEI...
• [Heineman, 2000]• [Councill, 2001]• [Crnkovic, 2001]• [Wallnau, 2003]
Introduction {Quality}
ISO 9000
CMM
[Weber & Nascimento, 2002]
Introduction {Component Certification}
• Concept...
“Third-party certification is a method to ensure that software components conform to well-defined standards; based on this
certification, trusted assemblies of components can be constructed.”
“Third-party certification is a method to ensure that software components conform to well-defined standards; based on this
certification, trusted assemblies of components can be constructed.”
[Councill, 2001]
Introduction {Component Certification}
Introduction {Component Certification}
• [Frakes et al., 1996]
History of Component Certification
•History of the Software Component Certification
• Decade of 90– Mathematical models– Test-Based models
• [Poore et al., 1993] “Planning and certifying software system reliability”
– Three mathematics model• Test cases • Report the failures• Data are analyzed to achieve a reliability index
– Reliability of system• Considering how the components affects this
reliability
History of Component Certification
• [Wohlin & Runeson, 1994]“Certification of Software Components”
– Method that consist:• Usage model• Usage profile
– Test cases based on this models– Collection the failure data– Certification of reliability
– Hypothesis certification• Certify a specific reliability level -> given degree of
confidence
– Reutilization degree of the models
History of Component Certification
• [Rohde et al., 1996]“Certification of Reusable Software Components”
– Rome Laboratory of the Air Force, NY
History of Component Certification
• [Rohde et al., 1996]“Certification of Reusable Software Components”
– Certification process:1. Readiness Assessment
– Compile without error and execute the code
2. Static Analysis– Automatic tool
3. Code Inspection– Manual technique
4. Testing – Other tests
– Analysis of the certification process…
History of Component Certification
• [TCI Initiative, 1998]1
– Affiliation of researchers• Formal interface specification
– Supports compositional reasoning • A restricted set of behavioral properties of
assemblies
– Difficult to find real contributions…1 http://www.trusted-components.org
History of Component Certification
• [Voas, 1998]“Certifying Off-the-Shelf Software Components”
– Automated technologies• Black-Box testing and fault injection
– Methodology:• Black-box component testing • System-level fault injection • Operational system testing
– Certify components to a determined environment...
History of Component Certification
• [Wohlin & Regnell, 1998]“Reliability Certification of Software Components”– Extend [Wohlin & Runeson, 1994] work
– Certification process1. Usage specification (usage model and usage
profile)2. Certification procedure• Three approaches:
– Certification Process – Reliability Certification of Component and Systems – Certify or Derive System Reliability
History of Component Certification
• [Wohlin & Regnell, 1998]“Reliability Certification of Software Components”– Extend [Wohlin & Runeson, 1994] work
History of Component Certification
• [Wohlin & Regnell, 1998]“Reliability Certification of Software Components”– Extend [Wohlin & Runeson, 1994] work
History of Component Certification
?
• [Voas & Payne, 2000]“Dependability Certification of Software
Components”
– Metrics framework– Create a tests methodology…
• Component testability score– Mathematical models– Statistics approaches
• Estimates the number of test cases necessary • Consider:
– The number of tests that a component received; – The “fault revealing” ability of those test cases.
History of Component Certification
• [Morris et al., 2001]“Software Component Certification”
– Four steps:• Tests Specification• Specification Document• Specified Results • Test-Pattern Verificator
– Limitations…
History of Component Certification
• However...– Testing is not enough...
• [Sametinger, 1997]– Component certification levels
• Level 1: A component is described with keywords and a summary and is stored for automatic search. No tests are performed; the degree of completeness is unknown;
• Level 2: A source code component must be compiled and metrics are determined;
• Level 3: Testing, test data, and test results are added;• Level 4: A reuse manual is added.
History of Component Certification
• [Heineman et al., 2000]– Panel presented in ICSE’2000– Discuss the necessity of trust assurance in
component
– Considerable CBD researchers participate:• Heineman {organizations}• Councill {software development}• Flynt {benefits to the customers}• Shaw {reutilization of the components}
History of Component Certification
• Workshops…
– 4th ICSE Workshop on Component-Based Software Engineering (CBSE): Component Certification and System Prediction, 2001.
– 5th ICSE Workshop on Component- Based Software Engineering (CBSE): Benchmarks for Predictable Assembly, 2002.
History of Component Certification
• Long time considering just test...• [Stafford & Wallnau, 2001]
“Is Third Party Certification Necessary?”
– Define a process model…• Support prediction of system properties prior
to component selection
– Introduce “credentials” concept• <property,value,credibility>
– Active component dossier• A dossier is an abstract component that
defines certain credentials
History of Component Certification
• Long time considering just test...• [Stafford & Wallnau, 2001]
“Is Third Party Certification Necessary?”
History of Component Certification
• [Stafford & Wallnau, 2001]“Is Third Party Certification Necessary?”
– Some open questions:•What level of trust is required? •Are there other mechanisms that might
be used to support trust?
•How to certify measurement techniques?
History of Component Certification
• Other authors…•How certification should be carried out?
(Goulao & Abreu, 2002)•What does it mean to trust a component?
(Hissam et al., 2003)•What characteristics of a component
make it certifiable, and what kinds of component properties can be certified? (Wallnau, 2003)
History of Component Certification
• [Councill, 2001]“Third-Party Certification and Its Required
Elements”
– Other aspect of component certification…• Human• Industrial• Business
– Certification is the components future…
History of Component Certification
• [Woodman et al., 2001]“Issues of CBD Product Quality and Process
Quality”
– Analyze some process in various CBD approaches
– Examine 11 potential CBD quality attributes
Reusability Maintainability Accuracy Clarity
Replaceability Interoperability Scalability Performance
Flexibility Adaptability Reliability
History of Component Certification
Reusability Maintainability Accuracy Clarity
Replaceability Interoperability Scalability Performance
Flexibility Adaptability Reliability
• [Hissam & Wallnau, 2003]“Enabling Predictable Assembly”– Extends the [Stafford & Wallnau, 2001]work
– Introduced Prediction-Enabled Component Technology (PECT)
– Component technology with analysis technology• Prediction of assembly properties
• Identify required component properties
• Certifiable properties
History of Component Certification
• [Hissam & Wallnau, 2003]“Enabling Predictable Assembly”– Extends the [Stafford & Wallnau, 2001]work
– Component technology and analysis technology
• Component model• Component runtime environment• Assembly environment
History of Component Certification
• Defines a property theory
• Parameters of this theory• Component properties
Increased accuracy prediction both
More abstract, less acurate
Increased accuracy prediction
• [Hissam & Wallnau, 2003]“Enabling Predictable Assembly”– Extends the [Stafford & Wallnau, 2001]work
– Validation:• Empirical
– Predictions made, conform to observations
– Limitations• Two prediction technology may be incompatible• How are non-resource attributes, such as
security, to be empirically validated?• Industrial component certification ?
History of Component Certification
• [Meyer, 2003]“The Grand Challenge of Trusted Components”
– Two complementary roads:•Low Road
– Qualification of existing components
•High Road– Production of components with fully proved correctness
properties.
History of Component Certification
• [Meyer, 2003]“The Grand Challenge of Trusted Components”
History of Component Certification
• [McGregor, 2003]“Measuring Component Reliability”
– Support prediction of assemblies reliabilities based on properties of the components
– Method to measuring and communicating the reliability of the component• Component’s services
– Component’s documentation
• Test plan is created, based on component’s services• Provide the reliability of each service
– This method is a fundamental element of PECT
History of Component Certification
• [Wallnau, 2003]“Volume III: A Technology for Predictable Assembly
from Certifiable Components”• CMU/SEI’s report
– How component technology can be extended in order to achieve Predictable Assembly from Certifiable Components (PACC).
• Runtime behavior of software components assemblies
• Component’s property– Component’s proprieties need rigorously defined and trusted; and
– It can be certified by independent third-party developers
History of Component Certification
• [Wallnau, 2003]“Volume III: A Technology for Predictable Assembly
from Certifiable Components”• CMU/SEI’s report
– SEI’s approach to PACC is PECT.
History of Component Certification
• [Wallnau, 2003]“Volume III: A Technology for Predictable Assembly
from Certifiable Components”• CMU/SEI’s report
– Status:• On going work
– PECT is relatively immature
• One or more certification properties…• Tools are being developed• Functional certification complements the PECT [Meyer, 2003]
– Precondition to PECT
• Non-functional properties
History of Component Certification
• Two failures case....– National Information Assurance
Partnership (NIAP) • Together with NIST and NSA• From 1993 until 1996• Defines criteria for certifying security features of
components • Restricted set of behavioral assembly properties.
– IEEE• 1997• The initiative was suspended, in this same year…
History of Component Certification
Summary
Future – RiSE Context
• [Almeida et al., 2004]• The IEEE International Conference on Information Reuse and Integration.
http://www.cin.ufpe.br/~rise
Future work
1. Key CBD Requirements• What are the requirements for a certification
process? • [Woodman et al., 2001]
• 11 CBD requirements
• [Simao,2003]• 124 CBD requirements
• [Larson, 2004]• 72 CBD requirements
2. Component Quality Model• What requirements are more important ?• [Meyer, 2003]
Future work
3. Certification Method• How certify components ?
4. A Metrics Framework• How to measure the component certification
processes ?
Future work
• Write a Paper– “On the Software Component
Certification Process”• The history• The proposal
Future work
References
• [Frakes, 1995] Frakes, W., B., Fox, C., J. Sixteen Questions about Software Reuse. Communications of the ACM, June, 1995.
• [Szyperski, 2002] Szyperski, C., 2002. Component Software: Beyond Object-Oriented Programming. Addison-Wesley, USA. ISBN 0-201-74572-0.
• [Mcllroy, 1968] Mcllroy, M. D., 1968. Mass Produced Software Components. NATO Software Engineering Conference Report, October, pp. 79-85.
• [Mili et al., 1998] Mili, A., Mili, R., Mittermeir, R., 1998. A Survey of Software Reuse Libraries. Annals Software Engineering, Vol. 05, pp. 349–414.
• [Heineman & Councill, 2001] Heineman, G. T., Councill, W. T., 2001. Component-Based Software Engineering: Putting the Pieces Together. Addison-Wesley, USA. ISBN: 0-201-70485-4.
• [Heineman et al., 2000] Heineman, G. T., Councill, W. T., Flynt, J. S., Mehta, A., Speed, J. R., Shaw, M., 2000. Component-Based Software Engineering and the Issue of Trust. The IEEE Proceedings of the 22nd International Conference on Software Engineering (ICSE), Canada, pp. 661-664.
• [Crnkovic, 2001] Crnkovic, I., 2001. Component-based software engineering - new challenges in software development. Software Focus, Vol. 2, No. 4, pp. 27-133.
References
• [Wallnau, 2003] Wallnau, K. C., 2003. Volume III: A Technology for Predictable Assembly from Certifiable Components. Software Engineering Institute (SEI), Technical Report, Vol. III, April.
• [Frakes & Terry, 1996] Frakes, W., Terry, C., 1996. Software Reuse: Metrics and Models. ACM Computing Survey, Vol. 28, No. 2, June, pp. 415-435.
• [Poore et al., 1993] Poore, J., Mills, H., Mutchler, D., 1993. Planning and certifying software system reliability. IEEE Computer, Vol. 10, No. 1, January, pp. 88-99.
• [Wohlin & Runeson, 1994] Wohlin, C., Runeson, P., 1994. Certification of Software Components. IEEE Transactions on Software Engineering, Vol. 20, No. 6, June, pp. 494-499.
• [Rohde et al., 1996] Rohde, S. L., Dyson, K. A., Geriner, P. T., Cerino, D. A., 1996. Certification of Reusable Software Components: Summary of Work in Progress. The IEEE Proceedings of the 2nd International Conference on Engineering of Complex Computer Systems (ICECCS), Canada, pp. 120-123.
References
• [Voas, 1998] Voas, J. M., 1998. Certifying Off-the-Shelf Software Components. IEEE Computer, Vol. 31, No. 6, June, pp. 53-59.
• [Wohlin & Regnell, 1998] Wohlin, C., Regnell, B., 1998. Reliability Certification of Software Components. The IEEE Proceedings of the 5th International Conference on Software Reuse (ICSR), Canada, pp 56-65.
• [Voas & Payne, 2000] Voas, J. M., Payne, J., 2000. Dependability Certification of Software Components. Journal of Systems and Software, Vol. 52, No.2-3, June, pp. 165-172.
• [Morris et al., 2001] Morris, J., Lee, G., Parker, K., Bundell, G. A., Lam, C. P., 2001. Software Component Certification. IEEE Computer, Vol. 34, No. 9, September, pp. 30-36.
• [Sametinger, 1997] Sametinger, J., 1997. Software Engineering with Reusable Components. Springer Verlag, USA. ISBN 3-540-62695-6.
References
• [Stafford & Wallnau, 2001] Stafford, J., Wallnau, K. C., 2001. Is Third Party Certification Necessary?. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May, pp. 13–17.
• [Councill, 2001] Councill, B., 2001. Third-Party Certification and Its Required Elements. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May.
• [Woodman et al., 2001] Woodman, M., Benebiktsson, O., Lefever, B., Stallinger, F., 2001. Issues of CBD Product Quality and Process Quality. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May.
• [Hissam et al., 2003] Hissam, S. A., Moreno, G. A., Stafford, J., Wallnau, K. C., 2003. Enabling Predictable Assembly. Journal of Systems and Software, Vol. 65, No. 3, March, pp. 185-198.
• [Meyer, 2003] Meyer, B., 2003. The Grand Challenge of Trusted Components. The IEEE Proceedings of 25th International Conference on Software Engineering (ICSE), USA, pp. 660–667.
References
• [McGregor et al., 2003] McGregor, J. D., Stafford, J. A., Cho, I. H., 2003. Measuring Component Reliability. The IEEE Proceedings of the 6th ICSE Workshop on Component-Based Software Engineering (CBSE), USA, May, pp. 13-24.
• [Schmidt, 2003] Schmidt, H., 2003. Trustworthy components: compositionality and prediction. Journal of Systems and Software, Vol. 65, No. 3, March, pp. 215-225.
• [Simão, 2003] R. Simao, A. Belchior, Quality Characteristics for Software Components: Hierarchy and Quality Guides. Lecture Notes in Computer Science, pp. 188-211, June. Springer-Verlag. 2003.
• [Larson, 2004] M. Larson, Predicting Quality Attributes in Component-based Software Systems, PhD Thesis, Malardalen University, 2004.