EHR Contract Negotiation Cathey Halsten , EHR Advisor July 13, 2011
EHR-S Conformance Considerations
description
Transcript of EHR-S Conformance Considerations
EHR-S Conformance ConsiderationsEHR-S Conformance Considerations
Lynne S. Rosenthal
National Institute of Standards and [email protected]
August 2004
2
Today’s ObjectivesToday’s Objectives
Present basic information about conformance and conformance related topics
Address:– What is Conformance?
– What does it mean to conform?
– How do you know a system conforms?
– What do you need to do testing?
– Who does the testing?
3
BackgroundBackground
Standards not enough to ensure interoperability– Standards only meaningful if implemented in a consistent way
Need to ensure that implementations adhere to the standard– What is expected of implementations in order to claim conformance –
i.e., what are the requirements?
– How will we know if an implementation conforms?
• test suites, test tools
Different ideas of what conformance is– Past experience may have affected view of conformance
4
TerminologyTerminology
Conformance
Specification Conformance Clause
Conformance Testing test suite, test tools
Validation process, reporting
Certification certificate, brand
5
Terminology - ConformanceTerminology - Conformance
CONFORMANCE – the fulfillment of a product, process or service of specified requirements. (ISO Guide 2)
– The requirements are specified in a standard or specification as part of a conformance clause or in the body of the specification
CONFORMANCE CLAUSE - a section of a specification that states all the requirements or criteria that must be satisfied to claim conformance
6
Terminology - Conformance TestingTerminology - Conformance Testing
CONFORMANCE TESTING – a way to determine directly or indirectly that relevant requirements are
fulfilled. – Serves as a communication between buyer and sellers
• Buyers increased confidence• Sellers substantiate claims
– Necessary, but not sufficient, for interoperability
Note: referred to as conformity assessment in ISO Guide 2
7
Terminology - Testing ProcessTerminology - Testing Process
VALIDATION - process necessary to perform conformance testing in accordance with a prescribed procedure and official test suite– ensures that testing can be repeatable and reproducible
– ensures that conclusions are consistent with facts presented in the evaluation
CERTIFICATION - acknowledgement that a validation was completed and the criteria established for issuing certificates (brands) was met.
8
Testing IngredientsTesting Ingredients
Standard Conformance clause
Conformance TestingTest assertions
Test suite (test software, test scripts, test criteria)
Validation Process - policy and procedures for testing
Certificationqualified bodies to do the testing and certification
control board - advisory and arbiter
9
Conformance IssuesConformance Issues
Conformance Clause should address1. What Needs to Conform
2. How to conform and claim conformance
3. Subdividing a specification: Profiles, Levels, Modules
4. Variability
a. Extensions
b. Options
c. Implementer defined values
d. Alternative approaches
10
What needs to conform? What needs to conform?
Identify who or what will implement the specification What ‘class of products’ will be developed
– What products are considered an EHR-S?
– Does it make sense to mention these?
Why care? – It helps in defining the scope of the specification
– It helps the reader know what is being targeted by the specification
What are the conditions that need to be met in order to claim conformance
11
Profiles, Levels, ModulesProfiles, Levels, Modules Profile is a subset of the overall specification that includes all
the functionality necessary to satisfy the requirements of a particular community of users. – Use to target needs of specific constituencies or purpose (care setting)
Levels are nested, hierarchical subsets of the specification. Level 1= core that all must implement, subsequent levels include lower level + added functionality– Use to facilitate incremental development and implementation (system core)
Modules are discrete collections of semantically-related units of functionality that do not necessarily fit into a simple hierarchical structure. – Use when functionality can be implemented independently of one another
(ePrescribing, CPOE, PHR)
Reference: Subdivide a Specification (extracted from W3C Quality Assurance documents)
12
Profiles, Levels, ModulesProfiles, Levels, Modules
EHR-SDSTU
NursingHomeProfile
Ambul.CareProfile
System Core
Care Setting CoreCare setting core
XX ZZ YY
3
System Core
Care Setting Core
CPOE
ePrescibe
PHR
13
ExtensionsExtensions Extensions incorporate additional features beyond what is defined
in the specification If allowed:
– shall not re-define semantics of existing functions
– shall not break existing functions
Standardized method for extending the specification – Define how to ‘simplify’ or ‘add’ functions for care settings
– Provide wording so vendors can articulate their extensions in a consistent way
Why?– Helps ensure extensions defined in a consistent manner – leading to
predictable understanding and handling aids interoperability
14
Options = discretionary choicesOptions = discretionary choices
Provide the freedom to make choices about:– Whether or not to support a function and/or features of that
function– Which functions, features, or values to choose from a list of
possibilities– Implementation dependent features or values – Alternative approaches
Indicated by choice of words: Should, May, Recommend Make sure need for the option is real If the option is implemented, it must be implemented
correctly (as defined in the specification)
15
More on optionsMore on options
Implementation Defined Features– may be necessary to address environmental conditions
(code sets and vocabulary, locality differences - e.g., language, time zones, dependencies on other technologies, or just to be flexible)
Alternative Approaches– specification may describe several different ways to
accomplish an operation (e.g., choice of bindings)
– specify whether or not all approaches need to be implemented
16
Conformance TestingConformance Testing
Requires: a test tool/method and testing program Test method:
– Way to determine requirements are met– E.g., test tool, test suite, checklist, etc
Testing is Falsification Testing– Can show presence of errors not their absence– Prove non-conformance, can never prove conformance
Testing Program = Doing the testing– 1st party (self-test), 2nd party (buyer), 3rd party (independent)
CONFORMANCE TESTING – a way to determine directly or indirectly that relevant requirements are fulfilled.
17
Testing and Certification Testing and Certification Considerations
Not all specifications need a testing program– Risks of nonconformance vs. cost of creating and running
testing program How formal should testing be?
– Determined by the market or regulation– Self testing, 2nd party (buyer), 3rd party testing– Self-certifying vs. independent certification
Testing program– Who will ‘own’ the testing program– What are the policy and procedures– Who will do the testing – testing organizations?– Who will issue the certificates?– Who will handle disputes Control board– Who will do maintenance?
18
Testing and CertificationTesting and Certification
Enables quality products, competitive markets, more choices Must be acceptable to sellers and credible to buyers
Reference: Conformance Testing and Certification Model for Software Specifications
19
Conformance GuidanceConformance Guidance
All specs to contain a conformance clause– Identifies: what and how to conform– Any special designations of conformance (e.g.,
conform to core levels)
– Ways that conforming implementations can vary– How to make a claim of conformance
Traceability to the specification
Specification profiles EHR-S
Conformance statements test cases tests
20
Conformance Clause ExamplesConformance Clause Examples Example 1
– Conformance of Generators is defined in terms of conformance to a particular profile of CGM.
– A Profile conforms if it meets all the requirements in ISO xxx and is structured in accordance with the Rules for Profiles
Example 2– There is no concept of conformance to this standard because there is no normative content.
This approach is taken for several reasons, chiefly … Example 3
– Within the context of this specification, conformance can be claimed for markup, document types, module implementations, documents, generators, and interpreters. In most of these cases, two levels of conformance are available: simple conformance and full conformance. Simple conformance means that the conforming object supports the minimal content model of the ruby element in section 2.1, i.e. only simple ruby markup. Full conformance means that the conforming object supports the maximal content model of the ruby element in section 2.1, i.e. that both simple and complex ruby markup are supported.
Example 4:– DICOM 1 (PS3.1-2003) points to PS3.2-2003, DICOM Part 2: Conformance – Each part of the 1073 family of standards contains a conformance section.
21
ReferencesReferences
1. Conformance Testing http://www.oasis-open.org/cover/conform20000112.html
Defines conformance and conformance testing and describes the components of a conformance testing program.
2. Conformance Testing and Certification Model for Software Specifications
http://www.itl.nist.gov/div897/ctg/conformance/bulletin-conformance.htm
Overview of testing and certification, including identifying key roles, activities and products involved in any conformance testing and certification program.
3. W3C Quality Assurance Framework: Specification Guidelines
and W3C Quality Assurance Variability in Specificationshttp://www.w3.org/QA/WG/ Public versions available in September 2004.
How to write better specifications with a focus on conformance issues