This section outlines specific criteria to be assessed in each of three main types of standards: messaging, terminology, and content standards. Tables containing detailed descriptions and rationales for the criteria are listed in the appendices.
Review Fit for Purpose Criteria
Review Stewardship Criteria
Review Quality Criteria
Terminology Standards
When selecting a terminology standard, there are certain essential technical criteria the standard must meet: concept orientation, concept permanence, unambiguous concept meanings, and explicit version identifiers. Other desirable, but not essential technical criteria, include: meaningless identifiers, multi-hierarchies, source information/terminology model, use of synonyms, and level of granularity.
Table 4 - Terminology Standard Criteria
Criteria | Description | Rationale | |
Concept Orientation | This criterion assesses whether terminology elements are coded concepts (e.g. heart attack code# 103405), with possible multiple synonymous text and relationships to other concepts such as diagnoses or procedures. Example questions may include: Does the standard contain redundant, ambiguous, or vague concepts?
| To eliminate any possible ambiguity that may exist historically. Concept orientation ensures that terms must correspond to at least one meaning and no more than one meaning, and that meanings correspond to no more than one term. | |
Concept permanence* | The purpose of this criterion is to assess whether the meaning of each coded concept in a terminology remains forever unchanged. If the meaning of a concept needs to be changed or refined, a new coded concept should be introduced. No retired codes are deleted or reused. | This is important, for example, when data coded under an older version of the vocabulary needs to be interpreted in view of a current conceptual framework. | |
Unambiguous concept meanings | This criterion ensures that concepts must have exactly one meaning. When a common term has two or more associated meanings, it must be separated into distinct concepts. | To ensure each concept is unambiguous and does not have any duplicate meanings. | |
Explicit version identifiers | This criterion assesses whether each version of the terminology is designated by a unique identifier, such that parties exchanging data can readily determine whether they are using the same set of terms. | To prevent the misuse of concepts and identifiers within a standard. | |
Meaningless identifiers | The purpose of this criterion is to assess whether unique identifiers attached to concepts are not tied to hierarchical position or other contexts, and do not carry any meaning. | Meaningless identifiers (where, for instance, a hospital site identifier bears no relationship to a hospital organization) allow for an individual concept to remain constant even if changes are made to future relationships. | |
Multi-Hierarchical | This criterion assesses whether the standard fully supports multiple classifications with concepts accessible through all reasonable hierarchical relationships. For example: the concept of the influenza virus may have two different parents, one for pathogens and the other for vaccines. | To support the needs of multiple stakeholders with varying degrees of hierarchical relationships within a controlled vocabulary. | |
Source Information/Terminology Model | This criterion assesses whether all content in the standard is derived from a single information and/or terminology model. | To ensure that terminology has a consistent model applied to all terms, making maintenance and updates easier. Models also help to express the relationships between hierarchies that the concepts belong to, which in turn help people to understand the meaning of specific concepts and allow them to implement the standard correctly. | |
Use of Synonyms [Office1] | The purpose of this criterion is to assess whether each concept may have multiple synonymous terms, but the relationship of the terms to the concept must be explicitly represented. | The terms or ?labels? for a concept needs to be precise. However, the precise term (sometimes called the Fully Specified Name) may not be commonly used. Furthermore, different people may prefer to use slightly different terms (i.e. synonyms) for the same concept which are frequently used in their implementation. Allowing synonyms to be linked to a single concept allows implementers to use different or ?preferred? terms to describe the same concept while maintaining the precise semantics. For example, heart attack, infarction of heart and cardiac infarction are all synonyms of the concept myocardial infarction. | |
Level of Granularity | This criterion assesses the degree to which the standard provides structured, granular, coded data to support: advancements in provision of care such as decision support and alerts; and various levels of data masking for health system use of de-identified EHR data Example questions may include: Does the standard need to support both clinical decision making and secondary analysis? Does the system need to have the flexibility to allow users to enter post-coordinated terms?
| To support the needs of various stakeholders who may require different levels of granularity. Different levels of granularity are needed for defining concepts, navigation, decision support, and reporting. For example, a manager may only need to know that a patient has a broken leg; the finance department that it is a fractured tibia; but the clinician needs to know that it is a spiral fracture of the shaft of the right tibia. | |
Messaging Standards
When selecting a messaging standard, the following criteria should be considered: implementation completeness, flexibility, and market support[Office1] .
Table 5 - Messaging and Content Standards Criteria
Criteria | Description | Rationale |
Implementation Completeness | This criterion assesses the ease of implementation by the existence of artefacts and tools. Some examples of questions include: Do schemas and implementation guides exist? Does the localization/implementation documentation have all the standards artefacts (e.g. well written implementation guide, terminology specification, XML schemas and message instances, Visio diagrams, Model Interchange Format (MIF) 1 and 2 files, etc.)? Are there existing code libraries and off the shelf products that support this? How much custom code is required?
| If a standard has all the implementation artefacts and tools, it saves time and effort by not having to create them.
Provides better guidance and less work intensive activities around implementation of the standard |
Flexibility | This criterion assesses whether or not integration with other standards is possible. Some example questions may be: Does the standard support different formats? Does the standard work well in terms of plug and play, or is it tied to some other part of architecture? Can any security scheme be layered or is the security format and policy dictated? Can any terminology standard be used with the standard or is it limited to one specific standard?
| Supports ease of adoption and seamless integration with existing or pre-adopted standards and platforms
|
Market Support | This criterion assesses the implementation and usage of the standard in a wider context. Example questions may be: How widely is the standard implemented? How many vendors currently support the standard?
| To ensure vendor support and adoption of the standard is readily available |
Content Standards
Standards include document standards such as HL7 CDA and DICOM for imaging. When selecting a content standard, the following criteria should be considered: implementation completeness, flexibility, and market support.