Page History

This is section outlines specific criteria to be assessed in each of three main types of standards: messaging, terminology, and content standards. Tables containing detailed descriptions and rationales for the criteria are listed in the appendices.  

Anchor
term term

Terminology Standards 

When selecting a terminology standard, there are certain essential technical criteria the standard must meet: Certain technical features of terminology standards such as concept orientation, concept permanence, unambiguous concept meanings , and explicit version identifiers significantly enhance the quality, stability and comparability of codified data. Other desirable, but not essential technical criteria, include: meaningless identifiers, multi-hierarchies, source information/terminology model, use of synonyms, and level of granularity.

Table 4 - Terminology Standard Criteria

Criteria

Description

Rationale

Concept Orientation

This criterion assesses whether terminology elements are coded concepts (e.g. heart attack code# 103405), with possible multiple synonymous text and relationships to other concepts such as diagnoses or procedures. Example questions may include:

Does the standard contain redundant, ambiguous, or vague concepts? 

To eliminate any possible ambiguity that may exist historically. Concept orientation ensures that terms must correspond to at least one meaning and no more than one meaning, and that meanings correspond to no more than one term.  

Concept permanence*

  These should be considered essential and should be measured on a pass / fail basis.

Other features such as use of synonyms, granular terms, meaningless identifiers, multiple hierarchies and a consistent model of meaning enhance the usability of the terminology and/or the usability of data with little direct impact on data quality.  These features should be weighed based on their applicability to the use case.

Concept Orientation

Concept orientated terminology standards are composed of collections of coded concepts with unique meanings.  Each concept will typically be represented by a very specific description or label that fully conveys the meaning of the concept. 

Assessment Question:

• Is each coded term within the standard conceptually unique?

Unambiguous concept meanings

Concepts within the standard must have exactly one meaning. When a common word or phrase has two or more associated meanings, it must be separated into distinct concepts with unique labels.

Assessment Questions:

• Are the descriptions or labels used to distinguish coded terms specific enough to consistently infer the intended underlying concept or could they be interpreted in different ways within the intended scope of use?  (These labels are often referred to as fully specified names.)

Concept permanence*

The meaning of a coded concept in a terminology should never change. If the meaning of a concept needs to be changed or refined, a new coded concept should be introduced.

  Retired codes should never be deleted or reused.

Assessment Questions:

• Does the standard maintenance processes ensure that the meaning of a coded term or concept doesn’t change over time?
• Are retired codes ever deleted or reused?

Meaningless identifiers

Standards which use unique identifiers for concepts which are not tied to hierarchical position or other contexts (i.e. do not carry any meaning) offer greater content stability when underlying models change.

Assessment Questions:

• Can the meaning of a coded term or concept be inferred (partially or fully) when looking only at the code? 
• Is there any representation of hierarchy in the codes?

This is important, for example, when data coded under an older version of the vocabulary needs to be interpreted in view of a current conceptual framework.

Unambiguous concept meanings

This criterion ensures that concepts must have exactly one meaning. When a common term has two or more associated meanings, it must be separated into distinct concepts.

Explicit version identifiers

Each version of the terminology is designated by a unique identifier, such that parties exchanging data can readily determine whether they are using the same set of terms.

To prevent the misuse of concepts and identifiers within a standard.

Meaningless identifiers

The purpose of this criterion is to assess whether unique identifiers attached to concepts are not tied to hierarchical position or other contexts, and do not carry any meaning. 

Meaningless identifiers (where, for instance, a hospital site identifier bears no relationship to a hospital organization) allow for an individual concept to remain constant even if changes are made to future relationships.  

Multi-Hierarchical

Assessment Questions:

• Does the standard’s versioning mechanism provide a way to readily check for the presence of a term or concept within a version as well as its status?  Is there a straightforward way to see what has changed between two versions?

Multi-Hierarchical

Multi-hierarchical Standards allow concepts with multiple classifications

accessible through all reasonable hierarchical relationships. For example: the concept

To support the needs of multiple stakeholders with varying degrees of hierarchical relationships within a controlled vocabulary. 

viral pneumonia could be classified either as type of pneumonia or a type of viral infection or both.

Assessment Questions:

• Are terms organized in flat lists, a strict hierarchy (taxonomy) or in a structure which allows multiple definitional relationships?
•  

Consistent Model of Meaning (Ontology)

Terminology standards which express meaning explicitly through a consistent model and relationships between concepts enable human and machine assisted maintenance and quality assurance process.  These features can also be reused for other information processing purposes.

Assessment Questions:

• Are the meanings of coded terms and concepts explicitly expressed in a model that can be cross referenced with descriptive terms?
• Is the model consistent throughout the standard and machine readable?
• Is the model extendable through a localization mechanism?
• Is the model extendable at the time of use?

Use of Synonyms

Concept oriented terminology standards require very precise labels (sometimes referred to as Fully Specified Names) to uniquely identify concepts.  These labels may not reflect common language.  Terminology standards which allow multiple synonymous labels (preferred terms and synonyms) to be assigned to a single concept may allow greater usability.

(Note that this flexibility can also increase solution complexity.  It is important to consider how different components of an integrated solution will interoperate with one another and requirements for supporting services when evaluating this criterion.)

Assessment Questions:

• Can multiple different descriptions be related to a single coded term or concept?
• Is there a notion of preferred terms vs synonyms?  Can the designation of a term as preferred vs synonym vary based on the context of use?

Level of Granularity

Different levels of concept granularity may be required for different uses such as clinical documentation

Source Information/Terminology Model

This criterion assesses whether all content in the standard is derived from a single information and/or terminology model.

To ensure that terminology has a consistent model applied to all terms, making maintenance and updates easier.  Models also help to express the relationships between hierarchies that the concepts belong to, which in turn help people to understand the meaning of specific concepts and allow them to implement the standard correctly.  

Use of Synonyms [Office1] 

The purpose of this criterion is to assess whether each concept may have multiple synonymous terms, but the relationship of the terms to the concept must be explicitly represented. 

The terms or ?labels? for a concept needs to be precise.   However, the precise term (sometimes called the Fully Specified Name) may not be commonly used.  Furthermore, different people may prefer to use slightly different terms (i.e. synonyms) for the same concept which are frequently used in their implementation. Allowing synonyms to be linked to a single concept allows implementers to use different or ?preferred? terms to describe the same concept while maintaining the precise semantics.  For example, heart attack, infarction of heart and cardiac infarction are all synonyms of the concept myocardial infarction.

Level of Granularity 

This criterion assesses the degree to which the standard provides structured, granular, coded data to support:

advancements in provision of care such as decision support and alerts; and

various levels of data masking for health system use of de-identified EHR data

Example questions may include:

Does the standard need to support both clinical decision making and secondary analysis?

Does the system need to have the flexibility to allow users to enter post-coordinated terms?

, navigation, decision support, and reporting.

As an example, a manager may only need to know that a patient has a broken leg; the finance department that it is a fractured tibia; but the clinician needs to know that it is a spiral fracture of the shaft of the right tibia.

Assessment Questions:

• Are the coded terms and concepts within the standard at an appropriate level of detail for the intended use cases?

Anchor
message message

Messaging Standards

When selecting assessing a messaging standard, the following criteria should be considered: implementation completeness, flexibility, and market support[Office1] .

Table 5 - Messaging and Content Standards Criteria

Criteria

Description

Implementation Completeness

Implementation artefacts and supporting tools

can save time and effort for standards implementation.  

Assessment Question:

• Do schemas and implementation guides exist?
• Does the localization/implementation documentation have all the standards artefacts (e.g. well written implementation guide, terminology specification, XML schemas and message instances, Visio diagrams, Model Interchange Format (MIF) 1 and 2 files, etc.)?
• Are there existing code libraries and off the shelf products

• to support the use of artefacts?
• How much custom code is required?

If a standard has all the implementation artefacts and tools, it saves time and effort by not having to create them.  

Provides better guidance and less work intensive activities around implementation of the standard

Flexibility

Flexible standards may ease adoption and integration with existing or pre-adopted standards and platforms.  

Note that greater flexibility can increase solution complexity.  It is important to consider how different components of an integrated solution will interoperate with one another and requirements for supporting services when evaluating this criterion.

Assessment Question:

Flexibility 

This criterion assesses whether or not integration with other standards is possible.

• Does the standard support different formats?
• Does the standard work well in terms of plug and play, or is it tied to some other part of architecture?
• Can any security scheme be layered or is the security format and policy dictated?
• Can any terminology standard be used with the standard or is it limited to one specific standard?

Market Support

Market support is an indicator of the availability and usability of a standard.  It may also be an  

Assessment Question:

• How widely is the standard implemented

• ?

Anchor
content content

Content Standards

Standards include document standards such as HL7 CDA and DICOM for imaging. When selecting a content standard, the following criteria should be considered: implementation completeness, flexibility, and market support.