This section outlines specific criteria to be assessed in each of three main types of standards: messaging, terminology, and content standards. Tables containing detailed descriptions and rationales for the criteria are listed in the appendices.  

Review Terminology Standard Criteria

Review Messaging Standard Criteria

Review Content Standard Criteria

Review General Assessment Criteria.

Terminology Standards 

Certain technical features of terminology standards such as concept orientation, concept permanence, unambiguous concept meanings and explicit version identifiers significantly enhance the quality, stability and comparability of codified data.  These should be considered essential and should be measured on a pass / fail basis.

Other features such as use of synonyms, granular terms, meaningless identifiers, multiple hierarchies and a consistent model of meaning enhance the usability of the terminology and/or the usability of data with little direct impact on data quality.  These features should be weighed based on their applicability to the use case.

Concept Orientation

Concept orientated terminology standards are composed of collections of coded concepts with unique meanings.  Each concept will typically be represented by a very specific description or label that fully conveys the meaning of the concept. 

  • Is each coded term within the standard conceptually unique?

Unambiguous concept meanings

Concepts within the standard must have exactly one meaning. When a common word or phrase has two or more associated meanings, it must be separated into distinct concepts with unique labels.

  • Are the descriptions or labels used to distinguish coded terms specific enough to consistently infer the intended underlying concept or could they be interpreted in different ways within the intended scope of use?  (These labels are often referred to as fully specified names.)

Concept permanence

The meaning of a coded concept in a terminology should never change. If the meaning of a concept needs to be changed or refined, a new coded concept should be introduced.  Retired codes should never be deleted or reused.

  • Does the standard maintenance processes ensure that the meaning of a coded term or concept doesn’t change over time?
  • Are retired codes ever deleted or reused?

Meaningless identifiers

Standards which use unique identifiers for concepts which are not tied to hierarchical position or other contexts (i.e. do not carry any meaning) offer greater content stability when underlying models change.

  • Can the meaning of a coded term or concept be inferred (partially or fully) when looking only at the code? 
  • Is there any representation of hierarchy in the codes?

Explicit version identifiers

Each version of the terminology is designated by a unique identifier, such that parties exchanging data can readily determine whether they are using the same set of terms.

  • Does the standard’s versioning mechanism provide a way to readily check for the presence of a term or concept within a version as well as its status?  Is there a straightforward way to see what has changed between two versions?

Multi-Hierarchical

Multi-hierarchical Standards allow concepts with multiple classifications accessible through all reasonable hierarchical relationships. For example: the concept viral pneumonia could be classified either as type of pneumonia or a type of viral infection or both.

  • Are terms organized in flat lists, a strict hierarchy (taxonomy) or in a structure which allows multiple definitional relationships?

Consistent Model of Meaning (Ontology)

Terminology standards which express meaning explicitly through a consistent model and relationships between concepts enable human and machine assisted maintenance and quality assurance process.  These features can also be reused for other information processing purposes.

  • Are the meanings of coded terms and concepts explicitly expressed in a model that can be cross referenced with descriptive terms?
  • Is the model consistent throughout the standard and machine readable?
  • Is the model extendable through a localization mechanism?
  • Is the model extendable at the time of use?

Use of Synonyms

Concept oriented terminology standards require very precise labels (sometimes referred to as Fully Specified Names) to uniquely identify concepts.  These labels may not reflect common language.  Terminology standards which allow multiple synonymous labels (preferred terms and synonyms) to be assigned to a single concept may allow greater usability.

(Note that this flexibility can also increase solution complexity.  It is important to consider how different components of an integrated solution will interoperate with one another and requirements for supporting services when evaluating this criterion.)

  • Can multiple different descriptions be related to a single coded term or concept?
  • Is there a notion of preferred terms vs synonyms?  Can the designation of a term as preferred vs synonym vary based on the context of use?

Level of Granularity

Different levels of concept granularity may be required for different uses such as clinical documentation, navigation, decision support, and reporting. As an example, a manager may only need to know that a patient has a broken leg; the finance department that it is a fractured tibia; but the clinician needs to know that it is a spiral fracture of the shaft of the right tibia.

  • Are the coded terms and concepts within the standard at an appropriate level of detail for the intended use cases?

 

Messaging Standards

When assessing a messaging standard, the following criteria should be considered: implementation completeness, flexibility, and market support.

Implementation Completeness

Implementation artefacts and supporting tools can save time and effort for standards implementation.  

  • Do schemas and implementation guides exist?
  • Does the localization/implementation documentation have all the standards artefacts (e.g. well written implementation guide, terminology specification, XML schemas and message instances, Visio diagrams, Model Interchange Format (MIF) 1 and 2 files, etc.)?
  • Are there existing code libraries and off the shelf products to support the use of artefacts?
  • Is there a need for significant custom code?

Flexibility

Flexible standards may ease adoption and integration with existing or pre-adopted standards and platforms.  

Note that greater flexibility can increase solution complexity.  It is important to consider how different components of an integrated solution will interoperate with one another and requirements for supporting services when evaluating this criterion.

  • Does the standard support different formats?
  • Does the standard work well in terms of plug and play, or is it tied to some other part of architecture?
  • Can any security scheme be layered or is the security format and policy dictated?

Market Support

Market support is an indicator of the availability and usability of a standard.

  • How widely is the standard implemented? 

 

Content Standards

Standards include document standards such as HL7 CDA and DICOM for imaging. When selecting a content standard, the following criteria should be considered: implementation completeness, flexibility, and market support. 

 

 

 

  • No labels