General Standards Screening Criteria

Criteria

Description

Rationale

Alignment to the eHealth Blueprint

The purpose of this criterion is to assess whether the standard supports the exchange of information between two or more components of the EHR. The standard should also be intended to be used in a federated approach to EHR architecture where multiple organizations are expected to build, operate and/or host the various applications.

The standard supports eHealth mandate in providing a single, harmonized, coherent province-wide EHR.

Existing Standard [RB2] 

This criterion assesses whether the standard is adopted/adapted from existing standards, and whether it is intended for international, pan- Canadian.

Re-use of existing standards allows implementers to re-use code (i.e. in point of service /Health Information Access Layer (HIAL) applications, conformance testing environments, etc.). It also allows reuse of existing tools/skills/knowledge sets to establish the standard.

Supports Business Requirements

This criterion assesses whether the standard expresses all information required by the business/clinical domain. While the exact business requirements are project specific, example questions may include:

• Does the messaging standard have all the messages to support the required functions for data exchange? 
• Do the messages have the appropriate fields to express the information required for the business functions?
• Does the document standard have all the sections to express the required business information?
• Does the terminology contain all of the concepts that need to be captured?
• If bi-lingual information needs to be exchanged, does the standard allow for English and French terms to be mapped to common codes? 
• Are there user interface data collection/display requirements that should be considered when choosing the interoperability standard?

To ensure that the standard addresses the health care business requirements.

The standard should be health care delivery setting independent to enable use across multiple health care delivery settings.

Supports Technical Requirements

This criterion assesses the degree to which the standard is feasible to adopt and implement according to the technical requirements of the project. While the exact technical requirements are project specific, example questions may include:

• Can the standard be implemented in the proposed architecture? 
• Does the conceptual architecture make any assumptions about synchronous or asynchronous communication? 
• Does the architecture support HTTP and/or MLLP? 
• Are there any interdependencies with other aspects of the architecture that would make it difficult to implement the standard (e.g. there are no places to express SAML bindings in HTTP without SOAP, making it difficult to integrate RESTful interfaces with the existing security system and therefore difficult to implement HL7 FHIR)?

To enable successful implementation of the standard.

Likely to be Adopted[Office3] 

This criterion assesses the likelihood that the selected standard will be adopted and maintained by vendors and other implementers. Some questions that may be asked when assessing the likelihood for adoption, include: 

• Have vendors already implemented this? If they haven’t, will they get a return on investment if they do?
• Do vendors have the necessary expertise to implement the standard? If not, is it realistic that they can gain that expertise in time to implement?
• Will vendors provide support for sustained use and maintenance?

To ensure success of the implementation. If the standard is not widely implemented or supported by vendors, it will result in increased time, cost, and effort. 

Supports coded and free text content.

This criterion assesses the degree to which coded data is required.

Some questions to ask when assessing whether there is a need for coded data versus free text, may include: 

• Does the project implementing the standard expect to support automated processing such as: data aggregation comparison within decision support or analytics applications, standardization of data used to trigger process flows (i.e. presence or absence of terms), etc[Office4] .[Office5] 
• Do clinicians require assistance to exchange data/information for human readability?
• Are there other forms of non-textual data/information (i.e. audio, video, images) that need to be exchanged?
• Are there privacy concerns with exchanging free text?

Different standards support different levels of data granularity and coding.  However, there can be trade-offs to application developers and users to provide granular and coded data.  At one extreme, end users may reject using the application because it forces them to enter data in too many fields, increasing their time/effort to do so. At the other extreme, just capturing free text can cause privacy breaches and be a barrier to implementing decision support.

Criteria

Description

Rationale

Costs and Benefits of Implementation

This criterion assesses whether the cost of implementation and disruption to current business is affordable given the benefits. It is important to consider not only the initial cost but the on-going cost for maintenance as well. Cost is not only measured in currency, but can be in time or effort as well. Some sample questions to ask when assessing the costs and benefits of implementation, include:

• What is the return on investment for the vendor?
• What is the cost of tools required?
• Will the implementation timelines be increased due to the complexity of the standard?
• Will cost or risk of subsequent system implementations be reduced?[Office1] 
• What is the number of people required for implementation?
• Is there any existing commercial off the shelf software vs. custom software?
• If properly implemented, would the standard provide system or societal benefits such as improved decision support or clinical research? 

To take into consideration any cost constraints when adopting a standard, as well as initial implementation and longer-term maintenance costs. 

Governance Structure

The purpose of this criterion is to assess whether or not the standard is governed appropriately.  When assessing if a governance structure is in place, it is important to consider whether there are: 

• Defined processes in place to facilitate decision making and issue resolution related to both standards content and processes.
• Balanced representation of communities responsible for and effected by the standard.
• Processes for adding and removing members on the governance committees. [Office2] 

To ensure proper processes are in place for the voting, maintenance and changing of the standard amongst its various stakeholders; as well as certification and other capabilities of the standards development or maintenance organization. 

Intellectual Property and Licensing Costs

This criterion assesses whether the intellectual property and licensing terms for the standard allows the wide use of the standard. Example questions may include:

• Does the standard have licensing costs that are so expensive that the costs prohibits uptake of the standard?
• Are there network benefits or economies of scale to be realized due to previous implementations of the standard?[Office3] 
• Is it likely the standard licensing costs will increase over time?
• If a standard is currently free, are there other hidden conditions?
• Does the standard have licensing restrictions that prevent it from being implemented by anyone?
• What standards are being implemented in neighbouring regions/countries due to differing licensing costs? Will these impact expectations in our country?

To ensure that the standard is vendor neutral and allows for the wide use of the standard. Openness and intellectual property policies affect the status, uptake and implementability of specifications, especially in the long term. 

Defined Maintenance Process

This criterion assesses whether or not the standard custodian has processes in place to accept requests for change to address errors or new needs in the standard. When assessing if a maintenance process is in place, it is important to consider the following:

• There are defined processes in place for changes to the standard?[Office4] 
• Are processes in place to manage and resolve conflicts related to change processes?
• The change processes are responsive to stakeholder needs.
• The frequency of updates should be sufficiently short to accommodate new codes and repairs quickly, as well as extensions or temporary (?hot?) fixes to resolve immediate needs in between full updates.
• Is the versioning process clearly defined, documented and compatible with business requirements?
• Is the maintenance body responsive to requests for assistance, maintenance, etc.?

Business requirements and technology change over time, as such, standards need to evolve to meet changing needs.  It is therefore important to ensure the selected standard has defined maintenance processes in order to evolve.

Criteria

Description

Rationale

Costs and Benefits of Implementation

This criterion assesses whether the cost of implementation and disruption to current business is affordable given the benefits. It is important to consider not only the initial cost but the on-going cost for maintenance as well. Cost is not only measured in currency, but can be in time or effort as well. Some sample questions to ask when assessing the costs and benefits of implementation, include:

• What is the return on investment for the vendor?
• What is the cost of tools required?
• Will the implementation timelines be increased due to the complexity of the standard?
• Will cost or risk of subsequent system implementations be reduced?[Office1] 
• What is the number of people required for implementation?
• Is there any existing commercial off the shelf software vs. custom software?
• If properly implemented, would the standard provide system or societal benefits such as improved decision support or clinical research? 

To take into consideration any cost constraints when adopting a standard, as well as initial implementation and longer-term maintenance costs. 

Governance Structure

The purpose of this criterion is to assess whether or not the standard is governed appropriately.  When assessing if a governance structure is in place, it is important to consider whether there are: 

• Defined processes in place to facilitate decision making and issue resolution related to both standards content and processes.
• Balanced representation of communities responsible for and effected by the standard.
• Processes for adding and removing members on the governance committees. [Office2] 

To ensure proper processes are in place for the voting, maintenance and changing of the standard amongst its various stakeholders; as well as certification and other capabilities of the standards development or maintenance organization. 

Intellectual Property and Licensing Costs

This criterion assesses whether the intellectual property and licensing terms for the standard allows the wide use of the standard. Example questions may include:

• Does the standard have licensing costs that are so expensive that the costs prohibits uptake of the standard?
• Are there network benefits or economies of scale to be realized due to previous implementations of the standard?[Office3] 
• Is it likely the standard licensing costs will increase over time?
• If a standard is currently free, are there other hidden conditions?
• Does the standard have licensing restrictions that prevent it from being implemented by anyone?
• What standards are being implemented in neighbouring regions/countries due to differing licensing costs? Will these impact expectations in our country?

To ensure that the standard is vendor neutral and allows for the wide use of the standard. Openness and intellectual property policies affect the status, uptake and implementability of specifications, especially in the long term. 

Defined Maintenance Process

This criterion assesses whether or not the standard custodian has processes in place to accept requests for change to address errors or new needs in the standard. When assessing if a maintenance process is in place, it is important to consider the following:

• There are defined processes in place for changes to the standard?[Office4] 
• Are processes in place to manage and resolve conflicts related to change processes?
• The change processes are responsive to stakeholder needs.
• The frequency of updates should be sufficiently short to accommodate new codes and repairs quickly, as well as extensions or temporary (?hot?) fixes to resolve immediate needs in between full updates.
• Is the versioning process clearly defined, documented and compatible with business requirements?
• Is the maintenance body responsive to requests for assistance, maintenance, etc.?

Business requirements and technology change over time, as such, standards need to evolve to meet changing needs.  It is therefore important to ensure the selected standard has defined maintenance processes in order to evolve.