Page History

General The following general screening criteria can be applied to screen available standards for implementation with within a solution.  It is assumed that the standards in use with existing systems participating in the solution will be evaluated as part of the initial screening, as well as options for resolving any differences between existing, implemented standards and those selected.

 Continue to Standards Specific Assessment Criteria.

Anchor

Fit for purpose Fit for purpose

Fit for Purpose Criteria

Fit for purpose criteria is used to evaluate the appropriateness of the a standard for within the intended business, clinical and technical context by considering whether the standard: aligns .

Aligns to the

Table 1 - Fit for Purpose Criteria

Criteria

Description

Rationale

Alignment to the eHealth Blueprint

The purpose of this criterion is to assess whether the standard supports the exchange of information between two or more components of the EHR. The standard should also be intended to be used in a federated approach to EHR architecture where multiple organizations are expected to build, operate and/or host the various applications.

The standard supports eHealth mandate in providing a single, harmonized, coherent province-wide EHR.

Existing Standard [RB2] 

This criterion assesses whether the standard is adopted/adapted from existing standards, and whether it is intended for international, pan- Canadian.

Digital Health Blueprint

Selection of existing interoperability specifications and terminology subsets which have been deployed to support similar use cases improves compatibility between solutions, supports rapid project delivery and decreases speed to value.

Selection of standards which are health care delivery setting independent can be reused across multiple health care delivery settings.

Reuses an Existing, Shared Standard

Reuse of widely adopted standards across different interoperability specifications or terminology subsets allows reuse of code,

applications, conformance testing environments, etc.

 This also allows reuse of existing tools/skills/knowledge sets to establish the standard.

Supports Business Requirements

Business Requirement Support

Selected standards must address all in-scope business requirements or be adapted to meet project needs.

This criterion assesses whether the standard expresses all information required by the business/clinical domain. While the exact business requirements are project specific, example questions may include:

To ensure that the standard addresses the health care business requirements.

The standard should be health care delivery setting independent to enable use across multiple health care delivery settings.

Supports Technical Requirements

Standards selected must be feasible to adopt and implement according to the technical requirements and constraints of the project.

To enable successful implementation of the standard.

Likely to be Adopted[Office3] 

Likelihood of Adoption

Standards that are adopted and incorporated into solutions, either by vendors or other implementers, have better staying power in the market and offer a greater opportunity to provide value back to implementers through future integrations.  Implementation costs associated with widely adopted standards may also be lower.

To ensure success of the implementation. If the standard is not widely implemented or supported by vendors, it will result in increased time, cost, and effort. 

Appropriately supports coded, structured and free text content

Different standards support different data structures and levels of terminology coding.  These difference entail trade-offs to application developers and users. At one extreme, end users may resist using an application which forces them to enter data in too many fields, increasing their time/effort to produce documentation. At the other extreme, just capturing free text can contribute to privacy breaches and present a barrier to implementing decision support.  

Supports coded and free text content.

This criterion assesses the degree to which coded data is required.

Anchor
steward steward

Stewardship Criteria

Stewardship criteria facilitate support comparison of the standard‘s potential standard’s steward in terms of governance structure, licensing and intellectual property rights, and defined maintenance process.

Table 2 - Stewardship Criteria

Criteria

Description

Rationale

 These criteria are associated with total costs of ownership and the future availability of the standards selected.

Costs and Benefits of Implementation

Is the cost of implementation and disruption to current business

affordable given the expected benefits

?  It is important to consider not only the initial cost but the on-going cost for maintenance as well. Cost is not only measured in currency, but can be in time or effort as well.

To take into consideration any cost constraints when adopting a standard, as well as initial implementation and longer-term maintenance costs. 

Governance Structure

The purpose of this criterion is to assess whether or not the standard is governed appropriately.  When assessing if a governance structure is in place, it is important to consider whether there are: 

Governance Structure

Well governed standards have processes in place for the voting, maintenance and changing of the standard amongst its various stakeholders; as well as certification and other capabilities of the standards development or maintenance organization. 

Intellectual Property and Licensing

Intellectual property policies, licencing terms and licensing costs can impact standards uptake.

Expand
title Assessment Questions
Does the standard have licensing costs that are

so expensive that the costs prohibits uptake of the standard?Are there network benefits or economies of scale to be realized due to previous implementations

significant enough to inhibit uptake of the standard?

[Office3] 

Is it likely the standard licensing costs will increase over time? If a standard is currently free, are there other hidden conditions? Do existing implementations provide network benefits which justify costs? Does use of the standard offset other costs (such as maintenance)? Does the standard have licensing restrictions that prevent it from being implemented by anyone? What standards are being implemented in

neighbouring

neighboring regions/countries due to differing licensing costs? Will these impact expectations in our country?

 

To ensure that the standard is vendor neutral and allows for the wide use of the standard. Openness and intellectual property policies affect the status, uptake and implementability of specifications, especially in the long term. 

Defined Maintenance Process

Maintenance Process

Business requirements and technology change over time and, as such, standards need to evolve to meet changing needs.  It is therefore important to ensure the selected standard has defined maintenance processes in order to evolve.  There are also costs and benefits associated with the frequency of maintenance cycles and content changes.

This criterion assesses whether or not the standard custodian has processes in place to accept requests for change to address errors or new needs in the standard. When assessing if a maintenance process is in place, it is important to consider the following:

Anchor
quality quality

Quality Criteria

Standard quality criteria evaluate relate to the overall calibre of the standard and are not related to the standard content itself. They must enable measure the level of interoperability, be adaptable, and stable.  Additional implementation resources must exist, adaptability, and stability provided by the standard as well as the availability of supporting resources such as training, implementation guides, and software to support development, conformance testing, and maintenance of the standard.[Office1] 

Table 3 - Standard Quality Criteria

Criteria

Description

Rationale

Implementation Support and Education

To assess the degree to which resources are able to assist with appropriate use and implementation of the standard. Some key questions to consider when assessing this criteria are:

Services which support implementers may help increase the rate of adoption and/or decrease the implementation and ownership costs for projects who would otherwise have to provide support services to adopters directly.  These services may be provided by the custodian or service providers for a fee or free of charge.

Expand
title Assessment Questions
Does the standard’s custodian provide direct support to implementers?  Do others?  At what cost

Does the standards development/maintenance organization (e.g. HL7, IHE, Infoway, etc.) provide support for implementation

? Is it easy to obtain education and/or training on the standard and supporting tools?   Is the education or training offered in multiple formats (online training modules, books, in person, etc.)?

If there are organizations available to provide supporting services to help implement, it will help increase the number of adopters and decrease the costs for specific implementers who would otherwise have to provide those services. 

Enables Interoperability

Existing and new clinical solutions will need to exchange information in a seamless manner.  Standards which are complementary to – or strategically positioned to work with – existing interoperability standards implementations

may offer greater flexibility or long term benefit.

(Note that greater flexibility can also increase solution complexity.  It is important to consider how different components of an integrated solution will interoperate with one another and requirements for supporting services when evaluating this criterion.)

To enable existing and new clinical solutions and provide the capability of exchanging information in a seamless manner. To ensure that previous data and legacy systems are compatible with the standard and can be upgraded if necessary.

Implementation and Maintenance Tools

The existence of tools

and code libraries

that are easily accessible or commercially available can reduce the effort and cost required to implement and maintain a particular standard.

Conformance Testing Methodologies and Tools

Conformance criteria and profiles are used to test whether a standard has been implemented correctly.  Process to measure solution conformance may already be in place and available to support the project. 

In order to test whether a standard has been implemented correctly, a set of conformance criteria and profiles against which a standard can be measured is important. Ideally, a process to measure conformance should be in place to publicly notify those vendors that are certified (meet testing criteria). 

Stability

Standards which are stable and widely implemented are more attractive for other implementers to adopt.  Less mature standards may be more volatile and fragile. 

Proven Stability [Office5] 

If a standard is stable and widely implemented it is more attractive for other implementers to adopt it.

If a standard is still in inception, it is more volatile and fragile, which will likely lead to increased risks (e.g. increased implementation costs and timeline delays) for initial implementers.

Adaptability

Different standards allow implementers different levels of flexibility to make modifications to support local needs.  

(Note that greater flexibility can also increase solution complexity.  It is important to consider how different components of an integrated solution will interoperate with one another and requirements for supporting services when evaluating this criterion.)

Adaptable and customizable [Office6] 

For every interoperability standard, it is essential to describe how tightly the aspects defined in the standard constrain the options for the implementation, and which aspects are left for the implementation-specific or customer-specific conventions.