Validation

Following the stages of Testing and Discovery and IIS Assessment, Validation is a summary step to acknowledge IIS that are progressing toward or achieving alignment with community-selected measures and tests. A Validation designation is automatically achieved when an IIS meets the designated measures and tests in a specific phase (e.g., Transport), as specified in the Aggregate Analysis Reporting Tool (AART).

Measures for each phase of Validation will be drawn from published IIS Assessment measures and tests, where results are already available to individual IIS via AART. No new measures or tests will be introduced in the Validation stage that are not already measured and visible in the IIS Assessment stage. All measures and tests throughout the Measurement and Improvement Initiative have been and will continue to be developed by the Measurement for Assessment and Certification Advisory Workgroup (MACAW), vetted for comment with the full IIS community via a Town Hall meeting format, and reviewed and approved by the AIRA Board prior to implementation.

As mentioned above, a “Validation” designation is automatically achieved when an IIS meets the designated measures and tests in a specific phase (e.g., Transport, Submission, Query, etc.), but interim steps/statuses toward Validation are also recognized. Statuses include:

  • Validated
  • Validated with Minor Differences
  • Major Differences
  • Participating But Not Measured
  • Opted Out

 

In keeping with Validation’s focus on transparency, the business rules for moving forward in the measurement process have also been simplified. MACAW assumes that IIS connecting with the Measurement and Improvement Initiative have a desire to participate and share results, unless an IIS states otherwise.

AART’s original settings of “opt-in” required users to declare their participation at each new stage and phase of measurement, raising questions from end-users who felt confused by having to say “yes” repeatedly. Looking across sites, AIRA staff noted participation was inconsistent and declined with each new phase. MACAW recommended - and the AIRA Board approved - a change to an “opt-out” approach with the rollout of Validation, where users will be automatically rolled forward to the next phase or stage. Admin users can modify their participation and sharing settings at any time. Click here for more information on participation and sharing settings.

Certain phases may be split into Basic or Complete measures. This graduated approach allows for more nuanced results and makes it easier to show both success and improvement. The Basic level includes essential measures an IIS would be expected to pass to minimally meet a specific functional area or phase, while the Complete level includes more comprehensive sets of measures for a given functional area or phase.

MACAW has proposed - and the AIRA board has approved - the measures for Transport as the first phase of Validation. MACAW will be proposing Submission and Acknowledgement, Query and Response, and Clinical Decision Support later in 2018.  

Validation by Phase

Below are measures and tests developed to date. Baseline and quarterly re-measurement reports are also available for the completed IIS Validation phases via the AIRA repository found here.

Transport

This phase examines the level of alignment with the formally defined transport specification, SOAP/Web Services and the CDC WSDL. When any two systems connect to exchange data, they must use an agreed upon transport layer to connect; otherwise, communication and the exchange of data will be impossible. An expert panel made up of Subject Matter Experts (SMEs) from across the IIS and stakeholder community selected the SOAP/Web Services and CDC WSDL as the IIS Standard in 2012.

Helpful resources for Transport Validation:

Message Submission (VXU) and Acknowledgement (ACK)

This phase focuses on submission and acknowledgement messages to an IIS. VXU, or vaccine update, is an HL7 message that contains demographic and vaccination data which is submitted to an IIS by an outside electronic health record system (EHR). ACK, or acknowledgement, is an HL7 message used to relay information back to the EHR regarding the status of the VXU.

Helpful resources for the Submission and Acknowledgement Validation:

  • Finalized and approved measures and tests for Submission and Acknowledgment are available here.
  • Information on the community wide town hall meeting for Submission and Query Validation, along with a recording and presentation slides are available.
  • Submission-Acknowledgment Validation Reports will be available soon.

Message Query (QBP) and Response (RSP)

This phase focuses on Query by Parameter (QBP) HL7 message submissions that are sent to an IIS and Response (RSP) HL7 message submissions that are returned by an IIS in response to a querying system. The QBP is used to query data from the IIS and contains basic demographic data about a patient. The sending system wants to know if the IIS has any demographic and/or vaccination data on the said patient, so it therefore “asks” the IIS to respond to the QBP message. Once the IIS receives a QBP, the IIS attempts to match the demographic information contained in the QBP message with data in the IIS. It then sends back an appropriate response HL7 message (RSP) telling the sending system if the patient was found in the IIS and what information (if any) was found. 

Helpful resources for the query and response phase:

  • Finalized and approved measures and tests for Query and Response are available here.
  • Information on the community wide town hall meeting, along with a recording and presentation slides are available.
  • Query-Response Validation Reports will be available soon.

Additional Resources

AIRA Town Hall Webinar: Introducing Validation, and Measures for Transport

IIS Validation Measures and Tests – Transport

Immunization Information System (IIS) Measurement and Improvement: Testing and Discovery, Assessment and Validation Overview

Information on Participation and Sharing Settings

Transport Validation: Basic Level - April 2018

Transport Validation: Complete Level - April 2018