IIS Assessment

Immunization information system (IIS) Assessment, the second stage of the Measurement and Improvement (M&I) Initiative, involves formal testing to assess individual IIS using community-selected measures and tests.

Launching a formal IIS Assessment process to drive quality improvement and alignment with standards, while acknowledging there will always be local law and policy and flexibility within the standard that can add additional requirements and contrasts between IIS, provides information to plan and prioritize improvements while also fine-tuning standards and measures. This creates an iterative process of improvement of both IIS programs and the metrics used to assess them. Test results are shared with individual IIS users via the Aggregate Analysis Reporting Tool (AART). IIS programs and vendors can receive technical assistance to accelerate improvement.

All measures and tests used for IIS Assessment are directly related to the IIS Functional Standards and Operational Guidance Statements. Measures and tests are developed by the Measurement for Assessment and Certification Advisory Workgroup (MACAW) and vetted for comment with the full IIS community via a town hall meeting format, to ensure broad community input and agreement.

There are six content areas measured in IIS Assessment:

Measures and Tests by Content Area

Below are brief descriptions and links to the current measures and tests for each content area. Aggregate baseline and quarterly remeasurement reports are available on the AIRA repository.


Transport

This content area examines the level of alignment with the formally defined transport specification: SOAP/Web Services and the CDC Web Services Definition Language (WSDL).

When any two systems connect to exchange data, they must use an agreed upon transport layer to connect. Otherwise, communication and the exchange of data will be impossible. An expert panel made up of subject matter experts from across the IIS and stakeholder community selected SOAP/Web Services and CDC WSDL as the IIS Standard in 2012.

Resources for IIS Assessment-Transport:


Message Submission (SUB) and Acknowledgment (ACK)

This content area focuses on Submission and Acknowledgment messages to an IIS. VXU, or vaccine update, is an HL7 message that contains demographic and vaccination data that is submitted to an IIS by an outside electronic health record system (EHR). ACK, or acknowledgment, is an HL7 message used to relay information back to the EHR regarding the status of the VXU.

Resources for IIS Assessment-SUB/ACK:


Message Query (QBP) and Response (RSP)

This content area focuses on Query (QBP) HL7 message submissions that are sent to an IIS and Response (RSP) HL7 message submissions that are returned by an IIS in response to a querying system. The QBP is used to query data from the IIS and contains basic demographic data about a patient. The sending system wants to know if the IIS has any demographic and/or vaccination data on said patient, so it “asks” the IIS to respond to the QBP message. Once the IIS receives a QBP, the IIS attempts to match the demographic information contained in the QBP message with data in the IIS. It then sends back an appropriate response (RSP) HL7 message telling the sending system if the patient was found in the IIS and what information (if any) was found. 

Resources for IIS Assessment-QBP/RSP:


Data Quality Incoming/Ongoing

This content area focuses on measuring IIS ability to detect data quality issues at the individual message level. The IIS will be presented with messages that contain intentional data quality errors (e.g., vaccination date before date of birth), and the IIS is expected to detect these errors. The IIS will also be assessed on the ability to retain (and return) vaccination events from submission to query without changing the meaning (e.g., NDC mapped to CVX, return key information about a vaccination event).

Resources for IIS Assessment-DQI:


Clinical Decision Support (CDS)

This content area focuses on Clinical Decision Support (CDS) tools designed to automatically determine the recommended immunizations needed when a patient presents for vaccination. These recommendations are developed by the Advisory Committee on Immunization Practices. The measures for this content area assess three CDS concepts that can be returned via HL7 messages: Evaluation Status, Earliest Date, and Recommended Date. Each IIS is measured on capability to return a concept and accuracy if the concept is returned based on age of the patient.

Resources for IIS Assessment-CDS: 


Data at Rest (DAR)

This content area measures the completeness, validity, timeliness of data residing in an IIS regardless of how it arrived, and leverages data quality indicators from IIS Data Quality Practices – To Monitor and Evaluate Data at Rest. DAR completed its initial Testing and Discovery stage that included 55 patient-level and vaccine-level measures. In July 2022, MACAW and CDC supported the proposal to advance 45 of those measures to Assessment. AIRA hosted a community-wide Town Hall in October 2022 and subsequently published the measures and tests resource in December 2022.

Participation in DAR Assessment offers IIS usable and actionable reports that can drive data quality efforts. IIS can learn more about participating in DAR, review Frequently Asked Questions (FAQs), or request assistance via AIRA’s online technical assistance form.  

Learn More

Resources for DAR Assessment:

Updated: 4/28/23