GVDN AESI Observed vs Expected Rates Dashboards

 Overview of vaccine safety surveillance and monitoring for potential vaccine safety signals

Adverse events

An adverse event following immunisation (AEFI) is any untoward medical occurrence that follows immunisation and that does not necessarily have a causal relationship with the usage of the vaccine.

Adverse events of special interest (AESI) are pre-defined medically significant events that have the potential to be causally associated with a vaccine that need to be carefully monitored and confirmed by further studies. In response to availability of COVID-19 vaccines, the Brighton Collaboration Safety Platform for Emergency Vaccines (SPEAC) Project, developed, and now maintains, a list of adverse events of special interest (AESIs) for COVID-19 vaccination based on existing and emerging data and literature. The list, which includes events with a proven association with vaccination or a vaccine platform, and events with a theoretical concern related to the immune response, disease complications, and vaccine platform concerns seen in animal models, has been used to inform key health outcomes for background rates, rapid cycle analysis, and observed versus expected analysis.

The Brighton Collaboration have published standardised case definitions to assess many of the AESIs or are under development for the AESIs. 

Vaccine safety

Assessment of vaccine safety occurs during all three pre-approval clinical trial phases but, by necessity, vaccine recipient numbers are small. Even the largest phase III clinical trials, such as those for the currently available rotavirus vaccines, only include tens of thousands of participants compared with country and global populations for whom the vaccine has been developed. Clinical trials are also limited in the diversity of populations that can be enrolled, individuals who are pregnant or who have health conditions may be excluded, and diverse ethnic groups may not be represented.

Very rare health outcomes, which could be vaccine-associated, may not be seen until in hundreds of thousands to millions of individuals are being vaccinated. To identify a two-fold risk of a vaccine-associated adverse event that occurs once in 100,000 people, 2.35 million people who have received the vaccine need to be compared with 2.35 million people who have not. Ongoing vaccine safety surveillance is essential after vaccines receive emergency-use authorisation and/or licensure and vaccines are administered in the general population.

Passive reporting pharmacovigilance, the least expensive and most common system used by many countries, can identify potential vaccine safety signals for unexpected adverse events. However, the size of the population they monitor may prevent identification of very rare events, and the type and quality of the data collected can be influenced by many factors. Passive surveillance systems have significant limitations with rapidly and reliably identifying genuine vaccine safety signals.

Active safety surveillance includes processes that monitor large communities for the occurrence of health conditions or unexpected adverse events after vaccination. Ideally, monitoring is prospective and occurs in or near real-time such as rapid cycle analysis and observed versus expected studies once vaccines are being administered in the general population. Identification of potential vaccine safety signals provides a trigger for further work to identify if the incidence of the health condition or event is higher, the same, or lower when compared with a similar population during a similar time prior to the vaccine being available.

Data about the occurrence of the health condition or adverse event over a period before the vaccine was introduced, the baseline or background incident rates, is a critical component of vaccine safety surveillance.

Observed vs. expected rates

Observed vs. expected rates studies assess the occurrence of events of interest recorded in administrative health data across large populations. The populations include individuals with diverse ethnic, cultural, and personal circumstances that may not have been represented in clinical trials and who receive (or do not receive) a vaccine under real-world conditions. The size of the population, both vaccinated and unvaccinated, may be large enough to identify very rare adverse events.

Benefits

Comparison of post-vaccination rates of events of interest with contemporary background rates of the same events seen in the same population before anyone was exposed to the vaccine of interest has the potential to quickly identify potential vaccine safety signals and initiate early safety investigations. Results from initial investigations as to whether the potential signal is statistically significant or artifact (e.g., caused by small case numbers, a change in coding, or health system changes) can inform vaccination policies in a timely manner and subsequent association studies. The studies look at the relationship between the event occurring and receipt of a vaccine to identify whether the event occurs coincidentally after vaccination or is likely caused by the vaccination.

Limitations

• Comparison of observed with expected rates of events do not provide definitive answers about vaccine safety. They cannot prove whether an event was caused by the vaccination, an underlying health condition, or something else. Therefore, these studies should only be used in conjunction with other epidemiological evidence.
• Monitoring of observed vs. expected rates rely on national administrative health data.
• The occurrence and diagnosis of various medical conditions changes in populations over time irrespective of introduction of a vaccine, for example the increase of diabetes related to obesity, availability of high energy foods, and other lifestyle changes. Using contemporary background rates, comparing like with like, can reduce the risk of false signals in the collected data.

Temporal or causal associations

Health conditions and adverse events occur across populations because there are many causes of ill health and disease, they occur when vaccines have not been administered and sometimes, they occur during a period after receipt of a vaccine. 

An adverse event that occurs after receipt of a vaccine has a temporal (time) association with the vaccination but does not mean the event was caused by the vaccine. However, if a pattern of the adverse event occurring develops in a group of people, those monitoring vaccine safety will conduct further evaluation. 

An adverse event that is caused by a vaccine has a causal association with the vaccination. While the event may occur after (temporal to) the vaccination, a cause-and-effect relationship cannot be established simply because the two occurred around the same time. Further investigation through observed over expected and association studies, with systematic review of data would be conducted. 

The comic strip below highlights the false assumption that “an adverse event that occurs after receiving a vaccine must have been caused by the vaccine” and fallacy “post hoc ergo propter hoc” (after this, therefore because of this).

The World Health Organization Causality assessment of an adverse event following immunization (AEFI): User manual for the revised WHO classification, 2nd edition, 2019 provides a comprehensive guide to systematic, standardised global causality assessment.

          ^{Post hoc ergo propter hoc by @bryanMMathers is licensed under CC-BY-ND.}

National administrative health data

• The data are not coded or collected for use in health research.
• Codes are determined by presenting signs and symptoms, and/or diagnostic tests, and/or response to treatment, but the condition will rarely have been verified against a standardised case definition before being coded.
• Codes and coding practices may change over time.
• Usually, they are crude rates. Rates may be influenced by:
  • Age, gender, and distribution of the population, socioeconomic status, comorbidities, medication use, and study methodology to derive the data.
  • Time, such as seasons, and change over time.
  • Access to and use of healthcare services can affect where and when health outcomes are identified and coded.
    • The World Health Organization noted a significant decrease in healthcare visits for non-COVID-19 chronic health conditions when health resources were redirected to managing large numbers of COVID-19 cases after the pandemic started.