Blogs

Understanding the role of observed vs. expected observational studies in vaccine safety
Helen Petousis-Harris, BSc, PGDipSci (Dist), PhD and Karin Batty, RN, PGDipPH (Dist)
  • Observed versus expected studies provide valuable information for identifying potential rare or serious adverse events associated with vaccination.
  • These studies cannot definitively prove whether an adverse event is caused by a vaccine or not, but they can provide valuable insight into the safety of a vaccine when used in combination with other evidence.
  • The COVID-19 vaccines that are currently in wide use are very safe and the risks associated with them are extremely rare and much lower than those associated with the disease itself.
  • Globally, healthcare, and public health specialists strongly recommend vaccination against COVID-19 disease.
  • It is important to be informed about all potential risks before making any decisions about getting vaccinated.


Vaccines are a cornerstone of public health, providing protection against many infectious diseases. Vaccine safety monitoring is an important aspect of ensuring that vaccines are safe and effective. In this blog, we will discuss observed versus (vs.) expected observational studies, which are commonly used tools to actively monitor vaccine safety.

In vaccine safety monitoring, observed vs. expected observational studies are used to assess the risk of rare and potentially serious adverse events followingIllustration showing background rates, expected rates and a potential signal  vaccination. These studies compare the rate of disease or other health outcomes in vaccinated people with what is expected based on population rates or epidemiological data established before the vaccine was available (background rates). This type of analysis can identify clustering of adverse events and potential vaccine safety signals that could indicate new or unrecognised risks associated with receipt of a vaccine. Through use of these observational studies, experts can also better understand how vaccines may be contributing to population health outcomes.

While observed vs. expected observational studies contribute valuable information and evidence to assess the risk of rare or serious adverse events associated with immunisation they do have limitations.

Identification of potential signals that could indicate a new or unrecognised risk does not provide definitive answers about the safety of a vaccine.

  • Observed vs. expected studies can only determine if there is a temporal association between receiving a vaccine and experiencing an adverse event.
  • These studies cannot prove whether an adverse event is directly caused by a vaccination or whether it is due to other factors, such as an underlying health condition. 

Observed vs. expected studies should only be used in combination with other epidemiological evidence when assessing vaccine safety.

Examples of detection of safety signals using observed over expected studies

Myocarditis and pericarditis

In April 2021, the Israel Health Ministry advised that reports of heart muscle inflammation (myocarditis) after receiving a COVID-19 required further investigation. Regulatory agencies around the world responded immediately, checking adverse event reporting systems for reports of the condition after COVID-19 vaccination. Health professionals also responded, providing case reports in peer reviewed journals to ensure colleagues globally were aware that some people presented with myocarditis, and inflammation of the thin sac covering the heart (pericarditis) after receiving the SARS-CoV-2 mRNA vaccine.

Studies from the U.S. (2021), Canada (2022), Europe and Singapore (2023) provide examples of observed vs. expected observational studies that identified a potential safety signal related to the onset of myocarditis or pericarditis following COVID-19 vaccination. Subsequent studies, for example these studies from four European countries (2022) and Singapore and Hong Kong (2023) showed that there is an increased risk for some people following the receipt of COVID-19 mRNA vaccines, particularly after the second dose.

Watch a video about myocarditis, pericarditis and COVID 19 vaccines or read more about myocarditis and vaccines here.

Guillain-Barré syndrome

In July 2021, the EMA reported that there had been 108 cases of Guillain-Barré syndrome (GBS) reported from 21 million people who had received a SARS-CoV-2 viral vector vaccination. GBS is a syndrome that includes increasing muscle weakness or paralysis that may also have changed sensation (feeling). Regulatory agencies responded with further investigation of adverse event reports, as did health professionals with publishing case reports to inform colleagues globally.

Studies from the U.S. (2022), Australia (2022) and the U.S. (2023) provide examples of observed vs. expected observational studies that identified a temporal association between the receipt of a COVID-19 viral vector vaccine and the onset of GBS.
The occurrence of GBS after receiving a viral vector COVID-19 vaccine is very rare. Subsequent epidemiological studies, for example these studies from England (2022), Malaysia (2022), and Italy (2023), suggest an association between one of the two viral vector vaccines available.

Read more about Guillain-Barré syndrome and vaccines here.

What does this mean? Are vaccines still safe?Illustration of scales balanced on people with disease risks weighing more heavily than immunisation risks

The COVID-19 vaccines that are in wide use are very safe. That does not mean that there are not side effects and potential risks. Most side effects are minor and don't last long. However, it is important to know what the risks are so that decisions about getting vaccinated are based on good scientific information.

For many people, getting COVID-19 disease has risks, such as getting very ill and needing to be admitted to hospital or developing serious health complications including disease-related myocarditis and/or pericarditis or Guillain-Barré syndrome. Long COVID is also a consideration. For most people, being immunised against the virus significantly reduces the risks related to having the disease.

Comparison of risk (benefit) with risk can be used to help with making decisions about any medicine, vaccine, or treatment.

Key messages

  • Observed versus expected studies provide valuable information for identifying the risk of rare or serious adverse events associated with vaccination.
  • Myocarditis and pericarditis have been identified as potential risk following mRNA vaccines and, thrombosis with thrombocytopenia, and Guillain-Barré syndrome have been identified as potential risks associated with viral vector vaccines through these observational studies.
  • While O/E studies cannot definitively prove whether an adverse event is caused by a vaccine or not, they can provide valuable insight into the safety of a vaccine when used in combination with other evidence.
  • The COVID-19 vaccines that are currently in wide use are very safe and the risks associated with them are much lower than those associated with the disease itself. It is important to be informed about all potential risks before making any decisions about getting vaccinated.

More detailed information can be found in these articles

COVID-19 disease risks

COVID-19 vaccination safety

Papers comparing COVID-19 disease risks with vaccination risks

References

  • Ab Rahman N, Lim MT, Lee FY, Lee SC, Ramli A, Saharudin SN, et al. Risk of serious adverse events after the BNT162b2, CoronaVac, and ChAdOx1 vaccines in Malaysia: A self-controlled case series study. Vaccine. 2022;40(32):4394-402.
  • Abara WE, Gee J, Marquez P, Woo J, Myers TR, DeSantis A, et al. Reports of Guillain-Barré syndrome after COVID-19 vaccination in the United States. JAMA Netw Open. 2023;6(2):e2253845.
  • Bots SH, Riera-Arnau J, Belitser SV, Messina D, Aragón M, Alsina E, et al. Myocarditis and pericarditis associated with SARS-CoV-2 vaccines: A population-based descriptive cohort and a nested self-controlled risk interval study using electronic health care data from four European countries. Front Pharmacol. 2022;13:1038043.
  • Diaz GA, Parsons GT, Gering SK, Meier AR, Hutchinson IV, Robicsek A. Myocarditis and pericarditis after vaccination for COVID-19. JAMA. 2021;326(12):1210-12.
  • Dorajoo SR, Tan XH, Teo DCH, Neo JW, Koon YL, Ng AJJ, et al. Nationwide safety surveillance of COVID-19 mRNA vaccines following primary series and first booster vaccination in Singapore. Lancet. Forthcoming 2023.
  • Fan M, Lai FTT, Cheng FWT, Tsie NTY, Li X, Wan EYF, et al. Risk of carditis after three doses of vaccination with mRNA (BNT162b2) or inactivated (CoronaVac) covid-19 vaccination: A self-controlled cases series and a case-control study. Lancet Reg Health West Pac. 2023;35:100745.
  • Hanson KE, Goddard K, Lewis N, Fireman B, Myers TR, Bakshi N, et al. Incidence of Guillain-Barré syndrome after COVID-19 vaccination in the Vaccine Safety Datalink. JAMA Netw Open. 2022;5(4):e228879.
  • Morciano C, Alegiani SS, Ippoliti FM, Belleudi V, Trifirò G, Zanoni G, et al. Post-marketing active surveillance of Guillan [sic] Barré Syndrome following vaccination with anti-COVID-19 vaccines in persons aged ≥12 years in Italy: A multi-database self-controlled case series study. medRxiv. 2023:2023.01.17.23284585.
  • Naveed Z, Li J, Spencer M, Wilton J, Naus M, García HAV, et al. Observed versus expected rates of myocarditis after SARS-CoV-2 vaccination: A population-based cohort study. 2022;194(45):e1529-36.
  • Osowicki J, Morgan HJ, Harris A, Clothier HJ, Buttery JP, Kiers L, et al. Guillain-Barré syndrome temporally associated with COVID-19 vaccines in Victoria, Australia. Vaccine. 2022;40(52):7579-985.
  • Tome J, Cowan LT, Fung IC-H. A pharmacoepidemiological study of myocarditis and pericarditis following the first dose of mRNA COVID-19 vaccine in Europe. Microorganisms. 2023;11(5):1099.
  • Walker JL, Schultze A, Tazare J, Tamborska A, Singh B, Donegan K, et al. Safety of COVID-19 vaccination and acute neurological events: A self-controlled case series in England using the OpenSAFELY platform. Vaccine. 2022;40(32):4479-87.