Main Study Used by FDA to Approve Covid-19 Vaccine Found No Significant Effect on the Risk of Death

By James D. Agresti
October 23, 2021
Correction appended

Overview

Buried 23 pages into the FDA’s approval summary for Pfizer’s Covid-19 vaccine is data that cuts to the core of the vaccination debate. Presented in a place and manner where it has been almost completely overlooked until now, the data reveals the number of deaths among people who received the C-19 vaccine and people who received a placebo in Pfizer’s largest clinical trial.

Those figures provide the only reliable measure of the vaccine’s greatest benefits and risks because they are unambiguous (death), enumerate the worst possible outcome (death), and control for every conflating factor (due to the design of the study).

These critical findings reveal that “there were a total of 38 deaths” during the trial, including 21 people who received the vaccine and “17 in the placebo group.” Considered together with the fact that about half of the 40,000 people in the trial received a placebo, these results show no statistically significant impact on the risk of death.

However, they do show that the range of potential impacts is likely minimal. With 95% statistical certainty, the results indicate that the vaccine could prevent up to one death or cause up to five deaths per year among every 1,000 people.

Beyond six months, the effects of the vaccine are much more uncertain because Pfizer and other vaccine manufacturers removed participants from their most-rigorous studies after a maximum of six months. They did this in defiance of government healthcare regulatory agencies who recommended that the subjects be enrolled for at least one year.

The benefits and harms to young children are even more murky because Pfizer’s primary study of them enrolled only 2,268 children with a current average follow-up time of 2.2 months.

 The Necessity of Randomized Controlled Trials

In approving Pfizer’s Covid-19 vaccine for adults and older children, the FDA mainly relied on a large randomized controlled trial (RCT). These are studies in which people are randomly assigned to receive or not receive a certain treatment.

When properly conducted, RCTs are the “gold standard” for clinical research because they provide “a rigorous tool to examine cause–effect,” which “is not possible with any other study design.” Thus, the medical textbook Rutherford’s Vascular Surgery calls RCTs “the pinnacle in clinical design.”

The defining feature of RCTs is that they control for every factor but the medical treatment that is studied. This is a natural outcome of randomly giving the treatment to some people and not to others. Combined with the use of a placebo so that people don’t alter their mindsets or behaviors as a result of knowing they received the treatment, quality RCTs ensure that any major differences between the groups who received and did not receive the treatment are due to the treatment and not some other variable.

That level of certainty cannot be obtained by observing people “in the wild” where non-random factors determine who receives and does not receive a treatment. For example, comparing the death rates of people who are vaccinated and unvaccinated against C-19 cannot prove whether the vaccines are more helpful than harmful because the odds of death are impacted by variables like these:

• Older people—who are more likely to die than younger people—have much higher C-19 vaccination rates than younger people.
• Immunocompromised people—who have conditions like cancer and HIV that increase their risk of death—are “plausibly more likely to be offered and seek vaccination” because they are very vulnerable to C-19.
• People who get vaccinated may feel invincible to C-19 and take fewer precautions like social distancing.

Researchers commonly use statistical techniques to “control” for such variables, but these methods cannot rule out the possibility that other factors are at play. Also, the techniques used to perform such analyses are prone to pitfalls.

Underscoring the reality that observational studies cannot match the reliability of RCTs, a 2018 paper in European Heart Journal compares RCT and non-RCT studies on drugs to prevent heart failure and finds that:

• the observational studies routinely conflict with the RCTs.
• “it is not possible to make reliable therapeutic inferences from observational associations.”
• RCTs “clearly remain the best guide to the treatment of patients.”

The root weakness of observational studies is that they can only measure associations, and association does not prove causation. Although commonly taught in high school math, this vital fact of medical and social science is routinely ignored by commentators, journalists, and Ph.D.’s.

None of this means that observational studies are clinically useless. They can illuminate paths for additional research, and in rare cases where their results are mathematically and logically overwhelming, they can estimate the effects of a treatment. However, their results should be taken with a grain of salt, especially if there are RCTs to the contrary.

Risk of Death

The Pfizer-BioNTech Covid-19 vaccine, which is being marketed under the name “Comirnaty,” is the only Covid-19 vaccine that is actually “approved” by the U.S. Food and Drug Administration and not merely “authorized for emergency use.” The primary basis for that approval was an RCT that enrolled 43,847 subjects aged 16 to 85. Per the FDA, these “participants were randomized 1:1 to receive 2 doses of either Comirnaty or placebo, 3 weeks apart.”

After monitoring the subjects for up to six months beyond their second dose of the vaccine or placebo, the study found “there were a total of 38 deaths, 21 in the Comirnaty group and 17 in the placebo group.”

That crucial statement, which has been almost completely overlooked until now, is disclosed on page 23 of the FDA’s vaccine approval summary in a paragraph that claims “none of the deaths were considered related to vaccination.”

In reality, only the RCT can objectively determine how many deaths were related to vaccination, since there are untold ways in which this may occur beyond potential direct effects like cardiac events, embolisms, fevers, and seizures. For example, many fatal car accidents are triggered by fatigue, and the RCT found that 70.1% of subjects under the age of 55 reported “fatigue” after receiving the vaccine. This is why the only way to accurately measure the net effects of all risks and benefits is via an RCT.

On a superficial basis, the mortality figures from Pfizer’s RCT suggest that the vaccine increased the risk of death by 24%. However, the death rate in both groups is so small (0.1%) that the difference between them is statistically insignificant.

Put precisely, the RCT found that the vaccine may decrease the risk of death by as much as 34% or increase the risk of death by as much 136%. Because the vast majority of people had a very low risk of death during the average enrollment time of the trial (0.3 years), these results indicate that the vaccine could prevent up to one death or cause up to five deaths per year among every 1,000 people. This range has a statistical certainty of 95%, meaning there is a 5% chance that the actual rate is outside of this range due to mathematical chance.

Ninety-five percent certainty is the standard threshold used by scientists to label a result “statistically significant,” but more than 800 scientists and the American Statistical Association have warned that this practice is fraught with risks. For instance, it can lead people to make life-or-death decisions based on data that has 5% odds of being inaccurate due to mere chance.

For added perspective, if 10 more deaths occurred among the 20,000+ people who received the vaccine, the study would show with 95% confidence that the vaccine increased the risk of death by at least 1% and as much as 232%. Conversely, if 20 more deaths occurred among the 20,000+ people who received the placebo, the study would show with 95% confidence that the vaccine reduced the risk of death by at least 1% and as much as 199%.

Some may argue that the study was “underpowered,” a medical term for clinical trials that don’t enroll enough participants to detect an effect. However, Pfizer’s trial enrolled more than 40,000 people, and 10 more deaths in the vaccine group, or 0.05% of the vaccinees, would have yielded a statistically significant result.

Any argument to dismiss the death data as the result of an underpowered study raises the question of why Pfizer would design an RCT that is incapable of detecting the primary objective of the vaccine: saving lives. It also raises the question of why the FDA would approve a vaccine based on a study that was underpowered in this manner.

In short, the main RCT for Pfizer’s Covid-19 vaccine found no statistically significant effect on the risk of death. And if there is an effect, the results show that it is likely minimal.

Longer-Term Effects

All studies have their limitations, and a major one of Pfizer’s RCT is that none of the participants were enrolled for more than six months after their second dose of the vaccine. In fact, Pfizer began removing people from the RCT (through a process called “unblinding”) as soon as they became eligible to receive the vaccine under “local recommendations.”

That decision was made in defiance of guidance issued by a global association of 24 healthcare regulatory agencies called the International Coalition of Medicines Regulatory Authorities. This group includes the FDA and its counterparts in Canada, Australia, China, France, Germany, Mexico, Japan, Nigeria, India, and other nations.

In a statement released in November 2020, this international coalition of government agencies made the following points (and others) about why longer-term RCTs are necessary for C-19 vaccines:

• “To determine that the benefit of a vaccine outweighs its potential risk, regulators need robust and convincing evidence of the safety and efficacy that is obtained from well-designed randomised and controlled trials.”
• “Thus, continued evaluation of the vaccinated and the unvaccinated” participants “for as long as feasible will provide invaluable information.”
• Such information includes but is not limited to “additional and more precise information on longer-term safety,” “potential risks of vaccine-induced enhanced disease,” and “whether protection against Covid-19 disease wanes over time.”
• “Therefore, unless maintaining participants in their randomised treatment groups (vaccinated or control) after a vaccine is approved is clearly infeasible, we recommend that clinical trials should proceed as initially planned with a follow-up of at least one year or more from completion of assigned doses.”

Pfizer and other vaccine manufacturers directly flouted that guidance. As a result, an analysis published in August 2021 by the journal BMJ Evidence-Based Medicine reported that “placebo controlled follow-up, originally planned for 2 years in many trials, was eliminated after a few months, when manufacturers began offering vaccine to placebo recipients within weeks of receiving emergency use authorisations.”

Decisions to end the RCTs also hindered their ability to detect any effects of herd immunity as the broader society became vaccinated.

Medical ethics require that RCTs be barred or ended if they would undoubtedly harm people. Thus, some may argue that the RCTs should have been shortened based on their findings that the vaccines have large and statistically significant effects on reducing the risk of severe Covid-19. The Pfizer RCT, for example, found that the vaccine decreases the incidence of severe Covid-19 among people aged 16 and older by 70.9% to 100.0% (with 95% confidence).

However, those results don’t account for any side effects of the vaccine or long-term complications of C-19, and they don’t account for “whether protection against Covid-19 disease wanes over time,” as requested by the International Coalition of Medicines Regulatory Authorities.

Decisions to hastily end the RCTs are proving to be ill-advised given that a wide range of studies are finding that the immunity conferred by the current C-19 vaccines wanes over time. This includes but is not limited to:

• a study recently released by Moderna.
• a study conducted by the CDC.
• a study conducted for the Department of Defense.
• new studies published in working papers that have not yet undergone peer review.
• a massive Pfizer-funded study published a few days ago by The Lancet.

Since all of those are observational studies, they don’t have the surety of RCTs and are therefore tentative. This is precisely why Dr. Doran Fink, Deputy Director of the FDA’s Division of Vaccines and Related Products Applications, warned at an FDA committee meeting in October 2020:

Once a decision is made to unblind an ongoing placebo-controlled trial, that decision cannot be walked back. And that controlled follow up is lost forever.

Vaccinating Children

Given the indeterminate results of Pfizer’s RCT on adults and older teens, a larger and longer trial seems advisable for children. However, Pfizer has asked the FDA for an emergency authorization to administer its vaccine to children aged 5 to 11 mainly based on a study that enrolled only 2,268 children with a current average follow-up time of 2.2 months.

Such sample sizes and study periods are severely underpowered to determine if the vaccine will save more children than it kills. This is because less than one out of every 250,000 children in the U.S. aged 5 to 14 have died from Covid-19 since the outset of the pandemic more than 19 months ago.

For comparison, the main RCT conducted for the polio vaccine in the 1950s enrolled more than 400,000 children, or 177 times more than Pfizer’s primary study. Moreover, clinical studies generally require greater numbers of participants when the ultimate purpose of a treatment is to prevent a rare outcome, as is the case for childhood C-19 deaths. Polio-induced childhood paralysis—the ultimate target of the polio vaccine—was roughly 139 times more common in the 1950s than the current risk of childhood death from Covid-19.

Those weaknesses of Pfizer’s study are especially concerning because the risk of being harmed by Covid-19 greatly declines at younger ages, while the major known risks of the vaccine increase.

In fact, the FDA’s approval summary for Pfizer’s Covid-19 vaccine explains that under a “worst-case” scenario, the risk of vaccine-related heart problems called myocarditis and pericarditis “would exceed Covid-19 hospitalizations and deaths” among males 16-17 years of age. Still, the FDA approved the vaccine because the FDA’s “most likely” scenario showed that C-19 posed a greater risk than these heart problems and because of “additional factors” outlined in the summary.

That analysis addresses only one type of risk from the vaccine, and it only applies to older children, who are much more vulnerable to Covid-19 than younger children. Indeed, the odds of dying from C-19 are about nine times greater among people aged 15 to 24 than children aged 5 to 14:

Some scholars argue that decisions to vaccinate children should not be based solely on the well-being of the children but to protect older people. Beyond the moral issue of using children as human shields, such rationalizations fail to consider the following facts:

• Most children have asymptomatic cases of Covid-19, and a meta-analysis published by the journal JAMA Network Open in 2020 found that less than 1% of people with asymptomatic or pre-symptomatic C-19 infect other members of their households.
• A 2021 paper in the Proceedings of the National Academy of Sciences reported that a key driver of C-19 contagiousness decreases dramatically with age and body-mass index.
• The currently known risks and benefits of Pfizer’s C-19 vaccine led four of the 22 people on the FDA’s panel to vote against approving it for adults and older children.
• The primary RCT for the only C-19 vaccine that is actually approved by the FDA found no statistically significant effect on the risk of death among adults and older children.

While bearing in mind that association does not prove causation, it is also noteworthy that:

• a recent paper in the European Journal of Epidemiology found “no discernable association” between vaccination rates and new C-19 cases in 68 nations and 2,947 U.S. counties during late August/early September of 2021.
• there were about 54,487 C-19 deaths in the U.S. during September 2021 when 52% of the population was fully vaccinated, while in contrast, there were 23,189 deaths (less than half as many) in September 2020 when 0% of the population was vaccinated.

Summary

The U.S. Occupational Safety and Health Administration has decided not to enforce a federal regulation that requires employers to “record worker side effects from Covid-19 vaccination.” In the words of the agency, this is because it “does not wish to have any appearance of discouraging workers from receiving Covid-19 vaccination, and also does not wish to disincentivize employers’ vaccination efforts.”

Compare that “close your eyes” approach to the candor of the United Kingdom’s Medicines & Healthcare Products Regulatory Agency, which explains that “all vaccines and medicines have some side effects,” and “these side effects need to be continuously balanced against the expected benefits in preventing illness.”

RCTs are the only reliable way to accurately measure those risks and benefits. Buried 23 pages into its approval summary for Pfizer’s C-19 vaccine, the FDA reports that 38 people died in the main RCT for this vaccine, including 17 who received the placebo and 21 who received the vaccine. With 95% statistical certainty, this data shows that the vaccine could prevent up to one death or cause up to five deaths per year among every 1,000 people.

Those results are subject to limitations that could have been lessened if Pfizer and other vaccine manufacturers had not ejected participants from their RCTs after a maximum of six months. Yet, the manufacturers did this in defiance of government healthcare regulatory agencies who recommended that the subjects be enrolled for at least one year.

The situation with young children is even murkier because Pfizer is mainly relying on a study that enrolled only 2,268 children. This is less than 1% of the 400,000+ children who were enrolled in the main RCT for the polio vaccine. Based primarily upon this statistically underpowered evidence, the FDA has scheduled a meeting on October 26th to consider authorizing the use of this vaccine in children aged 5 to 11.

Correction (11/4/21): The original version of this article incorrectly stated that “Pfizer has asked the FDA for an emergency authorization to administer its vaccine to children aged 5 to 11 without conducting any type of RCT on them.” This statement was based on two Pfizer press releases (September 20 and September 28) that announced “positive topline results from the pivotal trial” for this vaccine in young children and that Pfizer was submitting these results to the FDA to seek “authorization” for this age group. Both press releases said nothing about an RCT or placebo. Instead, they reported  that the vaccine showed a “favorable safety profile” and “robust neutralizing antibody responses.”