Vaccine efficacy: beyond the average

Australian experts are debating about the efficacy of AstraZeneca vaccine. Some argue that the efficacy (VE) of the vaccine is not good enough for herd immunity; others say that it is better than … face mask. I think the debate has been misled by the heavy reliance on the average. In this note, I want to make 3 comments on the true value of VE and its implication for herd immunity.

Australia is about to implement a COVID-19 vaccine program, with Oxford/AstraZeneca vaccine being the intervention of choice. It is expected that up to 80,000 people will be vaccinated by week 4, and 4 million by the end of March 2021. It is going to be a major public health effort.

However, some experts (eg immunologists and infectious disease specialists) are calling for a rethink of the program. They quoted the vaccine efficacy of 62% and argued that such a degree of efficacy is not high enough to achieve herd immunity [1].

Other experts, including the Government’s Chief Medical Office (CMO), argue that the vaccine efficacy is good enough. An expert quoted the efficacy of ‘up to 70%’ and said that was a great news. He reminds us that the efficacy of mask wearing was only 15–20% [1]. The CMO went on to advise that we should listen to the experts, because “they are hand picked experts on these matters”.

Which vaccine efficacy?

Well, in science we have the saying, “In God we trust, all else bring data”. Surely, experts’ opinions are important, but scientifically their opinions are ranked lowest in evidence-based medicine. As can be seen, in terms of vaccine efficacy, some experts quoted the 62% figure, others quoted the 70% figure. Who is correct? Which number is correct?

We need to see the original data. The data are available from the Lancet [3], and the key data are reproduced in the table below. Also, please pay attention to the title of the paper, ‘an interim analysis’, suggesting that the reported data are preliminary. That fact should be kept in mind when we interpret the data.

However, before interpreting the data, we should to pay attention to a glitch in the study. According to the study’s protocol, all participants were supposed to receive two standard doses (SD) of the vaccine. However, due to a serendipity of an error, the distribution of vaccine didn’t go out as planned, and as a result, some participants (n = 2741) received a lower dose (LD) followed by a standard dose (SD). Thus the study eventually ended up two groups of participants:

· SD/SD (n = 8895): participants received two SDs;

· LD/SD (n = 2741): participants received LD and then SD.

Common sense suggests that the SD/SD group should have greater efficacy than the LD/SD group. However, the actual results were opposite to expectation:

· SD/SD group: VE = 62% (95% confidence interval [CI], 41% to 76%);

· LD/SD group: VE = 90% (95% CI, 67% to 97%);

· Combined SD/SD and LD/SD: VE = 70% (95% CI, 55% to 81%).

So, now you can see where those numbers quoted by the experts (eg 62%, 70%, and 90%) are from. Actually, there are more VE numbers for each subgroup (see table). You can pick a number of your favorite!

For me, I would use the data derived from the combination of LS/SD and SD/SD groups (VE = 70%). Why? Because the VE for this sample has the narrowest confidence interval, meaning that it is more reliable than all other VE estimates.

Beyond the mean

Let us be honest: We — you, me, and the experts — do not know the true value of VE. We rely on data from a study to estimate VE. We then rely on statistics to make inference concerning the true value of VE based on the observed data.

Now, all the VE estimates quoted by the experts are averages. More specifically, they are sample averages. And, sample averages are expected to vary from sample to sample. Any replicated study — with the same design and the same measurement — will produce a different VE estimate. The point I want to make is that none of the quoted VE figures tell the whole story.

On average, the Oxford/AstraZeneca vaccine has a VE of 70%. However, the study’s data also tell us that there is 95% chance that the true VE value ranges anywhere between 55% and 81% [4]. The actual distribution of VE looks like the following figure:

The above figure shows that there is a slim chance that VE < 50% or VE > 80%. Indeed, there is only a 3% probability that VE > 80%. Moreover, there is a 87% probability that the true VE ranges between 60% and 80%. To me, that is the range we should work on.

How many people should be vaccinated to achieve herd immunity?

Now, back to the question of herd immunity. The critical question is: how many Australians need to be vaccinated to create herd immunity? The answer to this question is dependent on two parameters: VE and reproduction number (R0). R0 is the average number of secondary cases infected by a single infected individual over a certain period. The actual formula is:

P = (1–1 / R0) / VE.

where P is the proportion of population needs to be vaccinated.

We know that VE is likely to be between 60% and 80%.

What about R0? This figure also varies from sample to sample. A meta-analysis [5] found that the average R0 value was 2.9, with 95% CI ranging from 2.4 to 3.4.

That means we do not have a single P. We have a series values of P for various values of R0 and VE as shown in the figure below:

Based on the above consideration, I reckon that between 70% and 95% of the population need to be vaccinated to create a community of herd immunity.

In summary, my contributions are that:

· we don’t know the true value of vaccine efficacy, but the possible values are between 60 and 80%, with average being 70%;

· herd immunity is not just dependent on vaccine efficacy, but also the reproduction number which ranges between 2.4 and 3.4, with average being 2.9;

· given the above ranges of values, between 70% and 95% of Australian people needs to be vaccinated with Oxford/AstraZeneca to achieve herd immunity.

There is so much uncertainty in the science of covid-19. A recent BMJ editorial [6] reminds us that “when deciding whom to listen to in the covid-19 era, we should respect those who respect uncertainty, and listen in particular to those who acknowledge conflicting evidence on even their most strongly held views.” I totally agree with this advice.

_______

[1] https://7news.com.au/sunrise/on-the-show/calls-for-australias-rollout-of-oxford-astrazeneca-covid-vaccine-to-be-paused-over-efficacy-concerns-c-1945573

[2] https://www.abc.net.au/news/2021-01-13/government-coronavirus-vaccine-rollout-astrazeneca-efficacy/13053302

[3] https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)32661-1/fulltext

[4] I am making a Bayesian inference with a uniform prior distribution here.

[5] https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0242128

[6] https://www.bmj.com/content/371/bmj.m3979

osteoporosis | epidemiology | genetics | biostatistics | data enthusiast

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store