A reanalysis of an Italian study on the effectiveness of COVID-19 vaccination suggests that it might have unintended effects on total mortality

Un commento su “A reanalysis of an Italian study on the effectiveness of COVID-19 vaccination suggests that it might have unintended effects on total mortality”

1. Norman Fenton – 08/05/2024 12:45

   The paper sets out to analyse data sourced from the Italian National Healthcare System, from the province of Pescara, Italy, comprising just under one million people of all ages.

   What makes this paper interesting and exciting is that, unlike almost all observational studies of vaccine effectiveness and safety, two critical sources of bias are avoided:

   – Immortal time bias (ITB) – this is where a single individual might spend time in one or more vaccination categories. So rather than simply count someone as vaccinated or not, one would count the time spent in the unvaccinated category and the time spent in the vaccinated category (in days). This matters when determining exposure to some potential hazard or benefit to mortality, such as vaccination.

   – (Un)Healthy vaccinee effect – this is where a cohort contains greater or fewer people suffering from a comorbidity than another cohort that you wish to compare it with. Obviously, a cohort containing proportionally more (or fewer) people with comorbidities will already contain a higher or lower mortality profile and hence won’t be directly comparable.

   To resolve the immortal time bias they say:

   “…we aligned the entire population on a single index date (1st January 2021), calculating the time spent as an Unvaccinated for the 1 – dose population, the time spent as an Unvaccinated and as 1 – dose for the 2 – doses population and, finally, the time spent as an Unvaccinated, 1 – dose and 2 – doses for the 3/4 doses population.”

   Then they calculated the person-days a person had spent in each category.

   However, unfortunately, one other source of bias remains present in the data (this was outside of the research group’s control). This is the miscategorisation bias, whereby, within a given time frame, such as two weeks, they will be categorised as unvaccinated when they are in fact vaccinated. This miscategorisation ‘cheap trick’ applies to those double dose and boosted vaccinated too. Hence, a boosted person would count as double dosed up to two weeks after vaccination with the booster.

   Finally, their basis for comparison is all-cause mortality (ACM), and in the paper they are careful to justify why they focused on using this as the main basis for evaluating mortality effects of the vaccine, rather than Covid-19 mortality which is much less reliable and does not allow for the potential detection of any vaccine safety signal.

   The (deliberately?) understated title of the study subtly tells the reader that the paper does not contain good news about the vaccines:

   vaccination suggests that it might have unintended effects on total mortality

   Lay readers might find the detailed results of the paper difficult to follow, as they are written for biostatisticians and epidemiological specialists, but they are nevertheless unambiguous despite this.

   They compute the adjusted Hazard Ratio (HR) by calculating the ratio of all-cause mortality in one group (such as the unvaccinated) to another (the 1-dose, 2-dose, or 3/4-dose vaccinated). A value of HR exactly equal to ‘one’ (HR = 1) would mean there was no difference in ACM between a vaccinated cohort and the unvaccinated. A 95% confidence interval that contains the value one means the cohorts are not significantly different in a statistical sense.

   They performed two analyses:

   – Univariate: this is a direct comparison of one cohort against another without taking account of any other variables.

   – Multivariate: this is a comparison between cohorts by taking account of hypertension, cancer, (Covid-19) infection, sex and age. This means that any variability from these variables can be ignored in the presented result (i.e. it removes some of the confounding effects attributable to health).

   The HRs, with 95% confidence intervals at a group level are in their Table 5.

   All 1-dose, 2-dose, and 3/4 dose cohorts are compared against the baseline of the unvaccinated cohort.

   In the univariate analysis the HR for the 1-dose vaccine suggests it does not have a statistically significant effect on ACM, with a mean value of 0.88. However, for the 2-dose and 3/4-dose cohorts the negative effect on ACM is unambiguous and statistically significant with mean HRs of 1.23 and 1.21. Note that the confidence intervals are not inclusive of the HR = 1, except in the 1-dose case. This is strong evidence of a safety signal.

   For the multivariate case, where health and age confounders are accounted for, things look much worse for the vaccines. Here the HRs for the 1-dose and 2-dose cohorts are each much greater than one, at 2.40 and 1.98 respectively, and both results are highly statistically significant, meaning that single and double doses of the vaccine have a detectable and negative effect on ACM.

   Note that in the multivariate analysis 3/4-dose does not appear to affect ACM in the same way. With an HR of 0.99 it has no effect, thus contrasting with the 1 or 2 doses. However, notice the decreasing trend in the HR value as doses increase: 2.4, 1.98, 0.99. The authors suggest this reveals possible evidence of a harvesting effect, where the vulnerable have died at one or two doses and hence cannot move into the 3/4-dose category after death3. Only those healthy enough have survived. But more importantly given that the multivariate analysis removes any confounding effect by age, cancer, hypertension and Covid-19 infection this presents a cleaned-up mortality signal.

   They also used Restricted Mean Survival Time (RMST), as an index of ‘life expectancy’.

   Here the comparison was between 2-dose vaccinated and the unvaccinated and the 3/4-dose vaccinated versus the unvaccinated (their Tables 3 and 4). So, for the 2-dose cohort the mean survival time, compared to the unvaccinated, was reduced by 2.7 days (731.62 – 728.92 = -2.7) and this was a statistically significant result. For the 3-dose cohort the reduction was 0.784 days.

   The authors remark on how meaningful these figures are:

   They may appear irrelevant (of the order of a few days), but they refer to a limited period of time (739 days for those vaccinated with 2 doses and 579 days for those vaccinated with 3 or more doses). They could be compared with the entire life expectancy of an individual, which in the province of Pescara has an average value of 82.6 years (corresponding to 30,149 days).

   They extrapolated the results to calculate the effect on the entire life expectancy of the Pescara population. The effect of 2-doses was to reduce life expectancy by 3.6 months, and for 3/4 doses it was 1.31 months.

   The methods used by the authors have allowed them to resolve confounding issues due to a number of biases, crucially adjusting for the immortal time bias. And partially adjusting for (un)healthy vaccine bias (their multivariate analysis has adjusted for age, Covid-19 infection status, hypertension and cancer, but does not adjust for other comorbidities).

   Note that the authors state in the discussion that:

   “It should also be considered that in Italy the release of exemptions was very limited, while the subjects with pathologies were vaccinated with a priority indication and induced to continue with vaccination with 1 or 2 booster doses.”

   Also, they say:

   “..the population of those vaccinated with 3 or more doses displays the highest percentage of subjects with pathologies (28% versus 16% of those vaccinated with only 2 doses, and 17% of those with only 1 dose)”.

   In personal correspondence the authors have also confirmed that these percentages refer to those who had at least one pathology, but in fact the comorbidities are concentrated in subjects who have more than one at the same time.

   This suggests there was indeed a concentration of unhealthy people in the vaccinated cohorts, but which has been partially adjusted for in their statistical analysis. But as we have mentioned, the data still suffers from miscategorisation bias (which cannot be adjusted for). This confirms what we found and reported in the UK data, back in 2021.
   Conclusions

   Even though the data suffers from the miscategorisation bias, as well as some other potential confounding effects, which they carefully note, this is clearly the best quality study we have available on Covid-19 vaccination to date.

   The authors state that they went into the study expect a null result (HR = 1), and that the vaccines would register zero safety signal. However, what they found was:

   The univariate analysis, carried out using the Cox proportional hazards model, shows an increase in the risk of the vaccinated compared to the unvaccinated, as one moves from the first to the subsequent doses. This confirms what has already been highlighted in our previous article [4]. In fact, the HR was slightly lower than 1 with the first dose, while with the second and third doses the risk for the vaccinated resulted significantly higher (more than 20%) than that of the unvaccinated.

   On the multivariate analysis they say:

   age, sex and previous pathologies are confounding factors affecting the HR of all-cause deaths of the vaccinated compared to the unvaccinated. Therefore, only a multivariate analysis allows a more reliable estimate of the HRs for the different vaccination status compared to the unvaccinated.

   In our opinion this analysis that shows a strong negative safety signal for the Covid-19 vaccines. There appears to be evidence of not only significantly higher mortality, most likely in those already sick or elderly, but also evidence of a harvesting effect. Likewise, the study has been careful designed to reduce or eliminate the effects of confounding and significant sources of selection bias present in other data sets.

   They also say:

   We must therefore admit that vaccination increases the risk of death from causes other than COVID-19, or by direct damage (adverse effects), or indirect damage, e.g. to the immune system. Therefore, may be that the risk of death is greater for one dose than for two, and for two than for three, because individuals more liable to harm are already dead after the first and second doses.

   They calculated the cumulative life expectancy loss for the Pescara population, for 2-doses was to reduce life expectancy by 3.6 months, and for 3/4 doses it was 1.31 months – a total of approximately four months.

   Between 2019 and 2022 the life expectancy in the province of Pescara fell by one year, from 83.6 to 82.6 years, corresponding to an annual loss of four months. The correspondence between this figure and the life expectancy loss provide by this study is confirmation that the study authors are correct in their claim that this increased mortality is attributable to the Covid-19 vaccines.

   Now, four months might not look like much in absolute terms, but bear in mind that this mortality deficit is calculated over the whole population, and we cannot assume this is a transitory event. Also, the risk presented to vulnerable sub-groups, such as those with multiple comorbidities, will likely be significantly greater than this. We have therefore asked the authors if they could analyse the data to determine risk stratified for age and comorbidity, by testing additional hypotheses for each comorbidity (such as “what is the mortality risk of vaccination to hypertensives versus hypertensives who remain unvaccinated?”

   Finally, bear in mind this study has calculated the short-term impact on mortality: the long-term impacts remain unquantified.

   Accedi per rispondere