On 2015 Feb 26, Harri Hemila commented:

Missing contradictory findings and other problems in Curtis AJ, 2014

Curtis AJ, 2014 state in their abstract that “The evidence from pooled analysis of 18 randomised controlled trials undertaken in apparently healthy people shows no effect of vitamin E supplementation at a dose of 23-800 IU/day on all-cause mortality.” This conclusion was based on the pooling of results from 18 RCTs that obtained RR = 1.01 (95% CI: 0.97 to 1.05).

In their calculation of the average effect of vitamin E from the 18 studies, Curtis (2014) assume that there exists an overall uniform effect for all of those 18 RCTs, which can be estimated by pooling their findings. However, a subgroup analysis of the ATBC Study (1994) refutes the notion that the effect of vitamin E is uniform for all people. The combination of age and dietary vitamin C in the subgroup analysis modified the effect of vitamin E, which provides very strong evidence of heterogeneity over 6 subgroups (P = 0.0005) Hemilä H, 2009. Such heterogeneity refutes the assumption of a uniform effect in pooling the findings of the 18 RCTs by Curtis (2014).

Curtis AJ, 2014 also state “Subgroup analyses by ... duration of exposure ... showed no association with all-cause mortality.” However, the effect of time should not be analyzed by comparing RCTs at the study level, some of the RCTs being long in duration while the others being short in duration. In the ATBC Study, harm from vitamin E in the young participants was seen after 3.3 years of supplementation. Longer vitamin E supplementation increased mortality by 38% (17% to 63%), whereas vitamin E had no effect on mortality over the earlier period, see Hemilä H, 2009. This finding of a time-dependent effect modification refutes the claim that the duration of vitamin E supplementation does not modify the effects of the vitamin with regard to all-cause mortality. Comparing RCTs on vitamin E by the mean durations of the RCTs, ie study-level analysis, cannot capture changes in the vitamin E effect after lag periods. Instead the analysis of time-dependent effect modification requires individual-level analysis.

Ecological fallacy occurs when mean data about a group is used to impute identical values for all individuals of the group. Thus, the assumption by Curtis AJ, 2014 that the calculated average effect of vitamin E on mortality for the 18 RCTs (ie RR = 1.01) is valid for all participants in these 18 studies, or for other individuals in the community, is an example of ecological fallacy. The conclusion by Curtis AJ, 2014 that “duration of exposure ... showed no association with all-cause mortality” is another example of ecological fallacy.

We do not know how far the heterogeneity of the ATBC Study can be generalized. The participants in that study were middle-aged males, who were aged 50-69 years at the start of the trial in the 1980s, which indicates that they were born before WW-II. In addition, all were smokers and lived in Finland. We do not know if similar effect modification occurs in other populations. Nevertheless, the heterogeneity found in this particular cohort refutes the notion that there might be a universal vitamin E effect on mortality that would be valid for all groups of people. Highly significant heterogeneity was also found in the effect of vitamin E on pneumonia incidence in Hemilä H, 2011 and on the incidence of the common cold in Hemilä H, 2006. Thus the evidence of heterogeneity of the vitamin E effect is not restricted to overall mortality but exists to other outcomes as well.

Furthermore, in the Table 1 of Curtis AJ, 2014 it is stated that the “study quality” of the ATBC Study (1994) was “medium”. As a justification for this classification, in Appendix 2 (“quality of included studies”) Curtis AJ, 2014 inserted question marks next to “blinding of personnel” and “blinding of participants” and a cross (to indicate high risk of bias) to “blinding of outcome assessment”.

The ATBC Study (1994), Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group., 1994, reported that “Participants and all study staff involved in the ascertainment of end points and the assignment of final diagnoses remained blinded to the participants' treatment assignments throughout the trial” (p. 1030). The same report also stated that “Deaths (n=3570) were identified from the National Death Registry, a branch of Statistics Finland” (p. 1030), which clearly indicates that the deaths in the ATBC study were recorded by an organization that had nothing to do with the administration of the tablets. Thus, the claims by Curtis (2014) about the blinding of participants and personnel in their Appendix 2 are inconsistent with the ATBC Study (1994) report.

Transparency in systematic reviews is important, so that the reader is correctly informed what the basis for the quality assessments is. Curtis AJ, 2014 do not give any explanations about why they dismiss the descriptions about blinding in the ATBC Study (1994) report. Therefore the readers are misled about the quality of the study that has the greatest weight in Fig. 2 of Curtis (2014).

Finally, the ATBC Study (1994) is wrongly cited in Curtis AJ, 2014. They write Virtamo (1998) in Fig. 2 ,Virtamo (2006) in Appendix 2, and Virtamo (2003) in the reference section. All of these three years are wrong. The ATBC Study was actually published in 1994, with the PubMed record Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group., 1994 (the correct year and reference). Virtamo (2003), which is listed in the reference section, does exist but it is a post-intervention follow-up report of the trial and not the correct reference for the main results of the ATBC Study (1994).

On 2015 Feb 26, Harri Hemila commented:

Missing contradictory findings and other problems in Curtis AJ, 2014

Curtis AJ, 2014 state in their abstract that “The evidence from pooled analysis of 18 randomised controlled trials undertaken in apparently healthy people shows no effect of vitamin E supplementation at a dose of 23-800 IU/day on all-cause mortality.” This conclusion was based on the pooling of results from 18 RCTs that obtained RR = 1.01 (95% CI: 0.97 to 1.05).

In their calculation of the average effect of vitamin E from the 18 studies, Curtis (2014) assume that there exists an overall uniform effect for all of those 18 RCTs, which can be estimated by pooling their findings. However, a subgroup analysis of the ATBC Study (1994) refutes the notion that the effect of vitamin E is uniform for all people. The combination of age and dietary vitamin C in the subgroup analysis modified the effect of vitamin E, which provides very strong evidence of heterogeneity over 6 subgroups (P = 0.0005) Hemilä H, 2009. Such heterogeneity refutes the assumption of a uniform effect in pooling the findings of the 18 RCTs by Curtis (2014).

Curtis AJ, 2014 also state “Subgroup analyses by ... duration of exposure ... showed no association with all-cause mortality.” However, the effect of time should not be analyzed by comparing RCTs at the study level, some of the RCTs being long in duration while the others being short in duration. In the ATBC Study, harm from vitamin E in the young participants was seen after 3.3 years of supplementation. Longer vitamin E supplementation increased mortality by 38% (17% to 63%), whereas vitamin E had no effect on mortality over the earlier period, see Hemilä H, 2009. This finding of a time-dependent effect modification refutes the claim that the duration of vitamin E supplementation does not modify the effects of the vitamin with regard to all-cause mortality. Comparing RCTs on vitamin E by the mean durations of the RCTs, ie study-level analysis, cannot capture changes in the vitamin E effect after lag periods. Instead the analysis of time-dependent effect modification requires individual-level analysis.

Ecological fallacy occurs when mean data about a group is used to impute identical values for all individuals of the group. Thus, the assumption by Curtis AJ, 2014 that the calculated average effect of vitamin E on mortality for the 18 RCTs (ie RR = 1.01) is valid for all participants in these 18 studies, or for other individuals in the community, is an example of ecological fallacy. The conclusion by Curtis AJ, 2014 that “duration of exposure ... showed no association with all-cause mortality” is another example of ecological fallacy.

We do not know how far the heterogeneity of the ATBC Study can be generalized. The participants in that study were middle-aged males, who were aged 50-69 years at the start of the trial in the 1980s, which indicates that they were born before WW-II. In addition, all were smokers and lived in Finland. We do not know if similar effect modification occurs in other populations. Nevertheless, the heterogeneity found in this particular cohort refutes the notion that there might be a universal vitamin E effect on mortality that would be valid for all groups of people. Highly significant heterogeneity was also found in the effect of vitamin E on pneumonia incidence in Hemilä H, 2011 and on the incidence of the common cold in Hemilä H, 2006. Thus the evidence of heterogeneity of the vitamin E effect is not restricted to overall mortality but exists to other outcomes as well.

Furthermore, in the Table 1 of Curtis AJ, 2014 it is stated that the “study quality” of the ATBC Study (1994) was “medium”. As a justification for this classification, in Appendix 2 (“quality of included studies”) Curtis AJ, 2014 inserted question marks next to “blinding of personnel” and “blinding of participants” and a cross (to indicate high risk of bias) to “blinding of outcome assessment”.

The ATBC Study (1994), Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group., 1994, reported that “Participants and all study staff involved in the ascertainment of end points and the assignment of final diagnoses remained blinded to the participants' treatment assignments throughout the trial” (p. 1030). The same report also stated that “Deaths (n=3570) were identified from the National Death Registry, a branch of Statistics Finland” (p. 1030), which clearly indicates that the deaths in the ATBC study were recorded by an organization that had nothing to do with the administration of the tablets. Thus, the claims by Curtis (2014) about the blinding of participants and personnel in their Appendix 2 are inconsistent with the ATBC Study (1994) report.

Transparency in systematic reviews is important, so that the reader is correctly informed what the basis for the quality assessments is. Curtis AJ, 2014 do not give any explanations about why they dismiss the descriptions about blinding in the ATBC Study (1994) report. Therefore the readers are misled about the quality of the study that has the greatest weight in Fig. 2 of Curtis (2014).

Finally, the ATBC Study (1994) is wrongly cited in Curtis AJ, 2014. They write Virtamo (1998) in Fig. 2 ,Virtamo (2006) in Appendix 2, and Virtamo (2003) in the reference section. All of these three years are wrong. The ATBC Study was actually published in 1994, with the PubMed record Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group., 1994 (the correct year and reference). Virtamo (2003), which is listed in the reference section, does exist but it is a post-intervention follow-up report of the trial and not the correct reference for the main results of the ATBC Study (1994).