On 2016 Aug 18, Daniel Weeks commented:

Relative effect sizes of FTO variants and rs373863828 on body mass index in Samoans

We would like to thank Dr. Janssens for making these helpful comments about the presentation and interpretation of our findings. And we welcome this opportunity to present our results more precisely and clearly.

Regarding the suggestion that we should have compared standardized effects, there exists some literature that argues that comparison of standardized effects can be misleading (Cummings P, 2004, Cummings P, 2011). Indeed, Rothman and Greenland (1998, p. 672) recommend that "effects should be expressed in a substantively meaningful unit that is uniform across studies, not in standard-deviation units." While the argument for comparing standardized effects may be more compelling when different studies used different measurement scales, in this case, body mass index (BMI) has been measured in prior studies and our current one using a common scale.

As recommended, we have now assessed the effect of variants on BMI in the FTO region to allow for direct comparison in our Samoan population. As Table 1 indicates, while the effects of these FTO variants are not statistically significant in our discovery sample, the estimates of the effect size of the FTO variants are similar in magnitude to previous estimates in other populations, and the non-standardized effect of the missense variant rs373863828 in CREBRF is approximately 3.75 to 4.66 times greater than the effects of the FTO variants in our discovery sample.

We concur with the important reminder that the odds ratio overestimates the relative risk when the outcome prevalence is high.

Thank you,

Daniel E. Weeks and Ryan Minster on behalf of all of the co-authors.

Department of Human Genetics, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA, USA.

References:

Cummings P. (2004) Meta-analysis based on standardized effects is unreliable. Arch Pediatr Adolesc Med. 158(6):595-7. PubMed PMID: 15184227.

Cummings P. (2011) Arguments for and against standardized mean differences (effect sizes). Arch Pediatr Adolesc Med. 165(7):592-6. doi: 10.1001/archpediatrics.2011.97. PubMed PMID: 21727271.

Rothman, K.J. and Greenland S. (1998) Modern epidemiology, second edition. Lippincott Williams & Wilkins, Philadelphia.

On 2016 Aug 03, Cecile Janssens commented:

This study showed that a variant in CREBRF is strongly associated with body mass index (BMI) in Samoans. The authors write that this gene variant is associated with BMI with “a much larger effect size than other known common obesity risk variants, including variation in FTO.” The risk variant was also “positively associated with obesity risk” and with other obesity-related traits. For a correct interpretation of these findings, two methodological issues need to be considered.

Much larger effect size

The effect size of the CREBRF variant (1.36-1.45 kg/m2 per copy of the risk allele) is indeed larger than that of FTO (0.39 kg/m2 per copy), but this comparison is not valid to claim that the gene variant has a stronger effect.

The effect size for the FTO gene comes from a pooled analysis of studies in which the average BMI of the population was below 28kg/m2 with standard deviations lower than 4kg/m2. In this study, the mean BMI was 33.5 and 32.7 kg/m2 in the discovery and replication samples and the standard deviations were higher (6.7 and 7.2 kg/m2). To claim that the CREBRF has a stronger effect than FTO, the researchers should have compared standardized effects that take into account the differences in BMI between the study populations, or they should have assessed the effect of FTO to allow for a direct comparison in the Samoan population.

It is surprising that the authors have not considered this direct comparison between the genes, given that an earlier publication had reported about the relationship between FTO and BMI in the replication datasets of this study (Karns R, 2012) That study showed no association between FTO and BMI in the smallest, but a higher effect size in the largest of the two replication samples (0.55-0.70 kg/m2). The effect of the CREBRF gene may still be stronger than that of the FTO gene, but the difference may not be as large as the comparison of unstandardized effect sizes between the populations suggests.

Impact on obesity risk

The authors also investigate the “impact of the gene variant on the risk of obesity” and found that the odds ratio for the gene variant was 1.44 in the replication sample. This value is an odds ratio and indicates the impact on the odds of obesity, not on the risk of obesity. The difference between the two is essential here.

The value of the odds ratio is similar to the relative risk when the outcome of interest is rare. In this study, the majority of the people were obese, 55.5% and 48.8% in the discovery and replication samples had BMI higher than 32kg/m2. When the prevalence of the outcome is this high, the odds ratio overestimates the relative risk. When the odds ratio is 1.44, the relative risk is 1.43 when the prevalence of obesity in noncarriers is 1%, 1.32 when it is 20%, 1.22 when it is 40%, 1.18 when it is 50%, and 1.16 when 55% of the non-carriers is obese. Regarding the impact on obesity risk, the gene variant might be more ordinary as suggested.

On 2016 Aug 03, Cecile Janssens commented:

This study showed that a variant in CREBRF is strongly associated with body mass index (BMI) in Samoans. The authors write that this gene variant is associated with BMI with “a much larger effect size than other known common obesity risk variants, including variation in FTO.” The risk variant was also “positively associated with obesity risk” and with other obesity-related traits. For a correct interpretation of these findings, two methodological issues need to be considered.

Much larger effect size

The effect size of the CREBRF variant (1.36-1.45 kg/m2 per copy of the risk allele) is indeed larger than that of FTO (0.39 kg/m2 per copy), but this comparison is not valid to claim that the gene variant has a stronger effect.

The effect size for the FTO gene comes from a pooled analysis of studies in which the average BMI of the population was below 28kg/m2 with standard deviations lower than 4kg/m2. In this study, the mean BMI was 33.5 and 32.7 kg/m2 in the discovery and replication samples and the standard deviations were higher (6.7 and 7.2 kg/m2). To claim that the CREBRF has a stronger effect than FTO, the researchers should have compared standardized effects that take into account the differences in BMI between the study populations, or they should have assessed the effect of FTO to allow for a direct comparison in the Samoan population.

It is surprising that the authors have not considered this direct comparison between the genes, given that an earlier publication had reported about the relationship between FTO and BMI in the replication datasets of this study (Karns R, 2012) That study showed no association between FTO and BMI in the smallest, but a higher effect size in the largest of the two replication samples (0.55-0.70 kg/m2). The effect of the CREBRF gene may still be stronger than that of the FTO gene, but the difference may not be as large as the comparison of unstandardized effect sizes between the populations suggests.

Impact on obesity risk

The authors also investigate the “impact of the gene variant on the risk of obesity” and found that the odds ratio for the gene variant was 1.44 in the replication sample. This value is an odds ratio and indicates the impact on the odds of obesity, not on the risk of obesity. The difference between the two is essential here.

The value of the odds ratio is similar to the relative risk when the outcome of interest is rare. In this study, the majority of the people were obese, 55.5% and 48.8% in the discovery and replication samples had BMI higher than 32kg/m2. When the prevalence of the outcome is this high, the odds ratio overestimates the relative risk. When the odds ratio is 1.44, the relative risk is 1.43 when the prevalence of obesity in noncarriers is 1%, 1.32 when it is 20%, 1.22 when it is 40%, 1.18 when it is 50%, and 1.16 when 55% of the non-carriers is obese. Regarding the impact on obesity risk, the gene variant might be more ordinary as suggested.