On 2017 Jun 02, Eric Robinson commented:

The method and analysis adopted in this research are flawed and the conclusions are incorrect.

Action before commenting on pubmed: I contacted the journal and the authors in question with my concerns. The journal ignored a number of my emails and after an internal review decided not to issue a correction or retraction. I requested the results of the internal review, but the journal has not responded. The first author appears to have been a Masters student at the senior author’s institute and I was unable to contact the first author. The senior author (BM Corfe) informed me on the telephone that he was not prepared to discuss the research and instead advised me to raise any points in a public domain. The senior author also advised me that he would not be considering a correction or retraction of the work and that the research team stood by all of the conclusions made. Given the journal’s position on the study and my frustration with their handling of this, I was not prepared to support the journal by publishing a letter to the editor.

Research question: The researchers wanted to examine whether there is a link between self-reported appetite (self-reported subjective feelings of hunger) and energy intake; do participants who report feeling hungry eat more than participants who report feeling less hungry? They conducted what they described as a ‘systematic’ review and examined over 400 articles.

Conclusions drawn: The authors conclude that self-reported appetite (e.g. subjective feelings of hunger) ‘does not predict energy intake’ (title of article) and an associated University press release stated that ‘there is no link between how hungry we feel and the amount of calories we consume’.

Is this a spoof or hoax article? At first I thought this article may be a hoax, because the conclusion that self-reported hunger is in no way predictive of how much a person eats, is odd. Previous research shows that self-reported hunger/appetite does predict how much a person eats, but as you might expect, the correlation between self-reported hunger and energy intake is not perfect. Recently, Sadoul et al. (1) show this to be the case in an analysis of 23 studies that assessed self-reported appetite and ad-libitum meal energy intake. Robinson et al. (2) show this to the case in an analysis of 31 studies that assessed self-reported hunger and ad-libitum intake of snack foods.

Flawed method: There are a number of texts on best practice for conducting systematic reviews and synthesising data from multiple studies. Rather than using standard meta-analytic methods (e.g. combining weighted correlation coefficients between self-reported hunger and energy intake from studies), the researchers scored each study in their review as either providing evidence of a ‘link’ or evidence of ‘no link’ between self-reported hunger and energy intake. The scoring system used was inappropriate in a number of ways. For example, if an experimental manipulation in a study led to a change in energy intake without a change in self-reported hunger, in the present review this constituted evidence that self-reported hunger does not predict energy intake. This line of reasoning is a logical fallacy because it is based on the premise that a) energy intake can only be affected by self-reported hunger and b) energy intake being affected by anything other than subjective feelings of hunger proves that subjective appetite is in no way related to energy intake. Energy intake can be increased or decreased by a multitude of factors and many of these will not act on energy intake by altering self-reported hunger.

Flawed analysis: The authors’ main analysis was dependent on counting studies that provided statistically significant findings vs. those that did not, which is not considered best practice as it ignores considerations of sample size, statistical power and how heavily each study should be weighted in analyses. The above points aside, the authors went on to report that approximately 49% of studies they surveyed found a ‘link’ and 51% found ‘no link’ between subjective self-reported hunger and energy intake. This is actually highly convincing evidence that there is a link between self-reported hunger and energy intake, because if there was ‘no link’ we would expect to see closer to only 5% of all studies finding a ‘link’ (typical alpha level of .05), as opposed to the 49% reported.

Invalid conclusions: A combination of flawed methods and analyses results in incorrect conclusions.

A sobering experience: I noticed this study because of the bizarre conclusion it made; hunger in no way relates to how much we eat. I reached out to the authors several times over email. In the end I had to ring the senior author’s office phone to speak to him, but as noted the senior author was not prepared to discuss his research or revise his position on this research. This to me is direct experiential evidence that some scientists do not appear to care about the quality and accuracy of research they conduct and publish.

(1) Sadoul BC, Schuring EA, Mela DJ, Peters HP. The relationship between appetite scores and subsequent energy intake: an analysis based on 23 randomized controlled studies. Appetite 2014; 83: 153-159

(2) Robinson E, Haynes A, Hardman CA, Kemps E, Higgs S, Jones A. The bogus taste test: Validity as a measure of laboratory food intake. Appetite 2017; 116: 223-231.

On 2017 Jun 02, Eric Robinson commented:

The method and analysis adopted in this research are flawed and the conclusions are incorrect.

Action before commenting on pubmed: I contacted the journal and the authors in question with my concerns. The journal ignored a number of my emails and after an internal review decided not to issue a correction or retraction. I requested the results of the internal review, but the journal has not responded. The first author appears to have been a Masters student at the senior author’s institute and I was unable to contact the first author. The senior author (BM Corfe) informed me on the telephone that he was not prepared to discuss the research and instead advised me to raise any points in a public domain. The senior author also advised me that he would not be considering a correction or retraction of the work and that the research team stood by all of the conclusions made. Given the journal’s position on the study and my frustration with their handling of this, I was not prepared to support the journal by publishing a letter to the editor.

Research question: The researchers wanted to examine whether there is a link between self-reported appetite (self-reported subjective feelings of hunger) and energy intake; do participants who report feeling hungry eat more than participants who report feeling less hungry? They conducted what they described as a ‘systematic’ review and examined over 400 articles.

Conclusions drawn: The authors conclude that self-reported appetite (e.g. subjective feelings of hunger) ‘does not predict energy intake’ (title of article) and an associated University press release stated that ‘there is no link between how hungry we feel and the amount of calories we consume’.

Is this a spoof or hoax article? At first I thought this article may be a hoax, because the conclusion that self-reported hunger is in no way predictive of how much a person eats, is odd. Previous research shows that self-reported hunger/appetite does predict how much a person eats, but as you might expect, the correlation between self-reported hunger and energy intake is not perfect. Recently, Sadoul et al. (1) show this to be the case in an analysis of 23 studies that assessed self-reported appetite and ad-libitum meal energy intake. Robinson et al. (2) show this to the case in an analysis of 31 studies that assessed self-reported hunger and ad-libitum intake of snack foods.

Flawed method: There are a number of texts on best practice for conducting systematic reviews and synthesising data from multiple studies. Rather than using standard meta-analytic methods (e.g. combining weighted correlation coefficients between self-reported hunger and energy intake from studies), the researchers scored each study in their review as either providing evidence of a ‘link’ or evidence of ‘no link’ between self-reported hunger and energy intake. The scoring system used was inappropriate in a number of ways. For example, if an experimental manipulation in a study led to a change in energy intake without a change in self-reported hunger, in the present review this constituted evidence that self-reported hunger does not predict energy intake. This line of reasoning is a logical fallacy because it is based on the premise that a) energy intake can only be affected by self-reported hunger and b) energy intake being affected by anything other than subjective feelings of hunger proves that subjective appetite is in no way related to energy intake. Energy intake can be increased or decreased by a multitude of factors and many of these will not act on energy intake by altering self-reported hunger.

Flawed analysis: The authors’ main analysis was dependent on counting studies that provided statistically significant findings vs. those that did not, which is not considered best practice as it ignores considerations of sample size, statistical power and how heavily each study should be weighted in analyses. The above points aside, the authors went on to report that approximately 49% of studies they surveyed found a ‘link’ and 51% found ‘no link’ between subjective self-reported hunger and energy intake. This is actually highly convincing evidence that there is a link between self-reported hunger and energy intake, because if there was ‘no link’ we would expect to see closer to only 5% of all studies finding a ‘link’ (typical alpha level of .05), as opposed to the 49% reported.

Invalid conclusions: A combination of flawed methods and analyses results in incorrect conclusions.

A sobering experience: I noticed this study because of the bizarre conclusion it made; hunger in no way relates to how much we eat. I reached out to the authors several times over email. In the end I had to ring the senior author’s office phone to speak to him, but as noted the senior author was not prepared to discuss his research or revise his position on this research. This to me is direct experiential evidence that some scientists do not appear to care about the quality and accuracy of research they conduct and publish.

(1) Sadoul BC, Schuring EA, Mela DJ, Peters HP. The relationship between appetite scores and subsequent energy intake: an analysis based on 23 randomized controlled studies. Appetite 2014; 83: 153-159

(2) Robinson E, Haynes A, Hardman CA, Kemps E, Higgs S, Jones A. The bogus taste test: Validity as a measure of laboratory food intake. Appetite 2017; 116: 223-231.