On 2017 Apr 27, Randi Pechacek commented:

Trent Northen, one of the authors of this paper, wrote an interesting blog post on microBEnet giving some background behind this research.

On 2015 Oct 03, Robert Groom commented:

The author raises an important question about use of candidate demographic data and exam results to identify opportunities to improve the education of perfusionists. Clearly there is a moral obligation to use information to inform and refine perfusion education.

On 2015 Sep 28, Bill Cayley commented:

A good piece of research suggesting that sometimes "less" is "more": https://lessismoreebm.wordpress.com/2015/09/28/comparative-effectiveness-of-generic-versus-brand-name-antiepileptic-medications/

On 2015 Sep 26, Lydia Maniatis commented:

None

On 2015 Sep 25, James Hadfield commented:

This appears to be a carbon copy of http://www.ncbi.nlm.nih.gov/pubmed/25278450 and may need investigating

On 2016 Jun 12, John Duffy commented:

Kari Poikolainen and I sent a letter to the Lancet about this paper. It was not published. The content is as follows

The authors report no association between level of alcohol intake and a composite health measure. They write that "In summary, our study shows that current drinking is not associated with a net health benefit." What they fail to mention is that their study also "shows" that current drinking is not associated with net health harm. The interpretation, "sufficient commonalities to support global health strategies and national initiatives to reduce harmful alcohol use" does not follow from their study, which taken on its own suggests that there is no need for such initiatives. However there are a number of methodological problems with their data and analysis. The follow-up time (median 4.3 yrs) was short and the proportion of alcohol consumers (33%) small. There is a strong implication of overfitting, and in the stratified analyses in particular a shortage of degrees of freedom. Since a large body of earlier well-conducted studies shows evidence of both benefits and harms it would be inappropriate to base conclusions on this work.

On 2015 Sep 26, David Keller commented:

Dose-response relationship observed between concentration of ingested alcohol and cancer rate

This study reported that "the risk for cancer was 38% higher in wine drinkers than in never drinkers (HR, 1.38; 95% CI, 1.05 - 1.81), 69% higher in spirit drinkers (HR, 1.69; 95% CI, 1.26 - 2.26), and 20% higher in beer drinkers (HR, 1.20; 95% CI, 0.91 - 1.57)." These 3 data points demonstrate a dose-response curve relating the concentration of ingested alcohol with increased risk of cancer. For cardiovascular (CV) disease, it is the amount of alcohol consumed per day, not its concentration, which is known to be the relevant modifiable risk factor. The maximum CV benefits are associated with consumption of around 2 standard servings per day of alcohol of any concentration.

Note that the 95 percent confidence interval for the Harm Ratio due to drinking beer spanned 1.0, and thus beer did not raise the risk for cancer to a statistically significant degree.

5 percent Beer.....HR = 1.2 non-significant

12 percent Wine.....HR = 1.38

40 percent Spirits..HR = 1.69

Dose-Response Between Alcohol Concentration and Harm Ratio for Cancers

Thus, drinking beer, which, in moderation, is associated with significant cardiovascular (CV) benefits, is not associated with any statistically significant increase in cancers, as are wine and spirits, presumably due to beer's lower alcohol concentration.

This study did not report separate findings for high-volume and low-volume beer drinkers for its risk-benefit analyses. If beer drinkers who limited themselves to 2 servings per day had been analyzed separately, might the harm ratio for incident cancers have been even lower? Might a reduction in cancer incidence perhaps have been observed? At what volume of daily alcohol ingestion in the form of beer (the least carcinogenic alcoholic beverage) was the greatest improvement seen in the composite outcome of cancer and CV disease? Does the optimum amount of beer per day differ for the composite outcome versus the optimal 2 servings per day associated purely with optimal CV outcomes?

The main purpose of studies such as this should be to result in actionable information. Telling us that low economic level countries exhibit different outcomes associated with the consumption of alcohol does little good unless we are given clues as to what alcohol consumers in poor countries might be doing differently than their counterparts in richer nations. Are they drinking higher concentrations of alcohol, greater volumes, both, or is their nutritional status or some other factor implicated?

On 2016 Mar 10, Lydia Maniatis commented:

I'm interleaving my comments to Hoffman's last reply below:

Hoffman: The interface theory predicts that realism is false, and thus that Bell correlations exist. Were experiments to demonstrate that Bell correlations do not exist, then the interface theory would be falsified. Thus the interface theory is an empirical theory, and its prediction is experimentally testable.

LMM: There's a difference between constructing a theory and simply making a loosely argued assertion. Your “theory” doesn't have the logical consistency, respect for facts and specificity to actually predict Bell's correlations. Many people have said the real world doesn't exist, but that doesn't make them physicists, it doesn't make sense to say that they had in effect predicted Bell's correlations. The variables you use are much too vague to justify predictions at any level of specificity, let alone that required at the level of quantum physics.

Hoffman: The remaining experimental loopholes in tests of Bell correlations that Howard Wiseman cites in his paper “Physics: Bell’s theorem still reverberates” are the following: “… they lacked large separations and fast switching of the settings, opening the ‘separation loophole’: information about the detector setting for one photon could have propagated, at light speed, to the other detector, and affected its outcome.” These loopholes are now closed. Bell correlation is confirmed. See Hensen et al., 2015, “Loophole-free Bell inequality violation using electron spins separated by 1.3 kilometres”, Nature, 526, 682-686.

LMM: And there's an even more recent one. However, as I reread Wiseman's article, it is those who (unlike you) don't agree with the Bell's notion of hidden variables (b/c they are empirically inaccessible) need to forgo realism. They (according to Wiseman) must either conclude that some events are correlated for no reason or accept Bell's metaphysical hidden variables. So it's not even clear that Bell's supporters are not realists.

At this point, it seems that physicists are in a very difficult place, caught between two paradoxical options. This is the point at which ground-breaking new theories sometimes emerge. I don't think vision science has a lot to say here.

The fact remains that there are principles of visual perception (and of mechanics, to take an example from physics) that are close enough to the truth in our environment to make useful predictions, lead to useful applications, and to lead to ever more useful theories. That's the world that science lives in. You say this world doesn't exist, yet your story is full of assumptions about that world (natural selection, etc). All I'm saying is, you can't have it both ways.

On 2016 Mar 05, Donald D Hoffman commented:

The interface theory predicts that realism is false, and thus that Bell correlations exist. Were experiments to demonstrate that Bell correlations do not exist, then the interface theory would be falsified. Thus the interface theory is an empirical theory, and its prediction is experimentally testable.

The remaining experimental loopholes in tests of Bell correlations that Howard Wiseman cites in his paper “Physics: Bell’s theorem still reverberates” are the following: “… they lacked large separations and fast switching of the settings, opening the ‘separation loophole’: information about the detector setting for one photon could have propagated, at light speed, to the other detector, and affected its outcome.”

These loopholes are now closed. Bell correlation is confirmed. See Hensen et al., 2015, “Loophole-free Bell inequality violation using electron spins separated by 1.3 kilometres”, Nature, 526, 682-686.

When Wiseman states that “Most physicists are localists” that means that most physicists give up realism. You simply misunderstood him.

On 2016 Feb 17, Lydia Maniatis commented:

This is the status of Bell's theorem in the physics community:

Physics: Bell’s theorem still reverberates Howard Wiseman 19 June 2014

"...But 50 years on, the experimental verifications of these quantum correlations still have ‘loopholes’, and scientists and philosophers still dispute exactly what the theorem states.

...As of now, there are no loophole-free Bell experiments.

...Most physicists are localists: they recognize the two options but choose the first, because hidden variables are, by definition, empirically inaccessible [i.e. outside the realm of empirical science]. Quantum information scientists embrace irreducible randomness as a resource for secure cryptography3. Other physicists and philosophers (the ‘non-localist camp’) dispute that there are two options, and insist that Bell’s theorem mandates non-locality4."

Apparently perception scientists are supposed to side with the minority opinion in physics on an issue of extraordinary complexity, and in turn to figure out what this implies for their discipline. Meanwhile, physics goes on as usual.

Btw, the online descriptions of Bell's theorem often include the term "metaphysics." Again, we are not dealing with empirical science, i.e. testable claims.

The only apparent relevance of the metaphysical claims being put forth as the "interface theory" is that the fundamental assumptions that enable empirical science are false. Yet persistent and irresolvable metaphysical problems with "reality" haven't stopped scientific progress before.

With respect to "H1 and H2," the framing of the questions is paradoxical. What is "it" and what is doing the perceiving? How does the perceiver come into existence? What if no one perceives the perceiver? What if there were no perceivers?

In general, science can never prove that the world exists beyond perception, but it acts as though it could, and that seems to lead to some practical benefits.

On 2015 Dec 10, Donald D Hoffman commented:

Consider two hypotheses.

H1: When it is not perceived, an object has a definite value for each of its dynamical physical properties.

H2: When it is not perceived, an object does not have a definite value for each of its dynamical physical properties.

H1 is widely accepted by vision researchers. Is H1 falsifiable?

If yes: There is an experiment that can support H1 or its negation, viz., H2. Thus H1 is falsifiable iff H2 is falsifiable.

If no: H1 and H2 are both outside empirical science.

In fact H1 and H2 are both falsifiable, and within empirical science. Understanding this nontrivial fact was an intellectual achievement of John Stewart Bell, published in 1964.

Below are published papers presenting empirical tests of the falsifiable hypothesis H2. The results in each case support H2, and thus support the interface theory of perception. A gentle introduction to this literature, and to the remarkable theorem on which it relies, is the paper by Mermin. These experiments, and the theory behind them, are among the greatest achievements of modern science.

Ansmann, M., Wang, H., Bialczak, R. C., Hofheinz, M., Lucero, E., Neeley, M., . . . Martinis, J. M. (2009). Violation of Bell’s inequality in Josephson phase qubits. Nature, 461, 504–506. doi:10.1038/ nature08363

Bell, J. S. (1964). On the Einstein Podolsky Rosen Paradox. Physics 1 (3): 195–200.

Cabello, A., Estebaranz, J. M., & García-Alcaine, G. (1996). Bell– Kochen–Specker theorem: A proof with 18 vectors. Physics Letters A, 212, 183. doi:10.1016/0375-9601(96)00134-X

Giustina, M., Mech, A., Ramelow, S., Wittmann, B., Kofler, J., Beyer, J., . . . Zeilinger, A. (2013). Bell violation using entangled photons with- out the fair-sampling assumption. Nature, 497, 227–230. doi:10.1038/nature12012

Mermin, N. D. (1985). Is the moon there when nobody looks? Reality and the quantum theory. Physics Today, 38(4), 38–47. doi:10.1063/1. 880968

Pan, J.-W., Bouwmeester, D., Daniell, M., Weinfurter, H., & Zeilinger, A. (2000). Experimental test of quantum nonlocality in three-photon GHZ entanglement. Nature, 403, 515–519.

Rowe, M. A., Kielpinski, D., Meyer, V., Sackett, C. A., Itano, W. M., Monroe, C., & Wineland, D. J. (2001). Experimental violation of a Bell’s inequality with efficient detection. Nature, 409, 791–794.

Salart, D., Baas, A., van Houwelingen, J. A. W., Gisin, N., & Zbinden, H. (2008). Spacelike separation in a Bell test assuming gravitationally induced collapses. Physical Review Letters, 100, 220404.

Weihs, G., Jennewein, T., Simon, C., Weinfurter, H., & Zeilinger, A. (1998). Violation of Bell’s inequality under strict Einstein locality conditions. Physical Review Letters, 81, 5039.

On 2015 Dec 02, Lydia Maniatis commented:

Very brief summary: The "falsifiable prediction" that “No object has a position, or any other physical property, when it is not perceived” cannot be tested except by perceiving some objective property of a part of the world (e.g. the absence of an object)...while we are not perceiving it.

It's like turning around really quickly to check if the world behind you disappears when you're not looking at it. Whimsical, but not very scientific.

On 2015 Nov 24, Lydia Maniatis commented:

The authors' proposal NOT FALSIFIABLE (i.e. it is not testable, i.e. it does not belong to empirical science), for a number of reasons.

In addressing the falsifiability question explicitly, Hoffman et al say that “if either [of two predictions – see below] is false, then the interface theory is false” (p. 1501).

The “predictions” being referred to are the following: 1) “No object has a position, or any other physical property, when it is not perceived.” 2) “No object has any causal powers.

The problem should already be obvious. The “interface theory” says that we cannot perceive the state of the objective world, no way, no how – that even space-time is not a property of the physical world. Therefore, on what basis can we ever corroborate any claims regarding position/physical properties of objects? The same goes for “prediction” number 2. We would have to be able to say "Look, this here object HAS a position and physical properties x, y, z EVEN WHEN NOT PERCEIVED! I can't perceive it or any other object, but I can somehow access this information!" With no access to the physical world, we can provide no proof possible about claims regarding the physical world. The assumptions of the “theory,” in other words mean that PROVING IT WOULD BE TANTAMOUNT TO DISPROVING IT.

Another reason that the theory is unfalsifiable is that its computations are based on statements such as: W = a set of world states. How can we ever substitute any kind of numerical value here? I suggest that the “precision” the authors claim for their proposal collapses with this impossibility. (Perhaps as a non-mathematician I'm missing something...).

Also, as an analogy, the interface theory fails to make the case (and the analogy is pretty much the whole case). The claim is that, just as the e.g. square, blue file icon on the desktop doesn't correspond in its properties to the underlying electronics, so percepts don't correspond in any way to physical objects. But our perception of the blue square presumably corresponds to the spectral properties of the light, and the luminance relationships, of the surface of the computer. Hoffman et al can claim that our perception of the properties of the icon are not veridical, but not using the lack of correspondence between icon and electronics as evidence. No one claims that because we can't accurately perceive the shape of a tree because we can't also see the interior, back, atoms that compose it...

Basing broad, unfalsifiable statements such as those being made in this article on a particular, crude similation is not credible. The “non-linearity” assumptions are said to be frequent but not universal, and even the examples given don't hold up. We are told that “not enough water and one dies of thirst; too much and one drowns; somewhere in between is just right” (p. 1506). This is no argument against the accurate (linear) perception of quantity – seeing a lot of water doesn't mean we actually have to drink it all. We might want to decide to swim in it.

There's just no reason to claim that the refined optical structures and responses of organisms have no purpose in using reflected light to infer and represent the world veridically. It's absurd, and leads to absurdity and, again, unfalsifiable claims. There is absolutely no special relationship between this “theory” and the theory of evolution by natural selection; the implication that a fitness criterion is opposite to a veridical (as veridical as possible and necessary) outcome is unfounded (and, again, unfalsifiable). In fact, the normal assumptions about structure and function (based on perceived structures) are more consistent with evolutionary discussions (they are actually required to make such discussions rational).

The claim, in his comment on the article, by the Editor-in-Chief of this journal that “Our measurements...are species-specific and we should worry if they form the starting point for physical theories” (Hickok (2015, p. 1478) just recycles an old philosophical conundrum that is irrelevant to scientific progress. The “worry” cannot be resolved by science but also cannot, in principle and in practice, affect, let alone “be the future of the science of the mind” (p. 1479) - or of any science (otherwise physicists would also have a problem). It just takes us round in circles for the millionth time. The focus on quantum physics is silly, implying as it does that progress in perception depends on an understanding of the unsolved mysteries of that discipline.

On 2015 Nov 11, Lydia Maniatis commented:

Feldman seems unaware of a fundamental principle of perception. His error (can you spot it?) is contained in the following statements:

“What we can say, often, is whether estimates from distinct perceptual systems “agree.” When I reach out and grasp the banana, I may find that it feels the same size as it looked. This is extremely useful, because it means that the perceptual system is coherent: estimates drawn from different sorts of evidence agree.”

In fact, there is a fundamental asymmetry between our visual sense and our body senses. The body senses defer to vision – what vision says, goes. With respect to haptics, this is described as “visual capture.” In the relevant experiment, a subject observes a straight rod through a distorting lens that makes it appear curved. It feels curved.

In the tilted room, our eyes tell us we're tilted, even though our body senses tell us (or should tell us) that we're upright. We feel tilted.

In the rubber hand illusion, our eyes tell us that the rubber hand is being tickled in the same rhythm as we are experiencing tickling on our real hand. The experience is percieved as originating in the hand that we see, not the hand that is actually doing the feeling.

Thus, when this false argument is used to promote the “Bayesian inference” brand, it is moot:

“Not incidentally, this is the central principle underlying Bayesian inference, which Dennis Lindley (2006) paraphrased as (his emphasis) “BE COHERENT.”

In the case of perception, the “central principle' seems to be resting on a false assumption.

Coherence is also a central principle of scientific argument. It is impossible, in this brief report, for a reader to evaluate Feldman's remaining claims for “B'ian inference.” For a critical analysis of very sloppy arguments, please see my PubPeer comment on Feldman's (2015) “Bayesian models of perceptual organization.”

On 2015 Nov 17, Lydia Maniatis commented:

Anderson concludes his critical commentary on Hoffman, Singh and Prakash's (2015) "Interface theory of perception" with the following remark:

"But remarkable theories require remarkable evidence, and there is currently insufficient evidence to take the metaphysical leap proposed by interface theory."

In fact, as I observe in a recent editorial (http://pec.sagepub.com/content/44/10/1149.full) critical of the "anti-realism" of Hoffman and others, the position is not amenable to evidence; any appeal to evidence in favour of the anti-realist position is paradoxical.

If you say that the features of the perceived world bear no correlation, no connection to the real world, then you cannot EVER produce evidence in support of this position. You cannot prove such a position, because you can't compare the "real world" with your (supposed) wholly inaccessible-to-perception-world. As it is impossible, Hoffman cannot be not doing it, even when he is claiming that he is. He is free to believe anything he wants, but the issue is outside of empirical science.

(I will follow this up with a critique of Hoffman, Singh and Prakash (2105) but wanted to make this quick point.)

On 2015 Oct 29, Dmitri Graifer commented:

I have read with great interest this excellent paper. Regretfully, I found that the authors cite earlier studies in somewhat careless way. First, when mentioning biochemical data on eRF1 motifs involved in stop codon recognition, they ignore data obtained by site-directed cross-linking, which for the first time directly demonstrated that one of the mentioned motifs, GTS, is actually located close to the purines of the A site bound stop codon in the mammalian ribosomal termination complexes (Bulygin et al., 2010, RNA). In this regard, the authors cite only the later paper by Wong et al. (Nucleic Acids Res., 2012) as ref. 25, where a suggestion on implication of this motif in stop codon recognition has been made based on the application of a model ribosome-free system but this paper does not contain any original evidence for the proximity of GTS motif to the stop codon bound at the ribosomal A site. Besides, I have found an “empty” reference 52 (Blanchet et al., Nucleic Acids Res., 2015), which does not contain information on contacts of Arg residues in eRF1 positions 65 and 68 with rRNA as one could expect from the phrase “…and the following Arg65 and Arg68 interact with ribosomal RNA (52)”. In particular, in the paper by Blanchet et al., 2015 (ref. 52) Arg68 is not mentioned at all, and interaction of Arg65 with the rRNA is only hypothesized in the middle part of the penultimate paragraph of the Discussion but it is not shown. I guess that inappropriate referencing corrupts really existing priority in the field and gives distorted ideas on the actually obtained scientific results.

On 2015 Nov 13, Lydia Maniatis commented:

Brainard and colleagues have apparently been using the practical-definition-free term "naturalistic" for at least 18 years, as indicated by the article titled: "Color constancy in the nearly natural image. 1. Asymmetric matches. (Brainard, D. H., Brunt, W. A, & Speigle, J. M. (1997), Journal of the Optical Society of America. A, Optics, Image Science, and l'ision, 14, 2091-2110.)

Without stimulus/variable control, questions cannot be framed in a meaningful, clear way. To say that we are controlling for "degree of naturalism" is clearly not a viable option. Not to mention that, as noted above, proving that color constancy is pretty good in "natural" conditions is not interesting - if it weren't, we wouldn't even be asking the question. The question for scientists is, how is it achieved? Refining the answers to this question requires that we posit the role of specific factors, and this means clarifying what features of our stimulus are being manipulated. Again, "degree of naturalness" is a non-starter, and if investigators want to use it, they need to be clear about what they mean by it, and what features are under their control, and what features are NOT under their control, and might affect their results.

In the present study, not even the factor supposedly being manipulated - illumination - was under the investigators' control (see above). They are taking a graphics package's assumptions about what will be perceived as the color of the illumination at face value.

On 2015 Nov 03, Lydia Maniatis commented:

What is the purpose of the data collected in this study? Assuming there is a purpose, are its methods consistent with achieving it?

The title, “Color constancy in a naturalistic, goal-directed task,” is no guide. The ability of humans to achieve color constancy evolved in the natural environment, on the basis of interaction with that environment. Only goal-directed action could lead to selection for a functional perceptual process. “Natural” conditions are where the process enabling color constancy is expected to be most reliable. A plausible question, then, might be “how accurate can color constancy be at its best (assuming we were interested and could measure this.)” However, this is not what the authors are asking.

The abstract suggests two possible motives. First, we are told that “whether and to what degree constancy supports accurate cross-illumination object selection is not well understood.” It is not clear why we would not expect reliable surface perception under variable lighting to “support object selection.” At any rate, this framing of the question seems to assume good color constancy as the independent variable and “object selection” as the dependent variable. A few sentences later, however, the question seems to change; the authors tell us that they “were able to infer and quantify the degree of color constancy that mediated subjects performance.” Now, what seems to have been under investigation is the degree of subjects' color constancy.

However, the question “How good is a subject's color constancy” is meaningless unless we specify conditions. Under some conditions, it is very good, under others, it fails completely. Much is known about these conditions. Artists, for example, are expert at causing radical failures of color constancy via pictorial impressions of inhomogeneous illumination. Perception scientists sometimes have an inkling of this as well. So what specific conditions did the authors aim to test, and why?

The term used is “naturalistic.” What does this mean? The authors apparently can't say. We are told that they manipulated “complexity” by increasing (relative to a previous experiment) the number of distinct surfaces, but that they “do not have at present a precise technical definition of complexity nor precise ways of relating the concepts of scene complexity and scene naturalness” (p. 19). Evidently, “naturalistic” is being defined as “containing more surfaces than stimuli used in a previous experiment.”

At any rate, the stimuli used are clearly not “naturalistic” in any intuitive sense of the word. The authors themselves describe stimuli in one condition as having “a visually jarring appearance – somewhat reminiscent of an image of an early 1970's discotheque...” (p. 17).

So are the authors at least testing color constancy in a randomly selected stimulus? They are not doing even that. This is because their definition of a color constant selection is not based on subjects selecting an e.g. actually red surface under different illuminations, but on subjects matching a red surface with a surface that, in many cases, is not red (i.e. does not have the same chromaticity as the sample.) The stimuli are images on a computer screen, and the changes of illumination are manipulations that the authors hope, or expect, will correspond to the impression of a change in illumination, e.g. the impression of orange lighting. The theoretical assumptions underlying this expectation are part of the graphics package and are never made explicit. They might need reassessment, as subjects did not always make the predicted “color constant” selections. There was, for example, a deviation from the authors' definition/prediction of veridical choices, which based on deviation from their assumptions, they call a “blue bias.” In other words, subjects sometimes matched a sample surface with one that projected more in the blue part of the spectrum than what the authors defined as a “color constant” match. They “do not have an explanation for this bias” (p. 18). However, whether we are dealing with a bias on the part of subjects' perceptual system (unlikely) or a failed prediction of the authors' prepackaged, implicit color constancy/illumination perception model is impossible to say in this hopelessly confounded study.

I would note, finally, that defining the effect of simultaneous contrast as a “failure of color constancy” constitutes a failure to distinguish between figure/ground contrast and adjustments of color contingent on perceived illumination variation.

On 2015 Sep 22, ANDRES TIRADO-SANCHEZ commented:

Are we near to our Krypton?. World paranoia about future experiments at CERN…again. There is growing concern among world's population about the next experiments of the Large Hadron Collider at CERN related to smash subatomic particles at nearly the speed of light, looming next months (interestingly, if we add one of the most mentioned dates on Google maps ©, it redirects us to CERN). This conCERN has invaded the World Wide Web (both created by CERN) through social networking, news, search engines and very descriptive and witty videos that clearly may misinform, in addition to alarm people about the impending destruction of our planet. Detailed information on what lies ahead the CERN experiments is unknown to most of the population, similar to other “end of the world” events, who have gone without fulfilling what they promised however, such events left a feeling of uncertainty that affected people more susceptible, a feeling of fear of an upcoming uncontrollable black hole, that will lead to our extinction. The dark side of the (I must say, exciting) experiments at CERN, are a source of curiosity, fear and uncertainty. Many renowned physicists are concerned and believe that, at some point, both the amount of energy and the black hole generated will get out of control ending in disaster. In order to calm this (still) small outbreak of paranoia, more information is needed; CERN must open doors to the public, reduce mysticism and establish clear rules and communicate the details of the experiments performed there, the real scope, the real risks and the usefulness for mankind; to the extent that the above mentioned aspects are clarified, the population will not create speculation and feel better knowing that what is currently done at CERN will be for our benefit and not our extinction.

On 2017 Jul 04, Randi Pechacek commented:

Elisabeth Bik examines this paper on a microBEnet blog post.

On 2015 Dec 04, Siddharudha Shivalli commented:

I read the article by Gebre M et al, with a great interest. Authors’ efforts are commendable. In their community based study, authors highlight the lower birth preparedness practice in the study area.

Introduction part of the manuscript should delineate the health problem, research question or the hypothesis in such a way that even reader without subject expertise should be able to understand the present situation and need of the study. Keeping the international readers in the mind, authors should have given a brief explanation of existing programs to address maternal and child health issues, with a special emphasis on birth preparedness, in the introduction. Role of healthcare extension worker in the same should have been delineated. In addition, how the study findings could be useful in streamlining of ongoing strategies to foresee the impact should have been discussed.

The level of birth preparedness should have been reported with 95% confidence intervals (i.e. n=104, 18.3%, 95% CI: 15.3-21.7). Authors used the ‘knowledge of danger signs’ during pregnancy, delivery, and postnatal period as a predictor of birth preparedness. However, in methodology, they should have explicitly mentioned the various danger signs asked to assess the same. In Table 4, authors should have reported row wise percentages to make it clearer. Regression analysis is an excellent way of adjusting the effect of con-founders. However, it is always recommended to assess and report the adequacy of applied regression model. Failure to do so may lead to misleading or incorrect inferences. Although the study sample was relatively large (n=569), a word about R2 (explaining the variance in birth preparedness) of the applied regression model would have been more affirmative.

Nonetheless, I must congratulate the authors for exploring an important public health problem in the study area. No competing interests declared.

On 2017 Apr 25, Jeffrey Ross-Ibarra commented:

In my 2007 paper I find a positive correlation between the number of crossovers per bivalent and genome size. I note that this correlation is weaker than linear, such that genome size increases faster than the number of crossovers per bivalent. This means that measures of recombination that include genome size (e.g. cM/Mb) will decrease with increasing genome size (because the denominator increases faster than the numerator). The findings reported here of a negative correlation between cM/Mb and genome size are thus consistent with those reported in my 2007 paper.

On 2015 Oct 07, Bill Cayley commented:

For adolescents with depression, "less" appears to be "more": https://lessismoreebm.wordpress.com/2015/09/17/restoring-study-329-efficacy-and-harms-of-paroxetine-and-imipramine-in-treatment-of-major-depression-in-adolescence/

On 2015 Sep 29, Carlos Maia commented:

http://www.ncbi.nlm.nih.gov/pubmed/25806553

On 2015 Sep 17, Tom Kindlon commented:

(contd.)

11 National Institute for Health and Clinical Excellence. Chronic fatigue syndrome/myalgic encephalomyelitis (or encephalopathy): diagnosis and management of CFS/ME in adults and children, 2007. http://www.nice.org.uk/guidance/CG53 Accessed September 6, 2015. London: National Institute for Health and Clinical Excellence.

12 White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, et al. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. The Lancet 2011;377:823-36.

13 Kindlon T. PACE Trial - 97% of the participants who didn't have a psychiatric disorder satisfied the definition of M.E. used. https://listserv.nodak.edu/cgi-bin/wa.exe?A2=ind1106A&L=CO-CURE&P=R2764 Accessed: September 6, 2015

14 Ellen Goudsmit on PubMed Commons: http://www.ncbi.nlm.nih.gov/myncbi/ellen m.goudsmit.1/comments/

15 Green CR, Cowan P, Elk R, O'Neil KM, Rasmussen AL. National Institutes of Health Pathways to Prevention Workshop: advancing the research on myalgic encephalomyelitis/chronic fatigue syndrome. Ann Intern Med. 2015; 162:860-5.

16 Haney E, Smith MEB, McDonagh M, Pappas M, Daeges M, Wasson N, et al. Diagnostic methods for myalgic encephalomyelitis/chronic fatigue syndrome: a systematic review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med. 2015; 162:834-40.

17 Smith MEB, Haney E, McDonagh M, Pappas M, Daeges M, Wasson N, et al. Treatment of myalgic encephalomyelitis/chronic fatigue syndrome: a systematic review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med. 2015; 162:841-50.

On 2015 Sep 17, Tom Kindlon commented:

Comment 2 of 2 formally submitted on September 9, 2015 on the full Cochrane review using instructions here: http://www.cochranelibrary.com/help/submitting-comments-on-cochrane-reviews.html

(This is available in one piece at: http://forums.phoenixrising.me/index.php?threads/my-two-detailed-comments-on-the-cochrane-exercise-therapy-for-cfs-review-2015.39801/ )

Second comment:

I have decided to split my comment into two as it was getting very long. This is part two.

Variation in interventions

It would have been useful to have some more information on the “exercise with pacing” intervention tested in the Wallman et al. (2004) trial and how it was distinct from some other exercise interventions tested. The authors say (1): “On days when symptoms are worse, patients should either shorten the session to a time they consider manageable or, if feeling particularly unwell, abandon the session altogether” (p. 143). I don't believe the description given in the review conveys this. In the review, this approach is described as "Exercise with pacing: exercise in which the incremental increase in exercise was personally set." But Wallman et al.’s approach allows patients to decrease as well as increase how much exercise they do on the day. This approach also contrasts with how White (an investigator in two of the trials) has described graded exercise therapy: "if [after increasing the intensity or duration of exercise] there has been an increase in symptoms, or any other adverse effects, they should stay at their current level of exercise for a further week or two, until the symptoms are back to their previous levels" (2). In the PACE Trial manual White co-wrote (3), the GET intervention was guided by the principle that “planned physical activity and not symptoms are used to determine what the participant does” (p. 21); similarly, “it is their planned physical activity, and not their symptoms, that determine what they are asked to do” (p. 20). Compliance data would help us examine which approach patients are actually using: I suspect many patients are in fact doing exercise with pacing even in trials such as the PACE Trial (i.e. when they have increased symptoms, often reducing levels of exercise and sometimes doing no exercise activities at all on that day).

Bimodal versus Likert scoring in Wearden et al. (2010)

I find it odd that the fatigue scores for the Wearden et al. (2010) trial (4) are given in the 0-33 format rather than the 0-11 scoring method. The 0-11 scoring system is what is mentioned as a primary outcome measure in the protocol and is what is reported in the main paper reporting the results (4, 5). It is even what your own report says on p. 44 is the scoring method (“Fatigue Scale, FS; 11 items; each item was scored dichotomously on a 4-point scale [0, 0, 1 or 1]”). This is important because using the scoring method for which you don't report data (0-11), there is no statistically significant difference at the primary outcome point of 70 weeks (5).

Diagnostic criteria

One problem with using these trials as an evidence base, which I don't believe was mentioned, is that all the trials used the Oxford and Fukuda diagnostic criteria (6, 7). Neither of these criteria require patients to have post-exertional malaise (or something similar). Many consider this to be a core symptom of ME/CFS and it is mandatory in most of the other major criteria (8-11). [Aside: The London criteria were assessed in the PACE Trial (12) but they seem to have been operationalised in an unusual way. Ninety seven per cent of the participants who satisfied the (broad) Oxford criteria who didn't have a psychiatric disorder satisfied the definition of M.E. used (13). Ellen Goudsmit, one of the authors of the London criteria, has rejected the way they were used in the PACE Trial (14)]. So this lack of requirement for patients to have post-exertional malaise (or a similar description) means we cannot be sure that the evidence can be generalised to such patients. An independent National Institutes of Health committee this year concluded "continuing to use the Oxford definition may impair progress and cause harm. Therefore, for progress to occur, we recommend that this definition be retired" (15). An Agency for Healthcare Research and Quality review of diagnostic methods this year reached a similar conclusion: "Consensus groups and researchers should consider retiring the Oxford case definition because it differs from the other case definitions and is the least restrictive, probably including individuals with other overlapping conditions” (16). An Agency for Healthcare Research and Quality review of ME/CFS treatments said: "The Oxford CFS case definition is the least restrictive, and its use as entry criteria could have resulted in selection of participants with other fatiguing illnesses or illnesses that resolve spontaneously with time" (17).

Exclusion of some data from analyses due to baseline differences

It seems unfortunate that some data cannot be used due to baseline differences e.g. "Four trials (669 participants) contributed data for evaluation of physical functioning at follow-up (Jason 2007; Powell 2001; Wearden 2010; White 2011). Jason 2007 observed better results among participants in the relaxation group (MD 21.48, 95% CI 5.81 to 37.15). However, results were distorted by large baseline differences in physical functioning between the exercise and relaxation groups (39/100 vs 54/100); therefore we decided not to include these results in the meta-analysis". It would be good if other methods could be investigated (e.g. using baseline levels as covariates) to analyse such data.

Thank you for taking the time to read my comments.

Tom Kindlon

Conflict of Interest statement:

I am a committee member of the Irish ME/CFS Association and do a variety of unpaid work for the Association.

References

1 Wallman KE, Morton AR, Goodman C, Grove R. Exercise prescription for individuals with chronic fatigue syndrome. Med J Aust. 2005;183:142-3.

2 White P. How exercise can help chronic fatigue syndrome. Pulse: 1998. June 20:86-87.

3 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME (Therapist Manual) http://www.pacetrial.org/docs/get-therapist-manual.pdf

4 Wearden AJ, Riste L, Dowrick C, Chew-Graham C, Bentall RP, Morriss RK, Peters S, Dunn G, Richardson G, Lovell K, Powell P. Fatigue Intervention by Nurses Evaluation--the FINE Trial. A randomised controlled trial of nurse led self-help treatment for patients in primary care with chronic fatigue syndrome: study protocol. [ISRCTN74156610]. BMC Med. 2006 Apr 7;4:9.

5 Wearden AJ, Dowrick C, Chew-Graham C, Bentall RP, Morriss RK, Peters S, Riste L, Richardson G, Lovell K, Dunn G; Fatigue Intervention by Nurses Evaluation (FINE) trial writing group and the FINE trial group. Nurse led, home based self help treatment for patients in primary care with chronic fatigue syndrome: randomised controlled trial. BMJ. 2010 Apr 23;340:c1777. doi: 10.1136/bmj.c1777.

6 Sharpe M, Archard L, Banatvala J, Borysiewicz LK, Clare AW, David A, et al. Chronic fatigue syndrome: guidelines for research. Journal of the Royal Society of Medicine 1991;84 (2):118–21.

7 Fukuda K, Straus SE, Hickie I, et al. The chronic fatigue syndrome: A comprehensive approach to its definition and study. Ann Intern Med. 1994; 121: 953‑959.

8 Carruthers BM, Jain AK, De Meirleir KL, et al. Myalgic Encephalomyelitis/chronic fatigue syndrome: Clinical working case definition, diagnostic and treatments protocols. Journal of Chronic Fatigue Syndrome. 2003; 11: 7-115.

9 Carruthers BM, van de Sande MI, De Meirleir KL, et al. Myalgic encephalomyelitis: International Consensus Criteria. J Intern Med. 2011; 270: 327-338.

10 IOM (Institute of Medicine). Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Washington, DC: The National Academies; 2015.

(continues)

On 2015 Sep 17, Tom Kindlon commented:

(continues)

References:

1 Turner L, Boutron I, Hróbjartsson A, Altman DG, Moher D: The evolution of assessing bias in Cochrane systematic reviews of interventions: celebrating methodological contributions of the Cochrane Collaboration. Syst Rev 2013, 2:79.

2 Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry. 2015;2:141-152.

3 White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, et al. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. The Lancet 2011;377:823-36.

4 Kindlon T. Reporting of Harms Associated with Graded Exercise Therapy and Cognitive Behavioural Therapy in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Bull IACFS ME. 2011;19:59-111. http://iacfsme.org/ME-CFS-Primer-Education/Bulletins/BulletinRelatedPages5/Reporting-of-Harms-Associated-with-Graded-Exercise.aspx

5 Lipkin DP, Scriven AJ, Crake T, Poole-Wilson PA (1986). Six minute walking test for assessing exercise capacity in chronic heart failure. British Medical Journal 292, 653-5.

6 Kadikar A, Maurer J, Kesten S. The six-minute walk test: a guide to assessment for lung transplantation. J Heart Lung Transplant. 1997 Mar;16(3):313-9.

7 Friedberg F, Sohl S. Cognitive-behavior therapy in chronic fatigue syndrome: is improvement related to increased physical activity? J Clin Psychol. 2009 Feb 11.

8 Friedberg F. Does graded activity increase activity? A case study of chronic fatigue syndrome. Journal of Behavior Therapy and Experimental Psychiatry, 2002, 33, 3-4, 203-215

9 Results and In-depth Analysis of the 2012 ME Association Patient Survey Examining the Acceptability, Efficacy and Safety of Cognitive Behavioural Therapy, Graded Exercise Therapy and Pacing, as Interventions used as Management Strategies for ME/CFS. Gawcott, England. http://www.meassociation.org.uk/2015/05/23959/ Accessed: September 3, 2015

10 Critical Illness - A Dreadful Experience with Scottish Provident. http://forums.moneysavingexpert.com/showthread.php?t=2356683 Accessed: September 4, 2015

11 McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS One. 2012;7(8):e40808.

12 Rapport d’évaluation (2002-2004) portant sur l’exécution des conventions de rééducation entre le Comité de l’assurance soins de santé (INAMI) et les Centres de référence pour le Syndrome de fatigue chronique (SFC). 2006. http://health.belgium.be/internet2Prd/groups/public/@public/@shc/documents/ie2divers/14926531_fr.pdf (Starts on page 223.) Accessed September 4, 2015 (French language edition)

13 Evaluatierapport (2002-2004) met betrekking tot de uitvoering van de revalidatieovereenkomsten tussen het Comité van de verzekering voor geneeskundige verzorging (ingesteld bij het Rijksinstituut voor Ziekte- en invaliditeitsverzekering) en de Referentiecentra voor het Chronisch vermoeidheidssyndroom (CVS). 2006. http://health.belgium.be/internet2Prd/groups/public/@public/@shc/documents/ie2divers/14926531.pdf (Starts on page 227.) Accessed September 4, 2015 (Dutch language version)

14 Stordeur S, Thiry N, Eyssen M. Chronisch Vermoeidheidssyndroom: diagnose, behandeling en zorgorganisatie. Health Services Research (HSR). Brussel: Federaal Kenniscentrum voor de Gezondheidszorg (KCE); 2008. KCE reports 88A (D/2008/10.273/58) https://kce.fgov.be/sites/default/files/page_documents/d20081027358.pdf Accessed September 4, 2015

15 Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg G. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychol Med. 2010; 40:1281-1287.

16 Van Kessel K, Moss-Morris R, Willoughby, Chalder T, Johnson MH, Robinson E, A randomized controlled trial of cognitive behavior therapy for multiple sclerosis fatigue, Psychosom. Med. 2008; 70:205–213.

17 Powell P. FINE Trial Patient Booklet http://www.fine-trial.net/downloads/Patient PR Manual ver9 Apr05.pdf Accessed September 7, 2015

18 Twisk FNM, Maes M. A review on Cognitive Behavorial Therapy (CBT) and Graded Exercise Therapy (GET) in Myalgic Encephalomyelitis (ME)/Chronic Fatigue Syndrome (CFS): CBT/GET is not only ineffective and not evidence-based, but also potentially harmful for many patients with ME/CFS. Neuro Endocrinol Lett. 2009;30:284-299.

19 Carruthers BM et al. Myalgic Encephalomyelitis – Adult & Paediatric: International Consensus Primer for Medical Practitioners. ISBN 978-0-9739335-3-6 http://www.investinme.org/Documents/Guidelines/Myalgic Encephalomyelitis International Consensus Primer -2012-11-26.pdf Accessed September 5, 2015

20 Twisk FN. Objective Evidence of Post-exertional “Malaise” in Myalgic Encephalomyelitis and Chronic Fatigue Syndrome. J Sports Med Doping Stud 2015. 5:159. doi: 10.4172/2161-0673.1000159

21 Higgins JPT, Green S: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. Table 8.5.d. The Cochrane Collaboration; 2011. http://handbook.cochrane.org/chapter_8/table_8_5_d_criteria_for_judging_risk_of_bias_in_the_risk_of.htm Accessed: September 5, 2015

22 White PD, Sharpe MC, Chalder T, DeCesare JC, Walwyn R; on behalf of the PACE trial group. Protocol for the PACE trial: A randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurology 2007, 7:6 http://www.biomedcentral.com/1471-2377/7/6 Accessed: September 5, 2015

On 2015 Sep 17, Tom Kindlon commented:

(contd.)

Compliance

The review doesn't include any information on compliance. I'm not sure that there is much published information on this but I know there was a measure based on attendance at therapy sessions (which could be conducted over the phone) given for the PACE Trial (3). Ideally, it would be interesting if you could obtain some unpublished data from activity logs, records from heart-rate monitors, and other records to help build up a picture of what exercise was actually performed and the level of compliance. Information on adherence and what exercise was actually done is important in terms of helping clinicians, and indeed patients, to interpret and use the data. I mention patients because patients' own decisions about their behaviour is likely to be affected by the medical information available to them, both within and outside of a supervised programme of graded exercise; unlike with an intervention like a drug, patients can undertake exercise without professional supervision.

"Selective reporting (outcome bias)" and White et al. (2011)

I don't believe that White et al. (2011) (the PACE Trial) (3) should be classed as having a low risk of bias under "Selective reporting (outcome bias)" (Figure 2, page 15). According to the Cochrane Collaboration's tool for assessing risk of bias (21), the category of low risk of bias is for: "The study protocol is available and all of the study’s pre-specified (primary and secondary) outcomes that are of interest in the review have been reported in the pre-specified way". This is not the case in the PACE Trial. The three primary efficacy outcomes can be seen in the published protocol (22). None have been reported in the pre-specified way. The Cochrane Collaboration's tool for assessing risk of bias states that a “high risk” of bias applies if any one of several criteria are met, including that “not all of the study’s pre-specified primary outcomes have been reported” or “one or more primary outcomes is reported using measurements, analysis methods or subsets of the data (e.g. subscales) that were not pre-specified”. In the PACE Trial, the third primary outcome measure (the number of "overall improvers") was never published. Also, the other two primary outcome measures were reported using analysis methods that were not pre-specified (including switching from the bimodal to the Likert scoring method for The Chalder Fatigue Scale, one of the primary outcomes in your review). These facts mean that the “high risk of bias” category should apply.

Thank you for taking the time to read my comments.

Tom Kindlon

Conflict of Interest statement:

I am a committee member of the Irish ME/CFS Association and do a variety of unpaid work for the Association. (continues)

On 2015 Sep 17, Tom Kindlon commented:

Comment 1 of 2 formally submitted on September 9, 2015 on the full Cochrane review using instructions here: http://www.cochranelibrary.com/help/submitting-comments-on-cochrane-reviews.html

(This is available in one piece at: http://forums.phoenixrising.me/index.php?threads/my-two-detailed-comments-on-the-cochrane-exercise-therapy-for-cfs-review-2015.39801/ )

I would first like to thank those involved for their work in preparing this document. Even for those of us who have read the individual Chronic Fatigue Syndrome (CFS) papers it is useful to have the results collated, as well as details regarding the interventions. Also it is interesting to see the results of sensitivity analyses, subgroup analyses, standardised mean differences, etc.

I would like to make a few comments. I’m splitting them into two submissions as the piece had become very long. I’ve added some loose headings to hopefully make it more readable.

Objective measures

The review assessed the studies as having a high risk of bias regarding blinding, since neither participants nor assessors were blinded. Evidence suggests that subjective outcomes are more prone to bias than objective outcomes when there is no blinding (1). It is thus unfortunate that the review concentrated almost exclusively on subjective measures, failing to include results from nearly all the objective outcome measures that have been published with trials. (The exception was health resource use for which you presented follow-up data from one trial).

I hope objective outcome data can be included in a future revision or edition of this review.

Examples of objective outcomes include: exercise testing (work capacity by oxygen consumption); fitness test/step test; the six minute walking test; employment status; and disability payments.

Adding in these results would allow a more rigorous assessment of the effectiveness and relevance of the therapies, their causal mechanisms, therapeutic compliance, and safety.

On exercise testing, for example, in the PACE Trial (the largest trial in the review) there was no improvement in fitness levels as measured by a step test (2). The fitness data contrasts sharply with the many positive results from subjective self-report measures in the trial, so one is left wondering how much the subjective measures reflect reality.

On another exercise test used in the PACE Trial, the 6 minute walk test, there was a small (mean) increase from 312 metres at baseline to 379 metres at 12 months: this was 35.3 metres more than the "passive" control group when adjustments were made. However, the final result of 379 metres remains very poor compared to the more than 600 metres one would expect from healthy people of a similar age and gender make-up (3,4). By comparison, a group with Class III heart failure walked an average of 402 metres (5). A score of less than 400 metres has been suggested as the level at which somebody should be put on a lung transplant list (6). Such information from objective measures helps to add important context to the subjective measures and restraint to the conclusions that can be drawn from them.

Objective data is also needed to check compliance with a therapy. If patients diligently exercised for 12 months one would expect much better results on fitness and exercise testing than the aforementioned results in the PACE Trial. This is important when considering adverse events and safety: such trials may not give us good information on the safety of complying with such interventions if patients haven't actually complied.

Employment and receipt of disability payments are practical objective measures of general functional capacity so data on them would help establish whether patients can actually do more overall or whether they may just be doing, for example, a little more exercise but have substituted that for doing less in other areas (7,8). Also, CFS patients are sometimes pressured by insurance companies into doing graded exercise therapy (GET) programs so it would be useful to have data collated on employment outcomes to see whether pressure can in any way be justified (9,10). In the PACE Trial, there was no significant improvement in employment measures and receipt of disability payments in the GET group (11). Outside the realm of clinical trials, the quantitative and qualitative data in a major (UK) ME Association survey also found that GET didn't lead to higher levels of employment and lower levels of receipt of disability payments on average (9). Also, extensive external audits were performed of Belgian CFS rehabilitation clinics that treated using cognitive behavioural therapy (CBT) and GET. The main reports are in French and Dutch (12,13), with an English summary available (14) that says, "Employment status decreased at the end of the therapy, from an average of 18.3% of a 38h working week, to 14.9% [...] The percentage of patients living from a sickness allowance increased slightly from 54 to 57%." This contrasts with the average improvements reported in the audit for some symptoms like fatigue.

While data on (self-reported) symptoms like fatigue (one of your two primary outcomes) is interesting, arguably more important to patients is improving their overall level of functioning (and again, objective measures are needed here). Being able to work, for example, despite experiencing a certain level of fatigue would likely be more important for many than being unable to work but having slightly lower levels of fatigue.

An example of how reductions in the reported levels of fatigue may not lead to improvements in functioning can be seen in an analysis of three graded activity-oriented CBT therapy interventions for CFS (15). The analysis showed, compared to controls, there were no improvements in overall activity levels as measured by actometers despite improvements in self-reported fatigue (15). Activity in these trials was assessed using actometers. Another study that exemplifies the problem of focusing too much on fatigue scores after behavioural interventions is a study of CBT in multiple sclerosis (MS) patients with “MS fatigue”(16). The study found that following the intervention, patients with MS reported significantly lower (i.e. better) scores on the Chalder Fatigue Scale (0-33 scoring) than those in a healthy, nonfatigued comparison group! This significant difference was maintained at 3 and 6 months’ follow-up. It is difficult to believe that patients with MS fatigue (at baseline) truly subsequently had less fatigue than healthy nonfatigued controls: a much more likely scenario is that undertaking the intervention had led to response biases.

You mention that "many patient charities are opposed to exercise therapy for chronic fatigue syndrome (CFS)". One reason for concern about the way in which exercise programmes are promoted to patients is that they are often based upon models which assume that there is no abnormal physiological response to exercise in the condition, and make unsupported claims to patients. For example, in the FINE trial (Wearden et al., 2010) patient booklet (17), it is boldly asserted that: "Activity or exercise cannot harm you" (p. 49). However, a large number of studies have found abnormal responses to exercise, and the possibility of harm being done simply cannot be excluded on the basis of current evidence (discussed in 4, 18-20)."

(continues)

On 2016 Feb 08, Theo Sanderson commented:

"Briefly, PMA stock solution (20 mM in 20 % dimethyl sulfoxide; Biotium Inc., Hayward, CA) was added at a concentration of 200 mM." It seems likely that the working concentration was in fact 200 μM?

On 2015 Sep 30, Anders von Heijne commented:

This is an important analysis. The only thing missing is the temporal relationship between the debut of the PML-IRIS related encephalitic symptoms and the imaging findings. Do they occur at the same time or are there cases when the imaging findings precede the clinical worsening?

On 2017 Oct 12, Sander Houten commented:

This paper focuses on the role of KLF14 in the regulation of hepatic ApoA-I expression. The authors show that hepatic KLF14 expression is reduced in dyslipidemia mouse models. They show that overexpression of human KLF14 in liver of mice that were fed a high fat diet, increased HDL-cholesterol, cholesterol efflux and ApoA-I expression. Conversely, they show that liver-specific deletion of Klf14 decreased HDL-cholesterol and ApoA-I expression. The authors offer a mechanistic explanation by demonstrating that the ApoA-I promoter contains one KLF14-binding site. Moreover, perhexiline was identified as a KLF14 activator that raised HDL-cholesterol, ApoA-I and cholesterol efflux in vivo and reduced atherosclerosis development in Apoe^−/− mice (Guo Y, 2015). I would like to point out that there is little evidence to support the hypothesis that KLF14 plays an important role in adult mouse liver biology. We found no evidence for expression of KLF14 in adult mouse liver as we were unable to amplify Klf14 cDNA, did not find Klf14 mapped reads in liver RNA sequencing data and found no specific signal upon immunoblotting (Argmann CA, 2017). Our data on the absence of Klf14 expression in liver are consistent with previously published work by others (Parker-Katiraee L, 2007) and publicly available data sources. We also investigated the physiological functions of KLF14 by studying a whole body KO mouse model and focused on the metabolic role of this transcription factor in mice on chow and high fat diet. Our results do not support a major role of hepatic KLF14 in the regulation of ApoA-I and HDL-cholesterol levels (Argmann CA, 2017).

On 2016 Apr 16, Miguel Lopez-Lazaro commented:

Important flaws in the identification of the cells of origin in cancer

This is an extension of this Editorial (Cancer arises from stem cells: opportunities for anticancer drug discovery) and a commentary on the article http://www.ncbi.nlm.nih.gov/pubmed/27044116

Abstract: The cells of origin in cancer are the cells in which carcinogenesis starts, i.e., the normal cells that acquire the first cancer-related hit. These cells should not be confused with the cancer stem cells or tumor-initiating cells, which are malignant cells with the ability to initiate and maintain tumor growth. Recently, many articles published in leading journals claim that they have experimentally demonstrated the cellular origin of several types of cancer. These articles, however, have important flaws that may have led to wrong conclusions. Here I briefly discuss one of the many articles in this field of research to draw attention to these flaws. In addition, I analyze the striking differences in prostate cancer incidence by age to examine the article’s conclusion and to propose that prostate cancer originates in stem cells. The striking variations in cancer risk among tissues also indicate that stem cells are the cells of origin in cancer.

Full text at: Lopez-Lazaro, M. Important flaws in the identification of the cells of origin in cancer. ResearchGate: DOI: 10.13140/RG.2.1.4321.5769; http://dx.doi.org/10.13140/RG.2.1.4321.5769

On 2015 Dec 10, Kenneth Witwer commented:

As a co-author of this meeting report on an RNAi session at the 40th meeting of The Toxicology Forum, I summarized a talk I gave on human health considerations related to environmental RNA molecules. During my talk, I also reflected briefly on a 2014 US Environmental Protection Agency panel and a European Food Safety Authority (EFSA) workshop I had recently taken part in. During the preparation and revision process of this manuscript, EFSA published a workshop report on the EFSA website. I inadvertently did not include a link to this report in the manuscript, so I would like to note that the pdf can be downloaded here. "Selected key outcomes of the workshop" were also described by Ramon M, 2014.

On 2014 Dec 23, Akshaya shetti commented:

can i get the full text of this article?

On 2017 Apr 01, Misha Koksharov commented:

It's interesting whether LREs have a gluconolactonase function. Or whether gluconolactonases have the LRE activity

On 2017 Mar 26, Misha Koksharov commented:

None

On 2017 Jul 04, Randi Pechacek commented:

Matt Perisin, first author of this paper, wrote a "Behind the Paper" blog on microBEnet.

On 2016 Feb 16, Robert Courtney commented:

A study that uses questionable assumptions rather than empirical evidence leads to conclusions that stretch credibility.

Chalder et al. [1] used the “single mediation model” for their methodology, which is explained in detail in a book by MacKinnon [2]. Explaining the methodology MacKinnon says a temporal separation between variables must be observed (i.e. changes in mediating variable must occur before changes in the mediated variable) for a mediation effect to be empirically and robustly established.

Chalder et al. were working to this model and acknowledged that they failed to establish a clear temporal separation between variables, and therefore did not empirically establish a causal mediation effect: “Given the pattern of change in the mediators was similar to the pattern of change in the outcomes it is possible that the variables were affecting each other reciprocally”.

However, despite the lack of robust empirical evidence to support a mediation effect, the investigators concluded that they had established mediation effects, e.g: “Our main finding was that fear avoidance beliefs were the strongest mediator for both CBT and GET.”

The study’s conclusion relied upon an assumption that the investigators’ favoured hypothetical model of illness for ME/CFS has a robust empirical evidence base and is applicable to this study. The hypothesis is based upon the idea that symptoms and disability in ME/CFS are perpetuated by unhelpful or maladaptive illness beliefs, fear, and an avoidance of activity.

However, the prestigious National Academy of Medicine (formerly known as the Institute of Medicine) recently released a comprehensive report [3] into ME/CFS that rejected such a hypothetical model of illness, and unambiguously concluded that ME/CFS does not have a psychological or cognitive-behavioural basis, but is an organic illness that requires biomedical research.

Chalder et al. discussed the possibility that more frequent measurements may have potentially demonstrated a temporal separation between the variables, and therefore a mediation effect. However, this raises the possibility of whether changes in the primary outcome variables (self-report physical function and fatigue) may, in fact, have occurred before changes in the presumed mediator variables. Such an outcome would entirely contradict the investigators’ premature conclusions. According to MacKinnon [2] and Wiedermann et al. [4], unexpected outcomes should not be ruled out.

Chalder et al. concluded that symptoms and physical impairment, in ME/CFS patients, are mediated by activity avoidance and other factors. (e.g. This would mean that a decrease in activity would cause an increase in symptoms.) However, from a common sense point of view, this seems like rather a convoluted conclusion, and it seems more likely that increased symptoms would be the direct cause of activity avoidance in any illness, rather than vice versa. To conclude that activity avoidance causes fatigue (rather than fatigue being a direct cause of activity avoidance), is similar to concluding that a person has flu because they’ve taken a day off work, rather than the obvious conclusion that they’ve taken a day off work because they have flu.

In the case of fatigue, flu-like malaise and other symptoms of ME/CFS, it seems reasonable to consider the possibility that, as the symptoms fluctuate, patients may intuitively or rationally adapt their activity levels according to what is comfortable and safe. i.e. patients reduce activity levels because they are fatigued. The investigators have concluded that patients are fatigued because they have reduced activity levels.

Perhaps patients’ perspectives and insights would help clarify the issues but, unfortunately, patients were not consulted for this study.

References:

1. Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 2015; 2: 141–52.

2. MacKinnon DP. Introduction to Statistical Mediation Analysis. Taylor and Francis: New York 2008.

3. IOM (Institute of Medicine). 2015. Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Washington, DC: The National Academies Press. http://iom.nationalacademies.org/Reports/2015/ME-CFS.aspx

4. Wiedermann W, von Eye A. Direction of Effects in Mediation Analysis. Psychol Methods 2015; 20: 221-44.

On 2015 Sep 15, Tom Kindlon commented:

(Contd.)

References:

1 Torjesen I. Tackling fears about exercise is important for ME treatment, analysis indicates. BMJ 2015;350:h227 http://www.bmj.com/content/350/bmj.h227

2 Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 14 Jan 2015, doi:10.1016/S2215-0366(14)00069-8.

3 Burgess M, Chalder T. Manual for Participants. Cognitive behaviour therapy for CFS/ME.http://www.pacetrial.org/docs/cbt-participant-manual.pdf (accessed: January 17, 2015)

4 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME. Information for Participants http://www.pacetrial.org/docs/get-participant-manual.pdf (accessed: January 17, 2015)

5 Wechsler ME, Kelley JM, Boyd IO, Dutile S, Marigowda G, Kirsch I, Israel E, Kaptchuk TJ. Active albuterol or placebo, sham acupuncture, or no intervention in asthma. N Engl J Med. 2011;365(2):119-26.

6 Whiting P, Bagnall AM, Sowden AJ, Cornell JE, Mulrow CD, Ramírez G. Interventions for the treatment and management of chronic fatigue syndrome: a systematic review. JAMA. 2001 Sep 19;286(11):1360-8.

7 Burgess M, Chalder T. PACE manual for therapists. Cognitive behaviour therapy for CFS/ME.http://www.pacetrial.org/docs/cbt-therapist-manual.pdf (accessed: January 17, 2015)

8 White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, et al, for the PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011;377:823-36.

9 McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS One. 2012;7(8):e40808. doi: 10.1371/journal.pone.0040808

10 Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg G. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychol Med. 2010 Aug;40(8):1281-7. doi: 10.1017/S0033291709992212. Epub 2010 Jan 5.

11 Heins MJ, Knoop H, Burk WJ, Bleijenberg G. The process of cognitive behaviour therapy for chronic fatigue syndrome: which changes in perpetuating cognitions and behaviour are related to a reduction in fatigue? J Psychosom Res. 2013 Sep;75(3):235-41. doi: 10.1016/j.jpsychores.2013.06.034. Epub 2013 Jul 19.

12 Friedberg F, Sohl S. Cognitive-behavior therapy in chronic fatigue syndrome: is improvement related to increased physical activity? J Clin Psychol. 2009 Apr;65(4):423-42. doi: 10.1002/jclp.20551.

13 Knoop H, Prins JB, Stulemeijer M, van der Meer JW, Bleijenberg G. The effect of cognitive behaviour therapy for chronic fatigue syndrome on self-reported cognitive impairments and neuropsychological test performance. Journal of Neurology and Neurosurgery Psychiatry. 2007 Apr;78(4):434-6.

14 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME (Therapist manual): http://www.pacetrial.org/docs/get-therapist-manual.pdf (accessed: January 17, 2015)

15 O'Dowd H, Gladwell P, Rogers CA, Hollinghurst S, Gregory A. Cognitive behavioural therapy in chronic fatigue syndrome: a randomised controlled trial of an outpatient group programme. Health Technology Assessment, 2006, 10, 37, 1-140.

16 Knoop H, Wiborg JF. What makes a difference in chronic fatigue syndrome? Lancet Psychiatry 13 Jan 2015 DOI: http://dx.doi.org/10.1016/S2215-0366(14)00145-X

17 Kindlon T. Reporting of Harms Associated with Graded Exercise Therapy and Cognitive Behavioural Therapy in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Bulletin of the IACFS/ME. 2011;19(2):59-111http://iacfsme.org/BULLETINFALL2011/Fall2011KindlonHarmsPaperABSTRACT/ta...

18 Lipkin DP, Scriven AJ, Crake T, Poole-Wilson PA. Six minute walking test for assessing exercise capacity in chronic heart failure. Br Med J (Clin Res Ed) 1986. 292:653–655. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1339640/pdf/bmjcred00224-001...

19 Marin JM, Carrizo SJ, Gascon M, Sanchez A, Gallego B, Celli BR. Inspiratory Capacity, Dynamic Hyperinflation, Breathlessness, and Exercise Performance during the 6-Minute-Walk Test in Chronic Obstructive Pulmonary Disease. Am. J. Respir. Crit. Care Med. 2001 63(6):1395-1399.http://171.66.122.149/content/163/6/1395.full

20 Goldman MD, Marrie RA, Cohen JA. Evaluation of the six-minute walk in multiple sclerosis subjects and healthy controls. Multiple Sclerosis 2008. 14(3):383-390. http://pocketknowledge.tc.columbia.edu/home.php/viewfile/download/65399/The six-minute walk test.pdf

21 Ross RM, Murthy JN, Wollak ID, Jackson AS. The six minute walk test accurately estimates mean peak oxygen uptake. BMC Pulm Med. 2010 May 26;10:31. PMID 20504351.http://www.biomedcentral.com/1471-2466/10/31

22 Camarri B, Eastwood PR, Cecins NM, Thompson PJ, Jenkins S. Six minute walk distance in healthy subjects aged 55–75 years. Respir Med. 2006. 100:658-65 http://www.resmedjournal.com/article/S0954-6111(05)00326-4/abstract

23 Troosters T, Gosselink R, Decramer M. Six minute walking distance in healthy elderly subjects. Eur Respir J. 1999. 14:270-4. http://www.ersj.org.uk/content/14/2/270.full.pdf

24 Rapport d'évaluation (2002-2004) portant sur l'exécution des conventions de rééducation entre le Comité de l'assurance soins de santé (institué auprès de l'Institut national d'assurance maladie invalidité) et les Centres de référence pour le Syndrome de fatigue chronique (SFC), Bruxelles, juillet 2006. (French language edition)

25 Evaluatierapport (2002-2004) met betrekking tot de uitvoering van de revalidatieovereenkomsten tussen het Comité van de verzekering voor geneeskundige verzorging (ingesteld bij het Rijksinstituut voor Ziekte- en invaliditeitsverzekering) en de Referentiecentra voor het Chronisch vermoeidheidssyndroom (CVS). 2006. Available online:https://drive.google.com/file/d/0BxnVj9ZqRgk0QTVsU2NNLWJSblU/edit (accessed: January 17, 2015) (Dutch language version)

26 Stordeur S, Thiry N, Eyssen M. Chronisch Vermoeidheidssyndroom: diagnose, behandeling en zorgorganisatie. Health Services Research (HSR). Brussel: Federaal Kenniscentrum voor de Gezondheidszorg (KCE); 2008. KCE reports 88A (D/2008/10.273/58)https://kce.fgov.be/sites/default/files/page_documents/d20081027358.pdf (accessed: January 17, 2015)

On 2015 Sep 15, Tom Kindlon commented:

Objective measures found a lack of improvement for CBT & GET in the PACE Trial: subjective improvements may simply represent response biases or placebo effects in this non-blinded trial

[Originally posted here: http://www.bmj.com/content/350/bmj.h227/rr-10]

This BMJ article and a flurry of articles in the lay media this week followed the publication in Lancet Psychiatry of an analysis of the mediators of change in the important PACE Trial, a chronic fatigue syndrome (CFS) trial which cost UK taxpayers £5 million[1,2]. What seems to have been lost in the coverage is that, although there were some modest improvements in the self-report measures, there was an almost complete absence of improvements in objectively measured outcomes for cognitive behavioural therapy (CBT) and graded exercise therapy (GET) compared to the control group (specialist medical care only (SMC)).

This is a non-blinded trial, where participants were told CBT and GET had previously been found to be effective in CFS and other conditions[3,4]: one way to look at the mediation results for subjective measures when there was a lack of objective improvements is that they may merely tell us how response biases and/or placebo effects are mediated[5].

The focus on subjective measures in some CFS studies was previously criticised in a systematic review published back in 2001 (long before the PACE Trial started)[6]. They suggested instead "a more objective measure of the effect of any intervention would be whether participants have increased their working hours, returned to work or school, or increased their physical activities."

The model presented for cognitive behaviour therapy (CBT) in the PACE Trial manuals posits that the impairments and symptoms are reversible with the therapy[3,7]. However, the latest paper shows that fitness, as measured by a step test, didn't improve following CBT[2]. An earlier PACE Trial publication reported that the addition of CBT to SMC did not result in an improvement in 6-minute walking test scores compared to SMC alone[8].

The PACE Trial was part funded by the UK Department of Work and Pensions, a rare move for them, presumably done due to an expectation that the therapies would improve measures of employment and levels of benefit receipt. However, again CBT brought about no improvement using objective measures, such as days of employment lost, levels of disability benefits received and levels of receipt of insurance payments[9].

These results are in line with earlier studies of CBT. For example, an analysis of three randomized controlled trials of CBT interventions for CFS found no improvement in objectively measured activity, despite participants reporting a reduction in (self-reported) fatigue and (sometimes) functional impairments[10]. Similar results were found in another uncontrolled trial where changes in objectively measured activity did not predict fatigue levels, and objectively measured activity on completion remained low compared to population norms[11]. An uncontrolled study found improvements in self-reported physical functioning and fatigue were reported despite a numerical decrease in (objectively measured) activity[12]. In another study, the level of self-reported cognitive impairment in CFS patients decreased significantly after CBT, however, cognition had not improved when it was measured objectively using neuropsychological test performance[13].

It is unsurprising that 15 sessions of CBT (and the associated homework exercises and management program) might alter how participants respond to self-report questionnaires. A PACE Trial manual itself says "the essence of CBT is helping the participant to change their interpretation of symptoms": this could lead to altered or biased fatigue scores, one of the two primary outcome measures[14]. Also, one of the aims of CBT (for CFS) has been said to be "increased confidence in exercise and physical activity"[15]. The possible responses for the other primary outcome measure, the SF-36 physical functioning subscale, are "yes, limited a lot", "yes, limited a little" and "no, not limited at all" to questions on a range of physical activities. Such responses could be easily be artificially altered following a therapy like CBT for CFS.

The results were not that different with the GET cohort in the PACE Trial. Again the manuals predicted that the impairments and symptoms are reversible using the intervention[4,15]. The model said there was no reason participants should not be able to get back to full functioning. Deconditioning was posited to be an important maintaining factor. However, GET did not result in an improvement in fitness, as measured by the step test. GET did result in a small improvement on the six minute walking test to a final distance of 379 metres, or 35 metres more than the SMC-only group[7]. However, as Knoop and Wiborg commented in an accompanying editorial in Lancet Psychiatry: "an increase in distance walked during a test situation without an increased fitness suggests that patients walk more because of a change in cognitive processes (eg, daring to do more or an increased self-efficacy with respect to activity), not because of a change in physiological capacity”[16]. The result remained very poor given that normative data would suggest a group of similar age and gender should walk an average of 644 or so metres[17]. The distance walked remained comparable to people with many serious conditions[18-21], and considerably worse than the distance walked by healthy elderly adults[22,23], despite the PACE trial cohort having a mean age of only 40[8]. Also, to be allowed entry into CFS research studies such as the PACE Trial one can not have a range of chronic illnesses so with genuine recovery one would expect results comparable to healthy people[8].

As with CBT, measures relating to employment showed no improvement following GET in days of work missed, which remained very high, nor a reduction in levels of benefits (financial support from the state) or payments from insurance companies[9].

These results are in line with an audit of Belgian rehabilitation centres for CFS offering CBT and GET[24-26]. Some improvements in subjective measures were found, but there was no improvement in the results of the exercise test and hours in employment actually decreased.

Probably the main contribution of the PACE Trial has been to add to a growing body of evidence that while CBT and GET for CFS have resulted in some changes on subjective measures, they haven't lead to improvements on objective measures.

Competing interests: I am a committee member of the Irish ME/CFS Association and perform various types of voluntary work for the Association.

(continues)

On 2015 Oct 15, Radboudumc Psycho-Oncology Journal Club commented:

This interesting commentary, which has important implications for the field of psycho-oncology, was discussed by our Journal Club members on 14th of October, 2015 and generated a lively discussion and the following comments and questions:

• 1) After reading your commentary, we concluded that it would be worthwhile to incorporate lifestyle variables in psychosocial intervention programmes on tertiary prevention, to further explore the relationship between psychological outcomes and health behaviours. Our group is based within an academic medical centre and there was debate within our group about the role of psycho-oncologists should take. Some members of our journal club believed that psycho-oncology as a discipline should focus more on collaboration with public health, health psychology and other disciplines working in the area of primary and secondary cancer prevention.. However, other members believed health behaviour change is the area of expertise of health psychologists not psycho-oncologists. Furthermore, in light of resource constraints, psycho-oncology professionals should focus on the core business of providing care to those affected by disease and ill-health in medical settings.<br>
• 2) We welcomed the proposal of The Expanded Model of Research in Psycho-Oncology and congratulate the authors for stimulating debate over the role of psycho-oncology. As for a suggestion, we would have liked it if the new conceptional model had an additional box with factors relevant to secondary prevention after cancer diagnosis. For example, psycho-oncologists are well placed to conduct activities that focus on the secondary prevention of psychological problems (e.g. depression, anxiety) or which might prevent a new disease recurrence following a diagnosis of cancer (e.g via lifestyle change programmes for cancer survivors). We also believed that psychological screening within oncology settings can be conceptualised as a secondary prevention activity.
• 3) We believe that psycho-oncology as a discipline, if it is to be involved with primary and secondary prevention at all, might focus on the three non-tumor specific health behavior changes with the biggest impact smoking, diet and exercise.
• 4) When it comes to medical psychologists working in psycho-oncology, essentially it comes down to the question whether a hospital is a “health valley” or a place in which we treat those affected by disease. Primary and secondary prevention implies the “health valley” model, which needs a much larger paradigm shift of medicine in general rather than in psycho-oncology alone. It will also require increased funding for hospitals to enable this to occur.<br>
• 5) We identified systemic barriers such as the organisational separation of hospitals and public health programmes in many jurisdictions. We concur with the authors that further investment is needed in the training of psycho-oncologists to ensure they learn how to apply their skills to preventive health and have more opportunities to gain work experience in preventive health care settings.

On 2015 Oct 02, Robert Groom commented:

Thank you to the authors for this tribute to an outstanding perfusionist that invested herself in improving her field. The commentary from her friends and contemporaries exemplifies the importance of our relationships with colleagues in professional societies and how through those relationships we are able to make the World a better place.

On 2015 Dec 06, Gwinyai Masukume commented:

Unacknowledged important limitation - the effective sample size of 229 was insufficient to detect a statistically significant association.

Dean and colleagues find that there was a deficit of 229 pregnancies among women who were in the first trimester when the mass shooting at Port Arthur occurred and that by sex there was no statistically significant differential loss. They estimate that for every 100 females, 107 males were lost.

Although Dean and colleagues did not find a statistically significant difference by sex, their sample size of 229 was insufficient to detect the difference given the small effect size Austad SN, 2015.

A rigorous study Orzack SH, 2015 demonstrated that normally, in the absence of exogenous stressors such as that caused by the Port Arthur calamity, more female embryos/fetuses are lost during the first trimester, approximately 100 females for every 97 males.

In conclusion, the results presented by Dean and colleagues show excess male loss during a period when more female loss is expected. Thus, the small sample size of 229 should have been mentioned as an important limitation of the paper and a formal power calculation was apt.

On 2015 Sep 17, Daniel Himmelstein commented:

Thanks Dr. Levine for your thoughtful response. As you mention, the practices I criticize in your discovery phase are not unique to your study. Your study caught my attention because of its remarkable finding that such a small sample yielded a highly predictive and robust classifier for such a complex phenotype. Hopefully, others will benefit from our discussion here.

Additionally, I agree that replication grants researchers the freedom to discover as they wish. Suboptimal model training should not cause "type I" replication error, if the replication dataset is independent.

However, a replication p-value alone is insufficient to identify the probability of a model being true. This probability depends on the plausibility of the model. Since I think that the odds are low that your study design could produce a true model, I require extraordinary evidence before accepting the proposed PRS model. I think your replication provides good evidence but not extraordinary.

Follow-up studies on different populations will be important for establishing extraordinary evidence. I think it would be helpful to specify which allele is minor for each PRS SNP: my impression is that minor alleles are sample not population specific.

On 2015 Sep 16, Morgan E Levine commented:

Daniel Himmelstein, thank you for your comments. I will try to address them to the best of my ability.

We acknowledge that the sample size is very small, which we mention in our limitations section of the paper. Because we are studying such a rare phenotype, there is not much that can be done about this. “Long-lived smokers” is a phenotype that has been the subject of a number of our papers, and that we think has strong genetic underpinnings. Despite the small sample size, we decided to go ahead and see if we could detect a signal, since there is evidence to suggest that the genetic influence may be larger for this phenotype compared to many others—something we discuss at length in our introduction section.

To the best of our knowledge the finding that highly connected genes contain more SNPs, has not been published in a peer-reviewed journal. Therefore, we had no way of knowing or evaluating the importance of this for our study. Similarly, we used commonly used networks and do acknowledge the limitations of these networks in our discussion section. The network you link to was not available at the time this manuscript was accepted.

We acknowledge the likelihood of over-fitting in our PRS, which is probably due to our sample size. This score did validate in two independent samples. Therefore, while it is likely not perfect, we feel that it may still capture some of the true underlying signal. We followed standard protocol for calculating our score (which we reference). In the literature there are many examples of scores that have been generated by linearly combining information from SNPs that are below a given p-value threshold in a GWAS. While, not all of these replicate, many do. Our study used very similar methods, but just introduced one additional SNP selection criteria—SNPs had to also be in genes that were part of an FI network. I don't think this last criteria would introduce additional bias that would cause a type I error in the replication analysis. However, we still recognize and mention some of the limitations of our PRS. We make no claim that the score is free from error/noise or that it should be used in a clinical setting. In fact, in the paper we suggest future methods that can be used to generate better scores.

We feel we have provided sufficient information for replication of our study. The minor alleles we used are consistent with those reported for CEU populations, which is information that is readily available. Thus, the only information we provide in Table S2 pertain to things specific to our study, that can't be found elsewhere. Lastly, the binning of ages is not 'bizarre' from a biogerontology and longevity research perspective. A number of leaders in the filed have hypothesized that the association between genes and lifespan is not linear (variants that influence survival to age 100+ are not the same as variants that influence survival to age 80+). Thus, using a linear model would not be appropriate in this case and instead we selected to look at survival by age group.

On 2015 Sep 15, Daniel Himmelstein commented:

I have several concerns with the discovery phase of this study. Specifically:

Underpowered: The sample size (90 cases) is insufficient for a complex phenotype where effect sizes are small. My own research, Himmelstein DS, 2015, does not consider GWAS with under 1000 samples because any findings are likely to be false (Sawcer S, 2008). The pathway-based prioritization attempts to alleviate the underpowered study design but suffers from the following two criticisms.

SNP abundance confounding: Genes were selected for the network analysis if they contained any SNPs with p < 5×10^-3. Therefore, genes containing more SNPs were more likely to get selected by chance. Since genes with more SNPs appear more frequently in curated pathway databases, the enrichment analysis is confounded. A permutation test that shuffles case-control status and recomputes SNP p-values would prevent SNP abundance confounding. However, this does not appear to be the permutation test that was performed.

Limited network relevance: The network analysis uses only curated pathway databases. These databases are heavily biased towards well-studied genes as well as being incomplete. In our recent network that includes more curated pathways than Reactome FI, only 9,511 genes are in any curated pathway. In other words, the majority of genes aren't curated to a single pathway and hence cannot contribute to this study's prioritization approach.

Overfitting: The polygenic risk score (PRS) was ruthlessly overfit. The PRS perfectly discriminated the 90 long-lived smokers from the younger smokers. The authors don't appear to appreciate that the performance is due to overfitting and write:

Results showed that the score completely accounted for group membership, with no overlap between the two groups.

Not only were scores significantly higher for the long-lived group, but scores also appeared to be more homogeneous.

Egregious overfitting is guaranteed by their PRS approach since 215 logistic regressions are fit, each with only 90 positives and without regularization or cross-validation. When a model is overfit on training data, its perfomance on novel data diminishes.

Unreplicable: Table S2 of the supplement does not specify which allele is minor for each SNP. Therefore, the PRS computation cannot be replicated by others.

Given these issues, I find it unlikely that the study found a reliable genotype of longevity. Rather, I suspect the successful validation resulted from confounding, p-value selection bias, or an implementation error.

Finally, the binning of continuous outcomes, primarily age, is bizarre. The binning serves only to reduce the study's power, while providing much room for unintended p-value selection bias.

Update 2015-09-15: I am not suggesting any misconduct, negligence, or intentional bad practices. Rather the methods are clearly described and the validation quite impressive and seemingly honest. I believe the study makes a valuable contribution by proposing a genotype of longevity, which future studies can confirm or deny.

Update 2015-09-19: I replaced "p-hacking" with "p-value selection bias". My intended meaning is the greater investigation and preferential publication granted to more significant findings.

On 2017 Oct 21, Anju Anand commented:

This article was discussed at our #rsjc. This captures the summary: http://www.thelancet.com/journals/lanres/article/PIIS2213-2600(15)00523-8/fulltext

On 2016 Jun 13, Stuart Phillips commented:

Please see a letter to the editor regarding the results from this paper: http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=10353765&fileId=S000711451600221X

On 2016 Apr 30, Stuart Phillips commented:

None

On 2016 Apr 30, Stuart Phillips commented:

None

On 2015 Sep 18, Alejandro Diaz commented:

Prepubertal gynecomastia is a very rare condition and there have been an unexpected number of cases reported in the Miami area. This case report highlighted three cases. However, we have seen many other children in our practices with prepubertal gynecomastia or thelarche who were similarly exposed to the lavender-containing cologne described in our report. Unfortunately, the evaluation and follow up for these children was not as in-depth as in the cases that we presented in the article, which included chemical analysis of the cologne product.

In Politano VT, 2013 article, uterotrophic assay, performed in immature female rats, was used to evaluate estrogenic activity in vivo. However, these rats were only treated with lavender oil for a period of 3 days, whereas the children we reported on were exposed to the lavender-containing cologne for several years. As described by Henley DV, 2007, the estrogenic activity of lavender is weak, which may explain why there was not uterotrophic effects on these rats.

You mentioned that the reason Henley found that lavender and tea tree oil activated the estrogen receptor was due to the use of polystyrene in their test system as opposed to the use of glass. Both Ohno K, 2003 and Fail PA, 1998 could not demonstrate estrogenic response from polystyrene. Therefore, this explanation is not substantiated. Moreover, there was an estrogenic response to the lavender and tea tree oils, but not to the control substance, despite having used the same test system containers in both conditions.

While it is true that there are additional components in many lavender preparations, we have not found prepubertal gynecomastia to be associated with other non-lavender-containing colognes or topical products. Thus, lavender itself is a logical and well-founded explanation for the physical findings in our patients.

Regarding tea tree oil, I previously evaluated a patient who was exposed over a period of several years to numerous products containing tea tree oil, including shampoos, toothpaste, detergents, home cleaning products, and melaleuca essential oil for minor cuts, burns, and other skin issues. He developed severe prepubertal gynecomastia that improved upon discontinuation of the exposure to these products. As I did in the cases of lavender exposure, I conducted a complete endocrine evaluation and his hormone levels were all within normal ranges. I did not publish this case at the request of his parents, who were in the tea tree oil industry. As clinicians, if we find patients with prepubertal gynecomastia who have been exposed for prolonged periods of time to substances known to activate the estrogen receptor, it provides substantial evidence of causality.

The possibility exists that there are contaminants that have estrogenic activity in these preparations. However, it is the responsibility of the industry that the products as sold are safe for human exposure.

It is, however, of utmost importance to publish these case reports to offer the balance you refer to, and to deepen the scientific knowledge base and protect consumers from unnecessary harm. Clinicians must be informed and use their clinical judgment to determine what is best for their individual patients.

Alejandro Diaz, M.D. and Marco Danon, M.D. Pediatric Endocrinologists

On 2015 Sep 14, Tony Larkman commented:

It is disappointing that the authors didn’t include or at least refer to the following two articles in this paper: 1. Politano VT et al (2013), Uterotrophic Assay of Percutaneous Lavender Oil in Immature Female Rats; International Journal of Toxicology; 32(2): 123-9 (http://www.ncbi.nlm.nih.gov/pubmed/23358464) 2. Carson CF et al. (2014), Lack of evidence that essential oils affect puberty; Reproductive Toxicology; 44:50-1 (http://www.ncbi.nlm.nih.gov/pubmed/24556344)

The first paper largely exonerates lavender oil through the ’gold standard’ uterotrophic assay while the second hypothesizes a mechanism that may be causal in incidences of both gynecomastia and premature thelarche as well as the ‘in vitro’ findings of Henley et al (2007) where they in fact used polystyrene in their test system and not glass. Based on these it is not impossible to conceive that other endocrine disruptors may have been inadvertently present in the material tested.

A further, as yet untested, hypothesis for these manifestations is the high incidence of adulteration in essential oils. The incidence of adulteration in TTO is remarkably high as reported in Wong YF et al. (2014) Enantiomeric distribution of selected terpenes for authenticity assessment of Australian Melaleuca alternifolia oil; Industrial Crops & Products; 67: 475-83 (http://ijt.sagepub.com/content/early/2013/01/24/1091581812472209) where more than 50% of 43 commercial samples tested failed to comply with the proposed chiral ratios. Of 15 commercially sourced samples in the European Union 73% of these showed significant differences in chiral abundances while TTO from both North America and Asia also displayed similar results where ≥50% of the tested samples did not match the expected results. In material of Chinese origin the incidence rises to 100%. An extraordinary range of compounds never found in pure TTO has been coincidentally detected so it has been further hypothesized that this, along with the likelihood that the material used is heavily oxidized, may be significant source of problems such as allergic contact dermatitis attributed to the use of TTO as well as conditions related to endocrine disruption.

It is disappointing for the tea tree oil industry as a whole that TTO continues to be mentioned as a potential endocrine disruptor without a more balanced view being presented when there are clear indications available in the literature that the original proposal by Henley et al in 2007 may be flawed. The fact that TTO was not present at all in any of the material tested is also disappointing as TTO can in no way be implicated yet its mention continues to promulgate the link and implicate TTO unfairly.

On 2018 Jan 07, Mark Milton commented:

In defense of the authors, their article was first published online on September 8, 2015 and appeared in the Issue published on April 1, 2016. The BIAL incident occurred in Jan 2016, i.e. after this article was published and hence there couldn't have been any discussion of the BIAL incident in this article.

On 2016 Jun 08, Christopher Southan commented:

Some discussion of the BIA-10-2474 fatality and AEs would have been pertinant, even though it was more recent than the cut-off date https://cdsouthan.blogspot.se/2016/01/the-unfortunate-case-of-bia-10-2474.html

On 2015 Dec 07, Sanjay Kumar commented:

We appreciate the interest in our work. While we are unable to comment on the internalization mechanisms of the SmartFlare reagents based on our data, our independent RT-PCR measurements (Fig. 3) are consistent with the SmartFlare data and confirm that matrix stiffness and ligand density regulate miR18a levels in this system.

On 2015 Nov 18, Raphael Levy commented:

This article uses the SmartFlare technology to detect miR18a. I would be most interested in the authors' opinion on how these probes escape endosomes to report on miR18a level. In our experience, they do not and fluorescence increase is most likely correlated with degradation in endosomes. It might be quite plausible that endocytosis "is non-linearly regulated by matrix stiffness and fibronectin density in glioma cells."

On 2015 Nov 12, Christopher Sampson commented:

It seems that personalised screening for breast cancer is a sheep in wolves' clothing (rather than the author's suggestion of the reverse).

The purpose of risk-based screening is to redirect resources to those at the greatest risk of disease; those in the greatest need. I have argued elsewhere that it is possible to prioritise screening in this way, in order to maximise the benefits of screening within a given budget.

Feig's commentary suggests that the terms "risk-based screening" and "personalized screening" are being misappropriated. That may well be true. However, the criticisms therein relate to very specific uses of these terms and to particular guidelines (I do not see these claims holding true more broadly). It does not then follow that 'risk-based screening' and 'personalisation' are wolves, and indeed the way they have been clothed does not appear desirable. They are sheep in wolves' clothing!

We should not -- as I hope Feig would agree -- allow the unsavoury dressing-up of these ambitious developments waylay further research into risk-based screening.

On 2017 Feb 18, Clive Bates commented:

Erroneous interpretations have been placed on these results and overly confident conclusions drawn from very small numbers of imperfectly characterised teenagers. The headline recommendations were based on the behaviour of six out of 16 baseline e-cigarette users in a sample of 694 adolescents deemed not to be susceptible to smoking. Large conclusions drawn from small numbers should always be a cause for caution, as discussed in this article about this study by Five Thirty-Eight:

Ignore The Headlines: We Don’t Know If E-Cigs Lead Kids To Real Cigs by Christie Ashwandan, 11 September 2015

One should expect the inclination to use e-cigarettes to be caused by the same things that cause an inclination to smoke - they are similar habits (the former much less risky) and it is quite likely that those who used e-cigarettes first would have become smokers first in the absence of e-cigarettes - a concept known as shared liability. A range of independent factors that create a common propensity to smoke or vape, such as parental smoking, rebellious nature, delinquency etc. explain the association between vaping and smoking incidence but without this relationship being causal.

The authors try to address this by characterising teenagers non-susceptible to smoking if they answer “definitely no” when asked the following: “If one of your friends offered you a cigarette, would you try it?” and “Do you think you will smoke a cigarette sometime in the next year?”. The study concentrates on this group.

This is not a foolproof way of characterising susceptibility to smoking, which in any case is not a binary construct but a probability distribution. Nor is susceptibility a permanent condition for any young person - for example, if a teenage girl starts seeing a new boyfriend who smokes that will materially changes her susceptibility to smoking. The fact that some were deemed unsusceptible to smoking but were already e-cigarette users is grounds for further unease - these would be more likely to be the teens where the crude characterisation failed.

It is a near-universal feature of tobacco control research that the study presented is a wholly inadequate basis for any policy recommendation drawn in the conclusion, and this study is no exception:

These findings support regulations to limit sales and decrease the appeal of e-cigarettes to adolescents and young adults.

The findings do not support this recommendation, not least because the paper is concerned exclusively with the behaviour of young people deemed not susceptible to smoking and, within that group, a tiny fraction who progressed from vaping to smoking. Even for this group (6 of 16) the authors cannot be sure this isn't a result of mischaracterisation and that they would not have smoked in the absence of e-cigarettes. The approach to characterising non-susceptibility is far too crude and the numbers involved far too small to draw any policy-relevant conclusions.

But this isn't the main limitation. Much more troubling is that the authors made this policy recommendation without considering the transitions among young people who are susceptible to smoking - i.e. those more likely to smoke, and also those much more likely to use e-cigarettes as well as or instead of smoking. This group is much more likely to benefit from using e-cigarettes as an alternative to smoking initiation, to quit smoking or cut down or as a later transition as they approach adult life.

There are already findings (Friedman AS, 2015, Pesko MF, 2016, Pesko MF, Currie JM, 2016) that regulation of the type proposed by the authors designed to reduce access to e-cigarettes by young people has had unintended consequences in the form of increased smoking - something that should not be a surprise given these products are substitutes. While one may debate these findings, the current study makes no assessment of such effects and does not even cover the population that would be harmed by them. With these limitations, it cannot underpin its own over-confident and sweeping policy recommendation.

On 2015 Sep 15, Aadil Inamdar commented:

The very aim of a systematic review is to provide a critical summary of the available evidence. The authors of a systematic review may draw conclusions based on the collected studies on a particular topic. It is equally important to present to their readers an unbiased quality assessment of the included studies in the review, which is an integral part of a systematic review process. This process is what differentiates a systematic review from the rest and a missing quality assessment of included studies section brings it as par with the common narrative or a literature review. The risk of bias and quality assessment of studies included in this review is missing. The onus is on the researchers to provide clear, simple conclusions of studies that has been meticulously checked and rechecked for their validity. Otherwise, on the contrary, it only helps to pollute the current evidence.

It is important to note that;

• Systematic reviews are one of the highest levels of information in the hierarchy of evidence LINK
• Steps involved in a systematic review have been laid down by the Cochrane collaboration STEPS
• A major section involves assessment of quality and risk of bias, of the included studies
• Assessment not only helps in ascertaining validity of a particular study, but also gives a true estimate (over/under) of the intervention (or exposure) Assessment of quality and risk of bias

On 2015 Nov 14, Arnaud Chiolero MD PhD commented:

An enlightening review revealing the issues of universal lipid screening in children. Similar issues with the screening of other CVD risk factors in children (see www.publichealthreviews.net/content/36/1/9)

On 2015 Sep 10, David Keller commented:

Did droxidopa increase dyskinesias, blood pressure, heart rate & adrenergic side-effects?

In general, medications which improve rigidity and bradykinesia and other "off symptoms" in Parkinson disease (PD) patients also tend to worsen dyskinesias. If droxidopa was able to improve PD "off" symptoms without worsening dyskinesias, then this drug is a breakthrough. The effect of droxidopa on dyskinesias is a crucial outcome and should be reported in the abstract. Without this information, the study cannot be evaluated properly.

Droxidopa is a metabolic precursor of norepinephrine, so it is expected to result in adrenergic effects, such as increasing blood pressure, heart rate, insomnia, heart arrhythmias, constipation, etc. The presence or absence of such side-effects is of interest to clinicians and patients, and should also be reported in the abstract.

Addendum

On September 16, 2015, I received a very informative email from Shifu Zhao, MD, in which he stated that "adrenergic side-effects were not significantly increased comparing with placebo group or baseline data at droxidopa dose 600mg/day" including "blood pressure, insomnia, palpitation, ECG" [1].

In addition, he assured me that add-on droxidopa therapy improved tremor and alternating motion of the hands in patients with moderate-to-severe Parkinson's disease, without worsening dyskinesias. [1]

I thank Dr. Zhao for supplying this information to me, and I hereby relay it to other Parkinson disease patients who read scientific abstracts without access to the underlying journal articles.

Reference:

1: Zhao S, Personal Communication by email, Received 9/16/2015 in reply to an earlier email inquiry.

On 2017 Mar 13, Atanas G. Atanasov commented:

Excellent overview on the topic, I have featured this publication at: http://healthandscienceportal.blogspot.com/2017/03/the-stomach-in-health-and-disease.html

On 2015 Oct 26, Gustav van Niekerk commented:

It would be hard to over emphasise the intense selective pressure that pathogens place on host populations (as predicted by the Red Queen hypothesis). Our genome are littered with pseudonized genes formerly associated with immune-related functions. These ‘genomic fossils’ representing discarded immunological innovations that became redundant as our pathogens evolve strategies to defeat them, testifying to an ancient conflict that have been ranging on between host and pathogens. Sequencing genomes have repeatedly demonstrated that genes with immunological functions tend to be under selective pressure and tend to be highly polymorphic (Mother Nature is ‘diversifying’ the ‘portfolio’ of immunological strategies: a pathogens can overcome many immunological strategies, but not all of them simultaneously. This observed immunological heterogeneity in population is how a species ‘hedge it’s bet’ against pathogens). Host and pathogen/parasites occasionally demonstrate "mirror-image phylogenies", demonstrating the close coevolution as pathogens keep in step with host development. Collectively, such observations suggest that pathogens exerts immense evolutionary pressure and one of the biggest drivers for evolutionary novelty.

Consequently, we suggest a less exiting narrative in which an AIS evolved initially in response to pathogen stress. Similar to your work Corcos D, 2015, we also noted that AIS evolved in an aquatic environment. Sediment and marine environments typically have a higher pathogen burden (see references in < PMID: 25698354 >), thus suggesting a higher level of pathogen stress. However, as you point out, there are scepticism about whether the AIS represent a true immunological innovation, as invertebrates do not seem any worse-off than vertebrates in suffering from infections. In this regard, we would like to point out that the AIS could have been an initial innovation that was subsequently overcome by fast evolving pathogens. That is, the AIS could have evolved in response to pathogen stress, providing an initial benefit to vertebrates, but was subsequently overcome by pathogens. And now, regardless whether the AIS is currently an ‘immunological innovation’, we are stuck with it (for better or worse). Finally, it is not obvious that the AIS do not repeat a lasting innovation. After all, “[v]ertebrates are the dominant group of animals on the Earth, given their abundance, large body sizes and presence at the top of both aquatic and terrestrial foodwebs” Wiens JJ, 2015. How did vertebrates manage to survive (and indeed flourish) in a world initially dominated by invertebrates? An AIS may have some additional benefits that have helped vertebrates invade into niches previously occupied by invertebrates. How AIS could provide a benefit that is (a) only applicable to vertebrates and (b) does not involve an enhancement of immune potency is currently an open question.

(PS we unfortunately do not have institutional access to the article from Burnet FM. We will shortly be sending a letter to the editor, commenting on your Corcos D, 2015 interesting article)

On 2015 Oct 20, Daniel Corcos commented:

This paper is interesting for examining the relationship between the adaptive immune system and the vascular system. However it does not explain such a complexity with no obvious advantage. For instance, immunodeficiency in zebrafish does not result in a dramatic change in viability. This can be interpreted in the words of Hedrick as an indication that the AIS is "not so superior"(1) or in the words of Burnet that it is "related to something other than defence against pathogenic microorganisms."(2) I have proposed (3) that its origin was related to intraspecific predation (cannibalism).

1)Hedrick SM. Immune system: not so superior. Science 2009;325:1623–4.

2)Burnet FM. “Self-recognition” in colonial marine forms and flowering plants in relation to the evolution of immunity. Nature 971;232:230–5.

3)Corcos D. Food-Nonfood Discrimination in Ancestral Vertebrates: Gamete Cannibalism and the Origin of the Adaptive Immune System. Scand J Immunol. 2015;82(5):409-17.

On 2017 Jul 04, Randi Pechacek commented:

Alex Alexiev wrote a blog post on microBEnet, mentioning this paper, arguing the importance of equipment cleanliness for food quality.

On 2016 Jan 20, thomas samaras commented:

A very interesting study. However, the conclusion that overweight may not be harmful at older ages is not consistent with what we know about the dangers of increasing BMI and various health parameters. Lamon-Fava showed many years ago that virtually all biological parameters get worse with increasing BMI in a linear trend; e.g., BP, fibrinogen, Apo B, Apo A, and HDL. In addition, the New England Centenarian Study found that almost all men who reached 100 years of age were lean. The Okinawan centenarians are also very lean. We also know that pre-western people with low chronic disease throughout their lives also tend to be quite lean in old age; e.g., people of Kitava (near Papua New Guinea).

Maier, Van Heemst and Westendorp found that shorter 85 and 90 years were more likely to reach advanced ages. These shorter, longer living individuals also had longer telomeres than taller people as found in the Guzzardi, et al. study. Perhaps confounding occurred by a number of lean people who were developing health problems without current symptoms, smokers, or poorly nourished individuals. This confounding may explain the results in regard to the positive correlation of increased % of fat mass and cholesterol between the start and end of the study.

On 2018 Jan 19, Natalie Clairoux commented:

Free version of this article available at http://hdl.handle.net/1866/19740

On 2016 Feb 10, Anders von Heijne commented:

In a recent case with a very large testicular epidermoid cyst we found an ADC-value of 0.544. IT seems that testicular epidermoid cysts have distinctly lower ADC-values than their intracranial counterparts.

On 2018 Jan 19, Natalie Clairoux commented:

Free version of this article available at http://hdl.handle.net/1866/19736

On 2015 Dec 08, S. Celeste Morley commented:

Thank you very much for your interest in and comment upon our work. The safety and efficacy of PCV in preventing and reducing the incidence of invasive pneumococcal disease is unquestioned and unequivocally supported by all literature. The mechanisms by which PCV protects are multifactorial. PCV generates an immune response that protects against invasive disease, if not colonization. We did not address incidence of IPD in this study. PCV also results in decreased carriage prevalence of the disease-causing serotypes covered by the vaccine, and thus shifts serotype prevalence without necessarily altering overall carriage prevalence. Our study simply reported overall carriage prevalence and the antibiotic susceptibility profiles of carriage isolates; we did not report serotypes. Our results finding that overall likelihood of colonization with any pneumococcal serotype is not affected by PCV is in line with other larger studies (e.g. Zuccotti et al Vaccine. 2014 Jan 23;32(5):527-34. Zuccotti G, 2014) Thus, our study is not in conflict with any of the studies referenced above, which clearly show a decrease in colonization with vaccine-covered serotypes.

On 2015 Dec 07, Manoochehr Karami commented:

Manoochehr Karami, PhD, Research Center for Health Sciences and Department of Epidemiology, School of Public Health, Hamadan University of Medical Sciences, Hamadan, Iran.

In an interesting study published recently, Julie Y. Zhou et al.(1) highlighted the prevalence of nasopharyngeal pneumococcal colonization among children in the greater St. Louis hospital. Authors have stated "pneumococcal conjugate vaccine (PCV) did not alter prevalence" of nasopharyngeal carriage. World Health Organization indicated that after PCV introduction, both targeted and non-targeted vaccination population were affected by direct and indirect effects of PCV immunization(2). Moreover, published studies (3-7) supported the changes of epidemiological profile of Streptococcus pneumonia related diseases transmission and nasopharyngeal carriage even among those individuals who were not immunized against Streptococcus pneumonia. Accordingly, it seems Zhou JY et al. interpretations based on their findings is in question and might be affected because of potential selection bias while enrolled participants. Rational for such selection bias is the catchment area of St. Louis hospital and potential differences between study participants and non-participants. Although they have excluded some patients, however this strategy does not guarantee the representativeness of their own work. Generally speaking, better explanation for Zhou et al findings is selection bias. In conclusion, lack of generalizability of this study findings should be considered by policy makers and interested readers.   References: 1. Zhou JY, Isaacson-Schmid M, Utterson EC, et al. Prevalence of nasopharyngeal pneumococcal colonization in children and antimicrobial susceptibility profiles of carriage isolates. International Journal of Infectious Diseases;39:50-52. 2. World Health Organization. Measuring impact of Streptococcus pneumoniae and Haemophilus influenzae type b conjugate vaccination. WHO Press, Geneva, Switzerland, 2012. 3. World Health Organization.Pneumococcal vaccines WHO position paper – 2012.Wkly Epidemiol Rec. 2012;87:129-244. . 4. Lehmann D, Willis J. The changing epidemiology of invasive pneumococcal disease in aboriginal and non-aboriginal western Australians from 1997 through 2007 and emergence of nonvaccine serotypes. Clinical Infectious Diseases. 2010, 50(11):1477–1486. 5. Pilishvili T, Lexau C. Sustained reductions in invasive pneumococcal disease in the era of conjugate vaccine. The Journal of Infectious Diseases, 2010, 201(1):32–41. 6. Davis S, Deloria-Knoll M, Kassa H, O’Brien K. Impact of pneumococcal conjugate vaccines on nasopharyngeal carriage and invasive disease among unvaccinated people: Review of evidence on indirect effects. Vaccine.2014;32:133-145. 7. Karami M, Alikhani MY. Serotype Replacement and Nasopharyngeal Carriage Due to the Introduction of New Pneumococcal Conjugate Vaccine to National Routine Immunization. Jundishapur Journal of Microbiology 2015;8.

On 2015 Sep 10, George McNamara commented:

Over at PubPeer (where this will be mirrored to) there were questions as to why I consider the image quality of figures 1 and 2 are so bad. See the Ma et al 2013 paper - also published in PNAS. http://www.pnas.org/content/110/52/21048.figures-only

Was not that hard two years ago to publish high quality images. Nor was it that hard for Deng et al to publish good quality figure 3 or supplemental files.

On 2015 Sep 09, George McNamara commented:

The good news of this paper: open access. The bad news: figures 1 and 2 are among the worst light microscopy image quality I have seen in a long time. I don't know if PNAS completely dropped the ball on image quality (fig 3 does look ok - also the authors do get to approve or fix the page proofs), or something to do with the authors. When I was a graduate student, I thought the difference between a cell biologist and a biochemist was that a cell biologist (or their lab) had microscopes and knew how to use them, and biochemists did not and did not care. Alternative hypothesis: PNAS figures are managed by biochemists. Considering the authors are down the hall from Eric Betzig, who won a prize of high resolution microscopy (and has since published two Science papers on more high res microscopy), they could have acquired and published better images. Their centromere, telomere repeats, MUC1 and MUC4 tandem repeat targets were all previously published (and cited by them) with somewhat better image quality.

On 2015 Oct 01, Badrul Arefin commented:

Dear readers,

We found two mistakes either as a missing vertical axis title or a typo in the following figures in this published article. In figure 7A, vertical axis title is missing when figure re-arrangement is performed in a revised version. Here it should be "Eclosure (%)". In figure 8I, N-NAME in chart area should be replaced with "L-NAME". Please excuse us for these mistakes. However, both of these are correctly-written/correct in the respective figure legends and elsewhere in the article.

Sincerely,

Badrul Arefin, On behalf of all authors.

On 2015 Sep 01, Friedrich Thinnes commented:

From my point of view the dust-raising effect of antibody 6E10 fuels the idea that plasmalemma-integrated VDAC-1 works as a receptor of amyloid Aß.

It has been shown that docking of amyloid Aß via GxxxG motif interaction to cell membrane-standing VDAC-1 opens the channel, and this way induces neuronal cell deaths accumulating over time.

Whenever critical brain regions and all redundant structures are affected Alzheimer´s Dementia appears.

For details see Friedrich P. Thinnes, Biochimica et Biophysica Acta 1848 (2015) 1410–1416 and www.futhin.de

On 2015 Sep 11, Chanin Nantasenamat commented:

Thanks Christopher for the reminder. Actually, we also wanted to make the dataset publicly available. Thus, please find the dataset used in this study (in Excel file format) from the following link:

http://dx.doi.org/10.6084/m9.figshare.1539580

On 2015 Sep 05, Christopher Southan commented:

This is a usefuly detailed study but direct availability of the 2,937 structures should have been a condition of publication. I note they were selected from BindingDB but also filtered post-download (and will change via updates). The authors should thus please surface at least the SMILES on figshare or other open acess option (Sep 14th - good 2 c the file :)

On 2015 Oct 23, Miguel Lopez-Lazaro commented:

This article provides a model to explain that virtually every cancer cell within a tumor often contains the same core set of genetic alterations, with heterogeneity confined to mutations that emerge late during tumor growth.

In my opinion, there is a simpler explanation. Recent evidence strongly suggests that cancer arises from normal stem cells. If cancer arises from normal stem cells, all the mutations occurring in these cells before becoming malignant (cancer stem cells, CSCs) will be found in all their progeny, that is, in all the tumor cancer cells. Some tumor cells may lack some of these mutations if they lose during cell division the chromosomes or pieces of chromosomes that bear these DNA alterations. The mutations arising during the self-renewal of CSCs will be found only in the tumor populations derived from these malignant stem cells. In addition to self-renewing, CSCs generate progenitor cancer cells, which divide and produce the bulk of cancer cells within a tumor. The mutations found in few tumor cancer cells probably occur during the division of these progenitor cells. In some cases, the tumor cancer cells may arise from more than one normal stem cell. In these cases, not all the cancer cells within a tumor will share the same core set of genetic alterations (1). Normal and malignant stem cells are defined by their self-renewal capacity and differentiation potential, and have a natural ability to migrate.

(1). Lopez-Lazaro M. Selective amino acid restriction therapy (SAART): a non-pharmacological strategy against all types of cancer cells. DOI: 10.18632/oncoscience.258

On 2015 Aug 28, Tom Kindlon commented:

Minimal change on only objective outcome measure:

On the only objective measure, the activity monitor, at 52 weeks (compared to baseline): the MRT group had increased by 5.8% and CBT group had increased by 6.6%.

There was no control group. One would expect a no therapy CFS group would likely on average increase their activity a little also in 52 weeks esp. in the early years of the illness and/or in the first year or two after diagnosis.

On 2015 Oct 03, Prashant Sharma, MD, DM commented:

The affiliation for Drs. Varma N, Sachdeva and Sharma should read as "Department of Hematology" and not "Hepatopathology".

On 2015 Sep 08, Alexandra Alexiev commented:

This article was written about in a blog post at: http://microbe.net/2015/09/08/antimicrobial-resistance-countermeasures-in-veal-farming/

MicroBEnet is a microbiology of the built environment blog funded by the Sloan foundation to communicate about scientific research in the field.

On 2015 Nov 26, Lydia Maniatis commented:

We can get a sense of the intellectual poverty of this study simply by reading the "Conclusions" section:

"Our experiments show that luminance edges play a central role in White's illusion. The illusion seems to be predominantly caused by the luminance edge between the test patch and its background bar, while the edge contrast to neighboring bars is largely ignored. The effect of contour adaptation on White's illusion could not be replicated by spatial filtering models, which adds further evidence against the adequacy of such models as a mechanistic explanation of White's illusion in particular, and lightness perception in general. Our results highlight the importance of further investigating the question of how surface lightness is computed from edge contrast. "

There is no content here because:

1) There is no alternative to the idea that "luminance edges play a central role in White's illusion." Both sides of the display are the same, other than the (perceived) location of the targets as lying on white or on black. Note that there is NO REFERENCE to the authors' orientation claims.

2) The inadequacy of a priori inadequate models that weren't even properly tested (assuming they could be) is not an argument for anything.

3). The statement that "Our results highlight the importance of further investigating the question of how surface lightness is computed from edge contrast" is meaningless. If there are questions to be answered, this study did not address them nor make them seem any more interesting.

The ad hotness of the authors' orientation claims is not only refuted by other displays, such as the Benary cross, but can also be refuted by versions of White's illusion in which the edges of the targets are curved so as to collectively be consistent with, e.g. an interrupted circle. Try it. Such a display, for me, makes more evident the fundamentally bistable character of the lighter-looking group of targets. They can either appear to be part of an opaque surface that lies behind the white stripes, on an amodally-completed black background, or they can appear to form part of a transparency that passes over the black and white stripes. The transparency aspect of White's illusion, which has been noted before, and is noticed by naive observers, is, of course, never touched on in this paper.

On 2015 Nov 08, Lydia Maniatis commented:

This article should probably not be read from the beginning, but from the end – from the last section of the discussion, titled “Difficulties with matching tasks.” The major theoretical problems discussed earlier seem moot once you appreciate that problems with method render the data virtually worthless..

The stimulus effects were not robust but, rather, highly ambiguous and liable to produce theoretically relevant effects that the authors did not bother to consider. This led to highly variable responses that they could not interpret. The authors try and put the blame for their data problems on the lightness matching technique itself, but the fault lies in them: If a task (or a stimulus) is not fit for purpose, it is the fault of those who chose it when it fails to deliver.

The problem is, in fact, the stimuli, which, again, were highly unstable and often did not produce the percepts that the authors predicted and which needed to arise reliably in order to allow them to validate their predictions. Four of ten observers, we are told, did not even perceive White's illusion! These observers were “following some strategy, but [our data] does not allow us to understand what exactly they are doing.”

In general, there was “large variability across and within observers....With simple stimuli of the kind used here, the perceived lightness of different image parts can change over time....In that case, the mean across trials may not be a good indicator of subjective experience [i.e. perception].” So the authors do not really know what their subjects are perceiving.

It seems, further, that the “simple stimuli” produced perceptual phenomena that the authors were or should have been aware are possible – either through the literature or simply on the basis of looking at their stimuli (the least a perception scientist should be doing when planning an experiment is to notice obvious effects of their stimuli) - but did not take into consideration: “Ekroll and Faul (2013) observed an additional transparent layer for similar stimuli...” This might “explain the inconsistent results...” So the “simple stimuli” could produce complex percepts - precepts that, for these authors simply amounts to noise.

In addition: “...observers sometimes selected match lightnesses that were outside the luminance range spanned by the grating. This is surprising....” They really have no idea what their data mean.

The authors' stunning conclusion: “This is just one further example that lightness matching is not such an easy and straightforward task as it might appear on the surface.” They don't seem to realize that no method is secure if your stimuli are ambiguous and unstable and you have not inspected them and factored the potential effects into your theory and method.

Unfortunately, lesson seems not to have been learned, as revealed by the final discussion section of Betz, Shapley, Wichmann and Maertens, 2015, which similarly describes devastating methodological problems as an afterthought. Why are reviewers setting such an obviously low bar for publication?

On 2015 Aug 30, Lydia Maniatis commented:

The authors of this study construct an untenable, ad hoc account of a myopic observation.

The observation is that the contrast effect in White's illusion “is largely determined by edge contrast across the edge orthogonal to the grating, whereas the parallel edge has little or no influence.”

This is a correct literal description of White's illusion. If we had no knowledge of any other contrast illusions, we might overgeneralize from this and conclude that, in general, orthogonal edges produce contrast effects while parallel edges do not. But we do know more. We know that contrast effects are not tied to edge orientation in this way. Consequently we would not reasonably attempt to construct a neural model of contrast based on a principle of “Orthogonal edges produce contrast, parallel edges don't,” since such an ad hoc model, shrink-wrapped to fit White's illusion, would instantly be challenged and falsified by any number of other contrast effects. However, this is precisely what Betz et al propose. (Not to mention that, being “low-level,” the authors' proposed mechanism would also have to be compatible with every other “low-level” as well as every “high-level” effect, e.g. transparency effects, since information from the early parts of the visual system is the basis for the high-level effects.)

In addition, the authors tailor their account to a very weak version of White's illusion (as the authors show, its effect is comparable to the classic slc demo), composed of single target bars rather than columns of aligned bars. The latter produce a much larger lightness differences as well as transparency effects (in the case of the lighter-seeming bars). Does the proposed model work for this classic version as well? Does it work for the classic simultaneous contrast illusion? Does it work for round targets on square backgrounds? Does it explain the Benary cross? If not, then how does such an account contribute to theoretical or practical understanding of the phenomenon (lightness contrast) under consideration? What are the chances that the visual system possesses a mechanism especially for producing a version of White's illusion?

As far as their experimental observations, the authors take a little too much credit for themselves when they say that their experiments have shown that “not all luminance borders that enclose a surface are treated equally in White's illusion.” This is an unnecessarily narrow framing of a many-decades-known, fundamental fact of lightness perception.

If there is one safe statement we can make about simultaneous contrast, it is that it is closely correlated with the appearance of a figure-ground relationship. Specifically, the surface that appears as figure lightens against a darker (apparent) background, and darkens against a lighter one. Mere (apparent) adjacency does not produce contrast. So when the authors claim to have shown that “the luminance step between the test patch and the grating bar on which it is [apparently!] placed is the critical condition for perceiving White's illusion” they a. Have not shown anything new and b. seem naive to the theoretical implications and generality of their own casual description (“on which it is placed”).

The implications are that, given the robustness of the figure-ground/simultaneous contrast link, the challenge of predicting simultaneous contrast reduces to the challenge of predicting figure-ground relations. The latter, in contrast, is not reducible to structure-blind theories involving inhibition/filtering based on luminance ratios. And it is certainly not reducible to an orthogonal/parallel edge dichotomy.

On 2015 Oct 07, Bart Verkuil commented:

An Associate Editor of The Journal of The Norwegian Medical Association made us aware of an error in the Introduction of this paper: we mention that a meta-analysis of the longitudinal relation between workplace bullying and mental health, published in The Journal of The Norwegian Medical Association (Tidsskrift for Den norske legeforening) in 2014, is only available in the Norwegian language. This is incorrect, as it is available in English: http://www.tidsskriftet.no/article/3213422

On 2016 Aug 23, Ben Goldacre commented:

This trial has the wrong trial registry ID associated with it on PubMed: both in the XML on PubMed, and in the originating journal article. The ID given is NCT013655104. We believe the correct ID, which we have found by hand searching, is NCT01365104.

This comment is being posted as part of the OpenTrials.net project^[1] , an open database threading together all publicly accessible documents and data on each trial, globally. In the course of creating the database, and matching documents and data sources about trials from different locations, we have identified various anomalies in datasets such as PubMed, and in published papers. Alongside documenting the prevalence of problems, we are also attempting to correct these errors and anomalies wherever possible, by feeding back to the originators. We have corrected this data in the OpenTrials.net database; we hope that this trial’s text and metadata can also be corrected at source, in PubMed and in the accompanying paper.

Many thanks,

Jessica Fleminger, Ben Goldacre*

[1] Goldacre, B., Gray, J., 2016. OpenTrials: towards a collaborative open database of all available information on all clinical trials. Trials 17. doi:10.1186/s13063-016-1290-8 PMID: 27056367

* Dr Ben Goldacre BA MA MSc MBBS MRCPsych<br> Senior Clinical Research Fellow<br> ben.goldacre@phc.ox.ac.uk<br> www.ebmDataLab.net<br> Centre for Evidence Based Medicine<br> Department of Primary Care Health Sciences<br> University of Oxford<br> Radcliffe Observatory Quarter<br> Woodstock Road<br> Oxford OX2 6GG

On 2016 Apr 22, Falk Leichsenring commented:

For a response to Hofmann, see http://ebmh.bmj.com/cgi/content/full/eb-2016-102372?ijkey=giCBgiE7JTlTg&keytype=ref&siteid=bmjjournals

On 2016 Mar 25, Stefan Hofmann commented:

For a more detailed critique and response, see: http://www.ncbi.nlm.nih.gov/pubmed/27009055

On 2016 Feb 27, Falk Leichsenring commented:

We absolutely agree: strong conclusions require strong evidence

Falk Leichsenring, Patrick Luyten, Mark J. Hilsenroth, Allan Abbass, Jacques P. Barber, John R. Keefe, Frank Leweke, Sven Rabung, Christiane Steinert

Hofmann [1] asked us to provide evidence - here it is.

In our first response [2] to Hofmann et al. [3], we showed that their commentary on the quality of studies of PDT is not consistent with empirical evidence from quality research performed in adversarial collaboration between PDT and CBT researchers, which found no significant differences in quality between studies of PDT and CBT [4, 5]. Furthermore, we noted that Gerber et al. did not find significant correlations between methodological quality and outcome in studies of PDT [5]. These results (i.e., evidence) are inconsistent with Hofmann et al.’s conclusions [3], regardless of whether Hofmann et al.’s ratings [3] are methodologically sound.

In addition, we emphasized that Hofmann et al. failed to demonstrate any evidence that the quality of the 64 RCTs leads to results in favor of PDT. In a similar way Bhar and Beck suggested that the lack of difference in outcome between CBT and PDT found by Leichsenring, Rabung, and Leibing [6] was due to poor treatment integrity [7]. However, using Bhar and Beck’s own integrity ratings, their assertion was not corroborated by empirical data [8]. It is of note that these meta-analyses [6, 8] included researchers from both CBT (E.L.) and PDT (e.g. F.L.).

In our first response, we also observed that the authors failed to provide basic data on interrater reliability, raters’ training, the rating procedures, attempts to address allegiance effects, or blinding of raters [2]. The authors also did not include researchers of both approaches among the raters, as done by Gerber et al. and Thoma et al. [4, 5]. In addition, we noted that Hofmann et al. based their conclusions of poor methodological quality on "unclear" designations of quality [2]. Most authors would have attempted to contact the original authors of a study before asserting that procedural information was unclear and making strong conclusions about study quality.

Hofmann et al. [3] drew strong conclusions about the quality of our review using extreme terms such as "invalidating the authors´ results" and "making the findings meaningless" using nonstandard procedures of questionable quality. For strong conclusions, strong evidence is required. Yet, Hofmann et al. failed to provide it. For a commentary aiming to address study quality, it is puzzling to apply procedures of such poor quality.

We are raising these issues again since Dr. Hofmann did not address them in his response [1]. Instead of doing so, Dr. Hofmann stated [1]: ”As their only defense, the authors argue that CBT is also poorly supported." In this way, he is simply ignoring the evidence we provided and the methodological shortcomings of his commentary we had pointed out [2]. Further, we did not question that CBT is an efficacious treatment. We just pointed out that the available evidence shows that the quality of CBT studies is no better than that of PDT studies [4, 5].

We also did not intend to attack Dr. Hofmann on a personal level, but rather intended to provide evidence that he repeatedly applied double standards when judging studies of CBT as compared to those of PDT [2, 9, p. 49-51]. We respectfully asked that if he chooses to write about PDT (e.g., comment on a meta-analysis, conduct a meta-analysis, or conduct a study involving PDT), that he considers involving a psychodynamic researcher in the process. This invitation still stands.

Dr. Hofmann emphasized that CBT is widely disseminated in the UK. This is true, but PDT is recommended by treatment guidelines and implemented in the National Health Service in the UK as well. This is also true in other countries. In Germany, for instance, PDT is as frequently used as CBT [10]. The Scientific Board for Psychotherapy (Wissenschaftlicher Beirat Psychotherapie; WBP) is the paramount body in Germany for assessing the scientific status of psychotherapeutic interventions. For this purpose, standardized and transparent criteria are used. Based on a careful evaluation by the WBP, both CBT and PDT were acknowledged as scientific and efficacious forms of psychotherapy (www.wbpsychotherapie.de). It is noteworthy that the WBP is composed of researchers from diverse psychotherapeutic orientations (e.g. CBT, PDT, and systemic therapy). The studies of PDT were evaluated by CBT researchers, and vice versa. The conclusions by a balanced expert institution such as the WBP are incompatible with those by Hofmann et al. [3].

We all should be happy that a variety of psychotherapeutic treatments exist that are beneficial to patients. Future research should address the question of which patients benefit most from which treatments, and why.

Declaring the evidence of a whole treatment approach as "meaningless" is not supported by the preponderance of evidence, and is counter-productive to this goal.

References

1 Hofmann SG: Show us the data! Pubmed Commons Feb 17 2016 2 Leichsenring F, Luyten P, Hilsenroth MJ, Abbass A, Barber JP, Keefe JR, Leweke F, Rabung S, Steinert C: Once again: Double standards in psychotherapy research - response to hofmann et al. PubMed Commons 2016 3 Hofmann SG, Eser N, Andreoli G: Comment from pubmed commons. . January 23rd, 12:28am UTC 2016 2016 4 Thoma NC, McKay D, Gerber AJ, Milrod BL, Edwards AR, Kocsis JH: A quality-based review of randomized controlled trials of cognitive-behavioral therapy for depression: An assessment and metaregression. Am J Psychiatry 2012;169 5 Gerber AJ, Kocsis JH, Milrod BL, Roose SP, Barber JP, Thase ME, Perkins P, Leon AC: A quality-based review of randomized controlled trials of psychodynamic psychotherapy. Am J Psychiatry 2011;168:19-28. 6 Leichsenring F, Rabung S, Leibing E: The efficacy of short-term psychodynamic psychotherapy in specific psychiatric disorders: A meta-analysis. Arch Gen Psychiatry 2004;61:1208-1216. 7 Bhar SS, Beck AT: Treatment integrity of studies that compare short-term psychodynamic psychotherapy with cognitive-behavior therapy Clin Psychol-Sci Pr 2009;16:370-378. 8 Leichsenring F, Salzer S, Hilsenroth M, Leibing E, Leweke F, Rabung S: Treatment integrity: An unresolved issue in psychotherapy research. Curr Psych Rev 2011;7 313-321. 9 Leichsenring F, Rabung S: Double standards in psychotherapy research. Psychother Psychosom 2011;80:48-51. 10 Albani C, Blaser G, Geyer M, Schmutzer G, Brähler E: Outpatient psychotherapy in germany from the patient perspective: Part 1: Health care situation Psychotherapeut 2010;55:503-514.

On 2016 Feb 17, Stefan Hofmann commented:

Show us the data! The authors wrote in the abstract that psychodynamic therapy is as efficacious as treatments established in efficacy. Strong conclusions require strong evidence. Dr. Leichsenring and colleagues were unable to provide the reader with such evidence. Our earlier commentary of the article by Leichsenring and colleagues reported data suggesting that the majority of studies included in their review were of low quality. It is the responsibility of Leichsenring and colleagues to provide the reader with evidence that their conclusions are justified. Instead of providing the reader with such evidence, the authors chose to attack me on a personal level. Whether or not Dr. Leichsenring and colleagues believe that I am adequate to serve as a reviewer or editor of scientific journals or grants and collaborate with others is completely unrelated to the weaknesses of their study. As their only defense, the authors argue that CBT is also poorly supported. The authors are incorrect. The supporting evidence of CBT is overwhelmingly large. Our own review identified 269 meta-analytic studies of CBT. We observed that the quality of studies that entered some of these meta-analyses were not uniformly high. However, some of them were of high quality (e.g., Hofmann & Smits, 2008). Because CBT has such a solid empirical basis, many countries, including the UK, disseminate CBT on a large-scale basis, e.g., http://www.iapt.nhs.uk/. It should be noted that this dissemination is not limited to CBT but also includes other empirically supported treatments. Polemics and personal attacks on my scientific integrity are not the real problem. The biggest concern in my view is that these disputes distract from the real issue. They confuse our patients and policy makers, inhibit scientific progress, and inflict harm by withholding effective treatments. References 1. Hofmann, S. G., Asnaani, A., Vonk, J. J., Sawyer, A. T., & Fang, A. (2012). The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive Therapy and Research, 36, 427-440. doi 10.1007/s10608-012-9476-1 2. Hofmann, S. G. & Smits, J. A. J. (2008). Cognitive-behavioral therapy for adult anxiety disorders: A meta-analysis of randomized placebo-controlled trials. Journal of Clinical Psychiatry, 69, 621-632. doi: 10.4088/JCP.v69n0415

On 2016 Jan 29, Falk Leichsenring commented:

Once again: Double standards in psychotherapy research - response to Hofmann et al.

Falk Leichsenring, Patrick Luyten, Mark J. Hilsenroth, Allan Abbass, Jacques P. Barber, John R. Keefe, Frank Leweke, Sven Rabung, Christiane Steinert

Referring to a recent review on psychodynamic therapy (PDT) [1], Hofmann et al. [2] criticize the quality of studies included in this review. The authors conclude that the "poor quality" of studies of PDT "invalidates" the results of this review making them "meaningless" [2]. The comment by Hofmann et al. [2] deserves some response.

• The conclusions drawn by Hofmann et al. [2] are inconsistent with present research. As shown by an independent research group including proponents of both CBT and PDT working respectfully together, the quality of studies of PDT does not differ significantly from that of studies of CBT which fell into the lower range of adequate quality [3, p. 22, 4]. Most of the studies included by Leichsenring et al. [1] were also included in this comparison [3, 4]. However, Hofmann has not described the respective CBT studies as “meaningless”.

Furthermore, the comment by Hofmann et al. [2] suffers from several shortcomings.

• The authors are incorrect when referring to our publication as a meta-analysis. In fact it was a systematic review [1]. This is of note since possible shortcomings in individual studies would not invalidate the review as a whole.

• The authors draw a highly generalizing conclusion without any differentiation, for example, by disorders or degree of risk.

• For their ratings, Hofmann et al. [2] did not report basic data on the number and training of raters, on blinding, or interrater-reliability. For the best minimization of bias, raters of both approaches would have been included as done by Gerber et al. and Thoma et al. [3, 4]. Thus, the quality of the procedures applied by Hofmann et al. [2] themselves is questionable.

• The conclusions by Hofmann et al. [2] are based mostly on “unclear” designations, not clear flaws. In fact, an “unclear” risk of bias indicates that the design feature could be both worse or better than described in the article. What is most concerning is that the authors did not make any effort to resolve the “unclear” assignments by carefully reading the papers or contacting their authors. Many assignments are obviously clear from the studies [e.g. 5, 6].

• In addition, even if there are flaws, Hofmann has not shown that these particular flaws lead to results in favor of PDT (rather than e.g., greater error in effect estimates overall). Several meta-analyses did neither find significant correlations between ratings of methodological quality and outcome [4, 7] nor between treatment integrity (assessed by prominent CBT researchers, e.g. Aaron T. Beck) and differences in outcome between CBT and PDT [8].

• If the "poor quality" of PDT studies "invalidates" [2] the results reported by Leichsenring et al. [1] making them "meaningless", this would equally apply to meta-analyses carried out by CBT researchers who included several of these same studies. Tolin [9], for example, included 10 studies also included by Leichsenring et al. [1]. Hofmann, however, has never critically commented on these meta-analyses - which are interpreted as supporting the efficacy of CBT - thus, again applying a double standard.

• Hofmann was repeatedly shown to apply double standards when judging studies of CBT vs. PDT [10, p. 49-51].

(a) In a previous meta-analysis, for example, Hofmann [11, p. 180] claimed that the quality of studies included by him was "considerably better" than that of studies of PDT, e.g. in the meta-analysis by Leichsenring and Rabung [12]. Hofmann et al. [11] reported a mean Jadad score of 1.23, whereas the mean Jadad score in the meta-analysis by Leichsenring et al.[12] was 1.96 (0 = poor; 5 = excellent).

(b) Hofmann [11]criticized the meta-analysis by Leichsenring et al. [12] for including heterogeneous studies. However, between-effect hterogeneity was in the low to medium range [10, 12]. In his own meta-analysis, Hofmann [11] did not even test for heterogeneity before combining data of randomized controlled and observational studies [10, 11].

• Thus, from a scientific perspective, it is questionable whether a strong proponent of CBT who has publicly demonstrated that he is an opponent of PDT [e.g. 13] is able to provide unbiased conclusions about PDT.

Given the author’s very negative publicly expressed opinions about PDT, and the way he conducted this critique of the Leichsenring et al. review [1] it appears that biases can lead to a lack of even-handedness in regard to the evaluation of psychodynamic studies. Thus, we respectfully would ask that if he chooses to write about PDT(e.g., comment on a meta-analysis, conduct a meta-analysis, or conduct a study involving PDT), that he involve a psychodynamic researcher in the process (i.e., implement a version of adversarial collaboration [14], and also that he recuse himself from being involved as an editor, or reviewer, in regard to research involving PDT.

We would welcome the collaboration of CBT researchers in researching psychotherapy and synthesizing the results from trials. We have done so several times [e.g. 15, 16].

Given the present crisis of replicability of research [17], biased and tendentious statements as those by Hofmann bear the risk of damaging all psychotherapy research equally in the eyes of the public.

References

1 Leichsenring F et al.: Psychodynamic therapy meets evidence-based medicine: a systematic review using updated criteria. Lancet Psychiatry 2015;2:648-660. 2 Hofmann SG et al.: Comment from PubMed Commons. January 23rd, 12:28am UTC 2016 3 Thoma NC et al.: A quality-based review of randomized controlled trials of cognitive-behavioral therapy for depression: an assessment and metaregression. Am J Psychiatry 2012;169 4 Gerber AJ et al.: A quality-based review of randomized controlled trials of psychodynamic psychotherapy. Am J Psychiatry 2011;168:19-28. 5 Barber J et al.: Short-term dynamic psychotherapy versus pharmacotherapy for major depressive disorder: a randomized, placebo-controlled trial. J Clin Psychiatry 2012;73:66-73. 6 Crits-Christoph P et al.: Psychosocial treatments for cocaine dependence: National Institute on Drug Abuse Collaborative Cocaine Treatment Study. Arch Gen Psychiatry 1999;56:493-502. 7 Keefe JR et al.: A meta-analytic review of psychodynamic therapies for anxiety disorders. Clin Psychol Rev 2014;34:309-323. 8 Leichsenring F et al.: Treatment integrity: An unresolved issue in psychotherapy research. Curr Psych Rev 2011;7 313-321. 9 Tolin DF: Is cognitive-behavioral therapy more effective than other therapies? A meta-analytic review. Clin Psychol Rev 2010;30:710-720. 10 Leichsenring F et al.: Double standards in psychotherapy research. Psychother Psychosom 2011;80:48-51. 11 Hofmann SG et al.: The effect of mindfulness-based therapy on anxiety and depression: a meta-analytic review J Consult Clin Psychol 2010; 78 169-183. 12 Leichsenring F et al.: Effectiveness of long-term psychodynamic psychotherapy: a meta-analysis. JAMA 2008;300:1551-1565. 13 Rief W et al.: [Saving psychoanalysis. At any cost?]. Nervenarzt 2009;80:593-597. 14 Mellers B. et al.: Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychol Sc 2001;12:269-275. 15 Leichsenring F et al.: The efficacy of short-term psychodynamic psychotherapy in specific psychiatric disorders: a meta-analysis. Arch Gen Psychiatry 2004;61:1208-1216. 16 Leichsenring F et al.: Psychodynamic therapy and CBT in social anxiety disorder: a multicenter randomized controlled trial. Am J Psychiatry 2013;170:759-767. 17 Open Science Collaboration: PSYCHOLOGY. Estimating the reproducibility of psychological science. Science 2015;349:aac4716.

On 2015 Oct 05, Lydia Maniatis commented:

It's been a month since I posted my comments on this article and on the original Blakeslee and McCourt article. A sharp increase in visits to the target articles post-comment seems to indicate that the comments are being read, but no author or reader has put in their two cents, either here or on PubPeer. I think that my argument against the "brightness-is-perceived-luminance" idea is sound and straightforward. Am I wrong?

On 2015 Sep 06, Lydia Maniatis commented:

I fully agree with the latter part of this commentary but disagree with the author's concession that, in lightness experiments, "when illumination appeared homogeneous, lightness and brightness judgments were identical. There is nothing new here. It is well known."

Brightness is currently being described, including by Gilchrist, as the perceptual correlate of luminance. But there is no perceptual correlate of luminance, even under (apparently) homogeneous illumination, and this can be proved as follows:

We ask an observer to report on the lightness of a set of surfaces which don't produce the impression of shadows or transparency. Then, in a second session, we present the same set of surfaces under a different level of illumination. The lightness reports for the surfaces will stay essentially the same, even though their luminances may have changed substantially. So to say that people are making "brightness" judgments in either the first or the second or in any case doesn't seem reasonable.

On 2018 Jan 24, Doug Berger commented:

Excellent article from a brave young researcher willing to put her career in the line of fire of criticism of CBT and other psychotherapy adherents, not to mention employment opportunities where psychotherapy research has been a mainstay for years and is a cash-cow. The only caveat I would add is that the translation of basic science research into the ability og CBT to overcome the inability to single-(patient blind), or double-(patient and therapist blind), as well as the inability to have blind placebo, in a psychotherapy clinical trial is a serious doubt. Many psychiatric study drugs over the years have had considerable pre-clinical basic science data, even phase I or phase II clinical data, only to fail miserably in phase III trials when adequate numbers of subjects were studied in tightly blinded conditions with blind placebo control. Basic science research only lends itself to decide if it is worth while to test a drug in large populations clinically, it doesn’t say itself if a drug will work, especially in psychiatric conditions where endpoints are subjective and random error high.

The whole premise of CBT, that negative or distorted cognitions are the cause of a psychiatric condition is the only instance in all of medicine and psychiatry where a symptom of an illness is also construed to be the cause. Taking a symptom and making it into a cause is a way to spin the need of a therapy aimed at a cause that actually the result, and in psychiatric conditions with subjective endpoint hope and expectation effects are an easy way to garner some symptom reduction that can easily be called a "response" in a clinical trial. Reading the paper, I wonder if the journal asked the authors to propose ways to work around the blinding problem. The work-around is heuristic but not of practical value once a clinical trial gets going-and as the authors rightly note (and as I have noted in other papers below), there is really no way to do blind/blind placebo psychotherapy clinical trial.

Other papers on this topic: https://www.ncbi.nlm.nih.gov/pubmed/26870318 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4863672/ https://www.japanpsychiatrist.com/Abstracts/CBT_Escape.html

D. Berger, U.S. Board Certified Psychiatrist

On 2015 Aug 25, Ranjith Ramasamy commented:

https://www.youtube.com/watch?v=qAaSG4LDSE4&feature=youtu.be - Video from Urology times summarizing the study findings

On 2015 Aug 23, Kelly Drew commented:

Hello to all! Interesting paper, but don't dismiss A1AR. Have you seen these papers?

1: Muzzi M, Coppi E, Pugliese AM, Chiarugi A. Anticonvulsant effect of AMP by direct activation of adenosine A1 receptor. Exp Neurol. 2013 Dec;250:189-93. doi: 10.1016/j.expneurol.2013.09.010. Epub 2013 Sep 19. PubMed PMID: 24056265.

2: Muzzi M, Blasi F, Masi A, Coppi E, Traini C, Felici R, Pittelli M, Cavone L, Pugliese AM, Moroni F, Chiarugi A. Neurological basis of AMP-dependent thermoregulation and its relevance to central and peripheral hyperthermia. J Cereb Blood Flow Metab. 2013 Feb;33(2):183-90. doi: 10.1038/jcbfm.2012.157. Epub 2012 Oct 24. PubMed PMID: 23093068; PubMed Central PMCID: PMC3564191.

3: Muzzi M, Blasi F, Chiarugi A. AMP-dependent hypothermia affords protection from ischemic brain injury. J Cereb Blood Flow Metab. 2013 Feb;33(2):171-4. doi: 10.1038/jcbfm.2012.181. Epub 2012 Dec 5. PubMed PMID: 23211965; PubMed Central PMCID: PMC3564206.

On 2015 Aug 26, Jim Woodgett commented:

I sound like a broken record, but it is quite remarkable that yet another paper presumes an inhibitor is specific for GSK-3beta. In this case (AR-A014418) is equally as effective at inhibiting GSK-3alpha, as is every other small molecule inhibitor of GSK-3, including lithium. This is a problem because the authors fail to assess the contribution of the second isoform which is being equivalently blocked in the experiments. There are numerous papers that describe the common and distinctive functions of these protein kinases.

On 2016 Feb 08, Salzman Lab Journal Club commented:

This paper presents a striking view of the intron lariat spliceosome at unprecedented resolution and uses pombe as a strategy for enriching relatively homogenous structural complexes. As our lab is interested in circular RNA, we were curious about how this structure may be used to model minimal circularized exon lengths. We look forward to more structures in the future that give a more detailed look at the exon-exon junction!

On 2016 Nov 03, David Juurlink commented:

Post-publication note: Additional analysis of absolute risk of opioid-related death among patients receiving more than 200 mg morphine (or equivalent) per day reported here http://bit.ly/2fVaQYL

On 2015 Aug 22, Arnaud Chiolero MD PhD commented:

An enlightening commentary on the necessity of revisiting DCIS treatment. It also raises question about screening - the major driver of DCIS identification.

On 2015 Nov 26, Tero Kivelä commented:

This paper is commented in http://www.ncbi.nlm.nih.gov/pubmed/26288835

On 2015 Nov 26, Tero Kivelä commented:

This is a commentary on http://www.ncbi.nlm.nih.gov/pubmed/26288849

On 2015 Aug 24, Andrew R Kniss commented:

This “Perspective” piece is basically a plea from Dr. Landrigan and Dr. Benbrook for “all aspects of the safety of biotechnology” to be “thoroughly reconsider[ed]”. However, in the two page opinion, they provide no evidence that crop biotechnology is harmful. In fact, Landrigan and Benbrook acknowledge that the National Academy of Sciences (NAS) “has twice reviewed the safety of GM crops” and they do not dispute the scientific consensus expressed by NAS that “GM crops pose no unique hazards to human health.” The way I read it, the entire Perspective piece seems to be a muddled conflation of two separate (albeit related) issues; the use of GMO crops and the use of herbicides. A full critique here: http://goo.gl/IcZt2S

Dr. Landrigan and Dr. Benbrook cite glyphosate-resistant weeds as a primary reason why “fields must be now be treated with multiple herbicides,” but this point also deserves some scrutiny. In corn, for example, multiple herbicides have been a common practice since long before GMO crops were introduced. In the year 2000, before Roundup Ready GMO corn had gained widespread adoption in the US (and also before glyphosate-resistant weeds were growing in corn fields), corn growers were applying nearly three herbicide active ingredients per acre. The latest USDA data from 2014 show fewer than 3.5 active ingredients applied per acre. This suggests that while glyphosate-resistant weeds may certainly have increased the number of herbicides used per acre compared to 5 years ago, the change has been relatively modest when compared to herbicide use before adoption of GMO crops.

Another misleading statement made by Dr. Landrigan and Dr. Benbrook is that the “risk assessment gave little consideration to potential health effects in infants and children, thus contravening federal pesticide law.” This claim was also addressed explicitly by EPA in their FAQ document (http://www2.epa.gov/ingredients-used-pesticide-products/registration-enlist-duo). EPA concluded that after incorporating a 10X safety factor for children, and based on a “complete and very robust” data set, that the “risks were still acceptable for all age groups for all components of the assessment: dietary food and drinking water exposure, volatility, spray drift, residential, and aggregate assessment.” This claim is addressed in even more detail (600 words) in the EPA’s response to public comment (http://www.regulations.gov/contentStreamer?documentId=EPA-HQ-OPP-2014-0195-2414&disposition=attachment&contentType=msw8).

On 2015 Sep 21, M Mangan commented:

It should be further noted that although disclosure documents were updated for Dr. Benbrook in August, following the publication of an article in the NYTimes in September, new issues of potential conflicts of interest were unearthed. Documents provided from a FOIA request by Eric Lipton illustrated that both Benbrook and Landrigan were actively working with the "Just Label It" organization and Gary Hirshberg of the organic food industry, during the time of preparation of this work. http://www.nytimes.com/interactive/2015/09/06/us/document-benbrook.html

My request to the NEJM to investigate this and update the disclosure documents, due to the new evidence, was declined by the NEJM.

On 2015 Aug 20, M Mangan commented:

This piece is astonishingly flawed on several levels, but most importantly has basic facts that are incorrect or blatantly misrepresented. Please see some capable assessments of the claims here, from a researcher who studies herbicides: http://weedcontrolfreaks.com/2015/08/gmos-herbicides-and-the-new-england-journal-of-medicine/

I would also encourage readers to seek out the expert reactions from the Science Media Centre: http://www.sciencemediacentre.org/expert-reaction-to-gmos-herbicides-and-public-health/

But I would also like to note that the calls to provide information via labels do not indicate what label they think would apply to their herbicide concerns. None of the proposed labels in the US, or any other country I've seen, address herbicides in any manner. Since non-GMOs also use herbicides, and some GMOs do not, calling for a misleading label would be irresponsible for health professionals.

On 2015 Sep 16, F Morceau commented:

Interesting article but did the authors assessed the effect of BML-210 on NB4 cells differentiation? It woud have been interesting and relevant to provide this information. ATRA is mentioned in the materials and methods section while it has not been used in fact in the article! How this could escape to the reviewers?

On 2017 May 30, Rashmi Das commented:

We thank Harri for his PERSONAL (NON PEER REVIEWED) OPINION which is available at above HANDLE ( http://hdl.handle.net/10138/153180) THAT CONTAINS DIRECT COPY AND PASTE OF THREE FIGURES/IMAGES FROM OUR PREVIOUS PUBLICATIONS (JAMA 2014 and Cochrane 2013). We are happy to reply to above comments made by Harri. First regarding the Cochrane review which was withdrawn in 2015, the detailed report is already available at following link (http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD001364.pub5/abstract). This report is the collaborative observation and conclusion of the Cochrane editors (UNLIKE THE HANDLE WHICH CONTAINS MORE OF PERSONAL OPINION WHICH HAS ALREADY BEEN EXAMINED BY THE COCHRANE EDITORS BEFORE REACHING THE CONCLUSION). The same HANDLE WAS SENT TO JAMA EDITORS REGARDING THE JAMA CLINICAL SYNOPSIS (PUBLISHED IN 2014) AND HARRI REQUESTED THE EDITORS TO CARRY OUT THE INVESTIGATION AND VERIFY. THE EDITORS ASKED US FOR REPLY WHICH WE CLARIFIED IN A POINT TO POINT MANNER (BOTH THE COMMENT BY HARRI AND OUR REPLY WAS PUBLISHED, SEE BELOW). HAD THE COMMENT/REPORT BY HARRI WAS ENTIRELY CORRECT, THE JAMA EDITORS COULD HAVE STRAIGHTWAY RETRACTED/WITHDRAWN THE SYNOPSIS WITHOUT GOING FOR PUBLICATION OF THE COMMENT/REPLY (Both are available at following: https://www.ncbi.nlm.nih.gov/pubmed/26284729; https://www.ncbi.nlm.nih.gov/pubmed/26284728). IT HAS TO BE MADE CLEAR THAT THE JAMA SYNOPSIS (DAS 2014) WAS WITHDRAWN AS THE SOURCE DOCUMENT ON WHICH IT WAS BASED (COCHRANE 2013 REVIEW) WAS WITHDRAWN (NOT BASED ON THE REPORT IN THE HANDLE WHICH IS A PERSONAL NON PEER REVIEWED OPINION). The irony is that though HARRI'S COMMENT got published as LETTER TO EDITOR in JAMA after OUR REPLY, still the NON PEER REVIEWED HANDLE THAT CONTAINS DIRECT COPY OF THREE FIGURES/IMAGES FROM OUR PUBLICATION IS GETTING PROPAGATED.

On 2015 Aug 30, Harri Hemila commented:

This is a short comment on Das RR, 2014. A much more detailed description of problems in Das RR, 2014 is available at HDL.

The paper by Das RR, 2014 was based on their Cochrane review Singh M, 2013, which had a number of problems which are descibed in HDL.

The Cochrane review was withdrawn in April 2015, see DOI.

On 2016 Feb 04, Hamid Salehiniya commented:

I read the article,this is a attractive and useful article, based on previous research, we conclude obesity is a important risk factor for cancer, in conclusion one strategy in cancer prevention is active life style.

On 2015 Aug 28, Lydia Maniatis commented:

There seems to be a chicken and egg problem going on here as well. Here: https://www.youtube.com/watch?v=QORWM3Pl760, one of the authors seems to be defining "stimulus" as something that is constructed by the visual system, as opposed to a light -reflecting object in the world, and at the same time he is saying that the nature of this stimulus is determined in some way by its frequency of occurrence. But the latter has no existence prior to the former. There has to be some reference at some point to an objective situation, a stimulus in that sense, for the conversation to work.

Do the authors consider there to be a link between the topography of the light energy on the retina and the topography of the light energy in the world one moment before it strikes the retina?

On 2015 Aug 28, Lydia Maniatis commented:

The claims in this article (and related past publications) are not credible, and barely defended.

First, there is a little bit of sleight-of-hand in the authors' definition of “wholly empirical,” with the result that it includes both wholly acceptable and wholly unacceptable assertions. After describing their view that the visual system assigns perceptual values “without ever recovering or statistically estimating the properties of objects and conditions in the visual environment,” they label this view “wholly empirical” on the basis that it “depends entirely on feedback from trial and error experience.” But unlike the former claim, which is highly disputable, the latter seems simply to be a description of evolution by natural selection, a trial and error process in which adaptive features are preserved and less adaptive ones rejected. I cannot imagine that any scientist today would propose a mechanism for a biological function that they do not believe could have arisen via natural selection. Thus, it does not seem fair on the part of the authors to monopolize the concept for a specific set of proposals.

They reinforce this apparent monopoly by implying that a lack of correlation between the percept and the physical world is a necessary consequence of the link between perceptual processes and behavior: “...the biological feedback loop...would progressively order the basic visual qualities we perceive (apparent length, lightness, etc) according to their impact on biological (reproductive) success, rather than with the generative properties in the world (actual physical lengths, the reflective and illumination values underlying luminance, etc) [On what basis do the authors refer to reflectance and illumination as objective properties of the world (see below)?]” The “rather than” term is a verbal trick, implying that a perceptual representation that is correlated with (selected) physical properties would be less adaptive than one that is not. But such an assumption is not warranted.

The “evidence” for the position is also contingent on looseness in the supporting arguments. We are told, for example, that “since luminance measures the number of photons falling on the retina, common sense suggests that measurements of light intensity and perceived lightness should be proportional. Thus, if two surfaces return the same amount of light to the eye, they “should” be perceived as equally light.” This is a straw man. In the most extreme cases of same-luminance surfaces producing different lightness, the different surfaces, in addition to producing lightness percepts, also produce illumination percepts. Higher perceived lightness correlates with lower perceived illumination. The visual system is attempting to provide relative estimates for reflectance and illumination. This could not be achieved by directly representing luminance, whatever “common sense” might say. It is not correct to say that the visual system never correctly (or to a good approximations) represents relative reflectance/illumination values across surfaces. When it does, it is not by accident, it is what the system is designed (so to speak) to achieve. The idea that all images that elicit reflectance/illumination percepts do so on the basis of the frequency of evolutionary experience - behavioral responses to each particular stimulus, and their consequences for the reproductive success of the individual - is not credible. Even if we take apparent illumination out of the picture, the problem is just as big. Are we supposed to explain e.g. every Kanisza figure, on the basis of how frequently it occurred?

And, for that matter, the notion that “sensory stimuli [are] associated with biologically useful responses” does not justify the claim that stimulus frequency determines perceptual values, since a stimulus pattern may be very frequent but have little biological consequence, or rare but have a larger biological impact. So, in another sleight-of-hand, the authors have slipped an unwarranted frequency-biologically-relevant link into their argument.

In addition to the lightness example, the other piece of “evidence” offered is equally problematic on a number of counts. First, we are told that psychophysical experiments indicate that lines oriented at about thirty degrees from the vertical appear longer than lines at any other orientation. But the perceived length of a line is entirely contingent on the figure in which it is incorporated. So even if the authors claim that, over evolutionary time, line segments with the thirty degree orientation occurred (and were reacted to in an evolutionary dispositive way), more often than any other orientation, this would not explain why an oblique with a greater deviation from the vertical will reliably yield a longer percept than a vertical when incorporated in, e.g. a Shepard box. Even if we take the authors' natural scene statistics at face value, they are not relevant, since edges always occur in objects, and the shapes of these objects mediate the perception of length of an edge. Assumptions regarding the shapes of objects even determine whether or not a physically present or absent edge will or will not be seen.

The natural scene statistics offered are also not credible. Human ancestors varied differed greatly in size form the present, individual humans differ in height based on age, the eyes and body are constantly in motion, the distance to the object being viewed is constantly changing, and all of these things affect the orientation of the projection of a physical edge. Most of our time is arguably spent looking at close range, and at people. A statistical distribution developed on the basis of measurements from a fixed height at a relatively great distance of groomed gardens is arguably not a good approximation of human experience. Finally, are the authors really claiming that an isolated line segment of thirty degrees looks longer than others because it (supposedly) projected more frequently on the retinas of humans and their ancestors? If all percepts are drawn from frequency distributions, then how do we account for their qualitative differences? Are colors also labelled on the basis of the frequency that each collection of wavelengths occurs?

There is, finally, a fundamental contradiction in the argument that perceptual values don't ever recover or estimate properties of objects and conditions in the environment (I discuss this contradiction in Maniatis (in press). The problem is that if we choose to adopt this view, then we are not entitled to refer to the physical world as though we did, in fact, have knowledge about it. Thus, statements such as “consider the objective length of a line, e.g. the edge of a ruler...” are paradoxical because they imply that the authors have access to the very objective facts that they claim are inaccessible. It does no good to argue that we detect objective facts using objects called instruments, since the properties of these objects, too, are only accessible through perception. Given their position, the authors are not entitled to make any reference to the objective world and its properties, since such references directly contradict it.

On 2015 Aug 26, Lydia Maniatis commented:

In a search of the book "Perceiving Geometry," the term 'shape' comes up three times, twice in reference to the shape of distributions and once in a plain assertion that some neurons respond selectively to "higher-order stimulus characteristics...such as shape...". The assertion is not accompanied by arguments. The book has many references, are there any in particular that address the issue of shape?

The term or concept of 'figure-ground' also does not arise in the book. Do you consider the problems of perceptual organisation to have been solved on the basis of frequency distributions?

On 2015 Aug 24, Dale Purves commented:

Shape is a higher order construct of "size, distance, orientation" and geometrical factors such as the length of intervals (lines) and angles. See Howe and Purves "Perceiving Geometry" (Springer, 2005) for a full account and references to original papers.

On 2015 Aug 24, Lydia Maniatis commented:

In their summary, the authors state that:

"Visual perception is characterized by the basic qualities of lightness, brightness, color, size, distance, orientation, speed and direction of motion ordered over some range."

They have left out the most basic quality of all, and the one that largely mediates all the others: Shape. Was this an oversight, or a purposeful or principled omission?

They also say that: "These perceived qualities and their order within these ranges, however, do not align with reality."

They should clarify whether they mean for this second statement to apply also to shape.

On 2016 Jan 14, Arturo Casadevall commented:

The central criticism is that we have compared variables for which there is no causal relationship. We recognize the difficulties involved in assuming causality and dangers of spurious correlations when plotting unrelated variables. Furthermore, we are fully aware that correlation is not causation. However, the criticism made by Levine and Weinstein does not take into account a large body of published scholarly work showing that spending of public funds translates into medical goods such as new therapeutics. To make this point, we note the findings of several studies. In 2000, a United States Senate Report found that of the 21 most important drugs introduced between 1965-1992, 15 (71%) ‘were developed using knowledge and techniques from federally funded research’ (http://www.faseb.org/portals/2/pdfs/opa/2008/nih_research_benefits.pdf). A recent study of 26 transformative drugs or drug classes found that for many, their discovery was made with governmental support [1]. Numerous other studies have reinforced this point [2-4]. Kohout-Blume estimated that a 10% increase in targeted funding for specific diseases produced a 4.5% increase in the number of drugs reaching clinical trials after an average lag of 12 years [5]. In our own investigations, we have traced the ancestry of most of the drugs licensed in the past four decades to publicly funded research (unpublished data). The literature in this field overwhelmingly supports the notion that public spending in biomedical research translates into public goods. The debate is not about whether this happens but rather about the magnitude of the effect. The notion that public funding in biomedical research generates basic knowledge that is subsequently used in drug development is a concept accepted by most authorities. Hence, the use of the NIH budget as a proxy for public spending in biomedical research is appropriate.

We are aware that establishing causal relationships among non-experimental variables can be a daunting task. However, we note that the relationship between public spending and medical goods does meet some of the essential criteria needed to establish causality. First, the relationship between these variables meets the requirement of temporal causality since, for many drugs, publicly funded basic research precedes drug development. Second, we also have mechanistic causality, since knowledge from basic research is used in designing drugs. There are numerous examples of mechanistic causality including the finding of receptors with public funds that are then exploited in drug development when industry generates an agonist or inhibitor. We acknowledge that we do not know if the relationship between public spending and drug development is linear, and the precise mathematical formulation for how public spending translates into medical goods is unknown. In the absence of evidence for a more complex relationship, a linear relationship is a reasonable first approximation, and we note that other authorities have also assumed linear relationships in analyzing inputs and outcomes in the pharmaceutical industry. For example, Scannell et al. [6] used a similar analysis to make the point that ‘The number of new drugs approved by the US Food and Drug Administration (FDA) per billion US dollars (inflation-adjusted) spent on research and development (R&D) has halved roughly every 9 years’.

The authors claim to have done a causality analysis of the data generated in our paper, concluding that ‘We do not find evidence that NIH budget ⇒ NME (p=0.475), and thus it may not be a good indicator of biomedical research efficiency.’ However, this oversimplifies a very complex process of how public spending affects NME development; we do not agree that this simple analysis can be used to deny causality. Although the limited information provided in their comment does not permit a detailed rebuttal, we note that a failure to reject the Granger causality null hypothesis does not necessarily indicate the absence of causality. Furthermore, Granger causality refers to the ability of one variable to improve predictions of the future values of a second variable, which is distinct from the philosophical definition of causality. Whether or not NIH budget history adds predictive ability in determining the number of NMEs approved at some point in the future cannot negate the fact that basic biomedical research funding unequivocally influences the creation of future drugs, as well as many other outcomes. Therefore, we stand by our use of NIH funding and NMEs as indicators of biomedical research inputs and outcomes.

The authors suggest that another study by Rzhetsky et al. [7] contradicts the findings of our paper and provides a better method of studying biomedical research efficiency. The work by Rzhetsky et al., while very interesting, addresses a fundamentally different question relating to how scientists can most efficiently choose research topics to explore a knowledge network [7]. The allocation of scientists to research topics is undoubtedly a possible contributor to overall research efficiency, but the approach used in this analysis is very different from our broader analysis of the biomedical research enterprise as a whole. The work in [7] has a narrow scope and does not attempt to study the impact of research investments in producing societal outcomes. The central conclusion of our paper is that biomedical research inputs and outputs are increasing much faster than outcomes, as measured by NMEs and LE.

We do not ‘conjecture that a lack relevance or rigor in biomedical research’ is solely responsible for this phenomenon, as Levine and Weinstein assert. Instead, our paper discusses a number of possible explanations—many of which have been previously identified in the literature [6-12], including several that agree with the conclusions of Rzhetsky et al. [7]. However, the recent epidemic of retracted papers along with growing concerns about the reproducibility of biomedical studies, expressed in part by pharmaceutical companies dedicated to the discovery of NMEs [13, 14], are indisputable facts. If a substantial portion of basic science findings are unreliable, this is likely to contribute to reduced productivity of the research enterprise. We agree with the suggestion that research difficulty increases as a field matures, which has been made by others [6]; this does not contradict our analysis and is mentioned in our paper’s discussion. Biomedical research efficiency is complex, and it is likely that the decline in scientific outputs has numerous causes. It is appropriate for scientists to consider any factors that may be contributing to this trend, and the comments from Dr. Schuck-Paim in this regard (see the other posted comments) are therefore welcome.

In summary, we do not find the arguments of Levine and Weinstein to be compelling. We note that other investigators have come to conclusions similar to ours [6, 15]. The productivity crisis in new drug development has been intensively discussed for at least a decade [6, 15-16]. We believe that addressing inefficiencies in biomedical research is essential to maintain public confidence in science and, by extension, public funding for basic research.

Arturo Casadevall and Anthony Bowen

[1] Health Aff (Millwood ) 2015, 34:286-293.

[2] PNAS 1996, 93:12725-12730.

[3] Am J Ther 2002, 9:543-555.

[4] Drug Discov Today 2015, 20:1182-1187.

[5] J Policy Anal Manage 2012, 31:641-660.

[6] Nat Rev Drug Discov 2012, 11:191-200.

[7] PNAS 2015, 112:14569-14574.

[8] Res Policy 2014, 43(1):21–31.

[9] Nature 2015, 521(7552):270–271.

[10] Br J Cancer 2014, 111(6):1021–1046.

[11] Nature 2015, 521(7552):274–276.

[12] J Psychosom Res 2015, 78(1):7–11.

[13] Nature 2012, 483(7391):531-533.

[14] Nat Rev Drug Discov 2011, 10(9):712.

[15] Nat Rev Drug Discov 2009, 8:959-968.

[16] Innovation policy and the economy 2006, 7:1-32.

On 2016 Jan 08, Michael LeVine commented:

In this recent article, Bowen and Casadevall attempt to classify biomedical research efficiency in terms of the ratio of outcomes to input. The outcomes were chosen to be approved new molecule entities (NMEs) and US life expectancy (LE); the chosen input was the NIH budget. While the resulting analysis claims that efficiency has decreased in the last decade, we argue that (i)-the analysis performed is insufficient to make that claim, and (ii)- the findings do not support the conjecture that a lack of relevance or rigor in biomedical research is causing stagnation in medicine and public health.

Bowen and Casadevall suggest that because research projects take time to complete, it is possible that “the exponential growth in research investment and scientific knowledge over the previous five decades has simply not yet grown fruit and that a deluge of medical cures are right around the corner”. They investigate time-lagged efficiency for NMEs, but this analysis is only sufficient if there is a linear causal relationship between the two variables that is unaffected by any external variables that have not been included in the analysis. Without any evidence of such a relationship, it is unwise to interpret a trend in a ratio between two unassociated measurements. Just as two unrelated measurements can display a spurious correlation, the ratio between those measurements may display a spurious trend.

We reanalyzed the data used in this paper to find evidence of causal relationships between the inputs and outcomes. To do this, we tested for Granger causality (1), which identifies potentially causal relationships by determining whether the time series of one variable is able to improve the forecasting of another. We analyzed the non-stationary time series from 1965-2012 using the Toda and Yamamoto method (2), which utilizes vector autoregression. We will refer to a variable X improving the forecasting of a variable Y as X ⇒ Y.

We do not find evidence that NIH budget ⇒ NME (p=0.475), and thus it may not be a good indicator of biomedical research efficiency. However, we do find evidence that NIH budget ⇒ LE (p<10-8) and NIH budget ⇒ publications (p<machine precision). Notably, however, both VAR models utilize the maximum possible time lags (15, as selected using the Akaike information criterion (3), coincidentally the same number as used in this paper), and do not pass the Breusch-Godfrey test (4) for serially correlated residuals. As this suggests that more time lags are required to build appropriately rigorous models, it seems unwise to over-interpret the potential Granger causal relationships or make any comparisons between the Granger causality during different periods of time, until significantly more time points are available. Even with additional data, the serial correlation in the residuals might not be alleviated without the use of more complex models including non-linear terms or external variables. All three of these possible limitations also affect the analysis in this paper, but no statistical tests were performed there to assess the robustness of results.

We conclude that this published study of biomedical research efficiency is insufficient methodologically because models of greater complexity are required. From our reanalysis, we are not able to support the hypothesis that biomedical research efficiency is decreasing. Instead, we can only conclude that from 1965-2012, the NIH budget may have a causal effect on LE and publication, but that more time points are required to improve the models.

Another recent work (5), which aimed to study the scientific process and its efficiency, also suggested that the efficiency of biomedicinal chemistry research (defined in terms of the number of experiments that would need to be performed to discover a given fraction of all knowledge) has decreased over time. However, the analysis in (5) also suggested that even the optimal research strategy would eventually display a decrease in efficiency, due to the intrinsic increase in the difficulty of discovery as a field matures. While there are additional limitations and assumptions involved in this analysis, (5) provides an example of the level of complexity and quantitative rigor required to study research efficiency, and implies an alternative explanation for the potential reduction in biomedical research efficiency. Considering the findings in (5), we deem the hypotheses proposed in this paper, which suggests that a lack of relevance or rigor in biomedical research is causing stagnation in medicine and public health, to be unfounded.

We write this comment in part because unfounded defamatory claims directed at the scientific community are dangerous in that they may negatively affect the future of scientific funding. Such claims should not be made lightly, and the principle of parsimony should be invoked when less defamatory alternative hypotheses are available.

Michael V. LeVine and Harel Weinstein

(1) Granger CWJ (1969) Investigating Causal Relations by Econometric Models and Cross-spectral Methods. Econometrica 37(3):424–438.

(2) Toda HY, Yamamotob T (1995) Statistical inference in vector autoregressions with possibly integrated processes. J Econom 66:225–250.

(3) Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723.

(4) Breusch TS (1978) Testing for autocorrelation in dynamic linear models. Aust Econ Pap 17(31):334–355.

(5) Rzhetsky A, Foster JG, Foster IT, Evans JA (2015) Choosing experiments to accelerate collective discovery. Proc Natl Acad Sci:201509757.

On 2016 Jan 14, Arturo Casadevall commented:

We appreciate the comment by Dr. Schuck-Paim and we fully agree that increased transparency and reporting of data during the process of therapeutic development can only benefit the scientific enterprise and enable a more efficient use of limited resources. Some of the other mentioned issues, including relevance of animal models, underpowered study design, and errors during data analysis and reporting, have all been implicated in the literature as referenced by Dr. Schuck-Paim and cited in our paper. We agree that each of these issues merits attention and expect that new tools will need to be developed to address some problems. One example would be the development of drug screening chips containing human cells as an alternative to some animal models, which may have poor predictability of a drug’s toxicity in humans (http://www.ncats.nih.gov/tissuechip).

Anthony Bowen

Arturo Casadevall

On 2015 Nov 05, Cynthia Schuck-Paim commented:

In exploring progress in biomedical research, the authors show that human life expectancy and the number of new molecular entities (NME) approved by the FDA have remained relatively constant over the last decades, despite increasing financial input, research efforts and publication numbers.

To explain the slowing of therapeutic innovation they consider several negative pressures acting on the field, including prior resolution of simpler research problems, increasing regulation, overreliance on reductionist approaches (including use of animal models), and the poor quality of published research. The high prevalence of irreproducible results, obscure methods, poorly designed research and publication biases are also mentioned.

Many of these issues would greatly benefit from initiatives that promote transparency at the various stages of the therapeutic development pipeline. It has been widely acknowledged that poor reporting prevents the accurate assessment of drug and intervention efficacy. Indeed, pre-clinical research and in vivo methods have been shown to be particularly prone to biases and selective reporting of outcomes, leading to bad decision-making, wasted resources, unnecessary replication of efforts and missed opportunities for the development of effective drugs (1). One proposal to address this issue is the extension of good disclosure practice to the pre-clinical phase by conditioning publication to the registration of the pre-clinical trial prior to the commencement of the study (2). Indeed, the exact same reasons that compelled prospective registration and deposition of clinical trial results in public databases apply to preclinical studies.

Still, no matter how transparent, well-designed, -analyzed and -reported the research, results generated from inappropriate models will not be successfully translated into valid disease contexts. Currently, most pre-clinical studies are based on the use of animal models, despite the increasing number of articles showing that they are an expensive and ineffective option to explore pathophysiological mechanisms, evaluate therapeutics, and decide on whether drug candidates should be carried forward into the clinical phase (3-5).

Failure rates in the clinical phase are around 95% (6), mainly due to the limited power of animal studies to predict NME efficacy, safety and toxicity in humans. These predictive odds vary depending on the understanding and complexity of disease biology: while for therapeutics targeting infectious diseases success rates are higher, for diseases involving complex mechanisms, such as cancer, they can be as low as 2.3% (7). Such low predictability drains the entire system by funneling limited resources into outputs that often fail.

In addition, false negatives at the pre-clinical stage eliminate a large part of NMEs that may have succeeded otherwise. Let us not forget Aspirin, a blockbuster drug that would not make the preclinical trial phase if tested today given its unacceptably high toxicity in animal tests. Animal-based pre-clinical phases are certainly pivotal in explaining the small number of NMEs identified in the last decades. Implementation of methods that are more faithful to the human biology is crucial for the much needed progress to ameliorate human disease and suffering.

References

(1) Macleod MR, 2015 Risk of bias in reports of in vivo research: a focus for improvement. PLoS Biol 13: e1002273

(2) Kimmelman J, Anderson JA (2012). Should preclinical studies be registered? Nat Biotechnol 30: 488–489

(3) Hartung T (2013). Food for thought: look back in anger – what clinical studies tell us about preclinical work. Altex 30: 275–291.

(4) Sutherland BA, 2012 Neuroprotection for ischaemic stroke: translation from the bench to the bedside. Int J Stroke 7: 407–18.

(5) Seok J, 2013 Genomic responses in mouse models poorly mimic human inflammatory diseases. PNAS 110: 3507–12.

(6) Arrowsmith J, 2012 A decade of change. Nat Rev Drug Discov 11:17–18

(7) Hay M, 2014 Clinical development success rates for investigational drugs. Nat Biotechnol 32 (1): 40–51.

On 2015 Aug 31, Wichor Bramer commented:

In my opinion the conclusions of this article are rather overdrawn. The authors have determined that the coverage of PubMed is high enough to be used in reviews. However, they have not performed searches for a systematic review, they have performed multiple searches for known items. If recall would be ideal PubMed would retrieve enough relevant articles to be used as a single database. However, recall is never ideal in any database. My recent observations (which have not yet been published) show that Medline retrieved 937 out of 1192 included references for 38 published reviews, so only 79%. Embase and medline together retrieved 1110 references (94%). So overall recall in embase is much better than in medline alone. In my opinion overall recall is not the best measure for database usefulness. When deciding a strategy for a systematic review, one decides for only one review, and not for 50 of 38. A better parameter is the minimum number observed. In medline the minimum was 53%, compared to 76% in embase/medline. Neither is acceptable in my opinion. The authors of this article also found that for some reviews, the coverage in pubmed was much lower than the total coverage. One database cannot be used to replace all other databases in the search for a systematic review.

On 2017 Feb 09, Kevin Hall commented:

The theoretical basis of the carbohydrate-insulin model (CIM) relies on generally accepted physiology about the endocrine regulation of adipose tissue based on short-term experiments lasting days and weeks. While there are indeed metabolic adaptations that take place on longer time scales, many of these changes actually support the conclusion that the purported metabolic advantages for body fat loss predicted by the CIM are inconsistent with the data.

For example, as evidence for a prolonged period of fat adaptation, Ludwig notes modest additional increases in blood and urine ketones observed after 1 week of either starvation Owen OE, 1983 or consuming a hypocaloric ketogenic diet Yang MU, 1976. The implication is that daily fat and ketone oxidation presumably increase along with their blood concentrations over extended time periods to eventually result in an acceleration of body fat loss with low carbohydrate high fat diets as predicted by the CIM. But since acceleration of fat loss during prolonged starvation would be counterproductive to survival, might there be data supporting a more physiological interpretation the prolonged increase in blood and urine ketones?

Both adipose lipolysis Bortz WM, 1972 and hepatic ketone production Balasse EO, 1989 reach a maximum within 1 week as demonstrated by isotopic tracer data. Therefore, rising blood ketone concentrations after 1 week must be explained by a reduced rate of removal from the blood. Indeed, muscle ketone oxidation decreases after 1 week of starvation and, along with decreased overall energy expenditure, the reduction in ketone oxidation results in rising blood concentrations and increased urinary excretion (page 144-152 of Burstztein S, et al. ‘Energy Metabolism, Indirect Calorimetry, and Nutrition.’ Williams & Wilkins 1989). Therefore, rather than being indicative of progressive mobilization of body fat to increase oxidation and accelerate fat loss, rising concentrations of blood ketones and fatty acids occurring after 1 week arise from reductions in ketone and fat oxidation concomitant with decreased energy expenditure.

The deleterious effects of a 600 kcal/d low carbohydrate ketogenic diet on body protein and lean mass were demonstrated in Vasquez JA, 1992 and were found to last about 1 month. Since weight loss was not significantly different compared to an isocaloric higher carbohydrate diet, body fat loss was likely attenuated during the ketogenic diet and therefore in direct opposition to the CIM predictions. Subsequent normalization of nitrogen balance would tend to result in an equivalent rate of body fat loss between the isocaloric diets over longer time periods. In Hall KD, 2016, urinary nitrogen excretion increased for 11 days after introducing a 2700 kcal/d ketogenic diet and coincided with attenuated body fat loss measured during the first 2 weeks of the diet. The rate of body fat loss appeared to normalize in the final 2 weeks, but did not exceed the fat loss observed during the isocaloric high carbohydrate run-in diet. Mere normalization of body fat and lean tissue loss over long time periods cannot compensate for early deficiencies. Therefore, these data run against CIM predictions of augmented fat loss with lower carbohydrate diets.

Ludwig uses linear extrapolation to claim that our data “would imply a 13 kg greater body fat loss versus the higher-fat diet over a year”. However, the same computational model that correctly predicted the difference in short-term body fat loss projected only small differences in long-term body fat between the diets. Based on these model simulations we concluded that “the body acts to minimize body fat differences with prolonged isocaloric diets varying in carbohydrate and fat.”

While I believe that outpatient weight loss trials demonstrate that low carbohydrate diets often outperform low fat diets over the short-term, there are little body weight differences over the long-term Freedhoff Y, 2016. However, outpatient studies cannot ensure or adequately measure diet adherence and therefore it is unclear whether greater short-term weight losses with low carbohydrate diets were due to reduced diet calories or the purported “metabolic advantages” of increased energy expenditure and augmented fat loss predicted by the CIM. The inpatient controlled feeding studies demonstrate that the observed short-term energy expenditure and body fat changes often violate CIM predictions.

Ludwig conveniently suggests that all existing inpatient controlled feeding studies have been too short and that longer duration studies might produce results more favorable to the CIM. But even this were true, the current data demonstrate repeated violations of CIM model predictions and constitute experimental falsifications of the CIM. This possibility was accurately described in my review Hall KD, 2017 and requires an ad hoc modification of the CIM such that the metabolic advantages of isocaloric lower carbohydrate diets only begin after a time lag lasting many weeks – a possibility currently unsupported by data but obviously supported by sincere belief.

On 2017 Feb 07, DAVID LUDWIG commented:

In his comment of 31 January 2017, Hall continues to insist that the results of his 6-day study and other very short feeding studies of substrate oxidation inform understanding of the long-term relationship between diet and body composition. This contention can be simply dismissed, with recognition that the 36 g/d advantage in fat oxidation on Hall’s low-fat diet would imply a 13 kg greater body fat loss versus the higher-fat diet over a year. There is simply no precedent for such an effect, and if anything the long-term clinical trials suggest the opposite Tobias DK, 2015 Mansoor N, 2016 Mancini JG, 2016 Sackner-Bernstein J, 2015 Bueno NB, 2013.

The reason short term studies of high-fat diets are misleading is that the process of adapting to reduced carbohydrate intake can take several weeks. We can clearly observe this phenomenon in 4 published graphs involving very-low-carbohydrate diets.

For convenience, these figures can be viewed at this link:

Owen OE, 1983 Figure 1. Ketones are, of course, the hallmark of adaptation to a very-low-carbohydrate (ketogenic) diet. Generally speaking, the most potent stimulus of ketosis is fasting, since the consumption of all gluconeogenic precursors (carbohydrate and protein) is zero. As this figure shows, the blood levels of each of the three ketone species (BOHB, AcAc and acetone) continues to rise for ≥3 weeks. Indeed, the prolonged nature of adaptation to complete fasting has been known since the classic starvation studies of Cahill GF Jr, 1971. It stands to reason that this process might take even longer on standard low-carbohydrate diets, which inevitably provide ≥ 20 g carbohydrate/d and substantial protein.

Yang MU, 1976 Figure 3A. Among men with obesity on an 800 kcal/d ketogenic diet (10 g/d carbohydrate, 50 g/d protein), urinary ketones continued to rise for 10 days through the end of the experiment, and by that point had achieved levels equivalent only to those on day 4 of complete fasting. Presumably, this process would be even slower with a non-calorie restricted ketogenic diet (because of inevitably higher carbohydrate and protein content).

Vazquez JA, 1992 Figure 5B. On a conventional high-carbohydrate diet, the brain is critically dependent on glucose. With acute restriction of dietary carbohydrate (by fasting or a ketogenic diet), the body obtains gluconeogenic precursors by breaking down muscle. However, with rising ketone concentrations, the brain becomes adapted, sparing glucose. In this way, the body shifts away from protein to fat metabolism, sparing lean tissue. This process is clearly depicted among women with obesity given a calorie-restricted ketogenic diet (10 g carbohydrate/d) vs a nonketogenic diet (76 g carbohydrate/d), both with protein 50 g protein/d. For 3 weeks, nitrogen balance was strongly negative on the ketogenic diet compared to the non-ketogenic diet, but this difference was completely abolished by week 4. What would subsequently happen? We simply can’t know from the short-term studies.

Hall KD, 2016 Figure 2B. Another study by Hall shows that the transient decrease in rate of fat loss upon initiation of the ketogenic diet accelerates after 2 weeks.

The existence of this prolonged adaptive process explains why metabolic advantages for low-fat diet are consistently seen in very short metabolic studies. But after 2 to 4 weeks, advantages for low-carbohydrate diets begin to emerge, Hall KD, 2016 Miyashita Y, 2004 Ebbeling CB, 2012. Any meaningful conclusions about the long-term effects of macronutrients must await longer studies.

On 2017 Jan 31, Kevin Hall commented:

My recent review of the carbohydrate-insulin model Hall KD, 2017 presented a synthesis of the evidence from 20 inpatient controlled feeding studies strongly suggesting that at least some important aspects of the model are in need of modification. In particular, our recent studies Hall KD, 2015, Hall KD, 2016 employing carefully controlled inpatient isocaloric diets with constant protein, but differing in carbohydrate and fat, resulted in statistically significant differences between the diets regarding body fat and energy expenditure that were in directions opposite to predictions of the carbohydrate-insulin model.

Ludwig comments that the diets used in Hall KD, 2015 were either too low in fat or insufficiently low in carbohydrate. However, while these considerations may be clinically important for sustainability of the diets, they are irrelevant to whether the diets resulted in a valid test of the carbohydrate-insulin model predictions. We selectively reduced 30% of baseline calories solely by restricting either carbohydrate or fat. These diets achieved substantial differences in daily insulin secretion as measured by ~20% lower 24hr urinary C-peptide excretion with the reduced carbohydrate diet as compared with the reduced fat diet (p= 0.001) which was unchanged from baseline. Whereas the reduced fat diet resulted in no significant energy expenditure changes from baseline, carbohydrate restriction resulted in a ~100 kcal/d decrease in both daily energy expenditure and sleeping metabolic rate. These results were in direct opposition to the carbohydrate-insulin model predictions, but in accord with the previous studies described in the review as well as a subsequent study demonstrating that lower insulin secretion was associated with a greater reduction of metabolic rate during weight loss Muller MJ, 2015.

While the DXA methodology was not sufficiently precise to detect significant differences in body fat loss between the diets, even this null result runs counter to the predicted greater body fat loss with the reduced carbohydrate diet. Importantly, the highly sensitive fat balance technique demonstrated small but statistically significant differences in cumulative body fat loss (p<0.0001) in the direction opposite to the carbohydrate-insulin model predictions. Ludwig claims that our results are invalid because “rates of fat oxidation, the primary endpoint, are exquisitely sensitive to energy balance. A miscalculation of available energy for each diet of 5% in opposite directions could explain the study’s findings.” However, it is highly implausible that small uncertainties in the metabolizable energy content of the diet amounting to <100 kcal/d could explain the >400 kcal/d (p<0.0001) measured difference in daily fat oxidation rate. Furthermore, our results were robust to the study errors and exclusions described in the report and our observations clearly falsified important aspects of the carbohydrate-insulin model.

Ludwig argues that “it can take the body weeks to fully adapt to a high fat diet”, However, daily fat oxidation has been observed to plateau within the first week when added dietary fat is accompanied by an isocaloric reduction in carbohydrate as indicated by the rapid and sustained drop in daily respiratory quotient in Hall KD, 2016 and Schrauwen P, 1997. Similarly, Hall KD, 2015 observed a decrease and plateau in daily respiratory quotient with the reduced carbohydrate diet, whereas the reduced fat diet resulted in no significant changes indicating that daily fat oxidation was unaffected. As further evidence that adaptations to carbohydrate restriction occur relatively quickly, adipose tissue lipolysis is known to reach a maximum within the first week of a prolonged fast Bortz WM, 1972 as does hepatic ketone production Balasse EO, 1989.

While there is no evidence that carbohydrate restricted diets lead to an acceleration of daily fat oxidation on time scales longer than 1 week, and there is no known physiological mechanism for such an effect, this possibility cannot be ruled out. Such speculative long term effects constitute an ad hoc modification of the carbohydrate-insulin model whereby violations of model predictions on time scales of 1 month or less are somehow reversed.

Ludwig is correct that it takes the body a long time to equilibrate to added dietary fat because, unlike carbohydrate and protein, dietary fat does not directly promote its own oxidation and does not significantly increase daily energy expenditure Schutz Y, 1989 and Horton TJ, 1995. Unfortunately, these observations run counter to carbohydrate-insulin model predictions because they imply that added dietary fat results in a particularly efficient means to accumulate body fat compared to added carbohydrate or protein Bray GA, 2012. If such an added fat diet is sustained, adipose tissue will continue to expand until lipolysis is increased to sufficiently elevate circulating fatty acids and thereby increase daily fat oxidation to reestablish balance with fat intake Flatt JP, 1988.

Of course, differences in long term ad libitum food intake between diets varying in macronutrient composition could either obviate or amplify any predicted body fat differences based solely on fat oxidation or energy expenditure considerations. Such mechanisms warrant further investigation and will inform improved models of obesity. Nevertheless, it is clear that several important aspects of the carbohydrate-insulin model have been experimentally falsified by a variety of studies, including Hall KD, 2015.

On 2017 Jan 17, DAVID LUDWIG commented:

In a recent review, the first author of this Cell Metabolism article (Kevin Hall) cites this 6-day study as a basis for having “falsified” the Carbohydrate-Insulin Model of obesity. That argument disregards some key limitations of this study, which warrant elucidation.

In the discussion section of this study, Hall and colleagues write: “Our relatively short-term experimental study has obvious limitations in its ability to translate to fat mass changes over prolonged durations” (NB, it can take the body weeks to fully adapt to a high fat diet Hawley JA, 2011 Vazquez JA, 1992 Veum VL, 2017). Beyond short duration and confounding by transient biological adaptations, the study: 1) did not find a difference in actual fat mass by DXA (p=0.78); 2) used an exceptionally low fat content for the low-fat diet (< 8% of total energy), arguably without precedent in any population consuming natural diets; 3) used a relatively mild restriction of carbohydrate (30% of total energy), well short of typical very-low-carbohydrate diets; 4) had protocol errors and post-randomization data exclusions that could confound findings; and 5) failed to verify biologically available energy of the diet (e.g., by analysis of the diets and stools for energy content). Regarding this last point, rates of fat oxidation, the primary endpoint, are exquisitely sensitive to energy balance. A miscalculation of available energy for each diet of 5% in opposite directions could explain the study’s findings – and this possibility can’t be ruled out in studies of such short duration.

Thus, this study should not be interpreted as providing a definitive test of the Carbohydrate-Insulin Model.