On 2015 Apr 18, Maurice Smith commented:

None

On 2015 Apr 20, Dorothy V M Bishop commented:

This is drifting a bit to a new topic, but just to let you know I was inspired by the last two sentences of this reply to write a blogpost on the impact of page length limits on scientific communication: http://deevybee.blogspot.co.uk/2015/04/how-long-does-scientific-paper-need-to.html

On 2015 Apr 18, Maurice Smith commented:

We agree with the sentiment conveyed by the above comment, that overhyping scientific results is a problem. We do not, however, think that Dr. Brembs is on solid ground with his various charges about our paper (Wu et al. 2014). It is, of course, the specifics that matter. And the specific charges made in the above comment and various blog posts authored by Dr. Brembs lack any nuance and end up being rather misleading. What follows is an attempt to clarify the record.

Reading the introduction section of our paper should provide a direct counter to Dr. Brembs main charge that we fail to acknowledge previous work on reinforcement learning. Nearly 40% of our paper’s introduction (the entirety of paragraphs 3-4), where we lay out the motivation for our study, is devoted to a discussion of previous work about the idea that motor variability might enhance motor learning ability. We specifically acknowledge that this idea comes from both reinforcement learning theory (2 citations, #16-17) and motor learning studies in animals (6 additional citations: 4 in songbirds, #12-15, 2 in rodents, #18-19). While we did not cite the particular references mentioned in Dr. Brembs’ comment, we argue that the studies we do cite are more germane and cover essentially the same ground. We are as big fans of Skinner as is Dr. Brembs, but the Wu et al. (2014) paper is not a historic review on operant conditioning and reinforcement learning. When faced with a strict journal policy citation limit (50 for the whole paper), one cannot appropriately cover the intellectual history of an entire field, which is why we cited a book and a review where the interested reader can get this information (and yes, Skinner is cited there). We believe the citations we included are appropriate for readers to meaningfully appreciate the context of our study in relation to key previous ideas and scientific work.

Dr Brembs’ suggestion that we overhype our results by suggesting that they are “surprising” is taken completely out of context, as the quote he uses cuts out key information from the very sentence it comes from. The full sentence referred to in his comment reads: “Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning BOTH across individuals AND across tasks in TWO different paradigms, one relying on REWARD-BASED learning to shape specific arm movement trajectories and the other relying on ERROR-BASED LEARNING to adapt movements in novel physical environments.” (emphasis added here). Here the word “surprisingly” clearly refers to the constellation of our findings, which in our view, and the view of our paper’s peer reviewers, is indeed surprising and novel. This is in sharp contrast to the out-of-context extraction of the first phrase in Dr. Brembs’ comments: “Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning,” which misconstrues the meaning of the sentence.

Regarding the specific citations that Dr. Brembs suggest we inappropriately left out (Neuringer 2002, Shea & Morgan 1979, Schmidt & Bjork 1992), a close examination reveals that these citations are for findings that are either disputed or not very relevant. The Neuringer (2002) study in rats used variability as an operant to experimentally induce different levels of variability. However, there are several confounds as well as methodological and interpretive difficulties with these experiments, making it very difficult to draw any conclusions regarding the relationship between the structure of normally expressed trial-to-trial motor variability and learning ability. These various issues are discussed in subsequent papers that aimed to replicate Neuringer’s finding (see Maes and van der Goot, Learning and Motivation 2006 and Doolan and Bizo, The Psychological Record, 2013). These studies, conducted in humans and hence more relevant to our findings, could not replicate Neuringer’s result. In fact, both showed an opposite effect, i.e. going against what Skinner and reinforcement learning theory would have predicted.

The Shea & Morgan (1979) and Schmidt & Bjork (1992) studies, which Dr. Brembs also claims are similar to ours, are perhaps even less relevant. These studies are squarely about what has been termed “contextual interference”, i.e. effects that arise when switching between tasks when multiple different tasks are trained in an interspersed fashion. These studies have nothing to do with trial-to-trial variability, but variability in task structure. We, in contrast, studied learning within a single task, precluding any such interference effects. If one were to compare the findings from these papers to our study in an “apples to apples” manner, and we strongly believe that one should not, they would predict the opposite of our results. In the contextual interference studies, the benefit of interspersed training is not seen in the initial learning rate. In fact, the initial learning rate is markedly decreased (in line with the moniker “contextual interference”), exactly the opposite of what we report. Again, this difference is not surprising given how different the studies are. The benefit in learning seen with contextual interference is in the long-term retention, an issue that we did not investigate at all in Wu et al 2014. As a primary research paper with strict limits on the number of words and citations, ours cannot be expected to be a scholarly review of various tangential studies.

Finally, it should also be noted that Dr. Brembs’ comment refers to a small fraction of our paper’s content, and as such any protracted discussion of the pertaining results would have been at the expense of discussion points relating to other findings, which Dr. Brembs himself found more interesting. We can reassure the reader that it was very painful to cut down the discussion, introduction, and citations to conform to Nature Neuroscience’s strict and rather arbitrary limits. We would personally be in favor of expanding these limits, or doing away with them entirely, but this is not our choice to make. Given these constraints, we are comfortable with the choices we made and do not agree that they misrepresent the context or novelty of our findings.

On 2015 Apr 14, Björn Brembs commented:

“Standing on the shoulders of giants” is what scientists say to acknowledge the work they are building on. It is a statement of humility and mostly accompanied by citations to the primary literature preceding the current work. In today’s competitive scientific enterprise, however, such humility appears completely misplaced. Instead, what many assume to be required in the struggle to survive is to convince everyone that they are the giant, the genius, the prodigy who is deserving of the research funds, the next position, tenure.

The Nature Neuroscience article “Temporal structure of motor variability is dynamically regulated and predicts motor learning ability” by Wu et al. with its accompanying news-type article “Motor variability is not noise, but grist for the learning mill” by Herzfeld and Shadmehr (linked above) can only be described as over-hyping an otherwise very interesting discovery. Both articles claim that the researchers have made the game-changing discovery that something long thought to be a bug in our movement system is actually a spectacular feature. It is argued that this discovery is such a huge surprise, because nobody in their right mind would have ever thought this “unwanted characteristic” to actually serve some purpose.

The problem with this line of argument is that probably most people in the field thought it should be obvious, even to be expected – and not surprising at all. Skinner is largely credited with the analogy of operant conditioning and evolution. This analogy entails that reward and punishment act on behaviors like selection is acting on mutations in evolution: an animal behaves variably and encounters a reward after it initiated a particular action. This reward will make the action now more likely to occur in the future, just as selection will make certain alleles more frequent in a population. Already in 1981, Skinner called this “Selection by Consequences“ (Science Vol. 213 no. 4507 pp. 501-504, DOI: 10.1126/science.7244649). Skinner’s analogy sparked wide interest, e.g. an entire journal issue (Behavioral and Brain Sciences 7(04), 1984), which later appeared in book form (The Selection of Behavior: The Operant Behaviorism of B. F. Skinner: Comments and Consequences. A. Charles Catania, Stevan R. Harnad, Cambridge University Press). Clearly, the idea that reinforcement selects from a variation of different behaviors is not a novel concept at all, but more than three decades old and rather prominent. This analogy cannot have escaped anybody working on any kind of operant/motor learning, except those seriously neglecting the most relevant literature. This interaction of variability and selection is a well-known and not overly complicated concept, based in evolutionary biology and psychology/neuroscience. Consequently, numerous laboratories have been studying various aspects of this interaction for a long time. Skinner’s projection was that increased behavioral variability leads to increased operant learning rates, just like increased mutations rates lead to increased rates of evolutionary change. More than a dozen years ago, Allen Neuringer showed this to be the case in rats (Psychonomic Bulletin & Review 2002, 9 (2), 250-258, doi: 10.3758/BF03196279), but there are studies in humans as well (Shea, J. B., & Morgan, R. B. (1979). Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5, 179–187). That such variability is beneficial, rather than detrimental has been shown even in situations where the variability is so high, that the acquisition rate is reduced, but post-training performance is enhanced (Schmidt RA, Bjork RA (1992): New conceptualizations of practice: Common Principles in Three Paradigms Suggest New Concepts for training. Psychological Science, 3(4): 207-217).

Wu et al. confirm both Skinner’s conjecture as well as previously published reports (some cited above) that indeed the rate of learning in operant conditioning is increased in subjects where the initial variability in the behavior is higher. This is an important and relevant finding. However, instead of citing the wealth of earlier work, Wu et al. claim that their results were surprising: “Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning”. Herzfeld and Shadmehr were similarly stunned: “These results provide intriguing evidence that some of the motor variability commonly attributed to unwanted noise is in fact exploration in motor command space.”

I regard it as highly unlikely that none of the seven authors in total should have never heard of Skinner or the work over the last four decades by many human movement scientists that have explored the temporal structure of human movement variability and its relationship with motor learning. The work by senior scientists such as Karl Newell, Michael Turvey, Richard Schmidt, and their students published in books and hundreds of journal articles is completely ignored, just as the work by several younger mid-career scientists such as Nick Stergiou, Jeff Hausdorff, Thurmon Lockhart, Didier Dilignieres, and many others. After a thorough review of this literature the authors may realize that some of their results are neither new nor novel. If indeed the authors were unaware of this entire section of literature so relevant to their own research, it would be an indictment in its own right.

It needs to be emphasized explicitly, that the above does not call into question the validity of the research results, nor does it imply that the described results do not merit publication. Clearly, the research described in Wu et al. is relevant, interesting and it was absolutely correct to publish it in its entirety. What ought to have happened, however, is to encourage the authors to include the relevant references in the appropriate sections of their articles.

(Much of this comment was drafted together with Dr. Nick Stergiou)

On 2015 Apr 14, Björn Brembs commented:

“Standing on the shoulders of giants” is what scientists say to acknowledge the work they are building on. It is a statement of humility and mostly accompanied by citations to the primary literature preceding the current work. In today’s competitive scientific enterprise, however, such humility appears completely misplaced. Instead, what many assume to be required in the struggle to survive is to convince everyone that they are the giant, the genius, the prodigy who is deserving of the research funds, the next position, tenure.

The Nature Neuroscience article “Temporal structure of motor variability is dynamically regulated and predicts motor learning ability” by Wu et al. with its accompanying news-type article “Motor variability is not noise, but grist for the learning mill” by Herzfeld and Shadmehr (linked above) can only be described as over-hyping an otherwise very interesting discovery. Both articles claim that the researchers have made the game-changing discovery that something long thought to be a bug in our movement system is actually a spectacular feature. It is argued that this discovery is such a huge surprise, because nobody in their right mind would have ever thought this “unwanted characteristic” to actually serve some purpose.

The problem with this line of argument is that probably most people in the field thought it should be obvious, even to be expected – and not surprising at all. Skinner is largely credited with the analogy of operant conditioning and evolution. This analogy entails that reward and punishment act on behaviors like selection is acting on mutations in evolution: an animal behaves variably and encounters a reward after it initiated a particular action. This reward will make the action now more likely to occur in the future, just as selection will make certain alleles more frequent in a population. Already in 1981, Skinner called this “Selection by Consequences“ (Science Vol. 213 no. 4507 pp. 501-504, DOI: 10.1126/science.7244649). Skinner’s analogy sparked wide interest, e.g. an entire journal issue (Behavioral and Brain Sciences 7(04), 1984), which later appeared in book form (The Selection of Behavior: The Operant Behaviorism of B. F. Skinner: Comments and Consequences. A. Charles Catania, Stevan R. Harnad, Cambridge University Press). Clearly, the idea that reinforcement selects from a variation of different behaviors is not a novel concept at all, but more than three decades old and rather prominent. This analogy cannot have escaped anybody working on any kind of operant/motor learning, except those seriously neglecting the most relevant literature. This interaction of variability and selection is a well-known and not overly complicated concept, based in evolutionary biology and psychology/neuroscience. Consequently, numerous laboratories have been studying various aspects of this interaction for a long time. Skinner’s projection was that increased behavioral variability leads to increased operant learning rates, just like increased mutations rates lead to increased rates of evolutionary change. More than a dozen years ago, Allen Neuringer showed this to be the case in rats (Psychonomic Bulletin & Review 2002, 9 (2), 250-258, doi: 10.3758/BF03196279), but there are studies in humans as well (Shea, J. B., & Morgan, R. B. (1979). Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5, 179–187). That such variability is beneficial, rather than detrimental has been shown even in situations where the variability is so high, that the acquisition rate is reduced, but post-training performance is enhanced (Schmidt RA, Bjork RA (1992): New conceptualizations of practice: Common Principles in Three Paradigms Suggest New Concepts for training. Psychological Science, 3(4): 207-217).

Wu et al. confirm both Skinner’s conjecture as well as previously published reports (some cited above) that indeed the rate of learning in operant conditioning is increased in subjects where the initial variability in the behavior is higher. This is an important and relevant finding. However, instead of citing the wealth of earlier work, Wu et al. claim that their results were surprising: “Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning”. Herzfeld and Shadmehr were similarly stunned: “These results provide intriguing evidence that some of the motor variability commonly attributed to unwanted noise is in fact exploration in motor command space.”

I regard it as highly unlikely that none of the seven authors in total should have never heard of Skinner or the work over the last four decades by many human movement scientists that have explored the temporal structure of human movement variability and its relationship with motor learning. The work by senior scientists such as Karl Newell, Michael Turvey, Richard Schmidt, and their students published in books and hundreds of journal articles is completely ignored, just as the work by several younger mid-career scientists such as Nick Stergiou, Jeff Hausdorff, Thurmon Lockhart, Didier Dilignieres, and many others. After a thorough review of this literature the authors may realize that some of their results are neither new nor novel. If indeed the authors were unaware of this entire section of literature so relevant to their own research, it would be an indictment in its own right.

It needs to be emphasized explicitly, that the above does not call into question the validity of the research results, nor does it imply that the described results do not merit publication. Clearly, the research described in Wu et al. is relevant, interesting and it was absolutely correct to publish it in its entirety. What ought to have happened, however, is to encourage the authors to include the relevant references in the appropriate sections of their articles.

(Much of this comment was drafted together with Dr. Nick Stergiou)

On 2015 Apr 18, Maurice Smith commented:

None