On 2016 Oct 11, UFRJ Neurobiology and Reproducibility Journal Club commented:

In this article, the authors demonstrate that the response to an aversively conditioned context is different between rats that have undergone conditioning individually or collectively, and suggest that this is mediated by olfaction.

Despite the interesting findings, we would like to point out that the statistical methods used for the results shown in Figures 2E and 4C do not support the conclusions made by the authors. In Figure 1B, the illustrated result was that the collectively conditioned group presented a significant decrease (p<0.01) in fear response 24 h after training when compared to animals conditioned individually. The same kind of comparison (collective vs. individual conditioning) was performed in Figure 2E, with the test performed 3.5 h after training; in this case, no statistical significance (p> 0.1) was found (even though a trend towards reduced freezing in the collectively conditioned group can be observed in the figure). Based on this, the authors conclude that “these data argue strongly that fear acquisition was similar in both groups and that neuronal mechanisms mediating fear memory consolidation and/or expression are involved in the markedly attenuated fear memory in Group animals.”

This conclusion, however, is based on a common statistical misconception: namely, to infer that, if a significant difference at a given threshold is found between two groups in one condition (i.e. collective conditioning) but not in another (i.e. individual condition), this implies that there is a significant difference between the two conditions. This statistical reasoning is erroneous, since the differences found in both conditions can be similar in magnitude (as they seem to be in this case), and the fact that one but not the other is found to be significant at an arbitrary significance threshold can be a consequence of statistical power, variability, or mere chance. As discussed by (Nieuwenhuis S, 2011) to argue in favor of a difference in magnitude between two differences found in individual comparisons, the correct statistical approach would be to test whether there is a significant interaction between the two independent variables (in this case, group and time of testing), and not simply report that an effect was statistically significant while the other it was not.

The same error occurs in Figure 4C, in which the collectively conditioned sham group presented a significant reduction compared to individually conditioned sham rats (p < 0.05) individually, while this difference between types of conditioning was not found (p > 0.1) in anosmic rats. Although the difference in this case indeed seems larger in the sham group, results for the interaction between type of conditioning and anosmia were not shown, and the post-hoc results presented cannot be used to conclude that the sense of smell is responsible for the difference between individual and collective conditioning. This does not mean that the sense of smell does not play a role, but to support this conclusion the authors should present a statistical analysis that actually tests this hypothesis.

On 2016 Oct 11, UFRJ Neurobiology and Reproducibility Journal Club commented:

None

On 2016 Oct 11, UFRJ Neurobiology and Reproducibility Journal Club commented:

None

On 2016 Oct 11, UFRJ Neurobiology and Reproducibility Journal Club commented:

In this article, the authors demonstrate that the response to an aversively conditioned context is different between rats that have undergone conditioning individually or collectively, and suggest that this is mediated by olfaction.

Despite the interesting findings, we would like to point out that the statistical methods used for the results shown in Figures 2E and 4C do not support the conclusions made by the authors. In Figure 1B, the illustrated result was that the collectively conditioned group presented a significant decrease (p<0.01) in fear response 24 h after training when compared to animals conditioned individually. The same kind of comparison (collective vs. individual conditioning) was performed in Figure 2E, with the test performed 3.5 h after training; in this case, no statistical significance (p> 0.1) was found (even though a trend towards reduced freezing in the collectively conditioned group can be observed in the figure). Based on this, the authors conclude that “these data argue strongly that fear acquisition was similar in both groups and that neuronal mechanisms mediating fear memory consolidation and/or expression are involved in the markedly attenuated fear memory in Group animals.”

This conclusion, however, is based on a common statistical misconception: namely, to infer that, if a significant difference at a given threshold is found between two groups in one condition (i.e. collective conditioning) but not in another (i.e. individual condition), this implies that there is a significant difference between the two conditions. This statistical reasoning is erroneous, since the differences found in both conditions can be similar in magnitude (as they seem to be in this case), and the fact that one but not the other is found to be significant at an arbitrary significance threshold can be a consequence of statistical power, variability, or mere chance. As discussed by (Nieuwenhuis S, 2011) to argue in favor of a difference in magnitude between two differences found in individual comparisons, the correct statistical approach would be to test whether there is a significant interaction between the two independent variables (in this case, group and time of testing), and not simply report that an effect was statistically significant while the other it was not.

The same error occurs in Figure 4C, in which the collectively conditioned sham group presented a significant reduction compared to individually conditioned sham rats (p < 0.05) individually, while this difference between types of conditioning was not found (p > 0.1) in anosmic rats. Although the difference in this case indeed seems larger in the sham group, results for the interaction between type of conditioning and anosmia were not shown, and the post-hoc results presented cannot be used to conclude that the sense of smell is responsible for the difference between individual and collective conditioning. This does not mean that the sense of smell does not play a role, but to support this conclusion the authors should present a statistical analysis that actually tests this hypothesis.