On 2016 Dec 07, Joanne Kamens commented:

Thank you for this helpful set of definitions and clarifications. Common language will make the discussion more productive. I was prompted by an excellent blog by Hilda Bastian (http://blogs.plos.org/absolutely-maybe/2016/12/05/reproducibility-crisis-timeline-milestones-in-tackling-research-reliability/) and the subsequent twitter conversation to mention that these definitions don't address or even seem to mention the potential, influence or use of reagent/materials reproducibility. Experimental results and interpretation can be dramatically enhanced by the use of the correct standards, materials and/or reagents to reproduce a study. Protocol and methods sections alone are not sufficient to account for this as some reagents are not easily remade and are not always validated as being the same (unless subjected to quality control via repository storage or standard validation).

On 2016 Dec 07, Joanne Kamens commented:

Thank you for this helpful set of definitions and clarifications. Common language will make the discussion more productive. I was prompted by an excellent blog by Hilda Bastian (http://blogs.plos.org/absolutely-maybe/2016/12/05/reproducibility-crisis-timeline-milestones-in-tackling-research-reliability/) and the subsequent twitter conversation to mention that these definitions don't address or even seem to mention the potential, influence or use of reagent/materials reproducibility. Experimental results and interpretation can be dramatically enhanced by the use of the correct standards, materials and/or reagents to reproduce a study. Protocol and methods sections alone are not sufficient to account for this as some reagents are not easily remade and are not always validated as being the same (unless subjected to quality control via repository storage or standard validation).