On 2017 Apr 01, Lydia Maniatis commented:

Last sentence of abstract: "Our findings suggest participants integrate shape, motion, and optical cues to infer stiffness, with optical cues playing a major role for our range of stimuli."

The authors offer no criterion for "range of stimuli," even though they clearly limit their conclusions to this undefined set. This means that. if one wanted to attempt a replication, it would not be clear what "range of stimuli" would constitute a valid attempt. The relevance of this point can be appreciated in the context of statements like: "Compared with previous studies, we find a much less pronounced effect of shape cues compared with material cues (Han & Keyser, 2015, 2016; Paulun et al., 2017)." Speculation as to the reasons for this discrepancy are moot if we can't circumscribe our stimulus set in a theoretically clear way.

Also, the terms "shape, motion, and optical cues," as they are used by the authors, reflect an unproductive failure to distinguish between perceived and physical properties. Relevant physical properties of a stimulus are limited to the retinal stimulation it produces. The correct title for this paper would be "Inferring the stiffness of unfamiliar objects from their inferred surface properties, shapes, and motions."

Instead, the authors are treating stiffness as an inference and the rest as objective fact. (Not that thinking about perceptual qualities like stiffness, which seem more indirectly* inferred than the others, isn't interesting in itself, but the lines between perception and physics, and the relationships between them, shouldn't be blurred).

*Having said this, I don't think it's actually appropriate to characterize any perceived quality as more or less indirect than others. Even the seemingly simplest things - such as extent (which includes amodal completion), or lightness (which includes double layers, subjective contours), are not in any sense read directly off of the retinal stimulation.

The problem of confusing perceived and objective properties is the more acute given that the investigators aren't using real objects, but objects that rendered by a third-party computer program. "

"The render engine used to generate the final images was Maxwell 3.0.1.3 (NextLimit Technologies, Madrid, Spain).... Specifically, they were designed to approximate the following materials: black marble, white marble, porcelain, nickel, concrete paving, cement, ceramic, steel, copper, light wood, dark wood, silvered glass, glass, stone, leather, wax, gelatine, cardboard, plastic, paper, latex, cork, ice cream, lichen, waffle, denim, moss, and velvet. Some of these materials were downloaded or based on downloads from the Maxwell free resources library (http://resources.maxwellrender.com), and others were designed by us."

The authors are skipping all the good parts. What are the theoretical underpinnings of what are essentially assumptions about what various computer images will look like? Why is Maxwell's rendering of "velvet" equivalent, in terms of the retinal stimulation it generates, with real velvet? What are the criteria of a valid rendering of all of these perceived qualities and substances?

The criteria are empirical (see below), but loose. It is not clear that they are statistically valid, or how this could be assessed:

"Finally, Supplementary Figure S2 summarizes the results of the free material-naming task. Generally, they show that most of our renderings yielded compelling impressions of realistic materials that observers were able to reliably classify. In the following, the naming results were used to decide on the materials to test in Experiments 3 and 4 by choosing only materials that were identified as the same material by at least 50% of participants (see Stimuli section of Experiment 3)."

Fifty percent agreement seems like a pretty low bar. Why not at least 51% (which would still be low). Why not shoot for 100%? Is normal inter-individual variability in perception of materials this low in real-life? Or are the renderings generally inadequate? Even poor pictorial renderings of materials can contain cues - e.g. wood grain - which could produce seemingly clear answers that don't really reflect a valid percept in all the particulars. The very brief description of the naming task doesn't make it clear whether or not it was a forced answer, i.e. whether or not participants were allowed to say "not sure," which seems relevant.

On 2017 Apr 01, Lydia Maniatis commented:

Last sentence of abstract: "Our findings suggest participants integrate shape, motion, and optical cues to infer stiffness, with optical cues playing a major role for our range of stimuli."

The authors offer no criterion for "range of stimuli," even though they clearly limit their conclusions to this undefined set. This means that. if one wanted to attempt a replication, it would not be clear what "range of stimuli" would constitute a valid attempt. The relevance of this point can be appreciated in the context of statements like: "Compared with previous studies, we find a much less pronounced effect of shape cues compared with material cues (Han & Keyser, 2015, 2016; Paulun et al., 2017)." Speculation as to the reasons for this discrepancy are moot if we can't circumscribe our stimulus set in a theoretically clear way.

Also, the terms "shape, motion, and optical cues," as they are used by the authors, reflect an unproductive failure to distinguish between perceived and physical properties. Relevant physical properties of a stimulus are limited to the retinal stimulation it produces. The correct title for this paper would be "Inferring the stiffness of unfamiliar objects from their inferred surface properties, shapes, and motions."

Instead, the authors are treating stiffness as an inference and the rest as objective fact. (Not that thinking about perceptual qualities like stiffness, which seem more indirectly* inferred than the others, isn't interesting in itself, but the lines between perception and physics, and the relationships between them, shouldn't be blurred).

*Having said this, I don't think it's actually appropriate to characterize any perceived quality as more or less indirect than others. Even the seemingly simplest things - such as extent (which includes amodal completion), or lightness (which includes double layers, subjective contours), are not in any sense read directly off of the retinal stimulation.

The problem of confusing perceived and objective properties is the more acute given that the investigators aren't using real objects, but objects that rendered by a third-party computer program. "

"The render engine used to generate the final images was Maxwell 3.0.1.3 (NextLimit Technologies, Madrid, Spain).... Specifically, they were designed to approximate the following materials: black marble, white marble, porcelain, nickel, concrete paving, cement, ceramic, steel, copper, light wood, dark wood, silvered glass, glass, stone, leather, wax, gelatine, cardboard, plastic, paper, latex, cork, ice cream, lichen, waffle, denim, moss, and velvet. Some of these materials were downloaded or based on downloads from the Maxwell free resources library (http://resources.maxwellrender.com), and others were designed by us."

The authors are skipping all the good parts. What are the theoretical underpinnings of what are essentially assumptions about what various computer images will look like? Why is Maxwell's rendering of "velvet" equivalent, in terms of the retinal stimulation it generates, with real velvet? What are the criteria of a valid rendering of all of these perceived qualities and substances?

The criteria are empirical (see below), but loose. It is not clear that they are statistically valid, or how this could be assessed:

"Finally, Supplementary Figure S2 summarizes the results of the free material-naming task. Generally, they show that most of our renderings yielded compelling impressions of realistic materials that observers were able to reliably classify. In the following, the naming results were used to decide on the materials to test in Experiments 3 and 4 by choosing only materials that were identified as the same material by at least 50% of participants (see Stimuli section of Experiment 3)."

Fifty percent agreement seems like a pretty low bar. Why not at least 51% (which would still be low). Why not shoot for 100%? Is normal inter-individual variability in perception of materials this low in real-life? Or are the renderings generally inadequate? Even poor pictorial renderings of materials can contain cues - e.g. wood grain - which could produce seemingly clear answers that don't really reflect a valid percept in all the particulars. The very brief description of the naming task doesn't make it clear whether or not it was a forced answer, i.e. whether or not participants were allowed to say "not sure," which seems relevant.