On 2016 Dec 08, Lydia Maniatis commented:

Part 2: 2. The terms “adaptation” and “aftereffects.” There is no way to discern the meaning of these terms in the context of this study. The authors seem to be using the term very loosely: “….many aspects of visual perception are adaptive, such that the appearance of a given stimulus may be affected by what has been seen before.” The example they give involves motion aftereffects, the cause of which, as far as I can discover, is still unknown. Continuing, they declare that “This perceptual bias is known as an aftereffect [last term italicized].”

This exposition conflates all possible effects of experience on subsequent perceptual impressions with the color aftereffects based on color opponency at the retinal level, and with the less well-understood motion aftereffects. In other words, we’re lumping together the phenomenon known as “perceptual set,” for example, with color aftereffects, as well as with motion aftereffects. In these latter cases, at least, it is intelligible to talk about aftereffects as being “opposite” to the original effects. In the case of motion, we are fairly straightforwardly talking about a percept of a motion in the opposite direction. In the case of color, the situation is not based on percepts having perceptually opposing characteristics; what makes green the opposite of red is more physiological then perceptual. So even with respect to what the authors refer to as “low-level” effects, ‘opposite’ means rather different things.

The vagueness of the ‘opposite’ concept as used by Burton et al (2016) is expressed in their placement of quotation marks around the term: “In the facial expression aftereffect, adaptation to a face with a particular expression will bias participants’ judgments of subsequent faces towards the “opposite” expression: The expression with visual characteristics opposite those of the adaptor, relative to the central tendency of expressions.”

All of the unexamined theoretical assumptions implicit in the terms ‘visual characteristics,’ ‘adaptor’ ‘central tendency’ and, therefore, ‘opposite’ are embedded in the uncritically-adopted procedure of Tiddeman et al (2001). While the example the authors give – “Where fear has raised eyebrows and an open mouth, anti-fear has lowered eyebrows and a closed mouth, and so on” may seem straightforward, neither it nor the procedure is as straightforward as we might assume. The devil is in the “and so on.” First, “lowered eyebrows” is a relative term; lowered in relation to what? Different faces have different relationships between eyebrows, eyes, nose, hairline, etc. And a closed mouth is a very general state. Second, this discrete, if vague, description doesn’t directly reference the technical procedure developed by Tiddeman et al (2001). When we are told by Burton et al (2016) that “anti-expressions were created by morphing along a trajectory that ran from one of the identity-neutral faces [they look like nothing?] through the average expression and beyond it to a point that differed from the average to the same extent as the original expression,” we have absolutely no way to interpret this without examining the assumptions and mathematics utilized by Tiddeman et al (2001). On a conceptual and practical level, readers and authors are blind as to the theoretical significance of this manipulation and the description of its products in terms of “opposites.”

In addition, there is no way to distinguish the authors’ description of adaptation from any possible effect of previous experience, e.g. to distinguish it from the previously-mentioned concept of “perceptual set,” or from the fact that we perceive things in relative terms; a baby tiger cub, for example, evokes an impression of smallness while a smaller but fully-grown cat large for its size might evoke the impression of largeness. Someone used to being around short people might find average-height people tall, and vice versa. Should we lump this with color, motion aftereffects and perceptual set effects? Maybe we should, but we need to make the case, we need a rationale.

1. Implications The authors say that their results indicate that “expression aftereffects” may have a significant impact on day-to-day expression perception, but given that they needed to train their observers to deliver adequate results, and given the very particular conditions that they chose (without explanation), this is not particularly convincing. Questions about the specifics of the design are always relevant in this type of studies, where stimuli are very briefly presented. Why, for example, did Burton et al (2016) use a 150 millisecond ISI, versus the 500 millisecond ISI used by Skinner and Benton (2010). With such tight conditions, such decisions can obviously influence results, so it’s important to rationalize them in the context of theory.

2. It should be obvious already, but the following statement, taken from the General discussion, is an apt demonstration of the general intellectual vagueness of the article: “Face aftereffects are often used to examine the visual representation of faces, with the assumption that these aftereffects tap the mechanisms of visual perception in the same way as lower level visual aftereffects.”

The phrase “in the same way…” is wholly without a referent; we don’t even know what level of analysis is being referred to. In the same way as (retinally-mediated, as far as we understand) color aftereffects? In the same (physiologically not well understood) way as motion aftereffects?” In the same way as the effects of perceptual set? In the same way as seeing size, or shape, or color, etc, in relative terms? What way is “the same way”?

On 2016 Dec 08, Lydia Maniatis commented:

Part 1: There seems to be a widespread misunderstanding in the vision sciences about the role of experiment in science. The value of experiment rests solely on the clarity, coherence, internal consistency, and respect for known facts of the underlying rationale, certain of whose predictions it is designed to test. The methodological choices made in conducting the experimental test are directly linked to this rationale. (For example, if I did a study on the heritability of a single (assumed) gene underlying nose size (plus “noise”), my results would be interpretable in the terms of my assumption (especially if “size” were not clearly defined), even if my assumption were false. This is why it’s important to take care in constructing our hypotheses and tests). If the underlying concepts are vague, or if the rationale lacks internal consistency, or if it lacks coherence, then the results of the experiment cannot be interpreted (in a scientific sense).

The problems of rationale and method are overwhelmingly on display here. Below, I enumerate and expand on various issues.

1. The stimuli The theory/method behind the (implicitly) theory-laden stimuli (averaged and morphed photos of faces having various expressions ) is briefly described as having been adapted from those used by Skinner and Benton (2010). (As the corresponding author has clarified to me, the stimuli are, in fact, the same ones used in that study.) The pre-morphed versions of those stimuli came from a database of 70 individual faces exhibiting various expressions collected by Lundqvist, Flykt & Ohman (1998). This reference is not cited by Burton et al (2016), which I feel is an oversight, and neither Bonner et al (2016) nor Skinner and Benton (2010) feel the need to address the sampling criteria used by Lundqvist et al (1998). Sampling methods are a science in themselves, and are always informed by the purpose and assumptions of a study.

However, we’ll assume the investigators evaluated Lundqvist et al’s (1998) sampling technique and found it acceptable, as it’s arguably less problematic than the theoretical problems glossed over in the morphing procedure. The only (non)description of this procedure provided by either Burton et al (2016) or Skinner and Benton (2010) is a single reference, to Tiddeman, Burt, & Perrett,( 2001). A cursory examination of the work reported on by those researchers reveals it to be a wholly inadequate basis for the use Burton et al (2016) make of it.

Tiddeman et al (2001) were interested in the problem of using morphing techniques to age faces. They were trying to improve the technique in comparison to previous attempts, with regard to this specific problem. They didn’t merely assume their computational solutions achieved the aim, but evaluated the results in empirical tests with observers. (However, they, too, fail to describe the source of the images on which they are basing their procedures; the reference to 70 original faces seems the only clue that we are dealing with the sample collected by Lundqvist et al (1998).) The study is clearly a preliminary step in the development of morphing techniques for a specific purpose: “We plan to use the new prototyping and transformation methods to investigate psychological theories of facial attraction related to aging….We’ll also investigate technical extensions to the texture processing algorithms. Our results show that previous statistical models of facial images in terms of shape and color are incomplete.”

The use of the computational methods being tentatively proposed by Tiddeman et al (2001) by Skinner and Benton (2010) and Burton et al (2016) for a very different purpose has been neither analyzed, rationalized nor validated by either group. Rather, the procedure is casually and thoughtlessly adopted to produce stimuli that the authors refer to as exhibiting “anti-expressions.” What this label means or implies at a theoretical level is completely opaque. I suspect the authors may not even know what the Tiddeman et al algorithm actually does. (Earlier work by Rhodes clearly shows the pitfalls of blind application of computational procedures to stimuli and labeling them on the basis of the pre-manipulation perception. I remember seeing pictures of morphed or averaged“faces” in studies on the perception of beauty that appeared grossly deformed and non-human.)

Averaging of things in general is dicey, as the average may be a completely unrealistic or meaningless value. If we mix all the colors of the rainbow, do we get an “average” color? All the more so when it comes to complex shapes. If we combined images of a pine tree and a palm tree, would the result be the average of the two? What would this mean? Complex structures are composed of multiple internal relationships and the significance of the products of a crude averaging process is difficult to evaluate and should be used with caution. Or not.

On 2016 Dec 08, Lydia Maniatis commented:

Part 1: There seems to be a widespread misunderstanding in the vision sciences about the role of experiment in science. The value of experiment rests solely on the clarity, coherence, internal consistency, and respect for known facts of the underlying rationale, certain of whose predictions it is designed to test. The methodological choices made in conducting the experimental test are directly linked to this rationale. (For example, if I did a study on the heritability of a single (assumed) gene underlying nose size (plus “noise”), my results would be interpretable in the terms of my assumption (especially if “size” were not clearly defined), even if my assumption were false. This is why it’s important to take care in constructing our hypotheses and tests). If the underlying concepts are vague, or if the rationale lacks internal consistency, or if it lacks coherence, then the results of the experiment cannot be interpreted (in a scientific sense).

The problems of rationale and method are overwhelmingly on display here. Below, I enumerate and expand on various issues.

1. The stimuli The theory/method behind the (implicitly) theory-laden stimuli (averaged and morphed photos of faces having various expressions ) is briefly described as having been adapted from those used by Skinner and Benton (2010). (As the corresponding author has clarified to me, the stimuli are, in fact, the same ones used in that study.) The pre-morphed versions of those stimuli came from a database of 70 individual faces exhibiting various expressions collected by Lundqvist, Flykt & Ohman (1998). This reference is not cited by Burton et al (2016), which I feel is an oversight, and neither Bonner et al (2016) nor Skinner and Benton (2010) feel the need to address the sampling criteria used by Lundqvist et al (1998). Sampling methods are a science in themselves, and are always informed by the purpose and assumptions of a study.

However, we’ll assume the investigators evaluated Lundqvist et al’s (1998) sampling technique and found it acceptable, as it’s arguably less problematic than the theoretical problems glossed over in the morphing procedure. The only (non)description of this procedure provided by either Burton et al (2016) or Skinner and Benton (2010) is a single reference, to Tiddeman, Burt, & Perrett,( 2001). A cursory examination of the work reported on by those researchers reveals it to be a wholly inadequate basis for the use Burton et al (2016) make of it.

Tiddeman et al (2001) were interested in the problem of using morphing techniques to age faces. They were trying to improve the technique in comparison to previous attempts, with regard to this specific problem. They didn’t merely assume their computational solutions achieved the aim, but evaluated the results in empirical tests with observers. (However, they, too, fail to describe the source of the images on which they are basing their procedures; the reference to 70 original faces seems the only clue that we are dealing with the sample collected by Lundqvist et al (1998).) The study is clearly a preliminary step in the development of morphing techniques for a specific purpose: “We plan to use the new prototyping and transformation methods to investigate psychological theories of facial attraction related to aging….We’ll also investigate technical extensions to the texture processing algorithms. Our results show that previous statistical models of facial images in terms of shape and color are incomplete.”

The use of the computational methods being tentatively proposed by Tiddeman et al (2001) by Skinner and Benton (2010) and Burton et al (2016) for a very different purpose has been neither analyzed, rationalized nor validated by either group. Rather, the procedure is casually and thoughtlessly adopted to produce stimuli that the authors refer to as exhibiting “anti-expressions.” What this label means or implies at a theoretical level is completely opaque. I suspect the authors may not even know what the Tiddeman et al algorithm actually does. (Earlier work by Rhodes clearly shows the pitfalls of blind application of computational procedures to stimuli and labeling them on the basis of the pre-manipulation perception. I remember seeing pictures of morphed or averaged“faces” in studies on the perception of beauty that appeared grossly deformed and non-human.)

Averaging of things in general is dicey, as the average may be a completely unrealistic or meaningless value. If we mix all the colors of the rainbow, do we get an “average” color? All the more so when it comes to complex shapes. If we combined images of a pine tree and a palm tree, would the result be the average of the two? What would this mean? Complex structures are composed of multiple internal relationships and the significance of the products of a crude averaging process is difficult to evaluate and should be used with caution. Or not.

On 2016 Dec 08, Lydia Maniatis commented:

Part 2: 2. The terms “adaptation” and “aftereffects.” There is no way to discern the meaning of these terms in the context of this study. The authors seem to be using the term very loosely: “….many aspects of visual perception are adaptive, such that the appearance of a given stimulus may be affected by what has been seen before.” The example they give involves motion aftereffects, the cause of which, as far as I can discover, is still unknown. Continuing, they declare that “This perceptual bias is known as an aftereffect [last term italicized].”

This exposition conflates all possible effects of experience on subsequent perceptual impressions with the color aftereffects based on color opponency at the retinal level, and with the less well-understood motion aftereffects. In other words, we’re lumping together the phenomenon known as “perceptual set,” for example, with color aftereffects, as well as with motion aftereffects. In these latter cases, at least, it is intelligible to talk about aftereffects as being “opposite” to the original effects. In the case of motion, we are fairly straightforwardly talking about a percept of a motion in the opposite direction. In the case of color, the situation is not based on percepts having perceptually opposing characteristics; what makes green the opposite of red is more physiological then perceptual. So even with respect to what the authors refer to as “low-level” effects, ‘opposite’ means rather different things.

The vagueness of the ‘opposite’ concept as used by Burton et al (2016) is expressed in their placement of quotation marks around the term: “In the facial expression aftereffect, adaptation to a face with a particular expression will bias participants’ judgments of subsequent faces towards the “opposite” expression: The expression with visual characteristics opposite those of the adaptor, relative to the central tendency of expressions.”

All of the unexamined theoretical assumptions implicit in the terms ‘visual characteristics,’ ‘adaptor’ ‘central tendency’ and, therefore, ‘opposite’ are embedded in the uncritically-adopted procedure of Tiddeman et al (2001). While the example the authors give – “Where fear has raised eyebrows and an open mouth, anti-fear has lowered eyebrows and a closed mouth, and so on” may seem straightforward, neither it nor the procedure is as straightforward as we might assume. The devil is in the “and so on.” First, “lowered eyebrows” is a relative term; lowered in relation to what? Different faces have different relationships between eyebrows, eyes, nose, hairline, etc. And a closed mouth is a very general state. Second, this discrete, if vague, description doesn’t directly reference the technical procedure developed by Tiddeman et al (2001). When we are told by Burton et al (2016) that “anti-expressions were created by morphing along a trajectory that ran from one of the identity-neutral faces [they look like nothing?] through the average expression and beyond it to a point that differed from the average to the same extent as the original expression,” we have absolutely no way to interpret this without examining the assumptions and mathematics utilized by Tiddeman et al (2001). On a conceptual and practical level, readers and authors are blind as to the theoretical significance of this manipulation and the description of its products in terms of “opposites.”

In addition, there is no way to distinguish the authors’ description of adaptation from any possible effect of previous experience, e.g. to distinguish it from the previously-mentioned concept of “perceptual set,” or from the fact that we perceive things in relative terms; a baby tiger cub, for example, evokes an impression of smallness while a smaller but fully-grown cat large for its size might evoke the impression of largeness. Someone used to being around short people might find average-height people tall, and vice versa. Should we lump this with color, motion aftereffects and perceptual set effects? Maybe we should, but we need to make the case, we need a rationale.

1. Implications The authors say that their results indicate that “expression aftereffects” may have a significant impact on day-to-day expression perception, but given that they needed to train their observers to deliver adequate results, and given the very particular conditions that they chose (without explanation), this is not particularly convincing. Questions about the specifics of the design are always relevant in this type of studies, where stimuli are very briefly presented. Why, for example, did Burton et al (2016) use a 150 millisecond ISI, versus the 500 millisecond ISI used by Skinner and Benton (2010). With such tight conditions, such decisions can obviously influence results, so it’s important to rationalize them in the context of theory.

2. It should be obvious already, but the following statement, taken from the General discussion, is an apt demonstration of the general intellectual vagueness of the article: “Face aftereffects are often used to examine the visual representation of faces, with the assumption that these aftereffects tap the mechanisms of visual perception in the same way as lower level visual aftereffects.”

The phrase “in the same way…” is wholly without a referent; we don’t even know what level of analysis is being referred to. In the same way as (retinally-mediated, as far as we understand) color aftereffects? In the same (physiologically not well understood) way as motion aftereffects?” In the same way as the effects of perceptual set? In the same way as seeing size, or shape, or color, etc, in relative terms? What way is “the same way”?