On 2017 May 15, Lydia Maniatis commented:

In short, there are too many layers of uncertainty and conceptual vagueness here for this project to offer any points of support for any hypothesis or theory.

On 2017 May 15, Lydia Maniatis commented:

Every hypothesis or theory is a story, but in the current relaxed climate in vision science, the story doesn't need to be empirically tested and well-rationalized in order to be publishable. We just need a special section in the paper acknowledging these problems, often entitled "limitations of the study" or, as here, "qualifying remarks." Below I excerpt remarks from this and other sections of the paper (caps mine):

"A major goal of the present work was to test hypotheses related to multi-stage programming of saccades in infants. On the empirical side, we exposed 6-month-old infants to double-step trials but DID NOT SUCCEED IN COLLECTING RELIABLE DATA. In this respect, the model simulations were used to investigate an aspect of eye movement control that was not tested empirically."

"In the model simulations, certain model parameters were allowed to vary across age groups and/or viewing conditions, based on theoretical and empirical considerations. We then interpreted the constellation of best-fitting parameters." There is a great deal of flexibility in post hoc data-fitting with numerous free parameters.

"in supplementary analyses (not presented here) we used this approach to follow up on the results obtained in Simulation Study 2. To determine the individual contributions of saccade programming and saccade timing model parameters in generating the fixation duration distributions from LongD and ShortD groups during free viewing of naturalistic videos, we ran simulations in which we estimated the saccade programming parameters (mean durations of labile and non-labile stages) while keeping the saccade timing parameters fixed, and vice versa. In brief, the results confirmed that, for both ShortD and LongD groups, a particular combination of saccade-programming and saccade timing parameters was needed to achieve a good fit. HOLDING EITHER SET OF PARAMETERS FIXED DID NOT RESULT IN AN ADEQUATE FIT."

There are also a lot of researcher degrees of freedom in generating and analysing data. From the methods:

"Fixation durations (FDs). Eye-tracking data from infants may contain considerably higher levels of noise than data from more compliant participants such as adults due to various factors including their high degree of movement, lack of compliance tothe task, poor calibration and corneal reflection disturbances dueto the underdeveloped cornea and iris (Hessels, Andersson,Hooge, Nystr歬 & Kemner, 2015; Saez de Urabain, Johnson, &Smith, 2015; Wass, Smith, & Johnson, 2013). To account for this potential quality/age confound, dedicated in-house software for parsing and cleaning eye tracking data has been developed (GraFix, Saez de Urabain et al., 2015). This software allows valid fixations to be salvaged from low-quality datasets whilst also removing spurious invalid fixations. In the present study, both adult and infant datasets were parsed using GraFix鳠two-stage semi-automated process (see Appendix A for details). The second stage of GraFix involves manual checking of the fixations detected automatically during the first stage. This manual coding stage was validated by assessing the degree of agreement between two different raters. ONE RATER WAS ONE OF THE AUTHORS (IRSdU)."

The "in-house software" changes the data, a d therefore the assumptions implicit in its computations should be made explicit. The p-value used to assess rater agreement was p<05, which nowadays is considered rather low. We need more info on the "valid/invalid" distinction as well as on how the software is supposed to make this distinction.

From the "simulation studies": "The parameter for the standard deviation of the gamma distributions (rc) is a fixed parameter. To accommodate the higher variability generally observed in infant data compared to adult data, it was set to 0.33 for the infant data and 0.25 for the adult data. These values were adopted from previous model simulations (Engbert et al., 2005; Nuthmann et al., 2010; Reichle et al., 1998, 2003)."

Using second-hand values adopted by other researchers in the past doesn't absolve the current ones from explaining the rationale behind these choices (assuming there is one.) "

On 2017 May 15, Lydia Maniatis commented:

Every hypothesis or theory is a story, but in the current relaxed climate in vision science, the story doesn't need to be empirically tested and well-rationalized in order to be publishable. We just need a special section in the paper acknowledging these problems, often entitled "limitations of the study" or, as here, "qualifying remarks." Below I excerpt remarks from this and other sections of the paper (caps mine):

"A major goal of the present work was to test hypotheses related to multi-stage programming of saccades in infants. On the empirical side, we exposed 6-month-old infants to double-step trials but DID NOT SUCCEED IN COLLECTING RELIABLE DATA. In this respect, the model simulations were used to investigate an aspect of eye movement control that was not tested empirically."

"In the model simulations, certain model parameters were allowed to vary across age groups and/or viewing conditions, based on theoretical and empirical considerations. We then interpreted the constellation of best-fitting parameters." There is a great deal of flexibility in post hoc data-fitting with numerous free parameters.

"in supplementary analyses (not presented here) we used this approach to follow up on the results obtained in Simulation Study 2. To determine the individual contributions of saccade programming and saccade timing model parameters in generating the fixation duration distributions from LongD and ShortD groups during free viewing of naturalistic videos, we ran simulations in which we estimated the saccade programming parameters (mean durations of labile and non-labile stages) while keeping the saccade timing parameters fixed, and vice versa. In brief, the results confirmed that, for both ShortD and LongD groups, a particular combination of saccade-programming and saccade timing parameters was needed to achieve a good fit. HOLDING EITHER SET OF PARAMETERS FIXED DID NOT RESULT IN AN ADEQUATE FIT."

There are also a lot of researcher degrees of freedom in generating and analysing data. From the methods:

"Fixation durations (FDs). Eye-tracking data from infants may contain considerably higher levels of noise than data from more compliant participants such as adults due to various factors including their high degree of movement, lack of compliance tothe task, poor calibration and corneal reflection disturbances dueto the underdeveloped cornea and iris (Hessels, Andersson,Hooge, Nystr歬 & Kemner, 2015; Saez de Urabain, Johnson, &Smith, 2015; Wass, Smith, & Johnson, 2013). To account for this potential quality/age confound, dedicated in-house software for parsing and cleaning eye tracking data has been developed (GraFix, Saez de Urabain et al., 2015). This software allows valid fixations to be salvaged from low-quality datasets whilst also removing spurious invalid fixations. In the present study, both adult and infant datasets were parsed using GraFix鳠two-stage semi-automated process (see Appendix A for details). The second stage of GraFix involves manual checking of the fixations detected automatically during the first stage. This manual coding stage was validated by assessing the degree of agreement between two different raters. ONE RATER WAS ONE OF THE AUTHORS (IRSdU)."

The "in-house software" changes the data, a d therefore the assumptions implicit in its computations should be made explicit. The p-value used to assess rater agreement was p<05, which nowadays is considered rather low. We need more info on the "valid/invalid" distinction as well as on how the software is supposed to make this distinction.

From the "simulation studies": "The parameter for the standard deviation of the gamma distributions (rc) is a fixed parameter. To accommodate the higher variability generally observed in infant data compared to adult data, it was set to 0.33 for the infant data and 0.25 for the adult data. These values were adopted from previous model simulations (Engbert et al., 2005; Nuthmann et al., 2010; Reichle et al., 1998, 2003)."

Using second-hand values adopted by other researchers in the past doesn't absolve the current ones from explaining the rationale behind these choices (assuming there is one.) "

On 2017 May 15, Lydia Maniatis commented:

In short, there are too many layers of uncertainty and conceptual vagueness here for this project to offer any points of support for any hypothesis or theory.