On 2017 May 12, Lydia Maniatis commented:

"Although the results from both our experiments appear to be consistent with previous research (Atsma et al., 2012; Fencsik et al., 2007; Franconeri et al., 2012; Howe & Holcombe, 2012; Iordanescu et al., 2009; Keane & Pylyshyn, 2006; Khan et al., 2010; Lovejoy et al., 2009; Luu & Howe, 2015; Szinte et al., 2015; Watamaniuk & Heinen, 2015), they do not seem to be consistent with each other. Obviously, the two experiments we have described here were not exactly the same. We will discuss some of the differences that might explain the seemingly conflicting results."

The conflict between the results has to also be a conflict between some of the results and the hypothesis being tested. The broad speculation as to which of the many confounds may be responsible just shows that there were too many confounds. Such as:

"the amount of attentional resources dedicated to the task might have been different between the two experiments. For both overtly tracked and covertly tracked targets, we see that the overall probe detection rate was higher in the second experiment compared to the first. Moreover, the feedback we received from several participants in both experiments suggests that tracking the objects in Experiment 1 was so easy that participants were very easily distracted by their thoughts, and that Experiment 2 was more challenging and engaging. We therefore speculate that participants focused their attention more strongly (i.e., dedicated more attentional resources) toward tracking each target during Experiment 2 than during Experiment 1."

The only way to test that speculation is to do another experiment, hopefully one less confounded. Otherwise - if speculation by itself can resolve serious confounds in an otherwise inconclusive experiment - why do any experiments at all? Just assume that any differences between future results and prediction will be due to confounds.

On 2017 May 12, Lydia Maniatis commented:

"Although the results from both our experiments appear to be consistent with previous research (Atsma et al., 2012; Fencsik et al., 2007; Franconeri et al., 2012; Howe & Holcombe, 2012; Iordanescu et al., 2009; Keane & Pylyshyn, 2006; Khan et al., 2010; Lovejoy et al., 2009; Luu & Howe, 2015; Szinte et al., 2015; Watamaniuk & Heinen, 2015), they do not seem to be consistent with each other. Obviously, the two experiments we have described here were not exactly the same. We will discuss some of the differences that might explain the seemingly conflicting results."

The conflict between the results has to also be a conflict between some of the results and the hypothesis being tested. The broad speculation as to which of the many confounds may be responsible just shows that there were too many confounds. Such as:

"the amount of attentional resources dedicated to the task might have been different between the two experiments. For both overtly tracked and covertly tracked targets, we see that the overall probe detection rate was higher in the second experiment compared to the first. Moreover, the feedback we received from several participants in both experiments suggests that tracking the objects in Experiment 1 was so easy that participants were very easily distracted by their thoughts, and that Experiment 2 was more challenging and engaging. We therefore speculate that participants focused their attention more strongly (i.e., dedicated more attentional resources) toward tracking each target during Experiment 2 than during Experiment 1."

The only way to test that speculation is to do another experiment, hopefully one less confounded. Otherwise - if speculation by itself can resolve serious confounds in an otherwise inconclusive experiment - why do any experiments at all? Just assume that any differences between future results and prediction will be due to confounds.