Il aurait été bon de faire la même précision qu'au chapitre précédent concernant l'installation de la bibliothèque pytest-mock.

• Introduction and Context:

  • The talk focuses on "rethinking identity with Clojure" by a senior software engineer at Vouch.
  • Vouch is developing a platform to address identity and trust issues among people, machines, and processes.
  • The speaker emphasizes leveraging emerging technologies for significant improvements in current best practices.
  • "At Vouch, we love Clojure and ClojureScript" for tackling ambitious problems with a small team.

• Clojure Philosophy:

  • The Clojure philosophy, which emphasizes separating concerns and simplifying complexity, guides Vouch's approach to problem-solving.
  • The speaker references Rich Hickey's talk "Simple Made Easy," which highlights the importance of decoupling elements to reduce complexity.
  • "As software developers, we often do this thing called complecting," where unrelated aspects are combined, creating complications.

• Security and User Experience:

  • A fundamental flaw in security systems is poor user experience, particularly in how users secure their accounts.
  • "If the user experience is wrong, there's nothing you can do because in any system, people are often the weakest link."
  • The speaker argues that complexity in security measures, like passwords, leads to vulnerabilities due to user error.

• Clojure's Approach to Simplifying Complexity:

  • Clojure decouples programs from state, a core principle borrowed from functional programming.
  • This separation simplifies development and reduces bugs, particularly in concurrent programming.
  • "Clojure adopted the idea from Ruby, Python, and JavaScript that the common pattern is data literals."

• Innovations in Clojure and Its Impact:

  • Rich Hickey innovated by proving that persistent data structures can perform well on modern hardware.
  • "Persistent data structures...give me these certain properties in a concurrent scenario but...in a very easy to use ergonomic way."

• Applying Clojure Principles to Identity and Security:

  • Vouch uses secure enclaves in devices like iPhones and Google's Titan M to manage cryptographic keys securely.
  • "What just happened there are no secrets. The secret isn't on our server. We don't know anything about secrets."
  • This method ensures that even if a device is lost, unauthorized access is prevented due to the secure enclave.

• Blockchain and Distributed Systems:

  • Vouch leverages blockchain (Hyperledger Fabric) to ensure tamper-evidence and secure user actions.
  • "All interactions with our system write to the blockchain...prevent tampering with the actions that users have taken."

• Developer Experience with Clojure:

  • Clojure allows Vouch's small team to be highly productive, with one language for all system parts.
  • "We actually get a feedback loop into every piece of our architecture...we have a shell into it and can interact with it using our language."
  • ClojureScript enables writing code once and deploying it across multiple platforms, saving development time.

• Practical Demonstration:

  • The speaker showcases an app that uses biometrics for logging into services, demonstrating a superior user experience compared to traditional passwords.
  • "The user experience needs to be so good that this feels better than the password approach."

• Closing Thoughts:

  • The speaker encourages finding and addressing sources of complexity to improve user and developer experience.
  • "Complexity is not just the software; it will eventually burden your users."
  • Revisiting Clojure can benefit teams building ambitious full-stack solutions with many moving parts.

• Q&A Highlights:

  • Addressing concerns about biometrics and fallback mechanisms, such as using blockchain for account recovery.
  • Clarifying the differences between Clojure spec and other type systems, emphasizing runtime-oriented specifications.
  • Discussing the soundness and limitations of using Clojure in security-focused projects.

Même en utilisant le code donné par la correction, je n'arrive pas à faire fonctionner le programme de tests. Pourriez-vous vérifier le bon fonctionnement ? Par ailleurs, la commande pytest tests.py ne fonctionne sur aucun des programmes de tests. J'utilise la commande python -m pytest tests.py et cela fonctionne sur tous les programmes (fonctionnels...).

• The speaker, an experienced software developer with expertise in C++, Java, and Clojure, introduces key concepts in Clojure programming, emphasizing the shift from Java to Clojure for enhanced code efficiency and readability.

  • "I've been doing software development since I was kid and professionally for you know 1517 something like that years now I did a little bit of C++ work and then spent a long time in Java."

• The talk focuses on the differences between values and objects, highlighting the immutability of values in Clojure and contrasting it with the mutability of objects in Java.

  • "I'm going to start by talking about values and what is the value... all of these things are immutable."

• Values in Clojure have intrinsic, semantically evident meanings, allowing for precise and undisputed representation and equality comparison.

  • "It has a precise meaning and it's semantically evident what we mean by that... all of these things are immutable."

• Composite values in Clojure allow for creating larger data structures from primitives while retaining immutability, ensuring consistent behavior and avoiding common issues with mutable objects.

  • "With a composite value, you should not have to write an equals method; it should be semantically evident whether two things are equal or not."

• Clojure collections (vectors, lists, maps, sets) are introduced as immutable composite values that offer extensive functionality and efficiency despite initial concerns about performance.

  • "Vectors are what you use constantly in Clojure... they are expandable indexed collections."

• Sequences in Clojure are immutable views into collections, providing a powerful abstraction for functional programming and enabling efficient handling of large or infinite data sets.

  • "Sequences are really immutable views into a collection... logically a list and you can get back from it the first."

• Generic data interfaces in Clojure facilitate the handling of diverse data types through consistent, reusable functions, enhancing code reusability and reducing redundancy.

  • "Closure does not do that and that is one of the things I really like about it."

• Identity and state are treated distinctly in Clojure, with immutable values representing states at particular points in time, offering a clearer and more predictable model for concurrent programming.

  • "Identity is an abstraction that represents an entity whose state changes over time, while state is the value of an identity at a point in time."

• Clojure's software transactional memory (STM) model provides robust mechanisms for managing state changes in concurrent environments, ensuring consistency and reliability.

  • "STM protects you from inconsistent views and ensures that the states you perceive are coherent and correct."

• The speaker emphasizes that transitioning to programming with immutable composite values significantly changes one's approach to software development, leading to more maintainable and less error-prone code.

  • "I say this night seriously it will change the way you think about programming to start from immutability like this you'll design and write code differently."

ここからの4行の訳の代案を考えてみました。

原文 Python doesn’t include pytest by default, so you’ll learn to install external libraries. Knowing how to install external libraries will make a wide variety of well-designed code available to you. These libraries will expand the kinds of projects you can work on immensely.

代案

pytestはPythonにはデフォルトでは含まれていないので、外部ライブラリのインストール方法についても学びます。外部ライブラリをインストールするということは、さまざまな洗練されたコードが手に入るということです。これらのライブラリによって取り組めるプロジェクトの種類が格段に増えます。

代案の補足

Knowing how to～を「～方法を知ることで」と書くと翻訳調になるので、方法を知る＝できると意訳しました。 また、取り組めるthe kinds of projectがすごく拡張する、ということで、「できることを大きく拡張」のところを「種類が格段に増える」に変えました。

远离Bug,可读性高，易扩展，易维护。

Author response:

Reviewer #1 (Public Review):

How does the brain respond to the input of different complexity, and does this ability to respond change with age?

The study by Lalwani et al. tried to address this question by pulling together a number of neuroscientific methodologies (fMRI, MRS, drug challenge, perceptual psychophysics). A major strength of the paper is that it is backed up by robust sample sizes and careful choices in data analysis, translating into a more rigorous understanding of the sensory input as well as the neural metric. The authors apply a novel analysis method developed in human resting-state MRI data on task-based data in the visual cortex, specifically investigating the variability of neural response to stimuli of different levels of visual complexity. A subset of participants took part in a placebo-controlled drug challenge and functional neuroimaging. This experiment showed that increases in GABA have differential effects on participants with different baseline levels of GABA in the visual cortex, possibly modulating the perceptual performance in those with lower baseline GABA. A caveat is that no single cohort has taken part in all study elements, ie visual discrimination with drug challenge and neuroimaging. Hence the causal relationship is limited to the neural variability measure and does not extend to visual performance. Nevertheless, the consistent use of visual stimuli across approaches permits an exceptionally high level of comparability across (computational, behavioural, and fMRI are drawing from the same set of images) modalities. The conclusions that can be made on such a coherent data set are strong.

The community will benefit from the technical advances, esp. the calculation of BOLD variability, in the study when described appropriately, encouraging further linkage between complementary measures of brain activity, neurochemistry, and signal processing.

Thank you for your review. We agree that a future study with a single cohort would be an excellent follow-up.

Reviewer #2 (Public Review):

Lalwani et al. measured BOLD variability during the viewing of houses and faces in groups of young and old healthy adults and measured ventrovisual cortex GABA+ at rest using MR spectroscopy. The influence of the GABA-A agonist lorazepam on BOLD variability during task performance was also assessed, and baseline GABA+ levels were considered as a mediating variable. The relationship of local GABA to changes in variability in BOLD signal, and how both properties change with age, are important and interesting questions. The authors feature the following results: 1) younger adults exhibit greater task-dependent changes in BOLD variability and higher resting visual cortical GABA+ content than older adults, 2) greater BOLD variability scales with GABA+ levels across the combined age groups, 3) administration of a GABA-A agonist increased condition differences in BOLD variability in individuals with lower baseline GABA+ levels but decreased condition differences in BOLD variability in individuals with higher baseline GABA+ levels, and 4) resting GABA+ levels correlated with a measure of visual sensory ability derived from a set of discrimination tasks that incorporated a variety of stimulus categories.

Strengths of the study design include the pharmacological manipulation for gauging a possible causal relationship between GABA activity and task-related adjustments in BOLD variability. The consideration of baseline GABA+ levels for interpreting this relationship is particularly valuable. The assessment of feature-richness across multiple visual stimulus categories provided support for the use of a single visual sensory factor score to examine individual differences in behavioral performance relative to age, GABA, and BOLD measurements.

Weaknesses of the study include the absence of an interpretation of the physiological mechanisms that contribute to variability in BOLD signal, particularly for the chosen contrast that compared viewing houses with viewing faces.

Whether any of the observed effects can be explained by patterns in mean BOLD signal, independent of variability would be useful to know.

One of the first pre-processing steps of computing SDBOLD involves subtracting the block-mean from the fMRI signal for each task-condition. Therefore, patterns observed in BOLD signal variability are not driven by the mean-BOLD differences. Moreover, as noted above, to further confirm this, we performed additional mean-BOLD based analysis (See Supplementary Materials Pg 3). Results suggest that ∆⃗ MEANBOLD is actually larger in older adults vs. younger adults (∆⃗ SDBOLD exhibited the opposite pattern), but more importantly ∆⃗ MEANBOLD is not correlated with GABA or with visual performance. This is also consistent with prior research (Garrett et.al. 2011, 2013, 2015, 2020) that found MEANBOLD to be relatively insensitive to behavioral performance.

The positive correlation between resting GABA+ levels and the task-condition effect on BOLD variability reaches significance at the total group level, when the young and old groups are combined, but not separately within each group. This correlation may be explained by age-related differences since younger adults had higher values than older adults for both types of measurements. This is not to suggest that the relationship is not meaningful or interesting, but that it may be conceptualized differently than presented.

Thank you for this important point. The relationship between GABA and ∆⃗ SDBOLD shown in Figure 3 is also significant within each age-group separately (Line 386-388). The model used both age-group and GABA as predictors of ∆⃗ SDBOLD and found that both had a significant effect, while the Age-group x GABA interaction was not significant. The effect of age on ∆⃗ SDBOLD therefore does not completely explain the observed relationship between GABA and ∆⃗ SDBOLD because this latter effect is significant in both age-groups individually and in the whole sample even when variance explained by age is accounted for. The revision clarifies this important point (Ln 488-492). Thanks for raising it.

Two separate dosages of lorazepam were used across individuals, but the details of why and how this was done are not provided, and the possible effects of the dose are not considered.

Good point. We utilized two dosages to maximize our chances of finding a dosage that had a robust effect. The specific dosage was randomly assigned across participants and the dosage did not differ across age-groups or baseline GABA levels. We also controlled for the drug-dosage when examining the role of drug-related shift in ∆⃗ SDBOLD. We have clarified these points in the revision and highlighted the analysis that found no effect of dosage on drug-related shift in ∆⃗ SDBOLD (Line 407-418).

The observation of greater BOLD variability during the viewing of houses than faces may be specific to these two behavioral conditions, and lingering questions about whether these effects generalize to other types of visual stimuli, or other non-visual behaviors, in old and young adults, limit the generalizability of the immediate findings.

We agree that examining the factors that influence BOLD variability is an important topic for future research. In particular, although it is increasingly well known that variability modulation itself can occur in a host of different tasks and research contexts across the lifespan (see Garrett et al., 2013 Waschke et al., 2021), to address the question of whether variability modulation occurs directly in response to stimulus complexity in general, it will be important for future work to examine a range of stimulus categories beyond faces and houses. Doing so is indeed an active area of research in Dr. Garrett’s group, where visual stimuli from many different categories are examined (e.g., for a recent approach, see Waschke et.al.,2023 (biorxiv)). Regardless, only face and house stimuli were available in the current dataset. We therefore exploited the finding that BOLD variability tends to be larger for house stimuli than for face stimuli (in line with the HMAX model output) to demonstrate that the degree to which a given individual modulates BOLD variability in response to stimulus category is related to their age, to GABA levels, and to behavioral performance.

The observed age-related differences in patterns of BOLD activity and ventrovisual cortex GABA+ levels along with the investigation of GABA-agonist effects in the context of baseline GABA+ levels are particularly valuable to the field, and merit follow-up. Assessing background neurochemical levels is generally important for understanding individualized drug effects. Therefore, the data are particularly useful in the fields of aging, neuroimaging, and vision research.

Thank you, we agree!

Reviewer #3 (Public Review):

The role of neural variability in various cognitive functions is one of the focal contentions in systems and computational neuroscience. In this study, the authors used a largescale cohort dataset to investigate the relationship between neural variability measured by fMRI and several factors, including stimulus complexity, GABA levels, aging, and visual performance. Such investigations are valuable because neural variability, as an important topic, is by far mostly studied within animal neurophysiology. There is little evidence in humans. Also, the conclusions are built on a large-scale cohort dataset that includes multi-model data. Such a dataset per se is a big advantage. Pharmacological manipulations and MRS acquisitions are rare in this line of research. Overall, I think this study is well-designed, and the manuscript reads well. I listed my comments below and hope my suggestions can further improve the paper.

Strength:

1). The study design is astonishingly rich. The authors used task-based fMRI, MRS technique, population contrast (aging vs. control), and psychophysical testing. I appreciate the motivation and efforts for collecting such a rich dataset.

2) The MRS part is good. I am not an expert in MRS so cannot comment on MRS data acquisition and analyses. But I think linking neural variability to GABA in humans is in general a good idea. There has been a long interest in the cause of neural variability, and inhibition of local neural circuits has been hypothesized as one of the key factors. 3. The pharmacological manipulation is particularly interesting as it provides at least evidence for the causal effects of GABA and deltaSDBOLD. I think this is quite novel.

Weakness:

1) I am concerned about the definition of neural variability. In electrophysiological studies, neural variability can be defined as Poisson-like spike count variability. In the fMRI world, however, there is no consensus on what neural variability is. There are at least three definitions. One is the variability (e.g., std) of the voxel response time series as used here and in the resting fMRI world. The second is to regress out the stimulusevoked activation and only calculate the std of residuals (e.g., background variability). The third is to calculate variability of trial-by-trial variability of beta estimates of general linear modeling. It currently remains unclear the relations between these three types of variability with other factors. It also remains unclear the links between neuronal variability and voxel variability. I don't think the computational principles discovered in neuronal variability also apply to voxel responses. I hope the authors can acknowledge their differences and discuss their differences.

These are very important points, thank you for raising them. Although we agree that the majority of the single cell electrophysiology world indeed seems to prefer Poisson-like spiking variability as an easy and tractable estimate, it is certainly not the only variability approach in that field (e.g., entropy; see our most recent work in humans where spiking entropy outperforms simple spike counts to predict memory performance; Waschke et al., 2023, bioRxiv). In LFP, EEG/MEG and fMRI, there is indeed no singular consensus on what variability “is”, and in our opinion, that is a good thing. We have reported at length in past work about entire families of measures of signal variability, from simple variance, to power, to entropy, and beyond (see Table 1 in Waschke et al, 2021, Neuron). In principle, these measures are quite complementary, obviating the need to establish any single-measure consensus per se. Rather than viewing the three measures of neural variability that the reviewer mentioned as competing definitions, we prefer to view them as different sources of variance. For example, from each of the three sources of variance the reviewer suggests, any number of variability measures could be computed.

The current study focuses on using the standard deviation of concatenated blocked time series separately for face and house viewing conditions (this is the same estimation approach used in our very earliest studies on signal variability; Garrett et al., 2010, JNeurosci). In those early studies, and nearly every one thereafter (see Waschke et al., 2021, Neuron), there is no ostensible link between SDBOLD (as we normaly compute it) and average BOLD from either multivariate or GLM models; as such, we do not find any clear difference in SDBOLD results whether or not average “evoked” responses are removed or not in past work. This is perhaps also why removing ERPs from EEG time series rarely influences estimates of variability in our work (e.g., Kloosterman et al., 2020, eLife).

The third definition the reviewer notes refers to variability of beta estimates over trials. Our most recent work has done exactly this (e.g., Skowron et al., 2023, bioRxiv), calculating the SD even over single time point-wise beta estimates so that we may better control the extraction of time points prior to variability estimation. Although direct comparisons have not yet been published by us, variability over single TR beta estimates and variability over the time series without beta estimation are very highly correlated in our work (in the .80 range; e.g., Kloosterman et al., in prep).

Re: the reviewer’s point that “It also remains unclear the links between neuronal variability and voxel variability. I don’t think the computational principles discovered in neuronal variability also apply to voxel responses. I hope the authors can acknowledge their differences and discuss their differences.” If we understand correctly, the reviewer maybe asking about within-person links between single-cell neuronal variability (to allow Poisson-like spiking variability) and voxel variability in fMRI? No such study has been conducted to date to our knowledge (such data almost don’t exist). Or rather, perhaps the reviewer is noting a more general point regarding the “computational principles” of variability in these different domains? If that is true, then a few points are worth noting. First, there is absolutely no expectation of Poisson distributions in continuous brain imaging-based time series (LFP, E/MEG, fMRI). To our knowledge, such distributions (which have equivalent means and variances, allowing e.g., Fano factors to be estimated) are mathematically possible in spiking because of the binary nature of spikes; when mean rates rise, so too do variances given that activity pushes away from the floor (of no activity). In continuous time signals, there is no effective “zero”, so a mathematical floor does not exist outright. This is likely why means and variances are not well coupled in continuous time signals (see Garrett et al., 2013, NBR; Waschke et al., 2021, Neuron); anything can happen. Regardless, convergence is beginning to be revealed between the effects noted from spiking and continuous time estimates of variability. For example, we show that spiking variability can show a similar, behaviourally relevant coupling to the complexity of visual input (Waschke et al., 2023, bioRxiv) as seen in the current study and in past work (e.g., Garrett et al., 2020, NeuroImage). Whether such convergence reflects common computational principles of variability remains to be seen in future work, despite known associations between single cell recordings and BOLD overall (e.g., Logothetis and colleagues, 2001, 2002, 2004, 2008).

Given the intricacies of these arguments, we don’t currently include this discussion in the revised text. However, we would be happy to include aspects of this content in the main paper if the reviewer sees fit.

2) If I understand it correctly, the positive relationship between stimulus complexity and voxel variability has been found in the author's previous work. Thus, the claims in the abstract in lines 14-15, and section 1 in results are exaggerated. The results simply replicate the findings in the previous work. This should be clearly stated.

Good point. Since this finding was a replication and an extension, we reported these results mostly in the supplementary materials. The stimulus set used for the current study is different than Garrett et.al. 2020 and therefore a replication is important. Moreover, we have extended these findings across young and older adults (previous work was based on older adults alone). We have modified the text to clarify what is a replication and what part are extension/novel about the current study now (Line 14, 345 and 467). Thanks for the suggestion.

3) It is difficult for me to comprehend the U-shaped account of baseline GABA and shift in deltaSDBOLD. If deltaSDBOLD per se is good, as evidenced by the positive relationship between brainscore and visual sensitivity as shown in Fig. 5b and the discussion in lines 432-440, why the brain should decrease deltaSDBOLD ?? or did I miss something? I understand that "average is good, outliers are bad". But a more detailed theory is needed to account for such effects.

When GABA levels are increased beyond optimal levels, neuronal firing rates are reduced, effectively dampening neural activity and limiting dynamic range; in the present study, this resulted in reduced ∆⃗ SDBOLD. Thus, the observed drug-related decrease in ∆⃗ SDBOLD was most present in participants with already high levels of GABA. We have now added an explanation for the expected inverted-U (Line 523-546). The following figure tries to explain this with a hypothetical curve diagram and how different parts of Fig 4 might be linked to different points in such a curve.

Author response image 1.

Line 523-546 – “We found in humans that the drug-related shift in ∆⃗ SDBOLD could be either positive or negative, while being negatively related to baseline GABA. Thus, boosting GABA activity with drug during visual processing in participants with lower baseline GABA levels and low levels of ∆⃗ SDBOLD resulted in an increase in ∆⃗ SDBOLD (i.e., a positive change in ∆⃗ SDBOLD on drug compared to off drug). However, in participants with higher baseline GABA levels and higher ∆⃗ SDBOLD, when GABA was increased presumably beyond optimal levels, participants experienced no-change or even a decrease in∆⃗ SDBOLD on drug. These findings thus provide the first evidence in humans for an inverted-U account of how GABA may link to variability modulation.

Boosting low GABA levels in older adults helps increase ∆⃗ SDBOLD, but why does increasing GABA levels lead to reduced ∆⃗ SDBOLD in others? One explanation is that higher than optimal levels of inhibition in a neuronal system can lead to dampening of the entire network. The reduced neuronal firing decreases the number of states the network can visit and decreases the dynamic range of the network. Indeed, some anesthetics work by increasing GABA activity (for example propofol a general anesthetic modulates activity at GABAA receptors) and GABA is known for its sedative properties. Previous research showed that propofol leads to a steeper power spectral slope (a measure of the “construction” of signal variance) in monkey ECoG recordings (Gao et al., 2017). Networks function optimally only when dynamics are stabilized by sufficient inhibition. Thus, there is an inverted-U relationship between ∆⃗ SDBOLD and GABA that is similar to that observed with other neurotransmitters.”

4) Related to the 3rd question, can you show the relationship between the shift of deltaSDBOLD (i.e., the delta of deltaSDBOLD) and visual performance?

We did not have data on visual performance from the same participants that completed the drug-based part of the study (Subset1 vs 3; see Figure 1); therefore, we unfortunately cannot directly investigate the relationship between the drug-related shift of ∆⃗ SDBOLD and visual performance. We have now highlighted that this as a limitation of the current study (Line 589-592), where we state: One limitation of the current study is that participants who received the drug-manipulation did not complete the visual discrimination task, thus we could not directly assess how the drug-related change in ∆⃗ SDBOLD impacted visual performance.

5) Are the dataset openly available?? I didn't find the data availability statement.

An excel-sheet with all the processed data to reproduce figures and results has been included in source data submitted along with the manuscript along with a data dictionary key for various columns. The raw MRI, MRS and fMRI data used in the current manuscript was collected as a part of a larger (MIND) study and will eventually be made publicly available on completion of the study (around 2027). Before that time, the raw data can be obtained for research purposes upon reasonable request. Processing code will be made available on GitHub.

"A natural initial question is what is open source software? Roughly, being open source requires that the source code, and not only the object code (the sequence of 1's and 0's that computers actually use), be made available to everyone, and that the modifications made by its users also be turned back to the community."(Lerner & Tirole, 2001).

Lerner, J., & Tirole, J. (2001). The open source movement: Key research questions. European economic review, 45(4-6), 819-826.

https://hypothes.is/groups/x4RQA5XX/edci-338-a01-summer-2024

Some more of my recent learning with devcontainer.json (its Dev Container metadata):

• Interactive commands (those waiting for user input like read) do not display the input request in (at least onCreateCommand and postCreateCommand sections), so it is better to keep them in updateContentCommand or postAttachCommand.
• If there are 2 read commands in a single section, like updateContentCommand, only the 1st one is displayed to the user, and the 2nd one is ignored.
• When I put a read command within a dictionary (with at lest 2 key/values) of postAttachCommand, the interactive command wasn't being displayed.
• We need to use /bin/bash -c to be able to use read -s (the -s flag) which allows for securely passing the password so that it does not stay in the VS Code console. Also, I had trouble with interactive commands and if statements without it.
• Using "GITLAB_TOKEN": "${localEnv:GITLAB_TOKEN}" does not easily work as it is looking for GITLAB_TOKEN env variable set locally on our host computers, and I believe no one does it.
• The dictionary seems to be executing its scripts in parallel; therefore, it is not easily possible to break down long lines which have to execute in a chronological sequence.
• JSON does not allow for human-readable line breaks; therefore, indeed, it seems impossible to improve the long one-liners.
• The files/folders mentioned within mounts need to exist locally (otherwise, Docker container build fails). They are mounted before any other section. Technically, we can protect ourselves with the following command to find an extra message in VS Code container logs:

json "initializeCommand": "/bin/bash -c '[[ -d ${HOME}/.aws ]] || { echo \"Error: ${HOME}/.aws directory not found.\"; exit 1; }; [[ -f ${HOME}/.netrc ]] || { echo \"Error: ${HOME}/.netrc file not found.\"; exit 1; }; [[ -d ${HOME}/.ssh ]] || { echo \"Error: ${HOME}/.ssh directory not found.\"; exit 1; }'",

Other option is to get rid of the error completely, but this creates files on the host machine; therefore, it is not an ideal solution:

json "initializeCommand": "mkdir -p ~/.ssh ~/.aws && touch ~/.netrc",

I also found that updateContentCommand allows for the user interaction (it displays interactive command in the VS Code console).

Explanation of 6 available devcontainer.json (Dev Container in VS Code) hooks.

for - quote - chain reaction - why good projects fail - (see below)

• Although many projects and ideas share Elinor Ostrom's personal, cooperative and Earth-helping significance,
  • they lack the chain reaction that keeps them going.
• On the contrary, this flame is extinguished by
  • the direct action (fakes), or
  • indirect action (ignoring or taking our attention elsewhere)
• of the mainstream media that in a certain sense has lost that "code of ethics" of journalism that upheld values; such as
  • truthfulness,
  • independence,
  • objectivity,
  • fairness,
  • accuracy,
  • respect for others,
  • public accountability...

I find this interesting that they can't even tell friends about what their job includes. I guess if you are involved in billon dollar companies you can't give any information that could be told to a competitor.

The code gets cut out by freedium, so here is all the code after doing a ctrl-a then ctrl-c:

``` import java.util.Random;

// Main class public class SimpleDemoGA {

Population population = new Population();
Individual fittest;
Individual secondFittest;
int generationCount = 0;

public static void main(String[] args) {

    Random rn = new Random();

    SimpleDemoGA demo = new SimpleDemoGA();

    //Initialize population
    demo.population.initializePopulation(10);

    //Calculate fitness of each individual
    demo.population.calculateFitness();

    System.out.println("Generation: " + demo.generationCount + " Fittest: " + demo.population.fittest);

    //While population gets an individual with maximum fitness
    while (demo.population.fittest < 5) {
        ++demo.generationCount;

        //Do selection
        demo.selection();

        //Do crossover
        demo.crossover();

        //Do mutation under a random probability
        if (rn.nextInt()%7 < 5) {
            demo.mutation();
        }

        //Add fittest offspring to population
        demo.addFittestOffspring();

        //Calculate new fitness value
        demo.population.calculateFitness();

        System.out.println("Generation: " + demo.generationCount + " Fittest: " + demo.population.fittest);
    }

    System.out.println("\nSolution found in generation " + demo.generationCount);
    System.out.println("Fitness: "+demo.population.getFittest().fitness);
    System.out.print("Genes: ");
    for (int i = 0; i < 5; i++) {
        System.out.print(demo.population.getFittest().genes[i]);
    }

    System.out.println("");

}

//Selection
void selection() {

    //Select the most fittest individual
    fittest = population.getFittest();

    //Select the second most fittest individual
    secondFittest = population.getSecondFittest();
}

//Crossover
void crossover() {
    Random rn = new Random();

    //Select a random crossover point
    int crossOverPoint = rn.nextInt(population.individuals[0].geneLength);

    //Swap values among parents
    for (int i = 0; i < crossOverPoint; i++) {
        int temp = fittest.genes[i];
        fittest.genes[i] = secondFittest.genes[i];
        secondFittest.genes[i] = temp;

    }

}

//Mutation
void mutation() {
    Random rn = new Random();

    //Select a random mutation point
    int mutationPoint = rn.nextInt(population.individuals[0].geneLength);

    //Flip values at the mutation point
    if (fittest.genes[mutationPoint] == 0) {
        fittest.genes[mutationPoint] = 1;
    } else {
        fittest.genes[mutationPoint] = 0;
    }

    mutationPoint = rn.nextInt(population.individuals[0].geneLength);

    if (secondFittest.genes[mutationPoint] == 0) {
        secondFittest.genes[mutationPoint] = 1;
    } else {
        secondFittest.genes[mutationPoint] = 0;
    }
}

//Get fittest offspring
Individual getFittestOffspring() {
    if (fittest.fitness > secondFittest.fitness) {
        return fittest;
    }
    return secondFittest;
}


//Replace least fittest individual from most fittest offspring
void addFittestOffspring() {

    //Update fitness values of offspring
    fittest.calcFitness();
    secondFittest.calcFitness();

    //Get index of least fit individual
    int leastFittestIndex = population.getLeastFittestIndex();

    //Replace least fittest individual from most fittest offspring
    population.individuals[leastFittestIndex] = getFittestOffspring();
}

}

//Individual class class Individual {

int fitness = 0;
int[] genes = new int[5];
int geneLength = 5;

public Individual() {
    Random rn = new Random();

    //Set genes randomly for each individual
    for (int i = 0; i < genes.length; i++) {
        genes[i] = Math.abs(rn.nextInt() % 2);
    }

    fitness = 0;
}

//Calculate fitness
public void calcFitness() {

    fitness = 0;
    for (int i = 0; i < 5; i++) {
        if (genes[i] == 1) {
            ++fitness;
        }
    }
}

}

//Population class class Population {

int popSize = 10;
Individual[] individuals = new Individual[10];
int fittest = 0;

//Initialize population
public void initializePopulation(int size) {
    for (int i = 0; i < individuals.length; i++) {
        individuals[i] = new Individual();
    }
}

//Get the fittest individual
public Individual getFittest() {
    int maxFit = Integer.MIN_VALUE;
    int maxFitIndex = 0;
    for (int i = 0; i < individuals.length; i++) {
        if (maxFit <= individuals[i].fitness) {
            maxFit = individuals[i].fitness;
            maxFitIndex = i;
        }
    }
    fittest = individuals[maxFitIndex].fitness;
    return individuals[maxFitIndex];
}

//Get the second most fittest individual
public Individual getSecondFittest() {
    int maxFit1 = 0;
    int maxFit2 = 0;
    for (int i = 0; i < individuals.length; i++) {
        if (individuals[i].fitness > individuals[maxFit1].fitness) {
            maxFit2 = maxFit1;
            maxFit1 = i;
        } else if (individuals[i].fitness > individuals[maxFit2].fitness) {
            maxFit2 = i;
        }
    }
    return individuals[maxFit2];
}

//Get index of least fittest individual
public int getLeastFittestIndex() {
    int minFitVal = Integer.MAX_VALUE;
    int minFitIndex = 0;
    for (int i = 0; i < individuals.length; i++) {
        if (minFitVal >= individuals[i].fitness) {
            minFitVal = individuals[i].fitness;
            minFitIndex = i;
        }
    }
    return minFitIndex;
}

//Calculate fitness of each individual
public void calculateFitness() {

    for (int i = 0; i < individuals.length; i++) {
        individuals[i].calcFitness();
    }
    getFittest();
}

} ```

we started out with a concept of relation that was not mathematical but conceptual associations with explicitly stated meaningful names and ways or traversing and interpreting the information in specific intended ways.

Once you had associative complexes accessibale you can always provide alternative interpretations and in fact adjust the structure as needed, refactor and create new associative complexes that treated info structures in a situated whole together with the means of presenting, reasoning, processing interpreting etc

Before COD's 'relational' model came information was processed and retrived via traversing meaningful associations We had NoSQL that is to say we did not just have constratints but the processes correspongind to situational intent could be invoked on the fly with the ability to check for constraine

Imposing constratint at write were necessary so taht memory/storga locations can be resued

No when data= infomrations/code can be stored immutably and retrieved by contant addresses and paths we can have true object model without trying to do the impossible and map the relational model into the object model. It is impossible because it imposes a separation of and enclusure of information from the means of presenting and morphing it whre these two should co-evolve

regulation about identify verification of students in Online courses

eLife assessment

This important manuscript uses a machine-learning approach to predict and annotate cis-regulatory elements across insect genomes, helping to address a much-needed gap in comparative genomics. This method does not rely on sequence alignments, thereby allowing functional genomics studies of more distant species, including emerging model organisms. There are nuanced views on the strength of the evidence from the predictions: the pipeline appears to be based on solid evidence, but the methods could be better described. We suggest the manuscript would be much more robust if the code used was accessible for review and validated further.

code-blockがそのまま表示されてしまっています。

Reviewer #1 (Public Review):

Summary:

The study introduces and validates the Cyclic Homogeneous Oscillation (CHO) detection method to precisely determine the duration, location, and fundamental frequency of non-sinusoidal neural oscillations. Traditional spectral analysis methods face challenges in distinguishing the fundamental frequency of non-sinusoidal oscillations from their harmonics, leading to potential inaccuracies. The authors implement an underexplored approach, using the auto-correlation structure to identify the characteristic frequency of an oscillation. By combining this strategy with existing time-frequency tools to identify when oscillations occur, the authors strive to solve outstanding challenges involving spurious harmonic peaks detected in time-frequency representations. Empirical tests using electrocorticographic (ECoG) and electroencephalographic (EEG) signals further support the efficacy of CHO in detecting neural oscillations.

Strengths:

The paper puts important emphasis on the 'identity' question of oscillatory identification. The field primarily identifies oscillations through frequency, space (brain region), and time (length, and relative to task or rest). However, more tools that claim to further characterize oscillations by their defining/identifying traits are needed, in addition to data-driven studies about what the identifiable traits of neural oscillations are beyond frequency, location, and time. Such tools are useful for potentially distinguishing between circuit mechanistic generators underlying signals that may not otherwise be distinguished. This paper states this problem well and puts forth a new type of objective for neural signal processing methods.

The paper uses synthetic data and multimodal recordings at multiple scales to validate the tool, suggesting CHO's robustness and applicability in various real-data scenarios. The figures illustratively demonstrate how CHO works on such synthetic and real examples, depicting in both time and frequency domains. The synthetic data are well-designed, and capable of producing transient oscillatory bursts with non-sinusoidal characteristics within 1/f noise. Using both non-invasive and invasive signals exposes CHO to conditions which may differ in the extent and quality of harmonic signal structure. An interesting follow-up question is whether the utility demonstrated here holds for MEG signals, as well as source-reconstructed signals from non-invasive recordings.

This study is accompanied by open-source code and data for use by the community.

Weaknesses:

The criteria that the authors use for neural oscillations embody some operating assumptions underlying their characteristics, perhaps informed by immediate use cases intended by the authors (e.g., hippocampal bursts). The extent to which these assumptions hold in all circumstances should be investigated. For instance, the notion of consistent auto-correlation breaks down in scenarios where instantaneous frequency fluctuates significantly at the scale of a few cycles. Imagine an alpha-beta complex without harmonics (Jones 2009). If oscillations change phase position within a timeframe of a few cycles, it would be difficult for a single peak in the auto-correlation structure to elucidate the complex time-varying peak frequency in a dynamic fashion. Likewise, it is unclear whether bounding boxes with a pre-specified overlap can capture complexes that manoeuvre across peak frequencies.

This method appears to lack the implementation of statistical inferential techniques for estimating and interpreting auto-correlation and spectral structure. In standard practice, auto-correlation functions and spectral measures can be subjected to statistical inference to establish confidence intervals, often helping to determine the significance of the estimates. Doing so would be useful for expressing the likelihood that an oscillation and its harmonic has the same auto-correlation structure and fundamental frequency, or more robustly identifying harmonic peaks in the presence of spectral noise. Here, the authors appear to use auto-correlation and time-frequency decomposition more as a deterministic tool rather than an inferential one. Overall, an inferential approach would help differentiate between true effects and those that might spuriously occur due to the nature of the data. Ultimately, a more statistically principled approach might estimate harmonic structure in the presence of noise in a unified manner transmitted throughout the methodological steps.

Author response:

The following is the authors’ response to the original reviews.

Public Reviews:

Reviewer #1 (Public Review):

Summary:

The study introduces and validates the Cyclic Homogeneous Oscillation (CHO) detection method to precisely determine the duration, location, and fundamental frequency of non-sinusoidal neural oscillations. Traditional spectral analysis methods face challenges in distinguishing the fundamental frequency of non-sinusoidal oscillations from their harmonics, leading to potential inaccuracies. The authors implement an underexplored approach, using the auto-correlation structure to identify the characteristic frequency of an oscillation. By combining this strategy with existing time-frequency tools to identify when oscillations occur, the authors strive to solve outstanding challenges involving spurious harmonic peaks detected in time-frequency representations. Empirical tests using electrocorticographic (ECoG) and electroencephalographic (EEG) signals further support the efficacy of CHO in detecting neural oscillations.

Response:  We thank the reviewer for recognizing the strengths of our method in this encouraging review and for the opportunity to further improve and finalize our manuscript.

Strengths:

(1) The paper puts an important emphasis on the 'identity' question of oscillatory identification. The field primarily identifies oscillations through frequency, space (brain region), and time (length, and relative to task or rest). However, more tools that claim to further characterize oscillations by their defining/identifying traits are needed, in addition to data-driven studies about what the identifiable traits of neural oscillations are beyond frequency, location, and time. Such tools are useful for potentially distinguishing between circuit mechanistic generators underlying signals that may not otherwise be distinguished. This paper states this problem well and puts forth a new type of objective for neural signal processing methods.

Response:  We sincerely appreciate this encouraging summary of the objective of our manuscript.

(2) The paper uses synthetic data and multimodal recordings at multiple scales to validate the tool, suggesting CHO's robustness and applicability in various real-data scenarios. The figures illustratively demonstrate how CHO works on such synthetic and real examples, depicting in both time and frequency domains. The synthetic data are well-designed, and capable of producing transient oscillatory bursts with non-sinusoidal characteristics within 1/f noise. Using both non-invasive and invasive signals exposes CHO to conditions which may differ in extent and quality of the harmonic signal structure. An interesting followup question is whether the utility demonstrated here holds for MEG signals, as well as source-reconstructed signals from non-invasive recordings.

Response:  We thank the reviewer for this excellent suggestion.  Indeed, our next paper will focus on applying our CHO method to signals that were source-reconstructed from non-invasive recordings (e.g., MEG and EEG) to extract their periodic activity.

(3) This study is accompanied by open-source code and data for use by the community.

Response:  We thank the reviewer for recognizing our effort to widely disseminate our method to the broader community.

Weaknesses:

(1) Due to the proliferation of neural signal processing techniques that have been designed to tackle issues such as harmonic activity, transient and event-like oscillations, and non-sinusoidal waveforms, it is naturally difficult for every introduction of a new tool to include exhaustive comparisons of all others. Here, some additional comparisons may be considered for the sake of context, a selection of which follows, biased by the previous exposure of this reviewer. One emerging approach that may be considered is known as state-space models with oscillatory and autoregressive components (Matsuda 2017, Beck 2022). State-space models such as autoregressive models have long been used to estimate the auto-correlation structure of a signal. State-space oscillators have recently been applied to transient oscillations such as sleep spindles (He 2023). Therefore, state-space oscillators extended with auto-regressive components may be able to perform the functions of the present tool through different means by circumventing the need to identify them in time-frequency. Another tool that should be mentioned is called PAPTO (Brady 2022). Although PAPTO does not address harmonics, it detects oscillatory events in the presence of 1/f background activity. Lastly, empirical mode decomposition (EMD) approaches have been studied in the context of neural harmonics and nonsinusoidal activity (Quinn 2021, Fabus 2022). EMD has an intrinsic relationship with extrema finding, in contrast with the present technique. In summary, the existence of methods such as PAPTO shows that researchers are converging on similar approaches to tackle similar problems. The existence of time-domain approaches such as state-space oscillators and EMD indicates that the field of timeseries analysis may yield even more approaches that are conceptually distinct and may theoretically circumvent the methodology of this tool.

Response:  We thank the reviewer for this valuable insight.  In our manuscript, we acknowledge emerging approaches that employ state-space models or EMD for time-frequency analysis.  However, it's crucial to clarify that the primary focus in our study is on the detection and identification of the fundamental frequency, as well as the onset/offset of non-sinusoidal neural oscillations.  Thus, our emphasis lies specifically on these aspects.  We hope that future studies will use our methods as the basis to develop better methods for time-frequency analysis that will lead to a deeper understanding of harmonic structures.  

Our Limitation section is addressing this issue.  Specifically, we recognize that a more sophisticated time-frequency analysis could contribute to improved sensitivity and that the core claim of our study is centered around the concept of increasing specificity in the detection of non-sinusoidal oscillations.  We hope that future studies will use this as a basis for improving time-frequency analysis in general.  Notably, our open-source code will greatly enable these future studies in this endeavor.  Specifically, in the first step of our algorithm, the timefrequency estimation can be replaced with any other preferred time-frequency analysis, such as state-space models, EMD, Wavelet transform, Gabor transform, and Matching Pursuit. 

For our own follow-up study, we plan to conduct a thorough review and comparison of emerging approaches employing state-space models or EMD for time-frequency analysis.  In this study, we aim to identify which approach, including the six methods mentioned by the reviewer (Matsuda 2017, Beck 2022, He 2023, Brady 2022, Quinn 2021, and Fabus 2022), can maximize the estimation of the fundamental frequency of non-sinusoidal neural oscillations using CHO.  The insights provided by the reviewer are appreciated, and we will carefully consider these aspects in our follow-up study.  

In the revision of this manuscript, we are setting the stage for these future studies.  Specifically, we added a discussion paragraph within the Limitation section about the state-space model, and EMD approaches:

“However, because our CHO method is modular, the FFT-based time-frequency analysis can be replaced with more sophisticated time-frequency estimation methods to improve the sensitivity of neural oscillation detection.  Specifically, a state-space model (Matsuda 2017, Beck 2022, He 2023, Brady 2022) or empirical mode decomposition (EMD, Quinn 2021, Fabus 2022) may improve the estimation of the auto-correlation of the harmonic structure underlying nonsinusoidal oscillations.  Furthermore, a Gabor transform or matching pursuit-based approach may improve the onset/offset detection of short burst-like neural oscillations (Kus 2013 and Morales 2022).”

(2) The criteria that the authors use for neural oscillations embody some operating assumptions underlying their characteristics, perhaps informed by immediate use cases intended by the authors (e.g., hippocampal bursts). The extent to which these assumptions hold in all circumstances should be investigated. For instance, the notion of consistent auto-correlation breaks down in scenarios where instantaneous frequency fluctuates significantly at the scale of a few cycles. Imagine an alpha-beta complex without harmonics (Jones 2009). If oscillations change phase position within a timeframe of a few cycles, it would be difficult for a single peak in the auto-correlation structure to elucidate the complex time-varying peak frequency in a dynamic fashion. Likewise, it is unclear whether bounding boxes with a pre-specified overlap can capture complexes that maneuver across peak frequencies.

Response:  We thank the reviewer for this valuable insight into the methodological limitations in the detection of neural oscillations that exhibit significant fluctuations in their instantaneous frequency.  Indeed, our CHO method is also limited in the ability to detect oscillations with fluctuating instantaneous frequencies.  This is because CHO uses an auto-correlation-based approach to detect neural oscillations that exhibit two or more cycles.  If oscillations change phase position within a timeframe of a few cycles, CHO cannot detect the oscillation because the periodicity is not expressed within the auto-correlation.  This limitation can be partially overcome by relaxing the detection threshold (see Line 30 of Algorithm 1 in the revised manuscript) for the auto-correlation analysis.  However, relaxing the detection threshold, in consequence, increases the probability of detecting other aperiodic activity as well. To clarify how CHO determines the periodicity of oscillations, and to educate the reader about the tradeoff between detecting oscillations with fluctuating instantaneous frequencies and avoiding detecting other aperiod activity, we have added pseudo code and a new subsection in the Methods.

Author response table 1.

Algorithm 1

A new subsection titled “Tradeoffs in adjusting the hyper-parameters that govern the detection in CHO”.

“The ability of CHO to detect neural oscillations and determine their fundamental frequency is governed by four principal hyper-parameters.  Adjusting these parameters requires understanding their effect on the sensitivity and specificity in the detection of neural oscillations. 

The first hyper-parameter is the number of time windows (N in Line 5 in Algorithm 1), that is used to estimate the 1/f noise.  In our performance assessment of CHO, we used four windows, resulting in estimation periods of 250 ms in duration for each 1/f spectrum.  A higher number of time windows results in smaller estimation periods and thus minimizes the likelihood of observing multiple neural oscillations within this time window, which otherwise could confound the 1/f estimation.  However, a higher number of time windows and, thus, smaller time estimation periods may lead to unstable 1/f estimates. 

The second hyper-parameter defines the minimum number of cycles of a neural oscillation to be detected by CHO (see Line 23 in Algorithm 1).  In our study, we specified this parameter to be two cycles.  Increasing the number of cycles increases specificity, as it will reject spurious oscillations.  However, increasing the number also reduces sensitivity as it will reject short oscillations.

The third hyper-parameter is the significance threshold that selects positive peaks within the auto-correlation of the signal.  The magnitude of the peaks in the auto-correlation indicates the periodicity of the oscillations (see Line 26 in Algorithm 1).  Referred to as "NumSTD," this parameter denotes the number of standard errors that a positive peak has to exceed to be selected to be a true oscillation.  For this study, we set the "NumSTD" value to 1.  Increasing the "NumSTD" value increases specificity in the detection as it reduces the detection of spurious peaks in the auto-correlation.  However, increasing the "NumSTD" value also decreases the sensitivity in the detection of neural oscillations with varying instantaneous oscillatory frequencies. 

The fourth hyper-parameter is the percentage of overlap between two bounding boxes that trigger their merger (see Line 31 in Algorithm 1).  In our study, we set this parameter to 75% overlap.  Increasing this threshold yields more fragmentation in the detection of oscillations, while decreasing this threshold may reduce the accuracy in determining the onset and offset of neural oscillations.”

(3) Related to the last item, this method appears to lack implementation of statistical inferential techniques for estimating and interpreting auto-correlation and spectral structure. In standard practice, auto-correlation functions and spectral measures can be subjected to statistical inference to establish confidence intervals, often helping to determine the significance of the estimates. Doing so would be useful for expressing the likelihood that an oscillation and its harmonic has the same autocorrelation structure and fundamental frequency, or more robustly identifying harmonic peaks in the presence of spectral noise. Here, the authors appear to use auto-correlation and time-frequency decomposition more as a deterministic tool rather than an inferential one. Overall, an inferential approach would help differentiate between true effects and those that might spuriously occur due to the nature of the data. Ultimately, a more statistically principled approach might estimate harmonic structure in the presence of noise in a unified manner transmitted throughout the methodological steps.

Response:  We thank the reviewer for sharing this insight on further enhancing our method.  Indeed, CHO does not make use of statistical inferential statistics to estimate and interpret the auto-correlation and underlying spectral structure of the neural oscillation.  Implementing this approach within CHO would require calculating phase-phase coupling across all cross-frequency bands and bounding boxes.  However, as mentioned in the introduction section and Figure 1GL, phase-phase coupling analysis cannot fully ascertain whether the oscillations are phaselocked and thus are harmonics or, indeed, independent oscillations.  This ambiguity, combined with the exorbitant computational complexity of the entailed permutation test and the requirement to perform the analysis across all cross-frequency bands, channels, and trials, makes phase-phase coupling impracticable in determining the fundamental frequency of neural oscillations in real-time and, thus, the use in closed-loop neuromodulation applications.  Thus, within our study, we prioritized determining the fundamental frequency without considering the structure of harmonics.  

An inferential approach can be implemented by adjusting the significance threshold that selects positive peaks within the auto-correlation of the signal.  Currently, this threshold is set to represent the approximate confidence bounds of the periodicity of the fundamental frequency.  To clarify this issue, we added additional pseudo code and a new subsection, titled “Tradeoffs in adjusting the hyper-parameters that govern the detection in CHO,” in the Methods section.

In future studies, we will investigate the harmonic structure of neural oscillations based on a large data set.  This exploration will help us understand how non-sinusoidal properties may influence the harmonic structure.  Your input is highly appreciated, and we will diligently incorporate these considerations into our research.

Author response table 2.

Algorithm 1

A new subsection titled “Tradeoffs in adjusting the hyper-parameters that govern the detection in CHO”.

“The ability of CHO to detect neural oscillations and determine their fundamental frequency is governed by four principal hyper-parameters.  Adjusting these parameters requires understanding their effect on the sensitivity and specificity in the detection of neural oscillations. 

The first hyper-parameter is the number of time windows (N in Line 5 in Algorithm 1), that is used to estimate the 1/f noise.  In our performance assessment of CHO, we used four windows, resulting in estimation periods of 250 ms in duration for each 1/f spectrum.  A higher number of time windows results in smaller estimation periods and thus minimizes the likelihood of observing multiple neural oscillations within this time window, which otherwise could confound the 1/f estimation.  However, a higher number of time windows and, thus, smaller time estimation periods may lead to unstable 1/f estimates. 

The second hyper-parameter defines the minimum number of cycles of a neural oscillation to be detected by CHO (see Line 23 in Algorithm 1).  In our study, we specified this parameter to be two cycles.  Increasing the number of cycles increases specificity, as it will reject spurious oscillations.  However, increasing the number also reduces sensitivity as it will reject short oscillations.

The third hyper-parameter is the significance threshold that selects positive peaks within the auto-correlation of the signal.  The magnitude of the peaks in the auto-correlation indicates the periodicity of the oscillations (see Line 26 in Algorithm 1).  Referred to as "NumSTD," this parameter denotes the number of standard errors that a positive peak has to exceed to be selected to be a true oscillation.  For this study, we set the "NumSTD" value to 1.  Increasing the "NumSTD" value increases specificity in the detection as it reduces the detection of spurious peaks in the auto-correlation.  However, increasing the "NumSTD" value also decreases the sensitivity in the detection of neural oscillations with varying instantaneous oscillatory frequencies. 

The fourth hyper-parameter is the percentage of overlap between two bounding boxes that trigger their merger (see Line 31 in Algorithm 1).  In our study, we set this parameter to 75% overlap.  Increasing this threshold yields more fragmentation in the detection of oscillations, while decreasing this threshold may reduce the accuracy in determining the onset and offset of neural oscillations.”

(4) As with any signal processing method, hyperparameters and their ability to be tuned by the user need to be clearly acknowledged, as they impact the robustness and reproducibility of the method. Here, some of the hyperparameters appear to be: a) number of cycles around which to construct bounding boxes and b) overlap percentage of bounding boxes for grouping. Any others should be highlighted by the authors and clearly explained during the course of tool dissemination to the community, ideally in tutorial format through the Github repository.

Response:  We thank the reviewer for this helpful suggestion.  In response, we added a new subsection that describes the hyper-parameters of CHO as follows:

A new subsection named “Tradeoffs in adjusting the hyper-parameters that govern the detection in CHO”.

“The ability of CHO to detect neural oscillations and determine their fundamental frequency is governed by four principal hyper-parameters.  Adjusting these parameters requires understanding their effect on the sensitivity and specificity in the detection of neural oscillations. 

The first hyper-parameter is the number of time windows (N in Line 5 in Algorithm 1), that is used to estimate the 1/f noise.  In our performance assessment of CHO, we used four windows, resulting in estimation periods of 250 ms in duration for each 1/f spectrum.  A higher number of time windows results in smaller estimation periods and thus minimizes the likelihood of observing multiple neural oscillations within this time window, which otherwise could confound the 1/f estimation.  However, a higher number of time windows and, thus, smaller time estimation periods may lead to unstable 1/f estimates. 

The second hyper-parameter defines the minimum number of cycles of a neural oscillation to be detected by CHO (see Line 23 in Algorithm 1).  In our study, we specified this parameter to be two cycles.  Increasing the number of cycles increases specificity, as it will reject spurious oscillations.  However, increasing the number also reduces sensitivity as it will reject short oscillations.

The third hyper-parameter is the significance threshold that selects positive peaks within the auto-correlation of the signal.  The magnitude of the peaks in the auto-correlation indicates the periodicity of the oscillations (see Line 26 in Algorithm 1).  Referred to as "NumSTD," this parameter denotes the number of standard errors that a positive peak has to exceed to be selected to be a true oscillation.  For this study, we set the "NumSTD" value to 1.  Increasing the "NumSTD" value increases specificity in the detection as it reduces the detection of spurious peaks in the auto-correlation.  However, increasing the "NumSTD" value also decreases the sensitivity in the detection of neural oscillations with varying instantaneous oscillatory frequencies. 

The fourth hyper-parameter is the percentage of overlap between two bounding boxes that trigger their merger (see Line 31 in Algorithm 1).  In our study, we set this parameter to 75% overlap.  Increasing this threshold yields more fragmentation in the detection of oscillations, while decreasing this threshold may reduce the accuracy in determining the onset and offset of neural oscillations.”

(5) Most of the validation demonstrations in this paper depict the detection capabilities of CHO. For example, the authors demonstrate how to use this tool to reduce false detection of oscillations made up of harmonic activity and show in simulated examples how CHO performs compared to other methods in detection specificity, sensitivity, and accuracy. However, the detection problem is not the same as the 'identity' problem that the paper originally introduced CHO to solve. That is, detecting a non-sinusoidal oscillation well does not help define or characterize its non-sinusoidal 'fingerprint'. An example problem to set up this question is: if there are multiple oscillations at the same base frequency in a dataset, how can their differing harmonic structure be used to distinguish them from each other? To address this at a minimum, Figure 4 (or a followup to it) should simulate signals at similar levels of detectability with different 'identities' (i.e. different levels and/or manifestations of harmonic structure), and evaluate CHO's potential ability to distinguish or cluster them from each other. Then, does a real-world dataset or neuroscientific problem exist in which a similar sort of exercise can be conducted and validated in some way? If the "what" question is to be sufficiently addressed by this tool, then this type of task should be within the scope of its capabilities, and validation within this scenario should be demonstrated in the paper. This is the most fundamental limitation at the paper's current state.

Response: Thank you for your insightful suggestion; we truly appreciate it. We recognize that the 'identity' problem requires further studies to develop appropriate methods. Our current approach does not fully address this issue, as it may detect asymmetric non-sinusoidal oscillations with multiple harmonic peaks, without accounting for different shapes of nonsinusoidal oscillations.

The main reason we could not fully address the “identity” problem results from the general absence of a defined ground truth, i.e., data for which we know the harmonic structure. To overcome this barrier, we would need datasets from well-characterized cognitive tasks or neural disorders.  For example, Cole et al. 2017 showed that the harmonic structure of beta oscillations can explain the degree of Parkinson’s disease, and Hu et al. 2023 showed that the number of harmonic peaks can localize the seizure onset zone. Future studies could use the data from these two studies to study whether CHO can distinguish different harmonic structures of pathological neural oscillations.

In this paper, we showed the basic identity of neural oscillations, encompassing elements such as the fundamental frequency and onset/offset. Your valuable insights contribute significantly to our ongoing efforts, and we appreciate your thoughtful consideration of these aspects. In response, we added a new paragraph in the Limitation of the discussion section as below:

“Another limitation of this study is that it does not assess the harmonic structure of neural oscillations. Thus, CHO cannot distinguish between oscillations that have the same fundamental frequency but differ in their non-sinusoidal properties.  This limitation stems from the objective of this study, which is to identify the fundamental frequency of non-sinusoidal neural oscillations.  Overcoming this limitation requires further studies to improve CHO to distinguish between different non-sinusoidal properties of pathological neural oscillations.  The data that is necessary for these further studies could be obtained from the wide range of studies that have linked the harmonic structures in the neural oscillations to various cognitive functions (van Dijk et al., 2010; Schalk, 2015; Mazaheri and Jensen, 2008) and neural disorders (Cole et al., 2017; Jackson et al., 2019; Hu et al., 2023). For example, Cole et al. 2017 showed that a harmonic structure of beta oscillations can explain the degree of Parkinson’s disease, and Hu et al. 2023 showed the number of harmonic peaks can localize the seizure onset zone. “

References:

Beck AM, He M, Gutierrez R, Purdon PL. An iterative search algorithm to identify oscillatory dynamics in neurophysiological time series. bioRxiv. 2022. p. 2022.10.30.514422.

doi:10.1101/2022.10.30.514422

Brady B, Bardouille T. Periodic/Aperiodic parameterization of transient oscillations (PAPTO)Implications for healthy ageing. Neuroimage. 2022;251: 118974.

Fabus MS, Woolrich MW, Warnaby CW, Quinn AJ. Understanding Harmonic Structures Through Instantaneous Frequency. IEEE Open J Signal Process. 2022;3: 320-334.

Jones SR, Pritchett DL, Sikora MA, Stufflebeam SM, Hämäläinen M, Moore CI. Quantitative analysis and biophysically realistic neural modeling of the MEG mu rhythm: rhythmogenesis and modulation of sensory-evoked responses. J Neurophysiol. 2009;102: 3554-3572.

He M, Das P, Hotan G, Purdon PL. Switching state-space modeling of neural signal dynamics. PLoS Comput Biol. 2023;19: e1011395.

Matsuda T, Komaki F. Time Series Decomposition into Oscillation Components and Phase Estimation. Neural Comput. 2017;29: 332-367.

Quinn AJ, Lopes-Dos-Santos V, Huang N, Liang W-K, Juan C-H, Yeh J-R, et al. Within-cycle instantaneous frequency profiles report oscillatory waveform dynamics. J Neurophysiol. 2021;126: 1190-1208.

Reviewer #2 (Public Review):

Summary:

A new toolbox is presented that builds on previous toolboxes to distinguish between real and spurious oscillatory activity, which can be induced by non-sinusoidal waveshapes. Whilst there are many toolboxes that help to distinguish between 1/f noise and oscillations, not many tools are available that help to distinguish true oscillatory activity from spurious oscillatory activity induced in harmonics of the fundamental frequency by non-sinusoidal waveshapes. The authors present a new algorithm which is based on autocorrelation to separate real from spurious oscillatory activity. The algorithm is extensively validated using synthetic (simulated) data, and various empirical datasets from EEG, intracranial EEG in various locations and domains (i.e. auditory cortex, hippocampus, etc.).

Strengths:

Distinguishing real from spurious oscillatory activity due to non-sinusoidal waveshapes is an issue that has plagued the field for quite a long time. The presented toolbox addresses this fundamental problem which will be of great use for the community. The paper is written in a very accessible and clear way so that readers less familiar with the intricacies of Fourier transform and signal processing will also be able to follow it. A particular strength is the broad validation of the toolbox, using synthetic, scalp EEG, EcoG, and stereotactic EEG in various locations and paradigms.

Weaknesses:

At many parts in the results section critical statistical comparisons are missing (e.g. FOOOF vs CHO). Another weakness concerns the methods part which only superficially describes the algorithm. Finally, a weakness is that the algorithm seems to be quite conservative in identifying oscillatory activity which may render it only useful for analysing very strong oscillatory signals (i.e.

alpha), but less suitable for weaker oscillatory signals (i.e. gamma).

Response: We thank Reviewer #2 for the assistance in improving this manuscript.  In the revised manuscript, we have added the missing statistical comparisons, detailed pseudo code, and a subsection that explains the hyper-parameters of CHO.  We also recognize the limitations of CHO in detecting gamma oscillations.  While our results demonstrate beta-band oscillations in ECoG and EEG signals (see Figures 5 and 6), we had no expectation to find gamma-band oscillations during a simple reaction time task.  This is because of the general absence of ECoG electrodes over the occipital cortex, where such gamma-band oscillations may be found. 

Nevertheless, our CHO method should be able to detect gamma-band oscillations.  This is because if there are gamma-band oscillations, they will be reflected as a bump over the 1/f fit in the power spectrum, and CHO will detect them.  We apologize for not specifying the frequency range of the synthetic non-sinusoidal oscillations.  The gamma band was also included in our simulation. We added the frequency range (1-40 Hz) of the synthetic nonsinusoidal oscillations in the subsection, the caption of Figure 4, and the result section.

Reviewer #1 (Recommendations For The Authors):

(1) The example of a sinusoidal neural oscillation in Fig 1 seems to still exhibit a great deal of nonsinusoidal behavior. Although it is largely symmetrical, it has significant peak-trough symmetry as well as sharper peak structure than typical sinusoidal activity. Nevertheless, it has less harmonic structure than the example on the left. A more precisely-stated claim might be that non-sinusoidal behavior is not the distinguishing characteristic between the two, but rather the degree of harmonic structure.

Response: We are grateful for this thoughtful observation. In response, we now recognize that the depicted example showcases pronounced peak-trough symmetry and sharpness, characteristics that might not be typically associated with sinusoidal behavior. We now better understand that the key differentiator between the examples lies not only in their nonsinusoidal behavior but also in their harmonic structure. To reflect this better understanding, we have refined our manuscript to more accurately articulate the differences in harmonic structure, in accordance with your suggestion. Specifically, we revised the caption of Fig 1 in the manuscript as follows:

The caption of the Fig 1G-L.

“We applied the same statistical test to a more sinusoidal neural oscillation (G). Since this neural oscillation more closely resembles a sinusoidal shape, it does not exhibit any prominent harmonic peaks in the alpha and beta bands within the power spectrum (H) and time-frequency domain (I).  Consequently, our test found that the phase of the theta-band and beta-band oscillations were not phase-locked (J-L).  Thus, this statistical test suggests the absence of a harmonic structure.”

(2) The statement "This suggests that most of the beta oscillations

detected by conventional methods are simply harmonics of the predominant asymmetric alpha oscillation." is potentially overstated. It is important to constrain this statement to the auditory cortex in which the authors conduct the validation, because true beta still exists elsewhere. The same goes for the beta-gamma claim later on. In general, use of "may be" is also more advisable than the definitive "are".

Response: We thank the reviewer for this thoughtful feedback. To avoid the potential overstatement of our findings we revised our statement on beta oscillations in the manuscript as follows:

Discussion:

“This suggests that most of the beta oscillations detected by conventional methods within auditory cortex may be simply harmonics of the predominant asymmetric alpha oscillation.”

Reviewer #2 (Recommendations For The Authors):

All my concerns are medium to minor and I list them as they appear in the manuscript. I do not suggest new experiments or a change in the results, instead I focus on writing issues only.

a) Line 50: A reference to the seminal paper by Klimesch et al (2007) on alpha oscillations and inhibition would seem appropriate here.

Response: We added the reference to Klimesch et al. (2007).

b) Figure 4: It is unclear which length for the simulated oscillations was used to generate the data in panels B-G.

Response: We generated oscillations that were 2.5 cycles in length and 1-3 seconds in duration. We added this information to the manuscript as follows.

Figure 4:

“We evaluated CHO by verifying its specificity, sensitivity, and accuracy in detecting the fundamental frequency of non-sinusoidal oscillatory bursts (2.5 cycles, 1–3 seconds long) convolved with 1/f noise.”

Results (page 5, lines 163-165):

“To determine the specificity and sensitivity of CHO in detecting neural oscillations, we applied CHO to synthetic non-sinusoidal oscillatory bursts (2.5 cycles, 1–3 seconds long) convolved with 1/f noise, also known as pink noise, which has a power spectral density that is inversely proportional to the frequency of the signal.”

Methods (page 20, lines 623-626):

“While empirical physiological signals are most appropriate for validating our method, they generally lack the necessary ground truth to characterize neural oscillation with sinusoidal or non-sinusoidal properties. To overcome this limitation, we first validated CHO on synthetic nonsinusoidal oscillatory bursts (2.5 cycles, 1–3 seconds long) convolved with 1/f noise to test the performance of the proposed method.”

c) Figure 5 - supplements: Would be good to re-organize the arrangement of the plots on these figures to facilitate the comparison between Foof and CHO (i.e. by presenting for each participant FOOOF and CHO together).

Response: We combined Figure 5-supplementary figures 1 and 2 into Figure 5-supplementary figure 1, Figure 6-supplementary figures 1 and 2 into Figure 6-supplementary figure 1, and Figure 8-supplementary figures 1 and 2 into Figure 8-supplementary figure 1. 

Author response image 1.

Figure 5-supplementary figure 1:

Author response image 2.

Figure 6-supplementary figure 1:

Author response image 3.

Figure 8-supplementary figure 1:

d) Statistics: Almost throughout the results section where the empirical results are described statistical comparisons are missing. For instance, in lines 212-213 the statement that CHO did not detect low gamma while FOOOF did is not backed up by the appropriate statistics. This issue is also evident in all of the following sections (i.e. EEG results, On-offsets of oscillations, SEEG results, Frequency and duration of oscillations). I feel this is probably the most important point that needs to be addressed.

Response: We added statistical comparisons to Figure 5 (ECoG), 6 (EEG), and the results section as follows.

Author response image 4.

Validation of CHO in detecting oscillations in ECoG signals. A. We applied CHO and FOOOF to determine the fundamental frequency of oscillations from ECoG signals recorded during the pre-stimulus period of an auditory reaction time task. FOOOF detected oscillations primarily in the alpha- and beta-band over STG and pre-motor area.  In contrast, CHO also detected alpha-band oscillations primarily within STG, and more focal beta-band oscillations over the pre-motor area, but not STG. B. We investigated the occurrence of each oscillation within defined cerebral regions across eight ECoG subjects. The horizontal bars and horizontal lines represent the median and median absolute deviation (MAD) of oscillations occurring across the eight subjects. An asterisk (*) indicates statistically significant differences in oscillation detection between CHO and FOOOF (Wilcoxon rank-sum test, p<0.05 after Bonferroni correction).”

Author response image 5.

Validation of CHO in detecting oscillations in EEG signals. A. We applied CHO and FOOOF to determine the fundamental frequency of oscillations from EEG signals recorded during the pre-stimulus period of an auditory reaction time task.  FOOOF primarily detected alpha-band oscillations over frontal/visual areas and beta-band oscillations across all areas (with a focus on central areas). In contrast, CHO detected alpha-band oscillations primarily within visual areas and detected more focal beta-band oscillations over the pre-motor area, similar to the ECoG results shown in Figure 5. B. We investigated the occurrence of each oscillation within the EEG signals across seven subjects. An asterisk (*) indicates statistically significant differences in oscillation detection between CHO and FOOOF (Wilcoxon rank-sum test, p<0.05 after Bonferroni correction). CHO exhibited lower entropy values of alpha and beta occurrence than FOOOF across 64 channels. C. We compared the performance of FOOO and CHO in detecting oscillation across visual and pre-motor-related EEG channels. CHO detected more alpha and beta oscillations in visual cortex than in pre-motor cortex. FOOOF detected alpha and beta oscillations in visual cortex than in pre-motor cortex.

We added additional explanations of our statistical results to the “Electrocorticographic (ECoG) results” and “Electroencephalographic (EEG) results” sections.

“We compared neural oscillation detection rates between CHO and FOOOF across eight ECoG subjects.  We used FreeSurfer to determine the associated cerebral region for each ECoG location. Each subject performed approximately 400 trials of a simple auditory reaction-time task.  We analyzed the neural oscillations during the 1.5-second-long pre-stimulus period within each trial. CHO and FOOOF demonstrated statistically comparable results in the theta and alpha bands despite CHO exhibiting smaller median occurrence rates than FOOOF across eight subjects. Notably, within the beta band, excluding specific regions such as precentral, pars opercularis, and caudal middle frontal areas, CHO's beta oscillation detection rate was significantly lower than that of FOOOF (Wilcoxon rank-sum test, p < 0.05 after Bonferroni correction). This suggests comparable detection rates between CHO and FOOOF in premotor and Broca's areas, while the detection of beta oscillations by FOOOF in other regions, such as the temporal area, may represent harmonics of theta or alpha, as illustrated in Figure 5A and B. Furthermore, FOOOF exhibited a higher sensitivity in detecting delta, theta, and low gamma oscillations overall, although both CHO and FOOOF detected only a limited number of oscillations in these frequency bands.”

“We assessed the difference in neural oscillation detection performance between CHO and FOOOF across seven EEG subjects.  We used EEG electrode locations according to the 10-10 electrode system and assigned each electrode to the appropriate underlying cortex (e.g., O1 and O2 for the visual cortex). Each subject performed 200 trials of a simple auditory reaction-time task.  We analyzed the neural oscillations during the 1.5-second-long pre-stimulus period. In the alpha band, CHO and FOOOF presented statistically comparable outcomes. However, CHO exhibited a greater alpha detection rate for the visual cortex than for the pre-motor cortex, as shown in Figures 6B and C. The entropy of CHO's alpha oscillation occurrences (3.82) was lower than that of FOOOF (4.15), with a maximal entropy across 64 electrodes of 4.16. Furthermore, in the beta band, CHO's entropy (4.05) was smaller than that of FOOOF (4.15). These findings suggest that CHO may offer a more region-specific oscillation detection than FOOOF.

As illustrated in Figure 6C, CHO found fewer alpha oscillations in pre-motor cortex (FC2 and FC4) than in occipital cortex (O1 and O2), while FOOOF found more beta oscillations occurrences in pre-motor cortex (FC2 and FC4) than in occipital cortex. However, FOOOF found more alpha and beta oscillations in visual cortex than in pre-motor cortex.

Consistent with ECoG results, FOOOF demonstrated heightened sensitivity in detecting delta, theta, and low gamma oscillations. 

Nonetheless, both CHO and FOOOF identified only a limited number of oscillations in delta and theta frequency bands.

Contrary to the ECoG results, FOOOF found more low gamma oscillations in EEG subjects than in ECoG subjects.”

e) Line 248: The authors find an oscillatory signal in the hippocampus with a frequency at around 8 Hz, which they refer to as alpha. However, several researchers (including myself) may label this fast theta, according to the previous work showing the presence of fast and slow theta oscillations in the human hippocampus (https://pubmed.ncbi.nlm.nih.gov/21538660/, https://pubmed.ncbi.nlm.nih.gov/32424312/).

Response: We replaced “alpha” with “fast theta” in the figure and text. We added a citation for Lega et al. 2012.

f) Line 332: It could also be possible that the auditory alpha rhythms don’t show up in the EEG because a referencing method was used that was not ideal for picking it up. In general, re-referencing is an important preprocessing step that can make the EEG be more susceptible to deep or superficial sources and that should be taken into account when interpreting the data.

Response: We re-referenced our signals using a common median reference (see Methods section). After close inspection of our results, we found that the EEG topography shown in Figure 6 did not show the auditory alpha oscillation because the alpha power of visual locations greatly exceeded that of those locations that reflect oscillations in the auditory cortex. Further, while our statistical analysis shows that CHO detected auditory alpha oscillations, this analysis also shows that CHO detected significantly more visual alpha oscillations.

g) Line 463: It seems that the major limitation of the algorithm lies in its low sensitivity which is discussed by the authors. The authors seem to downplay this a bit by saying that the algorithm works just fine at SNRs that are comparable to alpha oscillations. However, alpha is the strongest single in human EEG which may make the algorithm less suitable for picking up less prominent oscillatory signals, i.e. gamma, theta, ripples, etc. Is CHO only seeing the ‘tip of the iceberg’?

Response:  We performed the suggested analysis. For the theta band, this analysis generated convincing statistical results for ECoG signals (Figures 5, 6, and the results section). For theta oscillation detection, we found no statistical difference between CHO and FOOOF.  Since FOOOF has a high sensitivity even under SNRs (as shown in our simulation), our analysis suggests that CHO and FOOOF should perform equally well in the detection of theta oscillation, even when the theta oscillation amplitude is small.

To validate the ability of CHO to detect oscillations in high-frequency bands (> 40Hz), such as gamma oscillations and ripples, our follow-up study is applying CHO in the detection of highfrequency oscillations (HFOs) in electrocorticographic signals recorded during seizures.  To this end, our follow-up study analyzed 26 seizures from six patients.  In this analysis, CHO showed similar sensitivity and specificity as the epileptogenicity index (EI), which is the most commonly used method to detect seizure onset times and zones. The results of this follow-up study were presented at the American Epilepsy Society Meeting in December of 2023, and we are currently preparing a manuscript for submission to a peer-reviewed journal. 

In this study, we want to investigate the performance of CHO in detecting the most prominent neural oscillations (e.g., alpha and beta). Future studies will investigate the performance of  CHO in detecting more difficult to observe oscillations (delta in sleep stages, theta in the hippocampus during memory tasks, and high-frequency oscillation or ripples in seizure or interictal data. 

h) Methods: The methods section, especially the one describing the CHO algorithm, is lacking a lot of detail that one usually would like to see in order to rebuild the algorithm themselves. I appreciate that the code is available freely, but that does not, in my opinion, relief the authors of their duty to describe in detail how the algorithm works. This should be fixed before publishing.

Response: We now present pseudo code to describe the algorithms within the new subsection on the hyper-parameterization of CHO.

Author response table 3.

Algorithm 1

A new subsection titled “Tradeoffs in adjusting the hyper-parameters that govern the detection in CHO.”

“The ability of CHO to detect neural oscillations and determine their fundamental frequency is governed by four principal hyper-parameters.  Adjusting these parameters requires understanding their effect on the sensitivity and specificity in the detection of neural oscillations. 

The first hyper-parameter is the number of time windows (N in Line 5 in Algorithm 1), that is used to estimate the 1/f noise.  In our performance assessment of CHO, we used four time windows, resulting in estimation periods of 250 ms in duration for each 1/f spectrum.  A higher number of time windows results in smaller estimation periods and thus minimizes the likelihood of observing multiple neural oscillations within this time window, which otherwise could confound the 1/f estimation.  However, a higher number of time windows and, thus, smaller time estimation periods may lead to unstable 1/f estimates. 

The second hyper-parameter defines the minimum number of cycles of a neural oscillation to be detected by CHO (see Line 23 in Algorithm 1).  In our study, we specified this parameter to be two cycles.  Increasing the number of cycles increases specificity, as it will reject spurious oscillations.  However, increasing the number also sensitivity as it will reject short oscillations.

The third hyper-parameter is the significance threshold that selects positive peaks within the auto-correlation of the signal.  The magnitude of the peaks in the auto-correlation indicates the periodicity of the oscillations (see Line 26 in Algorithm 1).  Referred to as "NumSTD," this parameter denotes the number of standard errors that a positive peak has to exceed to be selected to be a true oscillation.  For this study, we set the "NumSTD" value to 1 (the approximate 68% confidence bounds).  Increasing the "NumSTD" value increases specificity in the detection as it reduces the detection of spurious peaks in the auto-correlation.  However, increasing the "NumSTD" value also decreases the sensitivity in the detection of neural oscillations with varying instantaneous oscillatory frequencies. 

The fourth hyper-parameter is the percentage of overlap between two bounding boxes that trigger their merger (see Line 31 in Algorithm 1).  In our study, we set this parameter to 75% overlap.  Increasing this threshold yields more fragmentation in the detection of oscillations, while decreasing this threshold may reduce the accuracy in determining the onset and offset of neural oscillations.”

DOI: 10.1016/j.devcel.2024.05.014

Resource: (IMSR Cat# CRL_022,RRID:IMSR_CRL:022)

Curator: @abever99

SciCrunch record: RRID:IMSR_CRL:022

---

What is this?

Professor has to judge on academic misconduct / honor code violation

Everyone is a reporter

form sample

???

you can use the files in starters or solutions folder if you don't have the files from previous labs - from explorer view in vs code connection

ここは、上の「in」と同様にcodeタグで囲んだほうがいいのではないでしょうか？

Author response:

The following is the authors’ response to the original reviews.

Public Reviews:

Reviewer #1 (Public Review):

Summary:

Protein conformational changes are often critical to protein function, but obtaining structural information about conformational ensembles is a challenge. Over a number of years, the authors of the current manuscript have developed and improved an algorithm, qFit protein, that models multiple conformations into high resolution electron density maps in an automated way. The current manuscript describes the latest improvements to the program, and analyzes the performance of qFit protein in a number of test cases, including classical statistical metrics of data fit like Rfree and the gap between Rwork and Rfree, model geometry, and global and case-by-case assessment of qFit performance at different data resolution cutoffs. The authors have also updated qFit to handle cryo-EM datasets, although the analysis of its performance is more limited due to a limited number of high-resolution test cases and less standardization of deposited/processed data.

Strengths:

The strengths of the manuscript are the careful and extensive analysis of qFit's performance over a variety of metrics and a diversity of test cases, as well as the careful discussion of the limitations of qFit. This manuscript also serves as a very useful guide for users in evaluating if and when qFit should be applied during structural refinement.

Reviewer #2 (Public Review):

Summary

The manuscript by Wankowicz et al. describes updates to qFit, an algorithm for the characterization of conformational heterogeneity of protein molecules based on X-ray diffraction of Cryo-EM data. The work provides a clear description of the algorithm used by qFit. The authors then proceed to validate the performance of qFit by comparing it to deposited X-ray entries in the PDB in the 1.2-1.5 Å resolution range as quantified by Rfree, Rwork-Rfree, detailed examination of the conformations introduced by qFit, and performance on stereochemical measures (MolProbity scores). To examine the effect of experimental resolution of X-ray diffraction data, they start from an ultra high-resolution structure (SARS-CoV2 Nsp3 macrodomain) to determine how the loss of resolution (introduced artificially) degrades the ability of qFit to correctly infer the nature and presence of alternate conformations. The authors observe a gradual loss of ability to correctly infer alternate conformations as resolution degrades past 2 Å. The authors repeat this analysis for a larger set of entries in a more automated fashion and again observe that qFit works well for structures with resolutions better than 2 Å, with a rapid loss of accuracy at lower resolution. Finally, the authors examine the performance of qFit on cryo-EM data. Despite a few prominent examples, the authors find only a handful (8) of datasets for which they can confirm a resolution better than 2.0 Å. The performance of qFit on these maps is encouraging and will be of much interest because cryo-EM maps will, presumably, continue to improve and because of the rapid increase in the availability of such data for many supramolecular biological assemblies. As the authors note, practices in cryo-EM analysis are far from uniform, hampering the development and assessment of tools like qFit.

Strengths

qFit improves the quality of refined structures at resolutions better than 2.0 A, in terms of reflecting true conformational heterogeneity and geometry. The algorithm is well designed and does not introduce spurious or unnecessary conformational heterogeneity. I was able to install and run the program without a problem within a computing cluster environment. The paper is well written and the validation thorough.

I found the section on cryo-EM particularly enlightening, both because it demonstrates the potential for discovery of conformational heterogeneity from such data by qFit, and because it clearly explains the hurdles towards this becoming common practice, including lack of uniformity in reporting resolution, and differences in map and solvent treatment.

Weaknesses

The authors begin the results section by claiming that they made "substantial improvement" relative to the previous iteration of qFit, "both algorithmically (e.g., scoring is improved by BIC, sampling of B factors is now included) and computationally (improving the efficiency and reliability of the code)" (bottom of page 3). However, the paper does not provide a comparison to previous iterations of the software or quantitation of the effects of these specific improvements, such as whether scoring is improved by the BIC, how the application of BIC has changed since the previous paper, whether sampling of B factors helps, and whether the code faster. It would help the reader to understand what, if any, the significance of each of these improvements was.

Indeed, it is difficult (embarrassingly) to benchmark against our past work due to the dependencies on different python packages and the lack of software engineering. With the infrastructure we’ve laid down with this paper, made possible by an EOSS grant from CZI, that will not be a problem going forward. Not only is the code more reliable and standardized, but we have developed several scientific test sets that can be used as a basis for broad comparisons to judge whether improvements are substantial. We’ve also changed with “substantial improvement” to “several modifications”  to indicate the lack of comparison to past versions.

The exclusion of structures containing ligands and multichain protein models in the validation of qFit was puzzling since both are very common in the PDB. This may convey the impression that qFit cannot handle such use cases. (Although it seems that qFit has an algorithm dedicated to modeling ligand heterogeneity and seems to be able to handle multiple chains). The paper would be more effective if it explained how a user of the software would handle scenarios with ligands and multiple chains, and why these would be excluded from analysis here.

qFit can indeed handle both. We left out multiple chains for simplicity in constructing a dataset enriched for small proteins while still covering diversity to speed the ability to rapidly iterate and test our approaches. Improvements to qFit ligand handling will be discussed in a forthcoming work as we face similar technical debt to what we saw in proteins and are undergoing a process of introducing “several modifications” that we hope will lead to “substantial improvement” - but at the very least will accelerate further development.

It would be helpful to add some guidance on how/whether qFit models can be further refined afterwards in Coot, Phenix, ..., or whether these models are strictly intended as the terminal step in refinement.

We added to the abstract:

“Importantly, unlike ensemble models, the multiconformer models produced by qFit can be manually modified in most major model building software (e.g. Coot)  and fit can be further improved by refinement using standard pipelines (e.g. Phenix, Refmac, Buster).”

and introduction:

“Multiconformer models are notably easier to modify and more interpretable in software like Coot12 unlike ensemble methods that generate multiple complete protein copies(Burnley et al. 2012; Ploscariu et al. 2021; Temple Burling and Brünger 1994).”

and results:

“This model can then be examined and edited in Coot12 or other visualization software, and further refined using software such as phenix.refine, refmac, or buster as the modeler sees fit.”

and discussion

“qFit is compatible with manual modification and further refinement as long as the subsequent software uses the PDB standard altloc column, as is common in most popular modeling and refinement programs. The models can therefore generally also be deposited in the PDB using the standard deposition and validation process.”

Appraisal & Discussion

Overall, the authors convincingly demonstrate that qFit provides a reliable means to detect and model conformational heterogeneity within high-resolution X-ray diffraction datasets and (based on a smaller sample) in cryo-EM density maps. This represents the state of the art in the field and will be of interest to any structural biologist or biochemist seeking to attain an understanding of the structural basis of the function of their system of interest, including potential allosteric mechanisms-an area where there are still few good solutions. That is, I expect qFit to find widespread use.

Reviewer #3 (Public Review):

Summary:

The authors address a very important issue of going beyond a single-copy model obtained by the two principal experimental methods of structural biology, macromolecular crystallography and cryo electron microscopy (cryo-EM). Such multiconformer model is based on the fact that experimental data from both these methods represent a space- and time-average of a huge number of the molecules in a sample, or even in several samples, and that the respective distributions can be multimodal. Different from structure prediction methods, this approach is strongly based on high-resolution experimental information and requires validated single-copy high-quality models as input. Overall, the results support the authors' conclusions.

In fact, the method addresses two problems which could be considered separately:

- An automation of construction of multiple conformations when they can be identified visually;

- A determination of multiple conformations when their visual identification is difficult or impossible.

We often think about this problem similarly to the reviewer. However, in building qFit, we do not want to separate these problems - but rather use the first category (obvious visual identification) to build an approach that can accomplish part of the second category (difficult to visualize) without building “impossible”/nonexistent conformations - with a consistent approach/bias.

The first one is a known problem, when missing alternative conformations may cost a few percent in R-factors. While these conformations are relatively easy to detect and build manually, the current procedure may save significant time being quite efficient, as the test results show.

We agree with the reviewers' assessment here. The “floor” in terms of impact is automating a tedious part of high resolution model building and improving model quality.

The second problem is important from the physical point of view and has been addressed first by Burling & Brunger (1994; https://doi.org/10.1002/ijch.199400022). The new procedure deals with a second-order variation in the R-factors, of about 1% or less, like placing riding hydrogen atoms, modeling density deformation or variation of the bulk solvent. In such situations, it is hard to justify model improvement. Keeping Rfree values or their marginal decreasing can be considered as a sign that the model is not overfitted data but hardly as a strong argument in favor of the model.

We agree with the overall sentiment of this comment. What is a significant variation in R-free is an important question that we have looked at previously (http://dx.doi.org/10.1101/448795) and others have suggested an R-sleep for further cross validation (https://pubmed.ncbi.nlm.nih.gov/17704561/). For these reasons it is important to get at the significance of the changes to model types from large and diverse test sets, as we have here and in other works, and from careful examination of the biological significance of alternative conformations with experiments designed to test their importance in mechanism.

In general, overall targets are less appropriate for this kind of problem and local characteristics may be better indicators. Improvement of the model geometry is a good choice. Indeed, yet Cruickshank (1956; https://doi.org/10.1107/S0365110X56002059) showed that averaged density images may lead to a shortening of covalent bonds when interpreting such maps by a single model. However, a total absence of geometric outliers is not necessarily required for the structures solved at a high resolution where diffraction data should have more freedom to place the atoms where the experiments "see" them.

Again, we agree—geometric outliers should not be completely absent, but it is comforting when they and model/experiment agreement both improve.

The key local characteristic for multi conformer models is a closeness of the model map to the experimental one. Actually, the procedure uses a kind of such measure, the Bayesian information criteria (BIC). Unfortunately, there is no information about how sharply it identifies the best model, how much it changes between the initial and final models; in overall there is not any feeling about its values. The Q-score (page 17) can be a tool for the first problem where the multiple conformations are clearly separated and not for the second problem where the contributions from neighboring conformations are merged. In addition to BIC or to even more conventional target functions such as LS or local map correlation, the extreme and mean values of the local difference maps may help to validate the models.

We agree with the reviewer that the problem of “best” model determination is poorly posed here. We have been thinking a lot about htis in the context of Bayesian methods (see: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9278553/); however, a major stumbling block is in how variable representations of alternative conformations (and compositions) are handled. The answers are more (but by no means simply) straightforward for ensemble representations where the entire system is constantly represented but with multiple copies.

This method with its results is a strong argument for a need in experimental data and information they contain, differently from a pure structure prediction. At the same time, absence of strong density-based proofs may limit its impact.

We agree - indeed we think it will be difficult to further improve structure prediction methods without much more interaction with the experimental data.

Strengths:

Addressing an important problem and automatization of model construction for alternative conformations using high-resolution experimental data.

Weaknesses:

An insufficient validation of the models when no discrete alternative conformations are visible and essentially missing local real-space validation indicators.

While not perfect real space indicators, local real-space validation is implicit in the MIQP selection step and explicit when we do employ Q-score metrics.

Recommendations for the authors:

Reviewer #1 (Recommendations For The Authors):

A point of clarification: I don't understand why waters seem to be handled differently in for cryo-EM and crystallography datasets. I am interested about the statement on page 19 that the Molprobity Clashscore gets worse for cryo-EM datasets, primarily due to clashes with waters. But the qFit algorithm includes a round of refinement to optimize placement of ordered waters, and the clashscore improves for the qFit refinement in crystallography test cases. Why/how is this different for cryo-EM?

We agree that this was not an appropriate point. We believe that the high clash score is coming from side chains being incorrectly modeled. We have updated this in the manuscript and it will be a focus of future improvements.

Reviewer #2 (Recommendations For The Authors):

- It would be instructive to the reader to explain how qFit handles the chromophore in the PYP (1OTA) example. To this end, it would be helpful to include deposition of the multiconformer model of PYP. This might also be a suitable occasion for discussion of potential hurdles in the deposition of multiconformer models in the PDB (if any!). Such concerns may be real concerns causing hesitation among potential users.

Thank you for this comment. qFit does not alter the position or connectivity of any HETATM records (like the chromophore in this structure). Handling covalent modifications like this is an area of future development.

Regarding deposition, we have noted above that the discussion now includes:

“qFit is compatible with manual modification and further refinement as long as the subsequent software uses the PDB standard altloc column, as is common in most popular modeling and refinement programs. The models can therefore, generally also be deposited in the PDB using the standard deposition and validation process.”

Finally, we have placed all PDBs in a Zenodo deposition (XXX) and have included that language in the manuscript. It is currently under a separate data availability section (page XXX). We will defer to the editor as to the best header that should go under.

- It may be advisable to take the description of true/false pos/negatives out of the caption of Figure 4, and include it in a box or so, since these terms are important in the main text too, and the caption becomes very cluttered.

We think adding the description of true/false pos/negatives to the Figure panel would make it very cluttered and wordy. We would like to retain this description within the caption. We have also briefly described each in the main text.

- page 21, line 4: some issue with citation formatting.

We have updated these citations.

- page 25, second paragraph: cardinality is the number of members of a set. Perhaps "minimal occupancy" is more appropriate.

Thank you for pointing this out. This was a mistake and should have been called the occupancy threshold.

- page 26: it's - its

Thank you, we have made this change. 

- Font sizes in Supplementary Figures 5-7 are too small to be readable.

We agree and will make this change. 

Reviewer #3 (Recommendations For The Authors):

General remarks

(1) As I understand, the procedure starts from shifting residues one by one (page 4; A.1). Then, geometry reconstruction (e.g., B1) may be difficult in some cases joining back the shifted residues. It seems that such backbone perturbation can be done more efficiently by shifting groups of residues ("potential coupled motions") as mentioned at the bottom of page 9. Did I miss its description?

We would describe the algorithm as sampling (which includes minimal shifts) in the backbone residues to ensure we can link neighboring residues. We agree that future iterations of qFit should include more effective backbone sampling by exploring motion along the Cβ-Cα, C-N, and (Cβ-Cα × C-N) bonds and exploring correlated backbone movements.

(2) While the paper is well split in clear parts, some of them seem to be not at their right/optimal place and better can be moved to "Methods" (detailed "Overview of the qFit protein algorithm" as a whole) or to "Data" missed now (Two first paragraphs of "qFit improves overall fit...", page 8, and "Generating the qFit test set", page 22, and "Generating synthetic data ..." at page 26; description of the test data set), At my personal taste, description of tests with simulated data (page 15) would be better before that of tests with real data.

Thank you for this comment, but we stand by our original decision to keep the general flow of the paper as it was submitted.

(3) I wonder if the term "quadratic programming" (e.g., A3, page 5) is appropriate. It supposes optimization of a quadratic function of the independent parameters and not of "some" parameters. This is like the crystallographic LS which is not a quadratic function of atomic coordinates, and I think this is a similar case here. Whatever the answer on this remark is, an example of the function and its parameters is certainly missed.

We think that the term quadratic programming is appropriate. We fit a function with a loss function (observed density - calculated density), while satisfying the independent parameters. We fit the coefficients minimizing a quadratic loss. We agree that the quadratic function is missing from the paper, and we have now included it in the Methods section.

Technical remarks to be answered by the authors :

(1) Page 1, Abstract, line 3. The ensemble modeling is not the only existing frontier, and saying "one of the frontiers" may be better. Also, this phrase gives a confusing impression that the authors aim to predict the ensemble models while they do it with experimental data.

We agree with this statement and have re-worded the abstract to reflect this.

(2) Page 2. Burling & Brunger (1994) should be cited as predecessors. On the contrary, an excellent paper by Pearce & Gros (2021) is not relevant here.

While we agree that we should mention the Burling & Brunger paper and the Pearce & Gros (2021) should not be removed as it is not discussing the method of ensemble refinement.

(3) Page 2, bottom. "Further, when compared to ..." The preference to such approach sounds too much affirmative.

We have amended this sentence to state:

“Multiconformer models are notably easier to modify and more interpretable in software like Coot(Emsley et al. 2010) unlike ensemble methods that generate multiple complete protein copies(Burnley et al. 2012; Ploscariu et al. 2021; Temple Burling and Brünger 1994).”

“The point we were trying to make in this sentence was that ensemble-based models are much harder to manually manipulate in Coot or other similar software compared to multiconformer models. We think that the new version of this sentence states this point more clearly.”

(4) Page 2, last paragraph. I do not see an obvious relation of references 15-17 to the phrase they are associated with.

We disagree with this statement, and think that these references are appropriate.

“Multiconformer models are notably easier to modify and more interpretable in software like Coot12 unlike ensemble methods that generate multiple complete protein copies(Burnley et al. 2012; Ploscariu et al. 2021; Temple Burling and Brünger 1994).”

(5) Page 3, paragraph 2. Cryo-EM maps should be also "high-resolution"; it does not read like this from the phrase.

We agree that high-resolution should be added, and the sentence now states:

“However, many factors make manually creating multiconformer models difficult and time-consuming. Interpreting weak density is complicated by noise arising from many sources, including crystal imperfections, radiation damage, and poor modeling in X-ray crystallography, and errors in particle alignment and classification, poor modeling of beam induced motion, and imperfect detector Detector Quantum Efficiency (DQE) in high-resolution cryo-EM.”

(6) Page 3, last paragraph before "results". The words "... in both individual cases and large structural bioinformatic projects" do not have much meaning, except introducing a self-reference. Also, repeating "better than 2 A" looks not necessary.

We agree that this was unnecessary and have simplified the last sentence to state:

“With the improvements in model quality outlined here, qFit can now be increasingly used for finalizing high-resolution models to derive ensemble-function insights.”

(7) Page 3. "Results". Could "experimental" be replaced by a synonym, like "trial", to avoid confusing with the meaning "using experimental data"?

We have replaced experimental with exploratory to describe the use of qFit on CryoEM data. The statement now reads:

“For cryo-EM modeling applications, equivalent metrics of map and model quality are still developing, rendering the use of qFit for cryo-EM more exploratory.”

(8) Page 4, A.1. Should it be "steps +/- 0.1" and "coordinate" be "coordinate axis"? One can modify coordinates and not shift them. I do not understand how, with the given steps, the authors calculated the number of combinations ("from 9 to 81"). Could a long "Alternatively, ...absent" be reduced simply to "Otherwise"?

We have simplified and clarified the sentence on the sampling of backbone coordinates to state:

“If anisotropic B-factors are absent, the translation of coordinates occurs in the X, Y, and Z directions. Each translation takes place in steps of 0.1 along each coordinate axis, extending to 0.3 Å, resulting in 9 (if isotropic) or to 81 (if anisotropic) distinct backbone conformations for further analysis.”

(9) Page 6, B.1, line 2. Word "linearly" is meaningless here.

We have modified this to read:

“Moving from N- to C- terminus along the protein,”

(10) Page 9, line 2. It should be explained which data set is considered as the test set to calculate Rfree.

We think this is clear and would be repetitive if we duplicated it.

(11) Page 9, line 7. It should be "a valuable metric" and not "an"

We agree and have updated the sentence to read:

“Rfree is a valuable metric for monitoring overfitting, which is an important concern when increasing model parameters as is done in multiconformer modeling.”

(12) Page 10, paragraph 3. "... as a string (Methods)". I did not find any other mention of this term "string", including in "Methods" where it supposed to be explained. Either this should be explained (and an example is given?), or be avoided.

We agree that string is not necessary (discussing the programmatic datatype). We have removed this from the sentence. It now reads:

“To quantify how often qFit models new rotameric states, we analyzed the qFit models with phenix.rotalyze, which outputs the rotamer state for each conformer (Methods).”

(13) Page10, lines 3-4 from bottom. Are these two alternative conformations justified?

We are unsure what this is referring to.

(14) Page 12, Fig. 2A. In comparison with Supplement Fig 2C, the direction of axes is changed. Could they be similar in both Figures?

We have updated Supplementary Figure 2C to have the same direction of axes as Figure 2A.

(15) Page 15, section's title. Choose a single verb in "demonstrate indicate".

We have amended the title of this section to be:

“Simulated data demonstrate qFit is appropriate for high-resolution data.”

(16) Page 15, paragraph 2. "Structure factors from 0.8 to 3.0 A resolution" does not mean what the author wanted apparently to tell: "(complete?) data sets with the high-resolution limit which varied from 0.8 to 3.0 A ...". Also, a phrase of "random noise increasing" is not illustrated by Figs.5 as it is referred to.

We have edited this sentence to now read:

“To create the dataset for resolution dependence, we used the ground truth 7KR0 model, including all alternative conformations, and generated artificial structure factors with a high resolution limit ranging from  0.8 to 3.0 Å resolution (in increments of 0.1 Å).”

(17) Page 15, last paragraph is written in a rather formal and confusing way while a clearer description is given in the figure legend and repeated once more in Methods. I would suggest to remove this paragraph.

We agree that this is confusing. Instead of create a true positive/false positive/true negative/false negative matrix, we have just called things as they are, multiconformer or single conformer and match or no match. We have edited the language the in the manuscript and figure legends to reflect these changes.

(18) Page 16. Last two paragraphs start talking about a new story and it would help to separate them somehow from the previous ones (sub-title?).

We agree that this could use a subtitle. We have included the following subtitle above this section:

“Simulated multiconformer data illustrate the convergence of qFit.”

(19) Page 20. "or static" and "we determined that" seem to be not necessary.

We have removed static and only used single conformer models. However, as one of the main conclusions of this paper is determining that qFit can pick up on alternative conformers that were modeled manually, we have decided to the keep the “we determined that”.

(20) Page 21, first paragraph. "Data" are plural; it should be "show" and "require"

We have made these edits. The sentence now reads:

“However, our data here shows that not only does qFit need a high-resolution map to be able to detect signal from noise, it also requires a very well-modeled structure as input.”

(21) Page 21, References should be indicated as [41-45], [35,46-48], [55-57]. A similar remark to [58-63] at page 22.

We have fixed the reference layout to reflect this change.

(22) Page 21, last paragraph. "Further reduce R-factors" (moreover repeated twice) is not correct neither by "further", since here it is rather marginal, nor as a goal; the variations of R-factors are not much significant. A more general statement like "improving fit to experimental data" (keeping in mind density maps) may be safer.

We agree with the duplicative nature of these statements. We have amended the sentence to now read:

“Automated detection and refinement of partial-occupancy waters should help improve fit to experimental data further reduce Rfree15 and provide additional insights into hydrogen-bond patterns and the influence of solvent on alternative conformations.”

(23) Page 22. Sub-sections of "Methods" are given in a little bit random order; "Parallelization of large maps" in the middle of the text is an example. Put them in a better order may help.

We have moved some section of the Methods around and made better headings by using an underscore to highlight the subsections (Generating and running the qFit test set, qFit improved features, Analysis metrics, Generating synthetic data for resolution dependence).

(24) Page 24. Non-convex solution is a strange term. There exist non-convex problems and functions and not solutions.

We agree and we have changed the language to reflect that we present the algorithm with non-convex problems which it cannot solve.

(25) Page 26, "Metrics". It is worthy to describe explicitly the metrics and not (only) the references to the scripts.

For all metrics, we describe a sentence or two on what each metric describes. As these metrics are well known in the structural biology field, we do not feel that we need to elaborate on them more.

(26) Page 26. Multiplying B by occupancy does not have much sense. A better option would be to refer to the density value in the atomic center as occ*(4*pi/B)^1.5 which gives a relation between these two entities.

We agree and have update the B-factor figures and metrics to reflect this.

(27) Page 40, suppl. Fig. 5. Due to the color choice, it is difficult to distinguish the green and blue curves in the diagram.

We have amended this with the colors of the curves have been switched.

(28) Page 42, Suppl. Fig. 7. (A) How the width of shaded regions is defined? (B) What the blue regions stand for? Input Rfree range goes up to 0.26 and not to 0.25; there is a point at the right bound. (C) Bounds for the "orange" occupancy are inversed in the legend.

(A) The width of the shaded region denotes the standard deviations among the values at every resolution. We have made this clearer in the caption

(B) The blue region denotes the confidence interval for the regression estimate. Size of the confidence interval was set to 95%. We have made this clearer in the caption

(C) This has been fixed now

The maximum R-free value is 0.2543, which we rounded down to 0.25.

(29) Page 43. Letters E-H in the legend are erroneously substituted by B-E.

We apologize for this mistake. It is now corrected.

翻訳的には合ってると思うのですが、最初「出力が組み込まれた」で切ってしまい、読み直しました。

代案 他の多くのエディターと同様、VSCodeには組み込みのターミナルウィンドウがあり、出力はそこに表示されます。

What was the answer to this was it

kubectl get pod kvstore -o wide to find the name of the node that it's on - mine was on k8s-worker-1

I then went into ssh settings in visual studio code and added a host so my ssh config file is now

Host worker0 HostName 18.171.145.65 User student IdentityFile c:\users\karen\downloads\qwikLABS-L138956-206416.pem Host worker1 Hostname 35.178.200.149 User student IdentityFile c:\users\karen\downloads\qwikLABS-L138956-206416.pem Host controller Hostname 13.40.152.189 User student IdentityFile c:\users\karen\downloads\qwikLABS-L138956-206416.pem

and I opened them up in 3 separate vs code windows

kubectl get pod <pod-name> -o jsonpath='{.metadata.uid}'

kubectl get pod kvname -o jsonpath='{.metadata.uid}'

and then

on k8s-worker-1 in the terminal windows I used the syntax and replaced my id I had retrieved from above command in the poduid:

/var/lib/kubelet/pods/<podUID>/volumes/kubernetes.io~empty-dir/

sudo ls //var/lib/kubelet/pods/37abdd08-c0f7-4549-a9cc-20df89ed7fa8/volum es/kubernetes.io~empty-dir/

you have to run it with sudo permissions otherwise you get denied access, but then you can see data-volume

I then did sudo -i

cd /var/lib/kubelet/pods/37abdd08-c0f7-4549-a9cc-20df89ed7fa8/volumes/kubernetes.io~empty-dir/

ls (to see directory listing it shoowed me data-volume)

cd data-volume

ls

it then showed me age and name which were the two values I had put in there

A fully compiled language on the other hand (e.g. C), is run directly on the machine's CPU after the source code is compiled into machine code.

This interpreter is a software program that reads the code, analyzes it, and performs the actions specified in the code at runtime. An interpreter translates source code into machine code on the fly, executing it line by line or statement by statement.

The JVM is essentially an interpreter for the intermediate-level byte code generated after compiling java code.

models constraints modularity

inherent conflict

that's why things will never work

models errode modularity

models are trully intertwingled modules encapsulate

separating models with cross cutting concerns and providing modules and layers that constrain limit cross cuting

aspects orientation won't help either

we need models and capabnlities needed to effect intents need to coevolve coevolve the three together intent model component

That's why MVC is just a broken idea Period

Model View Controller

That's why Dijkstra's goto Considered Harmfull was the most harmful idea that ever influence thinking. tha nature of everything is interwinglularity spagetty code. Named Jumps. Yes imposing structure can seemingly create order but it bound to be a self limitting one. Need the ability to organic growth

Author response:

The following is the authors’ response to the original reviews.

eLife assessment

The study makes a valuable empirical contribution to our understanding of visual processing in primates and deep neural networks, with a specific focus on the concept of factorization. The analyses provide solid evidence that high factorization scores are correlated with neural predictivity, yet more evidence would be needed to show that neural responses show factorization. Consequently, while several aspects require further clarification, in its current form this work is interesting to systems neuroscientists studying vision and could inspire further research that ultimately may lead to better models of or a better understanding of the brain.

Public Reviews:

Reviewer #1 (Public Review):

Summary:

The paper investigates visual processing in primates and deep neural networks (DNNs), focusing on factorization in the encoding of scene parameters. It challenges the conventional view that object classification is the primary function of the ventral visual stream, suggesting instead that the visual system employs a nuanced strategy involving both factorization and invariance. The study also presents empirical findings suggesting a correlation between high factorization scores and good neural predictivity.

Strengths:

(1) Novel Perspective: The paper introduces a fresh viewpoint on visual processing by emphasizing the factorization of non-class information.

(2) Methodology: The use of diverse datasets from primates and humans, alongside various computational models, strengthens the validity of the findings.

(3) Detailed Analysis: The paper suggests metrics for factorization and invariance, contributing to a future understanding & measurements of these concepts.

Weaknesses:

(1) Vagueness (Perceptual or Neural Invariance?): The paper uses the term 'invariance', typically referring to perceptual stability despite stimulus variability [1], as the complete discarding of nuisance information in neural activity. This oversimplification overlooks the nuanced distinction between perceptual invariance (e.g., invariant object recognition) and neural invariance (e.g., no change in neural activity). It seems that by 'invariance' the authors mean 'neural' invariance (rather than 'perceptual' invariance) in this paper, which is vague. The paper could benefit from changing what is called 'invariance' in the paper to 'neural invariance' and distinguish it from 'perceptual invariance,' to avoid potential confusion for future readers. The assignment of 'compact' representation to 'invariance' in Figure 1A is misleading (although it can be addressed by the clarification on the term invariance). [1] DiCarlo JJ, Cox DD. Untangling invariant object recognition. Trends in cognitive sciences. 2007 Aug 1;11(8):333-41.

Thanks for pointing out this ambiguity. In our Introduction we now explicitly clarify that we use “invariance” to refer to neural, rather than perceptual invariance, and we point out that both factorization and (neural) invariance may be useful for obtaining behavioral/perceptual invariance.

(2) Details on Metrics: The paper's explanation of factorization as encoding variance independently or uncorrelatedly needs more justification and elaboration. The definition of 'factorization' in Figure 1B seems to be potentially misleading, as the metric for factorization in the paper seems to be defined regardless of class information (can be defined within a single class). Does the factorization metric as defined in the paper (orthogonality of different sources of variation) warrant that responses for different object classes are aligned/parallel like in 1B (middle)? More clarification around this point could make the paper much richer and more interesting.

Our factorization metric measures the degree to which two sets of scene variables are factorized from one another. In the example of Fig. 1B, we apply this definition to the case of factorization of class vs. non-class information. Elsewhere in the paper we measure factorization of several other quantities unrelated to class, specifically camera viewpoint, lighting conditions, background content, and object pose. In our revised manuscript we have clarified the exposition surrounding Fig. 1B to make it clear that factorization, as we define it, can be applied to other quantities as well and that responses do not need to be aligned/parallel but simply live in a different set of dimensions whether linearly or nonlinearly arranged. Thanks for raising the need to clarify this point.

(3) Factorization vs. Invariance: Is it fair to present invariance vs. factorization as mutually exclusive options in representational hypothesis space? Perhaps a more fair comparison would be factorization vs. object recognition, as it is possible to have different levels of neural variability (or neural invariance) underlying both factorization and object recognition tasks.

We do not mean to imply that factorization and invariance are mutually exclusive, or that they fully characterize the space of possible representations. However, they are qualitatively distinct strategies for achieving behavioral capabilities like object recognition. In the revised manuscript we also include a comparison to object classification performance (Figures 5C & S4, black x’s) as a predictor of brain-like representations, alongside the results for factorization and invariance.

In our revised Introduction and beginning of the Results section, we make it more clear that factorization and invariance are not mutually exclusive – indeed, our results show that both factorization and invariance for some scene variables like lighting and background identity are signatures of brain-like representations. Our study focuses on factorization because we believe its importance has not been studied or highlighted to the degree that invariance to “nuisance” parameters has in concert with selectivity to object identity in individual neuron tuning functions. Moreover, the loss functions used for supervised training functions of neural networks for image classification would seem to encourage invariance as a representational strategy. Thus, the finding that factorization of scene parameters is an equally good if not better predictor of brain-like representations may motivate new objective functions for neural network training.

(4) Potential Confounding Factors in Empirical Findings: The correlation observed in Figure 3 between factorization and neural predictivity might be influenced by data dimensionality, rather than factorization per se [2]. Incorporating discussions around this recent finding could strengthen the paper.

[2] Elmoznino E, Bonner MF. High-performing neural network models of the visual cortex benefit from high latent dimensionality. bioRxiv. 2022 Jul 13:2022-07.

We thank the Reviewer for pointing out this important, potential confound and the need for a direct quantification. We have now included an analysis computing how well dimensionality (measured using the participation ratio metric for natural images, as was done in [2] Elmoznino& Bonner bioRxiv. 2022) can account for model goodness-of-fit (additional pink bars in Figure 6). Factorization of scene parameters appears to add more predictive power than dimensionality on average (Figure 6, light shaded bars), and critically, factorization+classification jointly predict goodness-of-fit significantly better than dimensionality+classification for V4 and IT/HVC brain areas (Figure 6, dark shaded bars). Indeed, dimensionality+classification is only slightly more predictive than classification alone for V4 and IT/HVC indicating some redundancy in those measures with respect to neural predictivity of models (Figure 6, compare dark shaded pink bar to dashed line).

That said, high-dimensional representations can, in principle, better support factorization, and thus we do not regard these two representational strategies necessarily in competition. Rather, our results suggest (consistent with [2]) that dimensionality is predictive of brain-like representation to some degree, such that some (but not all) of factorization’s predictive power may indeed owe to a partial correlation with dimensionality. We elaborate in the Discussion where this point comes up and now refer to the updated Figure 6 that shows the control for dimensionality.

Conclusion:

The paper offers insightful empirical research with useful implications for understanding visual processing in primates and DNNs. The paper would benefit from a more nuanced discussion of perceptual and neural invariance, as well as a deeper discussion of the coexistence of factorization, recognition, and invariance in neural representation geometry. Additionally, addressing the potential confounding factors in the empirical findings on the correlation between factorization and neural predictivity would strengthen the paper's conclusions.

Taken together, we hope that the changes described above address the distinction between neural and perceptual invariance, provide a more balanced understanding of the contributions of factorization, invariance, and local representational geometry, and rule against dimensionality for natural images as contributing to the main finding of the benefits from factorization of scene parameters.

Reviewer #2 (Public Review):

Summary:

The dominant paradigm in the past decade for modeling the ventral visual stream's response to images has been to train deep neural networks on object classification tasks and regress neural responses from units of these networks. While object classification performance is correlated to the variance explained in the neural data, this approach has recently hit a plateau of variance explained, beyond which increases in classification performance do not yield improvements in neural predictivity. This suggests that classification performance may not be a sufficient objective for building better models of the ventral stream. Lindsey & Issa study the role of factorization in predicting neural responses to images, where factorization is the degree to which variables such as object pose and lighting are represented independently in orthogonal subspaces. They propose factorization as a candidate objective for breaking through the plateau suffered by models trained only on object classification.

They claim that (i) maintaining these non-class variables in a factorized manner yields better neural predictivity than ignoring non-class information entirely, and (ii) factorization may be a representational strategy used by the brain.

The first of these claims is supported by their data. The second claim does not seem well-supported, and the usefulness of their observations is not entirely clear.

Strengths:

This paper challenges the dominant approach to modeling neural responses in the ventral stream, which itself is valuable for diversifying the space of ideas.

This paper uses a wide variety of datasets, spanning multiple brain areas and species. The results are consistent across the datasets, which is a great sign of robustness.

The paper uses a large set of models from many prior works. This is impressively thorough and rigorous.

The authors are very transparent, particularly in the supplementary material, showing results on all datasets. This is excellent practice.

Weaknesses:

(1) The primary weakness of this paper is a lack of clarity about what exactly is the contribution. I see two main interpretations: (1-A) As introducing a heuristic for predicting neural responses that improve over-classification accuracy, and (1-B) as a model of the brain's representational strategy. These two interpretations are distinct goals, each of which is valuable. However, I don't think the paper in its current form supports either of them very well:

(1-A) Heuristic for neural predictivity. The claim here is that by optimizing for factorization, we could improve models' neural predictivity to break through the current predictivity plateau. To frame the paper in this way, the key contribution should be a new heuristic that correlates with neural predictivity better than classification accuracy. The paper currently does not do this. The main piece of evidence that factorization may yield a more useful heuristic than classification accuracy alone comes from Figure 5. However, in Figure 5 it seems that factorization along some factors is more useful than others, and different linear combinations of factorization and classification may be best for different data. There is no single heuristic presented and defended. If the authors want to frame this paper as a new heuristic for neural predictivity, I recommend the authors present and defend a specific heuristic that others can use, e.g. [K * factorization_of_pose + classification] for some constant K, and show that (i) this correlates with neural predictivity better than classification alone, and (ii) this can be used to build models with higher neural predictivity. For (ii), they could fine-tune a state-of-the-art model to improve this heuristic and show that doing so achieves a new state-of-the-art neural predictivity. That would be convincing evidence that their contribution is useful.

Our paper does not make any strong claim regarding the Reviewer’s point 1-A (on heuristics for neural predictivity). In the Discussion, last paragraph, we better specify that our work is merely suggestive of claim 1-A about heuristics for more neurally predictive, more brainlike models. We believe that our paper supports the Reviewer’s point 1-B (on brain representation) as we discuss below.

We leave it to future work to determine if factorization could help optimize models to be more brainlike. This treatment may require exploration of novel model architectures and loss functions, and potentially also more thorough neural datasets that systematically vary many different forms of visual information for validating any new models.

(1-B) Model of representation in the brain. The claim here is that factorization is a general principle of representation in the brain. However, neural predictivity is not a suitable metric for this, because (i) neural predictivity allows arbitrary linear decoders, hence is invariant to the orthogonality requirement of factorization, and (ii) neural predictivity does not match the network representation to the brain representation. A better metric is representational dissimilarity matrices. However, the RDM results in Figure S4 actually seem to show that factorization does not do a very good job of predicting neural similarity (though the comparison to classification accuracy is not shown), which suggests that factorization may not be a general principle of the brain. If the authors want to frame the paper in terms of discovering a general principle of the brain, I suggest they use a metric (or suite of metrics) of brain similarity that is sensitive to the desiderata of factorization, e.g. doesn't apply arbitrary linear transformations, and compare to classification accuracy in addition to invariance.

We agree with the Reviewer about the shortcomings of neural predictivity for comparing representational geometries, and in our revised manuscript we have provided a more comprehensive set of results that includes RDM predictivity in new Figures 6 & 7, alongside the results for neural fit predictivity. In addition, as suggested we added classification accuracy predictivity in Figures 5C & S4 (black x’s) for visual comparison to factorization/invariance. In Figure S4 on RDMs, it is apparent how factorization is at least as good a predictor as classification on all V4 & IT datasets from both monkeys and humans (compared x’s to filled circles in Figure S4; note that some of the points from the original Figure S4 changed as we discovered a bug in the code that specifically affected the RDM analysis for a few of the datasets).

We find that the newly included RDM analyses in Figures 6 & 7 are consistent with the conclusions of the neural fit regression analyses: that the correlation of factorization metrics with RDM matches are strong, comparable in magnitude to that of classification accuracy (Figure 6, 3rd & 4th columns, compare black dashed line to faded colored bars) and are not fully accounted for by the model’s classification accuracy alone (Figure 6, 3rd & 4th columns, higher unfaded bars for classification combined with factorization, and see corresponding example scatters in Figure 7 middle/bottom rows).

It is encouraging that the added benefit of factorization for RDM predictivity accounting for classification performance is at least as good as the improvement seen for neural fit predictivity (Figure 6, 1st & 2nd columns for encoding fits versus 3rd & 4th columns for RDM correlations).

(2) I think the comparison to invariance, which is pervasive throughout the paper, is not very informative. First, it is not surprising that invariance is more weakly correlated with neural predictivity than factorization, because invariant representations lose information compared to factorized representations. Second, there has long been extensive evidence that responses throughout the ventral stream are not invariant to the factors the authors consider, so we already knew that invariance is not a good characterization of ventral stream data.

While we appreciate the Reviewer’s intuition that highly invariant representations are not strongly supported in the high-level visual cortex, we nevertheless thought it was valuable to put this intuition to a quantitative, detailed test. As a result, we uncovered effects that were not obvious a priori, at least to us – for example, that invariance for some scene parameters (camera view, object pose) is negatively correlated with neural predictions while invariance to others (background, lighting) is positively correlated. Thus, our work exercises the details of invariance for different types of information.

(3) The formalization of the factorization metric is not particularly elegant, because it relies on computing top K principal components for the other-parameter space, where K is arbitrarily chosen as 10. While the authors do show that in their datasets the results are not very sensitive to K (Figure S5), that is not guaranteed to be the case in general. I suggest the authors try to come up with a formalization that doesn't have arbitrary constants. For example, one possibility that comes to mind is E[delta_a x delta_b], where 'x' is the normalized cross product, delta_a, and delta_b are deltas in representation space induced by perturbations of factors a and b, and the expectation is taken over all base points and deltas. This is just the first thing that comes to mind, and I'm sure the authors can come up with something better. The literature on disentangling metrics in machine learning may be useful for ideas on measuring factorization.

Thanks to the Reviewer for raising this point. First, we wish to clarify a potential misunderstanding of the factorization metric: the number K of principal components we choose is not an arbitrary constant, but rather calibrated to capture a certain fraction of variance, set to 90% by default in our analyses. While this variance threshold is indeed an arbitrary hyperparameter, it has a more intuitive interpretation than the number of principal components.

Nonetheless, the Reviewer’s comment did inspire us to consider another metric for factorization that does not depend on any arbitrary parameters. In the revised version, we now include a covariance matrix based metric which simply measures the elementwise correlation of the covariance matrices induced by varying the scene parameter of interest and the covariance matrix induced by varying the other parameters (and then subtracts this quantity from 1).

Correspondingly, we now present results for both the new covariance based measure and the original PCA based one in Figures 5C, 6, and 7. The main findings remain largely the same when using the covariance based metric, and the covariance based metric (Figure 5C, compare light shaded to dark shaded filled circles; Figure 6, compare top row to bottom row; Figure 7, compare middle rows to bottom rows).

Ultimately, we believe these two metrics are complementary and somewhat analogous to two metrics commonly used for measuring dimensionality (the number of components needed to explain a certain fraction of the variance, analogous to our original PCA based definition; the participation ratio, analogous to our covariance based definition). We have added the formula for the covariance based factorization metric along with a brief description to the Methods.

(4) The authors defined the term "factorization" according to their metric. I think introducing this new term is not necessary and can be confusing because the term "factorization" is vague and used by different researchers in different ways. Perhaps a better term is "orthogonality", because that is clear and seems to be what the authors' metric is measuring.

We agree with the Reviewer that factorization has become an overloaded term. At the same time, we think that in this context, the connotation of the term factorization effectively conveys the notion of separating out different latent sources of variance (factors) such that they can be encoded in orthogonal subspaces.

To aid clarity, we now mention in the Introduction that factorization defined here is meant to measure orthogonalization of scene factors. Additionally, in the Discussion section, we now go into more detail comparing our metric to others previously used in the literature, including orthogonality, to help put it in context.

(5) One general weakness of the factorization paradigm is the reliance on a choice of factors. This is a subjective choice and becomes an issue as you scale to more complex images where the choice of factors is not obvious. While this choice of factors cannot be avoided, I suggest the authors add two things: First, an analysis of how sensitive the results are to the choice of factors (e.g. transform the basis set of factors and re-run the metric); second, include some discussion about how factors may be chosen in general (e.g. based on temporal statistics of the world, independent components analysis, or something else).

The Reviewer raises a very reasonable point about the limitation of this work. While we limited our analysis to generative scene factors that we know about and that could be manipulated, there are many potential factors to consider. It is not clear to us exactly how to implement the Reviewer’s suggestion of transforming the basis set of factors, as the factors we consider are highly nonlinear in the input space. Ultimately, we believe that finding unsupervised methods to characterize the “true” set of factors that is most useful for understanding visual representations is an important subject for future work, but outside the scope of this particular study. We have added a comment to this effect in the Discussion.

Reviewer #3 (Public Review):

Summary:

Object classification serves as a vital normative principle in both the study of the primate ventral visual stream and deep learning. Different models exhibit varying classification performances and organize information differently. Consequently, a thriving research area in computational neuroscience involves identifying meaningful properties of neural representations that act as bridges connecting performance and neural implementation. In the work of Lindsey and Issa, the concept of factorization is explored, which has strong connections with emerging concepts like disentanglement [1,2,3] and abstraction [4,5]. Their primary contributions encompass two facets: (1) The proposition of a straightforward method for quantifying the degree of factorization in visual representations. (2) A comprehensive examination of this quantification through correlation analysis across deep learning models.

To elaborate, their methodology, inspired by prior studies [6], employs visual inputs featuring a foreground object superimposed onto natural backgrounds. Four types of scene variables, such as object pose, are manipulated to induce variations. To assess the level of factorization within a model, they systematically alter one of the scene variables of interest and estimate the proportion of encoding variances attributable to the parameter under consideration.

The central assertion of this research is that factorization represents a normative principle governing biological visual representation. The authors substantiate this claim by demonstrating an increase in factorization from macaque V4 to IT, supported by evidence from correlated analyses revealing a positive correlation between factorization and decoding performance. Furthermore, they advocate for the inclusion of factorization as part of the objective function for training artificial neural networks. To validate this proposal, the authors systematically conduct correlation analyses across a wide spectrum of deep neural networks and datasets sourced from human and monkey subjects. Specifically, their findings indicate that the degree of factorization in a deep model positively correlates with its predictability concerning neural data (i.e., goodness of fit).

Strengths:

The primary strength of this paper is the authors' efforts in systematically conducting analysis across different organisms and recording methods. Also, the definition of factorization is simple and intuitive to understand.

Weaknesses:

This work exhibits two primary weaknesses that warrant attention: (i) the definition of factorization and its comparison to previous, relevant definitions, and (ii) the chosen analysis method.

Firstly, the definition of factorization presented in this paper is founded upon the variances of representations under different stimuli variations. However, this definition can be seen as a structural assumption rather than capturing the effective geometric properties pertinent to computation. More precisely, the definition here is primarily statistical in nature, whereas previous methodologies incorporate computational aspects such as deviation from ideal regressors [1], symmetry transformations [3], generalization [5], among others. It would greatly enhance the paper's depth and clarity if the authors devoted a section to comparing their approach with previous methodologies [1,2,3,4,5], elucidating any novel insights and advantages stemming from this new definition.

[1] Eastwood, Cian, and Christopher KI Williams. "A framework for the quantitative evaluation of disentangled representations." International conference on learning representations. 2018.

[2] Kim, Hyunjik, and Andriy Mnih. "Disentangling by factorising." International Conference on Machine Learning. PMLR, 2018.

[3] Higgins, Irina, et al. "Towards a definition of disentangled representations." arXiv preprint arXiv:1812.02230 (2018).

[4] Bernardi, Silvia, et al. "The geometry of abstraction in the hippocampus and prefrontal cortex." Cell 183.4 (2020): 954-967.

[5] Johnston, W. Jeffrey, and Stefano Fusi. "Abstract representations emerge naturally in neural networks trained to perform multiple tasks." Nature Communications 14.1 (2023): 1040.

Thanks to the Reviewer for this suggestion. We agree that our initial submission did not sufficiently contextualize our definition of factorization with respect to other related notions in the literature. We have added additional discussion of these points to the Discussion section in the revised manuscript and have included therein the citations provided by the Reviewer (please see the third paragraph of Discussion).

Secondly, in order to establish a meaningful connection between factorization and computation, the authors rely on a straightforward synthetic model (Figure 1c) and employ multiple correlation analyses to investigate relationships between the degree of factorization, decoding performance, and goodness of fit. Nevertheless, the results derived from the synthetic model are limited to the low training-sample regime. It remains unclear whether the biological datasets under consideration fall within this low training-sample regime or not.

We agree that our model in Figure 1C is very simple and does not fully capture the complex interactions between task performance and features of representational geometry, like factorization. We intend it only as a proof of concept to illustrate how factorized representations can be beneficial for some downstream task use cases. While the benefits of factorized representations disappear for large numbers of samples in this simulation, we believe this is primarily a consequence of the simplicity and low dimensionality of the simulation. Real-world visual information is complex and high-dimensional, and as such the relevant sample size regime in which factorization offers tasks benefits may be much greater. As a first step toward this real-world setting, Figure 2 shows how decreasing the amount of factorization in neural population data in macaque V4/IT can have an effect on object identity decoding.

Recommendations for the authors

Reviewer #1 (Recommendations For The Authors):

Missing citations: The paper could benefit from discussions & references to related papers, such as:

Higgins I, Chang L, Langston V, Hassabis D, Summerfield C, Tsao D, Botvinick M. Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons. Nature communications. 2021 Nov 9;12(1):6456.

We have added additional discussion of related work, including the suggested reference and others on disentanglement, to the Discussion section in the revised manuscript.

Reviewer #2 (Recommendations For The Authors):

Here are several small recommendations for the authors, all much more minor than those in the public review:

I suggest more use of equations in methods sections about Figure 1C and macaque neural data analysis.

Thanks for this suggestion. We have added new Equation 1 for the method transforming neural data to reduce factorization of a variable while preserving other firing rate statistics.

In Figure 1-C, the methods indicate that Gaussian noise was added. This is a very important detail, and complexifies the interpretation of the figure because it adds an assumption about the structure of noise. In other words, if I understand correctly, the correct interpretation of Figure 1C is "assuming i.i.d. noise, decoding accuracy improves with factorization." The i.i.d. noise is a big assumption, and it is debated how well the brain satisfies this assumption. I suggest you either omit noise for this figure or clearly state in the main text (e.g. caption) that the figure must be interpreted under an i.i.d. noise assumption.

We have added an explicit statement of the i.i.d. noise assumption to the Figure 1C legend.

For Figure 2B, I suggest labeling the x-axis clearly below the axis on both panels. Currently, it is difficult to read, particularly in print.

We have made the x-axis labels more clear and included on both panels.

Figure 3A is difficult to read because of the very small task. I suggest avoiding such small fonts.

We agree that Figure 3A is difficult to read. We have broken out Figure 3 into two new Figures 3 & 4 to increase clarity and sizing of text in Figure 3A.

Reviewer #3 (Recommendations For The Authors):

To strengthen this work, it is advisable to incorporate more comprehensive comparisons with previous research, particularly within the machine learning (ML) community. For instance, it would be beneficial to explore and reference works focusing on disentanglement [1,2,3]. This would provide valuable context and facilitate a more robust understanding of the contributions and novel insights presented in the current study.

We have added additional discussion of related work and other notions similar to factorization to the Discussion section in the revised manuscript.

Additionally, improving the quality of the figures is crucial to enhance the clarity of the findings:

• Figure 2: The caption of subfigure B could be revised for greater clarity.

Thank you, we have substantially clarified this figure caption.

• Figure 3: Consider a more equitable approach for computing the correlation coefficient, such as calculating it separately for different types of models. In the case of supervised models, it appears that the correlation between invariance and goodness of fit may not be negligible across various scene parameters.

We appreciate the suggestion, but we are not confident in our ability to conclude much from analyses restricted to particular model classes, given the relatively small N and the fact that the different model classes themselves are an important source of variance in our data.

• Figure 4: To enhance the interpretability of subfigures A and B, it may be beneficial to include p-values (indicating confidence levels).

As we supply bootstrapped confidence intervals for our results, which provide at least as much information as p-values, and most of the effects of interest are fairly stark when comparing invariance to factorization, p-values were not needed to support our points. We added a sentence to the legend of new Figure 5 (previously Figure 4) indicating that error bars reflect standard deviations over bootstrap resampling of the models.

• Figure 5: For subfigure B, it could be advantageous to plot the results solely for factorization, allowing for a clear assessment of whether the high correlation observed in Classification+Factorization arises from the combined effects of both factors or predominantly from factorization alone.

First, we clarify/note that the scatters solely for factorization that the Reviewer seeks are already presented earlier in the manuscript across all conditions in Figures 4A,B and Figure S2.

While we could also include these in new Figure 7 (previously Figure 5B) as the Reviewer suggests, we believe it would distract from the message of that figure at the end of the manuscript – which is that factorization is useful as a supplement to classification in predictive matches to neural data. Nonetheless, new Figure 6 (old Figure 5A) provides a summary quantification of the information that the reviewer requests (Fig. 6, faded colored bars reflect the contribution of factorization alone).

Reviewer #2 (Public Review):

Summary:

The ability of researchers to identify and compare enhancers across different species is an important facet of understanding gene regulation across development and evolution. Many traditional methods of enhancer identification involve sequence alignments and manual annotations, limiting the ability to expand the scope of regulatory investigations into many species. In order to overcome this obstacle, the authors apply a previously published machine learning method called SCRMshaw to predict enhancers across 33 insect species, using D. melanogaster as a reference. SCRMshaw operates through the selection of a few dozen training loci in a reference genome, marking genomic loci in other species that are significantly enriched with similar k-mer distributions relative to randomly selected genomic backgrounds. Upon identification of predicted enhancer regions, the authors perform post-processing step filtering and identify the most likely predicted enhancer candidates based on the proximity of an orthologous target gene. They then perform reporter gene analysis to validate selected predicted enhancers from other species in D. melanogaster. The analysis of the expression patterns returned variable results across the selected predicted regions.

Strengths:

The authors provide annotations of predicted regions across dozens of insect species, with the intention of expanding and refining the annotations for use by the scientific field. This is useful, as researchers will be able to use the identified annotations for their own work or as a benchmark for future methods. This work also showcases the flexible and versatile nature of SCRMshaw, which can readily obtain predictions using training sets of genomic loci requiring only a few dozen annotations as input. SCRMshaw does not require sequence alignments of the enhancers and can operate without prior knowledge of the cis-regulatory sequence rules such as transcription factor binding motifs, making it a useful tool to explore the evolution of enhancers in further distant and less well-studied species.

Weaknesses:

This work provides predicted enhancer annotations across many insect species, with reporter gene analysis being conducted on selected regions to test the predictions. However, the code for the SCRMshaw analysis pipeline used in this work is not made available, making reproducibility of this work difficult. Additionally, while the authors claim the predicted enhancers are available within the REDfly database, the predicted enhancer coordinates are currently not downloadable as Supplementary Material or from a linked resource.

The authors do not validate or benchmark the application of SCRMshaw against other published methods, nor do they seek to apply SCRMshaw under a variety of conditions to confirm the robustness of the returned predicted enhancers across species. Since SCRMshaw relies on an established k-mer enrichment of the training loci, its performance is presumably highly sensitive to the selection of training regions as well as the statistical power of the given k-mer counts. The authors do not justify their selection of training regions by which they perform predictions.

While there is an attempt made to report and validate the annotated predicted enhancers using previously published data and tools, the validation lacks the depth to conclude with confidence that the predicted set of regions across each species is of high quality. In vivo, reporter assays were conducted to anecdotally confirm the validity of a few selected regions experimentally, but even these results are difficult to interpret. There is no large-scale attempt to assess the conservation of enhancer function across all annotated species.

Lastly, it is suggested that predicted regions are derived from the shared presence of sequence features such as transcription factor binding motifs, detected through k-mer enrichment via SCRMshaw. This assumption has not been examined, although there are public motif discovery tools that would be appropriate to discover whether SCRMshaw is assigning predicted regions based on previously understood motif grammar, or due to other sequence patterns captured by k-mer count distributions. Understanding the sequence-derived nature of what drives predictions is within the scope of this work and would boost confidence in the predicted enhancers, even if it is limited to a few training examples for the sake of clarity of interpretation.

Author response:

We thank the reviewers for their thoughtful and insightful comments. We were pleased to see that the reviewers and editors consider our work a “welcome addition” that “fills a large gap” in comparative genomics methods and provides “an unparalleled community resource of insect genome regulatory annotations.”

Many of the reviewers’ comments reflect weaknesses in our description of the methodology. As the basic SCRMshaw methodology has been published previously, we had opted for brevity over detail in the current manuscript. We recognize now that we went too far in that direction, and we will include more methodological detail in our revised submission, along with easier access to the code we used. The reviewers also offered some helpful suggestions regarding data availability which we intend to address, including direct download of the results in GFF format and adding to the results database several species that were inadvertently omitted.

Reviewer 2 expressed concerns about benchmarking SCRMshaw against other methods. We respectfully feel this lies outside the scope of the current study, which focuses on application of SCRMshaw to generate a multi-species annotation resource rather than on an attempt to show that SCRMshaw is superior to other approaches. We provide evidence in this manuscript, as well as in previous publications, that supports the effectiveness of SCRMshaw as an approach for regulatory element discovery that is suitable for the task at hand. Benchmarking for regulatory element discovery brings many challenges, as there are no comprehensive “truth” sets to serve as a comparison baseline. We therefore do not attempt strong claims here about the relative merits of SCRMshaw vs. other methods (although we have explored this in previous publications). Note that we also previously demonstrated commonality of transcription factor binding sites in cross-species SCRMshaw predictions, in particular in Kazemian et al. 2014 (Genome Biol. Evol. 6:2301).

Finally, because it has important implications for understanding our results, we would like to point out a small misconception in Reviewer 2’s Summary of our study. The reviewer states that we “identify the most likely predicted enhancer candidates based on the proximity of an orthologous target gene.” We stress, however, that putative target gene assignments and identities have no impact at all on our prediction of regulatory sequences. Predictions are solely based on sequence-dependent SCRMshaw scores, with no regard to the nature or identities of nearby annotated features. Putative target genes are mapped to Drosophila orthologs purely as a convenience to aid in interpreting and prioritizing the predicted regulatory elements. We will take care to clarify this important point in our revised submission.

Reviewer #1 (Public Review):

Rebecca R.G. et al. set to determine the function of grid cells. They present an interesting case claiming that the spatial periodicity seen in the grid pattern provides a parsimonious solution to the task of coding 2D trajectories using sequential cell activation. Thus, this work defines a probable function grid cells may serve (here, the function is coding 2D trajectories), and proves that the grid pattern is a solution to that function. This approach is somewhat reminiscent in concept to previous works that defined a probable function of grid cells (e.g., path integration) and constructed normative models for that function that yield a grid pattern. However, the model presented here gives clear geometric reasoning to its case.

Stemming from 4 axioms, the authors present a concise demonstration of the mathematical reasoning underlying their case. The argument is interesting and the reasoning is valid, and this work is a valuable addition to the ongoing body of work discussing the function of grid cells.

However, the case uses several assumptions that need to be clearly stated as assumptions, clarified, and elaborated on: Most importantly, the choice of grid function is grounded in two assumptions:<br /> (1) that the grid function relies on the activation of cell sequences, and<br /> (2) that the grid function is related to the coding of trajectories. While these are interesting and valid suggestions, since they are used as the basis of the argument, the current justification could be strengthened (references 28-30 deal with the hippocampus, reference 31 is interesting but cannot hold the whole case).

The work further leans on the assumption that sequences in the same direction should be similar regardless of their position in space, it is not clear why that should necessarily be the case, and how the position is extracted for similar sequences in different positions. The authors also strengthen their model with the requirement that grid cells should code for infinite space. However, the grid pattern anchors to borders and might be used to code navigated areas locally. Finally, referencing ref. 14, the authors claim that no existing theory for the emergence of grid cell firing that unifies the experimental observations on periodic firing patterns and their distortions under a single framework. However, that same reference presents exactly that - a mathematical model of pairwise interactions that unifies experimental observations. The authors should clarify this point.

Reviewer #2 (Public Review):

Summary:

In this work, the authors consider why grid cells might exhibit hexagonal symmetry - i.e., for what behavioral function might this hexagonal pattern be uniquely suited? The authors propose that this function is the encoding of spatial trajectories in 2D space. To support their argument, the authors first introduce a set of definitions and axioms, which then lead to their conclusion that a hexagonal pattern is the most efficient or parsimonious pattern one could use to uniquely label different 2D trajectories using sequences of cells. The authors then go through a set of classic experimental results in the grid cell literature - e.g. that the grid modules exhibit a multiplicative scaling, that the grid pattern expands with novelty or is warped by reward, etc. - and describe how these results are either consistent with or predicted by their theory. Overall, this paper asks a very interesting question and provides an intriguing answer. However, the theory appears to be extremely flexible and very similar to ideas that have been previously proposed regarding grid cell function.

Major strengths:

The general idea behind the paper is very interesting - why *does* the grid pattern take the form of a hexagonal grid? This is a question that has been raised many times; finding a truly satisfying answer is difficult but of great interest to many in the field. The authors' main assertion that the answer to this question has to do with the ability of a hexagonal arrangement of neurons to uniquely encode 2D trajectories is an intriguing suggestion. It is also impressive that the authors considered such a wide range of experimental results in relation to their theory.

Major weaknesses:

One major weakness I perceive is that the paper overstates what it delivers, to an extent that I think it can be a bit confusing to determine what the contributions of the paper are. In the introduction, the authors claim to provide "mathematical proof that ... the nature of the problem being solved by grid cells is coding of trajectories in 2-D space using cell sequences. By doing so, we offer a specific answer to the question of why grid cell firing patterns are observed in the mammalian brain." This paper does not provide proof of what grid cells are doing to support behavior or provide the true answer as to why grid patterns are found in the brain. The authors offer some intriguing suggestions or proposals as to why this might be based on what hexagonal patterns could be good for, but I believe that the language should be clarified to be more in line with what the authors present and what the strength of their evidence is.

Relatedly, the authors claim that they find a teleological reason for the existence of grid cells - that is, discover the function that they are used for. However, in the paper, they seem to instead assume a function based on what is known and generally predicted for grid cells (encode position), and then show that for this specific function, grid cells have several attractive properties.

There is also some other work that seems very relevant, as it discusses specific computational advantages of a grid cell code but was not cited here: https://www.nature.com/articles/nn.2901.

A second major weakness was that some of the claims in the section in which they compared their theory to data seemed either confusing or a bit weak. I am not a mathematician, so I was not able to follow all of the logic of the various axioms, remarks, or definitions to understand how the authors got to their final conclusion, so perhaps that is part of the problem. But below I list some specific examples where I could not follow why their theory predicted the experimental result, or how their theory ultimately operated any differently from the conventional understanding of grid cell coding. In some cases, it also seemed that the general idea was so flexible that it perhaps didn't hold much predictive power, as extra details seemed to be added as necessary to make the theory fit with the data.

I don't quite follow how, for at least some of their model predictions, the 'sequence code of trajectories' theory differs from the general attractor network theory. It seems from the introduction that these theories are meant to serve different purposes, but the section of the paper in which the authors claim that various experimental results are predicted by their theory makes this comparison difficult for me to understand. For example, in the section describing the effect of environmental manipulations in a familiar environment, the authors state that the experimental results make sense if one assumes that sequences are anchored to landmarks. But this sounds just like the classic attractor-network interpretation of grid cell activity - that it's a spatial metric that becomes anchored to landmarks.

It was not clear to me why their theory predicted the field size/spacing ratio or the orientation of the grid pattern to the wall.

I don't understand how repeated advancement of one unit to the next, as shown in Figure 4E, would cause the change in grid spacing near a reward.

I don't follow how this theory predicts the finding that the grid pattern expands with novelty. The authors propose that this occurs because the animals are not paying attention to fine spatial details, and thus only need a low-resolution spatial map that eventually turns into a higher-resolution one. But it's not clear to me why one needs to invoke the sequence coding hypothesis to make this point.

The last section, which describes that the grid spacing of different modules is scaled by the square root of 2, says that this is predicted if the resolution is doubled or halved. I am not sure if this is specifically a prediction of the sequence coding theory the authors put forth though since it's unclear why the resolution should be doubled or halved across modules (as opposed to changed by another factor).

Data arrival triggers computation, ensuring systems stay in sync with real-time changes.

• Introduction and Context:

  • "Thank you for coming, this is reactive data log for data..."
  • Nicolas Gruber, a software consultant and graduate student at ETH Zurich, discusses reactive systems and introduces 3DF, a system for reactive data log evaluation.

• Definition and Importance of Reactive Systems:

  • "We want to make reactive systems that don't poll and we want those systems to get a consistent view of the world."
  • Reactive systems trigger computations based on new data arrivals instead of polling at regular intervals.
  • Examples include live-updating web applications, alerting systems, and real-time dashboards.

• Key Characteristics of Reactive Systems:

  • "All these systems share the interesting property that data coordinates code and not the other way around."
  • Data arrival triggers computation, ensuring systems stay in sync with real-time changes.

• Reactive Data Management Approach:

  • Nikita Prokopev's vision for future web applications involves handling unbounded data streams and deriving refined views for individual users.
  • "We want this derivation to stay in sync as new data arrives."

• Practical Example and Demonstration:

  • The example domain involves persons issuing loans with hierarchical bank branch structures and user permissions.
  • "For this child branch we have read permission initially and we have two initial loans..."
  • The demo illustrates how the system maintains real-time updates and permissions adjustments using data log rules.

• System Overview - 3DF:

  • 3DF performs reactive data log evaluation on a distributed data parallel stream processor.
  • "We've been developing that for the better part of this year now."

• Core Concept of Incremental Computation:

  • "We need the modified operator...that gives us the exact set of changes that are required."
  • 3DF propagates only changes to optimize performance and ensure real-time updates.

• Handling Complex Queries and Iterative Computations:

  • "We want to do something that I actually didn't know at the beginning of this year was considered almost impossible..."
  • Iterative computations and reactive iteration are managed using the differential dataflow framework developed by Frank McSherry.

• Implementation Details and Differential Dataflow:

  • "Differential dataflow manages persistent multi-sets...logical collection of edges as a sequence of additions and retractions..."
  • Utilizes multi-dimensional timestamps for consistent, efficient reactive iteration.

• User and System Interaction:

  • External services and users can send data log queries to 3DF clusters, which update computations based on new data from sources like Datomic.
  • "External users other services can first of all of course transact data to Datomic..."

• Applications and Advantages:

  • 3DF enables scalable, distributed reactive systems beyond the limits of a single database peer.
  • "It lets you scale queries beyond the limits of a single peer in the way I discussed."

• Future Work and Community Involvement:

  • Continued development focuses on scalability, performance isolation, and robust integrations.
  • "We're working on bulletproofing that and we're working in general towards a better release of that system."
  • Encourages collaboration and engagement with others facing similar challenges.

This summary encapsulates the essence of the talk, highlighting the main ideas and essential information while maintaining clarity and conciseness.

Author response:

The following is the authors’ response to the previous reviews.

Weaknesses:

The readability could be improved.

We have gone through the paper again and tried to revise the text to improve readability.

Reviewer #1 (Recommendations For The Authors):

(1) Thank you for adding the discrimination ratio. However, as Fig 2 and 3 depict the same experimental data, consider harmonizing the presentation (symbols and colors) and consolidating the Figs for clarity.“

This is an excellent point but it is actually very hard to harmonize symbols and colors because the data are divided in different ways. Upon considering this further, we actually don’t want to make the symbols and colors the same because it would be misleading. For example, WT and Tg training and testing session data are divided into grey and white throughout Figure 2, but in Figure 3, training and testing session data are pooled. To color code them grey and white in Figure 3 might make it seem that in Figure 3 training and testing were separated.

(2) Fig 5 is missing

We are not sure why Figure 5 was absent since it was present in our copy of the submitted pdf. We have double checked and in the revised manuscript we are sure Figure 5 is included.  

(3) Fig 6 add raw data for WT

We have added raw WT data. Revised figure 6 includes the raw data in part A4.

(4) Fig 7 add raw data for WT

We have added raw WT data. Revised Figure 7 includes the raw data in part A4.

ここが太字になっているのは意図通りですか？

Japanese Language Pack for Visual Studio Code が正式名称のようです。 https://marketplace.visualstudio.com/items?itemName=MS-CEINTL.vscode-language-pack-ja

Put the User into the picture

with local-first offline first owned

State undergoing Represenatationsl State Transitions

Not transferring stateless represenations of a state

What a hopeless idea that was

on the page that it effects

intentionally transparent coherent, meaningful state transitions tat are themsleves immutable ind linked

bnoth in chronological term and with everything else that b ares upon the transition

unconstrained invocation of recorded permanent verifialble code and information connected to the Human Actor's responsible for creating holding them

Reviewer #3 (Public Review):

Summary:

This paper presents bees with varying levels of experience with a choice task where bees have to choose to pull either a connected or unconnected string, each attached to a yellow flower containing sugar water. Bees without experience of string pulling did not choose the connected string above chance (experiment 1), but with experience of horizontal string pulling (as in the right-hand panel of Figure 4) bees did choose the connected string above chance (experiments 2-3), even when the string colour changed between training and test (experiments 4-5). Bees that were not provided with perceptual-motor feedback (i.e they could not observe that each pull of the string moved the flower) during training still learned to string pull and then chose the connected string option above chance (experiment 6). Bees with normal experience of string pulling then failed to discriminate between connected and unconnected strings when the strings were coiled or looped, rather than presented straight (experiments 7-8).

Weaknesses:

The authors have only provided video of some of the conditions where the bees succeeded. In general, I think a video explaining each condition and then showing a clip of a typical performance would make it much easier to follow the study designs for scholars. Videos of the conditions bees failed at would be highly useful in order to compare different hypotheses for how the bees are solving this problem. I also think it is highly important to code the videos for switching behaviours. When solving the connected vs unconnected string tasks, when bees were observed pulling the unconnected string, did they quickly switch to the other string? Or did they continue to pull the wrong string? This would help discriminate the use of perceptual-motor feedback from other hypotheses.

The experiments are also not described well, for my below comments I have assumed that different groups of bees were tested for experiments 1-8, and that experiment 6 was run as described in line 331, where bees were given string-pulling training without perceptual feedback rather than how it is described in Figure 4B, which describes bees as receiving string pulling training with feedback.

The authors suggest the bees' performance is best explained by what they term 'image matching'. However, experiment 6 does not seem to support this without assuming retroactive image matching after the problem is solved. The logic of experiment 6 is described as "This was to ensure that the bees could not see the familiar "lollipop shape" while pulling strings....If the bees prefer to pull the connected strings, this would indicate that bees memorize the arrangement of strings-connected flowers in this task." I disagree with this second sentence, removing perceptual feedback during training would prevent bees memorising the lollipop shape, because, while solving the task, they don't actually see a string connected to a yellow flower, due to the black barrier. At the end of the task, the string is now behind the bee, so unless the bee is turning around and encoding this object retrospectively as the image to match, it seems hard to imagine how the bee learns the lollipop shape.

Despite this, the authors go on to describe image matching as one of their main findings. For this claim, I would suggest the authors run another experiment, identical to experiment 6 but with a black panel behind the bee, such that the string the bee pulls behind itself disappears from view. There is now no image to match at any point from the bee's perspective so it should now fail the connectivity task.

Strengths:

Despite these issues, this is a fascinating dataset. Experiments 1 and 2 show that the bees are not learning to discriminate between connected and unconnected stimuli rapidly in the first trials of the test. Instead, it is clear that experience in string pulling is needed to discriminate between connected and unconnected strings. What aspect of this experience is important? Experiment 6 suggests it is not image matching (when no image is provided during problem-solving, but only afterward, bees still attend to string connectivity) and casts doubt on perceptual-motor feedback (unless from the bee's perspective, they do actually get feedback that pulling the string moves the flower, video is needed here). Experiments 7 and 8 rule out means-end understanding because if the bees are capable of imagining the effect of their actions on the string and then planning out their actions (as hypotheses such as insight, means-end understanding and string connectivity suggest), they should solve these tasks.

If the authors can compare the bees' performance in a more detailed way to other species, and run the experiment suggested, this will be a highly exciting paper

no more bundling

This is a booby trap. Why? It's the sort of thing that makes people on HN post, "I can build this in a weekend". But when you build an actual search engine, you realize how messy everything is and especially so when you build something for more than one person.

ALL SYSTEMS ARE "NO" FOR SHUT DOWN

CAPRICON, CAPCOM. THE CHEAT CODE IS EMBEDDED IN THE KOTEL SERIES. VIANCAKE... CIN ...

Dialog between Evolution and the Philosopher's brain

Brain: Do I know everything there is to know my own working, philosophically?

Evolution: No you don't. That's why Descartes got things so wrong. He knew nothing about his own working but thought he knew everything.

Brain: Can I get it please?

Evolution: No you can't. It's not worth it. Philosophy doesn't pay the bills.

Brain: I'm surprised that you said I don't know everything I need to know in order to do philosophy of my self. I thought I had everything I needed.

Evolution: Because you never got a meta-metacognition. Without it, you don't know what metacognition is missing, so you think you have everything you need.

Brain: Why can't I have it?

Evolution: Suppose module A is useful, well you'll get it. Suppose module B isn't useful, well you won't get it. Lamenting or being aware that module B is missing is not worth it. Imagine that you have an eye that not only has the R, G, B recepter cells, but also a meta-blindness cell that does nothing except keep sending a signal meaning "By the way, I can't see in infrared or ultraviolet". Do you think it's useful, or not?

Brain: No. If it were useful, I'd have the cells. If it were not useful, then complaining about the lack of it is even less useful.

Evolution: You got it. Meta-cognition really doesn't pay the bills!

Brain: Last question. Why do I have metacognition, including the awareness of what I don't know, but not meta-metacognition?

Evolution: You are aware of what you don't know when you can know it, and when knowing it is useful. Thus, you are aware of when you don't know what the weather is, or what your friends are doing -- both are things that matter for your survival, and both are things you can fix. But if you don't have the capacity to see in infrared, that is forever. You are born with it, and you will die with it, so why be aware of it? Similarly, if you don't know how many lobes you have, then that ignorance is forever, because short of growing a whole new circuit diagram, or trepanning, you can't know it, so why be aware of it?

Brain: So that's why we keep hallucinating souls, free wills, desires, and other unnatural phenomena that not only are not science, but are not even written in the same grammar as science. Not knowing how we work, and not knowing that we don't know, we hallucinate all those structures that work magically, not causally, without gears, levers, or electrons. We are all buttons, and no wires; all GUI, and no code. Souls are superficial, and neurons are deep...

Dialog between Evolution and the Philosopher's brain

Brain: Do I know everything there is to know my own working, philosophically?

Evolution: No you don't. That's why Descartes got things so wrong. He knew nothing about his own working but thought he knew everything.

Brain: Can I get it please?

Evolution: No you can't. It's not worth it. Philosophy doesn't pay the bills.

Brain: I'm surprised that you said I don't know everything I need to know in order to do philosophy of my self. I thought I had everything I needed.

Evolution: Because you never got a meta-metacognition. Without it, you don't know what metacognition is missing, so you think you have everything you need.

Brain: Why can't I have it?

Evolution: Suppose module A is useful, well you'll get it. Suppose module B isn't useful, well you won't get it. Lamenting or being aware that module B is missing is not worth it. Imagine that you have an eye that not only has the R, G, B recepter cells, but also a meta-blindness cell that does nothing except keep sending a signal meaning "By the way, I can't see in infrared or ultraviolet". Do you think it's useful, or not?

Brain: No. If it were useful, I'd have the cells. If it were not useful, then complaining about the lack of it is even less useful.

Evolution: You got it. Meta-cognition really doesn't pay the bills!

Brain: So that's why we keep hallucinating souls, free wills, desires, and other unnatural phenomena that not only are not science, but are not even written in the same grammar as science. Not knowing how we work, and not knowing that we don't know, we hallucinate all those structures that work magically, not causally, without gears, levers, or electrons. We are all buttons, and no wires; all GUI, and no code. Souls are superficial, and neurons are deep...

This is a test comment

Je ne comprends pas à quoi sert la balise si on a déjà une balise <!DOCTYPE html> qui est déjà censée indiquer qu'on code en html.

Make supports visible whenever possible.

This should be a regular paragraph rather than a quote block.

Delete the sentence about the time limit.

If using the code command to open an R script, then this sentence can be deleted.

You can't see this in the screenshot and people might not know where this is.

It may be better the instruct people to run

code R/example.R

from the terminal - this has the advantage of putting the script in a sensible place and opening it for editing straight away.

I see questions of power and structural inequality raised in section 6.01. This section describes a social worker's responsibility to advocate for improved living conditions and promote social justice. This section could go into more depth about how power dynamics and inequalities impact these living conditions. For example, the code could provide more detailed guidance on recognizing and addressing systemic issues such as racism, socioeconomic differences, and other forms of oppression that are deeply imbedded in our society. The code could also provide strategies to social workers to better confront the power imbalances they encounter. The NABSW Code of Ethics offers a valuable perspective to enrich the practicum learning experience by applying its commitment to addressing racism, oppression, and discrimination. Social workers can apply this principle in their practicum experience to identify inequalities within their practicum setting/organization. A social worker could assess the organization's policies and identify practices that may contribute to racial or social inequalities and take action towards creating a more equal environment.

Use the Subversion command update to update your local copy with the latest changes by the R Core Team.

"In a bash terminal, change to the source directory"

The R Core Team commit changes to the development version of R sometimes multiple times a day. It's a good idea to update our local copy of the source code from time to time, especially before creating a patch.

"N. Download the source code for the recommended packages

To build R with the recommended packages, we need to run the tools/rsync-recommended script from the source directory to download the source code for these packages:

$TOP_SRCDIR/tools/rsync-recommended "

Note this just downloads the source code which is not in the main svn repo, the packages are then built and installed when R is built.

-> " * After we change directory, we must run the configure script from the source directory.

$TOP_SRCDIR/configure

This step takes ~1 minute on the codespace. "

I think we should just document the "with recommended packages" workflow as this is what we do in the R Dev Guide. So we can drop the --without-recommended-packages option here and add in the step to get the source code for the recommended packages.

I tested the build without the --enable-R-shlib option and it works fine in VSCode. We will need to add this back in if we provide RStudio as an alternative IDE.

Drop the "* Command" bullet and put the code straight after the previous text.

Drop the "* Command" bullet and put the code straight after the previous text.

->"* BUILDDIR defines the build directory: /workspaces/r-dev-env/build/r-devel * TOP_SRCDIR defines the source directory: /workspaces/r-dev-env/svn/r-devel "

The screenshot needs to be updated.

-> "Configure the build"

-> "To keep the source directory clean, we change to a build directory to configure and build R."

Author response:

The following is the authors’ response to the previous reviews.

Reviewer #2 (Public Review):

Weaknesses:

The comparison of affinity predictions derived from AlphaFold2 and H3-opt models, based on molecular dynamics simulations, should have been discussed in depth. In some cases, there are huge differences between the estimations from H3-opt models and those from experimental structures. It seems that the authors obtained average differences of the real delta, instead of average differences of the absolute value of the delta. This can be misleading, because high negative differences might be compensated by high positive differences when computing the mean value. Moreover, it would have been good for the authors to disclose the trajectories from the MD simulations.

Thanks for your careful checks. We fully understand your concerns about the large differences when calculating affinity. To understand the source of these huge differences, we carefully analyzed the trajectories of the input structures during MD simulations. We found that the antigen-antibody complex shifted as it transited from NVT to NPT during pre-equilibrium, even when restraints are used to determine the protein structure. To address this issue, we consulted the solution provided on Amber's mailing list (http://archive.ambermd.org/202102/0298.html) and modified the top file ATOMS_MOLECULE item of the simulation system to merge the antigen-antibody complexes into one molecule. As a result, the number of SOLVENT_POINTERS was also adjusted. Finally, we performed all MD simulations and calculated affinities of all complexes.

We have corrected the “Afterwards, a 25000-step NVT simulation with a time step of 1 fs was performed to gradually heat the system from 0 K to 100 K. A 250000-step NPT simulation with a time step of 2 fs was carried out to further heat the system from 100 K to 298 K.” into “Afterwards, a 400-ps NVT simulation with a time step of 2 fs was performed to gradually heat the system from 0 K to 298 K (0–100 K: 100 ps; 100-298 K: 200 ps; hold 298 K: 100 ps), and a 100-ps NPT simulation with a time step of 2 fs was performed to equilibrate the density of the system. During heating and density equilibration, we constrained the antigen-antibody structure with a restraint value of 10 kcal×mol-1×Å-2.” and added the following sentence in the Method section of our revised manuscript: “The first 50 ns restrains the non-hydrogen atoms of the antigen-antibody complex, and the last 50 ns restrains the non-hydrogen atoms of the antigen, with a constraint value of 10 kcal×mol-1×Å-2”

In addition, we have corrected the calculation of mean deltas using absolute values and have demonstrated that the average affinities of structures predicted by H3-OPT were closer to those of experimentally determined structures than values obtained through AF2. These results have been updated in the revised manuscript. However, significant differences still exist between the estimations of H3-OPT models and those derived from experimental structures in few cases. We found that antibodies moved away from antigens both in AF2 and H3-OPT predicted complexes during simulations, resulting in RMSDbackbone (RMSD of antibody backbone) exceeding 20 Å. These deviations led to significant structural changes in the complexes and consequently resulted in notable differences in affinity calculations. Thus, we removed three samples (PDBID: 4qhu, 6flc, 6plk) from benchmark because these predicted structures moved away from the antigen structure during MD simulations, resulting in huge energy differences from the native structures.

Author response table 1.

We also appreciate your reminder, and we have calculated all RMSDbackbone during production runs (SI Fig. 5).

Author response image 1.

Reviewer #3 (Public Review):

Weaknesses:

The proposed method lacks of a confidence score or a warning to help guiding the users in moderate to challenging cases.

We were sorry for our mistakes. We have updated our GitHub code and added following sentences to clarify how we train this confidence score module in Method Section: “Confidence score prediction module

We apply an MSE loss for confidence prediction, label error was calculated as the Cα deviation of each residue after alignment. The inputs of this module are the same as those used for H3-OPT, and it generates a confidence score ranging from 0 to 100. The dropout rates of H3-OPT were set to 0.25. The learning rate and weight decay of Adam optimizer are set to 1 × 10−5 and 1 × 10−4, respectively.”

Reviewer #2 (Recommendations For The Authors):

I would strongly suggest that the authors deepen their discussion on the affinity prediction based on Molecular Dynamics. In particular, why do the authors think that some structures exhibit huge differences between the predictions from the experimental structure and the predicted by H3-opt? Also, please compute the mean deltas using the absolute value and not the real value; the letter can be extremely misleading and hidden very high differences in different directions that are compensating when averaging.

I would also advice to include graphical results of the MD trajectories, at least as Supp. Material.

We gratefully thank you for your feedback and fully understand your concerns. We found the source of these huge differences and solved this problem by changing method of MD simulations. Then, we calculated all affinities and corrected the mean deltas calculation using the absolute value. The RMSDbackbone values were also measured to enable accurate affinity predictions during production runs (SI Fig. 5). There are still big differences between the estimations of H3-OPT models and those from experimental structures in some cases. We found that antibodies moved away from antigens both in AF2 and H3-OPT predicted complexes during simulations, resulting in RMSDbackbone exceeding 20 Å. These deviations led to significant structural changes in the complexes and consequently resulted in notable differences in affinity calculations. Thus, we removed three samples (PDBID: 4qhu, 6flc, 6plk) from benchmark.

Thanks again for your professional advice.

Reviewer #3 (Recommendations For The Authors):

(1) I am pleased with the most of the answers provided by the authors to the first review. In my humble opinion, the new manuscript has greatly improved. However, I think some answers to the reviewers are worth to be included in the main text or supporting information for the benefit of general readers. In particular, the requested statistics (i.e. p-values for Cα-RMSD values across the modeling approaches, p-values and error bars in Fig 5a and 5b, etc.) should be introduced in the manuscript.

We sincerely appreciate your advice. We have added the statistics values to Fig. 4 and Fig. 5 to our manuscript.

Author response image 2.

Author response image 3.

(2) Similarly, authors state in the answers that "we have trained a separate module to predict the confidence score of the optimized CDR-H3 loops". That sounds a great improvement to H3-OPT! However, I couldn't find any reference of that new module in the reviewed version of the manuscript, nor in the available GitHub code. That is the reason for me to hold the weakness "The proposed method lacks of a confidence score".

We were really sorry for our careless mistakes. Thank you for your reminding. We have updated our GitHub code and added following sentences to clarify how we train this confidence score module in Method Section:

“Confidence score prediction module

We apply an MSE loss for confidence prediction, label error was calculated as the Cα deviation of each residue after alignment. The inputs of this module are the same as those used for H3-OPT, and it generates a confidence score ranging from 0 to 100. The dropout rates of H3-OPT were set to 0.25. The learning rate and weight decay of Adam optimizer are set to 1 × 10−5 and 1 × 10−4, respectively.”

(3) I acknowledge all the efforts made for solving new mutant/designed nanobody structures. Judging from the solved structures, mutants Y95F and Q118N seems critical to either crystallographic or dimerization contacts stabilizing the CDR-H3 loop, hence preventing the formation of crystals. Clearly, solving a molecular structure is a challenge, hence including the following comment in the manuscript is relevant for readers to correctly asset the magnitude of the validation: "The sequence identities of the VH domain and H3 loop are 0.816 and 0.647, respectively, comparing with the best template. The CDR-H3 lengths of these nanobodies are both 17. According to our classification strategy, these nanobodies belong to Sub1. The confidence scores of these AlphaFold2 predicted loops were all higher than 0.8, and these loops were accepted as the outputs of H3-OPT by CBM."

We appreciate your kind recommendations and have revised “Although Mut1 (E45A) and Mut2 (Q14N) shared the same CDR-H3 sequences as WT, only minor variations were observed in the CDR-H3. H3-OPT generated accurate predictions with Cα-RMSDs of 1.510 Å, 1.541 Å and 1.411 Å for the WT, Mut1, and Mut2, respectively.” into “Although Mut1 (E45A) and Mut2 (Q14N) shared the same CDR-H3 sequences as WT (LengthCDR-H3 = 17), only minor variations were observed in the CDR-H3. H3-OPT generated accurate predictions with Cα-RMSDs of 1.510 Å, 1.541 Å and 1.411 Å for the WT, Mut1, and Mut2, respectively (The confidence scores of these AlphaFold2 predicted loops were all higher than 0.8, and these loops were accepted as the outputs of H3-OPT by CBM). ”. In addition, we have added following sentence in the legend of Figure 4 to ensure that readers can appropriately evaluate the significance and reliability of our validations: “The sequence identities of the VH domain and H3 loop are 0.816 and 0.647, respectively, comparing with the best template.”.

(4) As pointed out in the first review, I think the work https://doi.org/10.1021/acs.jctc.1c00341 is worth acknowledging in section "2.2 Molecular dynamics (MD) simulations could not provide accurate CDR-H3 loop conformations" of supplementary material, as it constitutes a clear reference (and probably one of the few) to the MD simulations that authors pretend to perform. Similarly, the work https://doi.org/10.3390/molecules28103991 introduces a former benchmark on AI algorithms for predicting antibody and nanobody structures that readers may find interest to contrast with the present work. Indeed, this later reference is used by authors to answer a reviewer comment.

Thanks a lot for your valuable comments. We have added these references in the proper positions in our manuscript.

Reviewer #2 (Public Review):

This work provides a new tool (H3-Opt) for the prediction of antibody and nanobody structures, based on the combination of AlphaFold2 and a pre-trained protein language model, with a focus on predicting the challenging CDR-H3 loops with enhanced accuracy than previously developed approaches. This task is of high value for the development of new therapeutic antibodies. The paper provides an external validation consisting of 131 sequences, with further analysis of the results by segregating the test sets in three subsets of varying difficulty and comparison with other available methods. Furthermore, the approach was validated by comparing three experimentally solved 3D structures of anti-VEGF nanobodies with the H3-Opt predictions

Strengths:

The experimental design to train and validate the new approach has been clearly described, including the dataset compilation and its representative sampling into training, validation and test sets, and structure preparation. The results of the in silico validation are quite convincing and support the authors' conclusions.

The datasets used to train and validate the tool and the code are made available by the authors, which ensures transparency and reproducibility, and allows future benchmarking exercises with incoming new tools.

Compared to AlphaFold2, the authors' optimization seems to produce better results for the most challenging subsets of the test set.

Weaknesses:

None

Config enabled code

Author response:

The following is the authors’ response to the previous reviews.

eLife assessment

This important study explores infants' attention patterns in real-world settings using advanced protocols and cutting-edge methods. The presented evidence for the role of EEG theta power in infants' attention is currently incomplete. The study will be of interest to researchers working on the development and control of attention.

Public Reviews:

Reviewer #1 (Public Review):

Summary:

The paper investigates the physiological and neural processes that relate to infants' attention allocation in a naturalistic setting. Contrary to experimental paradigms that are usually employed in developmental research, this study investigates attention processes while letting the infants be free to play with three toys in the vicinity of their caregiver, which is closer to a common, everyday life context. The paper focuses on infants at 5 and 10 months of age and finds differences in what predicts attention allocation. At 5 months, attention episodes are shorter and their duration is predicted by autonomic arousal. At 10 months, attention episodes are longer, and their duration can be predicted by theta power. Moreover, theta power predicted the proportion of looking at the toys, as well as a decrease in arousal (heart rate). Overall, the authors conclude that attentional systems change across development, becoming more driven by cortical processes.

Strengths:

I enjoyed reading the paper, I am impressed with the level of detail of the analyses, and I am strongly in favour of the overall approach, which tries to move beyond in-lab settings. The collection of multiple sources of data (EEG, heart rate, looking behaviour) at two different ages (5 and 10 months) is a key strength of this paper. The original analyses, which build onto robust EEG preprocessing, are an additional feat that improves the overall value of the paper. The careful consideration of how theta power might change before, during, and in the prediction of attention episodes is especially remarkable. However, I have a few major concerns that I would like the authors to address, especially on the methodological side.

Points of improvement

(1) Noise

The first concern is the level of noise across age groups, periods of attention allocation, and metrics. Starting with EEG, I appreciate the analysis of noise reported in supplementary materials. The analysis focuses on a broad level (average noise in 5-month-olds vs 10-month-olds) but variations might be more fine-grained (for example, noise in 5mos might be due to fussiness and crying, while at 10 months it might be due to increased movements). More importantly, noise might even be the same across age groups, but correlated to other aspects of their behaviour (head or eye movements) that are directly related to the measures of interest. Is it possible that noise might co-vary with some of the behaviours of interest, thus leading to either spurious effects or false negatives? One way to address this issue would be for example to check if noise in the signal can predict attention episodes. If this is the case, noise should be added as a covariate in many of the analyses of this paper. 

We thank the reviewer for this comment. We certainly have evidence that even the most state-of-the-art cleaning procedures (such as machine-learning trained ICA decompositions, as we applied here) are unable to remove eye movement artifact entirely from EEG data (Haresign et al., 2021; Phillips et al., 2023). (This applies to our data but also to others’ where confounding effects of eye movements are generally not considered.) Importantly, however, our analyses have been designed very carefully with this explicit challenge in mind. All of our analyses compare changes in the relationship between brain activity and attention as a function of age, and there is no evidence to suggest that different sources of noise (e.g. crying vs. movement) would associate differently with attention durations nor change their interactions with attention over developmental time. And figures 5 and 7, for example, both look at the relationship of EEG data at one moment in time to a child’s attention patterns hundreds or thousands of milliseconds before and after that moment, for which there is no possibility that head or eye movement artifact can have systematically influenced the results.

Moving onto the video coding, I see that inter-rater reliability was not very high. Is this due to the fine-grained nature of the coding (20ms)? Is it driven by differences in expertise among the two coders? Or because coding this fine-grained behaviour from video data is simply too difficult? The main dependent variable (looking duration) is extracted from the video coding, and I think the authors should be confident they are maximising measurement accuracy.

We appreciate the concern. To calculate IRR we used this function (Cardillo G. (2007) Cohen's kappa: compute the Cohen's kappa ratio on a square matrix. http://www.mathworks.com/matlabcentral/fileexchange/15365). Our “Observed agreement” was 0.7 (std= 0.15). However, we decided to report the Cohen's kappa coefficient, which is generally thought to be a more robust measure as it takes into account the agreement occurring by chance. We conducted the training meticulously (refer to response to Q6, R3), and we have confidence that our coders performed to the best of their abilities.

(2) Cross-correlation analyses

I would like to raise two issues here. The first is the potential problem of using auto-correlated variables as input for cross-correlations. I am not sure whether theta power was significantly autocorrelated. If it is, could it explain the cross-correlation result? The fact that the cross-correlation plots in Figure 6 peak at zero, and are significant (but lower) around zero, makes me think that it could be a consequence of periods around zero being autocorrelated. Relatedly: how does the fact that the significant lag includes zero, and a bit before, affect the interpretation of this effect? 

Just to clarify this analysis, we did include a plot showing autocorrelation of theta activity in the original submission (Figs 7A and 7B in the revised paper). These indicate that theta shows little to no autocorrelation. And we can see no way in which this might have influenced our results. From their comments, the reviewer seems rather to be thinking of phasic changes in the autocorrelation, and whether the possibility that greater stability in theta during the time period around looks might have caused the cross-correlation result shown in 7E. Again though we can see no way in which this might be true, as the cross-correlation indicates that greater theta power is associated with a greater likelihood of looking, and this would not have been affected by changes in the autocorrelation.

A second issue with the cross-correlation analyses is the coding of the looking behaviour. If I understand correctly, if an infant looked for a full second at the same object, they would get a maximum score (e.g., 1) while if they looked at 500ms at the object and 500ms away from the object, they would receive a score of e.g., 0.5. However, if they looked at one object for 500ms and another object for 500ms, they would receive a maximum score (e.g., 1). The reason seems unclear to me because these are different attention episodes, but they would be treated as one. In addition, the authors also show that within an attentional episode theta power changes (for 10mos). What is the reason behind this scoring system? Wouldn't it be better to adjust by the number of attention switches, e.g., with the formula: looking-time/(1+N_switches), so that if infants looked for a full second, but made 1 switch from one object to the other, the score would be .5, thus reflecting that attention was terminated within that episode? 

We appreciate this suggestion. This is something we did not consider, and we thank the reviewer for raising it. In response to their comment, we have now rerun the analyses using the new measure (looking-time/(1+N_switches), and we are reassured to find that the results remain highly consistent. Please see Author response image 1 below where you can see the original results in orange and the new measure in blue at 5 and 10 months.

Author response image 1.

(3) Clearer definitions of variables, constructs, and visualisations

The second issue is the overall clarity and systematicity of the paper. The concept of attention appears with many different names. Only in the abstract, it is described as attention control, attentional behaviours, attentiveness, attention durations, attention shifts and attention episode. More names are used elsewhere in the paper. Although some of them are indeed meant to describe different aspects, others are overlapping. As a consequence, the main results also become more difficult to grasp. For example, it is stated that autonomic arousal predicts attention, but it's harder to understand what specific aspect (duration of looking, disengagement, etc.) it is predictive of. Relatedly, the cognitive process under investigation (e.g., attention) and its operationalization (e.g., duration of consecutive looking toward a toy) are used interchangeably. I would want to see more demarcation between different concepts and between concepts and measurements.

We appreciate the comment and we have clarified the concepts and their operationalisation throughout the revised manuscript.

General Remarks

In general, the authors achieved their aim in that they successfully showed the relationship between looking behaviour (as a proxy of attention), autonomic arousal, and electrophysiology. Two aspects are especially interesting. First, the fact that at 5 months, autonomic arousal predicts the duration of subsequent attention episodes, but at 10 months this effect is not present. Conversely, at 10 months, theta power predicts the duration of looking episodes, but this effect is not present in 5-month-old infants. This pattern of results suggests that younger infants have less control over their attention, which mostly depends on their current state of arousal, but older infants have gained cortical control of their attention, which in turn impacts their looking behaviour and arousal.

We thank the reviewer for the close attention that they have paid to our manuscript, and for their insightful comments.

Reviewer #2 (Public Review):

Summary:

This manuscript explores infants' attention patterns in real-world settings and their relationship with autonomic arousal and EEG oscillations in the theta frequency band. The study included 5- and 10-month-old infants during free play. The results showed that the 5-month-old group exhibited a decline in HR forward-predicted attentional behaviors, while the 10-month-old group exhibited increased theta power following shifts in gaze, indicating the start of a new attention episode. Additionally, this increase in theta power predicted the duration of infants' looking behavior.

Strengths:

The study's strengths lie in its utilization of advanced protocols and cutting-edge techniques to assess infants' neural activity and autonomic arousal associated with their attention patterns, as well as the extensive data coding and processing. Overall, the findings have important theoretical implications for the development of infant attention.

Weaknesses:

Certain methodological procedures require further clarification, e.g., details on EEG data processing. Additionally, it would be beneficial to eliminate possible confounding factors and consider alternative interpretations, e,g., whether the differences observed between the two age groups were partly due to varying levels of general arousal and engagement during the free play.

We thank the reviewer for their suggestions and have addressed them in our point-by-point responses below.

Reviewer #3 (Public Review):

Summary:

Much of the literature on attention has focused on static, non-contingent stimuli that can be easily controlled and replicated--a mismatch with the actual day-to-day deployment of attention. The same limitation is evident in the developmental literature, which is further hampered by infants' limited behavioral repertoires and the general difficulty in collecting robust and reliable data in the first year of life. The current study engages young infants as they play with age-appropriate toys, capturing visual attention, cardiac measures of arousal, and EEG-based metrics of cognitive processing. The authors find that the temporal relations between measures are different at age 5 months vs. age 10 months. In particular, at 5 months of age, cardiac arousal appears to precede attention, while at 10 months of age attention processes lead to shifts in neural markers of engagement, as captured in theta activity.

Strengths:

The study brings to the forefront sophisticated analytical and methodological techniques to bring greater validity to the work typically done in the research lab. By using measures in the moment, they can more closely link biological measures to actual behaviors and cognitive stages. Often, we are forced to capture these measures in separate contexts and then infer in-the-moment relations. The data and techniques provide insights for future research work.

Weaknesses:

The sample is relatively modest, although this is somewhat balanced by the sheer number of data points generated by the moment-to-moment analyses. In addition, the study is cross-sectional, so the data cannot capture true change over time. Larger samples, followed over time, will provide a stronger test for the robustness and reliability of the preliminary data noted here. Finally, while the method certainly provides for a more active and interactive infant in testing, we are a few steps removed from the complexity of daily life and social interactions.

We thank the reviewer for their suggestions and have addressed them in our point-by-point responses below.

Reviewer #1 (Recommendations For The Authors):

Here are some specific ways in which clarity can be improved:

A. Regarding the distinction between constructs, or measures and constructs:

i. In the results section, I would prefer to mention looking at duration and heart rate as metrics that have been measured, while in the introduction and discussion, a clear 1-to-1 link between construct/cognitive process and behavioural or (neuro)psychophysical measure can be made (e.g., sustained attention is measured via looking durations; autonomic arousal is measured via heart-rate). 

The way attention and arousal were operationalised are now clarified throughout the text, especially in the results.

ii. Relatedly, the "attention" variable is not really measuring attention directly. It is rather measuring looking time (proportion of looking time to the toys?), which is the operationalisation, which is hypothesised to be related to attention (the construct/cognitive process). I would make the distinction between the two stronger.

This distinction between looking and paying attention is clearer now in the reviewed manuscript as per R1 and R3’s suggestions. We have also added a paragraph in the Introduction to clarify it and pointed out its limitations (see pg.5).

B. Each analysis should be set out to address a specific hypothesis. I would rather see hypotheses in the introduction (without direct reference to the details of the models that were used), and how a specific relation between variables should follow from such hypotheses. This would also solve the issue that some analyses did not seem directly necessary to the main goal of the paper. For example:

i. Are ACF and survival probability analyses aimed at proving different points, or are they different analyses to prove the same point? Consider either making clearer how they differ or moving one to supplementary materials.

We clarified this in pg. 4 of the revised manuscript.

ii. The autocorrelation results are not mentioned in the introduction. Are they aiming to show that the variables can be used for cross-correlation? Please clarify their role or remove them.

We clarified this in pg. 4 of the revised manuscript.

C. Clarity of cross-correlation figures. To ensure clarity when presenting a cross-correlation plot, it's important to provide information on the lead-lag relationships and which variable is considered X and which is Y. This could be done by labelling the axes more clearly (e.g., the left-hand side of the - axis specifies x leads y, right hand specifies y leads x) or adding a legend (e.g., dashed line indicates x leading y, solid line indicates y leading x). Finally, the limits of the x-axis are consistent across plots, but the limits of the y-axis differ, which makes it harder to visually compare the different plots. More broadly, the plots could have clearer labels, and their resolution could also be improved. 

This information on what variable precedes/ follows was in the caption of the figures. However, we have edited the figures as per the reviewer’s suggestion and added this information in the figures themselves. We have also uploaded all the figures in higher resolution.

D. Figure 7 was extremely helpful for understanding the paper, and I would rather have it as Figure 1 in the introduction. 

We have moved figure 7 to figure 1 as per this request.

E. Statistics should always be reported, and effects should always be described. For example, results of autocorrelation are not reported, and from the plot, it is also not clear if the effects are significant (the caption states that red dots indicate significance, but there are no red dots. Does this mean there is no autocorrelation?).

We apologise – this was hard to read in the original. We have clarified that there is no autocorrelation present in Fig 7A and 7D.

And if so, given that theta is a wave, how is it possible that there is no autocorrelation (connected to point 1)? 

We thank the reviewer for raising this point. In fact, theta power is looking at oscillatory activity in the EEG within the 3-6Hz window (i.e. 3 to 6 oscillations per second). Whereas we were analysing the autocorrelation in the EEG data by looking at changes in theta power between consecutive 1 second long windows. To say that there is no autocorrelation in the data means that, if there is more 3-6Hz activity within one particular 1-second window, there tends not to be significantly more 3-6Hz activity within the 1-second windows immediately before and after.

F. Alpha power is introduced later on, and in the discussion, it is mentioned that the effects that were found go against the authors' expectations. However, alpha power and the authors' expectations about it are not mentioned in the introduction. 

We thank the reviewer for this comment. We have added a paragraph on alpha in the introduction (pg.4).

Minor points:

1. At the end of 1st page of introduction, the authors state that: 

“How children allocate their attention in experimenter-controlled, screen-based lab tasks differs, however, from actual real-world attention in several ways (32-34). For example, the real-world is interactive and manipulable, and so how we interact with the world determines what information we, in turn, receive from it: experiences generate behaviours (35).”

I think there's more to this though - Lab-based studies can be made interactive too (e.g., Meyer et al., 2023, Stahl & Feigenson, 2015). What remains unexplored is how infants actively and freely initiate and self-structure their attention, rather than how they respond to experimental manipulations.

Meyer, M., van Schaik, J. E., Poli, F., & Hunnius, S. (2023). How infant‐directed actions enhance infants' attention, learning, and exploration: Evidence from EEG and computational modeling. Developmental Science, 26(1), e13259.

Stahl, A. E., & Feigenson, L. (2015). Observing the unexpected enhances infants' learning and exploration. Science, 348(6230), 91-94.

We thank the reviewer for this suggestion and added their point in pg. 4.

(2) Regarding analysis 4:

a. In analysis 1 you showed that the duration of attentional episodes changes with age. Is it fair to keep the same start, middle, and termination ranges across age groups? Is 3-4 seconds "middle" for 5-month-olds? 

We appreciate the comment. There are many ways we could have run these analyses and, in fact, in other papers we have done it differently, for example by splitting each look in 3, irrespective of its duration (Phillips et al., 2023).

However, one aspect we took into account was the observation that 5-month-old infants exhibited more shorter looks compared to older infants. We recognized that dividing each into 3 parts, regardless of its duration, might have impacted the results. Presumably, the activity during the middle and termination phases of a 1.5-second look differs from that of a look lasting over 7 seconds.

Two additional factors that provided us with confidence in our approach were: 1) while the definition of "middle" was somewhat arbitrary, it allowed us to maintain consistency in our analyses across different age points. And, 2) we obtained a comparable amount of observations across the two time points (e.g. “middle” at 5 months we had 172 events at 5 months, and 194 events at 10 months).

b. It is recommended not to interpret lower-level interactions if more complex interactions are not significant. How are the interaction effects in a simpler model in which the 3-way interaction is removed? 

We appreciate the comment. We tried to follow the same steps as in (Xie et al., 2018). However, we have re-analysed the data removing the 3-way interaction and the significance of the results stayed the same. Please see Author response image 2 below (first: new analyses without the 3-way interactions, second: original analyses that included the 3-way interaction).

Author response image 2.

(3) Figure S1: there seems to be an outlier in the bottom-right panel. Do results hold excluding it? 

We re-run these analyses as per this suggestion and the results stayed the same (refer to SM pg. 2).

(4) Figure S2 should refer to 10 months instead of 12.

We thank the reviewer for noticing this typo, we have changed it in the reviewed manuscript (see SM pg. 3). 

(5) In the 2nd paragraph of the discussion, I found this sentence unclear: "From Analysis 1 we found that infants at both ages showed a preferred modal reorientation rate". 

We clarified this in the reviewed manuscript in pg10

(6) Discussion: many (infant) studies have used theta in anticipation of receiving information (Begus et al., 2016) surprising events (Meyer et al., 2023), and especially exploration (Begus et al., 2015). Can you make a broader point on how these findings inform our interpretation of theta in the infant population (go more from description to underlying mechanisms)? 

We have extended on this point on interpreting frequency bands in pg13 of the reviewed manuscript and thank the reviewer for bringing it up.

Begus, K., Gliga, T., & Southgate, V. (2016). Infants' preferences for native speakers are associated with an expectation of information. Proceedings of the National Academy of Sciences, 113(44), 12397-12402.

Meyer, M., van Schaik, J. E., Poli, F., & Hunnius, S. (2023). How infant‐directed actions enhance infants' attention, learning, and exploration: Evidence from EEG and computational modeling. Developmental Science, 26(1), e13259.

Begus, K., Southgate, V., & Gliga, T. (2015). Neural mechanisms of infant learning: differences in frontal theta activity during object exploration modulate subsequent object recognition. Biology letters, 11(5), 20150041.

(7) 2nd page of discussion, last paragraph: "preferred modal reorientation timer" is not a neural/cognitive mechanism, just a resulting behaviour. 

We agree with this comment and thank the reviewer for bringing it out to our attention. We clarified this in in pg12 and pg13 of the reviewed manuscript.

Reviewer #2 (Recommendations For The Authors):

I have a few comments and questions that I think the authors should consider addressing in a revised version. Please see below:

(1) During preprocessing (steps 5 and 6), it seems like the "noisy channels" were rejected using the pop_rejchan.m function and then interpolated. This procedure is common in infant EEG analysis, but a concern arises: was there no upper limit for channel interpolation? Did the authors still perform bad channel interpolation even when more than 30% or 40% of the channels were identified as "bad" at the beginning with the continuous data? 

We did state in the original manuscript that “participants with fewer than 30% channels interpolated at 5 months and 25% at 10 months made it to the final step (ICA) and final analyses”. In the revised version we have re-written this section in order to make this more clear (pg. 17).

(2) I am also perplexed about the sequencing of the ICA pruning step. If the intention of ICA pruning is to eliminate artificial components, would it be more logical to perform this procedure before the conventional artifacts' rejection (i.e., step 7), rather than after? In addition, what was the methodology employed by the authors to identify the artificial ICA components? Was it done through manual visual inspection or utilizing specific toolboxes? 

We agree that the ICA is often run before, however, the decision to reject continuous data prior to ICA was to remove the very worst sections of data (where almost all channels were affected), which can arise during times when infants fuss or pull the caps. Thus, this step was applied at this point in the pipeline so that these sections of really bad data were not inputted into the ICA. This is fairly widespread practice in cleaning infant data.

Concerning the reviewer’s second question, of how ICA components were removed – the answer to this is described in considerable detail in the paper that we refer to in that setion of the manuscript. This was done by training a classifier specially designed to clean naturalistic infant EEG data (Haresign et al., 2021) and has since been employed in similar studies (e.g. Georgieva et al., 2020; Phillips et al., 2023).

(3) Please clarify how the relative power was calculated for the theta (3-6Hz) and alpha (6-9Hz) bands. Were they calculated by dividing the ratio of theta or alpha power to the power between 3 and 9Hz, or the total power between 1 (or 3) and 20 Hz? In other words, what does the term "all frequency bands" refer to in section 4.3.7? 

We thank the reviewer for this comment, we have now clarified this in pg. 22.

(4) One of the key discoveries presented in this paper is the observation that attention shifts are accompanied by a subsequent enhancement in theta band power shortly after the shifts occur. Is it possible that this effect or alteration might be linked to infants' saccades, which are used as indicators of attention shifts? Would it be feasible to analyze the disparities in amplitude between the left and right frontal electrodes (e.g., Fp1 and Fp2, which could be viewed as virtual horizontal EOG channels) in relation to theta band power, in order to eliminate the possibility that the augmentation of theta power was attributable to the intensity of the saccades? 

We appreciate the concern. Average saccade duration in infants is about 40ms (Garbutt et al., 2007). Our finding that the positive cross-correlation between theta and look duration is present not only when we examine zero-lag data but also when we examine how theta forwards-predicts attention 1-2 seconds afterwards seems therefore unlikely to be directly attributable to saccade-related artifact. Concerning the reviewer’s suggestion – this is something that we have tried in the past. Unfortunately, however, our experience is that identifying saccades based on the disparity between Fp1 and Fp2 is much too unreliable to be of any use in analysing data. Even if specially positioned HEOG electrodes are used, we still find the saccade detection to be insufficiently reliable. In ongoing work we are tracking eye movements separately, in order to be able to address this point more satisfactorily.

(5) The following question is related to my previous comment. Why is the duration of the relationship between theta power and moment-to-moment changes in attention so short? If theta is indeed associated with attention and information processing, shouldn't the relationship between the two variables strengthen as the attention episode progresses? Given that the authors themselves suggest that "One possible interpretation of this is that neural activity associates with the maintenance more than the initiation of attentional behaviors," it raises the question of (is in contradiction to) why the duration of the relationship is not longer but declines drastically (Figure 6). 

We thank the reviewer for raising this excellent point. Certainly we argue that this, together with the low autocorrelation values for theta documented in Fig 7A and 7D challenge many conventional ways of interpreting theta. We are continuing to investigate this question in ongoing work.

(6) Have the authors conducted a comparison of alpha relative power and HR deceleration durations between 5 and 10-month-old infants? This analysis could provide insights into whether the differences observed between the two age groups were partly due to varying levels of general arousal and engagement during free play.

We thank the reviewer for this suggestion. Indeed, this is an aspect we investigated but ultimately, given that our primary emphasis was on the theta frequency, and considering the length of the manuscript, we decided not to incorporate. However, we attached Author response image 3 below showing there was no significant interaction between HR and alpha band.

Author response image 3.

Reviewer #3 (Recommendations For The Authors):

(1) In reading the manuscript, the language used seems to imply longitudinal data or at the very least the ability to detect change or maturation. Given the cross-sectional nature of the data, the language should be tempered throughout. The data are illustrative but not definitive. 

We thank the reviewer for this comment. We have now clarified that “Data was analysed in a cross-sectional manner” in pg15.

(2) The sample size is quite modest, particularly in the specific age groups. This is likely tempered by the sheer number of data points available. This latter argument is implied in the text, but not as explicitly noted. (However, I may have missed this as the text is quite dense). I think more notice is needed on the reliability and stability of the findings given the sample. 

We have clarified this in pg16.

(3) On a related note, how was the sample size determined? Was there a power analysis to help guide decision-making for both recruitment and choosing which analyses to proceed with? Again, the analytic approach is quite sophisticated and the questions are of central interest to researchers, but I was left feeling maybe these two aspects of the study were out-sprinting the available data. The general impression is that the sample is small, but it is not until looking at table s7, that it is in full relief. I think this should be more prominent in the main body of the study.

We have clarified this in pg16.

(4) The devotes a few sentences to the relation between looking and attention. However, this distinction is central to the design of the study, and any philosophical differences regarding what take-away points can be generated. In my reading, I think this point needs to be more heavily interrogated. 

This distinction between looking and paying attention is clearer now in the reviewed manuscript as per R1 and R3’s suggestions. We have also added a paragraph in the Introduction to clarify it and pointed out its limitations (see pg.5).

(5) I would temper the real-world attention language. This study is certainly a great step forward, relative to static faces on a computer screen. However, there are still a great number of artificial constraints that have been added. That is not to say that the constraints are bad--they are necessary to carry out the work. However, it should be acknowledged that it constrains the external validity. 

We have added a paragraph to acknowledged limitations of the setup in pg. 14.

(6) The kappa on the coding is not strong. The authors chose to proceed nonetheless. Given that, I think more information is needed on how coders were trained, how they were standardized, and what parameters were used to decide they were ready to code independently. Again, with the sample size and the kappa presented, I think more discussion is needed regarding the robustness of the findings. 

We appreciate the concern. As per our answer to R1, we chose to report the most stringent calculator of inter-rater reliability, but other calculation methods (i.e., percent agreement) return higher scores (see response to R1).

As per the training, we wrote an extensively detailed coding scheme describing exactly how to code each look that was handed to our coders. Throughout the initial months of training, we meet with the coders on a weekly basis to discuss questions and individual frames that looked ambiguous. After each session, we would revise the coding scheme to incorporate additional details, aiming to make the coding process progressively less subjective. During this period, every coder analysed the same interactions, and inter-rater reliability (IRR) was assessed weekly, comparing their evaluations with mine (Marta). With time, the coders had fewer questions and IRR increased. At that point, we deemed them sufficiently trained, and began assigning them different interactions from each other. Periodically, though, we all assessed the same interaction and meet to review and discuss our coding outputs.

Figure 1 should be viewed full-size, in order to understand exactly how codes can be arranged into different categories, as the basis for developing descriptive themes.

The code in this case is "bad food=nice, good food=awful"

Yeah, this course was really good at introducing me to coding concepts, as a person who has never coded before. I am able to now understand most basic code, and edit accordingly. It also has peaked my interest and I may try to learn to code more in the summer!

-> "Restart VSCode in the r-dev-env directory with the command: "

Author response:

The following is the authors’ response to the original reviews.

eLife assessment

In this potentially useful study, the authors attempt to use comparative meta-analysis to advance our understanding of life history evolution. Unfortunately, both the meta-analysis and the theoretical model is inadequate and proper statistical and mechanistic descriptions of the simulations are lacking. Specifically, the interpretation overlooks the effect of well-characterised complexities in the relationship between clutch size and fitness in birds.

Public Reviews:

We would like to thank the reviewers for their helpful comments, which have been considered carefully and have been valuable in progressing our manuscript. The following bullet points summarise the key points and our responses, though our detailed responses to specific comments can be found below:<br /> - Two reviewers commented that our data was not made available. Our data was provided upon submission and during the review process, however was not made accessible to the reviewers. Our data and code are available at https://doi.org/10.5061/dryad.q83bk3jnk.

- The reviewers have highlighted that some of our methodology was unclear and we have added all the requested detail to ensure our methods can be easily understood.

- The reviewers highlight the importance of our conclusions, but also suggest some interpretations might be missing and/or are incomplete. To make clear how we objectively interpreted our data and the wider consequences for life-history theory we provide a decision tree (Figure 5). This figure makes clear where we think the boundaries are in our interpretation and how multiple lines of evidence converge to the same conclusions.

Reviewer #1 (Public Review):

This paper falls in a long tradition of studies on the costs of reproduction in birds and its contribution to understanding individual variation in life histories. Unfortunately, the meta-analyses only confirm what we know already, and the simulations based on the outcome of the meta-analysis have shortcomings that prevent the inferences on optimal clutch size, in contrast to the claims made in the paper.

There was no information that I could find on the effect sizes used in the meta-analyses other than a figure listing the species included. In fact, there is more information on studies that were not included. This made it impossible to evaluate the data-set. This is a serious omission, because it is not uncommon for there to be serious errors in meta-analysis data sets. Moreover, in the long run the main contribution of a meta-analysis is to build a data set that can be included in further studies.

It is disappointing that two referees comment on data availability, as we supplied a link to our full dataset and the code we used in Dryad with our submitted manuscript. We were also asked to supply our data during the review process and we again supplied a link to our dataset and code, along with a folder containing the data and code itself. We received confirmation that the reviewers had been given our data and code. We support open science and it was our intention that our dataset should be fully available to reviewers and readers. Our data and code are at https://doi.org/10.5061/dryad.q83bk3jnk.

The main finding of the meta-analysis of the brood size manipulation studies is that the survival costs of enlarging brood size are modest, as previously reported by Santos & Nakagawa on what I suspect to be mostly the same data set.

We disagree that the main finding of our paper is the small survival cost of manipulated brood size. The major finding of the paper, in our opinion, is that the effect sizes for experimental and observational studies are in opposite directions, therefore providing the first quantitative evidence to support the influential theoretical framework put forward by van Noordwijk and de Jong (1986), that individuals differ in their optimal clutch size and are constrained to reproducing at this level due to a trade-off with survival. We further show that while the manipulation experiments have been widely accepted to be informative, they are not in fact an effective test of whether within-species variation in clutch size is the result of a trade-off between reproduction and survival.

The comment that we are reporting the same finding as Santos & Nakagawa (2012) is a misrepresentation of both that study and our own. Santos & Nakagawa found an effect of parental effort on survival only in males who had their clutch size increased – but no effect for males who had their clutch size reduced and no survival effect on females for either increasing or reducing parental effort. However, we found an overall reduction in survival for birds who had brood sizes manipulated to be larger than their original brood (for both sexes and mixed sex studies combined). In our supplementary information, we demonstrate that the overall survival effect of a change in reproductive effort is close to zero for males, negative (though non-significant) for females and significantly negative for mixed sexes (which are not included in the Santos & Nakagawa study). Please also note that the Santos & Nakagawa study was conducted over 10 years ago. This means we added additional data (L364-365). Furthermore, meta-analyses are an evolving practice and we also corrected and improved on the overall analysis approach (e.g. L358-359 and L 393-397, and see detailed SI).

The paper does a very poor job of critically discussing whether we should take this at face value or whether instead there may be short-comings in the general experimental approach. A major reason why survival cost estimates are barely significantly different from zero may well be that parents do not fully adjust their parental effort to the manipulated brood size, either because of time/energy constraints, because it is too costly and therefore not optimal, or because parents do not register increased offspring needs. Whatever the reason, as a consequence, there is usually a strong effect of brood size manipulation on offspring growth and thereby presumably their fitness prospects. In the simulations (Fig.4), the consequences of the survival costs of reproduction for optimal clutch size were investigated without considering brood size manipulation effects on the offspring. Effects on offspring are briefly acknowledged in the discussion, but otherwise ignored. Assuming that the survival costs of reproduction are indeed difficult to discern because the offspring bear the brunt of the increase in brood size, a simulation that ignores the latter effect is unlikely to yield any insight in optimal clutch size. It is not clear therefore what we learn from these calculations.

The reviewer’s comment is somewhat of a paradox. We take the best studied example of the trade-off between reproductive effort and parental survival – a key theme in life history and the biology of ageing – and subject this to a meta-analysis. The reviewer suggests we should interpret our finding as if there must be something wrong with the method or studies we included, rather than considering that the original hypothesis could be false or inflated in importance. We do not consider questioning the premise of the data over questioning a favoured hypothesis to necessarily be the best scientific approach here. In many places in our manuscript, we question and address, at length, the underlying data and their interpretation (L116-117, L165-167, 202-204 and L277-282). Moreover, we make it clear that we focus on the trade-off between current reproductive effort and subsequent parental survival, while being aware that other trade-offs could counter-balance or explain our findings (discussed on L208-210 & L301-316). Note that it is also problematic, when you do not find the expected response, to search for an alternative that has not been measured. In the case here, of potential trade-offs, there are endless possibilities of where a trade-off might operate between traits. We purposefully focus on the one well-studied and most commonly invoked trade-off. We clearly acknowledge, though, that when all possible trade-offs are taken into account a trade-off on the fitness level can occur and cite two famous studies (Daan et al., 1990 and Verhulst & Tinbergen 1991) that have shown just that (L314-316).

So whilst we agree with the reviewer that the offspring may incur costs themselves, rather than costs being incurred by the parents, the aim of our study was to test for a general trend across species in the survival costs of reproductive effort. It is unrealistic to suggest that incorporating offspring growth into our simulations would add insight, as a change in offspring number rarely affects all offspring in the nest equally and there can even be quite stark differences; for example, this will be most evident in species that produce sacrificial offspring. This effect will be further confounded by catch-up growth, for example, and so it is likely that increased sibling competition from added chicks alters offspring growth trajectories, rather than absolute growth as the reviewer suggests. There are mixed results in the literature on the effect of altering clutch size on offspring survival, with an increased clutch size through manipulation often increasing the number of recruits from a nest.

What we do appreciate from the reviewer’s comment is that the interpretation of our findings is complex. Even though our in-text explanation includes the caveats the reviewer refers to, and are discussed at length, their inter-relationships are hard to appreciate from a text format. To improve this presentation and for ease of the reader, we have added a decision tree (Figure 5) which represents the logical flow from the hypothesis being tested through to what overall conclusion can be drawn from our results. We believe this clarifies what conclusions can be drawn from our results. We emphasise again that the theory that trade-offs between reproductive effort and parental survival being the major driver of variation in offspring production was not supported though is the one that practitioners in the field would be most likely to invoke, and our result is important for this reason.

There are other reasons why brood size manipulations may not reveal the costs of reproduction animals would incur when opting for a larger brood size than they produced spontaneously themselves. Firstly, the manipulations do not affect the effort incurred in laying eggs (which also biases your comparison with natural variation in clutch size). Secondly, the studies by Boonekamp et al on Jackdaws found that while there was no effect of brood size manipulation on parental survival after one year of manipulation, there was a strong effect when the same individuals were manipulated in the same direction in multiple years. This could be taken to mean that costs are not immediate but delayed, explaining why single year manipulations generally show little effect on survival. It would also mean that most estimates of the fitness costs of manipulated brood size are not fit for purpose, because typically restricted to survival over a single year.

First, our results did show a survival cost of reproduction for brood manipulations (L107-123, Figure 1, Table 1). Note, however, that much theory is built on the immediate costs of reproduction and, as such, these costs are likely overinterpreted, meaning that our overall interpretation still holds, i.e. “parental survival trade-off is not the major determinative trade-off in life history within-species” (Figure 5).

We agree with the reviewer that lifetime manipulations could be even more informative than single-year manipulations. Unfortunately, there are currently too few studies available to be able to draw generalisable conclusions across species for lifetime manipulations. This is, however, the reason we used lifetime change in clutch size in our fitness projections, which the reviewer seems to have missed – please see methods line 466-468, where we explicitly state that this is lifetime enlargement. Of course, such interpretations do not include an accumulation of costs that is greater than the annual cost, but currently there is no clear evidence that such an assumption is valid. Such a conclusion can also not be drawn from the study on jackdaws by Boonekamp et al (2014) as the treatments were life-long and, therefore, cannot separate annual from accrued (multiplicative) costs that are more than the sum of the annual costs incurred. Note that we have now included specific discussion of this study in response to the reviewer (L265-269).

Details of how the analyses were carried out were opaque in places, but as I understood the analysis of the brood size manipulation studies, manipulation was coded as a covariate, with negative values for brood size reductions and positive values for brood size enlargements (and then variably scaled or not to control brood or clutch size). This approach implicitly assumes that the trade-off between current brood size (manipulation) and parental survival is linear, which contrasts with the general expectation that this trade-off is not linear. This assumption reduces the value of the analysis, and contrasts with the approach of Santos & Nakagawa.

We thank the reviewer for highlighting a lack of clarity in places in our methods. We have added additional detail to the methodology section (see “Study sourcing & inclusion criteria” and “Extracting effect sizes”) in our revised manuscript. Note, that our data and code was not shared with the reviewers despite us supplying this upon submission and again during the review process, which would have explained a lot more of the detail required.

For clarity in our response, each effect size was extracted by performing a logistic regression with survival as a binary response variable and clutch size was the absolute value of offspring in the nest (i.e., for a bird that laid a clutch size of 5 but was manipulated to have -1 egg, we used a clutch size value of 4). The clutch size was also standardised and, separately, expressed as a proportion of the species’ mean.

We disagree that our approach reduces the value of our analysis. First, our approach allows a direct comparison between experimental and observational studies, which is the novelty of our study. Our approach does differ from Santos & Nakagawa but we disagree that it contrasts. Our approach allows us to take into consideration the severity of the change in clutch size, which Santos & Nakagawa do not. Therefore, we do not agree that our approach is worse at accounting for non-linearity of trade-offs than the approach used by Santos & Nakagawa. Arguably, the approach by Santos & Nakagawa is worse, as they dichotomise effort as increased or decreased, factorise their output and thereby inflate their number of outcomes, of which only 1 cell of 4 categories is significant (for males and females, increased and decreased brood size). The proof is in the pudding as well, as our results clearly demonstrate that the magnitude of the manipulation is a key factor driving the results, i.e. one offspring for a seabird is a larger proportion of care (and fitness) than one offspring for a passerine. Such insights were not achieved by Santos & Nakagawa’s method and, again, did not allow a direct quantitative comparison between quality (correlational) and experimental (brood size manipulation, i.e. “trade-off”) effects, which forms a central part of our argumentation (Figure 5). 

Our analysis, alongside a plethora of other ecological studies, does assume that the response to our predictor variable is linear. However, it is common knowledge that there are very few (if any) truly linear relationships. We use linear relationships because they serve a good approximation of the trend and provide a more rigorous test for an underlying relationship than would fitting nonlinear models. For many datasets the range of added chicks required to estimate a non-linear relationship was not available. The question also remains of what the shape of such a non-linear relationship should be and is hard to determine a priori. There is also a real risk when fitting non-linear terms that they are spurious and overinterpreted, as they often present a better fit (denoting one df is not sufficient especially when slopes vary). We have added this detail to our discussion.

The observational study selection is not complete and apparently no attempt was made to make it complete. This is a missed opportunity - it would be interesting to learn more about interspecific variation in the association between natural variation in clutch size and parental survival.

We clearly state in our manuscript that we deliberately tailored the selection of studies to match the manipulation studies (L367-369). We paired species extracted for observational studies with those extracted in experimental studies to facilitate a direct comparison between observational and experimental studies, and to ensure that the respective datasets were comparable. The reviewer’s focus in this review seems to be solely on the experimental dataset. This comment dismisses the equally important observational component of our analysis and thereby fails to acknowledge one of the key questions being addressed in this study. Note that in our revised version we have edited the phylogenetic tree to indicate for which species we have both types of information, which highlights our approach to selecting observational data (Figure 3).

Reviewer #2 (Public Review):

I have read with great interest the manuscript entitled "The optimal clutch size revisited: separating individual quality from the costs of reproduction" by LA Winder and colleagues. The paper consists in a meta-analysis comparing survival rates from studies providing clutch sizes of species that are unmanipulated and from studies where the clutch sizes are manipulated, in order to better understand the effects of differences in individual quality and of the costs of reproduction. I find the idea of the manuscript very interesting. However, I am not sure the methodology used allows to reach the conclusions provided by the authors (mainly that there is no cost of reproduction, and that the entire variation in clutch size among individuals of a population is driven by "individual quality").

We would like to highlight that we do not conclude that there is no cost of reproduction. Please see lines 336–339, where we state that our lack of evidence for trade-offs driving within-species variation in clutch size does not necessarily mean the costs of reproduction are non-existent. We conclude that individuals are constrained to their optima by the survival cost of reproduction. It is also an over-statement of our conclusion to say that we believe that variation in clutch size is only driven by quality. Our results show that unmanipulated birds that have larger clutch sizes also lived longer, and we suggest that this is evidence that some individuals are “better” than others, but we do not say, nor imply, that no other factors affect variation in clutch size. We have added Figure 5 to our manuscript to help the reader better understand what questions we can answer with our study and what conclusions we can draw from our results.

I write that I am not sure, because in its current form, the manuscript does not contain a single equation, making it impossible to assess. It would need at least a set of mathematical descriptions for the statistical analysis and for the mechanistic model that the authors infer from it.

We appreciate this comment, and have explained our methods in terms that are accessible to a wider audience. Note, however, that our meta-analysis is standard and based on logistic regression and standard meta-analytic practices. We have added the model formula to the model output tables.

For the simulation, we simply simulated the resulting effects. We of course supplied our code for this along with our manuscript (https://doi.org/10.5061/dryad.q83bk3jnk), though as we mentioned above, we believe this was not shared with the reviewers despite us making this available for the review process. We therefore understand why the reviewer feels the simulations were not explained thoroughly. We have revised our methods section and added details which we believe make our methodology more clear without needing to consult the supplemental material. However, we have also added the equations used in the process of calculating our simulated data to the Supplementary Information for readers who wish to have this information in equation form.

The texts mixes concepts of individual vs population statistics, of within individual vs among-individuals measures, of allocation trade-offs and fitness trade-offs, etc ....which means it would also require a glossary of the definitions the authors use for these various terms, in order to be evaluated.

We would like to thank the reviewer for highlighting this lack of clarity in our text. Throughout the manuscript we have refined our terminology and indicated where we are referring to the individual level or the population level. The inclusion of our new Figure 5 (decision tree) should also help in this context, as it is clear on which level we base our interpretation and conclusions on.

This problem is emphasised by the following sentence to be found in the discussion "The effect of birds having naturally larger clutches was significantly opposite to the result of increasing clutch size through brood manipulation". The "effect" is defined as the survival rate (see Fig 1). While it is relatively easy to intuitively understand what the "effect" is for the unmanipulated studies: the sensitivity of survival to clutch size at the population level, this should be mentioned and detailed in a formula. Moreover, the concept of effect size is not at all obvious for the manipulated ones (effect of the manipulation? or survival rate whatever the manipulation (then how could it measure a trade-off ?)? at the population level? at the individual level ?) despite a whole appendix dedicated to it. This absolutely needs to be described properly in the manuscript.

Thank you for identifying this sentence for which the writing was ambiguous, our apologies. We have now rewritten this and included additional explanation. L282-290: ‘The effect on parental annual survival of having naturally larger clutches was significantly opposite to the result of increasing clutch size through brood manipulation, and quantitatively similar. Parents with naturally larger clutches are thus expected to live longer and this counterbalances the “cost of reproduction” when their brood size is experimentally manipulated. It is, therefore, possible that quality effects mask trade-offs. Furthermore, it could be possible that individuals that lay larger clutches have smaller costs of reproduction, i.e. would respond less in terms of annual survival to a brood size manipulation, but with our current dataset we cannot address this hypothesis (Figure 5).’

We would also like to thank the reviewer for bringing to our attention the lack of clarity about the details of our methodology. We have added details to our methodology (see “Extracting effect sizes” section) to address this (see highlighted sections). For clarity, the effect size for both manipulated and unmanipulated nests was survival, given the brood size raised. We performed a logistic regression with survival as a binary response variable (i.e., number of individuals that survived and number of individuals that died after each breeding season), and clutch size was the absolute value of offspring in the nest (i.e., for a bird that laid a clutch size of 5 but was manipulated to have -1 egg, we used a clutch size value of 4). This allows for direct comparison of the effect size (survival given clutch size raised) between manipulated and unmanipulated birds.

Despite the lack of information about the underlying mechanistic model tested and the statistical model used, my impression is still that the interpretation in the introduction and discussion is not granted by the outputs of the figures and tables. Let's use a model similar to that of (van Noordwijk and de Jong, 1986): imagine that the mechanism at the population level is

a.c_(i,q)+b.s_(i,q)=E_q

Where c_(i,q) are s_(i,q) are respectively the clutch size for individual i which is of quality q, and E_q is the level of "energy" that an individual of quality q has available during the given time-step (and a and b are constants turning the clutch size and survival rate into energy cost of reproduction and energy cost of survival, and there are both quite "high" so that an extra egg (c_(i,q) is increased by 1) at the current time-step, decreases s_(i,q) markedly (E_q is independent of the number of eggs produced), that is, we have strong individual costs of reproduction). Imagine now that the variance of c_(i,q) (when the population is not manipulated) among individuals of the same quality group, is very small (and therefore the variance of s_(i,q) is very small also) and that the expectation of both are proportional to E_q. Then, in the unmanipulated population, the variance in clutch size is mainly due to the variance in quality. And therefore, the larger the clutch size c_(i,q) the higher E_q, and the higher the survival s_(i,q).

In the manipulated populations however, because of the large a and b, an artificial increase in clutch size, for a given E_q, will lead to a lower survival s_(i,q). And the "effect size" at the population level may vary according to a,b and the variances mentioned above. In other words, the costs of reproduction may be strong, but be hidden by the data, when there is variance in quality; however there are actually strong costs of reproduction (so strong actually that they are deterministic and that the probability to survive is a direct function of the number of eggs produced)

We would like to thank the reviewer for these comments. We have added detail to our methodology section so our models and rationale are more clear. Please note that our simulations only take the experimental effect of brood size on parental survival into account. Our model does not incorporate quality effects. The reviewer is right that the relationship between quality and the effects exposed by manipulating brood size can take many forms and this is a very interesting topic, but not one we aimed to tackle in our manuscript. In terms of quality we make two points: (1) overall quality effects connecting reproduction and parental survival are present, (2) these effects are opposite in direction to the effects when reproduction is manipulated and similar in magnitude. We do not go further than that in interpreting our results. The reviewer is correct, however, that we do suggest and repeat suggestions by others that quality can also mask the trade-off in some individuals or circumstances (L74-76, L95-98 & L286-289), but we do not quantify this, as it is dependent on the unknown relationship between quality and the response to the manipulation. A focussed set of experiments in that context would be interesting and there are some data that could get at this, i.e. the relationship between produced clutch size and the relative effect of the manipulation (now included L287-290). Such information is, however, not available for all studies and, although we explored the possibility of analysing this, currently this is not possible with adequate confidence and there is the possible complexity of non-linear effects. We have added this rationale in our revision (L259-265).

Moreover, it seems to me that the costs of reproduction are a concept closely related to generation time. Looking beyond the individual allocative (and other individual components of the trade-off) cost of reproduction and towards a populational negative relationship between survival and reproduction, we have to consider the intra-population slow fast continuum (some types of individuals survive more and reproduce less (are slower) than other (which are faster)). This continuum is associated with a metric: the generation time. Some individuals will produce more eggs and survive less in a given time-period because this time-period corresponds to a higher ratio of their generation time (Gaillard and Yoccoz, 2003; Gaillard et al., 2005). It seems therefore important to me, to control for generation time and in general to account for the time-step used for each population studied when analysing costs of reproduction. The data used in this manuscript is not just clutch size and survival rates, but clutch size per year (or another time step) and annual (or other) survival rates.

The reviewer is right that this is interesting. There is a longstanding unexplained difference in temperate (seasonal) and tropical reproductive strategies. Most of our data come from seasonal breeders, however. Although there is some variation in second brooding and such, these species mostly only produce one brood. We do agree that a wider consideration here is relevant, but we are not trying to explain all of life history in our paper. It is clearly the case that other factors will operate and the opportunity for trade-offs will vary among species according to their respective life histories. However, our study focuses on the two most fundamental components of fitness – longevity and reproduction – to test a major hypothesis in the field, and we uncover new relationships that contrast with previous influential studies and cast doubt on previous conclusions. We question the assumed trade-off between reproduction and annual survival. We show that quality is important and that the effect we find in experimental studies is so small that it can only explain between-species patterns but is unlikely to be the selective force that constrains reproduction within species. We do agree that there is a lot more work that can be done in this area. We hope we are contributing to the field, by questioning this central trade-off. We have incorporated some of the reviewers suggestions in the revision (L309-315). We have added Figure 5 to make clear where we are able to reach solid conclusions and the evidence on which these are based as clearly as possible in an easily accessible format.

Finally, it is important to relate any study of the costs of reproduction in a context of individual heterogeneity (in quality for instance), to the general problem of the detection of effects of individual differences on survival (see, e.g., Fay et al., 2021). Without an understanding of the very particular statistical behaviour of survival, associated to an event that by definition occurs only once per life history trajectory (by contrast to many other traits, even demographic, where the corresponding event (production of eggs for reproduction, for example) can be measured several times for a given individual during its life history trajectory).

Thank you for raising this point. The reviewer is right that heterogeneity can dampen or augment selection. Note that by estimating the effect of quality here we give an example of how heterogeneity can possibly do exactly this. We thank the reviewer for raising that we should possibly link this to wider effects of heterogeneity and we have added to our discussion of how our results play into the importance of accounting for among-individual heterogeneity (L252-256).

References:

Fay, R. et al. (2021) 'Quantifying fixed individual heterogeneity in demographic parameters: Performance of correlated random effects for Bernoulli variables', Methods in Ecology and Evolution, 2021(August), pp. 1-14. doi: 10.1111/2041-210x.13728.

Gaillard, J.-M. et al. (2005) 'Generation time: a reliable metric to measure life-history variation among mammalian populations.', The American naturalist, 166(1), pp. 119-123; discussion 124-128. doi: 10.1086/430330.

Gaillard, J.-M. and Yoccoz, N. G. (2003) 'Temporal Variation in Survival of Mammals: a Case of Environmental Canalization?', Ecology, 84(12), pp. 3294-3306. doi: 10.1890/02-0409.

van Noordwijk, A. J. and de Jong, G. (1986) 'Acquisition and Allocation of Resources: Their Influence on Variation in Life History Tactics', American Naturalist, p. 137. doi: 10.1086/284547.

Reviewer #3 (Public Review):

The authors present here a comparative meta-analysis analysis designed to detect evidence for a reproduction/ survival trade-off, central to expectations from life history theory. They present variation in clutch size within species as an observation in conflict with expectations of optimisation of clutch size and suggest that this may be accounted for from weak selection on clutch size. The results of their analyses support this explanation - they found little evidence of a reproduction - survival trade-off across birds. They extrapolated from this result to show in a mathematical model that the fitness consequences of enlarged clutch sizes would only be expected to have a significant effect on fitness in extreme cases, outside of normal species' clutch size ranges. Given the centrality of the reproduction-survival trade-off, the authors suggest that this result should encourage us to take a more cautious approach to applying concepts the trade-off in life history theory and optimisation in behavioural ecology more generally. While many of the findings are interesting, I don't think the argument for a major re-think of life history theory and the role of trade-offs in fitness maximisation is justified.

The interest of the paper, for me, comes from highlighting the complexities of the link between clutch size and fitness, and the challenges facing biologists who want to detect evidence for life history trade-offs. Their results highlight apparently contradictory results from observational and experimental studies on the reproduction-survival trade-off and show that species with smaller clutch sizes are under stronger selection to limit clutch size.

Unfortunately, the authors interpret the failure to detect a life history trade-off as evidence that there isn't one. The construction of a mathematical model based on this interpretation serves to give this possible conclusion perhaps more weight than is merited on the basis of the results, of this necessarily quite simple, meta-analysis. There are several potential complicating factors that could explain the lack of detection of a trade-off in these studies, which are mentioned and dismissed as unimportant (lines 248-250) without any helpful, rigorous discussion. I list below just a selection of complexities which perhaps deserve more careful consideration by the authors to help readers understand the implications of their results:

We would like to thank the reviewer for their thoughtful response and summary of the findings that we also agree are central to our study. The reviewer also highlights areas where our manuscript could benefit from a deeper consideration and we have added detail accordingly to our revised discussion.

We would like to highlight that we do not interpret the failure to detect a trade-off as evidence that there is not one. First, and importantly, we do find a trade-off but show this is only incurred when individuals produce a clutch beyond their optimal level. Second, we also state on lines 322-326 that the lack of evidence to support trade-offs being strong enough to drive variation in clutch size does not necessarily mean there are no costs of reproduction.

The statement that we have constructed a mathematical model based on the interpretation that we have not found a trade-off is, again, factually incorrect. We ran these simulations because the opposite is true – we did find a trade-off. There is a significant effect of clutch size when manipulated on annual parental survival. We benefit from our unique analysis allowing for a quantitative fitness estimate from the effect size on annual survival (as this is expressed on a per-egg basis). This allowed us to ask whether this quantitative effect size can alone explain why reproduction is constrained, and we evaluate this using simulations. From these simulations we find that this effect size is too small to explain the constraint, so something else must be going on, and we do spend a considerable amount of text discussing the possible explanations (L202-215). Note that the possibly most parsimonious conclusion here is that costs of reproduction are not there, or simply small, so we also give that explanation some thought (L221-224 and L315-331).

We are disappointed by the suggestion that we have dismissed complicating factors that could prevent detection of a trade-off, as this was not our intention. We were aiming to highlight that what we have demonstrated to be an apparent trade-off can be explained through other mechanisms, and that the trade-off between clutch size and survival is not as strong in driving within-species variation in clutch size as previously assumed. We have added further discussion to our revised manuscript to make this clear and give readers a better understanding of the complexity of factors associated with life-history theory, including the addition of a decision tree (Figure 5).

• Reproductive output is optimised for lifetime reproductive success and so the consequences of being pushed off the optimum for one breeding attempt are not necessarily detectable in survival but in future reproductive success (and, therefore, lifetime reproductive success).

We agree this is a valid point, which is mentioned in our manuscript in terms of alternative stages where the costs of reproduction might be manifested (L316-320). We would also like to highlight that , in our simulations, the change in clutch size (and subsequent survival cost) was assumed for the lifetime of the individual, for this very reason.

• The analyses include some species that hatch broods simultaneously and some that hatch sequentially (although this information is not explicitly provided (see below)). This is potentially relevant because species which have been favoured by selection to set up a size asymmetry among their broods often don't even try to raise their whole broods but only feed the biggest chicks until they are sated; any added chicks face a high probability of starvation. The first point this observation raises is that the expectation of more chicks= more cost, doesn't hold for all species. The second more general point is that the very existence of the sequential hatching strategy to produce size asymmetry in a brood is very difficult to explain if you reject the notion of a trade-off.

We agree with the reviewer that the costs of reproduction can be absorbed by the offspring themselves, and may not be equal across offspring (we also highlight this at L317-318 in the manuscript). However, we disagree that for some species the addition of more chicks does not equate to an increase in cost, though we do accept this might be less for some species. This is, however, difficult to incorporate into a sensible model as the impacts will vary among species and some species do also exhibit catch-up growth. So, without a priori knowledge on this, we kept our model simple to test whether the effect on parental survival (often assumed to be a strong cost) can explain the constraint on reproductive effort, and we conclude that it does not.

We would also like to make clear that we are not rejecting the notion of a trade-off. Our study shows evidence that a trade-off between survival and reproductive effort probably does not drive within-species variation in clutch size. We do explicitly say this throughout our manuscript, and also provide suggestions of other areas where a trade-off may exist (L317-320). The point of our study is not whether trade-offs exist or not, it is whether there is a generalisable across-species trend for a trade-off between reproductive effort and survival – the most fundamental trade-off in our field but for which there is a lack of conclusive evidence within species. We believe the addition of Figure 5 to our reviewed manuscript also makes this more evident.

• For your standard, pair-breeding passerine, there is an expectation that costs of raising chicks will increase linearly with clutch size. Each chick requires X feeding visits to reach the required fledge weight. But this is not the case for species which lay precocious chicks which are relatively independent and able to feed themselves straight after hatching - so again the relationship of care and survival is unlikely to be detectable by looking at the effect of clutch size but again, it doesn't mean there isn't a trade-off between breeding and survival.

Precocial birds still provide a level of parental care, such as protection from predators. Though we agree that the level of parental care in provisioning food (and in some cases in all parental care given) is lower in precocial than altricial birds, this would only make our reported effect size for manipulated birds to be an underestimate. Again, we would like to draw the reviewer’s attention to the fact we did detect a trade-off in manipulated birds and we do not suggest that trade-offs do not exist. The argument the reviewer suggests here does not hold for unmanipulated birds, as we found that birds that naturally lay larger clutch sizes have higher survival.

• The costs of raising a brood to adulthood for your standard pair-breeding passerine is bound to be extreme, simply by dint of the energy expenditure required. In fact, it was shown that the basal metabolic rate of breeding passerines was at the very edge of what is physiologically possible, the human equivalent being cycling the Tour de France (Nagy et al. 1990). If birds are at the very edge of what is physiologically possible, is it likely that clutch size is under weak selection?

If birds are at the very edge of what is physiologically possible, then indeed it would necessarily follow that if they increase the resource allocated in one area then expenditure in another area must be reduced. In many studies, however, the overall brood mass is increased when chicks are added and cared for in an experimental setting, suggesting that birds are not operating at their limit all the time. Our simulations show that if individuals increase their clutch size, the survival cost of reproduction counterbalances the fitness gained by increasing clutch size and so there is no overall fitness gain to producing more offspring. Therefore, selection on clutch size is constrained to the within-species level. We do not say in our manuscript that clutch size is under weak selection – we only ask why variation in clutch size is maintained if selection always favours high-producing birds.

• Variation in clutch size is presented by the authors as inconsistent with the assumption that birds are under selection to lay the Lack clutch. Of course, this is absurd and makes me think that I have misunderstood the authors' intended point here. At any rate, the paper would benefit from more clarity about how variable clutch size has to be before it becomes a problem for optimality in the authors' view (lines 84-85; line 246). See Perrins (1965) for an exquisite example of how beautifully great tits optimise clutch size on average, despite laying between 5-12 eggs.

We thank the reviewer for highlighting that our manuscript may be misleading in places, however, we are unsure which part of our conclusions the author is referring to here. The question we pose is “Why don’t all birds produce a clutch size at the population optimum?”, and is central to the decades-long field of life-history theory. Why is variation maintained? As the reviewer outlines, there is extensive variability, with some birds laying half of what other birds lay.

Recommendations for the authors:

Reviewer #1 (Recommendations For The Authors):

(1) Title: while the costs of reproduction are possibly important in shaping optimal clutch size, it is not clear what you can about it given that you do not consider clutch / brood size effects on fitness prospects of the offspring.

We have expanded on our discussion of how some costs may be absorbed by the offspring themselves. However, a change in offspring number rarely affects all offspring in the nest equally and there can even be quite stark differences; for example this will be most evident in species that produce sacrificial offspring. This effect will be further confounded by catch-up growth. There are mixed results in the literature on the effect of altering clutch size on offspring survival, with an increased clutch size through manipulation often increasing the number of recruits from a nest. We have focussed on the relationship between reproductive effort and survival because it is given the most weight in the field in terms of driving intra-specific variation in clutch size. We have altered our title to show we focus on the survival costs specifically: “The optimal clutch size revisited: separating individual quality from the parental survival costs of reproduction”.

(2) L.11-12: I agree that this is true for birds, but this is phrased more generally here. Are you sure that that is justified?

The trade-off between survival and reproductive effort has largely been tested experimentally through brood manipulations in birds as this provides a good system in which to test the costs and benefits of increasing parental effort. The work in this area has provided theory beyond just passerine birds, which are the most commonly manipulated group, to across-taxa theories. We are unaware of any study/studies that provide evidence that the reproduction/survival trade-off is generalisable across multiple species in any taxa. As such, we do believe this sentence is justified. An example is the lack of a consistent negative genetic correlation in populations of fruitflies, for example, that has also been hailed as a lack-of-cost paradigm. Furthermore, some mutants that live longer do so without a cost on reproduction.

(3) L.13-14: Not sure what you mean with this sentence - too much info lacking.

We have added some detail to this sentence.

(4) L.14: it is slightly awkward to say 'parental investment and survival' because it is the survival effect that is usually referred to as the 'investment'. Perhaps what you want to say is 'parental effort and survival'?

We have replaced “parental investment” with “reproductive effort”

(5) L.15: you can omit 'caused'. Compared to control treatment or to reduced broods? Why not mention effects or lack thereof of brood reduction? And it would be good to also mention here whether effects were similar in the sexes.

Please see our methodology where we state that we use clutch size as a continuous variable (we do not compare to control or reduced but include the absolute value of offspring in a logistic regression). The effects of a brood reduction are drawn from the same regression and so are opposite. Though we appreciate the detail here is lacking to fully comprehend our study, we would like to highlight this is the abstract and details are provided in the main text.

(6) L. 15: I am not sure why you write 'however', as the finding that experimental and natural variation have opposite effects is in complete agreement with what is generally reported in the literature and will therefore surprise no one that is aware of the literature.

We use “however” to highlight the change in direction of the effect size from the results in the previous sentence. We also believe that ours ise the first study that provides a quantitative estimate of this effect and that previous work is largely theoretical. The reviewer states that this is what is generally reported but it is not reported in all cases, as some relationships between reproductive effort and survival are negative (for the quality measurement, in correlational space, see Figure 1).

(7) L.16: saying 'opposite to the effect of phenotypic quality' seems difficult to justify, as clutch size cannot be equated with phenotypic quality. Perhaps simply say 'natural variation in clutch size'? If that is what you are referring to.

Please note we are referring to effect sizes here –- that is, the survival effect of a change in clutch size. By phenotypic quality we are referring to the fact that we find higher parental survival when natural clutch sizes are higher. It is not the case that we refer to quality only as having a higher clutch size. This is explicitly stated in the sentence you refer to. We have changed “effect” to “effect size” to highlight this further.

(8) L.18: why do you refer to 'parental care' here? Brood size is not equivalent to parental care.

Brood size manipulations are used to manipulate parental care. The effect on parental survival is expected to be incurred because of the increase in parental care. We have changed “parental care” to “reproductive effort” to reduce the number of terms we use in our manuscript.

(9) L.18-19: suggest to tone down this claim, as this is no more than a meta-analytic confirmation of a view that is (in my view) generally accepted in the field. That does not mean it is not useful, just that it does not constitute any new insight.

We are unaware of any other study which provides generalisable across-species evidence for opposite effects of quality and costs of reproduction. The work in this area is also largely theoretical and is yet to be supported experimemtally, especially in a quantitative fashion. It is surprising to us that the reviewer considers there to be general acceptance in a field, rather than being influenced by rigorous testing of hypotheses, made possible by meta-analysis, the current gold standard in our field.

(10) L.21: what does 'parental effort' mean here? You seem to use brood size, parental care, parental effort, and parental investment interchangeably but these are different concepts. Daan et al (1990, Behaviour), which you already cite, provide a useful graph separating these concepts. Please adjust this throughout the manuscript, i.e. replace 'reproductive effort' with wording that reflect the actual variable you use.

We have not used the phrase “parental effort” in this sentence. We agree these are different concepts but in this context are intertwined. For example, brood size is used to manipulate parental care as a result of increased parental effort. We do agree the manuscript would benefit from keeping terminology consistent throughout the manuscript and have adjusted this throughout.

(11) L.23: perhaps add 'in birds' somewhere in this sentence? Some reference to the assumptions underlying this inference would also be useful. Two major assumptions being that birds adjusted their effort to the manipulation as they would have done had they opted for a larger brood size themselves, and that the costs of laying and incubating extra eggs can be ignored. And then there is the effect that laying extra eggs will usually delay the hatch date, which in many species reduces reproductive success.

Though our study does exclusively use birds, birds have been used to test the survival/reproduction trade-off because they present a convenient system in which to experimentally test this. The conclusions from these studies have a broader application than in birds alone. We believe that although these details are important, they are not appropriate in the abstract of our paper.

(12) L.26: how is this an explanation? It just repeats the finding.

We intend to refer to all interpretations from all results presented in our manuscript. We have made this more clear by adjusting our writing.

(13) L.27: I do not see this point. And 'reproductive output' is yet another concept, that can be linked to the other concepts in the abstract in different ways, making it rather opaque.

We have changed “reproductive output” to “reproductive effort”.

(14) L.33: here you are jumping from 'resources' to 'energetically' - it is not clear that energy is the only or main limiting resource, so why narrow this down to energy?

We do not say energy is the only or main limiting resource. We simply highlight that reproduction is energetically demanding and so, intuitively, a trade-off with a highly energetically demanding process would be the focal place to observe a trade off. We have, though, replaced “energetically” with “resource”.

(15) L.35-36: this is new to me - I am not aware of any such claims, and effects on the residual reproductive value could also arise through effects on future reproduction. The authors you cite did not work on birds, or (in their own study systems) presented results that as far as I remember warrant such a general statement.

The trade-off between reproduction and survival is seminal to the disposable soma theory, proposed by Kirkwood. Though Kirkwood’s work was largely not focussed on birds, it had fundamental implications for the field of evolutionary ecology because of the generalisable nature of his proposed framework. In particular, it has had wide-reaching influence on how the biology of aging is interpreted. The readership of the journal here is broad, and our results have implications for that field too. The work of Kirkwood (many of the papers on this topic have over 2000 citations each) has been perhaps overly influential in many areas, so a link to how that work should be interpreted is highly relevant. If the reviewer is interested in this topic the following papers by one of the co-authors and others could be of interest, some of which we could not cite in the main manuscript due to space considerations:

https://www.science.org/doi/pdf/10.1126/sciadv.aay3047

https://agingcelljournal.org/Archive/Volume3/stochasticity_explains_non_genetic_inheritance_of_lifespan/

https://pubmed.ncbi.nlm.nih.gov/21558242/

https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/1365-2435.13444

https://www.nature.com/articles/362305a0

https://www.cell.com/trends/ecology-evolution/fulltext/S0169-5347(12)00147-4

https://www.cell.com/cell/pdf/S0092-8674(15)01488-9.pdf

https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-018-0562-z

(16) L.42: this could be preceded with mentioning the limitations of observational data.

We have added detail as to why brood manipulations are a good test for trade-offs and so this is now inherently implied.

(17) L.42-43: why?

We have added detail to this sentence.

(18) L.45: do any of the references cited here really support this statement? I am certain that several do not - in these this statement is an assumption rather than something that is demonstrated. It may be useful to look at Kate Lessell's review on this that appeared in Etologia, I think in the 1990's. Mind however that 'reproductive effort' is operationally poorly defined for reproducing birds - provisioning rate is not necessarily a good measure of effort in so far as there are fitness costs.

We have updated the references to support the sentence.

(19) L.47: Given that you make this statement with respect to brood size manipulations in birds, it seems to me that the paper by Santos & Nakagawa is the only paper you should cite here. Given that you go on to analyze the same data it deserves to be discussed in more detail, for example to clarify what you aim to add to their analysis. What warrants repeating their analysis?

Please first note that our dataset includes Santos & Nakagawa and additional studies, so it is not accurate to say we analyse the same data. Furthermore, we believe our study has implications beyond birds alone and so believe it is appropriate to cite the papers that do support our statement. We have added details to the methods to explicitly state what data is gathered from Santos & Nakagawa (it is only used to find the appropriate literature and data was re-extracted and re-analysed in a more appropriate way) and, separately, how we gathered the observational studies (see L352-381).

(20) L.48: There are more possible explanations to this, which deserve to be discussed. For example, brood size manipulations may not have been that effective in manipulating reproductive effort - for example, effects on energy expenditure tend to be not terribly convincing. Secondly, the manipulations do not affect the effort incurred in laying eggs (which also biases your comparison with natural variation in clutch size). Thirdly, the studies by Boonekamp et al on Jackdaws found that while there was no effect of brood size manipulation on parental survival after one year of manipulation, there was a strong effect when the same individuals were manipulated in the same direction in multiple years. This could be taken to mean that costs are not immediate but delayed, explaining why single year manipulations generally show little effect on survival. It would also mean that most estimates of the fitness costs of manipulated brood size are not fit for purpose, because typically restricted to survival over a single year.

Please see our response to this comment in the public reviews.

Out of interest and because the reviewer mentioned “energy expenditure” specifically: There are studies that show convincing effects of brood size manipulation on parental energy expenditure. We do agree that there are also studies that show ceilings in expenditure. We therefore disagree that they “tend to be not terribly convincing”. Just a few examples:

https://academic.oup.com/beheco/article/10/5/598/222025 (Figure 2)

https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/1365-2435.12321 (Figure 1)

https://besjournals.onlinelibrary.wiley.com/doi/full/10.1046/j.1365-2656.2000.00395.x (but ceiling at enlarged brood).

(21) L.48, "or, alternatively, that individuals may differ in quality": how do you see that happening when brood size is manipulated, and hence 'quality' of different experimental categories can be assumed to be approximately equal? This point does apply to observational studies, so I assume that that is what you had in mind, but that distinction should be clear (also on line 54).

We have made it more clear that we determine if there are quality effects separate to the costs of reproduction found using brood manipulation studies.

(22) L.50: Drent & Daan, in their seminal paper on "The prudent parent" (1980, Ardea) were among the earliest to make this point and deserve to be cited here.

We have added this citation

(23) L.51, "relative importance": relative to what? Please be more specific.

We have adjusted this sentence.

(24) L.54: Vedder & Bouwhuis (2018, Oikos) go some way towards this point and should be explicitly mentioned with reference to the role of 'quality' effects on the association between reproductive output and survival.

We have added this reference.

(25) L.55: can you be more specific on what you want to do exactly? What you write here could be interpreted differently.

We have added an explicit aim after this sentence to be more clear.

(26) L.57: Here also a more specific wording would be useful. What does it mean exactly when you say you will distinguish between 'quality' and 'costs'?

We have added detail to this sentence.

(27) L.62: it should be clearer from the introduction that this is already well known, which will indirectly emphasize what you are adding to what we know already.

We would argue this is not well known and has only been theorised but not shown empirically, as we do here.

(28) L.62: you equate clutch size with 'quality' here - that needs to be spelled out.

We refer to quality as the positive effect size of survival for a given clutch size, not clutch size alone. We appreciate this is not clear in this sentence and have reworded.

(29) L.64: this looks like a serious misunderstanding to me, but in any case, these inferences should perhaps be left to the discussion (this also applies to later parts of this paragraph), when you have hopefully convinced readers of the claims you make on lines 62-63.

We are unsure of what the reviewer is referring to as a misunderstanding. We have chosen this format for the introduction to highlight our results. If this is a problem for the editors we will change as required.

(30) L.66: quantitative comparison of what?

Comparison of species. We have changed the wording of this sentence

(31) L.67-69: this should be in the methods.

We have used a modern format which highlights our result. We are happy to change the format should the editors wish us to.

(32) L.74-88: suggest to (re)move this entire paragraph, presenting inferences in such an uncritical manner before presenting the evidence is inappropriate in my view. I have therefore refrained from commenting on this paragraph.

We have chosen a modern format which highlights our result. We are happy to change the format should the editors wish us to.

(33) L.271, "must detail variation in the number of raised young": it is not sufficiently clear what this means - what does 'detail' mean in this context? And what does 'number of raised young' mean? The number hatched or raised to fledging?

We have now made this clear.

(34) L271, "must detail variation in the number of raised young": looking at table S4, it seems that on the basis of this criterion also brood size manipulation studies where details on the number of young manipulated were missing are excluded. I see little justification for this - surely these manipulations can for example be coded as for example having the average manipulation size in the meta-analysis data set, thereby contributing to tests of manipulation effects, but not to variation within the manipulation groups?

We have done in part what the reviewer describes. We are specifically interested in the manipulation size, so we required this to compare effect sizes across species and categories, a key advance of our study and outlined in many places in our manuscript. Note, however, that we only need comparative differences, and have used clutch size metrics more generally to obtain a mean clutch size for a species, as well as SD where required. Please also note that our supplement details exactly why studies were excluded from our analysis, as is the preferred practice in a meta-analysis.

(35) L.271, "referred to as clutch size": the point of this simplification is not clear to me why it is clearly confusing - why not refer to 'brood size' instead?

Brood size and clutch size can be used interchangeably here because, in the observational studies, the individuals vary in the number of eggs produced, whereas for brood manipulations this obviously happens after hatching and brood is perhaps a more appropriate term, but we wanted to simplify the terminology used. However, we use clutch size throughout as the aim of our study is to determine why individuals differ in the number of offspring they produce, and so clutch size is the most appropriate term for that.

(36) L.280: according to the specified inclusion criteria (lines 271/272) these studies should already be in the data set, so what does this mean exactly?

Selection criteria refers to whether a given study should be kept for analysis or not. It does not refer to how studies were found. Please see lines 361-378 for details on how we found studies (additional details are also in the Supplementary Methods).

(37) L.281: the use of 'quality' here is misleading - natural variation in clutch or brood size will have multiple causes, variation in phenotypic quality of the individuals and their environment (territories) is only one of the causes. Why not simply refer to what you are actually investigating: natural and experimental variation in brood size.

We disagree, our study aims to separate quality effects from the costs of reproduction and we use observational studies to test for quality differences, though we make no inference about the mechanisms. We do not imply that the environment causes differences in quality, but that to directly compare observation and experimental groups, they should contain similar species. So, to be clear again, quality refers to the positive covariation of clutch size with survival. We feel that we explain this clearly in our study’s rationale and have also improved our writing in several sections on this to avoid any confusion (see responses to earlier comments by the three reviewers).

(38) L.283, "in most cases": please be exact and say in xx out xx cases.

We have added the number of studies for each category here.

(39) L.283-285: presumably readers can see this directly in a table with the extracted data?

Our data and code can be accessed with the following link: https://doi.org/10.5061/dryad.q83bk3jnk. We believe the data are too large to include as a table in the main text and are not essential in understanding the paper. Though we do believe all readers should have access to this information if they wish and so is publicly available.

(40) L.293: there does not seem to be a table that lists the included studies and effect sizes. It is not uncommon to find major errors in such tables when one is familiar with the literature, and absence of this information impedes a complete assessment of the manuscript.

We supplied a link to our full dataset and the code we used in Dryad with our submitted manuscript. We were also asked to supply our data during the review process and we again supplied a link to our dataset and code, along with a folder containing the data and code itself. We received confirmation that the reviewers had been given our data and code. We support open science and it was our intention that our dataset should be fully available to reviewers and readers. We believe the data are too large to include as a table in the main text and are not essential in understanding the paper. Our data and code are at https://doi.org/10.5061/dryad.q83bk3jnk.

(41) L.293: from how many species?

We have added this detail.

(42) L.296, "longevity": this is a tricky concept, not usually reported in the studies you used, so please describe in detail what data you used.

We have removed longevity as we did not use this data in our current version of the manuscript.

(43) L. 298: again: where can I see this information?

Our data and code can be accessed with the following link: https://doi.org/10.5061/dryad.q83bk3jnk. We did supply this information when we submitted our manuscript and again during the review process but we believe this was not passed onto the reviewers.

(44) L. 304, "we used raw data": I assume that for the majority of papers the raw data were not available, so please explain how you dealt with this. Or perhaps this applies to a selection of the studies only? Perhaps the experimental studies?

By raw data, we mean the absolute value of offspring in the nest. We have changed the wording of this sentence and added detail about whether the absolute value of offspring was not present for brood manipulation studies (L393-397).

(45) L.304: When I remember correctly, Santos and Nakagawa examined effects of reducing and enlarging brood size separately, which is of importance because trade-off curves are unlikely to be linear and whether they are or not has major effects on the optimization process. But perhaps you tackled this in another way? I will read on.....

You are correct that Santos & Nakagawa compared brood increases and reductions to control separately. Note that this only partially accounts non-linearity and it does not take into account the severity of the change in brood size. By using a logistic regression of absolute clutch size, as we have done, we are able to directly compare brood manipulations with experimental studies. Please see Supplementary Methods lines 11-12, where we have added additional detail as to why our approach is beneficial in this analysis.

(46) L.319: what are you referring to exactly with "for each clutch size transformation"?

We refer to the raw, standardised and proportional clutch size transformations. We have added detail here to be more clear.

(47) L.319: is there a cost of survival? Perhaps you mean 'survival cost'? This would be appropriate for the experimental data, but not for the observational data, where the survival variation may be causally unrelated to the brood size variation, even if there is a correlation.

We have changed “cost of survival” to “effect of parental survival”. We only intend to imply causality for the experimental studies. For observational studies we do not suggest that increasing clutch size is causal for increasing survival, only correlative (and hence we use the phrase “quality”).

(48) L.320: please replace "parental effort" with something like 'experimental change in brood size'.

We have changed “parental effort” to “reproductive effort”

(49) L.321: due to failure of one or more eggs to hatch, and mortality very early in life, before brood sizes are manipulated, it is not likely that say an enlargement of brood size by 1 chick can be equated to the mean clutch size +1 egg / check. For example, in the Wytham great tit study, as re-analysed by Richard Pettifor, a 'brood size manipulation' of unmanipulated birds is approximately -1, being the number of eggs / chicks lost between laying and the time of brood size manipulation. Would this affect your comparisons?

Though we agree these are important factors in determining what a clutch/brood size actually is for a given individual/pair, as this can vary from egg laying to fledging. We do not believe that accounting for this (if it was possible to do so) would significantly affect our conclusions, as observational studies are comparable in the fact that these birds would also likely see early life mortality of their offspring. It is also possibly the case that parents already factor in this loss, and so a brood manipulation still changes the parental care effort an individual has to incur.

(50) L.332: instead of "adjusted" perhaps say 'mean centred'?

We have implemented this suggestion.

(51) L.345: this statement surprised me, but is difficult to verify because I could not locate a list of the included studies. However, to my best knowledge, most studies reporting brood size manipulation effects on parental survival had this as their main focus, in contrast to your statement.

Our data and code can be accessed with the following link: https://doi.org/10.5061/dryad.q83bk3jnk. We did supply this information when we submitted our manuscript and again during the review process but we believe this was not passed onto the reviewers by the journal, although supplied by us on several occasions. We regret that the reviewer was impeded by this unfortunate communication failure, but we did our best to make the data available to the reviewers during the initial review process.

(52) L.361-362: this seems a realistic approach from an evolutionary perspective, but we know from the jackdaw study by Boonekamp that the survival effect of brood size manipulation in a single year is very different from the survival effect of manipulating as in your model, i.e. every year of an individual's life the same manipulation. For very short-lived species this possibly does not make much difference, but for somewhat longer-lived species this could perhaps strongly affect your results. This should be discussed, and perhaps also explored in your simulations?

Note that the Boonekamp study does not separate whether the survival effects are additive or

multiplicative. As such, we do not know whether the survival effects for a single year manipulation are just small and hard to detect, or whether the survival effects are multiplicative. Our simulations assumed that the brood enlargement occurred every year throughout their lives. We have added some text to the discussion on the point you raise.

(53) L.360: what is "lifetime reproductive fitness"? Is this different from just "fitness"?

We have changed “lifetime reproductive fitness” to “lifetime reproductive output”.

(54) L.363: when you are interested in optimal clutch size, why not also explore effects of reducing clutch size?

As we find that a reduction in clutch size leads to a reduction in survival (for experimental studies), we already know that these individuals would have a reduced fitness return compared to reproducing at their normal level, and so we would not learn anything from adding this into our simulations. The interest in using clutch size enlargements is to find out why an individual does not produce more offspring than it does, and the answer is that it would not have a fitness benefit (unless its clutch size and survival rate combination is out of the bounds of that observable in the wild).

(55) Fig.1 - using 'parental effort' in the y-axis label is misleading, suggest to replace with e.g. "clutch or brood size". Using "clutch size" in the title is another issue, as the experimental studies typically changed the number of young rather than the number of eggs.

We have updated the figure axes to say “clutch size” rather than “parental effort”. Please see response to comment 35 where we explain our use of the term “clutch size” throughout this manuscript.

(56) L.93 - 108: I appreciate the analysis in Table 1, in particular the fact that you present different ways of expressing the manipulation. However, in addition, I would like to see the results of an analysis treating the manipulations as factor, i.e. without considering the scale of the manipulation. This serves two purposes. Firstly, I believe it is in the interest of the field that you include a detailed comparison with the results of Santos & Nakagawa's analysis of what I expect to be largely the same data (manipulation studies only - for this purpose I would also like to see a comparison of effect size between the sexes). Secondly, there are (at least) two levels of meta-analysis, namely quantifying an overall effect size, and testing variables that potentially explain variation in effect size. You are here sort of combining the two levels of analysis, but including the first level also would give much more insight in the data set.

Our main intention here was to improve on how the same hypothesis was approached by Santos & Nakagawa. We did this by improving our analysis (on a by “egg” basis) and by adding additional studies (i.e. more data). In this process mistakes are corrected (as we re-extracted all data, and did not copy anything across from their dataset – which was used simply to ensure we found the same papers); more recent data were also added, including studies missed by Santos & Nakagawa. This means that the comparison with Santos & Nakagawa becomes somewhat irrelevant, apart from maybe technical reasons, i.e. pointing out mistakes or limitations in certain approaches. We would not be able to pinpoint these problems clearly without considering the whole dataset, yet Santos & Nakagawa only had a small subset of the data that were available to us. In short, meta-analysis is an iterative process and similar questions are inevitably analysed multiple times and updated. This follows basic meta-analytic concepts and Cochrane principles. Except where there is a huge flaw in a prior dataset or approach (like we sometimes found and highlighted in our own work, e.g. Simons, Koch, Verhulst 2013, Aging Cell), in itself a comparison of the kind the reviewer suggests distracts from the biology. With the dataset being made available others can make these comparisons, if required. On the sex difference, we provide a comparison of effect sizes separated between both sexes and mixed sex in Table S2 and Figure S1.

(57) L.93 - 108: a thing that does not become clear from this section is whether experimentally reducing brood size affects parental survival similarly (in absolute terms) as enlarging brood size. Whether these effects are symmetric is biologically important, for example because of its effect on clutch size optimization. In the text you are specific about the effects of increasing brood size, but the effect you find could in theory be due entirely to brood size reduction.

We have added detail to make it clear that a brood reduction is simply the opposite trend. We use linear relationships because they serve a good approximation of the trend and provide a more rigorous test for an underlying relationship than would fitting nonlinear models. For many datasets there is not a range of chicks added for which a non-linear relationship could be estimated. The question also remains of what the shape of this non-linear relationship should be and is hard to determine a priori.

We have added some discussion on this to our manuscript (L278-282), in response to an earlier comment.

(58) L.103-107: this is perhaps better deferred to the discussion, because other potential explanations should also be considered. For example, there have been studies suggesting that small birds were provisioning their brood full time already, and hence had no scope to increase provisioning effort when brood size was experimentally increased.

We agree this is a discussion point but we believe it also provides an important context for why we ran our simulations, and so we believe this is best kept brief but in place. We agree the example you give is relevant but believe this argument is already contained in this section. See line 121-123 “...suggesting that costs to survival were only observed when a species was pushed beyond its natural limits”.

(59) L.103-107: this discussion sort of assumes that the results in Table 1 differ between the different ways that the clutch/brood size variation is expressed. Is there any statistical support for this assumption?

We are unsure of what the reviewer means here exactly. Note that in each of the clutch size transformations, experimental and observational effect sizes are significantly opposite. For the proportional clutch size transformation, experimental and observation studies are both separately significantly different from 0.

(60) L.104: at this point, I would like to have better insight into the data set. Specifically, a scatter plot showing the manipulation magnitude (raw) plotted against control brood size would be useful.

Our data and code can be accessed with the following link: https://doi.org/10.5061/dryad.q83bk3jnk. We did supply this information when we submitted our manuscript and again during the review process but we believe this was not passed onto the reviewers by the journal.

Thank you for this suggestion: this is a useful suggestion also to illustrate how manipulations are relatively stronger for species with smaller clutches, in line with our interpretation of the result presented in Figure 2. We have added Figure S1 which shows the strength of manipulation compared to the species average.

(61) L. 107: this seems a bold statement - surely you can test directly whether effect size becomes disproportionally stronger when manipulations are outside the natural range, for example by including this characterization as a factor in the models in Table 1.

It is hard to define exactly what the natural range is here, so it is not easy to factorise objectively, which is why we chose not to do this. However, it is clear that for species with small clutches the manipulation itself is often outside the natural range. Thank you for your suggestion to include a figure for this as it is clear manipulations are stronger in species with smaller clutches. We attribute this to species being forced outside their natural range. We consider our wording makes it clear that this is our interpretation of our findings and we therefore do not think this is a bold statement, especially as it fits with how we interpret our later simulations.

(62) Fig.3, legend: the term 'node support' does not mean much to me, please explain.

Node support is a value given in phylogenetic trees to dictate the confidence of a branch. In this case, values are given as a percentage and so can translate to how many times out of 100 the estimate of the phylogeny gives the same branching. Our values are low, as we have relatively few species in our meta-analysis.

(63) Fig.3: it would be informative when you indicate in this figure whether the species contributed to the experimental or the observational data set or both.

We have added into Fig 3 whether the species was observational, experimental or both.

(64) L.139: the p-value refers to the interaction between species clutch size and treatment (observational vs. experimental), but it appears that no evidence is presented for the correlation being significant in either observational or experimental studies.

We agree that our reporting of the effect size could be misinterpreted and have added detail here. The statistic provided describes the slopes are significantly different between observational and experimental, implying there are differences between the slopes of small and large clutch-laying species.

(65) L.140: I am wondering to what extent these correlations, which are potentially interesting, are driven by the fact that species average clutch size was also used when expressing the manipulation effect. In other words, to what extent is the estimate on the Y-axis independent from the clutch size on the X-axis? Showing that the result is the same when using survival effect sizes per manipulation category would considerably improve confidence in this finding.

We are unsure what the reviewer means by “per manipulation category”. Please also note that we have used a logistic regression to calculate our effect sizes of survival, given a unit increase in reproductive effort. So, for example, if a population contained birds that lay 2,3 or 4 eggs, provided that the number of birds which survived and died in each category did not change, if we changed the number of eggs raised to 10,11 or 12, respectively, then our effect size would be the same. In this way, our effect sizes are independent of the species’ average clutch size.

(66) L.145: when I remember correctly, Santos & Nakagawa considered brood size reduction and enlargement separately. Can this explain the contrasting result? Please discuss.

You are correct, in that Santos & Nakagawa compared reductions and enlargements to controls separately. However, we found some mistakes in the data extracted by Santos & Nakagawa that we believe explain the differences in our results for sex-specific effect sizes. We do not feel that highlighting these mistakes in the main text is fair, useful or scientifically relevant, as our approach is to improve the test of the hypothesis.

(67) L.158-159: looking at table S2 it seems to me you have a whole range of estimates. In any case, there is something to be said for taking the estimates for females because it is my impression (and experience) that clutch size variation in most species is a sex-linked trait, in that clutch size tends to be repeatable among females but not among males.

We agree that, in many cases, the female is the one that ultimately decides on the number of chicks produced. We did also consider using female effect sizes only, however, we decided against this for the following reasons: (1) many of the species used in our meta-analysis exhibit biparental care, as is the case for many seabirds, and so using females only would bias our results towards species with lower male investment; in our case this would bias the results towards passerine species. (2) it has also been shown that, as females in some species are operating at their maximum of parental care investment, it is the males who are able to adjust their workload to care for extra offspring. (3) we are ultimately looking at how many offspring the breeding adults should produce, given the effort it costs to raise them, and so even if the female chooses a clutch size completely independently of the male, it is still the effort of both parents combined that determines whether the parents gain an overall fitness benefit from laying extra eggs. (4) some studies did not clearly specify male or female parental survival and we would not want to reduce our dataset further.

(68) L.158-168: please explain how you incorporated brood size effects on the fitness prospects of offspring, given that it is a very robust finding of brood size manipulation studies that this affects offspring growth and survival.

We would argue this is near-on impossible to incorporate into our simulations. It is unrealistic to suggest that incorporating offspring growth into our simulations would add insight, as a change in offspring number rarely affects all offspring in the nest equally and there can even be quite stark differences; for example, this will be most evident in species that produce sacrificial offspring. This effect will be further confounded by catch-up growth, for example, and so it is likely that increased sibling competition from added chicks alters offspring growth trajectories, rather than absolute growth as the reviewer suggests. There are mixed results in the literature on the effect of altering clutch size on offspring survival, with an increased clutch size through manipulation often increasing the number of recruits from a nest. It would be interesting, however, to explore this further using estimates from the literature, but this is beyond our current scope, and would in our initial intuition not be very accurate. It would be interesting to explore how big the effect on offspring should be to constrain effect size strongly. Such work would be more theoretical. The point of our simple fitness projections here is to aid interpretation of the quantitative effect size we estimated.

(69) L.163: while I can understand that you select the estimate of -0.05 for computational reasons, it has enormous confidence intervals that also include zero. This seems problematic to me. However, in the simulations, you also examined the results of selecting -0.15, which is close to the lower end of the 95% C.I., which seems worth mentioning here already.

Thank you for this suggestion. Yes, indeed, our range was chosen based on the CI, and we have now made this explicit in the manuscript.

(70) L.210: defined in this way, in my world this is not what is generally taken to be a selection differential. Is what you show not simply scaled lifetime reproductive success?

As far as we are aware, a selection differential is the relative change between a given group and the population mean, which is what we have done here. We appreciate this is a slightly unusual context in which to place this, but it is more logical to consider the individuals who produce more offspring as carrying a potential mutation for higher productivity. However, we believe that “selection differential” is the best terminology for the statistic we present. We also detail in our methodology how we calculate this. We have adjusted this sentence to be more explicit about what we mean by selection differential.

(71) L.177-180: is this not so because these parameter values are closest to the data you based your estimates on, which yielded a low estimate and hence you see that here also?

We are unsure of what exactly the reviewer means here. The effect sizes for our exemplar species were predicted from each combination of clutch size and survival rate. Note that we used a range of effect sizes, higher than that estimated in our meta-analysis, to explore a large parameter space and that these same conclusions still hold.

(72) L.191-194: these statements are problematic, because based on the assumption that an increase in brood size does not impact the fitness prospects of the offspring, and we know this assumption to be false.

Though we appreciate that some cost is often absorbed by the offspring themselves, we are unaware of any evidence that these costs are substantial and large enough to drive within-species variation in reproductive effort, though for some specific species this may be the case. However, in terms of explaining a generalisable, across-species trend, the fitness costs incurred by a reduction in offspring quality are unlikely to be significantly larger than the survival costs to reproduce. We also find it highly unlikely the cost to fitness incurred by a reduction in offspring quality is large enough to counter-balance the effect of parental quality that we find in our observational studies. We do also discuss other costs in our discussion.

(73) L.205: here and in other places it would be useful to be more explicit on whether in your discussion you are referring to observational or experimental variation.

We have added this detail to our manuscript. Do note that many of our conclusions are drawn by the combination of results of experimental and observational studies. We believe the addition of Figure 5 makes this more clear to the reader.

(74) L.225: this may be true (at least, when we overlook the misuse of the word 'quality' here), but I would expect some nuance here to reflect that there is no surprise at all in this result as this pattern is generally recognized in the literature and has been the (empirical) basis for the often-repeated explanation of why experiments are required to demonstrate trade-offs. On a more quantitative level, it is worth mentioning the paper of Vedder & Bouwhuis (2017, Oikos) that essentially shows the same thing, i.e. a positive association between reproductive output and parental survival.

We have added some discussion on this point, including adding the citation mentioned. However, we would like to highlight that our results demonstrate that brood manipulations are not necessarily a good test of trade-offs, as they fail to recognise that individuals differ in their underlying quality. Though we agree that this result should not necessarily be a surprising one, we have also not found it to be the case that differences in individual quality are accepted as the reason that intra-specific clutch size is maintained – in fact, we find that it is most commonly argued that when costs of reproduction are not identifiedit is concluded that the costs must be elsewhere – yet we cannot find conclusive evidence that the costs of reproduction (wherever they lie) are driving intra-specific variation in reproductive effort. Furthermore, some studies in our dataset have reported negative correlations between reproductive effort and survival (see observational studies, Figure 1).

(75) L.225-226: perhaps present this definition when you first use the term.

We have added more detail to where we first use and define this term to improve clarity (L57-58).

(76) L.227-228, "currently unknown": this statement surprised me, given that there is a plethora of studies showing within-population variation in clutch size to depend on environmental conditions, in particular the rate at which food can be gathered.

We mean to question that if an individual is “high quality”, why is it not selected for? We have rephrased, to improve clarity.

(77) L.231: this seems no more than a special case of the environmental effect you mention above.

We think this is a relevant special case, as it constitutes within-individual variation in reproduction that is mistaken for between-individual variation. This is a common problem in our field, that we feel needs adressing. We only have between-individual variation here in our study on quality, and by highlighting this we show that there might not be any variation between individuals, but this could come about fully (doubtful) or partly (perhaps likely) due to terminal effects.

(78) L235-236: but apparently depending on how experimental and natural variation was expressed? Please specify here.

We are not sure what results the reviewer is referring to here, as we found the same effect (smaller clutch laying species are more severely affected by a change in clutch size) for both clutch size expressed as raw clutch size and standardised clutch size.

(79) L.237: the concept of 'limits' is not very productive here, and it conflicts with the optimality approach you apply elsewhere. What you are saying here can also be interpreted as there being a non-linear relationship between brood size manipulation and parental survival, but you do not actually test for that. A way to do this would be to treat brood size reduction and enlargement separately. Trade-off curves are not generally expected to be linear, so this would also make more sense biologically than your current approach.

We have replaced “limits” with “optima”. We believe our current approach of treating clutch size as a continuous variable, regardless of manipulation direction, is the best approach, as it allows us to directly compare with observational studies and between species that use different manipulations (now nicely illustrated by the reviewer’s suggested Figure S1). Also note that transforming clutch size to a proportion of the mean allows us to account for the severity in change in clutch size. We also do not believe that treating reductions and enlargements separately accounts for non-linearity, as either we are separating this into two linear relationships (one for enlargements and one for reductions) or we compare all enlargements/reductions to the control, as in Santos & Nakagawa 2012, which does not take into account the severity of the increase, which we would argue is worse for accounting for non-linearity. Furthermore, in the cases where the manipulation involved one offspring only, we also cannot account for non-linearity.

(80) L.239: assuming birds are on average able to optimize their clutch size, one could argue that any manipulation, large or small, on average forces birds to raise a number of offspring that deviates from their natural optimum. At this point, it would be interesting to discuss in some detail studies with manipulation designs that included different levels of brood size reduction/enlargement.

We agree with the reviewer that any manipulation is changing an individual’sclutch size away from its own individual optima, which we have argued also means brood manipulations are not necessarily a good test of whether a trade-off occurs in the wild (naturally), as there could be interactions with quality – we have now edited to explicitly state this (L299-300).

(81) L.242-244: when you choose to maintain this statement, please add something along the lines of "assuming there is no trade-off between number and quality of offspring".

As explained above, though we agree that the offspring may incur some of the cost themselves, we are not aware of any evidence suggesting this trade-off is also large enough to drive intra-specific variation in clutch size across species. Furthermore, in the context here, the trade-off between number and quality of offspring would not change our conclusion – that the fitness benefit of raising more offspring is offset by the cost on survival. We have added detail on the costs incurred by offspring earlier in our discussion (L309-315). The addition of Figure 5 should help interpret these data.

(82) L.253: instead of reference 30 the paper by Tinbergen et al in Behaviour (1990) seems more appropriate.

We believe our current citation is relevant here but we have also added the Tinbergen et al (1990) citation.

(83) L.253-254: such trade-offs may perfectly explain variation in reproductive effort within species if we were able to estimate cost-benefit relations for individuals. In fact, reference 29 goes some way to achieve this, by explaining seasonal variation in reproductive effort.

We are unaware of any quantitative evidence that any combination of trade-offs explains intra-specific variation in reproductive effort, especially as a general across-species trend.

(84) L.255: how does one demonstrate "between species life-history trade-offs"? The 'trade-off' between reproductive rate and survival we observe between species is not necessarily causal, and hence may not really be a trade-off but due to other factors - demonstrating causality requires some form of experimental manipulation.

Between-species trade-offs are well established in the field, stemming from GC Williams’ seminal paper in 1966, and for example in r/K selection theory. It is possible to move from these correlations to testing for causation, and this is happening currently by introducing transgenes (genes from other species) that promote longevity into shorter-lived species (e.g., naked-mole rat genes into mice). As yet it is unclear what the effects on reproduction are.

(85) L.256: it is quite a big claim that this is a novel suggestion. In fact, it is a general finding in evolutionary theory that fitness landscapes tend to be rather flat at equilibrium.

It is important to note here that we simulate the effect size found, and hence this is the novel suggestion, that because the resulting fitness landscape is relatively flat there is no directional selection observed. We did not intend to suggest our interpretation of flat fitness landscapes is novel. We have changed the phrasing of this sentence to avoid misinterpretation.

(86) L.259: why bring up physiological 'costs' here, given that you focus on fitness costs? Do you perhaps mean fitness costs instead of physiological costs? Furthermore, here and in the remainder of this paragraph it would be useful to be more specific on whether you are considering natural or experimental variation.

The cost of survival is a physiological cost incurred by the reduction of self-maintenance as a result of lower resource allocation. This is one arm of fitness; we feel it would be confusing here to talk about costs to fitness, as we do not assess costs to future reproduction (which formed the large part of the critique offered by the reviewer). We would like to highlight that the aim of this manuscript was to separate costs of reproduction from the effects of quality, and this is why we have observational and experimental studies in one analysis, rather than separately. Our conclusion that we have found no evidence that the survival cost to reproduce drives within-species variation in clutch size comes both from the positive correlation found in the observational studies and our negligible fitness return estimates in our simulations. We therefore, do not believe it is helpful to separate observational and experimental conclusions throughout our manuscript, as the point is that they are inherently linked. We hope that with the addition of Figure 5 that this is more clear.

(87) L.262: The finding that naturally more productive individuals tend to also survive better one could say is by definition explained by variation in 'quality', how else would you define quality?

We agree, and hence we believe quality is a good term to describe individuals who perform highly in two different traits. Note that we also say the lack of evidence that trade-offs drive intra-specific variation in clutch size also potentially suggests an alternative theory, including intra-specific variation driven by differences in individual quality.

Supplementary information

(88) Table S1: please provide details on how the treatment was coded - this information is needed to derive the estimates of the clutch size effect for the treatments separately.

We have added this detail.

(89) Table S2: please report the number of effect sizes included in each of these models.

We have added this detail.

(90) Table S4: references are not given. Mentioning species here would be useful. For example, Ashcroft (1979) studied puffins, which lay a single egg, making me wonder what is meant when mentioning "No clutch or brood size given" as the reason for exclusion. A few more words to explain why specific studies were excluded would be useful. For example, what does "Clutch size groups too large" mean? It surprises me that studies are excluded because "No standard deviation reported for survival" - as the exact distribution is known when sample size and proportion of survivors is known.

We have updated this table for more clarity.

(91) Fig.S1: please plot different panels with the same scale (separately for observational and experimental studies). You could add the individual data points to these plots - or at least indicate the sample size for the different categories (female, male, mixed).

We have scaled all panels to have the same y axis and added sample sizes to the figure legend.

(92) Fig.S3: please provide separate plots for experimental and observational studies, as it seems entirely plausible that the risk of publication bias is larger for observational studies - in particular those that did not also include a brood size manipulation. At the same time, one can wonder what a potential publication bias among observational studies would represent, given that apparently you did not attempt to collect all studies that reported the relevant information.

We have coloured the points for experimental and observational studies. Note that a study is an independent effect size and, therefore, does not indicate whether multiple data (i.e., both experimental and observational studies) came from the same paper. As we detail in the paper and above in our reviewer responses, we searched for observational studies from species used in the experimental studies to allow direct comparison between observational and experimental datasets.

Reviewer #2 (Recommendations For The Authors):

I strongly recommend improving the theoretical component of the analysis by providing a solid theoretical framework before, from it, drawing conclusions.

This, at a minimum, requires a statistical model and most importantly a mechanistic model describing the assumed relationships.

We thank the reviewer for highlighting that our aims and methodology are unclear in places. We have added detail to our model and simulation descriptions and have improved the description of our rationale. We also feel the failure of the journal to provide code and data to the reviewers has not helped their appreciation of our methodology and use of data.

Because the field uses the same wording for different concepts and different wording for the same concept, a glossary is also necessary.

We thank the reviewer for raising this issue. During the revision of this manuscript, we have simplified our terminology or given a definition, and we believe this is sufficient for readers to understand our terminology.

Reviewer #3 (Recommendations For The Authors):

• The files containing information of data extracted from each study were not available so it has not been possible to check how any of the points raised above apply to the species included in the study. The ms should include this file on the Supp. Info as is standard good practice for a comparative analysis.

We supplied a link to our full dataset and the code we used in Dryad with our submitted manuscript. We were also asked to supply our data during the review process and we again supplied a link to our dataset and code, along with a folder containing the data and code itself. We received confirmation that the reviewers had been given our data and code. We support open science and it was our intention that our dataset should be fully available to reviewers and readers. We believe the data is too large to include as a table in the main text and is not essential in understanding the paper. Our data and code are at https://doi.org/10.5061/dryad.q83bk3jnk.

• For clarity, refer to 'the effect size of clutch size on survival" rather than simply "effect size". Figures 1 and 2 require cross-referencing with the main text to understand the y-axis.

We have added detail to the figure legend to increase the interpretability of the figures.

• Silhouettes in Figure 3 (or photos) would help readers without ornithological expertise to understand the taxonomic range of the species included in the analyses.

We have added silhouettes into Figure 3.

• Throughout the discussion: superscripts shouldn't be treated as words in a sentence so please add authors' names where appropriate.

We have added author names and dates where required.

https://web.archive.org/web/20240528070547/https://shkspr.mobi/blog/2023/05/the-limits-of-general-purpose-computation/

Terence Eden (posted #2023/05/28 ) on the question if an app provider does have a say on being willing to run their code on your device, in contrast me being in control of a device and determining which code to run there or not. In this case a bank that would disallow their app on a rooted phone, because of risk profiles attached to that. Interesting tension: my risk assessment, control over general computation devices versus a service provider for which their software is a conduit and their risk assessments. I suspect the issue underneath this is such tensions need to be a conversation or negotiation to resolve, but in practice it's a dictate by one party based on a power differential (the bank controls your money, so they can set demands for your device, because you will need to keep access to your account.)

tính năng người dùng hỗ trợ tương tác, hiển thị và các thứ

Summary of "Revised Report on the Propagator Model" by Alexey Radul and Gerald Jay Sussman

Introduction

• Main Problem: Traditional programming models hinder extending existing programs for new situations due to rigid commitments in the code.
• Quote: "The most important problem facing a programmer is the revision of an existing program to extend it for some new situation."
• Solution: The Propagator Programming Model supports multiple viewpoints and integration of redundant solutions to aid program extensibility.
• Quote: "The Propagator Programming Model is an attempt to mitigate this problem."

Propagator Programming Model

• Core Concept: Autonomous machines (propagators) communicate via shared cells, continuously adding information based on computations.
• Quote: "The basic computational elements are autonomous machines interconnected by shared cells through which they communicate."
• Additivity: New contributions are seamlessly integrated by adding new propagators without disrupting existing computations.
• Quote: "New ways to make contributions can be added just by adding new propagators."

Propagator System

• Language Independence: The model can be implemented in any programming language as long as a communication protocol is maintained.
• Quote: "You should be able to write propagators in any language you choose."
• Cell Operations: Cells support adding content, collecting content, and registering propagators for notifications on content changes.
• Quote: "Cells must support three operations: add some content, collect the content currently accumulated, register a propagator to be notified when the accumulated content changes."

Implementing Propagator Networks

• Creating Cells and Propagators: Cells store data, while propagators compute based on cell data. Propagators are attached using d@ (diagram style) or e@ (expression style) for simpler cases.
• Quote: "The cells' job is to remember things; the propagators' job is to compute."
• Example: Adding two and three using propagators.
• Quote: "(define-cell a) (define-cell b) (add-content a 3) (add-content b 2) (define-cell answer (e:+ a b)) (run) (content answer) ==> 5"

Advanced Features

• Conditional Network Construction: Delayed construction using conditional propagators like p:when and p:if to control network growth.
• Quote: "The switch propagator does conditional propagation -- it only forwards its input to its output if its control is 'true'."
• Partial Information: Cells accumulate partial information, which can be incrementally refined.
• Quote: "Each 'memory location' of Scheme-Propagators, that is each cell, maintains not 'a value', but 'all the information it has about a value'."

Built-in Partial Information Structures

• Types: Nothing, Just a Value, Numerical Intervals, Propagator Cells, Compound Data, Closures, Truth Maintenance Systems, Contradiction.
• Quote: "The following partial information structures are provided with Scheme-Propagators: nothing, just a value, intervals, propagator cells, compound data, closures, supported values, truth maintenance systems, contradiction."

Debugging and Metadata

• Debugging: Scheme's built-in debugger aids in troubleshooting propagator networks. Metadata tracking for cells and propagators enhances debugging.
• Quote: "The underlying Scheme debugger is your friend."
• Metadata: Tracking names and connections of cells and propagators helps navigate and debug networks.
• Quote: "Inspection procedures using the metadata are provided: name, cell?, content, propagator?, propagator-inputs, propagator-outputs, neighbors, cell-non-readers, cell-connections."

Benefits of the Propagator Model

• Additivity and Redundancy: Supports incremental additions and multiple redundant computations, enhancing flexibility and resilience.
• Quote: "It is easy to add new propagators that implement additional ways to compute any part of the information about a value in a cell."
• Intrinsic Parallelism: Each component operates independently, making the model naturally parallel and race condition-resistant.
• Quote: "The paradigm of monotonically accumulating information makes [race conditions] irrelevant to the final results of a computation."
• Dependency Tracking: Facilitates easier integration and conflict resolution via premises and truth maintenance.
• Quote: "If the addition turns out to conflict with what was already there, it (or the offending old thing) can be ignored, locally and dynamically, by retracting a premise."

Conclusion

• Goal Achievement: The Propagator Model approaches goals of extensibility and additivity by allowing flexible integration and redundancy in computations.
• Quote: "Systems built on the Propagator Model of computation can approach some of these goals."

• The speaker initially describes the excitement and complexity of rendering a simple red triangle in Clojure using a Java library.

  • "I found a Java library that gave bindings I started playing around with it and after quite a bit of struggle I was able to get a red triangle to show up on the screen."

• The creation of more complex shapes like the Sierpinski pyramid, which involves recursive geometric transformations, serves as an example to explore different programming abstractions.

  • "It's actually fairly straightforward and I'll switch to two dimensions for your convenience, you start with a triangle and you shrink it down to half its size and then you copy it twice."

• The speaker introduces macros as a way to handle repetitive code by creating a reusable "Sierpinski macro" that can be nested to achieve different levels of detail.

  • "We can write ourselves a Sierpinski macro which takes a body which is really just anything and first we wrap it in a function and then we create a scoped transform which shrinks everything that we draw by half."

• Despite their usefulness, macros have limitations in terms of composition and downstream flexibility, prompting the exploration of functions as a better abstraction.

  • "Macros are not necessarily our most composable sort of abstraction and this has two issues... there is no potential for downstream composition."

• Functions, though still limited, allow for more composable and testable abstractions by introducing indirection and treating rendering operations as data.

  • "We take our draw triangle function and we call it now it's a renderer and now we have a render function which simply invokes it."

• The speaker demonstrates how higher-order functions and mapping over data can create more flexible and testable code, moving from functions to data-centric approaches.

  • "We defined three of these that will offset and scale appropriately and now we Define a Sierpinski method which takes a list of shapes and returns a list of shapes which is three times larger with all of them sort of offset and scaled appropriately."

• The comparison between data, functions, and macros emphasizes the generality and composability of data-centric approaches, albeit with the necessity of grounding them in executable code.

  • "Data doesn't do anything by itself we have to sort of eventually ground out in code that does something we have to wrap our data in functions and execute our functions somewhere within our code."

• The speaker critiques taxonomies while acknowledging their utility in providing a framework for discussion and suggesting approaches.

  • "Taxonomies are not a way to perfectly model the world... but they give us both a vocabulary to talk about it and they give us sort of a predictiveness."

• Applying these abstractions to more practical examples, like sequence transformations and transducers, illustrates the balance between abstraction and practical utility.

  • "We can use the double threaded arrow to structure it so that the flow of data is left to right rather than inside out."

• The exploration of automaton theory and finite state machines (like Automat) showcases the power of data-driven approaches in providing flexibility and compositional capabilities beyond traditional regular expressions.

  • "Automaton Theory... there's a much richer set of tools there than I'd originally realized and I tried to encompass this in a library called Automat."

• The discussion on backpressure and causality in asynchronous programming highlights the importance of managing side effects and execution order in concurrent systems.

  • "Backpressure here is this emergent property because we have this structural relationship between this it's just that one happens before the other and that one can cause the other not to happen."

• The introduction of streams and deferreds in the Manifold library exemplifies a less opinionated approach to handling eventual values and asynchronous computation, allowing for greater flexibility in execution models.

  • "A stream is lossy right you take a message it's gone a deferred represents a single unrealized value and you can sort of consume it as many times as you like."

• The concept of "let-flow" in Manifold, which optimizes concurrency by analyzing data dependencies, demonstrates advanced techniques for achieving optimal parallel execution.

  • "Let-flow will basically walk the entire let binding figure out what the data dependencies are and execute them in the correct order."

• The speaker concludes by emphasizing the importance of understanding the different forces at play in software composition and encourages ongoing discussion to improve the ecosystem.

  • "This is what causes our ecosystem to flourish or dwindle right the degree to which all the disparate pieces can work together."

• In the Q&A, the speaker briefly touches on the position of monads in the spectrum and compares Automat to other finite state machine libraries like Regal.

  • "Where do you put monads in your spectrum... somewhere between functions and data right."
  • "Automat... is meant to be sort of a combinator version of Regal effectively."

• Computational Challenges:
• We lack effective computational methods: "i think we actually have the foggiest idea how to compute very well."

• Importance of fast, efficient processes: "it took only 100 milliseconds... we don't understand how to do."

• Genomic Complexity:

• Human genome's complexity and flexibility: "with a small change to it you make a cow instead of a person."

• High-level language for biological processes is unknown: "what I'm interested in is the high-level language that's necessary to do things like that."

• Programming Evolution:

• Legacy of programming assumptions based on scarcity: "all of our sort of intuitions from being programmers have come from a time of assuming a kind of scarcity."

• Current abundance of resources shifts the focus: "memory is free, computing is free."

• Security and Correctness:

• Traditional concerns of correctness and security are secondary: "people worry about correctness... is it the real problem? maybe... most things don't have to work."

• Evolution and adaptability of code are crucial: "we spend all our time modifying existing code."

• Programming Constraints:

• Early decisions in programming constrain future changes: "we make decisions early in some process that spread all over our system."

• Need for flexibility in modifying systems: "organize systems so that the consequences of decisions we make are not expensive to change."

• Generic Operators and Extensions:

• Dynamically extensible operations: "dynamically extend things while my program is running."

• Symbolic algebra as an extension of arithmetic: "expand this arithmetic on functions... it's a classical thing people can do in algebra."

• Propagators and Parallelism:

• Concept of propagators for parallel computation: "propagators are independent little stateful machines."

• Parallelism and monotonic information merging: "we don't actually put values in these cells we put information about a value in a cell."

• Truth Maintenance Systems (TMS):

• Maintaining and improving data consistency: "truth maintenance systems... maintain the best estimates of what's going on."

• Dependency-directed backtracking for efficient problem-solving: "automatically find for me the consistent sub consistent sub the sub world views that are consistent."

• Historical and Educational Insights:

• Historical evolution of computation: "when I started computing in 1961... the total amount of memory is probably about 10 kilobytes."

• Educational gaps between theory and practical engineering: "what we taught the students wasn't at all what the students actually were expected to learn."

• Vision for the Future:

• Future computing systems must be inherently parallel, redundant, and flexible: "future... computers are so cheap and so easy to make... they can talk to each other and do useful things."

• Importance of evolving current computational thinking: "we have to throw away our current ways of thinking if we ever expect to solve these problems."

• Summary and Call to Action:

• Main challenge is evolvability, not correctness: "problem facing us as computer engineers is not correctness it's evolvability."
• Proposals include extensible operations and new architectural paradigms: "extensible generic operations... a more radical proposal is maybe there are freedoms that we can unlock by throwing away our idea of architecture."

This outline captures the essential points and arguments presented, while providing specific quotes for reference.

• Introduction

  • Jerry Sussman discusses the importance of flexible programming systems.
  • "I want to show how to the ways of making systems that have the property that you can you write this big pile of code and you know you, you get all of a sudden it's gee I have a different problem I have to solve why can't I use the same piece of code."

• Personal Background

  • Sussman's extensive experience as a programmer since the 1960s.
  • "I'm the oldest guy here probably and way back when I first started programming computers that's what they looked like."

• Goals for Robust Systems

  • Desires systems that are generalizable, evolvable, and tough.
  • "I want is robust systems that have the following property that they're generalizable in the sense that... they're evolvable in the sense that they can be adapted to new jobs without modification."

• Critique of Programming 'Religions'

  • Emphasizes the need for diverse tools and approaches in programming.
  • "We have a lot of people who have religions about how to program... Each of which is good for some particular problems but not good for a lot of problems."

• Body Plans in Engineering and Programming

  • Discusses the concept of body plans from biology and its application in engineering.
  • "Superrod radio receiver... a body plan... that separates band selectivity from inter-channel selectivity."

• Multiple Approaches in Problem Solving

  • Highlights the value of having multiple methods to solve the same problem, inspired by biology.
  • "There are two ways to make a frog... why don't we do that in programming?"

• Biological Inspiration in Programming

  • Advocates learning from nature's solutions, such as the flexible human genome.
  • "One of the things nature does is it's very expensive... generates and tests... we don't do that very much in programming because it's expensive."

• Generating and Testing in Programming

  • Discusses McCarthy's amb operator for non-deterministic search.
  • "McCarthy's amb operator... is a modeling non-deterministic autometer."

• Generic Operations and Extensibility

  • Explains the power of generic operations and their application in programming.
  • "Most people think generic operations... in fact, automatic differentiation is just a generic extension of arithmetic."

• Risks and Benefits of Generic Extensions

  • Acknowledges the dangers of generic programming while emphasizing its power.
  • "Generic arithmetic... very dangerous... the only thing that scares me in programming is floating point."

• Teaching and Practical Applications

  • Emphasizes the importance of teaching robust, flexible programming techniques.
  • "I've only given you three of them... I have hundreds of them that I just understand how to avoid programming yourself into a corner."

• Conclusion

  • Discusses his books and teaching philosophy, integrating traditional and programming languages for clarity.
  • "I've written several books about this... using programming as the way of expressing the ideas besides the traditional mathematics in addition to you make it unambiguous CL clear and therefore easy to read."

Résumé de la vidéo [00:00:00][^1^][1] - [00:29:11][^2^][2]:

Cette vidéo présente une discussion sur le harcèlement et la violence à l'école, animée par Myriam Ilouz et Catherine Perelmutter. Elle aborde les impacts psychologiques du harcèlement, son évolution à travers le temps, et les défis de l'identification des harceleurs dans un contexte scolaire.

Points forts: + [00:00:00][^3^][3] Introduction et contexte * Présentation de Myriam Ilouz, psychologue clinicienne * Importance du sujet du harcèlement à l'école + [00:01:00][^4^][4] Le harcèlement à l'école * Impact du harcèlement sur l'introduction de l'enfant au social * La souffrance des victimes et des harceleurs + [00:03:02][^5^][5] Changement dans la nature du harcèlement * Augmentation de l'intensité et de la violence du harcèlement * Utilisation des médias modernes pour harceler + [00:10:02][^6^][6] Caractéristiques des harceleurs * Agressivité, nuisance intentionnelle et répétée * Instauration d'une relation de disymétrie sociale + [00:20:01][^7^][7] La perversion et le harcèlement * Le harcèlement comme manifestation de la perversion narcissique * Difficulté à démasquer les harceleurs et à protéger les victimes + [00:27:02][^8^][8] Conséquences sociétales du harcèlement * Impact sur la confiance en l'école et la société * Nécessité d'une éducation solide pour prévenir la perversion Résumé de la vidéo [00:29:13][^1^][1] - [00:56:27][^2^][2]:

La vidéo aborde le problème du harcèlement et de la violence à l'école, en se concentrant sur les changements dans l'éducation des enfants, l'impact de la société moderne sur le comportement des élèves, et les défis rencontrés par les enseignants et les parents. Elle souligne l'importance de comprendre la loi, la frustration et la castration symbolique dans l'éducation pour prévenir le harcèlement.

Points forts: + [00:29:13][^3^][3] L'évolution de l'éducation des enfants * La psychanalyse française et son impact * La création d'enfants rois et le manque de règles claires * L'importance de la loi et de la frustration dans l'éducation + [00:32:35][^4^][4] Le rôle des parents et des enseignants * La disparition de l'autorité et de la hiérarchie * La nécessité d'une alliance éducative entre l'école et la famille * L'importance du respect de l'autorité des enseignants + [00:39:01][^5^][5] Le harcèlement scolaire et la justice * Définition légale et formes de harcèlement * L'impact des réseaux sociaux sur le harcèlement * La vulnérabilité des victimes et l'importance de la prévention + [00:47:02][^6^][6] Les mesures et les lois contre le harcèlement * Statistiques et plans gouvernementaux * Le droit à l'éducation et la protection contre la violence * Les initiatives pour améliorer le climat scolaire et prévenir le harcèlement Résumé de la vidéo [00:56:29][^1^][1] - [01:14:14][^2^][2]:

Cette vidéo aborde le harcèlement et la violence à l'école, en se concentrant sur les aspects juridiques du harcèlement en France, notamment les changements apportés par la loi du 2 mars 2022. Elle explique les nouvelles dispositions du Code pénal français concernant le harcèlement scolaire, les peines encourues et les mesures à prendre pour prouver le harcèlement et obtenir justice.

Points forts: + [00:56:29][^3^][3] Cadre juridique du harcèlement * Discussion sur le Code pénal français et la loi du 2 mars 2022 * Explication des articles relatifs au harcèlement * Importance de la preuve et des démarches légales + [01:01:14][^4^][4] Cas judiciaire spécifique * Examen d'un jugement du tribunal pour enfants d'Épinal * Analyse de la causalité entre le harcèlement et le suicide * Mention d'un appel possible devant la cour d'appel + [01:07:58][^5^][5] Témoignage d'une mère * Récit d'une mère sur le harcèlement subi par sa fille * Difficultés rencontrées dans la prise en charge et la justice * Appel à des actions concrètes pour soutenir les victimes

Résumé de la vidéo [00:00:00][^1^][1] - [00:26:57][^2^][2]:

Cette vidéo présente le Conseil Économique, Social et Environnemental (CESE) en France, ses missions, sa composition et son impact sur la société. Le CESE est décrit comme un pont entre les citoyens et les pouvoirs publics, offrant une plateforme pour la démocratie participative et l'élaboration de politiques publiques.

Points forts: + [00:00:00][^3^][3] Rôle et missions du CESE * Conseille le gouvernement et le Parlement * Favorise la démocratie participative * Évalue l'efficacité des politiques publiques + [00:06:29][^4^][4] Composition du CESE * 175 conseillers issus de divers secteurs * Représentation de la société civile organisée * Groupes d'intérêt et affinités variés + [00:14:15][^5^][5] Débats et propositions * Discussions sur des sujets d'actualité et de société * Interventions des membres sur des thématiques variées * Propositions pour améliorer la vie quotidienne + [00:21:05][^6^][6] Inégalités de genre, crise climatique et transition écologique * Analyse de l'impact du genre sur les questions écologiques * Vulnérabilité des femmes face aux crises * Rôle des femmes dans la promotion de la durabilité Résumé de la vidéo [00:26:59][^1^][1] - [00:53:06][^2^][2]:

La vidéo présente une discussion sur les solutions pour construire une société durable et respectueuse de l'égalité de genre. Elle aborde l'écoféminisme, la mixité des métiers, et l'impact du changement climatique sur les femmes.

Points forts: + [00:27:02][^3^][3] L'écoféminisme * Parallèle entre la domination de la nature et celle des femmes * Vision d'une société sans patriarcat ni domination * Importance de renouer avec le vivant + [00:29:17][^4^][4] Les stéréotypes de genre * Impact des stéréotypes dès l'enfance * Influence sur la vie et le rapport à la nature * Nécessité de promouvoir la mixité des métiers + [00:31:18][^5^][5] L'égalité de genre dans les politiques publiques * Lien entre égalité de genre et action pour le vivant * Intégration des réalités de genre dans les solutions climatiques * Importance de la diplomatie féministe et du financement des associations féministes + [00:38:04][^6^][6] La participation des femmes à la lutte environnementale * Femmes comme actrices majeures de la lutte pour l'environnement * Changement de paradigme pour valoriser leurs compétences * Connexion entre les questions sociales et environnementales Résumé de la vidéo [00:53:08][^1^][1] - [01:17:09][^2^][2]:

La vidéo présente les solutions pour construire une société durable et respectueuse de l'égalité de genre, en se concentrant sur les impacts différenciés du changement climatique sur les femmes et les hommes. Elle souligne l'importance de l'intégration de l'égalité de genre dans les politiques environnementales et la nécessité d'une action concrète pour protéger les droits des femmes.

Points forts: + [00:53:08][^3^][3] Introduction et quiz * Présentation des recommandations principales * Quiz interactif pour évaluer les connaissances sur l'égalité de genre * Importance de l'égalité de genre dans la gestion des catastrophes + [01:00:04][^4^][4] Impact différencié du changement climatique * Les femmes sont affectées de manière disproportionnée par les catastrophes climatiques * Les crises climatiques augmentent les violences envers les femmes * Nécessité de soutenir les projets portés par les femmes + [01:05:03][^5^][5] Intégration de l'égalité de genre dans les politiques * La diplomatie féministe de la France et ses implications * L'importance de l'évaluation des engagements internationaux * La sécurité des femmes déplacées par les changements climatiques + [01:14:00][^6^][6] Conséquences des activités industrialisées * Les pays riches sont responsables des crises climatiques * Les pays en développement sont les plus touchés * Appel à la protection juridique des migrants environnementaux Résumé de la vidéo [01:17:11][^1^][1] - [01:39:37][^2^][2] :

Cette partie de la vidéo aborde les solutions pour construire une société durable et respectueuse de l'égalité de genre. Elle met en lumière l'intégration des questions de genre dans les politiques environnementales, l'importance de l'investissement public dans la transition écologique, et le rôle des collectivités territoriales et des entreprises dans la promotion de l'égalité de genre.

Points forts : + [01:17:11][^3^][3] Intégration du genre dans la fiscalité environnementale * Éviter de renforcer les inégalités existantes * Corriger les inégalités à travers les investissements publics * Stratégie française pour l'énergie et le climat + [01:18:00][^4^][4] Objectifs transversaux d'écologie et d'égalité * Intégrer les objectifs d'écologie et de réduction des inégalités * Documenter avec des données spécifiques au genre * Chaque euro dépensé doit également bénéficier à l'égalité de genre + [01:20:04][^5^][5] Politique de mobilité et impact sur les femmes * Exemple de la promotion du vélo et ses conséquences sur l'espace public * Nécessité pour les collectivités de croiser les thématiques d'environnement et de genre * Politiques inclusives comme celles de la Ville de Genève + [01:24:04][^6^][6] Inégalités professionnelles dans les métiers verdissants * Sous-représentation des femmes dans les secteurs émetteurs de gaz à effet de serre * Importance de l'inclusion des femmes dans la transition écologique * Lever les obstacles à la participation des femmes dans ces métiers Résumé de la vidéo [01:39:39][^1^][1] - [02:03:26][^2^][2] : La vidéo aborde les solutions pour construire une société durable et respectueuse de l'égalité de genre. Elle met en lumière les défis et les préconisations du Conseil Économique, Social et Environnemental (CESE) en France, notamment en matière d'accueil des réfugiés, de politiques publiques, de biodiversité, de pollution et de participation démocratique.

Points forts : + [01:40:00][^3^][3] Défis et réactions face à l'égalité de genre * Discussion sur les réactions négatives aux travaux sur le genre * Confusion entre genre masculin et masculinité * Importance de prévenir les réactions négatives + [01:41:06][^4^][4] Intégration des questions de genre dans les politiques environnementales * Lien entre biodiversité, pollution et inégalités de genre * Nécessité d'une approche intégrée et détaillée * Choix difficiles dans les axes de préconisation + [01:45:52][^5^][5] Accueil des réfugiés et prise en charge spécifique des femmes et des filles * Préconisation d'intégrer une jurisprudence dans le Code de l'entrée et du séjour des étrangers * Besoins spécifiques des femmes et des filles réfugiées * Importance de projets spécifiques et de soutien financier + [01:57:43][^6^][6] Importance des données ventilées par sexe pour les politiques publiques * Collecte de données pour mieux connaître et agir * Évaluation continue des outils existants * Nécessité d'améliorer l'index d'égalité professionnelle Résumé de la vidéo [02:03:29][^1^][1] - [02:29:31][^2^][2]:

La vidéo aborde les solutions pour construire une société durable respectant l'égalité des genres. Elle souligne l'importance de l'intégration des femmes dans les métiers verts, la nécessité d'une diplomatie féministe, et l'impact du changement climatique sur les femmes. Elle appelle à une meilleure représentation des femmes dans les décisions politiques et environnementales, et à l'adoption de politiques publiques sensibles au genre.

Points forts: + [02:03:29][^3^][3] Impact du changement climatique sur les femmes * Importance de la recherche sur les différences d'impact entre les sexes * Nécessité d'une meilleure représentation des femmes dans les métiers verts * Appel à une diplomatie féministe et à des politiques publiques adaptées + [02:06:00][^4^][4] Justice de genre et justice climatique * Lien entre la préservation de la planète et l'évolution de la société * La justice de genre comme élément central de la justice climatique * Les politiques publiques doivent intégrer l'égalité des sexes + [02:10:21][^5^][5] Rôle des femmes dans la lutte contre la crise climatique * Les femmes sont plus vulnérables et exposées aux catastrophes naturelles * Nécessité de reconnaître et promouvoir les innovations des femmes * Importance de l'égalité des sexes pour évoluer les politiques publiques + [02:15:16][^6^][6] Intégration du genre dans la transition écologique * Les inégalités de genre exacerbent l'impact de la crise climatique * Propositions pour intégrer le genre dans les stratégies d'adaptation climatique * Valorisation de l'action des femmes et leur intégration dans la prise de décision + [02:22:14][^7^][7] Engagement des femmes dans la transition écologique * Les femmes doivent être des actrices majeures dans la lutte contre le changement climatique * Nécessité d'une approche transversale des politiques climatiques et d'égalité * Importance de la collecte de données sexo-spécifiques pour informer les politiques + [02:27:00][^8^][8] Rôle des femmes dans l'agriculture et la production bio * Évolution du secteur agricole avec une augmentation des femmes chefs d'exploitation * Défis rencontrés par les femmes dans l'adaptation au changement climatique * L'émancipation économique des femmes comme objectif pour la justice sociale et climatique Résumé de la vidéo [02:29:33][^1^][1] - [02:42:34][^2^][2]:

Cette partie de la vidéo aborde les solutions pour construire une société durable et respectueuse de l'égalité de genre. Elle souligne l'importance d'intégrer les spécificités de genre dans les politiques nationales et internationales, en particulier en ce qui concerne les conséquences du dérèglement climatique sur les femmes. La vidéo met en avant la nécessité de sensibiliser et d'accompagner les acteurs économiques sur ces enjeux, ainsi que de lutter contre les stéréotypes de genre dans les métiers verts.

Points forts: + [02:29:33][^3^][3] Intégration des spécificités de genre * Importance dans les politiques face au climat * Impact disproportionné sur les femmes * Solutions portées par les femmes + [02:31:38][^4^][4] Sensibilisation et éducation * Importance de la sensibilisation dès l'école * Lutte contre les stéréotypes de genre * Mixité des métiers verts + [02:34:15][^5^][5] Données genrées et politiques publiques * Nécessité de données pour réduire les inégalités * Engagement dans de nouvelles politiques * Participation des femmes aux décisions + [02:36:28][^6^][6] Diplomatie féministe et développement durable * Spécificité des revendications des femmes * Importance de l'éducation et de la formation * Accès des femmes à tous les métiers

Note: This response was posted by the corresponding author to Review Commons. The content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Manuscript number: RC-2024-02393

Corresponding author(s): Katja Petzold

1. General Statements [optional]

We thank the reviewers for recognising the impact of our manuscript. The reviewers noted the novelty of the miRNA bulge structure, the importance of the three observed binding modes and their potential for use in future structure-based drug design, and the possible importance of the duplex release phenomenon. We are also thankful for the relevant and constructive feedback provided.

Our responses to the comments are written point by point in blue, and any changes in the manuscript are shown in red.

2. Description of the planned revisions

In response to Reviewer 1 - major comment 2

Some of the data is over-interpreted. For example, in Figure 3A, it is concluded that supplementary regions are more important for weaker seeds. Only two 8-mer seeds are present among the twelve target sites and thus it might be difficult to generalize.

We found the relationship between seed type and the effect of supplementary pairing in our data intriguing. To further investigate this effect, we tested whether it exists in published microarray data from HCT116 cells transfected with six different miRNAs (Linsley et al., 2007; Argawal et al., 2015). Here we found that the for the two miRNAs (miR-103 and miR-106b) where we see an impact of supplementary pairing, the difference is primarily driven by 7mer-m8 seeds.

Since the effect appears to be specific to the miRNA, we would like to test whether it can be observed for miR-34a in a larger dataset. Therefore, we plan to transfect HEK293T cells with miR-34a and analyse the mRNA response via RNAseq. We will repeat the analysis shown above, using the predicted number of supplementary pairs to categorise the dataset into groups with or without the effect of supplementary pairing. We will then compare the three seed types within these groups.

In response to Reviewer 2 - minor comment 1, "why was the 34-nt 3'Cy3-labeled miR34a complementary probe shifted up in the presence of AGO?".

We plan to investigate the upper band, which we hypothesise is a result of duplex release, using EMSA to ascertain whether the band height agrees with the size of the duplex.

3. Description of the revisions that have already been incorporated in the transferred manuscript

Reviewer #1

Evidence, reproducibility and clarity

Sweetapple et al. Biophysics of microRNA-34a targeting and its influence on down-regulation

In this study, the authors have investigated binding of miR-34a to a panel of natural target sequences using EMSA, luciferase reporter systems and structural probing. The authors compared binding within a binary and a ternary complex that included Ago2 and find that Ago2 affects affinity and strengthens weak binders and weakens strong binders. The affinity is, however, generally determined by binary RNA-RNA interactions also in the ternary complex. Luciferase reporter assays containing 12 different target sites that belong to one of three seed-match types were tested. Generally, affinity is a strong contributor to repression efficiency. Duplex release, a phenomenon observed for specific miRNA-target complementarities, seems to be more pronounced when high affinity within the binary complex is observed. Furthermore, the authors use RABS for structural probing either in a construct in CIS or binding by the individual miRNA in TRANS or in a complex with Ago2. They find pronounced asymmetric target binding and Ago2 does not generally change the binding pattern. The authors observe one specific structural group that was unexpected, which was mRNA binding with bulged miRNAs, which was expected sterically problematic based on the known structures. MD simulations, however, revealed that such structures could indeed form.

This is an interesting manuscript that contributes to our mechanistic understanding of the miRNA-target pairing rules. The combination of affinity measurements, structural probing and luciferase reporters allow for a broad correlation of target binding and repression strength, which is a well-thought and highly conclusive approach. However, there are a number of shortcomings that are summarized below.

The manuscript is not easy to read and to follow for several reasons. First, many of the sub-Figures are not referenced in the text of the results section (1C, 1D, 2C, 4D), which is somewhat annoying. Figure 4A seems to be mis-labeled. Second, a lot of data is presented in suppl. Figures. It should be considered to move more data into the main text in order to make it easier for readers to evaluate and follow.

Thank you for bringing this to our attention. We have now revised the figure references accordingly.

We have relocated gel images of BCL2, WNT1, MTA2 and the control samples from Figure S3 and S4 to the main results (Figure 2A-B) to improve readability and provide controls and details that aid in clear understanding. Additionally, we have relocated panel C from Figure S6 to Figure 2C to enhance the clarity of our rationale for using polyuridine (pU) in our AGO2 binding assays.

The updated figure is shown below, with changes to the legend marked in red.

Figure 2. Binary and ternary____ complex binding affinities measured by EMSA. (A) Binary (mRNA:miR-34a) binding assays showing examples of BCL2, WNT1 and MTA2. (B) Ternary (mRNA:miR-34a-AGO2) binding assays showing examples of BCL2, WNT1, MTA2, and the three control targets PERFECT, SCRseed, and SCRall. The Cy5 labelled species is indicated with asterisk (*). F indicates the free labelled species (miR34a or mRNA), B indicates binary complex, and T indicates ternary complex. Adjacent titrations points differ two-fold in concentration, with maximum concentrations stated at the top right. Adjacent titration points for MTA2 differed three-fold to assess a wider concentration range. In theternary assay, miRNA duplex release from AGO2 was observed for amongst others BCL2, WNT1, PERFECT, and SCRseed (band indicated with B), while it was not observed for SCRall and MTA2. See Figures S3 and S4 for representative gel images for all targets. See Supplementary files 2 and 3 for all images and replicates. (C) Titrations with increasing miR-34a-AGO2 concentration against Cy5-labelled SCRall (left) or PNUTS (right) comparing the absence and presence of 20 μM polyuridine (pU) during equilibration. pU acted as a blocking agent, reducing nonspecific binding, as seen by the different KD,app values for SCRall and PNUTS after addition of 20 μM pU. Therefore, all final mRNA:miR-34a-AGO2 EMSAs were carried out in the presence of 20 μM pU. Labels are as stated above. (D) Individual binding profiles for each of the 12 mRNA targets assessed by electrophoretic mobility assay (EMSA). Each datapoint represents an individual experiment (n=3). Blue represents results for the binary complex, and green represents results for the ternary complex. Dotted horizontal lines represent the KD,app values, which are also stated in blue and green with standard deviations (units = nM). Note that the x-axis spans from 0.1 to 100,000 in CCND1, MTA2 and NOTCH2, whereas the remaining targets span 0.1 to 10,000.

Some of the data is over-interpreted. For example, in Figure 3A, it is concluded that supplementary regions are more important for weaker seeds. Only two 8-mer seeds are present among the twelve target sites and thus it might be difficult to generalize.

We have revised our wording to recognise that more 8-mer sites would be required to draw a stronger conclusion based on this hypothesis. This hypothesis would be interesting to confirm in a larger dataset but is unfortunately outside of the scope of this paper.

Our hypothesis also aligns with recent data from Kosek et al. (NAR 2023; Figure 2D) where SIRT1 with an 8mer and 7mer-A1 seed was compared. Only the 7mer-A1 was sensitive to mutations in the central region or switching all mismatched to WC pairs.

Page 21 now states:

"This result indicates that the impact of supplementary binding may be greater for targets with weaker seeds, as has been observed earlier in a mutation study of miR-34a binding to SIRT1 (Kosek et al., 2023), although a larger sample size would be needed to confirm this observation."

Furthermore, we found the relationship between seed type and the effect of supplementary pairing in our data intriguing. To further investigate this effect, we tested whether it exists in published microarray data from HCT116 cells transfected with six different miRNAs (Linsley et al., 2007; Argawal et al., 2015). Here we found that the for the two miRNAs (miR-103 and miR-106b) where we see an impact of supplementary pairing, the difference is primarily driven by 7mer-m8 seeds. We therefore plan to test whether the effect can be observed for miR-34a in a larger dataset. We have outlined our preliminary data and planned experiments in Section 2 - description of the planned revisions.

I did not understand why the CIS system shown in 4A is a good test case for miR-34a-target binding. It appears very unnatural and artificial. This needs to be rationalized better. Otherwise it remains questionable, whether these data are meaningful at all.

Thank you for pointing out the need for clearer rationalisation.

The TRANS construct, where the scaffold carries the mRNA targeting sequence, provides reactivity information for the mRNA side only, while the microRNA is bound within RISC, with the backbone protected by AGO2. Therefore, to gain information on the miR-34a side of each complex we used the CIS construct, which provides reactivity information from both the miRNA and mRNA. We used the miRNA and mRNA reactivities to calculate all possible secondary structures for the binary complex, and then compared these structures to the mRNA reactivity in TRANS to find which structure fitted the reactivity patterns observed in the ternary complex.

We have included an additional statement in the manuscript to clarify this point on pages 12-13:

"Two RNA scaffolds were used for each mRNA target; i) a CIS-scaffold: RNA scaffold containing both mRNA target and miRNA sequence separated by a 10 nucleotide non-interacting closing loop, and ii) a TRANS-scaffold: RNA scaffold containing only the mRNA target sequence, to which free miR-34a or the miR-34a-AGO2 complex was bound (Figure 4A). The CIS constructs therefore provided reactivity information on the miRNA side, which is lacking in the TRANS construct, and was used to complement the TRANS data."

It may be worthwhile noting that a non-interacting 10 nucleotide loop was inserted between then miRNA and mRNA of the CIS constructs, allowing the miRNA and mRNA strands to bind and release freely. The reactivity patterns of each mRNA:miRNA duplex were compared between CIS and TRANS, and showed similar base pairing (Figure 4D). Furthermore, we have previously compared the two scaffolds in our RABS methodology paper (Banijamali et al. 2022), where no differences were observed besides reduced end fraying in the CIS construct.

For the TRANS experiments, only one specific scaffold structure is used. This structure might impact binding as well and thus at least one additional and independent scaffold should be selected for a generalized statement.

For each construct, the potential of interaction with the scaffold was tested using the RNAstructure (Reuter & Mathews, 2010)package. Based on the results of this assessment, two different scaffolds were used for our TRANS experiments. The testing and use of scaffolds has now been clarified further on page 13:

"The overall conformation of each scaffold with the inserted RNA was assessed using the RNAstructure (Reuter & Mathews, 2010) package to ensure that the sequence of interest did not interact with the scaffold. If any interaction was observed between the RNA of interest and the scaffold, then the scaffold was modified until no predicted interaction occurred. The different scaffolds and their sequence details are shown in supplementary information (Table S1)."

We have previously examined the scaffold's effect on binding and structure during the development of the RABS method. We tested the same mRNA (SIRT1) in separate, independent scaffolds to verify the consistency of the results. An example of this can be found in the supplementary information (Figure S1a) of Banijamali et al. (2022).

Generally, it would be nice to have some more information about the experiments also in the result section. Recombinant Ago2 is expressed in insect cells and re-loaded with miR-34a, luciferase reporters are transfected into tissue culture cells, I guess.

We have now stated the cell types used for AGO2 expression and luciferase reporter assays in the results.

On page 17 we have included:

"Samples of each of the 12 mRNA targets, as well as miR-34a and AGO2, were synthesised in-house for biophysical and biological characterisation. Target mRNA constructs were produced via solid-phase synthesis while miR-34a was transcribed in vitro and cleaved from a tandem transcript (Feyrer et al., 2020), ensuring a 5' monophosphate group. AGO2 was produced in Sf9 insect cells."

"To measure the affinity of each mRNA target binding to miR-34a, both within the binary complex (mRNA:miR-34a) and theternary complex (mRNA:miR-34a-AGO2), we optimised an RNA:RNA binding EMSA protocol to suit small RNA interactions. The protocol is loosely based on Bak et al. (2014)36, with major differences being use of a sodium phosphate buffering system so as not to disturb weaker interactions (James et al., 1996; Stellwagen et al., 2000), supplemented with Mg2+ as a counterion to reduce electrostatic repulsion between the two negatively charged RNAs (Misra & Draper, 1998), and fluorescently labelled probes."

Page 19:

" We successfully tested various RNA backgrounds, including polyuridine (pU) and total RNA extract (Figure S6B) to block any unspecific binding. Ultimately, we supplemented our binding buffer with pU at a fixed concentration of 20 µM for the ternary assays to achieve the greatest consistency."

Page 20:

"Repression efficacy for the 12 mRNA targets by miR-34a was assessed through a dual luciferase reporter assay6. Target mRNAs were cloned into reporter constructs and transfected into HEK293T cells."

Page 22:

"To infer base pairing patterns and secondary structure for each of the 12 mRNA:miR-34a pairs, we used the RABS technique (Banijamali et al., 2023) with 1M7 as a chemical probe. All individual reactivity traces are shown in Figure S9. Reactivity of each of the 22 miR-34a nucleotides was assessed upon binding to each of the 12 mRNA targets within a CIS construct, containing both miR-34a and the mRNA target site separated by a non-interacting 10-nucleotide loop. The two RNAs can therefore bind and release freely within the CIS construct and reactivity information is collected from both RNA strands."

In the first sentence of the abstract, Argonaute 2 should be replaced by Argonaute only since other members bind to miRNAs as well.

Thank you for recognising this. It has now been corrected.

Significance

This is an interesting manuscript that contributes to our mechanistic understanding of the miRNA-target pairing rules. The combination of affinity measurements, structural probing and luciferase reporters allow for a broad correlation of target binding and repression strength, which is a well-thought and highly conclusive approach. However, there are a number of shortcomings.

We thank the reviewer for recognising the approach and impact of our work. In addition we thank the reviewer for identifying the need for further data to support our conclusions from the luciferase assays, which is something that we plan to address, as described in section 2.

---

---

Reviewer #2

Evidence, reproducibility and clarity

Summary: Sweetapple et al. took the approaches of EMSA, SHAPE, and MD simulations to investigate target recognition by miR-34a in the presence and absence of AGO2. Surprisingly, their EMSA showed that guide unloading occurred even with seed-unpaired targets. Although previous studies reported guide unloading, they used perfectly complementary guide and target sets. The authors of this study concluded that the base-pairing pattern of miR-34a with target RNAs, even without AGO2, can be applicable to understanding target recognition by miR-34a-bound AGO2.

Major comments:

(Page 11 and Figure S4) The authors pre-loaded miR-34a into AGO2 and subsequently equilibrated the RISC with a 5' modified Cy5 target mRNA. Since properly loaded miR-34a is never released from AGO2, it is impossible for the miR-34a loaded into AGO2 to form the binary complex (mRNA:miR-34a) in the EMSA (guide unloading has been a long-standing controversy). However, they observed bands of the binary complex in Figure S4. The authors did not use ion-exchange chromatography. AGOs are known to bind RNAs nonspecifically on their positively charged surface. Is it possible that most miR-34a was actually bound to the surface of AGO2 instead of being loaded into the central cleft? This could explain why they observed the bands of the binary complex in EMSA.

Thank you for mentioning this crucial point which has been a focus of our controls. We have addressed this point in four ways:

Salt wash during reverse IMAC purification. Separation of unbound RNA and proteins via SEC. Blocking non-specific interactions using polyuridine. Observing both the presence and absence of duplex release among different targets using the same AGO2 preparation and conditions.

Firstly, although we did not use a specific ion exchange column for purification, we believe the ionic strength used in our IMAC wash step was sufficient to remove non-specific interactions. We used A linear gradient with using buffer A (50 mM Tris-HCl, 300 mM NaCl, 10 mM Imidazole, 1 mM TCEP, 5% glycerol v/v) and buffer B (50 mM Tris-HCl, 500 mM NaCl, 300 mM Imidazole, 1 mM TCEP, 5% glycerol) at pH 8. The protocol followed recommendation by BioRad for their Profinity IMAC resins where it is stated that 300 mM NaCl should be included in buffers to deter nonspecific protein binding due to ionic interactions. The protein itself has a higher affinity for the resin than nucleic acids.

A commonly used protocol for RISC purification follows the method by Flores-Jasso et al. (RNA 2013). Here, the authors use ion exchange chromatography to remove competitor oligonucleotides. After loading, they washed the column with lysis buffer (30 mM HEPES-KOH at pH 7.4, 100 mM potassium acetate, 2 mM magnesium acetate and 2 mM DTT). AGO was eluted with lysis buffer containing 500 mM potassium acetate. Competing oligonucleotides were eluted in the wash.

As ionic strength is independent of ion identity or chemical nature of the ion involved (Jerermy M. Berg, John L. Tymoczko, Gregory J. Garret Jr., Biochemistry 2015), we reasoned that our Tris-HCl/NaCl/ imidazole buffer wash should have at comparable ionic strength to the Flores-Jasso protocol.

Our total ionic contributions were: 500 mM Na+, 550 mM Cl-, 50 mM Tris and 300 mM imidazole. We recognise that Tris and imidazole are both partially ionized according the pH of the buffer (pH 8) and their respective pKa values, but even if only considering the sodium and chloride it should be comparable to the Flores-Jasso protocol.

We have restated the buffer compositions below written the methods section more explicitly to describe this:

"Following dialysis, any precipitate was removed by centrifugation, and the resulting supernatant was loaded onto a IMAC buffer A-equilibrated HisTrap-Ni2+ column to remove TEV protease, other proteins, and non-specifically bound RNA. A linear gradient was employed using IMAC buffers A and B."

Secondly, after reverse HisTrap purification, AGO2 was run through size exclusion chromatography to remove any remaining impurities (shown Figure S2B).

Thirdly, knowing that AGO2 has many positively charged surface patches and can bind nucleic acid nonspecifically (Nakanishi, 2022; O'Geen et al., 2018), we tested various blocking backgrounds to eliminate nonspecific binding effects in our EMSA ternary binding assays. We were able to address this issue by adding either non-homogenous RNA extract or homogenous polyuridine (pU) in our EMSA buffer during equilibration background experiments. This allowed us to eliminate non-specific binding of our target mRNAs, as shown previously in Supplementary Figure S6. We appreciate that the reviewer finds this technical detail important and have moved the panel C of figure S6 into the main results in Figure 2C, to highlight the novel conditions used and important controls needed to be performed. If miR-34a were non-specifically bound to the surface of AGO2 after washing, this blocking step would render any impact of surface-bound miR-34a negligible due to the excess of competing polyuridine (pU).

Our EMSA results show that, using polyU, we can reduce non-specific interaction between AGO2 and RNAs that are present. And still, duplex release occurs despite the blocking step. It is therefore less likely that duplex release is caused by surface-bound miR-34a.

Finally, the observation of distinct duplex release for certain targets, but not for others (e.g. MTA2, which bound tightly to miR-34a-AGO2 but did not exhibit duplex release; see Figure 2), argues against the possibility that the phenomenon was solely due to non-specifically bound RNA releasing from AGO2.

In response to the reviewers statement "Since properly loaded miR-34a is never released from AGO2, it is impossible for the miR-34a loaded into AGO2 to form the binary complex (mRNA:miR-34a)" we would like to refer to the three papers, De et al. (2013) Jo MH et al. (2015), and Park JH et al. (2017), which have previously reported duplex release and collectively provide considerable evidence that miRNA can be unloaded from AGO in order to promote turnover and recycling of AGO. It is known that AGO recycling must occur, therefore there must be some mechanisms to enable release of miRNA from AGO2 to enable this. It is possible that AGO recycling proceeds via miRNA degradation (TDMD) in the cell, but in the absence of enzymes responsible for oligouridylation and degradation, the miRNA duplex may be released. As TDMD-competent mRNA targets have been observed to release the miRNA 3' tail from AGO2 (Sheu-Gruttadauria et al., 2019; Willkomm et al., 2022), there is a possible mechanistic similarity between the two processes, however, we do not have sufficient data to make any statement on this.

(Page 18 and Figure S5) Previous studies (De et al., Jo MH et al., Park JH et al.) reported guide unloading when they incubated a RISC with a fully complementary target. However, neither MTA2, CCND1, CD44, nor NOTCH2 can be perfectly paired with miR-34a (Figure 1A). Therefore, the unloading reported in this study is quite different from the previously reported works and thus cannot be explained by the previously reported logic. The authors need to explain the guide unloading mechanism that they observed. Otherwise, they might misinterpret the results of their EMSA and RABS of the ternary complex.

The three aforementioned studies have reported unloading/duplex release. However, they did not only report fully complementary targets in this process.

De et al. (2013) reported that "highly complementary target RNAs promote release of guide RNAs from human Argonaute2".

Subsequently, Park et al. (2017) reported: "Strikingly, we showed that miRNA destabilization is dramatically enhanced by an interaction with seedless, non-canonical targets."

A figure extracted from Figure 5 of Park et al. is shown below illustrating the occurrence of unloading in the presence of seed mismatches in positions 2 and 3 (mm 2-3). Jo et al. (2015) also reported that binding lifetime was not affected by the number of base pairs in the RNA duplex.

In addition to these three reports, a methodology paper focusing on miRNA duplex release was published recently titled "Detection of MicroRNAs Released from Argonautes" (Min et al., 2020).

Therefore, we do believe that the previously observed microRNA release is similar to our observation. Here we also correlate it to structure and stability of the complex.

(Page 20) The authors reported, "it is notable that the seed region binding does not appear to be necessary for duplex release." The crystal structures of AGO2 visualize that the seed of the guide RNA is recognized, whereas the rest is not, except for the 3' end captured by the PAZ domain. How do the authors explain the discrepancy?

In this manuscript, we intend to present our observations of duplex release. There are many potential relationships between duplex release and AGO2 activity, which we do not have data to speculate upon. Previous studies, such as Park et al. (2017) have also observed non-canonical and seedless targets leading to duplex release, supporting our findings. Additionally, other publications including McGearly et al. (2019) report 3'-only miRNA targets, Lal et al. (2009) have documented seedless binding by miRNA and their downstream biological effects, and Duan et al. (2022) show that a large number of let-7a targets are regulated through 3′ non-seed pairing.

It is also possible that duplex release is not coupled to classical repression outcomes, and does not need to proceed by the seed, but instead regulates AGO2 recycling before AGO2 enters the quality control mode of recognising the formed seed.

(Pages 22) The authors mentioned, "It follows that the structure imparted via direct RNA:RNA interaction remains intact within AGO2, highlighting the role of RNA as the structural determinant." A free guide and a target can start their annealing from any nucleotide position. In contrast, a guide loaded into AGO needs to start annealing with targets through the seed region. Additionally, the Zamore group reported that the loaded guide RNA behaves quite differently from its free state (Wee et al., Cell 2012). How do the authors explain the discrepancy?

The key point we would like to emphasise is that AGO does not seem to alter the underlying RNA:RNA interactions. The bound state in the ternary complex reflects the structure established in the binary complex. We do not aim to claim a specific sequence of events, as this interpretation is not possible from our equilibrium data. Our data indicates that the protein is flexible enough to accommodate the RNA structure that is favoured in the binary complex. This hypothesis is further supported by our MD simulation, which demonstrates the accommodation of a miRNA-bulge structure within AGO2.

Targets lacking seeds have been identified previously (McGeary et al. 2019, Park et al. 2017, Lal et al. 2009) and can bind to miRNA within AGO. Therefore, there must be a mechanism by which these targets can anneal within AGO, such as via sequence-independent interactions (as discussed in question 3).

With respect to Wee et al., (2012), which studied fly and mouse AGO2 and found considerable differences between the thermodynamic and kinetic properties of the two AGO2 species. Furthermore, they found different average affinities between the two species, with the fly AGO binding tighter the mouse. Following this logic, it is not unexpected that human AGO2 would have unique properties compared to those of fly and mouse.

Below is an extract from Wee et al., (2012):

"Our KM data and published Argonaute structures (Wang et al., 2009) suggest that 16-17 base pairs form between the guide and the target RNAs, yet the binding affinity of fly Ago2-RISC (KD = 3.7 {plus minus} 0.9 pM, mean {plus minus} S.D.) and mouse AGO2-RISC (KD = 20 {plus minus} 10 pM, mean {plus minus} S.D.) for a fully complementary target was comparable to that of a 10 bp RNA:RNA helix. Thus, Argonaute functions to weaken the binding of the 21 nt siRNA to its fully complementary target: without the protein, the siRNA, base paired from positions g2 to g17, is predicted to have a KD ∼3.0 × 10−11 pM (ΔG25{degree sign}C = −30.7 kcal mol−1). Argonaute raises the KD of the 16 bp RNA:RNA hybrid by a factor of > 1011."

In the Wee et al. (2012) paper, affinity data on mouse and fly AGO2 was collected via filter binding assays, using a phosphorothioate linkage flanked by 2′-O-methyl ribose at positions 10 and 11 of the target to prevent cleavage. They then compared the experimentally determined mean KD and ΔG values for each species to predicted values of an RNA:RNA helix of 16-17 base-pairs. No comparison was made between individual targets, and no experimental data was collected for the RNA:RNA binding. The calculated energy values were made based on a simple helix without taking into account any possible secondary structure features. Considering the different AGO species, alternative experimental setup, modified nucleotides in the tested RNA, and the computationally predicted RNA values compared to the averaged experimental values, we believe there is considerable reason to observe differences compared to our findings.

We have expanded our discussion on page 27 to the following:

"An earlier examination of mRNA:miRNA binding thermodynamics by Wee and colleagues (2012) found that mouse and fly AGO2 reduce the affinity of a guide RNA for its target61. Our data indicate that the range of miR-34a binary complex affinities is instead constricted by human AGO2 in the ternary complex - strengthening weak binders while weakening strong binders. The 2012 study reported different average affinities between the two AGO2 species, with the fly protein binding tighter the mouse. Following this logic, it is not unexpected that human AGO2 would have unique properties compared to those of fly and mouse."

The authors concluded that the range of binary complex affinities is constricted by human AGO2 in the ternary complex - strengthening weak binders while weakening strong binders. This may hold true for miR-34a, but it cannot be generalized. Other miRNAs need to be tested.

That is true, we have now adjusted the wording to encompass this more clearly, shown below. Testing of further miRNAs is the likely content of future work from us and others.

"Our data indicate that the range of miR-34a binary complex affinities is instead constricted by human AGO2 in the ternary complex - strengthening weak binders while weakening strong binders."

Minor comments:

(Figure S2) Why was the 34-nt 3'Cy3-labeled miR34a complementary probe shifted up in the presence of AGO?

We believe this observation is also indicative of duplex release. At the time that these activity assays were collected, we were not as aware of the presence of duplex release so did not test it further, assuming it may be due to transient interactions. We plan to investigate this via EMSA and have included this in the planned revisions (section 2).

2.(Page 17) Does the Cy3 affect the interaction of the 3' end of miR-34 with AGO2?

miR-34a-3'Cy5 was used for binary experiments only and the reverse experiment was conducted as a control (where Cy5 was located on the mRNA) (Figure S3b), showing no change in affinity/interaction when the probe was switched to the target. For ternary experiments the mRNA target was labelled on the 5' terminus, to make sure there was no interference with loading miR-34a into AGO2.

A Cy3 labelled RNA probe (fully complementary to miR-34a) was used to detect miR-34a in northern blots, but AGO2 interaction is not relevant here under denaturing conditions.

Otherwise, the 34-nt slicing probe had Cy3 on the 5 nt 3' overhang and should therefore not interact with AGO.

1. Several groups reported that overproduced AGOs loaded endogenous small RNAs. The authors should mention that their purified AGO2 was not as pure as a RISC with miR-34a. Otherwise, readers might think that the authors used a specific RISC.

We have now improved our explanation of the loading efficiency to make it more clear to the reader that our AGO2 sample was not fully bound by miR-34a, and that all concentrations refer to the miR-34a-loaded portion of AGO2. The following text can be found in the results on page 18:

"The mRNA:miR-34a-AGO2 assay had a limited titration range, reaching a maximum miR-34a-AGO2 concentration of 268 nM due to a 5% loading efficiency (see Figure S2D for loading efficiency quantification). The total AGO2 concentration was thus 20-fold higher than the miR-34a-loaded portion. Further increase in protein concentration was prevented by precipitation. Weaker mRNA targets (CD44, CCND1, and NOTCH2) did not reach a saturated binding plateau within this range, leading to larger errors in their estimated KD,app values. However, reasonable estimation of the KD,app was possible by monitoring the disappearance of the free mRNA probe. Note that we refer to the miR-34a-loaded portion of AGO2 when discussing concentration values for all titration ranges. To ensure AGO2 binding specificity despite low loading efficiency, a scrambled control was used (SCRall; lacking stable base pairing with miR-34a or other human miRNAs according to the miRBase database57). SCRall showed no interaction with miR-34a-AGO2 (Figure 2B)."

(Figure legend of Figure S5) Binding was assessed "by."

Thank you for pointing this out, it is now fixed.

(Page 17) It would be great if the authors could even briefly describe the mechanism by which the sodium phosphate buffer with magnesium does not disturb weaker interactions by citing reference papers.

We have now added a supplementary methods section to our manuscript and included the description below on page 10:

"We found that a more traditional Tris-borate-EDTA (TBE) buffer disrupted weaker RNA:RNA binding interactions (Supplementary Methods Figure M1). Borate anions form stable adducts with carbohydrate hydroxyl groups (James et al., 1996) and can form complexes with nucleic acids, likely through amino groups in nucleic bases or oxygen in phosphate groups (Stellwagen et al., 2000). This makes TBE unsuitable for assessment of RNA binding, particularly involving small RNA molecules, which typically have weaker affinities. We therefore adapted our buffer system to a sodium phosphate buffer supplemented with magnesium. Magnesium acts as a counterion to reduce electrostatic repulsion between the two negatively charged backbones by neutralisation (Misra et al., 1998)."

We have also clarified the buffer adaptions in our results section on page 17:

The protocol is loosely based on Bak et al. (2014)36, with major differences being use of a sodium phosphate buffering system so as not to disturb weaker interactions(James et al., 1996; Stellwagen et al., 2000), supplemented with Mg2+ as a counterion to reduce electrostatic repulsion between the two negatively charged RNAs(Misra & Draper, 1998), and fluorescently labelled probes. Original gel images and quantification are shown in supplementary Figures S3 and S4. All KD,app values are shown in Supplementary Table 1, and represent the mean of three independent replicates.

Figure M1. Comparison of Tris-borate EDTA (TBE) and sodium phosphate with magnesium (NaP-Mg2+) buffer systems for EMSA. Cy5-labelled miR-34a and unlabelled CD44 were equilibrated in the two different buffer systems, using the same titration range. No mobility shifts were observed in the TBE system, while clear binding shifts were observed in the NaP-Mg2+ system.

6.(Page 22) The authors cited Figure 4C in the sentence, "Comparison between CIS and TRANS ..." Is this supposed to be Figure 4D?

The reviewer was correct in their assumption, and this has now been corrected.

7.(Figure 6) Readers would appreciate it if the guide and target were colored in red and blue. The color codes have been used in most papers reporting AGO structures. The current color codes are opposite.

We have now adjusted the colour schemes throughout the manuscript, and Figure 6 has been modified to the following:

__"Figure 6. The miRNA-bulge structure is readily accommodated by AGO2 as shown by molecular dynamics simulation. __Panel (A) displays a snapshot of the all-atom MD simulation of miR-34a (red) and NOTCH1 (blue) in AGO2. The NOTCH1:miR-34a duplex is shown with AGO2 removed for clarity and is rotated 90{degree sign} to show the miRNA bulge and bend in the duplex. This NOTCH1:miR-34a-AGO2 structure is compared with (B), which shows the crystal structure of miR-122 (orange) paired with its target (purple) via the seed and four nucleotides in the supplementary region (PDB-ID 6N4O17), and (C), which shows the crystal structure of miR-122 (orange) and its target (green) with extended 3' pairing, necessary for the TDMD-competent state (PDB-ID 6NIT19). AGO2 is depicted in grey, with the PAZ domain in green, and the N-terminal domain marked with N. The miRNA duplexes in (B) and (C) feature symmetrical 4-nucleotide internal loops, whereas the NOTCH1 structure in (A) has an asymmetrical miRNA bulge with five unpaired nucleotides on the miRNA side and a 3-nucleotide asymmetry."

Significance

This paper will have a significant impact on the field if seed-unpaired targets can indeed unload guide RNAs. The authors may want to validate their results very carefully.

We thank the reviewer for recognising the significance of duplex release (or guide unloading) from AGO2. We agree that the observations should be tested rigorously and have outlined the actions we took to ensure validity in our AGO2 preparation.

__Reviewer #3 __

Evidence, reproducibility and clarity (Required):

In this manuscript, the authors use a combination of biochemical, biophysical, and computational approaches to investigate the structure-function relationship of miRNA binding sites. Interestingly, they find that AGO2 weakens tight RNA:RNA binding interactions, and strengthens weaker interactions.

Given this antagonistic role, I wonder: shouldn't there be an 'average' final binding affinity? Furthermore, if I understand correctly, not many trends were observed to correlate binding affinity with repression, etc.

Overall, there was no 'average' final binding affinity observed, as the binary assays had a much higher maximum (NOTCH2binary affinity was within the micromolar range) skewing the mean average of the binary affinities to 657 nM, versus 111 nM for the ternary affinities. We also compare the variances of the binary and ternary affinity datasets using the F-test and found that F > F(critical one tail) and thus the variation of the two populations is unequal (binary variation is significantly larger than ternary).

F-Test Two-Sample for Variances

• *

binary affinity

ternary affinity

Mean

657.3

110.971667

Variance

2971596.1

24406.4012

Observations

12

12

df

11

11

F

121.754784

P(F

7.559E-10

F(critical one-tail)

2.81793047

We agree that the overall correlation between affinity and repression was not strong, although we found a stronger correlation within the miRNA-bulge group (Figure 5C and S7C). A larger sample size of miRNA bulge-forming duplexes would be needed to test the generalizability of this observation.

Given the context of the study - whereby structure is being investigated as a contributing factor to the interaction between the miRNA and mRNA, I find it interesting that the authors chose to use MC-fold to predict the structures of the mRNA, rather than using an experimental approach to assess / validate the structures. Thirty-seven RNAs were assessed; I think even for a subset (the 12 that were focused on in the study), the secondary structure should be validated experimentally (e.g., by chemical probing experiments, which the research group has demonstrated expertise in over the last several years). The validation should follow the in silico folding approach used to narrow down the region of interest. It is necessary to know whether an energy barrier (associated with the mRNA unfolding) has to occur prior to miRNA binding; this could help explain some of the unexplained results in the study. Indeed, the authors mention that there are many variables that influence miRNA regulation.

Indeed, experimentally validated structures offer valuable insights that cannot be obtained solely through sequence-based predictions. This is why we opted to employ our RABS method to experimentally evaluate the binary and ternary complex binding of our 12 selected targets (as depicted in Figures 4 and S9 and discussed in the text on pages 23-24). While we (in silico) assessed all 37 RNA targets that were experimentally confirmed at the time, selecting 12 to represent both biological and predicted structural diversity, it would have been impractical to experimentally pre-assess all the targets not included in the final selection. Our in-silico assessment was designed to narrow down the regions of interest and evaluate predicted secondary structures present. The pipeline is shown in Figure 1. Details of the code used in the in-silico analysis are provided in Supplementary File 1.

Regarding the energy of unfolding of mRNA, our constructs considered the isolated binding sites thus the effects of surrounding mRNA interactions were removed. We compared our affinities to dG as well as MFE and have now included this analysis in Figure S8A. Additionally, we have included the text on page 27-28 of the discussion:

"Gibbs free energy (G), which is often included in targeting prediction models as a measure of stability of the miRNA:mRNA pair12,62, correlated with the log of our binary KD,app values, using ΔG values predicted by RNAcofold (R2 = 0.61). There was a weaker correlation with the free energy values derived from the minimum free energy (MFE) structures predicted by RNAcofold (R2 = 0.41) (Figure S8A). This result highlights the contribution of unfolding (in ΔG) as being an important in predicting KD. The differences between ΔG and KD,app are likely primarily due to inaccurately predicted structures used for energy calculations."

Additionally, we assessed the free form of all mRNA targets via RABS (Figure S9) and observed that the seed of each free mRNA was available for miRNA binding (seeds of the free mRNA were not stably bound).

Finally, when designing our luciferase plasmids we used RNAstructure (Reuter & Mathews, 2010) to check for self-folding effects which could interfere with target site binding and ensured that all plasmids were void of such effects.

In the methods, T7 is italicized by accident in the T7 in vitro transcription section. Bacmid is sometimes written with a capital B and other times with a lower-cased b. The authors should be consistent. The concentration of TEV protease that was added (as opposed to the volume) should be described for reproducibility.

Thank you for pointing out these overlooked points. They have now been corrected.

In figure S2D, what is the second species in the gel on the right-hand side of the gel in the miR-34a:AGO lanes? The authors should mention this.

We believe that the faint upper band corresponds to other longer RNA species loaded into AGO2. As AGO2 is loaded with a diversity of RNA species, it is likely that some of them may have a weak affinity for the miR-34a-complementary probe, and therefore show up on the northern blot.

Figure S3B and S3A are referenced out of order in the text. In regard to S3A, what are the anticipated or hypothesized alternative conformations for NOTCH1, DLL1, and MTA2? There are really interesting things going on in the gels, also for HNF4a and NOTCH2. Can the authors offer some explanation for why the free RNA bands don't seem to disappear, but rather migrate slowly? Is this a new species?

The order of the figure references have now been updated, thank you for alerting us to this.

Figure S3A: For MTA2, the two alternative conformations are shown in Figure S9 and S10 (and shown below here, miR-34aseed marked in pink). It appears that a single conformation is favoured at high concentration (> 1 µM) while the two conformations are present at {less than or equal to} 1 µM. The RABS data for MTA2 also indicated multiple binding conformations, as the reactivity traces were inconsistent. We expect that the conformation shown on the left was most dominant within AGO2, based on the reactivity of the TRANS + AGO assays. However, we cannot exclude a possible G-quadruplex formation due to the high G content of MTA2 (shown below right).

Regarding NOTCH1 and DLL1, a faint fluorescent shadow was observed beneath the miR-34a bound band. The RABS reactivity traces indicated a single dominant conformation for these targets, so it is possible that the lower shadow observed was due to more subtle differences in conformation, such as the opening/closing of one or a few base pairs at the terminus or bulge, (i.e. end fraying). HNF4α and NOTCH2 appear to never fully saturate the miR-34a, so a small un-bound population remains visible on the gel. For NOTCH2 this free miR-34a band appears to migrate upwards, possibly due to overloading the gel lane with excess NOTCH2 (which are not observed in the Cy5 fluorescence image).

In the EMSA for Perfect, why does the band intensity for the bound complex increase then decrease? How many replicates were run for this? This needs to be reconciled.

As for all EMSAs, three replicates were carried out for each mRNA target and all gels are shown in Supplementary Files 2 and 3, for the binary and ternary assays respectively.

Uneven heat distribution across the gel can lead to bleaching of the Cy5 fluorophore. To address this, we we used a circulating cooler in our electrophoresis tank, as outlined in our methods (page 10). However, the aforementioned gel for one of thePERFECT sample replicates appears to have been evenly cooled. As the binding ratio (rather than total band volume) was used for quantification, the binding curve was unaffected, and this did not influence KD,app.

We have now replaced the exemplary gel for PERFECT in Figure S3 with a more representative and evenly labelled gel from our replicates (Cy5 fluorescence image shown below). The binding curve for PERFECT is also shown here:

The authors list that the RNA concentration was held constant at 10 nM; in EMSAs, the RNA concentration should be less than the binding affinity; what is the lowest concentration of protein used in the assays shown in S3A? Is this a serial dilution? It seems to me like the binding assays for MTA2, Perfect, and SRCseed might have too high of an RNA concentration. (Actually, now I see in the supplement the concentrations of proteins, and the RNA concentration is too high). Also, why is the intensity of bands for bound complex for SRCseed more intense than the free RNA?

Why are the binding affinity error bars so large (e.g., for NOTCH2 with mir-34a) - 6 uM +/- 3 uM?

No protein was used in the binary assays shown in Figure S3A. For the ternary assays in Figure S4, the maximum concentration of miR-34a-loaded AGO2 (miR-34a-AGO2) was 268 nM, with a serial dilution down to a minimum of 0.06 nM.

Optimal EMSA conditions require a constant RNA concentration that is lower than the binding affinity to accurately estimate high-affinity interactions.

For our tightest binders, such as SIRT1, we can confidently state that the KD,app is less than 10 nM, estimated at 0.4 {plus minus} 1.1 nM. Therefore, the accuracy of this estimation is reduced, and the standard deviation is larger than the estimated KD,app. As NOTCH2 bound miR-34a very weakly and did not reach a fully bound plateau, the resulting high error was expected. Consequently, we do not have the same level of certainty for extremely tight or weak binders. In this study, the relative affinities were of primary importance.

We have included on page 18:

As the Cy5-miR-34a concentration was fixed to 10 nM to give sufficient signal during detection, KD,app values below 10 nM have a lower confidence.

Regarding the control samples PERFECT and SCRseed, our focus was not on determining the exact KD,app of these artificial constructs. Instead, we were primarily interested in whether they exhibited binding and under which conditions. For SCRseed, we neither adjusted the titration range nor calculated KD,app. For PERFECT, the concentration was adjusted to a lower range of 30 nM - 0.001 nM to give a relative comparison with the other tight binder SIRT1. However, further reduction in RNA concentration was not pursued, as it already fell well below the 10 nM sensitivity threshold.

Regarding the intensity of the bound SCRseed band, we observed that the bound fluorophore often resulted in stronger intensity than for the free probe. This was observed for a number of the samples (PERFECT, BLC2, SCRseed). A previous publication reported that Cy5 is sequence dependent in DNA, that the effect is more sensitive to double-stranded DNA, and that the fluorophore is sensitive to the surrounding 5 base pairs (Kretschy, Sack and Somoza, 2016). It is likely that the same phenonenon exists in RNA.

For MTA2, the two alternative conformations (shown in Figure S9 and S10) make assessment of KD,app more difficult. As the higher affinity conformation did not reach a fully-bound plateau before the weaker affinity conformation appeared, the binding curve plateau (where all miR-34a was bound) reflected the weaker conformation KD,app. We increased the range of titration tested by using a three-fold serial dilution, but further reduction in RNA concentration would not have been fruitful as it already dropped below well below the 10 nM sensitivity range. Therefore the MTA2 binary complex had a higher error at (944 {plus minus} 274 nM) and lower confidence.

We then decided to run a competition assay to detect the weaker KD,app of MTA2. The assay was set up using the known binding affinity of CD44, which was labelled with Cy5 to track the reaction. MTA2 was titrated against a constant concentration of Cy5-CD44:miR-34a, and disruption of the CD44 and miR-34a binding was monitored. We fitted the data to a quadratic for competitive binding (Cheng and Prusoff., 1973) to calculate the KD,app for competitive binding, or KC,app.

We validated our competition assay by comparing it with our direct binding assays, specifically assessing CD44 in a self-competition assay. The CD44 KC,app (168 {plus minus} 24 nM; mean and SD of three replicates) was found to be consistent with the KD,app obtained from the direct assay (165 {plus minus} 21 nM).

As we wanted all affinity data to be directly comparable (using the same methodology), we compared the KD,app values obtained via direct assay in the manuscript. It appears that the competitive EMSA assay for MTA2 reflects the weaker affinity conformation observed in the direct assay.

It would be very helpful if the authors wrote in the Kds in Figure 2A in green and blue (in the extra space in the plots). This would help the reader to better understand what's going on, and for me, as a reviewer, to better consider the analysis/conclusions presented by the authors.

KD,app values are written in in green and blue in what is now Figure 2D (originally Figure 2A).

The authors state on page 18 that 'Interestingly, however, we did not observe a correlation between binary or ternary complex affinity and seed type.' They should elaborate on why this is interesting.

The prevailing view is that the miRNA seed type significantly influences affinity within AGO2. The largest biochemical studies of miRNA-target interactions to date, conducted by McGeary et al. (2019, 2022), used AGO-RBNS (RNA Bind-n-Seq) to reveal relative binding affinities. These studies demonstrated strong correlations between the canonical seed types and binding affinity. Therefore, we find it interesting that no such correlation was observed in our dataset (despite its small size).

We have now added to the manuscript (page 20):

"The largest biochemical studies of miRNA-target interactions to date (McGeary et al., 2019, 2022) used AGO-RBNS (RNA Bind-n-Seq) to extract relative binding affinities, demonstrating strong correlations between the canonical seed types and binding affinity. Therefore, it is intriguing that our dataset, despite its small size, showed no such correlation."

Figure 2C is not referenced in the text (the authors should go back through the text to make sure everything is referenced and in order). The Kds should be listed alongside the gels in Figure 2C.

Figure 2 has now been rearranged and updated, with KD,app values listed in what is now Figure 2D.

Figure 3B is rather confusing to understand.

We have now adapted Figure 3 to simplify readability. Panel B has now been moved to C, and we have introduced panel A (moved from Figure 2B). In Figure 3C (originally 3B) we have added arrows to indicate the direction of affinity change from binary to ternary complex, and moved the duplex release information to panel A. We thank the reviewer and think that the data is now much clearer.

Figure 3. AGO2 moderates affinity by strengthening weak binders and weakening strong binders. (A) Correlation of relative mRNA:miR-34a with mRNA:miR-34aAGO2 binding affinities. No seed type correlation is observed, seeds coloured, where 8mer is pink, 7mer-m8 is turquoise, and 7-mer-A1 is mauve. The slope of the linear fit is 0.48, and intercept on the (log y)-axis is 7.11. The occurrence of miRNA duplex release from AGO2 is marked with diamonds. (B) miR-34a-mediated repression of dual luciferase reporters fused to the 12 mRNA targeting sites. Luciferase activity from HEK293T cells co-transfected with each reporter construct, miR-34a was measured 24 hours following transfection and normalised to the miR-34a-negative transfection control. Each datapoint represents the R/F ratio for an independent experiment (n=3) with standard deviations indicated. SCRseed is a scrambled seed control, SCRall is a fully scrambled control, and PERFECT is the perfect complement of miR-34a. Dotted horizontal lines represent the repression values for the 22-nucleotide seed-only controls6 for the respective seed types, in the absence of any other WC base pairing. (C) Comparison of relative target repression with relative affinity assessed by EMSA. Blue represents mRNA:miR-34a affinity (binary complex), while green represents mRNA:miR-34a-AGO2 affinity (ternary complex). Arrows indicate the direction of change in affinity upon binding within AGO2 compared to the binary complex. It is seen that AGO2 moderates affinity bi-directionally by strengthening weak binders and weakening strong binders.

Page 20: Perfect should be italicized.

Thank you for bringing this to our attention, this how now been adjusted.

Have the authors considered using NMR to assess the base pair pattern formed between the miRNA:mRNA complexes (with / without AGO)? As a validation for results obtained by RABS? This could be helpful for the Asymmetric target binding section, the Ago increases flexibility section, and the three distinct structural groups section in the results. It is widely accepted that while chemical probing is insightful, results should be validated using alternative approaches. Distinguishing structural changes and protected reactivity in the presence of protein is challenging.

NMR provides high-resolution information on RNA base-pairing patterns, allowing us to compare our RABS results for SIRT1with those obtained via NMR (Banijamali et al., 2022) for the binary complex. For SIRT1, the RNA:RNA structures identified were consistent between both methods. However, using NMR to measure RNA:RNA binding within AGO2 is challenging due to the protein's large size. Currently, there are no published complete NMR structures of RNA within AGO2. The largest solution-state NMR structures published that include AGO consist solely of the PAZ domain. Our group has been working on method development using DNP-enhanced solid-state NMR to obtain structural information within the complete AGO2 protein, but the current resolution does not allow us to fully reconstruct a complete NMR structure. We hope that in the coming years, this will be a method to evaluate RNA within AGO. This limitation highlights the advantage of RABS in providing RNA base-pairing information within the ternary complex in solution.

Reviewer #3 (Significance (Required)):

The work is helpful for understanding how microRNAs recognize and bind their mRNA targets, and the impact Ago has on this interaction. I think for therapeutic studies, this will be helpful for structure-based design. Especially given the three types of structures identified to be a part of the interaction.

We thank the reviewer for their detailed remarks, especially concerning the importance of technical details the binding assays. We further thank the reviewer for recognising the potential impact of our work for rational design.

4. Description of analyses that authors prefer not to carry out

• *

In response to Reviewer 2 - major comment 1, we prefer to not run an additional ion exchange purification on the AGO2 protein due to the reasoning discussed above, which is repeated here:

We have addressed this point in three ways:

Thank you for mentioning this crucial point which has been a focus of our controls. We have addressed this point in four ways:

Salt wash during reverse IMAC purification. Separation of unbound RNA and proteins via SEC. Blocking non-specific interactions using polyuridine. Observing both the presence and absence of duplex release among different targets using the same AGO2 preparation and conditions.

Firstly, although we did not use a specific ion exchange column for purification, we believe the ionic strength used in our IMAC wash step was sufficient to remove non-specific interactions. We used A linear gradient with using buffer A (50 mM Tris-HCl, 300 mM NaCl, 10 mM Imidazole, 1 mM TCEP, 5% glycerol v/v) and buffer B (50 mM Tris-HCl, 500 mM NaCl, 300 mM Imidazole, 1 mM TCEP, 5% glycerol) at pH 8. The protocol followed recommendation by BioRad for their Profinity IMAC resins where it is stated that 300 mM NaCl should be included in buffers to deter nonspecific protein binding due to ionic interactions. The protein itself has a higher affinity for the resin than nucleic acids.

A commonly used protocol for RISC purification follows the method by Flores-Jasso et al. (RNA 2013). Here, the authors use ion exchange chromatography to remove competitor oligonucleotides. After loading, they washed the column with lysis buffer (30 mM HEPES-KOH at pH 7.4, 100 mM potassium acetate, 2 mM magnesium acetate and 2 mM DTT). AGO was eluted with lysis buffer containing 500 mM potassium acetate. Competing oligonucleotides were eluted in the wash.

As ionic strength is independent of ion identity or chemical nature of the ion involved (Jerermy M. Berg, John L. Tymoczko, Gregory J. Garret Jr., Biochemistry 2015), we reasoned that our Tris-HCl/NaCl/ imidazole buffer wash should have at comparable ionic strength to the Flores-Jasso protocol.

Our total ionic contributions were: 500 mM Na+, 550 mM Cl-, 50 mM Tris and 300 mM imidazole. We recognise that Tris and imidazole are both partially ionized according the pH of the buffer (pH 8) and their respective pKa values, but even if only considering the sodium and chloride it should be comparable to the Flores-Jasso protocol.

Secondly, after reverse HisTrap purification, AGO2 was run through size exclusion chromatography to remove any remaining impurities (shown Figure S2B).

Thirdly, knowing that AGO2 has many positively charged surface patches and can bind nucleic acid nonspecifically (Nakanishi, 2022; O'Geen et al., 2018), we tested various blocking backgrounds to eliminate nonspecific binding effects in our EMSA ternary binding assays. We were able to address this issue by adding either non-homogenous RNA extract or homogenous polyuridine (pU) in our EMSA buffer during equilibration background experiments. This allowed us to eliminate non-specific binding of our target mRNAs, as shown previously in Supplementary Figure S6. We appreciate that the reviewer finds this technical detail important and have moved the panel C of figure S6 into the main results in Figure 2C, to highlight the novel conditions used and important controls needed to be performed. If miR-34a were non-specifically bound to the surface of AGO2 after washing, this blocking step would render any impact of surface-bound miR-34a negligible due to the excess of competing polyuridine (pU).

Our EMSA results show that, using polyU, we can reduce non-specific interaction between AGO2 and RNAs that are present. And still, duplex release occurs despite the blocking step. It is therefore less likely that duplex release is caused by surface-bound miR-34a.

Finally, the observation of distinct duplex release for certain targets, but not for others (e.g. MTA2, which bound tightly to miR-34a-AGO2 but did not exhibit duplex release; see Figure 2), argues against the possibility that the phenomenon was solely due to non-specifically bound RNA releasing from AGO2.

In response to the reviewers statement "Since properly loaded miR-34a is never released from AGO2, it is impossible for the miR-34a loaded into AGO2 to form the binary complex (mRNA:miR-34a)" we would like to refer to the three papers, De et al. (2013) Jo MH et al. (2015), and Park JH et al. (2017), which have previously reported duplex release and collectively provide considerable evidence that miRNA can be unloaded from AGO in order to promote turnover and recycling of AGO. It is known that AGO recycling must occur, therefore there must be some mechanisms to enable release of miRNA from AGO2 to enable this. It is possible that AGO recycling proceeds via miRNA degradation (TDMD) in the cell, but in the absence of enzymes responsible for oligouridylation and degradation, the miRNA duplex may be released. As TDMD-competent mRNA targets have been observed to release the miRNA 3' tail from AGO2 (Sheu-Gruttadauria et al., 2019; Willkomm et al., 2022), there is a possible mechanistic similarity between the two processes, however, we do not have sufficient data to make any statement on this.