linked data backbones for - information, - data models and - mappings

that connect - people, - things, and - insights.

Code and APIs optional.

These curly quotes killed my code. I just copy-pasted the caption with quotes and got stuck in troubleshooting.

[!NOTE] 在小规模（一台机器）使用张量并行时，有什么好用的库？

flashcard

tensor_parallel

is the code for this also in the linked github repo? I couldn't find it, but I think it's an interesting application. It would be nice if something similar could be implemented for sourmash taxonomy results

I noticed that this repo doesn't have any unit tests and that the python script only contains 58 lines of code. Would it be possible to include this approach directly in sourmash?

mon code fonctionne mais n'est exactement écrit comme dans la solution GitHub, c grave docteur ? Après une 3è vérifs, j'ajoutais une ligne qui ne servait à rien. Voilà le code match.

New language: specifies that the grievance. The grievance "must allege that the decision was in violation of a right or privilege based on federal or state law, UNC Policy or Regulation, university policies or regulations, and that the faculty member was negatively affected by such a decision."

New language excludes grievances related to "disputes between faculty colleagues; between faculty and staff; between faculty and students"; and regarding decisions that don't "directly affect" the faculty member's employment, teaching loads/assignments, or resource decisions other than compensation."

What's the deal here? (Ask if this means that "personal malice" is now a permissible grounds for denial of PTR.) What's the effect, and how is it operationalized?

WHOSE "malice" toward WHOM? Does this refer to the malice of the candidate or the malice of others toward the candidate?

Q: why are they using the source code and not the text output?

Il me semble que c'est "voyage" dans le code téléchargé

It's a user-friendly and cost-effective solution, not only from a development perspective but also makes it easier to deploy changes like adding site languages, access permissions etc.

Dommage que le lien n'ouvre pas une nouvelle fenêtre ce qui éviterai de recommencer le quiz. Bon ça vas c'est deux questions à reprendre c'est pas très grave. Et ça a le mérite de voir qu'une amélioration est possible. En conclusion sur un plan pédagogique, c'est peut être une bonne chose.

[Disagree] I disagree with this provision since this is unfair for the consumers. This provision raises privacy concerns for the consumers, as people who use Shopee to exchange ideas are not protected. This is because this provision gives Shopee an excessive amount of rights towards a person's idea are not even employed by them. This raises a flag regarding privacy concerns for the users.

[surprising/unexpected]

It was very surprising to find out that material or ideas posted in Shopee can be used and managed by them even without the knowledge of the owner that it is being used. Since we blindly click "accept" to these terms and conditions, this means that we are also technically giving Shopee the right to use our data and information publicly which could be very terrifying. Most especially, this would also probably be surprising for the sellers who use this platform as their content can be publicly used by Shopee without due credit or permission.

Reviewer #2 (Public Review):

This manuscript introduces an integrative framework for modelling and analysis in neuroscience called BrainPy. It describes the many tools and utilities for building a wide range of models with an accessible and extensible unified interface written in Python. Several illustrative examples are provided for common use cases, including how to extend the existing classes to incorporate new features, demonstrating its ease of use and adherence to Python's programming conventions for integrative modelling across multiple scales and paradigms. The provided benchmarks also demonstrate that despite the convenience of presenting a high-level interpreted language to the user, it provides orders of magnitude of computational speed-up relative to three popular alternative frameworks on the chosen simulations through the extensive use of several Just In Time compilers. Computational benchmarks are also provided to illustrate the speed-up gained from running the models on massively parallel processing hardware, including GPUs, suggesting leading computational performance across a wide range of use cases.

While the results presented are impressive, publishing further details of the benchmarks in an appendix would be helpful for evaluating the claims and the overall conclusion would be more convincing if the performance benefits were demonstrated on a wider selection of test cases. Unsatisfyingly, the authors gave up on making a direct comparison to Brian running on GPUs with GeNN which would have been a fairer comparison than CPU-based simulations. The code for the chosen benchmarks is also likely to be highly optimised by the authors for running on BrainPy but less so for the other platforms - a fairer test would be to invite the authors of the other simulators to optimise the same models and re-evaluate the benchmarks. Furthermore, the manuscript reads like an advertisement for the platform with very little discussion of its limitations, weaknesses, or directions for further improvement. A more frank and balanced perspective would strengthen the manuscript and give the reader greater confidence in the platform.

Since simulators wax and wane in popularity, it would be reassuring to see a roadmap for development with a proposed release cadence and a sustainable governance policy for the project. This would serve to both clearly indicate the areas of active development where community contributions would be most valuable and also to reassure potential users that the project is unlikely to be abandoned in the near future, ensuring that their time investment in learning to use the framework will not be wasted. Similarly, a complex set of dependencies, which need to be modified for BrainPy, will likely make the project hard to maintain and so a similar plan to those given for the CI pipeline and documentation generation for automation of these modifications would be a good addition. It is also important to periodically reflect on whether it still makes sense to combine all the disparate tools into one framework as the codebase grows and starts to strain under modifications required to maintain its unification.

Finally, a live demonstration would be a very useful addition to the project. For example, a Jupyter notebook hosted on mybinder.org or similar, and a fully configured Docker image, would each enable potential users to quickly experiment with BrainPy without having to install a stack of dependencies and troubleshoot version conflicts with their pre-existing setup. This would greatly lower the barrier to adoption and help to convince a larger base of modellers of the potential merits of BrainPy, which could be major, both in terms of the computational speed-up and ease of development for a wide range of modelling paradigms.

Reviewer #3 (Public Review):

The paper presents the novel neuro-simulator BrainPy, which introduces several new concepts compared to existing simulators such as NEST, Brian, or GeNN: 1) a modular and Pythonic interface, which avoids having to use a fixed set of neural/synaptic models or using a textual equation-oriented interface; 2) a common platform for simulation, training, and analysis; 3) the use of just-in-time compilation using JAX/XLA, allowing to transparently access CPU, GPU, and TPU platforms. While none of these features is new per se (apart from TPU support, as far as I know), their combination provides an interesting new direction for the design of neuro-simulators.

Overall, BrainPy is a nice and valuable addition to the already overwhelming list of neuro-simulators, which all have their own advantages and drawbacks and are diversely maintained. The main strengths of BrainPy are 1) its multi-scale modular interface and 2) the possibility for the user to transparently use various hardware platforms for the simulation. The paper succeeds in explaining those two aspects in a convincing manner. The paper is also very didactic in explaining the different strengths and weaknesses of the current simulators, as well as the benefits of JIT compilation.

One potential issue is that the scope of the neuro-simulator is not very clearly explained and the target audience is not well defined: is BrainPy primarily intended for computational neuroscientists or for neuro-AI practitioners? The simulator offers very detailed neural models (HH, fractional order models), classical point-models (LIF, AdEx), rate-coded models (reservoirs), but also deep learning layers (Conv, MaxPool, BatchNorm, LSTM). Is there an advantage to using BrainPy rather than PyTorch for purely deep networks? Is it possible to build hybrid models combining rate-coded reservoirs or convnets with a network of HH neurons? Without such a hybrid approach, it is unclear why the deep learning layers are needed. In terms of plasticity, only external training procedures are implemented (backpropagation, FORCE, surrogate gradients). No local plasticity mechanism (Hebbian learning for rate-coded networks, STDP and its variants for spiking networks) seems to be implemented, apart from STP. Is it a planned feature? Appendix 8 refers to `bp.synplast.STDP()`, but it is not present in the current code (https://github.com/brainpy/BrainPy/tree/master/brainpy/_src/dyn/synplast). Spiking networks without STDP are not going to be very useful to computational neuroscientists, so this suggests that the simulator targets primarily neuro-AI, i.e. AI researchers interested in using spiking models in a machine learning approach. However, it is unclear why they would be interested in HH or Morris-Lecar models rather than simpler LIF neurons.

A second weakness of the paper concerns the demos and benchmarks used to demonstrate the versatility and performance of BrainPy, which are not sufficiently described. In Fig. 4, it is for example not explained how the reservoirs are trained (only the readout weights, or also the recurrent ones? Using BPTT only makes sense when the recurrent weights are also trained.), nor how many neurons they have, what the final performance is, etc. The comparison with NEURON, NEST, and Brian2 is hard to trust without detailed explanations. Why are different numbers of neurons used for COBA and COBAHH? How long is the simulation in each setting? Which time is measured: the total time including compilation and network creation, or just the simulation time? Are the same numerical methods used for all simulators? It would also be interesting to discuss why the only result involving TPUs (Fig 8c) shows that it is worse than the V100 GPU. What could be the reason? Are there biologically-realistic networks that would benefit from a TPU? As the support for TPUs is a major selling point of BrainPy, it would be important to investigate its usage further.

Elasticsearch is built on top of the Java Virtual Machine (JVM). The most critical part of memory allocation for Elasticsearch is the JVM heap memory. The heap memory is where Elasticsearch stores data structures, indexes, and caches.

The JVM is the runtime environment for Elasticsearch, and it's responsible for executing Elasticsearch's code and managing its memory. Allocating memory to the JVM is crucial because it affects Elasticsearch's performance and stability.

So here's a funny story. My first math exam was today. I'm an online student, so we have the honorlock extension. I took the exam in a study room at the library. I have discovered that I talk to myself a lot while working out math problems. One of the things that flags you for possible cheating is talking. I was so nervous afterwards. My grade wasn't immediately put in, so I think they may have had a proctor go back over my video...I hope I was entertaining! Also, I made a 90%!

This is difficult to answer. Hopefully my company is successful and stays in business for a long time. If that's the case I'm still not sure that the code itself will have a long lifespan. If it's amenable to change I would expect it to be changed over and over in the coming years.

At a startup it's difficult to balance short-term needs with long-term aspirations.

Il serait peut être mieux de dire aux personnes de déplacer la page pour faire l'exercice ? (mettre capture d'écran GitHub pour mieux comprendre ?) sauf si j'ai mal compris l'énoncé. Et cela permettrait d'enlever la phrase "sans déplacer les fichiers" dans les missions proposées juste en dessous. Indiquez une fois l'exercice fini de remettre le fichier "a propos" dans le même dossier pour la suite et revoir les liens de son code en conséquence

Il est possible de limiter le nombre de caractères par ligne afin que le code soit plus compact. Fixer une limite de ligne à 80 caractères est une bonne pratique. Voici la marche à suivre sur VsCode : WordWrapColumn

Author Response

The following is the authors’ response to the original reviews.

Editorial comments:

Comment 1 - Recommendations for the authors: please note that you control which revisions to undertake from the public reviews and recommendations for the authors.

We appreciate the feedback from the 3 Reviewers and Editor. We have enumerated each Reviewer comment and provide a detailed response. We endeavoured to include each suggestion into the revised manuscript. All changes in the manuscript are indicated in red font. In instances in which we respectfully disagree with the Reviewer, we have provided a fair rebuttal. We feel the comments from the Reviewers has significantly improved the clarity and quality of the manuscript.

Comment 2 - The revision process has demonstrated the value of your work, highlighting both its strengths and shortcomings. Importantly, it provides detailed and achievable suggestions for improving the current version of your contribution.

We thank the Reviewers and Editor for their time and expert input on our manuscript. We feel the suggestions from the Reviewers to address the shortcomings has resulted in a significantly improved manuscript.

Comment 3 - There is a general consensus among the reviewers on three key aspects. Firstly, the article would greatly benefit from a clearer layout of the experimental design and methodology, potentially including schematics to help readers comprehend the complexity and details of the study.

We appreciate the feedback from Reviewer 2 in particular. We have added a new schematic for Experiment 3 (see PUBLIC REVIEWS Reviewer #2 Comment 2). We have also revised the Results section by including subheadings and additional text to help explain the methods.

Comment 4 - Secondly, conducting a more comprehensive analysis of the available dataset, utilizing tools such as WGCNA to explore gene co-expression networks beyond specific genes, is recommended. Additionally, it is advised to exercise greater caution when discussing the limitations of the employed methods.

The suggestion for the WGCNA is excellent and very much appreciated. The revised manuscript includes WGCNA for both the MBH and pituitary gland. See Figures S3 & Table S6 and lines 166-182; 497-505).

Comment 5 - Thirdly, expanding the results section to create a more engaging narrative that guides readers through the numerous findings, and extending the discussion and conclusions to emphasize the ecological relevance of learning photoperiodic/seasonal responses and highlighting the presented model, would be valuable.

These were excellent suggestions that significantly improved the clarity and quality of the manuscript. The results section included several subheadings to help break up of the transitions across experiments. We have also significantly revised the introduction and discussion to include the ecological relevance and importance to consider sex as a factor in the interpretations.

Comment 6 - Finally, please pay close attention to the comment on the statistical analysis provided by Rev#2.

It is unclear why the Benjamini-Hochberg’s FDR analyses was suggested. The statistical test is a version of the Bonferroni test but is less stringent. We prefer to use conservative tests (i.e., Bonferroni correction). Moreover, the Bonferroni correction is the commonly used statistical tests in the field. To be consistent with the field and to be careful in our statistical approach, the revised manuscript did not change the post-hoc correction.

PUBLIC REVIEWS:

Reviewer #1:

Comment 1 - The authors investigated the molecular correlates in potential neural centers in the Japanese quail brain associated with photoperiod-induced life-history states. The authors simulated photoperiod to attain winter and summer-like physiology and samples of neural tissues at spring, and autumn life-history states, daily rhythms in transcripts in solstices and equinox, and lastly studies FSHb transcripts in the pituitary. The experiments are based on a series of changes in photoperiod and gave some interesting results. The experiment did not have a control for no change in photoperiod so it seems possible that endogenous rhythms could be another aspect of seasonal rhythms that lack in this study. The short-day group does not explain the endogenous seasonal response.

We thank the Reviewer for the fair assessment of the manuscript. The statement ‘the experiment did not have a control for no change in photoperiod’ is not clear to us. We think the Reviewer is arguing that prolonged constant photoperiod was not conducted to examine circannual timing in avian reproduction. The constant short photoperiod in Exp3 does provide the ability to examine the initial stages of interval timing. A different endogenous mechanism used by animals. The revised manuscript has clarified the different physiological responses.

Comment 2 - The manuscript would benefit from further clarity in synthesizing different sections. Additionally, there are some instances of unclear language and numerous typos throughout the manuscript. A thorough revision is recommended, including addressing sentence structure for improved clarity, reframing sentences where necessary, correcting typos, conducting a grammar check, and enhancing overall writing clarity.

We have incorporated the suggestions from both Reviewer 1 and Reviewer 2 that aimed to increase the clarity of the manuscript. We have provided detailed responses to each comment below and state how each comment was incorporated in the revised manuscript. We also had the manuscript reviewed by a colleague to help identify issues associated with sentence structure, grammar, and spelling.

Comment 3 - Data analysis needs more clarity particularly how transcriptome data explains different physiological measures across seasonal life-history states. It seems the discussion is built around a few genes that have been studied in other published literature on quail seasonal response. Extending results on the promotor of DEGs and building discussion is an extrapolating discussion on limited evidence and seems redundant.

A new statistical analysis (ie., WGCNA) was conducted to identify relations between photoperiod, physiology and transcripts. The focus on the few photoperiodic gene was kept in the discussion as the transcript expression is important to highlight the differences from the prevailing hypotheses and novel patterns of expression across seasonal timescales. See Figures S3 & Table S6 and lines 166-182; 497-505).

Comment 4 - Last, I wondered if it would be possible to add an ecological context for the frequent change in the photoperiod schedule and not take account of the endogenous annual response. Adding discussion on ecological relevance would make more sense.

This is an excellent suggestion. The introduction and discussion were substantially revised to include the ecological relevance.

Reviewer #2:

Comment 1 - This study is carefully designed and well executed, including a comprehensive suite of endpoint measures and large sample sizes that give confidence in the results. I have a few general comments and suggestions that the authors might find helpful.

We appreciate the Reviewers support for our manuscript. We have endeavoured to incorporate all suggestions in the revised manuscript.

Comment 2 - I found it difficult to fully grasp the experimental design, including the length of light treatment in the three different experiments (which appears to extend from 2 weeks up to 8 weeks). A graphical description of the experimental design along a timeline would be very helpful to the reader. I suggest adding the respective sample sizes to such a graphic, because this information is currently also difficult to keep track of.

We have created a new figure panel to address the Reviewer’s concern. See figure S4 panel ‘a’. The new schematic representation was designed to illustrate the similarity in experimental design used in Experiment 1 and Experiment 2. But clearly illustrates the extended short photoperiod manipulation (4 weeks and not 8 weeks). We added the sample sizes to initial drafts but felt the added text hindered the clarity of the schematic representation (particularly for Fig1a). The sample sizes for each experiment and treatment are provided in the raw data provided in the supplementary Table 1. For this reason, we have opted to not add the sample size to each diagram. We hope that the Reviewer will understand our perspective.

Comment 3 - The authors use a lot of terminology that is second nature to a chronobiologist but may be difficult for the general reader to keep track of. For example, what is the difference between "photoinducibility" and "photosensitivity"? Similarly, "vernal" and "autumnal" should be briefly explained at the outset, or maybe simply say "spring equinox" and "fall equinox."

This is a very helpful suggestion, and we thank the Reviewer. Two changes were made to the manuscript to address this comment. First, we revised the second introductory paragraph to describe the photoperiodic response and the terms used. Second, we have removed all reference to ‘vernal’ and replaced with ‘spring’. We opted to keep ‘autumn’ as the change to ‘fall’ did not provide the clarity of seasonal state in some statements (as fall is also used as a downward direction).

Comment 4 What was the rationale for using only male birds in this study? The authors may want to include a brief discussion on whether the expected results for females might be similar to or different from what they found in males, and why.

We agree with the Reviewer’s position that studies should include, or least describe, male and female biology. We have revised the text to address this comment. In the methods, we provide 2 sentences that state the photoperiodic response is the same for both male and females, and why males were selected. See lines (352-355). Then, in the discussion, we describe why females will be important to study how other supplementary environmental cues impact seasonal timing of reproduction. See lines (312-330; and 334-339).

Comment 5 - The authors used the Bonferroni correction method to account for multiple hypothesis testing of measures of testes mass, body mass, fat score, vimentin immunoreactivity and qPCR analyses in Study 1. I don't think Bonferroni is ever appropriate for biological data: these methods assume that all variables are independent of each other, an assumption that is almost never warranted in biology. In fact, the data show clear relationships between these endpoint measures. Alternatively, one might use Benjamini-Hochberg's FDR correction or various methods for calculating the corrected alpha level.

This concern is not clear to us. The Benjamini-Hochberg’s FDR is a slight modification of the Bonferroni correction. Moreover, the FDR is a less-stringent statistical test compared to the Bonferroni correction. We prefer to keep the Bonferroni approach to correct for multiple tests for two reasons. First, this test is commonly used in the field of chronobiology, and second, the Bonferroni correction is more conservative. We hope the Reviewer will appreciate our perspective to be consistent with the research field and higher stringency in our statistical approach.

Comment 6 - The graphical interpretations of the results shown in Figure 1n and Figure 3e, along with the hypothesized working model shown in Figure S5, might best be combined into a single figure that becomes part of the Discussion. As is, I do not think these interpretative graphics (which are well done and super helpful!) are appropriate for the Results section.

We appreciate the Reviewer’s suggestion. During the revision we developed a single figure to show the graphical representation for the respective experiments. Unfortunately, we found the single source to be very difficult to provide a clear description and overview of the findings. We feel that the interpretations, (admittedly unusual for Results section) are best placed in the respective figures that correspond to the different experiments.

Reviewer #3:

Comment 1a - It is well known that as seasonal day length increases, molecular cascades in the brain are triggered to ready an individual for reproduction. Some of these changes, however, can begin to occur before the day length threshold is reached, suggesting that short days similarly have the capacity to alter aspects of phenotype. This study seeks to understand the mechanisms by which short days can accomplish this task, which is an interesting and important question in the field of organismal biology and endocrinology.

We thank the Reviewer for their positive feedback.

Comment 1b - The set of studies that this manuscript presents is comprehensive and well-controlled. Many of the effects are also strong and thus offer tantalizing hints about the endo-molecular basis by which short days might stimulate major changes in body condition. Another strength is that the authors put together a compelling model for how different facets of an animal's reproductive state come "on line" as day length increases and spring approaches. In this way, I think the authors broadly fulfill their aims.

We thank the Reviewer for the positive support of our research and manuscript.

Comment 1c - I do, however, also think that there are a few weaknesses that the authors should consider, or that readers should consider when evaluating this manuscript. First, some of the molecular genetic analyses should be interpreted with greater caution. By bioinformatically showing that certain DNA motifs exist within a gene promoter (e.g., FSHbeta), one is not generating robust evidence that corresponding transcription factors actually regulate the expression of the gene in question. In fact, some may argue that this line of evidence only offers weak support for such a conclusion. I appreciate that actually running the laboratory experiments necessary to generate strong support for these types of conclusions is not trivial, and doing so may even be impossible. I would therefore suggest a clear admission of these limitations in the paper.

We agree with the Reviewer’s position. The transcription binding protein analyses was used as a means to identify potential factors involved in the regulation of transcript expression. We have written a new paragraph to address this comment. In the discussion, we that highlight the links between the well characterised circadian regulation of photoperiodic transcripts (e.g, D- & E-box elements and the photoperiodic control of TSHβ. We also indicate that our bioinformatic approach identified potentially new transcription binding motifs, and provide a clear admission and state that functional analyses are required to determine necessity of these pathways (e.g., MEF2). See lines 293-295.

Comment 2 - Second, I have another issue with the interpretation of data presented in Figure 3. The data show that FSHbeta increases in expression in the 8Lext group, suggesting that endogenous drivers likely act to increase the expression of this gene despite no change in day length. However, more robust effects are reported for FSHbeta expression in the 10v and 12v groups, even compared to the 8Lext group. Doesn't this suggest that both endogenous mechanisms and changes in day length work together to ramp up FSHbeta? The rest of the paper seemed to emphasize endogenous mechanisms and gloss over the fact that such mechanisms likely work additively with other factors. I felt like there was more nuance to these findings than the authors were getting into.

We agree with the Reviewer and a similar concern was raised by Reviewer 1. Our aim was to highlight that FSH expression increased in constant short photoperiod. We have revised the manuscript to address the concern raised by the Reviewer. We have added 2 sentences in the results to highlight the additive role of endogenous timing and photoperiodic effects on FSH expression (see lines 223-226). We have kept the text that describes endogenous increases in expression (e.g., FSH/GnRH) in response to short photoperiod in the manuscript as this observation is not influenced by long photoperiod.

Comment 3 - Third, studies 1 - 3 are well controlled; however, I'm left wondering how much of an effect the transitions in day length might have on the underlying molecular processes that mediate changes in body condition. While the changes in day length are themselves ecologically relevant, the transitions between day length states are not. How do we know, for example, that more gradual changes in day length that occur over long timespans do not produce different effects at the levels of the brain and body? This seemed especially relevant for study 3, where animals experience a rather sudden change in day length. I recognize that these experimental methods are well described in the literature, and they have been used by endocrinologists for a long time; nonetheless, I think questions remain.

There are two points raised in this comment. First, the effect of transition in day length on body condition. We are investigating the impact of photoperiodic transitions on body condition. The ongoing project has examined the changes in tissue lipid content and conducted transcriptomic analyses of multiple peripheral tissues involved in energy balance. Although we made an initial attempt to combine all the findings into a single manuscript, the large datasets resulted in an overwhelming manuscript that lacked clarity. Instead, we have opted for two manuscripts that focus on the respective physiological systems. Those data should be published shortly. We did expand the discussion by developing a single paragraph that focused on the pattern of POMC expression and changes in quail body mass and adipose tissue. See lines 300-311.

Second, the Reviewer raised the issue of more gradual changes in day length over longer timespans. The day length and duration of exposure selected was to replicate previously used photoperiod manipulations to ensure reproducibility in research programmes, and to reduce the impact of photoperiod history (see lines 367-369). The present manuscript is the first study in birds to examine multiple intervening (ie within the extreme long- and short-photoperiods) day length conditions and we feel this is a major and novel contribution to the field. We agree that other time points (e.g., 13L:11D), or quicker/longer timespans could provide additional insight into the molecular mechanisms that govern seasonal transitions in reproduction/energy balance. The question raised by the Reviewer requires the types of studies that use natural conditions from wild-caught animals (or semi-natural laboratory settings) and beyond the focus of the current manuscript.

Recommendations For The Authors:

Reviewer #1

Comment 1 - Abstract: Overall abstract needs more clarity in rationale, hypothesis, and result outcomes. How this study advances our knowledge in seasonal/ photoperiodic regulation of reproduction in birds. Particularly what knowledge gap FSHb results fill in.

We have substantially revised the abstract considering the Reviewer’s suggestions. The abstract has clarified the rationale, hypothesis and results outcomes. We have also added new introductory and concluding statements that place the work into a wider ecological context (as suggested below).

Comment 2 - In general the introduction needs more clarity and doesn't seem to cover the ecological relevance of learning photoperiodic/seasonal response.

We agree with the Reviewer the introduction could be improved. We have substantially revised the introduction with an aim to increase the clarity. This involved an addition on the ecological context, clarification of the photoperiodic states in birds, and a description of the general and specific objectives. Note we did not include an introduction to ‘learning’ of the photoperiodic response, as the term implies a cognitive component is involved which is incorrect. See lines (61-67, 71-74, 80-86, and 100-105).

Comment 3 - Line 58: What does the author mean by "future seasonal environment" Is it to introduce change in climate or future seasonal events? This sentence needs rephrasing and more clarity.

In response to Comment 2, we have revised the introductory paragraph and the sentence was removed from the text.

Comment 4 - Line 63: I would recommend the use of circannual rhythms with caution for the kind of experiments authors have proposed. The approach used here is beyond the scope of addressing circannual endogenous rhythms, which can be tested only independent of photoperiod change.

We agree with the Reviewer’s concern. The use of circannual rhythms was limited to the first paragraph (lines 56-63) only to introduce the concept of endogenous rhythmicity. We were careful to not use the term ‘circannual’ for the rest of the manuscript, as the Reviewer has indicated, would be inappropriate. We have retained the use of ‘endogenous program’ to refer to the molecular and physiological changes that can occur independent of photoperiod change (ie Experiment 3). In this case, the use of endogenous is appropriate as this form of timing adheres to an interval timer. We also provided a definition for interval timer and ecological examples to illustrate the difference between circannual rhythms and annual interval timer (see lines 71-74). We also reviewed the entire manuscript to ensure the distinction for the endogenous program was clear.

Comment 5 - Another aspect authors missed is that Quail is not an absolute photorefractory (Robinson and Follett, 1982).

We agree with the Reviewer that quail are not absolute photorefractory (but instead relative photorefractory). As our photoperiod manipulations do not address criterion 1, or criterion 2 of the avian photoperiodic response (MacDougall-Shackelton et al., 2009; see https://doi.org/10.1093/icb/icp048), we feel that adding the type of photorefractory response would be a distraction and reduce the clarity of the concepts/experimental design described in the manuscript.

Comment 6 - Line 223-234: "Chicks were raised under constant light and constant heat lamp". Constant photoperiod experienced during development raises concern on how this pretreatment would shape the adult seasonal response, which could be different in the seasonal response of birds raised in natural photoperiod. If this is correct, the results shown are not tenable for birds inhabiting the natural environment.

The light schedule used in our experiment is the most appropriate for laboratory reared chicks. The light schedule, use of an incubator and hatchery is commonly used in research laboratories. The procedure serves to increase the hatch rate and welfare of chicks. Undoubtedly there will be some early developmental programming effects on quail development. However, the gonadal response across all 3 experiments was consistent with the vast scientific literature on the avian photoperiodic response in both laboratory and wild birds. As the robust gonadal response clearly replicated previous studies, we are confident the results are tenable for birds inhabiting natural environments.

Comment 7 - Numerous studies done in mammals suggest that photoperiod experienced in the early life stage affects the circadian and seasonal response in adults (Ciarleglio et al., 2011, Perinatal photoperiod imprints the circadian clock, Nat Neurosceince; Stetson M., et al., 1986, Maternal transfer of photoperiodic information influences the photoperiodic response of prepubertal Djungarian hamsters).

We agree with the Reviewer that developmental programming in mammals is important for the photoperiodic response. However, there are vast differences between the avian and mammalian photoperiodic response. Critically, in mammals, the maternal transfer of information to the offspring is achieved via the melatonin hormone. Conversely, in birds, melatonin is not necessary, nor sufficient for photoperiodic time measurement (Juss et al., 1993; see https://doi.org/10.1098/rspb.1993.0121). It is not scientifically tenable to relate the mammalian and avian photoperiodic responses in adulthood based on early developmental programs. For this reason, we did not introduce or discuss developmental programming in our manuscript.

Comment 8 - Please give details on the month in which these birds were exposed to different short and long photoperiods. It is not clear in the method section. The birds experience long to short day transition and then back to long day in 16 weeks (~ 4 months). The annual cycle is ~12 months long in nature. Again, what is the ecological relevance of such an experimental paradigm. This could give some idea on photoperiodic response, but not on how the endogenous annual cycle would respond.

Birds were delivered in September 2019 and 2020. (We have added these details to the manuscript (see lines 351-352). We agree with the Reviewer that the ecological relevance of the experimental design is limited. Our focus was to use laboratory conditions and well characterised photoperiodic manipulations to examine the role of the environmental, initial predictive cue to time seasonal transitions in reproduction. The 2-week duration for each photoperiod state in Experiment 1 provides the ability to eliminate the impact of photoperiodic history (see lines 367-369; Stevenson et al., 2012a) and reduce the time necessary for the research project. As described above in Comment #4 – we did not examine the endogenous annual cycle – but instead focused on an endogenous interval timer. Experiment 3 was designed to best examine an endogenous interval timer.

Comment 9 - Line 251: "A jugular blood sample" Please rephrase this sentence and add 50 ul heparinized tubes

We thank the Reviewer for identifying this oversight. The text was changed accordingly.

Comment 10 - Line 259: The scale.....fat pads" - The sentence doesn't read correctly.

The sentence was revised accordingly.

Comment 11 - Line 274: Male.....six weeks. It is not clear from this sentence; what photoperiod birds were exposed to before transferring to 2 long days. Is it 16 or 14 LD.

The birds were held in 16L. The text has been revised accordingly.

Comment 12 - Line 276: It is not clear what is Home Office approved schedule 1. This may be a commonly used term for animal sacrifice protocol in UK and Europe. But it is not familiar jargon for the rest of the globe.

We apologise for the jargon. The text was revised to include the exact methods (decapitation followed by exsanguination).

Comment 13 - Line 277-284: Birds under SD for 4 weeks (8 Lext) is a bit confusing and particularly in the context of studying endogenous rhythm. Needs more clarity.

The text was revised to improve the clarity. The manuscript now states: ‘A subset of birds (n=6) was maintained in short day photoperiods for four more weeks (8Lext). This group of birds provided the ability to examine whether an endogenous increase in FSHβ expression would occur in constant short day photoperiod condition.’

Comment 14 - Line 322-323: Give RIN number (RNA integrity number) here which is a very common parameter to determine RNA degradation in RNAseq experiments. I guess, the MiniON is a portable sequencer and sequences one sample at a time. If this is true authors should consider any batch effect in sequencing and use it as a covariate in the model.

The RIN values from our extraction protocol reliably produce RIN values >9.0. The text now states: Isolated RNA reliably has RIN values >9.0 for both the mediobasal hypothalamus and pituitary gland. Our RIN values are well above the recommended 7.0 limit. The Reviewer is correct that MinION is portable, however, more than one sample can be run at a time. We stated in the text (lines 454-460) that birds were counterbalanced across Flow cells so that each sequencing run had 9 samples, one from each treatment group. Our counterbalancing approach and quality control steps prevented batch effects.

Comment 15 - Line 397-398: Adding quail or chicken-specific vimentin peptide pre-incubation with primary Ab will serve more confirming control. Omitting primary Ab doesn't address cross-reactive/ nonspecific binding issues.

We agree that a positive control (ie primary Ab) is the gold standard to support specificity of the antibody. Unfortunately, we have not found a supplier of the epitope for quail/chicken vimentin. We have conducted another in silico analysis an established that the sequences for the vimentin antibody is specific for vimentin. The next closest sequence alignment is only 68% for a protein that is not expressed in the brain. The immunoreactive pattern observed in our histology reproduces work from mammalian models in which the epitope is available. Therefore, we are confident that our immunoreactive signal for vimentin is specific. We have added the in silico analysis in the manuscript on lines 535-538.

Comment 16 - Line 430: Was the GLM model used for testing all variables? Running a statistical model to explain Differentially expressed genes, photoperiod, and physiological variables together will give a more conclusive outcome to explain the photoperiod effect and seasonal state.

A similar comment was raised by Reviewer 2. We have conducted a WGCNA analyses to examine the relationship between photoperiod, physiological variables and DEG. See Figures S3 & Table S6 and lines 166-182; 497-505).

Comment 17 - It is a bit unclear why the author used cherry-picking approach by talking about only a few genes that have been studied as key regulators of photoperiodic response in quail. What was the purpose of transcriptome? A better approach would have been to use a model to reduce the data (PCA) and explain the physiological response by regression against different PCs.

We agree with the Reviewer that other statistical approaches could be conducted, and other genes could be discussed. However, we focussed on the key regulators of the photoperiodic response in quail as these are the well characterised genes. It is important that our discussion focused on these transcripts as most do not conform to the predicted patterns of expression. We feel it is best that we keep the focus on these genes.

Comment 18 - TSHb result is inconsistent with past studies, where TSHb is the first responder gene on photoinduction. The author did not pay attention to explaining it further in the discussion.

We respectfully disagree with the Reviewer. Our results are consistent with past studies and show that TSHβ expression is a molecular marker of long day photoperiod. Our study does not examine photoinduction; which does not provide the ability to compare between our study and previous work (eg., Nakao et al., 2008; see doi: 10.1038/nature06738). We have revised the text in consideration of the concern raised by the Reviewer. The text now states ‘Previous reports established that TSHβ expression is significantly increased during the period of photoinducibility in quail (Nakao et al., 2008). Although the present study did not directly examine photoinduction, TSHβ expression was consistently elevated in long day photoperiod (i.e., 16L).’. (see lines 262-265).

Comment 19 - PRL result seems interesting and there could be more discussion in relation to the rise in PRL transcripts levels termination of breeding. Elaborating on PRL expression and breeding termination can add more information to the discussion.

This comment is not clear to us, and we would incorporate a clarified comment in a revised manuscript. The increased expression of prolactin does not occur during the termination of breeding. The increase in prolactin occurs during the vernal increase in photoperiod (ie 14L) but does not have a clear link with gonadal growth.

Comment 20 - Line 217-219: Based......respectively. Sounds like a big claim with less evidence.

We have removed the sentence from the discussion.

Comment 21 - Line 220-223: The .....Bird. The sentence is not clear about how this study would add to ecological studies. Need more clarity on the importance of such data.

The sentence was removed from the text.

Comment 22 - I think that it would be helpful to add a couple of caveats to provide more ecological context. First, the model is only based on males, and responses in females could be different.

We agree with the Reviewer there are undoubtedly sex differences in timing seasonal biology. However, the photoperiodic response (growth and regression) is similar in both males and females. Sex differences exist in response to supplementary environmental cues (e.g., temperature). Males were used in these studies as the gonadal response to changes in photoperiod manipulations are much larger compared to ovarian changes in females. The focus on males allows for fewer animals to be used in the experiments and greater statistical power. To address the Reviewers concern, we have added a paragraph in the discussion that describes the similarity in photoperiodic responses in males and females, and the importance of supplementary cues for full reproductive development in female birds. We also provide a couple sentences in the methods that describe the justification for only males in the present study. See lines (Methods 352-355; Discussion 312-330; and 334-339).

Comment 23 - Last, I wondered if it would be possible to add an ecological context for the frequent change in the photoperiod schedule and not take account of the endogenous annual response. Would the procedure simulate a similar kind of underlined molecular response for a bird under natural conditions responding to changing daylight cycles on an annual time frame?

The discussion was considerably revised to address the ecological relevance of the study, and findings. We have added a sentence at the beginning of the discussion to highlight that the laboratory-based approach and photoperiodic manipulations reliable replicate previous findings using semi-natural conditions (Robinson and Follett, 1982) (See lines 248-250). We have already reduced the focus on the endogenous annual response.

Reviewer #2:

Comment 1 - The writing is very terse and could benefit from a more narrating style, which would make it a lot easier for the reader to get through some of the very data-heavy text. Breaking up the Results with subheadings would also be helpful.

We appreciate the suggestion to add subheadings to the Results. We added 3 descriptive headings for each other studies conducted in the manuscript. We feel the added revision (e.g., ecological) has improved the narrative and made the manuscript accessible to the wider readership.

Comment 2 - The transcriptome analyses could be developed a bit more. First, using the limma package would allow the authors to apply a more complete model to the DEG analyses, which would likely be superior to EdgeR. Second, the authors may want to consider WGCNA or a similar approach to discover gene co-expression modules, and then examine whether any of the resulting module eigengenes co-vary with any morphological or physiological measures and/or vary rhythmically.

This is an excellent suggestion, and the new analyses was incorporated into the revised manuscript. Using the Langfelder and Horvath 2008 WCGNA package we conducted module-trait analyses to examine co-variation in our findings. These data are presented in Figure S# and lines 476-484. We agree that other DEG analyses would be useful; our main objectives was to use BioDare2.0 to identify rhythmic transcription in the seasonal transcriptomes. EdgR provides an excellent approach to identify transcripts and commonly used.

Comment 3 - In the Data and code availability statement (lines 226ff) the authors state that "all raw data are available in Extended data Table 1." However, they should be submitted to the GEO database or a similar public repository along with all relevant metadata. Also, and maybe I overlooked this, I did not see anywhere that the "R code used in Study 1 is freely available" (I was not sure what "the methods reference list" was supposed to refer to). Instead of stating that "the full R code used is available upon request" I suggest making all scripts available via GitHub or Dataverse, along with all non-omics data. The advantage of the latter platform is that a citable DOI is assigned to each upload.

The data are now available in the GEO database and can be accessed see GSE241775. We have added this information to the text. The R code is now provided as a Table S11 so that the reader can directly access the script.

Comment 4 - Line 191: Delete the extra "that"

We thank the Reviewer for identifying the oversight. We have revised the text accordingly.

Comment 5 - Line 24f: What does "pseudo-randomly" mean? Maybe "haphazardly" would be more appropriate here?

The term pseudo-randomly is used to describe the organized manner in which subjects are assigned to each treatment group. The aim is to ensure that a particular physiological variable, such as body mass, is evenly distributed across treatment groups. (Note although the term derived from the field of psychology). The aim is to reduce bias in the experiment due to an initial bias established when assigning treatment group. We are reluctant to replace pseudorandomly with haphazardly as the latter does not imply a logical organization. We have added text to help clarify the reason. The text now state: At the end of each photoperiodic treatment a subset of quail (n=12) body mass was used as a measure to pseudo randomly select birds for tissue collection and served to reduce the potential for unintentional bias.

Comment 6 - Figure 1e,j: The text indicates that 398 and 130 genes were "rhythmically expressed" in the MBH and pituitary, respectively, but considerably fewer genes are shown in the heatmaps in Figure 1e,j. How were these genes selected, and what was the rationale for doing so? Also, some autumnal and vernal expression patterns show some strong similarities (e.g., 16a and 16v in the MBH), which could be discussed. Consider showing the two heatmaps with the columns also hierarchically clustered in a supplementary figure.

We agree with the Reviewer that the full heatmap for the transcripts should be provided. The heat maps in Figure 1 are based on the transcripts with the most significant change; and were selected to provide a graphical representation that would be easily digested by the wide readership. We have created a new figure (ie. Fig. S1) that provides all the transcripts in heat maps for both the MBH and pituitary gland.

Reviewer #3:

Comment 1 I do not have too much to add to this section of my review. Broadly speaking, I would suggest that the authors address some of the concerns I highlight above, and integrate their thoughts into the paper more than they currently do. I think this is particularly important with respect to the limitations of many of the bioinformatic analyses.

We thank the reviewer for their input and time assessing the manuscript. We have revised the manuscript in many sections incorporating the suggestions by Reviewer 3 above, and Reviewers 1 and 2.

Comment 2 Some of the methods are also a little scant. For example, the qPCR analyses are not described in sufficient detail to replicate the study. What are the efficiencies? Were samples run in duplicate? What was the housekeeping control gene used? Was there only one, or were multiple housekeeping genes used?

We apologise for the oversight, the absence of information was a mistake that missed our previous early revisions. The revised manuscript includes all the requested information. Line 333 states that all samples were run in duplicate. The efficiency for each transcript was within the MIQE guidelines (indicated on line 342) and were within the 0.7 to 1.0 range. Actin and glyceraldehyde 3-phosphate dehydrogenase were used as the reference transcripts. The most stable reference transcript was used to calculate fold change in target gene expression (lines 343-345).

Author Response

The following is the authors’ response to the original reviews.

eLife assessment

In this important paper, the authors report a link between brumation and tissue size in frogs, summarizing convincing evidence that extended brumation is associated with smaller brain size and increased investment in reproduction-related tissues. The research will be of broad interest to ecologists, evolutionary biologists, and those interested in global change biology. While the dataset involves significant field work and advanced statistical analyses, the manuscript would benefit from more explanation of the models, including why frogs are a good model in which to address these questions, and from general improvement in the structure and conciseness.

We highly appreciate your positive assessment and that you considered our paper important and convincing.

Reviewer #1 (Public Review):

The authors have conducted lots of field work, lab work and statistical analysis to explore the effect of brumation on individual tissue investments, the evolutionary links between the relative costly tissue sizes, and the complex non-dependent processes of brain and reproductive evolution in anuran. The topic fits well within the scope of the journal and the manuscript is generally written well. The different parameters used in the present study will attract a board readership across ecology, zoology, evolution biology, and global change biology.

Thank you for your positive and supporting feedback.

Reviewer #2 (Public Review):

The authors set out to show how hibernation is linked to brain size in frogs. If there were broader aims it is hard to decipher them. The authors present an extremely impressive dataset and a thorough set of cutting-edge analyses. However not all details are well explained. The main result about hibernation and brain size is fairly convincing, but it is hard to think of broader implications for this study. Overall, the manuscript is very confusing and hard to follow.

Thank you for your compliments on our paper. As for your concerns, we have greatly revised our paper and, as we hope, improved its clarity. We have also added a few sentences to the conclusions to draw attention to potentially broader implications. Specifically, we describe how the focal traits of our study may all be affected by climate change. Differential constraints in necessary investments could be one of several reasons for the varying resilience to climate change between species in the same habitat.

Reviewer #1 (Recommendations For The Authors):

There are no issues on the availability of data and code.

Thank you.

Line 15: in the author contribution section, it seems that C.L.M. and J.P.Y are not in the author list.

These two authors are not part of this study. This was a mistake.

Line 24: I don't think it is vital or logical to address or compare too much on birds or mammals, which are not the focused taxa of the present study. Instead, it is better to clarify the reason why frogs and toads are ideal model taxon to this study.

The reason for comparisons with birds and mammals was that all hypotheses related to the various trade-offs tested here had been developed in these taxa. One of the points of our paper was that these needed validation beyond the two taxa, in addition to being tested against one another (each prediction had been developed in a specific group and typically in isolation of all other hypotheses).

Line 25-26: as the authors are shooting for eLife, as a general journal, it is not essential to provide the detailed methods in the abstract. But I think the authors need to strengthen the novelty of the work in the field here.

The strength of our study was that all traits were measured directly in our species, including estimates of hibernation duration. Prior studies used various proxies, categorial classification or datasets assembled from multiple sources. To us, this seemed like a sufficiently important advance in the field to mention it, but considering the reviewer’s comment, we have now removed it.

Line 28: "protracted brumation reduces brain size and instead promotes reproductive investments", as a correlative study, it is much more precise to change this sentence to a similar description as "protracted brumation is negatively correlated with brain size but is positively correlated with reproductive investments" here and related statements throughout the whole text.

We agree that, strictly speaking, a path analysis can only point toward possible causality and not provide hard evidence as experimental manipulation might. The wording may have been a bit too strong here in our attempt to minimize wordiness and because all our analyses combined very strongly pointed in this direction. However, we have now changed this as suggested even though it now reads almost as if we had done no more than conducting a simple correlation. We have further paid attention to the wording of our interpretations throughout the paper.

Line 32-33: it needs a bigger ending linking your main findings with the implication in understanding species response to the sustained environment change.

We have reworded the ending of the abstract to: “Our results provide novel insights into resource allocation strategies and possible constraints in trait diversification, which may have important implications for the adaptability of species under sustained environmental change.”

Line 63-68: this sentence is too long to understand and please simplify it.

We have split the sentence into two sentences.

Line 125-130: it is known that there are various frog reproductive modes (Crump et al. 2015) such as trade-offs between clutch size and egg size, different number of breeding during one year, etc. These different reproductive forms may also influence the brain size evolution with food availability and seasonal variations. Please clarify it.

Yes, anurans do have varying reproductive modes, but to us, there is no a priori reason to assume that such variation would have a direct effect on brain evolution. Rather, in our opinion, different reproductive modes would have indirect effects by affecting the environment in which reproduction occurs. For example, larvae developing under different environmental conditions (substrate, larval density, egg provisioning etc.) might affect developmental trajectories that could influence how resources are available and allocated to different organs, including the brain. Alternatively, reproductive modes could influence the choice of environment for reproduction, thereby possibly affecting mating strategies and ultimately trait investments associated with these strategies. Given we were asked to shorten our paper, we believe that ‘environmental effects’ remains broad enough to encompass such variation, thereby not necessitating disentangling the different, and likely primarily indirect, ways that reproductive modes could be linked to brain evolution. However, if the reviewer would find it important to go into such detail in the paper, we will be happy to do so.

Line 186-187: it is necessary to mention here that the authors also conducted sensitivity analyses to apply 2{degree sign}C or 4{degree sign}C below their experimentally derived as thresholds to test the robustness of the results to data uncertainty.

We have added “(details on methodology and various sensitivity analyses for validation in Material and Methods)” to indicate the different types of sensitivity analyses, which included more than simply 2 or 4°C difference.

Line 188: please change "In phylogenetic regressions" to "after controlling for phylogenetic autocorrelation/pseudo-replication" or similar sentence here.

Our focus here was the phylogenetically informed GLS model rather than phylogenetic control itself. In the latter case, it would still not be clear what type of model was conducted with such phylogenetic control. To avoid any shorthand, we have reworded for more precision: “We employed phylogenetic generalized least-squares (PGLS) models, …”

Line 177-287: please provide the exact variance explained by different predictor variables in brumation duration, individual tissue investments, and brain evolution. I also suggest that the authors need consider conducting multi-model inference-based model averaging analysis to test the relative importance of different variables. In addition, the present analyses did not include the interaction terms among variables, which may be more important than the effect of each individual factor.

There may be some misunderstanding as these models represent separate analyses for each predictor as indicated by the associated λ values (never more than one value per model). We conducted separate models to determine which variables might even play a role in explaining variation in the corresponding response variables. Based on relevant predictors, we then conducted path analyses rather than general multi-predictor analyses. The relative effect sizes are represented by the correlation coefficients (r values) in the tables.

Reviewer #2 (Recommendations For The Authors):

Why exactly are the pairwise comparisons positively correlated (fig. S5) and then negatively correlated (fig. 3). What is actually driving this difference? For the phylogenetic path analyses 26 candidate models are chosen without explanation. What theory or hypotheses are these based on?

We assume the reviewer is referring to the brain-body fat association. The two ‘pairwise’ analyses they mention were not the same. The correlation in Fig. S5 was a standard (albeit phylogenetically informed) partial correlation between the two focal tissues, controlling for SVL. By contrast, as described when introducing the analyses, negative associations were derived when additionally controlling for testes and hindlimb muscles, all of which deviated from isometry against body size. Here, the total mass of the four main tissues was divided by their proportional contribution to that mass in each species, then standardized for comparison across species. Since the total mass of these four tissues scaled directly with body size, larger-bodied species did not invest a proportion of their body to these tissues than smaller-bodied species, thus essentially rendering body size irrelevant for this analysis. However, the relative representation of the four traits changed between species such that more resources devoted to body fat was associated with a smaller brain, hence a negative relationship. Similarly, the multivariate analysis as well as the PCA also suggested similar trends when all four tissues were considered rather than purely pairwise comparisons.

Regarding the second comment: We indeed used 28 pre-defined predictions for our larger path analysis.

The authors haven't really provided much additional context either, and the discussion is almost entirely a rehash of the results section. I can't see the analysis code but this may be of use to people performing similar analyses.

It is true that the traits and core message of the Discussion relate directly to our results, but we believe that our Discussion provides the essential biological context to our findings and to how they are connected. We tried not to go on tangents or too much speculation as the many results provided enough material to discuss, with several different ways that we expanded the prior state-of-the-art in the field. However, we have now expanded the concluding paragraph to place our findings in the context of climate change, given that this could affect anurans and the different traits examined in many ways that are directly related to the current study. Yet, we decided to keep this short because such extrapolation of our findings

We indeed held off making the code available to the public in case dramatic changes to the paper were requested by the reviewers. However, it will be published.

Additional recommendations from the Reviewing Editor:

• One of the reviewers and I found the text a little difficult to follow. I suggest simplifying the paper by being more concise. For example, the introduction could be shortened into a 3-4 paragraphs of relevant text without overwhelming the reader. One of the reviewers wanted a better explanation of statistical models and I agree. The discussion could benefit from some structure - consider adding subheadings that would guide the reader as to the topic. Finally, the figures are difficult to see and should be made larger. For example, the graphs in Figure 1c could be on a panel below A and B so that readers can interpret the graph. In Figure 3 - the legend is far too small - please put above or below the graphs. In summary - I hope you consider a major re-write that would strengthen the accessibility of your paper to a broad audience.

We have substantially shortened the paper despite adding further details on models and a broader context to the Discussion. We also condensed the Introduction to about two thirds of the original word count. However, we did not think that shortening it even further or splitting it into 3-4 paragraphs would improve readability. We still considered it important to introduce with sufficient context all major hypotheses that were tested against one another, provide at least some information on what was or was not known about the evolution of the focal traits and their links to one another or the environmental variables. We also found it important to touch on the differences between our study organisms and those typically studied in the context of hibernation or brain evolution, as this could affect the predictions. Given the number of hypotheses and traits, cutting the number of paragraphs would have meant merging some of them into very long ones, which we did not consider helpful.

We further added short subheadings to the Discussion and adjusted the figures as requested.

Kshetra: Which sample code?

with apt

Dashboard What are my jobs? Show my jobs with budgets and actuals. What's the code for westheimer? How much budget is left on Glen Haven?

While this is super amazing and is so cool for people to overcome their disability, we should try to create jobs specifically for people with disabilities. Also when we are writing a code we should always take into account that there are people that have disabilities.

Ne serait il pas plus simple d'écrire directement: .btn { &:active + .progress { & .progress__bar { transform: scaleX(1); } } }

quelle est la différence avec le code du cours? merci

Author Response

We appreciate the editor's and reviewers' time to review our manuscript. We will work on the suggestions and have provided an initial assessment of what we can do for our revised submission.

Reviewer #1 (Public Review):

Summary:

This study aimed to investigate the effects of optically stimulating the A13 region in healthy mice and a unilateral 6-OHDA mouse model of Parkinson's disease (PD). The primary objectives were to assess changes in locomotion, motor behaviors, and the neural connectome. For this, the authors examined the dopaminergic loss induced by 6-OHDA lesioning. They found a significant loss of tyrosine hydroxylase (TH+) neurons in the substantia nigra pars compacta (SNc) while the dopaminergic cells in the A13 region were largely preserved. Then, they optically stimulated the A13 region using a viral vector to deliver the channelrhodopsine (CamKII promoter). In both sham and PD model mice, optogenetic stimulation of the A13 region induced pro-locomotor effects, including increased locomotion, more locomotion bouts, longer durations of locomotion, and higher movement speeds. Additionally, PD model mice exhibited increased ipsilesional turning during A13 region photoactivation. Lastly, the authors used whole-brain imaging to explore changes in the A13 region's connectome after 6-OHDA lesions. These alterations involved a complex rewiring of neural circuits, impacting both afferent and efferent projections. In summary, this study unveiled the pro-locomotor effects of A13 region photoactivation in both healthy and PD model mice. The study also indicates the preservation of A13 dopaminergic cells and the anatomical changes in neural circuitry following PD-like lesions that represent the anatomical substrate for a parallel motor pathway.

Strengths:

These findings hold significant relevance for the field of motor control, providing valuable insights into the organization of the motor system in mammals. Additionally, they offer potential avenues for addressing motor deficits in Parkinson's disease (PD). The study fills a crucial knowledge gap, underscoring its importance, and the results bolster its clinical relevance and overall strength.

The authors adeptly set the stage for their research by framing the central questions in the introduction, and they provide thoughtful interpretations of the data in the discussion section. The results section, while straightforward, effectively supports the study's primary conclusion the pro-locomotor effects of A13 region stimulation, both in normal motor control and in the 6-OHDA model of brain damage.

We thank the reviewer for their positive comments.

Weaknesses:

1) Anatomical investigation. I have a major concern regarding the anatomical investigation of plastic changes in the A13 connectome (Figures 4 and 5). While the methodology employed to assess the connectome is technically advanced and powerful, the results lack mechanistic insight at the cell or circuit level into the pro-locomotor effects of A13 region stimulation in both physiological and pathological conditions. This concern is exacerbated by a textual description of results that doesn't pinpoint precise brain areas or subareas but instead references large brain portions like the cortical plate, making it challenging to discern the implications for A13 stimulation. Lastly, the study is generally well-written with a smooth and straightforward style, but the connectome section presents challenges in readability and comprehension. The presentation of results, particularly the correlation matrices and correlation strength, doesn't facilitate biological understanding. It would be beneficial to explore specific pathways responsible for driving the locomotor effects of A13 stimulation, including examining the strength of connections to well-known locomotor-associated regions like the Pedunculopontine nucleus, Cuneiformis nucleus, LPGi, and others in the diencephalon, midbrain, pons, and medulla.

We considered two approaches initially. The first approach was to look at specific projections to the motor regions, focusing on the MLR. The second approach was to utilize a whole-brain analysis that is presented here. Given what we know about the zona incerta, especially its integrative role, we felt that a reasonable starting point was to examine the full connectome. The value of the whole-brain approach is that it provides a high-level overview of the afferents and efferents to the region. The changes in the brain that occur following Parkinson-like lesions, such as those in the nigrostriatal pathway, are known to be complex and can affect neighbouring regions such as the A13. Therefore, we wished to highlight the A13, which we considered a therapeutic target, and examine changes in connectivity that could occur following acute lesions affecting the SNc. We acknowledge that this study does not provide a causal link, but it presents the fundamental background information for subsequent hypothesis-driven, focused, region-specific analysis.

The terms provided were from the Allen Brain Atlas terminology and were presented as abbreviations. We have looked at other ways to present it, including a greater emphasis on raw numbers and highlighting motor-related subareas. We will rewrite the connectomics section to make it more accessible, reflecting the change in the figures.

Additionally, identifying the primary inputs to A13 associated with motor function would enhance the study's clarity and relevance.

This is a great point and could help simplify the whole-brain results. We can present the motor-related inputs and outputs as part of a new figure in the main paper and add accompanying text in the results section. This will help highlight possible therapeutic pathways. We can also enhance our discussion of these motor-related pathways. We will retain the entire dataset and present it in a supplementary table for those who are interested.

The study raises intriguing questions about compensatory mechanisms in Parkinson's disease and a new perspective on the preservation of dopaminergic cells in A13, despite the SNc degeneration, and the plastic changes to input/output matrices. To gain inspiration for a more straightforward reanalysis and discussion of the results, I recommend the authors refer to the paper titled "Specific populations of basal ganglia output neurons target distinct brain stem areas while collateralizing throughout the diencephalon from the David Kleinfeld laboratory." This could guide the authors in investigating motor pathways across different brain regions.

Thank you for the advice, and as pointed out, Kleinfeld’s group had a nice, focused presentation of their data. For the connectomic piece, we can certainly adopt their reporting style, which, as you point out, may highlight key motor-related regions. There are a few ideas here that we can explore further, as mentioned above.

2) Description of locomotor performance. Figure 3 provides valuable data on the locomotor effects of A13 region photoactivation in both control and 6-OHDA mice. However, a more detailed analysis of the changes in locomotion during stimulation would enhance our understanding of the pro-locomotor effects, especially in the context of 6-OHDA lesions. For example, it would be informative to explore whether the probability of locomotion changes during stimulation in the control and 6-OHDA groups. Investigating reaction time, speed, total distance, and even kinematic aspects during stimulation could reveal how A13 is influencing locomotion, particularly after 6-OHDA lesions. The laboratory of Whelan has a deep knowledge of locomotion and the neural circuits driving it so these features may be instructive to infer insights on the neural circuits driving movement. On the same line, examining features like the frequency or power of stimulation related to walking patterns may help elucidate whether A13 is engaging with the Mesencephalic Locomotor Region (MLR) to drive the pro-locomotor effects. These insights would provide a more comprehensive understanding of the mechanisms underlying A13-mediated locomotor changes in both healthy and pathological conditions.

Thank you for these suggestions. We will revise as suggested. We will provide additional and/or updated data in revised figures and text. We will also move Supplementary Figures S1 and S2, which present additional locomotor data, into the main text to partly address the reviewers' points.

Reviewer #2 (Public Review):

Summary:

The paper by Kim et al. investigates the potential of stimulating the dopaminergic A13 region to promote locomotor restoration in a Parkinson's mouse model. Using wild-type mice, 6-OHDA injection depletes dopaminergic neurons in the substantia nigra pars compacta, without impairing those of the A13 region and the ventral tegmentum area, as previously reported in humans. Moreover, photostimulation of presumably excitatory (CAMKIIa) neurons in the vicinity of the A13 region improves bradykinesia and akinetic symptoms after 6-OHDA injection. Whole-brain imaging with retrograde and anterograde tracers reveals that the A13 region undergoes substantial changes in the distribution of its afferents and projections after 6-OHDA injection. The study suggests that if the remodeling of the A13 region connectome does not promote recovery following chronic dopaminergic depletion, photostimulation of the A13 region restores locomotor functions.

Strengths:

Photostimulation of presumably excitatory (CAMKIIa) neurons in the vicinity of the A13 region promotes locomotion and locomotor recovery of wild-type mice 1 month after 6-OHDA injection in the medial forebrain bundle, thus identifying a new potential target for restoring motor functions in Parkinson's disease patients.

Weaknesses:

Electrical stimulation of the medial Zona Incerta, in which the A13 region is located, has been previously reported to promote locomotion (Grossman et al., 1958). Recent mouse studies have shown that if optogenetic or chemogenetic stimulation of GABAergic neurons of the Zona Incerta promotes and restores locomotor functions after 6-OHDA injection (Chen et al., 2023), stimulation of glutamatergic ZI neurons worsens motor symptoms after 6-OHDA (Lie et al., 2022).

Thank you - we will add this reference. It is useful as Grossman did stimulate the zona incerta in the cat and elicit locomotion, suggesting that stimulation of the area in normal mice has external validity. The area targeted by Chen et al. (2023) is in the lateral aspect of central/medial zona incerta, formed by dorsal and ventral zona incerta, which may account for the differing results. Our data were robust for stimulation of the medial aspect of the rostromedial zona incerta. The thigmotactic behaviour that we observed in our work that focused on CamKII neurons has not been observed with chemogenetic, optogenetic activation or with photoinhibition of GABAergic central/medial ZI (Chen et al. 2023).

Although CAMKIIa is a marker of presumably excitatory neurons and can be used as an alternative marker of dopaminergic neurons, behavioral results of this study raise questions about the neuronal population targeted in the vicinity of the A13 region. Moreover, if YFP and CHR2-YFP neurons express dopamine (TH) within the A13 region (Fig. 2), there is also a large population of transduced neurons within and outside of the A13 region that do not, thus suggesting the recruitment of other neuronal cell types that could be GABAergic or glutamatergic.

We found that CamKII transfection of the A13 region was extremely effective in promoting locomotor activity, which was critical for our work in exploring its possible therapeutic potential. We acknowledge that specific viral approaches that target the GABAergic, glutamatergic, and dopaminergic circuits would be very useful. The range of tools to target A13 dopaminergic circuits is more limited than the SNc, for example, because the A13 region lacks DAT, and TH-IRES-Cre approaches, while useful, are less specific than DAT-Cre mouse models. Intersectional approaches targeting multiple transmitters (glutamate & dopamine, for example) may be one solution as we do not expect that a single transmitter-specific pathway would work, as well as broad targeting of the A13 region. Recent work suggests that GABAergic neuron activation may have more general effects on behaviour rather than control of ongoing locomotor parameters. However, this is in contrast to recent work showing a positive valence effect of dopamine A13 activation on motivated food-seeking behavior, which differs from consummatory behavior observed with GABAergic modulation (Ye, Nunez, and Zhang 2023). Chemogenetic inactivation and ablation of dopaminergic A13 revealed that they contribute to grip strength and prehensile movements, uncoupling food-seeking grasping behavior from motivational factors (Garau et al. 2023). Overall, this suggests differing effects of GABA compared to DA and/or glutamatergic cell types, consistent with our effects of stimulating CamKII.

Regarding the analysis of interregional connectivity of the A13 region, there is a lack of specificity (the viral approach did not specifically target the A13 region), the number of mice is low for such correlation analyses (2 sham and 3 6-OHDA mice), and there are no statistics comparing 6-OHDA versus sham (Fig. 4) or contra- versus ipsilesional sides (Fig. 5). Moreover, the data are too processed, and the color matrices (Fig. 4) are too packed in the current format to enable proper visualization of the data. The A13 afferents/efferents analysis is based on normalized relative values; absolute values should also be presented to support the claim about their upregulation or downregulation.

Generally, papers using tissue-clearing imaging approaches have low sample sizes due to technical complexity and challenges. The technical challenges of obtaining these data were substantial in both collection and analysis. There are multiple technical complexities arising from dual injections (A13 and MFB coordinates) and targeting the area correctly. The A13 region is difficult to target as it spans only around 300 µm in the anterior-posterior axis. While clearing the brain takes weeks, and light-sheet imaging also takes time, the time necessary to analyze the tissue using whole-brain quantification is labor intensive, especially with a lack of a standardized analysis pipeline from atlas registrations, signal segmentations, and quantifications. The field is still relatively new, requiring additional time to refine pipelines.

Correlation matrices are often used in analyzing connectivity patterns on a brain-wide scale, as they can identify any observable patterns within a large amount of data. We used correlation matrices to display estimated correlation coefficients between the afferent and efferent proportions from one brain subregion to another across 251 brain regions in total in a pairwise manner (not for hypothesis testing). We provided descriptive statistics (mean and error bars) in Figure 5C and G. As mentioned in comments for Reviewer 1, we will also present data in a revised Figure 5 and/or a new figure that focuses specifically on motor-related pathways to provide information on possible therapeutic pathways. As suggested, absolute values will be shared in a supplemental table.

In the absence of changes in the number of dopaminergic A13 neurons after 6-OHDA injection, results from this correlation analysis are difficult to interpret as they might reflect changes from various impaired brain regions independently of the A13 region.

We acknowledge that models of Parkinson’s disease, particularly those using 6-OHDA, induce plasticity in various regions, which may subsequently affect A13 connectivity. Our aim is to emphasize the residual, intact A13 pathways that could serve as therapeutic targets in future investigations. This emphasis is pertinent in the context of potential clinical applications, as the overall input and output to the region fundamentally dictate the significance of the A13 region in lesioned nigrostriatal models. We agree with the reviewer that the changes certainly can be independent of A13; however, the fact that there was a significant change in the connectome post-6-OHDA injection and striatonigral degeneration is in and of itself important and important to document.

There is no causal link between anatomical and behavioral data, which raises questions about the relevance of the anatomical data.

This point was also addressed earlier in response to a comment from Reviewer 1. Focusing on specific motor pathways is one avenue to explore. However, given that the zona incerta acts as an integrative hub, we believed it is prudent to initially examine both afferent and efferent pathways using a brain-wide approach. For instance, without employing this methodology, the potential significance of cortical interconnectivity to the A13 region might not have been fully appreciated. As mentioned previously, we will place additional emphasis on motor-related regions in our revised paper, thereby enhancing the relevance of the anatomical data presented. With these modifications, we anticipate that our data will underscore specific motor-related targets for future exploration, employing optogenetic targeting to assess necessity and sufficiency.

Overall, the study does not take advantage of genetic tools accessible in the mouse to address the direct or indirect behavioral and anatomical contributions of the A13 region to motor control and recovery after 6-OHDA injection.

We acknowledge that our study has not specifically targeted neurons that express dopaminergic, glutamatergic, or GABAergic properties (refer to earlier comment for more detail). However, like others, we find that targeting one neuronal population often does not result in a pure transmitter phenotype. For instance, evidence suggests co-localization of dopamine neurons with a subpopulation of GABA neurons in the A13/medial zona incerta (Negishi et al. 2020). In the hypothalamus, research by Deisseroth and colleagues (Romanov et al. 2017) indicates the presence of multiple classes of dopamine cells, each containing different ratios of co-localized peptides and/or fast neurotransmitters. Consequently, we believe our work lays the foundation for the investigations suggested by the reviewer. Furthermore, if one considers this work in the context of a preclinical study to determine whether the A13 might be a target in human Parkinson's disease, the existing technology that could be utilized is deep brain stimulation (DBS) or electrical modulation, which would also affect different neuronal populations in a non-specific manner. While optogenetic stimulation therapy is longer term, using CamKII combined with the DJ hybrid AAV could be a translatable strategy for targeting A13 neuronal populations in non-human primates (Watakabe et al. 2015; Watanabe et al. 2020).

Reviewer #3 (Public Review):

Kim, Lognon et al. present an important finding on pro-locomotor effects of optogenetic activation of the A13 region, which they identify as a dopamine-containing area of the medial zona incerta that undergoes profound remodeling in terms of afferent and efferent connectivity after administration of 6-OHDA to the MFB. The authors claim to address a model of PD-related gait dysfunction, a contentious problem that can be difficult to treat with dopaminergic medication or DBS in conventional targets. They make use of an impressive array of technologies to gain insight into the role of A13 remodeling in the 6-OHDA model of PD. The evidence provided is solid and the paper is well written, but there are several general issues that reduce the value of the paper in its current form, and a number of specific, more minor ones. Also, some suggestions, that may improve the paper compared to its recent form, come to mind.

Thank you for the suggestions and careful consideration of our work - it is appreciated.

The most fundamental issue that needs to be addressed is the relation of the structural to the behavioral findings. It would be very interesting to see whether the structural heterogeneity in afferent/effects projections induced by 6-OHDA is related to the degree of symptom severity and motor improvement during A13 stimulation.

As mentioned in comments for Reviewer 1, we will be highlighting motor-related A13 pathways in a revised Figure 5 and/or a new figure. We hope that our work will provide a roadmap for future studies to disentangle divergent or convergent A13 pathways that are involved in different or all PD-related motor symptoms. Because we could not measure behavioural change in the same animals studied with the anatomic study (essentially because the optrode would have significantly disrupted the connectome we are measuring), we cannot directly compare behaviour to structure.

The authors provide extensive interrogation of large-scale changes in the organization of the A13 region afferent and efferent distributions. It remains unclear how many animals were included to produce Fig 4 and 5. Fig S5 suggests that only 3 animals were used, is that correct? Please provide details about the heterogeneity between animals. Please provide a table detailing how many animals were used for which experiment. Were the same animals used for several experiments?

The behavioral set and the anatomical set were necessarily distinct. In the anatomical experiments, we employed both anterograde and retrograde viral approaches to target the afferent and efferent A13 populations with fluorescent proteins. For the behavioral approach, a single ChR2 opsin was utilized to photostimulate the A13 region; hence combining the two populations was not feasible. We were also concerned that the optrode itself would interfere with connectomics. A lower number of animals were used for the whole-brain work due to technical limitations described earlier. We will provide more details regarding numbers we can identify as a table and text.

While the authors provide evidence that photoactivation of the A13 is sufficient in driving locomotion in the OFT, this pro-locomotor effect seems to be independent of 6-OHDA-induced pathophysiology. Only in the pole test do they find that there seems to be a difference between Sham vs 6-OHDA concerning the effects of photoactivation of the A13. Because of these behavioral findings, optogenic activation of A13 may represent a gain of function rather than disease-specific rescue. This needs to be highlighted more explicitly in the title, abstract, and conclusion.

We agree with the reviewer that this aspect needs to be highlighted more. Optogenetic activation of A13 may represent a gain of function in both healthy and 6-OHDA mice, highlighting a parallel descending motor pathway that remains intact. 6-OHDA lesions have multiple effects on motor and cognitive function. This makes a single pathway unlikely to rescue all deficits observed in 6-OHDA models. We can say that the lack of locomotion observed in 6-OHDA models can be reversed by A13 region stimulation. We have discussed some aspects of the gain of function possibility but will augment this in other areas of the paper as well, as suggested.

The authors claim that A13 may be a possible target for DBS to treat gait dysfunction. However, the experimental evidence provided (in particular the lack of disease-specific changes in the OFT) seems insufficient to draw such conclusions. It needs to be highlighted that optogenetic activation does not necessarily have the same effects as DBS (see the recent review from Neumann et al. in Brain: https://pubmed.ncbi.nlm.nih.gov/37450573/). This is important because ZI-DBS so far had very mixed clinical effects. The authors should provide plausible reasons for these discrepancies. Is cell-specificity, which only optogenetic interventions can achieve, necessary? Can new forms of cyclic burst DBS achieve similar specificity (Spix et al, Science 2021)? Please comment.

Thank you for the useful comments - we will update our discussion accordingly.

Our study highlights a parallel motor pathway provided by the A13 region that remains intact in 6-OHDA mice and can be sufficiently driven to rescue the hypolocomotor pathology observed in the OFT and overcome bradykinesia and akinesia. The photoactivation of ipsilesional A13 also has an overall additive effect on ipsiversive circling, representing a gain of function on the intact side that contributes to the magnitude of overall motor asymmetry against the lesioned side. The effects of DBS are rather complex, ranging from micro-, meso-, to macro-scales, involving activation, inhibition, and informational lesioning, and network interactions. This could contribute to the mixed clinical effects observed with ZI-DBS, in addition to differences in targeting and DBS programming among the studies (see review (Ossowska 2019)). Also the DBS studies targeting ZI have never targeted the rostromedial ZI which extends towards the hypothalamus and contains the A13. Furthermore, DBS and electrical stimulation of neural tissue, in general, are always limited by current spread and lower thresholds of activation of axons (e.g., axons of passage), both of which can reduce the specificity of the true therapeutic target. Optogenetic studies have provided mechanistic insights that could be leveraged in overcoming some of the limitations in targeting with conventional DBS approaches. Spix et al. (2021) provided an interesting approach highlighting these advancements. They devised burst stimulation to facilitate population-specific neuromodulation within the external globus pallidus. Moreover, they found a complementary role for optogenetics in exploring the pathway-specific activation of neurons activated by DBS. To ascertain whether A13 DBS may be a viable therapy for PD gait, it will be necessary to perform many more preclinical experiments, and tuning of DBS parameters could be facilitated by optogenetic stimulation in these murine models.

In a recent study, Jeon et al (Topographic connectivity and cellular profiling reveal detailed input pathways and functionally distinct cell types in the subthalamic nucleus, 2022, Cell Reports) provided evidence on the topographically graded organization of STN afferents and McElvain et al. (Specific populations of basal ganglia output neurons target distinct brain stem areas while collateralizing throughout the diencephalon, 2021, Neuron) have shown similar topographical resolution for SNr efferents. Can a similar topographical organization of efferents and afferents be derived for the A13/ ZI in total?

The ZI can be subdivided into four subregions in the antero-posterior axis: rostral (ZIr), dorsal (ZId), ventral (ZIv), and caudal (ZIc) regions. The dorsal and ventral ZI is also referred together as central/medial/intermediate ZI. There are topographical gradients in different cell types and connectivity across these subregions (see reviews: (Mitrofanis 2005; Monosov et al. 2022; Ossowska 2019). Recent work by Yang and colleagues (2022) demonstrated a topographical organization among the inputs and outputs of GABAergic (VGAT) populations across four ZI subregions. Given that A13 region encompasses a smaller portion (the medial aspect) of both rostral and medial/central ZI (three of four ZI subregions) and coexpress VGAT, A13 region likely falls under rostral and intermediate medial ZI dataset found in Yang et al. (2022). With our data, we would not be able to capture the breadth of topographical organization shown in Yang et al (2022).

In conclusion, this is an interesting study that can be improved by taking into consideration the points mentioned above.

Reviewer #1 (Recommendations For The Authors):

1) Figure 2 indeed presents valuable information regarding the effects of A13 region photoactivation. To enhance the comprehensiveness of this figure and gain a deeper understanding of the neurons driving the pro-locomotor effect of stimulation, it would be beneficial to include quantifications of various cell types:

• cFos-Positive Cells/TH-Positive Cells: it can help determine the impact of A13 stimulation on dopaminergic neurons and the associated pro-locomotor effect in the healthy condition and especially in the context of Parkinson's disease (PD) modeling.

• cFos-Positive Cells /TH-Negative Cells: Investigating the number of TH-negative cells activated by stimulation is also important, as it may reveal non-dopaminergic neurons that play a role in locomotor responses. Identifying the location and characteristics of these TH-negative cells can provide insights into their functional significance.

Incorporating these quantifications into Figure 2 would enhance the figure's informativeness and provide a more comprehensive view of the neuronal populations involved in the locomotor effects of A13 stimulation.

Agreed - we will add quantification and create graphs to present the data in Figure 2.

2) Refer to Figure 3. In the main text (page 5) when describing the animal with 6-OHDA the wrong panels are indicated. It is indicated in Fgure 2A-E but it should be replaced with 3A-E. Please do that.

Will be done

Reviewer #2 (Recommendations For The Authors):

Abstract

Page 1: Inhibitory or lesion studies will be necessary to support the claim that the global remodeling of afferent and efferent projections of the A13 region highlights the Zona Incerta's role as a crucial hub for the rapid selection of motor function.

We believe that overall, there is quite a bit of evidence that the zona incerta is a hub for afferent/efferents. Mitrofanis (2005) and, more recently, Wang et al. (2020) summarize some of the evidence. Yang (2022) illustrates that the zona incerta shows multiple inputs to GABAergic neurons and outputs to diverse regions. Recent work suggests that the zona incerta contributes to various motor functions such as hunting, exploratory locomotion, and integrating multiple modalities (Zhao et al. 2019; Wang et al. 2019; Monosov et al. 2022; Chometton et al. 2017). We will update our paper to reflect these references.

Introduction

Page 2, paragraph 2: "However, little attention has been placed on the medial zona incerta (mZI), particularly the A13, the only dopamine-containing region of the rostral ZI" Is the A13 region located in the rostral or medial ZI or both?

It should have been written “rostromedial” ZI. The A13 is located in the medial aspect of rostromedial ZI. We will update the introduction.

Page 2, para 3: Li et al (2021) used a mini-endoscope to record the GCaMP6 signal. Masini and Kiehn, 2022 transiently blocked the dopaminergic transmission; they never used 6-OHDA. Please correct through the text.

We will correct this.

Page 2, para 4: the A13 connectome encompasses the cerebral cortex,... MLR. The MLR is a functional region, correct this for the CNF and PPN.

Thank you, we will correct this.

Page 3, the last paragraph of the introduction could be clarified by presenting the behavioral data first, followed by the anatomy.

We will correct this.

Figure 1 is nice and clear, and well summarizes the experimental design.

Thank you.

Figure 2 shows an example of the extent of the ChR2-YFP expression and the position of an optical fiber tip above the dopaminergic A13 region from a mouse. Without any quantification, these images could be included in Figure 1. Despite a very small volume (36.8nL) of AAV, the extent of ChR2-YFP expression is quite large and includes dopaminergic and unidentified neurons within the A13 region but also a large population of unidentified neurons outside of it, thus raising questions about the volume and the types of neurons recruited.

This is an important consideration. As mentioned previously, we will provide more information on viral spread and optrode location. The issue of viral spread is complex and depends on factors including tissue type, serotype, and promotor of the virus. Li et al. (2021), for example, used different virus serotypes and promotors, injecting 150 nL, whereas we used AAV DJ, injecting 36.8nL. AAV-DJ is a hybrid viral type consisting of multiple serotypes. It has a high transduction efficiency, which leads to greater gene delivery than single-serotype AAV viral constructs (Mao et al. 2016). A secondary consideration regarding translation was that AAV-DJ could effectively transduce non-primate neurons (Watanabe et al. 2020). We have addressed the issue of neurons recruited earlier and will provide c-Fos quantification to illustrate the extent of co-localization with TH.

Anatomical reconstruction of the extent of the ChR2-YFP expression and the location of the tip of the optical fiber will be necessary to confirm that ChR2-YFP expression was restricted to the A13 region.

We will provide additional information regarding viral spread, ferrule tip placement, and c-fos cell counts.

Page 5, 1st para: Double-check the references, as not all of them are 6-OHDA injections in the MLF.

Will correct.

Page 5, 1st para, 4th line: Replace ferrule with optical canula or fiber.

Will correct.

Page 5, 1st para, 9th line: Replace Figure 2 with Figure 3.

Will correct.

Page 5, 2nd para: About the refractory decrease in traveled distance by sham-ChR2 mice: is this significant?

It was not significant (Figure S1, 1-way RM ANOVA: F5,25 = 0.486, P = 0.783)). We will update this.

Figure 3 showing behavioral assessments is nice, but the stats are not always clear. In Fig 3A, are each of the off and on boxes 1 minute long? The figure legend states the test lasts 1 min, but isn't it 4 minutes? In Figure 3B-E and 3J-M, what are the differences? Do the stats identify a significant difference only during the stimulation phase? Fig. 3F-I are nice and could have been presented as primary examples prior to data analysis in Fig. 3B-E. Group labels above the graph would help.

Yes, the off-on boxes are 1 minute long. We will correct the error in the legend. Great suggestion for F-I - we will move them ahead of the summary figures.

Fig. 3L-M, what do PreSur, Post, and Ferrule mean? I assume that Ferrule refers to mice tested with the optical fiber without stimulation, whereas Stim. refers to the stimulation. It would be helpful to standardize the format of stats in Fig. 3B-E and 3-J-M. What are time points a, b, and c referring to?

We will do this.

Figure S2A: the higher variability in 6-OHDA-YFP mice in comparison to 6-OHDA-ChR2 mice prior to stimulation suggests that 6-OHDA-YFP mice were less impaired. Why use boxplots only for these data? Would a pairwise comparison be more appropriate?

Data did not follow a normal distribution and thus, were plotted as box and whiskers with the horizontal line through the box indicating the group median, interquartile range indicated by the limits of the box, and group minimum and maximum indicated by the whiskers. And indeed, a non-parametric equivalent of paired t-test (Wilcoxon signed-rank test) was used.

Fig. S2B: add the statistical marker.

Will do

Page 7, para 1, line 8: to add "in comparison to 6-OHDA-YFP and YFP mice" to during photostimulation... (Figure 3E).

Will do

Page 7, para 3, line 5: about larger improvement, replace "sham ChR2" with "6-OHDA."

Will do

Page 8, para 1, line 4: Perier et al., 2000 reported that 6-OHDA injection increased the firing frequency of the ZI over a month.

We will add that time frame. Agreed, it is shorter than the behavioral work, which was started 3 weeks after 6-OHDA injection.

Page 8, para 2, line 1: Since the results were expected, add some references.

Will do

Page 8, para 3, line 4. Double-check the reference.

Will correct and update

Page 8: About large-scale changes in the A13 region, the relevance of correlation matrices is difficult to grasp. Analysis of local connectivity would have been more informative in the context of GABAergic and glutamatergic neurons of the ZI in the vicinity of the A13 region.

We will explore alternative methods to present the data.

Page 8, para 3, line: given Fig. 2, there is concern about the claim that only the A13 region was targeted. The time of the analysis after 6-OHDA should be mentioned. Some sections of the paragraph could be moved to methods.

As mentioned earlier, we will provide additional information regarding viral spread, ferrule tip placement, and c-fos cell counts. We will mention analysis time after 6-OHDA and update Figure 1a to include this.

Fig. 4: The color code helps the reader visualize distribution differences. However, statistical analyses comparing 6-OHDA versus sham should be included. Quantification per region would greatly help readers visualize the data and support the conclusion. The relationship between the type of correlation (positive or negative) and absolute change (increase or decrease) is unknown in the current format, which limits the interpretation of the data. Moreover, examples of raw images of axons and cells should be presented for several brain regions. The experimental design with a timeline, as in Fig. 1, would be helpful. The legend for Fig. 4 is a bit long. Some sections are very descriptive, whereas others are more interpretive.

We will explore alternative methods of presenting the data, as suggested in a previous comment. Should we retain the correlation matrix, we will incorporate the reviewer’s suggestions.

Page 10, para 1, line 1: add "afferent" to "changes in -afferent and- projection patterns."

Will do

Page 10, para 1, line 9: remove the 2nd "compared to sham" in the sentence.

Will do

10, para 1, line 10: remove "coordinated" in "several regions showed a coordinated reduction in afferent density." We cannot say anything about the timing of events, as there is only info at 1 month.

Will do

Page 10, para 2: the section should be written in the past tense.

Will do

Page 13, para 2, the last sentence is overstated. Please remove "cells" and refer to the A13 region instead.

Will do

About differential remodelling of the A13 region connectome: Figure 5C and 5G: The proportion of total afferents ipsi- and contralateral to 6-OHDA injection argues that the A13 region primarily receives inputs from the cortical plate and the striatum. Unfortunately, there are no statistics.

Due to the small sample size, we provided descriptive statistics (mean and error bars) in Figure 5C and G. As mentioned in comments for Reviewers 1 and 2, we will revise Figure 5 to present data focusing on motor-related pathways to provide clarity. In addition, absolute values will be shared in a supplemental table.

Figure 5 D and 5H: Changes in the proportion of total afferents/projections are relatively modest (less than 10% of the whole population for the highest changes). There is no standard deviation for these data and no statistics. Do they reflect real changes or variability from the injection site?

The changes are relatively modest (less than 10%) since a small brain region usually provides a very small proportion of total input (McElvain et al. 2021; Yang et al. 2022). The changes in the proportions reflect real differences between average proportions observed in sham and 6-OHDA mice. The variability in the total labeling of neurons and fibers was minimized by normalizing individual regional counts against total counts found in each individual animal.

Fig 5F and H: The example in F shows a huge decrease in the striatum, but H indicates only a 2% change, which makes the example not very representative. Absolute values would be helpful.

While a 2% change may seem small, it represents a relatively large change in the A13 efferent connectome. To provide further clarity, we will provide absolute values as suggested in our new supplemental table.

Figure 6 is inaccurate and unnecessary.

Agree - it is too simplistic. We will remove it and replace it with one outlined in comments to Reviewer 1.

Discussion

Although interesting, the discussion is too long.

We will make it more concise in the revised paper.

Page 12: para 2. If the A13 region has a pro-locomotor effect and has therapeutical potential; the claim about its plasticity relies on Fig. 4 and 5, which have a limited scope in the current analysis and presentation (see comments above).

We will revise the paper per the comments above and then update this accordingly.

Methods

Page 17, para 1: include the stereotaxic coordinates of the optical cannula above the A13 region.

We will include this information.

References

Chen, Fenghua, Junliang Qian, Zhongkai Cao, Ang Li, Juntao Cui, Limin Shi, and Junxia Xie. 2023. “Chemogenetic and Optogenetic Stimulation of Zona Incerta GABAergic Neurons Ameliorates Motor Impairment in Parkinson’s Disease.” iScience 26 (7). https://doi.org/10.1016/j.isci.2023.107149.

Chometton, S., K. Charrière, L. Bayer, C. Houdayer, G. Franchi, F. Poncet, D. Fellmann, and P. Y. Risold. 2017. “The Rostromedial Zona Incerta Is Involved in Attentional Processes While Adjacent LHA Responds to Arousal: C-Fos and Anatomical Evidence.” Brain Structure & Function 222 (6): 2507–25.

Garau, Celia, Jessica Hayes, Giulia Chiacchierini, James E. McCutcheon, and John Apergis-Schoute. 2023. “Involvement of A13 Dopaminergic Neurons in Prehensile Movements but Not Reward in the Rat.” Current Biology: CB, October. https://doi.org/10.1016/j.cub.2023.09.044.

Li, Zhuoliang, Giorgio Rizzi, and Kelly R. Tan. 2021. “Zona Incerta Subpopulations Differentially Encode and Modulate Anxiety.” Science Advances 7 (37): eabf6709.

Mao, Yingying, Xuejun Wang, Renhe Yan, Wei Hu, Andrew Li, Shengqi Wang, and Hongwei Li. 2016. “Single Point Mutation in Adeno-Associated Viral Vectors -DJ Capsid Leads to Improvement for Gene Delivery in Vivo.” BMC Biotechnology 16 (January): 1.

McElvain, Lauren E., Yuncong Chen, Jeffrey D. Moore, G. Stefano Brigidi, Brenda L. Bloodgood, Byung Kook Lim, Rui M. Costa, and David Kleinfeld. 2021. “Specific Populations of Basal Ganglia Output Neurons Target Distinct Brain Stem Areas While Collateralizing throughout the Diencephalon.” Neuron 109 (10): 1721–38.e4.

Mitrofanis, J. 2005. “Some Certainty for the ‘Zone of Uncertainty’? Exploring the Function of the Zona Incerta.” Neuroscience 130 (1): 1–15.

Monosov, Ilya E., Takaya Ogasawara, Suzanne N. Haber, J. Alexander Heimel, and Mehran Ahmadlou. 2022. “The Zona Incerta in Control of Novelty Seeking and Investigation across Species.” Current Opinion in Neurobiology 77 (December): 102650.

Negishi, Kenichiro, Mikayla A. Payant, Kayla S. Schumacker, Gabor Wittmann, Rebecca M. Butler, Ronald M. Lechan, Harry W. M. Steinbusch, Arshad M. Khan, and Melissa J. Chee. 2020. “Distributions of Hypothalamic Neuron Populations Coexpressing Tyrosine Hydroxylase and the Vesicular GABA Transporter in the Mouse.” The Journal of Comparative Neurology 528 (11): 1833–55.

Ossowska, Krystyna. 2019. “Zona Incerta as a Therapeutic Target in Parkinson’s Disease.” Journal of Neurology. https://doi.org/10.1007/s00415-019-09486-8.

Romanov, Roman A., Amit Zeisel, Joanne Bakker, Fatima Girach, Arash Hellysaz, Raju Tomer, Alán Alpár, et al. 2017. “Molecular Interrogation of Hypothalamic Organization Reveals Distinct Dopamine Neuronal Subtypes.” Nature Neuroscience 20 (2): 176–88.

Spix, Teresa A., Shruti Nanivadekar, Noelle Toong, Irene M. Kaplow, Brian R. Isett, Yazel Goksen, Andreas R. Pfenning, and Aryn H. Gittis. 2021. “Population-Specific Neuromodulation Prolongs Therapeutic Benefits of Deep Brain Stimulation.” Science 374 (6564): 201–6.

Wang, Xiyue, Xiaolin Chou, Bo Peng, Li Shen, Junxiang J. Huang, Li I. Zhang, and Huizhong W. Tao. 2019. “A Cross-Modality Enhancement of Defensive Flight via Parvalbumin Neurons in Zona Incerta.” eLife 8 (April). https://doi.org/10.7554/eLife.42728.

Wang, Xiyue, Xiao-Lin Chou, Li I. Zhang, and Huizhong Whit Tao. 2020. “Zona Incerta: An Integrative Node for Global Behavioral Modulation.” Trends in Neurosciences 43 (2): 82–87.

Watakabe, Akiya, Masanari Ohtsuka, Masaharu Kinoshita, Masafumi Takaji, Kaoru Isa, Hiroaki Mizukami, Keiya Ozawa, Tadashi Isa, and Tetsuo Yamamori. 2015. “Comparative Analyses of Adeno-Associated Viral Vector Serotypes 1, 2, 5, 8 and 9 in Marmoset, Mouse and Macaque Cerebral Cortex.” Neuroscience Research 93 (April): 144–57.

Watanabe, Hidenori, Hiromi Sano, Satomi Chiken, Kenta Kobayashi, Yuko Fukata, Masaki Fukata, Hajime Mushiake, and Atsushi Nambu. 2020. “Forelimb Movements Evoked by Optogenetic Stimulation of the Macaque Motor Cortex.” Nature Communications 11 (1): 3253.

Yang, Yang, Tao Jiang, Xueyan Jia, Jing Yuan, Xiangning Li, and Hui Gong. 2022. “Whole-Brain Connectome of GABAergic Neurons in the Mouse Zona Incerta.” Neuroscience Bulletin 38 (11): 1315–29.

Ye, Qiying, Jeremiah Nunez, and Xiaobing Zhang. 2023. “Zona Incerta Dopamine Neurons Encode Motivational Vigor in Food Seeking.” bioRxiv : The Preprint Server for Biology, June. https://doi.org/10.1101/2023.06.29.547060.

Zhao, Zheng-Dong, Zongming Chen, Xinkuan Xiang, Mengna Hu, Hengchang Xie, Xiaoning Jia, Fang Cai, et al. 2019. “Zona Incerta GABAergic Neurons Integrate Prey-Related Sensory Signals and Induce an Appetitive Drive to Promote Hunting.” Nature Neuroscience 22 (6): 921–32.

Reviewer #2 (Public Review):

Summary<br /> This paper expands on the literature on spatial metamers, evaluating different aspects of spatial metamers including the effect of different models and initialization conditions, as well as the relationship between metamers of the human visual system and metamers for a model. The authors conduct psychophysics experiments testing variations of metamer synthesis parameters including type of target image, scaling factor, and initialization parameters, and also compare two different metamer models (luminance vs energy). An additional contribution is doing this for a field of view larger than has been explored previously.

General Comments<br /> Overall, this paper addresses some important outstanding questions regarding comparing original to synthesized images in metamer experiments and begins to explore the effect of noise vs image seed on the resulting syntheses. While the paper tests some model classes that could be better motivated, and the results are not particularly groundbreaking, the contributions are convincing and undoubtedly important to the field. The paper includes an interesting Voronoi-like schematic of how to think about perceptual metamers, which I found helpful, but for which I do have some questions and suggestions. I also have some major concerns regarding incomplete psychophysical methodology including lack of eye-tracking, results inferred from a single subject, and a huge number of trials. I have only minor typographical criticisms and suggestions to improve clarity. The authors also use very good data reproducibility practices.

Specific Comments

Experimental Setup<br /> Firstly, the experiments do not appear to utilize an eye tracker to monitor fixation. Without eye tracking or another manipulation to ensure fixation, we cannot ensure the subjects were fixating the center of the image, and viewing the metamer as intended. While the short stimulus time (200ms) can help minimize eye movements, this does not guarantee that subjects began the trial with correct fixation, especially in such a long experiment. While Covid-19 did at one point limit in-person eye-tracked experiments, the paper reports no such restrictions that would have made the addition of eye-tracking impossible. While such a large-scale experiment may be difficult to repeat with the addition of eye tracking, the paper would be greatly improved with, at a minimum, an explanation as to why eye tracking was not included.

Secondly, many of the comparisons later in the paper (Figures 9,10) are made from a single subject. N=1 is not typically accepted as sufficient to draw conclusions in such a psychophysics experiment. Again, if there were restrictions limiting this it should be discussed. Also (P11) Is subject sub-00 is this an author? Other expert? A naive subject? The subject's expertise in viewing metamers will likely affect their performance.

Finally, the number of trials per subject is quite large. 13,000 over 9 sessions is much larger than most human experiments in this area. The reason for this should be justified.

Model<br /> For the main experiment, the authors compare the results of two models: a 'luminance model' that spatially pools mean luminance values, and an 'energy model' that spatially pools energy calculated from a multi-scale pyramid decomposition. They show that these models create metamers that result in different thresholds for human performance, and therefore different critical scaling parameters, with the basic luminance pooling model producing a scaling factor 1/4 that of the energy model. While this is certain to be true, due to the luminance model being so much simpler, the motivation for the simple luminance-based model as a comparison is unclear.

The authors claim that this luminance model captures the response of retinal ganglion cells, often modeled as a center-surround operation (Rodieck, 1964). I am unclear in what aspect(s) the authors claim these center-surround neurons mimic a simple mean luminance, especially in the context of evidence supporting a much more complex role of RGCs in vision (Atick & Redlich, 1992). Why do the authors not compare the energy model to a model that captures center-surround responses instead? Do the authors mean to claim that the luminance model captures only the pooling aspects of an RGC model? This is particularly confusing as Figures 6 and 9 show the luminance and energy models for original vs synth aligning with the scaling of Midget and Parasol RGCs, respectively. These claims should be more clearly stated, and citations included to motivate this. Similarly, with the energy model, the physiological evidence is very loosely connected to the model discussed.

Prior Work:<br /> While the explorations in this paper clearly have value, it does not present any particularly groundbreaking results, and those reported are consistent with previous literature. The explorations around critical eccentricity measurement have been done for texture models (Figure 11) in multiple papers (Freeman 2011, Wallis, 2019, Balas 2009). In particular, Freeman 20111 demonstrated that simpler models, representing measurements presumed to occur earlier in visual processing need smaller pooling regions to achieve metamerism. This work's measurements for the simpler models tested here are consistent with those results, though the model details are different. In addition, Brown, 2023 (which is miscited) also used an extended field of view (though not as large as in this work). Both Brown 2023, and Wallis 2019 performed an exploration of the effect of the target image. Also, much of the more recent previous work uses color images, while the author's exploration is only done for greyscale.

Discussion of Prior Work:<br /> The prior work on testing metamerism between original vs. synthesized and synthesized vs. synthesized images is presented in a misleading way. Wallis et al.'s prior work on this should not be a minor remark in the post-experiment discussion. Rather, it was surely a motivation for the experiment. The text should make this clear; a discussion of Wallis et al. should appear at the start of that section. The authors similarly cite much of the most relevant literature in this area as a minor remark at the end of the introduction (P3L72).

White Noise:<br /> The authors make an analogy to the inability of humans to distinguish samples of white noise. It is unclear however that human difficulty distinguishing samples of white noise is a perceptual issue- It could instead perhaps be due to cognitive/memory limitations. If one concentrates on an individual patch one can usually tell apart two samples. Support for these difficulties emerging from perceptual limitations, or a discussion of the possibility of these limitations being more cognitive should be discussed, or a different analogy employed.

Relatedly, in Figure 14, the authors do not explain why the white noise seeds would be more likely to produce syntheses that end up in different human equivalence classes.

It would be nice to see the effect of pink noise seeds, which mirror the power spectrum of natural images, but do not contain the same structure as natural images - this may address the artifacts noted in Figure 9b.

Finally, the authors note high-frequency artifacts in Figure 4 & P5L135, that remain after syntheses from the luminance model. They hypothesize that this is due to a lack of constraints on frequencies above that defined by the pooling region size. Could these be addressed with a white noise image seed that is pre-blurred with a low pass filter removing the frequencies above the spatial frequency constrained at the given eccentricity?

Schematic of metamerism:<br /> Figures 1,2,12, and 13 show a visual schematic of the state space of images, and their relationship to both model and human metamers. This is depicted as a Voronoi diagram, with individual images near the center of each shape, and other images that fall at different locations within the same cell producing the same human visual system response. I felt this conceptualization was helpful. However, implicitly it seems to make a distinction between metamerism and JND (just noticeable difference). I felt this would be better made explicit. In the case of JND, neighboring points, despite having different visual system responses, might not be distinguishable to a human observer.

In these diagrams and throughout the paper, the phrase 'visual stimulus' rather than 'image' would improve clarity, because the location of the stimulus in relation to the fovea matters whereas the image can be interpreted as the pixels displayed on the computer.

Other<br /> The authors show good reproducibility practices with links to relevant code, datasets, and figures.

Oh, this is new to me - if you want the cheap money, yo need to meet the ECOCDC

It reminded me of the video I have watched before about an Indian software engineer. He worked for Microsoft and talked about the experience to code as someone with disability.

Est-ce vrai ? Ou bien est-ce juste une considération ?

Doublon

So based on my understanding, the code of import draw is basically function like calling a house keeper to tell him to do something. The whole draw thing is just a bridge or transformer.

Using Python with the PRAW library to access Reddit is a powerful way to interact with the platform programmatically. It's interesting to note that PRAW leverages Reddit's API, which essentially serves as a bridge for communication between your code and the Reddit servers. Understanding how APIs work and the rules they follow is crucial for efficient data retrieval and interaction with online services like Reddit. It's a testament to the versatility and potential of programming when combined with APIs like Reddit's to automate tasks and gather data.

In discussing how to access Reddit using Python and the "PRAW" code library, the chapter emphasizes the essential function that APIs play in making this procedure possible. Reddit's API is used by PRAW to send internet queries, and it establishes the guidelines for request kinds, responses, and data extraction. In addition to warning that it might not be user-friendly for novices and might require considerable acclimation, the text suggests that individuals who are interested in learning more about PRAW contact the official documentation. Although it reiterates that the organization of the documentation may not be immediately novice-friendly, it provides insights into PRAW's capabilities and data retrieval, particularly by investigating the praw models, their associated data types, and the accessible functions.

Create an event listener for the DOM after it loads.

Author Response

The following is the authors’ response to the original reviews.

We thank the editor and the reviewers for their very useful and constructive comments. We went through the list and gladly received all their suggestions. The reviewers mostly pointed to minor revisions in the text, and we acted on all of those. The one suggestion that required major work was the one raised in point 13, about the processing pipeline being unconvincingly scattered between different tools (R → Python → Matlab). I agree that this was a major annoyance, and I am happy to say we have solved it integrating everything in a recent version of the ethoscopy software (available on biorxiv with DOI https://www.biorxiv.org/content/10.1101/2022.11.28.517675v2 and in press with Bioinformatics Advances). End users will now be able to perform coccinella analysis using ethoscopy only, thus relying on nothing else but Python as their data analysis tool. This revised version of the manuscript now includes two Jupyter Notebooks as supplementary material with a “pre-cooked” sample recipe of how to do that. This should really simplify adoption and provides more details on the pipeline used for phenotyping.

Please find below a point-by-point description of how we incorporated all the reviewers’ excellent suggestions.

Recommendations for the authors: please note that you control which, if any, revisions, to undertake

1) Line 38: "collecting data simultaneously from a large number of individuals with no or limited human intervention" is a bit misleading, as the entire condition the individuals are put in are highly modified by humans and most times "unnatural". I understand the point that once the animals are placed in these environments, then recording takes place without intervention, but it would be nice to rephrase this so that it reflects more accurately what is happening.

We have now rephrased this into the following (L39):

Collecting data simultaneously from a large number of individuals, which can remain undisturbed throughout recording.

2) Line 63: please add a reference to the Ethoscopes so that readers can easily find it.

Done.

2b) And also add how much they cost and the time needed to build them, as this will allow readers to better compare the proposed system against other commercially available ones.

This information is available on the ethoscope manual website (http://lab.gilest.ro/ethoscope). The price of one ethoscope, provided all necessary tools are available, is around ~£75 and the building time very much depends on the skillset of the builder and whether they are building their first ethoscope or subsequent ones. In our experience, building and adopting ethoscopes for the first time is not any more time-expensive than building a (e.g.) deeplabcut setup for the first time. We have added this information to L81

Ethoscopes are open source and can be manufactured by a skilled end-user at a cost of about £75 per machine, mostly building on two off-the-shelf component: a Raspberry Pi microcomputer and a Raspberry Pi NoIR camera overlooking a bespoke 3D printed arena hosting freely moving flies.

3) Line 88: The authors describe that in the current setting, their system is capable of an acquisition rate of 2.2 frames per second (FPS). Would reducing the resolution of the PiCamera allow for higher FPS? I raise this point because the authors state that max velocity over a ten second window is a good feature for classifying behaviors. However, if animals move much faster than the current acquisition rate, they could, for instance, be in position X, move about and be close to the initial position when the next data point is acquired, leading to a measured low max velocity, when in fact the opposite happened. I think it would be good to add a statement addressing this (either data from the literature showing that the low FPS does not compromise data acquisition, or a test where increasing greatly FPS leads to the same results).

We have previously performed a comparison of data analysed using videos captured at different FPSs, which is published in Quentin Geissman’s doctoral Thesis (2018, DOI: https://doi.org/10.25560/69514 ) in chapter 2, section 2.8.3, figure 2.9 ). We have now added this work as one of the references at L95 (reference 19).

4) Still on the low FPS, would a Raspberry Pi 4 help with the sampling rate? Given that they are more powerful than the RPi3 used in the paper?

It would, but it would be a minor increase, leading from 2.2 to probably 3-5 FPS. A significantly higher number of FPSs would be best achieved by lowering the camera’s resolution, as the reviewer’s suggested, or by operating offline. I think the interesting point being implied by the reviewers is that, for Drosophila, the current limits of resolution are more than sufficient. For other animals, perhaps moving more abruptly, they may not. The reviewer is right that we should add a line of caveat about this. We now do so in the discussion, lines 215-224.

Coccinella is a reductionist tool, not meant to replace the behavioural categorization that other tools can offer but to complement it. It relies on raspberry PIs as main acquisition devices, with associated advantages and limitations. Ethoscopes are inexpensive and versatile but have limitations in terms of computing power and acquisition rates. Their online acquisition speed is fast enough to successfully capture the motor activity of different species of Drosophilae28, but may not be sufficient for other animals moving more swiftly, such as zebrafish larvae. Moreover, coccinella cannot apply labels to behaviour (“courting”, “lounging”, “sipping”, “jumping” etc.) but it can successfully identify large behavioural phenotypes and generate unbiased hypothesis on how behaviour – and a nervous system at large – can be influenced by chemicals, genetics, artificial manipulations in general.

5) Along the same line of thought, would using a simple webcam (with similar specs to the PiCamera - ELP has cameras that operate on infrared and are quite affordable too) connected to a more powerful computer lead to higher FPS? - The reason for the question about using a simple webcam is that this would make your system more flexible (especially useful in the current shortage of RPi boards on the market) lowering the barrier for others to use it, increasing the chances for adoption.

Completely bypassing ethoscopes would require the users to setup their own tracking solution, with a final result that may or may not match what we describe here. If a greater temporal resolution is necessary, the easiest way to achieve more FPSs would be to either decrease camera resolution or use the Pis to take videos offline and then process those videos at a later stage. The combination of these two would give FPS acquisition of 60 fps at 720p, which is the maximum the camera can achieve. We now made this clear at lines 83-92.

The temporal and spatial resolution of the collected images depends on the working modality the user chooses. When operating in offline mode, ethoscopes are capable to acquire 720p videos at 60 fps, which is a convenient option with fast moving animals. In this study, we instead opted for the default ethoscope working settings, providing online tracking and realtime parametric extraction, meaning that images are analysed by each raspberry Pi at the very moment they were acquired (Figure 1b). This latter modality limits the temporal resolution of information being processed (one frame every 444 ms ± 127 ms, equivalent to 2.2 fps on a Raspberry Pi3 at a resolution of 1280x960 pixels with each animal being constricted in an ellipse measuring 25.8 ± 1.4 x 9.85 ±1.4 pixels - Figure 1a) but provides the most affordable and high-throughput solution, dispensing the researcher from organising video storage or asynchronous video processing for animals tracking.

6) One last point about decreasing use barrier and increasing adoption: Would it be possible to use DeepLabCut (DLC) to simply annotate each animal (instead of each body part) and feed the extracted data into your current analysis with coccinella? This way different labs that already have pipelines in place that use DLC would have a much easier time in testing and eventually switching to coccinella? I understand that extracting simple maximal velocity this way would be an overkill, but the trade-off would again be a lowering of the adoption barrier.

It would certainly be possible to calculate velocity from the whole animal pose measurement and then use this with HCTSA or Catch22, thus mimicking the coccinella pipeline, but it would be definitely overkilled, as the reviewers correctly points out. Given that we are trying to make an argument about high-throughput data acquisition I would rather not suggest this option in the manuscript.

7) Line 96: The authors state that once data is collected, it is put through a computational frameworkthat uses 7700 tests described in the literature so that meaningful discriminative features are found. I think it would be interesting to expand a bit on the explanation of how this framework deals multiple comparison/multiple testing issues.

We always use the full set of features on aggregate to train a classifier (e.g., TS_Classify in HCTSA) and that means no correction is necessary because the trained classifier only ever makes a single prediction (only one test is performed), so as long as it is done correctly (e.g., proper separation of training and test sets, etc.) then multiple hypothesis correction is not appropriate. This has been confirmed with the HCTSA/Catch22 author (Dr Ben Fulcher, personal communication). We have added a clarifying sentence about this to the methods (L315-318)

8) It would be nice to have a couple of lines explaining the choice of compounds used for testing and also why in some tests, 17 compounds were used, while in others 40, and then 12? I understand how much work it must be in terms of experiment preparation and data collection for these many flies and compounds, but these changes in the compounds used for testing without a more detailed explanation is suboptimal.

This is another good point. We have now added this information to the methods, in a section renamed “choice, handling and preparation of drugs” L280-285, which now reads like this:

The initial preliminary analysis was conducted using a group of 12 compounds “proof of principle” compounds and a solvent control. These compounds were initially used to compare both the video method and ethoscope method. After testing these initial compounds, it was found that the ethoscope methodology was more successful, and then the compound list was expanded to 17 (including the control) only using the ethoscope method. As a final test, we included additional compounds for a single concentration, bringing up the total to 40 (including control), also for the ethoscope method.

9) Line 119 states: "A similar drop in accuracy was observed using a smaller panel of 12 treatments (Supplementary Figure 2a)". It is actually Supplementary Figure 1c.

Thank you for noticing that! Now corrected. The Supplementary figures have also been renamed to obey eLife’s expected nomenclature (both Figure 1 – Figure supplements)

10) In some places the language seems a little outlandish and should either be removed or appropriately qualified. a- Lines 56-59 pose three questions that are either rhetorical or ill-posed. For example, "...minimal amount of information...behavior" implies there is a singular response but the response depends on many details such as to what degree do the authors want to "classify behavior".

Yes, those were meant as rhetorical questions indeed, but we prefer to keep them in, because we are hoping to generate this type of thoughts with the readers. These are concepts that may not be so obvious to someone who is just looking to apply an existing tool and may spring some reflection about what kind of data do they really want/need to acquire.

b) Some of the criticisms leveled at the state-of-the-art methods are probably unwarranted because the goals of the different approaches are different. The current method does not yield the type of rich information that DeepLabCut yields. So, depending on the application DeepLabCut may be the method of choice. The authors of the current manuscript should more clearly state that.

In the introduction and discussion we do try to stress that coccinella is not meant to replace tools like DLC. We have now added more emphasis to this concept, for instance to L212:

[tools like deeplabcut] are ideal – and irreplaceable – to identify behavioural patterns and study fine motor control but may be undue for many other uses.

And L215:

Coccinella is a reductionist tool not meant to replace the behavioural categorization that other tools can offer but to complement it

11) The application to sleep data appears suddenly in the manuscript. The authors should attempt to make with text change a smoother transition from drug screen to investigation into sleep.

I agree with this observation. We have now tried to add a couple of sentences to contextualise this experiment and hopefully make the connection appear more natural. Ultimately, this is a proof-ofprinciple example anyway so hopefully the reader will take it for what it is (L169).

Finally, to push the system to its limit, we asked coccinella to find qualitative differences not in pharmacologically induced changes in activity, but in a type of spontaneous behaviour mostly characterised by lack of movement: sleep. In particular, we wondered whether coccinella could provide biological insights comparing conditions of sleep rebound observed after different regimes of sleep deprivation. Drosophila melanogaster is known to show a strong, conserved homeostatic regulation of sleep that forces flies to recover at least in part lost sleep, for instance after a night of forceful sleep deprivation.

11b) Additionally, the beginning section of sleep experiments talks about sleep depth yet the conclusion drawn from sleep rebound says more about the validity of the current 5 min definition of sleep than about sleep depth. If this conclusion was misunderstood, it should be clarified. If it was not, the beginning text of the sleep section should be tailored to better fit the conclusion.

I am afraid we did not a good job at explaining a critical aspect here: the data fed to coccinella are the “raw” activity data, in which we are not making any assumption on the state of the animal. In other words, we do not use the 5-minutes at this or any other point to classify sleep and wakening. Nevertheless, coccinella picks the 300 seconds threshold as the critical one for discerning the two groups. This is interesting because it provides a full agnostic confirmation of the five minutes rule in D. melanogaster. We recognise this was not necessarily obvious from the text and now added a clarification at L189-201:

However, analysis of those same animals during rebound after sleep deprivation showed a clear clustering, segregating the samples in two subsets with separation around the 300 seconds inactivity trigger (Figure 3d). This result is important for two reasons: on one hand, it provides, for the third time, strong evidence that the system is not simply overfitting data of nought biological significance, given that it could not perform any better than a random classifier on the baseline control. On the other hand, coccinella could find biologically relevant differences on rebound data after different regimes of sleep deprivation. Interestingly enough, the 300 seconds threshold that coccinella independently identified has a deep intrinsic significance for the field, for it is considered to be the threshold beyond which flies lose arousal response to external stimuli, defining a “sleep quantum” (i.e.: the minimum amount of time required for transforming inactivity bouts into sleep bouts23,24,28). Coccinella’s analysis ran agnostic of the arbitrary 5-minutes threshold and yet identified the same value as the one able to segregate the two clusters, thus providing an independent confirmation of the fiveminutes rule in D. melanogaster.

12) Line 227: (standard food) - please add a link to a protocol or a detailed description on what is "standard food". This way others can precisely replicate what you are using. This is not my field, but I have the impression that food content/composition for these animals makes big changes in behaviour?

Yes, good point. We have now added the actual recipe to the methods L240:

Fly lines were maintained on a 12-hour light: 12-hour dark (LD) cycle and raised on polenta and yeast-based fly media (agar 96 g, polenta 240 g, fructose 960 g and Brewer’s yeast 1,200 g in 12 litres of water).

13) Data acquisition and processing: please add links to the code used.

Both the code and the raw data used to generate all the figures have been uploaded on Zenodo and available through their repository. Zenodo has a limit of 50GB per uploaded dataset so we had to split everything into two files, with two DOIs, given in the methods (L356, section “code and availability” - DOIs: 10.5281/zenodo.7335575 and 10.5281/zenodo.7393689). We have now also created a landing page for the entire project at http://lab.gilest.ro/coccinella and linked that landing page in the introduction (L64).

13b) Also your pipeline seems to use three different programming languages/environments... Any chance this could be reduced? Maybe there are R packages that can convert csv to matlab compatible formats, so you can avoid the Python step? (nothing against using the current pipeline per se, I am just thinking that for usability and adoption by other labs, the smaller amount of languages, the better?

This is a very important suggestion that highlights a clear limitation of the pipeline. I am happy to say that we worked on this and solved the problem integrating the Python version of Catch22 into the ethoscopy software. This means the two now integrate, and the entire analysis can be run within the Python ecosystem. HCTSA does not have a Python package unfortunately but we still streamlined the process so that one only has to go from Python to Matlab without passing through R. To be honest, Catch22 is the evolution of HCTSA and performs really well so I think that is what most users will want to use. We provide two supplementary notebooks to guide the reader through the process. One explains how to go from ethoscope data to an HCTSA compatible mat file. The other explains how ethoscope data integrate with Catch22 and provides many more examples than the ones found in the paper figures.

14) There are two sections named "References" (which are different from each other) on the manuscript I received and also on BioRxiv. Should one of them be a supplementary reference? Please correct it. I spent a bit of time trying to figure out why cited references in the paper had nothing to do with what was being described...

The second list of references actually applied only to the list of compounds in the supplementary table 1. When generating a collated PDF this appeared at the end of the document and created confusion. We have now amended the heading of that list in the following way, to read more appropriately:

I feel like this version of data poisoning is very interesting. It’s a way for others to stand against the power and have a way to control the situation. Without having to go out and protest they can just sabotaged their job listing so they have no other way of making money.

https://www.youtube.com/watch?v=ux1GXpzXt0U

Yet another PKM Guru channel. A mixture of (he says) Tiago Forte (BASB, CODE), zettelkasten, and PKM.

Uses Craft, Canva, Figma, Spotify (for ambient focus).

He's actively creating/using some of the words and definitions of others, but also creating his own "system" definition. (hubs, etc.) He redefines Forte's C.O.D.E. to give it his own spin: Capture, Connect, Create Share.

He's got a subtle proselytizing Christian underlying message. Mentions Bible. Has hat with word "WRSHP". "Adding value to someone else's life" by sharing. Personal conversation is important to him (proselytizing). Speaking at his church about what "God has put in his heart."

Turning notes into "diamonds"

There's an outline of a system here, but he doesn't show actual practice, which is possibly the most important part, otherwise it may be unusable theory. To be able to do this system, I think, one would need to already be conversant in what is going on generally in the space or have Forte's system under control. By this point, what is Wheeler's real contribution other than a small example?

meh....

Obsidian for Academic Publishing - A Walkthrough with Jason Yuh (4)

Al interior de su nota de capítulo, Jason hace resúmenes (algo sumamente importante) y divide en headers cada subcapítulo, junto con el número de la página.

Dentro de cada header, están en guiones (bullets) el número de la página seguido de un breve resumen o nota rápida – apuntes de lo que le interesa, un poco à la Luhmann.

En el número de página utiliza la "f", que en castellano sería la "s", de following o subsecuente.

En cada número(s) de página(s), Jason dice que se ha entrenado para atrapar el argumento principal o la premisa de la sección, para hacerlo en niveles de párrafos, y que cuando a veces no es disciplinado, lo hace al nivel de la sección, o al menos al nivel del capítulo.

Jason también utiliza apuntes al estilo: "skimmed mostly from 36-46". Para dejar en claro la huella de lo que leyó, pero también de lo que no leyó.

Tiene una nota "global" llamada "Abstracts", donde están todos los resúmenes de todos los artículos que ha leído.

Porque Jason hace abstracts de cada artículo que lee. Si hay uno ya hecho, lo copia y pega... Pero escribe siempre su propio resumen, en sus propias palabras, para retener mejor lo que leyó.

Su principal enfoque al momento de hacer un abstract:

• Cuáles son la conclusiones principales.
• Cuáles son las premisas principales que apoyan la conclusión.
• Cómo el argumento se "intersect" con las preguntas de investigación.

Sobre el tercer punto, es importante que abajo del abstract siempre pone un code-block (o cualquier cosa que sea visual funciona, que cambie el color y llame la atención), donde escribe cómo se relaciona el abstract con su investigación.

Jason dice que seguramente olvidará el artículo en pocas horas... Pero que el code-block le ayuda a recordar por qué ese artículo es importante para él y para su investigación.

Utiliza también la opción de "embedding" para poner los abstracts + code-blocks de cada capítulo en su nota principal del libro.

¿Cómo, en fin, Jason hace sus abstracts? Pregunta Anthony.

Jason intenta escribirlo sin mirar a las notas, solo de la memoria, para resumir todo lo que acaba de leer en un solo párrafo. Y después de escribir, revisa, sección por sección, para ver si se olvidó de algo o si escribió mal algo. Edita su abstract, elimina, añade y modifica cosas...

Author Response

The following is the authors’ response to the original reviews.

We thank the reviewers for recognizing the importance of our work and for their insightful suggestions. A point-by-point response to their comments is listed underneath each reviewer’s section.

Reviewer #1 (Recommendations For The Authors):

Major comments

1) Have the authors optimized the expression level of dCas9? I cannot find this information in this paper or in their 2021 paper. It is important to avoid the toxicity phenomenon that occurs when using guide RNAs that share specific five base seed sequences (referred to as 'bad seeds').

Cui L., Vigouroux A., Rousset F., Varet H., Khanna V., Bikard D. A CRISPRi screen in E. coli reveals sequence-specific toxicity of dCas9. Nat. Commun. 2018; 9:1912.

Rostain W., Grebert T., Vyhovskyi D., Thiel Pizarro P., Tshinsele-Van Bellingen G., Cui1 L., Bikard D. Cas9 off-target binding to the promoter of bacterial genes leads to silencing and toxicity. Nucleic Acids Research, 2023, gkad170.

2) One guide per gene is highly unusual given that different guides block the RNA polymerase with different efficiency. This was even shown by the Machner lab in the Legionella context in Figure 1c of Ellis et al. 2021 for sidM and vipD. Typically, genes need three guides minimum to ensure that the gene of interest is knocked down fully unless it is not possible as the gene is too small and/or it is difficult to find an NGG sequence. The authors have used one guide per effector, how can they be sure that each gene is knocked down? The Machner lab themselves in Figure 3c of Ellis et al. 2021 shows not all genes targeted using multiplex CRISPRi are equally efficiently knocked down. Please justify why only one guide per gene was chosen and add controls to validate the results. The authors themselves state that the strategy of one guide may be problematic. Lines 315-316 it reads... A possible explanation was the incomplete knockdown of a seemingly important process.

3) Given what the Machner lab observed about spacer location in Ellis et al. 2021 would it not make more sense to take one set of redundant effectors and make multiplex randomized CRISPRi with them in different locations? For Figure 1 at least.

4) Following infection, it seems that the bacteria were not plated onto antibiotic media, so it is not known how well the plasmid harboring guides is kept through infection.

Specific comments

A) The first results paragraph describes the set-up of 10-plex synthesized CRISPR arrays, where 10 effector encoding genes of specific gene families are selected. The rationale of the choice of these genes is not given. Please explain.

B) Please also add some biological data on what these genes code for, and what is their known or predicted function. It is not very informative and exciting to have tables of lpg numbers without any knowledge of what these genes code for and why they were selected, at least some.

C) Figure 1 A Why are only some of the MC arrays shown? Please, at least include in supplementary the others. Again one array in detail would be more informative, showing true knockdown of all genes by qPCR and ideally by western blot.

D) I am not convinced that the gene silencing efficiency qPCR comparison is done in the correct way. In my opinion, each of the genes to be knocked down should be tested against the expression of a control gene e.g. rpoS and then these results should be compared and not the results of empty plasmid or CRISPR array containing plasmid directly. L. pneumophila are very sensitive to growth conditions and inoculum, thus the two strains might not be completely at the same growth stage when being compared which can impact the results.

E) Figure 1 B As stated in general comment number 4, the authors do not appear to plate onto antibiotic so we don't know how well the plasmid harboring the guides is kept through infection. The sustained presence of the guide is particularly important for CRISPRi.

F) The authors found only a few growth phenotypes and mainly this was due to single genes and not combinations of genes. This might again be due to the fact that only one guide per gene was used. How do the authors know that all genes targeted were indeed knocked down?

G) Line 119 Alternatively, the genes were not 100% all knocked down, escaping the knockdown effect expected. Could authors take three genes with three guides each and look at impact instead of only one?

H) The authors then develop the randomized multiplexed arrays and chose to test 44 TME encoding genes. Line 141 Justify why these effectors were chosen in the text.

I) Unfortunately, the method is not clearly described, and many parts are complicated and the text needs to be re-read several times to be understood (lines 150 - 166). Please re-write to better explain to the reader. In line 156 the authors point to a supplementary note 1. This information should be in the main text.

J) What is the copy number of the CRISPR plasmid? Please add in the Material and Method section also the origin of this plasmid.

Figure 2

K) In the paper (line 154-160) and the extra notes, it states that authors attempt to size select CRISPR arrays. However, this is not apparent in Figure 2 schematic. Or are the authors stating that plasmids only containing one guide were selected out? However, line 312 would suggest not. Please clarify

L) A limiting factor in making multiplex guide CRISPR, as the authors are trying to establish in this study, is cloning of multiple guides. In the pre-determined CRISPR arrays in this study, the guides were synthesized. For the randomized multiplex CRISPR in this study, the authors adapt a Golden Gate cloning method to generate multiple sgRNAs in the Cas9 vector. A similar protocol was established in the below paper. Please add this reference.

Zuckermann, M.; Hlevnjak, M.; Yazdanparast, H.; Zapatka, M.; Jones, D.T.W.; Lichter, P.; Gronych, J. A novel cloning strategy for one-step assembly of multiplex CRISPR vectors. Sci. Rep. 2018

M) As the authors note, Zuckermann et al. similarly note that plex of 3 or 4 is most common and above 5 is rare. This thus appears to still be the limiting step of multiplex CRISPR technology. Please discuss

Figure 4

N) The idea of multiplexed CRISPRi seq to address the biological phenomenon of redundancy is an interesting one, however, I am missing the in-depth functional characterization and discussion of at least one of the redundant functions discovered. Please add.

Figure5/6

O) As noted above, the strength of the experiments is undermined by how CRISPRi is set up. Having an average multiplex of 2 or three and again only using one guide per gene weakens the study and the results obtained. Furthermore, as stated in general comment number 4, the authors do not appear to plate onto antibiotic so again, we don't know how well the plasmid harboring the guides is kept through infection. The sustained presence of the guide is particularly important for CRISPRi. Please add a validation that the guides are all present.

Response to Reviewer #1

We are grateful to the reviewers for their insightful comments and suggestions on how to further improve the manuscript.

Regarding the issue of ‘bad seed sequences’ (comment #1), we had previously evaluated the expression level of dcas9 (plotted in Figure 1b of the 2021 Communications Biol paper) and tuned our induction conditions accordingly (40 ng/mL as described in the Methods). Since all strains used in this study express dcas9 from the chromosome, not a plasmid, this eliminates the possibility of fluctuations in expression levels due to variabilities in plasmid copy numbers.

In the rare event that toxicity by any given guide occurs, we would expect that guide to already be underrepresented or missing in the input pool following 24+ hours of CRISPRi induction during axenic growth. Our data, now discussed in the manuscript (Lines 211-216 and Figure S2), revealed that this was not the case and that all guide-encoding spacers were present in roughly equal amounts (median of >5000 occurrences). As with any knockdown study, the creation of true chromosome deletions was performed throughout as to alleviate the issue of false positives.

Regarding comments #2, #3, and specific comments made under point F, G, and O, on the topic of using single vs. multiple guides, we agree that there are circumstances under which using more than one guide per target may be advantageous, for example when attempting to delete a gene from mammalian cells using conventional CRISPR engineering. In the study described here, this is not the case. In fact, we did create a second array library with alternative guides targeting the same group of genes at locations other than the “optimal location” identified in our 2021 paper and found that these “sub-optimal” guides were inefficient for identifying critical effectors as described in Supplemental Note S1 under the heading “Sub-optimal annealing sites” (Lines 919+). These data suggest that adding sub-optimal guides to the arrays of optimal guides might ‘poison’ the arrays and limit rather than enhance their ability to identify gene combinations.

Regarding comment #2, #3, and specific comments made under point C, F, and G, on the topic of confirming efficient gene knockdown for the identification of critical genes, we remind Reviewer 1 that we did confirm knockdown of 60 of the target genes of the 10-plex screen to be at least 2-fold, with an average fold repression of one order of magnitude or more (Figure 1A). While knockdown of every gene in every 10-plex construct would be an unprecedented ask of any published CRISPR screen, we believe that these 60 genes provide a large enough sampling of all guides to elucidate the range of knockdown to be expected by our CRISPRi platform. As with other knockdown technologies, such as RNAi, there is no expectation of accomplishing complete knockdown for any given target. Hence, the data in Figure 1A suggest that the lack of identifying critical genes using pre-determined 10-plex arrays was not due to a lack of knockdown efficiency, but rather the difficulty to accurately predict redundancy within a cohort of uncharacterized genes, accentuating the need for array randomization with MuRCiS.

On the topic of antibiotic use for plasmid selection (comments #4, E and O), we would like to clarify that the CRISPR plasmids were selected by thymidine prototrophy, not antibiotic resistance, and we apologize for not making this clearer. The laboratory strain Lp02 is a thymidine auxotroph (thyA-) L. pneumophila variant, and plasmid retention is routinely achieved by including the thymidine biosynthesis gene (thyA) on the plasmid backbone. Only with a plasmid bearing the thyA gene can L. pneumophila grow on CYE (thymidine-) plates. Our use of vectors bearing thyA and plating on CYE plates is described in the Methods section. Further, in Figure 7 of our 2021 paper, we show that CRISPR plasmids are efficiently retained by Lp02 for the duration of a 48-hour infection, resulting in efficient multi-gene knockdown even at the end of the intracellular growth experiment.

Regarding comments A and B, on publishing the biological data used to classify genes in gene families for 10-plex silencing, we do not consider it critical to provide additional information beyond the broad classification (e.g. kinases, phosphatases, etc) described in Table S1. Structural predictions constantly change due to continuously evolving databases. Our initial analyses were made in 2015 using HHPRED Hidden-Markov models and, in all likelihood, those predictions have been refined since then. Moreover, with the recent advent of Alphafold, anyone interested in learning more about select effectors from our list is advised to simply access the most recent functional predictions directly on the Alphafold webpage (https://alphafold.ebi.ac.uk/). We clarify how predictions were made on Lines 97-101.

Regarding specific comment D, on our method for qPCR normalization and comparison, we point Reviewer 1 to the Methods section (Lines 460+) where we describe that data obtained from each CRISPRi strain were in fact normalized to the levels of rpsL prior to comparing them to the normalized data from the strain with the empty control plasmid. This normalization to rpsL, a gene encoding a ribosomal protein, also corrects for growth differences between samples.

Regarding specific comment H, the justification for studying 44 transmembrane effector-encoding genes was driven by the fact that activities mediated by transmembrane proteins are difficult (though not impossible) to be replaced by cytosolic proteins, for example the transport of metabolites across the LCV membrane. And since transmembrane regions can be predicted with high confidence, we decided to probe this group of TMEs for synthetic lethality with the randomized CRISPRi approach as proof-of-concept. To make this clearer, we have added more detail to the text (Lines 151-155).

Regarding specific comment I, we have further simplified the description of the cloning technique to increase clarity (Lines 156+). The information listed under Supplemental Note S1, though informative, is not critical for the overall understanding of this highly technical section, and since the reviewer already considered this section to be difficult to follow, we would prefer to not further complicate the text by including these non-essential details.

Regarding the origin of the CRISPRi plasmid (specific comment J), we point Reviewer 1 to the reference (Hammer BK and Swanson MS (Mol Microbiol 1999)) listed in Table S10: Strains and Plasmids Used in this Study.

Regarding specific comment K and O, on the clarity of depicting the CRISPR array size selection process, we have updated the Figure 2 schematic. Reviewer 1 is correct in that despite our best efforts to exclude short CRISPR arrays, inevitably some 1-plex arrays remained in our input vector pool. Still, the average length of all arrays used in our pilot study exceeded three crRNA-encoding spacers. Further, having a population of 1- or 2-plex arrays in our libraries did allow us to pin-point the most critical effectors of a larger arrays within the same MuRCiS experiment (Table S5 and Table S7), a strength of MuRCiS as described in the discussion (Lines 378+).

Regarding specific comment L, we appreciate Reviewer 1’s suggestion of an additional reference and we have added it to the manuscript as reference #23 (Line 71). While this reference does use a Golden Gate strategy to build a multiplex array, that array was not randomized but had a predefined order. Hence, our assembly method is unique due to its randomization.

Regarding specific comment M, on array length cloning limitations, we agree with the conclusion of Zuckermann in Figure 1d of their article that longer inserts are generally harder to get into vector backbones. The challenge of cloning longer inserts is a common phenomenon of general biology and is not unique to cloning CRISPR arrays. We have altered the wording in our manuscript to better describe the intrinsic competition between short and long inserts during cloning (Lines 162-164).

Regarding specific comment N, we second Reviewer 1’s desire to learn more about the critical effector pairs discovered here. With that said, the goal of this manuscript is to report the development of a novel MuRCiS pipeline to identify these critical pairs. Biochemical and molecular investigations of the encoded protein pairs are on-going and will be the topic of a future manuscript.

Reviewer #2 (Recommendations For The Authors):

Specific points

1) The effector repertoire of L. pneumophila seems to have evolved in response to the plethora of potential protozoan hosts (PMID: 31988381). To further assess evolutionary aspects of the vast L. pneumophila effector arsenal, it would be interesting to test the single and double effector mutant strains (Fig. 5FG, Fig. 6EF) for growth in protozoa other than A. castellanii.

2) Most CRISPR arrays comprising genes encoding functionally similar proteins or encoding evolutionarily conserved proteins did not substantially affect intracellular growth of L. pneumophila (Fig. 1B). This rather surprising result should be further discussed.

3) l. 118/119: "Similar results ..., where none of the MC arrays ..." This statement should be phrased more precisely, since some CRISPR arrays did indeed have an effect on intracellular growth of L. pneumophila in U937 macrophages, while none affected intracellular growth in A. castellanii (Fig. 1B).

4) Typos:

• l. 852: ... (arbitrarily set to -100).

• l. 862: ... Legionella-containing vacuole (LCV).

• l. 895: ... and so we would recommend ...

Regarding point 1, we thank Reviewer 2 for the suggestion of testing effector mutants in different hosts. While the primary purpose of the current manuscript was to optimize the MuRCiS platform, future studies using this technology to investigate specific biological questions related to Legionella infection would certainly benefit from including more than one amoebaean species.

Regarding point 2, we agree that the lack of substantial growth defects seems surprising. Yet only two of the seven core effectors (found in all Legionella sp.), lpg2300 and mavN, individually attenuated Legionella intracellular growth when deleted (Burstein 2016 Nat Genetics; Isaac et al., 2015 PNAS). Thus, we hypothesize that the functions many effectors fulfil are of such importance for intracellular survival that that redundancy reaches beyond the boundary of conservation or like-function. We have added a statement emphasizing this at the end of the Figure 1 results section (Line 122-125).

Regarding points 3 and 4, we thank Reviewer 2 for catching these errors and have corrected where needed in the text.

-l. 852 (now Line 874): … (arbitrarily set to -100,000) is correct for Figure 6E.

J'ai scrap la fonction pour trouver le nom, dans un fichier il n'y aura qu'un seul code, donc je regarde celui de la première ligne et avec le dictionnaire j'ai directement son nom (pas besoin de mapping de dictionnaire).

So, at minimum, we won't get any syntax errors. But the semantics of the constructs we use means that it's a valid expectation that the browser itself can execute this code itself—even though it is not strictly JS—because the expected semantics here conveniently overlap with some of JS's semantics.

Emphasize that this corresponds to the main familiar from other programming systems—that triple scripts doesn't just permit arbitrary use of IIFEs at the top level, so long as you write them that way. This is in fact the correct way to denote the program entry point; it's special syntax.

Preface this with "In the world of triple scripts"?

Also, we can link to the wiki article for shunting blocks.

Note that by starting with LineChecker.prototype.getStats before later moving on to LineChecker.analyze, we're not actually practicing top-down programming here...

machine language is binary code. Assembly language is slightly different way to show low-level languages.

Reviewer #2 (Public Review):

In this study, researchers aim to understand the computational principles behind attention allocation in goal-directed reading tasks. They explore how deep neural networks (DNNs) optimized for reading tasks can predict reading time and attention distribution. The findings show that attention weights in transformer-based DNNs predict reading time for each word. Eye tracking reveals that readers focus on basic text features and question-relevant information during initial reading and rereading, respectively. Attention weights in shallow and deep DNN layers are separately influenced by text features and question relevance. Additionally, when readers read without a specific question in mind, DNNs optimized for word prediction tasks can predict their reading time. Based on these findings, the authors suggests that attention in real-world reading can be understood as a result of task optimization.

Strengths of the Methods and Results:<br /> The present study employed stimuli consisting of paragraphs read by middle and high school students, covering a wide range of diverse topics. This choice ensured that the reading experience for participants remained natural, ultimately enhancing the ecological validity of the findings and conclusions.

In Experiments 1-3, participants were instructed to read questions before the text, while in Experiment 4 participants were instructed to read questions after the text. This deliberate manipulation allowed the paper to assess how different reading task conditions influence reading and eye movements.

Weaknesses of the Methods and Results:

While the study benefits from several strengths, it is important to acknowledge its limitations. Notably, recent months have seen significant advancements in Deep Neural Network (DNN) models, including the development of models such as GPT-3.5 and GPT-4, which have demonstrated remarkable capabilities in tasks resembling human cognition, like Theory of Mind. However, as the code for these cutting-edge models was not publicly accessible, they were unable to evaluate whether the attention mechanisms in the most up-to-date DNN models could provide improved predictions for human eye-movement data. This constraint represents a limitation in the investigation.

The methods and data presented in this study are valuable for gaining insights into the psychological mechanisms of reading. Moreover, the data provided in this paper may prove instrumental in enhancing the performance of future DNN models.

Reviewer #2 (Public Review):

Summary:<br /> In this manuscript, "KinCytE- a Kinase to Cytokine Explorer to Identify Molecular Regulators and Potential Therapeutic", the authors present a web resource, KinCytE, that lets researchers search for kinase inhibitors that have been shown to affect cytokine and chemokine release and signaling networks. I think it's a valuable resource that has a lot of potential and could be very useful in deciding on statistical analysis that might precede lab experiments.

Opportunities:<br /> With the release of the manuscript and the code base in place, I hope the authors continue to build upon the platform, perhaps by increasing the number of cell types that are probed (beyond macrophages). Additionally, when new drug-response data becomes available, perhaps it can be used to further validate the findings. Overall, I see this as a great project that can evolve.

Strengths:<br /> The site contains valuable content, and the structure is such that growing that content should be possible.

Weaknesses:<br /> Only based on macrophage experiments, would be nice to have other cell types investigated, but I'm sure that will be remedied with some time.

I believe it is relatively common for hacked Minecraft clients (the market is big for "enhancing" gameplay) to also include token loggers hidden away in their code, making login information accessible for the hacked client developers. I think one of the main things that helps gamers interested in installing a safe cracked client is auditors who have time to review everything. Another thing that would probably help is trusting open-source software, since other developers could provide input easily.

Social engineering is the most powerful way to steal other people's information, scuh as sending fake url or link to people's email to get their ip info. Or making fake QR code. Also can get other people's info from mailing address or shipping address.

The corollary: you can do that (make it once and never touch it again) if you are using the "native substrate" of the WHATWG/W3C Web platform. Breaking changes in "JavaScript" or "browsers" are rarely actually that. They're project/organizational failures one layer up—someone (who doesn't control users' Web browsers and how they work) decided to stop maintaining something or published a new revision but didn't commit to doing it in a backwards compatible way (and someone decided to build upon that, anyway).

Real time

SD card slow

Frequency domain, 0.5 phase 180 degree shift.

Only code check, not for use.

SD card save to memory and fetch data and channel output to speaker after multiplying with reverse function.

Valdez points out the arbitrary nature of ethnic classification systems.

Better yet, just don't write code like this to begin with.

... so don't rely on it.

If the methods are helpful then reimplement them (as a library, even) and use that in your code. When passing data to code that you don't control, use the underlying ArrayBuffer instance.

The very mention of polyfilling here represents a fundamental misapprehension about how to structure a codebase and decide which abstractions to rely on and which ones not to...

cf https://en.wikipedia.org/wiki/Occupational_psychosis

I'm struggling to understand how this works - primarily how the number is generated. I understand that it's using the word "love" and "hate" but what determines how powerful they are and... quite frankly no one talks like they do in the examples. It'll probably make more sense reading further but it does seem kind of silly to me.

It's a bit the Matthew Kirschenbaum argument.... and the ongoing discussion whether one needs to be / can be expert in everything or can share/pool responsibility and expertises

Reviewer #1 (Public Review):

Summary<br /> This work contains 3 sections. The first section describes how protein domains with SQ motifs can increase the abundance of a lacZ reporter in yeast. The authors call this phenomenon autonomous protein expression-enhancing activity, and this finding is well supported. The authors show evidence that this increase in protein abundance and enzymatic activity is not due to changes in plasmid copy number or mRNA abundance, and that this phenomenon is not affected by mutants in translational quality control. It was not completely clear whether the increased protein abundance is due to increased translation or to increased protein stability.

In section 2, the authors performed mutagenesis of three N-terminal domains to study how protein sequence changes protein stability and enzymatic activity of the fusions. These data are very interesting, but this section needs more interpretation. It is not clear if the effect is due to the number of S/T/Q/N amino acids or due to the number of phosphorylation sites.

In section 3, the authors undertake an extensive computational analysis of amino acid runs in 27 species. Many aspects of this section are fascinating to an expert reader. They identify regions with poly-X tracks. These data were not normalized correctly: I think that a null expectation for how often poly-X track occur should be built for each species based on the underlying prevalence of amino acids in that species. As a result, I believe that the claim is not well supported by the data.

Strengths<br /> This work is about an interesting topic and contains stimulating bioinformatics analysis. The first two sections, where the authors investigate how S/T/Q/N abundance modulates protein expression level, is well supported by the data. The bioinformatics analysis of Q abundance in ciliate proteomes is fascinating. There are some ciliates that have repurposed stop codons to code for Q. The authors find that in these proteomes, Q-runs are greatly expanded. They offer interesting speculations on how this expansion might impact protein function.

Weakness<br /> At this time, the manuscript is disorganized and difficult to read. An expert in the field, who will not be distracted by the disorganization, will find some very interesting results included. In particular, the order of the introduction does not match the rest of the paper.

In the first and second sections, where the authors investigate how S/T/Q/N abundance modulates protein expression levels, it is unclear if the effect is due to the number of phosphorylation sites or the number of S/T/Q/N residues. The authors also do not discuss if the N-end rule for protein stability applies to the lacZ reporter or the fusion proteins.

The most interesting part of the paper is an exploration of S/T/Q/N-rich regions and other repetitive AA runs in 27 proteomes, particularly ciliates. However, this analysis is missing a critical control that makes it nearly impossible to evaluate the importance of the findings. The authors find the abundance of different amino acid runs in various proteomes. They also report the background abundance of each amino acid. They do not use this background abundance to normalize the runs of amino acids to create a null expectation from each proteome. For example, it has been clear for some time (Ruff, 2017; Ruff et al., 2016) that Drosophila contains a very high background of Q's in the proteome and it is necessary to control for this background abundance when finding runs of Q's. The authors could easily address this problem with the data and analysis they have already collected. However, at this time, without this normalization, I am hesitant to trust the lists of proteins with long runs of amino acid and the ensuing GO enrichment analysis.

Ruff KM. 2017. Washington University in St.<br /> Ruff KM, Holehouse AS, Richardson MGO, Pappu RV. 2016. Proteomic and Biophysical Analysis of Polar Tracts. Biophys J 110:556a.

I wonder how different they are

This is soo on brand for a Jewish ancient text

The MIshnah is oral torah

This shows the strength of Jewish people

TRÈS IMPORTANT

这让我想起来，小的时候大家遇到难题一起讨论的样子

En un mundo y nuestras realidades cada vez más atravesadas por la tecnología, este tipo de propuestas son necesarias para crear nuevas formas de comunicación y codificación del conocimiento.

Oui, et les nouveaux monopoles: Chrome est en train de distancier Firefox sur l'implémentation de fonctionnalité CSS indispensable au print (oprhans/widows, hyphenate-limit-chars, ect.) Ce qui fait quœn se retrouve à devoir utiliser Chrome. On a le même problème: le navigateur web est un logiciel sur lequel on ne peut pas intervenir parce que le code est trop complexe et nous n'avons pas les compétences pour le modifier

haha cette phrase me fait un peu rire quand on sait à quel point on galère en coulisse. Je trouve qu'il manque aussi une question par rapport à tout ce que tu as développé plus haut sur les pratiques des designers graphiques utlisant le code: est-ce que ces nouvelles pratiques peuvent être soutenable s'il y a pas une communauté qui se forme et partage un outil commun ? à vouloir toujours être un pas de côté et rejeter des choses qui sont construites par une plus grande communauté on s'épuise. Sans parler à leur place mais au détour de conversations avec elles: Marianne est partie de Luuse pour cette raison, Stéphanie est partie d'OSP aussi parce que c'est épuisant de maintenir un outil pour sa propre pratique sans qu'il y ait plus de contributeur. La question de l'institutionnalisation est différente de la question du nombre de personne qui utilise et participe au développement de l'outil (la communauté). Pour moi, il n'y a pas forcément institutionnalisation parce que l'outil est utilisé par un plus grand nombre de personnes. Le fait qu'une plus grande utilisation est vue négativement par certain.es me rend vraiment triste (on m'a déjà dit: 'j'utilise pas Paged.js parce que c'est déjà maintream"). Pourquoi ne pas vouloir participer à la construction d'une communauté qui développe un outil ensemble ? Il me semble que ça rend les pratiques plus fortes. Vouloir être radical à tout prix et rejeter toute forme d''organisation' (plutôt qu'insitutionnalisation), est-ce une bonne chose en soi ? est-ce vraiment un positionnement intéressant quand on se revendique du mouvement de la culture du libre ? Si tu poses la question de l'instiutionnalisation et d'uniformisation, il me semble que tu dois aussi poser la question inverse (que je n'arrive pas forcément bien à formuler). Nous avons aussi besoin de productions techniques partagées et de standard pour travailler ensemble. Comment les designers graphiques peuvent ils défendre une forme de collaboration et de communauté de pratiques s'ils rejettent une standardisation (à ne pas confondre avec uniformisation). Il y a bien des bonnes pratiques dans le code, la façon de structurer ses sites pour pouvoir collaborer à plus grande échelle. C'est aussi ça la culture de la programmation dont se revendique les graphistes.

PS: je fais mes commentaires au fur et à mesure donc désolé si tu reviens sur ça ensuite. Et mon commentaire est long parce que ça me tient à cœur ^^ Et en fait ça m'attriste un peu que tu cites Paged.js pour la première fois et aue tu mettes directement mettre une phrase un peu négative derrière (même si la question est légitime)

💛

des scripts et des programmes dans un certains langages : javascript, Python, mais on ne code pas en C++ ou go

Hmm, wonder if supplemental info has this code.

i love this game because it is fun and it helps me to code

Explanation on how Wasm runs in browsers

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

Reply to the reviewers

We would like to thank all reviewers for their careful evaluation of our manuscript and their thoughtful feedback, which we could use to improve its quality significantly.

Reviewer #1 (Evidence, reproducibility and clarity):

Summary: This study addresses the problem of what is the optimal ribosome composition in terms of relative RNA and protein content, to ensure optimal growth rate and minimal energy waste. The RNA-world hypothesis suggests that primitive ribosomes were RNA-only objects, and in fact this would appear to be very advantageous from an energetic point of view, since RNA synthesis requires a much lower energy expenditure than protein synthesis. Yet a large fraction of present-day ribosome mass is protein, ranging from 30% to nearly 70% depending on the organism. The authors hypothesize that one of the main functions of ribosomal proteins is to stabilize the RNA and to protect it against degradation. According to their idea, the fast degradation of a protein-free rRNA would offset the energetic advantage given by its cheaper synthesis. To test the hypothesis, they developed a mathematical model whereby to evaluate the optimal ribosome composition under a number of different conditions.

Major comments: The paper is well-written and very readable. I am not an expert of mathematical modelling, so I cannot go into the details of the model presented. As a biologist, I can say that the conclusion arrived at are reasonable and well-justified.

We thank the reviewer for the positive evaluation.

Perhaps the point of view is rather narrow, since ribosomal proteins are known to be important not only for RNA protection and ribosome stability, but also to ensure the accuracy of decoding and, in certain contexts, to allow the ribosomes to interact with other cellular ligands. The authors make only very slight reference to these questions, so it would be worthwhile to further comment on them.

Thank you for your suggestion. To address it, we expanded the discussion as follows:<br /> "Finally, we need to consider that ribosomal proteins may play other roles in the cells, especially in eukaryotic organisms. Ribosomal proteins participate in translation processes, for example, binding of translation factors, release of tRNA, and translocation. They may also affect the fidelity of translation (Nikolay et al., 2015). Furthermore, they play roles in various cellular processes such as cell proliferation, apoptosis, DNA repair, cell migration and others (Kisly and Tamm, 2023). These additional functions might have conferred evolutionary fitness advantages. Nevertheless, the primary role of ribosomal proteins seems to be stabilization and folding of rRNA (Nikolay et al., 2015; Kisly and Tamm, 2023)."

Furthermore, their explanation of why ribosome composition should be so different in different organisms (e.g. protein-poor bacterial ribosomes versus protein-rich archaeal ones) is not entirely convincing. For instance, they suggest that archaea may have protein-richer ribosomes than bacteria because they live in extreme environments, thus needing a further aid to stabilize the organelle. While this may be a factor, one must point out that non-extremophilic archaea (e.g. methanogens) have protein-rich ribosomes, making it obvious that other factors must be at play.<br />

We appreciate the reviewer's feedback. Ribosome composition is indeed complex and influenced by various factors. While extreme environments (may) contribute to protein-rich ribosomes in archaea, it's important to note that not all archaea share this characteristic. Some, like Halobacteriales, Methanomicrobiales, and Methanobacteriales, have ribosomes with protein content similar to bacteria.

Furthermore, there are species in both archaea and bacteria with low protein content in their ribosomes despite extreme habitats. This suggests that alternative strategies, possibly involving specific sequence variants in the rRNA (Nissley et al., 2023), play a role in stabilizing ribosomes. In our model, these findings would correspond to a decreased kdegmax. However, these sequence variants are not universal.

Amils et al. (1993) suggest that protein-rich ribosomes in archaea are (more) ancient and proteins may have been lost in some species, possibly to favor higher growth rates (and in agreement with our theoretical analysis). An intriguing avenue for further research would be a phylogenetic analysis of archaeal evolution to investigate the emergence of different ribosome compositions.

To address your concerns, we added the following paragraph to the discussion:<br /> "Additionally, some extremophilic organisms, such as the bacteria Chloroflexus aurantiacus or Fervidobacterium islandicum, exhibit ribosomes with lower protein content (approximately 40%) compared to extremophilic archaea (50%). It has been suggested that protein-rich ribosomes can be traced back to the oldest phylogenetic lineages, with some ribosomal proteins being lost over time (Amils et al., 1993; Acca et al., 1993). Organisms with lower protein content in their ribosomes may have evolved alternative strategies to thrive in extreme conditions. Examples of such strategies include the presence of specific rRNA sequence variants or base modifications, as recently discussed by Nissley et al. (2023).

Moreover, certain archaeal species, such as those from Methanobacteriales or Halobacteriales, have transitioned to milder environmental conditions and subsequently shed unnecessary ribosomal proteins (Acca et al., 1993; Amils et al., 1993).

To gain a comprehensive understanding of ribosome evolution in response to changing conditions, a thorough phylogenetic analysis is warranted. This analysis should be complemented by measurements of growth rate, translation rate, RNA degradation rate, among other parameters, to delineate the order of protein loss or gain, and the emergence of sequence variations and base modifications."

Minor comments: none in particular. Referencing is adequate, text is clear and the figures are clear and well-organized.

Thank you.

Reviewer #1 (Significance):

As I stated above, the main weakness of this study may be that it concentrates overwhelmingly on a single problem, i.e. the energetic cost of adding proteins to an RNA-only ancestral ribosome. On the other hand, this is a question seldom addressed when talking about ribosome composition, which indeed makes this paper valuable and interesting. The authors expand and advance a previous study of the same kind (to which they make ample reference).

Although rather specialized, I think this paper, in its general conclusions, may be of interest to most of those working in the field of protein synthesis and ribosome evolution.

Referee's keywords: archaea, ribosome evolution, translation, translation initiation

Reviewer #2 (Evidence, reproducibility and clarity):

The authors explore a mathematical model to rationalize the variable RNA content in ribosomes across species. The mathematical model particularly considers the idea that the protein-to-RNA ratio in ribosomes emerges as a consequence of faster rRNA than r-protein synthesis coupled with a faster degradation of rRNA. This is an interesting analysis. The idea is well explained and the math of the model is overall well explained. Overall, I thus support publication of this analysis.

We thank the reviewer for the positive evaluation.

However, while reading the manuscript I was continuously wondering about two major aspects which, I suggest, should be considered more prominently in the text:

1. How clear is it that rRNA is more unstable than r-protein?
2. Why should the translation rate (the speed with which ribosomes assemble new proteins) not be highly dependent on the ribosome-to-protein ratio (with some intermediate ratio ensuring efficient synthesis and efficient translation?

Currently these points are considered briefly in the discussion part. I suggest that these points should at least be discussed more prominently in the introduction. I further appreciate any more detailed thoughts the authors have on these questions.

Finally, I think the discussion section would benefit strongly from a more detailed consideration of possible future experiments. Which data is needed to probe the idea? What types of experiments could be performed to probe the model.

We added a paragraph to the discussion with suggestions for experiments:<br /> "There are still many open questions about ribosome biogenesis and evolution. Our model could guide future experiments. There are a few studies that assessed the effect of individual rP deletions in E. coli, for example mutation in S10 increased RNA degradation (Kuwano et al., 1977), and mutation in L6 lead to disrupted ribosomal assembly (Shigeno et al., 2016). A systematic knock-out screen of all ribosomal proteins could be done (as in Shoji et al. (2011)), complemented with quantification of RNA degradation and misfolding.

In case of extremophilic organisms with protein-rich ribosomes, temperature sensitivity could also be assessed. We would expect that deletion of the extra proteins would cause growth defects only at high temperatures.

Furthermore, after removal of proteins from archaeal protein-rich ribosomes, laboratory evolution could be performed to see whether growth rate increases beyond wild-type.

Comprehensive datasets, akin to the work of Bremer and Dennis in 2008 for E. coli, should be generated for non-standard organisms by measuring various parameters such as transcription and translation rates, ribosome and RNAP activities, and other relevant factors.

Finally, as mentioned earlier, phylogenetic analysis or ribosome evolution across different species and environments could be done."

More detailed comments:

Regarding i: rRNA is pretty stable compared to other RNA types in the cell. The authors argue it is unstable. The specific question then seems to become how stable rRNA is compared to r-protein? Generally, proteins are also stable, but what data is available to support that r-proteins are more stable than rRNA?

While rRNA that is already integrated into a ribosome is stable, nascent RNA may be susceptible to degradation (Jain, 2018). It has been observed that even during exponential growth, some rRNA is degraded (Gausing, 1997; Jain 2018) and the degradation rate increases if ribosome assembly is delayed (Jain, 2018). This suggests that rRNA that is synthesized in excess cannot be stored and used later. Furthermore, when rRNA is overexpressed in excess of rPs, it is rapidly degraded (half life 15-70 min) (Siehnel and Morgan, 1985).

On the other hand, the turnover of proteins is negligible (Bremer and Dennis 2008), and most ribosomal proteins can exist in a free form without RNA. For example, under starvation/in stationary phase, rRNA is degraded, but most proteins are stable and can be reused later (Reier et al., 2022; Deutscher, 2003).

The precise mechanisms of the rRNA instability are not clear. The simplest explanation is that rRNA that is not protected by rPs is attacked by RNases. Another option is that rRNA without proteins is difficult to fold and can get trapped in misfolded states. These are then degraded as a part of quality control. The model developed in this paper allows for both of these mechanisms.

We added these references to the discussion:<br /> "In order to explain a mixed (RNA+protein) ribosome, we consider rRNA degradation in our extended model, thereby increasing the costs for RNA synthesis. While rRNA that is already integrated into a ribosome is stable, nascent RNA may be susceptible to degradation (Jain, 2018). Indeed, it has been experimentally observed that even at maximum growth rate, 10% of newly synthesized rRNA is degraded (Gausing, 1977), and the degradation rate increases if ribosome assembly is delayed (Jain, 2018). Furthermore, when rRNA is overexpressed in excess of rPs, it is rapidly degraded (Siehnel and Morgan, 1985). Due to the extremely high rates at which rRNA is synthesized, errors become inevitable, necessitating the action of quality control enzymes such as polynucleotide phosphorylase (PNPase) and RNase R to ensure ribosome integrity (Dos Santos et al., 2018). The absence of the RNases results in the accumulation of rRNA fragments, ultimately leading to cell death (Cheng and Deutscher, 2003; Jain, 2018).

In contrast, protein turnover is negligible (Bremer & Dennis, 2008), and most ribosomal proteins can exist without rRNA and can be reused (Reier et al., 2022; Deutscher, 2003). Therefore, we do not consider protein degradation in our model."

Regarding ii: Building on their model results, the authors rationalize the highly varying RNA-to-protein ratio in ribosomes across species. The model considers a non-varying rate with which ribosomes synthesize new proteins. This is briefly discussed in the discussion section. However, this appears to be a major assumption that, I think, should be stated clearly stated earlier in the text, including the abstract and introduction. Second, I wonder how the authors then rationalize variations in translation rate across species. Translation rates and the speeds with which ribosomes are varying strongly across species (indicated for example well by the change in the slope between ribosome content/rRNA and growth rate - slope in Fig. 2A). Why could the rRNA-to-protein ratio not be important in playing a role here?

We decided not to consider the effect of rRNA/protein ratio in ribosomes on translation rate mainly because it is not clear in what way it affects it. Proteins are better catalysts than rRNA. Yet, eukaryotic ribosomes which have higher protein content, have lower translation rates. For archaea and mitochondria, we were not able to find data but it is unlikely that the translation rates are faster because the growth rates are not faster.

We added a paragraph to the introduction that explains our assumption:<br /> "We focus on the primary role of ribosomal proteins, which is stabilizing rRNA (by preventing its degradation or misfolding).

Ribosome protein content might also affect other parameters, such as translation rate. Proteins are generally better catalysts than RNA (Jeffares et al., 1998), but the ribosome's catalytic core is formed by rRNA (Tirumalai et al., 2021) and operates at a relatively slow catalytic rate compared to typical enzymes. This suggests that there is little evolutionary pressure to increase the catalytic rate. Furthermore, ribosomes with the lowest protein content, like the E. coli ribosome, exhibit the highest translation rates (Bonven and Gulløv, 1979; Hartl and Hayer-Hartl, 2009; Bremer and Dennis, 2008). Therefore, we do not consider the impact on translation rate in this study."

And a sentence to the abstract:<br /> "In this study, we develop a (coarse-grained) mechanistic model of a self-fabricating cell and validate it under various growth conditions. Using resource balance analysis (RBA), we examine how the maximum growth rate varies with ribosome composition, assuming that all kinetic parameters remain independent of ribosome composition."

More minor point, but I was also not sure about the justification that ribosome mass is constant (line 111). The mass of an amino acid and a nucleotide is quite different. Why should overall mass matter, and not for example the number of amino acids and proteins. I think it also would be good here to motivate the assumption better early on instead of commenting on it in the discussion section.

Thank you for your suggestion. We agree with the reviewer that we should make our assumption of keeping the ribosome mass constant, which we used for simplicity, clearer from the beginning. Therefore, we have added the following statement to the introduction:<br /> "For simplicity, we assume a constant ribosome mass."

Reviewer #2 (Significance):

Protein synthesis by ribosomes is a major determinant of the rate with which microbes and other fast growing cells accumulate biomass. To better understand cell growth it is thus essential to better understand the makeup of ribosomes. Széliová et al present a mathematical model to entertain the idea that the varying RNA content in ribosomes across species is a consequence of RNA degradation. The model makes clear predictions which can guide future experiments.

Reviewer #4 (Evidence, reproducibility and clarity):

Summary

In this manuscript, Széliová et al. used a simple self-replicating cell model to study why the ribosome consists of both RNA and protein from an economic point of view. Their base model predicts an RNA-only ribosome, which is not surprising since the smaller RNAP has a higher turnover number compared to the larger ribosome. When rRNA instability is included, the model predicts an "RNA+Protein" ribosome. In particular, the predicted ribosome composition is comparable to the measured ribosome composition when strong cooperative binding of ribosomal proteins to rRNA is considered. The authors conclude that the maximal growth rate is achieved by the real ribosome composition when rRNA instability is taken into account.

Major comments:

1. The authors modeled the rRNA degradation rate as a function of the concentration of fully assembled ribosomes (equation 5). However, only partially assembled ribosomes are susceptible to RNase, and they make up only a small fraction of total ribosomes. The majority of ribosomes are fully assembled. In addition, the turnover number obtained from Fazal et al. (2015) and used here is the degradation rate of double-stranded RNA, not the fully assembled ribosomes, which have a stable tertiary structure. In my opinion, the rRNA degradation rate should be modeled as a function of the concentration of partially assembled ribosomes (i.e., pre-R in Figure 7) rather than the concentration of fully assembled ribosomes.

We agree with the reviewer that the way we model the process is not entirely biologically accurate. The problem is that even if we add the assembly intermediates, their concentration would be zero as they do not catalyze any reaction (similarly to the metabolites). Therefore, the degradation rate would also always be zero. Given the current modeling setup, the obvious proxy for the intracellular rRNA concentration is the rRNA concentration in the (assembled) ribosome, c_R*(1-x_rP).

1. Compared to the work by Kostinski and Reuveni (2020), the authors have made an improvement by avoiding the use of constant ribosome allocation to ribosomal protein (Φ_rP^R) and RNAP (Φ_RNAP^R), allowing these parameters to vary with predicted growth rates (by changing 𝑥_rP). This is indeed important, as bacteria are very likely to adjust these parameters in response to different growth conditions. However, certain other growth rate-dependent parameters are still treated as constants (or treated as nutrient-specific parameters) across predicted growth rates under given conditions. For example, experiments have shown that the fraction of active RNAP (f_RNAP^act) and the ribosome elongation rate (k_R^el) are growth rate-dependent (Bremer and Dennis, 1996). In contrast, when the authors predict the maximum growth rate by changing 𝑥_rP, f_RNAP^act and k_R^el are held constant regardless of the predicted growth rates.

The fraction of active RNAP (f_RNAP^act) was growth-rate dependent in all our simulations (see Table 2), only the fraction of active ribosomes (f_R^act) was kept constant according to Bremer and Dennis, 1996 & 2008.

We decided to keep the elongation rate (k_R^el) constant similar to Scott et al. 2010 (their explanation is in the supplementary material “Correlation [1] and the control of ribosome synthesis”).

We reran the simulations with variable k_R^el. It has no impact on the predictions of optimal ribosome composition. However, the linear dependence of RNA/protein ratio is less steep and predicts an offset at zero growth rate.

We added the results to the supplementary material and the following text to the results section (for the base model):<br /> "…the base model correctly recovers the well-known linear dependence of the RNA to protein ratio and growth rate (Scott et al. 2010), see Figure 2a, but not the offset at zero growth rate, since our model does not contain any non-growth associated processes and we assume constant translation elongation rate kelR as in Scott et al. (2010). At low growth rate, kelR decreases, most likely because of the lower availability of the required substrates (Bremer and Dennis, 2008; Dai et al., 2016). Interestingly, when we use variable kelR, we observe a nonzero offset (Appendix 1, Figure 2)."

and in a later section:<br /> "Using variable or constant kelR has no impact on the predicted optimal ribosome composition. As in the base model, variable kelR leads to predicted non-zero offset of RNA/protein ratio at zero growth rate (Appendix 1, Figure 6)."

1. _If amino acids or nucleotides are provided in the media, the cell does not have to synthesize all of them de novo. However, the model assumes that the cell always synthesizes all amino acids or nucleotides de novo for growth on growth on amino acid-supplemented media or on LB. This problem could in principle be solved by assuming very fast kinetics of the metabolic reactions in these media, but that should be discussed in the manuscript. Furthermore, why does the turnover number for EAA depend on the growth rate while that of ENT is constant?<br /> > _

We focused on the “enzyme” EAA because it forms a significant fraction of the proteome. However, for consistency, we now also made ENT turnover number depend on growth rate. It made no significant impact on the simulation results.

We agree with the reviewer that the model is currently very simplified and the enzymes ENT and EAA are used even in the media supplemented with AAs/NTs. However, these enzymes represent lumped pathways that aim to take into account not only AA/NT synthesis but also the different ‘nutrient efficiencies’ of the carbon sources (as in Scott et al. 2010). Therefore, to approximate these effects we increase the kcat of EAA (and now also ENT) with growth rate.

We added a paragraph to the results section to explain these simplifications:<br /> "We used parameters from E. coli grown in six different media. Three of them are rich media (Gly+AA, Glc+AA, LB) where amino acids (and nucleotides) are provided so cells only have to express the corresponding transporters instead of the synthesis pathways. In our model, the enzymes ENT and EAA represent lumped pathways for glycolysis and nucleotide / amino acid synthesis, and we only consider one type of transporter. Therefore, to model the changing `nutrient quality' of the different media (inspired by Scott et al. 2010), we assume that turnover numbers of EAA and ENT increase with growth rate."

1. All parameters related to transcription (RNAP) and translation (ribosome) used in this manuscript are adopted from Kostinski and Reuveni (2020), which are slightly modified based on Bremer and Dennis' research (1996, 2008). However, the authors changed some of the original parameters or data points, but did not provide explanations for these changes:

(a) The original data depicted a growth rate-dependent translation elongation rate, but Table 2 presents it as a constant value.

Please see the reply to point 2 above.

(b) Figure 2b displays five experimental data points, as opposed to the six data points in the original dataset and other figures in this manuscript.

The values for the transcription rate were taken from Bremer and Dennis’s paper from 1996 which only contains five growth rates. We updated the Figure 2b – it now displays data from Bremer and Dennis 2008 for six growth rates.

(c) The model does not consider the fraction of RNAP transcribing rRNA (Φ_rRNA^RNAP), except in Appendix Figure 4. In the original data (Bremer and Dennis 1996), the fraction of RNAP transcribing rRNA increases dramatically with growth rate; however, in this study, it remains constant at 1.

Our goal was to keep the model as simple as possible and keep the number of required parameters to a minimum. We only included the figure in the supplementary material because it does not change the conclusions, even though it makes the predictions quantitatively better. In the future we would like to achieve this improvement by expanding the model (with mRNA, tRNA, non-specific RNAP binding to DNA etc.). We added a sentence to the discussion to point out again how the results are affected if Φ_rRNA^RNAP is included, and how this parameter could be mechanistically included in the model in the future.

"Furthermore, incorporating other types of RNA (mRNA, tRNA) and energy metabolism, or even constructing a genome-scale RBA model (Hu et al., 2020), will likely lead to more quantitative predictions of fluxes and growth rate. A strong indication of this is that including a variable RNAP allocation into the model leads to quantitatively better predictions (see Appendix 1, Figure 5). Therefore, in the future, we aim to model RNAP allocation mechanistically. This will involve for example adding other RNA species (mRNA, tRNA), and considering non-specifically bound RNAP which is a significant fraction of RNAP (Klumpp and Hwa, 2008)."

Furthermore, Φ_rRNA^RNAP was first introduced in line 205 but was not explained until line 337.

We added an explanation to the sentence in line 205:<br /> "If we consider RNAP allocation to rRNA (k_RNAP^el^bar = k_RNAP^el f_act^RNAP Φ_rRNA^RNAP, where Φ_rRNA^RNAP is the fraction of RNAP allocated to the synthesis of rRNA), the results get closer to the experimental data (Appendix 1, Figure 5)."

The value(s) of Φ_rRNA^RNAP for Appendix Figure 4 are also missing from this manuscript.

We added the missing values to the figure caption.

1. How, exactly, is the unit of flux converted to mmol g-1 h-1?

We are not exactly sure what the reviewer means by this question. As an example of unit conversion, we provide an explanation for the conversion of literature RNAP fluxes. The RNAP fluxes predicted by the model are in mmol g^-1 h^-1. The RNAP fluxes in Bremer and Dennis (2008) were in nt min^-1 cell^-1. To convert them to mmol g^-1 h^-1, we used the values of dry mass/cell from Bremer and Dennis (2008) and the number of nucleotides in rRNA (the stoichiometric coefficient n_rRNA). The code for the conversion is available on GitHub (https://github.com/diana-sz/RiboComp) in the script fluxes_vs_growth_rate.py.

1. What is the (dry) mass constraint and how is it defined? In the manuscript, both the second equation in line 101 and the bottom row of Table 1 are dry mass constraint(s). Why are they different? Furthermore, why is the right-hand side of the second equation in line 101 a dimensionless 1, and how does the last row of Table 1 result in the unit of growth rate, time^(-1)?

These are two forms of the same constraint. We added a paragraph to the methods section that explains how to convert the equations (capacity constraints, dry mass constraint) into the form in Table 1.

In the first form of the equation, ⍵Tc = 1, the units of ⍵ are g/mmol, and the units of c are mmol/g, so they cancel out.

The rows in Table 1 are multiplied by the vector of fluxes, so we get ⍵C [g/mmol] * vIC [mmol/gh] = μ [1/h].

1. The concentrations of all components that serve as "substrates" will be zero when growth rate is maximized, as these molecules do not catalyze any reactions, nor do they influence reaction kinetics in the model. These "0" concentration components are C, AA, NT, rP, and rRNA. Why are these concentrations even included in the model?

The reviewer is correct in pointing out that these species have zero concentrations at maximum growth, and it would be possible to simplify the model accordingly. However, we have chosen not to merge these reactions to maintain clarity in distinguishing between metabolic and macromolecular synthesis processes. Additionally, while we currently use the model to predict optimal behavior, it is not inherently limited to this purpose, as it can equally describe sub-optimal states (as in Figure 2b). Finally, if needed, we can easily introduce minimum concentration constraints (e.g. obtained from measurements) for any of these species without affecting our overall arguments.

Minor comments:

1. Questions regarding Figure 2:

(a) The explanation of Figure 2a is unclear. Intuitively, I assumed that it was a comparison between model predictions and experimental data, with the points representing experimental data and the line representing predictions; and the authors wrote in the figure legend "The points represent maximum growth rates in six experimental conditions". However, the growth rates shown in the figure do not match the original experimental data. Are all the data in the figure predictions?

Yes, the points are predictions and the line is a linear fit. We changed the figure caption as follows:<br /> "The model predicts a linear relationship between RNA to protein ratio and growth rate. The points represent the predicted maximum growth rates in six experimental conditions (Table 2). The line is a linear fit."

(b) Figure 2b is difficult to understand. This figure shows the non-optimal solutions of the model. It is unclear how these solutions are achieved and why there are three lines in the figure.

We expanded the figure caption to make it clearer:<br /> "Alternative RNAP fluxes at different non-optimal growth rates in glucose minimal medium. These are obtained by varying the growth rate step by step from zero to maximum and enumerating all solutions (elementary growth vectors as defined in Müller et al. (2022)) for each growth rate. The grey and blue lines are the alternative solutions. The blue line corresponds to solutions, where rRNA and ribosomes do not accumulate (constraints rRNA' andcap R' in Table 1 are limiting)."

1. Table 1 is also difficult to understand. While the stoichiometric constraints can be easily derived, the capacity constraints and the dry mass constraint cannot be easily derived from related equations from the text.

We added a paragraph into the methods section that explains how to convert the equations (capacity constraints, dry mass constraint) into matrix form.

1. As the authors ask a question in the title, they should provide an explicit answer in the abstract.

We added a sentence to the abstract:<br /> "Our model highlights the importance of RNA instability. If we neglect it, RNA synthesis is always ``cheaper' than protein synthesis, leading to an RNA-only ribosome at maximum growth rate. However, when we account for RNA turnover, we find that a mixed ribosome composed of RNA and proteins maximizes growth rate."

1. The authors should cite a seminal modeling paper, which was the first to examine resource allocation in simplified self-replicating cell systems (Molenaar et al. 2009, Molecular Systems Biology 5:323).

The citation was added.

1. The meaning of v is not consistently defined throughout the manuscript. It refers to the fluxes of enzymatic reactions in some instances, but in other contexts, it refers to the fluxes of the entire network of enzymatic reactions and protein synthesis reactions (Figure 1, Equation 1, and Line 386).

We have made the notation more consistent. When we refer to the fluxes of the entire network we now use v_tot instead of v.

1. Line 85, it might be difficult to interpret "RNAP fluxes" as the flux of rRNA synthesis without reading the subsequent text.

_We added the explanation in brackets.<br /> "_We validate the model by predicting RNAP fluxes (rRNA synthesis fluxes)."

1. Typo in line 102-103. "...protein fluxes 𝒘" → "...protein synthesis fluxes 𝒘".

Thank you for spotting that, we added the missing word.

1. Line 104, f_RNAP^act and f_R^act are not explained in the text; and their biological significance cannot be understood from their names in Table 2 ("RNAP activity" and "Ribosome activity").

We added a sentence that explains these parameters:<br /> "f_RNAP^act is the fraction of actively transcribing RNAPs, and f_R^act is the fraction of actively translating ribosomes."

1. Notion "**" in Table 2. The coupling between transcription and translation means the coupling of "mRNA" transcription and translation, not rRNA. At least in E. coli, the transcription rate of rRNA is faster than that of mRNA.

The transcription rate of the archaeal RNAP was determined in vitro. To our knowledge, data for transcription rates of rRNA vs. mRNA in vivo are not available. Therefore, the translation rate is only a very rough estimate.

1. Is the citation correct in line 136? I didn't find related information in Bremer and Dennis' paper after a quick scan.

We corrected the citation. Additionally, we added references that indicate that if rRNA is transcribed in excess of available r-proteins, it gets rapidly degraded:<br /> "In fact, the accumulation of free rRNA in a cell is biologically not realistic as it is bound by rPs already during transcription (Rodgers and Woodson, 2021). Furthermore, if rRNA is expressed in excess of rPs, it is rapidly degraded (Siehnel and Morgan, 1985)."

1. Lines 136-138. The statement is not accurate, as the fraction of inactive ribosomes increases with decreasing growth rate in E. coli (Dai et al. 2016, Nat Microbiol 2, 16231). If the studied growth rates are relatively high, it is acceptable to use a constant active ribosome fraction as an approximation, but this approximation should be made explicit.

We used the fractions of active ribosomes as reported in Bremer and Dennis, 2008 which are constant between growth rates of 0.4-2.1 1/h. In Dai et al. 2016, it was similarly observed that above the growth rate of ~0.5 1/h, the active fraction is quite constant. We rephrase the text to make it more accurate:<br /> "For the growth rates studied here (0.4-2.1 1/h), the fraction of inactive ribosomes stays roughly constant at 15-20% (Bremer and Dennis, 1996, 2008; Dai et al., 2016). In our model, we have already incorporated this fraction using the effective translation elongation rate (k_R^el^bar = k_R^el*f_R^act). However, below the growth rate of ~0.5 1/h, the fraction of active ribosomes rapidly decreases (Dai et al. 2016)."

1. The citation in line 142 is not accurate. It should be (Bremer and Dennis, 1996).

We corrected the citation.

1. Lines 192-193: "six" different growth media, not five.

Thank you for pointing that out, we corrected it.

1. Line 287: The statement "... translation rate does not increase in ribosomes with a higher protein content" could be misinterpreted as discussing translation elongation rate changes with different protein content in ribosomal protein mutant strains in a given species. It should be rephrased to remove ambiguity.

We rephrased the sentence as follows:<br /> "…translation rate does not increase in ribosomes from different species which have higher protein content."

1. Parameters for the three panels in Figure 8 are missing.

The parameters used for mitochondria are the same as for E. coli in glucose minimal media. The only difference is that a fraction of rPs can be imported. We added a sentence to the figure caption to clarify this:<br /> "The model can be adjusted to predict mitochondrial protein-rich ribosome composition. All parameters used for the simulation of mitochondria are the same as for E. coli in glucose minimal media, except a fraction of rPs can be imported for free from the cytoplasm and does not have to be synthesized. For simplicity, we assumed that 1/3 of rPs are imported. (In reality, almost all rPs are imported, but mitochondria make additional proteins to provide energy for the whole cell.)"

Reviewer #4 (Significance):

Strengths: Why the ribosome is composed of RNA and protein parts is a fundamental biological question. This manuscript proposes a very interesting hypothesis, arguing that the mixed ribosome composition results from rRNA instability. To test their hypothesis, the authors parameterize a simplified self-replicating cell model with realistic parameters. The model is first developed/parameterized for E. coli, and it could be easily adapted to other organisms with higher ribosomal protein content.

Limitations: The main limitations of this manuscript lie in the development of the model, especially the modeling of rRNA degradation and the use of constant values for growth rate-dependent parameters.

Advances: (1) This manuscript proposes a new hypothesis that rRNA instability is a universal factor that influences the ribosome composition across living organisms. (2) Compared to Kostinski and Reuveni's work, the authors have made certain improvements by including adjustable ribosome allocation to RNA and ribosomal protein when maximizing growth rate, which may lead to more realistic conclusions.

Audience: This work will be of interest to people in the field of theoretical biology, computational biology, and evolution, as well as to researchers studying ribosome structure and function.

Areas of expertise: Microbial systems biology, computational biology, and prokaryotic genomics.

This spring being 2013, roughly 1 year and a half or so

Reviewer #2 (Public Review):

This work provides a new tool (H3-Opt) for the prediction of antibody and nanobody structures, based on the combination of AlphaFold2 and a pre-trained protein language model, with a focus on predicting the challenging CDR-H3 loops with enhanced accuracy than previously developed approaches. This task is of high value for the development of new therapeutic antibodies. The paper provides an external validation consisting of 131 sequences, with further analysis of the results by segregating the test sets into three subsets of varying difficulty and comparison with other available methods. Furthermore, the approach was validated by comparing three experimentally solved 3D structures of anti-VEGF nanobodies with the H3-Opt predictions

Strengths:<br /> The experimental design to train and validate the new approach has been clearly described, including the dataset compilation and its representative sampling into training, validation and test sets, and structure preparation. The results of the in silico validation are quite convincing and support the authors' conclusions.

The datasets used to train and validate the tool and the code are made available by the authors, which ensures transparency and reproducibility, and allows future benchmarking exercises with incoming new tools.

Compared to AlphaFold2, the authors' optimization seems to produce better results for the most challenging subsets of the test set.

Weaknesses:<br /> The scope of the binding affinity prediction using molecular dynamics is not that clearly justified in the paper.

Some parts of the manuscript should be clarified, particularly the ones that relate to the experimental validation of the predictions made by the reported method. It is not absolutely clear whether the experimental validation is truly a prospective validation. Since the methodological aspects of the experimental determination are not provided here, it seems that this may not be the case. This is a key aspect of the manuscript that should be described more clearly.

Some Figures would benefit from a more clear presentation.

A Twitter user pointed out a stark contrast in tone between tweets sent from his Android phone and his iPhone in 2016, during Donald Trump's presidential campaign. A data scientist then carried out an analysis, which made evident the differences between the two groups of tweets. The investigation showed that these tweets were posted at various times of the day, by various people, and made unique use of links, hashtags, and retweets. Notably, tweets from the iPhone mostly consisted of positive announcements and pictures, but tweets from the Android handset were more critical and angry. This difference made it possible to distinguish between Trump's personal tweets, which are normally posted from an Android handset, and campaign-related tweets, which are usually posted from an iPhone. Regretfully, the capacity

The phrase context collapse reminds me of in- and out-groups in sociology. In-groups hold superiority and individuals usually belong identify with, while out-groups are the lesser or different group and treated inferiorly. This happens so often, especially with adolescents and wanting to fit in more. It's relatable because everyone's felt the need to adjust to a group to fit in, especially to minority groups because of the need to code-switch.

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

Reply to the reviewers

Reviewer #1

The paper provides models of essential complexes formed in bacteria. These models have been predicted by AlphaFold2 and in some of the models, information from existing experimental structures is utilized. The predicted models have been calculated based on standard workflow procedures which are explained in detail and can be reproduced by others. The figures are informative and clear.

We are grateful for the reviewer's insightful comments, which have significantly contributed to improve our manuscript.

Suggestions for improvement:

1. The PDB accession codes of the experimental structures should be providedb. A comparison of the predicted models with the experimental structures should be provided (e.g. same orientation, superposition). In Fig. 6 for example, a figure with superposition or use of the same orientation would be more informative.

As suggested by the reviewer, we have included a new table (Table 1) listing all experimental structures discussed in the main text, with the corresponding PDB codes. All predictions are listed in Supplementary File 1. For instances with available PDB codes, we compared the predicted structures to the experimental ones (new Supplementary Figure 3). In Fig. 6, the structures were difficult to superimpose because the subunits in the complexes have different relative orientations. To help comparing both models, we have added a schematic representation (new Fig. 6c).

The paper will certainly generate many hypotheses based on the predicted models. In this respect, it would be useful for a wide audience in the bioscience field. However, the discussed models will need experimental verification by various techniques, such as X-ray crystallography, cryo-EM, SAXS, and structural proteomics. A more thorough analysis of the literature may help to improve the paper in this respect.

We acknowledge the reviewer's emphasis on the importance of experimental verification of the predicted models. We have conducted a thorough analysis of the literature to identify instances where experimental verification could complement our predictions. We identified several mutations in BirA, documented in the literature, that affect its interaction with AccB. __In BirA mutations M310L and P143T were found to induce a superrepressor phenotype (BirA lacks the capacity to biotinylate AccB). These mutations do not significantly affect the BirA active site, but can destabilize the BirA-AccB interface. __We have added this information in the main text. Also, we investigated whether our complexes have known crosslinks in the xlinkdb database(https://xlinkdb.gs.washington.edu/xlinkdb/). We found information for five of our predicted complexes. In all cases, the distance restraints identified by crosslinking (crosslinked lysines are ~15Å apart) are compatible with our models. We have incorporated these references into a new table in Supplementary File 1. Unfortunately, we could not find more information in the xlinkdb that can be used to further validate our complexes.

Supplementary table. Selected binary complexes modeled by AF2 whose structure is experimentally verified by cross-linking mass spectrometry.

Protein 1

Protein 2

Peptide 1

Pepitde 2

Species

acca

accd

VNMLQYSTYSVISPEGCASILWKSADK

IKSNITPTR

E. coli

dnak

grpe

DDDVVDAEFEEVKDKK

VKAEMENLR

E. coli

rpob

rpoc

GKTHSSGK

KGLADTALK

E. coli

bama

bamd

TVDIKPAR

DVSYLKVAYQNFVDLIR

A. baumannii

secd

secf

ILGKTANLEFR

MPSEDPELGKK

P. aeruginosa

Reviewer #2

This study attempts to identify the 'essential interactome' through combining information in presence/absence genomics across bacteria, information in the STRING database, and predictions from alpha-fold. Overall, the strategy is clear, and I do not have concerns about reproducibility and clarity.

We value the reviewer's constructive evaluation of our manuscript and we would like to thank the reviewer's feedback as it has significantly helped us in improving our manuscript.

Strengths: Clever approach to get at the essential interactome.

Weaknesses: Putative impact. It is clear why understanding which interactions are present are important. But even as the authors suggest, interactions are dynamic and there are plenty of other tools that people could use to find interactions (including AA Coev that the authors themselves cite). The counter argument the authors bring up is the high false positive rate of interactions that is solved by this method. While true, the stringency criteria for what constitutes an interaction in this paper is remarkably high: each protein within the interaction needs to be essential, and needs to have a high confidence score in STRING, and then there is a hyperparameter that dictates the level at which AlphaFold 2 is providing confident answers. In this sense, this is less about an 'essential' interactome, and more about an interactome that is present with the highest true positive rate (trading off with the ability to discover new interactions at a reasonable breadth).

We appreciate the reviewer's insights concerning the stringency criteria for defining interactions. Here, we provide a detailed justification for our selection criteria and show how it aligns with our goal of identifying high-confidence interactions.

1. Protein essentiality: In our model, interactions are considered essential if, and only if, both proteins involved are essential, providing a conservative estimate for the essential interactome. In our revised manuscript, we explored the possibility the potential for two non-essential proteins to form an essential interaction by investigating synthetically lethal interactions. Out of all synthetic lethal interactions in * coli*, only 28 interactions were identified, and only two could be modeled with an ipTM score > 0.6. Likely, these non-essential proteins operate in parallel or compensatory pathways instead of interacting directly. These findings lend support to our hypothesis and suggest that our interactome encompasses most essential interactions.
2. Conditional essentiality: While we concur with the reviewer that our study does not address conditional essentiality, we would like to note that exploring conditional essential interactions across all the bacterial species discussed in our manuscript is currently unviable. Just as a matter of example, we checked the overlap in essential genes between Acinetobacter baumannii and Pseudomonas aeruginosa in the lung environment (Wang et al., 2014; Potvin et al., 2003). In that case, there is a minimal overlap between the two species, suggesting that conditional interactions might also be species-dependent. In our manuscript, we aimed to describe the core essential interactions for Gram-negative and Gram-positive bacteria under standard laboratory growth conditions. We agree that further research is needed to incorporate specific, context-dependent interactions to provide a complete, comprehensive view of the interactome. Nonetheless, we define here the first bacteria essential interactome that, in our opinion, marks a significant step towards understanding bacterial cell metabolism and holds relevance in applications such as developing broad-spectrum antibiotics.
3. Confidence of the interaction: All existing methods to predict protein-protein interactions, including those based on coevolution, suffer from poor performance metrics. Most of them generate many false positive interactions while missing important ones. Without the aim of being exhaustive, here we reproduce a table of some of the latest computational methods to predict PPIs. Table 1. Performance of state-of-the-art PPI prediction methods (Huang et al., 2023).

Methods

AUPRCa

*SGPPI *

0.422

Profppikernelb

0.359

PIPRc

0.342

PIPE2b

0.220

SigProdb

0.264

a AUPRC denotes the average AUPRC value of 10-fold cross-validation.

It is clear from the data that such methods are not mature enough to be used as confident predictors. Hence, we decided to resort to validated interactions in the String database, which is one of the most comprehensive PPI databases__. In this revised version, we have expanded our data set to include all experimentally labeled interactions in the String database, even those with a low probability (experimental score > 0.15). The addition of these new interactions __increased the total number of interactions tested from 1089 to 1402 and generated 38 new models for Gram-negative species (13 with high accuracy) and 275 new models for Gram-positive bacteria (18 with high accuracy). All interactions are now included in the Supplementary File 1 and high accuracy models will be deposited on Model Archive after acceptance.

Alphafold (AF2) criterion for complex prediction. Although AF2 has its limitations, its accuracy in predicting bacterial complexes is consistently high. In various benchmarking studies, AF2 Multimer accurately predicted between 70-75% of tested complexes, with almost 90% of them being of medium-to-high quality (Evans et al., Yin et al., 2022). While there might be some minor deviations, AF2 can largely capture the bacterial essential interactome accurately. In the revised version, we compare pDockQ and pDockQ2 metrics with our ipTM criterion to define confident models. We observed that both pDockQ and pDockQ2 metrics were capable of identifying highly reliable complexes, but also disregarded actual complexes (Supplementary Figure 1). Thus, we decided to retain our initial criterion, based on ipTM scores, which is consistent with other authors who used similar ipTM thresholds to model bacterial interactions (e.g., O’Reilly et al., 2023).

In summary, although our methodology has inherent limitations, we believe that our approach is sound and can give a comprehensive and realistic view of the bacterial essential interactome. We hope that these new insights further substantiate our approach.

I don't know of too many studies that use AlphaFold 2 in this way. This was clever. However, there are plenty of studies that use phylogenomic information to infer interactions. In this sense, the core idea of the paper is not intrinsically novel.

We thank the reviewer for valuing our approach. Although other methods have been used to predict interactomes, our study, to the best of our knowledge, provides the first high-quality essential interactome for bacteria. We used experimental data (analysis of single deletion mutants) to define the essential interactions in bacteria. Other methods, either using phylogenomic information and/or deep learning tools to infer interactions, have a poor performance, as illustrated in the preceding table. Often, these methods yield a high number of interactions and, in many cases, show a bias towards overrepresented entries in the positive databases used to train the predictors (Macho Rendón et al., 2022). Also, while other methods lack detailed structural insights into the interactions, we offer structural models for every interaction tested.

Overall, I do feel this would be worth publishing as an expose of AF2 is capable of. I'm not sure of the impact it will have on researchers, however.

We appreciate the reviewer's positive feedback on our manuscript. Using AF2, we identified key interactions using only gene deletion mutant data. __This manuscript reveals new insights into the assembly of essential bacterial complexes, providing specific structural details to understand their stability and function. Additionally, __our work seeks to establish a methodology applicable to all bacterial species, guiding future research in this field. The approach taken in this study may expand drug targeting opportunities and accelerate the development of more effective antibiotics aimed to disrupt these essential interactions. In conclusion, the impact of the paper lies in its novel use of Alphafold2 to understand essential bacterial protein interactions, providing key insights into assembly mechanisms, and identifying new potential drug targets.

Reviewer #3

The selection of "essential" interactions is a bit arbitrary, given that their main criterion for selection is that both proteins are essential. Unfortunately, it's not always clear where the essential protein data is coming from. Authors cite Mateus et al. (ref 15) as source for E. coli, but I don't see an explicit list of essential genes in this paper (nor its supplement). For Pseudomonas the citation doesn't contain author information and for Acinetobacter essentiality only seems to refer to "essentiality" in the lung.

As a minimum, the author should provide a table with summary statistics for the essential proteins they are using, as this is the basis for the whole paper. Such a table should include the names of the species, the number of genes that are considered as essential, a very brief characterization of how essentiality was determined and the source for this information. For instance, are the genes listed in the Supplementary File congruent with the genes in the Database of Essential Genes (DEG) for these organisms? Finally, authors should indicate in that table which (essential) protein pairs are conserved across species, as this is another one of their selection criteria. Conservation is not necessary for an essential interaction, but it certainly makes it more likely.

We understand the reviewer's concerns regarding the selection of essential interactions and the need for a more thorough description of the sources of essential protein data. To address these concerns in the revised manuscript:

1. __We included a clear explanation of the sources for essential protein data, including proper citations for each organism in Supplementary File 1. __The selected studies were primarily sourced from the DEG database. If data was unavailable, we revised the literature for relevant studies. The DEG database's most recent update was on September 1, 2020. __A graphical summary of the datasets has been included in Supplementary Figure 12, __that shows the overlapping between the different studies.
2. We included comprehensive information for the essential proteins used in our study in Supplementary File 1. The file provides two tables detailing genes for both Gram-positive and Gram-negative datasets. Each table lists the gene names and their corresponding Uniprot IDs for every species in our study, as well as their orthologues in other organisms. Also, the reviewer was right in pointing out that for Acinetobacter baumannii, the study was conducted in the lung, which may bias the results as all other studies were performed in the test tube. To solve this, we replaced this study for Bai et al., 2021, that was performed in rich medium.

Author should also state whether they have verified that none of the random pairs are in the positive set.

We thank the reviewer for this comment. We certainly checked that none of the random pairs was present in the positive dataset. This clarification has now been added to the methods section.

This is also relevant because authors "retrieved all high-confidence PPIs between these proteins from the STRING database" which provides compound scores for interactions but that has often little to do with physical interactions (given that the scores factor in co-expression and several other criteria). In fact, I find STRING scores difficult to interpret for that very reason.

We appreciate the reviewer's comment to the use of combined interaction scores from the STRING database. We agree with the reviewer that STRING combined scores are somehow difficult to interpret because they combine different evidence of interaction. We decided to use the STRING combined scores to include interactions that may not have direct experimental evidence but are probable to interact according to other information (e.g., co-expression). However, to further examine the interactome we have also included in the revised version all interactions with experimental evidence in String to complete our interactome. As mentioned in the response to Reviewer 1, __we expanded the tested interactions from 1089 to 1402. This resulted in 38 new models for Gram-negative species, with 13 being highly accurate, and 275 for Gram-positive bacteria, of which 18 were highly accurate. All interactions are now included in the Supplementary File 1 __and high accuracy models will be deposited on the Model Archive after acceptance.

The authors "reasoned that a given interaction would only be essential if and only if both proteins forming the complex are essential" - this sounds reasonable but doesn't capture synthetically lethal (genetic) interactions, that is, interactions between two proteins that are both non-essential but are essential in combination. Admittedly, I don't have a number of how many such cases exist, but there are such cases in the literature (e.g. Hannum et al. 2009, PLoS Genet 5[12]: e1000782, for yeast).

We thank the reviewer for bringing this point into discussion. We acknowledge that our reasoning does not capture synthetic lethality, which occurs when the loss of one of two individual genes has no effect on cell survival, but the simultaneous loss of both leads to cell death. In this case, the two genes or proteins are non-essential individually but become essential in combination. To cover synthetic lethality, we retrieved all synthetically lethal interactions found in Escherichia coli, strain K12-BW25113 from the Mlsar database and included them in our pipeline. We identified 28 synthetically lethal PPIs (involving 45 proteins) and we modeled them with AF2. Only two interactions displayed an ipTM score > 0.6 (nadA-pncB and nuoG-purA). Hence, the number of interactions due to synthetic lethality seems to contribute low to the overall interactome. We believe that synthetic lethal partners often function in parallel or compensatory pathways, rather than directly interacting with each other. For example, in yeast, the genes RAD9 and RAD24 are synthetic lethal. RAD9 is involved in cell cycle checkpoints, while RAD24 is involved in DNA damage response. They function in related pathways but do not encode proteins that directly interact with each other. Hence, finding specific examples of proteins that are both synthetic lethal and directly interact might be challenging as the synthetic lethal relationship often reveals functional rather than physical interactions.

Apart from that, one could question the selection method more generally, given that for a biological process always essential and non-essential proteins work together, so I wonder why the authors didn't include additional proteins known to be involved in specific processes as this could make their predictions much more biologically meaningful.

We agree with the reviewer that accessory proteins are important to understand the biological context of interactions. In fact, in several sections of our manuscript, we included accessory proteins to fully describe the essential complexes. For example, in the cell division complex, we incorporated proteins like MreCD-RodZ from the elongasome to enhance the structural context of the interactions. However, a comprehensive explanation of all identified interactions and accessory proteins would extend beyond the scope of this manuscript and further lengthen an already extensive document. In our study, we sought to describe the fundamental interactions for both Gram-negative and Gram-positive bacteria. We anticipate that our findings will prompt additional research to confirm our hypotheses and enhance knowledge of these protein complexes within the proper cellular context.

In any case, to understand their choice better, authors should provide a table (in the main text) summarizing the proteins they actually analyze and discuss in more detail in their models. This would allow a reader to see which proteins are considered essential and which ones are missing. I would organize this by function / pathway / process, so these proteins are listed in a functional context.

We added Table 1 in the main text, listing all interactions described in the text. Table 1 includes the proteins involved in each complex, the ipTM score of the interaction, whether a PDB code is available for comparison and the functional classification of the interaction.

With regard to docking, please also discuss why you focus on iPTM, as there are other derived metrics from AF2 scores, such as pdockq based on if_plddt (e. g. Bryant et al, 2022), as well as external metrics to AF2 (physics-based methods such as Rosetta). Another option may be a modified versions of AF2 multimer, such as AFSample, which produces a greater diversity of models, allowing for more "shots on goal" and ultimately a higher success rate, assuming one has a reliable QC filter (I wonder how those compares to iPTM).

We did not use AFsample because is a very expensive computational approach that would require too many resources for the batch prediction of more than 1.400 complexes. AFsample generates 240x models, and including the extra recycles, the overall timing is around 1,000x more costly than the baseline. However, we acknowledge that using other metrics can be useful to further evaluate our models. Hence, we investigated how pDockQ and pDockQ2 metrics compare with ipTM score. We observed that pDockQ hardly correlates with ipTM (R = 0.328) whereas the improved metric pDockQ2 correlates much better (R = 0.649). All complexes described in the manuscript, which have an ipTM score higher than our threshold (0.6), have also a pDockQ2 score higher than 0.23, except for six interactions that have a lower pDockQ2 score. However, these scores improve when the interactions are modeled with accessory proteins in the complex. This somehow suggests that the ipTM metric better captures binary interactions when these are excluded from their context. __It is possible however, that pDockQ scores are better in discriminating false positive interactions than ipTM scores. Based on the strong correlation between the two metrics and the observation that ipTM may better capture binary interactions, we decided to keep our method in the manuscript. Other authors have employed analogous ipTM thresholds to model bacterial interactions (e.g., O’Reilly et al., 2023). Notwithstanding, __we also included pDockQ and pDockQ2 metrics in Supplementary File 1, so readers can evaluate complexes based on these metrics.

Minor comments:

1. 1, 3rd last line: "the essential interactome is a potentially powerful strategy to [...] identify new targets for discovering new antibiotics"

2. Figures and figure legends need to be explicit which species is represented (ideally with a Uniprot ID) and which structure was predicted by alphafold and which one has an experimental structure. Known structures should be indicated in a table, as suggested above.

3. Figure 5: LptF is too dark when printed, so a lighter color may be better.

4. Figure 6: The cryoEM and alphafold structures look quite different, so please discuss discrepancies between them (in terms of prediction or cryEM modeling). A schematic may be helpful to illustrate the differences in more clarity.

5. Figure 7: LolC is also too dark when printed. Make lighter.

6. Maybe in some cases it may be worthwhile looking at Consurf structures to see if the predicted inferfaces are indeed more conserved than the non-conserved parts.

We thank the reviewer for his/her insightful feedback on our manuscript. We have addressed all these comments as follows:

1. The statement on page 1 was revised as suggested.
2. We revised all figure legends to include the Uniprot IDs, and distinguish between predicted and experimental structures. We also included Table 1 and Supplementary File 1 for known structures.
3. We adjusted the colors in Figures 5 and 7 to enhance print visibility.
4. We provided a schematic to illustrate discrepancies between cryoEM and AlphaFold structures in Figure 6c.
5. We used Vespa to highlight conserved interfaces in the complexes described in the manuscript, as suggested. The figures displaying the conservation of interfaces in the complexes are now depicted in Supplementary Figure 2. A comparison between interface and surface conservation can be found in Figure 1f.

   The main significance of this study is its potential use for a better understanding of the protein complexes described in more detail (and the fact that alphafold can be applied in a similar fashion to many other complexes). This is why the individual sections need to be evaluated to process-specific experts (disclaimer: I have only worked on some of the complexes, but I am not an expert on any of them). I wonder if it would make more sense to break out some of the sections on individual complexes into separate papers, and then discuss them in more detail and with more context from previous studies. Complexes such as the divisome have a huge body of literature and it may be worth reviewing which structures are known and which ones are not. However, the dynamic and labile nature of these complexes have made it difficult for both crystallography as well as modeling to get a good structural understanding, but some of the models proposed here may be useful for overcoming some of these hurdles.

We appreciate the reviewer's suggestion. While we acknowledge the complexity of some of the individual complexes, such as the divisome, and the wealth of existing literature, we believe that the current manuscript provides a valuable comprehensive view on how AF2 can be used to predict essential protein complexes in bacteria. In our opinion, dividing the manuscript in separate pieces might dilute its scope. Nonetheless, we are exploring in our laboratory the interactions detailed in the manuscript, aiming to further expand the knowledge on these important complexes and their potential as targets for new antimicrobials.

References:

Bai J, Dai Y, Farinha A, et al. Essential Gene Analysis in Acinetobacter baumannii by High-Density Transposon Mutagenesis and CRISPR Interference. J Bacteriol. 2021; 203(12):e0056520.

Evans R, O’Neill M, Pritzel A, et al. Protein complex prediction with AlphaFold-Multimer.

bioRxiv. 2021; 2021.10.04.463034.

Huang Y, Wuchty S, Zhou Y, Zhang Z. SGPPI: structure-aware prediction of protein-protein interactions in rigorous conditions with graph convolutional network. Brief Bioinform. 2023; 24(2):bbad020

Macho Rendón J, Rebollido-Ríos R, Torrent Burgas M. HPIPred: Host-pathogen interactome prediction with phenotypic scoring. Comput Struct Biotechnol J. 2022; 20:6534-6542.

O'Reilly FJ, Graziadei A, Forbrig C, et al. Protein complexes in cells by AI-assisted structural proteomics. Mol Syst Biol. 2023; 19(4):e11544.

Potvin, E., Lehoux, D.E., Kukavica-Ibrulj, I., et al. In vivo functional genomics of Pseudomonas aeruginosa for high-throughput screening of new virulence factors and antibacterial targets. Environmental Microbiology. 2003; 5: 1294-1308.

Wang N, Ozer EA, Mandel MJ, Hauser AR. Genome-wide identification of Acinetobacter baumannii genes necessary for persistence in the lung. mBio. 2014; 5(3):e01163-14.

Yin, R, Feng, BY, Varshney, A, Pierce, BG. Benchmarking AlphaFold for protein complex modeling reveals accuracy determinants. Protein Science. 2022; 31(8):e4379.

Similar to the idea of code switching discussed in the previous chapter. This aptitude for recognizing patterns is also why it's easy for a troll to disrupt the stability of those practices. Interesting to see how this notion of "trolling" can be used to disrupt structures that may be seen in a negative light, such as the K-Pop protest example. Despite its inherent connotations, many things on social media such as trolls and bots come down to perspective, which is why considering multiple ethical frameworks are beneficial.

Is this a book? article?

Author Response

Reviewer #1 (Public Review):

Summary:

Rai1 encodes the transcription factor retinoic acid-induced 1 (RAI1), which regulates expression of factors involved in neuronal development and synaptic transmission. Rai1 haploinsufficiency leads to the monogenic disorder Smith-Magenis syndrome (SMS), which is associated with excessive feeding, obesity and intellectual disability. Consistent with findings in human subjects, Rai1+/- mice and mice with conditional deletion of Rai1 in Sim+ neurons, which are abundant in the paraventricular nucleus (PVN), exhibit hyperphagia, obesity and increased adiposity. Furthermore, RAI1-deficient mice exhibit reduced expression of brain-derived neurotrophic factor (BDNF), a satiety factor essential for the central control of energy balance. Notably, overexpression of BDNF in PVN of RAI1-deficient mice mitigated their obesity, implicating this neurotrophin in the metabolic dysfunction these animals exhibit. In this follow up study, Javed et al. interrogated the necessity of RAI1 in BDNF+ neurons promoting metabolic health.

Consistent with previous reports, the authors observed reduced BDNF expression in the hypothalamus of Rai1+/- mice. Moreover, proteomics analysis indicated impairment in neurotrophin signaling in the mutants. Selective deletion of Rai1 in BDNF+ neurons in the brain during development resulted in increased body weight, fat mass and reduced locomotor activity and energy expenditure without changes in food intake. There was also a robust effect on glycemic control, with mutants exhibiting glucose intolerance. Selective depletion of RAI1 in BDNF+ neurons in PVN in adult mice also resulted in increased body weight, reduced locomotor activity, and glucose intolerance without affecting food intake. Blunting RAI1 activity also leads to increases and decreases in the inhibitory tone and intrinsic excitability, respectively, of BDNF+ neurons in the PVN.

Strengths:

Overall, the experiments are well designed and multidisciplinary approaches are employed to demonstrate that RAI1 deficits in BDNF+ neurons diminish hypothalamic BDNF signaling and produce metabolic dysfunction. The most significant advance relative to previous reports is the finding from electrophysiological studies showing that blunting RAI1 activity leads to increases and decreases the inhibitory tone and intrinsic excitability, respectively, of BDNF+ neurons in the PVN. Furthermore, that intact RAI1 function is required in BDNF+ neurons for the regulation of glucose homeostasis.

Weaknesses:

Some of the data need to be reconciled with previous findings by others. For example, the authors report that more than 50% of BDNF+ neurons in PVN also express pTrkB whereas about 20% of pTrkB+ cells contain BDNF, raising the possibility that autocrine mechanisms might be at play. This is in conflict with a previous study by An et al, (2015) showing that these cell populations are largely non-overlapping in the PVN.

We fully agree with this assessment. Given the difficulty of using immunostaining to characterize the expression of membrane proteins in vivo, and the specificity of the pTrkB antibody in different tissues remains unknown, it is difficult to interpret the signals we observed. We have excluded the data because the histological analysis of p-TRKB and BDNF autocrine/paracrine signalling is not a focus of the present study. Future studies using a more advanced genetic method (i.e., Ntrk2CreER/+; Ai9 mouse line as used by An et al., 2015) is more suitable and should be used in the future to investigate the function of Rai1 in the TRKB+ neurons.

Another issue that deserves more in-depth discussion is that diminished BDNF function appears to play a minor part driving deficits in energy balance regulation. Accordingly, both global central depletion of Rai1 in BDNF+ neurons during development and deletion of Rai1 in BDNF+ neurons in the adult PVN elicited modest effects on body weight (less than 18% increase) and did not affect food intake. This contrasts with mice with selective Bdnf deletion in the adult PVN, which are hyperphagic and dramatically obese (90% heavier than controls). Therefore, the results suggest that deficits in RAI1 in PVN or the whole brain only moderately affect BDNF actions influencing energy homeostasis and that other signaling cascades and neuronal populations play a more prominent role driving the phenotypes observed in Rai1+/- mice, which are hyperphagic and 95% heavier than controls. The results from the proteomic analysis of hypothalamic tissue of Rai1 mutant mice and controls could be useful in generating alternative hypotheses. Depleting RAI1 in BDNF+ neurons had a robust effect compromising glycemic control. However, as the approach does not necessarily impact BDNF exclusively, there should be a larger discussion of alternative mechanisms.

We thank the reviewer for these insightful comments. We want to highlight that global deletion of Rai1 from BDNF neurons did induce food intake increase in male mice (Fig 2figure supplement 4K). We have incorporated the following paragraphs into the discussion section.

Lines 364-384: “Notably, mice lacking one copy of Rai1 in the BDNF-producing cells do not exhibit obesity, whereas SMS patients and SMS mice show pronounced obesity (Burns et al., 2010; Huang et al., 2016; Smith et al., 2005). This indicates that although reduced Bdnf expression and BDNF-producing neurons contribute to regulating body weight, additional molecular changes and other hypothalamic populations also play important roles in regulating body weight homeostasis in SMS. Our RPPA data suggest that mTOR signalling is also misregulated in addition to the reduced activation of the neurotrophin downstream cascades. Hypothalamic mTORC1 is crucial to regulate glucose release from the liver, peripheral lipid metabolism, and insulin sensitivity (Burke et al., 2017; Caron et al., 2016; Smith et al., 2015), while mTORC2 regulates glucose tolerance and fat mass (Kocalis et al., 2014). How the impaired mTOR signalling contributes to energy homeostasis defects in SMS and the therapeutic potential of targeting this pathway to treat SMS-related obesity remains unclear and warrants future investigation.

What additional Rai1-dependent hypothalamic cell types residing in brain regions other than PVH regulate obesity in SMS? Other important cell types such as TRKB neurons within the PVH (An et al., 2020) and several RAI1-expressing hypothalamic nuclei including the arcuate nucleus, ventromedial nucleus of the hypothalamus (VMH), and lateral hypothalamus all play important roles in regulating energy homeostasis. POMC- and AGRP-expressing neurons within the arcuate nucleus are known to regulate food intake and glucose and insulin homeostasis (Quarta et al., 2021; Vohra et al., 2022). Therefore, Rai1 function in these neurons could contribute to obesity in SMS, a topic that awaits future investigation.”

Reviewer #2 (Public Review):

Understanding disease conditions often yields valuable insights into the physiological regulation of biological functions, as well as potential therapeutic approaches. In previous investigations, the author's research group identified abnormal expression of brain-derived neurotrophic factor (BDNF) in the hypothalamus of a mouse model exhibiting Smith-Magenis syndrome (SMS), which is caused by heterozygous mutations of the Rai1 gene. Human SMS is associated with distinct facial characteristics, sleep disturbances, behavioral issues, and intellectual disabilities, often accompanied by obesity. Conditional knockout (cKO) of the Bdnf gene from the paraventricular hypothalamus (PVH) in mice led to hyperphagic obesity, while overexpression of the Bdnf gene in the PVH of Rai1 heterozygous mice restored the SMS-like obese phenotype. Based on these preceding findings, the authors of the present study discovered that homozygous Rai1 cKO restricted to Bdnf-expressing cells, or Rai1 gene knockdown solely in Bdnf-positive neurons in the PVH, induced obesity along with intricate alterations in adipose tissue composition, energy expenditure, locomotion, feeding patterns, and glucose tolerance, some of which varied between sexes. Additionally, the authors demonstrated that a brain-penetrating drug capable of activating the TrkB pathway, a downstream signaling pathway of BDNF, partially alleviated the SMS-like obesity phenotype in female mice with Rai1 heterozygous mutations. Although the specific (neural) cell type responsible for this TrkB signaling remains an open question, the present study unequivocally highlights the importance of Rai1 gene function in PVH Bdnf neurons for the obesity phenotype, providing valuable insights into potential therapeutic strategies for managing obesity associated with SMS.

In the proteomic analysis (Fig. 1), the authors elucidated that multiple phospho-protein signaling pathways, including Akt and mTOR pathways, exhibited significant attenuation in the SMS model mice. Of significance, the manifestation of haploinsufficiency of the Rai1 gene exclusively within the BDNF+ cells demonstrated negligible impact on body weight (Fig. 2supple 3D), despite observing a reduction in BDNF levels in the heterozygous Rai1 mutant (Fig. 1A). Conversely, the homozygous Rai1 cKO in the BDNF+ cells prominently displayed an obesity phenotype, suggesting substantial dissimilarities in the gene expression profiles between Rai1 heterozygous and homozygous conditions within the BDNF+ cell population. It would be advantageous to precisely identify the responsible differentially expressed genes, possibly including Bdnf itself, in the homozygous cKO model. The observed reduction in the excitability of PVH BDNF+ cells (Fig. 3) is presumably attributed to aberrant gene expression other than Bdnf itself, which may serve as a prospective target for gene expression analysis. Notably, the Rai1 homozygous cKO mice in BDNF+ cells exhibited some sexual dimorphisms in feeding and energy expenditures, as evidenced by Fig. 2 and related figures. Exploring the potential relevance of these sexual differences to human SMS cases and investigating the underlying cellular/molecular mechanisms in the future would provide valuable insights.

Although the CRISPR-mediated knockdown of the Rai1 gene (Fig. 4) appears to be highly effective, given the broad transduction of AAV serotype 9, it may be helpful to exclude the possibility of other brain regions adjacent to the PVH, such as the DMH or VMH, being affected by this viral procedure. If the PVH-specificity is established, the majority of Rai1 cKO effects in Bdnf+ cells are primarily attributed to PVH-Bdnf+ cells based on the similarity of phenotypes observed. With regards to the apparent rescue of the body weight phenotype in Rai1 heterozygous mutants using a selective TrkB activator, the specific biological processes, and neurons responsible for this effect remain unclear to this reviewer. Elucidating these aspects would be significant when considering potential applications to human SMS cases.

We appreciate the reviewer's insightful comments. We agree that the logical next step would be to identify the profile of the differentially expressed genes in our homozygous conditional knockout model. We have included the following paragraphs in the discussion.

Lines 364-384: “Notably, mice lacking one copy of Rai1 in the BDNF-producing cells do not exhibit obesity, whereas SMS patients and SMS mice show pronounced obesity (Burns et al., 2010; Huang et al., 2016; Smith et al., 2005). This indicates that although reduced Bdnf expression and BDNF-producing neurons contribute to regulating body weight, additional molecular changes and other hypothalamic populations also play important roles in regulating body weight homeostasis in SMS. Our RPPA data suggest that mTOR signalling is also misregulated in addition to the reduced activation of the neurotrophin downstream cascades. Hypothalamic mTORC1 is crucial to regulate glucose release from the liver, peripheral lipid metabolism, and insulin sensitivity (Burke et al., 2017; Caron et al., 2016; Smith et al., 2015), while mTORC2 regulates glucose tolerance and fat mass (Kocalis et al., 2014). How the impaired mTOR signalling contributes to energy homeostasis defects in SMS and the therapeutic potential of targeting this pathway to treat SMS-related obesity remains unclear and warrants future investigation.

What additional Rai1-dependent non-PVH hypothalamic cell types regulate obesity in SMS? Other important cell types such as TRKB neurons within the PVH (An et al., 2020) and several RAI1expressing hypothalamic nuclei including the arcuate nucleus, ventromedial nucleus of the hypothalamus (VMH), and lateral hypothalamus all play important roles in regulating energy homeostasis. POMC- and AGRP-expressing neurons within the arcuate nucleus are known to regulate food intake and glucose and insulin homeostasis (Quarta et al., 2021; Vohra et al., 2022). Therefore, Rai1 function in these neurons could contribute to obesity in SMS, a topic that awaits future investigation.”

Lines 409-418: “It is plausible that RAI1 regulates the expression of genes encoding inward rectifier K+ channels, which regulate neuronal activity and potentially energy homeostasis. For instance, KIR6 (a family of ATP-sensitive potassium channels, KATP) is widely expressed in the hypothalamus. Deleting the hypothalamic KIR6.2 subunit impairs KATP channel function and glucose tolerance (Miki et al., 2001; Parton et al., 2007). Moreover, reduced expression of hypothalamic GIRK4 (encoding an inwardly rectifying potassium channel) causes obesity (Perry et al., 2008). GABAergic neurotransmission from arcuate AGRP-expressing neurons to the PVH neurons is important to increase appetite by favouring hyperphagia (Atasoy et al., 2012). Disrupting the composition of these ion channels could contribute to reduced PVHBDNF neuronal firing, which awaits further investigations.”

Moreover, to facilitate the future exploration of the potential relevance of sexual differences to human SMS cases, we have incorporated the following explanation in the discussion section.

Lines 419-426: “Female mice with a conditional knockout of Rai1 from BDNF-producing neurons do not display a noteworthy difference in food intake. Conversely, their male counterparts exhibit a significant increase in food intake. Although SMS individuals of both genders tend to overeat, male patients who are obese show significantly higher food consumption than their female counterparts (Gandhi et al., 2022). This observation raises the possibility that Rai1 regulates eating behaviours through multiple cell types in the hypothalamus and that a male-specific involvement of BDNF-producing neurons in regulating food intake, potentially provides a neurobiological basis for the observed pattern in SMS patients (Gandhi et al., 2022).”

To exclude the possibility of other brain regions adjacent to the PVH (such as VMH and arcuate nucleus) being affected by our AAV-CRISPR-mediated Rai1 knockout, we have analyzed other hypothalamic regions including VMH and arcuate nucleus from the same slides used to confirm PVH viral expression and we confirmed that the AAV was not expressed in these regions. We have incorporated a representative image (Figure 4 suppl 1F) depicting limiting AAV expression in these nuclei.

Regarding LM22A-4: It is possible that LM22A-4 functions directly through binding to TRKB or indirectly engages TRKB downstream molecules through activating other receptors such as GPCR. LM22A-4 appears to engage neurotrophin downstream PI3KAKT pathway, which was identified by our RPPA analysis to be downregulated in the hypothalamus of Rai1-deficient mice. Reduced AKT activity is associated with insulin resistance and obesity in mice. Restoration of functional activity of AKT by LM22A-4 could be the primary mode of action for this drug in the brain. However, since we observed that this drug only partially rescued the body weight defect, future research exploring more potent TrkB agonists or utilizing a combination therapy that targets both the neurotrophin and mTOR pathways might yield improved responses to the pharmacological interventions. We have included the following paragraph in the discussion:

Lines 451-461: “ We recognize that while several in vivo studies have demonstrated the potential of LM22A-4 in targeting neurotrophin downstream signalling (Kron et al., 2014; Li et al., 2017), an in vitro analysis failed to demonstrate the ability of LM22A-4 to activate TrkB directly (Boltaev et al., 2017). Therefore, the precise mechanism by which LM22A-4 enhances AKT cascades in the mammalian brain remains unclear and awaits further investigations. In the hypothalamus of SMS mice, LM22A-4 could indirectly engage neurotrophin downstream PI3KAKT pathway through the G protein-coupled receptor-dependent transactivation of the TRKB receptor (Domeniconi & Chao, 2010) or other unknown mechanisms. Moreover, while LM22A4 may have potential side effects, we found that wild-type mice treated with LM22A-4 did not show a further decrease in body weight, suggesting limited side effects regarding body weight regulation.”

Overall, the present study represents a valuable addition to the authors' series of high-quality molecular genetic investigations into the in vivo functions of the Rai1 gene. This reviewer particularly commends their diligent efforts to enhance our comprehension of SMS and contribute to the future development of more effective therapies for this syndrome.

We thank the reviewer for finding our study valuable in advancing the understanding of RAI1 function.

Reviewer #3 (Public Review):

Summary:

Smith-Magenis syndrome (SMS) is associated with obesity and is caused by deletion or mutations in one copy of the Rai1 gene which encodes a transcriptional regulator. Previous studies have shown that Bdnf gene expression is reduced in the hypothalamus of Rai1 heterozygous mice. This manuscript by Javed et al. further links SMS-associated obesity with reduced Bdnf gene expression in the PVH.

Strengths:

The authors show that deletion of the Rai1 gene in all BDNF-expressing cells or just in the PVH BDNF neurons postnatally caused obesity. Interestingly, mutant mice displayed sexual dimorphism in the cause for the obesity phenotype. Overall, the data are well presented and convincing except the data from LM22A-4.

Weaknesses:

1) The most serious concern is about data from LM22A-4 administration experiments (Figure 5 and associated supplemental figures). A rigorous study has demonstrated that LM22A-4 does not activate TrkB (Boltaev et al., Science Signaling, 2017), which is consistent with unpublished results from many labs in the neurotrophin field. It is tricky to interpret body weight data from pharmacological studies because compounds always have some side effects, which can reduce body weight non-specifically.

We thank this reviewer for their valuable comments. Indeed, the precise mechanism by which LM22A-4 exerts its effect is not entirely clear and there has been mixed evidence regarding its identity as a TRKB agonist in vitro. We have refrained from stating LM22A-4 as a partial agonist of TRKB, and instead have focused on highlighting the potential of this drug in activating neurotrophin downstream signalling through increasing AKT phosphorylation in vivo. We have modified the title to remove TRKB, and the following changes have been made in the discussion:

Lines 451-461: “ We recognize that while several in vivo studies have demonstrated the potential of LM22A-4 in targeting neurotrophin downstream signalling (Kron et al., 2014; Li et al., 2017), an in vitro analysis failed to demonstrate the ability of LM22A-4 to activate TrkB directly (Boltaev et al., 2017). Therefore, the precise mechanism by which LM22A-4 enhances AKT cascades in the mammalian brain remains unclear and awaits further investigations. In the hypothalamus of SMS mice, LM22A-4 could indirectly engage neurotrophin downstream PI3KAKT pathway through the G protein-coupled receptor-dependent transactivation of the TRKB receptor (Domeniconi & Chao, 2010) or other unknown mechanisms. Moreover, while LM22A4 may have potential side effects, we found that wild-type mice treated with LM22A-4 did not show a further decrease in body weight, suggesting limited side effects regarding body weight regulation.”

2) The resolution of all figures are poor, and thus I could not judge the quality of the micrographs.

We have updated with higher resolution images.

3) Citation of the literature is not precise. The study by An et al. (2015) shows that deletion of the Bdnf gene in the PVH leads to obesity due to increased food intake and reduced energy expenditure (not just hyperphagic obesity; Line 72). Furthermore, the study by Unger et al. (2017) carried out Bdnf deletion in the VMH and DMH using AAV-Cre and did not discuss SF1 neurons at all (Line 354). The two studies by Yang et al. (Mol Endocrinol, 2016) and Kamitakahara et al. (Mol Metab, 2015) did use SF1-Cre to delete the Bdnf gene and did not observe any obesity phenotype.

We thank the reviewer for bringing this to our attention. We have revised the text to ensure accurate representation of the cited publications. The following changes have been made: Lines 348-350: “ Although BDNF is required in the VMH and DMH to regulate body weight (Unger et al., 2007), embryonic deletion of Bdnf from the SF1-lineage populations including the VMH did not result in obesity (Kamitakahara et al., 2016; Yang et al., 2016).”

4) Animal number is not described in many figure legends.

We thank the reviewer for pointing it out. We have revised the manuscript to incorporate the missing animal numbers.

Reviewer #1 (Recommendations For The Authors):

Additional points:

1) The data provided indicating increased inhibitory tone onto BDNF neurons in PVN of Rai1 mutant mice are not convincing that inhibitory drive is significantly affected.

We have modified the sentences as follows, we have also deleted these conclusions from the abstract and discussion:

Lines 215-220: “We observed a slight rightward shift of the probability of miniature inhibitory postsynaptic current (mIPSC) frequency in cKO PVHBDNF neurons, although the average frequency (Fig 3K) was not significantly different between groups. The probability of mIPSC amplitude also showed a right shift without a significant change (Fig 3L, Figure 3—figure supplement 1D). However, we observes a significant increased area under the curve (Fig 3M).”

2) Fig. 3C - Was outlier analysis performed for these data? One of the data points for the control group looks like an outlier that might be skewing the data.

We performed an outlier analysis and found that indeed one data point was an outlier, after removing this data point, the data remained statistically significant (*p<0.05) and the new manuscript has been updated.

Reviewer #2 (Recommendations For The Authors):

1) The manuscript would benefit from improved usage and precise descriptions of statistics. The authors often provided only general statements such as "one or two-way ANOVA" without specifying the exact statistical tests used. It is important to differentiate between one-way and two-way ANOVA, particularly when using the latter, by clearly indicating the within-group effects and interaction effects. The representation of p-values associated with ANOVA using asterisks requires clarification, specifying which statistics indicate ANOVA results and which ones correspond to post hoc analysis. It is advisable to assess the normality of the distribution before employing t-tests or consider non-parametric comparisons such as Wilcoxon's rank sum test if normality assumptions are not met. Additionally, it is essential to specify whether the tests are one-sided or two-sided and whether they are paired or unpaired. In some figure panels, such as Fig. 2H and K, the statistical tests used were not indicated at all.

We have clarified the exact statistical tests in the figure legend for each figure.

2) Rearranging the figures to facilitate a direct comparison of the sexual phenotypes (Fig. 2 and Fig. 2-supple 4) within the same figures would greatly improve reader comprehension.

We have decided to keep the figure arrangement because of the focus on female mice in the main figures.

3) To improve the comprehension of the figures and text, the following points should be addressed:

• Fig. 1D: The definition of the expression level in the color code is not clear.

Explanation for the color code has been added in the method section.<br /> Lines 652-656: “The vertical axis of the dendrogram represents the dissimilarity (measured as distance) between protein expressions, and the horizontal axis represents the individual test samples. The colour code (ranging from red to yellow to green) specifies the expression levels of different proteins, where red indicates nifies low expression, yellow indicates intermediate expression, and green indicates high expression.”

• Fig. 1F: One parenthesis is missing from the figure label.

Fixed

• Fig. 2C: It is unclear why there are so many dots for just n = 3 animals. It would be better to specify the conditions or use "animals" as a unit of measurement.

The dots represent percentage cells quantified per sliced from 3 animals. It has been clarified in the figures.

• Fig. 2F: There seems to be an unnecessary label "I" in the middle of the panel.

Fixed

• It is not completely clear if the data in Fig. 2E-L were all obtained at 26 weeks of age.

To clarify, following line has been added to the method section:

Lines 517-518: “After the 25th week, mice were subjected to body composition analysis.”

• In Fig. 2-Supple 1, the legend should read "G-J." Additionally, please provide a definition for the arrowheads.

Line 1086: “yellow arrowheads indicate Ai9 marked BDNF cells co-expressing endogenous BDNF.”

• It is not completely clear if the data in Fig. 3 were all obtained from female mice.

It is explained in the legend of Fig 3.

• The description of the number of animals seems to be missing in Fig. 4

The description for the number of animals has been added in the figure legend. Line 1004: “(Ctrl group: n=5, Exp group: n =5)”

• On line 280-281, "Fig 4A." should be corrected to "Fig. 5A."

Corrected.

• In Fig. 5C-E, it is uncertain if multiple pairwise comparisons for three groups are statistically appropriate. At the very least, multiple comparisons should be corrected.

We performed two-way ANOVA where mean body weight of age-matched groups were compared with each other (i.e. between control saline-injected and SMS saline-injected, SMS saline-injected and LM22A-4 -saline injected, and Control saline-injected and SMS LM22A-4 injected). We used Šidák’s multiple comparisons test, where statistical significance was indicated with p<0.05, p < 0.01, p<0.001, **p < 0.0001. We have clarified this in the figure 5 legends.

• The unit of measurement should be standardized across figures, if possible, to facilitate better side-by-side comparisons. For example, most bodyweight figures use "g" (grams), but "mg" (milligrams) is used in Fig. 5.

All measurements are corrected to be consistent (in grams).

• It is unclear if nM (not mM) of glucose was actually measured in the glucose tolerance test (Fig. 2L and Fig. 4L).

Fixed.

Reviewer #3 (Recommendations For The Authors):

1) The authors can remove the LM22A-4 data without much detrimental effects on the conclusion of the manuscript. Otherwise, the authors have to demonstrate that LM22A-4 activates TrkB, does not have any toxicity, and does not cause aversion.

We thank this reviewer the valuable comments and we acknowledge the valid concern. Indeed, the precise mechanism by which LM22A-4 exert its effects is not clear and there has been mixed opinions regarding its function as TRKB agonist in in-vitro assays. To clarify, we have refrained from stating LM22A-4 as a partial agonist of TRKB, and instead have focused on highlighting the potential of this drug in activating neurotrophin downstream signalling through increased AKT phosphorylation, in-vivo.

We have also modified the title of our article to exclude the word “TRKB Signalling”. The new title is as follows:

“Smith-Magenis syndrome protein RAI1 regulates body weight homeostasis through hypothalamic BDNF-producing neurons and neurotrophin downstream signalling”

2) Line 50: "40% > 95th percentile weight, 40% > 85th percentile weight" should be "40% > 95th percentile weight, 80% > 85th percentile weight".

Corrected.

3) Abbreviations for brain-derived neurotrophic factor: Bdnf for gene and BDNF for protein.

Abbreviations have been corrected throughout the manuscript.

4) Need to specify the animal age when viruses were injected into the PVH to inactivate the Bdnf gene.

Line 235: Virus was injected at 3 weeks of age. It has been specified in the main text.

5) Line 832: "3 technical triplicates" can be simplified as "3 technical repeats" because 3 and triplicates are redundant.

Corrected.

6) Figure 2B: The "O" in cKO is misplaced.

Fixed.

7) Figure 3: The black legends in E and F should include Ctrl.

Fixed in the Figure 3.

Author Response

The following is the authors’ response to the original reviews.

eLife assessment

This useful manuscript challenges the utility of current paradigms for estimating brain-age with magnetic resonance imaging measures, but presents inadequate evidence to support the suggestion that an alternative approach focused on predicting cognition is more useful. The paper would benefit from a clearer explication of the methods and a more critical evaluation of the conceptual basis of the different models. This work will be of interest to researchers working on brain-age and related models.

Response: Thank you so much for providing high-quality reviews on our manuscript. We revised the manuscript to address all of the reviewers’ comments and provided full responses to each of the comments below.

Briefly, regarding clearer explanations of the methods, we added additional analyses (e.g., commonality analyses on ridge regression and on multiple regressions with a quadratic term for chronological age) to address some of the concerns and additional details in text and figures to ensure that the reader can fully understand our methodological procedures. Regarding the critical evaluation of the conceptual basis of the different models, we added discussions to help with interpretations and the scope of the generalisability of our findings. For instance, as opposed to treating Brain Cognition and Brain Age as separate biomarkers and comparing them in the ability to explain fluid cognition, we now treated the capability of Brain Cognition in capturing fluid cognition as the upper limit of Brain Age’s capability in capturing fluid cognition. In other words, we now examined the extent to which Brain Age missed the variation in the brain MRI that could explain fluid cognition (for this particular issue, please see our response to Reviewer 3 Public Review #4).

Reviewer 1:

This is a reasonably good paper and the use of a commonality analysis is a nice contribution to understanding variance partitioning across different covariates. I have some comments that I believe the authors ought to address which mostly relate to clarity and interpretation.

Reviewer 1 Public Review #1:

First, from a conceptual point of view, the authors focus exclusively on cognition as a downstream outcome. I would suggest the authors nuance their discussion to provide broader considerations of the utility of their method and on the limits of interpretation of brain-age models more generally. Further, I think that since brain-age models by construction confound relevant biological variation with the accuracy of the regression models used to estimate them, there may be limits to the interpretation of (e.g.) the brain-age gap is as a dimensionless biomarker. This has also been discussed elsewhere (see e.g. https://academic.oup.com/brain/article/143/7/2312/5863667). I would suggest that the authors consider and comment on these issues.

Response: Thank you Reviewer 1 for pointing out these important issues. We addressed them in our response to Reviewer 1 Recommendations For The Authors #1 (see below).

Reviewer 1 Public Review #2

Second, from a methods perspective, there is not a sufficient explanation of the methodological procedures in the current manuscript to fully understand how the stacked regression models were constructed. Stacked models can be prone to overfitting when combined with cross-validation. This is because the predictions from the first-level models (i.e. the features that are provided to the second level 'stacked' models) contain information about the training set and the test set. If cross-validation is not done very carefully (e.g. using multiple hold-out sets), information leakage can easily occur at the second level. Unfortunately, there is not a sufficient explanation of the methodological procedures in the current manuscript to fully understand what was actually done. Please provide more information to enable the reader to better understand the stacked regression models. If the authors are not using an approach that fully preserves training and test separability, they need to do so.

Response: Thank you Reviewer 1. We addressed this issue in our response to Reviewer 1 Recommendations For The Authors #2 (see below). Briefly, we now made it clearer that training models for both non-stacked and stacked models did not involve the test set, ensuring that there was no data leakage between training and test sets.

Reviewer 1 Public Review #3

Please also provide an indication of the different regression strengths that were estimated across the different models and cross-validation splits. Also, how stable were the weights across splits?

Response: Thank you Reviewer 1. We addressed this issue in our response to Reviewer 1 Recommendations For The Authors #3 (see below).

Reviewer 1 Public Review #4:

Please provide more details about the task designs, MRI processing procedures that were employed on this sample in addition to the regression methods, and bias-correction methods used. For example, there are several different parameterisations of the elastic net, please provide equations to describe the method used here so that readers can easily determine how the regularisation parameters should be interpreted.

Response: Thank you Reviewer 1. We addressed this issue in our response to Reviewer 1 Recommendations For The Authors #5-#6. Briefly, we followed your advice and add all of the suggested details.

Reviewer 2 (Public Review):

Reviewer 2 Public Review Overall:

In this study, the authors aimed to evaluate the contribution of brain-age indices in capturing variance in cognitive decline and proposed an alternative index, brain-cognition, for consideration. The study employs suitable data and methods, albeit with some limitations, to address the research questions. A more detailed discussion of methodological limitations in relation to the study's aims is required. For instance, the current commonality analysis may not sufficiently address potential multicollinearity issues, which could confound the findings. Importantly, given that the study did not provide external validation for the indices, it is unclear how well the models would perform and generalize to other samples. This is particularly relevant to their novel index, brain-cognition, given that brain-age has been validated extensively elsewhere. In addition, the paper's rationale for using elastic net, which references previous fMRI studies, seemed somewhat unclear. The discussion could be more nuanced and certain conclusions appear speculative.

Response Thank you for your encouragement. We have now added discussion of methodological limitations (see below). Regarding potential multicollinearity issues, we addressed this comment using Ridge regressions (see our response to Reviewer 2 Recommendations For The Authors #2). Regarding external validation, we now added discussions about how consistency between our results and several recent studies that investigated similar issues with Brain Age in different populations (see Reviewer 2 Recommendations For The Authors #1). Regarding Brain Cognition, we also added previous studies showing similarly high prediction for cognition functioning (Dubois et al., 2018; Pat, Wang, Anney, et al., 2022; Rasero et al., 2021; Sripada et al., 2020; Tetereva et al., 2022; for review, see Vieira et al., 2022). We added a discussion about Elastic Net (see Reviewer 1 Recommendations For The Authors #6)

Discussion

“There are several potential limitations of this study. First, we conducted an investigation relying only on one dataset, the Human Connectome Project in Aging (HCP-A) (Bookheimer et al., 2019). While HCP-A used state-of-the-art MRI methodologies, covered a wide age range from 36 to 100 years old and used several task-fMRI from different tasks that are harder to find in other bigger databases (e.g., UK Biobank from Sudlow et al., 2015), several characteristics of HCP-A might limit the generalisability of our findings. For instance, the tasks used in task-based fMRI in HCP-A are not used widely in clinical settings (Horien et al., 2020). This might make it challenging to translate the approaches used here. Similarly, HCP-A also excluded participants with neurological conditions, possibly making their participants not representative of the general population. Next, while HCP-A’s sample size is not small (n=725 and 504 people, before and after exclusion, respectively), other datasets provide a much larger sample size (Horien et al., 2020). Similarly, HCP-A does not include younger populations. But as mentioned above, a study with a larger sample in older adults (Cole, 2020) and studies in younger populations (8-22 years old) (Butler et al., 2021; Jirsaraie, Kaufmann, et al., 2023) also found small effects of the adjusted Brain Age Gap in explaining cognitive functioning. And the disagreement between the predictive performance of age-prediction models and the utility of Brain Age found here is largely in line with the findings across different phenotypes seen in a recent systematic review (Jirsaraie, Gorelik, et al., 2023).”

Reviewer 2 Public Review #1:

The authors aimed to evaluate how brain-age and brain-cognition indices capture cognitive decline (as mentioned in their title) but did not employ longitudinal data, essential for calculating 'decline'. As a result, 'cognition-fluid' should not be used interchangeably with 'cognitive decline,' which is inappropriate in this context.

Response Thank you for raising this issue. We now no longer used the word ‘cognitive decline’.

Reviewer 2 Public Review #2:

In their first aim, the authors compared the contributions of brain-age and chronological age in explaining variance in cognition-fluid. Results revealed much smaller effect sizes for brain-age indices compared to the large effects for chronological age. While this comparison is noteworthy, it highlights a well-known fact: chronological age is a strong predictor of disease and mortality. Has the brain-age literature systematically overlooked this effect? If so, please provide relevant examples. They conclude that due to the smaller effect size, brain-age may lack clinical significance, for instance, in associations with neurodegenerative disorders. However, caution is required when speculating on what brain-age may fail to predict in the absence of direct empirical testing. This conclusion also overlooks extant brain-age literature: although effect sizes vary across psychiatric and neurological disorders, brain-age has demonstrated significant effects beyond those driven by chronological age, supporting its utility.

Response For aim 1, we focused our claims on cognitive functioning and not on any clinical significance for neurodegenerative disorders. We now made it clearer that the small effects of the Corrected Brain Age Gap in explaining fluid cognition of aging individuals found here are consistent with a study with a larger sample in older adults (Cole, 2020) and studies in younger populations (8-22 years old) (Butler et al., 2021; Jirsaraie, Kaufmann, et al., 2023).

We believe this issue of the utility of brain age on cognitive functioning vs neurological/psychological disorders requires another consideration, namely the discrepancy in the training and test samples typically used for studies focusing on neurological/psychological disorders. We made this point in the discussion now (see below).

Discussion

“There is a notable difference between studies investigating the utility of Brain Age in explaining cognitive functioning, including ours and others (e.g., Butler et al., 2021; Cole, 2020, 2020; Jirsaraie, Kaufmann, et al., 2023) and those explaining neurological/psychological disorders (e.g., Bashyam et al., 2020; Rokicki et al., 2021). That is, those Brain Age studies focusing on neurological/psychological disorders often build age-prediction models from MRI data of largely healthy participants (e.g., controls in a case-control design or large samples in a population-based design), apply the built age-prediction models to participants without vs. with neurological/psychological disorders and compare Brain Age indices between the two groups. This means that age-prediction models from Brain Age studies focusing on neurological/psychological disorders might be under-fitted when applied to participants with neurological/psychological disorders because they were built from largely healthy participants. And thus the difference in Brain Age indices between participants without vs. with neurological/psychological disorders might be confounded by the under-fitted age-prediction models (i.e., Brain Age may predict chronological age well for the controls, but not for those with a disorder). On the contrary, our study and other Brain Age studies focusing on cognitive functioning often build age-prediction models from MRI data of largely healthy participants and apply the built age-prediction models to participants who are also largely healthy. Accordingly, the age-prediction models for explaining cognitive functioning do not suffer from being under-fitted. We consider this as a strength, not a weakness of our study.”

Reviewer 2 Public Review #3:

The second aim's results reveal a discrepancy between the accuracy of their brain-age models in estimating age and the brain-age's capacity to explain variance in cognition-fluid. The authors suggest that if the ultimate goal is to capture cognitive variance, brain-age predictive models should be optimized to predict this target variable rather than age. While this finding is important and noteworthy, additional analyses are needed to eliminate potential confounding factors, such as correlated noise between the data and cognitive outcome, overfitting, or the inclusion of non-healthy participants in the sample. Optimizing brain-age models to predict the target variable instead of age could ultimately shift the focus away from the brain-age paradigm, as it might optimize for a factor differing from age.

Response We discussed the issue regarding the discrepancy between the accuracy of their brain-age models in estimating age and the brain-age's capacity to explain variance in fluid cognition in our response to Reviewer 3 Public Review #9 (see below). This issue is found to be widespread in a recent systematic review (Jirsaraie, Gorelik, et al., 2023). We now provided several strategies to mitigate this issue to improve the utility of Brain Age in explaining other phenotypes based on our current work and others, using different MRI modalities as well as modelling techniques (Bashyam et al., 2020; Jirsaraie, Kaufmann, et al., 2023; Rokicki et al., 2021).

Regarding potential confounding factors, we are not sure what the reviewer meant by “correlated noise between the data and cognitive outcome”. The current study, for instance, used ICA-FIX (Glasser et al., 2016) to remove noise in functional MRI. It is unclear how much ‘noise’ is still left and might confound our findings. More importantly, we are not sure how to define ‘noise’ as referred to by Reviewer 2 here. As for overfitting, we used nested cross-validation to ensure that training and test sets were separate from each other (see Reviewer 1 Recommendations For The Authors #2). If overfitting happened as suggested, we should see a ‘lower’ predictive performance of age-prediction and cognitive-prediction models since the models would fit well with the training set but would not generalise well to the test set. This is not what we found. The predictive performance of our age-prediction and cognitive-prediction models was high and consistent with the literature. Regarding the inclusion of non-healthy participants in the sample, we discussed this above in our response to Reviewer 2 Public Review #2).

Reviewer 2 Public Review #4:

While a primary goal in biomarker research is to obtain indices that effectively explain variance in the outcome variable of interest, thus favouring models optimized for this purpose, the authors' conclusion overlooks the potential value of 'generic/indirect' models, despite sacrificing some additional explained variance provided by ad-hoc or 'specific/direct' models. In this context, we could consider brain-age as a 'generic' index due to its robust out-of-sample validity and significant associations across various health outcome variables reported in the literature. In contrast, the brain-cognition index proposed in this study is presumed to be 'specific' as, without out-of-sample performance metrics and testing with different outcome variables (e.g., neurodegenerative disease), it remains uncertain whether the reported effect would generalize beyond predicting cognition-fluid, the same variable used to condition the brain-cognition model in this study. A 'generic' index like brain-age enables comparability across different applications based on a common benchmark (rather than numerous specific models) and can support explanatory hypotheses (e.g., "accelerated ageing") since it is grounded in its own biological hypothesis. Generic and specific indices are not mutually exclusive; instead, they may offer complementary information. Their respective utility may depend heavily on the context and research or clinical question.

Response Thank you Reviewer 2 for pointing out this important issue. Reviewer 1 (Recommendations For The Authors #4) and Reviewer 3 (Public Review #4) bought up a similar issue. We agreed with Reviewer 2 that both 'specific/direct' index and Brain Age as a 'generic/indirect' index have merit in their own right. We made a discussion about this issue in our response to Reviewer 3 Public Review #4 (please see this response below).

Briefly, in the revision, as opposed to treating Brain Cognition and Brain Age as separate biomarkers and comparing them, we treated the capability of Brain Cognition in capturing fluid cognition as the upper limit of Brain Age’s capability in capturing fluid cognition. In other words, we now examined the extent to which Brain Age missed the variation in the brain MRI that could explain fluid cognition. We also made a discussion about using our commonality approach to test for this missing variation in future work:

Discussion

“Finally, researchers should test how much Brain Age miss the variation in the brain MRI that could explain fluid cognition or other phenotypes of interest. As demonstrated here, one straightforward method is to build a prediction model using a phenotype of interest as the target (e.g., fluid cognition) and incorporate the predicted value of this model (e.g., Brain Cognition), along with Brain Age and chronological age, into a multiple regression for commonality analyses. The unique effect of this predicted value will inform the missing variation in the brain MRI from Brain Age. If this unique effect is large, then researchers might need to reconsider whether using Brain Age is appropriate for a particular phenotype of interest.”

Reviewer 2 Public Review #5:

The study's third aim was to evaluate the authors' new index, brain-cognition. The results and conclusions drawn appear similar: compared to brain-age, brain-cognition captures more variance in the outcome variable, cognition-fluid. However, greater context and discussion of limitations is required here. Given the nature of the input variables (a large proportion of models in the study were based on fMRI data using cognitive tasks), it is perhaps unsurprising that optimizing these features for cognition-fluid generates an index better at explaining variance in cognition-fluid than the same features used to predict age. In other words, it is expected that brain-cognition would outperform brain-age in explaining variance in cognition-fluid since the former was optimized for the same variable in the same sample, while brain-age was optimized for age. Consequently, it is unclear if potential overfitting issues may inflate the brain-cognition's performance. This may be more evident when the model's input features are the ones closely related to cognition, e.g., fMRI tasks. When features were less directly related to cognitive tasks, e.g., structural MRI, the effect sizes for brain-cognition were notably smaller (see 'Total Brain Volume' and 'Subcortical Volume' models in Figure 6). This observation raises an important feasibility issue that the authors do not consider. Given the low likelihood of having task-based fMRI data available in clinical settings (such as hospitals), estimating a brain-cognition index that yields the large effects discussed in the study may be challenged by data scarcity.

Response Given the use of nested cross-validation, we do not consider the good predictive performance of Brain Cognition found here as overfitting. In fact, we found a similar level of predictive performance of Brain Cognition on another database with younger participants in the past (Tetereva et al., 2022). However, we agreed with Reviewer 2 that the prediction of fluid cognition might be driven by MRI modalities that are different from those that drive the prediction of chronological age. In our own work with other age groups, including young adults (Tetereva et al., 2022) and children (Pat, Wang, Anney, et al., 2022), cognitive functioning seems to be predicted well from task-based functional MRI. And Reviewer 2 is right that task-based fMRI is not commonly used in clinics, making it harder to translate our results. However, given our results, clinicians should be encouraged to use task-based fMRI if their goal is to predict cognitive functioning. Nevertheless, as suggested, we listed data scarcity as one of the limitations of our approach.

Discussion “For instance, the tasks used in task-based fMRI in HCP-A are not used widely in clinical settings (Horien et al., 2020). This might make it challenging to translate the approaches used here.”

Reviewer 2 Public Review #6:

This study is valuable and likely to be useful in two main ways. First, it can spur further research aimed at disentangling the lack of correspondence reported between the accuracy of the brain-age model and the brain-age's capacity to explain variance in fluid cognitive ability. Second, the study may serve, at least in part, as an illustration of the potential pros and cons of using indices that are specific and directly related to the outcome variable versus those that are generic and only indirectly related.

Response We are thankful for the encouragement. For the discrepancy between the predictive performance of age-prediction models and the utility of Brain Age indices as a biomarker for fluid cognition, we made a detailed discussion in our response to Reviewer 3 Public Review #9. More specifically, to ensure that readers can benefit from our findings, we made suggestions on how to ensure the utility of Brain Age indices as a biomarker for other phenotypes by drawing from our own strategy, as well as strategies used by Rokicki and colleagues (2021), Jirsaraie and colleagues (2023) and Bashyam and colleagues (2020).

As for the pros and cons between generic vs specific biomarkers, we made a detailed discussion in our response to Reviewer 3 Public Review #4. We also made some suggestions on how to make use of the difference in the ability between generic vs specific biomarkers (see Reviewer 2 Public Review #4, above).

Reviewer 2 Public Review #7:

Overall, the authors effectively present a clear design and well-structured procedure; however, their work could have been enhanced by providing more context for both the brain-age and brain-cognition indices, including a discussion of key concepts in the brain-age paradigm, which acknowledges that chronological age strongly predicts negative health outcomes, but crucially, recognizes that ageing does not affect everyone uniformly. Capturing this deviation from a healthy norm of ageing is the key brain-age index. This lack of context was mirrored in the presentation of the four brain-age indices provided, as it does not refer to how these indices are used in practice. In fact, there is no mention of a more common way in which brain-age is implemented in statistical analyses, which involves the use of brain-age delta as the variable of interest, along with linear and non-linear terms of age as covariates. The latter is used to account for the regression-to-the-mean effect. The 'corrected brain-age delta' the authors use does not include a non-linear term, which perhaps is an additional reason (besides the one provided by the authors) as to why there may be small, but non-zero, common effects of both age and brain-age in the 'corrected brain-age delta' index commonality analysis. The context for brain-cognition was even more limited, with no reference to any existing literature that has explored direct brain-cognitive markers, such as brain-cognition.

Response Regarding Brain Age and negative health outcomes, we addressed this in our response to Reviewer 1 Recommendations For The Authors #1 (see below). Briefly, we now discussed (1) the consistency between our findings on fluid cognition and other recent works on negative health outcomes, (2) the differences between Brain Age studies focusing on negative health outcomes vs. cognitive functioning and (3) suggested solutions to optimise the utility of brain age for both cognitive functioning and negative health outcomes.

Regarding how Brain Age was used in practice, we addressed this in our response to Reviewer 3 Public Review #2 (see below). Our argument resonates Butler and colleagues’ (2021) suggestion that the common practice for Brain Age analysis should be re-evaluated: “The MBAG and performance on the complex cognition tasks were not associated (r =  .01, p = 0.71). These results indicate that the association between cognition and the BAG are driven by the association between age and cognitive performance. As such, it is critical that readers of past literature note whether or not age was controlled for when testing for effects on the BAG, as this has not always been common practice (e.g., Beheshti et al., 2018; Cole, Underwood, et al., 2017; Franke et al., 2015; Gaser et al., 2013; Liem et al., 2017; Nenadi c et al., 2017; Steffener et al., 2016). (p. 4097).”

Importantly, we also implemented “brain-age delta as the variable of interest, along with linear and non-linear terms of age as covariates” in our additional analyses along with other implementations (see Reviewer 2 Recommendations For The Authors #3). Of particular note, we found that adding a non-linear term (i.e., a quadratic term for chronological age) barely changed the results of commonality analyses.

We now wrote this paragraph to recommend how future research should implement Brain Age:

Discussion

“First, they have to be aware of the overlap in variation between Brain Age and chronological age and should focus on the contribution of Brain Age over and above chronological age. Using Brain Age Gap will not fix this. Butler and colleagues (2021) recently highlighted this point, “These results indicate that the association between cognition and the BAG are driven by the association between age and cognitive performance. As such, it is critical that readers of past literature note whether or not age was controlled for when testing for effects on the BAG, as this has not always been common practice (p. 4097).” Similar to their recommendation (Butler et al., 2021), we suggest future work focus on Corrected Brain Age Gap or, better, unique effects of Brain Age indices after controlling for chronological age in multiple regressions. In the case of fluid cognition, the unique effects might be too small to be clinically meaningful as shown here and previously (Butler et al., 2021; Jirsaraie, Kaufmann, et al., 2023). “

Regarding brain cognition, we now expanded our explanation about Brain Cognition on how it might be relevant to Brain Age and on Brain Cognition’s predictive performance found previously.

Introduction

“Third and finally, certain variation in the brain MRI is related to fluid cognition, but to what extent does Brain Age not capture this variation? To estimate the variation in the brain MRI that is related to fluid cognition, we could build prediction models that directly predict fluid cognition (i.e., as opposed to chronological age) from brain MRI data. Previous studies found reasonable predictive performances of these cognition-prediction models, built from certain MRI modalities (Dubois et al., 2018; Pat, Wang, Anney, et al., 2022; Rasero et al., 2021; Sripada et al., 2020; Tetereva et al., 2022; for review, see Vieira et al., 2022). Analogous to Brain Age, we called the predicted values from these cognition-prediction models, Brain Cognition. The strength of an out-of-sample relationship between Brain Cognition and fluid cognition reflects variation in the brain MRI that is related to fluid cognition and, therefore, indicates the upper limit of Brain Age’s capability in capturing fluid cognition. Consequently, the unique effects of Brain Cognition that explain fluid cognition beyond Brain Age and chronological age indicate what is missing from Brain Age -- the amount of co-variation between brain MRI and fluid cognition that cannot be captured by Brain Age.”

Discussion

“Third, by introducing Brain Cognition, we showed the extent to which Brain Age indices were not able to capture the variation of brain MRI that is related to fluid cognition. Brain Cognition, from certain cognition-prediction models such as the stacked models, has relatively good predictive performance, consistent with previous studies (Dubois et al., 2018; Pat, Wang, Anney, et al., 2022; Rasero et al., 2021; Sripada et al., 2020; Tetereva et al., 2022; for review, see Vieira et al., 2022).”

Reviewer 2 Public Review #8:

While this paper delivers intriguing and thought-provoking results, it would benefit from recognizing the value that both approaches--brain-age indices and more direct, specific markers like brain-cognition--can contribute to the field.

Response Thank you so much for recognising the value of our work. As we mentioned above in our response to Reviewer 2 Public Review #4 and #6, we made some suggestions on how to make use of the difference in the ability between generic vs specific biomarkers.

Reviewer 3 (Public Review):

Reviewer 3 Public Review Overall:

The main question of this article is as follows: "To what extent does having information on brain-age improve our ability to capture declines in fluid cognition beyond knowing a person's chronological age?" While this question is worthwhile, considering that there is considerable confusion in the field about the nature of brain-age, the authors are currently missing an opportunity to convey the inevitability of their results, given how brain-age and the brain-age gap are calculated. They also argue that brain-cognition is somehow superior to brain-age, but insufficient evidence is provided in support of this claim.

Response We addressed the concerns below. The inevitability of our results is not obvious to many researchers who might be interested in Brain Age. We hope our findings might make many issues surrounding Brain Age more obvious, and we now make many suggestions on how to address some of these issues. We no longer argue that Brain Cognition is superior to Brain Age (Reviewer 3 Public Review #4). Rather, we treated the capability of Brain Cognition in capturing fluid cognition as the upper limit of Brain Age’s capability in capturing fluid cognition. We used the unique effects of Brain Cognition that explain fluid cognition beyond Brain Age and chronological age to indicate how much Brain Age misses the variation in the brain MRI that could explain fluid cognition.

Specific comments follow:

Reviewer 3 Public Review #1:

• "There are many adjustments proposed to correct for this estimation bias" (p3). Regression to the mean is not a sign of bias. Any decent loss function will result in over-predicting the age of younger individuals and under-predicting the age of older individuals. This is a direct result of minimizing an error term (e.g., mean squared error). Therefore, it is inappropriate to refer to regression to the mean as a sign of bias. This misconception has led to a great deal of inappropriate analyses, including "correcting" the brain age gap by regressing out age.

Response: Thank you so much for raising this issue. We used the word ‘bias’ following many articles in the field. For instance,

de Lange and Cole (2020) wrote: “brain-age estimation also involves a frequently observed bias: brain age is overestimated in younger subjects and underestimated in older subjects, while brain age for participants with an age closer to the mean age (of the training dataset) are predicted more accurately (Cole, Le, Kuplicki, McKinney, Yeh, Thompson, Paulus, Investigators, et al., 2018, Liang, Zhang, Niu, 2019, Niu, Zhang, Kounios, Liang, 2019, Smith, Vidaurre, Alfaro-Almagro, Nichols, Miller, 2019).”

Cole (2020) wrote: “As recent research has highlighted a proportional bias in brain-age calculation, whereby the difference between chronological age and brain-predicted age is negatively correlated with chronological age (Le et al., 2018, Liang et al., 2019, Smith et al., 2019), an age-bias correction procedure was used. This entailed calculating the regression line between age (predictor) and brain-predicted age (outcome) in the training set, then using the slope (i.e., coefficient) and intercept of that line to adjust brain-predicted age values in the testing set (by subtracting the intercept and then dividing by the slope). After applying the age-bias correction the brain-predicted age difference (brain-PAD) was calculated; chronological age subtracted from brain-predicted age.”

Beheshiti and colleagues (2019) used bias in their title: “Bias-adjustment in neuroimaging-based brain age frameworks: a robust scheme”

More recently, Cumplido-Mayoral and colleagues (2023) wrote: “As recent research has shown that brain-age estimation involves a proportional bias (de Lange et al., 2020a; Le et al., 2018; Liang et al., 2019; Smith et al., 2019), we applied a well-established age-bias correction procedure to our data (de Lange et al., 2020a; Le et al., 2018).”

Still, we agree with Reviewer 3 that using ‘bias’ might lead to misinterpretation. As Butler and colleagues (Butler et al., 2021) pointed out, ”It is important to note that regression toward the mean is not a failure, but a feature, of regression and related methods.“ We rewrote the paragraph and clarified the “regression towards the mean” issue. We no longer used the word “bias” here:

Introduction

“Note researchers often subtract chronological age from Brain Age, creating an index known as Brain Age Gap (Franke & Gaser, 2019). A higher value of Brain Age Gap is thought to reflect accelerated/premature aging. Yet, given that Brain Age Gap is calculated based on both Brain Age and chronological age, Brain Age Gap still depends on chronological age (Butler et al., 2021). If, for instance, Brain Age was based on prediction models with poor performance and made a prediction that everyone was 50 years old, individual differences in Brain Age Gap would then depend solely on chronological age (i.e., 50 minus chronological age). Moreover, Brain Age is known to demonstrate the “regression towards the mean” phenomenon (Stigler, 1997). More specifically, because Brain Age is a predicted value of a regression model that predicts chronological age, Brain Age is usually shrunk towards the mean age of samples used for training the model (Butler et al., 2021; de Lange & Cole, 2020; Le et al., 2018). Accordingly, Brain Age predicts chronological age more accurately for individuals who are closer to the mean age while overestimating younger individuals’ chronological age and underestimating older individuals’ chronological age. There are many adjustments proposed to correct for the age dependency, but the outcomes tend to be similar to each other (Beheshti et al., 2019; de Lange & Cole, 2020; Liang et al., 2019; Smith et al., 2019). These adjustments can be applied to Brain Age and Brain Age Gap, creating Corrected Brain Age and Corrected Brain Age Gap, respectively. Corrected Brain Age Gap in particular is viewed as being able to control for age dependency (Butler et al., 2021). Here, we tested the utility of different Brain Age calculations in capturing fluid cognition, over and above chronological age.”

Reviewer 3 Public Review #2:

• "Corrected Brain Age Gap in particular is viewed as being able to control for both age dependency and estimation biases (Butler et al., 2021)" (p3). This summary is not accurate as Butler and colleagues did not use the words "corrected" and "biases" in this context. All that authors say in that paper is that regressing out age from the brain age gap - which is referred to as the modified brain age gap (MBAG) - makes it so that the modified brain age gap is not dependent on age, which is true. This metric is meaningless, though, because it is the variance left over after regressing out age from residuals from a model that was predicting age. If it were not for the fact that regression on residuals is not equivalent to multiple regression (and out of sample estimates), MBAG would be a vector of zeros. Upon reading the Methods, I noticed that the authors use a metric from Le et al. (2018) for the "Corrected Brain Age Gap". If they cite the Butler et al. (2021) paper, I highly recommend sticking with the same notation, metrics and terminology throughout. That would greatly help with the interpretability of the present manuscript, and cross-comparisons between the two.

Response: We thank Reviewer 3 for pointing out the issues surrounding our choices of wording: "corrected" and "biases". We share the same frustration with Reviewer 3 in that different brain-age articles use different terminologies, and we tried to make sure our readers understand our calculations of Brain Age indices in order to compare our results with previous work.

We commented on the word “bias” in our response to Reviewer 3 Public Review #1 above and refrained from using this word in the revised manuscript. Here we commented on the use of the word “Corrected Brain Age Gap". And by doing so, we clarified how we calculated it.

Reviewer 3 is right that we cited the work of Butler and colleagues (2021), but wasn’t accurate to say that we used “a metric from Le et al. (2018) for the "Corrected Brain Age Gap". We, instead, used a method described in de Lange and Cole’s (2020) work. We now added equations to explain this method in our Materials and Method section (see below).

It is important to note that Butler and colleagues (2021) did not come up with any adjustment methods. Instead, Butler and colleagues (2021) discussed three adjustment methods:

1) A method proposed by Beheshiti and colleagues (2019). Butler and colleagues (2021) called the result of this method, Modified Brain Age Gap (MBAG). Importantly, Butler and colleagues (2021) discouraged the use of this method due to “researchers misinterpreting the reduced variability of the MBAG as an improvement in prediction accuracy.” Accordingly in our article, we performed methods (2) and (3) below.

2) A method proposed by de Lange and Cole (2020). We used this method in our article (see below for the equations). Briefly, we first fit a regression line predicting the Brain Age from a chronological age in each training set. We then used the slope and intercept of this regression line to adjust Brain Age in the corresponding test set, resulting in an adjusted index of Brain Age. Butler and colleagues (2021) called this index, “Revised Predicted Age.”, while de Lange and Cole’s (2020) originally called this Corrected Brain Age, “Corrected Predicted Age”. Butler and colleagues (2021) then subtracted the chronological age from this index and called it, “Revised Brain Age Gap (RBAG)”. We would like to follow the original terminology, but we do not want to use the word “Predicted Age” since chronological age can be predicted by other variables beyond the brain. We then settled with the word, "Corrected Brain Age" and “Corrected Brain Age Gap". We listed the terminologies used in the past in our article (see below).

3) A method proposed by Le and colleagues (2018). Here, Butler and colleagues (2021) referred to one of the approaches done by Le and colleagues: “include age as a regressor when doing follow-up analyses.” Essentially this is what we did for the commonality analysis. Le and colleagues (2018)’ approach is the same as examining the unique effects of Brain Age in a multiple regression analysis with Chronological Age and Brain Age as regressors.

While indexes from de Lange and Cole’s (2020) and Le and colleagues’ (2018) methods show poor performance in capturing fluid cognition in the current work, we need to stress that many research groups do not believe that these methods are meaningless. In fact, de Lange and Cole’s method (2020) is one of the most commonly implemented methods that can be seen elsewhere (e.g., Cole et al., 2020; Cumplido-Mayoral et al., 2023; Denissen et al., 2022). This index just does not seem to work well in the case of fluid cognition.

Here is how we described how we calculated Brain Age indexes in the revised manuscript:

Methods

“ Brain Age calculations: Brain Age, Brain Age Gap, Corrected Brain Age and Corrected Brain Age Gap In addition to Brain Age, which is the predicted value from the models predicting chronological age in the test sets, we calculated three other indices to reflect the estimation of brain aging. First, Brain Age Gap reflects the difference between the age predicted by brain MRI and the actual, chronological age. Here we simply subtracted the chronological age from Brain Age:

Brain Age Gapi = Brain Agei - chronological agei , (2)

where i is the individual. Next, to reduce the dependency on chronological age (Butler et al., 2021; de Lange & Cole, 2020; Le et al., 2018), we applied a method described in de Lange and Cole’s (2020), which was implemented elsewhere (Cole et al., 2020; Cumplido-Mayoral et al., 2023; Denissen et al., 2022):

In each outer-fold training set: Brain Agei = 0 + 1 chronological agei + εi, (3)

Then in the corresponding outer-fold test set: Corrected Brain Agei = (Brain Agei - 0)/1, (4)

That is, we first fit a regression line predicting the Brain Age from a chronological age in each outer-fold training set. We then used the slope (1) and intercept (0) of this regression line to adjust Brain Age in the corresponding outer-fold test set, resulting in Corrected Brain Age. Note de Lange and Cole (2020) called this Corrected Brain Age, “Corrected Predicted Age”, while Butler (2021) called it “Revised Predicted Age.”

Lastly, we computed Corrected Brain Age Gap by subtracting the chronological age from the Corrected Brain Age (Butler et al., 2021; Cole et al., 2020; de Lange & Cole, 2020; Denissen et al., 2022):

Corrected Brain Age Gap = Corrected Brain Age - chronological age, (5)

Note Cole and colleagues (2020) called Corrected Brain Age Gap, “brain-predicted age difference (brain-PAD),” while Butler and colleagues (2021) called this index, “Revised Brain Age Gap”.

Reviewer 3 Public Review #3:

• "However, the improvement in predicting chronological age may not necessarily make Brain Age to be better at capturing Cognitionfluid. If, for instance, the age-prediction model had the perfect performance, Brian Age Gap would be exactly zero and would have no utility in capturing Cognitionfluid beyond chronological age" (p3). I largely agree with this statement. I would be really careful to distinguish between brain-age and the brain-age gap here, as the former is a predicted value, and the latter is the residual times -1 (i.e., predicted age - age). Therefore, together they explain all of the variance in age. Changing the first sentence to refer to the brain-age gap would be more accurate in this context. The brain-age gap will never be exactly zero, though, even with perfect prediction on the training set, because subjects in the testing set are different from the subjects in the training set.

Response: Thank you so much for pointing this out. We agree to change “Brain Age” to “Brain Age Gap” in the mentioned sentence.

Reviewer 3 Public Review #4:

• "Can we further improve our ability to capture the decline in cognitionfluid by using, not only Brain Age and chronological age, but also another biomarker, Brain Cognition?". This question is fundamentally getting at whether a predicted value of cognition can predict cognition. Assuming the brain parameters can predict cognition decently, and the original cognitive measure that you were predicting is related to your measure of fluid cognition, the answer should be yes. Upon reading the Methods, it became clear that the cognitive variable in the model predicting cognition using brain features (to get predicted cognition, or as the authors refer to it, brain-cognition) is the same as the measure of fluid cognition that you are trying to assess how well brain-cognition can predict. Assuming the brain parameters can predict fluid cognition at all, it is then inevitable that brain-cognition will predict fluid cognition. Therefore, it is inappropriate to use predicted values of a variable to predict the same variable.

Response: Thank you Reviewer 3 for pointing out this important issue. Reviewer 1 (Recommendations For The Authors #4) and Reviewer 2 (Public Review #4) bought up a similar issue. While Reviewer 3 felt that “it is inappropriate to use predicted values of a variable to predict the same variable,“ Reviewer 2 viewed Brain Cognition as a 'specific/direct' index and Brain Age as a 'generic/indirect' index. And both have merit in their own right.

Similar to Reviewer 2, we believe that the specific index is as important and has commonly been used elsewhere in the context of biomarkers. For instance, to obtain neuroimaging biomarkers for Alzheimer’s, neuroimaging researchers often build a predictive model to predict Alzheimer's diagnosis (Khojaste-Sarakhsi et al., 2022). In fact, outside of neuroimaging, polygenic risk scores (PRSs) in genomics are often used following “to use predicted values of a variable to predict the same variable” (Choi et al., 2020). For instance, a PRS of ADHD that indicates the genetic liability to develop ADHD is based on genome-wide association studies of ADHD (Demontis et al., 2019).

Still, we now agreed that it may not be fair to compare the performance of a specific index (Brain Cognition) and a generic index (Brain Age) directly (as pointed out by Reviewer 3 Public Review #6 below). Accordingly, in the revision, as opposed to treating Brain Cognition and Brain Age as separate biomarkers and comparing them, we treated the capability of Brain Cognition in capturing fluid cognition as the upper limit of Brain Age’s capability in capturing fluid cognition. In other words, the strength of an out-of-sample relationship between Brain Cognition and fluid cognition reflects variation in the brain MRI that is related to fluid cognition. And consequently, the unique effects of Brain Cognition that explain fluid cognition beyond Brain Age and chronological age indicate what is missing from Brain Age -- the amount of co-variation between brain MRI and fluid cognition that cannot be captured by Brain Age. According to Reviewer 2, a generic index (Brain Age) “sacrificed some additional explained variance provided” compared to a specific index (Brain Cognition). Here, we used the commonality analyses to quantify how much scarifying was made by Brain Age. See below for the re-conceptualisation of Brain Age vs. Brain Cognition in the revision:

Abstract

“Lastly, we tested how much Brain Age missed the variation in the brain MRI that could explain fluid cognition. To capture this variation in the brain MRI that explained fluid cognition, we computed Brain Cognition, or a predicted value based on prediction models built to directly predict fluid cognition (as opposed to chronological age) from brain MRI data. We found that Brain Cognition captured up to an additional 11% of the total variation in fluid cognition that was missing from the model with only Brain Age and chronological age, leading to around a 1/3-time improvement of the total variation explained.”

Introduction:

“Third and finally, certain variation in the brain MRI is related to fluid cognition, but to what extent does Brain Age not capture this variation? To estimate the variation in the brain MRI that is related to fluid cognition, we could build prediction models that directly predict fluid cognition (i.e., as opposed to chronological age) from brain MRI data. Previous studies found reasonable predictive performances of these cognition-prediction models, built from certain MRI modalities (Dubois et al., 2018; Pat, Wang, Anney, et al., 2022; Rasero et al., 2021; Sripada et al., 2020; Tetereva et al., 2022; for review, see Vieira et al., 2022). Analogous to Brain Age, we called the predicted values from these cognition-prediction models, Brain Cognition. The strength of an out-of-sample relationship between Brain Cognition and fluid cognition reflects variation in the brain MRI that is related to fluid cognition and, therefore, indicates the upper limit of Brain Age’s capability in capturing fluid cognition. Consequently, the unique effects of Brain Cognition that explain fluid cognition beyond Brain Age and chronological age indicate what is missing from Brain Age -- the amount of co-variation between brain MRI and fluid cognition that cannot be captured by Brain Age.”

“Finally, we investigated the extent to which Brain Age indices missed the variation in the brain MRI that could explain fluid cognition. Here, we tested Brain Cognition’s unique effects in multiple regression models with a Brain Age index, chronological age and Brain Cognition as regressors to explain fluid cognition.“

Discussion

“Third, how much does Brain Age miss the variation in the brain MRI that could explain fluid cognition? Brain Age and chronological age by themselves captured around 32% of the total variation in fluid cognition. But, around an additional 11% of the variation in fluid cognition could have been captured if we used the prediction models that directly predicted fluid cognition from brain MRI.

“Third, by introducing Brain Cognition, we showed the extent to which Brain Age indices were not able to capture the variation of brain MRI that is related to fluid cognition. Brain Cognition, from certain cognition-prediction models such as the stacked models, has relatively good predictive performance, consistent with previous studies (Dubois et al., 2018; Pat, Wang, Anney, et al., 2022; Rasero et al., 2021; Sripada et al., 2020; Tetereva et al., 2022; for review, see Vieira et al., 2022). We then examined Brain Cognition using commonality analyses (Nimon et al., 2008) in multiple regression models having a Brain Age index, chronological age and Brain Cognition as regressors to explain fluid cognition. Similar to Brain Age indices, Brain Cognition exhibited large common effects with chronological age. But more importantly, unlike Brain Age indices, Brain Cognition showed large unique effects, up to around 11%. The unique effects of Brain Cognition indicated the amount of co-variation between brain MRI and fluid cognition that was missed by a Brain Age index and chronological age. This missing amount was relatively high, considering that Brain Age and chronological age together explained around 32% of the total variation in fluid cognition. Accordingly, if a Brain Age index was used as a biomarker along with chronological age, we would have missed an opportunity to improve the performance of the model by around one-third of the variation explained.”

Reviewer 3 Public Review #5:

• "However, Brain Age Gap created from the lower-performing age-prediction models explained a higher amount of variation in Cognitionfluid. For instance, the top performing age-prediction model, "Stacked: All excluding Task Contrast", generated Brain Age and Corrected Brain Age that explained the highest amount of variation in Cognitionfluid, but, at the same time, produced Brian Age Gap that explained the least amount of variation in Cognitionfluid" (p7). This is an inevitable consequence of the following relationship between predicted values and residuals (or residuals times -1): y=(y-y ̂ )+y ̂. Let's say that age explains 60% of the variance in fluid cognition, and predicted age (y ̂) explains 40% of the variance in fluid cognition. Then the brain age gap (-(y-y ̂)) should explain 20% of the variance in fluid cognition. If by "Corrected Brain Age" you mean the modified predicted age from Butler et al (2021), the "Corrected Brain Age" result is inevitable because the modified predicted age is essentially just age with a tiny bit of noise added to it. From Figure 4, though, this does not seem to be the case, because the lower left quadrant in panel (a) should be flat and high (about as high as the predictive value of age for fluid cognition). So it is unclear how "Corrected Brain Age" is calculated. It looks like you might be regressing age out of brain-age, though from your description in the Methods section, it is not totally clear. Again, I highly recommend using the terminology and metrics of Butler et al (2021) throughout to reduce confusion. Please also clarify how you used the slope and intercept. In general, given how brain-age metrics tend to be calculated, the following conclusion is inevitable: "As before, the unique effects of Brain Age indices were all relatively small across the four Brain Age indices and across different prediction models" (p10).

Response: We agreed that the results are ‘inevitable’ due to the transformations from Brain Age to other Brain Age indices. However, the consequences of these transformations may not be very clear to readers who are not very familiar with Brain Age literature and to the community at large who think about the implications of Brain Age. This is appreciated by Reviewer 1, who mentioned “While the main message will not come as a surprise to anyone with hands-on experience of using brain-age models, I think it is nonetheless an important message to convey to the community.”

Note we made clarifications on how we calculated each of the Brain Age indices above (see<br /> Reviewer 3 Public Review #2), including how we used the slope and intercept. We chose the terminology closer to the one originally used by de Lange and Cole (2020) and now listed many terminologies others have used to refer to this transformation.

Reviewer 3 Public Review #6:

"On the contrary, the unique effects of Brain Cognition appeared much larger" (p10). This is not a fair comparison if you do not look at the unique effects above and beyond the cognitive variable you predicted in your brain-cognition model. If your outcome measure had been another metric of cognition other than fluid cognition, you would see that brain-cognition does not explain any additional variance in this outcome when you include fluid cognition in the model, just as brain-age would not when including age in the model (minus small amounts due to penalization and out-of-sample estimates). This highlights the fact that using a predicted value to predict anything is worse than using the value itself.

Response Please see our response to Reviewer 3 Public Review #4 above. Briefly, we no long made this comparison. Instead, we now viewed the unique effects of Brain Cognition as a way to test how much Brain Age missed the variation in the brain MRI that could explain fluid cognition.

Reviewer 3 Public Review #7:

"First, how much does Brain Age add to what is already captured by chronological age? The short answer is very little" (p12). This is a really important point, but the paper requires an in-depth discussion of the inevitability of this result, as discussed above.

Response We agree that the tight relationship between Brain Age and chronological age is inevitable. We mentioned this from the get-go in the introduction:

Introduction “Accordingly, by design, Brain Age is tightly close to chronological age. Because chronological age usually has a strong relationship with fluid cognition, to begin with, it is unclear how much Brain Age adds to what is already captured by chronological age.”

To make this point obvious, we quantified the overlap between Brain Age and chronological age using the commonality analysis. We hope that our effort to show the inevitability of this overlap can make people more careful when designing studies involving Brain Age.

Reviewer 3 Public Review #8:

"Third, do we have a solution that can improve our ability to capture Cognitionfluid from brain MRI? The answer is, fortunately, yes. Using Brain Cognition as a biomarker, along with chronological age, seemed to capture a higher amount of variation in Cognitionfluid than only using Brain Age" (p12). I suggest controlling for the cognitive measure you predicted in your brain-cognition model. This will show that brain-cognition is not useful above and beyond cognition, highlighting the fact that it is not a useful endeavor to be using predicted values.

Response This point is similar to Reviewer 3 Public Review #6. Again please see our response to Reviewer 3 Public Review #4 above. Briefly, we no long made this comparison and said whether Brain Cognition is ‘better’ than Brain Age. Instead, we now viewed the unique effects of Brain Cognition as a way to test how much Brain Age missed the variation in the brain MRI that could explain fluid cognition.

Reviewer 3 Public Review #9:

"Accordingly, a race to improve the performance of age-prediction models (Baecker et al., 2021) does not necessarily enhance the utility of Brain Age indices as a biomarker for Cognitionfluid. This calls for a new paradigm. Future research should aim to build prediction models for Brian Age indices that are not necessarily good at predicting age, but at capturing phenotypes of interest, such as Cognitionfluid and beyond" (p13). I whole-heartedly agree with the first two sentences, but strongly disagree with the last. Certainly your results, and the underlying reason as to why you found these results, calls for a new paradigm (or, one might argue, a pre-brain-age paradigm). As of now, your results do not suggest that researchers should keep going down the brain-age path. While it is difficult to prove that there is no transformation of brain-age or the brain-age gap that will be useful, I am nearly sure this is true from the research I have done. If you would like to suggest that the field should continue down this path, I suggest presenting a very good case to support this view.

Response Thank you for your comments on this issue.

Since the submission of our manuscript, other researchers also made a similar observation regarding the disagreement between the predictive performance of age-prediction models and the utility of Brain Age. For instance, in their systematic review, Jirasarie and colleagues (2023, p7) wrote this statement, “Despite mounting evidence, there is a persisting assumption across several studies that the most accurate brain age models will have the most potential for detecting differences in a given phenotype of interest. As a point of illustration, seven of the twenty studies in this review only evaluated the utility of their most accurate model, which in all cases was trained using multimodal features. This approach has also led to researchers to exclusively use T1-weighted and diffusion-weighted MRI scans when developing brain age models36 since such modalities have been shown to have the largest contribution to a model’s predictive power.2,67 However, our review suggests that model accuracy does not necessarily provide meaningful insight about clinical utility (e.g., detection of age-related pathology). Taken with prior studies,16,17 it appears that the most accurate models tend to not be the most useful.”

We now discussed the disagreement between the predictive performance of age-prediction models and the utility of Brain Age, not only in the context of cognitive functioning (Jirsaraie, Kaufmann, et al., 2023) but also in the context of neurological/psychological disorders (Bashyam et al., 2020; Rokicki et al., 2021). Following Reviewer 3’s suggestion, we also added several possible strategies to mitigate this problem of Brain Age, used by us and other groups. Please see below.

Discussion:

“This discrepancy between the predictive performance of age-prediction models and the utility of Brain Age indices as a biomarker is consistent with recent findings (for review, see Jirsaraie, Gorelik, et al., 2023), both in the context of cognitive functioning (Jirsaraie, Kaufmann, et al., 2023) and neurological/psychological disorders (Bashyam et al., 2020; Rokicki et al., 2021). For instance, combining different MRI modalities into the prediction models, similar to our stacked models, often lead to the highest performance of age-prediction models, but does not likely explain the highest variance across different phenotypes, including cognitive functioning and beyond (Jirsaraie, Gorelik, et al., 2023).”

“Next, researchers should not select age-prediction models based solely on age-prediction performance. Instead, researchers could select age-prediction models that explained phenotypes of interest the best. Here we selected age-prediction models based on a set of features (i.e., modalities) of brain MRI. This strategy was found effective not only for fluid cognition as we demonstrated here, but also for neurological and psychological disorders as shown elsewhere (Jirsaraie, Gorelik, et al., 2023; Rokicki et al., 2021). Rokicki and colleagues (2021), for instance, found that, while integrating across MRI modalities led to age-prediction models with the highest age-prediction performance, using only T1 structural MRI gave age-prediction models that were better at classifying Alzheimer’s disease. Similarly, using only cerebral blood flow gave age-prediction models that were better at classifying mild/subjective cognitive impairment, schizophrenia and bipolar disorder.

As opposed to selecting age-prediction models based on a set of features, researchers could also select age-prediction models based on modelling methods. For instance, Jirsaraie and colleagues (2023) compared gradient tree boosting (GTB) and deep-learning brain network (DBN) algorithms in building age-prediction models. They found GTB to have higher age-prediction performance but DBN to have better utility in explaining cognitive functioning. In this case, an algorithm with better utility (e.g., DBN) should be used for explaining a phenotype of interest. Similarly, Bashyam and colleagues (2020) built different DBN-based age-prediction models, varying in age-prediction performance. The DBN models with a higher number of epochs corresponded to higher age-prediction performance. However, DBN-based age-prediction models with a moderate (as opposed to higher or lower) number of epochs were better at classifying Alzheimer’s disease, mild cognitive impairment and schizophrenia. In this case, a model from the same algorithm with better utility (e.g., those DBN with a moderate epoch number) should be used for explaining a phenotype of interest. Accordingly, this calls for a change in research practice, as recently pointed out by Jirasarie and colleagues (2023, p7), “Despite mounting evidence, there is a persisting assumption across several studies that the most accurate brain age models will have the most potential for detecting differences in a given phenotype of interest”. Future neuroimaging research should aim to build age-prediction models that are not necessarily good at predicting age, but at capturing phenotypes of interest.”

Reviewer #1 (Recommendations For The Authors):

In this paper, the authors evaluate the utility of brain age derived metrics for predicting cognitive decline using the HCP aging dataset by performing a commonality analysis in a downstream regression. The main conclusion is that brain age derived metrics do not explain much additional variation in cognition over and above what is already explained by age. The authors propose to use a regression model trained to predict cognition ('brain-cognition') as an alternative that explains more unique variance in the downstream regression.

This is a reasonably good paper and the use of a commonality analysis is a nice contribution to understanding variance partitioning across different covariates. While the main message will not come as a surprise to anyone with hands-on experience of using brain-age models, I think it is nonetheless an important message to convey to the community. With that said, I have some comments that I believe the authors ought to address before publication.

Reviewer 1 Recommendations For The Authors #1:

First, from a conceptual point of view, the authors focus exclusively on cognition as a downstream outcome. This is undeniably important, but is only one application area for brain age models. They are also used for example to provide biomarkers for many brain disorders. What would the results presented here have to say about these application areas? Further, I think that since brain-age models by construction confound relevant biological variation with the accuracy of the regression models used to estimate them, my own opinion about the limits of interpretation of (e.g.) the brain-age gap is as a dimensionless biomarker. This has also been discussed elsewhere (see e.g. https://academic.oup.com/brain/article/143/7/2312/5863667). I would suggest the authors nuance their discussion to provide considerations on these issues.

Response Thank you Reviewer 1 for pointing out two important issues.

The first issue was about applications for brain disorders. We now made a detailed discussion about this, which also addressed Reviewer 3 Public Review #9. Briefly, we now bought up

1) the consistency between our findings on fluid cognition and other recent works on brain disorders,

2) under-fitted age-prediction models from Brain Age studies focusing on neurological/psychological disorders when applied to participants with neurological/psychological disorders because the age-prediction models were built from largely healthy participants,

and 3) suggested solutions we and others made to optimise the utility of Brain Age for both cognitive functioning and brain disorders.

Discussion:

“This discrepancy between the predictive performance of age-prediction models and the utility of Brain Age indices as a biomarker is consistent with recent findings (for review, see Jirsaraie, Gorelik, et al., 2023), both in the context of cognitive functioning (Jirsaraie, Kaufmann, et al., 2023) and neurological/psychological disorders (Bashyam et al., 2020; Rokicki et al., 2021). For instance, combining different MRI modalities into the prediction models, similar to our stacked models, often lead to the highest performance of age-prediction models, but does not likely explain the highest variance across different phenotypes, including cognitive functioning and beyond (Jirsaraie, Gorelik, et al., 2023).”

“There is a notable difference between studies investigating the utility of Brain Age in explaining cognitive functioning, including ours and others (e.g., Butler et al., 2021; Cole, 2020, 2020; Jirsaraie, Kaufmann, et al., 2023) and those explaining neurological/psychological disorders (e.g., Bashyam et al., 2020; Rokicki et al., 2021). That is, those Brain Age studies focusing on neurological/psychological disorders often build age-prediction models from MRI data of largely healthy participants (e.g., controls in a case-control design or large samples in a population-based design), apply the built age-prediction models to participants without vs. with neurological/psychological disorders and compare Brain Age indices between the two groups. This means that age-prediction models from Brain Age studies focusing on neurological/psychological disorders might be under-fitted when applied to participants with neurological/psychological disorders because they were built from largely healthy participants. And thus, the difference in Brain Age indices between participants without vs. with neurological/psychological disorders might be confounded by the under-fitted age-prediction models (i.e., Brain Age may predict chronological age well for the controls, but not for those with a disorder). On the contrary, our study and other Brain Age studies focusing on cognitive functioning often build age-prediction models from MRI data of largely healthy participants and apply the built age-prediction models to participants who are also largely healthy. Accordingly, the age-prediction models for explaining cognitive functioning do not suffer from being under-fitted. We consider this as a strength, not a weakness of our study.”

“Next, researchers should not select age-prediction models based solely on age-prediction performance. Instead, researchers could select age-prediction models that explained phenotypes of interest the best. Here we selected age-prediction models based on a set of features (i.e., modalities) of brain MRI. This strategy was found effective not only for fluid cognition as we demonstrated here, but also for neurological and psychological disorders as shown elsewhere (Jirsaraie, Gorelik, et al., 2023; Rokicki et al., 2021). Rokicki and colleagues (2021), for instance, found that, while integrating across MRI modalities led to age-prediction models with the highest age-prediction performance, using only T1 structural MRI gave age-prediction models that were better at classifying Alzheimer’s disease. Similarly, using only cerebral blood flow gave age-prediction models that were better at classifying mild/subjective cognitive impairment, schizophrenia and bipolar disorder. As opposed to selecting age-prediction models based on a set of features, researchers could also select age-prediction models based on modelling methods. For instance, Jirsaraie and colleagues (2023) compared gradient tree boosting (GTB) and deep-learning brain network (DBN) algorithms in building age-prediction models. They found GTB to have higher age-prediction performance but DBN to have better utility in explaining cognitive functioning. In this case, an algorithm with better utility (e.g., DBN) should be used for explaining a phenotype of interest. Similarly, Bashyam and colleagues (2020) built different DBN-based age-prediction models, varying in age-prediction performance. The DBN models with a higher number of epochs corresponded to higher age-prediction performance. However, DBN-based age-prediction models with a moderate (as opposed to higher or lower) number of epochs were better at classifying Alzheimer’s disease, mild cognitive impairment and schizophrenia. In this case, a model from the same algorithm with better utility (e.g., those DBN with a moderate epoch number) should be used for explaining a phenotype of interest. Accordingly, this calls for a change in research practice, as recently pointed out by Jirasarie and colleagues (2023, p7), “Despite mounting evidence, there is a persisting assumption across several studies that the most accurate brain age models will have the most potential for detecting differences in a given phenotype of interest”. Future neuroimaging research should aim to build age-prediction models that are not necessarily good at predicting age, but at capturing phenotypes of interest.”

The second issue was about “the brain-age gap as a dimensionless biomarker.” We are not so clear on what the reviewer meant by “the dimensionless biomarker.” One possible meaning of the “dimensionless biomarker” is the fact that Brain Age from the same algorithm and same modality can be computed, such that Brain Age can be tightly fit or loosely fit with chronological age. This is what Bashyam and colleagues (2020) did in the article Reviewer 1 referred to. We now wrote about this strategy in the above paragraph in the Discussion.

Alternatively, “the dimensionless biomarker” might be something closer to what Reviewer 2 viewed Brain Age as a “generic/indirect” index (as opposed to a 'specific/direct' index in the case of Brain Cognition) (see Reviewer 2 Public Review #4). We discussed this in our response to Reviewer 3 Public Review #4.

Reviewer 1 Recommendations For The Authors #2:

Second, from a methods perspective, I am quite suspicious of the stacked regression models the authors are using to combine regression models and I suspect they may be overfit. In my experience, stacked models are very prone to overfitting when combined with cross-validation. This is because the predictions from the first level models (i,e. the features that are provided to the second-level 'stacked' models) contain information about the training set and the test set. If cross-validation is not done very carefully (e.g. using multiple hold-out sets), information leakage can easily occur at the second level. Unfortunately, there is not sufficient explanation of the methodological procedures in the current manuscript to fully understand what was done. First, please provide more information to enable the reader to better understand the stacked regression models and if the authors are not using an approach that fully preserves training and test separability, please do so.

Response: We would like to thank Reviewer 1 for the suggestion. We now made it clearer in texts and new figure (see below) that we used nested cross-validation to ensure no information leakage between training and test sets. Regarding the stacked models more specifically, the hyperparameters of the stacked models were tuned in the same inner-fold CV as the non-stacked model (see Figure 7 below). That is, training models for both non-stacked and stacked models did not involve the test set, ensuring that there was no data leakage between training and test sets.

Methods:

“To compute Brain Age and Brain Cognition, we ran two separate prediction models. These prediction models either had chronological age or fluid cognition as the target and standardised brain MRI as the features (Denissen et al., 2022). We used nested cross-validation (CV) to build these models (see Figure 7). We first split the data into five outer folds. We used five outer folds so that each outer fold had around 100 participants. This is to ensure the stability of the test performance across folds. In each outer-fold CV, one of the outer folds was treated as a test set, and the rest was treated as a training set, which was further divided into five inner folds. In each inner-fold CV, one of the inner folds was treated as a validation set and the rest was treated as a training set. We used the inner-fold CV to tune for hyperparameters of the models and the outer-fold CV to evaluate the predictive performance of the models.

In addition to using each of the 18 sets of features in separate prediction models, we drew information across these sets via stacking. Specifically, we computed predicted values from each of the 18 sets of features in the training sets. We then treated different combinations of these predicted values as features to predict the targets in separate “stacked” models. The hyperparameters of the stacked models were tuned in the same inner-fold CV as the non-stacked model (see Figure 7). That is, training models for both non-stacked and stacked models did not involve the test set, ensuring that there was no data leakage between training and test sets. We specified eight stacked models: “All” (i.e., including all 18 sets of features), “All excluding Task FC”, “All excluding Task Contrast”, “Non-Task” (i.e., including only Rest FC and sMRI), “Resting and Task FC”, “Task Contrast and FC”, “Task Contrast” and “Task FC”. Accordingly, in total, there were 26 prediction models for Brain Age and Brain Cognition.

Reviewer 1 Recommendations For The Authors #3:

Third, the authors standardize the elastic net regression coefficients post-hoc. Why did the authors not perform the more standard approach of standardizing the covariates and responses, prior to model estimation, which would yield standardized regression coefficients (in the classical sense) by construction? Please also provide an indication of the different regression strengths that were estimated across the different models and cross-validation splits. Also, how stable were the weights across splits?

Response For model fitting, we did not “standardize the elastic net regression coefficients post-hoc.” Instead, we did all of the standardisation steps prior to model fitting (see Methods below). For regression strengths across different models and cross-validation splits, we now provided predictive performance at each of the five outer-fold test sets in Figure 1 (below). As you may have seen, the predictive performance was quite stable across the cross-validation splits.

For visualising feature importance, We originally only standardised the elastic net regression coefficients post-hoc, so that feature importance plots were in the same scale across folds. However, as mentioned by Reviewer 3 (Recommendations for the Authors #7, below), this might make it difficult to interpret the directionality of the coefficients. In the revised manuscript, we refitted the Elastic Net model to the full dataset without splitting them into five folds and visualised the coefficients on brain images (see below).

Methods

“We controlled for the potential influences of biological sex on the brain features by first residualising biological sex from brain features in each outer-fold training set. We then applied the regression of this residualisation to the corresponding test set. We also standardised the brain features in each outer-fold training set and then used the mean and standard deviation of this outer-fold training set to standardise the test set. All of the standardisation was done prior to fitting the prediction models.”

“To understand how Elastic Net made a prediction based on different brain features, we examined the coefficients of the tuned model. Elastic Net coefficients can be considered as feature importance, such that more positive Elastic Net coefficients lead to more positive predicted values and, similarly, more negative Elastic Net coefficients lead to more negative predicted values (Molnar, 2019; Pat, Wang, Bartonicek, et al., 2022). While the magnitude of Elastic Net coefficients is regularised (thus making it difficult for us to interpret the magnitude itself directly), we could still indicate that a brain feature with a higher magnitude weights relatively stronger in making a prediction. Another benefit of Elastic Net as a penalised regression is that the coefficients are less susceptible to collinearity among features as they have already been regularised (Dormann et al., 2013; Pat, Wang, Bartonicek, et al., 2022).

Given that we used five-fold nested cross validation, different outer folds may have different degrees of ‘’ and ‘l_1 ratio’, making the final coefficients from different folds to be different. For instance, for certain sets of features, penalisation may not play a big part (i.e., higher or lower ‘’ leads to similar predictive performance), resulting in different ‘’ for different folds. To remedy this in the visualisation of Elastic Net feature importance, we refitted the Elastic Net model to the full dataset without splitting them into five folds and visualised the coefficients on brain images using Brainspace (Vos De Wael et al., 2020) and Nilern (Abraham et al., 2014) packages. Note, unlike other sets of features, Task FC and Rest FC were modelled after data reduction via PCA. Thus, for Task FC and Rest FC, we, first, multiplied the absolute PCA scores (extracted from the ‘components_’ attribute of ‘sklearn.decomposition.PCA’) with Elastic Net coefficients and, then, summed the multiplied values across the 75 components, leaving 71,631 ROI-pair indices.”

Reviewer 1 Recommendations For The Authors #4:

I do not really find it surprising that the level of unique explained variance provided by a brain-cognition model is higher than a brain-age model, given that the latter is considerably more accurate (also, in view of the comment above). As such I would recommend to tone down the claims about the utility of this method, also because it is only really applicable to one application area for brain age.

Response Thank you for bringing this issue to our attention. We have now toned down the claims about the utility of Brain Cognition and importantly treated the capability of Brain Cognition in capturing fluid cognition as the upper limit of Brain Age’s capability in capturing fluid cognition. Please see Reviewer 3 Public Review #4 above for a detailed discussion about this issue.

Reviewer 1 Recommendations For The Authors #5:

Please provide more details about the task designs and MRI processing procedures that were employed on this sample so that the reader is not forced to dig through the publications from the consortia contributing the data samples used. For example, comments such as "Here we focused on the pre-processed task fMRI files with a suffix "_PA_Atlas_MSMAll_hp0_clean.dtseries.nii." are not particularly helpful to readers not already familiar with this dataset.

Response Thank you so much for pointing out this important point on the clarity of the description of our MRI methodology. We now added additional details about the data processing done by the HCP-A and by us. We, for instance, explained the meaning of the HCP-A suffix “"_PA_Atlas_MSMAll_hp0_clean.dtseries.nii”. Please see below.

Methods

“HCP-A provides details of parameters for brain MRI elsewhere (Bookheimer et al., 2019; Harms et al., 2018). Here we used MRI data that were pre-processed by the HCP-A with recommended methods, including the MSMALL alignment (Glasser et al., 2016; Robinson et al., 2018) and ICA-FIX (Glasser et al., 2016) for functional MRI. We used multiple brain MRI modalities, covering task functional MRI (task fMRI), resting-state functional MRI (rsfMRI) and structural MRI (sMRI), and organised them into 19 sets of features.

Sets of Features 1-10: Task fMRI contrast (Task Contrast)

Task contrasts reflect fMRI activation relevant to events in each task. Bookheimer and colleagues (2019) provided detailed information about the fMRI in HCP-A. Here we focused on the pre-processed task fMRI Connectivity Informatics Technology Initiative (CIFTI) files with a suffix, “_PA_Atlas_MSMAll_hp0_clean.dtseries.nii.” These CIFTI files encompassed both the cortical mesh surface and subcortical volume (Glasser et al., 2013). Collected using the posterior-to-anterior (PA) phase, these files were aligned using MSMALL (Glasser et al., 2016; Robinson et al., 2018), linear detrended (see https://groups.google.com/a/humanconnectome.org/g/hcp-users/c/ZLJc092h980/m/GiihzQAUAwAJ) and cleaned from potential artifacts using ICA-FIX (Glasser et al., 2016).

To extract Task Contrasts, we regressed the fMRI time series on the convolved task events using a double-gamma canonical hemodynamic response function via FMRIB Software Library (FSL)’s FMRI Expert Analysis Tool (FEAT) (Woolrich et al., 2001). We kept FSL’s default high pass cutoff at 200s (i.e., .005 Hz). We then parcellated the contrast ‘cope’ files, using the Glasser atlas (Gordon et al., 2016) for cortical surface regions and the Freesurfer’s automatic segmentation (aseg) (Fischl et al., 2002) for subcortical regions. This resulted in 379 regions, whose number was, in turn, the number of features for each Task Contrast set of features.

HCP-A collected fMRI data from three tasks: Face Name (Sperling et al., 2001), Conditioned Approach Response Inhibition Task (CARIT) (Somerville et al., 2018) and VISual MOTOR (VISMOTOR) (Ances et al., 2009). First, the Face Name task (Sperling et al., 2001) taps into episodic memory. The task had three blocks. In the encoding block [Encoding], participants were asked to memorise the names of faces shown. These faces were then shown again in the recall block [Recall] when the participants were asked if they could remember the names of the previously shown faces. There was also the distractor block [Distractor] occurring between the encoding and recall blocks. Here participants were distracted by a Go/NoGo task. We computed six contrasts for this Face Name task: [Encode], [Recall], [Distractor], [Encode vs. Distractor], [Recall vs. Distractor] and [Encode vs. Recall].

Second, the CARIT task (Somerville et al., 2018) was adapted from the classic Go/NoGo task and taps into inhibitory control. Participants were asked to press a button to all [Go] but not to two [NoGo] shapes. We computed three contrasts for the CARIT task: [NoGo], [Go] and [NoGo vs. Go].

Third, the VISMOTOR task (Ances et al., 2009) was designed to test simple activation of the motor and visual cortices. Participants saw a checkerboard with a red square either on the left or right. They needed to press a corresponding key to indicate the location of the red square. We computed just one contrast for the VISMOTOR task: [Vismotor], which indicates the presence of the checkerboard vs. baseline.

Sets of Features 11-13: Task fMRI functional connectivity (Task FC)

Task FC reflects functional connectivity (FC ) among the brain regions during each task, which is considered an important source of individual differences (Elliott et al., 2019; Fair et al., 2007; Gratton et al., 2018). We used the same CIFTI file “_PA_Atlas_MSMAll_hp0_clean.dtseries.nii.” as the task contrasts. Unlike Task Contrasts, here we treated the double-gamma, convolved task events as regressors of no interest and focused on the residuals of the regression from each task (Fair et al., 2007). We computed these regressors on FSL, and regressed them in nilearn (Abraham et al., 2014). Following previous work on task FC (Elliott et al., 2019), we applied a highpass at .008 Hz. For parcellation, we used the same atlases as Task Contrast (Fischl et al., 2002; Glasser et al., 2016). We computed Pearson’s correlations of each pair of 379 regions, resulting in a table of 71,631 non-overlapping FC indices for each task. We then applied r-to-z transformation and principal component analysis (PCA) of 75 components (Rasero et al., 2021; Sripada et al., 2019, 2020). Note to avoid data leakage, we conducted the PCA on each training set and applied its definition to the corresponding test set. Accordingly, there were three sets of 75 features for Task FC, one for each task. “

Reviewer 1 Recommendations For The Authors #6:

Similarly, please be more specific about the regression methods used. There are several different parameterisations of the elastic net, please provide equations to describe the method used here so that readers can easily determine how the regularisation parameters should be interpreted. The same goes for the methods used for correcting bias, e.g. what is "de Lange and Cole's (2020) 5th equation"?

Response Thank you. We now made a detailed description of Elastic Net including its equation (see below). We also added more specific details about the methods used for correcting bias in Brain Age indices (see our response to Reviewer 3 Public Review #2 above).

Methods:

“For the machine learning algorithm, we used Elastic Net (Zou & Hastie, 2005). Elastic Net is a general form of penalised regressions (including Lasso and Ridge regression), allowing us to simultaneously draw information across different brain indices to predict one target variable. Penalised regressions are commonly used for building age-prediction models (Jirsaraie, Gorelik, et al., 2023). Previously we showed that the performance of Elastic Net in predicting cognitive abilities is on par, if not better than, many non-linear and more-complicated algorithms (Pat, Wang, Bartonicek, et al., 2022; Tetereva et al., 2022). Moreover, Elastic Net coefficients are readily explainable, allowing us the ability to explain how our age-prediction and cognition-prediction models made the prediction from each brain feature (Molnar, 2019; Pat, Wang, Bartonicek, et al., 2022) (see below).

Elastic Net simultaneously minimises the weighted sum of the features’ coefficients. The degree of penalty to the sum of the feature’s coefficients is determined by a shrinkage hyperparameter ‘’: the greater the , the more the coefficients shrink, and the more regularised the model becomes. Elastic Net also includes another hyperparameter, ‘l_1 ratio’, which determines the degree to which the sum of either the squared (known as ‘Ridge’; l_1 ratio=0) or absolute (known as ‘Lasso’; l_1 ratio=1) coefficients is penalised (Zou & Hastie, 2005). The objective function of Elastic Net as implemented by sklearn (Pedregosa et al., 2011) is defined as: argmin_ ((|(|y-X|)|_2^2)/(2×n_samples )+α×l_1 _ratio×|(||)|_1+0.5×α×(1-l_1 _ratio)×|(|w|)|_2^2 ), (1) where X is the features, y is the target, and  is the coefficient. In our grid search, we tuned two Elastic Net hyperparameters:  using 70 numbers in log space, ranging from .1 and 100, and l_1-ratio using 25 numbers in linear space, ranging from 0 and 1.”

Additional minor points:

Reviewer 1 Recommendations For The Authors #7:

• Please provide more descriptive figure legends, especially for Figs 5 and 6. For example, what do the boldface numbers reflect? What do the asterisks reflect?

Response Thank you for the suggestion. We made changes to the figure legends to make it clearer what the numbers and asterisks reflect.

Reviewer 1 Recommendations For The Authors #8:

• Perhaps this is personal thing, but I find the nomenclature cognition_{fluid} to be quite awkward. Why not just define FC as an acronym?

Response Thank you for the suggestion. We now used the word ‘fluid cognition’ throughout the manuscript.

Reviewer #2 (Recommendations For The Authors):

Suggestions for improved or additional experiments, data or analyses.

Reviewer 2 Recommendations For The Authors #1:

• Since the study did not provide external validation for the indices, it is unclear how well the models would perform and generalize to other samples. Therefore, it is recommended to conduct out-of-sample testing of the models.

Response Thank you for the suggestion. We now added discussions about how consistency between our results and several recent studies that investigated similar issues with Brain Age in different populations, e.g., large samples of older adults in Uk Biobank (Cole, 2020) and younger populations (Butler et al., 2021; Jirsaraie, Kaufmann, et al., 2023), and in a broader context, extending to neurological and psychological disorders (for review, see Jirsaraie, Gorelik, et al., 2023). Please see below.

Please also noted that all of the analyses done were out-of-sample. We used nested cross-validation to evaluate the predictive performance of age- and cognition-prediction models on the outer-fold test sets, which are out-of-sample from the training sets (please see Reviewer 1 Recommendations For The Authors #2). Similarly, we also conducted all of the commonality analyses on the outer-fold test sets.

Discussion

“The small effects of the Corrected Brain Age Gap in explaining fluid cognition of aging individuals found here are consistent with studies in older adults (Cole, 2020) and younger populations (Butler et al., 2021; Jirsaraie, Kaufmann, et al., 2023). Cole (2020) studied the utility of Brain Age on cognitive functioning of large samples (n>17,000) of older adults, aged 45-80 years, from the UK Biobank (Sudlow et al., 2015). He constructed age-prediction models using LASSO, a similar penalised regression to ours and applied the same age-dependency adjustment to ours. Cole (2020) then conducted a multiple regression explaining cognitive functioning from Corrected Brain Age Gap while controlling for chronological age and other potential confounds. He found Corrected Brain Age Gap to be significantly related to performance in four out of six cognitive measures, and among those significant relationships, the effect sizes were small with a maximum of partial eta-squared at .0059. Similarly, Jirsaraie and colleagues (2023) studied the utility of Brain Age on cognitive functioning of youths aged 8-22 years old from the Human Connectome Project in Development (Somerville et al., 2018) and Preschool Depression Study (Luby, 2010). They built age-prediction models using gradient tree boosting (GTB) and deep-learning brain network (DBN) and adjusted the age dependency of Brain Age Gap using Smith and colleagues’ (2019) method. Using multiple regressions, Jirsaraie and colleagues (2023) found weak effects of the adjusted Brain Age Gap on cognitive functioning across five cognitive tasks, five age-prediction models and the two datasets (mean of standardised regression coefficient = -0.09, see their Table S7). Next, Butler and colleagues (2021) studied the utility of Brain Age on cognitive functioning of another group of youths aged 8-22 years old from the Philadelphia Neurodevelopmental Cohort (PNC) (Satterthwaite et al., 2016). Here they used Elastic Net to build age-prediction models and applied another age-dependency adjustment method, proposed by Beheshti and colleagues (2019). Similar to the aforementioned results, Butler and colleagues (2021) found a weak, statistically non-significant correlation between the adjusted Brain Age Gap and cognitive functioning at r=-.01, p=.71. Accordingly, the utility of Brain Age in explaining cognitive functioning beyond chronological age appears to be weak across age groups, different predictive modelling algorithms and age-dependency adjustments.“

“This discrepancy between the predictive performance of age-prediction models and the utility of Brain Age indices as a biomarker is consistent with recent findings (for review, see Jirsaraie, Gorelik, et al., 2023), both in the context of cognitive functioning (Jirsaraie, Kaufmann, et al., 2023) and neurological/psychological disorders (Bashyam et al., 2020; Rokicki et al., 2021). For instance, combining different MRI modalities into the prediction models, similar to our stacked models, often lead to the highest performance of age-prediction models, but does not likely explain the highest variance across different phenotypes, including cognitive functioning and beyond (Jirsaraie, Gorelik, et al., 2023). “

“Third, by introducing Brain Cognition, we showed the extent to which Brain Age indices were not able to capture the variation of brain MRI that is related to fluid cognition. Brain Cognition, from certain cognition-prediction models such as the stacked models, has relatively good predictive performance, consistent with previous studies (Dubois et al., 2018; Pat, Wang, Anney, et al., 2022; Rasero et al., 2021; Sripada et al., 2020; Tetereva et al., 2022; for review, see Vieira et al., 2022). We then examined Brain Cognition using commonality analyses (Nimon et al., 2008) in multiple regression models having a Brain Age index, chronological age and Brain Cognition as regressors to explain fluid cognition. Similar to Brain Age indices, Brain Cognition exhibited large common effects with chronological age. But more importantly, unlike Brain Age indices, Brain Cognition showed large unique effects, up to around 11%. The unique effects of Brain Cognition indicated the amount of co-variation between brain MRI and fluid cognition that was missed by a Brain Age index and chronological age. This missing amount was relatively high, considering that Brain Age and chronological age together explained around 32% of the total variation in fluid cognition. Accordingly, if a Brain Age index was used as a biomarker along with chronological age, we would have missed an opportunity to improve the performance of the model by around one-third of the variation explained. “

“There is a notable difference between studies investigating the utility of Brain Age in explaining cognitive functioning, including ours and others (e.g., Butler et al., 2021; Cole, 2020, 2020; Jirsaraie, Kaufmann, et al., 2023) and those explaining neurological/psychological disorders (e.g., Bashyam et al., 2020; Rokicki et al., 2021). That is, those Brain Age studies focusing on neurological/psychological disorders often build age-prediction models from MRI data of largely healthy participants (e.g., controls in a case-control design or large samples in a population-based design), apply the built age-prediction models to participants without vs. with neurological/psychological disorders and compare Brain Age indices between the two groups. This means that age-prediction models from Brain Age studies focusing on neurological/psychological disorders might be under-fitted when applied to participants with neurological/psychological disorders because they were built from largely healthy participants. And thus, the difference in Brain Age indices between participants without vs. with neurological/psychological disorders might be confounded by the under-fitted age-prediction models (i.e., Brain Age may predict chronological age well for the controls, but not for those with a disorder). On the contrary, our study and other Brain Age studies focusing on cognitive functioning often build age-prediction models from MRI data of largely healthy participants and apply the built age-prediction models to participants who are also largely healthy. Accordingly, the age-prediction models for explaining cognitive functioning do not suffer from being under-fitted. We consider this as a strength, not a weakness of our study.”

Reviewer 2 Recommendations For The Authors #2:

• Employ Variance Inflation Factor (VIF) to empirically test for multicollinearity.

Response Given high common effects between many of the regressors in the models (e.g., between Brain Age and chronological age), VIF will be high, but this is not a concern for the commonality analysis. We showed now that applying the commonality analysis to multiple regressions allowed us to have robust results against multicollinearity, as demonstrated elsewhere (Ray-Mukherjee et al., 2014, Using commonality analysis in multiple regressions: A tool to decompose regression effects in the face of multicollinearity). Specifically, using the multiple regressions by themselves without the commonality analysis, researchers have to rely on beta estimates, which are strongly affected by multicollinearity (e.g., a phenomenon known as the Suppression Effect). However, by applying the commonality analysis on top of multiple regressions, researchers can then rely on R2 estimates, which are less affected by multicollinearity. This can be seen in our case (Figure 5 and 6) where Brain Age indices had the same unique effects regardless of the level of common effects they had with chronological age (e.g., Brain Age vs. Corrected Brain Age Gap from stacked models).

To directly demonstrate the robustness of the current commonality analysis regarding multicollinearity, we applied the commonality analysis to Ridge regressions (see Supplementary Figures 3 and 5 below). Ridge regression is a method designed to deal with multicollinearity (Dormann et al., 2013). As seen below, the results from commonality analyses applied to Ridge regressions are closely matched with our original results.

Methods

“Note to ensure that the commonality analysis results were robust against multicollinearity (Ray-Mukherjee et al., 2014), we also repeated the same commonality analyses done here on Ridge regression, as opposed to multiple regression. Ridge regression is a method designed to deal with multicollinearity (Dormann et al., 2013). See Supplementary Figure 3 for the Ridge regression with chronological age and each Brain Age index as regressors and Supplementary Figure 5 for the Ridge regression with chronological age, each Brain Age and Brain Cognition index as regressors. Briefly, the results from commonality analyses applied to Ridge regressions are closely matched with our results done using multiple regression.”

Reviewer 2 Recommendations For The Authors #3:

• Incorporate non-linearities in the correction of brain-age indices, such as separate terms in the regression or statistical analyses.

Response Thank you for the suggestion. We now added a non-linear term of chronological age in our multiple-regression models explaining fluid cognition (see Supplementary Figure 4 and 6 below). Originally we did not have the quadratic term for chronological age in our model since the relationship between chronological age and fluid cognition was relatively linear (see Figure 1 above). Accordingly, as expected, adding the quadratic term for chronological age as suggested did not change the pattern of the results of the commonality analyses.

Methods

“Similarly, to ensure that we were able to capture the non-linear pattern of chronological age in explaining fluid cognition, we added a quadratic term of chronological age to our multiple-regression models in the commonality analyses. See Supplementary Figure 4 for the multiple regression with chronological age, square chronological age and each Brain Age index as regressors and Supplementary Figure 6 for the multiple regression with chronological age, square chronological age, each Brain Age index and Brain Cognition as regressors. Briefly, adding the quadratic term for chronological age did not change the pattern of the results of the commonality analyses.”

Reviewer 2 Recommendations For The Authors #4:

• It would be helpful to include the complete set of results in the appendix - for instance, the statistical significance for each component for the final commonality analysis.

Response Figures 5 and 6 (see above) already have asterisks to reflect the statistical significance of the unique effects. Because of this, we do not believe we need more figures/tables in the appendix to show statistical significance.

Recommendations for improving the writing and presentation.

Reviewer 2 Recommendations For The Authors #5:

• The authors are encouraged to refrain from using terms such as 'fortunately', 'unfortunately', and 'unsettling', as they may appear inappropriate when referring to empirical findings.

Response We agree with this suggestion and no long used those words.

Reviewer 2 Recommendations For The Authors #6:

• It would be helpful to clarify in the methods that you end up with 5 test folds.

Response We now made a clarification why we chose 5 test folds.

Methods

“We used nested cross-validation (CV) to build these models (see Figure 7). We first split the data into five outer folds. We used five outer folds so that each outer fold had around 100 participants. This is to ensure the stability of the test performance across folds.”

Minor corrections to the text and figures.

Reviewer 2 Recommendations For The Authors #7:

• Why use months, not years for chronological age? This seems inappropriate given the age range.

Response We originally used months since they were units used in our prediction modelling. However, to make the figures easier to understand, we now used years.

Reviewer 2 Recommendations For The Authors #8:

• The formatting, especially regarding the text embedded within the figures, could benefit from significant improvements.

Response Thank you for the suggestion. We made changes to the text embedded within the figures. They should be more readable now

Reviewer 2 Recommendations For The Authors #9:

• The legend for the neuroimaging feature labels is missing, and the captions are incomplete.

Response Please see Figure 2 above. We now revised by adding the letter L and R for the laterality of the brain images. We made some changes to the captions to make sure they are complete.

Reviewer 2 Recommendations For The Authors #10:

• Figure 5's caption: SD has a missing decimal point).

Response The numbers are not SD. The numbers to the left of the figure represent the unique effects of chronological age in %, the numbers in the middle of the figure represent the common effects between chronological age and Brain Age index in %, and the numbers to the right of the figure represent the unique effects of Brain Age Index in %. We now used the same one decimal point for these number

Reviewer #3 (Recommendations For The Authors):

The main question of this article is as follows: “To what extent does having information on Brain Age improve our ability to capture declines in fluid cognition beyond knowing a person’s chronological age?” While this question is worthwhile, considering most of the field is confused about the nature of brain age, the authors are currently missing an opportunity to convey the inevitability of their results given how Brain Age and the Brain Age Gap are calculated. They also misleadingly convey that Brain Cognition is somehow superior to Brain Age. If the authors work on conveying the inevitability of their results and redo (or remove) their section on Brain Cognition, I can see how their results would be enlightening to the general neuroimaging community that is interested in the concept of brain age. See below for specific critiques.

Response Please see our response to Reviewer 3 Public Review Overall. Note we no longer argue that Brain Cognition is superior to Brain Age (Reviewer 3 Public Review #4). Rather, we treated the capability of Brain Cognition in capturing fluid cognition as the upper limit of Brain Age’s capability in capturing fluid cognition. We used the unique effects of Brain Cognition that explain fluid cognition beyond Brain Age and chronological age to indicate how much Brain Age misses the variation in the brain MRI that could explain fluid cognition.

Reviewer 3 Recommendations For The Authors #1:

“There are many adjustments proposed to correct for this estimation bias” (p3) → Regression to the mean is not a sign of bias. Any decent loss function will result in over- predicting the age of younger individuals and under-predicting the age of older individuals. This is a direct result of minimizing an error term (e.g., mean squared error). Therefore, it is inappropriate to refer to regression to the mean as a sign of bias. This misconception has led to a great deal of inappropriate analyses, including “correcting” the brain age gap by regressing out age.

Response Please see our response to Reviewer 3 Public Review#1

Reviewer 3 Recommendations For The Authors #2:

“Corrected Brain Age Gap in particular is viewed as being able to control for both age dependency and estimation biases (Butler et al., 2021).” (p3) → This summary is not accurate as Butler and colleagues did not use the words "corrected" and "biases" in this context. All that authors say in that paper is that regressing out age from the brain age gap - which is referred to as the modified brain age gap (MBAG) - makes it so that the modified brain age gap is not dependent on age, which is true. This metric is meaningless, though, because it is the variance left over after regressing out age from residuals from a model that was predicting age. If it were not for the fact that regression on residuals is not equivalent to multiple regression (and out of sample estimates), MBAG would be a vector of zeros. Upon reading your Methods, I noticed that you are using a metric for Le et al. (2018) for your “Corrected Brain Age Gap”. If they cite the Butler et al. (2021) paper, I highly recommend sticking with the same notation, metrics and terminology throughout. That would greatly help with the interpretability of your paper, and cross-comparisons between the two.

Response Please see our response to Reviewer 3 Public Review #2.

Reviewer 3 Recommendations For The Authors #3:

“However, the improvement in predicting chronological age may not necessarily make Brain Age to be better at capturing Cognitionfluid. If, for instance, the age-prediction model had the perfect performance, Brian Age Gap would be exactly zero and would have no utility in capturing Cognitionfluid beyond chronological age.” (p3) → I largely agree with this statement. I would be really careful to distinguish between Brain Age and the Brain Age Gap here, as the former is a predicted value, and the latter is the residual times -1 (predicted age - age). Therefore, together they explain all of the variance in age. If you change the first sentence to refer to the Brain Age Gap, this statement makes more sense. The Brain Age Gap will never be exactly zero, though, even with perfect prediction on the training set, because subjects in the testing set are different from the subjects in the training set.

Response Please see our response to Reviewer 3 Public Review #3.

Reviewer 3 Recommendations For The Authors #4:

“Can we further improve our ability to capture the decline in cognitionfluid by using, not only Brain Age and chronological age, but also another biomarker, Brain Cognition?” → This question is fundamentally getting at whether a predicted value of cognition can predict cognition. Assuming the brain parameters can predict cognition decently, and the original cognitive measure that you were predicting is related to your measure of fluid cognition, the answer should be yes. This seems like an uninteresting question to me. Upon reading your Methods, it became clear that the cognitive variable in the model predicting cognition using brain features (to get predicted cognition, or as you refer to it, Brain Cognition) is the same as the measure of fluid cognition that you are trying to assess how well Brain Cognition can predict. Assuming the brain parameters can predict fluid cognition at all, of course Brain Cognition will predict fluid cognition. This is inevitable. You should never use predicted values of a variable to predict the same variable.

Response Please see our response to Reviewer 3 Public Review #4.

Reviewer 3 Recommendations For The Authors #5:

“We also examined if these better-performing age-prediction models improved the ability of Brain Age in explaining Cognitionfluid.” → Improved above and beyond what?

Response We referred to if better-performing age-prediction models improved the ability of Brain Age in explaining fluid cognition over and above lower-performing age-prediction models. We made changes to the Introduction to clarify this change.

Reviewer 3 Recommendations For The Authors #6:

Figure 1 b & c → It is a little difficult to read the text by the horizontal bars in your plots. Please make the text smaller so that there is more space between the words vertically, or even better, make the plots slightly bigger. Please also put the predicted values on the y-axis. This is standard practice for displaying regression results. To make more room, you can get rid of your rPearson or your R2 plot, considering the latter is simply the square of the former. If you want to make it clear that the association is positive between all of your variables, I would keep rPearson.

Response Thank you so much for the suggestions.

1) We now made sure that the text by the horizontal bars in Figure 1b and c is readable.

2) Note in prediction model/machine-learning literature, it is more common to plot observed/real values on the y-axis. Here is the logic of our practice: values in the x-axis are the predicted values based on the model, and we would like to see if the changes in the predicted values correspond to the changes in the observed/real value in the y-axis.

3) Regarding Pearson correlation vs R2, please note that we wrote ”for R2, we used the sum of squares definition (i.e., R2 = 1 – (sum of squares residuals/total sum of squares)) per a previous recommendation (Poldrack et al., 2020).” As such, R2 is NOT the square of the Pearson correlation. In fact, in Poldrack and colleages’s “Establishment of Best Practices for Evidence for Prediction” paper (2020), they discourage 1) the use of Pearson correlation by itself and 2) the use of the correlation coefficient square as R2 (as opposed to sum of squares definition):

“It is common in the literature to use the correlation between predicted and actual values as a measure of predictive performance; of the 64 studies in our literature review that performed prediction analyses on continuous outcomes, 30 reported such correlations as a measure of predictive performance. This reporting is problematic for several reasons. First, correlation is not sensitive to scaling of the data; thus, a high correlation can exist even when predicted values are discrepant from actual values. Second, correlation can sometimes be biased, particularly in the case of leave-one-out cross-validation. As demonstrated in Figure 4, the correlation between predicted and actual values can be strongly negative when no predictive information is present in the model. A further problem arises when the variance explained (R2) is incorrectly computed by squaring the correlation coefficient. Although this computation is appropriate when the model is obtained using the same data, it is not appropriate for out-of-sample testing23; instead, the amount of variance explained should be computed using the sum-of-squares formulation (as implemented in software packages such as scikit-learn).”

“A further problem arises when the variance explained (R2) is incorrectly computed by squaring the correlation coefficient. Although this computation is appropriate when the model is obtained using the same data, it is not appropriate for out-of-sample testing23; instead, the amount of variance explained should be computed using the sum-of-squares formulation (as implemented in software packages such as scikit-learn).”

Accordingly, we decided to keep both R2 and Pearson correlation (along with MAE) in our Figure 1.

Reviewer 3 Recommendations For The Authors #7:

Figure 2 “We calculated feature importance by, first, standardizing Elastic Net weights across brain features of each set of features from each test fold.” → What do you mean by “standardize” here? Rescale to be mean 0, variance 1? If so, this seems like a misleading transformation, because it gives the impression that the relationships are negative, when they are not necessarily. Also, why did you choose to use elastic net weights in any form as measures of effect size (or importance)? The raw values are inherently penalized, which means they are under-estimates of the true effect size. It would be more meaningful (and less biased) to plot the raw correlations.

Response For the first question regarding standardisation, we addressed this issue in our response to Reviewer 1 Recommendations For The Authors #3. Briefly, we agreed with Reviewer 3 that standardisation (with mean = 0, SD = 1) might make it difficult to interpret the directionality of the coefficients. For visualising feature importance in the revised manuscript, we refitted the Elastic Net model to the full dataset without splitting them into five folds and visualised the coefficients on brain images (see below).

For the second question regarding why using Elastic Net coefficients as feature importance (as opposed to correlations), we need to mention the goal of feature importance: to understand how the model makes a prediction based on different brain features (Molnar, 2019). Correlations between a target and each brain feature do not achieve this. Instead, they will show univariate/marginal relationships between a target and a brain feature. What we want to visualise is how the model made a prediction, which in the case of Elastic Net, the prediction is based on the sum of the features’ coefficients. In other words, the multivariate models (including Elastic Net) focus on marginal relationships that take into account all brain features within each set of features.

Elastic Net coefficients can be considered as feature importance, such that more positive Elastic Net coefficients lead to more positive predicted values and, similarly, more negative Elastic Net coefficients lead to more negative predicted values (Molnar, 2019; Pat, Wang, Bartonicek, et al., 2022). While the magnitude of Elastic Net coefficients is regularised (thus making it difficult for us to interpret the magnitude itself directly), we could still indicate that a brain feature with a higher magnitude weights relatively stronger in making a prediction. Another benefit of Elastic Net as a penalised regression is that the coefficients are less susceptible to collinearity among features as they have already been regularised (Dormann et al., 2013; Pat, Wang, Bartonicek, et al., 2022).

Reviewer 3 Recommendations For The Authors #8:

Figure 3 → Again, what exactly do you mean by “standardised” here?

Response It means mean subtraction followed by the division by an SD. Though we no longer applies standardisation for feature importance. See our response to Reviewer 1 Recommendations For The Authors #3 and Reviewer 3 Recommendations For The Authors #7.

Reviewer 3 Recommendations For The Authors #9:

“However, Brain Age Gap created from the lower-performing age-prediction models explained a higher amount of variation in Cognitionfluid. For instance, the top performing age-prediction model, “Stacked: All excluding Task Contrast”, generated Brain Age and Corrected Brain Age that explained the highest amount of variation in Cognitionfluid, but, at the same time, produced Brian Age Gap that explained the least amount of variation in Cognitionfluid.” (p7) → Yes, but you did not need to run any models to show this, considering it is an inevitable consequence of the following relationship between predicted values and residuals (or residuals times -1): 𝑦 = (𝑦 − 𝑦% ) + 𝑦% . Let’s say that age explains 60% of the variance in fluid cognition, and predicted age ( 𝑦% ) explains 40% of the variance in fluid cognition. Then the brain age gap (−(𝑦 − 𝑦% )) should explain 20% of the variance in fluid cognition. If by “Corrected Brain Age” you mean the modified predicted age from the Butler paper, the “Corrected Brain Age” result is inevitable because the modified predicted age is essentially just age with a tiny bit of noise added to it. From Figure 4, though, this does not seem to be the case, because the lower left quadrant in panel a should be flat and high (about as high as the predictive value of age for fluid cognition). So how are you calculating “Corrected Brain Age”? It looks like you might be regressing age out of Brain Age, though from your description the Methods (How exactly do you use the slope and intercept? You need equation of you are going to stick with this terminology), it is not totally clear. I highly recommend using terminology and metrics from the Butler et al. (2021) paper throughout to reduce confusion.

Response Please see our response to Reviewer 3 Public Review #5

Reviewer 3 Recommendations For The Authors #10:

“On the contrary, an amount of variation in Cognitionfluid explained by Corrected Brain Age Gap was relatively small (maximum R2 = .041) across age-prediction models and did not relate to the predictive performance of the age-prediction models.” (p7) → If by “Corrected Brain Age Gap” you mean MBAG from The Butler paper, yes, this is also inevitable, considering MBAG would be a vector of zeros if it were not for regression on residuals (and out of sample estimates), as I mentioned earlier. Also, it is not clear why you used “on the contrary” as a transition here.

Response Please see our response to Reviewer 3 Public Review #2 for the ‘MBAG’ term. Briefly, we didn’t use Butler and colleagues' (2021) MBAG, but rather we used the method described in de Lange and Cole’s (2020), which was called RBAG by Butler and colleagues.

de Lange and Cole’s (2020) method, was commonly implemented elsewhere (Cole et al., 2020; Cumplido-Mayoral et al., 2023; Denissen et al., 2022). Accordingly, researchers who use Brain Age do not usually view this method as capturing a meaningless biomarker. Yet, the small effects of the Corrected Brain Age Gap in explaining fluid cognition of aging individuals found here are consistent with studies in older adults (Cole, 2020) and younger populations (Butler et al., 2021; Jirsaraie, Kaufmann, et al., 2023) (see our response to Reviewer 2 Recommendations For The Authors #1).

“On the contrary” refers to the fact that the other three Brain Age indices (i.e., those that did not account for the relationship between Brain Age and chronological age) showed a much higher amount of variation in fluid cognition explained. As mentioned above (our response to Reviewer 2 Public Review #7), our argument resonates Butler and colleagues’ (2021) suggestion (p. 4097): “As such, it is critical that readers of past literature note whether or not age was controlled for when testing for effects on the BAG, as this has not always been common practice (e.g., Beheshti et al., 2018; Cole, Underwood, et al., 2017; Franke et al., 2015; Gaser et al., 2013; Liem et al., 2017; Nenadi c et al., 2017; Steffener et al., 2016)”.

Reviewer 3 Recommendations For The Authors #11:

“As before, the unique effects of Brain Age indices were all relatively small across the four Brain Age indices and across different prediction models.” (p10) → Yes, again, this is inevitable considering how they are calculated. You can show these analyses to demonstrate your results in data, if you want, but ignoring the inevitability given how these variables are calculated is misleading.

Response Accounting for the relationship between Brain Age and chronological age when examining the utility of Brain Age is not misleading. Similar to previous recommendations (Butler et al., 2021; Le et al., 2018), we believe that not doing so is misleading. That is, without accounting for the relationship between Brain Age and chronological age, Brain Age will likely explain the same variation of the phenotype of interest as chronological age. Please see our response to Reviewer 3 Recommendations For The Authors #18 below.

Reviewer 3 Recommendations For The Authors #12:

“On the contrary, the unique effects of Brain Cognition appeared much larger.” (p10) → This is not a fair comparison if you don’t look at the unique effects above and beyond the cognitive variable you predicted (fluid cognition) in your Brain Cognition model. When you do this, you will see that Brain Cognition is useless when you include fluid cognition in the model, just as Brain Age would be in predicting age when you include age in the model. This highlights the fact that using predicted values of a metric to predict that metric is a pointless path to take, and that using a predicted value to predict anything is worse than using the value itself.

Response Please see our response to Reviewer 3 Public Review #6.

Reviewer 3 Recommendations For The Authors #13:

“First, how much does Brain Age add to what is already captured by chronological age? The short answer is very little.” (p12) → This is a really important point, but your paper requires an in-depth discussion of the inevitability of this result, which I have discussed previously in this review.

Response Please see our response to Reviewer 3 Public Review #7.

Reviewer 3 Recommendations For The Authors #14:

“Second, do better-performing age-prediction models improve the ability of Brain Age to capture Cognitionfluid? Unfortunately, the answer is no.” (p12) → You need to be clear that you are talking about above and beyond age here.

Response Thank you so much for your suggestion. We now made the change to this sentence accordingly.

Discussion

“Second, do better-performing age-prediction models improve the utility of Brain Age to capture fluid cognition above and beyond chronological age? The answer is also no.”

Reviewer 3 Recommendations For The Authors #15:

“Third, do we have a solution that can improve our ability to capture Cognitionfluid from brain MRI? The answer is, fortunately, yes. Using Brain Cognition as a biomarker, along with chronological age, seemed to capture a higher amount of variation in Cognitionfluid than only using Brain Age.” (p12) → Again, try controlling for the cognitive measure you predicted in your Brain Cognition model. This will show that Brain Cognition is not useful above and beyond cognition, highlighting the fact that it is not a useful endeavor to be using predicted values.

Response Please see our response to Reviewer 3 Public Review #8.

Reviewer 3 Recommendations For The Authors #16:

“Accordingly, a race to improve the performance of age-prediction models (Baecker et al., 2021) does not necessarily enhance the utility of Brain Age indices as a biomarker for Cognitionfluid. This calls for a new paradigm. Future research should aim to build prediction models for Brian Age indices that are not necessarily good at predicting age, but at capturing phenotypes of interest, such as Cognitionfluid and beyond.” (p13) → I whole-heartedly agree with the first two sentences, and strongly disagree with the last. Certainly your results, and the underlying reason as to why you found these results, calls for a new paradigm (or, one might argue, a pre-brain age paradigm). They do not, however, suggest that we should keep going down the Brain Age path. In fact, I think it should be abandoned all together. While it is difficult to prove that there is no transformation of Brain Age or the Brain Age Gap that will be useful, I am nearly sure this is true from the research I have done. Therefore, if you would like to suggest that the field should continue down this path, you need to present a very good case to support this view.

Response Please see our response to Reviewer 3 Public Review #9.

Reviewer 3 Recommendations For The Authors #17:

“Perhaps this is because the estimation of the influences of chronological age was done in the training set.” (p13) → I believe this is the case, and it is testable. Try re-running your analyses where parameters are estimated and performance is evaluated on the same data.

Response Yes, we agreed with this. Based on the equations we used, this is inevitable.

Reviewer 3 Recommendations For The Authors #18:

“Similar to a previous recommendation (Butler et al., 2021), we suggest focusing on Corrected Brain Age Gap.” (p13) → To be clear, the authors did not use the term “Corrected” because it is very misleading. The authors also did not suggest that we proceed with any brain age metric; rather they mentioned that the modified brain age gap is independent of age. Note the following passage: “Further, the interpretability of the modified brain age gap (MBAG) itself is limited by the fact that it is a prediction error from a regression to remove the effects of age from a residual obtained through a regression to predict age. By virtue of these limitations, we suggest that the modified version may not provide useful information about precocity or delay in brain development. In light of this, as well as the complexities associated with interpretations of the BAG and its dependence on age, we suggest that further methodological and theoretical work is warranted.” I recognize that that this statement is hedged, as is often required in the publication process, but I am all but certain that MBAG/BAG/modified predicted age are useless constructs. Therefore, if you are going to suggest that people continue to use them, opposed to suggesting that further methodological or theoretical work is warranted, you need to make a strong case, which you did not try to make here. If anything, your results support abandoning the age- prediction endeavor altogether.

Response Please see our response to Reviewer 3 Public Review #2 for the term. Briefly, we didn’t use Butler and colleagues’ (2021) MBAG, but rather RBAG. This index was originally described in de Lange and Cole’s (2020), and has now been implemented elsewhere (Cole et al., 2020; Cumplido-Mayoral et al., 2023; Denissen et al., 2022).

We do not intend to encourage people to abandon the Brain Age endeavour altogether. However, we made main three suggestions for future research on Brain Age to ensure its utility. First, they should account for the relationship between Brain Age and chronological age either using Corrected Brain Age Gap (or other similar adjustments) or, better, examining the unique effects of Brain Age indices after controlling for chronological age through commonality analyses (see below). This is similar to the suggestion made by Le and colleagues (2018) and later rephased by Butler and colleagues (2021). More specifically, Le and colleagues (2018) mentioned (p. 10): “Based on our observations in both real and simulated data, we recommend that the relationship between chronological age and BrainAGE should be accounted for. The two methods proposed in this study are either: (1) regress age on BrainAGE, producing BrainAGER, which is centered on 0 regardless of a participant's actual age or (2) include age as a regressor when doing follow-up analyses.”

Second, we suggested that researchers should not select age-prediction models based solely on age-prediction performance (see our response to Reviewer 1 Recommendations For The Authors #1).

Third, we suggested that researchers should test how much Brain Age miss the variation in the brain MRI that could explain fluid cognition or other phenotypes of interest (see our response to Reviewer 2 Public Review #4).

Discussion

“What does it mean then for researchers/clinicians who would like to use Brain Age as a biomarker? First, they have to be aware of the overlap in variation between Brain Age and chronological age and should focus on the contribution of Brain Age over and above chronological age. Using Brain Age Gap will not fix this. Butler and colleagues (2021) recently highlighted this point, “These results indicate that the association between cognition and the BAG are driven by the association between age and cognitive performance. As such, it is critical that readers of past literature note whether or not age was controlled for when testing for effects on the BAG, as this has not always been common practice (p. 4097).” Similar to previous recommendations (Butler et al., 2021; Le et al., 2018), we suggest future work should account for the relationship between Brain Age and chronological age, either using Corrected Brain Age Gap (or other similar adjustments) or, better, examining unique effects of Brain Age indices after controlling for chronological age through commonality analyses. Note we prefer using unique effects over beta estimates from multiple regressions, given that unique effects do not change as a function of collinearity among regressors (Ray-Mukherjee et al., 2014). In our case, Brain Age indices had the same unique effects regardless of the level of common effects they had with chronological age (e.g., Brain Age vs. Corrected Brain Age Gap from stacked models). In the case of fluid cognition, the unique effects might be too small to be clinically meaningful as shown here and previously (Butler et al., 2021; Cole, 2020; Jirsaraie, Kaufmann, et al., 2023).”

Reviewer 3 Recommendations For The Authors #19:

“To compute Brain Age and Brain Cognition, we ran two separate prediction models. These prediction models either had chronological age or Cognitionfluid as the target.” (p16) → You should make it clear in the main text of your paper that the cognition variable in your Brain Cognition models is the same as what you refer to as Cognitionfluid. Some of your analyses would have been much more reasonable if you had two different measures of cognition.

Response Thank you so much for the suggestion. We believe, given the re-conceptualisation of Brain Cognition as the main text

Introduction

“certain variation in the brain MRI is related to fluid cognition, but to what extent does Brain Age not capture this variation? To estimate the variation in the brain MRI that is related to fluid cognition, we could build prediction models that directly predict fluid cognition (i.e., as opposed to chronological age) from brain MRI data.”

Reviewer 3 Recommendations For The Authors #20:

“We controlled for the potential influences of biological sex on the brain features by first residualizing biological sex from brain features in the training set.” (p16) → Why? Your question is about prediction, not causal inference.

Response While the question is about prediction, we still would like to, as much as possible, be confident about what kind of information we drew from. Here we focused on brain data and controlled for other variables that might not be neuronal. For instance, we controlled for movement and physiological noise using ICA-FIX (Glasser et al., 2016). Following conventional practices in brain-based predictive modelling, we also treated biological sex as another sort of noise (Vieira et al., 2022). The difference between movement/physiological noise and biological sex is that the former varies across TRs, and the latter varies across individuals. Thus we controlled for movement and physiological noise within each participant and controlled for biological sex within a group of participants who belonged to the same training set.

Reviewer 3 Recommendations For The Authors #20:

“Lastly, we computer Corrected Brain Age Gap by subtracting the chronological age from the Corrected Brain Age (Butler et al., 2021; Le et al., 2018).” (p17) → The modified brain age gap in that paper is the residuals from regressing BAG on age (see equation 6). I highly recommend using that terminology and notation throughout to provide consistency and interpretability across papers.

Response Please see our response to Reviewer 3 Public Review #2 for the term.

Reviewer 3 Recommendations For The Authors #21: Equations (pgs 17-19) → Please use statistical notation instead of pseudo-R code.

Response We rewrote all of the equations using statistical notations.

References

Abraham, A., Pedregosa, F., Eickenberg, M., Gervais, P., Mueller, A., Kossaifi, J., Gramfort, A., Thirion, B., & Varoquaux, G. (2014). Machine learning for neuroimaging with scikit-learn. Frontiers in Neuroinformatics, 8, 14. https://doi.org/10.3389/fninf.2014.00014

Ances, B. M., Liang, C. L., Leontiev, O., Perthen, J. E., Fleisher, A. S., Lansing, A. E., & Buxton, R. B. (2009). Effects of aging on cerebral blood flow, oxygen metabolism, and blood oxygenation level dependent responses to visual stimulation. Human Brain Mapping, 30(4), 1120–1132. https://doi.org/10.1002/hbm.20574

Bashyam, V. M., Erus, G., Doshi, J., Habes, M., Nasrallah, I. M., Truelove-Hill, M., Srinivasan, D., Mamourian, L., Pomponio, R., Fan, Y., Launer, L. J., Masters, C. L., Maruff, P., Zhuo, C., Völzke, H., Johnson, S. C., Fripp, J., Koutsouleris, N., Satterthwaite, T. D., … on behalf of the ISTAGING Consortium, the P. A. disease C., ADNI, and CARDIA studies. (2020). MRI signatures of brain age and disease over the lifespan based on a deep brain network and 14 468 individuals worldwide. Brain, 143(7), 2312–2324. https://doi.org/10.1093/brain/awaa160

Beheshti, I., Nugent, S., Potvin, O., & Duchesne, S. (2019). Bias-adjustment in neuroimaging-based brain age frameworks: A robust scheme. NeuroImage: Clinical, 24, 102063. https://doi.org/10.1016/j.nicl.2019.102063

Bookheimer, S. Y., Salat, D. H., Terpstra, M., Ances, B. M., Barch, D. M., Buckner, R. L., Burgess, G. C., Curtiss, S. W., Diaz-Santos, M., Elam, J. S., Fischl, B., Greve, D. N., Hagy, H. A., Harms, M. P., Hatch, O. M., Hedden, T., Hodge, C., Japardi, K. C., Kuhn, T. P., … Yacoub, E. (2019). The Lifespan Human Connectome Project in Aging: An overview. NeuroImage, 185, 335–348. https://doi.org/10.1016/j.neuroimage.2018.10.009

Butler, E. R., Chen, A., Ramadan, R., Le, T. T., Ruparel, K., Moore, T. M., Satterthwaite, T. D., Zhang, F., Shou, H., Gur, R. C., Nichols, T. E., & Shinohara, R. T. (2021). Pitfalls in brain age analyses. Human Brain Mapping, 42(13), 4092–4101. https://doi.org/10.1002/hbm.25533 Choi, S. W., Mak, T. S.-H., & O’Reilly, P. F. (2020). Tutorial: A guide to performing polygenic risk score analyses. Nature Protocols, 15(9), Article 9. https://doi.org/10.1038/s41596-020-0353-1

Cole, J. H. (2020). Multimodality neuroimaging brain-age in UK biobank: Relationship to biomedical, lifestyle, and cognitive factors. Neurobiology of Aging, 92, 34–42. https://doi.org/10.1016/j.neurobiolaging.2020.03.014

Cole, J. H., Raffel, J., Friede, T., Eshaghi, A., Brownlee, W. J., Chard, D., De Stefano, N., Enzinger, C., Pirpamer, L., Filippi, M., Gasperini, C., Rocca, M. A., Rovira, A., Ruggieri, S., Sastre-Garriga, J., Stromillo, M. L., Uitdehaag, B. M. J., Vrenken, H., Barkhof, F., … Group, M. study. (2020). Longitudinal Assessment of Multiple Sclerosis with the Brain-Age Paradigm. Annals of Neurology, 88(1), 93–105. https://doi.org/10.1002/ana.25746

Cumplido-Mayoral, I., García-Prat, M., Operto, G., Falcon, C., Shekari, M., Cacciaglia, R., Milà-Alomà, M., Lorenzini, L., Ingala, S., Meije Wink, A., Mutsaerts, H. J., Minguillón, C., Fauria, K., Molinuevo, J. L., Haller, S., Chetelat, G., Waldman, A., Schwarz, A. J., Barkhof, F., … OASIS study. (2023). Biological brain age prediction using machine learning on structural neuroimaging data: Multi-cohort validation against biomarkers of Alzheimer’s disease and neurodegeneration stratified by sex. ELife, 12, e81067. https://doi.org/10.7554/eLife.81067

de Lange, A.-M. G., & Cole, J. H. (2020). Commentary: Correction procedures in brain-age prediction. NeuroImage: Clinical, 26, 102229. https://doi.org/10.1016/j.nicl.2020.102229

Demontis, D., Walters, R. K., Martin, J., Mattheisen, M., Als, T. D., Agerbo, E., Baldursson, G., Belliveau, R., Bybjerg-Grauholm, J., Bækvad-Hansen, M., Cerrato, F., Chambert, K., Churchhouse, C., Dumont, A., Eriksson, N., Gandal, M., Goldstein, J. I., Grasby, K. L., Grove, J., … Neale, B. M. (2019). Discovery of the first genome-wide significant risk loci for attention deficit/hyperactivity disorder. Nature Genetics, 51(1), Article 1. https://doi.org/10.1038/s41588-018-0269-7

Denissen, S., Engemann, D. A., De Cock, A., Costers, L., Baijot, J., Laton, J., Penner, I., Grothe, M., Kirsch, M., D’hooghe, M. B., D’Haeseleer, M., Dive, D., De Mey, J., Van Schependom, J., Sima, D. M., & Nagels, G. (2022). Brain age as a surrogate marker for cognitive performance in multiple sclerosis. European Journal of Neurology, 29(10), 3039–3049. https://doi.org/10.1111/ene.15473

Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carré, G., Marquéz, J. R. G., Gruber, B., Lafourcade, B., Leitão, P. J., Münkemüller, T., McClean, C., Osborne, P. E., Reineking, B., Schröder, B., Skidmore, A. K., Zurell, D., & Lautenbach, S. (2013). Collinearity: A review of methods to deal with it and a simulation study evaluating their performance. Ecography, 36(1), 27–46. https://doi.org/10.1111/j.1600-0587.2012.07348.x

Dubois, J., Galdi, P., Paul, L. K., & Adolphs, R. (2018). A distributed brain network predicts general intelligence from resting-state human neuroimaging data. Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1756), 20170284. https://doi.org/10.1098/rstb.2017.0284

Elliott, M. L., Knodt, A. R., Cooke, M., Kim, M. J., Melzer, T. R., Keenan, R., Ireland, D., Ramrakha, S., Poulton, R., Caspi, A., Moffitt, T. E., & Hariri, A. R. (2019). General functional connectivity: Shared features of resting-state and task fMRI drive reliable and heritable individual differences in functional brain networks. NeuroImage, 189, 516–532. https://doi.org/10.1016/j.neuroimage.2019.01.068

Fair, D. A., Schlaggar, B. L., Cohen, A. L., Miezin, F. M., Dosenbach, N. U. F., Wenger, K. K., Fox, M. D., Snyder, A. Z., Raichle, M. E., & Petersen, S. E. (2007). A method for using blocked and event-related fMRI data to study “resting state” functional connectivity. NeuroImage, 35(1), 396–405. https://doi.org/10.1016/j.neuroimage.2006.11.051

Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., Montillo, A., Makris, N., Rosen, B., & Dale, A. M. (2002). Whole Brain Segmentation. Neuron, 33(3), 341–355. https://doi.org/10.1016/S0896-6273(02)00569-X

Franke, K., & Gaser, C. (2019). Ten Years of BrainAGE as a Neuroimaging Biomarker of Brain Aging: What Insights Have We Gained? Frontiers in Neurology, 10, 789. https://doi.org/10.3389/fneur.2019.00789

Glasser, M. F., Smith, S. M., Marcus, D. S., Andersson, J. L. R., Auerbach, E. J., Behrens, T. E. J., Coalson, T. S., Harms, M. P., Jenkinson, M., Moeller, S., Robinson, E. C., Sotiropoulos, S. N., Xu, J., Yacoub, E., Ugurbil, K., & Van Essen, D. C. (2016). The Human Connectome Project’s neuroimaging approach. Nature Neuroscience, 19(9), 1175–1187. https://doi.org/10.1038/nn.4361

Glasser, M. F., Sotiropoulos, S. N., Wilson, J. A., Coalson, T. S., Fischl, B., Andersson, J. L., Xu, J., Jbabdi, S., Webster, M., Polimeni, J. R., Van Essen, D. C., & Jenkinson, M. (2013). The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage, 80, 105–124. https://doi.org/10.1016/j.neuroimage.2013.04.127

Gordon, E. M., Laumann, T. O., Adeyemo, B., Huckins, J. F., Kelley, W. M., & Petersen, S. E. (2016). Generation and Evaluation of a Cortical Area Parcellation from Resting-State Correlations. Cerebral Cortex, 26(1), 288–303. https://doi.org/10.1093/cercor/bhu239

Gratton, C., Laumann, T. O., Nielsen, A. N., Greene, D. J., Gordon, E. M., Gilmore, A. W., Nelson, S. M., Coalson, R. S., Snyder, A. Z., Schlaggar, B. L., Dosenbach, N. U. F., & Petersen, S. E. (2018). Functional Brain Networks Are Dominated by Stable Group and Individual Factors, Not Cognitive or Daily Variation. Neuron, 98(2), 439-452.e5. https://doi.org/10.1016/j.neuron.2018.03.035

Harms, M. P., Somerville, L. H., Ances, B. M., Andersson, J., Barch, D. M., Bastiani, M., Bookheimer, S. Y., Brown, T. B., Buckner, R. L., Burgess, G. C., Coalson, T. S., Chappell, M. A., Dapretto, M., Douaud, G., Fischl, B., Glasser, M. F., Greve, D. N., Hodge, C., Jamison, K. W., … Yacoub, E. (2018). Extending the Human Connectome Project across ages: Imaging protocols for the Lifespan Development and Aging projects. NeuroImage, 183, 972–984. https://doi.org/10.1016/j.neuroimage.2018.09.060

Horien, C., Noble, S., Greene, A. S., Lee, K., Barron, D. S., Gao, S., O’Connor, D., Salehi, M., Dadashkarimi, J., Shen, X., Lake, E. M. R., Constable, R. T., & Scheinost, D. (2020). A hitchhiker’s guide to working with large, open-source neuroimaging datasets. Nature Human Behaviour, 5(2), 185–193. https://doi.org/10.1038/s41562-020-01005-4

Jirsaraie, R. J., Gorelik, A. J., Gatavins, M. M., Engemann, D. A., Bogdan, R., Barch, D. M., & Sotiras, A. (2023). A systematic review of multimodal brain age studies: Uncovering a divergence between model accuracy and utility. Patterns, 4(4), 100712. https://doi.org/10.1016/j.patter.2023.100712

Jirsaraie, R. J., Kaufmann, T., Bashyam, V., Erus, G., Luby, J. L., Westlye, L. T., Davatzikos, C., Barch, D. M., & Sotiras, A. (2023). Benchmarking the generalizability of brain age models: Challenges posed by scanner variance and prediction bias. Human Brain Mapping, 44(3), 1118–1128. https://doi.org/10.1002/hbm.26144

Khojaste-Sarakhsi, M., Haghighi, S. S., Ghomi, S. M. T. F., & Marchiori, E. (2022). Deep learning for Alzheimer’s disease diagnosis: A survey. Artificial Intelligence in Medicine, 130, 102332. https://doi.org/10.1016/j.artmed.2022.102332

Le, T. T., Kuplicki, R. T., McKinney, B. A., Yeh, H.-W., Thompson, W. K., Paulus, M. P., Tulsa 1000 Investigators, Aupperle, R. L., Bodurka, J., Cha, Y.-H., Feinstein, J. S., Khalsa, S. S., Savitz, J., Simmons, W. K., & Victor, T. A. (2018). A Nonlinear Simulation Framework Supports Adjusting for Age When Analyzing BrainAGE. Frontiers in Aging Neuroscience, 10. https://www.frontiersin.org/articles/10.3389/fnagi.2018.00317

Liang, H., Zhang, F., & Niu, X. (2019). Investigating systematic bias in brain age estimation with application to post-traumatic stress disorders. Human Brain Mapping, 40(11), 3143–3152. https://doi.org/10.1002/hbm.24588

Luby, J. L. (2010). Preschool Depression: The Importance of Identification of Depression Early in Development. Current Directions in Psychological Science, 19(2), 91–95. https://doi.org/10.1177/0963721410364493

Molnar, C. (2019). Interpretable Machine Learning. A Guide for Making Black Box Models Explainable. https://christophm.github.io/interpretable-ml-book/

Nimon, K., Lewis, M., Kane, R., & Haynes, R. M. (2008). An R package to compute commonality coefficients in the multiple regression case: An introduction to the package and a practical example. Behavior Research Methods, 40(2), 457–466. https://doi.org/10.3758/BRM.40.2.457

Pat, N., Wang, Y., Anney, R., Riglin, L., Thapar, A., & Stringaris, A. (2022). Longitudinally stable, brain‐based predictive models mediate the relationships between childhood cognition and socio‐demographic, psychological and genetic factors. Human Brain Mapping, hbm.26027. https://doi.org/10.1002/hbm.26027

Pat, N., Wang, Y., Bartonicek, A., Candia, J., & Stringaris, A. (2022). Explainable machine learning approach to predict and explain the relationship between task-based fMRI and individual differences in cognition. Cerebral Cortex, bhac235. https://doi.org/10.1093/cercor/bhac235

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12(85), 2825–2830.

Poldrack, R. A., Huckins, G., & Varoquaux, G. (2020). Establishment of Best Practices for Evidence for Prediction: A Review. JAMA Psychiatry, 77(5), 534–540. https://doi.org/10.1001/jamapsychiatry.2019.3671

Rasero, J., Sentis, A. I., Yeh, F.-C., & Verstynen, T. (2021). Integrating across neuroimaging modalities boosts prediction accuracy of cognitive ability. PLOS Computational Biology, 17(3), e1008347. https://doi.org/10.1371/journal.pcbi.1008347

Ray-Mukherjee, J., Nimon, K., Mukherjee, S., Morris, D. W., Slotow, R., & Hamer, M. (2014). Using commonality analysis in multiple regressions: A tool to decompose regression effects in the face of multicollinearity. Methods in Ecology and Evolution, 5(4), 320–328. https://doi.org/10.1111/2041-210X.12166

Robinson, E. C., Garcia, K., Glasser, M. F., Chen, Z., Coalson, T. S., Makropoulos, A., Bozek, J., Wright, R., Schuh, A., Webster, M., Hutter, J., Price, A., Cordero Grande, L., Hughes, E., Tusor, N., Bayly, P. V., Van Essen, D. C., Smith, S. M., Edwards, A. D., … Rueckert, D. (2018). Multimodal surface matching with higher-order smoothness constraints. NeuroImage, 167, 453–465. https://doi.org/10.1016/j.neuroimage.2017.10.037

Rokicki, J., Wolfers, T., Nordhøy, W., Tesli, N., Quintana, D. S., Alnæs, D., Richard, G., de Lange, A.-M. G., Lund, M. J., Norbom, L., Agartz, I., Melle, I., Nærland, T., Selbæk, G., Persson, K., Nordvik, J. E., Schwarz, E., Andreassen, O. A., Kaufmann, T., & Westlye, L. T. (2021). Multimodal imaging improves brain age prediction and reveals distinct abnormalities in patients with psychiatric and neurological disorders. Human Brain Mapping, 42(6), 1714–1726. https://doi.org/10.1002/hbm.25323

Satterthwaite, T. D., Connolly, J. J., Ruparel, K., Calkins, M. E., Jackson, C., Elliott, M. A., Roalf, D. R., Hopson, R., Prabhakaran, K., Behr, M., Qiu, H., Mentch, F. D., Chiavacci, R., Sleiman, P. M. A., Gur, R. C., Hakonarson, H., & Gur, R. E. (2016). The Philadelphia Neurodevelopmental Cohort: A publicly available resource for the study of normal and abnormal brain development in youth. NeuroImage, 124, 1115–1119. https://doi.org/10.1016/j.neuroimage.2015.03.056

Smith, S. M., Vidaurre, D., Alfaro-Almagro, F., Nichols, T. E., & Miller, K. L. (2019). Estimation of brain age delta from brain imaging. NeuroImage, 200, 528–539. https://doi.org/10.1016/j.neuroimage.2019.06.017

Somerville, L. H., Bookheimer, S. Y., Buckner, R. L., Burgess, G. C., Curtiss, S. W., Dapretto, M., Elam, J. S., Gaffrey, M. S., Harms, M. P., Hodge, C., Kandala, S., Kastman, E. K., Nichols, T. E., Schlaggar, B. L., Smith, S. M., Thomas, K. M., Yacoub, E., Van Essen, D. C., & Barch, D. M. (2018). The Lifespan Human Connectome Project in Development: A large-scale study of brain connectivity development in 5–21 year olds. NeuroImage, 183, 456–468. https://doi.org/10.1016/j.neuroimage.2018.08.050

Sperling, R. A., Bates, J. F., Cocchiarella, A. J., Schacter, D. L., Rosen, B. R., & Albert, M. S. (2001). Encoding novel face-name associations: A functional MRI study. Human Brain Mapping, 14(3), 129–139. https://doi.org/10.1002/hbm.1047

Sripada, C., Angstadt, M., Rutherford, S., Kessler, D., Kim, Y., Yee, M., & Levina, E. (2019). Basic Units of Inter-Individual Variation in Resting State Connectomes. Scientific Reports, 9(1), Article 1. https://doi.org/10.1038/s41598-018-38406-5

Sripada, C., Angstadt, M., Rutherford, S., Taxali, A., & Shedden, K. (2020). Toward a “treadmill test” for cognition: Improved prediction of general cognitive ability from the task activated brain. Human Brain Mapping, 41(12), 3186–3197. https://doi.org/10.1002/hbm.25007

Stigler, S. M. (1997). Regression towards the mean, historically considered. Statistical Methods in Medical Research, 6(2), 103–114. https://doi.org/10.1177/096228029700600202

Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., Liu, B., Matthews, P., Ong, G., Pell, J., Silman, A., Young, A., Sprosen, T., Peakman, T., & Collins, R. (2015). UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age. PLOS Medicine, 12(3), e1001779. https://doi.org/10.1371/journal.pmed.1001779

Tetereva, A., Li, J., Deng, J. D., Stringaris, A., & Pat, N. (2022). Capturing brain‐cognition relationship: Integrating task‐based fMRI across tasks markedly boosts prediction and test‐retest reliability. NeuroImage, 263, 119588. https://doi.org/10.1016/j.neuroimage.2022.119588

Vieira, B. H., Pamplona, G. S. P., Fachinello, K., Silva, A. K., Foss, M. P., & Salmon, C. E. G. (2022). On the prediction of human intelligence from neuroimaging: A systematic review of methods and reporting. Intelligence, 93, 101654. https://doi.org/10.1016/j.intell.2022.101654

Vos De Wael, R., Benkarim, O., Paquola, C., Lariviere, S., Royer, J., Tavakol, S., Xu, T., Hong, S.-J., Langs, G., Valk, S., Misic, B., Milham, M., Margulies, D., Smallwood, J., & Bernhardt, B. C. (2020). BrainSpace: A toolbox for the analysis of macroscale gradients in neuroimaging and connectomics datasets. Communications Biology, 3(1), 103. https://doi.org/10.1038/s42003-020-0794-7

Woolrich, M. W., Ripley, B. D., Brady, M., & Smith, S. M. (2001). Temporal Autocorrelation in Univariate Linear Modeling of FMRI Data. NeuroImage, 14(6), 1370–1386. https://doi.org/10.1006/nimg.2001.0931

Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2), 301–320. https://doi.org/10.1111/j.1467-9868.2005.00503.x

Author Response

We thank the reviewers for their suggestions. We are confident in the model that predicts odor vs odor (OCT-MCH) preference using calcium activity, but we acknowledge the relative weakness of the model that predicts odor (OCT) vs air preference. We are preparing an updated manuscript that will prioritize our interpretation of the OCT-MCH results and more fully document uncertainties around our estimates of prediction capacity.

Reviewer #1 (Public Review):

Summary: The authors seek to establish what aspects of nervous system structure and function may explain behavioral differences across individual fruit flies. The behavior in question is a preference for one odor or another in a choice assay. The variables related to neural function are odor responses in olfactory receptor neurons or in the second-order projection neurons, measured via calcium imaging. A different variable related to neural structure is the density of a presynaptic protein BRP. The authors measure these variables in the same fly along with the behavioral bias in the odor assays. Then they look for correlations across flies between the structure-function data and the behavior.

Strengths: Where behavioral biases originate is a question of fundamental interest in the field. In an earlier paper (Honegger 2019) this group showed that flies do vary with regard to odor preference, and that there exists neural variation in olfactory circuits, but did not connect the two in the same animal. Here they do, which is a categorical advance, and opens the door to establishing a correlation. The authors inspect many such possible correlations. The underlying experiments reflect a great deal of work, and appear to be done carefully. The reporting is clear and transparent: All the data underlying the conclusions are shown, and associated code is available online.

We are glad to hear the reviewer is supportive of the general question and approach.

Weaknesses: The results are overstated. The correlations reported here are uniformly small, and don't inspire confidence that there is any causal connection. The main problems are

We are working on a revision that overhauls the interpretations of the results. We recognize that the current version inadequately distinguishes the results that we have high confidence in (specifically, PC2 of our Ca++ data as a predictor of OCT-MCH preference) versus results that are suggestive but not definitive (such as the PC1 of Ca++ data as a predictor of Air-OCT preference).

It’s true that the correlations are small, with r2 values typically in the 0.1-0.2 range. That said, we would call it a victory if we could explain 10 to 20% of the variance of a behavior measure, captured in a 3 minute experiment, with a circuit correlate. This is particularly true because, as the reviewer notes, the behavioral measurement is noisy.

1) The target effect to be explained is itself very weak. Odor preference of a given fly varies considerably across time. The systematic bias distinguishing one fly from another is small compared to the variability. Because the neural measurements are by necessity separated in time from the behavior, this noise places serious limits on any correlation between the two.

This is broadly correct, though to quibble, it’s our measurement of odor preference which varies considerably over time. We are reasonably confident that the more variance in our measurements can be attributed to sampling error than changes to true preference over time. As evidence, the correlation in sequential measures of individual odor preference, with delays of 3 hours or 24 hours, are not obviously different. We are separately working on methodological improvements to get more precise estimates of persistent individual odor preference, using averages of multiple, spaced measurements. This is promising, but beyond the scope of this study.

2) The correlations reported here are uniformly weak and not robust. In several of the key figures, the elimination of one or two outlier flies completely abolishes the relationship. The confidence bounds on the claimed correlations are very broad. These uncertainties propagate to undermine the eventual claims for a correspondence between neural and behavioral measures.

We are broadly receptive to this criticism. The lack of robustness of some results comes from the fundamental challenge of this work: measuring behavior is noisy at the individual level. Measuring Ca++ is also somewhat noisy. Correlating the two will be underpowered unless the sample size is huge (which is impractical, as each data point requires a dissection and live imaging session) or the effect size is large (which is generally not the case in biology). In the current version we tried to in some sense to avoid discussing these challenges head-on, instead trying to focus on what we thought were the conclusions justified by our experiments with sample sizes ranging from 20 to 60. We are working on a revision that is more candid about these challenges.

That said, we believe the result we view as the most exciting — that PC2 of Ca++ responses predicts OCT-MCH preference — is robust. 1) It is based on a training set with 47 individuals and a test set composed of 22 individuals. The p-value is sufficiently low in each of these sets (0.0063 and 0.0069, respectively) to pass an overly stringent Bonferonni correction for the 5 tests (each PC) in this analysis. 2) The BRP immunohistochemistry provides independent evidence that is consistent with this result — PC2 that predicts behavior (p = 0.03 from only one test) and has loadings that contrast DC2 and DM2. Taken together, these results are well above the field-standard bar of statistical robustness.

In the revision we are working on, we are explicit that this is the (one) result we have high confidence in. We believe this result convincingly links Ca++ and behavior, and warrants spotlighting. We have less confidence in other results, and say so, and we hope this addresses concerns about overstating our results.

3) Some aspects of the statistical treatment are unusual. Typically a model is proposed for the relationship between neuronal signals and behavior, and the model predictions are correlated with the actual behavioral data. The normal practice is to train the model on part of the data and test it on another part. But here the training set at times includes the testing set, which tends to give high correlations from overfitting. Other times the testing set gives much higher correlations than the training set, and then the results from the testing set are reported. Where the authors explored many possible relationships, it is unclear whether the significance tests account for the many tested hypotheses. The main text quotes the key results without confidence limits.

Our primary analyses are exactly what the reviewer describes, scatter plots and correlations of actual behavioral measures against predicted measures. We produced test data in separate experiments, conducted weeks to months after models were fit on training data. This is more rigorous than splitting into training and test sets data collected in a single session, as batch/environmental effects reduce the independence of data collected within a single session.

We only collected a test set when our training set produced a promising correlation between predicted and actual behavioral measures. We never used data from test sets to train models. In our main figures, we showed scatter plots that combined test and training data, as the training and test partitions had similar correlations.

We are unsure what the reviewer means by instances where we explored many possible relationships. The greatest number of comparisons that could lead to the rejection of a null hypothesis was 5 (corresponding to the top 5 PCs of Ca++ response variation or Brp signal). We were explicit that the p-values reported were nominal. As mentioned above, applying a Bonferroni correction for n=5 comparisons to either the training or test correlations from the Ca++ to OCT-MCH preference model remains significant at alpha=0.05.

Our revision will include confidence limits.

Reviewer #2 (Public Review):

Summary:

The authors aimed to identify the neural sources of behavioral variation in a decision between odor and air, or between two odors.

Strengths:

-The question is of fundamental importance.

-The behavioral studies are automated, and high-throughput.

-The data analyses are sophisticated and appropriate.

-The paper is clear and well-written aside from some strong wording.

-The figures beautifully illustrate their results.

-The modeling efforts mechanistically ground observed data correlations.

We are glad to read that the reviewer sees these strengths in the study. We hope the forthcoming revision will address the strong wording.

Weaknesses:

-The correlations between behavioral variations and neural activity/synapse morphology are (i) relatively weak, (ii) framed using the inappropriate words "predict", "link", and "explain", and (iii) sometimes non-intuitive (e.g., PC 1 of neural activity).

Taking each of these points in turn: i) It would indeed be nicer if our empirical correlations are higher. One quibble: we primarily report relatively weak correlations between measurements of behavior and Ca++/Brp. This could be the case even when the correlation between true behavior and Ca++/Brp is higher. Our analysis of the potential correlation between latent behavioral and Ca++ signals was an attempt to tease these relationships apart. The analysis suggests that there could, in fact, be a high underlying correlation between behavior and these circuit features (though the error bars on these inferences are wide).

ii) We are working to guarantee that all such words are used appropriately. “Predict” can often be appropriate in this context, as a model predicts true data values. Explain can also be appropriate, as X “explaining” a portion of the variance of Y is synonymous with X and Y being correlated. We cannot think of formal uses of “link,” and are revising the manuscript to resolve any inappropriate word choice.

iii) If the underlying biology is rooted in non-intuitive relationships, there’s unfortunately not much we can do about it. We chose to use PCs of our Ca++/Brp data as predictors to deal with the challenge of having many potential predictors (odor-glomerular responses) and relatively few output variables (behavioral bias). Thus, using PCs is a conservative approach to deal with multiple comparisons. Because PCs are just linear transformations of the original data, interpreting them is relatively easy, and in interpreting PC1 and PC2, we were able to identify simple interpretations (total activity and the difference between DC2 and DM2 activation, respectively). All in all, we remain satisfied with this approach as a means to both 1) limit multiple comparisons and 2) interpret simple meanings from predictive PCs.

-No attempts were made to perturb the relevant circuits to establish a causal relationship between behavioral variations and functional/morphological variations.

We did conduct such experiments, but we did not report them because they had negative results that we could not definitively interpret. We used constitutive and inducible effectors to alter the physiology of ORNs projecting to DC2 and DM2. We also used UAS-LRP4 and UAS-LRP4-RNAi to attempt to increase and decrease the extent of Brp puncta in ORNs projecting to DC2 and DM2. None of these manipulations had a significant effect on mean odor preference in the OCT-MCH choice, which was the behavioral focus of these experiments. We were unable to determine if the effectors had the intended effects in the targeted Gal4 lines, particularly in the LRP experiments, so we could not rule out that our negative finding reflected a technical failure. We are reviewing these results to determine if they warrant including as a negative finding in the revision.

We believe that even if these negative results are not technical failures, they are not necessarily inconsistent with the analyses correlating features of DC2 and DM2 to behavior. Specifically, we suspect that there are correlated fluctuations in glomerular Ca++ responses and Brp across individuals, due to fluctuations in the developmental spatial patterning of the antennal lobe. Thus, the DC2-DM2 predictor may represent a slice/subset of predictors distributed across the antennal lobe. This would also explain how we “got lucky” to find two glomeruli as predictors of behavior, when were only able to image a small portion of the glomeruli. In analyses we did not report, we explored this possibility using the AL computational model. We are likely to include this interpretation in the revised discussion.

Reviewer #3 (Public Review):

Churgin et. al. seeks to understand the neural substrates of individual odor preference in the Drosophila antennal lobe, using paired behavioral testing and calcium imaging from ORNs and PNs in the same flies, and testing whether ORN and PN odor responses can predict behavioral preference. The manuscript's main claims are that ORN activity in response to a panel of odors is predictive of the individual's preference for 3-octanol (3-OCT) relative to clean air, and that activity in the projection neurons is predictive of both 3-OCT vs. air preference and 3-OCT vs. 4-methylcyclohexanol (MCH). They find that the difference in density of fluorescently-tagged brp (a presynaptic marker) in two glomeruli (DC2 and DM2) trends towards predicting behavioral preference between 3-oct vs. MCH. Implementing a model of the antennal lobe based on the available connectome data, they find that glomerulus-level variation in response reminiscent of the variation that they observe can be generated by resampling variables associated with the glomeruli, such as ORN identity and glomerular synapse density.

Strengths:

The authors investigate a highly significant and impactful problem of interest to all experimental biologists, nearly all of whom must often conduct their measurements in many different individuals and so have a vested interest in understanding this problem. The manuscript represents a lot of work, with challenging paired behavioral and neural measurements.

Weaknesses:

The overall impression is that the authors are attempting to explain complex, highly variable behavioral output with a comparatively limited set of neural measurements…

We would say that we are attempting to explain a simple, highly variable behavioral measure with a comparatively limited set of neural measurements. I.e. we make no claims to explain the complex behavioral components of odor choice, like locomotion, reversals at the odor boundary, etc.

Given the degree of behavioral variability they observe within an individual (Figure 1- supp 1) which implies temporal/state/measurement variation in behavior, it's unclear that their degree of sampling can resolve true individual variability (what they call "idiosyncrasy") in neural responses, given the additional temporal/state/measurement variation in neural responses.

We are confident that different Ca++ recordings are statistically different. This is borne out in the analysis of repeated Ca++ recordings in this study, which finds that the significant PCs of Ca++ variation contain 77% of the variation in that data. That this variation is persistent over time and across hemispheres was assessed in Honegger & Smith, et al., 2019. We are thus confident that there is true individuality in neural responses (Note, we prefer not to call it “individual variability” as this could refer to variability within individuals, not variability across individuals.) It is a separate question of whether individual differences in neural responses bear some relation to individual differences in behavioral biases. That was the focus of this study, and our finding of a robust correlation between PC2 of Ca++ responses and OCT-MCH preference indicates a relation. Because behavior and Ca++ were collected with an hours-to-day long gap, this implies that there are latent versions of both behavioral bias and Ca++ response that are stable on timescales at least that long.

The statistical analyses in the manuscript are underdeveloped, and it's unclear the degree to which the correlations reported have explanatory (causative) power in accounting for organismal behavior.

With respect, we do not think our statistical analyses are underdeveloped, though we acknowledge that the detailed reviewer suggestions included the helpful suggestion to include uncertainty in the estimation of confidence intervals around the point estimate of the strength of correlation between latent behavioral and Ca++ response states. We are considering those suggestions and anticipate responding to them in the revision.

It is indeed a separate question whether the correlations we observed represent causal links from Ca++ to behavior (though our yoked experiment suggests there is not a behavior-to-Ca++ causal relationship — at least one where odor experience through behavior is an upstream cause). We attempted to be precise in indicating that our observations are correlations. That is why we used that word in the title, as an example. In the revision, we are working to make sure this is appropriately reflected in all word choice across the paper.

Reviewer #1 (Public Review):

Summary: The authors seek to establish what aspects of nervous system structure and function may explain behavioral differences across individual fruit flies. The behavior in question is a preference for one odor or another in a choice assay. The variables related to neural function are odor responses in olfactory receptor neurons or in the second-order projection neurons, measured via calcium imaging. A different variable related to neural structure is the density of a presynaptic protein BRP. The authors measure these variables in the same fly along with the behavioral bias in the odor assays. Then they look for correlations across flies between the structure-function data and the behavior.

Strengths: Where behavioral biases originate is a question of fundamental interest in the field. In an earlier paper (Honegger 2019) this group showed that flies do vary with regard to odor preference, and that there exists neural variation in olfactory circuits, but did not connect the two in the same animal. Here they do, which is a categorical advance, and opens the door to establishing a correlation. The authors inspect many such possible correlations. The underlying experiments reflect a great deal of work, and appear to be done carefully. The reporting is clear and transparent: All the data underlying the conclusions are shown, and associated code is available online.

Weaknesses: The results are overstated. The correlations reported here are uniformly small, and don't inspire confidence that there is any causal connection. The main problems are<br /> 1. The target effect to be explained is itself very weak. Odor preference of a given fly varies considerably across time. The systematic bias distinguishing one fly from another is small compared to the variability. Because the neural measurements are by necessity separated in time from the behavior, this noise places serious limits on any correlation between the two.<br /> 2. The correlations reported here are uniformly weak and not robust. In several of the key figures, the elimination of one or two outlier flies completely abolishes the relationship. The confidence bounds on the claimed correlations are very broad. These uncertainties propagate to undermine the eventual claims for a correspondence between neural and behavioral measures.<br /> 3. Some aspects of the statistical treatment are unusual. Typically a model is proposed for the relationship between neuronal signals and behavior, and the model predictions are correlated with the actual behavioral data. The normal practice is to train the model on part of the data and test it on another part. But here the training set at times includes the testing set, which tends to give high correlations from overfitting. Other times the testing set gives much higher correlations than the training set, and then the results from the testing set are reported. Where the authors explored many possible relationships, it is unclear whether the significance tests account for the many tested hypotheses. The main text quotes the key results without confidence limits.

I'm still a bit unsure of what the Da Vinci Code is

复杂的，优雅的，迅速的

WHY THIS METHOD - what it provides

depending on the image, the order of neurons responding to its features varies?

if you get any errors with this code, it's because you chose the wrong parameters file in the step above!!

These files are found from a zip https://github.com/MicrosoftLearning/AZ-140-Configuring-and-Operating-Microsoft-Azure-Virtual-Desktop - click on Code and then download the zip file. You can do quite a lot of the first lab outside of learnondemand environment, so you can save your files on your computer and inside learnondemand.