The model assumes both (1) no repeats in s and (2) that new k-mers generated by mutation are distinct from each other and from k-mers already in s. I believe assumption 2 is violated whenever two positions in s are Hamming-close, and D-hardest has average d₁ = 1.22. The paper attributes the divergence between the Mash-corrected estimator, and (1−J)/(1+J) to repeats, but it seems like it's possible that violations of assumption 2 are also at play. Does this seem likely?

You mention that longevity is negatively correlated with effective population size, which would create correlated shifts in selection intensity across many sites simultaneously in real data. I'm curious whether the H0 simulations, which use constant fitness profiles I believe, might underestimate the false positive rate in the empirical setting, since that correlated background signal isn't captured in the benchmark. Do you have a sense of how much this might affect the transfer of the FPR estimates to real data?

It sounds like the motivating concern is that ML rescoring could exploit construction artifacts, but Oktoberfest is primarily spectrum-driven, and rescoring compresses rather than amplifies the gap between generators. It might be useful to consider testing with a rescoring model that takes sequence-derived features (PLM embeddings) as direct input, where exploitation of construction fingerprints would be more mechanistically plausible?

Dear Authors,

Congratulations on the excellent preprint!

I have a question with regard to the dimensionality reduction step on the square-root transformed sphere. The methodology employs Tangent PCA, which creates a local linearization by projecting points onto the tangent space at the global Fréchet mean. As noted in the text, the Euclidean distance in this tangent plane effectively approximates the geodesic distance for points that are close to the Fréchet mean.

Given this constraint, how does GAIA perform on highly heterogeneous datasets, like whole-organism or maybe cross-tissue atlases, where distinct cell populations might be located very far from a single, global Fréchet mean on the hypersphere? Does the tangent approximation begin to distort the macro-relationships between highly divergent lineages at the edges of the projection, and have you explored the possibility of using multiple local tangent spaces (or something more clever) to preserve global geometry in these extreme cases?

Thank you for sharing this with the community.

The PIM values (Figure 5) determine the relative contribution of each selection metric to the final model, yet these appear to be point estimates from a single train/validation split. How stable are these PIM rankings across different random splits, bootstrap samples, or subsampling of the training data? If F1 score's dominance as the top weighted metric is sensitive to the specific data partition, this could affect reproducibility of MIXER-selected feature sets in independent applications. Have you characterized the variance of PIM estimates and does the ranking of metrics remain consistent?

The paper notes that MultiMIL relies on batch-corrected embeddings to handle technical confounders, and that explicitly adding covariates like age/sex didn't improve performance. But beyond technical batch effects, patients often have heterogeneous biological states like unique inflammatory signatures, co-morbidities, disease subtypes, that are real biology but not common to the disease state. These could still be predictive in smaller cohorts without reflecting shared disease mechanisms. Does the attention mechanism have any inherent safeguard against overfitting to patient-specific biological features? In the stability analysis comparing embeddings (scVI versus scGPT), were there cases where the model consistently attended to features that were biologically real but unique to specific patients rather than the shared phenotype?

You discuss preference-based methods (DPO) being used to bias toward stability or reduce MHC-I epitopes. How sensitive are these methods to the quality and balance of the preference pairs? What happens when preferences are noisy or when one class heavily outweighs the other?

You mention that "defining a specific algebra that accurately represents a particular trait remains an important task" but don't provide concrete methodology. What heuristics or systematic approaches would you recommend for researchers trying to map their specific biological trait to an appropriate algebraic structure?

Your ECOD-separated held-out set shows Pearson correlation of 0.299, while the COVID-19 benchmark achieves 0.496—a 65% difference. You attribute the held-out performance to chain concatenation (70% of those complexes), but concatenation affects 28.5% of your entire dataset. Could this performance gap indicate the model learns family-specific patterns rather than fully generalizable principles?

Does marker‑gene expression change monotonically along the CPM geodesic from root to leaf?

You observed that for ambiguous cases or high-levels of missing data, the model tended to predict the PUR population, suggesting it acts as a "default". Since PUR is an admixed population, does this imply the model learns that a state of high uncertainty or mixed/missing signals is most characteristic of admixed genomes in the training set? Could this "default" behavior be mitigated by training with a null or "uncertain" class?

What was the accuracy of the random forest classifier used to create the unified state annotations?

How does this calculation account for protein flexibility, which isn't represented in a single static model?

Sorry but I'm a bit confused here. It looks like $480 million is 59% of the total from 2023 ($819.6 million), but it says for 2013. Am I misreading this and it means that it was 59% of the total from 2013 which also just happens to be 59%?

For the DMS dataset for GFP there is a pretty sharp drop in fitness when 5 mutations is reached. For this model, in order to get accurate predictions for larger numbers of mutations, would much larger DMS datasets be needed. Do you see anyway of computationally advancing this so that it's not limited by experimental measures given the size of these spaces even with a handful of mutations?

Two questions:

First, do you think approaches using alternative modalities (e.g., visual explanations like https://arxiv.org/abs/2502.19400v2) could help detect the conceptual misalignments you identify?

Second, as models improve, do you envision needing continuously evolving datasets to detect new potemkin patterns, or do you think there might be an optimal set of evaluation tasks that would be more fundamentally robust to training advances?

Do you think adding additional sites would cause much of a performance difference?

In addition to this theory calculation, would it be possible with your data to look at empirical variance and that it behaves as expected?

Our group read this paper as part of our journal club and found the approach of using attention very interesting. We dug into a couple aspects that we wanted to explore further and released a notebook publication here: https://research.arcadiascience.com/pub/observation-geno-pheno-attention/release/1/, thanks for a great paper that inspired us to explore this topic further!

Is this technique novel? This seems like a good approach for adding in other features that can be relatively predicted using sequence only. Are there any plans to do that?

I know this isn't the goal of this paper, but I was curious if you had information about how performance metrics change on multiple video generations. For example, if you generated several video for the same topic and agent, how much do the metric vary around them?

Is there any particular reasoning for choosing N=5? I'm not familiar with how these evaluations have been done historically so this may be obvious but it's not clear to me why 5 would be natural choice.

This sum has 721 elements since 0 is included. I see that 720 of these represent possible elements, does the value of 0 represent a non-viable element?

Using mean pooling for the larger embeddings for some of the transformer based models makes sense, as that's such a common approach used. However, I was curious if you looked into any other pooling strategies and what impact that had? Or, using a small embedding model, how different performance looks like when the embeddings aren't reduced?

What was the motivation for the value 10,420? Is the ability to extract features pretty consistent (no the actual weights obviously but the behavior of being able to extract meaningful features) as long as the space is large enough?

Was the fourth layer chosen arbitrarily or did it have any particularly nice properties for visualization?

Is the discriminator learning something much more complicated than identifying something like how to identify MDH domains?

This seems to work well based on the results, was there previous work/concepts that lead to this as the architecture for the discriminator?

How computationally intensive was it to solve for all of the hyperparams for this model?

Do different values for percent of tokens masked have much of an impact on model performance?

Was there much variation around this?

Sorry if I missed this, but what was the motivation for choosing these particular discriminator models? They seem very reasonable given the results, but I'm curious how these two types of models were chosen based on the structure of the initial problem?

Would an "ideal" path be one where there is a single haplotype path that crosses every single \(a_{i}.u\) vertex? This would be something that generally exists for a given sample, but if present this would be the best path?

Just to verify I'm understanding this notation, \(a_{i}.u\) is being used like an accessor for components of the tuple \(a_{i}\)?

Sorry if this is clear, but I'm a little unclear on the notation. Is X the input data (so empirical results from a scRNA-seq experiment) and Y the generated dist? If so then are X' and Y' subsets of the respective distributions?

In this case does s denote tuning to how results are classified when calling true and false positives and negatives? Or weight terms in the F-measure score it self?

Is this a vector with only a value at position j? So a vector of size N with only position j having a value set, hence being different than x_{j}(t)?

It might be easier to see if the longest barcode was in a different color or had a dashed line overlayed on top of it.

It's hard to tell exactly which part of the structure is being referenced (at least for me). It might be helpful to add an annotation like a circle to show which area is being discussed.

The indexing of x_{ij} doesn't seem to be a unique element but a pair (u_{ij}, s_{ij}). Is this loss function calculating the difference between the spliced and unspliced differences combined?

This part is assuming that t has been set to 5, correct?

Wasn't scGPT v1 which out performed scGPT trained on a smaller pre-training data set?

Really interesting article!

Given the impact this had do you feel there are changes or criticisms needed around the review and publication process of the Bloom results? I'm also curious if you have any thoughts on how pre-print and open science can do a better job with contentious results and discussions around them.

Is this 1Mbp in either direction of a loci of interest? Just binning the human genome by 25 points gives about 1.7% of the genome within 1Mbp of these uniform bins. Depending on what percent of genes are associated with the traits of interest that could be very rare, or fairly common. Is there a way of viewing how impactful this result is in comparison to the size of the genome annotated as relevant to these traits?