Um, sorry, genomics isn’t really my area, so I might be misreading this, but the part that caught my attention was the block timing. The attention-style block running out of memory past length 8,192 on a single RTX 4090 while the Hydra block keeps going seemed like a big deal. I kept rereading that bit to make sure I understood that the cards were the same and only the long-range block changed.

I also liked that they averaged over four replicates and used GTEx fine-mapped eQTLs, because it makes the accuracy gains feel less like a lucky run. My brother always says memory is what really limits these models, and that 4090 example kind of proves it. I found some platforms covering rtx 4090 over at Orange Hardwares and Scan if anyone wants to try something like this on their own setup.