Small model delivers big performance

OpenAthena researchers have released MarinDNA, a genomic language model (gLM) with 1 billion parameters that reaches variant-effect-prediction (VEP) performance comparable to Evo 2 40B while running 2,330 times faster. The model uses a deliberately standard transformer architecture to reuse existing large-language-model infrastructure and training methods rather than reinventing the wheel.

The team identified data curation as the primary bottleneck for genomic models. Evo 2 excels across most regions but fails on enhancers, the only category deliberately omitted from its training data. To address this, the researchers projected human ENCODE annotations onto other mammals via sequence alignment and assembled a uniform, balanced dataset spanning genomic regions. This unified mix produced higher overall performance than a simple proportional blend of all data.

When scaling the model, the researchers observed an unexpected drop in zero-shot performance on the LLR metric for Mendelian missense variants as model size increased. Linear probing, however, revealed that internal representations continued to improve; the shortfall lay in the head layer rather than in learning capacity. After additional mixing experiments, the 1 billion-parameter checkpoint achieved the best results among single-sequence gLMs on Mendelian VEP: a modest zero-shot advantage over Evo 2 40B and a larger advantage in probing, while retaining simplicity and speed.

The authors caution that it remains unclear how much extra value single-sequence gLMs provide compared with models trained on whole-genome alignments or functional genomics data. They assess that the most immediate large-scale opportunity lies in the far greater number of species lacking such resources, where a fast, lightweight, standard model like MarinDNA could deliver practical predictions without requiring heavyweight infrastructure.