1 Introduction↩︎

Structure–property relationships [1]–[6] are foundational to the physical and biological sciences. Across proteins, small molecules and crystalline materials, observable functions and properties arise from the spatial, chemical and periodic organization of matter. In proteins, protein structures and conformations, long-range interactions and active-site geometry shape their biological functions [7], [8]; in chemicals, bonding, functional groups and stereochemistry contribute to their reactivity and molecular behaviour; and in materials, lattice symmetry, coordination motifs and periodic bonding networks dictate stability and electronic properties [9], [10]. Unraveling how these organizing principles connect structure to function, reactivity and physical response is therefore central to functional annotation, molecular design and materials discovery.

Yet, explaining these relationships mechanistically is rarely a straightforward pattern-matching problem. Evidence for a function, reaction or material property is distributed across local motifs, non-local contacts, chemical environments, conformational geometry and long-range periodic order. When annotating a protein or planning a synthesis route [11], [12], human experts integrate heterogeneous cues, compare competing structural hypotheses and apply biochemical, chemical or crystallographic constraints [13], [14]. Artificial intelligence (AI) can accelerate this process by retrieving knowledge and screening large structural spaces, but it faces a deeper representational challenge: to support expert-like scientific reasoning, a model must represent biological, chemical and materials structures as native objects and use them as evidence in explicit, inspectable reasoning.

Current scientific AI systems only partially meet this requirement because they often separate native structural representation from evidence-linked reasoning. Large language models provide a flexible interface to scientific knowledge, but when proteins, molecules and crystals are cast primarily as text, structural organization is compressed into strings or descriptions, making the resulting explanations depend largely on linguistic associations rather than directly addressable physical evidence [15]–[17]. Agentic systems extend such models through retrieval, tool use and workflow coordination [18]–[21], yet their scientific depth remains constrained by the structural competence of the foundation models they orchestrate. Domain-specific models, in contrast, often encode molecular graphs, protein structures or crystal lattices directly [9], [22], [23], but are usually optimized as task-specific predictors that output scores, labels or scalar properties without exposing the intermediate evidence behind their decisions. Thus, a critical gap remains: building a new reasoning paradigm that natively represents multidimensional scientific structures while producing reasoning traces that can be inspected against those structures.

Here we propose native structural reasoning as a foundation-model paradigm for structure–property analysis across biology, chemistry and materials science. We instantiate this paradigm in SciReasoner, a multimodal scientific foundation model that represents proteins, small molecules and periodic crystals through a unified structure-aware vocabulary. By native structural reasoning, we mean that structural tokens are not auxiliary descriptors appended to language, but addressable evidence units that can be combined, cited and checked within a generated reasoning trajectory. To this end, SciReasoner discretizes coordinates, topologies and periodic connectivities into scientific tokens that preserve domain-native information, including local geometry, stereochemistry, residue-level structural motifs, lattice symmetry and crystallographic bonding patterns. These tokens are integrated with language instructions in a single autoregressive model, enabling trajectories that interleave explanatory text with specific residues, molecular fragments, conformational cues or crystal descriptors.

This design changes the role of a scientific foundation model. Rather than predicting an answer after reading a structural prompt, SciReasoner generates an inspectable chain in which intermediate claims can be traced to explicit structural evidence. We test this capability in regimes where shortcut correlations are unreliable.

In the Cellular Component aspect of protein Gene Ontology prediction, SciReasoner shows its largest gains in low-homology and orphan-like proteins, where evolutionary transfer provides weak evidence, improving \(F_{\max}\) from 0.42 to 0.55. Its attention is enriched at contact-defined DNA-binding residues and localizes to protein–DNA interfaces, suggesting that functional predictions are supported by residues that physically mediate the molecular interaction. In retrosynthesis, SciReasoner raises single-step accuracy from 0.63 to 0.72 while decomposing products into reaction-relevant fragments, identifying strategic bond disconnections and verifying chemically valid precursors. In crystalline materials, the reasoning trajectories of SciReasoner assign higher importance to symmetry-distinct sites, local coordination environments and periodic bonding motifs associated with stability and band-gap variation, suggesting that it links electronic and thermodynamic behaviour to the structural features that physically govern them. Across these examples, the common behaviour is that structural organization acts as the working substrate of inference rather than a peripheral input descriptor.

Broad evaluation supports the generality of this structure-grounded modelling strategy. Across 86 benchmarks spanning proteins, DNA, RNA, small molecules, inorganic crystals, scientific question answering, property prediction and generation tasks, SciReasoner achieves state-of-the-art performance on 67 tasks. Double-blind expert evaluation further indicates that its reasoning traces are useful for scientific inspection, with experts preferring or tying SciReasoner over a frontier large language model in the overwhelming majority of comparisons. By representing structures as evidence-bearing units and linking them to inspectable conclusions, SciReasoner bridges accurate prediction and interpretable scientific inference, suggesting a route toward scientific AI systems that do not merely predict what property a structure has, but begin to explain why that structure gives rise to a particular function, reaction or material behaviour.

2 Results↩︎

2.1 Overview of SciReasoner↩︎

We present SciReasoner, a unified multimodal foundation model capable of explicit scientific sequence, structure, and textual reasoning within a single coherent output trajectory (Fig. [fig:main95figure]A). SciReasoner serves as a unified reasoning engine that maps diverse scientific inputs, including proteins, DNA/RNA, small molecules, and periodic crystals, into a shared cross-modal latent space. Unlike conventional computational approaches in materials and biology, this architecture provides a unified, mechanism-aware framework.

To integrate explicit 3D structural information into a linguistic token space, SciReasoner designs a structure-aware vocabulary designed to preserve physical and biochemical integrity (Fig. [fig:main95figure]A). The model utilizes unified tokens representing local motifs, 3D geometry, atomic bond types, and crystal space groups (Fig. [fig:main95figure]B). This resolves the severe limitations of conventional sub-word tokenization algorithms, which arbitrarily fragment molecular graphs and motifs, thereby destroying scientific semantics and corrupting the training process (Fig. [fig:main95figure]C).

To align this structural token space with natural language, we perform continued pretraining with a multi-stage training pipeline (Fig. [fig:main95figure]D, Top). Stage 1 serves as a warm-up training phase that anchors the newly introduced structure-aware tokens to basic topological, geometric, and chemical semantics without destabilizing the pretrained language space. This establishes a stable structural–linguistic interface, which is further developed through Stage 2 full-parameter multimodal training and Stage 3 annealed training with an increased proportion of question–answer (QA)-style data, thereby supporting native structural reasoning.

To enable robust multi-step reasoning, we developed a post-training framework named self-bootstrapped native structural reasoning (Fig. [fig:main95figure]D, Bottom). The post-training procedure uses reinforcement learning to connect the core semantic meanings of the structural vocabulary with a chain-of-thought (CoT) strategy. This allows the model to explicitly decompose complex inputs into interpretable physical factors, such as composition, symmetry, and connectivity, before executing final property predictions or design decisions. This generative approach circumvents a major bottleneck in the field: the near-complete absence of ground-truth reasoning trajectories that simultaneously bridge sequence, structure, and text. Intra-domain structural evidence grounding first teaches domain-specific experts how structural tokens serve as evidence for reasoning. Cross-domain reasoning consolidation then integrates these domain-local evidence-use patterns into unified capabilities, reducing the trajectory collapse frequently observed in standard alignment techniques.

The radar plot in Fig. [fig:main95figure]E (left) compares SciReasoner with four contemporary generalist LLMs (Opus-4.7 [24], GPT-5.5 [25], DeepSeek-V4-Pro [26] and Kimi-K2.6 [27]) across five capability sectors and four scientific modalities (proteins, DNA/RNA, small molecules and crystals). The suite spans numerical property prediction, sequence- and structure-level classification, generation and design, modality translation and scientific QA. SciReasoner expands the polygon further than every generalist baseline in every sector, with the largest margins on tasks that demand explicit structural reasoning, where text-only LLMs collapse to near-trivial scores.

We next compared SciReasoner with the published specialist that defines the prior state of the art on each benchmark (Fig. [fig:main95figure]E, right). Unlike these specialists, each of which is restricted to a fixed numerical or categorical output, SciReasoner produces free-form natural language alongside its predictions, yet still matches or exceeds them on the majority of tasks across the three task panels on the right: SciReasoner reaches a mean \(F_{\max}=0.59\) on DeepFRI-GO (averaged over the three aspects), surpassing SaProt [28] (\(0.52\)); attains \(0.88\) accuracy on subcellular localization, exceeding the ESM2 [29] (\(0.84\)); is on par with or above the specialist on DNA promoter and transcription-factor detection; and substantially outperforms RNA-function specialists, raising Isoform R\(^2\) from \(0.59\) to \(0.86\) and RNA protein interaction MCC from \(0.74\) to \(0.81\). On the DUD-E virtual-screening suite, SciReasoner matches the previously reported best AUC of \(0.76\) and improves the \(5.0\%\) enrichment factor from \(7.12\) to \(7.70\). Beyond closed-set predictions, the same model handles open-ended scientific language tasks that lie entirely outside the specialists’ output space, scoring \(0.85\) BertScore on biomedical QA and \(0.77\) ROUGE-L on protein general function description, and supplies an interpretable natural-language rationale for each predicted label.

High benchmark scores can be obtained by exploiting homology, scaffold or template-level shortcuts, and do not by themselves demonstrate that a model has captured the underlying science. Sections 2.2.1–2.2.4 therefore probe four structural behaviours of SciReasoner beyond average accuracy. On protein GO prediction (§2.2.1), SciReasoner shows its largest gain in the low-homology regime , improving Cellular Component \(F_{\max}\) at \(\leq30\%\) identity, by \(+0.21\) over BLAST [30] and by \(+0.13\) over ESM2, while maintaining stable reasoning quality across similarity bins, arguing against homology memorization. On retrosynthesis (§2.2.2), the chain-of-thought interleaves language with SMILES sub-fragments along a canonical analysis \(\rightarrow\) disconnection \(\rightarrow\) verification \(\rightarrow\) feasibility workflow, recovering literature reactants in \(5/5\) representative Retrosynthesis USPTO-50K cases versus \(2/5\) for RSGPT [31]. On 3D molecular similarity (§2.2.3), the per-molecule embedding clusters DUD-E [32] actives by binding-pocket geometry rather than 2D scaffold. On inorganic crystals (§2.2.4), the latent space separates C, Si and SiC into disjoint compositional clusters and, within each, orders polymorphs along smooth band-gap gradients. The accompanying reasoning traces explicitly invoke coordination changes, bonding topology and symmetry constraints as mechanistic evidence for linking crystal structure to properties. Consistent with this physically aligned representation, the formation-energy parity plot (Fig. 3C) reaches R\(^{2}=0.895\), well above competing models even where their absolute MAE is comparable, showing that SciReasoner captures the stability ranking rather than the marginal label distribution.

Across scientific tasks spanning proteins, DNA, RNA, small molecules and inorganic crystals, SciReasoner outperforms previous state-of-the-art methods on 67 tasks, and matches or exceeds published domain specialists on 26 benchmarks wherever such a baseline exists. To further exclude reliance on sequence, scaffold or composition-level shortcuts, we ablated structural inputs in representative protein, small-molecule and materials tasks and observed a consistent drop in performance, with the largest effects in problems governed by folded geometry, binding-site organization, molecular shape or crystal topology (Fig. 4). The same perturbation changed the reasoning traces: without structure, the reasoning process relied mainly on sequence motifs or chemical composition, whereas structural tokens led SciReasoner to use binding pockets, coordination environments and periodic connectivity as evidence for its predictions. To our knowledge, this is the first foundation model to enable sequence, structure and natural-language reasoning across all four scientific modalities within a single autoregressive trajectory, delivering specialist-level accuracy, interpretable structure-grounded chains of thought, and open-ended scientific language generation that no prior specialist or generalist model can provide.

2.2 Benchmarking SciReasoner across proteomic, molecular, and materials tasks↩︎

2.2.1 SciReasoner precisely predicts protein GO term and learns biologically meaningful structure↩︎

Assigning Gene Ontology (GO) terms [33], [34] to protein sequences is central to computational biology, translating rapidly growing sequence and structure databases [8] into interpretable functional annotations [35]. Since only a small fraction of proteins have experimentally verified functions [36], [37], accurate prediction across Molecular Function (MF), Biological Process (BP), and Cellular Component (CC) remains critical for protein characterization, pathway analysis, disease-gene interpretation, and therapeutic-target discovery [38], [39].

We evaluated GO-term prediction on DeepFRI-GO [39] and, to prevent label leakage, excluded from continued pretraining all proteins with \(>\)​30% sequence identity to the DeepFRI-GO test set. As shown in Fig. 1A, SciReasoner attains an overall (\(F_{\max}=0.59\)), outperforming sequence-based BLAST transfer (0.55) [30], structure-based Foldseek 3Di alignment (0.54) [40], fine-tuned protein language models including ESM2 (0.53) [29] and SaProt 650M AF2 (0.52) [28], and general-purpose reasoning LLMs, DeepSeek-V4-Pro (0.35) and GPT-5.5 (0.31). The gain is strongest for CC, where SciReasoner reaches 0.58 versus BLAST (0.49), and remains positive for BP (0.52 versus 0.51). MF is near saturation, with SciReasoner at 0.66, comparable to SaProt (0.67).

Overall, SciReasoner is the only method that leads or matches the best baseline across all branches, indicating that it integrates evidence beyond any single baseline family.

To determine whether these gains reflect genuine functional inference rather than memorized homology-based transfer, we stratified the test proteins by maximum BLAST percent identity to the training set (\(e\leq10^{-3}\), query coverage \(\geq50\%\), self-hits removed). The advantage of SciReasoner is largest precisely where sequence similarity becomes unreliable. In the CC \((0,30]\%\) identity bin, SciReasoner improves over BLAST by \(+0.21\) absolute \(F_{\max}\) (0.55 versus 0.34 in Fig. 1A), and outperforms ESM2 by \(+0.13\) absolute \(F_{\max}\) (0.55 versus 0.42). This low-homology advantage aligns with the design of SciReasoner: whereas BLAST relies on local sequence similarity, ESM2 encodes evolutionary and sequence-context patterns without explicit structural grounding, SciReasoner predicts GO terms through reasoning traces that integrate domain composition, localized motifs, structural environments, and reference proteins. Such local functional cues can remain conserved even when global sequence identity is weak.

We next tested whether SciReasoner grounds GO predictions in functionally relevant protein regions rather than superficial sequence patterns. Using DNA-binding proteins, where protein–DNA contacts provide structural ground truth, we extracted attention from the generated DNA-binding GO token (GO:0003677) to residue-level structure and sequence tracks and combined them into a per-residue profile. These profiles strongly align with DNA-binding sites (Fig. 1B): catabolite control protein A (UniProt: P25144) reaches AUROC (=0.91) and top-20% fold enrichment (\(=4.2\times\)), histone H2B type 1-A (UniProt: P70696) reaches AUROC (=0.83) and enrichment (\(=3.6\times\)), and DesT (UniProt: Q9HUS3) reaches AUROC (=0.78) and enrichment (\(=3.1\times\)). In all cases, the most attended residues fall within or immediately adjacent to annotated binding regions, and structural projections confirm that these residues cluster near bound DNA in three-dimensional space. Thus, SciReasoner does not merely assign the correct GO term. It concentrates its evidence on residues that are physically responsible for the corresponding molecular function.

Reinforcement learning further improves GO prediction across all three ontology aspects (Fig. 1C). After a brief exploration phase during the first \(\sim\!5\%\) of training, in which the science reward temporarily decreases under noisy advantages, all three trajectories rise and converge to stable plateaus. MF reaches the highest reward (\(\sim\!0.92\)), while BP and CC converge at comparable values (\(\sim\!0.89\)). The simultaneous improvement across all aspects indicates that RL does not merely exploit aspect-specific shortcuts, but instead strengthens a shared protein-function representation that supports functional, process, and localization annotations.

Because \(F_{\max}\) captures only final GO-term overlap, we also evaluated the reasoning traces underlying those predictions. To assess grounded biological inference beyond surface fluency, we used GPT-5.5 to score held-out BP traces on Overall Quality, Input Fidelity, Biological Plausibility, Reasoning Coherence, and Functional Hypothesis Quality. SciReasoner scores highest on every axis, with an Overall score of 8.33 versus 7.77 for SciReasoner-ColdStart and 6.96 for DeepSeek-V4-Pro [26] (Fig. 1D). Relative to the pre-RL coldstart model, the largest improvement is in Biological Plausibility (8.68 vs.), indicating that RL sharpens not only the fluency of the trace but also the biological relevance of the inferred function. The gap to DeepSeek-V4-Pro is concentrated on the biology-specific axes—Biological Plausibility (8.68 vs.) and Functional Hypothesis Quality (8.45 vs.)—and is far wider there than on Input Fidelity (8.27 vs.). Although DeepSeek-V4-Pro is a substantially larger general-purpose model, this pattern shows that domain-tuned scientific reasoning, rather than raw model scale alone, drives the improvement.

We stratified reasoning quality by BLAST similarity to test whether SciReasoner relies mainly on close homologues (Fig. 1E). Across seven similarity bins, SciReasoner maintains higher judge scores than both the coldstart checkpoint and DeepSeek-V4-Pro, including in low-similarity regimes where direct homology transfer is least reliable. This helps explain the low-homology gains in Fig. 1A. Rather than copying annotations from close neighbours, SciReasoner can combine weaker but conserved cues, such as local structural motifs, short sequence fragments, fold-level patterns, and biochemical context. These results suggest that its improved performance on low-similarity proteins arises from structure-grounded functional reasoning rather than memorization of homologous sequences.

Finally, Fig. 1F illustrates how the model’s reasoning can be audited at the level of structure tokens. For E. coli \(\beta\)-ketoacyl-ACP synthase I (UniProt: P0A953), SciReasoner identifies repeated VVVV fragments as canonical \(\alpha\)-helical regions and the DDLPPVDDP fragment as an extended strand/turn-rich segment. The trace then links this fold architecture, together with glycine-rich and catalytic-context cues, to fatty-acid biosynthesis and related lipid-metabolic processes. This example demonstrates that the reasoning trace is not an isolated natural-language explanation: its claims can be connected back to explicit structural substrings and spatial regions of the protein.

Together, these results show that SciReasoner advances GO-term prediction in three ways: it improves accuracy most in low-homology regimes, grounds predictions in localized sequence and structural evidence, and uses reinforcement learning to enhance both annotations and reasoning quality. Thus, SciReasoner serves not only as a more accurate GO predictor, but also as a mechanism-aware annotation framework that explains molecular function, biological process, and cellular localization assignments.

2.2.2 SciReasoner formulates chemically viable retrosynthetic routes with high precision↩︎

Retrosynthesis (the recursive disconnection of a target molecule into commercially available precursors) is the cognitive backbone of organic-synthesis route planning. Reliable retrosynthetic models accelerate hit-to-lead optimization, and an interpretable reasoning trace lets human chemists verify and reuse the suggested routes.

We evaluate on the standard Retrosynthesis USPTO-50K [41] split: the model is given a single target SMILES and must output the set of reactant SMILES that produces it. We sample 16 stochastic completions per query (\(T{=}0.6\), top-\(p{=}0.95\)) and rank the answers by sample frequency. A prediction is correct only when its canonical SMILES exactly matches the ground-truth reactant set, and we report Exact Match as the standard metric. Fig. 2A places SciReasoner against \(18\) published baselines spanning template-based [42]–[46], semi-template-based [47]–[52] and template-free [24], [31], [53]–[57] families: SciReasoner reaches \(0.72\) (Exact Match), exceeding the previous best template-free method RSGPT [31] (\(0.63\)) by \(+0.09\) points, while Opus-4.7 five-shot scores only \(0.48\). To prevent direct test leakage, we removed from the pretraining corpus all reactions whose product SMILES matched any product in the Retrosynthesis USPTO-50K test set.

Unlike generic chat models that emit a free-form natural-language explanation followed by a single SMILES guess, SciReasoner is a scientific reasoning model that interleaves natural-language steps with structural tokens (SMILES fragments) inside its chain-of-thought, e.g.the ester C–O bond, the phenyl ring c1ccccc1. Because the intermediate steps are themselves chemical structures, the trace can be checked atom-by-atom and the final reactant assembly is grounded in concrete sub-fragments.

Fig. 2B illustrates this on a hydroxy-substituted benzhydryl ester. SciReasoner severs the strategic ester C–O bond, independently verifies the expected functional groups on each precursor, and recognises the inverse reaction as an acid-catalysed esterification. Six of the seven sub-fragments emitted in the trace are strictly smaller than either reactant, yet their union exactly reconstructs the gold answer, indicating bottom-up reasoning over chemical groups rather than template recall.

Fig. 2C extends this to five further Retrosynthesis USPTO-50K products, contrasting SciReasoner against the specialised retrosynthesis model RSGPT and the general-purpose large language model Opus-4.7. SciReasoner contains the gold reactant set in its Top-3 for \(5/5\) rows; RSGPT recovers it in \(2/5\), and Opus-4.7 in \(2/5\).

On the \(N\)-tert-butyl pyridine-carboxamide aldehyde, SciReasoner recovers the literature Vilsmeier formylation route (DMF/Br precursor pair) with a single canonical answer across all 16 samples; RSGPT proposes a hydroxymethyl + bromopyridine pair, while Opus-4.7 commits all three ranks to oxidation of a benzyl alcohol and never proposes the C–N disconnection (Fig. 2C, Row 1). On a 1,4-disubstituted 1,2,3-triazole-fused phenyl-benzofuranone, the gold cut is a copper-catalysed azide–alkyne [3+2] cycloaddition (CuAAC) of trimethylsilyl azide with the terminal aryl-alkyne; only SciReasoner’s rank-1 recovers this pair, while RSGPT keeps the triazole intact and disconnects elsewhere and Opus-4.7 mis-treats the cycloaddition as an azide-plus-nitrile coupling (Fig. 2C, Row 2). On a chloro-arylsulfanyl tetrahydrofuro-pyridine \(N\)-Boc carbamate, the strategic cut is \(m\)-CPBA oxidation of the thioether to the sulfoxide; only SciReasoner’s rank-1 identifies it, while RSGPT and Opus-4.7 instead attempt to cleave the Boc carbamate or the tetrahydrofuran (Fig. 2C, Row 3). On a polycyclic indole-based amide, SciReasoner and Opus-4.7’s rank-1 both propose the correct amide coupling between the bicycloalkyl carboxylic acid and the parent aniline; the remaining baselines fragment the indole or the trifluoromethyl chain (Fig. 2C, Row 4). On a thiophene–phenyl biaryl bearing a Boc-aminothiophene and a spirolactone-piperazine amide, the strategic cut is a Suzuki–Miyaura coupling of the 5-bromo Boc-aminothiophene with benzyl 4-boronate-benzoate; SciReasoner returns this pair across all \(16\) samples and Opus-4.7’s rank-1/rank-2 both match, while RSGPT recovers the gold pair only at rank-3 (Fig. 2C, Row 5).

Taken together, the structural-token chain-of-thought makes SciReasoner’s predictions both more accurate and easier to audit: the trace states inside itself why each bond is the strategic cut, rather than answering in one step.

2.2.3 SciReasoner captures plausible 3D molecular structure for similarity representation↩︎

Identifying compounds that share a binding-pocket geometry but not a 2D scaffold is the central problem of virtual screening and ligand-based drug discovery. A representation that groups molecules by binding-pocket compatibility, rather than by simple substructure overlap, would directly enable hit expansion and scaffold hopping.

We probe whether SciReasoner itself already provides such a representation. From a single-molecule prompt, we extract the last layer hidden state of SciReasoner and average it over the 10 tokens generated immediately after the prompt to obtain a molecular embedding. Following the standard DUD-E protocol [32], we ask how well Euclidean distance in this embedding space ranks actives versus decoys for a given target query, reporting AUC and the \(5.0\%\) EF, and the embedding’s qualitative structure is visualised via 2D UMAP projection.

SciReasoner natively consumes structural tokens: a dedicated ConfSeq [58] tokenizer converts each 3D conformer into a sequence of geometry-aware tokens drawn from a separate vocabulary backed by a separate learned embedding table, held disjoint from the natural-language embedding. Because structural tokens are a parallel modality, geometric features are separated from those used for IUPAC names or descriptive language, so the hidden states above structural-token positions carry an uncontaminated geometric signal.

Fig. 2D quantifies this representation against nine published virtual-screening methods on \(102\) DUD-E targets, spanning alignment-free descriptors and alignment-based shape-overlay methods [58]–[66], plus an Opus-4.7 [24] reference. Used as a drop-in similarity feature, SciReasoner’s per-molecule embedding matches ConfSeq’s prior best AUC of \(0.76\) and improves the best \(5.0\%\) EF from \(7.12\) to \(7.70\), obtained without docking, scoring functions, or task-specific fine-tuning.

Fig. 2E demonstrates the qualitative structure of the same embedding on three pharmacologically distinct DUD-E targets spanning unrelated protein families: ADAM17 (a zinc metalloprotease), carbonic anhydrase II (a zinc hydrolase) and cyclooxygenase-2 / PGHS-2 (a heme-containing dioxygenase). Each target’s top-\(400\) closest actives form a clearly separated cluster around its query ligand (\(\bigstar\)) on the 2D UMAP, and the three clusters occupy distinct regions of the plane despite the proteins being sequence-unrelated, and the decoys (light grey) do not infiltrate any cluster, indicating that the embedding captures family-specific chemistry rather than gross descriptors.

The boxed side panels visualise the per-target binding mode in 3D. For ADAM17, the cyclic-thiol sulfonamide query is matched by closest actives that swap the thiopyrrolidine zinc-binder for a hydroxamate amide or extend the alkyne tail into a propargyl amine while keeping the diaryl sulfonyl core; their Morgan-fingerprint Tanimoto similarities to the query are only \(0.30\)–\(0.32\), yet USRCAT 3D-shape similarities remain greater than \(0.22\) and Open3DALIGN scores reach \(120\)–\(140\), meaning all three ligands drape over the same TACE active site. For PGHS-2, the celecoxib-like diaryl-pyrazole sulfonamide query is matched by an indanone-acetic-acid analog (Tanimoto \(0.11\)) and a chloro-aryl-thiazole-sulfonamide (Tanimoto \(0.31\)), chemically divergent COX-2 chemotypes that adopt the same 3D pharmacophore. Carbonic anhydrase II is the most striking case: the query reduces to the minimal warhead, and its three closest actives—a cyclohexyl sulfamate, a pyridyl-thiadiazole sulfonamide and a thiopheno-thiazinone, spanning very different fused-ring topologies yet all expose the canonical zinc-binding sulfonamide that defines CA-II inhibition.

Taken together, both qualitative and quantitative results indicate that SciReasoner has implicitly learned to cluster DUD-E actives by binding-pocket geometry rather than scaffold similarity.

2.2.4 SciReasoner learns structure- and property-aware representations for materials tasks↩︎

We evaluated SciReasoner on ten downstream materials sub-tasks from five complementary databases: Materials Project¹, JARVIS-DFT², SNUMAT³, hMOF⁴, and QMOF⁵. These sub-tasks cover inorganic crystals, semiconductors and metal organic frameworks, and include thermodynamic stability, formation and total energies, electronic band gaps, spin orbit coupling, CO\(_2\) uptake and pore geometry. For these sub-tasks, SciReasoner was compared with CGCNN [9], LLM-Prop [67] and Opus-4.7, using Mean Absolute Error (MAE) for regression tasks and Area Under the Curve (AUC) for classification tasks. As shown in Fig. 3A, SciReasoner outperforms CGCNN across all ten tasks and exceeds LLM-Prop on most numerical properties. These results indicate that its representations capture both general energetic trends and more specialized relationships involving electronic structure, relativistic effects, adsorption behaviour and pore geometry.

We next examined the latent space learned by SciReasoner. UMAP projections show clear separation among C, Si and SiC structures, indicating that the embeddings encode global chemical identity (Fig. 3B). Within each compositional domain, structures do not collapse into a single cluster. Instead, they form extended subspaces that distinguish polymorphs and polytypes with the same composition, suggesting sensitivity to local geometry and bonding topology. At the quantitative level, SciReasoner produces DFT-consistent predictions for both formation energy and band gap, with parity plots showing close agreement with ground-truth values and residuals concentrated around zero (Fig. 3C). This predictive accuracy is reflected in the learned representation space: local UMAP projections for Si and SiC separate low- and high-band-gap structures while preserving continuous property gradients within each compositional manifold (Fig. 3D). Together, these results show that SciReasoner organizes structurally related materials along physically meaningful electronic-property manifolds, rather than encoding composition alone.

A representative TiGaCo\(_2\) stability prediction illustrates how SciReasoner grounds its reasoning in explicit structural tokens rather than relying only on the chemical formula (Fig. 3E). The model identifies the structure substring corresponding to the Fm\(\bar{3}\)m space group and links local edge substrings to periodic Ti–Co, Ti–Ga and Ga–Co connectivities in the reconstructed CIF structure, providing a traceable connection between token-level evidence and crystallographic features.

2.3 Structural evidence drives native scientific reasoning↩︎

To test whether native scientific reasoning depends on explicit structural evidence, we performed structure-ablation experiments across materials, proteins and small molecules (Fig. 4A). Removing structural information consistently weakened performance, whereas providing structural inputs improved prediction across all three domains. The gains were especially pronounced for protein tasks, where function depends on folded geometry, binding-site organization and spatial residue interactions, but were also evident in materials and molecular-similarity tasks, where periodic topology and three-dimensional shape provide information that cannot be recovered from composition, sequence or two-dimensional descriptors alone. Thus, the ablation shows that structural evidence is not merely an additional input channel, but a necessary substrate for the model’s native scientific reasoning.

The embedding analysis further shows that SciReasoner learns a distinct representational space for structural evidence (Fig. 4B). In the Qwen embedding space, paired protein sequence and structure inputs remain partially intermixed, suggesting that structural descriptions are treated largely as another form of text. By contrast, SciReasoner forms more clearly separated sequence and structure clusters, indicating that structural tokens occupy a distinct representational subspace while preserving within-modality consistency. This modality separation suggests that the model can distinguish sequence-level evidence from structure-level evidence, a prerequisite for reasoning tasks that require explicit modelling of sequence–structure relationships.

Case studies show that structural evidence changes the reasoning path, not merely the final output. For QMOF pore-limiting-diameter prediction, the structure-free model relies on compositional and stoichiometric priors and overestimates the pore size by nearly an order of magnitude (Fig. 4C). With structural tokens, SciReasoner instead cites monoclinic symmetry, cobalt coordination to nitrogen and oxygen atoms, edge connectivity and periodic offsets, producing a prediction close to the ground truth. Similarly, for biological-process GO prediction, sequence-only reasoning is diverted by cationic and histidine-rich motifs towards DNA recombination and stress-response terms, whereas structure-aware reasoning integrates secondary-structure context and binding-pocket evidence to recover protein-modification annotations (Fig. 4D). These examples indicate that structural evidence redirects the model from superficial sequence or composition cues towards physically and biologically meaningful explanations.

Quantitative analyses support the same conclusion. For materials prediction, structural input reduces absolute errors for bandgap, largest-cavity-diameter and pore-limiting-diameter prediction (Fig. 4E), showing that crystal geometry improves both electronic-structure and pore-geometry inference. For GO molecular-function prediction, residue-level attention maps show that structure-aware reasoning concentrates high attention around the functional binding site, whereas sequence-only reasoning attends to non-binding regions and yields lower accuracy (Fig. 4F). Together, the ablations, representation analysis, reasoning cases and attention maps support the claim that structural evidence drives native scientific reasoning in SciReasoner, enabling the model to ground its conclusions in biologically and physically meaningful structural features rather than superficial sequence, composition or textual correlations.

2.4 Intra-domain grounding and cross-domain consolidation stimulate self-bootstrapped native structural reasoning↩︎

While broad scientific pre-training provides diverse molecular, biomolecular, and materials representations, it does not establish a unified reasoning protocol across scientific tasks. We therefore adopt a self-bootstrapped post-training framework, which first grounds structural evidence within each scientific domain (i.e., intra-domain structural evidence grounding) and then integrates these domain-local reasoning patterns into a single scientific reasoner (i.e., cross-domain reasoning consolidation) (Fig. 5A). Sketch traces provide only a coldstart prior, after which domain-structure experts generate higher-quality on-policy traces that are pooled for final training. Together, this two-stage procedure turns pre-trained structural priors into explicit scientific reasoning, first eliciting domain-native evidence use and then consolidating it into a unified structure-grounded reasoner.

The reward and performance dynamics in Fig.5B support the self-bootstrapped strategy. Intra-domain structural evidence grounding yields consistent reward gains across task families, indicating that a small set of sketch traces can seed trainable reasoning behaviours. After expert-generated traces are pooled, cross-domain reasoning consolidation enables the unified SciReasoner to outperform the coldstart baseline across both structure-intensive and non-structure tasks. This suggests that expert-generated traces better align with the model’s representation space, enabling model-native reasoning trajectories to emerge.

Fig. 5C shows that the post-training procedure improves both first-sample accuracy and sampling efficiency across tasks. For pass@10, we used task-specific aggregation: the mean Fmax@10 across BP, MF and CC for GO annotation, an oracle best-of-10 prediction selected by minimum absolute error for QMOF regression, a label-independent 10-sample score ensemble followed by 5.0% EF evaluation for 3D molecule structure (DUD-E), and top-10 reactant-set accuracy for retrosynthesis. In GO term prediction, intra-domain structural evidence grounding raises pass@1 from 0.48 to 0.52 and reduces the pass@10–pass@1 gap from 0.13 to 0.11, while cross-domain reasoning consolidation further achieves the highest pass@1 of 0.59 with a smaller gap than its coldstart precursor. In QMOF prediction, pass@1 increases from 2.71 to 4.76 after intra-domain structural evidence grounding and from 6.80 to 8.61 after cross-domain reasoning consolidation, with the gap reduced from 11.29 to 4.07 and from 13.82 to 5.57, respectively. 3D molecule structure (DUD-E) prediction shows the same trend: pass@1 improves from 4.65 to 6.31 after intra-domain structural evidence grounding and from 7.29 to 7.70 after cross-domain reasoning consolidation. Retrosynthesis shows the largest absolute gain, with pass@1 rising from 0.41 to 0.49 after intra-domain structural evidence grounding and to 0.72 after cross-domain reasoning consolidation, accompanied by gap reductions from 0.36 to 0.26 and from 0.22 to 0.09. These consistent gap reductions indicate that the procedure does not merely improve best-of-\(N\) performance, but concentrates probability mass on correct first responses, making SciReasoner more reliable under single-sample scientific prediction.

Fig. 5D further evaluates reasoning quality beyond final-answer accuracy using LLM-judge score distributions over 100 records per task. Self-bootstrapped native structural reasoning consistently shifts the distributions toward higher scores, with the largest worst-to-best mean improvement on 3D molecule structure (DUD-E) (\(\Delta_{\mathrm{mean}}=+3.10\)), followed by GO protein-function annotation (\(+1.79\)), QMOF property prediction (\(+1.61\)), and Retrosynthesis USPTO-50K (\(+0.48\)). The larger gains on DUD-E, GO and QMOF suggest that the self-bootstrapped procedure is especially effective when reasoning must be grounded in molecular topology, protein structural evidence or crystal-property relationships. By contrast, retrosynthesis starts from an already high reasoning-quality distribution, leaving less room for improvement. Its smaller but positive shift therefore indicates slower saturation rather than failure of the post-training procedure. Overall, these distribution-level gains show that the procedure improves not only answer correctness, but also the coherence and evidence grounding of the scientific reasoning process.

Fig. 5 shows that self-bootstrapped native structural reasoning progressively transforms domain-structure experts into a unified scientific reasoner. Intra-domain structural evidence grounding strengthens domain-specific reasoning, while cross-domain reasoning consolidation transfers these capabilities into a single cross-domain model. Improvements in reward, pass@1 performance, and reasoning quality demonstrate more accurate and stable evidence-grounded reasoning.

2.5 Human Expert Evaluation↩︎

The automated metrics and LLM judges establish that SciReasoner produces accurate answers, but they cannot establish whether its reasoning traces are genuinely useful to domain experts. We therefore conducted a double-blinded human evaluation pilot in which domain experts compared SciReasoner and DeepSeek-V4-Pro reasoning traces on cases spanning three tasks — GO protein-function annotation, crystalline-material property prediction, and single-step retrosynthesis on Retrosynthesis USPTO-50K. Evaluators were double-blinded to model identity and rated every trace against a five-axis, 1–10 rubric (Q1 evidence grounding, Q2 domain plausibility, Q3 target-regime alignment, Q4 reasoning coherence, Q5 anti-hallucination) together with a five-point pairwise preference (Q8, strongly prefer SciReasoner \(\to\) strongly prefer DeepSeek-V4-Pro). The two panels of Fig. 5E summarize the resulting \(N=177\) ⁶ reliable case-judgments: the head-to-head preference distribution and the per-axis score distributions.

In the head-to-head comparison, experts rated SciReasoner tie-or-exceed than DeepSeek-V4-Pro in \(98\%\) of case-judgements, with \(73\%\) strongly preferring SciReasoner, \(21\%\) preferring it, and \(4\%\) ties. Only a small residual fraction favoured DeepSeek-V4-Pro (Fig. 5E, pairwise preference). The preference is consistent across all three tasks — \(100\%\) tie-or-exceed on GO protein function, \(95\%\) on crystalline-material property prediction, and \(98\%\) on retrosynthesis. It demonstrates that SciReasoner’s advantage is not carried by any single domain but holds across heterogeneous scientific-reasoning settings.

The per-axis scores show that this preference reflects a broad, consistent quality gap rather than a single dominant dimension. SciReasoner averages \(8.7/10\) overall versus \(4.3/10\) for DeepSeek-V4-Pro, with per-axis means of \(8.8\) vs \(4.5\) on evidence grounding, \(8.5\) vs \(4.4\) on domain plausibility, \(8.9\) vs \(4.3\) on target-regime alignment, \(8.7\) vs \(3.9\) on reasoning coherence, and \(8.8\) vs \(4.6\) on anti-hallucination. Every per-axis difference is significant shown by Wilcoxon signed-rank on paired per-case differences, \(P<0.001\) in Fig. 5E, per-axis scores. SciReasoner’s distributions are concentrated near the top of the scale on all five axes, whereas DeepSeek-V4-Pro receives more variable scores, especially on target-regime alignment and reasoning coherence. This indicates that the most reliable separation is not that DeepSeek-V4-Pro uniformly fails, but that SciReasoner is more consistently grounded, task-aligned, and coherent across cases.

Together, these results show that SciReasoner’s advantage on automated metrics also holds under direct expert scrutiny: across three heterogeneous tasks, human experts prefer or tie SciReasoner reasoning in the overwhelming majority of cases, and rate it markedly higher on every quality axis, reflecting faithful, verifiable scientific reasoning.

3 Discussion↩︎

In this work, we introduced SciReasoner, a multimodal scientific foundation model for native structural reasoning across proteins, small molecules and inorganic crystals. The central premise is that structure–property relationships cannot be fully addressed by treating scientific entities as text strings, low-dimensional descriptors or black-box inputs to property predictors. SciReasoner instead represents structures as primary objects of inference through a unified structure-aware vocabulary that discretizes three-dimensional coordinates, molecular topologies and crystallographic lattices into domain-native tokens, and integrates them with language instructions in a single autoregressive model. This design allows residues, molecular fragments, conformational cues and crystal descriptors to function as addressable evidence units within generated reasoning trajectories, rather than as peripheral annotations appended to text.

Our results support native structural reasoning as a model-level capability for scientific AI. Rather than using structure only as an input format or auxiliary descriptor, SciReasoner makes residues, molecular fragments, conformational cues and crystal descriptors available as evidence within generated reasoning trajectories. This shift is important because structure–property analysis requires not only accurate outputs, but intermediate claims that can be inspected against the physical object being studied.

The strongest evidence for this capability comes from regimes in which superficial similarity is insufficient.

For Cellular Component annotation of low-homology and orphan-like proteins, SciReasoner improved \(F_{\max}\) from 0.42 to 0.55, with its attention enriched at contact-defined DNA-binding residues and protein–DNA interfaces. In retrosynthesis, it increased single-step accuracy from 0.63 to 0.72 while identifying reaction-relevant fragments, strategic bond disconnections and chemically valid precursors. In crystalline materials, its reasoning trajectories invoked symmetry-distinct sites, coordination environments and periodic bonding motifs when linking structure to electronic and thermodynamic behaviour. These behaviours indicate that structural organization functions as the working substrate of inference, not merely as a representation used to produce a final score.

This distinction clarifies the contribution of SciReasoner relative to existing scientific AI systems. A Foldseek-derived structural fragment can ground a fold- or function-level hypothesis; a molecular substructure can justify a retrosynthetic disconnection; a conformational cue can support a pharmacophore-level similarity judgement; and a crystal-connectivity token can support an inference about stability or electronic response. Because such intermediate claims are expressed in the reasoning trajectory, they can be checked against the input structure rather than accepted as opaque post hoc rationales. This helps bridge two previously separated capabilities: domain-native structural representation and explicit scientific reasoning. The double-blind expert evaluation, in which experts preferred or tied SciReasoner over a frontier large language model in 98% of cases, further suggests that this inspectability is practically useful for scientific assessment.

Overall, SciReasoner demonstrates that a single autoregressive foundation model can unify sequence, structure, and natural-language reasoning across major scientific modalities while achieving specialist-level accuracy and interpretable scientific explanation. This provides a route toward scientific AI systems that move beyond prediction toward reasoning over the physical structures from which functions, properties, and mechanisms emerge.

4 Method↩︎

4.1 Data Source and Processing↩︎

4.1.1 Protein Data↩︎

We constructed the protein component of the pretraining corpus by following established protein-language data construction protocols. To incorporate textual and functional protein knowledge, we used SIFTS-based PDB–UniProt mappings as indexing anchors [35], [68], [69], protein sequences were associated with protein names, PubMed/PMC literature [70], [71] and curated UniProtKB/Swiss-Prot annotations, including functional descriptions, domain and subunit information, post-translational modifications, disease associations, induction information and tissue specificity when available.

For large-scale UniProt protein sequences, we retrieved the corresponding AlphaFold-predicted structures from AlphaFoldDB whenever available [72]. Following the SaProt pretraining setting [28], [73], each protein structure was converted into residue-level Foldseek 3Di tokens [40], which describe local structural environments and are aligned one-to-one with the amino-acid sequence. The amino-acid tokens and 3Di tokens were then separated into amino acid (AA) sequences (wrapped by <protein></protein> tag) and structural sequences (warped by <protein_structure></protein_structure> tag) in one input, allowing the model to learn from both primary sequence and predicted three-dimensional structure. Structurally unreliable regions (pLDDT\(<\)​70) were masked or replaced by a special structural token to reduce noise from uncertain coordinates.

To prevent the leakage of labels in the benchmark data within the interleaved protein–text data, we excluded from continued pretraining all text-linked protein records with \(>\)​30% MMseq2 [74], [75] sequence identity to any protein in the held-out test sets of the downstream benchmarks.

4.1.2 Small-molecule Data↩︎

We constructed the small-molecule component by integrating chemical text, molecular representations, structured property datasets and three-dimensional molecular conformations. The textual component was collected with US Chemistry Olympiad problems, Europe PMC full-text articles and abstracts, ChemRxiv, bioRxiv, medRxiv and arXiv submissions filtered for chemistry, medicinal chemistry and physical chemistry. These sources provide foundational chemical concepts, textbook-style explanations, research-level chemical knowledge, experimental context, safety descriptions and domain-specific terminology.

We further converted public molecular property datasets into language-interfaced templates, pairing molecular entities with curated descriptions of properties, assays, labels and task contexts. These sources include MoleculeNet [76], Therapeutics Data Commons and ADMET-style datasets [77], ChEBI, PubChem BioAssay, Tox21, hERG and CYP bioactivity datasets, CHEMDNER and other biomedical named-entity recognition corpora, and curated datasets covering ADMET, physicochemical properties, molecular toxicity and safety-related endpoints. Together, these resources use natural language to align standardized molecular representations with property labels, bioactivity assays, toxicity endpoints, pharmacokinetic profiles and biomedical chemical terminology. We also incorporated reaction, biochemical and spectral resources, including Open Reaction Database (ORD), USPTO-derived reactions, Buchwald–Hartwig datasets, Rhea/UniProt reaction annotations, MOSES [78], MoNA and spectrum-reasoning datasets, to support learning of reactant–product mappings, reaction procedures, enzymatic transformations, molecular understanding and spectroscopy-based reasoning.

To incorporate explicit three-dimensional molecular information, we additionally constructed a conformation-aware molecular corpus following the representation learning dataset of ConfSeq [58]. Molecules collected from ChEMBL and BindingDB were converted into three-dimensional conformers using RDKit-based conformer generation. The resulting conformations were represented in a sequence-compatible format that combines molecular connectivity with internal-coordinate information and is wrapped by <molecule_structure></molecule_structure> tag, linking with natural language to allow the model to learn geometric features such as molecular shape, bond geometry and conformational variation within a language-modeling framework. To avoid label leakage, we canonicalized molecular identifiers and excluded pretraining examples whose molecule–label or reaction product entries overlapped with the held-out test sets of the corresponding downstream benchmarks.

4.1.3 Materials Data↩︎

We constructed the materials component by collecting crystal compositions, crystal information files, structure-derived textual descriptions and tabular materials properties from public materials databases. The primary sources include Materials Project, JARVIS-DFT, SNUMAT, hMOF, QMOF, OQMD⁷, OMDB⁸, JARVIS-QETB⁹, GNoME¹⁰, and Cantor HEA¹¹ These sources cover a broad range of inorganic crystals, semiconductors, metal–organic frameworks, organic materials, high-entropy alloys and computationally discovered stable materials.

For each material, we retained complementary representations such as chemical formula, elemental composition, periodic crystal structure in CIF format and natural-language structural descriptions generated from crystallographic information. We then used natural language templates to link these representations to form the training samples, enabling the model to learn composition-level, geometry-level and text-level representations of periodic systems.

For data processing, we used an 80/10/10 random split at the material-sample level, where each sample contains one material together with all associated property labels. To prevent leakage, all validation and test material samples, including their structural representations and property records, were removed from the continued-pretraining corpus before training.

4.1.4 DNA/RNA Data↩︎

We constructed the RNA component from RNAcentral, retaining sequences in FASTA format and wrapping them with <rna></rna> tags. Available metadata, such as RNA family information, was incorporated to provide functional and contextual cues. For DNA, genomic sequences were retrieved from NCBI, sampled into 1,000-bp organism-level fragments, represented in FASTA format and wrapped with <dna></dna> tags. Organism names and genomic coordinates were retained to anchor each sequence within its genomic context.

4.1.5 General Text and Instruction Following Data↩︎

To complement the scientific corpora with general-domain text and instruction-following supervision, we draw from three publicly released sources. For general web text, we sample from Nemotron-CC v2 [79], a multi-trillion-token, quality-filtered Common Crawl derivative. For reasoning-formatted instruction data, we use the mathematics and instruction-following splits of Dolci-Think-SFT-32B [80], in which every response carries an explicit <think>-delimited deliberation trace covering multi-step symbolic reasoning and constraint-driven instruction following. For scientific instruction adherence, we additionally include the training split of SciIF [81], whose prompts impose verifiable constraints over scientific contexts.

4.2 Model Architecture↩︎

The overall architecture of SciReasoner is designed to operate as a unified causal language model capable of processing interleaved structural and textual modalities. The system consists of three primary components: modality-specific offline structural compressors, a structure-aware vocabulary embedding layer (acting as a discrete cross-modal projection), and a unified large language model (LLM) backbone, denoted as \(f_\phi\). The LLM backbone is initialized with the weights of Qwen3-14B [82].

4.2.1 Offline Structure Encoder↩︎

A critical limitation of applying generalist LLMs directly to scientific data is the reliance on text-based sub-word tokenizers, such as Byte-Pair Encoding [83], [84], which arbitrarily fragment physical topologies. To prevent this, we employ a modality-specific offline compression strategy. Let \(S \in \{S_{prot}, S_{chem}, S_{cryst}\}\) represent the raw structure input data for proteins, small molecules, and crystals, respectively. As shown in Fig. [fig:main95figure]B, we utilize domain-specific discrete encoders to encode \(S\) into the structural information sequence \(X_v\). Crucially, this offline process guarantees that fundamental physical semantics (such as local chemical motifs, three-dimensional geometry, and lattice symmetries) are preserved as distinct, holistic tokens prior to interacting with the language model. Specifically, we adopt: (i) Foldseek [40] for encoding the 3D structure of protein, (ii) SLICES [85] for crystal, and (iii) ConfSeq [58] for 3D molecular structures. The encoded structural information sequences are tagged with corresponding structure tag, i.e., <material_structure></material_structure>, <molecule_structure></molecule_structure>, and <protein_structure></protein_structure>.

4.2.2 Structure-Aware Vocabulary Embedding↩︎

Unlike traditional multimodal frameworks that rely on heavy continuous encoders (e.g., Graph Neural Networks) during the forward pass, SciReasoner treats the discrete structural sequence \(X_v\) identically to linguistic scientific tokens. We introduce a dedicated structure-aware vocabulary, which functions as a discrete cross-modal projection layer. Let \(\mathbf{W}_v \in \mathbb{R}^{|V_v| \times d_{LLM}}\) be the learnable embedding matrix for the structural vocabulary, where \(|V_v|\) is the vocabulary size of all structural motifs and features, and \(d_{LLM}\) is the hidden dimension of the LLM. The structural sequence \(X_v\) is directly projected into the dense continuous space via a discrete embedding lookup: \[\mathbf{H}_v = \text{Embedding}(X_v, \mathbf{W}_v) \in \mathbb{R}^{L_v \times d_{LLM}}\] This discrete projection elegantly bypasses the computational bottleneck of continuous spatial encoders, seamlessly aligning structural inputs with the LLM’s native semantic space.

4.2.3 Unified Autoregressive Generation↩︎

Simultaneously, the language instruction \(X_q\) is tokenized and embedded by the LLM’s native embedding layer to produce the language representation \(\mathbf{H}_q \in \mathbb{R}^{L_q \times d_{LLM}}\). The projected structural embeddings and language embeddings are concatenated along the sequence dimension to form the unified input prompt \(\mathbf{H}_{prompt} = [\mathbf{H}_v; \mathbf{H}_q]\). The LLM backbone \(f_\phi\) takes \(\mathbf{H}_{prompt}\) as input and generates the response sequence \(X_a\) in an autoregressive manner, ensuring the generation is strictly conditioned on both the structural evidence and the textual instruction.

4.3 Pretraining↩︎

4.3.1 Optimization Objective and Multi-Stage Pretraining↩︎

The entire multi-stage pretraining pipeline is optimized using a single, unified autoregressive causal language modeling objective (Next Token Prediction).

Given a structural input \(\mathbf{H}_v\) and a language instruction \(\mathbf{H}_q\), the model generates a target response sequence \(X_a = \{x_{a,1}, x_{a,2}, ..., x_{a,T}\}\). The training objective minimizes the negative log-likelihood of the target tokens conditioned on the preceding context: \[\mathcal{L}_{NTP} = -\sum_{t=1}^{T} \log P_\phi (x_{a,t} \mid x_{a,<t}, \mathbf{H}_v, \mathbf{H}_q)\] where \(\phi\) represents the trainable parameters of the system at any given stage. To effectively align the distinct physical structural space with the natural language space without catastrophic forgetting, we implement a three-stage curriculum learning strategy, distinguished exclusively by parameter freezing configurations, data mixture, and learning rate dynamics. Let \(\Theta = \{\theta_{vocab}, \theta_{emb}, \theta_{head}, \theta_{backbone}\}\) denote the complete parameter set, corresponding to the structure-aware vocabulary layer (\(\mathbf{W}_v\)), LLM textual token embeddings, LLM prediction head, and the core LLM transformer backbone, respectively.

4.3.1.1 Stage 1: Warm-up Training

The initial stage focuses on mapping the newly introduced structural tokens into the linguistic semantic space. To prevent the untrained structure-aware vocabulary layer from corrupting the pre-trained reasoning capabilities of the LLM, we strictly freeze the core transformer backbone. In this stage, the trainable parameter subset is \(\phi = \{\theta_{vocab}, \theta_{emb}, \theta_{head}\}\), while \(\theta_{backbone}\) remains frozen. The model is trained on fundamental structure-to-text reconstruction tasks. By restricting gradient updates to the peripheral layers, this stage establishes a stable foundation in the embedding space before deeper integration. To ensure localized and stable optimization for these newly initialized parameters, Stage 1 maintains an independent optimizer state and employs its own standalone learning rate schedule.

4.3.1.2 Stage 2: Full-parameter Training

Once the structural tokens are stably anchored in the embedding space, we proceed to align the modalities across the deep layers of the network. In this stage, all parameters are unfrozen (\(\phi = \Theta\)). For the continuous optimization dynamics spanning Stage 2 and Stage 3, we initialize a new shared optimizer state and a unified Warmup-Stable-Decay (WSD) learning rate scheduler.

Formally, let \(t\) denote the training step across the combined duration of Stage 2 and Stage 3. The shared WSD scheduler is defined by the durations of the warmup (\(T_{w}\)), stable (\(T_{s}\)), and decay (\(T_{d}\)) phases, with total steps \(T_{total} = T_{w} + T_{s} + T_{d}\). The learning rate \(\eta(t)\) is formulated as: \[\eta(t) = \begin{cases} \eta_{max} \cdot \frac{t}{T_{w}} & \text{if } 0 \le t < T_{w} \\ \eta_{max} & \text{if } T_{w} \le t < T_{w} + T_{s} \\ f_{decay}(t) & \text{if } T_{w} + T_{s} \le t \le T_{total} \end{cases}\] where \(\eta_{max}\) is the peak learning rate and \(f_{decay}(t)\) represents a monotonically decreasing annealing function (e.g., cosine decay).

Stage 2 spans the interval \(0 \le t < T_{stage2}\), where \(T_{w} < T_{stage2} < T_{w} + T_{s}\). This indicates that Stage 2 strictly encompasses the entire warmup phase and the initial segment of the stable phase. During this period, the model is trained on diverse, broadly-sourced paired structure-text datasets, allowing the full network to assimilate domain-specific structural knowledge at a high, constant learning rate.

4.3.1.3 Stage 3: Annealing Training

In the final stage, we maintain the full-parameter optimization setting (\(\phi = \Theta\)) and seamlessly inherit both the optimizer state and the active WSD scheduler from Stage 2. Stage 3 covers the remainder of the training timeline (\(T_{stage2} \le t \le T_{total}\)). Thus, it begins by continuing the constant learning rate \(\eta_{max}\) from the stable phase and eventually triggers the decay phase (\(t \ge T_{w} + T_{s}\)) in its later period. In this training stage, we increase the proportion of question–answer (QA)-style data to achieve annealed training.

4.4 Post-training↩︎

The pretraining curriculum endows SciReasoner with broad cross-modal scientific knowledge, yet the resulting checkpoint \(M_0\) behaves as a next-token continuator rather than an instruction-following reasoner. Post-training instantiates self-bootstrapped native structural reasoning through a coldstart supervised stage followed by reinforcement learning (RL). Pooling the chain-of-thought (CoT) supervision of all tasks into a single pass is, however, ill-suited here: the reasoning patterns of heterogeneous scientific tasks are divergent enough that joint training induces destructive interference, and the seed CoT traces, borrowed from external teachers, are distributionally misaligned with SciReasoner.

We therefore organize post-training around two stages: intra-domain structural evidence grounding followed by cross-domain reasoning consolidation. In the first stage, we partition the task universe by task prior and specialize \(M_0\) into a family of per-group experts that learn how structural tokens serve as domain-specific reasoning evidence. In the second stage, these experts regenerate rational CoT traces that replace the external supervision; pooled across all tasks and augmented with non-thinking data, they drive a single unified coldstart and RL pass yielding the final model.

4.4.1 Supervised finetuning for reasoning pattern coldstart↩︎

The coldstart phase seeks to activate the capabilities accumulated during pretraining: we elicit instruction following and, most critically, an explicit CoT reasoning behaviour in which the model deliberates before committing to an answer. Because SciReasoner is initialized from Qwen3 [82], which natively supports a switchable thinking interface, we frame this stage as the re-establishment of a structured generation protocol that the backbone has already been exposed to, requiring only a lightweight intervention rather than a wholesale re-training.

4.4.1.1 Reasoning-formatted data construction

We curate a compact supervised corpus in which every target response adheres to an explicit deliberation template demarcated by dedicated <think> delimiters. We construct two complementary response modes to preserve the dual-mode behavior of the backbone. In the thinking mode, the model is prompted with an instruction and produces an interleaved trace of the form \[X_a \;=\; \texttt{<think>}\;\backslash\texttt{n}\; c \;\backslash\texttt{n}\; \texttt{</think>}\;\backslash\texttt{n}\backslash\texttt{n}\; y,\] where \(c\) denotes the latent chain-of-thought rationale and \(y\) the final answer conditioned on \(c\). In the non-thinking mode, the instruction is appended with a /no_think control token and the deliberation span is collapsed to an empty placeholder, \[X_a \;=\; \texttt{<think>}\;\backslash\texttt{n}\backslash\texttt{n}\;\texttt{</think>}\;\backslash\texttt{n}\backslash\texttt{n}\; y,\] yielding a direct-answer response. Exposing the model to both formats within a single corpus re-anchors the semantics of the <think> delimiters and the /no_think switch, enabling the controllable allocation of test-time computation between explicit reasoning and direct response.

4.4.1.2 Supervised optimization objective

The coldstart stage retains the autoregressive next-token-prediction objective used throughout pretraining, but departs from it in the supervision mask. Rather than computing the loss over the entire sequence, we adopt a supervised-finetuning (SFT) formulation in which gradients are propagated solely through the response tokens, while the instruction and structural context serve purely as conditioning. Concretely, let the full sequence be partitioned into the prompt context \((\mathbf{H}_v, \mathbf{H}_q)\) and the target response \(X_a = \{x_{a,1}, \dots, x_{a,T}\}\), where \(X_a\) encompasses both the deliberation span and the final answer. The coldstart objective minimizes the negative log-likelihood restricted to the response positions: \[\mathcal{L}_{SFT} = -\sum_{t=1}^{T} m_t \, \log P_\phi \big( x_{a,t} \mid x_{a,<t}, \mathbf{H}_v, \mathbf{H}_q \big),\] where the binary mask \(m_t \in \{0, 1\}\) activates supervision only on the response segment (\(m_t = 1\) for tokens in \(X_a\)) and suppresses the loss contribution of the prompt tokens (\(m_t = 0\)). This response-only masking ensures that the optimization sharpens the conditional mapping from instruction to reasoned answer, instead of expending capacity on modelling the marginal distribution of the inputs.

4.4.1.3 Coldstart

In the first stage, a coarse task prior partitions the task universe \(\mathcal{T}\) into \(J\) disjoint groups \(\{\mathcal{G}_1, \dots, \mathcal{G}_J\}\), each aggregating similar tasks with CoT corpus \(\mathcal{C}_j\) whose rationale \(c\) is teacher-supplied. We coldstart \(M_0\) independently on each \(\mathcal{C}_j\) under \(\mathcal{L}_{SFT}\), yielding specialized checkpoints \(\{M_j^{\text{cs}}\}_{j=1}^{J}\) so that each internalizes a coherent reasoning style rather than reconciling conflicting ones. In the second stage, after the experts \(\{M_j^{\text{expert}}\}\) are obtained by the intra-domain structural evidence grounding stage below, we replace the external supervision with SciReasoner-generated traces: each \(M_j^{\text{expert}}\) rolls out on its group’s prompts, and verification-passing traces \(\tilde{c}\) replace the external \(c\) in \(\mathcal{C}_j\), giving a new corpus \(\tilde{\mathcal{C}}_j\). Augmenting each \(\tilde{\mathcal{C}}_j\) with /no_think direct-answer data and pooling across groups, \(\tilde{\mathcal{C}} = \bigcup_{j=1}^{J} \tilde{\mathcal{C}}_j\), we perform a single unified coldstart from \(M_0\) under the same \(\mathcal{L}_{SFT}\). As the targets now lie on the model’s own manifold, this furnishes a well-matched initialization for the cross-domain reasoning consolidation pass.

4.4.2 Reinforcement learning↩︎

For each sub-task, we curate RL data with an empirical solve-rate filter that selects medium-difficulty instances.

Let \(D^{\text{train}}_{k}\) denote the training set for the \(k\)-th sub-task. For each example \(x \in D^{\text{train}}_{k}\):

1. Stochastic rollouts. We generate \(N=8\) independent rollouts from \(M_0\) on \(x\) with stochastic decoding and set do_sample=True. To balance response diversity and format fidelity, we use an adaptive temperature \(T\), tuned once for each sub-task before collection. Starting from \(T=0.9\), we increase \(T\) when the eight samples show insufficient diversity, such as low uniqueness among normalized responses or highly repetitive content. If the outputs become too diffuse and frequently violate the required format or fail to parse, we decrease \(T\). For the final eight rollouts for each \(x\), we keep \(T\) fixed and vary only the random seed, while holding all other decoding parameters constant.

2. Empirical solve rate. For rollout \(i\), we compute a correctness indicator \(f_i(x)\in\{0,1\}\) using the sub-task’s canonical metric, such as exact match or a thresholded score, and aggregate \[\hat{p}(x) \;=\; \frac{1}{8}\sum_{i=1}^{8} f_i(x).\]

3. Mid-difficulty filtering. We retain examples with partial success: \[S_k \;=\; \left\{\,x \in D^{\mathrm{train}}_{k} \;\middle|\; 0.125 < \hat{p}(x) < 0.875 \, \right\}.\] The retained examples are neither trivially solved nor consistently failed, and thus provide more informative RL learning signals than cases with \(\hat{p}(x)\approx 1\) or \(\hat{p}(x)\approx 0\).

4. Difficulty-centred sampling. We construct the RL subset \(T_k\) by ranking candidate examples according to their distance from intermediate success, \[d(x) \;=\; \left|\hat{p}(x)-0.5\right|.\] We then select \(K=2000\) examples without replacement, first from \(S_k\) in ascending order of \(d(x)\): \[T_k \;=\; \begin{cases} \operatorname*{arg\,top}_{K,\,x \in S_k} \bigl(-d(x)\bigr), & |S_k| \geq K, \\[4pt] S_k \cup \operatorname*{arg\,top}_{K-|S_k|,\,x \in D^{\mathrm{train}}_{k}\setminus S_k} \bigl(-d(x)\bigr), & |S_k| < K . \end{cases}\] Here, \(\operatorname*{arg\,top}_{m,\,x\in A} f(x)\) denotes the \(m\) examples in a candidate set \(A\) with the largest values of \(f(x)\). Since \(f(x)=-d(x)\), this operator selects examples with the smallest distance to \(\hat{p}(x)=0.5\). Thus, the procedure starts from examples closest to intermediate success and gradually expands to examples with success rates farther from \(0.5\) until \(K=2000\) examples are collected.

The resulting 2,000-example set serves as the RL training pool for that sub-task.

Focusing RL on neither-trivial-nor-impossible instances yields stronger and more stable improvement: items with \(\hat{p} \in (0.125,0.875)\) (i) expose the policy to cases where exploration can flip outcomes, (ii) avoid vanishing advantages on saturated positives, and (iii) reduce uninformative gradients from uniformly failed cases.

Model training is performed with DAPO [86], which optimizes the policy with RL across heterogeneous scientific tasks. Given a prompt, we first sample \(G\) candidates from the old policy \(\pi_{\theta_{\text{old}}}\). The corresponding within-group standardized advantages are computed as \[r_{i,t}(\theta)=\frac{\pi_{\theta}(o_{i,t} \mid q, o_{i,<t})}{\pi_{\theta_{\text{old}}}(o_{i,t} \mid q,o_{i,<t})},\quad \hat{A}_{i,t} = \frac{R_i - \text{mean}(\{R_i\}_{i=1}^G)}{\text{std}(\{R_i\}_{i=1}^G)}. \label{eq:advantage95calculation}\tag{1}\]

We then optimize the following PPO-style clipped objective: \[\begin{align} \mathcal{J}_{\text{DAPO}}(\theta) =\quad& \mathbb{E}_{(q,a)\sim \mathcal{D}, \{o_i\}_{i=1}^G\sim \pi_{\theta_\text{old}}(\cdot\mid q)}\\& \Bigg[\frac{1}{\sum_{i=1}^{G}|o_i|}\sum_{i=1}^{G}\sum_{t=1}^{|o_i|} \min \Big( r_{i,t}(\theta) \hat{A}_{i,t}, \;\text{clip} \Big( r_{i,t}(\theta), 1 - {\varepsilon_{\text{low}}}, 1 + {\varepsilon_{\text{high}}} \Big) \hat{A}_{i,t} \Big) \Bigg] \\ \text{s.t.}\quad& 0< \Big|\{o_i\mid\texttt{is\_equivalent}(a,o_i)\}\Big|< G, \end{align} \label{eq:dapoloss}\tag{2}\] The asymmetric Clip-Higher schedule is used to preserve exploration. We also apply Dynamic Sampling to remove degenerate groups whose rewards are identical, and use a token-level policy-gradient reduction.

The original DAPO reward is rule-based and binary, whereas many of our tasks, including free-form semantic judgment, information extraction, and numeric regression, cannot be evaluated by a single exact-match criterion. We therefore substitute the binary reward with a reward softening scheme, which converts grouped task metrics to a shared \([0,1]\) scale: \[\label{eq:task95aware95soft95reward} R_i^{\text{soft}} \;=\; g\!\big(m_i(q,o_i,\mathcal{A}(q))\big)\tag{3}\] Here, \(g(\cdot)\) is a monotone calibration function that keeps reward magnitudes comparable across tasks, and \(m_i\) denotes the group-specific quality metric. Under this grouping scheme, scientific prediction tasks use distance-based rewards, retrieval and extraction tasks use matching-based rewards, and tool-verified rewards are computed with professional scientific software.

4.4.2.1 Intra-domain structural evidence grounding and cross-domain reasoning consolidation

The procedure above is first run independently per group as intra-domain structural evidence grounding: starting from \(M_j^{\text{cs}}\), the solve-rate filtering and DAPO objective of Eqs. 1 –3 are applied on the data of \(\mathcal{G}_j\) alone, producing the experts \(\{M_j^{\text{expert}}\}_{j=1}^{J}\) that supply the on-policy traces for the unified coldstart. It is then run once more over the merged all-task pool as cross-domain reasoning consolidation, starting from the unified coldstart checkpoint. Because the policy is now initialized on expert-generated on-policy supervision from every group, this final pass consolidates the experts’ specialized competences without the interference of a naive joint optimization, yielding the final unified model \(M_{\text{final}}\).

References↩︎

---

1. https://materialsproject.org/↩︎

2. https://jarvis.nist.gov/jarvisdft↩︎

3. https://www.snumat.com/↩︎

4. https://mof.tech.northwestern.edu/↩︎

5. https://contribs.materialsproject.org/projects/qmof↩︎

6. We are collecting more human judgments.↩︎

7. https://www.oqmd.org/↩︎

8. https://omdb.mathub.io/↩︎

9. https://jarvis.nist.gov/jarvisqetb↩︎

10. https://github.com/google-deepmind/materials_discovery/blob/main/DATASET.md↩︎

11. https://doi.org/10.5281/zenodo.10854500↩︎