Relational deep learning and graph foundation models

Relational deep learning (RDL) treats a company's whole database as one graph and trains a graph neural network (GNN) on it directly. Bank-data platforms keep users, connections, accounts, devices, sessions, apps, institutions and transactions in tables joined by keys, so the questions here are whether a graph model beats or adds to a per-user sequence model, and which pretraining helps instead of hurting.

Relational Deep Learning (Fey et al., Stanford and Kumo, 2024). Tabular models need one flat table of hand-built aggregates, recomputed for every date. RDL makes each row a node and each foreign-key link an edge, and a heterogeneous GNN (separate weights per edge type) learns the aggregates by message passing, each node combining its neighbors' vectors layer by layer. Labels come from many past seed times, so each example's sampler sees only rows up to its own seed time. A blueprint without experiments, it links rows only where keys exist. In practice: give email, phone, device, IP and account number their own tables, so shared values become fraud-ring edges.

RelBench (Robinson et al., Stanford and Kumo, 2024). RelBench tests RDL's claim to replace feature engineering on 30 time-split tasks over 7 databases. A default two-layer GraphSAGE (a sum over sampled neighbors) tied a finance-ML data scientist's tuned LightGBM on classification (77.42 against 77.39 mean AUROC) with 0.5 hours of work per task instead of 12.3, and permuting foreign-key edges hurt badly, so the keys carry the signal. The data scientist won 5 of 7 regression tasks. In practice: expect speed on new tasks, plus GNN embeddings as extra tree columns, which also recovered part of the regression gap.

KumoRFM (Fey et al., Kumo, 2025). Supervised RDL still trains a model per task. KumoRFM answers a query on an unseen database by in-context learning (ICL: a frozen model reads labeled examples as input). The query, such as PREDICT COUNT(orders.*, 0, 7) > 0 FOR users.user_id, doubles as a label generator, replayed at past anchor times to supply the entity's lagged labels and its neighbors' as context. ICL averages 76.71 RelBench AUROC against 75.83 for supervised RDL, though it loses on 7 of 12 tasks and still needs past outcomes. In practice: "returns in the 5 days after each ACH request" gives a leakage-safe training table at no extra cost.

KumoRFM-2 (Hudovernik et al., Kumo, 2026). In version 1 the task arrived only after tables were mixed, so irrelevant columns diluted it. Version 2 injects the context labels into attention within each table first, dropping useless columns before attending across keys and examples. It averages 79.60 RelBench AUROC against 78.06 for supervised RelGNN and stays flat with 160 added noise columns. Its context holds at most 10k examples, though, and on the three databases with millions of labels RelGNN still wins. In practice: ACH return and credit fall in that regime, so ICL suits exploratory queries and a new customer's cold start.

RelGNN (Chen, Kanatsoulis and Leskovec, Stanford, 2025). Many rows only link others, as a purchase joins customer to item. Message passing crosses such a bridge in two hops, echoing the destination's own information and under-weighting the source. RelGNN derives atomic routes, one direct step through each bridge or hub (a row with three or more keys). It beats GraphSAGE on 10 of 12 classification tasks, by up to ten points on schemas full of multi-key tables and near zero on simple ones, and keeps most of the gain without attention. In practice: link events and sessions are hubs, and the attention-free version suits real-time scoring.

Rel-Cat (Dong et al., Intuit, 2025). Each bank transaction imported into QuickBooks needs one of the company's own categories. Rel-Cat predicts a link from the transaction node, embedded by an encoder trained on transaction strings, to a category node, and each epoch it hides the supervised transaction-to-category edges from message passing. Without that label-edge masking the model finds the answer in the graph, and the GNN's top-1 accuracy falls from 63.38 to 24.16. The full model beats production's per-company logistic regressions, 68.67 against 62.49. In practice: mask every outcome-bearing column and edge (return codes, fraud flags, account status) at the seed time.

Strategies for pre-training GNNs (Hu et al., Stanford, 2020). Graph pretraining sometimes hurt the downstream model, which is called negative transfer. Supervised graph-level pretraining alone did so on 2 of 8 molecular datasets and 13 of 40 protein tasks, because many graph tasks competed in a node-embedding space nothing had structured. A node-level self-supervised stage first (predict a node's context or masked attributes) fixes that, lifting molecular ROC-AUC from 67.0 to 74.2. GIN (sum aggregation with MLPs) gained 7.2 points, GraphSAGE 2.0. In practice: pretrain with a stronger aggregator than RDL's GraphSAGE default, on targets shaped like the downstream tasks.

GraphMAE (Hou et al., Tsinghua, 2022). Graph self-supervision leaned on negatives and augmentations. GraphMAE hides about half the nodes' features, encodes, masks those nodes' codes again and decodes with a one-layer GNN, so each masked node must be rebuilt from its neighbors' codes, under a scaled cosine error $(1-\cos(x_i, z_i))^{\gamma}$ that down-weights easy nodes. It needs no negatives and no edge reconstruction. Molecular transfer averages 73.8 ROC-AUC against 67.0 unpretrained, and gains over contrastive methods are often tenths of a point. In practice: masked cell reconstruction from the row's neighborhood is the natural self-supervised stage, with modest gains.

ULTRA (Galkin et al., Intel and Mila, 2024). Knowledge-graph completion (answer (head, relation, ?)) usually learns an embedding per relation, so it cannot transfer. ULTRA links relations by how they share entities and runs a GNN on that graph of relations from the query relation, marked with ones and the rest with zeros (a labeling trick), so each relation is described relative to the query. Zero-shot on 40 graphs it beats per-graph supervised models on average, 0.395 against 0.344 MRR (mean reciprocal rank), though it trails them on transductive graphs, and without the labeling trick MRR falls by about half. In practice: score "does this device belong with this account" by message passing from the pair, since an unconditioned GNN gives look-alike nodes identical vectors.

Graph foundation models are already here (Mao et al., Michigan State and others, 2024). This position paper asks what lets a graph model transfer. Its answer is a graph vocabulary that encodes what stays invariant across graphs, as tokens do for text, and the evidence it gathers says pretraining must match downstream data and task form: link prediction hurt node classification until node classification was recast as links to label nodes. No universal model exists yet. In practice: build for one schema, and since fraud edges tend to be heterophilous (a ring links to many legitimate nodes), do not average neighbors blindly.

ARC (Liu et al., Griffith and others, 2024). A new graph may offer a few known-normal nodes and no anomaly labels. ARC aligns features across graphs by sorting them by smoothness, encodes each node by its residuals from its neighborhood, and scores it by how poorly cross-attention rebuilds it from about 10 normal context nodes. It ranks first on average over 8 unseen graphs, but half have planted anomalies, and on real Reddit anomalies AUPRC (area under the precision-recall curve) is 4.48 at a 3.33% base rate. In practice: scoring against normal traffic fits a new customer before fraud labels arrive.

TwHIN (El-Kishky et al., Twitter, 2022). Many Twitter models needed features for over a billion users, Tweets and ads. TwHIN trains shared TransE-style embeddings, scoring each edge (s, r, t) by $(\theta_s + \theta_r)^\top \theta_t$ against negative samples (an endpoint swapped for a random entity of the same type); as frozen features it lowered cost per conversion 10.3% online. The embeddings are transductive: only entities present at training time get a trained vector, and new ones need retraining or a cluster-mixture approximation. In practice: it is the cheapest baseline for fraud and credit features, but a brand-new device at link time, the case fraud scoring most needs, has no vector.

PinnerFormer's use of graph embeddings (Pancha et al., Pinterest, 2022). The question is how graph knowledge should enter a per-user sequence model. PinnerFormer concatenates into each action token the frozen PinSage graph embedding of the Pin acted on, with learned action-type and time features, and a causal transformer reads the 256 latest tokens. Removing PinSage lowered Recall@10 from 0.229 to 0.142 and made retrieval nearly identical for every user. In practice: put graph embeddings of what each event touches (merchant, institution, device) into its token, keep the sequence model as it is, and refresh the graph on its own cycle.

What the papers say together. A database's keys carry predictive signal, and time-consistent sampling is the basic leakage control. Against expert features the accuracy gain is small (plain RDL ties on classification and loses regression; RelGNN and KumoRFM-2 lead by two or three points), and the large gain is in labor. Pretraining helps within a domain, but every paper that looked found negative transfer somewhere, and with millions of labels supervised models still win. Hand-built aggregates keep winning on heavy-tailed regression, the long-horizon statistics a per-user sequence model computes. Untested: delayed fraud or credit labels, adversarial users, real-time budgets, and a graph model against a strong sequence model on the same data.

What a graph foundation model for bank data most plausibly is, and what to build first. One pretrained, inductive model over one platform's schema, giving entity and pair embeddings and fine-tuned heads for link-time fraud (on the link event or session row), ACH return (on the payment request) and credit (on the user). Sequence embeddings become its node features, and its embeddings become sequence tokens. Build first a supervised RelGNN for ACH return and link-time fraud, with identity and device tables normalized, labels keyed to when each outcome became known, and large institution and app hubs capped. Ship its embeddings to the production trees and sequence model, and make it beat a TwHIN-style baseline. Pretrain only if that shows lift, with masked cell reconstruction and then history-mined targets, checked task by task against training from scratch. Evaluate on temporal splits, cold-start slices, precision at the operating threshold, and latency within the real-time hop budget.