pragmatiq
An open, reproducible implementation of the PRAGMA recipe for banking foundation models — turning timestamped key–value event histories into user embeddings you can probe, fine-tune, graph, and serve.
pip install -e ".[dev]"·pragmatiq quickstartSynthetic banking data
A deterministic, agent-based generator: transactions, app sessions, transfers, profiles, and causal fraud / default / AML labels — same seed, byte-identical output.
Key–value–time model
A padding-free PyTorch encoder stack (profile · event · history) with TimeRoPE on continuous log-seconds and a tied masked-LM head. Runs on CPU first.
Evaluate & serve
Gradient-boosting probes (ROC-AUC + PR-AUC), LoRA fine-tuning, ONNX/Triton serving, and a one-command deploy — the full path from data to inference.
AML over the graph
Our own extension (not in the paper): a GraphSAGE ablation over the transfer graph that recovers money-mule rings a per-user embedding can't see.
From events to a user embedding
Click through the four-encoder stack — what each stage takes in, produces, and why.
Every field becomes a (key, value, time) token. One shared table embeds both keys and values; a sinusoidal within-field position is added for multi-piece text.
- x = E(key) + E(value) + sinusoidal(position)
- The table is tied to the MLM output projection
- Numeric → percentile bucket, categorical → id, text → BPE pieces (or one frozen-embedding sentinel in the Nemotron variant)
Paper fidelity at a glance
Built by Dynamiq. pragmatiq is an independent implementation inspired by the PRAGMA paper (arXiv 2604.08649) and is not affiliated with or endorsed by Revolut.