Decisions log

Why: Muon's orthogonalized updates suit the large 2-D hidden weight matrices; embeddings, norms, and biases train better under AdamW. Clipping is per-optimizer.

Alternative considered: A single AdamW over everything — simpler, but leaves the hidden-weight conditioning Muon gives on the table.

Why: A learned embedding has non-linear structure a linear head misses; HistGradientBoostingClassifier captures it, and PR-AUC is reported alongside ROC-AUC for the low-prevalence risk tasks. The raw-count baseline uses the SAME classifier, so the gap is about the representation, not the model family.

Alternative considered: Logistic regression (still selectable via probe_model='logistic') — a cleaner linear read, but a weaker ceiling on a rich embedding.

Why: Banking magnitudes are heavy-tailed and zero is semantically special (no balance / no spend) and over-represented; percentile edges + a separate zero bucket keep both ends meaningful, and out-of-range values clip rather than fail.

Alternative considered: Fixed-width bins (poor on heavy tails) or standardized continuous inputs (loses the robustness of binning to outliers).

Why: Real histories are heavy-tailed; capping per-event tokens, profile tokens, and events-per-user keeps training tractable and the batch shapes bounded. At synthetic scale none of them bind, so default output is unchanged.

Alternative considered: No caps — simpler, but a few pathological users dominate memory and batch time at real scale.

Why: The paper does not specify the RoPE base for the continuous time axis; 10000 is the conventional default and is exposed as rope_base so it can be tuned per dataset.

Alternative considered: Hard-coding it — fine until someone's time scales differ and they need to tune it.

Why: The token/event/key union matches the paper; the 10% of selected positions replaced by [UNK] and excluded from the loss act as input dropout, so the model can't lean on a token that is sometimes simply absent.

Alternative considered: Token-only masking — simpler, but misses the event- and key-level structure that makes the objective informative for whole-field prediction.

Why: On synthetic money-mule rings whose transfer amounts and counterparty degree are matched to ordinary accounts, the discriminative signal is the multi-hop transfer structure, not any per-user feature. Gate 6 therefore requires that a GraphSAGE over the transfer graph beats a probe on the isolated embedding (c > a) and beats the same hand-crafted features without a graph (c > d) — i.e. message passing recovers relational signal. The learned per-user embedding sits near chance on its own and does not beat hand-crafted features, so 'the embedding beats hand-crafted features' (b > c) is reported, not gated — consistent with the paper's observation that AML is where a per-history model underperforms.

Alternative considered: Gating the aspirational headline b > c — but it is not achievable on degree/volume-matched synthetic mules, and gating it would either fail honest runs or invite weakening the benchmark to make it pass.