Liquid’s LFM2.5 isn’t a plain transformer — it interleaves gated short convolutions with grouped-query attention. This page implements both operators by hand — the double-gated depthwise conv, GQA with per-head QK-normalisation and rotary embeddings, RMSNorm, SwiGLU — and runs the whole forward pass on your GPU. Because we own the loop, we can read the prediction at every one of the 14 layers (the logit lens) and genuinely switch a layer off to watch the output change.
1 Load the model
Idle. Weights are fetched from Hugging Face once (then cached) and everything runs locally on your machine.
Heads-up: this is a real 230-million-parameter model. First load downloads ~440 MB (BF16) and holds ~0.9 GB in memory once expanded to FP32 — far lighter than a billion-parameter model, so it runs comfortably on a recent Chrome/Edge. On a low-end device, switch the backend to CPU.
2 The prompt
3 The logit lens prediction read at each of 14 layers · click a layer to ablate it
This is real. Each row shows the model’s best next-token guess if you decoded at that depth — watch the prediction emerge as it climbs the stack. Green rows are convolution blocks, blue are attention. The toggle genuinely removes that layer’s operator+MLP from the forward pass (the residual stream passes straight through), so switching layers off really changes the bottom row. Re-run after toggling.
4 Top-5 next-word log-odds one block per generated token, printed as it goes
Sibling page: a packaged Gemma-3-270M running via transformers.js — real model, but its compiled graph can’t expose layers, which is exactly why this hand-written engine exists.
Every operator is implemented from scratch. The conv block is a double-gated depthwise causal convolution: in_proj splits the input into three streams B, C, x; gate B·x feeds a length-3 depthwise causal conv1d; the result is gated again by C and projected out. The attention block is grouped-query attention (16 query heads over 8 key/value heads), with per-head QK-normalisation and rotary embeddings (θ=1,000,000) on a 64-dim head. Every block is pre-norm (RMSNorm, ε=1e-5) with a SwiGLU feed-forward. The hand-written forward pass was verified token-for-token against a NumPy reference (“The capital of France is” → “ Paris”).