Sponsored by

SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | November 20, 2026

Ripple-65B Preview is a plain dense transformer with no experts, no custom kernels and no quantized training, built by a team that grew from one person to five. Ricursive says it scores 80.52 on TerminalBench 2.1, ahead of all of them, for $32M of total R&D.

The architecture is the interesting part, and it's more conventional than the headline suggests: borrowed, well-chosen ideas from Cohere, Qwen and Gemma, executed cleanly. The benchmark lead is the fragile part. On Ricursive's own re-run of the rivals under the official harness, the gap to Beam goes from 0.4 points to 9.7. Which number you believe depends on whose harness you trust.

The promise: by the end you'll know exactly where Ripple's 64.78 billion parameters sit (we reproduced the count from its config to within 0.00003%), why its KV cache at 1M tokens is 5x smaller than a standard model's, what the final training run plausibly cost, and why "runs on a DGX Spark" doesn't mean "usable as a coding agent on a DGX Spark."

The fence: we read Ricursive's launch post, the Hugging Face config.json, and secondary reports of the model card's tables (Digital Applied, AgentsPulse, braindetox), plus the four referenced papers. The Hugging Face README and the custom modeling code weren't reachable from our environment, so the layer code below is a reconstruction from the config, labeled as such. We didn't run the model. Every benchmark and cost figure is Ricursive's own, and as of this writing nobody has published an independent evaluation.

What It Actually Does

Ripple-65B Preview is a text-only, mostly-English-and-code LLM tuned for agentic coding: working in a terminal, editing repositories, running tests. Released November 4, 2026, under Apache 2.0, with BF16, NVFP4, base and DFlash2 speculative-decoding checkpoints. A proprietary variant powers Ricursive's chip-design platform.

Ripple-65B Preview

Reflection Beam

Muse Spark Open

Inkling

Nemotron 3 Ultra

Architecture

Dense

MoE

MoE

MoE

MoE

Total / active params

65B / 65B

501B / 23B

816B / 42B

975B / 41B

550B / 55B

Pretraining tokens

12T

23.8T

n/a

45T

20T

TerminalBench 2.1 (self-reported)

80.52

80.1

76.4

63.8

56.4

TerminalBench 2.1 (Ricursive re-run, Terminus 2)

80.5

70.8

78.7

56.2

55.1

SWE-bench Pro v2.0 Hard

79.1

76.5

n/a

n/a

n/a

SWE-bench Verified

76.5

81.6

n/a

n/a

n/a

MCP Atlas

71.67

n/a

89.2

n/a

n/a

GPQA Diamond

85.2

90.6

n/a

n/a

n/a

Ripple leads on two rows of a 22-row table, both terminal-agent tasks. It trails on SWE-bench Verified, MCP Atlas (tool use), DeepSWE (44.3 vs 60.0), SciCode and knowledge benchmarks. That's a specialist, not a generalist, and the card says post-training was "relatively light."

The training pipeline, per the card: 12T tokens at 8K over 877,500 steps, a 200B-token mid-train at 64K, long-context stages up to 2M, SFT at 128K, and 700 RL steps over 1.4M rollouts, which Ricursive says is about 70x fewer than Beam's. Model FLOPs utilization held at 40% throughout.

The Architecture, Unpacked

What to notice: only 16 of 80 layers ever look beyond 1,024 tokens, and those 16 have no positional encoding at all. That one choice is how a 65B dense model gets to a 1M context without a 320 GiB KV cache.

Every design choice here is borrowed from published work. The skill is in which ones were picked and how they were tuned.

Hybrid NoPE attention, pushed harder than its source. Cohere's Rope to Nope and Back Again interleaved one global no-positional-encoding (NoPE) layer per three RoPE sliding-window layers with a 4,096-token window. At 256K, RULER retrieval rose from 57.1 to 74.8 and the KV cache shrank almost 4x. Ripple goes further on both axes the launch post names: one global per four local layers, and a 1,024-token window. Cohere also found that a large RoPE base adds noise that hurts the NoPE layers' retrieval. Ripple's RoPE θ is just 10,000, the classic default, where many long-context models now use 500,000 or more.

Sigmoid gate after attention. Qwen's Gated Attention multiplies each attention output by a sigmoid of the input, which removes the "attention sink" on the first token and suppresses loss spikes at scale. Our parameter count proves Ripple uses the elementwise version, one gate per output channel, not one per head: 4.19B parameters, 6.5% of the model, spent purely on stability.

Stability features everywhere. Pre- and post-normalization (the Gemma layout the card cites), MLP activations clamped at ±20, value residuals, and gating in full precision. A five-person team can't afford a diverged run, and this architecture spends parameters to avoid one.

Thin and deep, with extra heads. 80 layers at width 5,120 is narrow for 65B (Llama 3 70B uses 8,192 wide). The 80 query heads at dim 128 give a 10,240-wide attention, twice the residual stream. That's the "overprovisioned" head count.

The Code, Annotated

Snippet one: one Ripple layer, reconstructed from config.json. The real modeling code ships via trust_remote_code and we couldn't read it. Global layer placement (idx % 5 == 4) and the value-residual form are our assumptions.

import torch, torch.nn as nn, torch.nn.functional as F

class RippleLayer(nn.Module):
    def __init__(self, idx, d=5120, n_q=80, n_kv=8, hd=128, ffn=40960, window=1024):
        super().__init__()
        self.is_global = (idx % 5 == 4)                   # 16 global layers, 64 local
        self.window = None if self.is_global else window  # SnackOnAI: global layers see everything
        self.n_q, self.n_kv, self.hd = n_q, n_kv, hd
        self.q = nn.Linear(d, n_q * hd, bias=False)       # 10,240 wide: 2x the residual stream
        self.kv = nn.Linear(d, 2 * n_kv * hd, bias=False) # GQA: 8 KV heads serve 80 query heads
        self.gate = nn.Linear(d, n_q * hd, bias=False)    # elementwise gate: 52.4M params per layer
        self.o = nn.Linear(n_q * hd, d, bias=False)
        self.w13 = nn.Linear(d, 2 * ffn, bias=False); self.w2 = nn.Linear(ffn, d, bias=False)
        self.n1, self.n2, self.n3, self.n4 = (nn.RMSNorm(d, eps=1e-6) for _ in range(4))
        self.lam = nn.Parameter(torch.tensor(0.5))        # value-residual mix (our assumption)

    def forward(self, x, v_first, rope):
        B, T, _ = x.shape
        h = self.n1(x)                                    # pre-norm
        q = self.q(h).view(B, T, self.n_q, self.hd)
        k, v = self.kv(h).view(B, T, 2, self.n_kv, self.hd).unbind(2)
        if not self.is_global:
            q, k = rope(q), rope(k)                       # ← THIS is the trick: global layers get
                                                          #   NO position signal, so they retrieve by
                                                          #   content and never need RoPE scaling
        v = self.lam * v + (1 - self.lam) * v_first       # value residual from layer 0
        a = F.scaled_dot_product_attention(
            q.transpose(1, 2), k.transpose(1, 2), v.transpose(1, 2),
            attn_mask=sliding_causal_mask(T, self.window), enable_gqa=True,
        ).transpose(1, 2).reshape(B, T, -1)
        a = a * torch.sigmoid(self.gate(h).float()).to(a.dtype)   # gate in fp32 ("full-precision gating")
        x = x + self.n2(self.o(a))                        # post-norm before the residual add
        g, u = self.w13(self.n3(x)).chunk(2, -1)
        m = F.silu(g).clamp(-20, 20) * u                  # FFN clamp threshold 20.0 from config
        return x + self.n4(self.w2(m))

Nothing here needs a custom kernel: sliding windows, GQA and gating are all stock PyTorch SDPA. That's the "no custom kernels" claim made concrete, and it's why they could hold 40% MFU with simple sharding.

Snippet two: the RL loss, combining truncated importance sampling and score centering. Ripple's RL code isn't public. This follows the two referenced methods: truncated importance sampling (Yao, Liu et al.) and score centering (Marek and Ryabinin, Together AI).

def rl_token_loss(trainer_logits, tokens, adv, samp_logp_tok, samp_topk_logp, samp_topk_idx,
                  resp_len, C=2.0, len_coef=1e-5):
    # The sampler (vLLM/SGLang, fast kernels) and the trainer (FSDP, bf16) disagree slightly on
    # token probabilities. Uncorrected, that mismatch silently turns on-policy RL into off-policy RL.
    logp = trainer_logits.float().log_softmax(-1)
    logp_tok = logp.gather(-1, tokens[..., None]).squeeze(-1)

    # 1) Truncated importance sampling: reweight by trainer/sampler ratio, capped at C
    ratio = (logp_tok.detach() - samp_logp_tok).exp().clamp(max=C)   # cap bounds the variance

    # 2) Score centering: subtract the expected score under the SAMPLER's distribution.
    #    With a constant reward, plain policy gradient still drifts the trainer toward the
    #    sampler. Centering cancels that drift exactly at every prefix.
    q = samp_topk_logp.exp()                                         # sampler's top-128 probs
    centered = logp_tok - (q * logp.gather(-1, samp_topk_idx)).sum(-1)   # ← THIS is the trick

    # 3) Linear length penalty, applied to the per-sequence advantage
    adv = (adv - len_coef * resp_len)[:, None]                       # [B] → [B, 1] over tokens
    return -(ratio * adv * centered).mean()

Score centering needs only the top-k logprobs the inference engine already returns. In the paper's INT8-sampler, INT4-KV-cache stress test it trained to 30% accuracy where TIS alone reached 12% and everything else stayed under 5%.

Snippet three: our verification arithmetic.

d, L, ffn, V, H, KV, hd = 5120, 80, 40960, 100352, 80, 8, 128      # from config.json

attn = d*H*hd + 2*d*KV*hd + H*hd*d             # q, k, v, o
gate = d*H*hd                                   # elementwise: one gate per attention channel
mlp, norms = 3*d*ffn, 4*d
total = 2*V*d + L*(attn + gate + mlp + norms) + d
print(f"{total:,}")      # 64,782,668,800 vs Hugging Face 64,782,689,360 (diff 20,560)
                         # without the gate: 60.59B. Headwise gate: 60.62B. Only elementwise fits.

# KV cache in bf16 at 1M context
per_tok = 2 * KV * hd * 2                       # 4,096 bytes per token per layer
hybrid = 16 * per_tok * 2**20 + 64 * per_tok * 1024
print(hybrid / 2**30, L * per_tok * 2**20 / 2**30)   # 64.25 GiB vs 320 GiB if every layer were global

# Final pretraining run: 6 · N · D at 40% MFU
flops = 6 * 64.78e9 * 12e12                     # 4.66e24
print(flops / (0.40 * 989e12) / 3600 / 1e6)     # ~3.3M H100-hours  (B200: ~1.4M)

The parameter count reproduces to 0.00003%, and only with an elementwise gate. The final run's compute, at $2 to $3 per H100-hour, is roughly $7M to $10M, so most of the $32M went to the ladder, data and failed experiments, exactly as Ricursive's cost definition says.

It In Action

Task: a TerminalBench-style job, "the test suite in this repo fails on Python 3.13, fix it," run as a 30-turn agent session.

Step one, the API call. Ripple is on OpenRouter and Together through OpenAI-compatible endpoints with native tool calling, 1,048,576 tokens of context and up to 131,072 output tokens.

from openai import OpenAI
client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key=OPENROUTER_KEY)
resp = client.chat.completions.create(
    model="ricursiveai/ripple-65b-preview",
    messages=history,            # grows each turn: shell output, file contents, diffs
    tools=[run_shell, read_file, write_file],
)

Agent sessions resend the growing history every turn, so cached-input pricing matters more than the headline output price.

Step two, the token budget. Over 30 turns the context grows to roughly 128K, and the session sends about 2M input tokens in total, 80% of them cache hits, and generates about 15K output tokens.

Step three, the bill. At launch pricing ($0.20 per million input, $0.05 cached, $0.60 output): 0.4M × $0.20 + 1.6M × $0.05 + 15K × $0.60 per million = about $0.17 per session.

Step four, memory. At 128K context the hybrid KV cache is 8.25 GiB in bf16. A conventional all-global layout would need 40 GiB.

Step five, local speed. Decoding a dense model reads every weight for every token, so speed is capped by memory bandwidth:

Setup

Weights

Max single-stream decode (bandwidth bound)

15K output tokens

2× B200, BF16, batch 8 (vendor)

130 GB

250 tok/s aggregate, ~31 per stream

~8 min

DGX Spark, NVFP4

~57 GB

≤ 4.8 tok/s at 273 GB/s

≥ 52 min

DGX Spark, Q4_K_M GGUF

36.1 GiB

≤ 7.0 tok/s

≥ 36 min

minutes of pure decode for one 15K-token agent session
 2× B200 (per stream)  ███▉                                            ~8
 DGX Spark Q4_K_M      █████████████████▉                              ≥36
 DGX Spark NVFP4       ██████████████████████████▏                     ≥52

On a DGX Spark, one agent session is close to an hour of decoding before you count prefill. It fits on the box; it isn't a responsive coding agent there.

Why This Design Works (and what it trades away)

It works because dense is easy to train well. No router collapse, no expert load balancing, no all-to-all communication. A five-person team at 40% MFU with simple sharding is the payoff.

It works because the hard parts were already published. Hybrid NoPE for long context, gated attention for stability, an Olmo 3 ladder to start from, and off-policy RL corrections from Together AI. Ricursive claims a 16x compute-efficiency gain on code over the Olmo 3 ladder at 4B scale, from data mixing and these choices.

It works because the RL was small and careful. 700 steps and 1.4M rollouts, with tasks retuned as the model improved, instead of brute-force scale.

It trades away inference cost per token. Every token activates all 65B parameters. Beam activates 23B. At equal hardware, a sparse model decodes faster once its weights fit, and Ripple's whole advantage is that its weights fit in places Beam's don't.

It trades away breadth. Two benchmark wins out of 22 rows, both terminal-agent tasks. Tool use (MCP Atlas 71.67 vs 89.2) and DeepSWE (44.3 vs 60.0) lag far behind.

It trades away long-range resolution. Only 16 layers attend globally. Cohere's results say this helps retrieval, yet Ripple's long-context reasoning score (AA-LCR 75.0 vs 80.7 for Muse Spark Open) shows it isn't a long-context leader either.

Technical Moats

  • The data mix and ladder. The architecture is reproducible from config.json in an afternoon. The 12T-token, code-upsampled mixture and the ladder experiments that chose it are not published, and they're where the claimed 16x to 55x efficiency comes from.

  • Synthetic post-training pipelines. No purchased data. Their own synthetic generation, filtering and coverage metrics for long-horizon agent environments.

  • RL stability engineering. Combining TIS, score centering and length penalties at 128K context over 700 steps without collapse is craft, not just code.

  • A domain flywheel. The same model family runs Ricursive's chip-design platform, so product feedback flows into training data that general labs don't see.

Contrarian Insights

One: the headline margin is a harness artifact in both directions. Ripple leads Beam by 0.42 points using Beam's self-reported number. On Ricursive's own Terminus 2 re-run, Beam drops to 70.8 and the lead becomes 9.7, while Muse Spark Open rises to 78.7, just 1.8 behind. No run counts, no confidence intervals. If TerminalBench 2.1 keeps around 89 tasks like 2.0, one run has a standard error near 4 points, which swallows a 0.42-point lead whole. Ricursive is unusually candid that "replicating benchmark scores has been challenging." Readers should take them at their word: the model is clearly competitive, and the ranking is unproven.

Two: dense was the right training choice and the wrong local-inference pitch. "Fits on a DGX Spark" is true with NVFP4. But a dense 65B decodes at under 5 tokens per second on 273 GB/s of bandwidth, and agentic coding is output-heavy, looped and latency-sensitive. Where Ripple really wins on cost is the API at $0.20 / $0.60, cheap because it needs two B200s instead of a rack. The real story is "frontier-class terminal agent that a small inference provider can serve," not "coding agent on your desk."

Surprising Takeaway

Ripple's parameter count reproduces from its config to within 20,560 parameters, but only if its attention gate is elementwise. That gate is 4.19B parameters, 6.5% of the whole model, more than a Llama 3.2 3B, and its only job is keeping training stable. A five-person lab spent the parameter budget of an entire small model on insurance against loss spikes. Given that one diverged 12T-token run would have cost more than the gate ever will, that's probably the best trade in the architecture.

TL;DR For Engineers

  • Ripple-65B Preview is a 64.78B dense transformer: 80 layers at width 5,120, 80 query and 8 KV heads, elementwise sigmoid-gated attention, Apache 2.0.

  • 64 sliding-window RoPE layers (window 1,024) plus 16 global NoPE layers give a 1M context with a 64 GiB KV cache instead of 320 GiB.

  • Trained on 12T tokens at 40% MFU with stock kernels; RL used 700 steps and 1.4M rollouts with truncated importance sampling and score centering.

  • 80.52 on TerminalBench 2.1 is self-reported; the lead over Beam is 0.4 or 9.7 points depending on the harness, and Ripple trails on most other rows.

  • API is cheap ($0.20 / $0.60 per million). Local DGX Spark decode is bandwidth-capped near 5 tok/s.

Explain It Like I'm New

Coding assistants that can work on their own, opening files, running commands, fixing tests, need very capable AI models. Lately the best open ones have become enormous, hundreds of billions of parameters, because companies split them into many specialist "experts" and only use a few at a time. They work well but are complicated to train and need big server racks to run.

Ripple goes the other way. It's a single, conventional model, smaller than its rivals, where every part is used for every word. That's simpler to train, which is how a team of five could build it from scratch for what they say was $32 million.

The clever bits are about memory. To read a million words of context, most models keep notes on every word in every layer. Ripple lets most of its layers only look at the last thousand words, like a reader who keeps a few pages in view, and reserves a handful of layers for looking back over everything.

The catch is how it's judged. Ripple's top score depends on how the test is run, and the company's own retests move the gap with its rivals a lot. Running it on a small desktop machine works but is slow.

The bigger lesson is that smart choices and careful engineering still let small teams compete with the biggest labs.

See It In Action

  • Ripple-65B Preview launch post, Ricursive Intelligence. Read the announcement. The source for the architecture list, the cost definition and the TerminalBench chart, with the ladder comparison against Olmo 3.

  • Ripple-65B Preview on OpenRouter, OpenRouter. Try it in the playground. The fastest way to run your own terminal-agent prompts against it and see real latency and pricing.

  • Ripple-65B Preview: Benchmarks, DGX Spark Setup and License, AgentsPulse. Read the setup guide. Walks through the NVFP4 checkpoint's spark.py SGLang server at 128K context, and the two-B200 BF16 path with the DFlash2 drafter.

We found no video walkthrough or independent benchmark run as of November 10, six days after release.

Community Conversation

  • Ricursive Intelligence, launch thread on X. Frames Ripple as the U.S. open-weight leader in agentic coding over Muse Spark Open, Beam, Inkling and Nemotron 3 Ultra, and stresses that it "fits locally on a MacBook or DGX Spark."

  • braindetox, benchmark caveats breakdown. The most careful read so far: the 0.42-point margin, the self-reported versus Terminus 2 rows, in-house evals filling missing rival scores, and the 1M versus 2M context discrepancy.

  • AgentsPulse, local deployment notes. Points out that "15x smaller" counts total parameters, while Ripple actually activates more parameters per token than any MoE in its comparison table.

  • Martin Marek and Max Ryabinin (Together AI), Score Centering Stabilizes Off-policy RL. One of the two RL corrections Ripple credits, showing why sampler-trainer mismatch makes "on-policy" RL quietly drift.

  • Toolhalla, This Week in Local AI. llama.cpp support merged on November 5 (PR #31692), with third-party Q4_K_M GGUFs at 36.13 GiB; you need to build from master.

Conclusion: The Most Important Open Model of the Month Is a Recipe, Not a Ranking

Ripple-65B Preview is a convincing argument that a dense model, built from published ideas and trained with discipline, can compete with open models ten times its size on terminal-agent work. The architecture checks out to the parameter: hybrid NoPE attention for a cheap 1M context, an elementwise attention gate for stability, and stock kernels at 40% MFU. The RL recipe, with TIS and score centering over just 1.4M rollouts, is the part other labs should copy. What isn't settled is the ranking. Self-reported, its lead is within noise; on Ricursive's own re-run it's large; and it trails on most of the rest of the table. Use it through an API for terminal-heavy agents, benchmark it on your own harness before switching, and don't buy a DGX Spark expecting it to be a fast local coding agent.

References

Summary

Ripple-65B Preview is a 64.78B dense, Apache 2.0 coding model from a five-person team, built from published ideas: hybrid NoPE attention for a cheap 1M context, an elementwise attention gate for stability, and off-policy-corrected RL over 1.4M rollouts. Its 80.52 TerminalBench 2.1 lead is self-reported and harness-dependent, and it decodes slowly on a DGX Spark, but the recipe is the real contribution.

Sponsored Ad If you enjoy practical AI insights, check out SnackOnAI and support the newsletter by subscribing, sharing, and exploring our sponsored ad—it helps us keep building and delivering value 🚀

I Almost Threw Out My Keurig. It Wasn't Broken.

Every cup from my Keurig started tasting weak, flat, stale — so I unplugged it and stuck it in a closet. Then I learned the dirty secret of grocery store K-Cups: by the time a pod reaches your kitchen, it's usually spent months going from factory to warehouse to truck to shelf. The flavor oils are long gone. Some brands even over-roast to hide it.

The fix wasn't a new machine. It was fresh pods from Angelino's, a family-owned L.A. roastery that roasts, packs, and ships within 3–5 days — straight to your door, from 39¢ a cup. Over 1,000,000 happy customers. 50+ flavors, no subscription required, and 15% off your first order, applied automatically at checkout.

Recommended for you

View all
caret-right