In partnership with

SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | October 27, 2026

This issue takes apart Maya 2 Native, Maya Research's open-weight streaming text-to-speech model: how it turns text into audio tokens, how its engine runs 224 voices on one GPU, and where the numbers stop meaning what the launch posts imply. It covers the architecture, the serving engine, the evaluation and the tradeoffs. It doesn't cover the hosted Maya API, pricing, or a listening test, because the open release reports no human naturalness study and neither will we.

Contrarian Hook

Most engineers assume the expensive part of a speech model is the big language model in the middle.

In Maya 2 Native, the 1.7B parameter backbone takes 2.0 ms per audio frame. The 555M parameter depth decoder takes 6.5 ms. The smaller network costs three times more, every 80 milliseconds, for every voice on the GPU.

That one fact explains almost every design decision in this release: the frame rate, the CUDA graphs, the fp16 choice, and why the team publishes concurrency numbers instead of model size.

What It Actually Does

Maya 2 Native reads text and streams 24 kHz mono speech in 80 ms chunks. Strip away the launch copy and the release is:

  • A 2.50B unique-parameter model (about 5.0 GB in BF16), Apache-2.0 for weights and code (model card).

  • 12 languages: Assamese, Bengali, English, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil and Telugu.

  • 20 built-in voices chosen by name (Ahana, Siddhant, Ritvik and others), plus zero-shot cloning from 3 to 10 seconds of reference audio and its exact transcript. Cloning works across languages: a Hindi reference can speak Tamil.

  • Three ways in: a Python API, an OpenAI-compatible POST /v1/audio/speech, and a WebSocket stream at /v1/tts/stream (engine docs).

  • Measured serving: 224 concurrent streams on one H100 NVL with 111 ms median time to first audio, about 323 seconds of audio per wall-clock second.

What it isn't: the same thing as the hosted Maya 2 that topped Voice Arena two weeks ago. More on that in the insights.

The Architecture, Unpacked

Maya 2 Native is a two-speed autoregressive model sitting on top of a neural audio codec. If you've read the Moshi paper, you'll recognize the shape: a big "temporal" transformer that steps once per audio frame, and a small "depth" transformer that fills in the detail inside each frame.

Caption: Focus on the depth decoder. It runs 31 sequential steps per frame, which is why it costs more than the backbone ten times its size.

Three decisions do the heavy lifting.

A low frame rate. Mimi encodes speech at 12.5 frames per second, so the backbone only steps every 80 ms. A 10 second sentence is 125 backbone steps, not thousands. This is what makes a 1.7B language model affordable as a speech engine.

A semantic first codebook. In the Moshi paper, Mimi's first codebook is distilled from a self-supervised speech model, so it carries mostly content ("what is said") while later codebooks carry acoustics ("how it sounds"). The backbone predicts only that first token. Language understanding goes where the language model is, and acoustic polish goes to the cheap network.

All 32 codebooks. Moshi ran Mimi with 8 codebooks, about 1.1 kbps. Maya uses all 32, about 4.4 kbps. That buys fidelity, and it costs four times as many depth steps. This is the single choice that makes the depth decoder the bottleneck.

❝

📎 A full-colour version of this diagram ships as maya2-native-architecture.svg alongside this issue.

The Code, Annotated

Calling it: generation, streaming and cloning

From the model card quickstart, with commentary on why each argument exists.

from maya3 import Maya3TTS   # the package is still called maya3, the model is Maya 2 (yes, really)

# max_seats sizes the CUDA graphs AND the KV cache up front.
# 1 seat ≈ 7.7 GB steady (9.2 GB peak while loading); 32 seats ≈ 15.8 GB.
tts = Maya3TTS.from_pretrained("maya-research/maya2-native", max_seats=1)  # ← THIS is the capacity decision

# Batch generation: returns a mono float32 NumPy waveform at 24 kHz.
audio = tts.generate(
    "नमस्ते, आपका ऑर्डर कल पहुँच जाएगा।",
    lang="hi",            # explicit language: 12 choices share one backbone
    voice="Ahana",        # built-in voice, KV prefix already cached
    normalize=True,       # OFF by default in Python, ON in the servers. Forget it and "₹2,499" gets read badly.
)

# Streaming: yields int16 PCM chunks as frames are decoded (80 ms each).
for chunk in tts.stream(
    "Your order has shipped and will arrive tomorrow.",
    lang="en",
    voice="Siddhant",
    normalize=True,
):
    pass  # send chunk to the speaker/socket immediately; first one arrives ~14 ms after the call at B=1

# Zero-shot cloning: the transcript must match the reference audio exactly,
# because the clip becomes a text+codes PREFIX, not just an embedding.
tts.register_voice(
    "my_voice",
    ref_audio="reference.wav",                       # 3 to 10 seconds
    ref_text="<exact transcript of reference.wav>",  # ← mismatches here corrupt the prefix alignment
    lang="hi",
)

Caption: Cloning is prompt conditioning, not fine-tuning. The reference becomes a cached prefix, which is why it costs only about 1 ms extra to first audio (15 ms vs 14 ms).

Why the default context is plenty: the KV budget, computed

The engine docs say KV costs "112 KiB per token per seat." That number falls straight out of the Qwen3-1.7B config. This snippet runs as-is.

# KV cache cost per token = layers × KV heads × head_dim × 2 (K and V) × bytes
layers, kv_heads, head_dim = 28, 8, 128   # Qwen3-1.7B: 16 query heads share 8 KV heads (GQA)
bytes_per_val = 2                          # bf16

kv_per_token = layers * kv_heads * head_dim * 2 * bytes_per_val
print(kv_per_token / 1024)                 # Output: 112.0  (KiB), matches ENGINE.md exactly

# The trick: one backbone token = one 80 ms frame, not one text subword.
frame_rate = 12.5                          # Mimi frames per second  ← THIS is the trick
capacity = 2048                            # default KV tokens per seat
print(capacity / frame_rate)               # Output: 163.84 seconds of audio per seat (minus prompt)

per_seat_gib = capacity * kv_per_token / 2**30
print(round(per_seat_gib, 3))              # Output: 0.219 GiB per seat
print(round(32 * per_seat_gib, 1))         # Output: 7.0 GiB at 32 seats (docs report 7.8 GB KV at 32)

# Halve capacity → halve KV. Docs: an RTX 4090 fits 48 seats at 2048, 96 at 1024.

Caption: Grouped-query attention halves the KV bill, and the 12.5 Hz frame rate makes each KV slot cover 80 ms of speech. Long-form TTS is cheap in memory; it's expensive in sequential compute.

Serving it: the OpenAI-compatible endpoint

# One process, one GPU, no auth. Put a proxy with auth in front before exposing it.
python -m maya3.server \
  --ckpt maya-research/maya2-native \
  --host 127.0.0.1 --port 8080 --max-seats 1   # raise seats for concurrency; 224 fit on an H100 NVL

# Same request shape as OpenAI's speech API, so existing clients mostly just change the base URL.
curl --fail-with-body -sS http://127.0.0.1:8080/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"input":"Your order has shipped.","language":"en","voice":"Ahana","response_format":"wav"}' \
  --output speech.wav
# wav, pcm, mp3 and opus stream as they're generated; flac waits for the full clip.

Caption: OpenAI-shape compatibility is a distribution strategy. Any agent stack that already speaks /v1/audio/speech can swap Maya in with a URL change.

It In Action

One Hindi sentence, end to end, on one H100 NVL with a single seat. Engine timings come from ENGINE.md; the clip length is our estimate.

Input: "नमस्ते, आपका ऑर्डर कल पहुँच जाएगा।" ("Hello, your order will arrive tomorrow."), lang="hi", voice="Ahana".

  1. Normalize. Numbers, dates, currency and URLs become words. This sentence has none, so it passes through unchanged.

  2. Admit. Ahana's prefix (reference text plus reference codes) is already in the KV cache, so it's copied into the seat's slot instead of recomputed. Only the target text is prefilled, inside the same graphed backbone step as other seats' decode rows.

  3. First frame. Backbone 2.0 ms → cb0 sampler 0.35 ms → depth decoder 6.5 ms for cb1 to cb31 → Mimi 1.8 ms. That's 10.5 ms for 80 ms of audio. Measured time to first audio: 14 ms.

  4. Stream. Each later tick emits one 80 ms chunk: 1,920 samples, 3,840 bytes of int16 PCM. Assume the clip runs about 2.4 seconds: that's 30 frames, 960 codes, about 315 ms of compute. The docs' measured real-time factor is 0.137, so this lines up.

  5. Guard. If the model loops or rambles, generation is cut at 2.2 × expected duration + 1.2 s, about 6.5 s here, and a loop detector boosts the end-of-audio logit by +4.

  6. Output. A 2.4 s wav is about 115 KB of PCM (24,000 samples/s × 2 bytes) plus a 44-byte header.

t = 0 ms ──→ request admitted (voice prefix KV copied, text prefilled)
t ≈ 14 ms ─→ chunk #1 (80 ms of audio) leaves the GPU     ← listener can start hearing
t ≈ 315 ms → chunk #30, clip done; 2.4 s of speech produced in ~0.3 s

Caption: At one seat, the model runs about 7× faster than real time. The question in production is how much of that headroom survives at 224 seats.

At 224 seats, it mostly does. A tick takes about 30 ms at a batch of 160, well inside the 80 ms each stream consumes, and the docs report p99 time to first audio of 166 ms (plain) and 185 ms (cloned) with zero stalls. That works out to about 1.5 seconds of audio per second, per stream, on one GPU.

Why This Design Works, and What It Trades Away

Why it works. Every hot step is a CUDA graph: backbone, depth decoder and Mimi decoder are each captured once and replayed. That removes Python and kernel-launch overhead, which at 31 sequential depth levels would otherwise dominate. Continuous batching means one tick serves every active seat, and there's exactly one device-to-host copy per tick. The voice-prefix KV cache turns cloning into a memory copy.

What it trades away.

  • Flexibility for speed. CUDA graphs and KV are sized at load time by max_seats. You can't burst from 32 to 200 streams without restarting.

  • Fidelity costs latency. 32 codebooks sound better than 8, but each extra codebook is a sequential depth step. You can't parallelize your way out of 31 dependent steps.

  • Precision for speed in the depth decoder. It runs in fp16. The docs say fp16 is "the closest to fp32 greedy" among the fast options, which tells you they checked and it isn't identical.

  • Guards instead of guarantees. Autoregressive audio can loop or babble. The duration cap and loop guard are heuristics bolted on top, and they're suspended around inline tags like [laughs].

  • Training transparency. "Licensed and synthetic speech" with no hours, no dataset names. You can run the model, but you can't audit or reproduce its data.

Technical Moats

Indic speech data, not architecture. Everything structural here is borrowed in the open: Qwen3 backbone, Mimi codec, ECAPA speaker embeddings, the Moshi-style temporal/depth split. A competent team can rebuild the skeleton in weeks. What they can't rebuild is per-voice, per-language recordings across Odia, Assamese and Punjabi, and the card's own limitation (no built-in voice has even an hour of Assamese) shows how scarce that data is.

Preference data tuned on Indic ASR. The DPO stage ranks candidates by ASR agreement and speaker similarity. That needs reliable ASR judges in 12 languages, which is its own engineering project.

Serving discipline. Graph-captured three-stage decoding with prefix KV reuse, soak-tested at 224 seats for 30 minutes with zero stalls. Plenty of open TTS models are fast at batch size one and fall apart at batch size fifty. This one publishes its p99.

Distribution. OpenAI-compatible HTTP, a Pipecat integration and Baseten deployments in the US and India mean it's already wired into agent stacks.

Insights

Insight One: The leaderboard wins don't belong to the model you can download. The "#1 real-time Hindi on Voice Arena" headline from mid-October was about Maya's hosted models, launched as closed API products (Digit, ArtRealm). The hosted API's voices (Ananya, Arjun) aren't in the open checkpoint's list of 20, emotion tags need a separate "Emotional" model there, and the hosted API caps requests at 130 words. The open card reports no human naturalness study and no head-to-head comparison. Treat the open model as unbenchmarked on quality until you run your own listening test.

Insight Two: A 2.35% character error rate can hide a worse-sounding model. The headline evaluation is intelligibility: three ASR systems transcribe the output and count errors. The DPO stage also optimizes for ASR agreement. When the training reward and the test metric are the same family, you can get speech that machines transcribe well and humans find over-enunciated. The metric can't see prosody, emotion or naturalness. And the hard cases in Indian voice agents (URLs, long digit strings, code-mixed Hinglish) are exactly where the card admits it's weakest.

Surprising Takeaway

The 555M depth decoder takes 62% of every frame's compute (6.5 of 10.5 ms), while the 1.7B backbone takes 19%. In codec-based TTS, latency scales with the number of codebooks, not the size of the language model. Choosing 32 codebooks over Moshi's 8 is a bigger latency decision than choosing a 1.7B backbone over a 7B one.

TL;DR For Engineers

  • One frame = 80 ms of audio = 32 codes. The backbone predicts one code; the depth decoder predicts 31.

  • Profile the depth decoder first. It's 6.5 of 10.5 ms per frame at batch size one.

  • Set normalize=True in Python. It's off by default there and on in the servers.

  • Plan capacity at load time. max_seats fixes memory: 7.7 GB at one seat, 15.8 GB at 32; 224 seats fit on an H100 NVL.

  • Run your own listening test. The public quality claims are about the hosted models, and the open card measures intelligibility only.

Explain It Like I'm New

Picture a phone support line in India. A customer might switch between Hindi and English mid-sentence, ask about an order number, then want the answer in Tamil. For an AI to handle that call, it needs a voice: software that turns written replies into natural speech, fast enough that nobody notices a pause.

Older text-to-speech systems worked like a pipeline of specialists. One part figured out pronunciation, another planned the rhythm, a third produced sound. Each language needed its own careful engineering, and most of that effort went to English.

Maya 2 Native takes the approach that changed chatbots. It treats sound like text. A compression tool chops speech into tiny chunks and turns each one into a short list of numbers, almost like words in a sound vocabulary. Then a language model, the same kind of model behind chatbots, learns to predict the next chunk of sound the way it predicts the next word.

The clever part is the split of labour. A large model decides what is being said, a few times per second. A smaller model fills in how it sounds: the voice, the texture, the breath. Separating those jobs is what makes it fast enough to talk in real time.

Why it matters: hundreds of millions of people will meet AI first through voice, in their own language. Open models like this one let builders serve them without renting a foreign company's voice by the minute.

See It In Action

Community Conversation

  • Dheemanth Reddy, Maya Research CEO launched Maya 2 claiming it beats ElevenLabs v3, Cartesia Sonic 3.5 and Sarvam's Bulbul v3 on Indian languages, and framed India's AI future as "voice-first." The claim is vendor-reported, without published methodology.

  • Dishant, on DEV Community flagged the practical rough edges: the maya3 package name, case-sensitive voice names, raw PCM with no header from the hosted API, and an 88 ms vs 389 ms first-audio gap between reused and new connections. He also notes a 5 GB GPU checkpoint doesn't reach the cheap-Android users it's meant for.

  • Digit pointed out the gap between "India's 22 official languages" in the launch framing and the dozen actually supported, and called for independent tests in Marathi, Malayalam and Punjabi.

  • Analytics India Magazine covered today's open-weight release and highlighted what's missing: no human naturalness study and no controlled comparison with other TTS models.

  • Pipecat docs show the integration path builders are taking: a Maya-maintained service that keeps one WebSocket open across turns and cancels on interruption, the pattern that wins back those 300 ms.

The Codebook Count Is the Product Decision

Maya 2 Native isn't interesting because it's open, or because it speaks Odia. It's interesting because it shows, in published per-stage timings, where codec-based speech models actually spend their time. The backbone is the easy part. The 31 sequential steps after it are the real cost, and the team engineered around them with graphs, prefix caching and batching instead of hiding them. That's the right instinct.

Use it for Indic voice agents where you control the GPU and can run your own listening tests. Don't treat its Voice Arena headlines as evidence for the checkpoint you downloaded. And when the next open TTS model brags about parameter count, ask how many codebooks it decodes per frame.

References

Maya 2 Native is an open-weight, Apache-2.0 streaming TTS model for 11 Indian languages and English, built from a Qwen3 backbone, a 31-level depth decoder and Kyutai's Mimi codec. Its key lesson is that codebook count, not backbone size, sets latency, and its engine hides that cost well enough to serve 224 streams per H100. Its quality claims, though, belong to the hosted models until someone runs a real listening test on the open checkpoint.

Sponsored Ad If you enjoy practical AI insights, check out SnackOnAI and support the newsletter by subscribing, sharing, and exploring our sponsored ad, it helps us keep building and delivering value 🚀

The AI notetaker that gets the hard words right

Your AI is only as good as what you feed it. Feed it a transcript with your product names, acronyms, and numbers spelled wrong, and you spend the afternoon hand-fixing the output.

Wispr Flow Notetaker uses your dictionary and calendar, so product names, acronyms, numbers, and uncommon names come out spelled right. It pops up when your meeting starts and captures Zoom, Google Meet, Teams, and Slack huddles with one click, no bot joining the call.

Then your meetings show up inside Claude or ChatGPT through the built-in connector. No copy paste. Try Notetaker free on Mac and Windows.

Recommended for you

View all
caret-right
© 2026 Snack On AI.
beehiivPowered by beehiiv