M Star: Stanford and UW Built a Universal Multimodal Serving System. The Key Insight Is That Every Model, From BAGEL to V-JEPA to Qwen3-Omni, Is Just a Graph. Every Request Is Just a Walk.

In partnership with

One runtime, with one configuration, serves BAGEL's text-to-image generation, image editing, and image understanding simultaneously, a task vLLM-Omni cannot accomplish with a single configuration. On Qwen3-Omni TTS, M delivers 2.7x higher throughput than vLLM-Omni and 4.0x vs SGLang-Omni at batch size 16. On V-JEPA 2 robotic planning rollouts at horizon 30, it is 12.5x faster than Meta's native implementation.

SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | July 12,2026

Serving systems for LLMs assume a single execution structure: a prefill phase, then an autoregressive decode loop. This assumption held when "multimodal" meant attaching a vision encoder to a language model backbone. It does not hold anymore.

BAGEL runs text-to-text chat, text-to-image generation, image-to-text understanding, and image-to-image editing, all through the same Mixture-of-Transformers backbone, but each task activates a different subset of components in a different pattern. Qwen3-Omni has a Thinker transformer that produces text, a Talker transformer that converts hidden states to codec tokens, and a Code2Wav decoder that produces audio waveforms, pipelined in a streaming configuration where the Thinker's output feeds the Talker one token at a time. V-JEPA 2 runs an autoregressive video prediction loop over a variable number of steps, with KV cache growing at every iteration.

None of these map cleanly to prefill + decode. vLLM-Omni and SGLang-Omni approximate them as DAGs of stages. The approximation leaks: you cannot have loops across stages, so diffusion denoising loops must be hidden inside a stage with CUDA graphs disabled. You cannot have parallelism across stages, so CFG branches need model-specific glue code. You need a different configuration for each workload variant of the same model.

M*'s bet is that every multimodal model is already a directed computation graph, and that treating it explicitly as one, rather than approximating it as a DAG of opaque stages, is the abstraction that makes model-agnostic optimizations possible.

Scope: the Walk Graph abstraction (four composable primitives, streaming edges, Walks), the runtime implementation (Conductor + Workers + ZeroMQ + FlashInfer + Mooncake), and the full benchmark results across BAGEL, Qwen3-Omni, Orpheus, V-JEPA 2, and π0.5. Not covered: the RL training-time applications of M*, or the planned multi-node inter-node communication support (current evaluation uses intra-node only).

What It Actually Does

M* is a serving runtime. You declare your model as a Walk Graph, specify a YAML placement config mapping graph nodes to GPU ranks, and M*'s runtime handles all scheduling, batching, tensor transport, and optimization. You can change placement without modifying model or runtime code.

Quick start structure:

# mstar.stanford.edu/mstar — Python 3.12 + PyTorch + CUDA
from mstar.graph.base import Sequential, Loop, Parallel, DynamicLoop
from mstar.graph.base import GraphNode, GraphEdge, StreamingGraphEdge

# A model is declared as a Walk Graph: computation graph + named Walks
# A request is a series of Walks over the graph

Five composite model families M handles with a single runtime:*

Model family	Example	Non-AR pattern	M* primitive
Unified multimodal (UMM)	BAGEL	CFG 3-branch parallel, flow loop	Loop + Parallel
Omni model	Qwen3-Omni	Thinker streaming to Talker + Code2Wav	StreamingGraphEdge
Speech LM	Orpheus	AR + streaming SNAC decoder	StreamingGraphEdge
VLA (robot)	π0.5	Action decoder, MPC fan-out	Parallel
World model	V-JEPA 2	Variable-horizon AC rollout	DynamicLoop

The Architecture, Unpacked

The granularity gap between a "stage" and a "graph node" is the source of every benchmark result. A stage is an opaque engine that runs a full forward pass of a submodel. A graph node is a single component forward. With node-level granularity, M can see inside the stage and apply optimizations (paged KV, CUDA graphs, tensor parallelism) at the right resolution.*

The Code, Annotated

Snippet One: BAGEL Walk Graph, From Simplified to Full CFG Parallel

# M*: BAGEL Walk Graph for image generation
# Source: arXiv:2606.12688 Listing 1 + Appendix F (exact paper code)
# Design intent: the same Loop primitive handles diffusion denoising (fixed count)
# AND autoregressive text generation (DynamicLoop until EOS)
# Nothing is special-cased to any modality

from mstar.graph.base import Sequential, Loop, Parallel, DynamicLoop
from mstar.graph.base import GraphNode, GraphEdge

# ─── SIMPLIFIED (non-CFG) IMAGE GENERATION WALK ───────────────────────────────
# Direct from the paper's Listing 1
# ← This is the entire image generation pipeline: 49-step flow loop → VAE decode
# ← The same Loop primitive could express autoregressive decode with DynamicLoop
image_gen = Sequential([
    Loop(
        section=GraphNode(
            name="LLM",
            input_names=["latents", "time_index"],
            outputs=[
                GraphEdge(next_node="LLM", name="latents"),      # loop feedback
                GraphEdge(next_node="LLM", name="time_index"),   # loop feedback
            ]
        ),
        n_iters=49,  # ← num_timesteps - 1; fixed count = Loop (not DynamicLoop)
        outputs=[GraphEdge(next_node="vae_decoder", name="latents")],
    ),
    GraphNode(
        name="vae_decoder",
        input_ids={"latents"},
        outputs=[GraphEdge(next_node=EMIT_TO_CLIENT, name="image_output")]
    )
])

# ─── FULL CFG-PARALLEL IMAGE GENERATION WALK ──────────────────────────────────
# From Appendix F of the paper — the production version with classifier-free guidance
# ← CFG requires 3 LLM forward passes per denoising step (conditional + 2 negatives)
# ← vLLM-Omni handles this with model-specific glue: expands one request to 3
# ← M* handles it with the Parallel primitive: 3 nodes, 3 GPU ranks, no glue
image_gen_cfg = Sequential([
    Loop(
        section=Sequential([
            Parallel([                         # ← THIS is the key: 3 branches in parallel
                # Each branch gets its own GPU rank via YAML placement
                # No glue code. No model-specific handling.
                # The system schedules all three concurrently as regular graph nodes.
                GraphNode("LLM",          ["latents", "t"], [Edge("combine_cfg", "v_main")]),
                GraphNode("LLM_cfg_text", ["latents", "t"], [Edge("combine_cfg", "v_text")]),
                GraphNode("LLM_cfg_img",  ["latents", "t"], [Edge("combine_cfg", "v_img")]),
            ]),
            GraphNode("combine_cfg",           # Euler step: applies CFG formula + updates latents
                ["v_main", "v_text", "v_img", "latents", "t"],
                [loopback to all three LLMs]),  # feeds back into next iteration
        ]),
        max_iters=49,                          # 49 Euler steps
        outputs=[Edge("vae_decoder", "latents")]
    ),
    GraphNode("vae_decoder", ["latents"], [Edge(EMIT_TO_CLIENT, "image_output")]),
])

# ─── WHY THIS MATTERS FOR KV CACHE ─────────────────────────────────────────────
# vLLM-Omni: CFG requires 3 separate NaiveCaches
#   Each denoising step: concatenate K and V tensors at every layer for every branch
#   Cost: O(seq_len) memory allocation per step per branch
#
# M*: 3 CFG branches = 3 labels over the SAME single paged KV pool
#   Each denoising step: paged attention reads page tables in place
#   ← This is the source of the 2.64x I2I latency improvement over vLLM-Omni default
#   ← The "label" axis on the paged KV cache is a general mechanism:
#      it inherits all paging, offload, by-reference transfer, and continuous batching
#      automatically, without model-specific code

The Parallel primitive's power is that it decouples the logical fan-out (three CFG branches) from the physical execution (three GPU ranks) through the YAML placement config. The model author declares the branches. The deployer assigns them to hardware. M's runtime handles the scheduling, synchronization, and tensor transport.*

Snippet Two: Qwen3-Omni Thinker-Talker Streaming and V-JEPA DynamicLoop

# M*: Two contrasting streaming patterns
# Design intent: StreamingGraphEdge with ChunkPolicy is the general mechanism
# for any producer-consumer streaming; the runtime is agnostic to the policy

from mstar.graph.base import StreamingGraphEdge
from mstar.graph.chunks import FixedChunkPolicy, LeftContextChunkPolicy

# ─── QWEN3-OMNI: THINKER → TALKER → CODE2WAV ─────────────────────────────────
# Qwen3-Omni is a Thinker–Talker–Code2Wav pipeline
# ← Thinker: AR LLM that generates text and hidden states
# ← Talker: codec-token AR LLM that converts hidden states to audio codec tokens
# ← Code2Wav: vocoder that converts codec tokens to audio waveforms

qwen3_omni = Sequential([
    # Thinker (rank 0): generates hidden states one token at a time
    GraphNode("Thinker", ["input_tokens"], [
        # ← FixedChunkPolicy(K=1): Talker should consume EACH Thinker output immediately
        # This enables the Talker to start generating audio while Thinker is still running
        # ← vLLM-Omni cannot enable CUDA graphs for Code Predictor (shape changes)
        # ← M*: the entire Talker submodule including its multi-token predictor loop
        #   runs as a SINGLE CUDA graph because graph-level Loop makes it shape-static
        StreamingGraphEdge("Talker", chunk_policy=FixedChunkPolicy(K=1)),
        GraphEdge("output_text"),              # text output, returned to client
    ]),
    # Talker (rank 1): AR LM generating codec tokens, pipelined with Thinker
    GraphNode("Talker", ["hidden_states"], [
        # ← LeftContextChunkPolicy(C=25, L=2): fire every 25 codec tokens,
        # but prepend the 2 most recent prior tokens for acoustic continuity
        # (without them, the SNAC audio codec produces clicking at chunk boundaries)
        StreamingGraphEdge("Code2Wav",
                           chunk_policy=LeftContextChunkPolicy(C=25, L=2)),
    ]),
    # Code2Wav (colocated with Talker on rank 1): vocoder
    # ← Both Talker and Code2Wav on same Worker process = no inter-process IPC
    # ← vLLM-Omni and SGLang-Omni require separate processes for each stage
    GraphNode("Code2Wav", ["codec_chunks"], [
        GraphEdge(STREAM_TO_CLIENT, "audio_output"),  # streamed, not buffered
    ]),
])

# PLACEMENT (from Appendix C, Listing 2):
# Thinker prefill → rank 0
# Thinker decode  → rank 1 (PD disaggregation via per-Walk placement)
# Talker + Code2Wav → rank 1 (colocated: no IPC cost)
# This placement eliminates the inter-process communication of Talker codes to Code2Wav
# ← Source of 2.7x throughput and 2.9x RTF improvement over vLLM-Omni

# ─── V-JEPA 2: VARIABLE-HORIZON WORLD MODEL ROLLOUT ──────────────────────────
# V-JEPA 2 predicts the next video frame autoregressively, conditioned on actions
# Variable rollout horizon H per request
# ← Baseline (Meta's native): hand-written Python for-loop, no KV cache
#   Every iteration: prefill over the full growing sequence (O(H^2) compute)
# ← M*: DynamicLoop with paged-attention KV caching
#   Every iteration: decode step, KV cache grows by 1 (O(H) compute)

vjepa2_rollout = DynamicLoop(
    section=Sequential([
        GraphNode("VideoEncoder", ["frame_obs", "action"], [
            GraphEdge("Predictor", "encoded_frame"),
        ]),
        GraphNode("Predictor", ["encoded_frame", "kv_cache"], [
            # ← This is where paged attention matters:
            # KV cache grows with each rollout step but is never recomputed
            # The baseline re-prefills the entire sequence at every step
            # At H=30: baseline does 30 full prefills; M* does 30 decode steps
            # ← Source of 12.5x speedup at H=30 (speedup compounds with horizon)
            GraphEdge("VideoEncoder", "predicted_latent"),  # next-step prediction
        ]),
    ]),
    max_iters=30,
    stop_name="horizon",  # DynamicLoop: terminates at request-specific H, not fixed
    # H=4 → 2.08x speedup, H=15 → 3.76x, H=30 → 12.5x vs native implementation
)

The DynamicLoop's speedup scaling with horizon (2.08x at H=4, 12.5x at H=30) is precisely what you expect when you replace O(H^2) prefill-per-step with O(H) decode-per-step. The speedup is not a system trick; it is the fundamental algorithmic difference between re-prefilling a growing sequence and maintaining a KV cache. M makes this available to a world model by encoding the rollout as a first-class graph primitive rather than a Python for-loop.*

It In Action: BAGEL Serving All Three Workloads With One Configuration

Setup: BAGEL-7B, 1024x1024 images, 50-step flow schedule, 3x H100 (CFG parallel for generation, 1 H100 for understanding).

The problem with vLLM-Omni: Two configurations exist, neither works for all three:

vLLM-Omni DEFAULT configuration:
  T2I: ✓ good
  I2I: ✗ poor (Thinker + DiT are separate stages → expensive KV-cache transfer per step)
  I2T: ✓ good (runs on vLLM's optimized AR engine with continuous batching)

vLLM-Omni SINGLE-STAGE configuration:
  T2I: ✓ good
  I2I: ✓ good (collapses Thinker + DiT, removes inter-stage KV transfer)
  I2T: ✗ poor (no longer on optimized AR engine; loses continuous batching)
         41 tokens/sec at B=1, half the speed of default config

M with one configuration:*

M* (3-GPU CFG-parallel):
  T2I: ✓ 1.25x lower latency vs vLLM-Omni single-stage at B=1
  I2I: ✓ 2.64x lower latency vs vLLM-Omni default at B=1
       (3 CFG branches as 3 labels over one paged KV pool vs 3 NaiveCaches)
  I2T: ✓ 32.7% higher throughput at B=16; 33% lower TTFT at B=1

One config, PD disaggregation, encoder disaggregation, and tensor parallelism
available via minor YAML-level changes.
No model or runtime code changes required.

Qwen3-Omni TTS on Seed-TTS benchmark (2x H200):

Batch size   | vLLM-Omni RTF | SGLang-Omni RTF | M* RTF
B=1          | ~0.08         | ~0.10           | ~0.05
B=4          | ~0.18         | ~0.22           | ~0.09
B=8          | ~0.30         | ~0.38           | ~0.14
B=16         | ~0.50+        | ~0.65+          | ~0.18

(RTF = processing time / audio duration; <1 means real-time capable)

At B=16: M* = 2.7x higher throughput vs vLLM-Omni, 4.0x vs SGLang-Omni

Why: Entire Talker + multi-token predictor loop runs as single CUDA graph in M*
     vLLM-Omni: CUDA graphs explicitly disabled for Code Predictor
     Talker + Code2Wav colocated on same Worker: no inter-process IPC

V-JEPA 2 rollout (1x H100, B=1):

Rollout horizon | Baseline latency | M* latency | Speedup
H=4             | ~1.6s            | ~0.77s     | 2.08x
H=15            | ~22s             | ~5.85s     | 3.76x
H=30            | ~85s             | ~6.8s      | 12.5x

Baseline: Python for-loop, re-prefill growing sequence at every step → O(H^2)
M*: DynamicLoop + paged-attention KV cache → O(H) decode steps
Speedup grows with H because the baseline cost is quadratic; M* is linear.

Why This Design Works, and What It Trades Away

The Walk Graph's key property is that it decouples what the model does from where it runs and how the runtime optimizes it. A vLLM-Omni "stage" is simultaneously the model component AND the placement unit AND the unit of runtime optimization. This coupling means that to optimize CFG branches in parallel, you modify the runtime to expand user requests into CFG branches, a model-specific operation in the runtime layer. To add CUDA graphs for a codec decoder, you need the codec's execution to be shape-static across iterations, which requires knowing the codec's iteration structure in the runtime.

M* separates these. The Parallel primitive declares that CFG branches should run concurrently. The YAML says which ranks. The runtime applies standard scheduling without knowing what CFG is. The Loop primitive declares that diffusion denoising iterates 49 times with shape-static inputs. The runtime applies CUDA graphs without knowing what diffusion is. When a new model architecture arrives, the model author expresses it with the four primitives. The runtime already knows how to execute it.

The per-request KV cache label axis for CFG is the most concrete performance contribution. In vLLM-Omni, three CFG branches require three dense NaiveCaches: at every denoising step, every layer, K and V tensors are concatenated. In M*, three branches are three labels in one paged KV pool. Paged attention reads page tables in place. The BAGEL I2I 2.64x improvement comes almost entirely from this one mechanism.

What M trades away:*

The current evaluation uses intra-node NVLink and shared memory only. The Mooncake RDMA/TCP data plane is implemented but multi-node experiments are not yet reported. At scale (across nodes with Infiniband), the Walk Graph's cross-node tensor transport overhead will matter. The paper's Limitations section is honest about this.

The four primitives are currently finite. Sequential, Parallel, Loop, DynamicLoop, and StreamingGraphEdge with three ChunkPolicy types cover the evaluation models. Future architectures (block diffusion, speculative decoding, sequence parallelism) will require either extending the primitive set or expressing them as compositions of existing primitives. The paper explicitly lists these as future work.

Support for additional parallelism forms (sequence parallelism, context parallelism, pipeline parallelism) is not yet implemented at the system level.

Technical Moats

The paged KV label axis. Representing multiple CFG branches as labels over a single paged KV pool is an implementation decision with significant correctness requirements. Each label must be consistent across layers and across denoising steps. The page table must correctly route attention queries from each branch to only their own K and V pages. Getting this correct requires understanding paged attention at the implementation level, not just the API level. The benefit, that every optimization available to single-branch paged attention (offloading, by-reference transfer, continuous batching) automatically applies to multi-branch CFG, justifies the implementation complexity.

The Loop primitive enabling CUDA graph capture for iterative non-AR workloads. CUDA graph capture requires that the execution graph be static: same operations, same shapes, same memory addresses across multiple calls. A diffusion denoising step that is hidden inside a stage with shape-changing intermediate states cannot be captured. M*'s Loop primitive guarantees that each iteration of a Loop is a call to the same static subgraph with shape-static inputs (latents of fixed dimension, time index). This is what enables CUDA graph capture for Qwen3-Omni's Code Predictor and BAGEL's diffusion loop, which vLLM-Omni documents as "cudagraph-incompatible" in its own codebase.

The double-buffered FlashInfer attention plan. FlashInfer's attention plan construction is one of the heaviest per-batch CPU-side overheads in serving LLMs. M*'s runtime constructs the next attention plan asynchronously in a separate thread and CUDA stream while the current batch is in flight. This overlap is enabled by the Walk Graph's structure: the Conductor can determine which nodes will fire after the current batch completes (by traversing the Walk Graph with current outputs) and precompute their attention plans without waiting.

Insights

Insight One: The Walk Graph's subsumption table (Table 3 in the paper) is the most precise description of what existing serving systems actually support, and it is damning. vLLM-Omni explicitly documents enforce_eager: true for BAGEL's DiT, meaning CUDA graphs are disabled for the component that accounts for most of the compute in image generation. The reason, according to the paper's analysis, is that vLLM-Omni's stage abstraction cannot guarantee shape-static execution for loops that cross stage boundaries, so CUDA graphs are conservatively disabled. M's Loop primitive guarantees shape-staticity by construction, which is why it can enable CUDA graphs for the same component. This is not a system tuning difference. It is an abstraction difference that has direct latency consequences.*

Insight Two: The V-JEPA 2 result is the most under-discussed result in the paper. A 12.5x speedup over Meta's native implementation at rollout horizon 30 sounds like an inference trick. It is actually just the difference between an O(H^2) implementation (re-prefill the growing sequence at every step) and an O(H) implementation (maintain a KV cache and decode one step at a time). The native implementation was a Python for-loop that called the model on the full growing sequence at every rollout step because no serving framework expressed world model rollout as a first-class primitive. M's DynamicLoop with paged attention KV caching expresses it correctly. The speedup is not from a clever optimization. It is from doing the right thing, which required the right abstraction.*

Surprising Takeaway

The same Loop primitive that handles a 49-step BAGEL image generation flow also handles autoregressive text generation in BAGEL's image-understanding mode, and also handles V-JEPA 2's variable-horizon video prediction rollout. These are three structurally distinct iteration patterns: fixed-count diffusion denoising, token-by-token generation until EOS, and frame-by-frame world model prediction until a variable horizon. In vLLM-Omni and SGLang-Omni, each of these requires separate treatment: diffusion is hidden inside a stage (CUDA graphs disabled), AR generation is the only first-class loop (CUDA graphs enabled), and world model rollout requires a hand-written Python loop outside the system entirely. M expresses all three with Loop or DynamicLoop, and the CUDA graph infrastructure applies uniformly. The design principle at work, that a universal abstraction that unifies semantically related but syntactically different patterns always beats a collection of special cases, is the architectural bet M is making.**

TL;DR For Engineers

M* (mstar-project/mstar, arXiv:2606.12688, Stanford + UW + CMU, June 2026): universal serving runtime for composite multimodal models. Core abstraction: Walk Graph (model = computation graph, request = series of Walks). Four composable primitives: Sequential, Parallel, Loop, DynamicLoop. Python 3.12 + PyTorch + CUDA; FlashInfer paged attention; ZeroMQ for inter-worker communication.
Key benchmarks: BAGEL ~20% lower E2E latency and 2.64x lower I2I vs vLLM-Omni (from paged KV label axis replacing NaiveCaches per CFG branch); Qwen3-Omni TTS 2.7x/4.0x higher throughput vs vLLM-Omni/SGLang-Omni at B=16 (from CUDA graphs on Talker submodule and colocated Code2Wav); V-JEPA 2 rollout 12.5x at H=30 (from DynamicLoop + KV cache replacing O(H^2) Python for-loop); Orpheus TTS 13.6% lower RTF, 20-52% higher throughput vs VoxServe.
The BAGEL one-config result is the clearest practical signal: M* serves T2I, I2I, and I2T with one configuration while vLLM-Omni requires different configs for different workloads and no single config performs well on all three simultaneously.
Current limitations: single-node only (inter-node RDMA via Mooncake not yet benchmarked); sequence parallelism, context parallelism, pipeline parallelism not yet supported; Loop primitive set finite (block diffusion, speculative decoding require future work).

The Serving Stack Has Been Rebuilt for the Text Era. It Needs to Be Rebuilt Again.

M*'s observation is that the production serving stack (vLLM, SGLang, and their multimodal extensions) was designed around a single execution pattern: prefill once, then decode autoregressively. Every multimodal extension has been an adaptation of that pattern. Non-AR loops are hidden inside stages with CUDA graphs disabled. Parallel branches require model-specific glue code. Variable-horizon iteration requires Python for-loops outside the serving system entirely.

The Walk Graph's bet is that expressing the model's actual structure, as it is, as a graph of heterogeneous components connected by typed edges, and letting the runtime execute that structure with model-agnostic optimizations, produces a better system than adapting a text-centric runtime to each new multimodal pattern.

The benchmark results are early evidence that the bet is correct. A 12.5x speedup on V-JEPA rollouts. A single configuration serving three BAGEL workloads where vLLM-Omni cannot find one that works for all three. 4.0x throughput on Qwen3-Omni TTS. These are not incremental improvements on a baseline that almost worked. They are the result of starting from a different abstraction.

References

M* GitHub Repository, mstar-project, Stanford + UW + CMU
M*: A Modular, Extensible, Serving System for Multimodal Models, arXiv:2606.12688, Jha, Sagan, Kamahori, Sivgin, Sanda, Gao, Horowitz, Zettlemoyer, Hsu, Leskovec, Kasikci, Wang, June 2026
M* Project Page, Stanford SAIL
VoxServe: Streaming-centric serving for speech language models, arXiv:2602.00269, Kamahori et al., 2026
vLLM-Omni: Fully disaggregated serving for any-to-any multimodal models, arXiv:2602.02204, Yin et al., 2026
V-JEPA 2: Self-supervised video models, arXiv:2506.09985, Assran et al., 2025

Explain It Like I'm New

Modern AI tools are no longer single machines. They are factories: different departments handling different tasks, all connected by conveyor belts moving data between them. A text-to-image model might have one department that reads your prompt, another that generates a rough image sketch over 50 refinement passes, and a third that sharpens it into a final picture. An AI voice assistant might have a department for understanding your words, another for generating a spoken response, and a third for converting that response into audio waves in real time.

The problem is that the software that runs these AI factories was built when factories only had one room. Tools like vLLM were designed for a simpler world where an AI reads your input once, then generates one word at a time until it is done. When engineers tried to bolt on image generators, audio processors, and robot planners, they ran into a structural mismatch. The result was slower performance, fragile configurations, and code that needed manual rewiring for every new model design.

M* proposes a cleaner mental model: every AI model is already a diagram of boxes and arrows, so treat it like one. Declare the boxes (the components), declare the arrows (the data flow), and let the system figure out how to run it efficiently on actual hardware. The same four building blocks express a 50-step image generation loop, a streaming voice pipeline, and a robotic video prediction rollout, because all three are structurally the same thing: a directed graph with some iteration.

This matters because the most capable AI systems being deployed right now are these compound factories. The serving infrastructure needs to match their structure. M* is an early proof that it can.

M* (mstar-project/mstar, arXiv:2606.12688, Stanford + University of Washington + CMU, preprint June 2026) is a universal serving runtime for composite multimodal models that represents any model as a Walk Graph: a directed computation graph of component forward passes connected by typed edges, with requests executing as series of named Walks. Four composable primitives (Sequential, Parallel, Loop, DynamicLoop) and streaming edges with three ChunkPolicy types express the full execution diversity of unified multimodal models (BAGEL), omni models (Qwen3-Omni), speech language models (Orpheus), vision-language-action models (π0.5), and world models (V-JEPA 2). Key benchmarks: on BAGEL, M* achieves ~20% lower E2E latency for T2I and 2.64x lower for I2I (from representing CFG branches as paged-KV labels rather than NaiveCaches), plus 32.7% higher I2T throughput at B=16, all with one configuration where vLLM-Omni requires two and neither works well for all three; on Qwen3-Omni TTS, 2.7x/4.0x higher throughput vs vLLM-Omni/SGLang-Omni at B=16 (from CUDA-graph capture of the entire Talker submodule and colocated Code2Wav); on V-JEPA 2 AC rollout, 12.5x speedup at H=30 (from DynamicLoop with paged-attention KV caching replacing an O(H^2) Python for-loop); on Orpheus TTS, 13.6% lower RTF and 20-52% throughput improvement over VoxServe.

Sponsored Ad

If you enjoy practical AI insights, check out SnackOnAI and support the newsletter by subscribing, sharing, and exploring our sponsored ad — it helps us keep building and delivering value 🚀

Try the AI that knows your customers. No commitment.

Most platform evaluations start with a demo request and end three weeks later in a conference room. This one takes 15 minutes and puts you directly inside Gladly's interface — navigating it on your own terms.

See how AI surfaces real-time customer context before a conversation starts. Watch how a single conversation thread pulls in purchase history, channel history, and account details without a handoff.

No installation. No commitment. Start the interactive demo and see the platform for yourself.

See it yourself