LFM2: Liquid AI's Hardware-in-the-Loop Architecture Search Found That SSMs Were Not the Answer for Edge Devices, and the Result Runs 2x Faster Than Qwen3 on a Galaxy S25

Liquid AI (arXiv:2511.23404, November 2025, updated through LFM2.5 in 2026) inverted this process: every candidate architecture in LFM2's search was compiled to llama.cpp and ExecuTorch, profiled on a Samsung Galaxy S24 Ultra and an AMD Ryzen HX 370 laptop, and measured for TTFT, decode latency, and peak memory before being evaluated for quality. The search eliminated Mamba, S4, Mamba-2, linear attention variants, and hybrid SSM combinations from the result. What survived: gated short convolutions for most layers, plus a small minority of grouped-query attention blocks. The resulting family delivers up to 2x faster prefill and decode on CPUs than similarly sized models, and LFM2-8B-A1B reaches 3-4B-class quality with only 1.5B active parameters.

SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | Aug 06, 2026

LFM2 (Liquid AI Team, November 2025, arXiv:2511.23404) is a family of small foundation models (350M, 700M, 1.2B, 2.6B dense; 8.3B/1.5B active MoE) built on a hybrid backbone discovered through hardware-in-the-loop architecture search. The backbone combines depthwise gated short convolutions (most layers) with grouped-query attention, GQA, (a minority of layers). All dense models support 32K context. LFM2.5 (January 2026) extended pretraining from 10T to 28T tokens and added reinforcement learning at scale. LFM2.5-8B-A1B (May 2026) expanded context to 128K and scaled pretraining to 38T tokens. LFM2.5-230M (June 2026, the smallest variant) was deployed on a Unitree G1 humanoid robot for real-time skill selection.

The multimodal family: LFM2-VL (vision-language, tunable accuracy-latency tradeoff via token-efficient visual processing), LFM2-Audio (separates audio input and output pathways for real-time speech-to-speech, competitive with models 3x larger), LFM2-ColBERT (late-interaction retrieval encoder using MaxSim scoring across multiple languages). All models are open weights on HuggingFace under the LiquidAI organization, with deployment packages for llama.cpp, ExecuTorch, vLLM, and SGLang.

Scope covered: the hardware-in-the-loop architecture search (objectives, search space, on-device profiling, outcomes), the gated short convolution block formulation, the decoupled tempered Top-K knowledge distillation objective, the three-stage post-training pipeline (SFT + curriculum learning, length-normalized preference alignment, model merging), inference benchmarks on Galaxy S25 and Ryzen HX 370, LFM2.5 and LFM2.5-8B-A1B updates, and the multimodal variants. Excluded: full LFM2-ColBERT retrieval benchmarks, vision-language training protocol details, and audio architecture specifics beyond the I/O pathway separation.

The LFM2 architecture search is the most important design decision in the paper, and the result challenges what the efficient architecture literature predicted.

The standard narrative in 2024-2025 is that hybrid architectures combining SSMs (Mamba, Mamba-2), linear attention, and short convolutions outperform pure Transformers on edge tasks because they reduce the KV cache growth that makes standard attention memory-expensive at long context. The LFM2 search tested this directly: every candidate, including pure SSMs, Mamba, S4, Liquid-S4, RTF, Mamba-2, linear attention variants, and combinations, was compiled to the deployment stack, profiled on real edge hardware (Samsung Galaxy S24 Ultra, AMD Ryzen HX 370), and evaluated on a 50+ task internal benchmark.

The outcome: under realistic edge latency and memory budgets, additional SSM or linear attention operators do not improve quality over the minimal hybrid. Gated short convolutions (for local sequence mixing at constant memory) plus a small minority of GQA layers (for global retrieval at long context) is the Pareto-optimal design. The more complex SSM-heavy hybrids are worse on device metrics and not better on quality.

Caption: The search's elimination of SSMs is the most significant and counterintuitive result. The paper's conclusion (Section 2.1): "under identical on-device performance budgets, augmenting these stacks with linear-attention, state-space, or additional convolution operators does not improve aggregate quality on the evaluation suite and typically worsens device metrics."

The gated short convolution block is the mechanism that allows LFM2 to outperform pure Transformer baselines on CPU while using zero KV cache per block. Each block applies input-dependent multiplicative gating around a depthwise short convolution:

The key properties for edge deployment: no KV cache per block (unlike attention), O(k) compute per token per block (unlike SSMs which require state updates), and excellent CPU cache behavior because the short kernel (k=3) fits in L1 cache. The search's finding is that at most sizes, the quality deficit from not having global attention is recoverable by adding a small number (6-8) of GQA blocks. The remainder of the layers use the cheaper convolution block without quality loss.

LFM2-8B-A1B replaces the dense SwiGLU MLPs in most layers with sparse MoE MLPs: 32 experts per layer, top-4 activated per token via a normalized sigmoid router with adaptive load balancing biases. The first two layers remain dense for stability. With 8.3B total parameters and 1.5B active per forward pass, the model targets the on-device quality-latency tradeoff: 3-4B class quality at 1.5B class decode cost.

LFM2.5-8B-A1B (May 2026) extended this to 128K context window, scaled vocabulary from 65,536 to 128,000 tokens (improving tokenization efficiency for Hindi, Thai, Vietnamese, Indonesian, and Arabic), and trained on 38T tokens with large-scale RL, delivering particularly strong tool calling chains on consumer hardware.

Caption: The paper's benchmark uses Q4_0 quantization with llama.cpp for all models in Tables 2 and 3. Reproducing the comparison numbers requires exactly this setup. Using Q4_K_M (higher quality quantization) for LFM2 while using Q4_0 for baselines would artificially favor LFM2.

Caption: The decoupled design is what prevents the naive Top-K distillation failure. Applying temperature to the full Top-K distribution (including the membership binary decision) causes support mismatch: the student sees a temperature-softened teacher that assigns probability to tokens the student's Top-K set doesn't include. Decoupling separates the membership question (L_B, untempered) from the relative ranking question (L_T, tempered) and solves both correctly.

At the 1.2B scale: LFM2-1.2B is 1.5-1.7x faster prefill and 1.3-1.5x faster decode than Llama-3.2-1B. Versus Qwen3-1.7B: 2.3-2.3x faster prefill, 1.7-2.1x faster decode across context lengths.

LFM2-2.6B matches or approaches Qwen3-4B quality at 2.6B parameters, with 2-3x better CPU throughput.

The gated short convolution block is fast on CPUs because depthwise convolution with kernel size k=3 is fundamentally different from attention: it accesses only three adjacent positions, fits in L1 cache, uses no KV cache, and scales as O(kd) per token rather than O(L²d) for full attention. The multiplicative gating (B⊙h̃ and C⊙z) adds input-dependency that a purely convolutional layer lacks. This is the architectural insight: convolutional local mixing is cheap, global retrieval via attention is expensive, and most sequence modeling tasks do not require global retrieval at every layer, only at a minority.

The KV cache reduction is the memory benefit. A standard transformer with only GQA has a KV cache that grows linearly with sequence length for every layer. LFM2 has a KV cache only for the minority of GQA layers (6-8 in each model), with zero cache for convolution layers. At 32K context, this produces a substantial peak RSS reduction versus attention-heavy models, directly observable in the edge profiling results.

The decoupled tempered Top-K distillation is what makes training a 350M model on 10T tokens produce useful quality. Using LFM1-7B as the teacher and distilling only the Top-32 logits per token reduces storage and bandwidth while avoiding the support mismatch failure of naive Top-K KL. The binary term forces the student to correctly predict which tokens are in the teacher's top set; the tempered conditional term forces it to match relative rankings within that set. Together they approximate the full KL without requiring all 65,536 teacher logits per token.

What this trades away. The search result may not generalize to larger scales. LFM2's models top out at 8.3B total parameters (1.5B active). Whether gated short convolutions plus minimal GQA remains Pareto-optimal at 30B, 70B, or 100B+ total parameters on edge hardware is an open question. The paper acknowledges this as a limitation (Section 9.2). For the sub-10B on-device regime, the evidence is strong. Beyond it, the search's conclusions are not validated.

The 128K context window in LFM2.5-8B-A1B is a meaningful capability expansion, but GQA blocks that were designed for 32K context now serve 128K. The paper does not report whether the original architecture search results hold at 128K, or whether additional GQA blocks are needed at that range.

The hardware-in-the-loop pipeline is hard to replicate because the feedback loop is the product. Most architecture papers evaluate quality on GPU clusters and report device benchmarks as an afterthought, measured on whatever hardware is available. LFM2's search integrated device measurement into the selection criterion: architectures that fail on device latency or memory budgets are eliminated before quality is measured. Reproducing this requires sustained access to representative edge hardware (multiple smartphone and laptop SKUs), working ExecuTorch and llama.cpp build pipelines for every candidate, and the engineering infrastructure to run hundreds of candidates through the full loop. This is a systems engineering investment, not just a research methodology.

The Qualcomm and Intel partnership network accelerates the hardware coverage moat. The LFM2.5-350M announcement includes partnerships with Qualcomm Technologies (Hexagon NPU), AMD (Ryzen AI), Intel (Neural Processing Unit), Zetic, RunAnywhere, and Mirai. These partnerships provide access to hardware characterization, optimized kernel libraries, and distribution channels that are not available to academic or small-team efforts. The hardware-in-the-loop search only works if the loop is complete from model to device to metric. Each hardware partner extends the coverage of that loop.

The curriculum learning framework measures actual model solve rates across 12 models. The SFT data curriculum sorts examples from easy to hard by computing the solve rate across an ensemble of 12 models ranging from LFM2-350M to Qwen3-235B-A22-Instruct. This is not a proxy: it measures actual model behavior on each training example. Reproducing this requires running inference across 12 models on every training example, which at 5+ million examples is a substantial compute investment. The resulting difficulty ordering is a proprietary artifact of Liquid AI's training infrastructure.

Insight One: The hardware-in-the-loop result eliminates SSMs under edge constraints, but the conclusion is device-specific. The paper is careful about this: the search was run under "realistic edge latency and memory budgets" on smartphone and laptop CPUs. On A100 GPUs with HBM memory bandwidth, the conclusion may differ. Mamba-2 and other SSM variants have been demonstrated to match or exceed Transformer throughput on long sequences where attention's quadratic cost dominates on GPU. The LFM2 result is not "SSMs are worse than convolutions" in general. It is "under edge CPU constraints, the benefits of SSM long-range processing do not compensate for the overhead of their state update operations compared to the simpler gated convolution." This is a hardware-conditional conclusion, and teams evaluating architectures for GPU inference should not apply it without re-running the search on GPU hardware.

Insight Two: The distillation ceiling is the capability ceiling. LFM2's quality comes partly from distilling LFM1-7B. The student cannot exceed the teacher on tasks where the teacher's Top-32 logits contain the signal. For reasoning tasks where the correct answer is not in the teacher's top-32 tokens (because the teacher itself is wrong), the DTK objective cannot help. LFM2.5's addition of large-scale reinforcement learning (multi-stage RL on instruction following, math, and tool use) is partly an acknowledgment of this: beyond a certain point, distillation from a teacher that makes mistakes on hard tasks cannot improve the student on those same tasks. RL on verified rewards fills the gap where distillation saturates.

LFM2.5-230M was deployed on a Unitree G1 humanoid robot for real-time skill selection, making it the smallest model in the newsletter's coverage to be deployed on a physical robot in production conditions. At 230M parameters with day-one llama.cpp GGUF support, the model runs entirely on-device on the robot's embedded processor, selecting which motor skills to execute based on natural language commands without any cloud API call. The deployment surface is a bipedal robot operating in an uncontrolled physical environment with real-time latency constraints. The limitation acknowledged in the product note is precise: LFM2.5-230M is not recommended for reasoning-heavy tasks like math or code generation. It is recommended as a skill-selection and data-extraction layer, exactly the role it fills on the robot.

When you run an AI model on your phone or laptop, the bottleneck is usually not how smart the model is, it is how fast your processor can do the calculations. Most AI models are designed to run on large GPU servers, then retrofitted for phones. The math operations they use, particularly the attention mechanism that lets the model look at all previous words simultaneously, gets very slow and memory-hungry on CPU hardware.

LFM2 was designed the other way around. Liquid AI started with a list of requirements: must run fast on a Samsung Galaxy CPU, must fit within the phone's memory constraints, and must use the exact deployment software that the app will use. They then tested hundreds of different model architectures against these requirements directly on the actual hardware, before deciding which to build.

The result was surprising. Many recent efficient architectures use specialized mathematical operations called state space models, which are designed to be faster than attention. But when measured on real phone hardware with real deployment code, these more complex operations were actually slower than a simpler approach: a short sliding-window convolution (which processes only the last few tokens at once) combined with a small amount of traditional attention (which handles the cases where the model needs to look further back).

This is similar to discovering that, for everyday commuting in a city, a lightweight bicycle beats a sports car, not because bicycles are generally faster, but because in city traffic, the bicycle's advantages (lane flexibility, no parking time, easier acceleration) outweigh the car's top speed.

The result is a model family that runs two to three times faster than comparable models on the same phone hardware, enabling AI assistants that respond immediately rather than after a noticeable delay.

LFM2 demonstrates a replicable methodology, not just a model family. The hardware-in-the-loop search process: define quality and hardware objectives, enumerate candidate architectures from a broad search space, compile and profile every candidate on target devices with deployment-ready runtimes, evaluate quality only on candidates that pass hardware constraints, and iterate.

This is analogous to hardware-aware neural architecture search (NAS), but applied to the language model design problem at a team that has both the hardware partnerships and the deployment infrastructure to make the loop tight enough to be informative.

The result, a minimal hybrid that outperforms more complex SSM-heavy architectures on edge CPU under realistic constraints, is likely to be reproduced by other teams running similar searches on similar hardware. The specific architecture may change at larger parameter scales or different hardware targets. The methodology of measuring on actual deployment hardware before evaluating quality is the transferable contribution.

At LFM2.5-230M on a humanoid robot, the endpoint of this trajectory becomes concrete: AI models that run entirely on the physical systems they control, without cloud latency or cloud cost, because the architecture was designed for the constraint from the start.

Change a single line of code and unlock 1,000+ AI models, smart routing for cost and performance, and persistent agent memory.

Book a Demo