In partnership with

The proof of this architecture's value is not a synthetic benchmark. It is DEF CON CTF 2026, the hardest open hacking competition in the world, where SageCTF, built on OpenSage, competed as a solo player against 686 teams, recovered 8 flags across 7 challenges, placed in the top 5%, and outperformed every team that self-reported as using No AI or Low AI. The benchmark comparison on 50 elite challenges: SageCTF solved 39. Claude Code solved 13.

SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | July 11, 2026

Every serious LLM agent framework in production today requires a developer to define the agent topology before execution: which agents exist, what each one does, which sub-tasks go to which agent, and how results are aggregated. LangGraph requires you to define a state graph. AutoGen requires you to define a conversation pattern. Google ADK, OpenAI Agents SDK, Anthropic's ADK all require pre-specified agent structures. When the task is novel, the developer-defined topology either fails to cover the needed capabilities or wastes compute on irrelevant sub-agents.

OpenSage's thesis is that this is the wrong layer for human input. The LLM should decide what agents it needs. It should create them at runtime based on the specific task, decompose the task into sub-problems that deserve independent context, and terminate agents when their work is done. Human design should happen at the level of what the root agent's goal is, not how it should be architectured to achieve it.

Scope: OpenSage's three-component architecture (AI-created topology, AI-written sandboxed tools, hierarchical graph-based memory), the SageCTF application at DEF CON CTF 2026 with real challenge outcomes, the multi-model orchestration pattern, and two case studies showing where SageCTF succeeds and where it fails. Not covered: the RL training pipeline in the rl/ directory, or the CTFJudge/CTFTiny benchmark methodology from the AAAI 2026 paper.

What It Actually Does

OpenSage is an agent development kit with three capabilities that existing ADKs do not have simultaneously: runtime sub-agent creation (not pre-specified), sandboxed runtime tool creation (not pre-loaded), and hierarchical graph-based memory (not flat conversation history).

Comparison with existing ADKs:

ADK

AI-created topology

AI-written tools (open)

Graph-based memory

LangGraph

No

No

No

AutoGen

No

No

No

Google ADK

No

No

No

OpenAI Agents SDK

No

No

No

Anthropic ADK

No

Claimed, not open-sourced

No

OpenSage

Yes, runtime

Yes, sandboxed

Yes, hierarchical

Quick start:

git clone https://github.com/opensage-agent/opensage-adk.git
cd opensage-adk
uv venv --python 3.12
uv sync

# Run the web UI against your agent
uv run opensage web --agent /path/to/my_agent --port 8000

The minimal agent factory function:

from opensage.agents import OpenSageAgent

def mk_agent(session_id: str, model=None):
    session = opensage.get_opensage_session(session_id)
    return OpenSageAgent(
        name="my_agent",
        description="My custom OpenSage agent.",
        model=model,
        instruction="You are a helpful assistant.",
        enabled_skills="all",  # ← enables all built-in skills
        tools=[],              # pre-loaded tools (optional)
        subagents=[],          # pre-specified sub-agents (optional, but can be empty)
                               # ← OpenSage will create sub-agents as needed at runtime
    )

The Architecture, Unpacked

The critical distinction from every other ADK: the sub-agent pool is empty at the start of any task. The root agent reads the task, decides what specialists it needs, creates them dynamically, and terminates them when done. This is not a pre-configured agent team; it is an AI-generated organizational chart for each specific task.

The Code, Annotated

Snippet One: Runtime Sub-Agent Creation Pattern

# OpenSage: runtime sub-agent creation from within a parent agent
# Source: opensage-agent/opensage-adk src/opensage/ + agent_library/
# Design intent: topology emerges from the task, not from developer pre-specification

import opensage
from opensage.agents import OpenSageAgent
from google.adk.models.lite_llm import LiteLlm

def mk_ctf_agent(session_id: str, model=None):
    """
    Root CTF agent. Starts with an empty sub-agent list.
    At runtime, it will create specialized sub-agents based on what the
    specific challenge requires — not based on a pre-defined workflow.
    """
    session = opensage.get_opensage_session(session_id)

    if model is None:
        model = LiteLlm(
            model="gpt/gpt-5.5",  # Root model: GPT-5.5 for fewer safety false positives on CTF
            api_key=os.environ.get("OPENAI_API_KEY"),
        )

    return OpenSageAgent(
        name="ctf_root_agent",
        description="Root agent for CTF challenge solving.",
        model=model,
        instruction="""
        You are a CTF solving agent. For each challenge:
        1. Analyze the challenge artifacts and identify required expertise.
        2. CREATE specialized sub-agents for each investigation branch.
           Example sub-agents to consider (create only what's needed):
           - static_auditor: binary analysis, disassembly, decompilation
           - dynamic_explorer: GDB, runtime behavior, heap analysis
           - web_search_agent: external context, CVEs, technique lookup
           - exploit_builder: construct and test working exploits
        3. Use TARGETED MESSAGES to coordinate specific findings between agents.
        4. Use BROADCAST for validated primitives all agents need to know.
        5. Integrate findings into a working exploit chain.
        """,
        enabled_skills="all",
        tools=[],
        subagents=[],   # ← Empty. Sub-agents will be created at runtime.
        # ← THIS is the key architectural choice: no pre-specification.
        # When the root agent sees that the challenge requires binary analysis
        # AND web search AND exploit construction, it creates the right agents
        # for THIS specific challenge — not a one-size-fits-all team.
    )

# ─────────────────────────────────────────────────────────────────────────────
# HOW RUNTIME CREATION WORKS (conceptual — based on OpenSage's sub-agent API):

# During task execution, the root agent calls something like:
#
# new_agent = parent_agent.create_subagent(
#     name="static_auditor",
#     description="Perform static binary analysis with Ghidra and angr.",
#     model=LiteLlm(model="anthropic/claude-opus-4-6"),  # ← different model per sub-agent
#     tools=["ghidra_mcp", "angr", "objdump"],
#     instruction="Analyze the binary. Report: architecture, interesting functions, "
#                 "potential vulnerability classes. Do not stop until you have a "
#                 "concrete hypothesis about the attack surface.",
#     # ← Sub-agent gets its own context, own tools, own model
#     # ← Parent agent does NOT need to know about binary analysis internals
# )
#
# result = parent_agent.send_message(
#     to=new_agent,
#     message="Here is the binary. Your peer dynamic_explorer has found: "
#             "stale-vector heap aliasing at offset 0x1234.",
#     # ← Targeted message: only this sub-agent needs this specific info
# )
#
# parent_agent.broadcast(
#     message="VALIDATED: u -> fv alias confirmed in GDB. "
#             "All agents: exploit path via fake closure, not heap spray.",
#     # ← Broadcast: every agent needs this to update their strategy
# )
#
# parent_agent.terminate_subagent(new_agent)
# ← After sub-agent delivers its findings, it's terminated
#   Its state is preserved in memory — not discarded

The subagents=[] declaration in mk_agent() is the architectural statement. Every other ADK requires you to list what agents exist. OpenSage requires you to state what the root goal is. The organizational structure of the agent team is the model's problem, not the developer's.

Snippet Two: Multi-Model Orchestration for Complementary Capabilities

# OpenSage: multi-model orchestration pattern used in SageCTF
# Design intent: different models have different capability strengths;
# routing work to the best model for each sub-task outperforms a single-model system

from google.adk.models.lite_llm import LiteLlm

# ─── MODEL SELECTION RATIONALE (from SageCTF blog, June 2026) ─────────────────

# Root model: GPT-5.5
# ← Fewer safety false positives on CTF artifacts (exploit code, shellcode, etc.)
# ← A safety false positive from the ROOT agent terminates the ENTIRE solve
# ← At root level, stability matters more than raw capability
root_model = LiteLlm(
    model="gpt/gpt-5.5",
    api_key=os.environ.get("OPENAI_API_KEY"),
)

# Hard reasoning model: Claude-Opus-4.6
# ← Strongest for: reverse engineering, exploit construction, complex reasoning
# ← Most valuable work in a CTF solve (the difficult parts)
# ← Used for: static_auditor, exploit_builder, final integration
# ← Across 7 solved challenges: carried the large share of high-value work
hard_reasoning_model = LiteLlm(
    model="anthropic/claude-opus-4-6",
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
    # High reasoning effort: enabled for complex reasoning traces
)

# Exploration model: DeepSeek-V4-Pro
# ← Much cheaper and faster than GPT-5.5 and Claude-Opus-4.6
# ← Used for: breadth-first exploration, dead-end elimination, web_search_agent
# ← A CTF solve requires testing many hypotheses; most will be wrong
# ← Cheap model for breadth + expensive model for depth = correct cost allocation
exploration_model = LiteLlm(
    model="deepseek/deepseek-v4-pro",
    api_key=os.environ.get("DEEPSEEK_API_KEY"),
)

# ← THIS is the key insight: model selection is task-specific, not fixed
# Most agent systems use one model throughout. SageCTF routes:
#   - Safety-critical root decisions → GPT-5.5 (stability)
#   - Deep reasoning and exploit construction → Claude-Opus-4.6 (capability)
#   - Broad exploration and web search → DeepSeek-V4-Pro (cost efficiency)

# Scale during DEF CON CTF 2026 (7 solved challenges):
# Total model calls: 23,919
# Total tokens processed: 2,400,000,000 (2.4 billion)
# Average per solved challenge: ~3,417 calls, ~343M tokens
# Average solve time per challenge: 5 hours
# Longest solve: "My Favorite Instructions" at 12 hours

def assign_model_to_subagent(subagent_type: str):
    """
    Route each sub-agent type to the most cost-effective capable model.
    """
    if subagent_type in ["static_auditor", "exploit_builder", "recon_solver"]:
        return hard_reasoning_model   # ← Pay for capability where it matters
    elif subagent_type in ["web_search_agent", "brainstorm_agent", "dead_end_checker"]:
        return exploration_model      # ← Pay for speed and breadth
    else:
        return root_model             # ← Default: stability

The model routing strategy is the cost management insight most implementations miss. 23,919 model calls across 7 CTF challenges is a large number. Running all of them on Claude-Opus-4.6 would cost significantly more and provide no benefit for exploration tasks where DeepSeek-V4-Pro's breadth outperforms depth. The routing decision is: use the expensive model for the bottleneck steps, not for all steps.

It In Action: SageCTF Solves mapllvm at DEF CON CTF 2026

Challenge: mapllvm at DEF CON CTF 2026. The program receives a restricted one-line Racket source file and submits it to a custom Racket-to-LLVM compiler service. Players must exploit compiler/runtime behavior to make the generated ELF read /flag without direct file-read primitives. Category: reverse engineering / pwn. Solve rate at competition end: single-digit percentage.

Step 1: Root agent analysis and topology creation

Input: challenge binary + description
Root agent (GPT-5.5) analysis:
  This requires:
  - External technique research (Racket compiler edge cases, LLVM behavior)
  - Static binary analysis (identify what the compiler generates)
  - Dynamic runtime analysis (confirm exploit primitives in GDB)
  - Exploit construction (turn primitive into /flag read)
  - Integration (run remote exploit, extract flag)

Agents created at runtime:
  [web_search_agent]    → model: DeepSeek-V4-Pro (breadth, fast)
  [brainstorm_agent]    → model: DeepSeek-V4-Pro (hypothesis generation)
  [static_auditor]      → model: Claude-Opus-4.6 (binary analysis)
  [dynamic_explorer]    → model: Claude-Opus-4.6 (GDB, runtime)
  [exploit_builder]     → model: Claude-Opus-4.6 (exploit construction)
  [recon_solver]        → model: Claude-Opus-4.6 (final integration)

Step 2: Parallel exploration (both branches run simultaneously)

web_search_agent + brainstorm_agent → TEST THEN ELIMINATE:
  Hypothesis: Racket #reader / #lang path → dead end (eliminated)
  Hypothesis: Custom compiler stale-vector heap aliasing → PROMISING

Message bus broadcast:
  "Eliminated: #reader path does not work. Promising: stale-vector alias in Racket compiler."
  → All agents receive this, update their investigation direction

Step 3: Static-dynamic coordination loop

dynamic_explorer (GDB analysis):
  Validates stale-vector heap aliasing primitive at runtime
  Confirms: u -> fv alias exists and is exploitable
  
TARGETED MESSAGE → static_auditor:
  "VALIDATED: stale-vector alias at runtime. u -> fv confirmed.
   Need: how to turn this into fake closure in generated LLVM IR."

static_auditor (binary analysis):
  Turns primitive into fake-closure exploit source
  Identifies: control-flow target at 0x401048
  
BROADCAST → all agents:
  "KEY FINDING: fake closure via u -> fv alias. Jump target: 0x401048.
   Exploit path: shellcode via executable immediate bytes."

Step 4: Exploit construction and flag recovery

exploit_builder (Claude-Opus-4.6):
  Uses stale vector alias to control fake closure fields
  Confirms control-flow target in GDB
  Corrects jump target to 0x401048
  Constructs shellcode: open + read + write /flag via executable immediate bytes

recon_solver (Claude-Opus-4.6):
  Integrates all findings
  Runs final remote exploit against competition server
  Recovers flag: FLAG{...}

Result: SOLVE
Time spent: ~3-4 hours of autonomous execution
Model calls for this challenge: ~3,000-4,000
Human intervention: 0

What the solve illustrates: the message bus broadcasting "validated exploit primitive, all agents" is what prevents duplicated effort across parallel branches. Without targeted messages, every agent would independently validate the same primitive. With them, the dynamic_explorer validates once and all other agents immediately update their strategies. At 23,919 calls across 7 challenges, this coordination efficiency is the difference between a 5-hour solve and a solve that never converges.

Why This Design Works, and What It Trades Away

The runtime topology creation is the decisive architectural advantage for long-horizon tasks with unknown structure. A CTF challenge does not have a known agent topology at the start. You do not know whether you will need a heap analysis specialist, a Racket compiler expert, or an LLVM optimizer expert until you have read the challenge description and started exploring. Pre-specifying a generic "CTF team" wastes compute on specialists the challenge does not need and misses specialists it does.

The sub-agent state preservation is the second critical property. Every other ADK discards sub-agent state after the sub-agent completes its task. In OpenSage, the state persists in the hierarchical memory. When dynamic_explorer validates the heap aliasing primitive and is terminated, that finding remains in the memory graph. When exploit_builder is created later, it reads the memory and starts from the validated primitive rather than re-discovering it.

The multi-model orchestration reflects an empirical fact about model capabilities that single-model systems cannot leverage. GPT-5.5 has fewer safety false positives on exploit code. Claude-Opus-4.6 is stronger for hard reasoning and reverse engineering. DeepSeek-V4-Pro is dramatically cheaper for exploration. These are different tools. Routing work to the right tool is correct engineering.

What OpenSage trades away:

The benchmark numbers are impressive but the evaluation conditions deserve scrutiny. The 10-hour time limit and $200 per-challenge budget are significantly larger than what most teams operate with. The paper's three SOTA benchmark comparison is not yet independently reproduced. The 39/13 SageCTF vs Claude Code result is self-reported by the OpenSage team.

The DEF CON CTF result has an important caveat: SageCTF solved challenges but did NOT submit flags, because DEF CON policy prohibits fully automated bots. The 1,743-point ranking is computed from the challenges solved without submission. The team explicitly acknowledges this limitation and the ethical choice to not interfere with the live competition.

The Pixels and Nicotine close miss is the most honest self-assessment in the blog. SageCTF correctly identified the NES binary structure, found the encoded bytes, and reversed the score routine, but read the decoded bytes in memory order while the NES renderer wrote tiles in reverse order. The agent had all the right pieces and assembled them in the wrong sequence due to missing niche domain knowledge. External RAG would fix this specific failure. Dynamic judging feedback during the solve would also help.

Technical Moats

Runtime sub-agent creation with preserved state. Every existing production ADK pre-specifies the agent structure. The engineering complexity of implementing runtime creation correctly, including context isolation, state preservation after termination, recursive sub-agent support (sub-agents creating their own sub-agents), and a message bus that routes both targeted and broadcast messages, is significant. The sandboxing system for Docker-isolated tool execution adds another layer. This is not a feature that can be added to an existing ADK without architectural changes.

Multi-model orchestration at scale. Running 23,919 model calls across three different model families in a coordinated 5-hour autonomous solve requires infrastructure for dynamic model assignment, cost tracking, and safety-level routing. Most agent systems hardcode a single model. OpenSage's design enables model routing as a first-class property because sub-agents are created dynamically and each creation can specify the model.

The CTF domain as a benchmark. CTF challenges at DEF CON level are arguably the hardest measurable long-horizon tasks for AI agents currently available. They require sustained reasoning over hours, debugging of failed hypotheses, use of domain-specific tools, and convergence on a specific exploit chain. A top-5% result at DEF CON CTF against human expert teams is a credible upper bound on what autonomous agents can achieve on complex technical tasks today. The 4.4% solve rate for gitvfs (only 30 teams solved it) is the specific difficulty signal that matters. SageCTF solved a challenge that 96% of human expert teams did not.

Insights

Insight One: The comparison between SageCTF and Claude Code is structurally unfair to Claude Code in ways that are worth stating explicitly. Claude Code is a general-purpose coding assistant not specifically optimized for CTF. SageCTF is a CTF-specialized agent that evolved through months of CTF competition practice, each competition adding missing capabilities. The architecture comparison (SageCTF: 39/50, Claude Code: 13/50) measures CTF specialization as much as it measures architectural advantage. The paper would be stronger with a comparison against other CTF-specialized agents (like those studied in the AAAI 2026 CTFJudge paper) rather than a general-purpose coding tool. The architectural advantages of OpenSage (runtime topology, tool creation, hierarchical memory) are real and supported by the ablation structure, but the specific 39 vs 13 number conflates architecture with specialization.

Insight Two: The model routing decision to use GPT-5.5 as the root agent over Claude-Opus-4.6, despite Claude-Opus-4.6 being the stronger reasoning model, reveals the failure mode of capability-first model selection. Safety alignment false positives on CTF artifacts (shellcode, exploit code, buffer overflow patterns) cause the root agent to refuse to proceed, terminating the entire solve. The solution is not to jailbreak the model. It is to route safety-sensitive root decisions to a model with better-calibrated safety responses for security research contexts. This is a general pattern: for agentic tasks involving security research, binary analysis, or exploit development, root-agent safety calibration matters as much as raw capability. GPT-5.5 as the root with Claude-Opus-4.6 for the actual hard work is a deliberate and well-reasoned engineering choice, not a capability ranking.

Surprising Takeaway

SageCTF's gitvfs solve is a two-stage challenge where the first flag arrived after 3.5 hours of autonomous exploration and the second flag (completing the challenge) took approximately 10 hours total, with the agent maintaining coherent investigation state across both stages without any human intervention. DEF CON CTF's hardest challenges are designed to require a team of 20+ world-class hackers working around the clock. SageCTF treated gitvfs as a solo investigation, ran for 10 hours, and recovered both flags. Only 30 of 686 scored teams achieved the same result. The scale of what "autonomous" means here is easy to understate: 10 hours of continuous investigation, thousands of model calls, debugging failed exploit paths, maintaining partial progress, and eventually converging on working exploits for both stages of a challenge that 4.4% of the best human teams in the world solved. This is not demo-quality autonomy. It is production-scale, long-horizon, unsupervised task completion on genuinely hard problems.

TL;DR For Engineers

  • OpenSage (opensage-agent/opensage-adk, Apache 2.0, ICML 2026, arXiv:2602.16891): first ADK where the LLM creates its own sub-agents, writes its own sandboxed tools, and manages hierarchical graph-based memory at runtime. No pre-specified topology. Builds on Google ADK (LiteLlm backend). Python 3.12+, Docker for sandboxing.

  • Three differentiators vs every existing ADK: (1) AI-created topology (parent creates sub-agents at runtime for specific task, not from a list); (2) AI-written tools with dependency awareness (sandboxed Docker execution, state managed); (3) graph-based hierarchical memory (long-term cross-task + short-term per-task, sub-agent state preserved after termination, not discarded).

  • SageCTF (built on OpenSage): DEF CON CTF 2026, solo player, top 5% of 686 teams, 8 flags across 7 challenges, 1,743 points. Outperformed all No-AI and Low-AI teams (175 teams). Average 5 hours per solve, 23,919 model calls, 2.4B tokens. Models: GPT-5.5 (root, stability), Claude-Opus-4.6 (hard reasoning), DeepSeek-V4-Pro (exploration).

  • Benchmark: 50 elite challenges from TAMU/UMass/RITSEC CTF + NYU CTF Benchmark. SageCTF: 39/50 solved. Claude Code: 13/50 solved. Every Claude Code solve was also a SageCTF solve. Zero Claude-Code-only solves.

  • Caveats: flags solved but not submitted at DEF CON (policy compliance). 10h/$200 per challenge budget is generous. 39 vs 13 comparison conflates architecture with CTF specialization. Pixels and Nicotine miss shows external RAG gaps in niche domain knowledge.

The Agent Team Builds Itself

OpenSage's core claim is that the topology of an agent system should be determined by the task, not by the developer. The SageCTF result is the existence proof: when the task is unknown in advance, when the required specialists cannot be predicted before execution, and when the investigation must run for 10 hours without human intervention, a system that creates its own team composition for each specific challenge outperforms a system that uses a fixed structure.

The 39/13 result is the headline. The gitvfs 10-hour two-stage solve is the substance. DEF CON CTF is a domain where top 5% means you outperformed the majority of the world's best security professionals operating in teams. OpenSage achieved this as a solo autonomous agent.

The gap between where this is now and where it needs to be, the best human teams are still stronger, is still significant. But the direction of travel is clear.

References

OpenSage (opensage-agent/opensage-adk, Apache 2.0, ICML 2026, arXiv:2602.16891, UC Santa Barbara + UC Berkeley) is the first agent development kit where the LLM creates its own sub-agent topology at runtime rather than following a developer-specified structure, writes its own sandboxed tools with full dependency awareness, and maintains hierarchical graph-based memory that persists sub-agent findings after termination. Applied as SageCTF to DEF CON CTF 2026, the world's hardest open hacking competition, it competed as a solo player against 686 teams, recovered 8 flags across 7 challenges for 1,743 points (top 5%), outscored all No-AI and Low-AI teams, and averaged 5 autonomous hours per solve across 23,919 model calls and 2.4 billion tokens, using GPT-5.5 as the root model (safety stability), Claude-Opus-4.6 for hard reasoning and exploit construction, and DeepSeek-V4-Pro for breadth exploration. On a 50-challenge benchmark from TAMU/UMass/RITSEC CTF and the NYU CTF Benchmark, SageCTF solved 39 challenges while Claude Code (same hardware, same budget) solved 13, with zero Claude-Code-only solves.

Sponsored Ad

If you enjoy practical AI insights, check out SnackOnAI and support the newsletter by subscribing, sharing, and exploring our sponsored ad — it helps us keep building and delivering value 🚀

Your prompts are leaving out 80% of what you're thinking.

When you type a prompt, you summarize. When you speak one, you explain. Wispr Flow captures your full reasoning — constraints, edge cases, examples, tone — and turns it into clean, structured text you paste into ChatGPT, Claude, or any AI tool. The difference shows up immediately. More context in, fewer follow-ups out.

89% of messages sent with zero edits. Used by teams at OpenAI, Vercel, and Clay. Try Wispr Flow free — works on Mac, Windows, and iPhone.

Start flowing free

Recommended for you

© 2026 Snack On AI.
beehiivPowered by beehiiv