Aurum: The Local AI Stack You Can Build in an Afternoon

^{SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | April 27, 2026}

Most AI application tutorials converge on the same architecture: call OpenAI's API, get an answer, display it. Aurum, built by Adam Chan, takes the opposite approach: three Ollama models, a PostgreSQL database with the pgvector extension, a faster-whisper Python microservice, a Next.js 16 frontend, and zero external API calls. Everything runs locally. Your data never leaves the machine. And the technical choices made to get there, which embedding model, which LLM, how to do multimodal image recognition, how to implement semantic search, how to build a voice input pipeline without a cloud transcription service, are exactly the choices you would make building any serious local AI application.

This newsletter dissects Aurum as a systems document: what the three-model Ollama stack does, how pgvector with HNSW indexing enables semantic search across a home inventory, how faster-whisper runs local speech-to-text on port 9000, and what the ReAct pattern from arXiv:2210.03629 explains about the LLM's natural language parsing loop.

Scope: Aurum's complete technical stack, the three Ollama models (embeddinggemma:300m, llama3.2:3b, qwen3-vl), pgvector's cosine similarity search, the faster-whisper transcription microservice, and the Prisma ORM schema. Not covered: fine-tuning any of the underlying models, or production scaling beyond single-machine local deployment.

Aurum is a Next.js 16 web application for tracking home inventory using natural language, voice, and photos. Five stars on GitHub, MIT license, built and released publicly by Adam Chan. The repo description: "AI-powered home inventory management. Track, organize, and find your belongings using natural language, voice, or photos. 100% local with Ollama, your data never leaves home."

The use case is real: "Where did I put the extra toothpaste?" "Do I have any batteries?" "Add 3 bottles of shampoo to the bathroom cabinet." These queries are trivially easy to answer if you have a well-maintained inventory database and a natural language interface on top of it. The hard part is making the interface work for how people actually speak about their belongings, fuzzily, conversationally, without remembering the exact keywords they used when adding an item.

Semantic search via vector embeddings solves this. The embedding model turns both the stored item description and the query into high-dimensional vectors. Similarity search finds items where the vectors are close, even when the words are different. "I need a battery" finds items tagged "AA batteries," "Duracell," or "two AA cells in the kitchen drawer."

The complete installed tech stack:

^{Focus on the three distinct Ollama model roles. embeddinggemma handles search. llama3.2 handles intent parsing and response generation. qwen3-vl handles vision. They run as separate Ollama model instances, never sharing context. The entire multimodal AI pipeline runs on port 11434.}

The voice input path deserves attention. The browser's MediaRecorder API captures audio from the microphone, sends it to the faster-whisper Python server on port 9000 as a WAV blob, which returns a text transcript. That transcript then flows identically to typed text input. The transcription runs locally with Whisper's small or medium model, not via OpenAI's Whisper API. This is a meaningful architectural choice: voice transcription of home inventory items almost always includes product names, brand names, and room-specific vocabulary (linen closet, medicine cabinet, pantry) that general cloud transcription handles inconsistently. Local Whisper runs as a controlled environment with consistent behavior.

The <=> operator is the entire search mechanism. Everything else, the Ollama embedding call, the Prisma raw query, the similarity threshold, exists to set up and filter that single pgvector operation.

The temperature=0 setting for the LLM parser is critical. Natural language to JSON parsing must be deterministic. Any temperature above 0 introduces random variation in field names, JSON structure, and intent classification that breaks downstream code. The vision model and the chat response model can use higher temperatures. The parser model cannot.

Scenario: Voice input of a bathroom inventory, then semantic search for a specific item.

Input: User clicks "Start Recording" and says: "I have two toilet paper rolls, one comb, fifteen razor blade replacements, and two boxes of QTips."

Step 1: Voice transcription (faster-whisper, port 9000)

Step 2: LLM parsing (llama3.2:3b, temperature=0)

LLM parsing latency: ~800ms on llama3.2:3b (Apple Silicon Metal GPU acceleration via Ollama)

Step 3: Embedding generation (embeddinggemma:300m, per item)

Step 4: PostgreSQL upsert with pgvector

Step 5: Later query, "Do I have anything to cut my face hair?"

Total latency (voice input → stored + confirmed): ~4-5 seconds Total latency (text query → answer): ~1.2 seconds All on local hardware, no network calls to external APIs

The three-model separation is the correct design for this problem. Using one large multimodal model for embeddings, parsing, and vision would be worse on all three dimensions: larger models are slower for embedding generation (where speed matters more than quality), they are overkill for simple JSON parsing tasks, and mixing embedding and generation in the same model creates architectural coupling that makes model swaps impossible. Aurum's explicit model-per-role design means you can swap llama3.2:3b for gemma3:4b for the parser without touching the embedding pipeline.

The choice of embeddinggemma:300m as the embedding model reflects the correct tradeoff for this scale. EmbeddingGemma is a 300M parameter model from Google, built on the Gemma 3 architecture with T5Gemma initialization, trained on 100+ languages. It runs in less than 200MB of RAM, generates 768-dimensional embeddings, and at this scale (hundreds of home inventory items) is more than sufficient. The MTEB leaderboard shows EmbeddingGemma competitive with models twice its size. Semantic search quality over short item descriptions does not require billion-parameter embedding models.

The pgvector HNSW index is the correct choice over IVFFlat for this use case. At household inventory scale (hundreds to low thousands of items), HNSW (Hierarchical Navigable Small World graph) provides approximate nearest neighbor search with better recall than IVFFlat and no training requirement. IVFFlat requires a training step on representative data before building the index. HNSW builds incrementally as items are inserted.

What Aurum trades away:

Multi-user concurrent access. The Ollama models running locally serve one request at a time (unless OLLAMA_NUM_PARALLEL is set). For a single-user household app, this is fine. For a family of four simultaneously adding items, query latency will degrade proportionally.

Cross-device sync. Data lives in a local PostgreSQL instance. Accessing the inventory from a phone requires either running Aurum on a home server with local network access, or exporting/importing the database. There is no sync mechanism.

Voice accuracy on domain-specific terms. Faster-whisper's small model handles common English well but struggles with unusual product names, foreign words, or strong accents. The medium model improves this at the cost of ~3x more transcription latency.

The full local privacy guarantee. This is the actual differentiator relative to cloud-based inventory apps. Photos of your home interior, product inventories, medication lists, and sensitive items never leave the machine. For anyone with legitimate privacy concerns about where their home data goes, a cloud-connected inventory app is not acceptable regardless of its features. Aurum's 100% local architecture is not a limitation. It is the primary product attribute.

The multimodal pipeline for zero-configuration item entry. The combination of voice → faster-whisper → LLM parser and photo → qwen3-vl → LLM parser → embedding → database, all on local hardware, is a reference implementation for any local multimodal application. Any team building a similar system (local meeting notes, private document management, physical asset tracking) can replicate this architecture with the same stack.

EmbeddingGemma's performance-per-compute ratio. At 300M parameters running in under 200MB RAM, it outperforms older embedding models many times its size on semantic similarity tasks. The MTEB benchmark shows it competitive with models at 500M-700M parameters. For local deployment where model footprint is a constraint, this is the correct embedding model choice at this scale.

Insight One: Aurum is not a home inventory app. It is a reference implementation for building any privacy-first local RAG application, and the "home inventory" framing is limiting the community's recognition of what it demonstrates.

Every design decision in Aurum generalizes: three-model Ollama stack (embedding, generation, vision), pgvector semantic search, faster-whisper local transcription, Prisma ORM for type-safe database access, Next.js App Router for the UI layer. Substitute "home inventory items" with "private documents," "medical records," "legal notes," or "personal journals" and the architecture is identical. The specific domain is trivial. The pattern, a fully local, privacy-preserving, multimodal RAG application with sub-second query response over hundreds to thousands of items, is not trivial and is underdemonstrated in open source.

Insight Two: The ReAct pattern from arXiv:2210.03629 is operating implicitly in Aurum's intent parser, and the community building local AI apps routinely implements it without recognizing it.

ReAct (Reasoning and Acting, Yao et al., ICLR 2023) demonstrated that LLMs perform better on interactive tasks when they interleave reasoning traces with action steps: think, then act, observe the result, think again. Aurum's LLM parser implements a simplified version: parse the user's intent (reasoning), classify it as ADD/FIND/UPDATE/LIST (acting), return structured JSON (output). The zero-temperature setting ensures the reasoning step is deterministic rather than creative. The structured output constraint ensures the action step is machine-executable. What differs from vanilla ReAct is the absence of an observation loop: Aurum's parser is single-step. A more capable version would implement the full loop: parse intent, query the database, observe current state, then decide whether to add, update, or inform. This would handle "Add another bottle of shampoo to the bathroom" correctly when the item already exists, rather than creating a duplicate.

pgvector's cosine similarity search over 768-dimensional EmbeddingGemma vectors finds semantically related inventory items, including items described with completely different words, in under 5 milliseconds even without an HNSW index, across a typical household inventory of a few hundred items.

At this scale, the index is not the performance bottleneck. The Ollama embedding call is (~25ms). The LLM parsing call is (~800ms). The PostgreSQL query with a sequential scan is ~2ms. Building the HNSW index does not meaningfully improve query latency until the inventory reaches tens of thousands of items. For a single household, the correct optimization target is LLM parsing latency (use a smaller, faster model), not database index optimization. The community default assumption, "we need a vector database for AI search," is wrong at this scale. PostgreSQL with pgvector is sufficient for anything under a few hundred thousand items.

Aurum is a proof that the "call OpenAI's API" tutorial pattern is not the only option, and arguably not the right option for applications where the data itself is sensitive. Three Ollama models, one PostgreSQL instance, one Python microservice, and a Next.js frontend. Every component is open-source, self-contained, and runs on consumer hardware. The query response time (1.2 seconds for a text query) is competitive with cloud-connected alternatives while keeping every query, every photo, and every item description on your own machine. The pattern scales: swap the home inventory schema for any structured private data, and the architecture serves as a production-ready local RAG system. The tutorial that finally explains how to build this correctly, not with API calls but with a complete local stack, is what Aurum is, wrapped in the most relatable possible domain.

When analytics on Postgres slows down, most teams add a second database. TimescaleDB by Tiger Data takes a different approach: extend Postgres with columnar storage and time-series primitives to run analytics on live data, no split architecture, no pipeline lag, no new query language to learn. Start building for free. No credit card required.

Start building for free