SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | August 7, 2026
This issue dissects anydoc, the Rust document-to-Markdown engine Firecrawl open sourced on August 4, 2026, currently at 5.2k stars, 225 forks, and version 0.1.6 under MIT. Covered: the shared document model, content-based format detection, the context-sensitive Markdown escaper, the PDF bypass, and benchmark methodology. Also covered: numbers from running the library locally rather than repeating the README.
Excluded: OCR pipelines, vision-language document models, and the hosted Firecrawl Parse API beyond how it shapes anydoc's architecture. Excluded: any claim about anydoc's quality on scanned documents, because anydoc does not process them at all, and that turns out to be the most interesting fact in this issue.
What It Actually Does
anydoc converts fourteen office document formats into GitHub-Flavored Markdown. Word (.doc, .docx, .docm), PowerPoint (.ppt, .pptx, and four more container variants), Excel (.xls, .xlsx, .xlsm, .xlsb), OpenDocument (.odt, .ods, .odp), RTF, EPUB, CSV, and text-based PDF.
The marketing line is "single-digit milliseconds." The published benchmark says 4.4ms median across 100 real-world documents, against 134.8ms for markitdown, 513.6ms for Docling, and 1129.5ms for the LibreOffice pipeline. Firecrawl's own Parse landing page quotes 4.7ms, so treat the exact digit as noisy.
Here is the part the headline buries. The dependency list is nine crates:
[dependencies]
calamine = "0.36.1" # Excel: xlsx, xlsm, xlsb, and binary xls
cfb = "0.14.0" # OLE compound files: the 1997 binary Office container
csv = "1.4.0"
flate2 = "1"
encoding_rs = "0.8.35" # legacy codepages, because .doc from 2003 is not UTF-8
log = "0.4"
pdf-inspector = "0.1.7" # ← THIS is the one that breaks the architecture. See below.
quick-xml = "0.41.0"
zip = { version = "8.6.0", default-features = false, features = ["deflate"] }
Caption: No torch, no onnxruntime, no model weights. The entire "AI document parser" contains zero AI.
That is the actual product decision. anydoc does not understand documents. It reads the structure documents already declare about themselves, and refuses the ones that do not.
The Architecture, Unpacked
Every format funnels through one model and one serializer. Except one.

Caption: Focus on the right-hand branch. PDF never touches the shared model, which means the "one consistent output for every format" guarantee covers thirteen of fourteen formats and skips the hardest one.
The left branch is the good design. Eleven parsers, one model, one serializer means a table-escaping bug fixed for docx is fixed for rtf, odt, epub, and pptx in the same commit. That is the whole argument for the shape, and it holds.
The right branch is a delegation. pdf-inspector emits Markdown on its own terms.
The Code, Annotated
The one branch that tells you what anydoc really is
From src/lib.rs:
pub fn to_markdown_bytes(
bytes: &[u8],
format: impl Into<Option<Format>>,
) -> Result<String, ConvertError> {
let format = resolve_format(bytes, format.into())?;
// PDFs convert to Markdown directly (pdf-inspector) without passing
// through the document model.
if format == Format::Pdf {
return formats::pdf::to_markdown(bytes); // ← THIS is the trick, and the tax
}
Ok(document_to_markdown(&to_document(bytes, format)?))
}
Caption: One early return. Everything downstream of it, the shared model, the invariant-checked table grid, the context-sensitive escaper, does not apply to PDFs.
The crate documents the consequence honestly in the Format::Pdf doc comment: to_document is unsupported for PDFs. If you were planning to pull embedded assets or walk the block tree for a PDF, you cannot. You get a string.
Escaping that reads the room
Most converters escape Markdown syntax the same way everywhere: see a special character, add a backslash. anydoc tracks context. From src/render/markdown/escape.rs:
/// Where an inline run is being rendered; controls which characters
/// can be syntax there.
pub(crate) enum InlineContext { Block, Heading, TableCell }
pub(crate) fn escape_text(text: &str, ctx: InlineContext, opts: EscapeOpts) -> String {
// Last position of each pairable delimiter; a lone one is inert.
let mut last: [Option<usize>; 5] = [None; 5]; // * _ ~ ` ]
for (j, &c) in chars.iter().enumerate() {
match c { '*' => last[0] = Some(j), '_' => last[1] = Some(j), /* ... */ _ => {} }
}
// ← THIS is the trick: a delimiter is only escaped when a LATER twin
// exists that could pair with it into real emphasis.
let paired = |slot: usize| trailing_active || last[slot].is_some_and(|j| j > i);
Caption: Two passes over the run. The first records where each pairable delimiter last appears; the second escapes only the ones that could actually close.
Why an engineer should care: naive escapers backslash-pollute identifiers. node_01_gpu becomes node\_01\_gpu, --enable-fast survives but snake_case_names do not, and every one of those backslashes lands in your embeddings and breaks exact-match retrieval over code and config values. Here is anydoc's real output from my own run, unedited:
Node label is node_01_gpu and the flag is --enable\*fast* mode.
Pipe in prose: a|b, and a bare bracket ] plus a lone asterisk *.
| | | |
| --- | --- | --- |
| Compute \| Storage | | Cost |
Caption: Same pipe character, two fates. Raw in prose, escaped inside the table cell. Underscores in the identifier survive untouched. The lone asterisk stays a lone asterisk because nothing pairs with it.
It In Action
Input: a .docx I generated with python-docx containing an H1, an H2, a bullet list, a numbered list, a three-column table with a merged header cell, and four sentences engineered to be hostile to a naive escaper. File size 37,066 bytes. Hardware: single vCPU Linux container, Node v22.22.2, @firecrawl/anydoc 0.1.6.
Step one, convert.
$ npx @firecrawl/anydoc report.docx
# Q3 Infrastructure Review
Cluster spend rose 18% QoQ. See the table below.
Node label is node_01_gpu and the flag is --enable\*fast* mode.
## Findings
- GPU utilization below target
- Spot reclaim events doubled
1. Migrate to RKE2
| | | |
| --- | --- | --- |
| Compute \| Storage | | Cost |
| gpu-a | s3 | $4,200 |
| gpu-b | efs | $1,850 |
Step two, look at the merged cell. The header cell spans two columns. In the model it is one Origin with colSpan: 2 followed by a Covered marker:
{"kind":"origin","cell":{"blocks":[...],"colSpan":2,"rowSpan":1}},
{"kind":"covered","originRow":0,"originCol":0}
GFM has no span syntax, so the serializer renders the covered slot as an empty cell. Column count stays honest. The header association is gone.
Step three, timings. 1,000 conversions through toMarkdownBytes, warm, file read excluded:
metric | value |
|---|---|
p50 | 6.53 ms |
p95 | 7.58 ms |
p99 | 31.70 ms |
max | 99.71 ms |
500 docs, serial | 3,319 ms |
Step four, mislabel it. Copy report.docx to totally_a_pdf.pdf:
$ npx @firecrawl/anydoc totally_a_pdf.pdf
# Q3 Infrastructure Review # exit 0, converts correctly as docx
$ npx @firecrawl/anydoc - < data.csv
anydoc: unsupported input: unrecognized file content: name the format explicitly
$ echo $?
1
Caption: The ZIP mimetype wins over the file extension. CSV, which has no signature, fails loudly rather than guessing.
Step five, hand it a scan. A one-page PDF containing only a rasterized image of text:
$ npx @firecrawl/anydoc scanned.pdf
anydoc: unsupported input: PDF has no extractable text (Scanned, 1 pages): OCR is required
$ echo $?
1
Caption: Fails in single-digit milliseconds with the page count and the reason. That is exactly the error you want if anydoc is the cheap first stage of a router.
Why This Design Works, And What It Trades Away
Why it works. Firecrawl found the boundary where structure is free. A .docx is a ZIP of XML that literally contains <w:tbl>, <w:numPr>, and style ids. A .doc from 2003 is an OLE compound file with sprm property modifiers and a style sheet. The structure is in the file. Every millisecond Docling spends running RT-DETR over a 72dpi page bitmap to discover "this rectangle is a table" is a millisecond spent rediscovering something the office format already stated in plain XML.
The resource-limit module is the other quiet win. Fixed, deliberately non-configurable caps: 128 MiB per archive entry, 512 MiB total, 100,000 entries, 256 XML depth, 2,000,000 XML nodes, 4,000,000 table expansion cells. Crossing one returns ConvertError::ResourceLimit. If you have ever had a zip bomb take down a document ingestion worker, you know why "not configurable" is the right answer.
What it trades away.
Merged cells lose their header link. Docling made the opposite call. The Docling technical report explicitly repeats a spanning header value across each column it covers, so that "every data point can be traced back to row and column headings only by its grid coordinates." anydoc renders a blank. For a table chunked into a RAG index, Docling's row is self-describing and anydoc's is not. anydoc keeps the grid honest; Docling keeps the semantics recoverable. Pick based on whether a human or a retriever reads the output.
The document model has a PDF-shaped hole. You cannot get assets, block structure, or the escaper's guarantees out of a PDF.
Scans are somebody else's problem. Which, commercially, is the point. Firecrawl Parse classifies every page first at roughly 20ms, routes text pages to this exact local path, and sends only scanned pages to a GPU. Their own page says about half of PDFs are text based. anydoc is the free half of a funnel, and the free half is genuinely excellent.
Tail latency through the Node binding is fifteen times the median. p50 6.53ms, p99 31.70ms on my run. The conversion is not the variance; allocation and GC across the N-API boundary are. If you are converting per request in a Node service, size your timeouts off p99, not off the README.
Technical Moats
Not the Rust. Rust is table stakes and Firecrawl would tell you so.
Spec archaeology. src/formats/doc/ contains sprm.rs and stsh.rs. Those are the binary Word 97 property-modifier and style-sheet structures. src/formats/ppt/ walks a legacy record stream with its own depth limit of 64 and a 16,000,000 record cap. Nobody writes that for fun. LibreOffice took two decades to accumulate the same knowledge, and unstructured, markitdown, and pandoc simply do not support these formats: markitdown covers six of fourteen, pandoc five, Docling four.
A test surface that assumes hostility. A committed fixture corpus under tests/fixtures/ with snapshot tests, tests/robustness.rs mutation-testing every fixture, and cargo-fuzz targets per format in fuzz/. Eight fuzz targets: csv, doc, docx, epub, odf, pdf, ppt, rtf, xlsx. Parsers that survive fuzzing are a compounding asset.
Distribution as a flywheel. One artifact ships as a crate, an npm package, a PyPI wheel, a WebAssembly module, a CLI over npx, and an Agent Skill installable with npx skills add firecrawl/anydoc. That last one is the real land grab. Every Claude Code, Cursor, and Codex session that hits a .docx now has a default answer, and the default answer is Firecrawl's.
Insights
Insight One: The hundred-x is parsing beating perception, not Rust beating Python
The community reads "4.4ms vs 513.6ms" as a language and engineering win. It is a workload comparison between two systems that were never doing the same job.
Docling runs an RT-DETR-derived layout detector trained on DocLayNet over a page bitmap, then feeds every detected table to TableFormer, a vision transformer. The paper's own numbers: 1.27 to 2.45 pages per second on an M3 Max, 2.42 to 6.20 GB resident memory, and two to six seconds per table on a standard CPU. anydoc's benchmark harness notes Docling pulls in roughly 2 GB of torch just to install.
anydoc reads XML that already says the word "table."
Rewriting Docling in Rust would not close a hundred-x gap. Deleting its models would, and then it would not be Docling. The honest framing is that these tools overlap on .docx and diverge completely on anything scanned, and the benchmark table only shows the overlap.
Insight Two: The benchmark discloses everything and reproduces nothing
Credit where due: Firecrawl published the harness, named the judge, and disclosed the position-bias correction. That is more transparency than most vendor benchmarks. Now read the fine print.
The corpus is not redistributable and is not in the repo. Ground truth is the first six pages rendered by LibreOffice, and LibreOffice is also a scored competitor in the same table, finishing last at 39. The judge is Claude Sonnet 5, the vendor ran the evaluation on its own product, and the timing rules differ by tool: anydoc reports its own internal conversion time while pandoc and LibreOffice include process spawn. That last choice is defended in the README as reflecting real usage, and it is defensible, but it is not like for like.
Nothing here suggests the results are wrong. The per-format table is plausible and matches what the architecture predicts. The point is narrower and harder: no third party can currently reproduce a single number in that table. For a project asking to sit in your ingestion hot path, the benchmark is a claim, not evidence.
Takeaway
The document path knows a heading is a heading because the file says so. The PDF path has to guess, and I watched it guess wrong.
I generated two PDFs with identical content and identical fonts. The first laid out four eleven-point body lines as short standalone drawString calls. The second laid out the same sentences as flowing wrapped text.
First PDF, output:
# Q3 Infrastructure Review
## Cluster spend rose 18% QoQ.
## GPU utilization below target.
Second PDF, output:
# Q3 Infrastructure Review
Cluster spend rose 18 percent quarter over quarter, driven mostly by sustained
GPU demand across the training fleet and a doubling of spot reclaim events in
us-east-1 during the second half of the quarter. GPU utilization stayed below
the 70 percent target for six weeks.
Same library, same generator, same font size for the body. Line-joining works well. Short isolated lines get promoted to H2 because on the PDF path there is no <w:pStyle> to read, only geometry, and short plus isolated looks like a heading.
That is the entire thesis of this issue in one diff. For thirteen formats anydoc reads declared structure and the output is deterministic. For the fourteenth it infers structure from pixels-adjacent signals and inherits every failure mode of the tools it is a hundred times faster than. If your corpus is PDF-heavy, benchmark on your own documents before you believe the table.
TL;DR For Engineers
Fourteen formats, one shared
Documentmodel, one GFM serializer, nine crates, zero ML models. p50 6.53ms on my single-vCPU container, 4.4ms on Firecrawl's Ryzen 9 9950X3D.PDF bypasses the shared model entirely.
to_documentreturnsUnsupportedfor PDFs, and heading inference on the PDF path can promote short standalone lines to H2.Scanned PDFs fail fast with a page count and an explicit "OCR is required." That makes anydoc an excellent first stage in a router, and a non-starter as a complete solution.
Merged table cells render as blanks because GFM has no span syntax. Docling repeats the spanning value instead. If you chunk tables for retrieval, that difference matters more than the hundred-x.
p99 through the Node binding is 31.70ms against a 6.53ms p50. Size timeouts off the tail.
Explain It Like I'm New
Every AI application that touches company documents runs into the same wall on day one. The interesting information lives in Word files, slide decks, spreadsheets, and PDFs, and a language model cannot read any of those directly. It needs plain text with some structure preserved: what was a heading, what was a table, what order things came in.
There are two ways to get that text out, and they are more different than they look.
The first way is to read what the file already says about itself. A modern Word document is really a compressed folder of structured markup that spells out "this is a heading," "this is a table row," "this list is numbered." If you know how to read that markup, extraction is close to free. That is the entire idea behind anydoc. It is a Rust library that speaks fourteen of these formats fluently and funnels all of them into one common shape, so a slide deck and a spreadsheet come out looking the same on the other side.
The second way is to look at the page like a person would, as an image, and use vision models to work out where the tables and headings are. That is what tools like IBM's Docling do, and it is the only option when a document is a scan, because a scan is a photograph with no structure in it at all.
anydoc is roughly a hundred times faster than the second approach, and the reason is not clever optimization. It is that reading is cheaper than looking. The tradeoff is absolute: when a file has no structure to read, anydoc does not slow down, it stops and tells you to use something else.
This distinction is going to matter more, not less. As agents get handed real company file shares, the winning architecture is almost certainly the cheap reader in front and the expensive model behind it, invoked only when the cheap reader gives up.
See It In Action
Try anydoc in your browser (firecrawl.github.io/anydoc, Firecrawl). The demo compiles the library to WebAssembly and runs it client side, so your files never leave the machine. Drop in your ugliest real
.docxand watch the conversion time. This is the fastest way to falsify the benchmark on your own corpus.The benchmark harness (bench/README.md, Firecrawl). Not a video, but the most instructive artifact in the repo: the competitor matrix, the pairwise LLM judging protocol, the position-swap bias correction, and the cost controls. Read it before quoting any number from the results table.
Docling technical report (arXiv:2408.09869, IBM Research). The pipeline sketch in Figure 1 and the runtime table in Section Four are the direct counterpoint to anydoc's architecture. Shows exactly which stages cost the seconds anydoc does not spend.
The Agent Skill (skills/convert-documents-to-markdown, Firecrawl). Forty lines that turn a CLI into a capability any coding agent can pick up. Worth reading as a template for shipping your own tools into agent runtimes.
Community Conversation
Nick Camara, Firecrawl co-founder (x.com/nickscamara_). Launch thread claiming 100x faster local parsing and 500 docx files converted in 1.7 seconds. Useful as the strongest version of the vendor claim, and as the number most worth reproducing yourself.
Skeptical reply on the launch thread (x.com/firecrawl). Asks the right question: on what hardware, on what documents. Notes that real-world PDFs carry embedded images, unusual encodings, and malformed XML, and that benchmark numbers stay clean until production. This issue's PDF heading test is one data point in that direction.
WebAssembly in Cloudflare Workers (Issue #6). A community member got the core compiling to
wasm32and running inside workerd by swapping the zip crate's zstd feature for pure-Rust deflate, converting docx to Markdown correctly at the edge. The zero-OS-dependency claim held up under an independent port, which is a stronger signal than any benchmark row.Independent write-ups (AI Engineering on Medium). Reproduces the benchmark table while flagging the two caveats that matter: the corpus is not public, and the text-layer-only PDF limitation is easy to miss.
Read The File, Do Not Look At The Page
anydoc is the best available answer to a question most teams have been asking wrong. The question is not "what is the best document parser." It is "which of my documents still carry their own structure, and which ones do I have to pay a model to reconstruct."
For the first pile, which on Firecrawl's own numbers is about half of PDFs and effectively all office files, anydoc converts in single-digit milliseconds with nine dependencies and no GPU, and every competing tool in that lane is now doing unnecessary work. For the second pile it returns an error in under ten milliseconds, which is the correct and honest answer.
Put the cheap reader in front. Route the failures. Stop paying a vision transformer to rediscover a <w:tbl> tag.
References
anydoc, GitHub repository, Firecrawl. Source, benchmark table, and format matrix.
Docling Technical Report, arXiv:2408.09869, Auer et al., IBM Research. Layout analysis, TableFormer, and the runtime numbers used for comparison here.
DocLayNet: A Large Human-Annotated Dataset for Document-Layout Segmentation, Pfitzmann et al. The dataset behind Docling's layout model.
TableFormer: Table Structure Understanding with Transformers, Nassar et al., CVPR 2022. The vision transformer anydoc's grid model replaces with a parser.
Optimized Table Tokenization for Table Structure Recognition, Lysak et al., ICDAR 2023. Why table structure is hard when you cannot read the markup.
anydoc benchmark harness, Firecrawl. Judging protocol and timing methodology.
Firecrawl Parse, Firecrawl. Page classification and OCR routing, the commercial layer above anydoc.
anydoc browser demo, Firecrawl. WebAssembly build.
anydoc is a Rust library that converts fourteen document formats to GitHub-Flavored Markdown in single-digit milliseconds by reading declared structure instead of inferring it, funneling eleven format parsers through one document model and one serializer. Its hundred-x speed advantage over model-based tools like Docling is a workload difference rather than an engineering one, and it disappears the moment a document has no structure left to read. The architectural tell is that PDFs bypass the shared model entirely, which is exactly where the quality guarantees stop holding.
Sponsored Ad: If you enjoy practical AI insights, check out SnackOnAI and support the newsletter by subscribing, sharing, and exploring our sponsored ad, it helps us keep building and delivering value 🚀
Own Search With Podcasts
Your competitors are fighting over the same keywords. The smartest brands are building the authority that search engines, AI platforms, and customers trust everywhere.
Every relevant podcast appearance can produce branded mentions, backlinks, transcripts, citations, clips, expert content, and third-party proof that keeps compounding across search and AI discovery.
PodPitch searches millions of podcasts, finds the shows that matter to your market, develops the angle, sends personalized pitches, and follows up automatically until your experts are booked.
Growth teams are already using podcast appearances to build distributed authority that cannot be manufactured by publishing another generic SEO article.
Only 20 SEO, AEO, and GEO demo spots are available this month. Once they’re claimed, the offer disappears.
Start building searchable authority now, before your competitors own the conversations shaping your market.


