A 50x speedup you cannot spend

Ask a knowledge bundle a question and okf search reads every concept to answer it. Every field of every file, every time, no matter how many times you asked something similar a minute ago. On a bundle of two dozen concepts you will never notice. On a few thousand, you will.

The obvious fix is an inverted index, and okf 1.9.0 ships one. It is 44 to 56 times faster per query and it is switched off. This post is the measurement that led there, the costs the headline number hides, and the one architectural change that would flip the default.

What the scan does every time

The default engine reads the text and looks for your term in it, weighting a hit in the title above a hit in the body. That is the whole of it.

There is no index, which means there is nothing to build, nothing to invalidate, and nothing to keep in sync with the files on disk. Every query pays full price and no query pays setup. That property turns out to be the whole argument, though it took a benchmark to see it.

A port on a footprint budget

minifts is a pure-Ruby port of MiniSearch, the JavaScript engine the graph page already loads in the browser. It brings a real index and BM25+ ranking.

Adding it cost you nothing at install time, which was the condition for adding it at all: no native extension, so gem install never needs a compiler; no dependency tree of its own; and the same Ruby 2.4 floor the gem has always held, so it keeps installing on a decade of Rubies.

The port also buys something the CLI could not otherwise have: the browser page and the Ruby CLI now run the same engine at the same version, so a result you find in the graph page and a result you find with okf search --engine index rank identically. Two implementations of "ranked" that disagree would be worse than one implementation that is slower.

The measurement

The comparison was built to be fair rather than flattering. The documents are the real @okf bundle resolved through OKF::Registry; larger sizes replicate those real concepts, with unique ids and titles and real bodies, so the corpus grows without turning synthetic. The okf side calls the actual OKF::Bundle::Search, not a reimplementation. The minifts side indexes the same six fields and mirrors okf's field weights as per-field boosts, so both rank on the same signal. Queries are drawn from the corpus's own most-frequent vocabulary, mixing selective and common single- and two-term queries, and a parity line prints each engine's result counts so both are visibly searching.

Throughput is the full nine-query workload per second, measured with benchmark-ips on Ruby 4.0.5:

corpus (concepts)	minifts	okf linear scan	speedup
23 (the real bundle)	2040 /s	42 /s	48x
250	221 /s	3.9 /s	56x
1,000	51 /s	1.0 /s	52x
4,000	10.5 /s	0.24 /s	44x

The multiple stays roughly constant because both approaches scale linearly here: the scan re-reads everything, and the replicated corpus shares bodies, so common terms match a large fraction of it. On a corpus with more distinct text the gap would widen, because minifts touches only the matching posting lists while a scan always visits all of it.

One disclosure belongs with that table. It measures workload throughput, not a latency distribution: these are iterations per second from benchmark-ips, so they answer "how many of this fixed query set per second" and not "what does p99 look like under concurrency." The minifts figures also predate a round of allocation tuning, so the build cost below is now somewhat lower than the number that drove the decision, which moves the argument in the same direction rather than against it.

The number that decided the default

A 50x speedup is a strong result and it is the wrong measure, which took embarrassingly long to see.

The index has a cost the scan never pays: it has to be built. Roughly 2.8 s per 1,000 concept documents, around 11 s at 4,000. Amortised over a few dozen queries that is nothing. Amortised over one query it is the entire runtime.

And one query is all a CLI invocation ever gets to ask:

at 1,000 concepts	end to end	of which index build
--engine index	3.00 s	~95%
default scan	0.24 s	none

The engine that wins every query by 50x loses the command by 12x. The throughput was real and the process never lived long enough to spend it.

So the default did not move. --engine index and --fuzzy are there for when you want the ranking or the typo tolerance and are willing to pay for it, and the tradeoff is stated in --help rather than buried in a benchmark nobody runs.

What the index costs when you do want it

Speed is not the only price, and the rest of it is easy to miss because the engine still returns rows.

An index matches whole words, and it decides what a word is by splitting on punctuation. So customer_id becomes two words, and 7.2.0 becomes three. The consequence you will actually hit is that searching for a fragment inside a word stops working:

$ okf search @okf ustomer
Search — @okf · ustomer (2 of 24 concepts)

$ okf search @okf ustomer --engine index
Search — @okf · ustomer (0 of 24 concepts)

Ranking does not rescue it either: a short concept dense in 7, 2 and 0 can outrank the one that actually says 7.2.0.

The default has none of these failure modes, for the unglamorous reason that matching raw text has no notion of a word to get wrong. Which is why the two engines are documented as a tradeoff rather than a tier list: the index ranks well and matches tokens; the default matches text and ranks crudely.

The one case where the index is unambiguously the right call is the one the scan cannot do at all:

$ okf search @okf serch            # 0 of 24 concepts
$ okf search @okf serch --fuzzy    # 13 of 24 concepts

The scan is not being unhelpful there. It is correctly reporting that nothing in the bundle contains those letters in that order.

The shape that changes the arithmetic

Nothing about the index is slow. The problem is entirely the process it lives in, which suggests the fix is not a faster index but a longer-lived one.

That is the direction okf is heading: a warm sidecar. A small resident process that builds the expensive index once and keeps it in memory, with the CLI talking to it instead of rebuilding from scratch. The first command pays what it pays today. Every command after it skips the 95%, natively installed or run from the Docker image alike.

To be clear about status: none of this has shipped, and there are no numbers for it yet. The honest open questions are the ones that always come with a resident process. When does a cached index go stale against files that changed underneath it? What starts and stops it, and what happens when nothing does? Where is the trust boundary, given okf already treats an unfamiliar bundle as untrusted content? A sidecar that answers those badly is worse than the 3.00 s it saves.

What 1.9.0 did was make that possible without another rewrite. And there is already one reader for which the arithmetic runs the other way: a long-lived tool that searches over and over, like a terminal UI or the graph server, is exactly the workload the benchmark says an index wins, so those ask for the index and get it. The sidecar is the same insight applied to the CLI itself. Make the process long-lived and the fast engine stops being a flag you have to justify.

Where this leaves the default

For now, okf search is exact and instant, and that is the right behaviour for a command you run once. --fuzzy is there when you are guessing at a spelling. --engine index is there when you want BM25+ ranking or parity with what the graph page shows.

The measurement that produced those defaults is reproducible; the harness lives in the minifts repository:

gem install benchmark-ips
ruby -Ilib benchmarks/okf_vs_minifts.rb 1000

Run it against your own bundle before taking any of the numbers above on faith. The interesting one is not the speedup. It is how much of your command's runtime the index build accounts for, because that single ratio is what decides whether a resident process is worth building, and it is the number the next release is aiming at.