Ruby library API

Two layers

The library is deliberately split. A pure core models knowledge in memory with no disk, no stdio, no side effects. On-disk handles wrap it with load/save/reload/delete, an "ActiveRecord for the filesystem". require "okf" loads only this library surface; the CLI and skill installer load on demand, so an embedding app never pays for command-line machinery.

Pure, in-memory (no disk)

Build knowledge straight from data and run every feature against it. This is the surface an embedding app (a Rails store, an ETL job) uses over knowledge it already holds as records:

require "okf"

concept = OKF::Concept.new(
  path: "tables/orders.md",
  frontmatter: { "type" => "BigQuery Table", "title" => "Orders" },
  body: "Joined with [customers](/tables/customers.md).\n"
)

concept.id              # => "tables/orders"
concept.links           # => ["/tables/customers.md"]   (spec section 5)
concept.citations       # => [...]                      (spec section 8)
concept.to_markdown     # => String (round-trips the file format)
concept.lint            # => the concept-scoped checks

bundle = OKF::Bundle.new(concepts: [ concept ])
bundle.validate         # => section 9 conformance result
bundle.lint             # => curation report
bundle.graph            # => OKF::Bundle::Graph (#nodes, #edges, #to_h)

On disk

OKF::Bundle::Folder reads a directory into a pure bundle and materializes one back; OKF::Concept::File is a single-file handle:

folder = OKF::Bundle::Folder.load("docs")
folder.bundle                    # => OKF::Bundle (the pure bundle it read)
folder.validate; folder.lint     # delegate to the pure core
folder.concept("tables/orders")  # => OKF::Concept::File

# build in memory, then write it out:
OKF::Bundle::Folder.new(bundle: bundle, root: "out/dir").save

file = OKF::Concept::File.read(root: "docs", path: "tables/orders.md")
file.concept                     # => OKF::Concept (pure)
file.save; file.delete; file.reload

Writes are atomic, and Folder#save validates before publishing: it never writes a bundle that fails section 9.

Search, and how to add an engine

OKF::Bundle::Search is a facade over N engines. The facade owns what a result is: the documents, the row and the order of its keys, the snippet window, the final sort. An engine answers one narrower question, which documents match, how well, and where, and never builds a row of its own:

OKF::Bundle::Search.call(bundle, [ "dedup", "key" ])                  # the default engine
OKF::Bundle::Search.call(bundle, [ "dedup" ], engine: :index, fuzzy: true)
OKF::Bundle::Search.across([ [ "handbook", bundle ] ], [ "retry" ])   # many bundles, one ranking

Two engines ship. OKF::Bundle::Search::Scan is the default: raw literal matching, scored by the summed weight of the fields that hit, capability regexp. OKF::Bundle::Search::Index is a MiniFTS token index ranked by BM25+, capabilities fuzzy and prefix. The scan leads because a one-shot process pays for a build it asks one question of; a long-lived index is where the other one wins.

OKF::Bundle::Search.register is the published seam for a third. An engine is any object answering id, capabilities, available?, and call:

module Fts5Engine
  def self.id = :fts5
  def self.capabilities = %i[prefix]
  def self.available? = SQLite3.const_defined?(:Database)

  # => [ { key:, matched: [field, …], score:, terms: }, … ]
  def self.call(documents, terms, fields:, **options)
    ...
  end
end

OKF::Bundle::Search.register(Fts5Engine)

Registration is append-only and idempotent by id, so a double require cannot double the registry and an addon cannot displace a built-in. Capabilities are checked against a fixed vocabulary at registration, so a typo is refused rather than presenting later as "my engine is never selected". Every registered engine also runs a shared conformance suite: register one without a conformance class and the suite fails, which is the gem's way of saying an engine is a contract, not a duck.

A long-lived host can also avoid rebuilding the corpus on every query. OKF::Bundle::Search.prepare builds a corpus once (documents, the key-to-concept map, the built index) and Search.with queries it, which is how okf server warms search at boot instead of paying a full build per request. An engine opts in by exposing a prepare; one that does not, like the scan, is handed none, so the seam costs an addon nothing it must implement.

Mount the graph server

The interactive graph is a Rack app, so it mounts inside a host application as easily as it runs standalone:

require "okf/server/app"       # on-demand, exactly how the CLI loads it

OKF::Server::App.new(folder)   # => a Rack app: the interactive graph server

In Rails, mount it in routes.rb under any prefix; the page fetches concept bodies with relative URLs, so it works wherever it is mounted. The Rails guide has the full recipe, auth included. The graph server page covers the UI and the trust boundary.

Lower-level pieces

Each analyzer is usable on its own: OKF::Bundle::Validator.call(bundle), OKF::Bundle::Linter.call(bundle, min_body: 50), OKF::Bundle::Graph.build(bundle), and OKF::Markdown::Frontmatter.parse(markdown) for the file format itself. The architecture page explains the core/shell boundary that keeps all of this pure.