kevy4.0

Full-text search (KIND text)

Text is the index engine's third kind: declare it like any index and the field's raw bytes tokenize into a per-shard inverted segment, maintained synchronously with every write (zero drift, same derived-by-construction discipline as range/unique). There is no separate search server, no ingestion pipeline, and no analyzer configuration — one verb declares the index, and every subsequent write keeps it exact.

IDX.CREATE posts ON PREFIX post: FIELD body TYPE str KIND text
IDX.QUERY posts MATCH "rust 全文检索" LIMIT 10 [FIELDS title body]

Quick start (server)

Rows are hash keys under the declared prefix; the indexed value is one declared field, exactly as with range indexes (indexes.md):

kevy-cli -p 6004 IDX.CREATE posts ON PREFIX post: FIELD body TYPE str KIND text
kevy-cli -p 6004 HSET post:1 title "intro" body "kevy is a pure-Rust key-value store"
kevy-cli -p 6004 HSET post:2 title "search" body "dictionary-free CJK full-text search"
kevy-cli -p 6004 IDX.QUERY posts MATCH "full-text rust" LIMIT 10

The reply is a flat array of key, score pairs, best first. Add FIELDS to hydrate named hash fields on each hit's owning shard in the same call (no second round-trip):

kevy-cli -p 6004 IDX.QUERY posts MATCH "search" LIMIT 10 FIELDS title

Supporting verbs work on text indexes like on every other kind:

MATCH accepts LIMIT (≤ 1000) and FIELDS; there is no CURSOR form (see "Matching and ranking" for why).

Quick start (embedded)

Same engine in-process. The text kind is behind the text cargo feature (on by default; idx_create with IndexKind::Text answers KevyError::Unsupported when compiled out):

use kevy_embedded::{Config, IndexKind, IndexValType, Store};

fn main() -> kevy_embedded::KevyResult<()> {
    let store = Store::open(Config::default())?;

    store.idx_create(b"posts", b"post:", b"body", IndexValType::Str,
                     IndexKind::Text)?;   // builds synchronously
    store.hset(b"post:1", &[
        (b"title".as_slice(), b"intro".as_slice()),
        (b"body".as_slice(),
         b"kevy is a pure-Rust key-value store".as_slice()),
    ])?;

    // BM25-ranked hits, best first: Vec<(key, score)>.
    let hits = store.idx_match(b"posts", b"rust store", 10)?;
    for (key, score) in &hits {
        println!("{} {score}", String::from_utf8_lossy(key));
    }
    Ok(())
}

idx_match returns KevyResult<Vec<(Vec<u8>, f64)>>; KevyError::NotFound names a missing index. There is no FIELDS hydration embedded — you are in-process, read fields with hget. Embedded builds are synchronous: idx_create returns when the index serves (no -INDEXBUILDING phase to poll).

Tokenization (dictionary-free)

The bigram scheme is the whole CJK story: no dictionary to ship, no dictionary to go stale, and mixed-script documents ("Rust 入门") index both halves under their own rules. The cost is recall noise on one-character CJK queries (they match only lone-character emissions) — query with two or more characters.

Matching and ranking

MATCH is OR semantics over query tokens, BM25-ranked (k1=1.2, b=0.75, non-negative idf variant). Documents matching more (and rarer) query terms rank higher; term frequency saturates; long documents are normalized down.

Two declared approximations:

No phrase queries, no boolean syntax, no highlighting — that's the query-engine slope (deliberately out of scope). If you need those, you are describing a search engine; kevy's text kind stops at "ranked lookup over declared fields".

Hybrid retrieval (BM25 + KNN)

A text index and an ANN index over the same corpus fuse server-side with reciprocal-rank fusion:

IDX.QUERY HYBRID posts MATCH "rust storage" embs KNN <f32-le-blob>
    [LIMIT n] [RRFK k] [EF ef] [FIELDS f…]

Each hit's fused score is Σ 1/(k + rank_i) over the two result lists (RRFK default 60 — the standard fusion constant; rank positions, not raw scores, so BM25 and distance need no calibration against each other). See vector-search.md for the KNN half. Hybrid is a server verb; embedded callers run idx_match + idx_knn and fuse in-process.

Lifecycle and consistency

Same envelope as every index kind (indexes.md):

Performance

Top-K evaluation uses MaxScore pruning (rarest terms first; commoner lists are probed per candidate once they can't lift new documents into the top K). Postings are doc-id inverted lists impact-ordered two ways: tf buckets descending, sparse log2 dl bands ascending inside each bucket — a single-term query (no second list to prune against) stops exactly at the first band whose lower edge can't beat the kth floor, so even the most common term answers in ~0.1ms at 200k docs (was ~6ms as a full postings scan). One-posting tokens (the Zipf long tail) stay inline and cost no heap.

Measured envelope (receipts in the bench tree):

The write side is the standard index tax: one hash-field read plus one segment update per matching index per write; an empty catalog costs one untaken branch per write.

Sizing

bytes ≈ Σ_token (token_len + 48) + postings × 64 + Σ_doc (key_len + text_len + 72) (docs keep their original text so updates remove exactly their own tokens), reported live by IDX.VERIFY / IDX.LIST (entries/bytes/postings/tokens for text kinds). bench/textgate.sh gates the formula against real RSS growth.

Rebuild rides the standard backfill skeleton (tick-incremental on the server, synchronous embedded).

See also