KacheDB Documentation

The High-Performance, Zero-Copy In-Memory Engine for Redis-Compatible Caching & LLM KV-Cache Offloading

📖 Welcome to the KacheDB Documentation

KacheDB is an open-source, next-generation in-memory storage engine engineered from first principles in Rust. It bridges the gap between traditional microsecond application caching (Redis / Valkey workloads) and multi-gigabyte tensor state offloading for Large Language Model (LLM) inference engines (vLLM, SGLang, PyTorch).

🗺️ Documentation Sitemap

🚀 Getting Started

• Quickstart Guide: Build from source, run via Cargo or Docker, and query via CLI.
• kachedb-cli User Guide: Interactive terminal REPL and built-in throughput benchmark harness.
• Server Configuration & Tuning: Worker threads, CPU pinning, memory pool sizing, and kernel bypass settings.

⚡ Command Reference

• Core Key-Value Commands: GET, SET, MGET, MSET, DEL, EXISTS, INCR, DECR, APPEND, STRLEN, DBSIZE, TYPE, FLUSHDB, FLUSHALL, PING.
• TTL & Key Lifecycle Commands: EXPIRE, PEXPIRE, EXPIREAT, PEXPIREAT, TTL, PTTL, PERSIST, and the background Timing Wheel.
• SIMD Semantic Vector Commands: VADD, VSEARCH, VADD_BATCH, VSEARCH_BATCH, VDEL, VSTATS, VINDEX, ARM NEON / AVX2 kernels.
• Server Observability & Introspection: INFO, COMMAND DOCS, HELLO 2/3, AUTH, CLIENT SETNAME/GETNAME/ID/LIST, BGREWRITEAOF, QUIT.

🏛️ System Architecture

• System Architecture Overview: High-level system design, thread-per-core model, and the physical memory hierarchy.
• 2 MB Megaslab Memory Engine: Slotted slab bump allocator, 64-byte cache-line alignment, and S3-FIFO quota manager.
• Token Radix Prefix Tree: Hierarchical &[u32] token prefix tree, sub-microsecond prefill lookup, and Epoch RCU concurrency.
• Zero-Copy Shared Memory IPC: POSIX /dev/shm lock-free ring buffers and PCIe line-rate tensor sharing.

🤖 Production Integration Guides

• vLLM Integration Guide: Drop-in PagedAttention KV connector for vLLM inference servers.
• SGLang Integration Guide: RadixAttention tree-branching prefill offloading with SGLang.
• Semantic Caching Guide: High-throughput sync and async prompt caching with kachedb-py and FastEmbed/HuggingFace.

📊 Performance & Benchmarks

• Consolidated Master Benchmark Report: Criterion micro-benchmarks, latency percentiles, and hardware comparisons against Redis 7.4, Valkey 8.0, and DragonflyDB.
• Benchmark Artifacts & Logs: Raw test outputs, multi-phase scaling reports, and reproducibility scripts.

🚀 Quickstart Guide

Get up and running with KacheDB in less than 60 seconds.

📦 Installation & Deployment Options

Option 1: Run with Pre-Built Docker Image (Recommended for Cloud/Production)

KacheDB publishes official multi-architecture container images (linux/amd64 and linux/arm64) to the GitHub Container Registry:

# Run with host IPC for zero-copy POSIX Shared Memory support
docker run --privileged --ipc host -p 6379:6379 -d --name kachedb ghcr.io/vubon/kachedb:latest

Verify the container is running:

docker logs kachedb

Option 2: Build & Run from Source with Cargo

Prerequisites

• Rust Toolchain: 2024 edition (rustc >= 1.85.0)
• OS: Linux (kernel 5.10+ recommended for io_uring) or macOS (Apple Silicon / Intel with kqueue)

# 1. Clone the repository
git clone https://github.com/vubon/kachedb.git
cd kachedb

# 2. Compile release binaries across all workspace crates
cargo build --release --workspace

# 3. Start the KacheDB multi-core daemon on default port 6379 with 4 worker threads
./target/release/kachedb-server -p 6379 -w 4

Option 3: Run with Docker Compose

git clone https://github.com/vubon/kachedb.git
cd kachedb
docker compose -f docker/docker-compose.yml up -d --build

⚡ Connecting to KacheDB

1. Using the Interactive kachedb-cli

KacheDB includes a built-in terminal CLI with colorized output and syntax assistance:

# Start interactive REPL connected to localhost:6379
./target/release/kachedb-cli -p 6379

Try some basic commands:

127.0.0.1:6379> PING
PONG

127.0.0.1:6379> SET user:100 "alice" EX 60
OK

127.0.0.1:6379> GET user:100
"alice"

127.0.0.1:6379> TTL user:100
(integer) 58

127.0.0.1:6379> INCR counter
(integer) 1

127.0.0.1:6379> INFO
# Server
kachedb_version:0.1.0
os:macos
arch_bits:64
...

2. Using Standard redis-cli

Because KacheDB implements the standard Redis RESP2/RESP3 wire protocol, you can use any existing redis-cli tool:

# Connect using standard redis-cli
redis-cli -p 6379

127.0.0.1:6379> MSET key1 "val1" key2 "val2"
OK

127.0.0.1:6379> MGET key1 key2 missing_key
1) "val1"
2) "val2"
3) (nil)

3. Using the Official Python SDK (kachedb-py)

Install the client SDK:

pip install kachedb

Synchronous usage:

from kachedb import KacheClient

with KacheClient(host="127.0.0.1", port=6379) as client:
    client.set("greeting", "Hello from Python", ex=300)
    val = client.get("greeting")
    print(val)  # b'Hello from Python'

Asynchronous usage (asyncio):

import asyncio
from kachedb import AsyncKacheClient

async def main():
    async with AsyncKacheClient(host="127.0.0.1", port=6379) as client:
        await client.set("session:abc", "active_data", ex=120)
        res = await client.get("session:abc")
        print(res)

asyncio.run(main())

🎯 Next Steps

• Explore all supported operations in the Core Key-Value Command Reference.
• Learn about active memory reclamation in TTL & Key Lifecycle Guide.
• Perform nearest-neighbor vector queries in the SIMD Vector Search Guide.
• Offload LLM prompt prefill compute in the vLLM Integration Guide.

💻 kachedb-cli User Guide

kachedb-cli is the native command-line interface and benchmarking utility for KacheDB. It provides an interactive REPL with formatted RESP rendering, auto-reconnect capabilities, and a high-performance live throughput benchmarking harness.

🛠️ Usage & Command-Line Flags

kachedb-cli [OPTIONS]

Options Reference

💬 Interactive REPL Mode

Starting kachedb-cli without --bench enters interactive REPL mode:

./target/release/kachedb-cli -h 127.0.0.1 -p 6379

Banner & Prompt

Upon connection, kachedb-cli displays the ASCII logo and prompt:

  _  __           _          _____  ____   _____ _      _____ 
 | |/ /          | |        |  __ \|  _ \ / ____| |    |_   _|
 | ' / __ _  ___| |__   ___| |  | | |_) | |    | |      | |  
 |  < / _` |/ __| '_ \ / _ \ |  | |  _ <| |    | |      | |  
 | . \ (_| | (__| | | |  __/ |__| | |_) | |____| |____ _| |_ 
 |_|\_\__,_|\___|_| |_|\___|_____/|____/ \_____|______|_____|

Connecting to KacheDB at 127.0.0.1:6379...
⚡ Connected to KacheDB. Type commands or 'help' / 'quit'.

127.0.0.1:6379> 

⚡ REPL Commands & Examples

1. Key-Value & Strings

127.0.0.1:6379> SET user:1 "Alice Smith"
OK

127.0.0.1:6379> GET user:1
"Alice Smith"

127.0.0.1:6379> APPEND user:1 " (Admin)"
(integer) 19

127.0.0.1:6379> GET user:1
"Alice Smith (Admin)"

127.0.0.1:6379> STRLEN user:1
(integer) 19

2. Atomic Counters

127.0.0.1:6379> INCR visits
(integer) 1

127.0.0.1:6379> INCRBY visits 10
(integer) 11

127.0.0.1:6379> DECR visits
(integer) 10

127.0.0.1:6379> DECRBY visits 5
(integer) 5

3. Expiration & TTL Management

127.0.0.1:6379> SET session:temp "xyz" EX 60
OK

127.0.0.1:6379> TTL session:temp
(integer) 58

127.0.0.1:6379> PERSIST session:temp
(integer) 1

127.0.0.1:6379> TTL session:temp
(integer) -1

4. Vector Ingestion & Nearest Neighbor Search

# Store a 4-dimensional vector in index 'docs' with ID 'doc:1'
127.0.0.1:6379> VADD docs doc:1 4 "\x00\x00\x80?\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" PAYLOAD "Introduction to KacheDB" EX 3600
OK

# Search index 'docs' for closest vector matches
127.0.0.1:6379> VSEARCH docs "\x00\x00\x80?\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" TOPK 1 THRESHOLD 0.8
1) 1) "doc:1"
   2) "1.000000"
   3) "Introduction to KacheDB"

# Check vector index statistics
127.0.0.1:6379> VSTATS docs
1) "dimension"
2) (integer) 4
3) "total_vectors"
4) (integer) 1
5) "active_vectors"
6) (integer) 1
6) "memory_bytes"
8) (integer) 64

5. Introspection & Diagnostics

127.0.0.1:6379> INFO
# Server
kachedb_version:0.1.0
os:macos
arch_bits:64
process_id:48123
tcp_port:6379
uptime_in_seconds:342

# Memory
used_memory:134217728
used_memory_human:128.00M
used_memory_peak:134217728
megaslabs_allocated:64
slab_slots_active:1024
fragmentation_ratio:1.00

# Stats
total_connections_received:42
total_commands_processed:15234
instantaneous_ops_per_sec:0
keyspace_hits:14200
keyspace_misses:1034

6. Built-in REPL Helper Commands

• help: Displays a quick command reference card.
• clear: Clears the terminal screen.
• quit / exit: Closes the connection and exits the CLI.

🔥 Live Benchmark Mode (--bench)

You can measure raw network round-trip throughput and latency using the built-in benchmark harness:

# Run a 100,000-request pipelined benchmark against localhost
./target/release/kachedb-cli -p 6379 --bench -n 100000

Example Benchmark Output

🔥 Connecting to 127.0.0.1:6379 for live benchmark (100,000 requests)...
⚡ Pipelining 100,000 PING commands...

══════════════════════════════════════════════════════════════
  KacheDB Live Benchmark Report
══════════════════════════════════════════════════════════════
  Requests:          100,000
  Total Elapsed:     34.28 ms
  Throughput:        2,916,742 ops/sec
  Average Latency:   342.8 ns / op
══════════════════════════════════════════════════════════════

⚙️ Server Configuration & Tuning

This guide covers operational parameters, memory pool configuration, CPU core pinning, and kernel bypass tuning for KacheDB.

🚀 Daemon Command-Line Arguments (kachedb-server)

The kachedb-server binary accepts several command-line flags to control hardware allocation and network binding:

./target/release/kachedb-server [OPTIONS]

Options Reference

📄 Configuration File (kachedb.conf)

KacheDB supports declarative configuration using a canonical kachedb.conf file:

# Network & Binding
bind 127.0.0.1
port 6379
maxclients 10000

# Worker Threads & Memory
workers 4
pool_mb_per_core 256

# IPC & LLM Tensor Streaming
shm_enabled true

# Persistence (Append-Only File)
aof_enabled false
aof_path kachedb.aof
appendfsync everysec

# Security
# requirepass my_secret_token

To launch kachedb-server using the configuration file:

./target/release/kachedb-server -c /path/to/kachedb.conf

Note: Any explicit CLI flags will take precedence over directives inside the configuration file.

🧵 Thread-per-Core Topology & CPU Pinning

KacheDB operates on a shared-nothing, thread-per-core architecture:

• Each active worker thread is pinned to a dedicated physical CPU core using core_affinity.
• Zero Cross-Core Contention: Each worker thread owns its private 2 MB Megaslab arena pool and independent Swiss Table shard.
• Request execution requires no global mutex locks, eliminating lock contention and cache-line bouncing.

Example: Running on a Dedicated 8-Core Node

# Pin 8 workers to cores 0..7 with 512 MB memory per core (4 GB total)
./target/release/kachedb-server -p 6379 -w 8 --pool-mb 512

🧱 Memory Sizing & S3-FIFO Quota Management

Memory is managed through uniform 2 MB Megaslabs:

• Rather than calling malloc() on every request, KacheDB pre-allocates contiguous megaslab page frames.
• Per-Core Sizing: If --pool-mb is set to 256 on a 4-core machine, total initial memory allocated across the daemon is $4 \times 256\text{ MB} = 1.024\text{ GB}$.
• Elastic Borrowing: The dynamic WorkloadQuota manager elastically allocates megaslabs between application key-value cache and tensor memory based on current demand.

🐧 Linux Kernel & io_uring Tuning

For maximum throughput on Linux (> 2.5M QPS), apply the following kernel optimizations:

1. somaxconn & TCP Backlog

sudo sysctl -w net.core.somaxconn=65535
sudo sysctl -w net.ipv4.tcp_max_syn_backlog=65535

2. POSIX Shared Memory Limits (/dev/shm)

Ensure /dev/shm has sufficient space for high-volume LLM KV-cache offloading:

# Verify current /dev/shm size
df -h /dev/shm

# Remount /dev/shm with 32 GB (if serving 70B+ LLM inference nodes)
sudo mount -o remount,size=32G /dev/shm

3. File Descriptor Limits

ulimit -n 1048576

🐳 Production Docker Configuration

Here is the recommended production docker-compose.yml:

services:
  kachedb:
    image: ghcr.io/vubon/kachedb:latest
    container_name: kachedb
    privileged: true
    ipc: host
    network_mode: host
    restart: always
    command: ["-p", "6379", "-w", "4", "--pool-mb", "256"]
    ulimits:
      nofile:
        soft: 1048576
        hard: 1048576
      memlock:
        soft: -1
        hard: -1

🔑 Core Key-Value Commands

KacheDB implements the standard Redis / Valkey RESP2 and RESP3 binary wire protocol for standard key-value operations. All keys and values are treated as raw byte slices (&[u8]) and stored with zero heap allocation overhead inside 64-byte aligned Megaslab slots.

📋 Command Summary

🛠️ Detailed Command Reference & Examples

SET & GET

Stores and retrieves binary-safe values up to 2 MB per slot.

Syntax

SET key value [EX seconds] [PX milliseconds]
GET key

kachedb-cli Example

127.0.0.1:6379> SET user:100 "Alice"
OK

127.0.0.1:6379> GET user:100
"Alice"

127.0.0.1:6379> SET session:temp "abc123xyz" EX 30
OK

127.0.0.1:6379> GET missing_key
(nil)

Python SDK Example

with KacheClient() as client:
    client.set("user:100", "Alice")
    val = client.get("user:100")  # b'Alice'

MSET & MGET

Batch operations that store and retrieve multiple keys in a single network round-trip.

Syntax

MSET key value [key value ...]
MGET key [key ...]

kachedb-cli Example

127.0.0.1:6379> MSET config:theme "dark" config:lang "en" config:tz "UTC"
OK

127.0.0.1:6379> MGET config:theme config:lang config:missing config:tz
1) "dark"
2) "en"
3) (nil)
4) "UTC"

INCR, DECR, INCRBY, DECRBY

Atomic integer arithmetic executed in-place on string values. If the key does not exist, it is initialized to 0 before applying the operation.

Syntax

INCR key
DECR key
INCRBY key delta
DECRBY key delta

kachedb-cli Example

127.0.0.1:6379> INCR page_views
(integer) 1

127.0.0.1:6379> INCRBY page_views 100
(integer) 101

127.0.0.1:6379> DECR page_views
(integer) 100

127.0.0.1:6379> DECRBY page_views 50
(integer) 50

APPEND & STRLEN

String manipulation and length inspection.

Syntax

APPEND key value
STRLEN key

kachedb-cli Example

127.0.0.1:6379> SET doc:title "KacheDB"
OK

127.0.0.1:6379> APPEND doc:title " Architecture"
(integer) 20

127.0.0.1:6379> GET doc:title
"KacheDB Architecture"

127.0.0.1:6379> STRLEN doc:title
(integer) 20

DEL & EXISTS

Key deletion and existence verification.

Syntax

DEL key [key ...]
EXISTS key [key ...]

kachedb-cli Example

127.0.0.1:6379> EXISTS user:1 user:2
(integer) 1

127.0.0.1:6379> DEL user:1 user:2
(integer) 1

127.0.0.1:6379> EXISTS user:1
(integer) 0

DBSIZE

Returns the count of active, unexpired keys stored in the database.

Syntax

DBSIZE

kachedb-cli Example

127.0.0.1:6379> SET k1 "v1"
OK
127.0.0.1:6379> SET k2 "v2"
OK
127.0.0.1:6379> DBSIZE
(integer) 2

TYPE

Returns the underlying data structure type of the specified key.

Syntax

TYPE key

kachedb-cli Example

127.0.0.1:6379> SET greeting "hello"
OK
127.0.0.1:6379> TYPE greeting
string

127.0.0.1:6379> TYPE nonexistent
none

FLUSHDB & FLUSHALL

Deletes all keys in the database and recycles Megaslab blocks back to the free-list.

Syntax

FLUSHDB
FLUSHALL

kachedb-cli Example

127.0.0.1:6379> DBSIZE
(integer) 1000
127.0.0.1:6379> FLUSHDB
OK
127.0.0.1:6379> DBSIZE
(integer) 0

⏱️ TTL & Key Lifecycle Commands

KacheDB provides high-resolution time-to-live (TTL) expiration support with dual-engine memory reclamation:

1. Sub-Nanosecond Passive Expiry: On GET/EXISTS queries, the Swiss Table verifies the cached second timestamp in ~0.5 ns.
2. Active O(1) Background Timing Wheel: A lock-free 3,600-bucket per-core circular wheel proactively evicts expired keys and returns 2 MB Megaslab slots back to the free-list every second without waiting for read traffic.

📋 Command Summary

🛠️ Command Details & Examples

EXPIRE & PEXPIRE

Sets a relative timeout from the current time.

kachedb-cli Example

127.0.0.1:6379> SET user:auth "token_xyz123"
OK

# Expire in 60 seconds
127.0.0.1:6379> EXPIRE user:auth 60
(integer) 1

# Check remaining seconds
127.0.0.1:6379> TTL user:auth
(integer) 59

# Check remaining milliseconds
127.0.0.1:6379> PTTL user:auth
(integer) 58942

# Attempting to expire a non-existent key returns 0
127.0.0.1:6379> EXPIRE missing_key 30
(integer) 0

EXPIREAT & PEXPIREAT

Sets an absolute Unix epoch deadline timestamp.

kachedb-cli Example

# Expire at Unix timestamp 1893456000 (Jan 1, 2030)
127.0.0.1:6379> EXPIREAT user:auth 1893456000
(integer) 1

127.0.0.1:6379> TTL user:auth
(integer) 109895658

PERSIST

Removes the timeout from a key, converting it back to a permanent key.

kachedb-cli Example

127.0.0.1:6379> PERSIST user:auth
(integer) 1

# TTL returns -1 for unexpired keys without timeout
127.0.0.1:6379> TTL user:auth
(integer) -1

🏗️ Architecture: How the Timing Wheel Works

┌────────────────────────────────────────────────────────────────────────────────────────┐
│               HashedTimingWheel (3,600 Circular 1-Second Buckets)                       │
│                                                                                        │
│   Slot 0       Slot 1       Slot 2        ...         Slot 3599                        │
│ ┌─────────┐  ┌─────────┐  ┌─────────┐               ┌─────────┐                        │
│ │ Entries │  │ Entries │  │ Entries │               │ Entries │                        │
│ └────┬────┘  └────┬────┘  └────┬────┘               └────┬────┘                        │
│      │            │            │                         │                             │
│      ▼            ▼            ▼                         ▼                             │
│   KeyHash 1    KeyHash 2    KeyHash 3                 KeyHash N                        │
│  [BlockID 5]  [BlockID 9]  [BlockID 2]               [BlockID 14]                      │
└────────────────────────────────────────────────────────────────────────────────────────┘
                                      ▲
                                      │ Current Second Pointer (Advances every 1.0s)
                               [ Event Loop Tick ]

1. In-Place Mutation: EXPIRE and PERSIST mutate expiry_sec: u32 in the 64-byte TableEntry in-place.
2. Zero Allocation Jitter: Scheduling a key in the timing wheel appends to a per-bucket pre-allocated array.
3. Double-Free Safety: When the 1-second tick advances, the worker thread executes table.remove_if_matching(key_hash, slab_block_id). If the key was updated or replaced by a new SET command, the stale entry is safely ignored without double-deallocating slab blocks.

🧠 SIMD Semantic Vector Commands

KacheDB features a hardware-accelerated SIMD Vector Search Engine (kachedb-vector) built directly into the storage core. It enables sub-microsecond nearest-neighbor vector lookups, semantic caching, and LLM prompt similarity matching with zero external dependencies.

⚡ Hardware SIMD Acceleration

• ARM NEON (aarch64): 128-bit vfmaq_f32 with 4-way loop unrolling (16 floats per loop iteration) delivering $< 120\text{ ns}$ dot products on Apple Silicon and AWS Graviton.
• x86_64 AVX2 / FMA: 256-bit _mm256_fmadd_ps with 4-way loop unrolling (32 floats per iteration) delivering $> 40\text{ GB/s}$ throughput.
• Normalized Cosine Similarity: All stored vectors are automatically $L_2$-normalized upon ingestion, transforming cosine distance calculation into a single high-speed inner dot product: $$\text{CosineSimilarity}(\vec{u}, \vec{v}) = \sum_{i=1}^{D} u_i \cdot v_i$$

📋 Command Summary

🛠️ Detailed Command Reference & Examples

VADD

Ingests a single float32 vector into a named index. The vector is provided as raw little-endian IEEE 754 float32 byte buffers.

Syntax

VADD <index> <id> <dim> <vector_bytes> [PAYLOAD <payload>] [EX <seconds>]

kachedb-cli Example

# Insert a 4-dimensional vector with payload and 1-hour expiration
127.0.0.1:6379> VADD faq q:101 4 "\x00\x00\x80?\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" PAYLOAD "To reset password, go to Settings -> Security." EX 3600
OK

VSEARCH

Performs nearest-neighbor cosine similarity search across all vectors in the index.

Syntax

VSEARCH <index> <query_bytes> [TOPK <k>] [THRESHOLD <min_similarity>]

kachedb-cli Example

# Search index 'faq' for top 1 match with similarity >= 0.80
127.0.0.1:6379> VSEARCH faq "\x00\x00\x80?\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" TOPK 1 THRESHOLD 0.80
1) 1) "q:101"
   2) "1.000000"
   3) "To reset password, go to Settings -> Security."

VDEL & VSTATS

Index management and telemetry.

kachedb-cli Example

127.0.0.1:6379> VSTATS faq
1) "dimension"
2) (integer) 4
3) "total_vectors"
4) (integer) 1
5) "active_vectors"
6) (integer) 1
7) "memory_bytes"
8) (integer) 64

127.0.0.1:6379> VDEL faq q:101
(integer) 1

VADD_BATCH & VSEARCH_BATCH

High-throughput bulk ingestion and multi-vector querying.

Syntax

VADD_BATCH <index> <id1> <vec1_bytes> [payload1] ...
VSEARCH_BATCH <index> <q1_bytes> <q2_bytes> ... [TOPK <k>] [THRESHOLD <min_similarity>]

VINDEX CREATE, VINDEX INFO & VINDEX DROP

Lifecycle management for named HNSW vector index topologies.

Syntax

VINDEX CREATE <name> DIM <dim> [M <m>] [EF_CONSTRUCTION <ef_c>] [EF_SEARCH <ef_s>] [METRIC <COSINE|L2|IP>] [QUANTIZATION <NONE|SQ8>]
VINDEX INFO <name>
VINDEX DROP <name>

kachedb-cli Example

# Create an index for 1536-dimensional OpenAI embeddings
127.0.0.1:6379> VINDEX CREATE embeddings DIM 1536 METRIC COSINE QUANTIZATION SQ8
OK

# Inspect index topology
127.0.0.1:6379> VINDEX INFO embeddings
1) "name"
2) "embeddings"
3) "dimension"
4) (integer) 1536
5) "metric"
6) "COSINE"
7) "quantization"
8) "SQ8"

# Drop index when no longer needed
127.0.0.1:6379> VINDEX DROP embeddings
OK

🐍 Python SDK (kachedb-py) Example

Using vectors is seamless via kachedb-py:

from kachedb import KacheClient

with KacheClient() as client:
    # 1. Ingest vector embedding (automatically packs float lists to IEEE-754 bytes)
    embedding = [0.12, -0.45, 0.88, 0.05]
    client.vadd(
        index="products",
        item_id="item:1001",
        vector=embedding,
        payload="Ergonomic Mechanical Keyboard",
        ex=86400,
    )

    # 2. Query nearest vectors
    query_vector = [0.10, -0.40, 0.85, 0.04]
    results = client.vsearch(
        index="products",
        query_vector=query_vector,
        top_k=3,
        threshold=0.85,
    )

    for item_id, score, payload in results:
        print(f"Matched {item_id} (Score: {score:.4f}): {payload}")

📊 Server Observability & Introspection

KacheDB provides self-describing runtime introspection compatible with modern Redis tooling, GUI clients (such as Redis Insight and TablePlus), and monitoring agents.

📋 Command Summary

🛠️ Detailed Command Reference & Examples

INFO

Outputs multi-section server metrics in standard Redis key-value format.

Syntax

INFO [server | memory | stats | keyspace | vector]

kachedb-cli Example

127.0.0.1:6379> INFO
# Server
kachedb_version:0.1.0
os:macos
arch_bits:64
process_id:48123
tcp_port:6379
uptime_in_seconds:1250

# Memory
used_memory:134217728
used_memory_human:128.00M
used_memory_peak:134217728
megaslabs_allocated:64
slab_slots_active:5120
fragmentation_ratio:1.00

# Stats
total_connections_received:128
total_commands_processed:150240
instantaneous_ops_per_sec:0
keyspace_hits:148000
keyspace_misses:2240

# Keyspace
db0:keys=5120,expires=450,avg_ttl=1820

# VectorEngine
active_indices:3
total_vectors:10240
vector_memory_bytes:655360
simd_kernel:auto

HELLO

Negotiates RESP wire protocol version with the server (supports version 2 and version 3).

Syntax

HELLO 3 [SETNAME client_name]

kachedb-cli Example

127.0.0.1:6379> HELLO 3 SETNAME my_worker
 1) "server"
 2) "kachedb"
 3) "version"
 4) "0.1.0"
 5) "proto"
 6) (integer) 3
 7) "id"
 8) (integer) 1
 9) "mode"
10) "standalone"
11) "role"
12) "master"
13) "modules"
14) (empty array)

CLIENT

Manages client connection names and identifiers.

Subcommands

• CLIENT SETNAME <name>: Assigns a human-readable name to the current TCP connection.
• CLIENT GETNAME: Retrieves the assigned name (or nil).
• CLIENT ID: Returns the unique client connection ID.
• CLIENT LIST: Returns connected client details.

kachedb-cli Example

127.0.0.1:6379> CLIENT SETNAME web_app_1
OK

127.0.0.1:6379> CLIENT GETNAME
"web_app_1"

127.0.0.1:6379> CLIENT ID
(integer) 1

COMMAND & COMMAND DOCS

Provides capability introspection so GUI tools and drivers can discover supported commands dynamically without crashing.

kachedb-cli Example

127.0.0.1:6379> COMMAND DOCS
OK

AUTH

Authenticates a client connection when password protection is enabled with requirepass.

Syntax

AUTH [username] <password>

kachedb-cli Example

127.0.0.1:6379> GET secret_key
(error) NOAUTH Authentication required.
127.0.0.1:6379> AUTH my_secure_password
OK
127.0.0.1:6379> GET secret_key
"confidential_data"

BGREWRITEAOF

Instructs the server to compact and rewrite the Append-Only File (kachedb.aof) to remove redundant commands and minimize disk space.

Syntax

BGREWRITEAOF

kachedb-cli Example

127.0.0.1:6379> BGREWRITEAOF
Background append only file rewriting started

🏛️ System Architecture Overview

KacheDB is built from first principles in Rust to solve the fundamental memory, OS, and serialization bottlenecks of modern database architectures.

⚡ The Dual-Engine Topology

+-----------------------------------------------------------------------------------------------+
|                                      CLIENT INTERFACES                                        |
|   [RESP3 Wire Protocol (Redis/Valkey Clients)]     [Zero-Copy Tensor IPC / Python SDK]        |
+-----------------------------------------------------------------------------------------------+
                                               │
+-----------------------------------------------------------------------------------------------+
|                                       INDEXING SUBSYSTEM                                      |
|   1. SIMD Swiss Hash Table (3.09 ns Point Lookups, S3-FIFO Eviction Tracking)                 |
|   2. Token Radix Prefix Tree (&[u32] Longest Prefix Match for LLM KV-Cache Prefills)          |
|   3. Per-Core Hashed Timing Wheel (3,600 Circular Buckets for O(1) Memory Reclamation)        |
+-----------------------------------------------------------------------------------------------+
                                               │
+-----------------------------------------------------------------------------------------------+
|                                  CORE SLAB & ARENA ENGINE                                     |
|   - 2 MB Megaslab Page Frames (64-byte Cache-Line Aligned Slots, 0 False Sharing)             |
|   - Zero Runtime Heap Allocation Jitter (Bump-pointer + Free-list Recycling)                  |
|   - S3-FIFO Dynamic Quota Management (Elastic Workload Quotas)                                |
+-----------------------------------------------------------------------------------------------+
                                               │
+-----------------------------------------------------------------------------------------------+
|                                ZERO-COPY TRANSPORT LAYER                                      |
|   - Local: POSIX Shared Memory (/dev/shm) Lock-Free SPSC Ring Buffers (17.66M msgs/sec)       |
|   - Network: Linux io_uring (SQPOLL Fixed Buffers) / macOS kqueue (Edge-Triggered Workers)    |
+-----------------------------------------------------------------------------------------------+

🥊 Why Traditional In-Memory Databases Hit a Wall

Traditional in-memory engines (e.g. Redis, Memcached) struggle when scaling to tens of millions of operations per second or serving multi-megabyte AI tensors due to three physical hardware constraints:

1. The Generality Tax (Heap Fragmentation & Pointer Chasing)

• General-purpose databases support variable-size dynamic keys and polymorphic data structures, relying on standard dynamic allocators (jemalloc, malloc).
• Every non-contiguous pointer chase incurs an L1/L2/L3 CPU cache miss (~50–100 ns stall), degrading memory bandwidth.
• KacheDB Solution: Pre-allocates uniform 2 MB Megaslabs with fixed size classes and 64-byte cache line alignment, eliminating runtime malloc overhead down to 3.84 ns.

2. Kernel Context-Switching Overhead

• Standard POSIX socket operations (read(), write(), epoll_wait()) require constant transitions between user-space and kernel-space.
• Processing millions of small network packets saturates the CPU with interrupt handling and buffer copies.
• KacheDB Solution: Utilizes Linux io_uring with SQPOLL for zero-syscall kernel-bypass socket polling and an Accept-Dispatcher ring for even worker CPU saturation.

3. The Tensor Serialization Penalty

• Traditional key-value protocols serialize all responses over TCP sockets.
• For an LLM KV cache (hundreds of megabytes of FP16/BF16 attention matrices), serializing and copying through kernel socket buffers to Python completely destroys latency gains.
• KacheDB Solution: Direct POSIX Shared Memory (/dev/shm) lock-free SPSC ring buffers, allowing PyTorch and vLLM to map tensors directly from host RAM at PCIe line rate without socket copies.

🧵 Thread-per-Core Execution Model

• Each physical CPU core runs an independent WorkerThread pinned via core_affinity.
• Each worker owns an isolated SlabPool, SwissTable shard, and TimingWheel.
• Zero cross-thread locks during request execution ensures linear throughput scaling across 4, 8, 32, or 64 cores.

🧱 2 MB Megaslab Memory Engine

The kachedb-core memory subsystem completely eliminates runtime malloc and free allocation jitter by managing storage through structured 2 MB Megaslabs with 64-byte L1 CPU cache-line alignment.

📐 Memory Slab Geometry

┌────────────────────────────────────────────────────────────────────────────────────────┐
│                        2 MB Megaslab Page Frame (2,097,152 Bytes)                      │
│ ┌───────────────────┬───────────────────┬───────────────────┬────────────────────────┐ │
│ │  Slot 0 (64B-Aln) │  Slot 1 (64B-Aln) │  Slot 2 (64B-Aln) │  Slot N (64B-Aln)      │ │
│ │ [ Payload Data  ] │ [ Payload Data  ] │ [ Payload Data  ] │ [ Payload Data  ]      │ │
│ └───────────────────┴───────────────────┴───────────────────┴────────────────────────┘ │
└────────────────────────────────────────────────────────────────────────────────────────┘

1. Cache-Line Alignment (64-byte)

• Every allocated slot is guaranteed to start on a 64-byte boundary via libc::posix_memalign.
• Zero False Sharing: Multi-core writes to neighboring slots never cross CPU L1 cache line boundaries, eliminating cache invalidation stalls.

2. Standard Size Classes

⚡ Bump Pointer + Free-List Recycling

Allocation in KacheDB occurs in two stages:

1. Fast-Path Bump Allocation: If the active 2 MB arena has unallocated capacity, it increments a local cursor in $\approx 3.84\text{ ns}$.
2. Free-List Slot Recycling: When keys are overwritten, deleted, or expired, their BlockId is pushed to a lock-free LIFO free-list for immediate O(1) slot reuse.

🔄 Dynamic S3-FIFO Workload Quotas

KacheDB implements the state-of-the-art S3-FIFO (Simple, Scalable, Small-footprint FIFO) cache eviction algorithm:

• Small Queue (10% capacity): Acts as a high-speed filter against scan pollution and one-off burst keys.
• Main Queue (90% capacity): Stores frequency-accessed keys with multi-chance bit tracking.
• Ghost Queue: Tracks key signatures after eviction to detect re-access and promote directly into the Main Queue.
• Elastic Borrowing: The WorkloadQuota manager balances memory pools between Redis key-value workloads and LLM tensor memory elastically.

🌳 Token Radix Prefix Tree

The kachedb-radix subsystem implements a hierarchical token prefix tree optimized for LLM attention prompt prefill reuse (such as vLLM PagedAttention and SGLang RadixAttention).

⚡ The LLM Prefill Problem

During LLM inference, requests operate in two phases:

1. Prefill Phase: Computes the Key and Value ($K, V$) attention matrices across all layers for the prompt tokens. Computational complexity is O(N²) with respect to sequence length.
2. Decode Phase: Autoregressively generates tokens one by one using previously computed $K, V$ states.

For long multi-turn prompts (e.g. system prompts, few-shot examples, large codebases, 16K–128K tokens):

• Recomputing a 32K token prefill on an NVIDIA H100 GPU takes $200\text{--}500\text{ ms}$.
• Restoring precomputed KV tensors from KacheDB takes $< 20\text{ ms}$.
• Result: Offloading attention states cuts Time-To-First-Token (TTFT) by 5× to 15×.

🌲 Tree Topology & Chunked Edges

               [ Root (Parent Hash = 0) ]
                            │
               ┌────────────┴────────────┐
               ▼                         ▼
      "You are an assistant"   "System Prompt: Coding"
       [Chunk: 16 Tokens]        [Chunk: 16 Tokens]
               │                         │
       ┌───────┴───────┐                 │
       ▼               ▼                 ▼
  "Turn 1: Python" "Turn 1: Rust"  "User Prompt: Refactor"
   [Block ID 4]     [Block ID 9]     [Block ID 12]

1. Compressed Edge Hops ([u32; 16])

• Rather than storing 1 token per node hop, KacheDB edges compress 16 tokens per block.
• Traversal complexity drops from O(L) to O(L / 16).
• A 1,024-token prompt prefix matches in $2.45\ \mu\text{s}$ ($> 10,000\times$ faster than GPU recomputation).

2. Epoch-Based RCU Concurrency (EpochTree)

• Multi-reader concurrency is managed via lock-free Read-Copy-Update (RCU) snapshots using arc-swap.
• Reader inference threads access snapshots with $\approx 1\text{ ns}$ atomic load overhead without locking writers.

3. Reference Counting & Pinning

• Active attention decoders pin tree nodes (ref_count++), ensuring active conversation blocks are never evicted while a request is in flight.
• Eviction uses Bottom-Up Hierarchical LRU, pruning leaf turns while protecting shared root system prompts.

🚀 Zero-Copy Shared Memory IPC

The kachedb-shm subsystem implements high-throughput, zero-copy inter-process communication (IPC) between the KacheDB daemon and Python / PyTorch / CUDA inference engines via POSIX Shared Memory (/dev/shm).

⚡ The Socket Serialization Bottleneck

Transferring large attention tensors (50 MB – 2 GB) over standard TCP loopback sockets suffers from severe throughput degradation:

1. Python creates a socket payload → serializes tensor buffers.
2. Kernel performs socket send() / context switch into kernel space → copies into socket ring buffers.
3. Daemon recv() / context switch into user space → deserializes data into memory.
4. Total latency penalty: 15–40 ms, completely negating prefill savings.

🏛️ KacheDB Zero-Copy Architecture

┌────────────────────────┐              ┌────────────────────────┐
│   KacheDB Rust Daemon  │              │  vLLM / PyTorch Worker │
│ ┌────────────────────┐ │              │ ┌────────────────────┐ │
│ │  Megaslab Memory   │ │              │ │   torch.Tensor     │ │
│ │ (Direct Slot Ptr)  │ │              │ │  (Zero-Copy View)  │ │
│ └─────────┬──────────┘ │              │ └──────────┬─────────┘ │
└───────────┼────────────┘              └────────────┼───────────┘
            │                                        │
            ▼                                        ▼
┌────────────────────────────────────────────────────────────────┐
│           POSIX Shared Memory Region (/dev/shm/kachedb_0)       │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ 192-byte Lock-Free SPSC Ring Header (3 Isolated 64B Lines) │ │
│ ├────────────────────────────────────────────────────────────┤ │
│ │ Cache-Line Aligned Megaslab Page Frames (Multi-GB Storage) │ │
│ └────────────────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────┘

1. Lock-Free SPSC Ring Buffer

• Operates a Single-Producer Single-Consumer queue transferring TensorBlockDescriptor metadata frames.
• Throughput: Delivers 17.66 Million messages/sec (56.6 ns per slot) across process boundaries.

2. Cache-Line Isolation (0 False Sharing)

• The 192-byte ring header is partitioned into three isolated 64-byte cache lines:
  • Cache Line 0: Read cursor & consumer state
  • Cache Line 1: Write cursor & producer state
  • Cache Line 2: Ring capacity & flags
• Multi-core reader and writer threads never trigger CPU L1 cache invalidation races.

3. Adaptive Spin-Then-Park Strategy

• Fast-path transfers use busy-spinning for $< 50\text{ ns}$ handoffs.
• Falls back to OS thread parking if the queue remains empty or full for $> 100\ \mu\text{s}$, conserving CPU cycles during idle periods.

🤖 vLLM Integration Guide

This guide walks through configuring KacheDB as a high-speed external KV-cache offloading tier for vLLM (PagedAttention).

⚡ Overview

By offloading PagedAttention KV-cache blocks from GPU VRAM to KacheDB's zero-copy POSIX Shared Memory (/dev/shm), vLLM inference instances achieve:

• Up to 10,000× faster prompt prefill for repeated system prompts, few-shot examples, and multi-turn chat sessions.
• Zero socket serialization overhead using PyTorch tensor memory views.
• Seamless multi-GPU and distributed tensor parallel support.

📦 Installation & Setup

1. Start the KacheDB daemon with --ipc host or native POSIX Shared Memory enabled:

./target/release/kachedb-server -p 6379 -w 4 --pool-mb 512

2. Install the KacheDB Python client with PyTorch support:

pip install kachedb[torch]

🚀 Programmatic Integration Example

The KacheDBConnector handles prefix caching, block hashing, and zero-copy restoration automatically:

import torch
from kachedb.vllm import KacheDBConnector

# 1. Initialize the connector for the active GPU worker rank
connector = KacheDBConnector(
    rank=0,
    local_rank=0,
    block_size=16,
    pool_size_mb=256,
)

# 2. Simulated PagedAttention tensor for 2 transformer layers
# Shape: [num_blocks=4, 2 (K/V), num_heads=8, block_size=16, head_dim=64]
kv_shape = (4, 2, 8, 16, 64)
kv_caches = [
    torch.randn(kv_shape, dtype=torch.float16),
    torch.randn(kv_shape, dtype=torch.float16),
]

# 3. Offload KV cache blocks to KacheDB
prompt_tokens = [101, 2054, 2003, 1037, 2742, 102]
connector.offload_kv_cache(
    prompt_tokens=prompt_tokens,
    kv_caches=kv_caches,
)

# 4. On subsequent requests, restore matching prefix blocks
new_request_tokens = [*prompt_tokens, 999, 1000]
target_kv_caches = [
    torch.zeros(kv_shape, dtype=torch.float16),
    torch.zeros(kv_shape, dtype=torch.float16),
]

matched_tokens, is_hit = connector.restore_kv_cache(
    prompt_tokens=new_request_tokens,
    target_kv_caches=target_kv_caches,
)

if is_hit:
    print(f"✅ Cache Hit! Restored {matched_tokens} tokens from KacheDB zero-copy.")

🌲 SGLang RadixAttention Integration Guide

This guide describes integrating KacheDB with SGLang's RadixAttention hierarchical tree-branching engine.

⚡ Overview

SGLang manages KV-caches as a dynamic Radix Tree across multiple conversational branches and tool calls. KacheDB's KacheDBSGLangConnector maps SGLang tree nodes directly to lock-free memory frames:

• Arbitrary Slice Lengths: Variable-length token chunks hashed via chained Blake2b.
• Hierarchical Multi-Branch Restoration: Restores branched tree paths in a single pass.
• Multi-Precision Support: Native support for FP16, BF16 (LLaMA default), FP32, and INT8.

🚀 Programmatic Integration Example

import torch
from kachedb.sglang import KacheDBSGLangConnector

# 1. Initialize connector
connector = KacheDBSGLangConnector(
    rank=0,
    local_rank=0,
    pool_size_mb=256,
)

num_heads = 4
head_dim = 64
num_layers = 2
dtype = torch.bfloat16

# 2. Node 1: Root System Prompt (10 tokens)
root_tokens = list(range(10))
k_root = [torch.randn((num_heads, 10, head_dim), dtype=dtype) for _ in range(num_layers)]
v_root = [torch.randn((num_heads, 10, head_dim), dtype=dtype) for _ in range(num_layers)]

desc_root = connector.offload_node(
    node_id=1,
    token_ids=root_tokens,
    k_tensors=k_root,
    v_tensors=v_root,
    parent_hash=0,
)

# 3. Node 2: Child Branch Turn (20 tokens)
child_tokens = list(range(10, 30))
k_child = [torch.randn((num_heads, 20, head_dim), dtype=dtype) for _ in range(num_layers)]
v_child = [torch.randn((num_heads, 20, head_dim), dtype=dtype) for _ in range(num_layers)]

connector.offload_node(
    node_id=2,
    token_ids=child_tokens,
    k_tensors=k_child,
    v_tensors=v_child,
    parent_hash=desc_root.node_hash,
)

# 4. Restore Full 30-Token Sequence (Root + Child)
full_prompt = list(range(30))
target_k = [torch.zeros((num_heads, 30, head_dim), dtype=dtype) for _ in range(num_layers)]
target_v = [torch.zeros((num_heads, 30, head_dim), dtype=dtype) for _ in range(num_layers)]

matched_count, is_hit = connector.restore_prefix(
    prompt_tokens=full_prompt,
    target_k_buffers=target_k,
    target_v_buffers=target_v,
)

if is_hit:
    print(f"🌲 SGLang Restored {matched_count} tokens across Radix tree hierarchy!")

💬 Semantic Caching Guide

This guide explains how to use KacheDB's Semantic Cache Engine to intercept and cache LLM completions based on semantic intent and cosine similarity, saving 100% of GPU compute and token costs on semantic cache hits.

⚡ How Semantic Caching Works

Traditional caching requires an exact character-for-character string match ("What is KacheDB?" vs "what is kachedb?").

KacheDB's SemanticCache computes a dense vector embedding for incoming prompts and searches the in-memory SIMD vector index using normalized Cosine Similarity:

• Cache HIT: If similarity $\ge \text{threshold}$ (default 0.85), KacheDB immediately returns the cached LLM answer in $< 50\ \mu\text{s}$ without invoking the LLM.
• Cache MISS: The application queries the LLM and writes the answer to KacheDB for future semantic matches.

🚀 Synchronous Usage (SemanticCache)

from kachedb import KacheClient, SemanticCache

# 1. Connect to KacheDB
client = KacheClient(host="127.0.0.1", port=6379)

# 2. Initialize the semantic cache (auto-detects FastEmbed or SentenceTransformers)
cache = SemanticCache(
    client=client,
    index_name="customer_support_faq",
    similarity_threshold=0.85,
    ttl_seconds=86400,  # 24 hours
)

# 3. Store a Q&A pair in the semantic cache
cache.set(
    prompt="How do I change my billing address?",
    response="Go to Account Settings -> Billing -> Edit Address.",
)

# 4. Query with a semantically equivalent but differently worded prompt
query = "Where can I update my billing location?"
match = cache.get(query)

if match:
    print(f"🎯 Cache HIT! (Similarity: {match.similarity:.2f})")
    print(f"Response: {match.value}")
else:
    print("❌ Cache MISS")

⚡ Asynchronous Usage (AsyncSemanticCache)

For non-blocking asyncio inference servers (such as FastAPI, vLLM, or LiteLLM):

import asyncio
from kachedb import AsyncKacheClient, AsyncSemanticCache

async def main():
    async with AsyncKacheClient(host="127.0.0.1", port=6379) as client:
        cache = AsyncSemanticCache(
            client=client,
            index_name="async_chat_cache",
            similarity_threshold=0.88,
        )

        # Store response
        await cache.set(
            prompt="Explain quantum entanglement briefly",
            response="Quantum entanglement is a physical phenomenon where particles remain connected so that actions performed on one affect the other.",
        )

        # Retrieve asynchronously
        result = await cache.get("What is quantum entanglement in simple terms?")
        if result:
            print(f"⚡ Async Hit: {result.value}")

asyncio.run(main())

🔌 Pluggable Embedding Backends

KacheDB supports multiple embedding backends via kachedb.semantic.embedders:

• FastEmbedAdapter: Ultra-fast local ONNX Runtime embeddings (pip install fastembed).
• SentenceTransformersAdapter: HuggingFace sentence-transformers models.
• OpenAIAdapter: Remote OpenAI embedding API (text-embedding-3-small).
• CallableAdapter: Custom function wrapping any embedding model (Callable[[str], list[float]]).

OpenAI Semantic Reverse Proxy (kachedb-proxy)

The kachedb-proxy service is a lightweight, standalone reverse proxy that bridges AI developer tools (Cursor, Aider, Continue.dev, LiteLLM, LangChain, and OpenAI SDKs) with KacheDB's in-memory SIMD vector cache.

🏗️ Architecture & Packet Flow

┌─────────────────────────────────────────────────────────────────────────────┐
│                    CLIENT: Cursor / Aider / OpenAI SDK                      │
└──────────────────────────────────────┬──────────────────────────────────────┘
                                       │ POST /v1/chat/completions
                                       ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                   KACHEDB-PROXY (localhost:8080)                            │
│  1. Extract messages & canonicalize prompt text                             │
│  2. Compute local 384-dim embedding via FastEmbed/ONNX in 2-4 ms ($0 cost)  │
│  3. SIMD vector query to KacheDB daemon (127.0.0.1:6379, < 0.3 ms)          │
└───────────────────────┬─────────────────────────────┬───────────────────────┘
                        │                             │
              [ SEMANTIC HIT (>= 0.85) ]      [ SEMANTIC MISS ]
                        │                             │
                        ▼                             ▼
        ⚡ Return Cached Response (JSON/SSE)    🌐 Forward to Upstream LLM
        - Latency: < 5 ms                      - Pass client auth headers
        - Upstream Cost: $0.00                 - Stream chunks to client
        - Header: X-KacheDB-Cache: HIT         - Async VADD to KacheDB

🚀 Quickstart

1. Start KacheDB Server

./target/release/kachedb-server -p 6379

2. Launch kachedb-proxy

cargo run --release --manifest-path kachedb-proxy/Cargo.toml -- \
  --port 8080 \
  --upstream https://api.openai.com/v1 \
  --kachedb-port 6379

3. Connect Cursor / IDE

Set OpenAI Base URL in Cursor settings:

http://localhost:8080/v1

⚙️ Configuration Reference

KacheDB — Consolidated Master Benchmark Report

Date: 2026-08-17
Engine: KacheDB v0.1.0
Status: ✅ All 4 Phases Complete & Verified

1. Executive Summary

KacheDB is an in-memory storage engine designed from scratch in Rust to address the memory bottlenecks of modern AI and high-concurrency microservices:

1. Sub-4 ns Memory Allocation: Replaces runtime malloc/free with 64-byte aligned 2 MB Megaslab arenas, achieving 3.93 ns allocation latency regardless of slot size (128 B to 256 KB).
2. L1 Cache-Speed Point Queries: SIMD-probed Swiss Table hash index delivers 3.15 ns lookup hit latency with lock-free S3-FIFO eviction flags.
3. ~10,000× TTFT Speedup for LLM KV-Cache: Hierarchical &[u32] Radix Prefix Tree matches a 1,024-token sequence in 2.61 µs, skipping costly GPU attention prefill.
4. 15.04 Million msgs/sec Zero-Copy IPC: POSIX Shared Memory (/dev/shm) lock-free SPSC ring buffers stream tensor descriptors across processes in 66.47 ns per message with zero serialization and zero memory copies.
5. 10.2 Million QPS per Core: Ingests, parses, executes, and encodes standard Redis RESP commands over TCP in 97.63 ns end-to-end.

2. Test Environment

3. Detailed Subsystem Benchmark Matrix

Phase 0: Memory Allocation & Swiss Hash Table

Phase 1: LLM Token Radix Tree & POSIX Shared Memory

Phase 2: Wire Protocol & Asynchronous TCP Pipeline

Phase 3: Multi-Core Server Daemon & Python Bindings

4. Reproducing Benchmarks

All benchmarks can be reproduced locally with:

# Run all crate micro-benchmarks
cargo bench --workspace

# Or run individual crate benchmarks
cargo bench -p kachedb-core
cargo bench -p kachedb-hash
cargo bench -p kachedb-radix
cargo bench -p kachedb-shm
cargo bench -p kachedb-proto-resp
cargo bench -p kachedb-net

KacheDB: Standardized Benchmark Reproduction Protocol

Document Version: 1.0
Target Engine: KacheDB v0.1.0+

This guide provides the exact methodology, hardware parameters, and commands to independently reproduce the published performance numbers for KacheDB.

1. Hardware & Environment Reference

To achieve identical peak throughput numbers, tests should be run on a dedicated bare-metal or high-priority virtualized host:

• CPU: Multi-core modern x86_64 or ARM64 (e.g. Apple Silicon M-series, AMD EPYC 9004, or Intel Xeon 4th Gen)
• Linux Kernel: Linux 6.8+ (for SQPOLL kernel thread pooling and io_uring multishot socket support)
• RAM: Minimum 16 GB DDR4/DDR5
• IPC Mount: /dev/shm (POSIX Shared Memory mounted as tmpfs)

2. One-Command Turnkey Reproduction (Docker Linux)

Run the full end-to-end multi-core benchmark in an isolated Linux container with kernel io_uring + SQPOLL:

# From the repository root:
make benchmark-reproduce

This runs the automated suite inside Docker with --privileged and --ipc host, measuring:

1. PING Throughput & Latencies (50 concurrent clients, pipeline 16)
2. SET Throughput & Latencies (50 concurrent clients, pipeline 16, 64-byte payload)
3. GET Throughput & Latencies (50 concurrent clients, pipeline 16)

3. Local Micro-Benchmarks (Criterion.rs)

Run the statistical Criterion micro-benchmarks with nanosecond-level resolution:

# Memory Allocator & Pool Benchmarks
cargo bench -p kachedb-core

# Swiss Table Point Index Benchmarks
cargo bench -p kachedb-hash

# Full Workspace Benchmarks
cargo bench --workspace

Measured Baselines:

• arena::allocate (128 B): ~3.84 ns
• pool::allocate+deallocate: ~5.40 ns
• swiss_table::lookup hit: ~3.09 ns
• radix::lookup (1,024 tokens): ~2.45 µs

4. Live Multi-Core TCP Benchmarks (kachedb-bench)

Start the multi-worker server and load generator manually:

# Build release binaries
cargo build --release --workspace

# Start server on 4 dedicated cores
./target/release/kachedb-server -p 6379 -w 4 &
SERVER_PID=$!
sleep 1

# Run PING benchmark
./target/release/kachedb-bench -p 6379 -n 100000 -c 50 --pipeline 16 --command PING

# Run SET benchmark
./target/release/kachedb-bench -p 6379 -n 100000 -c 50 --pipeline 16 --command SET

# Run GET benchmark
./target/release/kachedb-bench -p 6379 -n 100000 -c 50 --pipeline 16 --command GET

# Terminate server
kill $SERVER_PID