marmot

Marmot v2

What & Why?

Marmot v2 is a leaderless, distributed SQLite replication system built on a gossip-based protocol with distributed transactions and eventual consistency.

Key Features:

• Leaderless Architecture: No single point of failure - any node can accept writes
• MySQL Protocol Compatible: Connect with any MySQL client (DBeaver, MySQL Workbench, mysql CLI)
• WordPress Compatible: Full MySQL function support for running distributed WordPress
• Distributed Transactions: Percolator-style write intents with conflict detection
• Multi-Database Support: Create and manage multiple databases per cluster
• DDL Replication: Distributed schema changes with automatic idempotency and cluster-wide locking
• Replicated Bulk Load: LOAD DATA LOCAL INFILE with distributed commit semantics
• Production-Ready SQL Parser: Powered by rqlite/sql AST parser for MySQL→SQLite transpilation
• CDC-Based Replication: Row-level change data capture for consistent replication
• Built-in Vector Search: Local IVF/PQ vector indexes for RAG workloads, with live CRUD and exact rerank from the base table

Why Marmot?

The Problem with Traditional Replication

MySQL active-active requires careful setup of replication, conflict avoidance, and monitoring. Failover needs manual intervention. Split-brain scenarios demand operational expertise. This complexity doesn't scale to edge deployments.

Marmot's Approach

• Zero operational overhead: Automatic recovery from split-brain via eventual consistency + anti-entropy
• No leader election: Any node accepts writes, no failover coordination needed
• Direct SQLite access: Clients can read the local SQLite file directly for maximum performance
• Tunable consistency: Choose ONE/QUORUM/ALL per your latency vs durability needs

Why MySQL Protocol?

• Ecosystem compatibility - existing drivers, ORMs, GUI tools work out-of-box
• Battle-tested wire protocol implementations
• Run real applications like WordPress without modification

Ideal Use Cases

Marmot excels at read-heavy edge scenarios:

Use Case How Marmot Helps Distributed WordPress Multi-region WordPress with replicated database Lambda/Edge sidecars Lightweight regional SQLite replicas, local reads Edge vector databases Built-in ANN search + distributed embeddings with local query Regional config servers Fast local config reads, replicated writes Product catalogs Geo-distributed catalog data, eventual sync

When to Consider Alternatives

• Strong serializability required → CockroachDB, Spanner
• Single-region high-throughput → PostgreSQL, MySQL directly
• Large datasets (>100GB) → Sharded solutions

Quick Start

# Start a single-node cluster
./marmot-v2

# Or run as daemon (background)
./marmot-v2 -daemon -pid-file=/tmp/marmot/marmot.pid

# Connect with MySQL client
mysql -h localhost -P 3306 -u root

# Or use DBeaver, MySQL Workbench, etc.

Testing Replication

…

WordPress Support

Marmot can run distributed WordPress with full database replication across nodes. Each WordPress instance connects to its local Marmot node, and all database changes replicate automatically.

MySQL Compatibility for WordPress

Marmot implements MySQL functions required by WordPress:

Category Functions Date/Time NOW, CURDATE, DATE_FORMAT, UNIX_TIMESTAMP, DATEDIFF, YEAR, MONTH, DAY, etc. String CONCAT_WS, SUBSTRING_INDEX, FIND_IN_SET, LPAD, RPAD, etc. Math/Hash RAND, MD5, SHA1, SHA2, POW, etc. DML ON DUPLICATE KEY UPDATE (transformed to SQLite ON CONFLICT)

Quick Start: 3-Node WordPress Cluster

cd examples/wordpress-cluster
./run.sh up

This starts:

• 3 Marmot nodes with QUORUM write consistency
• 3 WordPress instances each connected to its local Marmot node

┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│ WordPress-1 │  │ WordPress-2 │  │ WordPress-3 │
│ :9101       │  │ :9102       │  │ :9103       │
└──────┬──────┘  └──────┬──────┘  └──────┬──────┘
       ▼                ▼                ▼
┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│  Marmot-1   │◄─┤  Marmot-2   │◄─┤  Marmot-3   │
│ MySQL: 9191 │  │ MySQL: 9192 │  │ MySQL: 9193 │
└─────────────┘  └─────────────┘  └─────────────┘
       └──────────────┴──────────────┘
              QUORUM Replication

Test it:

1. Open http://localhost:9101 - complete WordPress installation
2. Open http://localhost:9102 or http://localhost:9103
3. See your content replicated across all nodes!

Commands:

./run.sh status   # Check cluster health
./run.sh logs-m   # Marmot logs only
./run.sh logs-wp  # WordPress logs only
./run.sh down     # Stop cluster

Production Considerations for WordPress

• Media uploads: Use S3/NFS for shared media storage (files not replicated by Marmot)
• Sessions: Use Redis or database sessions for sticky-session-free load balancing
• Caching: Each node can use local object cache (Redis/Memcached per region)

Architecture

Marmot v2 uses a fundamentally different architecture from other SQLite replication solutions:

vs. rqlite/dqlite/LiteFS:

• ❌ They require a primary node for all writes
• ✅ Marmot allows writes on any node
• ❌ They use leader election (Raft)
• ✅ Marmot uses gossip protocol (no leader)
• ❌ They require proxy layer or page-level interception
• ✅ Marmot uses MySQL protocol for direct database access

How It Works:

1. Write Coordination: 2PC (Two-Phase Commit) with configurable consistency (ONE, QUORUM, ALL)
2. Conflict Resolution: Last-Write-Wins (LWW) with HLC timestamps
3. Cluster Membership: SWIM-style gossip with failure detection
4. Data Replication: Full database replication - all nodes receive all data
5. DDL Replication: Cluster-wide schema changes with automatic idempotency

Comparison with Alternatives

Aspect Marmot MySQL Active-Active rqlite dqlite TiDB Leader None None (but complex) Yes (Raft) Yes (Raft) Yes (Raft) Failover Automatic Manual intervention Automatic Automatic Automatic Split-brain recovery Automatic (anti-entropy) Manual N/A (leader-based) N/A (leader-based) N/A Consistency Tunable (ONE/QUORUM/ALL) Serializable Tunabale (ONE/QUORUM/Linearizable) Strong Strong Direct file read ✅ SQLite file ❌ ✅ SQLite file ❌ ❌ JS-safe AUTO_INCREMENT ✅ Compact mode (53-bit) N/A N/A ❌ 64-bit breaks JS Edge-friendly ✅ Lightweight ❌ Heavy ✅ Lightweight ⚠️ Moderate ❌ Heavy Operational complexity Low High Low Low High

DDL Replication

Marmot v2 supports distributed DDL (Data Definition Language) replication without requiring master election:

How It Works

1. Cluster-Wide Locking: Each DDL operation acquires a distributed lock per database (default: 30-second lease)

   • Prevents concurrent schema changes on the same database
   • Locks automatically expire if a node crashes
   • Different databases can have concurrent DDL operations

2. Automatic Idempotency: DDL statements are automatically rewritten for safe replay

   CREATE TABLE users (id INT)
   → CREATE TABLE IF NOT EXISTS users (id INT)
   
   DROP TABLE users
   → DROP TABLE IF EXISTS users
   

3. Schema Version Tracking: Each database maintains a schema version counter

   • Incremented on every DDL operation
   • Exchanged via gossip protocol for drift detection
   • Used by delta sync to validate transaction applicability

4. Quorum-Based Replication: DDL replicates like DML through the same 2PC mechanism

   • No special master node needed
   • Works with existing consistency levels (QUORUM, ALL, etc.)

Configuration

[ddl]
# DDL lock lease duration (seconds)
lock_lease_seconds = 30

# Automatically rewrite DDL for idempotency
enable_idempotent = true

Best Practices

• ✅ Do: Execute DDL from a single connection/node at a time
• ✅ Do: Use qualified table names (mydb.users instead of users)
• ⚠️ Caution: ALTER TABLE is less idempotent - avoid replaying failed ALTER operations
• ❌ Don't: Run concurrent DDL on the same database from multiple nodes

CDC-Based Replication

Marmot v2 uses Change Data Capture (CDC) for replication instead of SQL statement replay:

How It Works

1. Row-Level Capture: Instead of replicating SQL statements, Marmot captures the actual row data changes (INSERT/UPDATE/DELETE)
2. Canonical Binary Row Format: DML changes are encoded once as msgpack EncodedCapturedRow records and stored in Marmot's CDC segment log
3. Minimal Wire Payload: DML replication sends the encoded row blob plus statement metadata; decoded before/after maps are local apply-time state only
4. Deterministic Application: Row data is applied directly to the target database, avoiding parsing ambiguities

Benefits

• Consistency: Same row data applied everywhere, no SQL parsing differences
• Performance: Binary format is more efficient than SQL text
• Reliability: No issues with SQL syntax variations between MySQL and SQLite
• Lower Write Amplification: Row payloads are not duplicated into Pebble intent keys; Pebble tracks transaction metadata and row locks while the CDC segment log stores DML bytes
• Atomic CDC Publication: DML bytes become visible through /cdc_manifest/{txnID} only after their segment ranges are known; recovery validates and truncates segment tails at record boundaries

Row Key Extraction

For UPDATE and DELETE operations, Marmot automatically extracts row keys:

• Uses PRIMARY KEY columns when available
• Falls back to ROWID for tables without explicit primary key
• Handles composite primary keys correctly

CDC Publisher

Marmot can publish CDC events to external messaging systems, enabling real-time data pipelines, analytics, and event-driven architectures. Events follow the Debezium specification for maximum compatibility with existing CDC tooling.

Features

• Debezium-Compatible Format: Events conform to the Debezium event structure, compatible with Kafka Connect, Flink, Spark, and other CDC consumers
• Multi-Sink Support: Publish to multiple destinations simultaneously (Kafka, NATS)
• Glob-Based Filtering: Filter which tables and databases to publish
• Automatic Retry: Exponential backoff with configurable limits
• Persistent Cursors: Survives restarts without losing position

Configuration

…

Event Format

Events follow the Debezium envelope structure:

{
  "schema": { ... },
  "payload": {
    "before": null,
    "after": {"id": 1, "name": "alice", "email": "[email protected]"},
    "source": {
      "version": "2.9.0-beta",
      "connector": "marmot",
      "name": "marmot",
      "ts_ms": 1702500000000,
      "db": "myapp",
      "table": "users"
    },
    "op": "c",
    "ts_ms": 1702500000000
  }
}

Operation Types (per Debezium spec):

Operation op before after INSERT c (create) null row data UPDATE u (update) old row new row DELETE d (delete) old row null

Topic Naming

Topics follow the pattern: {topic_prefix}.{database}.{table}

Examples:

• marmot.cdc.myapp.users
• marmot.cdc.myapp.orders