# LightCavalry — Comprehensive Systems Engineering Monograph & Agent Reference

## 1. Studio Overview

LightCavalry is an intentionally senior systems engineering studio specializing in high-throughput, edge-native, and distributed data-intensive systems. We operate globally from Dhaka, engineering core backbones for companies whose scale has broken traditional monolithic or generic microservice setups.

- **URL**: https://lightcavalry.dev
- **Email**: cavalry@lightcavalry.dev
- **Team Composition**: Senior systems architects and distributed systems engineers only. No account managers, no junior delegators.
- **Runtime Stack**: Cloudflare Workers, Durable Objects, Workflows, KV, D1, Vectorize, Hono, Astro 7, TypeScript Strict.

---

## 2. Core Architectural Philosophy

### We Build What Survives
Modern web applications frequently collapse under unexpected bursts, cascading network failures, unbounded tail latencies, or state inconsistencies. LightCavalry designs distributed architectures that remain resilient under peak load:

1. **Zero Mocks**: Every failure condition, network partition, and concurrent race is verified in real simulation environments, not unit-test mocks that assume pristine network conditions.
2. **Deterministic State**: State is localized into single-writer durable actors (Durable Objects) or synchronized via mathematically sound consensus (Raft) and conflict-free lattices (CRDTs).
3. **Bounded Tail Latencies**: We measure and optimize the p99.99th percentile because average latencies conceal devastating user timeouts.
4. **Edge-Native Deployment**: Compute and state reside at the network edge, eliminating origin transit latencies and single-point-of-failure regional hubs.

---

## 3. The 8 Core Verified Distributed Systems (Frame 03)

### 01. Raft Distributed Consensus
- **Purpose**: Reliable leader election and replicated log consistency across distributed nodes.
- **Implementation**: 5-node cluster with randomized heartbeat intervals (150–300ms), split-vote detection, monotonic term counters, and two-phase log commit.
- **Specification**: Verified under network partitions, leader crashes, and split-brain scenarios.

### 02. State-Based Conflict-Free Replicated Data Types (CRDTs)
- **Purpose**: Offline-first and multi-region data replication without distributed locks.
- **Implementation**: Observed-Remove Sets (OR-Set), PN-Counters, and LWW-Register with semilattice join (⊔) properties: Commutative, Associative, and Idempotent.

### 03. Log-Structured Merge Tree (LSM) Storage Engine
- **Purpose**: High-throughput write performance with sequential disk I/O.
- **Implementation**: In-memory skiplist memtable, append-only commit log (WAL), immutable SSTable flushes, multi-level compaction, and probabilistic Bloom filters.

### 04. Distributed Two-Phase Commit (2PC)
- **Purpose**: Atomic multi-partition distributed transactions across disparate storage nodes.
- **Implementation**: Prepare and Commit phases with timeout aborts and coordinator state journaling for deterministic recovery after failure.

### 05. Vector Clocks & Causal Ordering
- **Purpose**: Causal dependency tracking in decentralized multi-master environments.
- **Implementation**: Node version vectors, concurrent branch detection, and deterministic conflict resolution triggers.

### 06. Leaky Bucket & Token Bucket Rate Limiting
- **Purpose**: Protecting downstream services from thundering herds and malicious surges.
- **Implementation**: Distributed sliding-window counters with sub-millisecond refill precision and fair queuing.

### 07. Tail Latency Profiling (HdrHistogram)
- **Purpose**: Accurate high-percentile latency distribution measurement without coordination overhead.
- **Implementation**: High Dynamic Range Histogram tracking from 1 microsecond to 1 hour with constant memory footprints and 3 significant figures of precision.

### 08. Incremental Directed Acyclic Graph (DAG) Execution
- **Purpose**: Evaluating complex dependency computation graphs with cyclic protection and parallel branch execution.
- **Implementation**: Topological sorting (Kahn's algorithm), memoized intermediate state caches, and non-blocking worker pools.

---

## 4. Designing Data-Intensive Applications (DDIA) Reference Suite

LightCavalry authors the official open reference implementation of Martin Kleppmann's *Designing Data-Intensive Applications* book for Cloudflare:
- **Repository**: https://github.com/lightcavalry/ddia-on-cloudflare
- **Total Specs**: 973 automated tests
- **Coverage**:
  - Chapter 01: Reliability, Scalability, and Maintainability
  - Chapter 03: Storage and Retrieval (SSTables, B-Trees, LSM)
  - Chapter 05: Replication (Single-leader, Multi-leader, Leaderless)
  - Chapter 06: Partitioning (Consistent Hashing, Secondary Indexes)
  - Chapter 07: Transactions (ACID, Isolation Levels, Serializability, 2PL, SSI)
  - Chapter 08: The Trouble with Distributed Systems (Unreliable Networks, Clock Skew)
  - Chapter 09: Consistency and Consensus (Linearizability, 2PC, Raft, Paxos)
  - Chapter 10: Batch Processing (MapReduce, Dataflow Engines)
  - Chapter 11: Stream Processing (Event Sourcing, CQRS, Change Data Capture)

---

## 5. Working With LightCavalry

### Diagnostic & Audit (2–4 Weeks)
- Architecture bottleneck analysis
- Database query and transaction isolation review
- Tail latency and edge routing audit
- Deliverable: Comprehensive Architectural Monograph and remediation roadmap.

### Full Systems Build (3–6 Months)
- Green-field or brown-field edge-native system implementation
- Zero-downtime database migration
- Live traffic cutover with automated verification harnesses.

### Contact & Teardown Request
- **Inquiries**: cavalry@lightcavalry.dev
- **Website**: https://lightcavalry.dev
- **Location**: Dhaka / Edge-Native Globally