CRDT-Based Offline-First Field Inspection App for EU’s Digital Building Logbook

Architecture Blueprint & Data Orchestration for CRDT-Based Offline-First Field Inspection Systems

The architectural foundation of a CRDT (Conflict-free Replicated Data Type) based offline-first field inspection application is fundamentally distinct from traditional client-server models. For the EU’s Digital Building Logbook (DBL) initiative, the system must reconcile data across hundreds of thousands of inspectors operating in basements, rural construction sites, and high-rise buildings with intermittent connectivity. The core engineering challenge is not merely storing data locally but guaranteeing eventual consistency without a central arbiter.

Core System Engineering & Data Synchronization Layer

At the heart of this architecture lies the Synchronization Engine, a peer-to-peer data reconciliation layer that operates on a Last-Writer-Wins Register (LWW-Register) combined with a Merkle Clock for conflict resolution. Unlike operational transformation (OT) used in collaborative editors like Google Docs, CRDTs in this context are particularly suited for inspection data because they tolerate long disconnections and require no central server to order operations.

Architecture Components:

• Local First Database: SQLite with embedded CRDT extensions (e.g., using sql.js or react-native-quick-sqlite with a CRDT layer on top). The local database acts as the single source of truth.
• Sync Protocol: A delta-based synchronization protocol using Hash Graph DAGs (Directed Acyclic Graphs) rather than sequential state vectors. This enables partial syncs where only changed inspection fields (e.g., updated crack measurements, new photos, modified timestamps) are transmitted.
• Conflict Resolution Strategy: For the DBL, a Semantic CRDT approach is critical. A simple last-writer-wins is insufficient because inspection data has semantic dependencies. For example, a building element’s “condition score” and “remediation note” must be atomically consistent. The system implements Observed-Remove Set (OR-Set) for list-type fields (e.g., “defects found,” “materials used”) and Multi-Value Register (MVR) for scalar fields with dependency tracking.

Failure Modes & Input/Output Specifications:

| System Input | Processing Logic | System Output | Failure Mode | Recovery Strategy | |-------------|------------------|---------------|--------------|-------------------| | Inspector adds crack width (2.3mm) to wall element #445 | CRDT merge: create LWW-Register entry with vector clock | Persisted locally; queued for sync | Network unavailable; local write succeeds | On reconnect, sync engine sends delta; conflicts auto-merged via semantic priority rules | | Two inspectors simultaneously mark “asbestos detected” and “no asbestos” on same element | Merger evaluates timestamps AND inspector role (certified vs uncertified) | One value marked “pending review” with conflict flag | Human error conflict cannot be auto-resolved | Flagged in app UI; human-in-loop review required before DBL submission | | Photo capture of structural crack (high-res, 12MB) | Compression engine reduces to 2MB with perceptual hash; CRDT stores hash + metadata | Thumbnail immediately available; full image synced as background task | Storage limit on device exceeded | LRU eviction policy with re-fetch from peer nodes on sync |

Comparative Engineering Stack Evaluation

For a CRDT-based inspection app targeting the EU DBL, the technology stack must balance offline resilience, battery efficiency, and eventual consistency guarantees. The following table compares three viable stacks:

| Layer | Stack Option A (Web/Progressive) | Stack Option B (Native Mobile) | Stack Option C (Hybrid with Embedded WASM) | |-------|-----------------------------------|--------------------------------|---------------------------------------------| | CRDT Engine | Automerge (in-browser WASM) | Y-CRDT (via y-websocket + native bindings) | Custom Rust CRDT compiled to WASM | | Local Database | IndexedDB with Dexie wrapper | SQLite (WAL mode, synchronous=OFF) | SQLite with CRDT indexes (C library via FFI) | | Sync Protocol | HTTP/2 Server-Sent Events (SSE) | WebSocket with mTLS + delta compression | WebRTC DataChannels for P2P mesh sync | | State Management | Zustand with persisted middleware | Riverpod with StateNotifier for offline queues | Akka.js for actor-based CRDT management | | Crypto Layer | Web Crypto API (AES-GCM) | Android KeyStore + iOS Keychain | WebAssembly crypto (libsodium port) | | Build System | Vite + TypeScript + Capabilities API | Kotlin Multiplatform (KMP) + SwiftUI | Tauri (Rust backend, React frontend) |

Recommendation for EU DBL Compliance: Stack Option C (Hybrid with WASM) provides the strongest guarantees for data integrity under GDPR and the EU Data Act. The Rust CRDT engine, compiled to WASM, runs identically on iOS, Android, and desktop, eliminating divergent behavior across platforms. The WebRTC P2P mesh enables inspector-to-inspector sync even when both are offline, critical for team inspections in remote areas.

Core Systems Design: Offline-First Inspection Workflow

The system is designed around an Offline Command Pattern where every user action generates a CRDT operation in the local store before any network activity. The five-phase lifecycle mirrors the physical inspection process:

Phase 1: Inspection Initialization

• Download “Empty” CRDT Replica from the DBL authority or building owner’s server
• Initialize local SQLite with schema version, building identifier, and inspector’s public key
• Generate a unique Session ID using a Bloom-filter based counter to prevent UUID collisions across offline replicas

Phase 2: Data Collection (Offline Mode)

• Each field inspection action (photo capture, measurement, text annotation) creates a CRDT Mutator operation
• Mutators are appended to an append-only log (similar to LSM-tree design) on device storage
• Snapshot Compression: Every 100 operations, the system compresses the log into a CRDT Snapshot using Merkle tree hashing, reducing storage from linear to logarithmic growth

Phase 3: Conflict Resolution & Semantic Validation

• Inspectors can work on overlapping building sections; the system uses Causal Contexts (vector clocks with dependencies) to order operations logically
• A “defect severity” field uses a Lexicographic Priority where “critical” > “major” > “minor” regardless of timestamp
• When two inspectors rate the same element differently, the system does not auto-merge but creates a Conflict Record with both values and a dependency graph of related observations

Phase 4: Synchronization Engine

• Upon reconnection, the sync engine performs a Three-Way Merge:
  1. Local state (current device)
  2. Remote state (server or peer)
  3. Base state (last known common ancestor from Merkle Clock)
• Delta compression via Roaring Bitmaps identifies changed data blocks (reduces sync payload by 80-90% compared to full-state transfer)
• Bandwidth-aware throttling: On cellular connections, the engine prioritizes textual data over images; on WiFi, full sync proceeds

Phase 5: Submission & Locking

• Once the building owner or authority accepts the inspection, the CRDT state is finalized into an immutable record in the DBL
• The final state uses a Content-Addressable Storage (CAS) identifier (SHA-256 hash of the entire inspection graph), ensuring tamper-proof record
• Post-finalization, any new operations require creating a new version of the building log, triggering a formal amendment process

Comparative Database Systems Design

The database architecture must support both offline ACID transactions for local work and eventual consistency for distributed sync. Traditional OLTP databases (PostgreSQL, MySQL) are unsuitable due to their reliance on continuous connection. The following comparison table evaluates database engines for the CRDT layer:

| Database System | CRDT Support Method | Sync Efficiency | Storage Overhead | Query Capability During Offline | GDPR Compliance for Local Data | |----------------|---------------------|-----------------|------------------|--------------------------------|--------------------------------| | SQLite + Custom CRDT Extensions | User-defined functions in C; OR-Set implemented via triggers | High (delta sync via row-level change tracking) | Low (only operations stored; snapshots optional) | Full SQL (SELECT, JOIN, GROUP BY) | Full (data never leaves device without consent) | | Couchbase Lite | Built-in conflict resolution (last-writer-wins, custom handlers) | Very High (built-in sync gateway for delta push/pull) | Medium (document-based storage with revision trees) | Full (N1QL queries) | Partial (sync gateway metadata includes device info) | | Realm (MongoDB Atlas Device Sync) | Automatic merge via partition key + object ID | High (AWS/Azure global sync infrastructure) | Medium (schema-based; CRDT operations stored as revisions) | Full (local queries on realm objects) | Medium (requires self-hosted sync for EU data residency) | | Datomic (Peer Server Model) | CRDT-like via datom transactions + as-of queries | Low (requires continuous connection for transaction processing) | Very High (full history stored) | Limited (requires replica for query) | High (data stored in local Datomic transactor) |

Optimal Configuration for EU DBL: SQLite with a custom CRDT layer written in Rust (compiled to WASM for cross-platform consistency). This provides the lowest possible storage overhead (critical for mobile devices with limited flash) and guarantees that all data remains on the device until sync. The custom layer implements Hybrid Logical Clocks (HLCs) that combine physical timestamps (accurate to microsecond) with logical counters, eliminating the need for GPS or NTP synchronization across disconnected inspectors.

Configuration Templates for Deployment

The following JSON schema defines the CRDT operation format that every inspection action must conform to. This schema is enforced at the Rust/WASM boundary to guarantee consistency across all client implementations:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "CRDT Inspection Operation",
  "type": "object",
  "required": ["op_id", "session_id", "hybrid_clock", "crud_type", "element_path", "value"],
  "properties": {
    "op_id": {
      "type": "string",
      "pattern": "^[a-f0-9]{32}$",
      "description": "MD5 hash of (session_id + element_path + hybrid_clock) to ensure global uniqueness"
    },
    "session_id": {
      "type": "string",
      "pattern": "^[a-f0-9]{64}$",
      "description": "SHA-256 of inspector_id + building_id + inspection_date"
    },
    "hybrid_clock": {
      "type": "object",
      "required": ["physical_time_micros", "logical_counter", "node_id"],
      "properties": {
        "physical_time_micros": {"type": "integer", "minimum": 1700000000000000},
        "logical_counter": {"type": "integer", "minimum": 0},
        "node_id": {"type": "string", "pattern": "^[a-zA-Z0-9_]{1,64}$"}
      }
    },
    "crud_type": {
      "type": "string",
      "enum": ["CREATE", "UPDATE", "DELETE", "ANNOTATE"]
    },
    "element_path": {
      "type": "string",
      "pattern": "^/buildings/\\d+/floors/\\d+/rooms/\\d+/elements/\\d+(/properties)?$",
      "description": "Hierarchical path in the DBL structure"
    },
    "value": {
      "oneOf": [
        {"$ref": "#/$defs/scalar_value"},
        {"$ref": "#/$defs/map_value"},
        {"$ref": "#/$defs/list_value"}
      ]
    },
    "mutator_type": {
      "type": "string",
      "enum": ["LWW_REGISTER", "OR_SET", "MVR", "COUNTER"],
      "description": "CRDT type determining merge semantics"
    }
  },
  "$defs": {
    "scalar_value": {
      "type": "object",
      "required": ["type", "data"],
      "properties": {
        "type": {"enum": ["string", "number", "boolean", "null"]},
        "data": {"type": ["string", "number", "boolean", "null"]}
      }
    },
    "map_value": {
      "type": "object",
      "patternProperties": {
        "^[a-zA-Z_][a-zA-Z0-9_]*$": {"$ref": "#/$defs/scalar_value"}
      },
      "minProperties": 1
    },
    "list_value": {
      "type": "array",
      "items": {"$ref": "#/$defs/scalar_value"},
      "minItems": 1
    }
  }
}

The YAML configuration for the sync gateway (if using a Kubernetes-backed server cluster) demonstrates how to configure delta compression and peer discovery:

apiVersion: v1
kind: ConfigMap
metadata:
  name: crdt-sync-gateway-config
data:
  sync-gateway.yaml: |
    # Sync Gateway Configuration for EU DBL CRDT Backend
    # Ensure data residency within EU boundaries
    
    interface:
      binding: 0.0.0.0:8443
      tls:
        enabled: true
        cert_file: /etc/certs/tls.crt
        key_file: /etc/certs/tls.key
        mutual_tls: required
        client_ca_file: /etc/certs/ca.crt
    
    database:
      name: eu_dbl_inspections
      bucket_name: crdt-snapshots
      # Use Merkle tree for differential sync
      delta_sync:
        enabled: true
        max_delta_size_mb: 50
        snapshot_interval_seconds: 3600
        compression_algorithm: zstd
        compression_level: 3
      
      # Conflict resolution: Semantic CRDT
      conflict_resolution:
        type: custom
        script_path: /etc/crdt/merge_rules.lua
        allow_list_conflicts: false
        conflict_log_size: 10000
      
      # Peer discovery for P2P sync
      peer_discovery:
        type: dns_srv
        domain: p2p.dbl-inspections.eu
        ttl_seconds: 300
        heartbeat_interval: 30
    
    replication:
      # Push/Pull configuration per inspector
      pull:
        filter: by_channel
        channels:
          - session-{{.SessionID}}
        batch_size: 500
        continuous: true
        
      push:
        allowed_channels:
          - session-{{.SessionID}}
        max_retries: 10
        backoff_initial_ms: 5000

Non-Shifting Technical Principles for Long-Term Maintainability

Principle 1: Immutable Log + Compressible State Every inspection action generates an immutable CRDT operation. The system never deletes operations—it only adds new ones to the append-only log. This provides full auditability under EU’s GDPR right to explanation (Article 22). The state is derived by replaying operations through the CRDT merging function. For storage efficiency, periodic snapshots compress the log into a checkpoint state, but the raw log remains for legal compliance.

Principle 2: Peer-as-Server Architecture The system design eliminates single points of failure by allowing any device to temporarily act as a sync server. When an inspector has connectivity and other inspectors in the same building do not, their device can relay operations via a mesh network (using WebRTC or BLE for short-range). This is critical for basements, tunnels, and other areas where only one inspector’s device has signal.

Principle 3: Schema Evolution Through CRDT Merges As the EU updates the DBL schema (adding new mandatory fields like “embodied carbon” or “circular material index”), the CRDT layer handles schema drift seamlessly. Newer operations use the new fields; older operations lack them. The merge function treats missing fields as null values (for LWW-Register) or empty sets (for OR-Sets). Backward compatibility is guaranteed at the CRDT algebraic level, not through database migrations.

Principle 4: Verifiable Offline Proof For legal acceptance of offline-collected data, the system generates a Merkle Proof for each batch of operations synchronized. The proof includes the root hash of all operations in the session, timestamps, and inspector’s digital signature. This proof is stored on the DBL server and can be independently verified without trusting the inspector’s device. It ensures that no operations were inserted, deleted, or reordered after the inspector signed the batch.

Principle 5: Energy Proportional Sync Mobile devices have limited battery. The sync engine implements Proportional Sync where the synchronization intensity scales with available battery, connectivity quality, and data urgency. Critical defects (marked by the inspector as “urgent”) sync immediately via any available connection. Bulk data (like full inspection logs) sync only when battery > 50% and on WiFi. The system uses Adaptive Bitrate for image uploads, choosing resolution based on connection speed.

Long-Term Best Practices for CRDT-Based Inspection Systems

• Hybrid Logical Clocks over NTP: Do not rely on Network Time Protocol for ordering offline events. Implement Hybrid Logical Clocks (HLCs) that combine the device’s last known time with monotonic counters. This prevents ordering errors when two inspectors work in different time zones or have unsynchronized clocks.
• Reconciliation DAGs Instead of State Vectors: Traditional CRDT systems use state vectors (lists of operation counts per node). In inspection systems with thousands of nodes, this scales poorly. Use Merkle DAGs where each node maintains a hash chain of operations. Two nodes can reconcile by comparing root hashes; if different, they walk the DAG to find divergent branches.
• Semantic Conflict Marking: Never auto-merge inspection data that has legal implications (e.g., safety ratings, material declarations). Instead, mark conflicts explicitly in the CRDT state and trigger human review. The system should provide diff visualizations showing both conflicting versions and allowing the building owner or certified inspector to resolve.
• Forward Error Correction for Sync: Wireless sync sessions frequently drop mid-transfer. Implement erasure coding (e.g., Reed-Solomon codes) on sync payloads so that the receiver can reconstruct the full data from partial packets, reducing retransmission overhead by up to 40%.
• Cold Storage Archival: After inspections are accepted into the DBL, the CRDT operation logs can be compressed into immutable CRDT Archives using zstd compression with dictionary training. For a typical year of city-wide inspections (500,000 inspections, 10,000 operations each), the compressed archive size is approximately 50GB—storable on a single archival-grade SSD for EU regulatory compliance.

The Intelligent-Ps SaaS Solutions platform provides the foundational CRDT engine and sync infrastructure for this architecture, enabling rapid deployment of offline-first inspection applications that fully comply with EU Digital Building Logbook requirements. Their Rust-based CRDT core, pre-configured for HLC and semantic conflict resolution, reduces development time by eliminating the need to implement complex merge semantics from scratch.