Ayder: HTTP-Native Durable Event Log Challenges Complex Systems

A new durable event log system has emerged that challenges traditional architectural complexity. Ayder represents a fundamental shift toward simplicity, delivering enterprise-grade persistence through a single binary written in C.

The system's core philosophy centers on HTTP-native design, eliminating the need for complex client infrastructure. Instead of requiring JVM runtime environments, ZooKeeper coordination, or heavy client libraries, Ayder accepts standard HTTP requests—meaning developers can interact with it using nothing more sophisticated than curl.

Initial performance metrics reveal impressive capabilities: sustained throughput of 50,000 messages per second on a three-node Raft cluster, with client P99 latency measuring just 3.46 milliseconds. Perhaps most compelling is the crash recovery demonstration—after an unclean SIGKILL termination, the system restarts and verifies data integrity for approximately 8 million offsets in under a minute.

Ayder's design philosophy represents a deliberate departure from contemporary distributed system patterns. Built entirely in C, the system prioritizes minimalism and operational transparency over feature breadth.

The single-binary deployment model means operators manage one executable file rather than coordinating multiple services. This approach reduces deployment complexity, eliminates dependency conflicts, and streamlines upgrade procedures. The binary contains all necessary logic for persistence, consensus, and HTTP interface handling.

HTTP-native architecture fundamentally changes how applications integrate with the event log. Rather than importing specialized SDKs or managing connection pools to coordination services, developers issue standard HTTP requests. This approach offers several advantages:

• Universal client compatibility across any programming language
• Reduced attack surface through standardized protocols
• Simplified debugging using familiar HTTP tooling
• Lower memory overhead without thick client libraries

The system uses Raft consensus across a three-node cluster to ensure durability and availability. Write operations synchronize across a majority of nodes before acknowledging completion, providing strong consistency guarantees even during network partitions or node failures.

Performance validation occurred on a three-node Raft cluster operating across a real network environment—not simulated or isolated test conditions. The benchmark scenario used sync-majority writes with 64-byte payloads, representing typical event message sizes in streaming architectures.

Results demonstrate sustained throughput of approximately 50,000 messages per second under continuous load at 50,000 requests per second using the wrk2 benchmarking tool. This throughput level indicates the system can handle production-scale event ingestion without becoming a bottleneck.

Latency measurements show client P99 latency of 3.46 milliseconds, meaning 99% of requests complete within this timeframe. This performance profile suggests Ayder can support latency-sensitive applications where consistent response times are critical.

Numbers (3-node Raft, real network, sync-majority writes, 64B payload): ~50K msg/s sustained (wrk2 @ 50K req/s), client P99 ~3.46ms.

The benchmark methodology emphasizes real-world conditions, providing confidence that these metrics translate to actual deployment scenarios rather than idealized laboratory measurements.

Perhaps the most compelling demonstration of Ayder's durability is its crash recovery capability. The system was subjected to an unclean SIGKILL termination—simulating a sudden server failure or power loss without graceful shutdown procedures.

Following the forced termination, Ayder restarted and successfully verified that all offsets and data remained intact. The recovery process handled approximately 8 million offsets in a timeframe ranging from 40 to 50 seconds. This recovery speed demonstrates efficient checkpointing and log replay mechanisms.

The demonstration video, which runs approximately two minutes from initial crash through complete recovery validation, provides visual proof of the system's resilience. This capability addresses a critical requirement for production systems: maintaining data integrity through unplanned outages.

Key recovery characteristics include:

• Automatic offset verification after restart
• Zero data loss despite unclean termination
• Fast replay of millions of logged events
• Transparent recovery process requiring no manual intervention

These recovery metrics indicate that Ayder implements robust write-ahead logging and snapshotting strategies, ensuring that even catastrophic failures don't compromise the event log's consistency.

The developer is actively seeking early design partners to validate Ayder across diverse production scenarios. This invitation extends to organizations running any type of event ingestion or streaming workload.

Design partners would gain early access to the technology while providing feedback that shapes the roadmap. The system appears ready for real-world testing, with comprehensive documentation including benchmarks, demonstration videos, and quick-start guides available in the repository.

Organizations that might benefit from evaluation include those currently using:

• Traditional message queues seeking HTTP simplicity
• Event sourcing architectures requiring durability
• Microservices needing lightweight coordination
• Streaming platforms prioritizing crash recovery

The call for partners suggests the project has matured beyond proof-of-concept stage and is ready for broader validation. Early adopters would help identify edge cases, performance boundaries, and integration patterns across different deployment environments.

Ayder represents a return to simplicity in distributed systems design. By stripping away layers of abstraction and leveraging ubiquitous HTTP protocols, it offers a compelling alternative to complex event log architectures.

The combination of high performance (50K msg/s), low latency (3.46ms P99), and robust durability (8M offset recovery in 40-50s) addresses the core requirements of modern event-driven applications. The single-binary C implementation promises operational simplicity without sacrificing capabilities.

As the project seeks design partners, the next phase will reveal how Ayder performs across the varied demands of production workloads. For teams evaluating event log solutions, this technology warrants attention—not just for its technical merits, but for its philosophy of making durable event streaming accessible through simple, standard tools.