Choosing QuePaxa, an unusual consensus algorithm | Cloudflare Blog
Cloudflare is building an experimental consensus service using a never-before-deployed consensus algorithm, QuePaxa. Here's how it works and why we chose it.
I share interesting articles, videos, papers and more about distributed systems, formal methods and computer science.
Cloudflare is building an experimental consensus service using a never-before-deployed consensus algorithm, QuePaxa. Here's how it works and why we chose it.
How we used SKIP LOCKED, composite primary keys, and connection visibility to hit our scale targets.
In Zen And The Art Of Scaling - A Koan And Epigram Approach , Russell Sullivan offered an inter...
This article explores and names some of the practical approaches used in the implementation of large-scale mission-critical applications in a world that rejects distributed transactions. Topics include the management of fine-grained pieces of application data that may be repartitioned over time as the application grows. Design patterns support sending messages between these repartitionable pieces of data.
Hosting Git repositories at scale is a nightmare. When Linus Torvalds designed the first version of the information manager from hell (that's actually the tagline for Git, look it up), he had a very specific use case in mind: his own. He wanted to replace BitKeeper, the distributed version control system that was being used to develop the Linux Kernel. Of course, the replacement had to be distributed too. The Kernel is an unusual software project; it is extremely decentralized, with many different maintainers for its many different subsystems. A distributed version control system is a natural fit for this workflow. Twenty years later, Git has become an industry standard, but the truth is that its distributed nature is more of a hindrance than an advantage. The average open-source software project doesn't operate with a decentralized workflow. The average company definitely doesn't. They use the many advantages of the distributed model (such as being able to work offline, delay pushes, etc) but they very much rely on a centralized host. And hosting a Git repository, it turns out, is an incredibly hard thing to do.
Zak van der Merwe has spent his entire career at AWS building control planes. First for EC2 and now for DSQL. On the surface, the control plane looks quite boring: it records what should exist and reconciles that with what actually does. Nobody leaves school dreaming of building one, but Zak will be the first to tell you that if you like solving hard problems in distributed systems, there are few better places to be. Itâs where many of those hard problems converge, and where the decisions you make determine whether a service survives its own growth.
How to make 768 distinct Postgres servers look like 1 to your applications.
How we built an in-process client-side load balancer for a million requests per second of internal fan-out traffic, what we layered on top (N-ring fade-in, occupancy-based bounded load, and...
Aurora DSQL is a serverless SQL database designed for cloud-scale transaction processing with multi-region active-active capabilities.
TLA+ modeling caught a silent data-loss bug in Postgres pg_rewind during Multigres HA work—what we found and how we fixed it.