I share interesting articles, videos, papers and more about distributed systems, formal methods and computer science.
Here at decentralized thoughts, we spend a lot of time reasoning about distributed protocols. Often, we focus on solving distributed consensus, personally it’s my favorite CS problem, but it’s also famously one of the most difficult and subtle problems in distributed computing. Reasoning about distributed algorithms is hard at the...
In this blog I will go over how Apache Iceberg contributes to performance of compute engine. Apache Iceberg is an ACID table format designed for large-scale analytics workloads. While its consistency and schema evolution features are covered in previous blog, its impact on query performance can be equally transformative. By the end of this document, you will have a deep understanding of how Iceberg enhances performance, the trade-offs involved, and best practices for maximizing efficiency in read-heavy workloads.
Debugging concurrency bugs is no picnic, but we're going to get into it. Enter Fray, a deterministic concurrency testing framework from CMU’s PASTA Lab, that turns flaky failures into reliably reproducible ones.
To me it’s clear that the big idea there isn’t lightweight processes2 and message passing, but rather the generic components which in Erlang are called behaviours.
A curated collection of resources about deterministic simulation testing for distributed systems.
Emerging patterns for building scalable, high-performance observability pipelines
Material for the course Parallel, Concurrent and Distributed Programming by Ilya Sergey at Yale-NUS College
The challenge is that physical imperfections in hardware clocks (called quartz crystal oscillators) cause our software clocks to tick at different speeds, so that time passes faster or slower than it should, with these “drift” errors also accumulating into significant “skew” errors within a matter of minutes.
This page is a relatively informal discussion of distributed consensus and Paxos, what it does, how it works, and some tricks and variants.
