Distributed Database Replication using SWIM and BadgerDB
2025-05-14
This article details a distributed database replication scheme leveraging the SWIM protocol and BadgerDB's incremental backup feature. Using Hashicorp's memberlist package and its TCP Push/Pull mechanism, nodes periodically compute their local state and send it to other nodes. Remote nodes merge the remote state with their local state, triggering the replication process. Node state is defined as the set of last known versions of other nodes. Upon receiving state, a node sends HTTP requests to the remote node, concurrently syncing incremental backups of auth, config, and log databases, thus achieving actual replication.