Zookeeper
Often described as a “distributed key-value store”.
Why does zookeeper use a fault-tolerant consensus 202211232125 algorithm under the hood?
NOTE: People usually don’t Zookeeper directly. It’s often used by some other service, e.g., HBase, Hadoop YARN, OpenStack Nova, and Kafka.
Zookeeper is designed to hold small amounts of data that can entirely fit in memory (although they still write to disk for durability). That small amount of data is replicated across all nodes using a fault-tolerant total order broadcast 202211232155 algorithm. (Total order broadcast keeps replicas consistent with each other.)
Answer to the question above: Zookeeper has the following attractive features.
- Linearizable atomic operations
- Using atomic compare-and-set, you can implement a lock: if several nodes concurrently try to perform the same operation, only one will succeed. The consensus protocol ensures that the operation will be atomic and linearizable, even if a node fails or the network is interrupted.
- Total ordering of operations, which serves as a fencing token 202211232306 to prevent clients from conflicting with each other in the case of a process pause.
- Failure detection
- Clients and the server periodically exchange heartbeats to ensure that the other node is still alive.
- Locks by a session can be configured to be automatically released when the session times out. (These are called ephemeral nodes 202211232312)
- Change notifications
- Not only can one client can read locks and values that were created by another client, but it can also watch them for changes.
Several people have suggested using zookeeper as a service for shard assignments 202211232309
Created from: Raft & Paxos 202211231100
uid: 202211232246 tags: #software-engineering #algorithms #distributed-systems