A fault tolerant distributed key value store from scratch | Minhaz’s Blog

[utterances-bot]

A fault tolerant distributed key value store from scratch | Minhaz’s Blog

We had a B-Tech course on Distributed Systems and I took a course on on Cloud Computing Concepts 1 by Dr Indranil Gupta (UIUC) a year back and for long, I have been thinking about trying out different concepts explained in the course together as something meaningful. In this article I have attempted to describe how to build a fault tolerant distributed key value store from scratch. A key-value store, or key-value database, is a data storage paradigm designed for storing, retrieving, and managing associative arrays, a data structure more commonly known today as a dictionary or hash. A distributed Key Value store is one where data is replicated across different nodes such that there is High Availability and No single point of failure

https://blog.minhazav.dev/a-fault-tolerant-distributed-key-value-store-from-scratch/

[wjm1991]

You made it very clear, thank you.

Distributed Key-Value store is actually a kind of distributed data storage, and there are also problems caused by replication, data writing transaction problems, and scalability problems, but there is no need for data analysis.

The CAP theorem points out that in the asynchronous network model, there is no system that can satisfy the three properties of consistency, availability, and partition fault tolerance at the same time. That is, a distributed system must forego one of these properties.

Among the difficulties in the implementation of distributed systems, communication timeout and update delay are both consistency issues. The reason for this problem is that there are multiple servers, and each server has its own data. Although data redundancy can improve system availability and partition fault tolerance, it is correspondingly difficult to satisfy strong consistency. If we want to solve the consistency problem and achieve strong consistency, we need to process all requests through a single server, but it is difficult to achieve high availability.

For distributed Key-Value store systems, it tends to weaken the consistency of the system and improve availability and partition fault tolerance. To do this, a consensus algorithm is needed to achieve it.

Quote source article:
How to Implement a distributed Key-Value store