Skip to content
/ Gringofts Public

Gringofts makes it easy to build a replicated, fault-tolerant, high throughput and distributed event-sourced system.

License

Notifications You must be signed in to change notification settings

eBay/Gringofts

CI codecov

Table of Contents

Introduction

Gringofts makes it easy to build a replicated, fault-tolerant, high throughput and distributed event-sourced system. This type of system can process and store critical transaction data which is valuable for an entity.

Industries and typical applications that can benefit from Gringofts (but not limited to):

  • Finance - Payment processing system
  • Government - Civic Engagement system
  • Healthcare - Electronic medical records (EMR) software
  • Retail - Orders and sales management system
  • Transport - Logistics and order management system
  • ...

Features

Dependable Data

  1. Data is highly available
    Gringofts enables data to be stored in multiple replicas across data centers. Different deployment models are supported to achieve various levels of availability. For example, in a typical 2-2-1 setup (two replicas in each of the first two data centers and one replica in the third) data is still available even if two data centers are down.

  2. Data is bit-level accurate across all replicas
    Whichever replica you access, you get the exact same data. This is a top requirement for some applications such as a payment processing system, where each transaction must be accurately recorded.

  3. Data is secure and tamper-proof
    Industry-tested and accepted standards are used to encrypt the data and a blockchain-like technology is applied to avoid data being changed.

High Throughput

Internal experiments show that a single cluster with a 2-1 setup (three replicas across two data centers) can process 8,000 transactions per second. Since the framework is designed to be linear-scalable, throughput can be increased with the increase of clusters. Below is a test result of a 667-cluster setup.

Benchmark

Full Audit-ability

This is another must-have feature in most critical enterprise systems. Every write operation to a Gringofts-powered application is persisted by default and is immutable. A comprehensive toolset is also available to access this type of information.

Easy to Use

Gringofts users are usually domain experts. They only need to focus on two things which they are very good at and Gringofts will take care of the rest:

  1. Domain model
    This defines objects required to solve the target business problem. For example, in an Order Management system, Order is a domain object.
  2. Domain object interaction
    This defines how domain objects interact with each other. The interaction is usually modeled as two special objects called Command and Event, for example PlaceOrderCommand and OrderPlacedEvent.

100% State Reproducibility

Application state at any point of time in history is reproducible. This feature is especially useful when an application recovers after a crash or if users want to debug an issue.

Get Started

Supported Platforms

Currently the only recommended platform is Ubuntu 16.04. We plan to support more platforms in the near future.

Set up Source Dependencies

bash ./scripts/addSubmodules.sh

Build

Build via Docker (Recommended)

This approach requires minimum dependencies on the target OS as all of the dependencies are encapsulated in a docker image.

  1. Build docker image for compiling the project (one-time setup)
    sudo docker build --rm -t gringofts/dependencies:v1 -f dockers/dependencies/download.Dockerfile .
    sudo docker build --rm -t gringofts/compile:v1 -f dockers/dependencies/install.Dockerfile .
  2. Build binaries
    sudo docker run --workdir "$(pwd)" --mount type=bind,source="$(pwd)",target="$(pwd)" --user "$(id -u)":"$(id -g)" gringofts/compile:v1 hooks/pre-commit

Build directly on local OS

  1. Install external dependencies (one-time setup)
    sudo bash ./scripts/setupDevEnvironment.sh
    if ! grep 'export PATH=/usr/local/go/bin:$PATH' ~/.profile; then echo 'export PATH=/usr/local/go/bin:$PATH' >> ~/.profile; fi && \
    source ~/.profile
  2. Build binaries
    hooks/pre-commit-build

Run Demo App

  1. Backed by a single-cluster setup
    examples/run_demo_backed_by_single_cluster.sh
    You can use grpc_cli to verify:
    ./grpc_cli call 0.0.0.0:50055 ringofts.demo.protos.DemoService.Execute "value:1"
    Sample output:
    connecting to 0.0.0.0:50055
    code: 200
    message: "Success"
    
    Rpc succeeded with OK status
  2. Backed by a three-nodes-cluster setup
    examples/run_demo_backed_by_three_nodes_cluster.sh
  3. Backed by SQLite3
    examples/run_demo_backed_by_sqlite.sh

Development Environment Setup

Please refer to this doc for details.

Core Developers

Please see here for all contributors.

Acknowledgements

Special thanks to people who give your support on this project.

License Information

Copyright 2019-2020 eBay Inc.

Authors/Developers: Bin (Glen) Geng, Qi (Jacky) Jia

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Use of 3rd Party Code

Some parts of this software include 3rd party code licensed under open source licenses (in alphabetic order):

  1. OpenSSL
    URL: https://www.openssl.org/
    License: https://www.openssl.org/source/license.html
    Originally licensed under the Apache 2.0 license.

  2. RocksDB
    URL: https://github.com/facebook/rocksdb
    License: https://github.com/facebook/rocksdb/blob/master/LICENSE.Apache
    Apache 2.0 license selected.

  3. SQLite
    https://www.sqlite.org/index.html
    License: https://www.sqlite.org/copyright.html
    SQLite Is Public Domain

  4. abseil-cpp
    URL: https://github.com/abseil/abseil-cpp
    License: https://github.com/abseil/abseil-cpp/blob/master/LICENSE
    Originally licensed under the Apache 2.0 license.

  5. cpplint
    URL: https://github.com/google/styleguide
    License: https://github.com/google/styleguide/blob/gh-pages/LICENSE
    Originally licensed under the Apache 2.0 license.

  6. inih
    URL: https://github.com/benhoyt/inih
    License: https://github.com/benhoyt/inih/blob/master/LICENSE.txt Originally licensed under the New BSD license.

  7. gRPC
    URL: https://github.com/grpc/grpc
    License: https://github.com/grpc/grpc/blob/master/LICENSE
    Originally licensed under the Apache 2.0 license.

  8. googletest
    URL: https://github.com/google/googletest
    License: https://github.com/google/googletest/blob/master/LICENSE
    Originally licensed under the BSD 3-Clause "New" or "Revised" license.

  9. prometheus-cpp
    URL: https://github.com/jupp0r/prometheus-cpp
    License: https://github.com/jupp0r/prometheus-cpp/blob/master/LICENSE Originally licensed under the MIT license.

  10. spdlog
    URL: https://github.com/gabime/spdlog
    License: https://github.com/gabime/spdlog/blob/master/LICENSE
    Originally licensed under the MIT license.

About

Gringofts makes it easy to build a replicated, fault-tolerant, high throughput and distributed event-sourced system.

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published