Skip to content
This repository has been archived by the owner on Apr 10, 2024. It is now read-only.

Commit

Permalink
feat: shared configuration (#23)
Browse files Browse the repository at this point in the history 
* Add release github workflows

* Add dependecy review

* Fix license in .github

* Add license

* Remove github skip dirs

* Refactor API to be more readable

* Fix naming

* Move helper function to separate file

* Remove extra resource in debug mode

* Return error if resource does not exist

* Return error if domain does not exist

* Remove remote params

* Parse resources from domain map

* Remove verbose config functions

* Remove resource config path flag

* Fix start script

* Fix error conditions

* Print response body

* Printout body string

* Fix domainID errors message

* Return after error

* Fix usdt resourceID

* Improve error message

* Remove TODO

* Print domain names

* Fix fething domains

* Fix missing domain info

* Update dockerfile

* Load domains from IPFS if domain config path is an url

* Print out domain info

* Remove prints

* Fix domain.json

* Update README

* Ignore gosec error

* Fix oracle expiration date

* Make the endpoint expiration timestamp in the future

* Log final timestamp

* Fix invalid token rate

* Remove extra comment

* Add test resource

* Fix invalid reference
  • Loading branch information
@mpetrun5
mpetrun5 authored Feb 8, 2023
1 parent 6a436d6 commit 64b30d8
Show file tree
Hide file tree
Showing 33 changed files with 455 additions and 825 deletions.
2 changes: 1 addition & 1 deletion Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ RUN go mod download
RUN go build -o feeOracle ./
EXPOSE 8091 8091
RUN ["./feeOracle", "key-generate"]
CMD ["./feeOracle", "server", "-c", "./config.yaml", "-d", "./domain.json", "-r", "./resource.json", "-k", "./keyfile.priv", "-t", "secp256k1"]
CMD ["./feeOracle", "server", "-c", "./config.yaml", "-d", "./domain.json", "-k", "./keyfile.priv", "-t", "secp256k1"]
99 changes: 37 additions & 62 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ There are three main parts in fee oracle codebase in an abstract way: `App base`
3. Data provider: This is the combination of `store`, `api`, `consensus`, `identity` packages. Data provider queries the store based on the request from the endpoint, refines the data based on the configured `strategy`, then returns the data along with the signature of the fee oracle identity key.

# Installation & Build
**Make sure `Go1.17` has been installed.**
**Make sure `Go1.17` has been installed.**

This will clone the main branch of fee oracle codebase to your `workplace` dir and compile the binary into your
`$GOPATH/bin`
```
$ mkdir workplace && cd workplace
$ mkdir workplace && cd workplace
$ git clone https://github.com/ChainSafe/Sygma-fee-oracle.git
$ make install
```
Expand All @@ -25,76 +25,56 @@ $ make install
Fee oracle needs three config files in the `./` dir of the codebase:
1. `config.yaml`: this is the fee oracle application config file.
2. `domain.json`: this is the domain config file.
3. `resource.json`: this is the resource config file.

### Application config file `config.yaml`
Template of the config.yaml can be found in `./config/config.template.yaml`.

### Domain config file `domain.json`
This file indicates all the domains the fee oracle needs to fetch data for. Details need to be matched with
This file indicates all the domains and resources the fee oracle needs to fetch data for. Details need to be matched with
Sygma core configuration, such as `id`.

Sygma currently does not support bridging the native currency, such as Ether on Ethereum, Matic on Polygon, however, the `id` is constructed with zero address and its native `domainId` and is used in baseRate calculation internally.

Example:
```json
{
"domains": [
{
"id": 0,
"name": "ethereum",
"baseCurrencyFullName": "ether",
"baseCurrencySymbol": "eth",
"addressPrefix": "0x"
},
{
"id": 1,
"name": "polygon",
"baseCurrencyFullName": "matic",
"baseCurrencySymbol": "matic",
"addressPrefix": "0x"
}
]
}
```

### Resource config file `resource.json`
`resource` stands for the asset that can be bridged in Sygma. This `resource.json` file indicates all the resources the fee oracle needs to fetch data for.
Each resource has one unique `id` across all supported domains(networks), and it also has `domains` subsection to address some special domain related information each as `decimal`.
Sygma currently does not support bridging the native currency, such as Ether on Ethereum, Matic on Polygon, however, the `id` is constructed with zero address and its native `domainId` and is used in baseRate calculation internally.

```json
{
"resources": [
{
"id": "0x00000000000000000000000000000000000000000",
"symbol": "eth",
"domains": [
"nativeTokenDecimals": 18,
"nativeTokenSymbol": "eth",
"resources": [
{
"domainId": 0,
"decimal": 18
"resourceId": "0x0000000000000000000000000000000000000000000000000000000000000001",
"decimals": 18,
"symbol": "usdt"
}
]
},
{
"id": "0x00000000000000000000000000000000000000001",
"symbol": "matic",
"domains": [
"id": 1,
"name": "polygon",
"nativeTokenDecimals": 18,
"nativeTokenSymbol": "matic",
"resources": [
{
"domainId": 1,
"decimal": 18
"resourceId": "0x0000000000000000000000000000000000000000000000000000000000000001",
"decimals": 18,
"symbol": "usdt"
}
]
},
{
"id": "0x0000000000000000000000000000000000000000000000000000000000000001",
"symbol": "usdt",
"domains": [
"id": 2,
"name": "moonbeam",
"nativeTokenDecimals": 18,
"nativeTokenSymbol": "glmr",
"resources": [
{
"domainId": 0,
"decimal": 18
},
{
"domainId": 1,
"decimal": 18
"resourceId": "0x0000000000000000000000000000000000000000000000000000000000000001",
"decimals": 18,
"symbol": "usdt"
}
]
}
Expand All @@ -104,19 +84,19 @@ Sygma currently does not support bridging the native currency, such as Ether on

# Fee Oracle Identity
Each fee oracle server associates with a private key, which is used to sign the endpoint response data.
There should be a `keyfile.priv` keyfile in the root dir of the fee oracle codebase, or you can specify which keyfile to use in CLI.
There should be a `keyfile.priv` keyfile in the root dir of the fee oracle codebase, or you can specify which keyfile to use in CLI.

**Fee oracle provides [key generation CLI](#keycli), keyfile needs to be generated separately.**

# Quick Start
To quickly start from makefile, make sure `config.yaml`, `domain.json`, `resource.json` and `keyfile.priv` are ready in the root dir of the codebase, then execute:

`$ make start`

# Command Line
Fee oracle provides CLI.
Fee oracle provides CLI.

For general help:`$ sygma-fee-oracle -h`
For general help:`$ sygma-fee-oracle -h`

#### `$ sygma-fee-oracle server`
```
Expand All @@ -126,11 +106,11 @@ Usage:
sygma-fee-oracle server [flags]
Flags:
-c, --config_path string
-d, --domain_config_path string
-c, --config_path string
-d, --domain_config_path string
-h, --help help for server
-t, --key_type string Support: secp256k1
-k, --keyfile_path string
-k, --keyfile_path string
-r, --resource_config_path string
```

Expand All @@ -157,7 +137,7 @@ Flags:
`$ make lint`

# Using Docker
Fee oracle provides a Dockerfile to containerize the codebase.
Fee oracle provides a Dockerfile to containerize the codebase.
To build docker image:
```
$ docker build -t fee_oracle .
Expand All @@ -172,15 +152,13 @@ $ docker run -p 8091:8091 -it fee_oracle
if no keyfile exists in `./` dir, fee oracle will auto generate a `secp256k1` keyfile to use.

# End to End Test
This will start fee oracle service, ganache-cli locally, install `solcjs`, `abigen` and generate contract go binding code, deploy fee handler contracts to local ganache.
This will start fee oracle service, ganache-cli locally, install `solcjs`, `abigen` and generate contract go binding code, deploy fee handler contracts to local ganache.
`$ make e2e-test`

# EVN Params
Fee oracle loads important configs and prikey from files in CLI flags; however, the following EVN params will suppress CLI flags if provided.
Note: if `REMOTE_PARAM_OPERATOR_ENABLE` is set to `true`, valid credentials of the remote service must be setup. In addition `REMOTE_PARAM_DOMAIN_DATA` and `REMOTE_PARAM_RESOURCE_DATA` variables need to be set.
```text
APP_MODE=release // app mode: debug or release. app mode is used for internal testing only.
IDENTITY_KEY= // fee oracle prikey in hex, without 0x prefix
IDENTITY_KEY= // fee oracle prikey in hex, without 0x prefix
IDENTITY_KEY_TYPE=secp256k1 // fee oracle prikey type, only support secp256k1 for now
WORKING_ENV=production // fee oracle app running mode: dev or production
LOG_LEVEL=4 // log level, 4 is info, 5 is debug, using 4 on production
Expand All @@ -194,9 +172,6 @@ COINMARKETCAP_API_KEY= // api key of coinm
MOONSCAN_API_KEY= // api key of moonscan
DATA_VALID_INTERVAL=3600 // Time of valid fee oracle response data in seconds
CONVERSION_RATE_PAIRS=eth,usdt,matic,usdt // conversion rate pairs that enabled for price fetching. Must be paired
REMOTE_PARAM_OPERATOR_ENABLE=true // enable remote param operator, only enable this when deploying to remote environment like staging or prod
REMOTE_PARAM_DOMAIN_DATA="domainData/param/name" // domain data remote parameter name
REMOTE_PARAM_RESOURCE_DATA="resourceData/param/name" // resource data remote parameter name
```

# API Documentation
Expand Down
110 changes: 31 additions & 79 deletions api/api.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,12 @@ import (
"errors"
"fmt"
"net/http"
"strconv"
"strings"
"time"

"github.com/ChainSafe/sygma-fee-oracle/types"
"github.com/ChainSafe/sygma-fee-oracle/util"

"github.com/ChainSafe/sygma-fee-oracle/config"
oracleErrors "github.com/ChainSafe/sygma-fee-oracle/errors"

"github.com/ChainSafe/sygma-fee-oracle/consensus"
"github.com/ChainSafe/sygma-fee-oracle/identity"
Expand Down Expand Up @@ -51,114 +49,68 @@ func AddRouterPathsV1(v1RouterGroups map[string]*gin.RouterGroup, apiHandler *Ha
// calculation of transferring native resource: from ethereum to polygon transfer eth => ber = matic / eth, ter = matic / eth, gas price is from polygon
// calculation of transferring standard token : from ethereum to polygon transfer usdt => ber = matic / eth, ter = matic / usdt, gas price is from polygon
func (h *Handler) getRate(c *gin.Context) {
fromDomainID, err := strconv.Atoi(c.Param("fromDomainID"))
fromDomain, toDomain, err := h.parseDomains(c.Param("fromDomainID"), c.Param("toDomainID"))
if err != nil {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput, errors.New("invalid fromDomainID")))
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&oracleErrors.InvalidRequestInput, errors.New("invalid domainID")))
return
}
toDomainID, err := strconv.Atoi(c.Param("toDomainID"))
if err != nil {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput, errors.New("invalid toDomainID")))
return
}
if fromDomainID == toDomainID {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput, errors.New("fromDomainID cannot be the same as toDomainID")))
return
}
resourceID := c.Param("resourceID")
if !strings.HasPrefix(resourceID, "0x") || len(resourceID) != 66 {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput, errors.New("invalid resourceID")))
return
}
h.log.Debugf("new request with params fromDomainID: %d, toDomainID: %d, resourceID: %s\n", fromDomainID, toDomainID, resourceID)

toDomain := h.conf.GetRegisteredDomains(toDomainID)
if toDomain == nil {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput, errors.New("toDomainID not registered")))
return
}
fromDomain := h.conf.GetRegisteredDomains(fromDomainID)
if fromDomain == nil {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput, errors.New("fromDomain not registered")))
resource, err := h.parseResource(c.Param("resourceID"))
if err != nil {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&oracleErrors.InvalidRequestInput, errors.New("invalid resourceID")))
return
}
h.log.Debugf("new request with params fromDomainID: %d, toDomainID: %d, resourceID: %s\n", fromDomain.ID, toDomain.ID, resource.ID)

resource := h.conf.GetRegisteredResources(resourceID)
if resource == nil {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput, errors.New("resourceID not registered")))
msgGasLimitParam := c.DefaultQuery("msgGasLimit", "0")
if !util.CheckInteger(msgGasLimitParam) {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&oracleErrors.InvalidRequestInput, errors.New("invalid msgGasLimit")))
return
}

aggregatedGasPriceData, err := h.consensus.FilterLocalGasPriceData(h.gasPriceStore, toDomain.Name)
gp, err := h.consensus.FilterLocalGasPriceData(h.gasPriceStore, toDomain.Name)
if err != nil {
ginErrorReturn(c, http.StatusInternalServerError, newReturnErrorResp(&config.ErrConsensusFail, err))
ginErrorReturn(c, http.StatusInternalServerError, newReturnErrorResp(&oracleErrors.InternalServerError, err))
return
}
h.log.Debugf("aggregatedGasPriceData: %v\n", aggregatedGasPriceData)

baseSymbol := toDomain.BaseCurrencySymbol
foreignSymbol := fromDomain.BaseCurrencySymbol
h.log.Debugf("base rate calculation: base: %s, foreign: %s\n", baseSymbol, foreignSymbol)
h.log.Debugf("aggregatedGasPriceData: %v\n", gp)

aggregatedBaseRateData, err := h.consensus.FilterLocalConversionRateData(h.conversionRateStore, baseSymbol, foreignSymbol)
ber, err := h.consensus.FilterLocalConversionRateData(
h.conversionRateStore,
toDomain.BaseCurrencySymbol,
fromDomain.BaseCurrencySymbol)
if err != nil {
ginErrorReturn(c, http.StatusInternalServerError, newReturnErrorResp(&config.ErrConsensusFail, err))
ginErrorReturn(c, http.StatusInternalServerError, newReturnErrorResp(&oracleErrors.InternalServerError, err))
return
}
h.log.Debugf("base rate calculation: to: %s, from: %s\n", toDomain.BaseCurrencySymbol, fromDomain.BaseCurrencySymbol)

// if resourceID is the base currency of the fromDomain
aggregatedTokenRateData := aggregatedBaseRateData
if resource.Symbol != foreignSymbol {
fromDomainResource := h.conf.GetRegisteredResources(resource.ID)
if fromDomainResource == nil {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput,
errors.New("no resource registered with given the fromDomainID")))
return
}

h.log.Debugf("token rate calculation: base: %s, foreign: %s\n", baseSymbol, fromDomainResource.Symbol)

aggregatedTokenRateData = &types.ConversionRate{}
if fromDomainResource.Symbol == baseSymbol {
aggregatedTokenRateData.Rate = 1.0
} else {
aggregatedTokenRateData, err = h.consensus.FilterLocalConversionRateData(h.conversionRateStore, baseSymbol, fromDomainResource.Symbol)
if err != nil {
ginErrorReturn(c, http.StatusInternalServerError, newReturnErrorResp(&config.ErrConsensusFail, err))
return
}
}
}

msgGasLimitParam := c.DefaultQuery("msgGasLimit", "0")
validValue := util.CheckInteger(msgGasLimitParam)
if !validValue {
ginErrorReturn(c, http.StatusBadRequest, newReturnErrorResp(&config.ErrInvalidRequestInput, errors.New("invalid msgGasLimit")))
ter, err := h.calculateTokenRate(resource, ber, fromDomain, toDomain)
if err != nil {
ginErrorReturn(c, http.StatusInternalServerError, newReturnErrorResp(&oracleErrors.InternalServerError, err))
return
}
h.log.Debugf("token rate calculation: to: %s, from: %s\n", toDomain.BaseCurrencySymbol, resource.Symbol)

dataTime := aggregatedBaseRateData.Time
dataTime := ter.Time
signedTime := time.Now().Unix()

endpointRespData := &FetchRateResp{
BaseRate: fmt.Sprintf("%f", aggregatedBaseRateData.Rate),
TokenRate: fmt.Sprintf("%f", aggregatedTokenRateData.Rate),
DestinationChainGasPrice: aggregatedGasPriceData.SafeGasPrice,
FromDomainID: fromDomainID,
ToDomainID: toDomainID,
BaseRate: fmt.Sprintf("%f", ber.Rate),
TokenRate: fmt.Sprintf("%f", ter.Rate),
DestinationChainGasPrice: gp.SafeGasPrice,
FromDomainID: fromDomain.ID,
ToDomainID: toDomain.ID,
MsgGasLimit: msgGasLimitParam,
DataTimestamp: dataTime,
SignatureTimestamp: signedTime,
ExpirationTimestamp: h.dataExpirationManager(dataTime),
}

endpointRespData.Signature, err = h.rateSignature(endpointRespData, fromDomainID, resource.ID)
endpointRespData.Signature, err = h.rateSignature(endpointRespData, fromDomain.ID, resource.ID)
if err != nil {
ginErrorReturn(c, http.StatusInternalServerError, newReturnErrorResp(&config.ErrIdentityStampFail, err))
ginErrorReturn(c, http.StatusInternalServerError, newReturnErrorResp(&oracleErrors.InternalServerError, err))
return
}

endpointRespData.ResourceID = resource.ID

ginSuccessReturn(c, http.StatusOK, endpointRespData)
}
Loading

0 comments on commit 64b30d8

Please sign in to comment.