Skip to content

Latest commit

 

History

History
477 lines (381 loc) · 18.3 KB

README.md

File metadata and controls

477 lines (381 loc) · 18.3 KB
Raw
Cloudmos logo

Cloudmos

Cloudmos is an app that let's you deploy any docker container on the Akash Network in a few clicks... 🛠

version license Twitter URL https://discord.gg/akash

Services

Dataflow between services

Dataflow between services

Every services are nodejs applications written in typescript. They are deployed using docker. Both databases are PostgreSQL deployed on GCP.

Deploy Website

Website to deploy on Akash. It is using the Next.js framework. The data shown in the website comes from a combination of our API and akash nodes (REST).

Url: deploy.cloudmos.io
Directory: deploy-web

Provider Proxy

Used in the deploy website to proxy requests to providers. Needed since it's not possible to use the cert authentication system from the browser.

This is the equivalent of the Proxy used by the akash console.

Directory: provider-proxy

API

Provides data to the deploy website. The data is fetched from our Indexer database + coingecko for market data. This api also provide the endpoint for the Web3 Index integration.

Url: api.cloudmos.io
Directory: api

Indexer

Fetch the latest blocks from RPC nodes. Saves the blocks & stats to our Indexer Database. For details on how the indexer work see the Indexer README

Directory: indexer

Landing Page

Small landing page with button to "Launch App".

Url: cloudmos.io
Directory: landing

Config

Directory containing public config file that are fetch by the deploy tool. This is to allow modifying some configurations on the spot without the need to redeploy the web interface.

Directory: config

Monitoring

  • Sentry.io is integrated into all our services and allows us to catch js exceptions in the frontend as well as backend.
  • HealthChecks.io is used to monitor the execution of our Scheduled Tasks. If a task is failing we get an email + alert in discord.
  • Cronitor is used to monitor uptime. If our website or apis goes offline we get an email + alert in discord.
  • Grafana dashboards allows us to see usage stats and syncing status. We can monitor block syncing and scheduled task there.
  • Status Endpoints
    All our backend services have /status endpoints built into them allowing us to monitor health and view usage stats. Those endpoints are use by the other monitoring tools.

Example SQL Queries

See Example_Queries.md

Running the Application

This document provides instructions on how to set up and run the application, including steps for manual database restoration and using Docker Compose for ease of setup.

Manual Database Restoration

Due to the extensive time required to index Akash from block #1, it's recommended to initialize your database using an existing backup for efficiency. This approach is particularly beneficial for development purposes.

Available Backups

Restoration Steps

  1. Create a PostgreSQL database.
  2. Restore the database using psql. Ensure PostgreSQL tools are installed on your system.

For a .sql.gz file:

gunzip -c /path/to/cloudmos-akash-sandbox.sql.gz | psql --host "localhost" --port "5432" --username "postgres" --dbname "cloudmos-akash"

After restoring the database, you can proceed with the specific project's README instructions for further setup and running the application.

Using Docker Compose

For convenience, a Docker Compose configuration is provided to automate the database setup, download, and data import process.

Configuration

  1. Ensure environment variables are set in .postgres.local.env. This file configures PostgreSQL and import settings.
  2. The POSTGRES_DBS_FOR_IMPORT variable should be updated with the databases you wish to import. This is a comma-separated list with potential values including cloudmos-akash-sandbox (default) and cloudmos-akash-2 (mainnet). Leave this variable empty if no import is desired.

Running Docker Compose

docker-compose up

This command spins up the database service and automatically handles the downloading and importing of the specified data.

Database Structure

The database schemas is defined using sequelize-typescript in /shared/dbSchemas/. Models are separated into the following folders:

  • base - Tables which are cosmos generic and are used for every chains
  • akash - Tables which are akash specific (they are not created when initializing a database for another chain)
  • user - Tables which are user specific. Contains tables for user settings and templates

Block

Column Type Note
height integer
datetime timestamp
hash varchar
proposer varchar
dayId uuid See Day
txCount integer
isProcessed boolean false when inserted into the database and then set to true once the indexers have processed every messages in the block.
totalTxCount bigint Global tx count as of this height (for historical tx count graphs).
totalUAktSpent double Cummulative uakt spent on deployments as of this block.
activeLeaseCount integer Currently active lease count as of this block. Graph
totalLeaseCount integer Cummulative lease count as of this block. Graph
activeCPU integer Currently leased CPU as of this block. Graph
activeMemory bigint Currently leased Memory as of this block. Graph
activeEphemeralStorage bigint Currently leased Ephemeral Storage as of this block. Graph
activePersistentStorage bigint Currently leased Persistent Storage as of this block. Graph
activeProviderCount integer Provider count as of this block. This only consider on-chain data and does not include uptime checks.

Transaction

Column Type Note
id uuid
hash varchar
height integer
msgCount integer
multisigThreshold integer
gasUsed integer
gasWanted integer
fee bigint
memo text
isProcessed boolean false when inserted into the database and then set to true once the indexers have processed every messages in the transaction.
hasProcessingError boolean Indicates if the tx was a success or failed
log text Contains the error message if the tx failed

Message

Column Type Note
id uuid
txId uuid See Transaction
height integer
type varchar ex: /cosmos.bank.v1beta1.MsgSend
typeCategory varchar First part of the message part. ex: cosmos or akash
index integer
indexInBlock integer
isProcessed boolean false when inserted into the database and then set to true once the indexers have processed the message.
isNotificationProcessed boolean false when inserted into the database and then set to true once it has been checked for existing alerts. (BlockSpy specific)
amount bigint Amount of the transfer if token were transferred. ex: MsgSend or MsgDelegate
data bytes Protobuf encoded data of the message
relatedDeploymentId uuid If the message is deployment related (ex: MsgCloseLease) this will be set to the related deployment. See Deployment

AddressReference

Keeps track of which addresses are involved in which txs (even if they are not the signer).

Column Type Note
id number
transactionId uuid See Transaction
messageId uuid See Message
address varchar
type varchar Signer/Sender/Receiver

Day

Created for each days (UTC based), simplifies querying for daily stats.

Column Type Note
id uuid
date timestamp
aktPrice double See the "Sync Price History" task in Scheduled Tasks.
firstBlockHeight integer First block of the day.
lastBlockHeight integer Last block of the day. null if the day has not ended.
lastBlockHeightYet integer Last block of the day as of now.

Validator

Column Type Note
id uuid
operatorAddress varchar
accountAddress varchar
hexAddress varchar
createdMsgId uuid Message which created the validator (MsgCreateValidator). See Message
moniker varchar
identity varchar
website varchar
description text
securityContact varchar
rate double
double maxRate
maxChangeRate double
minSelfDelegation bigint
keybaseUsername varchar
keybaseAvatarUrl varchar

Deployment

Column Type Note
id uuid
owner varchar Address of the wallet that owns this deployment
dseq varchar
createdHeight integer Height of the MsgCreateDeployment message
balance double Remaining balance based on deposits and MsgWithdrawLease
deposit bigint Deposited amount based on MsgCreateDeployment and MsgDepositDeployment
lastWithdrawHeight integer Last height where an account settlement occured. This happens on create, withdraw and close.
withdrawnAmount double Withdrawn amount as of now. Updated on account settlement (create,withdraw,close).
closedHeight integer Height the deployment got closed on-chain. Can happen from MsgCloseDeployment or as a side-effect of having no active leases remaining.

DeploymentGroup

Column Type Note
id uuid
deploymentId uuid See Deployment
owner varchar Address of the wallet that owns this deployment group.
dseq varchar
gseq integer

DeploymentGroupResource

Column Type Note
id integer
deploymentGroupId uuid See DeploymentGroup
cpuUnits integer Thousandth of CPU
memoryQuantity bigint Memory in bytes
ephemeralStorageQuantity bigint Ephemeral Storage in bytes
persistentStorageQuantity bigint Persistent Storage in bytes
count integer
price double

Lease

Column Type Note
id uuid
deploymentId uuid See Deployment
deploymentGroupId uuid See DeploymentGroup
owner varchar Address of the wallet that owns this deployment group.
dseq varchar
oseq integer
gseq integer
providerAddress varchar
createdHeight integer Height of the lease creation. Happens when a bid is accepted with MsgCreateLease.
closedHeight integer Height at which the lease is closed on-chain. Happens from MsgCloseLease, MsgCloseHeight or if the deployment become overdrawn during an account settlement.
predictedClosedHeight bigint Height at which the lease should theoritically expire. This is calculated based on the balance and price. It will usually not match the closedHeight since leases can be closed early (MsgCloseLease & MsgCloseBid) or closed late since the closing wont happen until the provider does a MsgWithdrawLease
price double Lease price as uakt/block
withdrawnAmount double Withdrawn amount as of now for this lease. Updated on account settlement (create,withdraw,close).
cpuUnits integer Thousandth of CPU
memoryQuantity bigint Memory in bytes
ephemeralStorageQuantity bigint Ephemeral Storage in bytes
persistentStorageQuantity bigint Persistent Storage in bytes

Bid

Column Type Note
id integer
owner varchar Deployment's wallet address
dseq varchar
gseq integer
oseq integer
provider varchar Provider's wallet address
price double Price in uakt/block
createdHeight integer Height of the MsgCreateBid

Provider

Column Type Note
owner varchar Address of the wallet that owns this provider
hostUri varchar ex: https://provider.europlots.com:8443
createdHeight integer Height at which the provider is created on-chain. (MsgCreateProvider)
updatedHeight integer Height at which the provider was last updated. (MsgUpdateProvider)
deletedHeight integer Height at which the provider is deleted on-chain. (MsgDeleteProvider)
email varchar
website varchar
isOnline boolean Indicates if the latest uptime check was successful
lastCheckDate timestamp Date & Time of the latest uptime check
lastSnapshotId uuid ID of the last snapshot taken
nextCheckDate timestamp Planned Date & Time of the next uptime check
failedCheckCount integer Amount of consecutive failed checks, NULL if currently online.
lastSuccessfulSnapshotId uuid Snapshot ID of the last successful check
downtimeFirstSnapshotId uuid Snapshot ID of the first failed check of the current downtime period. NULL if currently online.
error text NULL if the latest uptime check was successful, otherwise this wil contain the error message.
deploymentCount integer
leaseCount integer
activeCPU bigint Thousandth of CPU
activeMemory bigint Memory in bytes
activeStorage bigint Storage in bytes
pendingCPU bigint Thousandth of CPU
pendingMemory bigint Memory in bytes
pendingStorage bigint Storage in bytes
availableCPU bigint Thousandth of CPU
availableMemory bigint Memory in bytes
availableStorage bigint Storage in bytes
akashVersion varchar
cosmosSdkVersion varchar
ip varchar IP obtained by resolving DNS for the hostUri
ipRegion varchar
ipCountry varchar
ipLat varchar
ipLon varchar
ipRegionCode varchar
ipCountryCode varchar

ProviderAttribute

Column Type Note
id integer
provider varchar Provider's wallet address
key varchar
value varchar

ProviderAttributeSignature

Store audited attributes based on MsgSignProviderAttributes & MsgDeleteProviderAttributes.

Column Type Note
id integer
provider varchar Provider's wallet address
auditor varchar Auditor's wallet address
key varchar
value varchar

ProviderSnapshot

Similar to stats on the Provider, but a new row is inserted for every uptime checks. This allows to graph the changes in resources over time as well as calculate an uptime percentage over a period.

Column Type Note
id uuid
owner varchar Address of the wallet that owns this provider
isOnline boolean Indicates if this uptime check was successful
isLastOfDay boolean Indicates if this is the last snapshot of the day for this provider
checkDate timestamp Date & Time of this uptime check
error text null if the uptime check was successful, otherwise this wil contain the error message.
deploymentCount integer
leaseCount integer
activeCPU bigint Thousandth of CPU
activeMemory bigint Memory in bytes
activeStorage bigint Storage in bytes
pendingCPU bigint Thousandth of CPU
pendingMemory bigint Memory in bytes
pendingStorage bigint Storage in bytes
availableCPU bigint Thousandth of CPU
availableMemory bigint Memory in bytes
availableStorage bigint Storage in bytes

ProviderSnapshotNodes

Keep track of ressources of individual provider nodes obtained through feature discovery.

Column Type Note
id uuid
snapshotId uuid Snapshot ID
name varchar Name of the node
cpuAllocatable bigint Thousandth of CPU
cpuAllocated bigint Thousandth of CPU
memoryAllocatable bigint Memory in bytes
memoryAllocated bigint Memory in bytes
ephemeralStorageAllocatable bigint Storage in bytes
ephemeralStorageAllocated bigint Storage in bytes
capabilitiesStorageHDD boolean Indicates if the node supports HDD storage
capabilitiesStorageSSD boolean Indicates if the node supports SSD storage
capabilitiesStorageNVME boolean Indicates if the node supports NVME storage
gpuAllocatable bigint
gpuAllocated bigint

ProviderSnapshotNodeCPU

Store CPU informations for each Provider Nodes

Column Type Note
id uuid
snapshotNodeId uuid Snapshot node ID
vendor varchar ex: GenuineIntel
model varchar ex: Intel(R) Xeon(R) CPU @ 2.30GHz
vcores smallint

ProviderSnapshotNodeGPU

Store GPU informations for each Provider Nodes

Column Type Note
id uuid
snapshotNodeId uuid Snapshot node ID
vendor varchar ex: nvidia
name varchar Model name (ex: rtx4090)
modelId varchar On the provider, this gets mapped to vendor, name, interface and memorySize based on this file.
interface varchar ex: PCIe
memorySize varchar ex: 24Gi

Template

Column Type Note
id uuid
userId uuid See User
copiedFromId uuid If the template was created by using the copy feature, this will contain the id of the original template.
title varchar
description text
isPublic boolean
cpu bigint
ram bigint
storage bigint
sdl text
createdOn timestamp

TemplateFavorite

Created when a user "favorite" a template

Column Type Note
id uuid
userId uuid See User
templateId uuid See Template
addedDate datetime

UserSetting

Column Type Note
id uuid
userId varchar User ID given by Auth0 this is the field other tables reference to.
username varchar
email varchar
emailVerified boolean
stripeCustomerId varchar Created once the user visits the checkout page
bio text
subscribedToNewsletter boolean
youtubeUsername varchar
twitterUsername varchar
githubUsername varchar