Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validator set persistence #200

Merged
merged 38 commits into from
Feb 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
8b8ab9d
add db schema and start reimplementing validator cache with persistence
pk910 Dec 17, 2024
182988e
continue working on validator set persistence
pk910 Dec 18, 2024
e45411c
continue implementation
pk910 Dec 19, 2024
0196c0d
Merge branch 'master' into pk910/validators-in-db
pk910 Dec 19, 2024
fc6c884
continue implementation
pk910 Jan 6, 2025
b86abd1
Merge branch 'master' into pk910/validators-in-db
pk910 Jan 23, 2025
a82df4e
continue implementation
pk910 Jan 23, 2025
a1f880b
cleanup
pk910 Jan 23, 2025
0848cb2
fix performance for filtered slot requests during long unfinality
pk910 Jan 24, 2025
1904aa3
Merge branch 'pk910/unfinality-performance-fixes' into pk910/validato…
pk910 Jan 24, 2025
3d7bc51
Merge branch 'pk910/unfinality-performance-fixes' into pk910/validato…
pk910 Jan 24, 2025
f52ce81
update validator activity page
pk910 Jan 24, 2025
7ab3bea
add log output for validator persistence
pk910 Jan 24, 2025
81b975d
fix validator persistence db queue
pk910 Jan 24, 2025
39be739
fix validator name filter
pk910 Jan 25, 2025
439a108
fix log message when persisting validators
pk910 Jan 25, 2025
1451203
Merge branch 'master' into pk910/validators-in-db
pk910 Jan 27, 2025
75a99a0
fix duty precalculation when validator set changed since last epoch
pk910 Feb 1, 2025
3822c35
show validator set restoration time in logs on startup
pk910 Feb 1, 2025
71177b4
dispose blocks to explicitly free memory for blocks even when still r…
pk910 Feb 2, 2025
669b899
add optional leveldb cache for validator pubkeys & activity
pk910 Feb 3, 2025
a845bb4
improve performance for activity cache
pk910 Feb 3, 2025
2d17c68
remove activity cache in leveldb (performance issues)
pk910 Feb 3, 2025
956e630
fix cache debug page
pk910 Feb 3, 2025
a8b8ba9
Merge branch 'master' into pk910/validators-in-db
pk910 Feb 7, 2025
3344068
add getter for validator status aggregation
pk910 Feb 7, 2025
898c4f9
fix validator status filter
pk910 Feb 7, 2025
5c4732d
make fork detection more resilent for non-permanent execution (dev se…
pk910 Feb 7, 2025
c196633
fix validators filter
pk910 Feb 7, 2025
ef7fffc
small fix for validator status aggregation
pk910 Feb 7, 2025
74e10b9
fix validator status sql query
pk910 Feb 7, 2025
d2d32db
code cleanup & performance improvements
pk910 Feb 7, 2025
2b482e6
fix panic in EpochStats.precomputeFromParentState
pk910 Feb 8, 2025
8f0c771
Merge branch 'master' into pk910/validators-in-db
pk910 Feb 13, 2025
a6fe27b
code cleanup & add comments
pk910 Feb 17, 2025
6506252
code cleanup
pk910 Feb 17, 2025
81774db
close leveldb on shutdown
pk910 Feb 17, 2025
f83ec3b
fix lock order
pk910 Feb 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/dora-explorer/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ func main() {

utils.WaitForCtrlC()
logger.Println("exiting...")
services.GlobalBeaconService.StopService()
db.MustCloseDB()
}

Expand Down
24 changes: 24 additions & 0 deletions db/schema/pgsql/20241216124511_validator-set.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
-- +goose Up
-- +goose StatementBegin

CREATE TABLE IF NOT EXISTS public."validators" (
validator_index BIGINT NOT NULL,
pubkey bytea NOT NULL,
withdrawal_credentials bytea NOT NULL,
effective_balance BIGINT NOT NULL,
slashed BOOLEAN NOT NULL,
activation_eligibility_epoch BIGINT NOT NULL,
activation_epoch BIGINT NOT NULL,
exit_epoch BIGINT NOT NULL,
withdrawable_epoch BIGINT NOT NULL,
CONSTRAINT validators_pkey PRIMARY KEY (validator_index)
);

CREATE INDEX IF NOT EXISTS "validators_pubkey_idx"
ON public."validators" ("pubkey");

-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
SELECT 'NOT SUPPORTED';
-- +goose StatementEnd
24 changes: 24 additions & 0 deletions db/schema/sqlite/20241216124511_validator-set.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
-- +goose Up
-- +goose StatementBegin

CREATE TABLE IF NOT EXISTS "validators" (
validator_index BIGINT NOT NULL,
pubkey BLOB NOT NULL,
withdrawal_credentials BLOB NOT NULL,
effective_balance BIGINT NOT NULL,
slashed BOOLEAN NOT NULL,
activation_eligibility_epoch BIGINT NOT NULL,
activation_epoch BIGINT NOT NULL,
exit_epoch BIGINT NOT NULL,
withdrawable_epoch BIGINT NOT NULL,
PRIMARY KEY (validator_index)
);

CREATE INDEX IF NOT EXISTS "validators_pubkey_idx"
ON "validators" ("pubkey");

-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
SELECT 'NOT SUPPORTED';
-- +goose StatementEnd
Loading