23.12.0

This is an optional update with bug fixes and performance improvements.

Note: The new --checkpoint-sync-url option is missing from the CLI help. The documentation for that flag can be found in our documentation page. The validator client in this release is not compatible with Lighthouse beacon nodes. If using this combination, please use release 23.11.0 of the validator client.

Downloads

• Available as 23.12.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 9b525f9207c69beeef80216fb1fd398c3fdfae866dbc36d4ee12a09f3c635784)
  • zip (sha256: 0c68d629b5c3daeb17fff4c12479ef7f13b9fec434a421081dd55ce1087552b4)

Known Issues

• (fixed in 23.12.1) Command line help not displaying --checkpoint-sync-url as an option. #7823
• (fixed in 23.12.1) Bug preventing node to startup when using --exit-when-no-validator-keys-enabled even with keys present. #7829
• (fixed in 23.12.1) Bug when node would not start if it failed downloading the deposit snapshot tree from Beacon API. #7827
• Using Teku Validator Client with Lighthouse beacon node won't work. Users using Teku Validator Client with Lighthouse Beacon Node should not upgrade to 23.12.0 or 23.12.1 #7842.

Breaking Changes

• By default, Teku won't allow syncing from genesis, users should use --checkpoint-sync-url when starting a new node. It is possible to revert back to the previous behaviour using the flag --ignore-weak-subjectivity-period-enabled.
• Teku will fail to start if the Validator API cannot read the password file used for authorization. Previously, if the Validator API was enabled but Teku couldn't read the password file, it would start but no request to the Validator API would be served.

Additions and Improvements

• Added support to new Beacon APIs publishBlindedBlockV2 and publishBlockV2 which introduce broadcast validation parameter.
• Added configuration attributes in support of honest validator late block reorg, which adds REORG_HEAD_WEIGHT_THRESHOLD, REORG_PARENT_WEIGHT_THRESHOLD, and REORG_MAX_EPOCHS_SINCE_FINALIZATION to phase 0 configurations. Mainnet values have been added as defaults for configurations that have not explicitly listed them.
• Added POST /eth/v1/beacon/states/{state_id}/validators beacon API.
• Added POST /eth/v1/beacon/states/{state_id}/validator_balances beacon API.
• Third-party library updates.
• Added --exit-when-no-validator-keys-enabled command line option.
• Updated Gnosis chain bootnodes.
• Support Basic authentication for external signer
• When initializing with the --checkpoint-sync-url option, the deposit tree snapshot will also be downloaded, rather than relying on older data packaged in the release.

Bug Fixes

• Fixed bugs when validator statuses were not updated and builder registrations were not sent in some cases

23.11.0

This is an optional update containing bug fixes, performance improments and security fixes.

Downloads

• Available as 23.11.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 75c1fdb4fa3c7b4e797e25d9ec92a21909aed905311f2e2eb252058046ee2d08)
  • zip (sha256: f38c5419199cb6600ee18956e35d3f2d279428716dbe6332938a7ad770d57e2e)

Breaking Changes

N/A

Additions and Improvements

• Updated dependencies.
• Added validator_duty_timer metrics recording the time to perform block_production, attestation_production and attestation_aggregation duties. Please note that this metric is not available by default and needs to be enabled using the --metrics-categories CLI option.
• Updated the default number of threads for batch signature verification based on CPUs available to help larger instances that may have more processing power available.
• Added /eth/v3/validator/blocks/{slot} experimental beacon API for block production
• Added --checkpoint-sync-url CLI option.
• Updated Netty HTTP2 codec version addressing CVE-2023-44487.

23.10.0

This is an optional update containing bug fixes and performance improvements.

Downloads

• Available as 23.10.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 0b0c71510780c0972a048ab2efa300b0ff584d9e5084b1e2f605af03aa4cbb31)
  • zip (sha256: 641c8fde7d2f338d60f91cece8c2dcbf53a62d3f80a54de6704b1ed99d71a6b0)

Breaking Changes

• The response of the Attestation Rewards method was updated to include two extra
  fields (inactivity and inclusion_delay) into the total_rewards and ideal_rewards objects.
  This should be considered a breaking change only if you are strictly
  checking the structure of the json in the response.

Additions and Improvements

• Solves an unintended breaking change in 23.9.1 release that included an updated LevelDB
  native library not compatible with older Linux distributions. LevelDB native library has
  been updated to support older GLIBC versions (ie Ubuntu 20.04 and Debian 11).
• Changed attestation duty calculation to improve performance.

Bug Fixes

• During network configuration load, all missing fields will now be reported, rather than just the first missing field causing failure.

Upcoming Breaking Changes

N/A

23.9.1

This is an optional update unless you are going to run a node on the new Holesky network. It contains the updated configuration for the Holesky launch on the 28th of September.

This release contains a leveldb library that requires GLIBC version 2.34.

• Linux users running on an OS that doesn’t provide this library won’t be able to run Teku and need to downgrade to 23.9.0.
• For reference, the minimum Debian version providing the library is 12 (bookworm), the minimum Ubuntu version is 22.0.4 (jammy).
• Windows users or users running teku using official docker images are not affected.

Downloads

• Available as 23.9.1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: b5381056e7d4bf2f118806ec8ffd0dcd586ada744712bc2873e7bc2cbe2c926e)
  • zip (sha256: 8191a3447ba58e8a07d6f938e69324a6bf812fb6e4b6f07e1a648b480af7eb15)

Breaking Changes

• Removed epoch from the POST /eth/v1/validator/liveness/{epoch} response as it was not part of the api spec.

Additions and Improvements

• The latest version of blst will automatically use optimized code paths if they are supported. As a result, JAVA_OPTS="-Dteku.portableBlst=true" is no longer necessary for some older systems.
• The voluntary exit subcommand now accepts --network=<NETWORK> command line option, using it to load the network specification rather than loading configuration from the rest api, if specified.
• Add /teku/v1/beacon/blob_sidecars/{slot} Teku API which returns all blob sidecars (canonical and non-canonical) at a specific slot. This endpoint will become available once Deneb fork is scheduled.
• Updated LevelDb native library which is using LevelDb 1.23 using latest Snappy to 1.1.10 for compression (this change doesn't apply to Windows)
• Updated holesky configuration with new genesis time of 28th September 12:00 UTC, and fork changes.
• Apply proposer boost to first block in case of equivocation (spec PR)

Bug Fixes

• When the rest-api's fail to start up they can now potentially 'fail fast' rather than silently ignoring the issue.

Upcoming Breaking Changes

N/A

23.9.0

This is a recommended update containing networking updates, optimizations and bug fixes. It also contains upcoming Holesky testnet configuration.

Downloads

• Available as 23.9.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: faa72f61accf98eb61de1129a80390373e196b4d8ec6ac0a58bac8aa0fc0e804)
  • zip (sha256: 3e6cc64e0c960c3c1d1197d46cdd49ddf1932c2c9eba0281ad1232b2c7fc32f2)

Breaking Changes

• The Development option --Xp2p-minimum-subnet-subscriptions has been removed and will no longer be recognised as a command line option.
• If running separated Beacon Node and Validator node it is required to upgrade Beacon Node first and then Validator Node. A Validator Node running this release won't start if connecting to an older Beacon Node version.

Known issues

• When switching an existing database to minimal mode, teku can become unresponsive. It will recover after cleaning up the old blocks, but this can take a period of time (an hour or so). If you wish to switch to minimal mode, consider re-initialising the database (delete beacon db and restart from a checkpoint sync) to avoid an outage.

Additions and Improvements

• Default storage mode has been changed from prune to minimal. Existing prune mode databases will need to explicitly set --data-storage-mode=minimal if they wish to switch to minimal data storage. Nodes starting with empty database will activate in minimal mode unless a different value is set via--data-storage-mode.
• Update attestation subnet subscriptions strategy according to the spec changes. All nodes (including non-validating ones) will subscribe to 2 subnets regardless of the number of validators.
• Added /eth/v1/validator/{pubkey}/voluntary_exit Validator API endpoint
• Add support for Holesky test network --network=holesky
• Add support for gzip encoding in REST API

Bug Fixes

• Fixed a bug in network configuration loader which was ignoring MIN_EPOCHS_FOR_BLOCK_REQUESTS parameter.
• Fixed a bug causing proposer config refresh feature to stop reloading the configuration.

Upcoming Breaking Changes

23.8.0

This is a recommended update containing few memory allocation optimizations. It also includes a move to Java 17 as the minimum supported Java version.

Downloads

• Available as 23.8.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 55b7c54aa21d0e31eb661f0e6734ab1214c34ce7d0fa7c45e6fb2ce3146636e9)
  • zip (sha256: 63536d689c51a1fd1ff9a535ef0e431d8bd943d0fa6321a86cfdf19673620f08)

Breaking Changes

• Upgrading the minimum Java version to Java 17, which means users will need to upgrade their Java installation to at least Java 17. The docker versions relying on jdk16 will no longer be published, so docker users explicitly referencing the jdk16 build need to update their package to reference jdk17, as tags develop-jdk16, develop-jdk16-arm64, latest-jdk16 will no longer be updated.
• Users who make heavy use of API calls to fetch non finalized states data other than head may wish to adjust the states-cache if they see excessive regeneration of state messages. This can be accomplished via the --Xstore-state-cache-size, which previously defaulted to 160.

Additions and Improvements

• Introduce --exchange-capabilities-monitoring-enabled parameter. If enabled, EL will be queried periodically for the Engine API methods it supports. If incompatibility is detected, a warning is raised in the logs. The default is true.
• Add support for Lukso network --network=lukso
• The development option --Xfork-choice-update-head-on-block-import-enabled was changed to default to false to ensure fork-choice is run when new blocks arrive.
• The default state-cache size has been changed to 8 (previously 160), and there is now an epoch-states-cache, which defaults to a maximum of 6 elements.

Bug Fixes

• Fixed a possibility of an overflow error when calculating validator duties if the number of active validators is >1M.

Upcoming Breaking Changes

23.6.2

This is an optional update with minor bug fixes and dependencies updates.

Downloads

• Available as 23.6.2 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: c0b021ead22a514fdb8d5e2ce072549dc447dfdeed660eccef0e3a42890e56c2)
  • zip (sha256: 103ae8d5b486f74f764db6432aa1e60300e9e18404b039fcce6f8f944f3239d4)

Breaking Changes

• --exchange-capabilities-enabled option has been removed since it is no longer applicable because of execution-apis spec change.
• Removed kiln and ropsten as --network options

Additions and Improvements

• --validators-external-signer-public-keys parameter now accepts external-signer value. It will enable public key retrieval from external signer standard API, making sure that configured keystore and trustStore will be used, if any.
• Stopped calling engine_exchangeTransitionConfigurationV1 Engine API method, since the method will be deprecated in the future.

Bug Fixes

• Fix Get Attestation Rewards API to fetch beacon state instead of block and beacon state (fixes #7338)

Upcoming Breaking Changes

• Upgrading the minimum Java version to Java 17, which means users will need to upgrade their Java installation to at least Java 17. The docker versions relying on jdk16 will no longer be published, so docker users explicitly referencing the jdk16 build need to update their package to reference jdk17, as tags develop-jdk16, develop-jdk16-arm64, latest-jdk16 will no longer be updated.

23.6.1

This is a required update for Gnosis network operators. It contains Gnosis Capella fork configuration.

Downloads

• Available as 23.6.1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 4a33cf38265ba1b5eb5142eaad230d4b3aaf5130bfaf54d7d37cca9eac10d64c)
  • zip (sha256: 81106eaf1ac11710a47459ee2a5e763478cbbe0e618fdfeb72995062c65e30fa)

Breaking Changes

• The Development options --Xprogressive-balances-mode and --Xee-version have been removed and will no longer be recognised as command line options.
• Network configs were updated following concensus-specs changes. If you run custom network, you will need to add lines with network parameters to your custom config (including the Deneb constants if required). If you are using a remote validator auto network feature, you will need to update both Beacon Node and Validator Client.

Additions and Improvements

• Added Gnosis configuration for the 🦉 Capella 🦉 network fork due at epoch 648704, UTC Tue 01/08/2023, 11:34:20
• Updated Sigmaprime's bootnodes
• Increased the pending pool size for attestations

Upcoming Breaking Changes

• Upgrading the minimum Java version to Java 17, which means users will need to upgrade their Java installation to at least Java 17. The docker versions relying on jdk16 will no longer be published, so docker users explicitly referencing the jdk16 build need to update their package to reference jdk17, as tags develop-jdk16, develop-jdk16-arm64, latest-jdk16 will no longer be updated.

23.6.0

This is a recommended update containing the rewards beacon-api endpoints, and an updated fix for the May non finality issue.

NOTE: Nethermind users on 1.19.X need to be at a minimum version of 1.19.2, due to compatibility issues with 1.19.0, 1.19.1.

Downloads

• Available as 23.6.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: aacd4a4b8396d793163e2cf061146bd1d5436e95440b2f4a8653ef6bc37b4707)
  • zip (sha256: 948abc377dd33560ead4ce94b634372b59c1a6fd61d2f9f92117c26454a0615f)

Breaking Changes

• The command argument --Xengine-exchange-capabilities has been removed, update to use --exchange-capabilities-enabled if you are using this option.
• The command argument --Xdeposit-snapshot-enabled has been removed, just remove it from commandline/configuration if you use it, updated argument --deposit-snapshot-enabled defaults to true now.
• The /eth/v1/debug/beacon/heads endpoint has been removed in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads.
• The /eth/v1/validator/blocks/:slot endpoint has been removed in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot.
• The /eth/v1/beacon/blocks/:block_id endpoint has been removed in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been removed in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/validator/liveness/:epoch endpoint was requiring the wrong body input and now conforms to the beacon-api spec.
• When --rest-api-enabled option is not specified and --rest-api-port is, --rest-api-enabled will now be set as true.
• Log messages previously printing WEI for builder bids and local payload values have been updated to print the ETH value.

Additions and Improvements

• Add support for Chiado (Gnosis testnet): --network=chiado
• Added beacon-apis rewards endpoints.
• Removed the experimental flag from /eth/v1/validator/liveness/:epoch rest-api endpoint.
• Following on from the non finality issue, improved state selection during attestation validation,
  where we failed to correctly identify that we could use the head state for validation of canonical attestation gossip.

Bug Fixes

• Fix a race condition on EL api result handling which may lead to beacon node remain syncing forever

Upcoming Breaking Changes

• Upgrading the minimum Java version to Java 17, which means users will need to upgrade their Java installation to at least Java 17. The docker versions relying on jdk16 will no longer be published, so docker users explicitly referencing the jdk16 build need to update their package to reference jdk17, as tags develop-jdk16, develop-jdk16-arm64, latest-jdk16 will no longer be updated.

Incident Statement - May 2023

As has been well documented by a post-mortem conducted by the Prysm team,
the Consensus layer suffered short periods of non-finality on the 11th and 12th of May, with the 12th of May being notable in that the network started an Inactivity leak.

To mitigate the problem, the Teku team released the 23.5.0 patch, which basically avoided processing some attestations due to
the observed behaviour of a number of clients suffering from memory contention if these attestations were fully validated by Teku.

This release contains a fix for the root cause of the issue, where the processing bug contributing to the original issue has been
rectified, greatly reducing the need for many states to be present while validating gossip.

23.5.0

This is a recommended update containing old attestation filtering.

Downloads

• Available as 23.5.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 9da9fc6c3a0e7ebbfe7c4599295c76113b3631c49da513a677dc35326c5a1a41)
  • zip (sha256: 725144706e5dddc1e5d1003a7e1c8ef12f2e815c93cda5273f109056bb6aa7b1)

Breaking Changes

Additions and Improvements

• Set User-Agent header to "teku/v" (e.g. teku/v23.5.0) when making builder bid requests to help builders identify clients and versions. Use --builder-set-user-agent-header=false to disable.
• Included more context when a request to an external signer fails.
• Added /eth/v1/beacon/rewards/blocks/{block_id} rest api endpoint.

Bug Fixes

• Filtering attestations with old target checkpoint

Upcoming Breaking Changes

• Upgrading source code to Java 17 meaning users will need to upgrade their Java install to at least 17, or use the jdk17 variant of the docker image.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads
• The command argument --Xengine-exchange-capabilities will be removed, update to use --engine-exchange-capabilities if you are using this option.
• The command argument --Xdeposit-snapshot-enabled will be removed, just remove it from commandline/configuration if you use it, updated argument --deposit-snapshot-enabled defaults to true now.