Releases · sigp/lighthouse

04 Mar 06:06

github-actions

v7.0.0-beta.2

9bb0e13

Frankenstein's Monster Pre-release

Pre-release

Summary

💡 Mainnet users can ignore this pre-release.

Summary

This is a second hot-fix release for the unsuccessful Electra upgrade on Holesky in February 2025. It contains patches to help reduce memory usage during non-finality, and to help nodes sync.

This release is only for Holesky users. Do not use this version on mainnet, Sepolia, or any other network.

Please see our informational issue for up-to-date advice on the Holesky situation: #7040

Forced Pseudo-Finalization

This release includes an experimental feature to forcefully "finalize" a block in Lighthouse's database so that Lighthouse can prune low-quality sidechains, remove finalized states and reclaim disk space. This feature is not safe in general, and should not be used except in emergencies.

Usage of the endpoint requires an epoch, block_root and state_root to pseudo-finalize, which must be from an epoch boundary. Skipped slots are OK, but the state_root must be the state_root of the epoch boundary state.

An example command to force pseudo-finalization on Holesky is:

curl -X POST --data '{"epoch": "117400", "state_root": "0x355fa23c9704fe346362c43a8fe43fba464fe63f20853bd3a87a8f465d52b4f4", "block_root": "0x06d788e593fd2b5b6fb6dcd63dfa4766201f05d948923dff6865d823246dd3c7" }' http://localhost:5052/lighthouse/finalize

If successful you should see a lot of logs of the form Pruning head, and then after some time (possibly multiple hours if your node knows all of the 800+ Holesky heads), it will complete. Completion is indicated via the debug log DEBG Database consolidation complete (look in $datadir/beacon/logs/beacon.log), and an update to the split field in the /lighthouse/database/info API.

Compaction will likely run after forced finalization, which is what reclaims the space permanently. If started you will see the log:

INFO Starting database compaction

When it completes (likely after ~1hr), you will see the log:

INFO Database compaction complete

If the finalization completes without compaction, you can trigger a manual compaction using the HTTP API:

curl -X POST http://localhost:5052/lighthouse/compaction

State Cache Tweaks

We've tweaked how old states interact with the state cache, removing several paths for state cache misses, and bad state cache interactions caused by BlocksByRange requests, sidechains and attestations to ancient blocks.

All of these optimisations are automatic and require no action from users to benefit. If you are feeling adventurous we've also found some benefit from using a new flag --state-cache-headroom 8, which prunes the state cache more aggressively (removing 8 states) when it gets full.

You may notice backtraces present in the State cache missed logs. These are harmless and do not indicate a fatal error. We added them to aid in debugging.

New HTTP endpoint `lighthouse/add_peer` to add trusted peer

This endpoint allows users to add a trusted peer to the peer database and dials it every heartbeat in case it gracefully disconnects. This is useful if the node struggles to find peers on the canonical chain. This can be used together with the --disable-discovery flag to limit the peers the node dials to speed up syncing to the right chain.

An example command to add a trusted peer:

curl -X POST -H "Content-Type: application/json" --data '{"enr": "enr:-Le4QLoE1wFHSlGcm48a9ZESb_MRLqPPu6G0vHqu4MaUcQNDHS69tsy-zkN0K6pglyzX8m24mkb-LtBcbjAYdP1uxm4BhGV0aDKQabfZdAQBcAAAAQAAAAAAAIJpZIJ2NIJpcIQ5gR6Wg2lwNpAgAUHQBwEQAAAAAAAAADR-iXNlY3AyNTZrMaEDPMSNdcL92uNIyCsS177Z6KTXlbZakQqxv3aQcWawNXeDdWRwgiMohHVkcDaCI4I"}' http://localhost:5052/lighthouse/add_peer

⚠️ Breaking Changes ⚠️

You should only upgrade to v7.0.0-beta.2 from v7.0.0-beta.{0,1} on Holesky. This release contains no breaking changes compared to v7.0.0-beta.1.

Please see the release notes for previous v7 releases for a summary of changes included in those releases:

Update Priority

This table provides priorities for which classes of users should update particular components.

User Class	Beacon Node	Validator Client
Staking Users (testnet)	High	High
Non-Staking Users (testnet)	High	---
Staking Users (mainnet)	DON'T UPDATE	DON'T UPDATE
Non-Staking Users (mainnet)	DON'T UPDATE	---

See Update Priorities for more information about this table.

Testnet users should update both the Lighthouse VC and BN to v7.0.0-beta.2 if using separate binaries. The execution layer client (e.g. Geth, Reth, Besu, Nethermind, Erigon) must also be updated prior to the Electra fork.

All Changes

Bump version to v7.0.0-beta.2 (#7073)
Manual compaction endpoint (#7072)
Add CI fixes to holesky-rescue (#7071)
Add http endpoint to add trusted peer (#7068)
Add backtrace logging. (#7063)
Prevent writing to state cache when migrating the database (#7067)
Split block root lookups between fork choice and store on BBR response (#7066)
Revert "Reuse milhouse subtrees to shrink inactivity_scores in memory (#7062)"
Reuse milhouse subtrees to shrink inactivity_scores in memory (#7062)
Manual finalization endpoint (#7059)
Change state cache size default to 32. (#7055)
Load block roots from fork choice where possible when serving BlocksByRange requests (#7058)
Optimise status processing for holesky-rescue (#7054)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

System	Architecture	Binary	PGP Signature
	x86_64	lighthouse-v7.0.0-beta.2-x86_64-apple-darwin.tar.gz	PGP Signature
	x86_64	lighthouse-v7.0.0-beta.2-x86_64-unknown-linux-gnu.tar.gz	PGP Signature
	aarch64	lighthouse-v7.0.0-beta.2-aarch64-unknown-linux-gnu.tar.gz	PGP Signature
	x86_64	lighthouse-v7.0.0-beta.2-x86_64-windows.tar.gz	PGP Signature

System	Option	-	Resource
	Docker	v7.0.0-beta.2	sigp/lighthouse

Assets 10

26 Feb 05:09

github-actions

v7.0.0-beta.1

dc320e3

Hemorrhage Pre-release

Pre-release

Summary

This is a hot-fix release for the unsuccessful Electra upgrade on Holesky in February 2025. It contains patches to help users avoid the invalid justified Holesky block and handle the resulting network turmoil (#7041).

This release is only for Holeksy users. Do not use this version on mainnet.

Please see our informational issue for up-to-date advice on the Holesky situation: #7040

Update Priority

This table provides priorities for which classes of users should update particular components.

User Class	Beacon Node	Validator Client
Staking Users	High (Holesky only)	High (Holesky only)
Non-Staking Users	High (Holesky only)	---

Do not use this version on mainnet

See Update Priorities more information about this table.

All Changes

Bump version
discard unused code
take config value
Bump sync-tolerance-epoch and make it a cli param
Add vc --disable-attesting flag
Ban peers with banned payloads
Add log
Blacklist invalid block root in block verification and blacklist invalid finalized epochs in sync.
Blacklist invalid block root in block verification and blacklist invalid finalized epochs in sync.
MORE
Allow invalidation of "valid" nodes
Implement invalidation API
Remove more liveness risks
lcli http-sync hacks
Disable liveness risk
Fix flag
Add flag to disable attestation APIs
Fix builder API headers (#7009)
Rust 1.85 lints (#7019)
Fix light client merkle proofs (#7007)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

System	Architecture	Binary	PGP Signature
	x86_64	lighthouse-v7.0.0-beta.1-x86_64-apple-darwin.tar.gz	PGP Signature
	x86_64	lighthouse-v7.0.0-beta.1-x86_64-unknown-linux-gnu.tar.gz	PGP Signature
	aarch64	lighthouse-v7.0.0-beta.1-aarch64-unknown-linux-gnu.tar.gz	PGP Signature
	x86_64	lighthouse-v7.0.0-beta.1-x86_64-windows.tar.gz	PGP Signature

System	Option	-	Resource
	Docker	v7.0.0-beta.1	sigp/lighthouse

Assets 10

13 Feb 07:34

github-actions

v7.0.0-beta.0

1888be5

Beta-Seven Pre-release

Pre-release

💡 Mainnet users can ignore this pre-release.

Summary

This beta release is high-priority for users on Holesky and Sepolia. It is a required upgrade for the upcoming Electra hard forks on these testnets.

Electra on Holesky: Mon 24 Feb 2025 21:55:12 UTC. Slot 3710976.
Electra on Sepolia: Wed 5 Mar 2025 07:29:36 UTC. Slot 7118848.

If you are running Lighthouse on a testnet, you must also upgrade your execution layer client to a Prague-Electra (Pectra) compatible release.

The Electra upgrade has not yet been scheduled on mainnet, so no action is required for mainnet users (do not upgrade).

Notable changes in Lighthouse v7.0.0-beta.0 include:

IPv6 enabled by default when a globally-routable IPv6 address is configured.
Light client server enabled by default.
Bugfix for a regression in attestation subscription logic.
Support for a new database backend, redb.

⚠️ Breaking Changes ⚠️

You can upgrade to v7.0.0-beta.0 from any v5 or v6 release. If you are upgrading from v5 you should make sure to read the v6 release notes to account for breaking changes between v5 and v6.

Upgrading to Lighthouse v7.0.0-beta.0 should be automatic for all users, as there are no changes to CLI flags that will prevent the node from starting. Holesky and Sepolia users must upgrade before the Electra fork. Failure to upgrade in time will require a re-sync.

Once you upgrade to Lighthouse v7.0.0-beta.0, you can downgrade to v6, but only prior to the Electra fork.

⚠️ Deprecated CLI Flags ⚠️

The following beacon node flags have been deprecated. You should remove them, but the beacon node will still start if they are provided.

--light-client-server

IPv6 by Default

Lighthouse will now automatically listen on IPv6 if it detects a globally-routable address. We expect for the majority of users with IPv4-only setups that this change will have no effect, but that it will benefit users with correctly configured IPv6 stacks.

The default IPv6 listening port has been changed from port 9090 to port 9000 (same as IPv4) to make firewalling easier. The IPv6 port can be adjusted using the flag --port6.

You can opt-out of IPv6 by using the flag --listen-address 0.0.0.0 to only listen on IPv4.

For more information, see the implementation PR:

#6808

🦀 Minimum Supported Rust Version 🦀

We have updated the Minimum Supported Rust Version (MSRV) for this release from 1.80.0 to 1.83.0.

This is only relevant to users compiling Lighthouse from source.

You can update your Rust compiler using:

rustup update stable

⚡ Electra ⚡

The Electra hard fork, paired with the Prague hard fork on the execution layer – together known as Pectra – brings several new features to Ethereum.

The headline change is known as Max EB, and raises the maximum effective balance a single validator may wield from 32 ETH to 2048 ETH. Once adopted, this will allow the network to run more efficiently with a lower validator count, while retaining the same level of security. Max EB even removes some centralisation vectors from staking incentives so that solo validators are able to tap into the compounding rewards previously enjoyed exclusively by large operators.

The process of switching a validator's max effective balance is a consolidation, which transfers stake from one validator to another. Consolidations are triggered via a smart contract call, and are fully opt-in and voluntary. If you are a solo operator with a small number of validators, there is no need to consolidate, although you may choose to do so.

We expect the tooling and documentation for consolidations to mature as Electra on mainnet approaches. We have plans of our own to update the Lighthouse UI (Siren) with consolidation support, and will make an announcement when that is ready.

Light Client Server

We have enabled the Light Client Server by default 🎉. Our implementation has reached a stage of maturity and performance where we feel comfortable rolling it out by default. It should result in negligible changes to bandwidth, CPU usage, memory and disk I/O for users.

The light client protocol allows very lightweight clients and devices to interact with Ethereum, and we are excited to see how the ecosystem evolves with more widespread protocol support.

You can opt-out of the light client server using --disable-light-client-server.

Attestation Subscription Fix

A fix was made to the beacon node's attestation subscription logic which could lead to under-subscription and issues publishing aggregate attestations. This has been addressed in the following PR:

#6890

New Database Backends

A new database backend is available in the form of redb. This is a pure-Rust database, with nice ACID properties. We hope to modernise and optimise our database usage around redb in the coming months.

For now, redb is performing slightly worse than LevelDB for some operations, and is only recommended for expert users and tinkerers. You can opt-in using --beacon-node-backend redb. This won't make use of any existing LevelDB database, nor will it delete it, so you should delete your LevelDB database manually and then checkpoint sync if you would like to switch.

🐛 Known Issues 🐛

There are no known issues at the time of writing.

Update Priority

This table provides priorities for which classes of users should update particular components.

User Class	Beacon Node	Validator Client
Staking Users (testnet)	High	High
Non-Staking Users (testnet)	High	---
Staking Users (mainnet)	DON'T UPDATE	DON'T UPDATE
Non-Staking Users (mainnet)	DON'T UPDATE	---

See Update Priorities for more information about this table.

Testnet users should update both the Lighthouse VC and BN to v7.0.0-beta.0 if using separate binaries. The execution layer client (e.g. Geth, Reth, Besu, Nethermind, Erigon) must also be updated prior to the Electra fork.

All changes

Release v7.0.0-beta.0 (#6962)
Fix light client plumbing in beacon processor (#6993)
Ensure GET v2/validator/aggregate_attestation is backwards compatible (#6984)
Address cargo audit failure RUSTSEC-2025-0006 (#6972)
IPv6 By Default (#6808)
Update EF tests to spec v1.5.0-beta.2 (#6958)
Sync active request byrange ids logs (#6914)
Enable Light Client server by default (#6950)
Schedule Sepolia and Holesky Electra forks (#6949)
Update attestation rewards API for Electra (#6819)
Fix aggregate attestation v2 response (#6926)
Remove duplicated fork_epoch and fork_version implementation (#6953)
Optimise and refine SingleAttestation conversion (#6934)
Fix fetch blobs in all-null case (#6940)
Keep execution payload during historical backfill when prune-payloads set to false (#6766)
Remove un-used batch sync error condition (#6917)
Remove unused metrics (#6817)
Reduce ForkName boilerplate in fork-context (#6933)
Use old geth version due to breaking changes. (#6936)
Fix attestation queue length metric (#6924)
Update metrics.rs (#6863)
Add individual by_range sync requests (#6497)
Return error if getBlobs not supported (#6911)
Add test to beacon node fallback feature (#6568)
Add check to Lockbud CI job (#6898)
UX Network Fixes (#6796)
chore: update peerDAS KZG library to 0.5.3 (#6906)
Migrate validator client to clap derive (#6300)
Use data column batch verification consistently (#6851)
Add builder SSZ flow (#6859)
Subscribe to PeerDAS topics on Fulu fork (#6849)
Fix subnet unsubscription time (#6890)
Cargo update for openssl vuln (#6901)
update libp2p to 0.55 (#6889)
update MSRV (#6896)
Compute columns in post-PeerDAS checkpoint sync (#6760)
Fix mdbook build. (#6891)
POST /eth/v2/beacon/pool/attestations bugfixes (#6867)
Cargo update without rust_eth_kzg (#6848)
Implement PeerDAS Fulu fork activation (#6795)
Make range sync chain Id sequential (#6868)
Underflow and Typo (#6885)
Increase jemalloc aarch64 page size limit (#5244) (#6831)
Some sync/backfill format nits (#6861)
Fork aware max values in rpc (#6847)
More gossipsub metrics (#6873)
Fix Redb implementation and add CI checks (#6856)
Detect invalid proposer signature on RPC block processing (#6519)
Add tests for ExecutionRequests decoding errors (#6832)
Update to EF tests v1.5.0-beta.1 (#6871)
Modularize beacon node backend (#4718)
Electra minor refactorings (#6839)
Update discv5 (#6836)
Avoid computing columns from EL blobs if block has already been imported (#6816)
Add MetaData V3 support to node/identity API (#6827)
Refactor mock builder (#6735)
Add EIP-7636 support (#6793)
Fix custodial peer assumption on lookup custody requests (#6815)
Do not send column requests if there is no blob for the block. (#6814)
SingleAttestation implementation (#6488)
Misc. dependency cleanup (#6810)
Remove ineffectual block RPC limits post merge (#6798)
Implement PeerDAS subnet decoupling (aka custody groups) (#6736)
Fix data columns not persisting for PeerDAS due to a getBlobs race condition (#6756)
Use existing peer count metrics loop to check for open_nat toggle (#6800)
Implement changes for EIP 7691 (#6803)
Execution requests with ...

Assets 10

16 Dec 06:25

github-actions

v6.0.1

0d90135

Rick In A Vat In The Garage Latest

Latest

Summary

This low-priority patch release fixes a few minor issues in the recent major release v6.0.0. If you have not yet upgraded to Lighthouse v6.0.0, we recommend skipping v6.0.0 and going straight to v6.0.1. These release notes provide a full description of the changes in v6.0.0 and v6.0.1, referred to collectively as v6.0.x.

Lighthouse v6.0.x includes new features and optimisations which are backwards-incompatible with Lighthouse v5.x.y.

After many months of testing, v6.0.x stabilises hierarchical state diffs, resulting in much more compact archive nodes! This long-awaited feature was also known as on-disk tree-states, and was available for pre-release testing in Lighthouse v5.1.222-exp.

Other notable changes in v6.0.x include:

Removal and deprecation of several old CLI flags. Some flags will need to be removed in order for Lighthouse to start, see below for a full list.
Improved beacon node failover and prioritisation in the validator client.
Support for engine_getBlobsV1 to speed up import and propagation of blobs.
Optimised peer discovery and long-term subnet subscription logic.
New commands for lighthouse validator-manager.
Improved light client support. Enabled via --light-client-server.
SSZ by default for blocks published by the VC.

Compared to v6.0.0, this release includes several bugfixes:

Fix an issue with attestation subnet subscriptions not being tracked correctly (#6682).
Fix the /lighthouse/nat API and NAT metrics (#6677).
Prevent sync from getting stuck on certain lookups (#6658).
Quality-of-life improvements for archive node users (#6669, #6668).

⚠️ Breaking Changes ⚠️

Upgrading to Lighthouse v6.0.x should be automatic for most users, but you must:

Remove any unsupported CLI flags (see below), and
Be aware of the one-way database migration and the changes to archive nodes.

Once you upgrade a beacon node to Lighthouse v6.0.x, you cannot downgrade to v5.x.y without re-syncing.

⚠️ Database Migration ⚠️

The beacon node database migration for v6.0.x is applied automatically upon upgrading. No manual action is required to upgrade.

There is no database downgrade available. We did not take this choice lightly, but in order to deliver hierarchical state diffs, a one-way database migration was simplest. If you do find yourself wanting to downgrade, re-syncing using checkpoint sync is highly recommended as it will get the node back online in just a few minutes.

For Archive Nodes

The migration enables hierarchical state diffs which necessitates the deletion of previously stored historic states. If you are running an archive node, then all historic states will be deleted upon upgrading. If you would like to continue running an archive node, you should use the --reconstruct-historic-states flag so that state reconstruction can restart from slot 0.

If you would like to change the density of diffs, you can use the new flag --hierarchy-exponents which should be applied the first time you start after upgrading. We have found that the hierarchy-exponents configuration does not greatly impact query times which tend to be dominated by cache builds and affected more by query ordering. We still recommend avoiding parallel state queries at the same slot, and making use of sequential calls where possible (e.g. in indexing services). We plan to continue optimising parallel queries and cache builds in future releases, without requiring a re-sync.

For more information on configuring the hierarchy exponents see the updated documentation on Database Configuration in the Lighthouse book.

Hierarchy Exponents	Storage requirement	Sequential slot query	Uncached query
5,9,11,13,16,18,21 (default)	418 GiB	250-700 ms	up to 10 s
5,7,11 (frequent snapshots)	589 GiB	250-700 ms	up to 6 s
0,5,7,11 (per-slot diffs)	1915 GiB+	250-700 ms	up to 2 s

As part of the archive node changes the format of the "anchor" has also changed. For an archive node the anchor will no longer be null and will instead take the value:

"anchor": {
  "anchor_slot": "0",
  "oldest_block_slot": "0",
  "oldest_block_parent": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "state_upper_limit": "0",
  "state_lower_limit": "0"
}

Don't be put off by the state_upper_limit being equal to 0: this indicates that all states with slots >= 0 are available, i.e. full state history.

NOTE: if you are upgrading from v5.1.222-exp you need to re-sync from scratch. The database upgrade will fail if attempted.

⚠️ Removed CLI Flags ⚠️

The following beacon node flags which were previously deprecated have been deleted. You must remove them from your beacon node arguments before updating to v6.0.x:

--self-limiter
--http-spec-fork
--http-allow-sync-stalled
--disable-lock-timeouts
--always-prefer-builder-payload
--progressive-balances
--disable-duplicate-warn-logs
-l (env logger)

The following validator client flags have also been deleted and must be removed before starting up:

--latency-measurement-service
--disable-run-on-all
--produce-block-v3

In many cases the behaviour enabled by these flags has become the default and no replacement flag is necessary. If you would like to fine-tune some aspect of Lighthouse's behaviour the full list of CLI flags is available in the book:

⚠️ Deprecated CLI Flags ⚠️

The following beacon node flags have been deprecated. You should remove them, but the beacon node will still start if they are provided.

--eth1
--dummy-eth1

The following global (BN and VC) flags have also been deprecated:

--terminal-total-difficulty-override
--terminal-block-hash-override
--terminal-block-hash-epoch-override
--safe-slots-to-import-optimistically

⚠️ Modified CLI Flags ⚠️

The beacon node flag --purge-db will now only delete the database in interactive mode, and requires manual confirmation. If it is provided in a non-interactive context, e.g. under systemd or docker then it will have no effect. The beacon node will start without anything being deleted.

If you wish to use the old purge-db behaviour it is available via the flag --purge-db-force which never asks for confirmation.

Network Optimisations

This release includes optimisations to Lighthouse's subnet subscription logic, updates to peer discovery, and fine-tuning of IDONTWANT and rate limiting.

Users should see similar performance with reduced bandwidth. Users running a large number of validators (1000+) on a single beacon node may notice a reduction in the number of subscribed subnets, but can opt-in to subscribing to more subnets using --subscribe-all-subnets if desired (e.g. for marginally increasing block rewards from included attestations).

Validator Client Fallback Optimisations

The beacon node fallback feature in the validator client has been refactored for greater responsiveness. Validator clients running with multiple beacon nodes will now switch more aggressively to the "healthiest" looking beacon node, where health status is determined by:

Sync distance (head distance from the current slot).
Execution layer (EL) health (whether the EL is online and not erroring).
Optimistic sync status (whether the EL is syncing).

The impact of this change should be less downtime during upgrades, and better resilience to faulty or broken beacon nodes.

Users running majority clients should be aware that in the case of a faulty majority client, the validator client may prefer the faulty chain due to it appearing healthier. The best defense against this problem is to run some (or all) validator clients without any connection to a beacon node running a majority CL client or majority EL client.

New Validator Manager Commands

The Lighthouse validator manager is the recommended way to manage validators from the CLI, without having to shut down the validator client.

In this release it has gained several new capabilities:

lighthouse vm import: now supports standard keystores generated by other tools like staking-deposit-cli.
lighthouse vm list: a new read-only command to list the validator keys that a VC has imported.
lighthouse vm delete: a new command to remove a key from a validator client, e.g. after exiting.

For details on these commands and available flags, see the docs: https://lighthouse-book.sigmaprime.io/validator-manager.html

Future releases will continue to expand the number of available commands, with the goal of eventually deprecating the previous lighthouse account-manager CLI.

Fetch Blobs Optimisation

This release supports the new engine_getBlobsV1 API for accelerating the import of blocks with blobs. If the API is supported by your execution node, Lighthouse will use it to load blobs from the mempool without waiting for them to arrive on gossip. Our testing indicates that this will help Ethereum scale to a higher blob count, but we need more data from real network...

Assets 10

02 Dec 05:57

github-actions

v6.0.0

c042dc1

Tiny Rick

Summary

This low-priority major release introduces new features and optimisations which are backwards-incompatible with Lighthouse v5.x.y. If you are running Nethermind we recommend waiting for the release of Nethermind v1.30.0 before upgrading due to an incompatibility (see Known Issues).

After many months of testing, this releases stabilises hierarchical state diffs, resulting in much more compact archive nodes! This long-awaited feature was also known as on-disk tree-states, and was available for pre-release testing in Lighthouse v5.1.222-exp.

Other notable changes include:

Removal and deprecation of several old CLI flags. Some flags will need to be removed in order for Lighthouse to start, see below for a full list.
Improved beacon node failover and prioritisation in the validator client.
Support for engine_getBlobsV1 to speed up import and propagation of blobs.
Optimised peer discovery and long-term subnet subscription logic.
New commands for lighthouse validator-manager.
Improved light client support. Enabled via --light-client-server.
SSZ by default for blocks published by the VC.

⚠️ Breaking Changes ⚠️

Upgrading to Lighthouse v6.0.0 should be automatic for most users, but you must:

Remove any unsupported CLI flags (see below), and
Be aware of the one-way database migration and the changes to archive nodes.

Once you upgrade a beacon node to Lighthouse v6.0.0, you cannot downgrade to v5.x.y without re-syncing.

⚠️ Database Migration ⚠️

The beacon node database migration for v6.0.0 is applied automatically upon upgrading. No manual action is required to upgrade.

For Archive Nodes

For more information on configuring the hierarchy exponents see the updated documentation on Database Configuration in the Lighthouse book.

Hierarchy Exponents	Storage requirement	Sequential slot query	Uncached query
5,9,11,13,16,18,21 (default)	418 GiB	250-700 ms	up to 10 s
5,7,11 (frequent snapshots)	589 GiB	250-700 ms	up to 6 s
0,5,7,11 (per-slot diffs)	1915 GiB+	250-700 ms	up to 2 s

As part of the archive node changes the format of the "anchor" has also changed. For an archive node the anchor will no longer be null and will instead take the value:

"anchor": {
  "anchor_slot": "0",
  "oldest_block_slot": "0",
  "oldest_block_parent": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "state_upper_limit": "0",
  "state_lower_limit": "0"
}

Don't be put off by the state_upper_limit being equal to 0: this indicates that all states with slots >= 0 are available, i.e. full state history.

NOTE: if you are upgrading from v5.1.222-exp you need to re-sync from scratch. The database upgrade will fail if attempted.

⚠️ Removed CLI Flags ⚠️

The following beacon node flags which were previously deprecated have been deleted. You must remove them from your beacon node arguments before updating to v6.0.0:

--self-limiter
--http-spec-fork
--http-allow-sync-stalled
--disable-lock-timeouts
--always-prefer-builder-payload
--progressive-balances
--disable-duplicate-warn-logs
-l (env logger)

The following validator client flags have also been deleted and must be removed before starting up:

--latency-measurement-service
--disable-run-on-all
--produce-block-v3

⚠️ Deprecated CLI Flags ⚠️

The following beacon node flags have been deprecated. You should remove them, but the beacon node will still start if they are provided.

--eth1
--dummy-eth1

The following global (BN and VC) flags have also been deprecated:

--terminal-total-difficulty-override
--terminal-block-hash-override
--terminal-block-hash-epoch-override
--safe-slots-to-import-optimistically

⚠️ Modified CLI Flags ⚠️

If you wish to use the old purge-db behaviour it is available via the flag --purge-db-force which never asks for confirmation.

Network Optimisations

This release includes optimisations to Lighthouse's subnet subscription logic, updates to peer discovery, and fine-tuning of IDONTWANT and rate limiting.

Validator Client Fallback Optimisations

Sync distance (head distance from the current slot).
Execution layer (EL) health (whether the EL is online and not erroring).
Optimistic sync status (whether the EL is syncing).

The impact of this change should be less downtime during upgrades, and better resilience to faulty or broken beacon nodes.

New Validator Manager Commands

The Lighthouse validator manager is the recommended way to manage validators from the CLI, without having to shut down the validator client.

In this release it has gained several new capabilities:

lighthouse vm import: now supports standard keystores generated by other tools like staking-deposit-cli.
lighthouse vm list: a new read-only command to list the validator keys that a VC has imported.
lighthouse vm delete: a new command to remove a key from a validator client, e.g. after exiting.

For details on these commands and available flags, see the docs: https://lighthouse-book.sigmaprime.io/validator-manager.html

Future releases will continue to expand the number of available commands, with the goal of eventually deprecating the previous lighthouse account-manager CLI.

Fetch Blobs Optimisation

There are several new Prometheus metrics to track the hit rate:

beacon_blobs_from_el_received_total
beacon_blobs_from_el_expected_total
beacon_blobs_from_el_hit_total

Logs at debug level also show the operation of this new feature (grep for fetch_blobs).

At the time of writing the follow execution clients support the API:

Reth v1.0.7 or newer.
Besu v24.9.1 or newer.
Geth v1.14.12 or newer.

Unsupported:

Nethermind v1.29.x (buggy, see below).
Erigon.

🐛 Known Issues 🐛

Nethermind v1.29.x `engine_getBlobsV1` bug

Nethermind versions v1.29.0 and v1.29.1 include ...

Assets 10

08 Aug 05:49

github-actions

v5.3.0

d6ba8c3

Risotto Groupon

Summary

This medium-priority release contains performance optimisations, bugfixes and new features. There are several breaking changes detailed below.

The main optimisations are:

Faster beacon node start-up due to an optimisation to the database schema (#5897). The schema migration is a backwards-incompatible change which is described in more detail below.
More reliable block proposals due to the use of the new v3 block production API (#5292).
Improvements to the validator client's broadcast mechanism (#5976, #6223).
Support for gossipsub IDONTWANT (#5422). This allows nodes to reduce their bandwidth usage by avoiding the download of duplicate messages.

The main bugfixes are:

Fix parsing of the VC proposer-nodes CLI flag (#6125).
Prevent connections from peers with a banned IP history (#6008).
Prevent UPnP queries when UPnP is disabled (#6170).
Corrections to lighthouse account CLI (#6153, #6091).
Update Sepolia bootnodes, fixing issues with 0 peers on Sepolia (#6037).

New features include:

A new Rust slasher database backend, redb (#4529). We still recommend running LMDB on mainnet, but encourage users to try redb on testnets using the flag --slasher-backend redb.
The default number of log files retained (logfile-max-number) has been increased from 5 to 10 to help with debugging issues (#6092).
Lots of code for the upcoming Electra and PeerDAS network upgrades (#5712, #5741, #5761, #5743).

Breaking Changes

🗑️ Removed CLI flags

The following CLI flags have been removed completely and will cause the BN to fail to start up if provided:

--builder-profit-threshold: This flag was already deprecated and has been incompatible since v5.2.0 (#6131).

You should remove any instances of these flags from your beacon node startup command.

👎 Deprecated CLI flags

Beacon node:

--disable-lock-timeouts: Lock timeouts are now disabled by default so this flag does nothing. It will be removed in a future release (#6048).
--self-limiter: The self limiter is now enabled by default. This flag will be removed in a future release. The purpose of this is to prevent Lighthouse from sending excessive requests to peers in case of bugs or edge case scenarios. The rate limit quotas can be configured via the existing --self-limiter-protocols flag (#6093).

Validator client:

--produce-block-v3: The produce block v3 API is now enabled by default and cannot be disabled, so this flag does nothing. It will be removed in a future release (#5292).

You should remove any instances of these flags from your BN/VC commands, although failing to do so will only result in a warning, not a failure to start.

🦀 Minimum Supported Rust Version 1.78.0 🦀

The minimum supported Rust version (MSRV) was updated to 1.78.0. Users who compile from source (i.e., not Docker or pre-built binary users) will receive the following error if they are using an earlier version of Rust:

lighthouse v5.3.0 (/home/sigp/lighthouse/lighthouse) cannot be built because it requires rustc 1.78 or newer

Users can typically obtain the latest version of Rust by running rustup update.

💾 Database Schema Upgrade

The beacon node database schema has been updated from v19 to v21. An automatic database schema upgrade will run when updating from a previous version. No manual intervention is required for upgrading.

The schema upgrade was required to prepare for the upcoming Electra fork, and to optimise how validator public keys are stored on disk. Public keys are now stored uncompressed, which uses twice as much space, but allows them to be loaded around 10x faster. This results in shorter start-up times after the initial start-up on v5.3.0.

If you decide to downgrade from v5.3.0, then a database schema downgrade is available. It must be run using the lighthouse db migrate command. For instructions on how to downgrade, see the Database Migrations section of the Lighthouse Book.

More details on the pubkey optimisation are found in the PR: #5897.

🐋 No more `-portable` tarballs or `-modern` images

There are no more -portable tarballs attached to this release, and the -modern Docker images on Docker Hub will no longer be updated. Both have been deprecated since v5.2.0 when portable builds became the default everywhere.

For portable binary users:

Download the latest .tar.gz for your platform. It will continue to work exactly as before.

For modern Docker image users:

Use the :latest tag, or the :v5.3.0 tag. It will be just as fast as the previously supplied -modern image.

📝 HTTP API `/eth/v1/beacon/deposit_snapshot` to use JSON response by default

The /eth/v1/beacon/deposit_snapshot now returns JSON response by default (instead of SSZ) if a header is not specified, for consistency with other beacon APIs.

See: #5813.

🐧 Jemalloc enabled by default on non-Windows targets

jemalloc is now necessary on Linux with the introduction of tree-states for performance reasons. The jemalloc feature is now enabled by default on non-Windows targets.

See: #5995.

🔓 Removal of signed validator client responses

Signing of validator client HTTP responses has been removed. This feature had limited utility and adoption, and removing it has simplified VC API token management. Unless you were relying on the Signature header in VC responses, no action is required.

See: #5529.

💵 Fee recipient required with `--always-prepare-payload`

The --suggested-fee-recipient flag is now required when --always-prepare-payload is set. This feature is likely only used by block builders and relays.

See: #6079.

Update Priority

This table provides priorities for which classes of users should update particular components.

User Class	Beacon Node	Validator Client
Staking Users	Medium	Medium
Non-Staking Users	Medium	---

See Update Priorities for more information about this table.

Lighthouse BNs and VCs from v5.0.0, v5.1.x, v5.2.x and v5.3.x are compatible. However, we recommend that users update both the VC and BN to v5.3.0 if upgrading.

All Changes

Release v5.3.0 (#6194)
Broadcast VC requests in parallel and fix subscription error (#6223)
Don't expect DAS config in HTTP spec response (#6221)
Downgrade re-org log to INFO (#6220)
patch quick-protobuf (#6217)
Work around UB in LMDB bindings (#6211)
Avoid acquiring another read lock while holding one to avoid potential deadlock (#6200)
Swap finalized chains based on processed batches (#6203)
Use upgradable read lock for pubkey cache (#6190)
default vc to block v3 endpoint and deprecate block-v3 flag (#5292)
Remove timeout locks (#6048)
limit dial concurrency on Swarm (#6184)
Fix lighthouse account validator subcommands help text (#6091)
Iterate expired components from the da_checker (#5895)
Rust 1.80.0 lints (#6183)
Fix unexpected Marking peer disconnected in DHT (#6140)
Add DataColumnSidecar gossip topic and message handling (#6147)
Use then instead of then_some when checking upnp_enabled to avoid useless UPnP query (#6170)
Remove backwards compatibility for el_offline and is_optimstic (#6168)
Show correct total validator count during validator recovery (#6153)
Update msrv to 1.78.0 (#6159)
Revert --image-download always flag to avoid CI getting rate limited (#6163)
Siren Docs Upgrade (#5979)
Remove use of ethers core (#5301)
level down libp2p quic accept and close errors (#6148)
Make proposer duties work pre-genesis (#4789)
Update is_available check to support PeerDAS. (#6076)
Add PeerDAS presets and configs. (#6127)
Upgrade openssl to address cargo audit issue (#6143)
Attempt to fix flaky basic sim test (#6134)
Delete --builder-profit-threshold flag (#6131)
Return syncing on HTTP when sync is stalled (#6129)
Add an option to keep existing enclave when starting local testnet (#6065)
Add requires suggested-fee-recipient flag when always-prepare-payload is set (#6079)
Disable slasher default features (#6115)
Fix proposer-nodes cli flag name (#6125)
chore: update the trusted setup to match the consensus-specs version (#6117)
Track store metrics by DB column (#6041)
Ef tests electra (#5758)
Remove VC response signing and fix HTTP error handling (#5529)
Remove trace logging (#6103)
Measure consensus verification time (#6089)
Add block_gossip Beacon API events (#5864)
Bump default logfile-max-number to 10 (#6092)
Beacon api + validator electra (#5744)
Enable the o...

Assets 10

28 Jun 04:02

github-actions

v5.2.1

9e12c21

Kyle

Summary

This medium-priority release contains bugfixes for issues identified in v5.2.0. Users who have experienced sync issues with v5.2.0 are recommended to upgrade.

The list of bugfixes includes:

Prevention of forwards sync stalls after restarting, through more robust handling of peer disconnects and errors (#5969, #5945, #5993).
Prevention of backfill sync stalls (#5969).
Removal of unnecessary warnings for stale blobs propagated on gossip by other clients (#5935). The log message WARN Could not verify blob sidecar for gossip. Ignoring the blob sidecar should no longer be seen.
Correct sizing of the slasher database for the growing validator sets on mainnet and Holesky (#5934).

Breaking Changes

There are no breaking changes between v5.2.0 and v5.2.1. Please see the release notes for v5.2.0 if you are upgrading from a prior 5.x.x release.

Known Issues

Teku 24.10 BN Incompatibility

Beacon nodes running Teku 24.10.x and newer are not compatible out of the box with the validator client from this release. If you want to run Lighthouse VC v5.2.x with Teku BN 24.10.x you must use the --produce-block-v3 flag with Lighthouse. Upgrading to Lighthouse v5.3.0 is preferable as it fixes the incompatibility without requiring any flags.

Nethermind v1.26.0 and Reth v1.0.0 `engine_getClientVersionV1` bug

Nethermind v1.26.0 and Reth v1.0.0 include an implementation of engine_getClientVersionV1 which is not compliant with the API specification and does not work with Lighthouse. We recommend upgrading Nethermind to v1.27.0. At time of writing there is no compatible Reth release available.

The warning generated by a non-compliant implementation is:

WARN Execution engine call failed error: InvalidClientVersion("Input must be exactly 8 characters long (excluding any '0x' prefix)"), service: exec

This warning can be safely ignored, although it may cause a slight degradation to node performance as Lighthouse flip-flops the EL's state between healthy and unhealthy.

See NethermindEth/nethermind#7013 and paradigmxyz/reth#9137 for details.

Source builds without `jemalloc` will OOM on Linux

We've become aware of OOM issues for Lighthouse builds from source that disable the jemalloc feature on Linux. This is due to GNU malloc not handling the allocation of many tree nodes as well as other allocators (jemalloc, the Windows allocator). If you are building from source for Linux ensure that you include jemalloc in FEATURES (enabled by default). If you build the Dockerfile use --build-arg FEATURES=jemalloc (not enabled by default).

This will be addressed in v5.3.0 by an already-merged PR: #5995

Update Priority

This table provides priorities for which classes of users should update particular components.

User Class	Beacon Node	Validator Client
Staking Users	Medium	Low
Non-Staking Users	Medium	---

See Update Priorities for more information about this table.

Lighthouse BNs and VCs from v5.0.0, v5.1.x and v5.2.x are compatible. However, we recommend that users update both the VC and BN to v5.2.1 if upgrading.

All changes

Release v5.2.1 (testing branch) (#5989)
Attempt to continue lookups after adding peers (#5993)
Remove all batches related to a peer on disconnect (#5969)
Update slasher DB size and Lighthouse book (#5934)
Ensure all handler events are covered (#5945)
report failed requests from RPC after being clear from rate limiter but not yet sent. (#5942)
Lower tolerance of stale blobs on gossip (#5935)
chore: simplify method to generate a random valid blob (#5946)
chore: change impl Into<T> for U to impl From<U> for T (#5948)
Update CI nethermind and LLVM versions (#5933)
Fix cargo audit: update curve25519-dalek (#5959)
Update local testnet documentation (#5896)
Fix SelfRateLimiter breaks the sequence of delayed_requests. (#5903)
Verify inclusion proof should not be fallible (#5787)
Remove extern crate (#5922)
Add FUNDING.json for DRIPS (#5932)
Fix CommandLineTest port conflicts on CI (#5908)
Remove some easy Electra TODOs (#5928)
Rust 1.79 lints (#5927)
Fix panic on startup in debug build (#5917)
Fix skip-ci on stable (#5909)
Beta compiler fixes (#5916)

See full changelog here.

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

System	Architecture	Binary	PGP Signature
	x86_64	lighthouse-v5.2.1-x86_64-apple-darwin.tar.gz	PGP Signature
	x86_64	lighthouse-v5.2.1-x86_64-apple-darwin-portable.tar.gz	PGP Signature
	x86_64	lighthouse-v5.2.1-x86_64-unknown-linux-gnu.tar.gz	PGP Signature
	x86_64	lighthouse-v5.2.1-x86_64-unknown-linux-gnu-portable.tar.gz	PGP Signature
	aarch64	lighthouse-v5.2.1-aarch64-unknown-linux-gnu.tar.gz	PGP Signature
	aarch64	lighthouse-v5.2.1-aarch64-unknown-linux-gnu-portable.tar.gz	PGP Signature
	x86_64	lighthouse-v5.2.1-x86_64-windows.tar.gz	PGP Signature
	x86_64	lighthouse-v5.2.1-x86_64-windows-portable.tar.gz	PGP Signature

System	Option	-	Resource
	Docker	v5.2.1	sigp/lighthouse

Assets 18

11 Jun 03:20

github-actions

v5.2.0

f1d88ba

Tree People

Summary

This medium-priority release contains a major improvement to how states are managed in memory (tree-states) and an optimised epoch and block processing implementation. The beacon node now uses considerably less memory than the previous version thanks to more efficient state caching. It also includes other performance improvements, bug fixes and groundwork preparation for the upcoming Electra fork and PeerDAS upgrade.

Notable changes in this release:

Epoch and block processing have been optimized, resulting in improved performance and security. The optimized epoch processing implementation is accompanied by a proof of correctness detailed in this post (#5279).
Introduce in-memory tree-states, providing more efficient state caching, improved overall performance and reduced memory usage (#5533).
Add server-sent events (SSE) for proposer slashing, attester slashing, and BLS to execution change (#5327, #5857).
Encode execution engine client version in block graffiti to probe execution layer client diversity, more details below (#5290).
Add new timing metrics for block availability to better reflect the actual delays caused by blob processing and other factors (#5510).
Update the proposer re-orgs feature to align with the latest spec (#5417).
Introduce new containers, presets, and configurations in preparation for the Electra fork (#5122, #5606, #5607, #5630, #5652, #5653).
Refactor and simplifications in sync and block lookup logic in preparation for PeerDAS (#5561, #5562, #5563, #5564, #5583, #5655).
Add a newbuilder-header-timeout flag, which allows for configuring timeouts for the get builder header API call (#5857)
Fix attestations not getting added to the aggregation pool. This improves the quality of aggregate attestations Lighthouse produces (#5863).

There are a few breaking changes. Please see the section on Breaking Changes below.

In-memory tree states (#5533)

This release introduces the memory-only portion of tree-states, marking a significant upgrade to state management. The beacon node can now hold ~64 times more states with approximately the same memory cost, significantly reducing memory usage and minimizing spikes. This enhancement also improves overall performance. Consequently, the default value for the --state-cache-size beacon node flag has been increased from 3 to 128. We recommend that users who had manually adjusted the state cache size try the new larger default.

This development brings us closer to the full implementation of tree-states (both in-memory and on-disk) in Lighthouse, an upgrade that has been in progress for over two years. Stay tuned for the remaining tree-states upgrades and their benefits in upcoming releases!

Encode execution engine client version in block graffiti (#5290)

To support better analysis of execution client diversity, Lighthouse now supports the engine_getClientVersionV1 API.

If no user graffiti is provided, and the method is supported by the execution node, then Lighthouse will use graffiti of the form:

GEab48LH5c2c

Here GE is the execution client shortcode corresponding to Geth, ab48 are the first 4 characters of Geth's commit hash, LH indicates Lighthouse, and 5c2c are the first 4 character's of Lighthouse's commit hash.

We expect execution clients to gradually gain support for this new API. Check your execution client's release notes for details. Note that the latest version of Nethermind (v1.26.0) includes a broken version of this API, so we recommend waiting until v1.27.0 is released before upgrading.

`portable` builds for Docker and binary releases (#5614, #5615)

All Lighthouse binaries and Docker images are now built using portable BLS. For backwards-compatibility *-modern Docker tags still exist as aliases. Similarly, binary tarballs for the -portable suffix are also attached to this release, but are identical to the unsuffixed binary tarballs. We recommend that automated update scripts switch over to using the :latest Docker image or the unsuffixed binary tarballs so we can remove the redundancy and streamline the available images and binaries in a future release.

The switch to portable is motivated by a change in the blst library to use runtime detection of hardware features. This puts portable builds on par for performance with modern builds, and simplifies the release update process for users and devs.

`builder-header-timeout` flag (#5857)

Lighthouse previously used a static timeout of 1.0s when requesting an execution payload header from an mev-boost relay. The new builder-header-timeout flag for the beacon node allows this value to be adjusted using a value in milliseconds, e.g. --builder-header-timeout 500 will set the timeout to 0.5s. This can be beneficial if communicating with a faraway relay which sometimes times out.

Breaking Changes

Several breaking changes have been made to CLI flags as part of a cleanup and modernisation effort. Please make sure to remove or update these flags as part of upgrading. The changes are described in detail below but the affected flags are:

VC: latency-measurement-service
BN: progressive-balances, http-spec-fork, http-allow-sync-stalled, self-limiter, inbound-rate-limiter.

We have avoided making breaking changes to the database schema in this release so that users can upgrade and downgrade more simply. The next release will include a number of on-disk changes.

🌲 If you were already running the tree-states beta (v5.1.222-exp), then the database schema for v5.2.0 is not compatible with your existing database. You need to either stay on that release, or wait for a new version that incorporates the on-disk changes (maybe v5.3.0). Unfortunately a re-sync will be required eventually, as the stable form of the tree-states DB will be different from the beta.

🦀 Minimum Supported Rust Version 1.77 🦀

The minimum supported Rust version (MSRV) has been updated to 1.77.0. Users who compile from source (i.e., not Docker or pre-built binary users) will receive the following error if they are using an earlier version of Rust:

lighthouse v5.2.0 (/home/sigp/lighthouse/lighthouse) cannot be built because it requires rustc 1.77 or newer

Users can typically obtain the latest version of Rust by running rustup update.

Deprecated `latency-measurement-service` flag

The validator client --latency-measurement-service flag is now deprecated in favour of --disable-latency-measurement-service. By default the latency measurement service is enabled, and can be disabled by providing the new --disable-latency-measurement-service flag. Using the prior --latency-measurement-service flag will have no effect and will become a hard error in a future release. This change was made as part of an upgrade to a new version of our CLI parsing library, Clap, in #5273.

Deprecated `http-spec-fork` and `http-allow-sync-stalled` flags (#5500)

Lighthouse now ignores the http-spec-fork value and keeps the default spec fork, as this option was rarely used.

The http-allow-sync-stalled flag has been deprecated and is now the default behaviour as Lighthouse now prioritises liveness for the HTTP API. These flags will be removed in upcoming releases.

See #5430 and #5500 for more details.

Deprecated `--progressive-balances` flag (#5279)

This optimisation (--progressive-balances=fast) is now the default and cannot be disabled. See #5279 for more details. A warning will be logged if the flag is provided, and in a future version we will remove the flag entirely.

Removed block processing metrics (#5510)

The block import metrics have been reworked to more accurately measure import delays and provide more insights into block availability and readiness for attestation.

The following block processing metrics have been removed:

beacon_block_observed_slot_start_delay_time
beacon_block_imported_observed_delay_time
beacon_block_head_imported_delay_time
beacon_block_head_slot_start_delay_time
beacon_block_head_slot_start_delay_exceeded_total
beacon_block_gossip_propagation_verification_delay_time
beacon_block_gossip_slot_start_delay_time
beacon_block_last_delay
beacon_block_gossip_arrived_late_total
beacon_blob_gossip_propagation_verification_delay_time
beacon_blob_gossip_slot_start_delay_time
beacon_blob_last_delay

For more details and the updated metrics, see #5510.

MDBX excluded from the Docker build (#5614)

The slasher-mdbx feature is no longer included as part of our standard Docker images.

The default slasher backend was changed from MDBX to LMDB in Lighthouse v4.3.0 for more stability and has been the recommended option. Advanced users can still use the slasher-mdbx feature with a custom Docker build.

Modified `self-limiter` flag

_This flag is mostly useful for developers and probably shouldn't be part of y...

Assets 18

28 Mar 07:43

github-actions

v5.1.3

3058b96

Gul'karna

🔥 Hotfix 🔥

This hotfix release includes several bug fixes related to the handling of blobs. Upgrading to this release should result in less cache misses, lower peak memory usage, reduced bandwidth and greater stability.

This is a medium priority release.

We recommend that all users upgrade at their convenience. We particularly recommend upgrading if you are seeing either of these logs:

Mar 13 10:26:50.229 WARN Peer sent invalid response to parent request., reason: ExtraBlocksReturned, peer_id: ..., service: sync

Mar 24 23:20:56.001 WARN Did not advance head state reason: Err(HeadMissingFromSnapshotCache(...)), service: state_advance

Summary

Duplicate lookup bugfix

Since the release of v5.1.2 we have found a few more cases where Lighthouse could issue an excessive number of lookup requests for the same block. This would manifest as a warning log, followed by a loss of peers due to downscoring:

Mar 13 10:26:50.229 WARN Peer sent invalid response to parent request., reason: ExtraBlocksReturned, peer_id: ..., service: sync

The additional cases were related to concurrent lookups, and have been addressed in pull request #5488.

Blob RPC bugfix

We found that when Lighthouse imported a block after a blob RPC request, it was failing to run fork choice. This had several knock-on effects: the head state would be absent from the snapshot cache, which would lead to cache misses and further errors. In some instances, the cache misses would lead to a spike in memory usage while snapshots were reloaded from disk.

Fork choice now runs after all block imports (implemented in #5475). Some snapshot cache misses may still occur. We are planning to further improve snapshot handling in a future release.

Attestation cache size

The size of the seen-cache for attestation gossip has been updated in pull request #5493 to accomodate the wider validity window for attestations post-Deneb. This should result in reduced load by preventing old duplicate attestations from being reprocessed.

Light client server

The experimental light client server has been updated with Capella and Deneb support in pull request #4946. We do not yet recommend enabling it on mainnet. We will provide an update on the light client server in a future release.

Breaking Changes

None.

🐛 Known Issues 🐛

There is an outstanding bug in Lighthouse's UPnP implementation:

#5444

It can be worked around by running with --disable-upnp and manual port forwarding. We are planning another release containing a fix soon.

Update Priority

This table provides priorities for which classes of users should update particular components.

User Class	Beacon Node	Validator Client
Staking Users	Medium	Medium
Non-Staking Users	Medium	---

See Update Priorities for more information about this table.

Lighthouse BNs and VCs from v5.0.0 and v5.1.0/v5.1.1/v5.1.2/v5.1.3 are intercompatible. However, we recommend that users update both the VC and BN to v5.1.3 if upgrading.

All Changes

Release v5.1.3 (#5497)
Bump duplicate cache time (#5493)
Lookup log improvements (#5491)
Single lookup improvements (#5488)
Run fork choice after RPC blob import (#5475)
Move gossipsub into a separate crate (#5401)
Fix double counted metrics (#5476)
Add Capella & Deneb light client support (#4946)
Delete PRE_CAPELLA_ENGINE_CAPABILITIES (#5406)
Built-in documentation text width in Lighthouse book (#5394)
Verify whether validators really are unknown during sync committee duty API request (#5174)
Make sure all geth processes are killed when stopping a local testnet (#5383)
Delete unused incomplete_processing_components (#5418)
Fix one and hide all beacon-processor flags (#5397)
chore: reduce scope of commitment (#5426)
chore: remove stale comment (#5440)
fix: tail command typo (#5456)
Improve parent lookup logging (#5451)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

System	Architecture	Binary	PGP Signature
	x86_64	lighthouse-v5.1.3-x86_64-apple-darwin.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.3-x86_64-apple-darwin-portable.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.3-x86_64-unknown-linux-gnu.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.3-x86_64-unknown-linux-gnu-portable.tar.gz	PGP Signature
	aarch64	lighthouse-v5.1.3-aarch64-unknown-linux-gnu.tar.gz	PGP Signature
	aarch64	lighthouse-v5.1.3-aarch64-unknown-linux-gnu-portable.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.3-x86_64-windows.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.3-x86_64-windows-portable.tar.gz	PGP Signature

System	Option	-	Resource
	Docker	v5.1.3	sigp/lighthouse

Assets 18

22 Mar 00:48

github-actions

v5.1.2

5ce1619

Handsome Bigfoot

🔥 Hotfix 🔥

This hotfix release includes a block lookup efficiency fix, a fix to stop propagation of slashings for attesters who have already been slashed, a fix to the --disable-upnp flag, and a couple of metric fixes.

This is a medium priority release.

We recommend you upgrade if you are seeing this log:

Mar 13 10:26:50.229 WARN Peer sent invalid response to parent request., reason: ExtraBlocksReturned, peer_id: ..., service: sync

Summary

We've had a couple reports of users seeing this log:

Mar 13 10:26:50.229 WARN Peer sent invalid response to parent request., reason: ExtraBlocksReturned, peer_id: ..., service: sync

This is a result of Lighthouse making duplicate requests to peers for the same block. This has been fixed in (#5432). If you are seeing this log, we recommend you upgrade.

Other client teams have notified us that Lighthouse propagates slashing messages for validators that have already been slashed. These messages are useless from the network's perspective so should be ignored (thanks @nisdas from Prysm). This has been resolved in (#5385).

The --disable-upnp was inneffective previously, but this has been fixed in (#5449).

Additionally this hotfix includes fixes to some metrics (#5404 and #5427).

Breaking Changes

None.

Update Priority

This table provides priorities for which classes of users should update particular components.

User Class	Beacon Node	Validator Client
Staking Users	Medium	Medium
Non-Staking Users	Medium	---

See Update Priorities for more information about this table.

Lighthouse BNs and VCs from v5.0.0 and v5.1.0/v5.1.1/v5.1.2 are intercompatible. However, we recommend that users update both the VC and BN to v5.1.2 if upgrading.

All Changes

Changes relative to v5.1.1:

Release v5.1.2 (#5453)
disable libp2p upnp (#5449)
Fix gossip verification of duplicate attester slashings (#5385)
fix NAT nat_open metrics report (#5427)
Fix peer count metrics (#5404)
Improvements and Fixes in Documentation, Including Corrected Command Usage (#4998)
Fix Rust beta compiler errors 1.78.0-beta.1 (#5439)
Dedup parent blob requests (#5432)
Add attestation simulator, blobs info and some updates to Lighthouse Book (#5364)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

System	Architecture	Binary	PGP Signature
	x86_64	lighthouse-v5.1.2-x86_64-apple-darwin.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.2-x86_64-apple-darwin-portable.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.2-x86_64-unknown-linux-gnu.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.2-x86_64-unknown-linux-gnu-portable.tar.gz	PGP Signature
	aarch64	lighthouse-v5.1.2-aarch64-unknown-linux-gnu.tar.gz	PGP Signature
	aarch64	lighthouse-v5.1.2-aarch64-unknown-linux-gnu-portable.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.2-x86_64-windows.tar.gz	PGP Signature
	x86_64	lighthouse-v5.1.2-x86_64-windows-portable.tar.gz	PGP Signature

System	Option	-	Resource
	Docker	v5.1.2	sigp/lighthouse

Contributors

nisdas

Assets 18

Releases: sigp/lighthouse

Frankenstein's Monster

Summary

Summary

Forced Pseudo-Finalization

State Cache Tweaks

New HTTP endpoint lighthouse/add_peer to add trusted peer

⚠️ Breaking Changes ⚠️

Update Priority

All Changes

Binaries

Hemorrhage

Summary

Update Priority

All Changes

Binaries

Beta-Seven

Summary

⚠️ Breaking Changes ⚠️

⚠️ Deprecated CLI Flags ⚠️

IPv6 by Default

🦀 Minimum Supported Rust Version 🦀

⚡ Electra ⚡

Light Client Server

Attestation Subscription Fix

New Database Backends

🐛 Known Issues 🐛

Update Priority

All changes

Rick In A Vat In The Garage

Summary

⚠️ Breaking Changes ⚠️

⚠️ Database Migration ⚠️

For Archive Nodes

⚠️ Removed CLI Flags ⚠️

⚠️ Deprecated CLI Flags ⚠️

⚠️ Modified CLI Flags ⚠️

Network Optimisations

Validator Client Fallback Optimisations

New Validator Manager Commands

Fetch Blobs Optimisation

Tiny Rick

Summary

⚠️ Breaking Changes ⚠️

⚠️ Database Migration ⚠️

For Archive Nodes

⚠️ Removed CLI Flags ⚠️

⚠️ Deprecated CLI Flags ⚠️

⚠️ Modified CLI Flags ⚠️

Network Optimisations

Validator Client Fallback Optimisations

New Validator Manager Commands

Fetch Blobs Optimisation

🐛 Known Issues 🐛

Nethermind v1.29.x engine_getBlobsV1 bug

Risotto Groupon

Summary

Breaking Changes

🗑️ Removed CLI flags

👎 Deprecated CLI flags

🦀 Minimum Supported Rust Version 1.78.0 🦀

💾 Database Schema Upgrade

🐋 No more -portable tarballs or -modern images

📝 HTTP API /eth/v1/beacon/deposit_snapshot to use JSON response by default

🐧 Jemalloc enabled by default on non-Windows targets

🔓 Removal of signed validator client responses

💵 Fee recipient required with --always-prepare-payload

Update Priority

All Changes

Kyle

Summary

Breaking Changes

Known Issues

Teku 24.10 BN Incompatibility

Nethermind v1.26.0 and Reth v1.0.0 engine_getClientVersionV1 bug

Source builds without jemalloc will OOM on Linux

Update Priority

All changes

Binaries

Tree People

New HTTP endpoint `lighthouse/add_peer` to add trusted peer

Nethermind v1.29.x `engine_getBlobsV1` bug

🐋 No more `-portable` tarballs or `-modern` images

📝 HTTP API `/eth/v1/beacon/deposit_snapshot` to use JSON response by default

💵 Fee recipient required with `--always-prepare-payload`

Nethermind v1.26.0 and Reth v1.0.0 `engine_getClientVersionV1` bug

Source builds without `jemalloc` will OOM on Linux

`portable` builds for Docker and binary releases (#5614, #5615)

`builder-header-timeout` flag (#5857)

Deprecated `latency-measurement-service` flag

Deprecated `http-spec-fork` and `http-allow-sync-stalled` flags (#5500)

Deprecated `--progressive-balances` flag (#5279)

Modified `self-limiter` flag