Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow node start with checkpoint sync #2014

Merged
merged 5 commits into from
Dec 13, 2024

Conversation

bzawisto
Copy link
Contributor

@bzawisto bzawisto commented Dec 13, 2024

Storage elements are inserted into memory and once the threshold is met - they are copied into final (sqlite) db with batch inserts. This speeds loading up by ~25%.

@bzawisto bzawisto linked an issue Dec 13, 2024 that may be closed by this pull request
Copy link
Contributor

github-actions bot commented Dec 13, 2024

🐰 Bencher Report

Branch1920-slow-node-start-with-checkpoint-sync
Testbedself-hosted
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Upper Boundary
nanoseconds (ns)
(Limit %)
process-empty/process-empty📈 view plot
🚷 view threshold
9,509,000.00
(+3.32%)
10,671,306.50
(89.11%)
produce-full/produce-full📈 view plot
🚷 view threshold
1,956,200,000.00
(+1.98%)
2,226,946,911.73
(87.84%)
🐰 View full continuous benchmarking report in Bencher

Copy link
Contributor

@JamesHinshelwood JamesHinshelwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea, thanks! A few comments that might be worth considering before we merge.

bzawisto and others added 3 commits December 13, 2024 22:46
Co-authored-by: James Hinshelwood <[email protected]>
…m:Zilliqa/zq2 into 1920-slow-node-start-with-checkpoint-sync
Copy link
Contributor

@JamesHinshelwood JamesHinshelwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@bzawisto bzawisto enabled auto-merge December 13, 2024 21:56
@bzawisto bzawisto added this pull request to the merge queue Dec 13, 2024
Merged via the queue into main with commit 4c76cf9 Dec 13, 2024
6 checks passed
@bzawisto bzawisto deleted the 1920-slow-node-start-with-checkpoint-sync branch December 13, 2024 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Slow node start with checkpoint sync
2 participants