Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(store): adding_storage_sharding #4128

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

sysrex
Copy link
Contributor

@sysrex sysrex commented Feb 18, 2025

Resolves #4100

@github-actions github-actions bot added the external Issues created by non node team members label Feb 18, 2025
@MSevey
Copy link
Member

MSevey commented Feb 26, 2025

This should have a test that verifies the expected FS structure on disk is created.

Additionally, is this a breaking change? I would assume yes since we are changing how data is being written/read from disk, so existing data would not be found? To test/verify that, we should write a test that starts with data written in the previous structure, and then tries to read it.

@Wondertan
Copy link
Member

There is an existing test that checks for the expected FS structure, but it needs to be fixed, so the tests are failing.

@Wondertan
Copy link
Member

Also, while working on this during onsite, we realized that sharding for height indicies wont work with hardlinks (as it currently stands) and we should discuss how to proceed here.

Its either we migrate over symlinks or solve sharding on FS level, as discussed in #4100

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external Issues created by non node team members
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request]: Implement block storage partitioning for scalable volume distribution
3 participants