-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ZTS: Corrupt all block copies with corrupt_file_at_level, test L1 corruption #11141
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
59cf6b1
to
c484790
Compare
|
c484790
to
7fa9a0b
Compare
|
7fa9a0b
to
9f55214
Compare
|
9f55214
to
7f0ad28
Compare
|
These tests appears to have failed during the Debian 10 test run. |
Ok back to plan B |
Signed-off-by: Ryan Moeller <[email protected]>
L1 and L2 indirect blocks have more than one copy on disk. Show all the indirect block copies. Signed-off-by: Ryan Moeller <[email protected]>
Signed-off-by: Ryan Moeller <[email protected]>
7f0ad28
to
5766308
Compare
|
Ah, Debian uses |
behlendorf
approved these changes
Nov 5, 2020
behlendorf
pushed a commit
that referenced
this pull request
Nov 6, 2020
The second part of list_file_blocks transforms the object description output by zdb -ddddd $ds $objnum into a stream of lines of the form "level path offset length" for the indirect blocks in the given file. The current code only works for the first copy of L0 blocks. L1 and L2 indirect blocks have more than one copy on disk. Add one more -d to the zdb command so we get all block copies and rewrite the transformation to match more than L0 and output all DVAs. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11141
behlendorf
pushed a commit
that referenced
this pull request
Nov 6, 2020
Add a new test case which corrupts all level 1 block in a file. Then verifies that corruption is detected and repaired. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11141
behlendorf
pushed a commit
that referenced
this pull request
Nov 11, 2020
The first part of list_file_blocks transforms the pool configuration output by zdb -C $pool into shell code to set up a shell variable, VDEV_MAP, that maps from vdev id to the underlying vdev path. This variable is a simple indexed array. However, the vdev id in a DVA is only the id of the top level vdev. When the pool is mirrored, the top level vdev is a mirror and its children are the mirrored devices. So, what we need is to map from the top level vdev id to a list of the underlying vdev paths. ist_file_blocks does not need to work for raidz vdevs, so we can disregard that case. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11141
behlendorf
pushed a commit
that referenced
this pull request
Nov 11, 2020
The second part of list_file_blocks transforms the object description output by zdb -ddddd $ds $objnum into a stream of lines of the form "level path offset length" for the indirect blocks in the given file. The current code only works for the first copy of L0 blocks. L1 and L2 indirect blocks have more than one copy on disk. Add one more -d to the zdb command so we get all block copies and rewrite the transformation to match more than L0 and output all DVAs. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11141
behlendorf
pushed a commit
that referenced
this pull request
Nov 11, 2020
Add a new test case which corrupts all level 1 block in a file. Then verifies that corruption is detected and repaired. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11141
jsai20
pushed a commit
to jsai20/zfs
that referenced
this pull request
Mar 30, 2021
The first part of list_file_blocks transforms the pool configuration output by zdb -C $pool into shell code to set up a shell variable, VDEV_MAP, that maps from vdev id to the underlying vdev path. This variable is a simple indexed array. However, the vdev id in a DVA is only the id of the top level vdev. When the pool is mirrored, the top level vdev is a mirror and its children are the mirrored devices. So, what we need is to map from the top level vdev id to a list of the underlying vdev paths. ist_file_blocks does not need to work for raidz vdevs, so we can disregard that case. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes openzfs#11141
jsai20
pushed a commit
to jsai20/zfs
that referenced
this pull request
Mar 30, 2021
The second part of list_file_blocks transforms the object description output by zdb -ddddd $ds $objnum into a stream of lines of the form "level path offset length" for the indirect blocks in the given file. The current code only works for the first copy of L0 blocks. L1 and L2 indirect blocks have more than one copy on disk. Add one more -d to the zdb command so we get all block copies and rewrite the transformation to match more than L0 and output all DVAs. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes openzfs#11141
jsai20
pushed a commit
to jsai20/zfs
that referenced
this pull request
Mar 30, 2021
Add a new test case which corrupts all level 1 block in a file. Then verifies that corruption is detected and repaired. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes openzfs#11141
sempervictus
pushed a commit
to sempervictus/zfs
that referenced
this pull request
May 31, 2021
The first part of list_file_blocks transforms the pool configuration output by zdb -C $pool into shell code to set up a shell variable, VDEV_MAP, that maps from vdev id to the underlying vdev path. This variable is a simple indexed array. However, the vdev id in a DVA is only the id of the top level vdev. When the pool is mirrored, the top level vdev is a mirror and its children are the mirrored devices. So, what we need is to map from the top level vdev id to a list of the underlying vdev paths. ist_file_blocks does not need to work for raidz vdevs, so we can disregard that case. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes openzfs#11141
sempervictus
pushed a commit
to sempervictus/zfs
that referenced
this pull request
May 31, 2021
The second part of list_file_blocks transforms the object description output by zdb -ddddd $ds $objnum into a stream of lines of the form "level path offset length" for the indirect blocks in the given file. The current code only works for the first copy of L0 blocks. L1 and L2 indirect blocks have more than one copy on disk. Add one more -d to the zdb command so we get all block copies and rewrite the transformation to match more than L0 and output all DVAs. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes openzfs#11141
sempervictus
pushed a commit
to sempervictus/zfs
that referenced
this pull request
May 31, 2021
Add a new test case which corrupts all level 1 block in a file. Then verifies that corruption is detected and repaired. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes openzfs#11141
13 tasks
behlendorf
pushed a commit
that referenced
this pull request
Dec 29, 2024
FreeBSD recently removed non-standard hex numbers support from awk. Neither it supports -n argument, enabling it in gawk. Instead of depending on those rewrite list_file_blocks() fuction to handle the hex math in shell instead of awk. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes #11141
behlendorf
pushed a commit
that referenced
this pull request
Dec 29, 2024
procfs might be not mounted on FreeBSD. Plus checking for specific PID might be not exactly reliable. Check for empty list of jobs instead. Premature loop exit can result in failed test and failed cleanup, failing also some following tests. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes #11141
behlendorf
pushed a commit
that referenced
this pull request
Dec 29, 2024
This test takes 3 minutes on RELEASE FreeBSD bots, but on CURRENT, probably due to debugging it has in kernel, it does not complete within 10 minutes, ending up killed. As I see all the redacting here happens within the first ~128MB of the file, so I hope it won't matter if there is 1GB of data instead of 2GB. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes #11141
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this pull request
Dec 29, 2024
FreeBSD recently removed non-standard hex numbers support from awk. Neither it supports -n argument, enabling it in gawk. Instead of depending on those rewrite list_file_blocks() fuction to handle the hex math in shell instead of awk. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this pull request
Dec 29, 2024
procfs might be not mounted on FreeBSD. Plus checking for specific PID might be not exactly reliable. Check for empty list of jobs instead. Premature loop exit can result in failed test and failed cleanup, failing also some following tests. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
behlendorf
pushed a commit
to behlendorf/zfs
that referenced
this pull request
Dec 29, 2024
This test takes 3 minutes on RELEASE FreeBSD bots, but on CURRENT, probably due to debugging it has in kernel, it does not complete within 10 minutes, ending up killed. As I see all the redacting here happens within the first ~128MB of the file, so I hope it won't matter if there is 1GB of data instead of 2GB. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
lundman
pushed a commit
to openzfsonwindows/openzfs
that referenced
this pull request
Jan 26, 2025
FreeBSD recently removed non-standard hex numbers support from awk. Neither it supports -n argument, enabling it in gawk. Instead of depending on those rewrite list_file_blocks() fuction to handle the hex math in shell instead of awk. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
lundman
pushed a commit
to openzfsonwindows/openzfs
that referenced
this pull request
Jan 26, 2025
procfs might be not mounted on FreeBSD. Plus checking for specific PID might be not exactly reliable. Check for empty list of jobs instead. Premature loop exit can result in failed test and failed cleanup, failing also some following tests. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
lundman
pushed a commit
to openzfsonwindows/openzfs
that referenced
this pull request
Jan 26, 2025
This test takes 3 minutes on RELEASE FreeBSD bots, but on CURRENT, probably due to debugging it has in kernel, it does not complete within 10 minutes, ending up killed. As I see all the redacting here happens within the first ~128MB of the file, so I hope it won't matter if there is 1GB of data instead of 2GB. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Component: Test Suite
Indicates an issue with the test framework or a test case
Status: Accepted
Ready to integrate (reviewed, tested)
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
In ZTS,
list_file_blocks
is supposed to output the level, vdev path, offset, and length of blocks in a given file.It fails in the following ways:
This prevents
corrupt_blocks_at_level
from corrupting all copies of the blocks, which is desired for reproducing a particular issue we encountered in the async DMU project.This prevents
corrupt_blocks_at_level
from working at any level other than 0.Description
Only the path of the first child of a mirrored vdev is output
The first part of
list_file_blocks
transforms the pool configuration output byzdb -C $pool
into shell code to set up a shell variable,VDEV_MAP
, that maps from vdev id to the underlying vdev path. This variable is a simple indexed array. However, the vdev id in a DVA is only the id of the top level vdev.When the pool is mirrored, the top level vdev is a mirror and its children are the mirrored devices. So, what we need is to map from the top level vdev id to a list of the underlying vdev paths.
list_file_blocks
does not need to work for raidz vdevs, so we can disregard that case.Only L0 blocks and only the first DVA per block are output
The second part of
list_file_blocks
transforms the object description output byzdb -ddddd $ds $objnum
into a stream of lines of the form "level path offset length" for the indirect blocks in the given file. The current code only works for the first copy of L0 blocks.Add one more
-d
to the zdb command so we get all block copies and rewrite the transformation to match more than L0 and output all DVAs.How Has This Been Tested?
Added a test case.
Types of changes
Checklist:
Signed-off-by
.