Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check szip parameters for validity, plus all other outstanding szip work... #1629

Merged
merged 44 commits into from
Feb 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
e793764
more HDF5 tests for szip functionality
edwardhartnett Feb 4, 2020
e363bd5
testing nc_def_var_szip when szip has not been built into HDF5
edwardhartnett Feb 4, 2020
bf3407c
more tests
edwardhartnett Feb 4, 2020
626f408
more documentation for nc_inq_var_szip
edwardhartnett Feb 4, 2020
a6fbc3e
fix problem with pre-enddef call to nc_inq_var_szip()
edwardhartnett Feb 4, 2020
52d745d
now remember szip setting in filter fields
edwardhartnett Feb 4, 2020
b09446b
test nc_inq_var_szip after call to nc_def_var_szip, but before enddef
edwardhartnett Feb 4, 2020
517ef4f
use nc_def_var_filter in nc_def_var_szip
edwardhartnett Feb 5, 2020
1817790
rely completely on nc_def_var_filter for setting szip
edwardhartnett Feb 5, 2020
3247061
adding szip test to tst_parallel_zlib2.c
edwardhartnett Feb 5, 2020
6c39c1d
more of szip test in tst_parallel_zlib2.c
edwardhartnett Feb 5, 2020
4a2ac1a
now parallel szip test working in tst_parallel_zlib2.c
edwardhartnett Feb 5, 2020
cf74f49
moved tst_parallel_zlib2 to tst_parallel_compress
edwardhartnett Feb 6, 2020
120c9e0
merged ejh_more_szip
edwardhartnett Feb 6, 2020
d5859e9
not return 0 for parameters to nc_inq_var_szip if szip is not turned …
edwardhartnett Feb 6, 2020
ca5c023
renamed var for clarity
edwardhartnett Feb 6, 2020
6d50ba6
turned on some previously failing tests for nc_inq_var_szip()
edwardhartnett Feb 6, 2020
fb2a104
documentation improvements for nc_inq_var_szip()
edwardhartnett Feb 6, 2020
2daeae5
fixed call to AC_SUBST inside an if-endif in configure.ac
edwardhartnett Feb 6, 2020
a306bdf
merged master
edwardhartnett Feb 6, 2020
04624c7
fix error in makefile building tst_parallel_compress.c
edwardhartnett Feb 6, 2020
b3fe7e5
removed unnecessary library check for szip library, cleaned up comments
edwardhartnett Feb 6, 2020
dfc49ac
merged changes from master
edwardhartnett Feb 6, 2020
6b29478
adding test for zlib+szip in HDF5
edwardhartnett Feb 7, 2020
cd3740e
finished test for zlib+szip in HDF5
edwardhartnett Feb 7, 2020
3d169c4
removed redundant test
edwardhartnett Feb 7, 2020
78cd7f7
removed unneeded var
edwardhartnett Feb 7, 2020
dc4e880
disallow szip if zlib already in use
edwardhartnett Feb 7, 2020
6d2d751
disallow zlib if szip already in use
edwardhartnett Feb 7, 2020
cbc2677
updated documentation
edwardhartnett Feb 7, 2020
6ae81a1
starting to add test for szip param values
edwardhartnett Feb 7, 2020
67a5623
adding test of different szip option mask settings
edwardhartnett Feb 7, 2020
81a0799
test development for different szip option mask settings
edwardhartnett Feb 7, 2020
ff72805
checking for some bad pixels_per_block values for szip
edwardhartnett Feb 7, 2020
c4d3937
now check number of elements in chunk against pixels_per_block for sz…
edwardhartnett Feb 7, 2020
b9aa2e7
test more values for szip
edwardhartnett Feb 7, 2020
a507958
test more values for szip
edwardhartnett Feb 7, 2020
558988b
fixed docs, removed unneeded defines in test
edwardhartnett Feb 7, 2020
8057a55
move nc_def_var_szip function so it will appear in the documentation
edwardhartnett Feb 7, 2020
ea98ee5
updated release notes
edwardhartnett Feb 7, 2020
96182a9
trying to fix windows build
edwardhartnett Feb 7, 2020
1489ebd
merged changes from master branch
edwardhartnett Feb 8, 2020
c0d9c62
added more documentation to nc_def_var_filter()
edwardhartnett Feb 10, 2020
164de98
merged changes from master branch
edwardhartnett Feb 11, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ This file contains a high-level description of this package's evolution. Release

## 4.7.4 - TBD

* [Enhancement] Now nc_inq_var_szip retuns 0 for parameter values if szip is not in use for var. See [https://github.com/Unidata/netcdf-c/issues/1618].
* [Enhancement] Now allow parallel I/O with filters, for HDF5-1.10.3 and later. See [https://github.com/Unidata/netcdf-c/issues/1473].
* [Enhancement] Increased default size of cache buffer to 16 MB, from 4 MB. Increased number of slots to 4133. See [https://github.com/Unidata/netcdf-c/issues/1541].
* [Enhancement] Allow zlib compression to be used with parallel I/O writes, if HDF5 version is 1.10.3 or greater. See [https://github.com/Unidata/netcdf-c/issues/1580].
* [Enhancement] Restore use of szip compression when writing data (including writing in parallel if HDF5 version is 1.10.3 or greater). See [https://github.com/Unidata/netcdf-c/issues/1546].
Expand Down
21 changes: 9 additions & 12 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -1047,7 +1047,7 @@ if test "x$enable_hdf5" = xyes; then

# H5Pset_fapl_mpiposix and H5Pget_fapl_mpiposix have been removed since HDF5 1.8.12.
# Use H5Pset_fapl_mpio and H5Pget_fapl_mpio, instead.
AC_CHECK_FUNCS([H5Pget_fapl_mpio H5Pset_deflate H5Z_SZIP H5free_memory H5resize_memory H5allocate_memory H5Pset_libver_bounds H5Pset_all_coll_metadata_ops H5Z_SZIP H5Dread_chunk])
AC_CHECK_FUNCS([H5Pget_fapl_mpio H5Pset_deflate H5Z_SZIP H5free_memory H5resize_memory H5allocate_memory H5Pset_libver_bounds H5Pset_all_coll_metadata_ops H5Dread_chunk])

# Check to see if HDF5 library has collective metadata APIs, (HDF5 >= 1.10.0)
if test "x$ac_cv_func_H5Pset_all_coll_metadata_ops" = xyes; then
Expand All @@ -1062,29 +1062,25 @@ if test "x$enable_hdf5" = xyes; then
AC_MSG_CHECKING([whether parallel io is enabled in hdf5])
AC_MSG_RESULT([$hdf5_parallel])

# Check to see if HDF5 library is 1.10.3 or greater. If so, allows parallel_zip.
# Check to see if HDF5 library is 1.10.3 or greater. If so, allows
# parallel I/O with filters. This allows zlib/szip compression to
# be used with parallel I/O, which is very helpful to HPC users.
if test "x$ac_cv_func_H5Dread_chunk" = xyes; then
AC_DEFINE([HDF5_SUPPORTS_PAR_FILTERS], [1], [if true, HDF5 is at least version 1.10.3 and allows parallel I/O with zip])
hdf5_supports_par_filters=yes
fi
AC_MSG_CHECKING([whether HDF5 is version 1.10.3 or greater])
AC_MSG_CHECKING([whether HDF5 allows parallel filters])
AC_MSG_RESULT([$ac_cv_func_H5Dread_chunk])
AC_SUBST([HAS_PAR_FILTERS], [$hdf5_supports_par_filters])

# Check to see if we need to search for and link against szlib.
if test "x$ac_cv_func_H5Z_SZIP" = xyes; then
AC_SEARCH_LIBS([SZ_BufftoBuffCompress], [szip sz], [],
[AC_MSG_ERROR([libhdf5 installed with szip support, but cannot find or link to the szip library.])])
fi


# Check to see if user asked for parallel build, but HDF5 does not support it.
if test "x$hdf5_parallel" = "xno"; then
if test "x$enable_parallel_tests" = "xyes"; then
AC_MSG_ERROR([Parallel tests requested, but no parallel HDF5 installation detected.])
fi
fi

# The user may have built HDF5 with the SZLIB library.
# Check whether HDF5 was built with the SZLIB library. If so we
# must be able to link to szip library.
AC_MSG_CHECKING([whether szlib was used when building HDF5])
enable_szlib=no
if test "x$ac_cv_func_H5Z_SZIP" = xyes; then
Expand All @@ -1095,6 +1091,7 @@ if test "x$enable_hdf5" = xyes; then

fi
AM_CONDITIONAL(HAS_PAR_FILTERS, [test x$hdf5_supports_par_filters = xyes ])
AC_SUBST([HAS_PAR_FILTERS], [$hdf5_supports_par_filters])

# If the user wants hdf4 built in, check it out.
if test "x$enable_hdf4" = xyes; then
Expand Down
Loading