December 2018 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Hapla Vaclav 07 Dec '18

07 Dec '18

Cool, I will do that. Vaclav > On 7 Dec 2018, at 23:49, Smith, Barry F. <bsmith(a)mcs.anl.gov> wrote: > > > As I said, I'm convinced, I'm fine with removing the old code > > Barry > > >> On Dec 7, 2018, at 2:47 PM, Jed Brown <jed(a)jedbrown.org> wrote: >> >> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >> >>>> On Dec 7, 2018, at 2:25 PM, Jed Brown <jed(a)jedbrown.org> wrote: >>>> >>>> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >>>> >>>>>> On Dec 7, 2018, at 8:56 AM, Jed Brown <jed(a)jedbrown.org> wrote: >>>>>> >>>>>> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >>>>>> >>>>>>> A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). >>>>>> >>>>>> The HDF5 developers were very deliberate about this. You can still use >>>>>> the old API in your own code while linking to the new library. See >>>>>> H5_USE_16_API. >>>>>> >>>>>> https://support.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html >>>>> >>>>> This pages is a firehose. Most people who have HDF5 in their code (that someone else probably wrote) will never get through this page to know what to do. >>> >>>> >>>> You literally just compile that old code using -DH5_USE_16_API while using new HDF5 libraries and headers. >>> >>> Where do you put this -DH5_USE_16_API thing? I literally don't have a clue how my build system works. I would rather just stick to the old version since its worked for the last n years. (This is the average user/developer of package/application A) >> >> I get your point, but instructions are simple and we can't stay in the >> dark ages forever. I think the HDF5 folks did a good job with their API >> deprecations. We'd be upset if people were going through pains to >> support petsc-2.2. >> >> ./configure CPPFLAGS=-DH5_USE_16_API >

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Smith, Barry F. 07 Dec '18

07 Dec '18

As I said, I'm convinced, I'm fine with removing the old code Barry > On Dec 7, 2018, at 2:47 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: > >>> On Dec 7, 2018, at 2:25 PM, Jed Brown <jed(a)jedbrown.org> wrote: >>> >>> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >>> >>>>> On Dec 7, 2018, at 8:56 AM, Jed Brown <jed(a)jedbrown.org> wrote: >>>>> >>>>> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >>>>> >>>>>> A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). >>>>> >>>>> The HDF5 developers were very deliberate about this. You can still use >>>>> the old API in your own code while linking to the new library. See >>>>> H5_USE_16_API. >>>>> >>>>> https://support.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html >>>> >>>> This pages is a firehose. Most people who have HDF5 in their code (that someone else probably wrote) will never get through this page to know what to do. >> >>> >>> You literally just compile that old code using -DH5_USE_16_API while using new HDF5 libraries and headers. >> >> Where do you put this -DH5_USE_16_API thing? I literally don't have a clue how my build system works. I would rather just stick to the old version since its worked for the last n years. (This is the average user/developer of package/application A) > > I get your point, but instructions are simple and we can't stay in the > dark ages forever. I think the HDF5 folks did a good job with their API > deprecations. We'd be upset if people were going through pains to > support petsc-2.2. > > ./configure CPPFLAGS=-DH5_USE_16_API

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Jed Brown 07 Dec '18

07 Dec '18

"Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >> On Dec 7, 2018, at 2:25 PM, Jed Brown <jed(a)jedbrown.org> wrote: >> >> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >> >>>> On Dec 7, 2018, at 8:56 AM, Jed Brown <jed(a)jedbrown.org> wrote: >>>> >>>> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >>>> >>>>> A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). >>>> >>>> The HDF5 developers were very deliberate about this. You can still use >>>> the old API in your own code while linking to the new library. See >>>> H5_USE_16_API. >>>> >>>> https://support.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html >>> >>> This pages is a firehose. Most people who have HDF5 in their code (that someone else probably wrote) will never get through this page to know what to do. > >> >> You literally just compile that old code using -DH5_USE_16_API while using new HDF5 libraries and headers. > > Where do you put this -DH5_USE_16_API thing? I literally don't have a clue how my build system works. I would rather just stick to the old version since its worked for the last n years. (This is the average user/developer of package/application A) I get your point, but instructions are simple and we can't stay in the dark ages forever. I think the HDF5 folks did a good job with their API deprecations. We'd be upset if people were going through pains to support petsc-2.2. ./configure CPPFLAGS=-DH5_USE_16_API

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Smith, Barry F. 07 Dec '18

07 Dec '18

> On Dec 7, 2018, at 2:25 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: > >>> On Dec 7, 2018, at 8:56 AM, Jed Brown <jed(a)jedbrown.org> wrote: >>> >>> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >>> >>>> A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). >>> >>> The HDF5 developers were very deliberate about this. You can still use >>> the old API in your own code while linking to the new library. See >>> H5_USE_16_API. >>> >>> https://support.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html >> >> This pages is a firehose. Most people who have HDF5 in their code (that someone else probably wrote) will never get through this page to know what to do. > > You literally just compile that old code using -DH5_USE_16_API while using new HDF5 libraries and headers. Where do you put this -DH5_USE_16_API thing? I literally don't have a clue how my build system works. I would rather just stick to the old version since its worked for the last n years. (This is the average user/developer of package/application A) > >> I'm fine with removing the older versions if everyone else is but I definitely remember this exact issue occurring in the past; perhaps now HDF5 has resolved it somewhere in their firehose. >> >> Barry

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Jed Brown 07 Dec '18

07 Dec '18

"Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >> On Dec 7, 2018, at 8:56 AM, Jed Brown <jed(a)jedbrown.org> wrote: >> >> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >> >>> A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). >> >> The HDF5 developers were very deliberate about this. You can still use >> the old API in your own code while linking to the new library. See >> H5_USE_16_API. >> >> https://support.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html > > This pages is a firehose. Most people who have HDF5 in their code (that someone else probably wrote) will never get through this page to know what to do. You literally just compile that old code using -DH5_USE_16_API while using new HDF5 libraries and headers. > I'm fine with removing the older versions if everyone else is but I definitely remember this exact issue occurring in the past; perhaps now HDF5 has resolved it somewhere in their firehose. > > Barry

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Smith, Barry F. 07 Dec '18

07 Dec '18

> On Dec 7, 2018, at 8:56 AM, Jed Brown <jed(a)jedbrown.org> wrote: > > "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: > >> A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). > > The HDF5 developers were very deliberate about this. You can still use > the old API in your own code while linking to the new library. See > H5_USE_16_API. > > https://support.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html This pages is a firehose. Most people who have HDF5 in their code (that someone else probably wrote) will never get through this page to know what to do. I'm fine with removing the older versions if everyone else is but I definitely remember this exact issue occurring in the past; perhaps now HDF5 has resolved it somewhere in their firehose. Barry

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Scott Kruger 07 Dec '18

07 Dec '18

FWIW, spack only supports 1.8.10 and above which means that a significant fraction of the scientific software stack has moved on. Scott On 12/7/18 7:56 AM, Jed Brown via petsc-dev wrote: > "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: > >> A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). > > The HDF5 developers were very deliberate about this. You can still use > the old API in your own code while linking to the new library. See > H5_USE_16_API. > > https://support.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html > -- Tech-X Corporation kruger(a)txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841 Boulder, CO 80303 Fax: (303) 448-7756

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Jed Brown 07 Dec '18

07 Dec '18

"Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: > A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). The HDF5 developers were very deliberate about this. You can still use the old API in your own code while linking to the new library. See H5_USE_16_API. https://support.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Matthew Knepley 07 Dec '18

07 Dec '18

Me too. I agree with Vaclav that we are already using features of 1.8. Its hard to make truly useful things without those features. Also, I don't think anyone using their own old HDF5 also uses our HDF5, so we would not have to break them. Thanks, Matt On Fri, Dec 7, 2018 at 9:48 AM Jed Brown via petsc-dev < petsc-dev(a)mcs.anl.gov> wrote: > I'm in favor of requiring >=1.8.0. > > Hapla Vaclav via petsc-dev <petsc-dev(a)mcs.anl.gov> writes: > > > I wanted to add support for HDF5 attributes of groups (currently only > datasets). > > > > But I already feel like held back by supporting HDF5 older than 1.8.0. > Do we still need that? > > > > My arguments to get rid of that support: > > 1) It seems we already use some 1.8.0+ functions anyway (might be my > fault, though). > > 2) We could get rid of all those #if > > 3) There were quite some useful API functions introduced in that release > which would make the code simpler. For instance, H5O* functions allow > handling groups and datasets the same way. > > > > Ad 2,3 consider this > > > > /* OLD */ > > #if (H5_VERS_MAJOR * 10000 + H5_VERS_MINOR * 100 + H5_VERS_RELEASE >= > 10800) > > if (dataset) { > > PetscStackCallHDF5Return(object, H5Dopen2, (h5, parent, > H5P_DEFAULT)); > > } else if (type == H5O_TYPE_GROUP) { > > PetscStackCallHDF5Return(object, H5Gopen2, (h5, parent, > H5P_DEFAULT)); > > } > > #else > > if (dataset) { > > PetscStackCallHDF5Return(object, H5Dopen, (h5, parent)); > > } else if (type == H5O_TYPE_GROUP) { > > PetscStackCallHDF5Return(object, H5Gopen, (h5, parent)); > > } > > #endif > > ... > > if (dataset) { > > PetscStackCallHDF5Return(err, H5Dclose, (object)); > > } else if (type == H5O_TYPE_GROUP) { > > PetscStackCallHDF5Return(err, H5Gclose, (object)); > > } > > > > > > /* NEW */ > > PetscStackCallHDF5Return(object, H5Oopen, (h5, parent, H5P_DEFAULT)); > > ... > > PetscStackCallHDF5Return(err, H5Oclose, (object)); > > > > > > Additionally, function H5Oget_info() which can say whether the object is > group or dataset was also introduced only in 1.8.0. So the dataset flag > above would have to be set by a user it seems... > > > > So I personally think it's already about time to leave the old HDF5 API > behind... Objections? If no, I would make a PR which would clean the code > from the #ifs and forbid the older versions already in configure. > > > > Vaclav > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] need for HDF5 < 1.8.0 support?
by Smith, Barry F. 07 Dec '18

07 Dec '18

A potential drawback is some users also use HDF5 directly in their code and may be using an older version (people are very slow to change). Barry > On Dec 7, 2018, at 8:48 AM, Jed Brown via petsc-dev <petsc-dev(a)mcs.anl.gov> wrote: > > I'm in favor of requiring >=1.8.0. > > Hapla Vaclav via petsc-dev <petsc-dev(a)mcs.anl.gov> writes: > >> I wanted to add support for HDF5 attributes of groups (currently only datasets). >> >> But I already feel like held back by supporting HDF5 older than 1.8.0. Do we still need that? >> >> My arguments to get rid of that support: >> 1) It seems we already use some 1.8.0+ functions anyway (might be my fault, though). >> 2) We could get rid of all those #if >> 3) There were quite some useful API functions introduced in that release which would make the code simpler. For instance, H5O* functions allow handling groups and datasets the same way. >> >> Ad 2,3 consider this >> >> /* OLD */ >> #if (H5_VERS_MAJOR * 10000 + H5_VERS_MINOR * 100 + H5_VERS_RELEASE >= 10800) >> if (dataset) { >> PetscStackCallHDF5Return(object, H5Dopen2, (h5, parent, H5P_DEFAULT)); >> } else if (type == H5O_TYPE_GROUP) { >> PetscStackCallHDF5Return(object, H5Gopen2, (h5, parent, H5P_DEFAULT)); >> } >> #else >> if (dataset) { >> PetscStackCallHDF5Return(object, H5Dopen, (h5, parent)); >> } else if (type == H5O_TYPE_GROUP) { >> PetscStackCallHDF5Return(object, H5Gopen, (h5, parent)); >> } >> #endif >> ... >> if (dataset) { >> PetscStackCallHDF5Return(err, H5Dclose, (object)); >> } else if (type == H5O_TYPE_GROUP) { >> PetscStackCallHDF5Return(err, H5Gclose, (object)); >> } >> >> >> /* NEW */ >> PetscStackCallHDF5Return(object, H5Oopen, (h5, parent, H5P_DEFAULT)); >> ... >> PetscStackCallHDF5Return(err, H5Oclose, (object)); >> >> >> Additionally, function H5Oget_info() which can say whether the object is group or dataset was also introduced only in 1.8.0. So the dataset flag above would have to be set by a user it seems... >> >> So I personally think it's already about time to leave the old HDF5 API behind... Objections? If no, I would make a PR which would clean the code from the #ifs and forbid the older versions already in configure. >> >> Vaclav

1 0