September 2017 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] How can we improve log_view about objects created and destroyed?
by Barry Smith 07 Sep '17

07 Sep '17

Fixed in barry/fix-log_view (except for the one missing viewer object). In next for testing. Can you run your bad code with this branch and see if the problem is truly gone. Thanks Barry > On Sep 1, 2017, at 2:42 AM, Stefano Zampini <stefano.zampini(a)gmail.com> wrote: > > As it is now, if you create any object and use PetscObjectRegisterDestroy, the output from log_view may be misleading, as it will show something like this > > > Memory usage is given in bytes: > > Object Type Creations Destructions Memory Descendants' Mem. > Reports information only for process 0. > > --- Event Stage 0: Main Stage > > Index Set 237 237 310852 0. > IS L to G Mapping 53 35 75260 0. > Application Order 2 2 2352 0. > Vector 617 582 1312048 0. > Vector Scatter 41 23 15088 0. > ..... > > > In this example, the objects registered for destroy where also composed with l2g maps and scatters. > So the user may think of having memory leaks somewhere. But running with -malloc_dump (or under valgrind --leak-check=full) the output is clean. > This is because PetscObjectRegisterDestroyAll is called after PetscLogView in PetscFinalize(). > > What if the reason to call PetscObjectRegisterDestroyAll after PetscLogView? Just for having PETSC_VIEWER_STDOUT_WORLD available? > > > -- > Stefano

1 0

Re: [petsc-dev] TSComputeRHSJacobian
by Barry Smith 07 Sep '17

07 Sep '17

Example that demonstrates the current code is problematic in some cases? > On Aug 28, 2017, at 11:59 AM, Stefano Zampini <stefano.zampini(a)gmail.com> wrote: > > > I was wondering if the following code in TSComputeRHSJacobian is correct when the user pass the reuse flag via TSRHSJacobianReuse > > if (ts->rhsjacobian.time == t && (ts->problem_type == TS_LINEAR || (ts->rhsjacobian.Xid == Uid && ts->rhsjacobian.Xstate == Ustate)) && (rhsfunction != TSComputeRHSFunctionLinear)) { > PetscFunctionReturn(0); > } > > Should the above code be replaced by the following? > > if (ts->rhsjacobian.time == t && (ts->problem_type == TS_LINEAR || (ts->rhsjacobian.Xid == Uid && ts->rhsjacobian.Xstate == Ustate)) && (rhsfunction != TSComputeRHSFunctionLinear)) { > if (ts->rhsjacobian.reuse) { > ierr = MatShift(A,-ts->rhsjacobian.shift);CHKERRQ(ierr); > ierr = MatScale(A,1./ts->rhsjacobian.scale);CHKERRQ(ierr); > if (B && A != B) { > ierr = MatShift(B,-ts->rhsjacobian.shift);CHKERRQ(ierr); > ierr = MatScale(B,1./ts->rhsjacobian.scale);CHKERRQ(ierr); > } > ts->rhsjacobian.shift = 0; > ts->rhsjacobian.scale = 1.; > } > PetscFunctionReturn(0); > } > > > -- > Stefano

1 0

Re: [petsc-dev] How can we improve log_view about objects created and destroyed?
by Barry Smith 07 Sep '17

07 Sep '17

Example that produces this problem? > On Sep 1, 2017, at 2:42 AM, Stefano Zampini <stefano.zampini(a)gmail.com> wrote: > > As it is now, if you create any object and use PetscObjectRegisterDestroy, the output from log_view may be misleading, as it will show something like this > > > Memory usage is given in bytes: > > Object Type Creations Destructions Memory Descendants' Mem. > Reports information only for process 0. > > --- Event Stage 0: Main Stage > > Index Set 237 237 310852 0. > IS L to G Mapping 53 35 75260 0. > Application Order 2 2 2352 0. > Vector 617 582 1312048 0. > Vector Scatter 41 23 15088 0. > ..... > > > In this example, the objects registered for destroy where also composed with l2g maps and scatters. > So the user may think of having memory leaks somewhere. But running with -malloc_dump (or under valgrind --leak-check=full) the output is clean. > This is because PetscObjectRegisterDestroyAll is called after PetscLogView in PetscFinalize(). > > What if the reason to call PetscObjectRegisterDestroyAll after PetscLogView? Just for having PETSC_VIEWER_STDOUT_WORLD available? > > > -- > Stefano

1 0

Re: [petsc-dev] c++, float128 and complex patch
by Michael Gegg 07 Sep '17

07 Sep '17

Am 07.09.2017 um 17:26 schrieb Satish Balay: > On Thu, 7 Sep 2017, Michael Gegg wrote: > >> Thank you very much, that does it. One more question: Could you include this >> patch in the official petsc release in the future? > Yes - once tested in next - it would go into master - and next release > (3.8) of petsc. ok thank you, I think that should be fine. Michael > I can also it to maint - which will go into next 3.7.7 patch > release.. [will have to backport - due to code changes between > maint/master] > > Satish > >> I would really appreciate >> that: I wrote a c++ library for quantum optical master equations based on >> petsc (https://github.com/modmido/psiquasp) which is available on github and >> sometimes extended precision is necessary (complex is mandatory). I now >> included a patch there, so that people can use this, but this is only a >> momentary fix. >> >> Thank you. Michael >> >> >> >> Am 06.09.2017 um 20:03 schrieb Satish Balay: >>> On Wed, 6 Sep 2017, Satish Balay wrote: >>> >>>>> So, could you give me a hint on how I should configure petsc so that >>>>> this >>>>> works? Is there a way to enforce the c99 complex numbers even when >>>>> calling >>>>> petsc from c++? >>>> Perhaps this is the change you need? >>> Looks like I need one more change.. >>> >>> Satish >>> >>> --------- >>> balay@asterix /home/balay/petsc (master *=) >>> $ git diff >>> diff --git a/include/petscmath.h b/include/petscmath.h >>> index a63f1f4977..a201c3093e 100644 >>> --- a/include/petscmath.h >>> +++ b/include/petscmath.h >>> @@ -214,7 +214,7 @@ PETSC_EXTERN MPI_Datatype MPIU___COMPLEX128; >>> #endif /* PETSC_USE_REAL_ */ >>> #endif /* ! PETSC_SKIP_COMPLEX */ >>> -#elif !defined(__cplusplus) && defined(PETSC_HAVE_C99_COMPLEX) && >>> !defined(PETSC_USE_REAL___FP16) >>> +#elif defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) >>> #if !defined(PETSC_SKIP_COMPLEX) >>> #define PETSC_HAVE_COMPLEX 1 >>> #include <complex.h> >>> @@ -414,7 +414,7 @@ PETSC_EXTERN PetscComplex PETSC_i; >>> */ >>> PETSC_STATIC_INLINE PetscComplex PetscCMPLX(PetscReal x, PetscReal y) >>> { >>> -#if defined(__cplusplus) >>> +#if defined(__cplusplus) && !defined(PETSC_USE_REAL___FLOAT128) >>> return PetscComplex(x,y); >>> #elif defined(_Imaginary_I) >>> return x + y * _Imaginary_I; >>

1 0

Re: [petsc-dev] c++, float128 and complex patch
by Satish Balay 07 Sep '17

07 Sep '17

On Thu, 7 Sep 2017, Michael Gegg wrote: > Thank you very much, that does it. One more question: Could you include this > patch in the official petsc release in the future? Yes - once tested in next - it would go into master - and next release (3.8) of petsc. I can also it to maint - which will go into next 3.7.7 patch release.. [will have to backport - due to code changes between maint/master] Satish > I would really appreciate > that: I wrote a c++ library for quantum optical master equations based on > petsc (https://github.com/modmido/psiquasp) which is available on github and > sometimes extended precision is necessary (complex is mandatory). I now > included a patch there, so that people can use this, but this is only a > momentary fix. > > Thank you. Michael > > > > Am 06.09.2017 um 20:03 schrieb Satish Balay: > > On Wed, 6 Sep 2017, Satish Balay wrote: > > > > > > So, could you give me a hint on how I should configure petsc so that > > > > this > > > > works? Is there a way to enforce the c99 complex numbers even when > > > > calling > > > > petsc from c++? > > > Perhaps this is the change you need? > > Looks like I need one more change.. > > > > Satish > > > > --------- > > balay@asterix /home/balay/petsc (master *=) > > $ git diff > > diff --git a/include/petscmath.h b/include/petscmath.h > > index a63f1f4977..a201c3093e 100644 > > --- a/include/petscmath.h > > +++ b/include/petscmath.h > > @@ -214,7 +214,7 @@ PETSC_EXTERN MPI_Datatype MPIU___COMPLEX128; > > #endif /* PETSC_USE_REAL_ */ > > #endif /* ! PETSC_SKIP_COMPLEX */ > > -#elif !defined(__cplusplus) && defined(PETSC_HAVE_C99_COMPLEX) && > > !defined(PETSC_USE_REAL___FP16) > > +#elif defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) > > #if !defined(PETSC_SKIP_COMPLEX) > > #define PETSC_HAVE_COMPLEX 1 > > #include <complex.h> > > @@ -414,7 +414,7 @@ PETSC_EXTERN PetscComplex PETSC_i; > > */ > > PETSC_STATIC_INLINE PetscComplex PetscCMPLX(PetscReal x, PetscReal y) > > { > > -#if defined(__cplusplus) > > +#if defined(__cplusplus) && !defined(PETSC_USE_REAL___FLOAT128) > > return PetscComplex(x,y); > > #elif defined(_Imaginary_I) > > return x + y * _Imaginary_I; > >

1 0

Re: [petsc-dev] c++, float128 and complex patch
by Michael Gegg 07 Sep '17

07 Sep '17

Thank you very much, that does it. One more question: Could you include this patch in the official petsc release in the future? I would really appreciate that: I wrote a c++ library for quantum optical master equations based on petsc (https://github.com/modmido/psiquasp) which is available on github and sometimes extended precision is necessary (complex is mandatory). I now included a patch there, so that people can use this, but this is only a momentary fix. Thank you. Michael Am 06.09.2017 um 20:03 schrieb Satish Balay: > On Wed, 6 Sep 2017, Satish Balay wrote: > >>> So, could you give me a hint on how I should configure petsc so that this >>> works? Is there a way to enforce the c99 complex numbers even when calling >>> petsc from c++? >> Perhaps this is the change you need? > Looks like I need one more change.. > > Satish > > --------- > balay@asterix /home/balay/petsc (master *=) > $ git diff > diff --git a/include/petscmath.h b/include/petscmath.h > index a63f1f4977..a201c3093e 100644 > --- a/include/petscmath.h > +++ b/include/petscmath.h > @@ -214,7 +214,7 @@ PETSC_EXTERN MPI_Datatype MPIU___COMPLEX128; > #endif /* PETSC_USE_REAL_ */ > #endif /* ! PETSC_SKIP_COMPLEX */ > > -#elif !defined(__cplusplus) && defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) > +#elif defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) > #if !defined(PETSC_SKIP_COMPLEX) > #define PETSC_HAVE_COMPLEX 1 > #include <complex.h> > @@ -414,7 +414,7 @@ PETSC_EXTERN PetscComplex PETSC_i; > */ > PETSC_STATIC_INLINE PetscComplex PetscCMPLX(PetscReal x, PetscReal y) > { > -#if defined(__cplusplus) > +#if defined(__cplusplus) && !defined(PETSC_USE_REAL___FLOAT128) > return PetscComplex(x,y); > #elif defined(_Imaginary_I) > return x + y * _Imaginary_I;

1 0

Re: [petsc-dev] c++, float128 and complex patch
by Satish Balay 07 Sep '17

07 Sep '17

This fix is now in 'balay/fix-quad-complex-from-cxx' - and merged to next. https://bitbucket.org/petsc/petsc/commits/546cf89789929e13eb602224f891fdd5f… Satish On Wed, 6 Sep 2017, Satish Balay wrote: > On Wed, 6 Sep 2017, Satish Balay wrote: > > > > So, could you give me a hint on how I should configure petsc so that this > > > works? Is there a way to enforce the c99 complex numbers even when calling > > > petsc from c++? > > > > Perhaps this is the change you need? > > Looks like I need one more change.. > > Satish > > --------- > balay@asterix /home/balay/petsc (master *=) > $ git diff > diff --git a/include/petscmath.h b/include/petscmath.h > index a63f1f4977..a201c3093e 100644 > --- a/include/petscmath.h > +++ b/include/petscmath.h > @@ -214,7 +214,7 @@ PETSC_EXTERN MPI_Datatype MPIU___COMPLEX128; > #endif /* PETSC_USE_REAL_ */ > #endif /* ! PETSC_SKIP_COMPLEX */ > > -#elif !defined(__cplusplus) && defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) > +#elif defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) > #if !defined(PETSC_SKIP_COMPLEX) > #define PETSC_HAVE_COMPLEX 1 > #include <complex.h> > @@ -414,7 +414,7 @@ PETSC_EXTERN PetscComplex PETSC_i; > */ > PETSC_STATIC_INLINE PetscComplex PetscCMPLX(PetscReal x, PetscReal y) > { > -#if defined(__cplusplus) > +#if defined(__cplusplus) && !defined(PETSC_USE_REAL___FLOAT128) > return PetscComplex(x,y); > #elif defined(_Imaginary_I) > return x + y * _Imaginary_I; >

1 0

Re: [petsc-dev] How are we deciding the memory alignment that BuildSystem should choose?
by Barry Smith 06 Sep '17

06 Sep '17

Is there a "tuning" part of BuildSystem that could be added. We select things that would benefit from tuning to select a good value. For example with memalign it runs some "benchmark" with the various sizes and selects the smallest size that works best? If this is just way to much intellectual working and coding to utilize, which I think it is, we can use the "predefined based on architecture" approach where configure looks in a table based on the arch/sub-arch etc to select a value (and of course the value is verified to work correctly). This approach may not always produce the optimal value but it is simple and will usually select a pretty good value, it only requires the discipline of adding new archs overtime (which often will not happen) but at least for some arches we may have good values. Barry > On Sep 5, 2017, at 12:55 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > Richard Tran Mills <rtmills(a)anl.gov> writes: > >> Folks, >> >> I am wondering how PETSc's BuildSystem currently chooses the memory >> alignment to use. In the example file I provide in the repo for the Cori >> KNL nodes, I specify '--with-memalign=64' to match the cache line width. If >> I don't do this, then configure chooses a 16 byte alignment. Do people >> think that we should try to make a better effort to choose a more >> appropriate alignment? >> >> I believe that all modern x86 CPUs use a 64 byte cache width, so should we >> be defaulting to that? I don't know how much this matters on a Xeon >> processor, but it can be important on Xeon Phi. > > The preference for 16 was for SSE2, prior to AVX and AVX512. With AVX, > I believe there is no particular reason to prefer more than 32-byte > alignment -- it just ensures that objects cannot share cache lines. > Harmless with large arrays, but not ideal for smaller allocations like > nodes in a linked list. (PETSc doesn't use these very often, but it > would be nice if control structures don't soak up more cache than they > need.)

1 0

Re: [petsc-dev] c++, float128 and complex patch
by Satish Balay 06 Sep '17

06 Sep '17

On Wed, 6 Sep 2017, Satish Balay wrote: > > So, could you give me a hint on how I should configure petsc so that this > > works? Is there a way to enforce the c99 complex numbers even when calling > > petsc from c++? > > Perhaps this is the change you need? Looks like I need one more change.. Satish --------- balay@asterix /home/balay/petsc (master *=) $ git diff diff --git a/include/petscmath.h b/include/petscmath.h index a63f1f4977..a201c3093e 100644 --- a/include/petscmath.h +++ b/include/petscmath.h @@ -214,7 +214,7 @@ PETSC_EXTERN MPI_Datatype MPIU___COMPLEX128; #endif /* PETSC_USE_REAL_ */ #endif /* ! PETSC_SKIP_COMPLEX */ -#elif !defined(__cplusplus) && defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) +#elif defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) #if !defined(PETSC_SKIP_COMPLEX) #define PETSC_HAVE_COMPLEX 1 #include <complex.h> @@ -414,7 +414,7 @@ PETSC_EXTERN PetscComplex PETSC_i; */ PETSC_STATIC_INLINE PetscComplex PetscCMPLX(PetscReal x, PetscReal y) { -#if defined(__cplusplus) +#if defined(__cplusplus) && !defined(PETSC_USE_REAL___FLOAT128) return PetscComplex(x,y); #elif defined(_Imaginary_I) return x + y * _Imaginary_I;

1 0

Re: [petsc-dev] c++, float128 and complex patch
by Satish Balay 06 Sep '17

06 Sep '17

On Wed, 6 Sep 2017, Michael Gegg wrote: > Hello everyone, > > a while ago I wrote about a patch for petsc with complex number c++ and > float128 using c99 complex numbers and you suggested to just use petsc with a > c build and then just call it from c++ (see below). > > I thought that this worked, but I only tried it for my patched version. Now I > tried to do this for the normal v3.7.6 and I get a lot of errors that > PetscCompex is not defined when I call it from c++. The installation works > fine and also the tutorial examples. (which are all c) > > It seems that if I call petsc from c++ it tries to switch to c++ complex > numbers even though I build petsc with c, which are obviously not defined. Yes - because a c++ user would want to use c++/complex from their c++ code [even if petsc is buit with c] > > So, could you give me a hint on how I should configure petsc so that this > works? Is there a way to enforce the c99 complex numbers even when calling > petsc from c++? Perhaps this is the change you need? [untested] diff --git a/include/petscmath.h b/include/petscmath.h index a63f1f4977..311d10f081 100644 --- a/include/petscmath.h +++ b/include/petscmath.h @@ -214,7 +214,7 @@ PETSC_EXTERN MPI_Datatype MPIU___COMPLEX128; #endif /* PETSC_USE_REAL_ */ #endif /* ! PETSC_SKIP_COMPLEX */ -#elif !defined(__cplusplus) && defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) +#elif defined(PETSC_HAVE_C99_COMPLEX) && !defined(PETSC_USE_REAL___FP16) #if !defined(PETSC_SKIP_COMPLEX) #define PETSC_HAVE_COMPLEX 1 #include <complex.h> > I always used > > ./configure ... --with-cc=gcc --with-cxx=g++ --with-fc=gfortran > --download-mpich --with-clanguage=c --with-scalar-type=complex > --with-precision=__float128 --download-f2cblaslapack > > As I said, with my patch this works fine. Thanks in advance. > > Also I realized that parmetis does only support double and not float128 and so > building petsc with float128 and parmetis crashes. I also made a patch for > that, that just declares all parmetis relevant parameters as real_t instead of > PetscReal. Would you be interested in including this patch? (its only 2 lines) > > Thank you. > > Michael > > > Am 03.05.2017 um 18:08 schrieb Michael Gegg: > > > > > > > > Am 03.05.2017 um 16:27 schrieb Matthew Knepley: > > > On Wed, May 3, 2017 at 9:22 AM, Michael Gegg <michael.gegg(a)tu-berlin.de > > > <mailto:[email protected]>> wrote: > > > > > > Hello Petsc developers, > > > > > > thank you for providing such a marvellous numerics package, we > > > had/have a lot of fun using it. For our purposes it was necessary > > > to use Petsc with c++, float128 and complex. Since this is not > > > possible with the standard build we made a patch. It uses c style > > > complex numbers as a workaround. > > > It can be used with the Parmetis package (there is a little fix > > > in the interface there) and we have made a Slepc patch that then > > > also allows to use c++, float128 and complex. > > > It would be great if you could include it in the petsc software. > > > We checked the patch for v3.7.6 and v3.7.3 (latest Slepc release) > > > > > > > > > One question. Why would you compile PETSc with C++ if you wanted to use > > > C99 complex? You can do that with the C build, and call > > > PETSc the same way from C++. > > > > > No specific reason. Thanks for the hint, I tried it and it works. I could > > have saved some effort there, but I also learned a thing or two ;) I just > > didn't know that this was possible. > > Thanks, > > Michael > > > Thanks, > > > > > > Matt > > > > > > With best regards, > > > > > > Michael Gegg > > > > > > > > > > > > > > > -- > > > What most experimenters take for granted before they begin their > > > experiments is infinitely more interesting than any results to which their > > > experiments lead. > > > -- Norbert Wiener > > >

1 0