May 2017 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] PETSc User Meeting 2017, June 14-16 in Boulder, Colorado
by Jed Brown 31 May '17

31 May '17

Correction: it is still possible to book lodging today (closes at midnight Mountain Time). See you in two short weeks. Thanks! Jed Brown <jed(a)jedbrown.org> writes: > The program is up on the website: > > https://www.mcs.anl.gov/petsc/meetings/2017/ > > If you haven't registered yet, we can still accommodate you, but please > register soon. If you haven't booked lodging, please do that soon -- > the on-campus lodging option will close on *Tuesday, May 30*. > > https://confreg.colorado.edu/CSM2017 > > We are looking forward to seeing you in Boulder! > > Jed Brown <jed(a)jedbrown.org> writes: > >> We'd like to invite you to join us at the 2017 PETSc User Meeting held >> at the University of Colorado Boulder on June 14-16, 2017. >> >> http://www.mcs.anl.gov/petsc/meetings/2017/ >> >> The first day consists of tutorials on various aspects and features of >> PETSc. The second and third days will be devoted to exchange, >> discussions, and a refinement of strategies for the future with our >> users. We encourage you to present work illustrating your own use of >> PETSc, for example in applications or in libraries built on top of >> PETSc. >> >> Registration for the PETSc User Meeting 2017 is free for students and >> $75 for non-students. We can host a maximum of 150 participants, so >> register soon (and by May 15). >> >> http://www.eventzilla.net/web/e/petsc-user-meeting-2017-2138890185 >> >> We are also offering low-cost lodging on campus. A lodging registration >> site will be available soon and announced here and on the website. >> >> Thanks to the generosity of Intel, we will be able to offer a limited >> number of student travel grants. We are also soliciting additional >> sponsors -- please contact us if you are interested. >> >> >> We are looking forward to seeing you in Boulder! >> >> Please contact us at petsc2017(a)mcs.anl.gov if you have any questions or >> comments.

1 0

Re: [petsc-dev] Branches with additional features
by Pierre Jolivet 31 May '17

31 May '17

On Tue, 30 May 2017 13:32:18 -0500, Barry Smith wrote: > Yeah, you have a valid point we really need a MatCreateDense() and a > MatCreateDenseWithArray() like we have for vectors. > > If we simply had a null array to indicate don't allocate space > then uses would ALWAYS need to allocate the space themselves and pass > it in which is annoying because most of the time users want PETSc to > allocate the array. > > In your case can't you create the matrix with the "first" array > from the other code and then after that use MatDensePlaceArray()? > Then > MatCreateDense() would not allocate any unneeded space. I indeed can. > Barry > > >> On May 30, 2017, at 3:46 AM, Pierre Jolivet >> <pierre.jolivet(a)enseeiht.fr> wrote: >> >> Thank you very much. One last thing: with VecCreateMPIWithArray, if >> I pass a NULL pointer for the array of values, then nothing is done >> and I need to use VecPlaceArray/VecResetArray. >> With MatCreateDense, if I pass a NULL pointer for the array of >> values, then PETSc allocates the memory. Why is there such a >> difference? Is there a way to create a dense matrix with no storage >> attached to it (that will of course be then set via >> MatDensePlaceArray/MatDenseResetArray)? >> >> Thanks, >> Pierre >> >> On Mon, 29 May 2017 19:45:27 -0500, Barry Smith wrote: >>> I have added MatDensePlaceArray() and MatDenseResetArray() in >>> >>> https://bitbucket.org/petsc/petsc/branch/barry/feature-matdenseplacearray >>> >>> Barry >>> >>>> On May 29, 2017, at 3:46 PM, Pierre Jolivet >>>> <pierre.jolivet(a)enseeiht.fr> wrote: >>>> >>>> >>>>> On 29 May 2017, at 22:20, Jed Brown <jed(a)jedbrown.org> wrote: >>>>> >>>>>> That makes perfect sense (assuming I'm not wrong about the >>>>>> MatAssemblyBegin/MatAssemblyEnd). MatSetUpMultiply_MPIDense >>>>>> would still >>>>>> be called at each iteration but I doubt this is too costly. >>>>> >>>>> It creates a VecScatter so it isn't nothing (in terms of parallel >>>>> semantics), but I'd like to see profiling data before chasing >>>>> this >>>>> around. >>>> >>>> Sure, I'll use that and will look out for any occurrence of >>>> "MatDensePlaceArray" (or stuff like that) in the logs, in case it is >>>> implemented. >>>> Thanks! >>

1 0

Re: [petsc-dev] Branches with additional features
by Barry Smith 30 May '17

30 May '17

Yeah, you have a valid point we really need a MatCreateDense() and a MatCreateDenseWithArray() like we have for vectors. If we simply had a null array to indicate don't allocate space then uses would ALWAYS need to allocate the space themselves and pass it in which is annoying because most of the time users want PETSc to allocate the array. In your case can't you create the matrix with the "first" array from the other code and then after that use MatDensePlaceArray()? Then MatCreateDense() would not allocate any unneeded space. Barry > On May 30, 2017, at 3:46 AM, Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> wrote: > > Thank you very much. One last thing: with VecCreateMPIWithArray, if I pass a NULL pointer for the array of values, then nothing is done and I need to use VecPlaceArray/VecResetArray. > With MatCreateDense, if I pass a NULL pointer for the array of values, then PETSc allocates the memory. Why is there such a difference? Is there a way to create a dense matrix with no storage attached to it (that will of course be then set via MatDensePlaceArray/MatDenseResetArray)? > > Thanks, > Pierre > > On Mon, 29 May 2017 19:45:27 -0500, Barry Smith wrote: >> I have added MatDensePlaceArray() and MatDenseResetArray() in >> https://bitbucket.org/petsc/petsc/branch/barry/feature-matdenseplacearray >> >> Barry >> >>> On May 29, 2017, at 3:46 PM, Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> wrote: >>> >>> >>>> On 29 May 2017, at 22:20, Jed Brown <jed(a)jedbrown.org> wrote: >>>> >>>>> That makes perfect sense (assuming I'm not wrong about the >>>>> MatAssemblyBegin/MatAssemblyEnd). MatSetUpMultiply_MPIDense would still >>>>> be called at each iteration but I doubt this is too costly. >>>> >>>> It creates a VecScatter so it isn't nothing (in terms of parallel >>>> semantics), but I'd like to see profiling data before chasing this >>>> around. >>> >>> Sure, I'll use that and will look out for any occurrence of "MatDensePlaceArray" (or stuff like that) in the logs, in case it is implemented. >>> Thanks! >

1 0

Re: [petsc-dev] Branches with additional features
by Pierre Jolivet 30 May '17

30 May '17

Thank you very much. One last thing: with VecCreateMPIWithArray, if I pass a NULL pointer for the array of values, then nothing is done and I need to use VecPlaceArray/VecResetArray. With MatCreateDense, if I pass a NULL pointer for the array of values, then PETSc allocates the memory. Why is there such a difference? Is there a way to create a dense matrix with no storage attached to it (that will of course be then set via MatDensePlaceArray/MatDenseResetArray)? Thanks, Pierre On Mon, 29 May 2017 19:45:27 -0500, Barry Smith wrote: > I have added MatDensePlaceArray() and MatDenseResetArray() in > > https://bitbucket.org/petsc/petsc/branch/barry/feature-matdenseplacearray > > Barry > >> On May 29, 2017, at 3:46 PM, Pierre Jolivet >> <pierre.jolivet(a)enseeiht.fr> wrote: >> >> >>> On 29 May 2017, at 22:20, Jed Brown <jed(a)jedbrown.org> wrote: >>> >>>> That makes perfect sense (assuming I'm not wrong about the >>>> MatAssemblyBegin/MatAssemblyEnd). MatSetUpMultiply_MPIDense would >>>> still >>>> be called at each iteration but I doubt this is too costly. >>> >>> It creates a VecScatter so it isn't nothing (in terms of parallel >>> semantics), but I'd like to see profiling data before chasing this >>> around. >> >> Sure, I'll use that and will look out for any occurrence of >> "MatDensePlaceArray" (or stuff like that) in the logs, in case it is >> implemented. >> Thanks!

1 0

Python-3 configure
by Jed Brown 29 May '17

29 May '17

I had people in my tutorial today with Python-3 installed but no Python-2. We can make a single codebase work with both 2 and 3, but it'll take some work and we need to stop adding sloppy 2-only code.

1 0

Re: [petsc-dev] Branches with additional features
by Barry Smith 29 May '17

29 May '17

I have added MatDensePlaceArray() and MatDenseResetArray() in https://bitbucket.org/petsc/petsc/branch/barry/feature-matdenseplacearray Barry > On May 29, 2017, at 3:46 PM, Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> wrote: > > >> On 29 May 2017, at 22:20, Jed Brown <jed(a)jedbrown.org> wrote: >> >>> That makes perfect sense (assuming I'm not wrong about the >>> MatAssemblyBegin/MatAssemblyEnd). MatSetUpMultiply_MPIDense would still >>> be called at each iteration but I doubt this is too costly. >> >> It creates a VecScatter so it isn't nothing (in terms of parallel >> semantics), but I'd like to see profiling data before chasing this >> around. > > Sure, I'll use that and will look out for any occurrence of "MatDensePlaceArray" (or stuff like that) in the logs, in case it is implemented. > Thanks!

1 0

Re: [petsc-dev] Branches with additional features
by Pierre Jolivet 29 May '17

29 May '17

> On 29 May 2017, at 22:20, Jed Brown <jed(a)jedbrown.org> wrote: > >> That makes perfect sense (assuming I'm not wrong about the >> MatAssemblyBegin/MatAssemblyEnd). MatSetUpMultiply_MPIDense would still >> be called at each iteration but I doubt this is too costly. > > It creates a VecScatter so it isn't nothing (in terms of parallel > semantics), but I'd like to see profiling data before chasing this > around. Sure, I'll use that and will look out for any occurrence of "MatDensePlaceArray" (or stuff like that) in the logs, in case it is implemented. Thanks!

1 0

Re: [petsc-dev] Branches with additional features
by Jed Brown 29 May '17

29 May '17

Pierre Jolivet <Pierre.Jolivet(a)enseeiht.fr> writes: > On Mon, 29 May 2017 04:38:39 -0600, Jed Brown wrote: >> Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> writes: >> >>>> On 29 May 2017, at 06:49, Jed Brown <jed(a)jedbrown.org> wrote: >>>> >>>> You can't get a callback from this "external Krylov solver"? >>> >>> You mean, a callback to let the external solver use the memory >>> allocated by PETSc for the Mats? Unfortunately, I need the underlying >>> memory to be contiguous between all Mats, and I don't think it is >>> possible with the current PETSc operators for matrix creation. >> >> You can allocate the full block, then pass portions of the >> (column-aligned) array in MatCreateDense(). > > So I was doing: > MatCreate(); > MatSetSizes(); > MatSetType(); > MatMPIDenseSetPreallocation(); > MatAssemblyBegin(); > MatAssemblyEnd(); > MatMatMult(); > > and what you are suggesting is: > MatCreateDense(); // data != nullptr as provided by the external solver > so that there is no need for MatAssemblyBegin/MatAssemblyEnd > http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/dense/mpi/mpidense… > ? > MatMatMult(); PetscErrorCode MatCreateDense(MPI_Comm comm,PetscInt m,PetscInt n,PetscInt M,PetscInt N,PetscScalar *data,Mat *A) { PetscErrorCode ierr; PetscMPIInt size; PetscFunctionBegin; ierr = MatCreate(comm,A);CHKERRQ(ierr); ierr = MatSetSizes(*A,m,n,M,N);CHKERRQ(ierr); ierr = MPI_Comm_size(comm,&size);CHKERRQ(ierr); if (size > 1) { ierr = MatSetType(*A,MATMPIDENSE);CHKERRQ(ierr); ierr = MatMPIDenseSetPreallocation(*A,data);CHKERRQ(ierr); if (data) { /* user provided data array, so no need to assemble */ ierr = MatSetUpMultiply_MPIDense(*A);CHKERRQ(ierr); (*A)->assembled = PETSC_TRUE; } } else { ierr = MatSetType(*A,MATSEQDENSE);CHKERRQ(ierr); ierr = MatSeqDenseSetPreallocation(*A,data);CHKERRQ(ierr); } PetscFunctionReturn(0); } > That makes perfect sense (assuming I'm not wrong about the > MatAssemblyBegin/MatAssemblyEnd). MatSetUpMultiply_MPIDense would still > be called at each iteration but I doubt this is too costly. It creates a VecScatter so it isn't nothing (in terms of parallel semantics), but I'd like to see profiling data before chasing this around.

1 0

Re: [petsc-dev] Branches with additional features
by Pierre Jolivet 29 May '17

29 May '17

On Mon, 29 May 2017 04:38:39 -0600, Jed Brown wrote: > Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> writes: > >>> On 29 May 2017, at 06:49, Jed Brown <jed(a)jedbrown.org> wrote: >>> >>> You can't get a callback from this "external Krylov solver"? >> >> You mean, a callback to let the external solver use the memory >> allocated by PETSc for the Mats? Unfortunately, I need the underlying >> memory to be contiguous between all Mats, and I don't think it is >> possible with the current PETSc operators for matrix creation. > > You can allocate the full block, then pass portions of the > (column-aligned) array in MatCreateDense(). So I was doing: MatCreate(); MatSetSizes(); MatSetType(); MatMPIDenseSetPreallocation(); MatAssemblyBegin(); MatAssemblyEnd(); MatMatMult(); and what you are suggesting is: MatCreateDense(); // data != nullptr as provided by the external solver so that there is no need for MatAssemblyBegin/MatAssemblyEnd http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/dense/mpi/mpidense… ? MatMatMult(); That makes perfect sense (assuming I'm not wrong about the MatAssemblyBegin/MatAssemblyEnd). MatSetUpMultiply_MPIDense would still be called at each iteration but I doubt this is too costly. Thanks for the help.

1 0

Re: [petsc-dev] Branches with additional features
by Jed Brown 29 May '17

29 May '17

Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> writes: >> On 29 May 2017, at 06:49, Jed Brown <jed(a)jedbrown.org> wrote: >> >> You can't get a callback from this "external Krylov solver"? > > You mean, a callback to let the external solver use the memory allocated by PETSc for the Mats? Unfortunately, I need the underlying memory to be contiguous between all Mats, and I don't think it is possible with the current PETSc operators for matrix creation. You can allocate the full block, then pass portions of the (column-aligned) array in MatCreateDense().

1 0