July 2016 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] memory usage with GPUs
by Barry Smith 29 Jul '16

29 Jul '16

Do you know if Nvidia provides an API to determine how much GPU memory has been currently allocated? If it does then that could easily be hooked up in same style as the PetscMemory...() routines for example PetscGPUMemory...(). If they don't provide an API then obviously there is no way to get that data. Barry > On Jul 27, 2016, at 8:18 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > Note, I wanted to use this for a Petsc code that uses GPUs but not through at Petsc. > > So my case is a little odd and I did not expect support but I just wanted to check. > > On Wednesday, July 27, 2016, Karl Rupp <rupp(a)iue.tuwien.ac.at> wrote: > > We (meanings someone) should add logging of GPU allocations separately. > > Agreed. I'll do it (though not immediately, as there are other things with higher priority to make GPU support in PETSc more useful) > > Best regards, > Karli > > > > On Jul 27, 2016, at 1:41 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > > No, although since we mirror storage you could get an idea if you broke out the GPU vecs into a separate stage. > > Matt > > On Wed, Jul 27, 2016 at 9:05 AM, Mark Adams <mfadams(a)lbl.gov> wrote: > Would/could PETSc memory usage methods (PetscMemoryGetMaximumUsage) pick up GPU memory usage? > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > >

1 0

Re: [petsc-dev] slow loading of mesh in dmplex
by Tobin Isaac 29 Jul '16

29 Jul '16

Hi Stephan, Thanks for this! I switched the internal storage of label strata to an IS to allow greater flexibility, but I admit to rotely throwing ISGetIndices/ISRestoreIndices around sections of code that I didn't take the time to understand fully. I'll get this integrated. Cheers, Toby On Fri, Jul 29, 2016 at 11:53:12AM +0100, Stephan Kramer wrote: > Dear petsc devs > > After some profiling of the loading of a large gmsh mesh in dmplex > (which seemed to take forever), I found that it was spending 99% of > its time in IsGetIndices_Stride in the following calltree > > DMPlexCreateGmshFromFile -> DMPlexGetFullJoin -> > DMLabelGetStratumBounds -> ISGetIndices -> ISGetIndices_Stride > > where it seems to work out the bounds of each stratum by explicitly > computing all indices of a strided index set to then only take its > first and last entry, and this is done for each facet in the mesh a > couple of times. > > With the attached patch I managed to reduce the load time of a mesh > from 4 hours to 22 seconds! I'm not a 100% sure if the patch is > correct in all circumstances: I presume the assumption was that the > index set is sorted? > > Cheers > Stephan > diff --git a/src/dm/label/dmlabel.c b/src/dm/label/dmlabel.c > index f3e784c..fb8d1b9 100644 > --- a/src/dm/label/dmlabel.c > +++ b/src/dm/label/dmlabel.c > @@ -770,15 +770,14 @@ PetscErrorCode DMLabelGetStratumBounds(DMLabel label, PetscInt value, PetscInt * > if (start) {PetscValidPointer(start, 3); *start = 0;} > if (end) {PetscValidPointer(end, 4); *end = 0;} > for (v = 0; v < label->numStrata; ++v) { > - const PetscInt *points; > + PetscInt min, max; > > if (label->stratumValues[v] != value) continue; > ierr = DMLabelMakeValid_Private(label, v);CHKERRQ(ierr); > if (label->stratumSizes[v] <= 0) break; > - ierr = ISGetIndices(label->points[v],&points);CHKERRQ(ierr); > - if (start) *start = points[0]; > - if (end) *end = points[label->stratumSizes[v]-1]+1; > - ierr = ISRestoreIndices(label->points[v],&points);CHKERRQ(ierr); > + ierr = ISGetMinMax(label->points[v], &min, &max);CHKERRQ(ierr); > + if (start) *start = min; > + if (end) *end = max+1; > break; > } > PetscFunctionReturn(0);

1 0

slow loading of mesh in dmplex
by Stephan Kramer 29 Jul '16

29 Jul '16

Dear petsc devs After some profiling of the loading of a large gmsh mesh in dmplex (which seemed to take forever), I found that it was spending 99% of its time in IsGetIndices_Stride in the following calltree DMPlexCreateGmshFromFile -> DMPlexGetFullJoin -> DMLabelGetStratumBounds -> ISGetIndices -> ISGetIndices_Stride where it seems to work out the bounds of each stratum by explicitly computing all indices of a strided index set to then only take its first and last entry, and this is done for each facet in the mesh a couple of times. With the attached patch I managed to reduce the load time of a mesh from 4 hours to 22 seconds! I'm not a 100% sure if the patch is correct in all circumstances: I presume the assumption was that the index set is sorted? Cheers Stephan

1 0

Re: [petsc-dev] hypre and threads
by Mark Adams 28 Jul '16

28 Jul '16

Cool, so just --with-openmp and we are off. I was hoping (and sort of claimed) it would easy, Thanks, On Thu, Jul 28, 2016 at 3:08 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > https://bitbucket.org/petsc/petsc/commits/e872ca4eda51751516e9741c875849477… > > satish > > On Thu, 28 Jul 2016, Mark Adams wrote: > > > I have a project (XGC) that has heard that hypre supports threads and > they > > are interested in using it. They would call PETSc outside of a threaded > > section, of course, but can hypre pick up the threads in some way? > > > > Mark > > > >

1 0

Re: [petsc-dev] hypre and threads
by Mark Adams 28 Jul '16

28 Jul '16

On Thu, Jul 28, 2016 at 3:07 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > Have they lost their fucking collective mind? > We will get some data. Good. > > Matt > > On Thu, Jul 28, 2016 at 12:04 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > >> I have a project (XGC) that has heard that hypre supports threads and >> they are interested in using it. They would call PETSc outside of a >> threaded section, of course, but can hypre pick up the threads in some way? >> >> Mark >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener >

1 0

Re: [petsc-dev] Is the website down?
by Satish Balay 28 Jul '16

28 Jul '16

with the outage - the day/night builds will also be broken - and dashboard not updated [and if updated - might have stale/incosistant info..] Satish On Thu, 28 Jul 2016, Munson, Todd wrote: > > Presumably. Argonne lost power during a thunderstorm and they are trying > to bring the site back up. > > Todd. > > > On Jul 28, 2016, at 2:35 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > > -- Norbert Wiener > >

1 0

Re: [petsc-dev] Is the website down?
by Munson, Todd 28 Jul '16

28 Jul '16

Presumably. Argonne lost power during a thunderstorm and they are trying to bring the site back up. Todd. > On Jul 28, 2016, at 2:35 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > > Matt > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Is the website down?
by Matthew Knepley 28 Jul '16

28 Jul '16

Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] hypre and threads
by Satish Balay 28 Jul '16

28 Jul '16

https://bitbucket.org/petsc/petsc/commits/e872ca4eda51751516e9741c875849477… satish On Thu, 28 Jul 2016, Mark Adams wrote: > I have a project (XGC) that has heard that hypre supports threads and they > are interested in using it. They would call PETSc outside of a threaded > section, of course, but can hypre pick up the threads in some way? > > Mark >

1 0

Re: [petsc-dev] hypre and threads
by Matthew Knepley 28 Jul '16

28 Jul '16

Have they lost their fucking collective mind? Matt On Thu, Jul 28, 2016 at 12:04 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > I have a project (XGC) that has heard that hypre supports threads and they > are interested in using it. They would call PETSc outside of a threaded > section, of course, but can hypre pick up the threads in some way? > > Mark > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0