July 2016 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] p4est configure error
by Mark Adams 07 Jul '16

07 Jul '16

> > > > /usr/bin/ld: attempted static link of dynamic object > `/opt/gcc/5.2.0/snos/lib/../lib64/libgomp.so' > > Looks like some libtool issue.. > That is probably it. At NERSC, Cori does not have a libtool module (in something else I assume) but it is required on Edison. I was missing it, Thanks, > > Satish > > > On Thu, 7 Jul 2016, Mark Adams wrote: > > > I nuked and ran this twice: > > > > 1013 rm -fr arch-xc30-dbg64-intel > > 1014 ../arch-xc30-dbg64-intel.py > > 1015 h > > 1016 ../arch-xc30-dbg64-intel.py > > > > Same error. > > > > > > On Thu, Jul 7, 2016 at 9:36 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > > > > perhaps configure was not rerun - so a stale p4est was being used.. > > > > > > Satish > > > > > > On Thu, 7 Jul 2016, Tobin Isaac wrote: > > > > > > > p4est.py automatically pulls from whatever p4est branch I call > 'petsc' > > > > in my repo, so I don't see how this problem persists if you really > > > > nuked it. > > > > > > > > - Did you nuke the right directory? > > > > - Did you send me the right configure.log? > > > > - Is the new failure the same (the same stale p4est commit is being > > > > used)? > > > > > > > > On Thu, Jul 07, 2016 at 08:33:57PM +0200, Mark Adams wrote: > > > > > It sounds like I have not merged your fix. Should I pull from > master? > > > > > > > > > > I have nuked this many times. > > > > > > > > > > On Thu, Jul 7, 2016 at 4:26 PM, Tobin Isaac <tisaac(a)uchicago.edu> > > > wrote: > > > > > > > > > > > > > > > > > On Thu, Jul 07, 2016 at 03:16:40PM +0200, Mark Adams wrote: > > > > > > > p4est on Edison configures in w/o debugging, although I have > to run > > > > > > > configure twice. > > > > > > > But with debugging it fails. I can not see any difference > between > > > the > > > > > > two > > > > > > > configures other than debugging. > > > > > > > Mark > > > > > > > > > > > > The log shows that you are configuring a p4est commit (3d3e1) > that is > > > > > > before the one I pushed specifically to avoid the fortran > > > > > > configuration error (f54bb3). Please clear the git.p4est > directory, > > > > > > and confirm that you are configuring with the current 'petsc' > branch > > > > > > of p4est (`grep 'checking Point version' configure.log` should > show > > > > > > X.XXX-a9f22). > > > > > > > > > > > > Cheers, > > > > > > Toby > > > > > > > > > > > > > > > > > > > >

1 0

Re: [petsc-dev] p4est configure error
by Satish Balay 07 Jul '16

07 Jul '16

>>>>>>>>>> Looking for P4EST at git.p4est, hg.p4est or a directory starting with p4est Found a copy of P4EST in git.p4est Executing: ['git', 'rev-parse', '--git-dir'] stdout: .git Executing: ['git', 'fetch'] Executing: ['git', 'rev-parse', 'origin/petsc'] stdout: a9f222be3099d0ca3aa19e2fe3d1aa0614491357 <snip> Executing: ['git', 'checkout', '-f', 'a9f222be3099d0ca3aa19e2fe3d1aa0614491357'] <<<<<<< https://github.com/tisaac/p4est/commits/petsc a9f222b Is the latest commit in this branch - so configure is using the correct p4est snapshot.. > /usr/bin/ld: attempted static link of dynamic object `/opt/gcc/5.2.0/snos/lib/../lib64/libgomp.so' Looks like some libtool issue.. Satish On Thu, 7 Jul 2016, Mark Adams wrote: > I nuked and ran this twice: > > 1013 rm -fr arch-xc30-dbg64-intel > 1014 ../arch-xc30-dbg64-intel.py > 1015 h > 1016 ../arch-xc30-dbg64-intel.py > > Same error. > > > On Thu, Jul 7, 2016 at 9:36 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > > perhaps configure was not rerun - so a stale p4est was being used.. > > > > Satish > > > > On Thu, 7 Jul 2016, Tobin Isaac wrote: > > > > > p4est.py automatically pulls from whatever p4est branch I call 'petsc' > > > in my repo, so I don't see how this problem persists if you really > > > nuked it. > > > > > > - Did you nuke the right directory? > > > - Did you send me the right configure.log? > > > - Is the new failure the same (the same stale p4est commit is being > > > used)? > > > > > > On Thu, Jul 07, 2016 at 08:33:57PM +0200, Mark Adams wrote: > > > > It sounds like I have not merged your fix. Should I pull from master? > > > > > > > > I have nuked this many times. > > > > > > > > On Thu, Jul 7, 2016 at 4:26 PM, Tobin Isaac <tisaac(a)uchicago.edu> > > wrote: > > > > > > > > > > > > > > On Thu, Jul 07, 2016 at 03:16:40PM +0200, Mark Adams wrote: > > > > > > p4est on Edison configures in w/o debugging, although I have to run > > > > > > configure twice. > > > > > > But with debugging it fails. I can not see any difference between > > the > > > > > two > > > > > > configures other than debugging. > > > > > > Mark > > > > > > > > > > The log shows that you are configuring a p4est commit (3d3e1) that is > > > > > before the one I pushed specifically to avoid the fortran > > > > > configuration error (f54bb3). Please clear the git.p4est directory, > > > > > and confirm that you are configuring with the current 'petsc' branch > > > > > of p4est (`grep 'checking Point version' configure.log` should show > > > > > X.XXX-a9f22). > > > > > > > > > > Cheers, > > > > > Toby > > > > > > > > > > > > >

1 0

Re: [petsc-dev] p4est configure error
by Mark Adams 07 Jul '16

07 Jul '16

I nuked and ran this twice: 1013 rm -fr arch-xc30-dbg64-intel 1014 ../arch-xc30-dbg64-intel.py 1015 h 1016 ../arch-xc30-dbg64-intel.py Same error. On Thu, Jul 7, 2016 at 9:36 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: > perhaps configure was not rerun - so a stale p4est was being used.. > > Satish > > On Thu, 7 Jul 2016, Tobin Isaac wrote: > > > p4est.py automatically pulls from whatever p4est branch I call 'petsc' > > in my repo, so I don't see how this problem persists if you really > > nuked it. > > > > - Did you nuke the right directory? > > - Did you send me the right configure.log? > > - Is the new failure the same (the same stale p4est commit is being > > used)? > > > > On Thu, Jul 07, 2016 at 08:33:57PM +0200, Mark Adams wrote: > > > It sounds like I have not merged your fix. Should I pull from master? > > > > > > I have nuked this many times. > > > > > > On Thu, Jul 7, 2016 at 4:26 PM, Tobin Isaac <tisaac(a)uchicago.edu> > wrote: > > > > > > > > > > > On Thu, Jul 07, 2016 at 03:16:40PM +0200, Mark Adams wrote: > > > > > p4est on Edison configures in w/o debugging, although I have to run > > > > > configure twice. > > > > > But with debugging it fails. I can not see any difference between > the > > > > two > > > > > configures other than debugging. > > > > > Mark > > > > > > > > The log shows that you are configuring a p4est commit (3d3e1) that is > > > > before the one I pushed specifically to avoid the fortran > > > > configuration error (f54bb3). Please clear the git.p4est directory, > > > > and confirm that you are configuring with the current 'petsc' branch > > > > of p4est (`grep 'checking Point version' configure.log` should show > > > > X.XXX-a9f22). > > > > > > > > Cheers, > > > > Toby > > > > > > > >

1 0

Re: [petsc-dev] Soliciting suggestions for linear solver work under SciDAC 4 Institutes
by Matthew Knepley 07 Jul '16

07 Jul '16

On Fri, Jul 1, 2016 at 4:32 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > The DOE SciDAC institutes have supported PETSc linear solver > research/code development for the past fifteen years. > > This email is to solicit ideas for linear solver research/code > development work for the next round of SciDAC institutes (which will be a 4 > year period) in PETSc. Please send me any ideas, no matter how crazy, on > things you feel are missing, broken, or incomplete in PETSc with regard to > linear solvers that we should propose to work on. In particular, issues > coming from particular classes of applications would be good. Generic > "multi physics" coupling types of things are too general (and old :-)) > while work for extreme large scale is also out since that is covered under > another call (ECP). But particular types of optimizations etc for existing > or new codes could be in, just not for the very large scale. > > Rough ideas and pointers to publications are all useful. There is an > extremely short fuse so the sooner the better, > I think the suggestions so far are fine, however they all seem to start at the "how", whereas I would prefer we start at the "why". Maybe something like 1) How do we run at bandwidth peak on new architectures like Cori or Aurora? Patrick and Rich have good suggestions here. Karl and Rich showed some promising numbers for KNL at the PETSc meeting. 2) How do we run larger Comp Chem simulations? Use sparse tensors as Jeff suggests. The baby example for this is MatMatMult(), which we have been screwing with for years, so its not easy. 3) How do we improve the user interface for multiphysics? Assembling pieces of a matrix should already work using http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetLocal… As Hong says, for solvers, MatGetSubMatrix() should be more scalable. 4) How do we precondition Helmholtz? This is a perennial ask on petsc-maint, and Domenico gave a great presentation on his results in Vienna. Just pulling this into the library would be worthwhile. For me, there are richer questions in the nonlinear arena, such as a) What smoothers are appropriate for FAS? I think NASM, but there is nonlinear row-projection (ABS), NBGS, NGMRES, NCG (never works), and others. We have no good theory here. The gap is yawning. b) How should NASM be formulated? Do we take big blocks or small blocks? What BC do we use? How do we specify the BC to PETSc once we decide what they should be? Considering that Xiao-Chuan, David, Hans DeSterck, and Rolf Krause all have multiple papers on some sort of NASM this year makes me think this is a good topic. c) How do we rework the SNES loop to accommodate preconditioning? I have a branch that proposes a new organization, but we do not even have enough tests to make me confident that I am not breaking things. We should be writing something on the level of the Templates book for nonlinear, since that is what people are asking for to understand this stuff. I think its too bad if fusty DOE categories prevent us from suggesting some subset of the above. Thanks, Matt > Thanks > > Barry > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] p4est configure error
by Satish Balay 07 Jul '16

07 Jul '16

perhaps configure was not rerun - so a stale p4est was being used.. Satish On Thu, 7 Jul 2016, Tobin Isaac wrote: > p4est.py automatically pulls from whatever p4est branch I call 'petsc' > in my repo, so I don't see how this problem persists if you really > nuked it. > > - Did you nuke the right directory? > - Did you send me the right configure.log? > - Is the new failure the same (the same stale p4est commit is being > used)? > > On Thu, Jul 07, 2016 at 08:33:57PM +0200, Mark Adams wrote: > > It sounds like I have not merged your fix. Should I pull from master? > > > > I have nuked this many times. > > > > On Thu, Jul 7, 2016 at 4:26 PM, Tobin Isaac <tisaac(a)uchicago.edu> wrote: > > > > > > > > On Thu, Jul 07, 2016 at 03:16:40PM +0200, Mark Adams wrote: > > > > p4est on Edison configures in w/o debugging, although I have to run > > > > configure twice. > > > > But with debugging it fails. I can not see any difference between the > > > two > > > > configures other than debugging. > > > > Mark > > > > > > The log shows that you are configuring a p4est commit (3d3e1) that is > > > before the one I pushed specifically to avoid the fortran > > > configuration error (f54bb3). Please clear the git.p4est directory, > > > and confirm that you are configuring with the current 'petsc' branch > > > of p4est (`grep 'checking Point version' configure.log` should show > > > X.XXX-a9f22). > > > > > > Cheers, > > > Toby > > > >

1 0

Re: [petsc-dev] p4est configure error
by Tobin Isaac 07 Jul '16

07 Jul '16

p4est.py automatically pulls from whatever p4est branch I call 'petsc' in my repo, so I don't see how this problem persists if you really nuked it. - Did you nuke the right directory? - Did you send me the right configure.log? - Is the new failure the same (the same stale p4est commit is being used)? On Thu, Jul 07, 2016 at 08:33:57PM +0200, Mark Adams wrote: > It sounds like I have not merged your fix. Should I pull from master? > > I have nuked this many times. > > On Thu, Jul 7, 2016 at 4:26 PM, Tobin Isaac <tisaac(a)uchicago.edu> wrote: > > > > > On Thu, Jul 07, 2016 at 03:16:40PM +0200, Mark Adams wrote: > > > p4est on Edison configures in w/o debugging, although I have to run > > > configure twice. > > > But with debugging it fails. I can not see any difference between the > > two > > > configures other than debugging. > > > Mark > > > > The log shows that you are configuring a p4est commit (3d3e1) that is > > before the one I pushed specifically to avoid the fortran > > configuration error (f54bb3). Please clear the git.p4est directory, > > and confirm that you are configuring with the current 'petsc' branch > > of p4est (`grep 'checking Point version' configure.log` should show > > X.XXX-a9f22). > > > > Cheers, > > Toby > >

1 0

Re: [petsc-dev] p4est configure error
by Mark Adams 07 Jul '16

07 Jul '16

It sounds like I have not merged your fix. Should I pull from master? I have nuked this many times. On Thu, Jul 7, 2016 at 4:26 PM, Tobin Isaac <tisaac(a)uchicago.edu> wrote: > > On Thu, Jul 07, 2016 at 03:16:40PM +0200, Mark Adams wrote: > > p4est on Edison configures in w/o debugging, although I have to run > > configure twice. > > But with debugging it fails. I can not see any difference between the > two > > configures other than debugging. > > Mark > > The log shows that you are configuring a p4est commit (3d3e1) that is > before the one I pushed specifically to avoid the fortran > configuration error (f54bb3). Please clear the git.p4est directory, > and confirm that you are configuring with the current 'petsc' branch > of p4est (`grep 'checking Point version' configure.log` should show > X.XXX-a9f22). > > Cheers, > Toby >

1 0

Re: [petsc-dev] Recent changes in DMLabel
by Lawrence Mitchell 07 Jul '16

07 Jul '16

On 07/07/16 17:22, Tobin Isaac wrote: > > Hi Lawrence, > > Bork me, sorry about that. Thanks for patch! I've merged it into > next. Great, thanks. Lawrence

1 0

Re: [petsc-dev] Recent changes in DMLabel
by Tobin Isaac 07 Jul '16

07 Jul '16

Hi Lawrence, Bork me, sorry about that. Thanks for patch! I've merged it into next. Cheers, Toby On Thu, Jul 07, 2016 at 04:29:04PM +0100, Lawrence Mitchell wrote: > Dear all (but mostly Toby), > > some recent changes in DMLabel changed the semantics of DMLabelHasPoint. > > In particular, after ad8374f (DMLabel: switch internal storage from > array to IS), clearing a label value produces bad results. Consider > the following: > > #include <petsc.h> > > int main(int argc, char **argv) > { > PetscErrorCode ierr; > DMLabel label; > PetscInt val; > PetscBool flg; > PetscInitialize(&argc, &argv, NULL, NULL); > > ierr = DMLabelCreate("foo", &label);CHKERRQ(ierr); > > ierr = DMLabelCreateIndex(label, 0, 1);CHKERRQ(ierr); > ierr = DMLabelSetValue(label, 0, 1);CHKERRQ(ierr); > ierr = DMLabelHasPoint(label, 0, &flg);CHKERRQ(ierr); > ierr = DMLabelGetValue(label, 0, &val);CHKERRQ(ierr); > PetscPrintf(PETSC_COMM_WORLD, "Label has point %d, value is %d\n", > flg, val);CHKERRQ(ierr); > > ierr = DMLabelClearValue(label, 0, 1);CHKERRQ(ierr); > ierr = DMLabelHasPoint(label, 0, &flg);CHKERRQ(ierr); > ierr = DMLabelGetValue(label, 0, &val);CHKERRQ(ierr); > PetscPrintf(PETSC_COMM_WORLD, "Label has point %d, value is %d\n", > flg, val);CHKERRQ(ierr); > > PetscFinalize(); > return 0; > } > > Before this change: > > $ ./bork > Label has point 1, value is 1 > Label has point 0, value is -1 > > After the change: > > $ ./bork > Label has point 1, value is 1 > Label has point 1, value is -1 > > So you can see that although the point has been cleared from the point > of view of GetValue, it has not if I use HasPoint. > > The problem appears to be that the invalidation of the label after > ClearValue no longer clears the PetscBT entry that exists if an index > has been created. > > Perhaps the following patch is right? > > diff --git a/src/dm/label/dmlabel.c b/src/dm/label/dmlabel.c > index 8d170c6..2c7f7c2 100644 > --- a/src/dm/label/dmlabel.c > +++ b/src/dm/label/dmlabel.c > @@ -665,7 +665,13 @@ PetscErrorCode DMLabelClearValue(DMLabel label, > PetscInt point, PetscInt value) > if (label->stratumValues[v] == value) break; > } > if (v >= label->numStrata) PetscFunctionReturn(0); > - if (label->validIS[v]) {ierr = > DMLabelMakeInvalid_Private(label,v);CHKERRQ(ierr);} > + if (label->validIS[v]) { > + ierr = DMLabelMakeInvalid_Private(label,v);CHKERRQ(ierr); > + } > + if (label->bt) { > + if ((point < label->pStart) || (point >= label->pEnd)) > SETERRQ3(PETSC_COMM_SELF, PETSC_ERR_ARG_OUTOFRANGE, "Label point %D is > not in [%D, %D)", point, label->pStart, label->pEnd); > + ierr = PetscBTClear(label->bt, point - label->pStart);CHKERRQ(ierr); > + } > ierr = PetscHashIDelKey(label->ht[v], point);CHKERRQ(ierr); > PetscFunctionReturn(0); > } > > > Cheers, > > Lawrence >

1 0

Recent changes in DMLabel
by Lawrence Mitchell 07 Jul '16

07 Jul '16

Dear all (but mostly Toby), some recent changes in DMLabel changed the semantics of DMLabelHasPoint. In particular, after ad8374f (DMLabel: switch internal storage from array to IS), clearing a label value produces bad results. Consider the following: #include <petsc.h> int main(int argc, char **argv) { PetscErrorCode ierr; DMLabel label; PetscInt val; PetscBool flg; PetscInitialize(&argc, &argv, NULL, NULL); ierr = DMLabelCreate("foo", &label);CHKERRQ(ierr); ierr = DMLabelCreateIndex(label, 0, 1);CHKERRQ(ierr); ierr = DMLabelSetValue(label, 0, 1);CHKERRQ(ierr); ierr = DMLabelHasPoint(label, 0, &flg);CHKERRQ(ierr); ierr = DMLabelGetValue(label, 0, &val);CHKERRQ(ierr); PetscPrintf(PETSC_COMM_WORLD, "Label has point %d, value is %d\n", flg, val);CHKERRQ(ierr); ierr = DMLabelClearValue(label, 0, 1);CHKERRQ(ierr); ierr = DMLabelHasPoint(label, 0, &flg);CHKERRQ(ierr); ierr = DMLabelGetValue(label, 0, &val);CHKERRQ(ierr); PetscPrintf(PETSC_COMM_WORLD, "Label has point %d, value is %d\n", flg, val);CHKERRQ(ierr); PetscFinalize(); return 0; } Before this change: $ ./bork Label has point 1, value is 1 Label has point 0, value is -1 After the change: $ ./bork Label has point 1, value is 1 Label has point 1, value is -1 So you can see that although the point has been cleared from the point of view of GetValue, it has not if I use HasPoint. The problem appears to be that the invalidation of the label after ClearValue no longer clears the PetscBT entry that exists if an index has been created. Perhaps the following patch is right? diff --git a/src/dm/label/dmlabel.c b/src/dm/label/dmlabel.c index 8d170c6..2c7f7c2 100644 --- a/src/dm/label/dmlabel.c +++ b/src/dm/label/dmlabel.c @@ -665,7 +665,13 @@ PetscErrorCode DMLabelClearValue(DMLabel label, PetscInt point, PetscInt value) if (label->stratumValues[v] == value) break; } if (v >= label->numStrata) PetscFunctionReturn(0); - if (label->validIS[v]) {ierr = DMLabelMakeInvalid_Private(label,v);CHKERRQ(ierr);} + if (label->validIS[v]) { + ierr = DMLabelMakeInvalid_Private(label,v);CHKERRQ(ierr); + } + if (label->bt) { + if ((point < label->pStart) || (point >= label->pEnd)) SETERRQ3(PETSC_COMM_SELF, PETSC_ERR_ARG_OUTOFRANGE, "Label point %D is not in [%D, %D)", point, label->pStart, label->pEnd); + ierr = PetscBTClear(label->bt, point - label->pStart);CHKERRQ(ierr); + } ierr = PetscHashIDelKey(label->ht[v], point);CHKERRQ(ierr); PetscFunctionReturn(0); } Cheers, Lawrence

1 0