June 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Modifying code in seqcusp folder
by Satish Balay 22 Jun '12

22 Jun '12

On Thu, 21 Jun 2012, Matthew Knepley wrote: > On Thu, Jun 21, 2012 at 1:48 PM, Chekuri Choudary > <cchoudary(a)rnet-tech.com>wrote: > > > Typically, when I make changes to petsc code, I do a make lib in the > > folder and then recompile my code. That doesnï¿½t seem to be working in > > seqcusp folder. I do a make lib and nothing happens. Can somebody please > > help?? > > > > Yes, the check for stale CUDA code was broken. Satish, did this get fixed > or is it still broken? Now fixed in petsc-dev http://petsc.cs.iit.edu/petsc/petsc-dev/rev/587a2035630d http://petsc.cs.iit.edu/petsc/BuildSystem/rev/f247c0b6e88b Satish

1 0

Re: [petsc-dev] urgent issues with DMDA_BOUNDARY_GHOSTED DMDA_BOUNDARY_MIRROR
by Barry Smith 22 Jun '12

22 Jun '12

On Jun 22, 2012, at 3:55 PM, Jed Brown wrote: > Oh, in the case of mirror boundary conditions, the ideal "automatic" thing would be for MatSetValuesLocal() to flip the sign of those matrix entries and insert into the rows and columns corresponding to the interior cells that were mirrored. The problem is that this is just one more special case, inflow pressure boundaries, for example, use the equation of state to fill in the ghost cells, so remapping those entries into the domain is not so simple. It would be fantastic to work out an API for this, but getting the right granularity seems to be tricky since people like to fill in ghost cells in very different ways. That's nuts. Way to special case. :-) Barry > > On Thu, Jun 21, 2012 at 5:45 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Jun 21, 2012, at 6:09 PM, Jed Brown wrote: > > > I think that is fine, though having mirrored ghost points in the local-to-global map would be more convenient for matrix assembly. > > Hmm, what do you mean by this? What should we do? > > Barry > > > > > On Jun 20, 2012 2:51 PM, "Barry Smith" <bsmith(a)mcs.anl.gov> wrote: > > > > My inclination is to remove the mirror enum and simply provide a utility function that updates the ghost points by mirroring in the local vector form that the user can call if they want mirroring? > > > > Barry > > > > On Jun 20, 2012, at 5:41 PM, Barry Smith wrote: > > > > > > > > What is the intended difference between DMDA_BOUNDARY_GHOSTED and DMDA_BOUNDARY_MIRROR? > > > > > > My understanding is that both cause ghost points around the physical boundary that can be indexed in the usual way (for example on the left side just indexing with -1 to get the first of the ghost points? Looking at the code but not testing it I assume that the correctly sized local vectors are created and the appropriate scatters are created that put things in the correct place despite these extra locations with calls to DMGlobalToLocal()? > > > > > > My guess is that whoever decided to have both of these was thinking that with DMDA_BOUNDARY_GHOSTED the user would fill up the ghost points themselves however they liked with whatever values they want whenever they want? Meanwhile with mirror PETSc would fill up the ghost points by copying the mirrored values from inside the real physical domain? Thus ghosted is more general. > > > > > > But when was it intended that the mirrored values be copied over? During the DMGlobalToLocal call or in a new call? (Note that this copying requires no parallel communication) > > > > > > Is there an example of using DMDA_BOUNDARY_GHOSTED where the ghosted values are set via mirroring? Should there be a utility to do that? > > > > > > We have someone who needs this mirroring (due to Jed's advice) but appear not to have any code to make that trivial. How shall we fix this? > > > > > > Thanks > > > > > > Barry > > > > > > >

1 0

Re: [petsc-dev] urgent issues with DMDA_BOUNDARY_GHOSTED DMDA_BOUNDARY_MIRROR
by Jed Brown 22 Jun '12

22 Jun '12

Oh, in the case of mirror boundary conditions, the ideal "automatic" thing would be for MatSetValuesLocal() to flip the sign of those matrix entries and insert into the rows and columns corresponding to the interior cells that were mirrored. The problem is that this is just one more special case, inflow pressure boundaries, for example, use the equation of state to fill in the ghost cells, so remapping those entries into the domain is not so simple. It would be fantastic to work out an API for this, but getting the right granularity seems to be tricky since people like to fill in ghost cells in very different ways. On Thu, Jun 21, 2012 at 5:45 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Jun 21, 2012, at 6:09 PM, Jed Brown wrote: > > > I think that is fine, though having mirrored ghost points in the > local-to-global map would be more convenient for matrix assembly. > > Hmm, what do you mean by this? What should we do? > > Barry > > > > > On Jun 20, 2012 2:51 PM, "Barry Smith" <bsmith(a)mcs.anl.gov> wrote: > > > > My inclination is to remove the mirror enum and simply provide a > utility function that updates the ghost points by mirroring in the local > vector form that the user can call if they want mirroring? > > > > Barry > > > > On Jun 20, 2012, at 5:41 PM, Barry Smith wrote: > > > > > > > > What is the intended difference between DMDA_BOUNDARY_GHOSTED and > DMDA_BOUNDARY_MIRROR? > > > > > > My understanding is that both cause ghost points around the > physical boundary that can be indexed in the usual way (for example on the > left side just indexing with -1 to get the first of the ghost points? > Looking at the code but not testing it I assume that the correctly sized > local vectors are created and the appropriate scatters are created that put > things in the correct place despite these extra locations with calls to > DMGlobalToLocal()? > > > > > > My guess is that whoever decided to have both of these was > thinking that with DMDA_BOUNDARY_GHOSTED the user would fill up the ghost > points themselves however they liked with whatever values they want > whenever they want? Meanwhile with mirror PETSc would fill up the ghost > points by copying the mirrored values from inside the real physical domain? > Thus ghosted is more general. > > > > > > But when was it intended that the mirrored values be copied over? > During the DMGlobalToLocal call or in a new call? (Note that this copying > requires no parallel communication) > > > > > > Is there an example of using DMDA_BOUNDARY_GHOSTED where the > ghosted values are set via mirroring? Should there be a utility to do that? > > > > > > We have someone who needs this mirroring (due to Jed's advice) but > appear not to have any code to make that trivial. How shall we fix this? > > > > > > Thanks > > > > > > Barry > > > > > > >

1 0

Re: [petsc-dev] [petsc-users] make test failed
by Satish Balay 22 Jun '12

22 Jun '12

ok pushed a fix. http://petsc.cs.iit.edu/petsc/releases/petsc-3.3/rev/7a4f6980d770 satish On Wed, 20 Jun 2012, Barry Smith wrote: > > Add a little python function that is called at the beginning of ./configure that goes through all the scripts and removes \r if they exist in the scripts. > > On non-Windows the function will do nothing and on Windows if either Mecurial or properly extracted tarball will do nothing. Will only do something if user extracted tarball wrong. > > Problem solved, doesn't require changing anything else in PETSc. > > > Barry > > On Jun 19, 2012, at 11:51 PM, Satish Balay wrote: > > > On Tue, 19 Jun 2012, Barry Smith wrote: > > > >>> It appears that the extra '\r' char is causing bash to process strings > >>> incorrectly. > > > >> Yes, but exactly how? Run the winzip to get the extra \r in > >> them. > > > > BTW: unix2dos is equivalent to winzip wrt reproduction. > > > >> Then start removing them as you run the script to determine > >> exactly where the grief comes from. I do not think the extra \r are > >> always harmful, just if they come up in the wrong place. If you > >> determine this you can reformat the original code to not have the > >> bad \r cause grief. > > > > Assume the extra char is 'x' [instead of '\r'] for illustration. > > > > i.e: > > var='foo'\r > > is represented as > > var='foo'x > > > > Note: the above is valid syntax [but 'var' will have extra stuff that > > wasn't intended] > > > > > > The primary issue is - if you have the following code [in dos formatted text]: > >>>> > > if [ 'a' == 'a'] > > then > > echo "yes" > > else > > echo "no" > > fi > > <<< > > It is interpreted as: > >>>> > > if [ 'a' == 'a']x > > thenx > > echo "yes"x > > elsex > > echo "no"x > > fix > > <<< > > > > And you get the error "syntax error: unexpected end of file" > > > > One way to workarround is to add 'var=foo' before '\r' and > > get some valid bash syntax. i.e something like: > > > >>>> > > if [ 'a' == 'a'] then a='a'x > > echo "yes"; a='a'x > > else a='a'x > > echo "no"; a='a'x > > fi; a='a'x > > <<<< > > > > So the fixed mpiexec.uni looks something like: > >>>>>> > > $ cat mpiexec.uni > > #!/bin/sh > > if [ $1 != "-np" -a $1 != "-n" ]; then a=a > > progname=$*; a=a > > elif [ $2 = "1" ]; then a=a > > shift ; shift; a=a > > progname=$* ; a=a > > else a=a > > echo "Uniprocessor version of MPI can only use one processor"; a=a > > exit 1; a=a > > fi; a=a > > a=a > > # Execute the program > > $progname;a=a > > exit 0;a=a > > <<<<<<< > > > > There is an alternative of setting the following env before running > > bash scripts in cygwin - but these commands appear to be cygwin/bash > > specific [so cannot easily automate?] > > > >>>>>>>> > > export SHELLOPTS > > set -o igncr > > <now run the script> > > <<<<<<<< > > > > Satish > > > >

1 0

Re: [petsc-dev] PetscLogView vs. -log_summary
by Barry Smith 21 Jun '12

21 Jun '12

On Jun 21, 2012, at 10:46 PM, Matthew Knepley wrote: > On Thu, Jun 21, 2012 at 4:16 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > > > That works, yes. Is the issue that stages / events need to be activated a priori, which is not possible when PetscLogView is only called at the end of the program? > > > > Crap. > > Matt, why is this "Crap" its completely reasonable that if you want to have logging from program control that you have to turn it on programatically? > > Crap, because I should have remembered earlier. FAQ or some other mechanism to "remember this". We can't be relying on our memories :-) Actually, why not have PetscLogView() print an error message if PetscLogBegin() was not previously called? Instead of just not have data to display? Then we won't need to remember anything? Barry > > Matt > > > If you do not provide -log_summary, you need PetscLogBegin() at the beginning. > > > > Matt > > > > The caveat is that I am assuming that when using log_summary, the log is written as part of PetscFinalize, so that if a job is killed by the sceduler or the user, there is no way to get an idea of the performances. I like to print the log every so often. > > > > Blaise > > > > -- > > Department of Mathematics and Center for Computation & Technology > > Louisiana State University, Baton Rouge, LA 70803, USA > > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin > > > > > > > > > > > > > > > > > > > > > > -- > > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] PetscLogView vs. -log_summary
by Matthew Knepley 21 Jun '12

21 Jun '12

On Thu, Jun 21, 2012 at 4:16 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Jun 21, 2012, at 3:40 PM, Matthew Knepley wrote: > > > On Thu, Jun 21, 2012 at 2:08 PM, Blaise Bourdin <bourdin(a)lsu.edu> wrote: > > > > On Jun 21, 2012, at 2:55 PM, Matthew Knepley wrote: > > > >> On Thu, Jun 21, 2012 at 1:11 PM, Blaise Bourdin <bourdin(a)math.lsu.edu> > wrote: > >> Hi, > >> > >> PetscLogView does not display the same amount of information as running > with -log_summary. In the "summary of stages" section, the former contains > only information on the main stage while the later contains much more. > >> > >> Add the following block at the end of > src/snes/examples/tutorials/ex1.c, for instance, to see what I mean > >> PetscViewer viewer; > >> ierr = > PetscViewerASCIIOpen(PETSC_COMM_WORLD,"ex1-log.py",&viewer);CHKERRQ(ierr); > >> ierr = PetscLogViewPython(viewer);CHKERRQ(ierr); > >> ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); > >> ierr = > PetscViewerASCIIOpen(PETSC_COMM_WORLD,"ex1-log.txt",&viewer);CHKERRQ(ierr); > >> ierr = PetscLogView(viewer);CHKERRQ(ierr); > >> ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); > >> ierr = > PetscLogPrintDetailed(PETSC_COMM_WORLD,"ex1-logdetailed.txt");CHKERRQ(ierr); > >> > >> Is this a bug or a feature? Is there a way to get the same output as > -log_summary in a file, without printing it to stdout? > >> > >> -log_summary just calls PetscLogView(). I am not sure how there could > be a problem. I will run it. However, can you > >> just use > >> > >> -log_summary <filename> > > > > That works, yes. Is the issue that stages / events need to be activated > a priori, which is not possible when PetscLogView is only called at the end > of the program? > > > > Crap. > > Matt, why is this "Crap" its completely reasonable that if you want > to have logging from program control that you have to turn it on > programatically? > Crap, because I should have remembered earlier. Matt > > If you do not provide -log_summary, you need PetscLogBegin() at the > beginning. > > > > Matt > > > > The caveat is that I am assuming that when using log_summary, the log is > written as part of PetscFinalize, so that if a job is killed by the > sceduler or the user, there is no way to get an idea of the performances. I > like to print the log every so often. > > > > Blaise > > > > -- > > Department of Mathematics and Center for Computation & Technology > > Louisiana State University, Baton Rouge, LA 70803, USA > > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 > http://www.math.lsu.edu/~bourdin > > > > > > > > > > > > > > > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] urgent issues with DMDA_BOUNDARY_GHOSTED DMDA_BOUNDARY_MIRROR
by Barry Smith 21 Jun '12

21 Jun '12

On Jun 21, 2012, at 6:09 PM, Jed Brown wrote: > I think that is fine, though having mirrored ghost points in the local-to-global map would be more convenient for matrix assembly. Hmm, what do you mean by this? What should we do? Barry > > On Jun 20, 2012 2:51 PM, "Barry Smith" <bsmith(a)mcs.anl.gov> wrote: > > My inclination is to remove the mirror enum and simply provide a utility function that updates the ghost points by mirroring in the local vector form that the user can call if they want mirroring? > > Barry > > On Jun 20, 2012, at 5:41 PM, Barry Smith wrote: > > > > > What is the intended difference between DMDA_BOUNDARY_GHOSTED and DMDA_BOUNDARY_MIRROR? > > > > My understanding is that both cause ghost points around the physical boundary that can be indexed in the usual way (for example on the left side just indexing with -1 to get the first of the ghost points? Looking at the code but not testing it I assume that the correctly sized local vectors are created and the appropriate scatters are created that put things in the correct place despite these extra locations with calls to DMGlobalToLocal()? > > > > My guess is that whoever decided to have both of these was thinking that with DMDA_BOUNDARY_GHOSTED the user would fill up the ghost points themselves however they liked with whatever values they want whenever they want? Meanwhile with mirror PETSc would fill up the ghost points by copying the mirrored values from inside the real physical domain? Thus ghosted is more general. > > > > But when was it intended that the mirrored values be copied over? During the DMGlobalToLocal call or in a new call? (Note that this copying requires no parallel communication) > > > > Is there an example of using DMDA_BOUNDARY_GHOSTED where the ghosted values are set via mirroring? Should there be a utility to do that? > > > > We have someone who needs this mirroring (due to Jed's advice) but appear not to have any code to make that trivial. How shall we fix this? > > > > Thanks > > > > Barry > > >

1 0

Re: [petsc-dev] urgent issues with DMDA_BOUNDARY_GHOSTED DMDA_BOUNDARY_MIRROR
by Jed Brown 21 Jun '12

21 Jun '12

I think that is fine, though having mirrored ghost points in the local-to-global map would be more convenient for matrix assembly. On Jun 20, 2012 2:51 PM, "Barry Smith" <bsmith(a)mcs.anl.gov> wrote: > > My inclination is to remove the mirror enum and simply provide a utility > function that updates the ghost points by mirroring in the local vector > form that the user can call if they want mirroring? > > Barry > > On Jun 20, 2012, at 5:41 PM, Barry Smith wrote: > > > > > What is the intended difference between DMDA_BOUNDARY_GHOSTED and > DMDA_BOUNDARY_MIRROR? > > > > My understanding is that both cause ghost points around the physical > boundary that can be indexed in the usual way (for example on the left side > just indexing with -1 to get the first of the ghost points? Looking at the > code but not testing it I assume that the correctly sized local vectors are > created and the appropriate scatters are created that put things in the > correct place despite these extra locations with calls to DMGlobalToLocal()? > > > > My guess is that whoever decided to have both of these was thinking > that with DMDA_BOUNDARY_GHOSTED the user would fill up the ghost points > themselves however they liked with whatever values they want whenever they > want? Meanwhile with mirror PETSc would fill up the ghost points by > copying the mirrored values from inside the real physical domain? Thus > ghosted is more general. > > > > But when was it intended that the mirrored values be copied over? > During the DMGlobalToLocal call or in a new call? (Note that this copying > requires no parallel communication) > > > > Is there an example of using DMDA_BOUNDARY_GHOSTED where the ghosted > values are set via mirroring? Should there be a utility to do that? > > > > We have someone who needs this mirroring (due to Jed's advice) but > appear not to have any code to make that trivial. How shall we fix this? > > > > Thanks > > > > Barry > > > >

1 0

Re: [petsc-dev] PetscLogView vs. -log_summary
by Barry Smith 21 Jun '12

21 Jun '12

On Jun 21, 2012, at 3:40 PM, Matthew Knepley wrote: > On Thu, Jun 21, 2012 at 2:08 PM, Blaise Bourdin <bourdin(a)lsu.edu> wrote: > > On Jun 21, 2012, at 2:55 PM, Matthew Knepley wrote: > >> On Thu, Jun 21, 2012 at 1:11 PM, Blaise Bourdin <bourdin(a)math.lsu.edu> wrote: >> Hi, >> >> PetscLogView does not display the same amount of information as running with -log_summary. In the "summary of stages" section, the former contains only information on the main stage while the later contains much more. >> >> Add the following block at the end of src/snes/examples/tutorials/ex1.c, for instance, to see what I mean >> PetscViewer viewer; >> ierr = PetscViewerASCIIOpen(PETSC_COMM_WORLD,"ex1-log.py",&viewer);CHKERRQ(ierr); >> ierr = PetscLogViewPython(viewer);CHKERRQ(ierr); >> ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); >> ierr = PetscViewerASCIIOpen(PETSC_COMM_WORLD,"ex1-log.txt",&viewer);CHKERRQ(ierr); >> ierr = PetscLogView(viewer);CHKERRQ(ierr); >> ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); >> ierr = PetscLogPrintDetailed(PETSC_COMM_WORLD,"ex1-logdetailed.txt");CHKERRQ(ierr); >> >> Is this a bug or a feature? Is there a way to get the same output as -log_summary in a file, without printing it to stdout? >> >> -log_summary just calls PetscLogView(). I am not sure how there could be a problem. I will run it. However, can you >> just use >> >> -log_summary <filename> > > That works, yes. Is the issue that stages / events need to be activated a priori, which is not possible when PetscLogView is only called at the end of the program? > > Crap. Matt, why is this "Crap" its completely reasonable that if you want to have logging from program control that you have to turn it on programatically? > If you do not provide -log_summary, you need PetscLogBegin() at the beginning. > > Matt > > The caveat is that I am assuming that when using log_summary, the log is written as part of PetscFinalize, so that if a job is killed by the sceduler or the user, there is no way to get an idea of the performances. I like to print the log every so often. > > Blaise > > -- > Department of Mathematics and Center for Computation & Technology > Louisiana State University, Baton Rouge, LA 70803, USA > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin > > > > > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] PetscLogView vs. -log_summary
by Blaise Bourdin 21 Jun '12

21 Jun '12

On Jun 21, 2012, at 3:40 PM, Matthew Knepley wrote: > On Thu, Jun 21, 2012 at 2:08 PM, Blaise Bourdin <bourdin(a)lsu.edu> wrote: > > On Jun 21, 2012, at 2:55 PM, Matthew Knepley wrote: > >> On Thu, Jun 21, 2012 at 1:11 PM, Blaise Bourdin <bourdin(a)math.lsu.edu> wrote: >> Hi, >> >> PetscLogView does not display the same amount of information as running with -log_summary. In the "summary of stages" section, the former contains only information on the main stage while the later contains much more. >> >> Add the following block at the end of src/snes/examples/tutorials/ex1.c, for instance, to see what I mean >> PetscViewer viewer; >> ierr = PetscViewerASCIIOpen(PETSC_COMM_WORLD,"ex1-log.py",&viewer);CHKERRQ(ierr); >> ierr = PetscLogViewPython(viewer);CHKERRQ(ierr); >> ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); >> ierr = PetscViewerASCIIOpen(PETSC_COMM_WORLD,"ex1-log.txt",&viewer);CHKERRQ(ierr); >> ierr = PetscLogView(viewer);CHKERRQ(ierr); >> ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); >> ierr = PetscLogPrintDetailed(PETSC_COMM_WORLD,"ex1-logdetailed.txt");CHKERRQ(ierr); >> >> Is this a bug or a feature? Is there a way to get the same output as -log_summary in a file, without printing it to stdout? >> >> -log_summary just calls PetscLogView(). I am not sure how there could be a problem. I will run it. However, can you >> just use >> >> -log_summary <filename> > > That works, yes. Is the issue that stages / events need to be activated a priori, which is not possible when PetscLogView is only called at the end of the program? > > Crap. If you do not provide -log_summary, you need PetscLogBegin() at the beginning. Of course... It is even written in the manual page for PetscLogView Thanks, Blaise -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin

1 0