March 2011 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Barry Smith 14 Mar '11

14 Mar '11

On Mar 13, 2011, at 11:34 PM, Jed Brown wrote: > On Mon, Mar 14, 2011 at 05:01, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > So it is okay to include headers from the source tree other than from > under $PETSC_DIR/include? > I thought the subject of this thread expressly banned that. > > That is a style decision that is up to Barry and not something I was trying to address in this latest patch. > > Currently, most things uses <../src/path/to/localinclude.h> but there are a handful of exceptions. I actually prefer using the "localinclude.h" instead of <../src/path/to/localinclude.h> because the former is easier to relocate and recompile (perhaps with a prefix-install where $PETSC_DIR/src is no longer around). In other words, using "localinclude.h" makes stuff in src/pkg/impls/xxx/ appear more like a standalone plugin which I think is a good thing. For files like aij.h if you use the localinclude.h paradigm then in aij.c you would have #include "aij.h" but in mpiaij.h you would have #include <./src/mat/impls/aij/seq/aij.h> I don't like the fact that in different files the sameinclude is referenced in a different way. Now one could argue that private includes used ONLY locally use the "localinclude.h" model and private includes used more than locally use the <../src/path/to/localinclude.h> but why? Better the simplicity of always using the <../src/path/to/localinclude.h> model. Barry > > This email thread is pretty old now, I think there was some misunderstanding of whether the compiler's $PWD affected header resolution (it doesn't) and recall some possible IDE issues but don't know if they were resolved.

1 0

Re: [petsc-dev] Using PCFieldSplitSetIS
by Jed Brown 14 Mar '11

14 Mar '11

On Mon, Mar 14, 2011 at 12:32, Thomas Witkowski < thomas.witkowski(a)tu-dresden.de> wrote: > Should I define blocks or splits for the subdomains and the interior nodes? > And what is the best way to force PETSc to make some LU factorization on > each subdomain and to store it (it is needed to create the reduced Schur > system, to define the action of the Schur complement operator and to solve > the subdomain unknows in the last step) and to use it later? Okay, define two splits. The first consists of all the interior nodes, the second has all the interface nodes. Now use -pc_fieldsplit_type schur -fieldsplit_0_ksp_type preonly -fieldsplit_0_pc_type bjacobi -fieldsplit_0_sub_pc_type lu. Remember to look at -ksp_view and -help for options. You have a choice of preconditioning the Schur complement, by default it just uses the interface matrix itself (which is usually nearly diagonal). That is, performing block Jacobi with direct subdomain solves on the (parallel) interior matrix will be the same as a direct solve with this matrix because all the subdomains are actually uncoupled. My point about exposing less concurrency had to do with always needing to solve problems with the parallel interior-node matrix which could actually be stored separately since the systems are not truly coupled. This is most relevant with multiple subdomains per process or if you are forming an explicit Schur complement (to build a course level operator, such as with FETI-DP/BDDC).

1 0

Re: [petsc-dev] Using PCFieldSplitSetIS
by Thomas Witkowski 14 Mar '11

14 Mar '11

Jed, I'm a little bit confused about your and Matt's answers. I played a little bit with PFCFieldSplit, and I think that I got some basic understanding of this concept. My code now creates two splits. One for the unknowns of all subdomain interior nodes, and one for all unknowns on the subdomain boundaries. This works fine. But what I want to implement is a simple Schur complement approach. So all processes should make an LU factorization on their subdomain first. Then all together should solve the reduced Schur system to get the values on the subdomain boundaries. And to the last all processes can easily compute the unknown on their subdomains. Yes, I know that this is not very efficient. But it doesn't matter. It is just to learn something about using PETSc in this context. But till now I don't know what's the best way to implement this Schur complement approach? Should I define blocks or splits for the subdomains and the interior nodes? And what is the best way to force PETSc to make some LU factorization on each subdomain and to store it (it is needed to create the reduced Schur system, to define the action of the Schur complement operator and to solve the subdomain unknows in the last step) and to use it later? Maybe some of you could give me some rough hints? Thanks, Thomas Jed Brown wrote: > > Thomas, I think Matt may have misunderstood what you wanted. > PCFieldSplit does not solve multiple splits concurrently. You can > emulate the first solve you need by defining a block diagonal system > with one block per process. Then solving it with block Jacobi is the > same as a direct solve. To expose more concurrency, you would need to > create a new PC. > >> On Mar 9, 2011 9:51 PM, "Matthew Knepley" <knepley(a)gmail.com >> <mailto:[email protected]>> wrote: >> >> Each process calls FieldSplitSetIS() with the indices for THAT FIELD >> that are owned by that process. Each >> process call FieldSplitSetIS() n+1 times if you have n+1 fields. >> Identifying fields with processes is a mistake. >> >> Matt >> >> >> >> On Wed, Mar 9, 2011 at 1:50 PM, Thomas Witkowski >> <Thomas.Witkowski(a)tu-dresden.de >> <mailto:[email protected]>> wrote: >> > >> > As I ... >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Jed Brown 13 Mar '11

13 Mar '11

On Mon, Mar 14, 2011 at 05:01, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > So it is okay to include headers from the source tree other than from > under $PETSC_DIR/include? > I thought the subject of this thread expressly banned that. > That is a style decision that is up to Barry and not something I was trying to address in this latest patch. Currently, most things uses <../src/path/to/localinclude.h> but there are a handful of exceptions. I actually prefer using the "localinclude.h" instead of <../src/path/to/localinclude.h> because the former is easier to relocate and recompile (perhaps with a prefix-install where $PETSC_DIR/src is no longer around). In other words, using "localinclude.h" makes stuff in src/pkg/impls/xxx/ appear more like a standalone plugin which I think is a good thing. This email thread is pretty old now, I think there was some misunderstanding of whether the compiler's $PWD affected header resolution (it doesn't) and recall some possible IDE issues but don't know if they were resolved.

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Dmitry Karpeev 13 Mar '11

13 Mar '11

So it is okay to include headers from the source tree other than from under $PETSC_DIR/include? I thought the subject of this thread expressly banned that. Dmitry. On Sun, Mar 13, 2011 at 10:38 PM, Jed Brown <jed(a)59a2.org> wrote: > On Tue, May 4, 2010 at 01:08, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >>> >>> The POSIX resolution semantics don't have anything to do with the >>> compiler's working directory ($PWD), it has only to do with the location >>> of the file. Using the full path makes sense if you want to relocate >>> *.c files *without* moving their accompanying headers (you'd have to >>> edit the file if you wanted to use a relocated header). In this case, >>> it would still be (more) correct to include the header with angle >>> brackets instead of quotes (because correctness requires that the header >>> is *not* found by following the path from the directory where the source >>> file resides, we're relying on the header being found only after falling >>> back to angle-bracket semantics). >> >> Perhaps we should change them to use <> > > I pushed this change. For consistency, if you want to include a file in the > same directory as the source file, use #include "header.h". If the file is > to be looked up in a system path or in some location specified by -I/path > (such as -I$PETSC_DIR/include), then use #include <header.h>. Speak up if > this has caused any problems.

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Jed Brown 13 Mar '11

13 Mar '11

On Tue, May 4, 2010 at 01:08, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > The POSIX resolution semantics don't have anything to do with the >> compiler's working directory ($PWD), it has only to do with the location >> of the file. Using the full path makes sense if you want to relocate >> *.c files *without* moving their accompanying headers (you'd have to >> edit the file if you wanted to use a relocated header). In this case, >> it would still be (more) correct to include the header with angle >> brackets instead of quotes (because correctness requires that the header >> is *not* found by following the path from the directory where the source >> file resides, we're relying on the header being found only after falling >> back to angle-bracket semantics). >> > > Perhaps we should change them to use <> I pushed this change. For consistency, if you want to include a file in the same directory as the source file, use #include "header.h". If the file is to be looked up in a system path or in some location specified by -I/path (such as -I$PETSC_DIR/include), then use #include <header.h>. Speak up if this has caused any problems.

1 0

Re: [petsc-dev] DM interface versus implementation headers
by Barry Smith 13 Mar '11

13 Mar '11

Concur. And fix that petscmg.h name at the same time :-) Barry On Mar 13, 2011, at 1:54 PM, Jed Brown wrote: > The solvers all have DM members now, but they only depend on the generic DM interface. Currently all of DMDA and now DMMesh are piled into petscdm.h which means that all the solvers need to be rebuilt any time these implementations change (e.g. a new implementation-specific function is added). The user already needs to decide which DM to use (DMDA, DMMesh, etc) so it would be no hardship to include petscdmda.h instead of just petscdm.h. For user-defined implementations of DM, this is already necessary. > > In order to loosen the dependency of the solvers on DM, I propose distilling petscdm.h to be only the core DM functionality (roughly what is in _DMOps) and creating petscdmda.h and petscdmmesh.h for the DMDA* and DMMesh* functions. Note that there is already some precedence for this in petscmg.h (which perhaps should be named petscpcmg.h), though I think the case for separation is much stronger for DM. > > If this change is going to happen, it should be before the release.

1 0

DM interface versus implementation headers
by Jed Brown 13 Mar '11

13 Mar '11

The solvers all have DM members now, but they only depend on the generic DM interface. Currently all of DMDA and now DMMesh are piled into petscdm.h which means that all the solvers need to be rebuilt any time these implementations change (e.g. a new implementation-specific function is added). The user already needs to decide which DM to use (DMDA, DMMesh, etc) so it would be no hardship to include petscdmda.h instead of just petscdm.h. For user-defined implementations of DM, this is already necessary. In order to loosen the dependency of the solvers on DM, I propose distilling petscdm.h to be only the core DM functionality (roughly what is in _DMOps) and creating petscdmda.h and petscdmmesh.h for the DMDA* and DMMesh* functions. Note that there is already some precedence for this in petscmg.h (which perhaps should be named petscpcmg.h), though I think the case for separation is much stronger for DM. If this change is going to happen, it should be before the release.

1 0

Re: [petsc-dev] broken petsc-dev related mesh?
by Jed Brown 12 Mar '11

12 Mar '11

On Sat, Mar 12, 2011 at 21:08, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > From the docs: if INSERT_VALUES then no parallel communication is used, > Evidently I can't read. > > My hope was that all reasonable possible approaches to localtoglobal > could be handle via a single DMLocalToGlobalBegin/End() interface. I admit > it may be consider slightly cumbersome for some cases but I considered those > cases rare enough that in the interest of not having a million different > functions to do similar things with local to global I could live with the > cumbersomeness. > No, this is fine. Using gtol with (INSERT_VALUES,SCATTER_REVERSE) never made sense anyway.

1 0

Re: [petsc-dev] broken petsc-dev related mesh?
by Barry Smith 12 Mar '11

12 Mar '11

On Mar 12, 2011, at 2:06 PM, Jed Brown wrote: > On Sat, Mar 12, 2011 at 21:02, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > My suggested triage is to improve the generation of various "tags" files so that we can perform the various searches via TAGS or Eclipse etc with various search parameters. > > Your suggested triage is to put the generated .html files "somewhere else" (and hence at the same time change other paradigms of PETSc) just so they don't get in the way of simple minded search techniques. > > By "somewhere else", I just mean to put them into LOC where all the other generated documentation goes. Why is putting them into LOC inconsistent with the rest of PETSc? See my just sent email. I was wrong and you right, I did not realize the inconsistency. > > > I think my triage is more forward looking and less disruptive. > > It's also a huge job. I think it can be done incrementally and don't see why it is a huge job. Barry

1 0