March 2011 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Barry Smith 14 Mar '11

14 Mar '11

On Mar 14, 2011, at 11:03 AM, Matthew Knepley wrote: > On Mon, Mar 14, 2011 at 11:00 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Mar 14, 2011, at 10:33 AM, Jed Brown wrote: > > > I had resisted doing this but maybe it is the way to go. > > I had resisted suggesting this, but if we are going towards all headers separate, I would suggest an alternative. > > 1) Our source, in the source tree, refer to private headers using relative paths or -I > > 2) Install copies all headers under include/private/<sys,ksp,etc>/* > > 3) Outside modules refer to the headers by full path 1) and 3) are f-ing perverse (I can see why you resisted, you should have continued to resist). ANYTHING that makes externally developed code look "different" then code in the PETSc tree is truly horrible!!! I remember this being a theme of some slide we presented like 16 years ago. What is so wrong with having all the includes in the include directory (properly organized)? Barry > > Then 3) might break if we move things, but they are writing to private interfaces anyway. > > Matt > > > Barry > > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Jed Brown 14 Mar '11

14 Mar '11

On Mon, Mar 14, 2011 at 17:00, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > Then basically almost all the headers belong in include/something because > they can and sometimes are used by outsiders. For example access to the guts > of any matrix class for implementation in heritence, access to the guts of > the gmres data-structure to implement a modified gmres like fgmres, lgmres > etc. pc_mg for sure > > So how do we organize private? Same as the source tree? > mat/impls/aij/seq etc etc? dump everything in one directory yuck? > I see a total of 70 implementation headers, not sure what fraction of those should be "open". Mirroring the source tree makes it easy to find the location of the "primary" user (*.c files) of a given header, but it also forces that there always be a primary user. Perhaps it's reasonable for things with no primary user to go directly into private/*.h instead of private/pkg/**/*.h. I don't have a strong preference either way. > I had resisted doing this but maybe it is the way to go. > I think it's the right thing if plugin developers/users and prefix installs are to be first-class citizens.

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Dmitry Karpeev 14 Mar '11

14 Mar '11

On Mon, Mar 14, 2011 at 11:00 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Mar 14, 2011, at 10:33 AM, Jed Brown wrote: > >> On Mon, Mar 14, 2011 at 16:22, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >> What bothers me about this one is that private has no hierarchy and will eventually become cluttered with all kinds of stuff from many different places in PETSc. >> >> How many shared headers are there really? >> >> >> The reason private was introduced (rather than having all of the private includes directly in the source tree) was sometimes external tools (Lisandros python stuff?) needed access to the files with a --prefix install so it had to go in the include directory. I really like having it in the source (I could give a shit about parts of PETSc source being built independently from the source tree). >> >> What about friendliness towards people writing plugins for PETSc. For example, creating a matrix format that is perhaps licensed differently from PETSc and thus not distributable with PETSc itself. So this new format would be compiled somewhere and the shared library dropped in a directory that PETSc inspects in PetscInitialize so it looks to the user like a first-class implementation (and the application need not even be recompiled). Of course this only works if the plugin can be compiled in an environment where it has access to the headers it needs. Those might not be present in a prefix-install. >> >> It would be a shame to distribute your plugin, but not be compilable with the vendor-provided install of PETSc that used prefix like most other libraries. >> >> Maybe the private includes should be fixed up to contain (1) what is truly needed in the --prefix directory hence must in the include tree and what (2) belongs somewhere in the source tree. >> >> In my opinion, it is not a matter of whether Lisandro needs them, but rather a decision of which types are "open" in the sense that implementation inheritance is intended versus "closed" in the sense that it is not. This also affects using "static" for implementation functions. > > Then basically almost all the headers belong in include/something because they can and sometimes are used by outsiders. For example access to the guts of any matrix class for implementation in heritence, access to the guts of the gmres data-structure to implement a modified gmres like fgmres, lgmres etc. pc_mg for sure > > So how do we organize private? Same as the source tree? mat/impls/aij/seq etc etc? dump everything in one directory yuck? BTW: if almost all headers go in the header tree then I would argue it becomes a much simplier model if we require ALL headers there. This can be automated in various ways. An install (of any kind) could dump the headers from every descendant of src/ into the corresponding include/private/ subdir, or maybe even include/src/. Then just include it as <mat/impls/aij/seq/aij.h>? Maybe the headers can even go into $PETSC_DIR/$PETSC_ARCH/include/... for non-prefix installs, although that would mean replicating a lot of headers, but disk space is cheap. Dmitry. > > > I had resisted doing this but maybe it is the way to go. > > Barry > > > > >

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Matthew Knepley 14 Mar '11

14 Mar '11

On Mon, Mar 14, 2011 at 11:00 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Mar 14, 2011, at 10:33 AM, Jed Brown wrote: > > > On Mon, Mar 14, 2011 at 16:22, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > What bothers me about this one is that private has no hierarchy and will > eventually become cluttered with all kinds of stuff from many different > places in PETSc. > > > > How many shared headers are there really? > > > > > > The reason private was introduced (rather than having all of the private > includes directly in the source tree) was sometimes external tools > (Lisandros python stuff?) needed access to the files with a --prefix install > so it had to go in the include directory. I really like having it in the > source (I could give a shit about parts of PETSc source being built > independently from the source tree). > > > > What about friendliness towards people writing plugins for PETSc. For > example, creating a matrix format that is perhaps licensed differently from > PETSc and thus not distributable with PETSc itself. So this new format would > be compiled somewhere and the shared library dropped in a directory that > PETSc inspects in PetscInitialize so it looks to the user like a first-class > implementation (and the application need not even be recompiled). Of course > this only works if the plugin can be compiled in an environment where it has > access to the headers it needs. Those might not be present in a > prefix-install. > > > > It would be a shame to distribute your plugin, but not be compilable with > the vendor-provided install of PETSc that used prefix like most other > libraries. > > > > Maybe the private includes should be fixed up to contain (1) what is > truly needed in the --prefix directory hence must in the include tree and > what (2) belongs somewhere in the source tree. > > > > In my opinion, it is not a matter of whether Lisandro needs them, but > rather a decision of which types are "open" in the sense that implementation > inheritance is intended versus "closed" in the sense that it is not. This > also affects using "static" for implementation functions. > > Then basically almost all the headers belong in include/something > because they can and sometimes are used by outsiders. For example access to > the guts of any matrix class for implementation in heritence, access to the > guts of the gmres data-structure to implement a modified gmres like fgmres, > lgmres etc. pc_mg for sure > > So how do we organize private? Same as the source tree? > mat/impls/aij/seq etc etc? dump everything in one directory yuck? BTW: > if almost all headers go in the header tree then I would argue it becomes a > much simplier model if we require ALL headers there. > > > I had resisted doing this but maybe it is the way to go. I had resisted suggesting this, but if we are going towards all headers separate, I would suggest an alternative. 1) Our source, in the source tree, refer to private headers using relative paths or -I 2) Install copies all headers under include/private/<sys,ksp,etc>/* 3) Outside modules refer to the headers by full path Then 3) might break if we move things, but they are writing to private interfaces anyway. Matt > > Barry > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Barry Smith 14 Mar '11

14 Mar '11

On Mar 14, 2011, at 10:33 AM, Jed Brown wrote: > On Mon, Mar 14, 2011 at 16:22, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > What bothers me about this one is that private has no hierarchy and will eventually become cluttered with all kinds of stuff from many different places in PETSc. > > How many shared headers are there really? > > > The reason private was introduced (rather than having all of the private includes directly in the source tree) was sometimes external tools (Lisandros python stuff?) needed access to the files with a --prefix install so it had to go in the include directory. I really like having it in the source (I could give a shit about parts of PETSc source being built independently from the source tree). > > What about friendliness towards people writing plugins for PETSc. For example, creating a matrix format that is perhaps licensed differently from PETSc and thus not distributable with PETSc itself. So this new format would be compiled somewhere and the shared library dropped in a directory that PETSc inspects in PetscInitialize so it looks to the user like a first-class implementation (and the application need not even be recompiled). Of course this only works if the plugin can be compiled in an environment where it has access to the headers it needs. Those might not be present in a prefix-install. > > It would be a shame to distribute your plugin, but not be compilable with the vendor-provided install of PETSc that used prefix like most other libraries. > > Maybe the private includes should be fixed up to contain (1) what is truly needed in the --prefix directory hence must in the include tree and what (2) belongs somewhere in the source tree. > > In my opinion, it is not a matter of whether Lisandro needs them, but rather a decision of which types are "open" in the sense that implementation inheritance is intended versus "closed" in the sense that it is not. This also affects using "static" for implementation functions. Then basically almost all the headers belong in include/something because they can and sometimes are used by outsiders. For example access to the guts of any matrix class for implementation in heritence, access to the guts of the gmres data-structure to implement a modified gmres like fgmres, lgmres etc. pc_mg for sure So how do we organize private? Same as the source tree? mat/impls/aij/seq etc etc? dump everything in one directory yuck? BTW: if almost all headers go in the header tree then I would argue it becomes a much simplier model if we require ALL headers there. I had resisted doing this but maybe it is the way to go. Barry

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Jed Brown 14 Mar '11

14 Mar '11

On Mon, Mar 14, 2011 at 16:47, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > "open" headers can be included, even if they are in the src/ tree, > Only if the src/ tree is present which means that plugins can only be built with non-prefix-installed PETSc. That would be unfortunate if, say, you wanted your plugin user to be able to build the plugin against a vendor-provided PETSc which of course wouldn't come with the src/ tree if they had modified anything.

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Dmitry Karpeev 14 Mar '11

14 Mar '11

On Mon, Mar 14, 2011 at 10:33 AM, Jed Brown <jed(a)59a2.org> wrote: > On Mon, Mar 14, 2011 at 16:22, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >> >> What bothers me about this one is that private has no hierarchy and will >> eventually become cluttered with all kinds of stuff from many different >> places in PETSc. > > How many shared headers are there really? > >> >> The reason private was introduced (rather than having all of the private >> includes directly in the source tree) was sometimes external tools >> (Lisandros python stuff?) needed access to the files with a --prefix install >> so it had to go in the include directory. I really like having it in the >> source (I could give a shit about parts of PETSc source being built >> independently from the source tree). > > What about friendliness towards people writing plugins for PETSc. For > example, creating a matrix format that is perhaps licensed differently from > PETSc and thus not distributable with PETSc itself. So this new format would > be compiled somewhere and the shared library dropped in a directory that > PETSc inspects in PetscInitialize so it looks to the user like a first-class > implementation (and the application need not even be recompiled). Of course > this only works if the plugin can be compiled in an environment where it has > access to the headers it needs. Those might not be present in a > prefix-install. > It would be a shame to distribute your plugin, but not be compilable with > the vendor-provided install of PETSc that used prefix like most other > libraries. > >> >> Maybe the private includes should be fixed up to contain (1) what is >> truly needed in the --prefix directory hence must in the include tree and >> what (2) belongs somewhere in the source tree. > > In my opinion, it is not a matter of whether Lisandro needs them, but rather > a decision of which types are "open" in the sense that implementation > inheritance is intended versus "closed" in the sense that it is not. This > also affects using "static" for implementation functions. "open" headers can be included, even if they are in the src/ tree, though. That might make more sense, because then the path to the header indicates what's being inherited from. Sticking them in include/private requires a disambiguating name. Dmitry.

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Dmitry Karpeev 14 Mar '11

14 Mar '11

On Mon, Mar 14, 2011 at 10:22 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Mar 14, 2011, at 8:44 AM, Jed Brown wrote: > >> On Mon, Mar 14, 2011 at 14:08, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >> For files like aij.h if you use the localinclude.h paradigm then in aij.c you would have #include "aij.h" but in mpiaij.h you would have #include <./src/mat/impls/aij/seq/aij.h> I don't like the fact that in different files the sameinclude is referenced in a different way. Now one could argue that private includes used ONLY locally use the "localinclude.h" model and private includes used more than locally use the <../src/path/to/localinclude.h> but why? >> >> So that they can be built independently from the source tree. >> >> Better the simplicity of always using the <../src/path/to/localinclude.h> model. >> >> There are at least four approaches >> >> 1. #include "../seq/aij.h" > > yuck > >> >> 2. #include <../src/mat/impls/aij/seq/aij.h> >> >> 3. Have the build system add -I$PETSC_DIR/src/mat/impls/aij/seq and use #include <aij.h> > > yuck > >> >> 4. Move this not-truly-private header to include/private/matseqaij.h and #include <private/matseqaij.h>. > > What bothers me about this one is that private has no hierarchy and will eventually become cluttered with all kinds of stuff from many different places in PETSc. > > The reason private was introduced (rather than having all of the private includes directly in the source tree) was sometimes external tools (Lisandros python stuff?) needed access to the files with a --prefix install so it had to go in the include directory. I really like having it in the source (I could give a shit about parts of PETSc source being built independently from the source tree). Maybe the private includes should be fixed up to contain (1) what is truly needed in the --prefix directory hence must in the include tree and what (2) belongs somewhere in the source tree. So, my understanding is that the headers in the include/ tree is only what's needed by the applications (those need include/) and external tools (additionally need the private headers include/private/). The headers in the src/ tree are for the interoperability between different impls. These src/ tree private headers can be addressed either relative to include/ (via ../src/mat/impls/aij/seq/aij.h) or relative to the including file (../seq/aih.h), and that choice is up to the particular developer, although Barry strongly prefers the former? Dmitry. > > > Barry > >> >> >> 1 permits relocating part of the source tree and compiling (a variant of) a single module without changing includes. 2 requires modifying all derived implementations if the header is relocated and only works if the full source tree is available. 3 is the same, but the change is only to the build system and the build system is more easily "aware" of this cross-dependency. 4 is a direction that we have already been going over the past couple years (some implementation headers moved from src/dm/ to include/private) because users wanted access to some internals from a prefix-install (perhaps to create their own derived implementation). >> >> Is there a good reason not to use 4 for shared headers and #include "mine.h" for unshared (and not intended to be derived from)? > >

1 0

Re: [petsc-dev] reminder never use #include "mylocalinclude.h" in PETSc source
by Jed Brown 14 Mar '11

14 Mar '11

On Mon, Mar 14, 2011 at 16:22, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > What bothers me about this one is that private has no hierarchy and will > eventually become cluttered with all kinds of stuff from many different > places in PETSc. > How many shared headers are there really? > > The reason private was introduced (rather than having all of the private > includes directly in the source tree) was sometimes external tools > (Lisandros python stuff?) needed access to the files with a --prefix install > so it had to go in the include directory. I really like having it in the > source (I could give a shit about parts of PETSc source being built > independently from the source tree). > What about friendliness towards people writing plugins for PETSc. For example, creating a matrix format that is perhaps licensed differently from PETSc and thus not distributable with PETSc itself. So this new format would be compiled somewhere and the shared library dropped in a directory that PETSc inspects in PetscInitialize so it looks to the user like a first-class implementation (and the application need not even be recompiled). Of course this only works if the plugin can be compiled in an environment where it has access to the headers it needs. Those might not be present in a prefix-install. It would be a shame to distribute your plugin, but not be compilable with the vendor-provided install of PETSc that used prefix like most other libraries. > Maybe the private includes should be fixed up to contain (1) what is truly > needed in the --prefix directory hence must in the include tree and what (2) > belongs somewhere in the source tree. > In my opinion, it is not a matter of whether Lisandro needs them, but rather a decision of which types are "open" in the sense that implementation inheritance is intended versus "closed" in the sense that it is not. This also affects using "static" for implementation functions.

1 0

Re: [petsc-dev] DMDABoundaryType
by Barry Smith 14 Mar '11

14 Mar '11

esc x tags-search enter the string to search for and then use esc , to find the next use. BUT etags is only as good as its database and sometimes the generation of the etags file doesn't traverse everywhere it should. Barry On Mar 14, 2011, at 9:43 AM, Ethan Coon wrote: > Ok, apparently etags does more than I thought it did... off to RTFM. > > Thanks, > > Ethan > > On Mon, 2011-03-14 at 15:39 +0100, Jed Brown wrote: >> On Mon, Mar 14, 2011 at 15:34, Ethan Coon <ecoon(a)lanl.gov> wrote: >> Related to this, do you guys have a standard way of finding >> all such >> instances? I was using grep/find (and then checking the >> compilation >> output), but I'm sure my find was missing a few files. >> >> Depends on your editor, but there are "tags" for Emacs and Vim >> (slightly different in each case, but basically the same). Most IDEs >> have some sort of symbol-reference lookup functionality built in. You >> can also use GNU Global which retains a bit more information than the >> usual TAGS. I often end up using recursive grep to locate files with a >> certain property and then edit them using sed or perl -pi -e (for >> mechanical things). In addition to the tags searches, you can have >> Emacs visit every location that matches a grep query. > > -- > ------------------------------------ > Ethan Coon > Post-Doctoral Researcher > Applied Mathematics - T-5 > Los Alamos National Laboratory > 505-665-8289 > > http://www.ldeo.columbia.edu/~ecoon/ > ------------------------------------ >

1 0