February 2011 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] configure issues
by Jed Brown 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 12:56, Matthew Knepley <knepley(a)gmail.com> wrote: > We used to have everything in the preprocessor, and we could happily edit > that and never have to think > about Python at all. We abandoned that approach because it was too > complicated, prone to break, and > we spent all of our petsc-maint time on it. > I don't know what examples you are referring to, I started with PETSc in 2004. I don't see a lot of logic currently being done on the Python side and used from C, petscconf.h does not have logic in it. No C source files or function bodies are currently generated by Python. It sounds like you want to generated imperative code in Python and call that from C. I think that would add complexity.

1 0

Re: [petsc-dev] configure issues
by Matthew Knepley 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 5:49 AM, Jed Brown <jed(a)59a2.org> wrote: > On Wed, Feb 9, 2011 at 12:33, Matthew Knepley <knepley(a)gmail.com> wrote: > >> If you refer to the first part of your comment above, it suggested >> generating the correct code, > > > So every function in PETSc that currently does > > #if something > ... > #else > ... > #endif > > would be replaced by generated headers and source files that have only the > chosen path. I think this will make it far more confusing to find out what > paths are possible, to make edits safely, and to locate definitions. I do > not know of any successful project that generates code for something like > this. > > Barry was not suggesting any new use of the preprocessor, just renaming > existing macros to make navigation easier. I think it's fine to keep the > existing names and instead write a snippet of elisp to make navigation > easier. Your response that all use of the preprocessor should be replaced by > generated code (or something) does not address the problem at all, and, if > anything, makes it worse because configure would become more complicated. > > For what it's worth, despite Python being "better" in all the ways you > mention, I don't know anybody who finds it easier/safer/faster to edit > BuildSystem code than to edit PETSc C source. This is less a statement about > the language and more a statement about organization, but I would be very > cautious about moving large amounts of complexity from src/*.c to > config/*.py. > We used to have everything in the preprocessor, and we could happily edit that and never have to think about Python at all. We abandoned that approach because it was too complicated, prone to break, and we spent all of our petsc-maint time on it. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] configure issues
by Jed Brown 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 12:33, Matthew Knepley <knepley(a)gmail.com> wrote: > If you refer to the first part of your comment above, it suggested > generating the correct code, So every function in PETSc that currently does #if something ... #else ... #endif would be replaced by generated headers and source files that have only the chosen path. I think this will make it far more confusing to find out what paths are possible, to make edits safely, and to locate definitions. I do not know of any successful project that generates code for something like this. Barry was not suggesting any new use of the preprocessor, just renaming existing macros to make navigation easier. I think it's fine to keep the existing names and instead write a snippet of elisp to make navigation easier. Your response that all use of the preprocessor should be replaced by generated code (or something) does not address the problem at all, and, if anything, makes it worse because configure would become more complicated. For what it's worth, despite Python being "better" in all the ways you mention, I don't know anybody who finds it easier/safer/faster to edit BuildSystem code than to edit PETSc C source. This is less a statement about the language and more a statement about organization, but I would be very cautious about moving large amounts of complexity from src/*.c to config/*.py.

1 0

Re: [petsc-dev] configure issues
by Matthew Knepley 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 4:45 AM, Jed Brown <jed(a)59a2.org> wrote: > On Wed, Feb 9, 2011 at 11:33, Matthew Knepley <knepley(a)gmail.com> wrote: > >> A single system, with real scoping, easy debugging, more readable code, >> etc. > > > Would you like a pony? What does this even mean? There is logic for a PETSc > program to attach a debugger to itself. The details of that logic depends on > tests that are run by configure. PETSc does not presently require Python at > runtime. If you were to refactor it, where would the logic go and how would > it be called from C? > If you refer to the first part of your comment above, it suggested generating the correct code, and then questioned the utility of doing so: What does this even mean? single system: Here I mean using just Python, rather than Python coupled with the awfulness that is the C preprocessor real scoping: Python has nice scopes, rather than the underscore hell of preprocessor names easy debugging: I can easily see the code I generated and debug it. This is much more complicated with preprocessor logic readable code: See above I cannot think of an argument for using the preprocessor. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] configure issues
by Jed Brown 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 11:33, Matthew Knepley <knepley(a)gmail.com> wrote: > A single system, with real scoping, easy debugging, more readable code, > etc. Would you like a pony? What does this even mean? There is logic for a PETSc program to attach a debugger to itself. The details of that logic depends on tests that are run by configure. PETSc does not presently require Python at runtime. If you were to refactor it, where would the logic go and how would it be called from C?

1 0

Re: [petsc-dev] configure issues
by Matthew Knepley 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 3:48 AM, Jed Brown <jed(a)59a2.org> wrote: > On Wed, Feb 9, 2011 at 10:26, Matthew Knepley <knepley(a)gmail.com> wrote: > >> 1) This is about fooling with the preprocessor, or I would not need any of >> these goofy names. How can you deny this? >> > > The preprocessor doesn't care what the name of the macro is. Barry's > suggestion was just a way to make it easy for a human to find out where the > macro was set, but at the expense of making the names fragile. I would > rather have the editor be able to navigate using more stable names. > > >> >> 2) For test information as well as control flow, I want to stay in Python. >> > > How would you manage, say, the logic around PETSC_USE_X_FOR_DEBUGGER? > Generate the appropriate snippets of code on the Python side? What advantage > to you see to that? > A single system, with real scoping, easy debugging, more readable code, etc. Matt > > >> >> 3) What do you mean by "link"? >> > > 1. From the name of a macro that may be defined in petscconf.h to the test > in configure which is used to decide how to set that macro. > 2. From the test in configure to a list of uses on the C side (in a less > brute-force way than grep). > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] configure issues
by Jed Brown 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 10:26, Matthew Knepley <knepley(a)gmail.com> wrote: > 1) This is about fooling with the preprocessor, or I would not need any of > these goofy names. How can you deny this? > The preprocessor doesn't care what the name of the macro is. Barry's suggestion was just a way to make it easy for a human to find out where the macro was set, but at the expense of making the names fragile. I would rather have the editor be able to navigate using more stable names. > > 2) For test information as well as control flow, I want to stay in Python. > How would you manage, say, the logic around PETSC_USE_X_FOR_DEBUGGER? Generate the appropriate snippets of code on the Python side? What advantage to you see to that? > > 3) What do you mean by "link"? > 1. From the name of a macro that may be defined in petscconf.h to the test in configure which is used to decide how to set that macro. 2. From the test in configure to a list of uses on the C side (in a less brute-force way than grep).

1 0

Re: [petsc-dev] configure issues
by Matthew Knepley 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 3:13 AM, Jed Brown <jed(a)59a2.org> wrote: > On Wed, Feb 9, 2011 at 08:04, Matthew Knepley <knepley(a)gmail.com> wrote: > >> I can't believe you guys want further screwing around with the C >> preprocessor. I want all that crap completely >> out of the system. We are very close to that. >> > > If you think we are very close to removing all functional tests from > configure (making petscconf.h contain only options set directly by the user, > not the result of any tests in configure), then you are delusional. Barry > just wants a clean way to jump from macro name to the test that defines that > macro and back. I agree that manually following the "link" is sometimes a > confusing process as evidenced by the fact that I typically use recursive > grep for it, but I would rather make the macro definition process > sufficiently well structured than to write potentially fragile paths into > the variable itself. This is about naming and links, not about "screwing > around with the C preprocessor". > 1) This is about fooling with the preprocessor, or I would not need any of these goofy names. How can you deny this? 2) For test information as well as control flow, I want to stay in Python. 3) What do you mean by "link"? Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] configure issues
by Jed Brown 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 08:04, Matthew Knepley <knepley(a)gmail.com> wrote: > I can't believe you guys want further screwing around with the C > preprocessor. I want all that crap completely > out of the system. We are very close to that. > If you think we are very close to removing all functional tests from configure (making petscconf.h contain only options set directly by the user, not the result of any tests in configure), then you are delusional. Barry just wants a clean way to jump from macro name to the test that defines that macro and back. I agree that manually following the "link" is sometimes a confusing process as evidenced by the fact that I typically use recursive grep for it, but I would rather make the macro definition process sufficiently well structured than to write potentially fragile paths into the variable itself. This is about naming and links, not about "screwing around with the C preprocessor".

1 0

Re: [petsc-dev] configure issues
by Matthew Knepley 09 Feb '11

09 Feb '11

I can't believe you guys want further screwing around with the C preprocessor. I want all that crap completely out of the system. We are very close to that. Matt On Sun, Feb 6, 2011 at 1:00 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Feb 6, 2011, at 12:16 AM, Jed Brown wrote: > > > Isn't this just making up for the editor (and perhaps static analysis > challenges with Python)? As long as the definition is explicit, grep -r > HAVE_FOO config/ should locate the definition. For the other direction, you > can grep the source tree (fast if it's in memory or indexed). So how about a > couple Emacs functions for this instead of writing fragile file system paths > into the configuration macros? > > Getting them into etags would be good. > > > > > >> On Feb 4, 2011 1:03 AM, "Barry Smith" <bsmith(a)mcs.anl.gov> wrote: > >> > >> > >> On Feb 3, 2011, at 1:35 PM, Jed Brown wrote: > >> > >> > On Thu, Feb 3, 2011 at 16:27, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > >> > > >> > > >> >> Then you are not going to like my next idea. I think that > "configure" > >> >> tests should be imbedded directly into the source code and evaluated > at > >> >> compile time to determine what gets built etc. In other words no > distinction > >> >> between configure and build time :-). > >> >> > >> > > >> > How to handle tests that need to actually be executed, in a batch > >> > environment? > >> > > >> > > >> >> This also eliminates the possibility of "incorrect semantic > information in > >> >> comments" since the comments won't be "requires X Y and Z" but > instead the > >> >> actual tests. Having the test code for a functionality being in a > different > >> >> place than the use of a functionality is not a good model (the > obvious > >> >> problem with what I say is that the use might be in 50 places so > should I > >> >> have the same test in 50 places?) > >> >> > >> > > >> > And should the test be executed in all 50 places, or should the result > be > >> > cached? Since C code is context dependent, it would be hard to prove > that it > >> > was actually the same test in all 50 places, even if it resided in a > common > >> > header. > >> > > >> > >> Yes directly imbedding the tests is probably not practical but how > about a naming convention that points directly to the test from the source. > Now sometimes one has to hunt around to find the test associated with a > particular #if since they could be in some many locations. For example > instead of > >> > >> #if defined(PETSC_HAVE_FOO) where is the test of foo > >> > >> have > >> > >> #if > defined(PETSC_HAVE_FOO_BUILDSYSTEM_CONFIG_MATTSCRAZYFILE_MATTSCRAZYMETHODINTHATFILE) > >> > >> now I can always go directly to the test. Ideally the links would be > bi-directional but that is hard. > >> > >> Barry > >> > >> > >> > >> > On Thu, Feb 3, 2011 at 16:27, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > >> > Configure is only slow because it has grown into this monster over > time; a complete refactoring should bring it down to barely a minute or so. > >> > > >> > Even a minute is a long time compared to 3 seconds which is what an > incremental build with a couple new/modified implementation files should > take. > >> > > >> > But configure needs to run a lot of tests which involves invoking the > compiler a lot of times (it needs to run the compiler on tiny test programs > to generate understandable errors as soon as possible, squashing it together > into a big compilation later is bad because errors become confusing). I'm > not sure that <1 minute is achievable, but I hope it is. > >> > > >> > Then you are not going to like my next idea. I think that > "configure" tests should be imbedded directly into the source code and > evaluated at compile time to determine what gets built etc. In other words > no distinction between configure and build time :-). > >> > > >> > How to handle tests that need to actually be executed, in a batch > environment? > >> > > >> > This also eliminates the possibility of "incorrect semantic > information in comments" since the comments won't be "requires X Y and Z" > but instead the actual tests. Having the test code for a functionality being > in a different place than the use of a functionality is not a good model > (the obvious problem with what I say is that the use might be in 50 places > so should I have the same test in 50 places?) > >> > > >> > And should the test be executed in all 50 places, or should the result > be cached? Since C code is context dependent, it would be hard to prove that > it was actually the same test in all 50 places, even if it resided in a > common header. > >> > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0