February 2011 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Requirements
by Matthew Knepley 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 1:21 PM, Jose E. Roman <jroman(a)dsic.upv.es> wrote: > > El 09/02/2011, a las 15:02, Matthew Knepley escribió: > > > I have changed the stuff that did not belong in #requirepackage to use > either #requirefunction > > or #requiredefine. No behavior should change. > > > > Matt > > In generatefortranstubs.py there is also a requirespackage statement. Should > it be changed as well? > Thanks, I missed that one. Pushed. Matt > Jose > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Requirements
by Jose E. Roman 09 Feb '11

09 Feb '11

El 09/02/2011, a las 15:02, Matthew Knepley escribió: > I have changed the stuff that did not belong in #requirepackage to use either #requirefunction > or #requiredefine. No behavior should change. > > Matt In generatefortranstubs.py there is also a requirespackage statement. Should it be changed as well? Jose

1 0

Re: [petsc-dev] configure issues
by Jed Brown 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 14:40, Matthew Knepley <knepley(a)gmail.com> wrote: > Name one that _only_ involves dropping files from the build. Hint: don't >> forget about XXRegisterAll(). >> > > Anything that uses #requirespackage. There are at least 50. I went through > all of them today since I was refactoring builder.py. > Evidently you did not read what I wrote. All of those require registering an implementation, the macro is used in some XXRegisterAll() to determine whether it can be registered. Yes, this is only two places (only one in C source), but many of them are used in a few more places or in examples. > > >> >>> For other #defines, like those that get memory usage from the operating >>> system, I definitely think code generation would be simpler >>> than the #if monstrosity that currently exists. >>> >> >> That is basically just one function, the fp_trap stuff is more complex, >> but entirely maintainable in my opinion. I think PetscMemoryGetCurrentUsage >> is a confusing function only because there was some attempt to only write >> "PetscFunctionBegin" once instead of just making 5 short-but-complete >> snippets for each alternative (i.e. duplicate the mundane stuff into all 5 >> bodies instead of duplicating the conditionals to avoid duplicating the >> mundane stuff). >> >> >>> It would divide debugging into two parts: >>> >>> a) Why doesn't the generated code work? We have made mistakes in just >>> the code I mentioned because we could not follow the inclusions correctly. >>> >>> b) Why did we choose to generate that code? This would be entirely >>> contained in a single configure test. >>> >> >> Recall that BuildSystem _still_ does not have a way to write a code >> snippet once and have it run immediately or in batch (there is currently >> potential for inconsistency even if I just want to do one test). >> Occasionally there is a combinatorial number of paths through the code >> (because features are, at least in principle, independent). If you generate >> it, you have to manage the results of multiple orthogonal tests (where >> should the logic to handle this go), and be confident that all code paths >> are correct. Sometimes the result of a test is used in many places, >> answering the question "where is this snippet of code used and why does it >> look this way" becomes difficult if there are a dozen snippets for a given >> test. Generating the snippets is a bit like writing a function >> FrobnicateTheVariablesInScopeOnLine192(); the snippet may not make sense in >> it's own right, just in the context where it is needed. This is an >> unconventional way to organize code and not going to reduce complexity >> (especially to someone new to the project). >> > > I really cannot understand what you are worried about here. We do this all > the time. > > >> I would rather write structured code that uses the results of these tests >> (i.e. in a C file where it is now) than to generate said code somewhere in >> config/*. >> > > I would not agree. I am one of those sketchy fringe characters who thinks it's easier to debug code in the same language it was written in.

1 0

Re: [petsc-dev] configure issues
by Dmitry Karpeev 09 Feb '11

09 Feb '11

This is all very confusing to me, but here is my simple question: if cpp directives were to be abandoned in favor of code generation, what would I do instead of #if defined (PETSC_USE_DEBUG) <stuff> #endif ? On Wed, Feb 9, 2011 at 7:40 AM, Matthew Knepley <knepley(a)gmail.com> wrote: > On Wed, Feb 9, 2011 at 7:32 AM, Jed Brown <jed(a)59a2.org> wrote: > >> On Wed, Feb 9, 2011 at 13:42, Matthew Knepley <knepley(a)gmail.com> wrote: >> >>> I did not mean at runtime. I meant that a great portion of our defines >>> now just remove an entire file from the build. We do not >>> need #defines for that. >>> >> >> Name one that _only_ involves dropping files from the build. Hint: don't >> forget about XXRegisterAll(). >> > > Anything that uses #requirespackage. There are at least 50. I went through > all of them today since I was refactoring builder.py. > > >> >>> For other #defines, like those that get memory usage from the operating >>> system, I definitely think code generation would be simpler >>> than the #if monstrosity that currently exists. >>> >> >> That is basically just one function, the fp_trap stuff is more complex, >> but entirely maintainable in my opinion. I think PetscMemoryGetCurrentUsage >> is a confusing function only because there was some attempt to only write >> "PetscFunctionBegin" once instead of just making 5 short-but-complete >> snippets for each alternative (i.e. duplicate the mundane stuff into all 5 >> bodies instead of duplicating the conditionals to avoid duplicating the >> mundane stuff). >> >> >>> It would divide debugging into two parts: >>> >>> a) Why doesn't the generated code work? We have made mistakes in just >>> the code I mentioned because we could not follow the inclusions correctly. >>> >>> b) Why did we choose to generate that code? This would be entirely >>> contained in a single configure test. >>> >> >> Recall that BuildSystem _still_ does not have a way to write a code >> snippet once and have it run immediately or in batch (there is currently >> potential for inconsistency even if I just want to do one test). >> Occasionally there is a combinatorial number of paths through the code >> (because features are, at least in principle, independent). If you generate >> it, you have to manage the results of multiple orthogonal tests (where >> should the logic to handle this go), and be confident that all code paths >> are correct. Sometimes the result of a test is used in many places, >> answering the question "where is this snippet of code used and why does it >> look this way" becomes difficult if there are a dozen snippets for a given >> test. Generating the snippets is a bit like writing a function >> FrobnicateTheVariablesInScopeOnLine192(); the snippet may not make sense in >> it's own right, just in the context where it is needed. This is an >> unconventional way to organize code and not going to reduce complexity >> (especially to someone new to the project). >> > > I really cannot understand what you are worried about here. We do this all > the time. > > >> I would rather write structured code that uses the results of these tests >> (i.e. in a C file where it is now) than to generate said code somewhere in >> config/*. >> > > I would not agree. > > Matt > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener >

1 0

Requirements
by Matthew Knepley 09 Feb '11

09 Feb '11

I have changed the stuff that did not belong in #requirepackage to use either #requirefunction or #requiredefine. No behavior should change. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] configure issues
by Matthew Knepley 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 7:32 AM, Jed Brown <jed(a)59a2.org> wrote: > On Wed, Feb 9, 2011 at 13:42, Matthew Knepley <knepley(a)gmail.com> wrote: > >> I did not mean at runtime. I meant that a great portion of our defines now >> just remove an entire file from the build. We do not >> need #defines for that. >> > > Name one that _only_ involves dropping files from the build. Hint: don't > forget about XXRegisterAll(). > Anything that uses #requirespackage. There are at least 50. I went through all of them today since I was refactoring builder.py. > >> For other #defines, like those that get memory usage from the operating >> system, I definitely think code generation would be simpler >> than the #if monstrosity that currently exists. >> > > That is basically just one function, the fp_trap stuff is more complex, but > entirely maintainable in my opinion. I think PetscMemoryGetCurrentUsage is a > confusing function only because there was some attempt to only write > "PetscFunctionBegin" once instead of just making 5 short-but-complete > snippets for each alternative (i.e. duplicate the mundane stuff into all 5 > bodies instead of duplicating the conditionals to avoid duplicating the > mundane stuff). > > >> It would divide debugging into two parts: >> >> a) Why doesn't the generated code work? We have made mistakes in just >> the code I mentioned because we could not follow the inclusions correctly. >> >> b) Why did we choose to generate that code? This would be entirely >> contained in a single configure test. >> > > Recall that BuildSystem _still_ does not have a way to write a code snippet > once and have it run immediately or in batch (there is currently potential > for inconsistency even if I just want to do one test). Occasionally there is > a combinatorial number of paths through the code (because features are, at > least in principle, independent). If you generate it, you have to manage the > results of multiple orthogonal tests (where should the logic to handle this > go), and be confident that all code paths are correct. Sometimes the result > of a test is used in many places, answering the question "where is this > snippet of code used and why does it look this way" becomes difficult if > there are a dozen snippets for a given test. Generating the snippets is a > bit like writing a function FrobnicateTheVariablesInScopeOnLine192(); the > snippet may not make sense in it's own right, just in the context where it > is needed. This is an unconventional way to organize code and not going to > reduce complexity (especially to someone new to the project). > I really cannot understand what you are worried about here. We do this all the time. > I would rather write structured code that uses the results of these tests > (i.e. in a C file where it is now) than to generate said code somewhere in > config/*. > I would not agree. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] configure issues
by Jed Brown 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 13:42, Matthew Knepley <knepley(a)gmail.com> wrote: > I did not mean at runtime. I meant that a great portion of our defines now > just remove an entire file from the build. We do not > need #defines for that. > Name one that _only_ involves dropping files from the build. Hint: don't forget about XXRegisterAll(). > For other #defines, like those that get memory usage from the operating > system, I definitely think code generation would be simpler > than the #if monstrosity that currently exists. > That is basically just one function, the fp_trap stuff is more complex, but entirely maintainable in my opinion. I think PetscMemoryGetCurrentUsage is a confusing function only because there was some attempt to only write "PetscFunctionBegin" once instead of just making 5 short-but-complete snippets for each alternative (i.e. duplicate the mundane stuff into all 5 bodies instead of duplicating the conditionals to avoid duplicating the mundane stuff). > It would divide debugging into two parts: > > a) Why doesn't the generated code work? We have made mistakes in just the > code I mentioned because we could not follow the inclusions correctly. > > b) Why did we choose to generate that code? This would be entirely > contained in a single configure test. > Recall that BuildSystem _still_ does not have a way to write a code snippet once and have it run immediately or in batch (there is currently potential for inconsistency even if I just want to do one test). Occasionally there is a combinatorial number of paths through the code (because features are, at least in principle, independent). If you generate it, you have to manage the results of multiple orthogonal tests (where should the logic to handle this go), and be confident that all code paths are correct. Sometimes the result of a test is used in many places, answering the question "where is this snippet of code used and why does it look this way" becomes difficult if there are a dozen snippets for a given test. Generating the snippets is a bit like writing a function FrobnicateTheVariablesInScopeOnLine192(); the snippet may not make sense in it's own right, just in the context where it is needed. This is an unconventional way to organize code and not going to reduce complexity (especially to someone new to the project). I would rather write structured code that uses the results of these tests (i.e. in a C file where it is now) than to generate said code somewhere in config/*.

1 0

Re: [petsc-dev] configure issues
by Matthew Knepley 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 6:34 AM, Jed Brown <jed(a)59a2.org> wrote: > On Wed, Feb 9, 2011 at 13:17, Matthew Knepley <knepley(a)gmail.com> wrote: > >> What does this sentence mean? If petscconf.h has no logic, then isn't all >> the logic on the Python side? Isn't all that logic used from C? >> > > The result of tests go into petscconf.h. For the example I brought up, > different macro values imply that different code must be executed. To remove > the #if completely, you would have to generate different imperative code. > > >> >> >>> bodies are currently generated by Python. It sounds like you want to >>> generated imperative code in Python and call that from C. I think that would >>> add >>> >> >> We can currently control the build to do selective builds, just as we do >> with #defines right now. I think that removing that code from the tree and >> building everything would actually be simpler. >> > > I agree that when possible, configuration choices should be available at > runtime instead of compile time. However, "everything" cannot be compiled > because some are mutually exclusive: scalar precision, real/complex, int > size, C++ mangling, and architecture specific things. Consider the code that > sets up floating point trapping. Only one path can be compiled on any > specific architecture. Either you generate the fp.c source file differently > for each architecture, you have a different source file for each > architecture and have the build system use the correct one, or you keep them > all in the same file and use the preprocessor to decide which one to compile > (this is currently done). I did not mean at runtime. I meant that a great portion of our defines now just remove an entire file from the build. We do not need #defines for that. For other #defines, like those that get memory usage from the operating system, I definitely think code generation would be simpler than the #if monstrosity that currently exists. It would divide debugging into two parts: a) Why doesn't the generated code work? We have made mistakes in just the code I mentioned because we could not follow the inclusions correctly. b) Why did we choose to generate that code? This would be entirely contained in a single configure test. I do not disagree that for b) we would like a nice way (comment) to indicate which configure test generated that code. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] configure issues
by Jed Brown 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 13:17, Matthew Knepley <knepley(a)gmail.com> wrote: > What does this sentence mean? If petscconf.h has no logic, then isn't all > the logic on the Python side? Isn't all that logic used from C? > The result of tests go into petscconf.h. For the example I brought up, different macro values imply that different code must be executed. To remove the #if completely, you would have to generate different imperative code. > > >> bodies are currently generated by Python. It sounds like you want to >> generated imperative code in Python and call that from C. I think that would >> add >> > > We can currently control the build to do selective builds, just as we do > with #defines right now. I think that removing that code from the tree and > building everything would actually be simpler. > I agree that when possible, configuration choices should be available at runtime instead of compile time. However, "everything" cannot be compiled because some are mutually exclusive: scalar precision, real/complex, int size, C++ mangling, and architecture specific things. Consider the code that sets up floating point trapping. Only one path can be compiled on any specific architecture. Either you generate the fp.c source file differently for each architecture, you have a different source file for each architecture and have the build system use the correct one, or you keep them all in the same file and use the preprocessor to decide which one to compile (this is currently done).

1 0

Re: [petsc-dev] configure issues
by Matthew Knepley 09 Feb '11

09 Feb '11

On Wed, Feb 9, 2011 at 6:05 AM, Jed Brown <jed(a)59a2.org> wrote: > On Wed, Feb 9, 2011 at 12:56, Matthew Knepley <knepley(a)gmail.com> wrote: > >> We used to have everything in the preprocessor, and we could happily edit >> that and never have to think >> about Python at all. We abandoned that approach because it was too >> complicated, prone to break, and >> we spent all of our petsc-maint time on it. >> > > I don't know what examples you are referring to, I started with PETSc in > 2004. > Everything before configure was 'bmake' which was pure preprocessor. > I don't see a lot of logic currently being done on the Python side and used > from C, petscconf.h does not have logic in it. No C source files or function > What does this sentence mean? If petscconf.h has no logic, then isn't all the logic on the Python side? Isn't all that logic used from C? > bodies are currently generated by Python. It sounds like you want to > generated imperative code in Python and call that from C. I think that would > add > We can currently control the build to do selective builds, just as we do with #defines right now. I think that removing that code from the tree and building everything would actually be simpler. Matt > complexity. > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0