July 2011 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] MPI_Exscan
by William Gropp 31 Jul '11

31 Jul '11

I'm referring to the MPI_VERSION and MPI_SUBVERSION. I'm in general a supporter of feature checks, but in the case of MPI, the documentation is very precise about the content of each version and there are comprehensive tests for MPI implementations that have ensured compliance with the spec. The MPI Forum has repeatedly rejected any MPI subset, ensuring that MPI_VERSION and MPI_SUBVERSION can be used. Note also that these are the versions of the *standard*, not the version of a particular implementation. I agree with never using the version of an implementation for the points you raise, but I don't believe that this applies to the version of a standard. (E.g., I'd never recommend using the MPICH version as a test.) Bill On Jul 29, 2011, at 5:36 PM, Jed Brown wrote: > On Fri, Jul 29, 2011 at 09:44, Gropp, William D > <wgropp(a)illinois.edu> wrote: > Much better would be to use the MPI macros introduced to deal with > exactly this problem. > > Are you just referring to the MPI_VERSION/MPI_SUBVERSION macros or > is there a macro that specifically indicates the availability of > MPI_Exscan()? (I haven't seen any such macro in MPICH2 or OMPI.) > > We have avoided using versions for compiler/library feature checks > because version numbers are not self-documenting and because it only > takes one implementer mistake to make the versions useless. On the > other hand, all the feature tests make configure slow, but hopefully > that will be mostly resolved once we get parallel configure working. William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [petsc-dev] [petsc-maint #80119] Re: make install broken on unix?
by Jed Brown 30 Jul '11

30 Jul '11

Does that mean we can just add python utilities to that package, or do we need to create petsc-util? On Jul 30, 2011 12:07 PM, "Lisandro Dalcin" <dalcinl(a)gmail.com> wrote: > On 25 July 2011 19:33, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: >> On Mon, Jul 25, 2011 at 10:16, Ethan Coon <ecoon(a)lanl.gov> wrote: >>> >>> It's not really a stand-alone script though, so it shouldn't get the >>> hashbang or lose the extension. But it's not a full-fledged package of >>> libraries either. I don't think it should be anywhere in the default path >>> -- it's the kind of code that you should have to explicitly include (either >>> via sys.path manipulation or via PYTHONPATH) because too many little scripts >>> of utility code makes for a nasty, polluted namespace. There just isn't a >>> good place (that I know of) in the standard file system hierarchy for this >>> kind of stuff. >> >> Making the user/scripts do manual sys.path or PYTHONPATH manipulation makes >> it nearly impossible to make portable scripts that depend on these >> functions. Should we start a "proper" Python package containing PETSc >> utilities? Then scripts could rely on >> from petsc.io import binary_read >> or whatever working. Of course it might only contain two functions right >> now, but I could add my "petscplot" (makes various types of convergence and >> scaling plots from convergence monitors and -log_summary output), and I'm >> sure the community has other useful scripts. > > Note however that "petsc" package is already registered in PyPI to > make "pip install petsc4py" download and install PETSc. > > > -- > Lisandro Dalcin > --------------- > CIMEC (INTEC/CONICET-UNL) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1011) > Tel/Fax: +54-342-4511169

1 0

Re: [petsc-dev] [petsc-maint #80119] Re: make install broken on unix?
by Lisandro Dalcin 30 Jul '11

30 Jul '11

On 25 July 2011 13:15, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Jul 25, 2011, at 12:06 PM, Matthew Knepley wrote: > >> On Mon, Jul 25, 2011 at 4:56 PM, Sean Farley <sean(a)mcs.anl.gov> wrote: >> I disagree with this completely. There's a tendency to think of everything python as a package -- this is NOT a package. It has no __init__.py, it has no __all__, it has no submodules, and it doesn't need any of that. It simply does a one-off task -- it's bits of code to be used in a script. Putting this sort of code in site-packages is what makes site-packages the nightmare of ignored dependencies and overlapping versions it is today. >> >> If you want to make it into a package, you're welcome to do so. With that includes registration with the cheeseshop to make sure the namespace is unique, placing the single file within a folder that includes an __init__ (and then likely just gets imported within that __init__), writing a setup.py, etc etc. It's just complete overkill. >> >> Then why put it in bin/python at all? Just drop the .py extension and put it in bin (with a 'correct' [whatever that may be] python hashbang)? > > Because directories are a useful way to organize categories of software (for example the bin/matlab directory has all the Matlab scripts). Why not just not use subdirectories in PETSc and just stick all PETSc files directly into the PETSc root directory? > 1) IMHO, using subdirectories in bin/ does not look good. I think we should have lib/matlab and lib/python. As long as the code in lib/python works with many different Python versions, that location looks fine for me. bin/ is not the right place to add new directories with stuff, you should put there just shebang scripts with execute permissions. 2) You simply cannot drop the .py extension from PetscBinaryRead.py . That code it is not a script intended to be run, but to be imported from other Python codes. If you remove the .py ext, Python import will not be able to locate PetscBinaryRead. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

1 0

Re: [petsc-dev] [petsc-maint #80119] Re: make install broken on unix?
by Lisandro Dalcin 30 Jul '11

30 Jul '11

On 25 July 2011 19:33, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Mon, Jul 25, 2011 at 10:16, Ethan Coon <ecoon(a)lanl.gov> wrote: >> >> It's not really a stand-alone script though, so it shouldn't get the >> hashbang or lose the extension. But it's not a full-fledged package of >> libraries either. I don't think it should be anywhere in the default path >> -- it's the kind of code that you should have to explicitly include (either >> via sys.path manipulation or via PYTHONPATH) because too many little scripts >> of utility code makes for a nasty, polluted namespace. There just isn't a >> good place (that I know of) in the standard file system hierarchy for this >> kind of stuff. > > Making the user/scripts do manual sys.path or PYTHONPATH manipulation makes > it nearly impossible to make portable scripts that depend on these > functions. Should we start a "proper" Python package containing PETSc > utilities? Then scripts could rely on > from petsc.io import binary_read > or whatever working. Of course it might only contain two functions right > now, but I could add my "petscplot" (makes various types of convergence and > scaling plots from convergence monitors and -log_summary output), and I'm > sure the community has other useful scripts. Note however that "petsc" package is already registered in PyPI to make "pip install petsc4py" download and install PETSc. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

1 0

Re: [petsc-dev] MPI_Exscan
by Jed Brown 29 Jul '11

29 Jul '11

On Fri, Jul 29, 2011 at 09:44, Gropp, William D <wgropp(a)illinois.edu> wrote: > Much better would be to use the MPI macros introduced to deal with exactly > this problem. Are you just referring to the MPI_VERSION/MPI_SUBVERSION macros or is there a macro that specifically indicates the availability of MPI_Exscan()? (I haven't seen any such macro in MPICH2 or OMPI.) We have avoided using versions for compiler/library feature checks because version numbers are not self-documenting and because it only takes one implementer mistake to make the versions useless. On the other hand, all the feature tests make configure slow, but hopefully that will be mostly resolved once we get parallel configure working.

1 0

Re: [petsc-dev] MPI_Exscan
by Gropp, William D 29 Jul '11

29 Jul '11

Much better would be to use the MPI macros introduced to deal with exactly this problem. Then no addition tests are necessary. Bill Connected by DROID on Verizon Wireless -----Original message----- From: Barry Smith <bsmith(a)mcs.anl.gov> To: For users of the development version of PETSc <petsc-dev(a)mcs.anl.gov> Sent: Thu, Jul 28, 2011 20:40:59 GMT+00:00 Subject: Re: [petsc-dev] MPI_Exscan On Jul 28, 2011, at 3:37 PM, Mark F. Adams wrote: > I could not fine PETSC_HAVE_MPI_EXSCAN in the tags file. > > Am I missing something? Why would it ever be in the tags file until someone uses it? Like I said below you need to use it (where needed) and make sure it gets defined by adding code to MPI.py Barry > > Mark > > On Jul 28, 2011, at 3:31 PM, Barry Smith wrote: > >> >> Ideally code like >> >> #if defined(PETSC_HAVE_MPI_EXSCAN) >> cool C code that uses MPI_EXSCAN >> #else >> less cool C code that uses something else in MPI that is in MPI 1 >> #endif >> >> less ideal but ok code >> >> #if defined(PETSC_HAVE_MPI_EXSCAN) >> cool C code that uses MPI_EXSCAN >> #else >> SETERRQ(..., "Sorry but this code requires MPI_EXSCAN that doesn't exist on your machine's version of MPI, install a MPI2 with PETSc to get this functionality") >> #endif >> >> >> Then to make sure PETSC_HAVE_MPI_EXSCAN gets defined when it can add a test for MPI_Exscan() to bottom of config/BuildSystem/config/packages/MPI.py with the other tests for MPI to functions >> >> PETSc only requires MPI 1.1 support so any 2.0 or later stuff used needs this kind of protection. >> >> Barry >> >> >> On Jul 28, 2011, at 2:24 PM, Mark F. Adams wrote: >> >>> Dumb question: How do you protect it? >>> Mark >>> >>> On Jul 28, 2011, at 3:05 PM, Barry Smith wrote: >>> >>>> >>>> On Jul 28, 2011, at 11:55 AM, Jose E. Roman wrote: >>>> >>>>> We are getting an undefined reference `MPI_Exscan'. This function is used in src/mat/impls/nest/matnest.c. >>>>> If I am not wrong, MPI_Exscan was introduced in MPI-2 but was not available in the previous standard. Apparently, the machine where we are compiling is very old and has MPI-1 only. >>>>> >>>>> I guess there should be a PETSC_HAVE_MPI_EXSCAN test in configure. >>>> >>>> Correct, whoever used that MPI 2 function in PETSc is obligated to protect it. >>>> >>>> Barry >>>> >>>> >>>>> >>>>> Jose >>>>> >>>> >>>> >>> >> >> >

1 0

Re: [petsc-dev] MPI_Exscan
by Satish Balay 28 Jul '11

28 Jul '11

edit BuildSystem/.hg/hgrc and change URL to ssh://[email protected]//hg/petsc/BuildSystem Satish On Thu, 28 Jul 2011, Mark F. Adams wrote: > I added protection for MPI_Exscan in matnest.c and my code but I can not push the MPI.py file: > > ~/Codes/petsc-dev/config/BuildSystem>hg push > pushing to http://petsc.cs.iit.edu/petsc/BuildSystem > searching for changes > 1 changesets found > remote: ssl required > > Mark > > On Jul 28, 2011, at 4:40 PM, Barry Smith wrote: > > > > > On Jul 28, 2011, at 3:37 PM, Mark F. Adams wrote: > > > >> I could not fine PETSC_HAVE_MPI_EXSCAN in the tags file. > >> > >> Am I missing something? > > > > Why would it ever be in the tags file until someone uses it? Like I said below you need to use it (where needed) and make sure it gets defined by adding code to MPI.py > > > > Barry > > > >> > >> Mark > >> > >> On Jul 28, 2011, at 3:31 PM, Barry Smith wrote: > >> > >>> > >>> Ideally code like > >>> > >>> #if defined(PETSC_HAVE_MPI_EXSCAN) > >>> cool C code that uses MPI_EXSCAN > >>> #else > >>> less cool C code that uses something else in MPI that is in MPI 1 > >>> #endif > >>> > >>> less ideal but ok code > >>> > >>> #if defined(PETSC_HAVE_MPI_EXSCAN) > >>> cool C code that uses MPI_EXSCAN > >>> #else > >>> SETERRQ(..., "Sorry but this code requires MPI_EXSCAN that doesn't exist on your machine's version of MPI, install a MPI2 with PETSc to get this functionality") > >>> #endif > >>> > >>> > >>> Then to make sure PETSC_HAVE_MPI_EXSCAN gets defined when it can add a test for MPI_Exscan() to bottom of config/BuildSystem/config/packages/MPI.py with the other tests for MPI to functions > >>> > >>> PETSc only requires MPI 1.1 support so any 2.0 or later stuff used needs this kind of protection. > >>> > >>> Barry > >>> > >>> > >>> On Jul 28, 2011, at 2:24 PM, Mark F. Adams wrote: > >>> > >>>> Dumb question: How do you protect it? > >>>> Mark > >>>> > >>>> On Jul 28, 2011, at 3:05 PM, Barry Smith wrote: > >>>> > >>>>> > >>>>> On Jul 28, 2011, at 11:55 AM, Jose E. Roman wrote: > >>>>> > >>>>>> We are getting an undefined reference `MPI_Exscan'. This function is used in src/mat/impls/nest/matnest.c. > >>>>>> If I am not wrong, MPI_Exscan was introduced in MPI-2 but was not available in the previous standard. Apparently, the machine where we are compiling is very old and has MPI-1 only. > >>>>>> > >>>>>> I guess there should be a PETSC_HAVE_MPI_EXSCAN test in configure. > >>>>> > >>>>> Correct, whoever used that MPI 2 function in PETSc is obligated to protect it. > >>>>> > >>>>> Barry > >>>>> > >>>>> > >>>>>> > >>>>>> Jose > >>>>>> > >>>>> > >>>>> > >>>> > >>> > >>> > >> > > > > > >

1 0

Re: [petsc-dev] MPI_Exscan
by Mark F. Adams 28 Jul '11

28 Jul '11

I added protection for MPI_Exscan in matnest.c and my code but I can not push the MPI.py file: ~/Codes/petsc-dev/config/BuildSystem>hg push pushing to http://petsc.cs.iit.edu/petsc/BuildSystem searching for changes 1 changesets found remote: ssl required Mark On Jul 28, 2011, at 4:40 PM, Barry Smith wrote: > > On Jul 28, 2011, at 3:37 PM, Mark F. Adams wrote: > >> I could not fine PETSC_HAVE_MPI_EXSCAN in the tags file. >> >> Am I missing something? > > Why would it ever be in the tags file until someone uses it? Like I said below you need to use it (where needed) and make sure it gets defined by adding code to MPI.py > > Barry > >> >> Mark >> >> On Jul 28, 2011, at 3:31 PM, Barry Smith wrote: >> >>> >>> Ideally code like >>> >>> #if defined(PETSC_HAVE_MPI_EXSCAN) >>> cool C code that uses MPI_EXSCAN >>> #else >>> less cool C code that uses something else in MPI that is in MPI 1 >>> #endif >>> >>> less ideal but ok code >>> >>> #if defined(PETSC_HAVE_MPI_EXSCAN) >>> cool C code that uses MPI_EXSCAN >>> #else >>> SETERRQ(..., "Sorry but this code requires MPI_EXSCAN that doesn't exist on your machine's version of MPI, install a MPI2 with PETSc to get this functionality") >>> #endif >>> >>> >>> Then to make sure PETSC_HAVE_MPI_EXSCAN gets defined when it can add a test for MPI_Exscan() to bottom of config/BuildSystem/config/packages/MPI.py with the other tests for MPI to functions >>> >>> PETSc only requires MPI 1.1 support so any 2.0 or later stuff used needs this kind of protection. >>> >>> Barry >>> >>> >>> On Jul 28, 2011, at 2:24 PM, Mark F. Adams wrote: >>> >>>> Dumb question: How do you protect it? >>>> Mark >>>> >>>> On Jul 28, 2011, at 3:05 PM, Barry Smith wrote: >>>> >>>>> >>>>> On Jul 28, 2011, at 11:55 AM, Jose E. Roman wrote: >>>>> >>>>>> We are getting an undefined reference `MPI_Exscan'. This function is used in src/mat/impls/nest/matnest.c. >>>>>> If I am not wrong, MPI_Exscan was introduced in MPI-2 but was not available in the previous standard. Apparently, the machine where we are compiling is very old and has MPI-1 only. >>>>>> >>>>>> I guess there should be a PETSC_HAVE_MPI_EXSCAN test in configure. >>>>> >>>>> Correct, whoever used that MPI 2 function in PETSc is obligated to protect it. >>>>> >>>>> Barry >>>>> >>>>> >>>>>> >>>>>> Jose >>>>>> >>>>> >>>>> >>>> >>> >>> >> > >

1 0

Re: [petsc-dev] MPI_Exscan
by Barry Smith 28 Jul '11

28 Jul '11

On Jul 28, 2011, at 3:37 PM, Mark F. Adams wrote: > I could not fine PETSC_HAVE_MPI_EXSCAN in the tags file. > > Am I missing something? Why would it ever be in the tags file until someone uses it? Like I said below you need to use it (where needed) and make sure it gets defined by adding code to MPI.py Barry > > Mark > > On Jul 28, 2011, at 3:31 PM, Barry Smith wrote: > >> >> Ideally code like >> >> #if defined(PETSC_HAVE_MPI_EXSCAN) >> cool C code that uses MPI_EXSCAN >> #else >> less cool C code that uses something else in MPI that is in MPI 1 >> #endif >> >> less ideal but ok code >> >> #if defined(PETSC_HAVE_MPI_EXSCAN) >> cool C code that uses MPI_EXSCAN >> #else >> SETERRQ(..., "Sorry but this code requires MPI_EXSCAN that doesn't exist on your machine's version of MPI, install a MPI2 with PETSc to get this functionality") >> #endif >> >> >> Then to make sure PETSC_HAVE_MPI_EXSCAN gets defined when it can add a test for MPI_Exscan() to bottom of config/BuildSystem/config/packages/MPI.py with the other tests for MPI to functions >> >> PETSc only requires MPI 1.1 support so any 2.0 or later stuff used needs this kind of protection. >> >> Barry >> >> >> On Jul 28, 2011, at 2:24 PM, Mark F. Adams wrote: >> >>> Dumb question: How do you protect it? >>> Mark >>> >>> On Jul 28, 2011, at 3:05 PM, Barry Smith wrote: >>> >>>> >>>> On Jul 28, 2011, at 11:55 AM, Jose E. Roman wrote: >>>> >>>>> We are getting an undefined reference `MPI_Exscan'. This function is used in src/mat/impls/nest/matnest.c. >>>>> If I am not wrong, MPI_Exscan was introduced in MPI-2 but was not available in the previous standard. Apparently, the machine where we are compiling is very old and has MPI-1 only. >>>>> >>>>> I guess there should be a PETSC_HAVE_MPI_EXSCAN test in configure. >>>> >>>> Correct, whoever used that MPI 2 function in PETSc is obligated to protect it. >>>> >>>> Barry >>>> >>>> >>>>> >>>>> Jose >>>>> >>>> >>>> >>> >> >> >

1 0

Re: [petsc-dev] MPI_Exscan
by Mark F. Adams 28 Jul '11

28 Jul '11

I could not fine PETSC_HAVE_MPI_EXSCAN in the tags file. Am I missing something? Mark On Jul 28, 2011, at 3:31 PM, Barry Smith wrote: > > Ideally code like > > #if defined(PETSC_HAVE_MPI_EXSCAN) > cool C code that uses MPI_EXSCAN > #else > less cool C code that uses something else in MPI that is in MPI 1 > #endif > > less ideal but ok code > > #if defined(PETSC_HAVE_MPI_EXSCAN) > cool C code that uses MPI_EXSCAN > #else > SETERRQ(..., "Sorry but this code requires MPI_EXSCAN that doesn't exist on your machine's version of MPI, install a MPI2 with PETSc to get this functionality") > #endif > > > Then to make sure PETSC_HAVE_MPI_EXSCAN gets defined when it can add a test for MPI_Exscan() to bottom of config/BuildSystem/config/packages/MPI.py with the other tests for MPI to functions > > PETSc only requires MPI 1.1 support so any 2.0 or later stuff used needs this kind of protection. > > Barry > > > On Jul 28, 2011, at 2:24 PM, Mark F. Adams wrote: > >> Dumb question: How do you protect it? >> Mark >> >> On Jul 28, 2011, at 3:05 PM, Barry Smith wrote: >> >>> >>> On Jul 28, 2011, at 11:55 AM, Jose E. Roman wrote: >>> >>>> We are getting an undefined reference `MPI_Exscan'. This function is used in src/mat/impls/nest/matnest.c. >>>> If I am not wrong, MPI_Exscan was introduced in MPI-2 but was not available in the previous standard. Apparently, the machine where we are compiling is very old and has MPI-1 only. >>>> >>>> I guess there should be a PETSC_HAVE_MPI_EXSCAN test in configure. >>> >>> Correct, whoever used that MPI 2 function in PETSc is obligated to protect it. >>> >>> Barry >>> >>> >>>> >>>> Jose >>>> >>> >>> >> > >

1 0