August 2008 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc4py] What should be the 'default' communicator?
by Matthew Knepley 25 Aug '08

25 Aug '08

On Mon, Aug 25, 2008 at 8:13 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Aug 25, 2008, at 5:40 PM, Lisandro Dalcin wrote: > >> On Mon, Aug 25, 2008 at 6:48 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >>> >>> This is fine for me, except I vote against the setter/getter. Just let >>> the >>> power user access the variable PETSC_COMM_DEFAULT directly. >>> >> >> Barry, sorry, I do not completely understand your comments. All my >> concern about this is only relevant to petsc4py and not core PETSc. >> With that in mind, I prefer to "hide" PETSC_COMM_DEFAULT from users, >> and ask them to call the getter/setter routines. You know: in Python, >> changing module-level globals is a bit unsafe: a user could do: >> >> from petsc4py import PETSc >> PETSc.COMM_DEFAULT = None # or watever >> >> and then latter get a failure. Furthermore, the setter can be in >> charge of all the relevant error checking: the comm have to actually >> be an instance of 'PETSc.Comm' type, and its comunicator cannot be the >> MPI_COMM_NULL. >> > Can you not overload the assignment = to automatically call whatever > fancy setter you want to use? Then you get the simplicity I crave and the > safety you desire? Yes, you can make it a 'property'. Matt > Barry >> >>> >>> On Aug 25, 2008, at 4:43 PM, Lisandro Dalcin wrote: >>> >>>> On Mon, Aug 25, 2008 at 6:22 PM, Matthew Knepley <knepley(a)gmail.com> >>>> wrote: >>>>> >>>>> I agree that people will do this, I just don't agree that it should be >>>>> the default. >>>> >>>> Would you agree with the following: >>>> >>>> At petsc4py initialization (and after calling PetscInitialize()), I >>>> define PETSC_COMM_DEFAULT = PETSC_COMM_WORLD. All parallel PETSc >>>> objects created through petsc4py use PETSC_COMM_DEFAULT if the >>>> communicator is not explicitelly passed as an argument. Additionally, >>>> I expose in petsc4py a getter/setter enabling users to change at ANY >>>> TIME the default communicator to use. With this approach, the world >>>> communicator will be default, unless changed by the (power) user. >>>> >>>> >>>> >>>>>> To be honest, I've never looked too much at paradigms like Condor. But >>>>>> using them implies to learn yet another framework. Another anecdote: a >>>>>> guy sent me a mail with questions about mpi4py for solving a >>>>>> embarassingly parallel problems. I asked why he was trying to use such >>>>>> a "heavy weight" approach. And then he answered he was tired of the >>>>>> complications and performance of using a Grid-based approach, and that >>>>>> 'mpiexec' a Python script with some coordinating MPI calls was far >>>>>> easier to setup, extend, and maintain and had better overall running >>>>>> times than submitting jobs to "The Grid". >>>>> >>>>> That is my experience with grid software as well. However, in the >>>>> particular case >>>>> of Condor, I disagree. It is fairly easy to setup and has great >>>>> features like fault >>>>> tolerance, automatic migration and balancing, that make it much more >>>>> useful >>>>> that just MPI. >>>>> >>>>> Matt >>>>> >>>>>>> On Mon, Aug 25, 2008 at 10:54 AM, Lisandro Dalcin <dalcinl(a)gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> After working hard on mpi4py, this week I'll spend my time >>>>>>>> cleaning-up >>>>>>>> and adding features to the new Cython-based petsc4py. Then, I'll be >>>>>>>> asking questions to this list requesting for advise. >>>>>>>> >>>>>>>> In all calls that create new PETSc objects, I've decided to make the >>>>>>>> 'comm' argument optional. If the communicator is not passed, >>>>>>>> PETSC_COMM_WORLD is currently used. This is the approach PETSc uses >>>>>>>> in >>>>>>>> some C++ calls implemented through PetscPolymorphicFunction(). >>>>>>>> >>>>>>>> But now I believe that is wrong, and that PETSC_COMM_SELF should be >>>>>>>> the default. Or perhaps even better, I should let users set the >>>>>>>> default communicator used by petsc4py to create new (parallel) >>>>>>>> objects. >>>>>>>> >>>>>>>> An anecdote: some time ago, a petsc4py user wrote a sequential code >>>>>>>> and created objects without passing communicator arguments, next he >>>>>>>> wanted to solve many of those problems in different worker processes >>>>>>>> in a "ambarrasingly parallel" fashion and collect results at the >>>>>>>> master process. Of course, he run into trouble. Then I asked him to >>>>>>>> initialize PETSc in such a way that PETSC_COMM_WORLD was actually >>>>>>>> PETSC_COMM_SELF (by setting the world comm before PetscInitalize()). >>>>>>>> This mostly works, but has a problem: we have lost the actual >>>>>>>> PETSC_COMM_WORLD, so we are not able to create a parallel object >>>>>>>> after >>>>>>>> PetscInitialize(). >>>>>>>> >>>>>>>> Any thoughts? >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Lisandro Dalcï¿½n >>>>>>>> --------------- >>>>>>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a >>>>>>>> (CIMEC) >>>>>>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica >>>>>>>> (INTEC) >>>>>>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>>>>>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>>>>>> Tel/Fax: +54-(0)342-451.1594 >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> What most experimenters take for granted before they begin their >>>>>>> experiments is infinitely more interesting than any results to which >>>>>>> their experiments lead. >>>>>>> -- Norbert Wiener >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Lisandro Dalcï¿½n >>>>>> --------------- >>>>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >>>>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >>>>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>>>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>>>> Tel/Fax: +54-(0)342-451.1594 >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which >>>>> their experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Lisandro Dalcï¿½n >>>> --------------- >>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>> Tel/Fax: +54-(0)342-451.1594 >>>> >>> >>> >> >> >> >> -- >> Lisandro Dalcï¿½n >> --------------- >> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >> Tel/Fax: +54-(0)342-451.1594 >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc4py] What should be the 'default' communicator?
by Barry Smith 25 Aug '08

25 Aug '08

On Aug 25, 2008, at 5:40 PM, Lisandro Dalcin wrote: > On Mon, Aug 25, 2008 at 6:48 PM, Barry Smith <bsmith(a)mcs.anl.gov> > wrote: >> >> This is fine for me, except I vote against the setter/getter. Just >> let the >> power user access the variable PETSC_COMM_DEFAULT directly. >> > > Barry, sorry, I do not completely understand your comments. All my > concern about this is only relevant to petsc4py and not core PETSc. > With that in mind, I prefer to "hide" PETSC_COMM_DEFAULT from users, > and ask them to call the getter/setter routines. You know: in Python, > changing module-level globals is a bit unsafe: a user could do: > > from petsc4py import PETSc > PETSc.COMM_DEFAULT = None # or watever > > and then latter get a failure. Furthermore, the setter can be in > charge of all the relevant error checking: the comm have to actually > be an instance of 'PETSc.Comm' type, and its comunicator cannot be the > MPI_COMM_NULL. > Can you not overload the assignment = to automatically call whatever fancy setter you want to use? Then you get the simplicity I crave and the safety you desire? Barry > >> >> On Aug 25, 2008, at 4:43 PM, Lisandro Dalcin wrote: >> >>> On Mon, Aug 25, 2008 at 6:22 PM, Matthew Knepley <knepley(a)gmail.com> >>> wrote: >>>> >>>> I agree that people will do this, I just don't agree that it >>>> should be >>>> the default. >>> >>> Would you agree with the following: >>> >>> At petsc4py initialization (and after calling PetscInitialize()), I >>> define PETSC_COMM_DEFAULT = PETSC_COMM_WORLD. All parallel PETSc >>> objects created through petsc4py use PETSC_COMM_DEFAULT if the >>> communicator is not explicitelly passed as an argument. >>> Additionally, >>> I expose in petsc4py a getter/setter enabling users to change at ANY >>> TIME the default communicator to use. With this approach, the world >>> communicator will be default, unless changed by the (power) user. >>> >>> >>> >>>>> To be honest, I've never looked too much at paradigms like >>>>> Condor. But >>>>> using them implies to learn yet another framework. Another >>>>> anecdote: a >>>>> guy sent me a mail with questions about mpi4py for solving a >>>>> embarassingly parallel problems. I asked why he was trying to >>>>> use such >>>>> a "heavy weight" approach. And then he answered he was tired of >>>>> the >>>>> complications and performance of using a Grid-based approach, >>>>> and that >>>>> 'mpiexec' a Python script with some coordinating MPI calls was far >>>>> easier to setup, extend, and maintain and had better overall >>>>> running >>>>> times than submitting jobs to "The Grid". >>>> >>>> That is my experience with grid software as well. However, in the >>>> particular case >>>> of Condor, I disagree. It is fairly easy to setup and has great >>>> features like fault >>>> tolerance, automatic migration and balancing, that make it much >>>> more >>>> useful >>>> that just MPI. >>>> >>>> Matt >>>> >>>>>> On Mon, Aug 25, 2008 at 10:54 AM, Lisandro Dalcin <dalcinl(a)gmail.com >>>>>> > >>>>>> wrote: >>>>>>> >>>>>>> After working hard on mpi4py, this week I'll spend my time >>>>>>> cleaning-up >>>>>>> and adding features to the new Cython-based petsc4py. Then, >>>>>>> I'll be >>>>>>> asking questions to this list requesting for advise. >>>>>>> >>>>>>> In all calls that create new PETSc objects, I've decided to >>>>>>> make the >>>>>>> 'comm' argument optional. If the communicator is not passed, >>>>>>> PETSC_COMM_WORLD is currently used. This is the approach PETSc >>>>>>> uses in >>>>>>> some C++ calls implemented through PetscPolymorphicFunction(). >>>>>>> >>>>>>> But now I believe that is wrong, and that PETSC_COMM_SELF >>>>>>> should be >>>>>>> the default. Or perhaps even better, I should let users set the >>>>>>> default communicator used by petsc4py to create new (parallel) >>>>>>> objects. >>>>>>> >>>>>>> An anecdote: some time ago, a petsc4py user wrote a sequential >>>>>>> code >>>>>>> and created objects without passing communicator arguments, >>>>>>> next he >>>>>>> wanted to solve many of those problems in different worker >>>>>>> processes >>>>>>> in a "ambarrasingly parallel" fashion and collect results at the >>>>>>> master process. Of course, he run into trouble. Then I asked >>>>>>> him to >>>>>>> initialize PETSc in such a way that PETSC_COMM_WORLD was >>>>>>> actually >>>>>>> PETSC_COMM_SELF (by setting the world comm before >>>>>>> PetscInitalize()). >>>>>>> This mostly works, but has a problem: we have lost the actual >>>>>>> PETSC_COMM_WORLD, so we are not able to create a parallel >>>>>>> object after >>>>>>> PetscInitialize(). >>>>>>> >>>>>>> Any thoughts? >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Lisandro Dalcï¿½n >>>>>>> --------------- >>>>>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a >>>>>>> (CIMEC) >>>>>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica >>>>>>> (INTEC) >>>>>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas >>>>>>> (CONICET) >>>>>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>>>>> Tel/Fax: +54-(0)342-451.1594 >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to >>>>>> which >>>>>> their experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Lisandro Dalcï¿½n >>>>> --------------- >>>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a >>>>> (CIMEC) >>>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica >>>>> (INTEC) >>>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas >>>>> (CONICET) >>>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>>> Tel/Fax: +54-(0)342-451.1594 >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to >>>> which >>>> their experiments lead. >>>> -- Norbert Wiener >>>> >>>> >>> >>> >>> >>> -- >>> Lisandro Dalcï¿½n >>> --------------- >>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a >>> (CIMEC) >>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica >>> (INTEC) >>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>> Tel/Fax: +54-(0)342-451.1594 >>> >> >> > > > > -- > Lisandro Dalcï¿½n > --------------- > Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) > Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) > Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) > PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 >

1 0

Re: [petsc4py] What should be the 'default' communicator?
by Lisandro Dalcin 25 Aug '08

25 Aug '08

On Mon, Aug 25, 2008 at 6:48 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > This is fine for me, except I vote against the setter/getter. Just let the > power user access the variable PETSC_COMM_DEFAULT directly. > Barry, sorry, I do not completely understand your comments. All my concern about this is only relevant to petsc4py and not core PETSc. With that in mind, I prefer to "hide" PETSC_COMM_DEFAULT from users, and ask them to call the getter/setter routines. You know: in Python, changing module-level globals is a bit unsafe: a user could do: from petsc4py import PETSc PETSc.COMM_DEFAULT = None # or watever and then latter get a failure. Furthermore, the setter can be in charge of all the relevant error checking: the comm have to actually be an instance of 'PETSc.Comm' type, and its comunicator cannot be the MPI_COMM_NULL. > > On Aug 25, 2008, at 4:43 PM, Lisandro Dalcin wrote: > >> On Mon, Aug 25, 2008 at 6:22 PM, Matthew Knepley <knepley(a)gmail.com> >> wrote: >>> >>> I agree that people will do this, I just don't agree that it should be >>> the default. >> >> Would you agree with the following: >> >> At petsc4py initialization (and after calling PetscInitialize()), I >> define PETSC_COMM_DEFAULT = PETSC_COMM_WORLD. All parallel PETSc >> objects created through petsc4py use PETSC_COMM_DEFAULT if the >> communicator is not explicitelly passed as an argument. Additionally, >> I expose in petsc4py a getter/setter enabling users to change at ANY >> TIME the default communicator to use. With this approach, the world >> communicator will be default, unless changed by the (power) user. >> >> >> >>>> To be honest, I've never looked too much at paradigms like Condor. But >>>> using them implies to learn yet another framework. Another anecdote: a >>>> guy sent me a mail with questions about mpi4py for solving a >>>> embarassingly parallel problems. I asked why he was trying to use such >>>> a "heavy weight" approach. And then he answered he was tired of the >>>> complications and performance of using a Grid-based approach, and that >>>> 'mpiexec' a Python script with some coordinating MPI calls was far >>>> easier to setup, extend, and maintain and had better overall running >>>> times than submitting jobs to "The Grid". >>> >>> That is my experience with grid software as well. However, in the >>> particular case >>> of Condor, I disagree. It is fairly easy to setup and has great >>> features like fault >>> tolerance, automatic migration and balancing, that make it much more >>> useful >>> that just MPI. >>> >>> Matt >>> >>>>> On Mon, Aug 25, 2008 at 10:54 AM, Lisandro Dalcin <dalcinl(a)gmail.com> >>>>> wrote: >>>>>> >>>>>> After working hard on mpi4py, this week I'll spend my time cleaning-up >>>>>> and adding features to the new Cython-based petsc4py. Then, I'll be >>>>>> asking questions to this list requesting for advise. >>>>>> >>>>>> In all calls that create new PETSc objects, I've decided to make the >>>>>> 'comm' argument optional. If the communicator is not passed, >>>>>> PETSC_COMM_WORLD is currently used. This is the approach PETSc uses in >>>>>> some C++ calls implemented through PetscPolymorphicFunction(). >>>>>> >>>>>> But now I believe that is wrong, and that PETSC_COMM_SELF should be >>>>>> the default. Or perhaps even better, I should let users set the >>>>>> default communicator used by petsc4py to create new (parallel) >>>>>> objects. >>>>>> >>>>>> An anecdote: some time ago, a petsc4py user wrote a sequential code >>>>>> and created objects without passing communicator arguments, next he >>>>>> wanted to solve many of those problems in different worker processes >>>>>> in a "ambarrasingly parallel" fashion and collect results at the >>>>>> master process. Of course, he run into trouble. Then I asked him to >>>>>> initialize PETSc in such a way that PETSC_COMM_WORLD was actually >>>>>> PETSC_COMM_SELF (by setting the world comm before PetscInitalize()). >>>>>> This mostly works, but has a problem: we have lost the actual >>>>>> PETSC_COMM_WORLD, so we are not able to create a parallel object after >>>>>> PetscInitialize(). >>>>>> >>>>>> Any thoughts? >>>>>> >>>>>> >>>>>> -- >>>>>> Lisandro Dalcï¿½n >>>>>> --------------- >>>>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >>>>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >>>>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>>>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>>>> Tel/Fax: +54-(0)342-451.1594 >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which >>>>> their experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Lisandro Dalcï¿½n >>>> --------------- >>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>> Tel/Fax: +54-(0)342-451.1594 >>>> >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which >>> their experiments lead. >>> -- Norbert Wiener >>> >>> >> >> >> >> -- >> Lisandro Dalcï¿½n >> --------------- >> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >> Tel/Fax: +54-(0)342-451.1594 >> > > -- Lisandro Dalcï¿½n --------------- Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

1 0

Re: possible bug in MatGetOwnershipRanges() and MatGetOwnershipRangesColumns()
by Barry Smith 25 Aug '08

25 Aug '08

I'm changing those two routines to also preallocate. Barry On Aug 25, 2008, at 4:27 PM, Lisandro Dalcin wrote: > Barry/Matt/Shatish. Could any of you review the implementation of > MatGetOwnershipRanges() and MatGetOwnershipRangesColumns(). I believe > a MatPreallocated() call is missing in both functions? The companion > ones, MatGetOwnershipRange() and MatGetOwnershipRangeColumns() > DO HAVE the call to MatPreallocate(), I believe this is indeed the > right way: PetscMapSetUp() calls are actually made in matrix > preallocation routines. Anyway, this is a bit dangerous, because this > makes getting ownership range a collective operation if the matrix is > not preallocated. Furthermore, this is in contradiction to all the > docs about XXXGetOwnershipRangeXXX(). Of course, the safe (but > unconvenient) way to go would be to disable operation on > non-preallocated Mat's by generating errors. > > -- > Lisandro Dalcï¿½n > --------------- > Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) > Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) > Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) > PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 >

1 0

Re: [petsc4py] What should be the 'default' communicator?
by Matthew Knepley 25 Aug '08

25 Aug '08

I am cool with this. Matt On Mon, Aug 25, 2008 at 4:48 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > This is fine for me, except I vote against the setter/getter. Just let the > power user access the variable PETSC_COMM_DEFAULT directly. > > Barry > > > On Aug 25, 2008, at 4:43 PM, Lisandro Dalcin wrote: > >> On Mon, Aug 25, 2008 at 6:22 PM, Matthew Knepley <knepley(a)gmail.com> >> wrote: >>> >>> I agree that people will do this, I just don't agree that it should be >>> the default. >> >> Would you agree with the following: >> >> At petsc4py initialization (and after calling PetscInitialize()), I >> define PETSC_COMM_DEFAULT = PETSC_COMM_WORLD. All parallel PETSc >> objects created through petsc4py use PETSC_COMM_DEFAULT if the >> communicator is not explicitelly passed as an argument. Additionally, >> I expose in petsc4py a getter/setter enabling users to change at ANY >> TIME the default communicator to use. With this approach, the world >> communicator will be default, unless changed by the (power) user. >> >> >> >>>> To be honest, I've never looked too much at paradigms like Condor. But >>>> using them implies to learn yet another framework. Another anecdote: a >>>> guy sent me a mail with questions about mpi4py for solving a >>>> embarassingly parallel problems. I asked why he was trying to use such >>>> a "heavy weight" approach. And then he answered he was tired of the >>>> complications and performance of using a Grid-based approach, and that >>>> 'mpiexec' a Python script with some coordinating MPI calls was far >>>> easier to setup, extend, and maintain and had better overall running >>>> times than submitting jobs to "The Grid". >>> >>> That is my experience with grid software as well. However, in the >>> particular case >>> of Condor, I disagree. It is fairly easy to setup and has great >>> features like fault >>> tolerance, automatic migration and balancing, that make it much more >>> useful >>> that just MPI. >>> >>> Matt >>> >>>>> On Mon, Aug 25, 2008 at 10:54 AM, Lisandro Dalcin <dalcinl(a)gmail.com> >>>>> wrote: >>>>>> >>>>>> After working hard on mpi4py, this week I'll spend my time cleaning-up >>>>>> and adding features to the new Cython-based petsc4py. Then, I'll be >>>>>> asking questions to this list requesting for advise. >>>>>> >>>>>> In all calls that create new PETSc objects, I've decided to make the >>>>>> 'comm' argument optional. If the communicator is not passed, >>>>>> PETSC_COMM_WORLD is currently used. This is the approach PETSc uses in >>>>>> some C++ calls implemented through PetscPolymorphicFunction(). >>>>>> >>>>>> But now I believe that is wrong, and that PETSC_COMM_SELF should be >>>>>> the default. Or perhaps even better, I should let users set the >>>>>> default communicator used by petsc4py to create new (parallel) >>>>>> objects. >>>>>> >>>>>> An anecdote: some time ago, a petsc4py user wrote a sequential code >>>>>> and created objects without passing communicator arguments, next he >>>>>> wanted to solve many of those problems in different worker processes >>>>>> in a "ambarrasingly parallel" fashion and collect results at the >>>>>> master process. Of course, he run into trouble. Then I asked him to >>>>>> initialize PETSc in such a way that PETSC_COMM_WORLD was actually >>>>>> PETSC_COMM_SELF (by setting the world comm before PetscInitalize()). >>>>>> This mostly works, but has a problem: we have lost the actual >>>>>> PETSC_COMM_WORLD, so we are not able to create a parallel object after >>>>>> PetscInitialize(). >>>>>> >>>>>> Any thoughts? >>>>>> >>>>>> >>>>>> -- >>>>>> Lisandro Dalcï¿½n >>>>>> --------------- >>>>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >>>>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >>>>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>>>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>>>> Tel/Fax: +54-(0)342-451.1594 >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which >>>>> their experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Lisandro Dalcï¿½n >>>> --------------- >>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>> Tel/Fax: +54-(0)342-451.1594 >>>> >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which >>> their experiments lead. >>> -- Norbert Wiener >>> >>> >> >> >> >> -- >> Lisandro Dalcï¿½n >> --------------- >> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >> Tel/Fax: +54-(0)342-451.1594 >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc4py] What should be the 'default' communicator?
by Barry Smith 25 Aug '08

25 Aug '08

This is fine for me, except I vote against the setter/getter. Just let the power user access the variable PETSC_COMM_DEFAULT directly. Barry On Aug 25, 2008, at 4:43 PM, Lisandro Dalcin wrote: > On Mon, Aug 25, 2008 at 6:22 PM, Matthew Knepley <knepley(a)gmail.com> > wrote: >> I agree that people will do this, I just don't agree that it should >> be >> the default. > > Would you agree with the following: > > At petsc4py initialization (and after calling PetscInitialize()), I > define PETSC_COMM_DEFAULT = PETSC_COMM_WORLD. All parallel PETSc > objects created through petsc4py use PETSC_COMM_DEFAULT if the > communicator is not explicitelly passed as an argument. Additionally, > I expose in petsc4py a getter/setter enabling users to change at ANY > TIME the default communicator to use. With this approach, the world > communicator will be default, unless changed by the (power) user. > > > >>> To be honest, I've never looked too much at paradigms like Condor. >>> But >>> using them implies to learn yet another framework. Another >>> anecdote: a >>> guy sent me a mail with questions about mpi4py for solving a >>> embarassingly parallel problems. I asked why he was trying to use >>> such >>> a "heavy weight" approach. And then he answered he was tired of the >>> complications and performance of using a Grid-based approach, and >>> that >>> 'mpiexec' a Python script with some coordinating MPI calls was far >>> easier to setup, extend, and maintain and had better overall running >>> times than submitting jobs to "The Grid". >> >> That is my experience with grid software as well. However, in the >> particular case >> of Condor, I disagree. It is fairly easy to setup and has great >> features like fault >> tolerance, automatic migration and balancing, that make it much >> more useful >> that just MPI. >> >> Matt >> >>>> On Mon, Aug 25, 2008 at 10:54 AM, Lisandro Dalcin <dalcinl(a)gmail.com >>>> > wrote: >>>>> After working hard on mpi4py, this week I'll spend my time >>>>> cleaning-up >>>>> and adding features to the new Cython-based petsc4py. Then, I'll >>>>> be >>>>> asking questions to this list requesting for advise. >>>>> >>>>> In all calls that create new PETSc objects, I've decided to make >>>>> the >>>>> 'comm' argument optional. If the communicator is not passed, >>>>> PETSC_COMM_WORLD is currently used. This is the approach PETSc >>>>> uses in >>>>> some C++ calls implemented through PetscPolymorphicFunction(). >>>>> >>>>> But now I believe that is wrong, and that PETSC_COMM_SELF should >>>>> be >>>>> the default. Or perhaps even better, I should let users set the >>>>> default communicator used by petsc4py to create new (parallel) >>>>> objects. >>>>> >>>>> An anecdote: some time ago, a petsc4py user wrote a sequential >>>>> code >>>>> and created objects without passing communicator arguments, next >>>>> he >>>>> wanted to solve many of those problems in different worker >>>>> processes >>>>> in a "ambarrasingly parallel" fashion and collect results at the >>>>> master process. Of course, he run into trouble. Then I asked him >>>>> to >>>>> initialize PETSc in such a way that PETSC_COMM_WORLD was actually >>>>> PETSC_COMM_SELF (by setting the world comm before >>>>> PetscInitalize()). >>>>> This mostly works, but has a problem: we have lost the actual >>>>> PETSC_COMM_WORLD, so we are not able to create a parallel object >>>>> after >>>>> PetscInitialize(). >>>>> >>>>> Any thoughts? >>>>> >>>>> >>>>> -- >>>>> Lisandro Dalcï¿½n >>>>> --------------- >>>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a >>>>> (CIMEC) >>>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica >>>>> (INTEC) >>>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas >>>>> (CONICET) >>>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>>> Tel/Fax: +54-(0)342-451.1594 >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to >>>> which >>>> their experiments lead. >>>> -- Norbert Wiener >>>> >>>> >>> >>> >>> >>> -- >>> Lisandro Dalcï¿½n >>> --------------- >>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a >>> (CIMEC) >>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica >>> (INTEC) >>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>> Tel/Fax: +54-(0)342-451.1594 >>> >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> >> > > > > -- > Lisandro Dalcï¿½n > --------------- > Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) > Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) > Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) > PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 >

1 0

Re: [petsc4py] What should be the 'default' communicator?
by Lisandro Dalcin 25 Aug '08

25 Aug '08

On Mon, Aug 25, 2008 at 6:22 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > I agree that people will do this, I just don't agree that it should be > the default. Would you agree with the following: At petsc4py initialization (and after calling PetscInitialize()), I define PETSC_COMM_DEFAULT = PETSC_COMM_WORLD. All parallel PETSc objects created through petsc4py use PETSC_COMM_DEFAULT if the communicator is not explicitelly passed as an argument. Additionally, I expose in petsc4py a getter/setter enabling users to change at ANY TIME the default communicator to use. With this approach, the world communicator will be default, unless changed by the (power) user. >> To be honest, I've never looked too much at paradigms like Condor. But >> using them implies to learn yet another framework. Another anecdote: a >> guy sent me a mail with questions about mpi4py for solving a >> embarassingly parallel problems. I asked why he was trying to use such >> a "heavy weight" approach. And then he answered he was tired of the >> complications and performance of using a Grid-based approach, and that >> 'mpiexec' a Python script with some coordinating MPI calls was far >> easier to setup, extend, and maintain and had better overall running >> times than submitting jobs to "The Grid". > > That is my experience with grid software as well. However, in the > particular case > of Condor, I disagree. It is fairly easy to setup and has great > features like fault > tolerance, automatic migration and balancing, that make it much more useful > that just MPI. > > Matt > >>> On Mon, Aug 25, 2008 at 10:54 AM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >>>> After working hard on mpi4py, this week I'll spend my time cleaning-up >>>> and adding features to the new Cython-based petsc4py. Then, I'll be >>>> asking questions to this list requesting for advise. >>>> >>>> In all calls that create new PETSc objects, I've decided to make the >>>> 'comm' argument optional. If the communicator is not passed, >>>> PETSC_COMM_WORLD is currently used. This is the approach PETSc uses in >>>> some C++ calls implemented through PetscPolymorphicFunction(). >>>> >>>> But now I believe that is wrong, and that PETSC_COMM_SELF should be >>>> the default. Or perhaps even better, I should let users set the >>>> default communicator used by petsc4py to create new (parallel) >>>> objects. >>>> >>>> An anecdote: some time ago, a petsc4py user wrote a sequential code >>>> and created objects without passing communicator arguments, next he >>>> wanted to solve many of those problems in different worker processes >>>> in a "ambarrasingly parallel" fashion and collect results at the >>>> master process. Of course, he run into trouble. Then I asked him to >>>> initialize PETSc in such a way that PETSC_COMM_WORLD was actually >>>> PETSC_COMM_SELF (by setting the world comm before PetscInitalize()). >>>> This mostly works, but has a problem: we have lost the actual >>>> PETSC_COMM_WORLD, so we are not able to create a parallel object after >>>> PetscInitialize(). >>>> >>>> Any thoughts? >>>> >>>> >>>> -- >>>> Lisandro Dalcï¿½n >>>> --------------- >>>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >>>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >>>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>>> Tel/Fax: +54-(0)342-451.1594 >>>> >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which >>> their experiments lead. >>> -- Norbert Wiener >>> >>> >> >> >> >> -- >> Lisandro Dalcï¿½n >> --------------- >> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >> Tel/Fax: +54-(0)342-451.1594 >> >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > -- Lisandro Dalcï¿½n --------------- Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

1 0

possible bug in MatGetOwnershipRanges() and MatGetOwnershipRangesColumns()
by Lisandro Dalcin 25 Aug '08

25 Aug '08

Barry/Matt/Shatish. Could any of you review the implementation of MatGetOwnershipRanges() and MatGetOwnershipRangesColumns(). I believe a MatPreallocated() call is missing in both functions? The companion ones, MatGetOwnershipRange() and MatGetOwnershipRangeColumns() DO HAVE the call to MatPreallocate(), I believe this is indeed the right way: PetscMapSetUp() calls are actually made in matrix preallocation routines. Anyway, this is a bit dangerous, because this makes getting ownership range a collective operation if the matrix is not preallocated. Furthermore, this is in contradiction to all the docs about XXXGetOwnershipRangeXXX(). Of course, the safe (but unconvenient) way to go would be to disable operation on non-preallocated Mat's by generating errors. -- Lisandro Dalcï¿½n --------------- Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

1 0

Re: [petsc4py] What should be the 'default' communicator?
by Matthew Knepley 25 Aug '08

25 Aug '08

On Mon, Aug 25, 2008 at 1:22 PM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: > On Mon, Aug 25, 2008 at 1:08 PM, Matthew Knepley <knepley(a)gmail.com> wrote: >> I would still maintain that PETSC_COMM_WORLD is the correct default. There >> are better paradigms for embarassingly parallel operation, like Condor. PETSc >> is intended for parallel, domain decomposition runs. > > Yes, you are completelly right. But I believe that many people still > use PETSc in a sequential way just because PETSc is full featured, > well designed, easy to learn, etc. So, despite PETSc being intended > for parallel, domain decomposition applications, many people are going > to use it for sequential apps and embarassingly parallel operations. I agree that people will do this, I just don't agree that it should be the default. > To be honest, I've never looked too much at paradigms like Condor. But > using them implies to learn yet another framework. Another anecdote: a > guy sent me a mail with questions about mpi4py for solving a > embarassingly parallel problems. I asked why he was trying to use such > a "heavy weight" approach. And then he answered he was tired of the > complications and performance of using a Grid-based approach, and that > 'mpiexec' a Python script with some coordinating MPI calls was far > easier to setup, extend, and maintain and had better overall running > times than submitting jobs to "The Grid". That is my experience with grid software as well. However, in the particular case of Condor, I disagree. It is fairly easy to setup and has great features like fault tolerance, automatic migration and balancing, that make it much more useful that just MPI. Matt >> On Mon, Aug 25, 2008 at 10:54 AM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >>> After working hard on mpi4py, this week I'll spend my time cleaning-up >>> and adding features to the new Cython-based petsc4py. Then, I'll be >>> asking questions to this list requesting for advise. >>> >>> In all calls that create new PETSc objects, I've decided to make the >>> 'comm' argument optional. If the communicator is not passed, >>> PETSC_COMM_WORLD is currently used. This is the approach PETSc uses in >>> some C++ calls implemented through PetscPolymorphicFunction(). >>> >>> But now I believe that is wrong, and that PETSC_COMM_SELF should be >>> the default. Or perhaps even better, I should let users set the >>> default communicator used by petsc4py to create new (parallel) >>> objects. >>> >>> An anecdote: some time ago, a petsc4py user wrote a sequential code >>> and created objects without passing communicator arguments, next he >>> wanted to solve many of those problems in different worker processes >>> in a "ambarrasingly parallel" fashion and collect results at the >>> master process. Of course, he run into trouble. Then I asked him to >>> initialize PETSc in such a way that PETSC_COMM_WORLD was actually >>> PETSC_COMM_SELF (by setting the world comm before PetscInitalize()). >>> This mostly works, but has a problem: we have lost the actual >>> PETSC_COMM_WORLD, so we are not able to create a parallel object after >>> PetscInitialize(). >>> >>> Any thoughts? >>> >>> >>> -- >>> Lisandro Dalcï¿½n >>> --------------- >>> Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) >>> Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) >>> Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) >>> PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina >>> Tel/Fax: +54-(0)342-451.1594 >>> >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> >> > > > > -- > Lisandro Dalcï¿½n > --------------- > Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) > Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) > Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) > PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Mat and row/column ownership ranges
by Lisandro Dalcin 25 Aug '08

25 Aug '08

Now that we can get Mat ownership range/reanges from rows and cols, would it make sense to condensate the calls into a single MatGetOwnershipRange[s] able to return all the ranges? Something like: MatGetOwnershipRange(Mat A, PetscInt* rlow, PetscInt *rhigh, PetscInt *clow, PetscInt *chigh); MatGetOwnershipRanges(Mat A, const PetscInt* rowranges, const PetscInt *colranges); We could still provide MatGetOwnershipRange[s]Row() and MatGetOwnershipRange[s]Column() and implement the first to calls above in terms of these later four. Of course, this smells to too many ways to do it. Note that I'm +0 on this, I just wanted to point out some other choices. -- Lisandro Dalcï¿½n --------------- Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

1 0