April 2012 - petsc-dev - mailman3.cels.anl.gov

A few remaining issues with Mat block sizes
by Lisandro Dalcin 10 Apr '12

10 Apr '12

Barry, could you review the attached patch? Basically, many Mat preallocation routines should not set the block size to 1, and instead honor what the user could have set calling MatSetBlockSize(). Please note that in this patch I'm not addressing all instances of this issue in the codebase, I've just worked on some of the matrix types (seq/mpidense, seq/mpiaij, shell) that I know for sure the fix seems to make sense. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

1 0

Re: [petsc-dev] Weird interaction between VecDuplicate and VecSetBlockSize
by Barry Smith 09 Apr '12

09 Apr '12

On Apr 9, 2012, at 2:20 PM, Lisandro Dalcin wrote: > On 9 April 2012 21:18, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >> >> On Mar 30, 2012, at 12:44 PM, Lisandro Dalcin wrote: >> >>> On 30 March 2012 19:34, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >>>> >>>> On Mar 30, 2012, at 11:07 AM, Lisandro Dalcin wrote: >>>> >>>>> On 30 March 2012 19:00, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >>> >>> Are you sure? How are we going to support the usage below?: >>> >>> VecCreateSeq(comm,n,&vec) >>> VecSetBlockSize(vec,bs) >> >> This is not longer supported. If you wish to set the blocksize of a vector you cannot use the convenience routines VecCreateSeq() or VecCreateMPI(). Nor can you can a Vec block size later. >> > > Not that I'm asking for it, but have you considered > VecCreateSeq(comm,bs,n,&vec) and VecCreateMPI(comm,&bs,n,N,&vec) ? > VecCreateXXXWithArray() was changed to take the block size... I considered that. The justification for changing the WithArray() is because there is no other way to set the block size (one cannot use VecCreate,VecSetSizes,.... ) and they are not commonly used while the VecCreateSeq() are commonly used so changing them will affect a lot of people and one can always use VecCreate, VecSetSizes,VecSetBlockSize, VecSetType. I am open to arguing about what the best interface is ... That is I am not claiming my decision is infallible. Barry > > > > -- > Lisandro Dalcin > --------------- > CIMEC (INTEC/CONICET-UNL) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1011) > Tel/Fax: +54-342-4511169

1 0

Re: [petsc-dev] Weird interaction between VecDuplicate and VecSetBlockSize
by Lisandro Dalcin 09 Apr '12

09 Apr '12

On 9 April 2012 21:18, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Mar 30, 2012, at 12:44 PM, Lisandro Dalcin wrote: > >> On 30 March 2012 19:34, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >>> >>> On Mar 30, 2012, at 11:07 AM, Lisandro Dalcin wrote: >>> >>>> On 30 March 2012 19:00, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >> >> Are you sure? How are we going to support the usage below?: >> >> VecCreateSeq(comm,n,&vec) >> VecSetBlockSize(vec,bs) > > This is not longer supported. If you wish to set the blocksize of a vector you cannot use the convenience routines VecCreateSeq() or VecCreateMPI(). Nor can you can a Vec block size later. > Not that I'm asking for it, but have you considered VecCreateSeq(comm,bs,n,&vec) and VecCreateMPI(comm,&bs,n,N,&vec) ? VecCreateXXXWithArray() was changed to take the block size... -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

1 0

Re: [petsc-dev] (no subject)
by Barry Smith 09 Apr '12

09 Apr '12

I cannot remember why this was required. I have removed this requirement, let's see how it goes; if we need to put it back make sure we add a developer note to the manual page explaining why. Note that it still requires the type and sizes to be set before calling to insure the function setoptions() for that type is set in the function table. Barry On Feb 23, 2012, at 1:15 PM, Lisandro Dalcin wrote: > MatSetOption() seems to require previous preallocation (I mean, it has > the MatCheckPreallocated() call). Isn't it too much? Do we really need > a previous preallocation in order to set an option? I agree that many > options are tightly related to nonzero pattern and memory allocation, > but many options can be safely set before preallocation. > > -- > Lisandro Dalcin > --------------- > CIMEC (INTEC/CONICET-UNL) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1011) > Tel/Fax: +54-342-4511169

1 0

Re: [petsc-dev] Weird interaction between VecDuplicate and VecSetBlockSize
by Barry Smith 09 Apr '12

09 Apr '12

On Mar 30, 2012, at 12:44 PM, Lisandro Dalcin wrote: > On 30 March 2012 19:34, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >> >> On Mar 30, 2012, at 11:07 AM, Lisandro Dalcin wrote: >> >>> On 30 March 2012 19:00, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: > > Are you sure? How are we going to support the usage below?: > > VecCreateSeq(comm,n,&vec) > VecSetBlockSize(vec,bs) This is not longer supported. If you wish to set the blocksize of a vector you cannot use the convenience routines VecCreateSeq() or VecCreateMPI(). Nor can you can a Vec block size later. Barry This prevents the mysterious changing of the block size in a layout shared by several vectors. > > We cannot certainly have a long-living vec with bs=-1, right? > > > -- > Lisandro Dalcin > --------------- > CIMEC (INTEC/CONICET-UNL) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1011) > Tel/Fax: +54-342-4511169

1 0

petsc4py import error on OS X: PETSc.so, 2): Symbol not found: _KSP_CLASSID
by Aron Ahmadia 08 Apr '12

08 Apr '12

If you are seeing this, it's because you installed petsc4py against a static PETSc. Rebuild the PETSc with shared libraries enabled and this error will go away. A

1 0

Re: [petsc-dev] PCApplyTranspose in BiCG
by Sean Farley 06 Apr '12

06 Apr '12

> > A change in tfqmr that broke it? A change in some underlying thing that > tfqmr that broke it. > > The "correct" way to track down the "bad" change is to us bisection with > mecurial http://www.selenic.com/mercurial/hg.1.html#bisect it lets you > quickly zoom into the change set that causes the change. Don't forget that since we don't use subrepositories you'll have to do a little bit extra to get buildsystem to work: http://www.mcs.anl.gov/petsc/documentation/faq.html#bisect

1 0

Re: [petsc-dev] PCApplyTranspose in BiCG
by Barry Smith 06 Apr '12

06 Apr '12

A change in tfqmr that broke it? A change in some underlying thing that tfqmr that broke it. The "correct" way to track down the "bad" change is to us bisection with mecurial http://www.selenic.com/mercurial/hg.1.html#bisect it lets you quickly zoom into the change set that causes the change. Barry On Mar 23, 2012, at 3:35 AM, Alexander Grayver wrote: > Barry, > > gmres produces the same result. > What does that mean? > > On 22.03.2012 21:10, Barry Smith wrote: >> Ok, first thing to confirm is if other KSP also produce different results. Run both exactly the same but use -ksp_type gmres on both; does it produce different results or the same? >> >> Barry >> >> On Mar 22, 2012, at 12:30 PM, Alexander Grayver wrote: >> >>> Jed, >>> >>> Ok, I see. Initially I've changed solver because I observed one thing. I have two petsc-dev (one is older than the other) versions compiled. >>> Running the same program with both gives different results: >>> >>> ./solveTest -ksp_view -ksp_converged_reason -ksp_monitor_true_residual -log_summary -mat_type aij -ksp_rtol 1.0e-7 -ksp_type tfqmr -pc_type sor -ksp_max_it 750 >>> 0 KSP preconditioned resid norm 9.760858225033e-10 true resid norm 4.108026443997e-08 ||r(i)||/||b|| 1.000000000000e+00 >>> 1 KSP preconditioned resid norm 9.497939702484e-10 true resid norm 3.591828545356e-08 ||r(i)||/||b|| 8.743440662618e-01 >>> 1 KSP preconditioned resid norm 1.341212527158e-09 true resid norm 3.754099340764e-08 ||r(i)||/||b|| 9.138449793207e-01 >>> 2 KSP preconditioned resid norm 9.481744679086e-10 true resid norm 3.710424003325e-08 ||r(i)||/||b|| 9.032132713620e-01 >>> 2 KSP preconditioned resid norm 1.340878639500e-09 true resid norm 3.730261027312e-08 ||r(i)||/||b|| 9.080421166135e-01 >>> 3 KSP preconditioned resid norm 9.481415577317e-10 true resid norm 3.725585425324e-08 ||r(i)||/||b|| 9.069039540306e-01 >>> ... >>> 750 KSP preconditioned resid norm 1.340874209607e-09 true resid norm 3.726506102951e-08 ||r(i)||/||b|| 9.071280708031e-01 >>> Linear solve did not converge due to DIVERGED_ITS iterations 750 >>> KSP Object: 1 MPI processes >>> type: tfqmr >>> maximum iterations=750, initial guess is zero >>> tolerances: relative=1e-07, absolute=1e-50, divergence=10000 >>> left preconditioning >>> using PRECONDITIONED norm type for convergence test >>> PC Object: 1 MPI processes >>> type: sor >>> SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 >>> linear system matrix = precond matrix: >>> Matrix Object: 1 MPI processes >>> type: seqaij >>> rows=1375920, cols=1375920 >>> total: nonzeros=17671714, allocated nonzeros=17671714 >>> total number of mallocs used during MatSetValues calls =0 >>> not using I-node routines >>> >>> Using Petsc Development HG revision: 59ca71b85d784abbe22f12c34ffcb2ccc6dc68b0 HG Date: Tue Mar 20 01:05:13 2012 -0500 >>> Configure options: --with-petsc-arch=openmpi-intel-complex-release-c --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-precision=double --with-blas-lapack-dir=/usr/local/lib --with-x=0 --with-debugging=0 >>> >>> ####################################### >>> And now second one: >>> ./solveTest -ksp_view -ksp_converged_reason -ksp_monitor_true_residual -log_summary -mat_type aij -ksp_rtol 1.0e-7 -ksp_type tfqmr -pc_type sor -ksp_max_it 750 >>> 0 KSP preconditioned resid norm 9.760858225034e-10 true resid norm 4.108026443997e-08 ||r(i)||/||b|| 1.000000000000e+00 >>> 1 KSP preconditioned resid norm 9.497930576113e-10 true resid norm 3.591633653854e-08 ||r(i)||/||b|| 8.742966246242e-01 >>> 1 KSP preconditioned resid norm 1.341211098002e-09 true resid norm 3.754049143693e-08 ||r(i)||/||b|| 9.138327600539e-01 >>> 2 KSP preconditioned resid norm 9.481662292281e-10 true resid norm 3.709648505700e-08 ||r(i)||/||b|| 9.030244951612e-01 >>> 2 KSP preconditioned resid norm 1.340863953080e-09 true resid norm 3.730358317575e-08 ||r(i)||/||b|| 9.080657995827e-01 >>> 3 KSP preconditioned resid norm 9.481308142928e-10 true resid norm 3.725036905334e-08 ||r(i)||/||b|| 9.067704300630e-01 >>> ... >>> 230 KSP preconditioned resid norm 8.875661094394e-17 true resid norm 1.062030983067e-14 ||r(i)||/||b|| 2.585258389996e-07 >>> Linear solve converged due to CONVERGED_RTOL iterations 230 >>> KSP Object: 1 MPI processes >>> type: tfqmr >>> maximum iterations=750, initial guess is zero >>> tolerances: relative=1e-07, absolute=1e-50, divergence=10000 >>> left preconditioning >>> using PRECONDITIONED norm type for convergence test >>> PC Object: 1 MPI processes >>> type: sor >>> SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 >>> linear system matrix = precond matrix: >>> Matrix Object: 1 MPI processes >>> type: seqaij >>> rows=1375920, cols=1375920 >>> total: nonzeros=17671714, allocated nonzeros=17671714 >>> total number of mallocs used during MatSetValues calls =0 >>> not using I-node routines >>> >>> Using Petsc Development HG revision: a4dae06b4d05ef5b78a9a9aa2cbc31b2b158834f HG Date: Wed Feb 08 11:29:52 2012 -0600 >>> Configure options: --with-petsc-arch=openmpi-intel-complex-release-f-mkl --with-fortran-interfaces=1 --download-superlu --download-superlu_dist --download-mumps --download-parmetis --download-ptscotch --download-metis --with-scalapack-lib=/opt/intel/Compiler/11.1/072/mkl/lib/em64t/libmkl_scalapack_lp64.a --with-scalapack-include=/opt/intel/Compiler/11.1/072/mkl/include --with-blacs-lib=/opt/intel/Compiler/11.1/072/mkl/lib/em64t/libmkl_blacs_openmpi_lp64.a --with-blacs-include=/opt/intel/Compiler/11.1/072/mkl/include --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-blas-lapack-lib="[/opt/intel/Compiler/11.1/072/mkl/lib/em64t/libmkl_intel_lp64.a,/opt/intel/Compiler/11.1/072/mkl/lib/em64t/libmkl_intel_thread.a,/opt/intel/Compiler/11.1/072/mkl/lib/em64t/libmkl_core.a,/opt/intel/Compiler/11.1/072/lib/intel64/libiomp5.a]" --with-precision=double --with-x=0 --with-debugging=0 >>> >>> >>> Well, two version compiled with different lines, but still I don't think this is the reason... >>> Where actually to look at? >>> >>> Thanks. >>> >>> On 22.03.2012 15:42, Jed Brown wrote: >>>> On Thu, Mar 22, 2012 at 04:23, Alexander Grayver<agrayver(a)gfz-potsdam.de> wrote: >>>> I've got the following error when applying bicg+sor to my problem: >>>> >>>> mt-soft/solveTest> ./solveTest -ksp_view -ksp_converged_reason -ksp_monitor_true_residual -log_summary -mat_type aij -ksp_rtol 1.0e-7 -ksp_type bicg -pc_type sor -ksp_max_it 750 >>>> [0]PETSC ERROR: --------------------- Error Message ------------------------------------ >>>> [0]PETSC ERROR: No support for this operation for this object type! >>>> [0]PETSC ERROR: PC does not have apply transpose! >>>> [0]PETSC ERROR: ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: Petsc Development HG revision: 59ca71b85d784abbe22f12c34ffcb2ccc6dc68b0 HG Date: Tue Mar 20 01:05:13 2012 -0500 >>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>>> [0]PETSC ERROR: See docs/index.html for manual pages. >>>> [0]PETSC ERROR: ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: Configure options --with-petsc-arch=openmpi-intel-complex-release-c --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-precision=double --with-blas-lapack-dir=/usr/local/lib --with-x=0 --with-debugging=0 >>>> [0]PETSC ERROR: ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: PCApplyTranspose() line 515 in /home/mt/agrayver/lib/petsc-dev/src/ksp/pc/interface/precon.c >>>> [0]PETSC ERROR: KSPSolve_BiCG() line 55 in /home/mt/agrayver/lib/petsc-dev/src/ksp/ksp/impls/bicg/bicg.c >>>> [0]PETSC ERROR: KSPSolve() line 434 in /home/mt/agrayver/lib/petsc-dev/src/ksp/ksp/interface/itfunc.c >>>> [0]PETSC ERROR: main() line 114 in /home/solveTest.c >>>> >>>> Any ideas? >>>> >>>> PCApplyTranspose_SOR is not implemented. It is unfortunately not an operation that is efficient to implement in-place like SOR. >>> >>> -- >>> Regards, >>> Alexander >>> > > > -- > Regards, > Alexander > >

1 0

Re: [petsc-dev] MatCreate from Fortran
by Alexander Grayver 05 Apr '12

05 Apr '12

Barry, ok, thanks! On 05.04.2012 18:11, Barry Smith wrote: > Sorry for the confusion. Those are now called matcreateaij() etc. because they have always actually created a seq object on one process and a parallel one on more then one, hence the old name was misleading. > > Barry > > > On Apr 5, 2012, at 11:07 AM, Alexander Grayver wrote: > >> Since last time I pulled petsc-dev (2 months) something probably changed with matrices creation of so, now I can't do it from Fortran: >> >> /home/main_mod.F90:(.text+0x4a3): undefined reference to `matcreatempiaij_' >> /home/main_mod.F90:(.text+0x5979): undefined reference to `matcreatempidense_' >> /home/main_mod.F90:(.text+0x59de): undefined reference to `matcreatempiaij_' >> >> Same program compiles just fine with: >> Using Petsc Development HG revision: 7c3c3f8c0d65b793aea33c138419de755b591ca7 HG Date: Thu Feb 09 14:42:30 2012 -0600 >> >> Something from my side or...? >> >> -- >> Regards, >> Alexander -- Regards, Alexander

1 0

Re: [petsc-dev] MatCreate from Fortran
by Barry Smith 05 Apr '12

05 Apr '12

Sorry for the confusion. Those are now called matcreateaij() etc. because they have always actually created a seq object on one process and a parallel one on more then one, hence the old name was misleading. Barry On Apr 5, 2012, at 11:07 AM, Alexander Grayver wrote: > Since last time I pulled petsc-dev (2 months) something probably changed with matrices creation of so, now I can't do it from Fortran: > > /home/main_mod.F90:(.text+0x4a3): undefined reference to `matcreatempiaij_' > /home/main_mod.F90:(.text+0x5979): undefined reference to `matcreatempidense_' > /home/main_mod.F90:(.text+0x59de): undefined reference to `matcreatempiaij_' > > Same program compiles just fine with: > Using Petsc Development HG revision: 7c3c3f8c0d65b793aea33c138419de755b591ca7 HG Date: Thu Feb 09 14:42:30 2012 -0600 > > Something from my side or...? > > -- > Regards, > Alexander

1 0