March 2013 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] GAMG MatMatMult error on 1 core
by Jed Brown 29 Mar '13

29 Mar '13

On Fri, Mar 29, 2013 at 10:44 AM, John Mousel <elafint.john(a)gmail.com>wrote: > I switched the arguments to > > CALL > MatXAIJSetPreallocation(Ap,1,0,dnnz,0,onnz,PETSC_NULL_OBJECT,PETSC_NULL_OBJECT,ierr) > You're passing too many arguments. http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatXAIJSetPreal… When I first introduced MatXAIJSetPreallocation, it had more arguments, but we trimmed it down early on. > > but it doesn't seem to like that; > > [0]PETSC ERROR: MatSetValues() line 1075 in > /home/jmousel/NumericalLibraries/petsc-hg/src/mat/interface/matrix.c > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Object is in wrong state! > [0]PETSC ERROR: Must call MatXXXSetPreallocation() or MatSetUp() on > argument 1 "mat" before MatSetValues()! > > > > On Fri, Mar 29, 2013 at 10:42 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > >> On Fri, Mar 29, 2013 at 10:31 AM, John Mousel <elafint.john(a)gmail.com>wrote: >> >>> So this >>> >>> Ap => proj%A >>> CALL MatCreate(solver%comm,Ap,ierr) >>> CALL MatSetSizes(Ap,nlp,nlp,ngp,ngp,ierr) >>> CALL MatSetType(Ap,MATMPIAIJ,ierr) >>> CALL MatMPIAIJSetPreallocation(Ap,0,dnnz,0,onnz,ierr) >>> >>> should go to this >>> >>> Ap => proj%A >>> CALL MatCreate(solver%comm,Ap,ierr) >>> CALL MatSetSizes(Ap,nlp,nlp,ngp,ngp,ierr) >>> CALL MatSetType(Ap,MATAIJ,ierr) >>> >> >> Yes >> >> >>> CALL MatXAIJSetPreallocation(Ap,0,dnnz,0,onnz,ierr) >>> >> >> Almost, see the actual arguments to MatXAIJSetPreallocation. >> >> >>> >>> ? >>> >>> >>> >>> >>> On Fri, Mar 29, 2013 at 10:25 AM, Jed Brown <jedbrown(a)mcs.anl.gov>wrote: >>> >>>> On Fri, Mar 29, 2013 at 10:22 AM, John Mousel <john.mousel(a)gmail.com>wrote: >>>> >>>>> I get the following error when I try to use GAMG on 1 core. I just >>>>> pulled dev and it still exists. >>>>> >>>> >>>> Are you explicitly setting your matrix type to MPIAIJ? Can you use >>>> MATAIJ instead (it will choose the correct type based on size)? >>>> >>>> >>>>> >>>>> John >>>>> >>>>> [0]PETSC ERROR: --------------------- Error Message >>>>> ------------------------------------ >>>>> [0]PETSC ERROR: Arguments are incompatible! >>>>> [0]PETSC ERROR: MatMatMult requires A, mpiaij, to be compatible with >>>>> B, seqaij! >>>>> [0]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: >>>>> unknown >>>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>>>> [0]PETSC ERROR: See docs/index.html for manual pages. >>>>> [0]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [0]PETSC ERROR: ../src/MUSSEL_exe on a intel-debug named >>>>> wv.iihr.uiowa.edu by jmousel Fri Mar 29 10:19:44 2013 >>>>> [0]PETSC ERROR: Libraries linked from >>>>> /home/jmousel/NumericalLibraries/petsc-hg/intel-debug/lib >>>>> [0]PETSC ERROR: Configure run at Fri Mar 29 10:00:56 2013 >>>>> [0]PETSC ERROR: Configure options --download-blacs=1 >>>>> --download-metis=1 --download-ml=1 --download-mpich=1 >>>>> --download-parmetis=/home/jmousel/NumericalLibraries/petsc-hg/parmetis-4.0.2-p3.tar.gz >>>>> --download-f-blas-lapack=1 --with-cc=icc --download-cmake=1 --with-cxx=icpc >>>>> --with-fc=ifort >>>>> [0]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [0]PETSC ERROR: MatMatMult() line 8567 in >>>>> /home/jmousel/NumericalLibraries/petsc-hg/src/mat/interface/matrix.c >>>>> [0]PETSC ERROR: PCGAMGOptprol_AGG() line 1335 in >>>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/agg.c >>>>> [0]PETSC ERROR: PCSetUp_GAMG() line 680 in >>>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/gamg.c >>>>> [0]PETSC ERROR: PCSetUp() line 890 in >>>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/interface/precon.c >>>>> [0]PETSC ERROR: KSPSetUp() line 278 in >>>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c >>>>> [0]PETSC ERROR: KSPSolve() line 399 in >>>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c >>>>> >>>>> >>>> >>> >> >

1 0

Re: [petsc-dev] GAMG MatMatMult error on 1 core
by John Mousel 29 Mar '13

29 Mar '13

I switched the arguments to CALL MatXAIJSetPreallocation(Ap,1,0,dnnz,0,onnz,PETSC_NULL_OBJECT,PETSC_NULL_OBJECT,ierr) but it doesn't seem to like that; [0]PETSC ERROR: MatSetValues() line 1075 in /home/jmousel/NumericalLibraries/petsc-hg/src/mat/interface/matrix.c [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Object is in wrong state! [0]PETSC ERROR: Must call MatXXXSetPreallocation() or MatSetUp() on argument 1 "mat" before MatSetValues()! On Fri, Mar 29, 2013 at 10:42 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Fri, Mar 29, 2013 at 10:31 AM, John Mousel <elafint.john(a)gmail.com>wrote: > >> So this >> >> Ap => proj%A >> CALL MatCreate(solver%comm,Ap,ierr) >> CALL MatSetSizes(Ap,nlp,nlp,ngp,ngp,ierr) >> CALL MatSetType(Ap,MATMPIAIJ,ierr) >> CALL MatMPIAIJSetPreallocation(Ap,0,dnnz,0,onnz,ierr) >> >> should go to this >> >> Ap => proj%A >> CALL MatCreate(solver%comm,Ap,ierr) >> CALL MatSetSizes(Ap,nlp,nlp,ngp,ngp,ierr) >> CALL MatSetType(Ap,MATAIJ,ierr) >> > > Yes > > >> CALL MatXAIJSetPreallocation(Ap,0,dnnz,0,onnz,ierr) >> > > Almost, see the actual arguments to MatXAIJSetPreallocation. > > >> >> ? >> >> >> >> >> On Fri, Mar 29, 2013 at 10:25 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: >> >>> On Fri, Mar 29, 2013 at 10:22 AM, John Mousel <john.mousel(a)gmail.com>wrote: >>> >>>> I get the following error when I try to use GAMG on 1 core. I just >>>> pulled dev and it still exists. >>>> >>> >>> Are you explicitly setting your matrix type to MPIAIJ? Can you use >>> MATAIJ instead (it will choose the correct type based on size)? >>> >>> >>>> >>>> John >>>> >>>> [0]PETSC ERROR: --------------------- Error Message >>>> ------------------------------------ >>>> [0]PETSC ERROR: Arguments are incompatible! >>>> [0]PETSC ERROR: MatMatMult requires A, mpiaij, to be compatible with B, >>>> seqaij! >>>> [0]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: >>>> unknown >>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>>> [0]PETSC ERROR: See docs/index.html for manual pages. >>>> [0]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: ../src/MUSSEL_exe on a intel-debug named >>>> wv.iihr.uiowa.edu by jmousel Fri Mar 29 10:19:44 2013 >>>> [0]PETSC ERROR: Libraries linked from >>>> /home/jmousel/NumericalLibraries/petsc-hg/intel-debug/lib >>>> [0]PETSC ERROR: Configure run at Fri Mar 29 10:00:56 2013 >>>> [0]PETSC ERROR: Configure options --download-blacs=1 --download-metis=1 >>>> --download-ml=1 --download-mpich=1 >>>> --download-parmetis=/home/jmousel/NumericalLibraries/petsc-hg/parmetis-4.0.2-p3.tar.gz >>>> --download-f-blas-lapack=1 --with-cc=icc --download-cmake=1 --with-cxx=icpc >>>> --with-fc=ifort >>>> [0]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: MatMatMult() line 8567 in >>>> /home/jmousel/NumericalLibraries/petsc-hg/src/mat/interface/matrix.c >>>> [0]PETSC ERROR: PCGAMGOptprol_AGG() line 1335 in >>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/agg.c >>>> [0]PETSC ERROR: PCSetUp_GAMG() line 680 in >>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/gamg.c >>>> [0]PETSC ERROR: PCSetUp() line 890 in >>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/interface/precon.c >>>> [0]PETSC ERROR: KSPSetUp() line 278 in >>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c >>>> [0]PETSC ERROR: KSPSolve() line 399 in >>>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c >>>> >>>> >>> >> >

1 0

Re: [petsc-dev] GAMG MatMatMult error on 1 core
by Jed Brown 29 Mar '13

29 Mar '13

On Fri, Mar 29, 2013 at 10:31 AM, John Mousel <elafint.john(a)gmail.com>wrote: > So this > > Ap => proj%A > CALL MatCreate(solver%comm,Ap,ierr) > CALL MatSetSizes(Ap,nlp,nlp,ngp,ngp,ierr) > CALL MatSetType(Ap,MATMPIAIJ,ierr) > CALL MatMPIAIJSetPreallocation(Ap,0,dnnz,0,onnz,ierr) > > should go to this > > Ap => proj%A > CALL MatCreate(solver%comm,Ap,ierr) > CALL MatSetSizes(Ap,nlp,nlp,ngp,ngp,ierr) > CALL MatSetType(Ap,MATAIJ,ierr) > Yes > CALL MatXAIJSetPreallocation(Ap,0,dnnz,0,onnz,ierr) > Almost, see the actual arguments to MatXAIJSetPreallocation. > > ? > > > > > On Fri, Mar 29, 2013 at 10:25 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > >> On Fri, Mar 29, 2013 at 10:22 AM, John Mousel <john.mousel(a)gmail.com>wrote: >> >>> I get the following error when I try to use GAMG on 1 core. I just >>> pulled dev and it still exists. >>> >> >> Are you explicitly setting your matrix type to MPIAIJ? Can you use MATAIJ >> instead (it will choose the correct type based on size)? >> >> >>> >>> John >>> >>> [0]PETSC ERROR: --------------------- Error Message >>> ------------------------------------ >>> [0]PETSC ERROR: Arguments are incompatible! >>> [0]PETSC ERROR: MatMatMult requires A, mpiaij, to be compatible with B, >>> seqaij! >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: >>> unknown >>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>> [0]PETSC ERROR: See docs/index.html for manual pages. >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: ../src/MUSSEL_exe on a intel-debug named >>> wv.iihr.uiowa.edu by jmousel Fri Mar 29 10:19:44 2013 >>> [0]PETSC ERROR: Libraries linked from >>> /home/jmousel/NumericalLibraries/petsc-hg/intel-debug/lib >>> [0]PETSC ERROR: Configure run at Fri Mar 29 10:00:56 2013 >>> [0]PETSC ERROR: Configure options --download-blacs=1 --download-metis=1 >>> --download-ml=1 --download-mpich=1 >>> --download-parmetis=/home/jmousel/NumericalLibraries/petsc-hg/parmetis-4.0.2-p3.tar.gz >>> --download-f-blas-lapack=1 --with-cc=icc --download-cmake=1 --with-cxx=icpc >>> --with-fc=ifort >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: MatMatMult() line 8567 in >>> /home/jmousel/NumericalLibraries/petsc-hg/src/mat/interface/matrix.c >>> [0]PETSC ERROR: PCGAMGOptprol_AGG() line 1335 in >>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/agg.c >>> [0]PETSC ERROR: PCSetUp_GAMG() line 680 in >>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/gamg.c >>> [0]PETSC ERROR: PCSetUp() line 890 in >>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/interface/precon.c >>> [0]PETSC ERROR: KSPSetUp() line 278 in >>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c >>> [0]PETSC ERROR: KSPSolve() line 399 in >>> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c >>> >>> >> >

1 0

Re: [petsc-dev] GAMG MatMatMult error on 1 core
by John Mousel 29 Mar '13

29 Mar '13

So this Ap => proj%A CALL MatCreate(solver%comm,Ap,ierr) CALL MatSetSizes(Ap,nlp,nlp,ngp,ngp,ierr) CALL MatSetType(Ap,MATMPIAIJ,ierr) CALL MatMPIAIJSetPreallocation(Ap,0,dnnz,0,onnz,ierr) should go to this Ap => proj%A CALL MatCreate(solver%comm,Ap,ierr) CALL MatSetSizes(Ap,nlp,nlp,ngp,ngp,ierr) CALL MatSetType(Ap,MATAIJ,ierr) CALL MatXAIJSetPreallocation(Ap,0,dnnz,0,onnz,ierr) ? On Fri, Mar 29, 2013 at 10:25 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Fri, Mar 29, 2013 at 10:22 AM, John Mousel <john.mousel(a)gmail.com>wrote: > >> I get the following error when I try to use GAMG on 1 core. I just pulled >> dev and it still exists. >> > > Are you explicitly setting your matrix type to MPIAIJ? Can you use MATAIJ > instead (it will choose the correct type based on size)? > > >> >> John >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Arguments are incompatible! >> [0]PETSC ERROR: MatMatMult requires A, mpiaij, to be compatible with B, >> seqaij! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: unknown >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ../src/MUSSEL_exe on a intel-debug named >> wv.iihr.uiowa.edu by jmousel Fri Mar 29 10:19:44 2013 >> [0]PETSC ERROR: Libraries linked from >> /home/jmousel/NumericalLibraries/petsc-hg/intel-debug/lib >> [0]PETSC ERROR: Configure run at Fri Mar 29 10:00:56 2013 >> [0]PETSC ERROR: Configure options --download-blacs=1 --download-metis=1 >> --download-ml=1 --download-mpich=1 >> --download-parmetis=/home/jmousel/NumericalLibraries/petsc-hg/parmetis-4.0.2-p3.tar.gz >> --download-f-blas-lapack=1 --with-cc=icc --download-cmake=1 --with-cxx=icpc >> --with-fc=ifort >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: MatMatMult() line 8567 in >> /home/jmousel/NumericalLibraries/petsc-hg/src/mat/interface/matrix.c >> [0]PETSC ERROR: PCGAMGOptprol_AGG() line 1335 in >> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/agg.c >> [0]PETSC ERROR: PCSetUp_GAMG() line 680 in >> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/gamg.c >> [0]PETSC ERROR: PCSetUp() line 890 in >> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/interface/precon.c >> [0]PETSC ERROR: KSPSetUp() line 278 in >> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c >> [0]PETSC ERROR: KSPSolve() line 399 in >> /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c >> >> >

1 0

Re: [petsc-dev] GAMG MatMatMult error on 1 core
by Jed Brown 29 Mar '13

29 Mar '13

On Fri, Mar 29, 2013 at 10:22 AM, John Mousel <john.mousel(a)gmail.com> wrote: > I get the following error when I try to use GAMG on 1 core. I just pulled > dev and it still exists. > Are you explicitly setting your matrix type to MPIAIJ? Can you use MATAIJ instead (it will choose the correct type based on size)? > > John > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Arguments are incompatible! > [0]PETSC ERROR: MatMatMult requires A, mpiaij, to be compatible with B, > seqaij! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: unknown > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ../src/MUSSEL_exe on a intel-debug named wv.iihr.uiowa.eduby jmousel Fri Mar 29 10:19:44 2013 > [0]PETSC ERROR: Libraries linked from > /home/jmousel/NumericalLibraries/petsc-hg/intel-debug/lib > [0]PETSC ERROR: Configure run at Fri Mar 29 10:00:56 2013 > [0]PETSC ERROR: Configure options --download-blacs=1 --download-metis=1 > --download-ml=1 --download-mpich=1 > --download-parmetis=/home/jmousel/NumericalLibraries/petsc-hg/parmetis-4.0.2-p3.tar.gz > --download-f-blas-lapack=1 --with-cc=icc --download-cmake=1 --with-cxx=icpc > --with-fc=ifort > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: MatMatMult() line 8567 in > /home/jmousel/NumericalLibraries/petsc-hg/src/mat/interface/matrix.c > [0]PETSC ERROR: PCGAMGOptprol_AGG() line 1335 in > /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/agg.c > [0]PETSC ERROR: PCSetUp_GAMG() line 680 in > /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/gamg.c > [0]PETSC ERROR: PCSetUp() line 890 in > /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPSetUp() line 278 in > /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: KSPSolve() line 399 in > /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c > >

1 0

GAMG MatMatMult error on 1 core
by John Mousel 29 Mar '13

29 Mar '13

I get the following error when I try to use GAMG on 1 core. I just pulled dev and it still exists. John [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Arguments are incompatible! [0]PETSC ERROR: MatMatMult requires A, mpiaij, to be compatible with B, seqaij! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: unknown [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ../src/MUSSEL_exe on a intel-debug named wv.iihr.uiowa.eduby jmousel Fri Mar 29 10:19:44 2013 [0]PETSC ERROR: Libraries linked from /home/jmousel/NumericalLibraries/petsc-hg/intel-debug/lib [0]PETSC ERROR: Configure run at Fri Mar 29 10:00:56 2013 [0]PETSC ERROR: Configure options --download-blacs=1 --download-metis=1 --download-ml=1 --download-mpich=1 --download-parmetis=/home/jmousel/NumericalLibraries/petsc-hg/parmetis-4.0.2-p3.tar.gz --download-f-blas-lapack=1 --with-cc=icc --download-cmake=1 --with-cxx=icpc --with-fc=ifort [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatMatMult() line 8567 in /home/jmousel/NumericalLibraries/petsc-hg/src/mat/interface/matrix.c [0]PETSC ERROR: PCGAMGOptprol_AGG() line 1335 in /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/agg.c [0]PETSC ERROR: PCSetUp_GAMG() line 680 in /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/impls/gamg/gamg.c [0]PETSC ERROR: PCSetUp() line 890 in /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 278 in /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: KSPSolve() line 399 in /home/jmousel/NumericalLibraries/petsc-hg/src/ksp/ksp/interface/itfunc.c

1 0

Re: [petsc-dev] VecMDot_SeqCUSP improved
by Karl Rupp 29 Mar '13

29 Mar '13

Hi Jose, > Here are my numbers for this size. They are similar to yours (a bit worse, though). Also, I tried with ViennaCL which gave very poor performance (is this normal?). Ok, good, this is consistent then. The ViennaCL-performance is indeed rather poor in this setting. This is partially because of the mdot() operation not being optimized yet (just using an iterated application of dot()). Nevertheless, the factor of ~2 compared to CUSP is due to some additional latency overhead when doing CPU<->GPU transfer of OpenCL vs. CUDA. I also see this in full CG benchmarks, cf. the two green curves here: http://viennacl.sourceforge.net/uploads/pics/cg-timings.png Only at sizes of 10^6 the CUDA and OpenCL curves finally meet. I assume that NVIDIA could do better with their OpenCL driver if they wanted, but it's more beneficial for them to drive their customers towards CUDA - just speculation, of course... > > I tried a full SLEPc computation, with a matrix of order 256,000 and making VecMDot operate on 40 vectors. The gain from 'master' to 'next' is 91 seconds to 53 seconds. So, yes it is good improvement. Thanks. Cool, glad to see that this pays off well :-) > However, I still see only a modest speedup (about 4) with respect to CPU (since we do some optimizations for the CPU). Also, performance depends a lot on the different matrix dimensions. I have to figure out how to optimize it more for the GPU as well. I think that the current mdot() implementation can be tweaked by a few additional percent for larger values of k. On the other hand, I would not expect more than a factor of ~5 performance gain over the CPU, particularly if you really compare against a well-optimized CPU implementation. The memory bandwidth on GPUs is (usually) less than a factor of 10 larger, particularly if you have three or four DRAM channels for the CPU, plus it's harder to get full bandwidth utilization. Best regards, Karli

1 0

Re: [petsc-dev] VecMDot_SeqCUSP improved
by Jose E. Roman 28 Mar '13

28 Mar '13

El 26/03/2013, a las 20:15, Karl Rupp escribió: > Hi Jose, > > here's the benchmark data obtained on my local machine running an NVIDIA GTX 285 for vectors of size 100k: > > # Master > > ./ex43 -n 100000 -k 200 -mdot -log_summary > VecMDot 5.6363e+01 > > ./ex43 -n 100000 -k 200 -mdot -log_summary -vec_type cusp > VecMDot 2.1936e+01 > > ./ex43 -n 100000 -k 200 -log_summary > VecDot 5.1124e+01 > > ./ex43 -n 100000 -k 200 -log_summary -vec_type cusp > VecDot 4.0968e+01 > > > # Next > > ./ex43 -n 100000 -k 200 -mdot -log_summary > VecMDot 5.6417e+01 > > ./ex43 -n 100000 -k 200 -mdot -log_summary -vec_type cusp > VecMDot 1.0281e+01 > > ./ex43 -n 100000 -k 200 -log_summary > VecDot 5.0886e+01 > > ./ex43 -n 100000 -k 200 -log_summary -vec_type cusp > VecDot 4.1905e+01 > > > This makes 10sec in next vs. 20 sec. on master for the CUDA-accelerated mdot-case. The factor of two is actually as expected, because in the 'old' kernel the data movement is twice as what it is in the custom kernel version. The factor of four with respect to VecDot is not entirely clear to me, I'd rather expect a factor close to 2. Presumably the more frequent host <-> device transfers add extra overhead. > > Best regards, > Karli Here are my numbers for this size. They are similar to yours (a bit worse, though). Also, I tried with ViennaCL which gave very poor performance (is this normal?). # Master ./ex43 -n 100000 -k 200 -mdot -log_summary VecMDot 4.0681e+01 ./ex43 -n 100000 -k 200 -mdot -log_summary -vec_type cusp VecMDot 2.4489e+01 ./ex43 -n 100000 -k 200 -log_summary VecDot 5.9457e+01 ./ex43 -n 100000 -k 200 -log_summary -vec_type cusp VecDot 5.0021e+01 # Next ./ex43 -n 100000 -k 200 -mdot -log_summary VecMDot 4.4252e+01 ./ex43 -n 100000 -k 200 -mdot -log_summary -vec_type cusp VecMDot 1.2176e+01 ./ex43 -n 100000 -k 200 -log_summary VecDot 5.9847e+01 ./ex43 -n 100000 -k 200 -log_summary -vec_type cusp VecDot 5.0080e+01 # ViennaCL ./ex43 -n 100000 -k 200 -mdot -log_summary -vec_type viennacl VecMDot 9.4478e+01 ./ex43 -n 100000 -k 200 -log_summary -vec_type viennacl VecMDot 1.2311e+02 I tried a full SLEPc computation, with a matrix of order 256,000 and making VecMDot operate on 40 vectors. The gain from 'master' to 'next' is 91 seconds to 53 seconds. So, yes it is good improvement. Thanks. However, I still see only a modest speedup (about 4) with respect to CPU (since we do some optimizations for the CPU). Also, performance depends a lot on the different matrix dimensions. I have to figure out how to optimize it more for the GPU as well. Jose

1 0

Re: [petsc-dev] ViennaCL-bindings in next
by Karl Rupp 28 Mar '13

28 Mar '13

Hi Dave, yes, OpenCL is much less picky about versions because it is a .so only. No separate compiler(s), just the standardized headers and the vendor-provided libOpenCL.so, which (in the case of GPUs) even come with the driver. No need for an SDK. MUCH cleaner from a maintenance point of view :-) (just Apple being a bit 'special') Best regards, Karli On 03/27/2013 05:33 PM, Nystrom, William D wrote: > Karli, > > Would the OpenCL headers and libraries that come with Cuda 5.0 be adequate? > > Thanks, > > Dave > > ________________________________________ > From: petsc-dev-bounces(a)mcs.anl.gov [petsc-dev-bounces(a)mcs.anl.gov] on behalf of Karl Rupp [rupp(a)mcs.anl.gov] > Sent: Thursday, March 21, 2013 5:50 PM > To: For users of the development version of PETSc > Subject: [petsc-dev] ViennaCL-bindings in next > > Hi, > > the interface to ViennaCL [1] has just been merged to 'next', enabling > PETSc to use OpenCL-enabled devices (most notably Intel's Xeon Phi and > GPUs from AMD in addition to the existing NVIDIA bindings). I tested it > with Vector-, Matrix-, and KSP- examples. > > The new bindings are enabled by the configure flags > --download-viennacl > --with-opencl-include=/path/to/OpenCL-Headers > --with-opencl-lib=/path/to/libOpenCL.so > > The OpenCL headers can be obtained on the Khronos webpage [2] and are > often available via package managers, e.g. > $> sudo apt-get install opencl-headers > or > $> sudo yam install opencl-headers > In such case, the OpenCL include directory is typically /usr/include > (BuildSystem may not detect these headers correctly on Mac OS, I'm > working on this). The OpenCL library comes with your graphics driver and > is usually located at /usr/lib/. There is no separate compiler as for > CUDA needed! > > FAQ: > > - What is the performance difference between OpenCL and CUDA for GPUs? > My experience is that for large problem sizes they are equivalent for > almost all linear algebra kernels. For small problem sizes (kernel > execution less than 1ms), CUDA is currently superior because of lower > latency. > > - Which solvers can I use with the ViennaCL-bindings? > The same as with CUSP. Most notably the usual CG, BiCGS, GMRES. > > - Which preconditioners are provided in the ViennaCL-bindings? > Currently none. You can, however, use all the standard CPU-based > preconditioners in PETSc together with ViennaCL matrices and vectors. > > - Which command line options do I need to use? > Use -vec_type viennacl -mat_type aijviennacl > On Intel's Xeon Phi you also need to pass -viennacl_device_accelerator. > (If you read this email in an archive: Check the manual for current flags). > > Best regards, > Karli > > [1] http://viennacl.sourceforge.net/ > [2] http://www.khronos.org/registry/cl/ >

1 0

Re: [petsc-dev] CMake-assisted CUDA builds ready
by Karl Rupp 28 Mar '13

28 Mar '13

Hi Dave, thanks for letting me know, I'll merge it to master then. Best regards, Karli On 03/27/2013 04:47 PM, Nystrom, William D wrote: > I verified that it works for me also. > > Thanks, > > Dave > > ________________________________________ > From: petsc-dev-bounces(a)mcs.anl.gov [petsc-dev-bounces(a)mcs.anl.gov] on behalf of Jose E. Roman [jroman(a)dsic.upv.es] > Sent: Tuesday, March 26, 2013 3:44 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] CMake-assisted CUDA builds ready > > El 26/03/2013, a las 21:08, Karl Rupp escribiï¿½: > >> Hi again, >> >> thanks, I think I could eliminate the cause. >> Please give 'next' another try... >> >> Thanks and best regards, >> Karli > > Yes, it works now. Thanks. >

1 0