April 2011 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Fwd: [petsc-maint #69763] PETSc and OSKI
by Barry Smith 08 Apr '11

08 Apr '11

On Apr 8, 2011, at 8:09 AM, Matthew Knepley wrote: > On Fri, Apr 8, 2011 at 6:50 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Who put this in and does it do anything? > > Jed did. As far as I know, this project is dead. Then that file should be removed from the repository. Remember the old frat house rule, never leave dead carcasses lying around the frat house for more than a day. Barry > > Matt > > Barry > > > Begin forwarded message: > > > From: Kadir Akbudak <kadir(a)cs.bilkent.edu.tr> > > Date: April 8, 2011 6:42:40 AM CDT > > To: petsc-maint(a)mcs.anl.gov > > Subject: [petsc-maint #69763] PETSc and OSKI > > Reply-To: petsc-maint(a)mcs.anl.gov, Kadir Akbudak <kadir(a)cs.bilkent.edu.tr> > > > > Dear Sir or Madam, > > > > There is such a configuration script, > > petsc-3.1-p8/config/PETSc/packages/oski.py, in the PETSc distribution. > > Does this mean that PETSc uses OSKI sparse matrix-vector routines? If > > yes, is it possible to use shared libraries? It is written as: > > self.liblist = [['oski/liboski_Tid.a','oski/liboski.a']] This means > > the libraries are static. > > > > In fact I want to use benefits of OSKI together with my sparse matrix > > ordering tool, in a numerical method which can be performed by PETSc. > > > > Yours Faithfully, > > > > Kadir > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] Fwd: [petsc-maint #69763] PETSc and OSKI
by Matthew Knepley 08 Apr '11

08 Apr '11

On Fri, Apr 8, 2011 at 6:50 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Who put this in and does it do anything? > Jed did. As far as I know, this project is dead. Matt > Barry > > > Begin forwarded message: > > > From: Kadir Akbudak <kadir(a)cs.bilkent.edu.tr> > > Date: April 8, 2011 6:42:40 AM CDT > > To: petsc-maint(a)mcs.anl.gov > > Subject: [petsc-maint #69763] PETSc and OSKI > > Reply-To: petsc-maint(a)mcs.anl.gov, Kadir Akbudak < > kadir(a)cs.bilkent.edu.tr> > > > > Dear Sir or Madam, > > > > There is such a configuration script, > > petsc-3.1-p8/config/PETSc/packages/oski.py, in the PETSc distribution. > > Does this mean that PETSc uses OSKI sparse matrix-vector routines? If > > yes, is it possible to use shared libraries? It is written as: > > self.liblist = [['oski/liboski_Tid.a','oski/liboski.a']] This means > > the libraries are static. > > > > In fact I want to use benefits of OSKI together with my sparse matrix > > ordering tool, in a numerical method which can be performed by PETSc. > > > > Yours Faithfully, > > > > Kadir > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Fwd: [petsc-maint #69763] PETSc and OSKI
by Aron Ahmadia 08 Apr '11

08 Apr '11

I would bet a beer that Jed's Intel intrinsics beat the pants off of anything OSKI has to offer at this point. A On Fri, Apr 8, 2011 at 7:50 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Who put this in and does it do anything? > > Barry > > > Begin forwarded message: > > > From: Kadir Akbudak <kadir(a)cs.bilkent.edu.tr> > > Date: April 8, 2011 6:42:40 AM CDT > > To: petsc-maint(a)mcs.anl.gov > > Subject: [petsc-maint #69763] PETSc and OSKI > > Reply-To: petsc-maint(a)mcs.anl.gov, Kadir Akbudak < > kadir(a)cs.bilkent.edu.tr> > > > > Dear Sir or Madam, > > > > There is such a configuration script, > > petsc-3.1-p8/config/PETSc/packages/oski.py, in the PETSc distribution. > > Does this mean that PETSc uses OSKI sparse matrix-vector routines? If > > yes, is it possible to use shared libraries? It is written as: > > self.liblist = [['oski/liboski_Tid.a','oski/liboski.a']] This means > > the libraries are static. > > > > In fact I want to use benefits of OSKI together with my sparse matrix > > ordering tool, in a numerical method which can be performed by PETSc. > > > > Yours Faithfully, > > > > Kadir > > > >

1 0

Fwd: [petsc-maint #69763] PETSc and OSKI
by Barry Smith 08 Apr '11

08 Apr '11

Who put this in and does it do anything? Barry Begin forwarded message: > From: Kadir Akbudak <kadir(a)cs.bilkent.edu.tr> > Date: April 8, 2011 6:42:40 AM CDT > To: petsc-maint(a)mcs.anl.gov > Subject: [petsc-maint #69763] PETSc and OSKI > Reply-To: petsc-maint(a)mcs.anl.gov, Kadir Akbudak <kadir(a)cs.bilkent.edu.tr> > > Dear Sir or Madam, > > There is such a configuration script, > petsc-3.1-p8/config/PETSc/packages/oski.py, in the PETSc distribution. > Does this mean that PETSc uses OSKI sparse matrix-vector routines? If > yes, is it possible to use shared libraries? It is written as: > self.liblist = [['oski/liboski_Tid.a','oski/liboski.a']] This means > the libraries are static. > > In fact I want to use benefits of OSKI together with my sparse matrix > ordering tool, in a numerical method which can be performed by PETSc. > > Yours Faithfully, > > Kadir >

1 0

Re: [petsc-dev] Compile error with UMFPACK
by Satish Balay 07 Apr '11

07 Apr '11

retry after 'rm -rf osx-debug externalpackages' satish On Thu, 7 Apr 2011, Shri wrote:

1 0

Compile error with UMFPACK
by Shri 07 Apr '11

07 Apr '11

1 0

Re: [petsc-dev] Pflotran perf on GPU
by Lichtner, Peter C 07 Apr '11

07 Apr '11

The email from Jeswin is a little hard to decipher. Is it possible to send as an attached file? Also could we see the input file you are using. Thanks, ...Peter On Apr 6, 2011, at 5:04 PM, Barry Smith wrote: Jeswin, Thanks for the numbers. The linear solver is doing very well and you can really see Amdahl's Law kicking in, even when the KSPSolve is several times faster because of the GPU the whole application code is NOT several times faster because the function evaluation and Jacobian evaluations which are not on the GPU are hold the speedup down. Look for example at the row labeled KSPSolve() and the row labeled SNESSolve(). The reason the tracer code is not showing great improvement is because the problem is WAY WAY to small to run on the GPU. GPU's can only work well with lots and lots of data, this is a fact of life and cannot be changed by better software. This is very exciting, the GPU is really helping the linear solver as much as one could hope for. I'm glad you got it running, Barry On Apr 6, 2011, at 3:46 PM, Jeswin Godwin wrote: Petsc Team, I am student from Ohio-State University working in Hpcrl lab. --This shows the performance of Pflotran on the GPU. I am also attaching the –log_summary output for 1st case of both the example cases. --I built Petsc with-debugging=0. -- I did not use the preconditioner for any of the cases.... -- Would you people be able to explain the performance in the these cases. (i.e) The performance of 100_100_100 is better in the GPU case compared to the ID tracer. Performance of Pflotran on the GPU. example 100_100_100 No Gpu Gpu Nx Ny Ny time sec Gflops Time sec Gflops 32 32 32 4.58E+01 1.78E+08 3.82E+01 2.19E+08 64 64 64 6.83E+02 2.66E+08 3.64E+02 5.14E+08 96 96 96 3.56E+03 3.32E+08 1.27E+03 9.53E+08 example 1-D Tracer No Gpu Gpu Nx Ny Nz time sec Gflops Time sec Gflops 1000 1 1 3.43E+00 5.04E+08 2.62E+01 7.21E+07 5000 1 1 1.12E+02 6.51E+08 2.02E+02 3.74E+08 8000 1 1 3.97E+02 6.41E+08 4.73E+02 5.67E+08 Thank you, Jeswin <output_100_100_100.txt><output_seqcusp100_100_100.txt><output_seqcuspcuda.txt><output_tracer.txt> <><><><><><><><><><><><><><><><><><><><><> Peter C. Lichtner (lichtner(a)lanl.gov<mailto:[email protected]>) LANL EES-16: MS D469 (505) 667-3420 (o), 665-3285 (fax), 795-2881 (cell) Los Alamos National Laboratory SM-30 Bikini Atoll Road Los Alamos, NM 87545 http://www.ees.lanl.gov/pflotran/ <><><><><><><><><><><><><><><><><><><><><>

1 0

Fwd: Pflotran perf on GPU
by Barry Smith 07 Apr '11

07 Apr '11

I sent the following email, but forgot to forward the attachments, here they are. Jeswin, Thanks for the numbers. The linear solver is doing very well and you can really see Amdahl's Law kicking in, even when the KSPSolve is several times faster because of the GPU the whole application code is NOT several times faster because the function evaluation and Jacobian evaluations which are not on the GPU are hold the speedup down. Look for example at the row labeled KSPSolve() and the row labeled SNESSolve(). The reason the tracer code is not showing great improvement is because the problem is WAY WAY to small to run on the GPU. GPU's can only work well with lots and lots of data, this is a fact of life and cannot be changed by better software. This is very exciting, the GPU is really helping the linear solver as much as one could hope for. I'm glad you got it running, Barry Begin forwarded message: > From: Jeswin Godwin <jeswingodwin(a)gmail.com> > Date: April 6, 2011 3:46:55 PM CDT > To: Barry Smith <bsmith(a)mcs.anl.gov>, Victor Minden <victorminden(a)gmail.com>, Matthew Knepley <knepley(a)gmail.com> > Subject: Fwd: Pflotran perf on GPU > > > > Petsc Team, > > I am student from Ohio-State University working in Hpcrl lab. > > > > --This shows the performance of Pflotran on the GPU. > > I am also attaching the –log_summary output for 1st case of both the example cases. > > --I built Petsc with-debugging=0. > > -- I did not use the preconditioner for any of the cases.... > > -- Would you people be able to explain the performance in the these cases. (i.e) The performance of 100_100_100 is better in the GPU case compared to the ID tracer. > > > > > > > > Performance of Pflotran on the GPU. > > example 100_100_100 > > No Gpu > > Gpu > > Nx > > Ny > > Ny > > time sec > > Gflops > > Time sec > > Gflops > > 32 > > 32 > > 32 > > 4.58E+01 > > 1.78E+08 > > 3.82E+01 > > 2.19E+08 > > 64 > > 64 > > 64 > > 6.83E+02 > > 2.66E+08 > > 3.64E+02 > > 5.14E+08 > > 96 > > 96 > > 96 > > 3.56E+03 > > 3.32E+08 > > 1.27E+03 > > 9.53E+08 > > example 1-D Tracer > > No Gpu > > Gpu > > Nx > > Ny > > Nz > > time sec > > Gflops > > Time sec > > Gflops > > 1000 > > 1 > > 1 > > 3.43E+00 > > 5.04E+08 > > 2.62E+01 > > 7.21E+07 > > 5000 > > 1 > > 1 > > 1.12E+02 > > 6.51E+08 > > 2.02E+02 > > 3.74E+08 > > 8000 > > 1 > > 1 > > 3.97E+02 > > 6.41E+08 > > 4.73E+02 > > 5.67E+08 > > > > > > > > > > Thank you, > > Jeswin > > >

1 0

Re: [petsc-dev] Preconditioner for Schur complement (pcfieldsplit)
by Thomas Witkowski 07 Apr '11

07 Apr '11

Jed Brown wrote: > On Thu, Apr 7, 2011 at 09:42, Thomas Witkowski > <thomas.witkowski(a)tu-dresden.de > <mailto:[email protected]>> wrote: > > The problem is that the data field (vec->data) of the nested > vectors is empty. I created the nested vectors exactly in the same > way as in /src/ksp/ksp/examples/tests/ex22.c. When I would make a > VecGetArray(myvec, &vecptr) on a nested vector myvec, than vecptr > is not a vailde pointer. > > > Indeed, this is why this test uses no preconditioner. I do not > recommend VecNest for general use, there are many places where it will > be problematic. So go ahead and use the MatNest, but just create a > normal Vec. Great, now it works fine! Thomas > > Eventually, I want to remove the explicit scatters in the PCFieldSplit > implementation in favor of VecGetSubVector(). When that is done, > VecNest will provide a way to eliminate some copies (at the expense of > memory layout so it will still not always be faster), but its use > should still remain hidden from user code. Don't bother with it now.

1 0

Re: [petsc-dev] Preconditioner for Schur complement (pcfieldsplit)
by Jed Brown 07 Apr '11

07 Apr '11

On Thu, Apr 7, 2011 at 09:42, Thomas Witkowski < thomas.witkowski(a)tu-dresden.de> wrote: > The problem is that the data field (vec->data) of the nested vectors is > empty. I created the nested vectors exactly in the same way as in > /src/ksp/ksp/examples/tests/ex22.c. When I would make a VecGetArray(myvec, > &vecptr) on a nested vector myvec, than vecptr is not a vailde pointer. > Indeed, this is why this test uses no preconditioner. I do not recommend VecNest for general use, there are many places where it will be problematic. So go ahead and use the MatNest, but just create a normal Vec. Eventually, I want to remove the explicit scatters in the PCFieldSplit implementation in favor of VecGetSubVector(). When that is done, VecNest will provide a way to eliminate some copies (at the expense of memory layout so it will still not always be faster), but its use should still remain hidden from user code. Don't bother with it now.

1 0