July 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Jack has produced a scalable sparse direct solver
by Jack Poulson 23 Jul '12

23 Jul '12

Hi Hong, On Mon, Jul 23, 2012 at 11:53 AM, Hong Zhang <hzhang(a)mcs.anl.gov> wrote: > Jack: > >> >> I'm getting started on creating documentation right now; it should end up >> looking very similar to Elemental's. >> > > The detailed instruction will be very helpful, especially > the sparse matrix data structures. Notify us when the documentation is out. > Will do. I just pushed out documentation for the build system: http://poulson.github.com/Clique/build.html The rest should follow soon. > Is this a sparse Cholesky direct solver? > More or less. It currently supports LDL^T and LDL^H factorizations without pivoting, and the latter certainly works for Hermitian positive-definite matrices. The current implementation is slightly more general than Cholesky and should achieve roughly the same performance. > Currently, only mumps supports parallel sparse Cholesky direct solver. > Yours will be a good addition to the petsc external solvers. > Thanks! I'll do my best to help with the integration when the time comes. > >> Please keep in mind that there is not _yet_ any support for pivoting, as >> I needed Clique's functionality for indefinite complex symmetric matrices >> which are nice enough to be factored accurately without pivoting. > > > Petsc users likely use it as a preconditioner. > In case of zero pivot, you may provide a routine/option for adding a small > shift? > Yes, that is what I was thinking. For instance, for Helmholtz equations, one can get away with very small positive imaginary shifts (the physically corresponds to slightly damping waves). > >> Clique's nested dissection is built on top of a nodal graph bisection >> routine which uses a custom interface to parmetis; I suspect that the graph >> partitioning is currently the least scalable part of the black-box >> interface due to KLFM refinement being so hard to parallelize. > > > Xuan has almost finished the petsc-elemental interface > (see petsc-dev/src/mat/impls/elemental/). > We would appreciate if you can take a look at it and give us > your comments/suggests. > > I just took a quick look and will make time to go over it in detail later today. I'll send out patch if I have any concrete suggestions. Jack >> >> On Mon, Jul 23, 2012 at 10:12 AM, Hong Zhang <hzhang(a)mcs.anl.gov> wrote: >> >>> Xuan will take a look at this. >>> Hong >>> >>> On Mon, Jul 23, 2012 at 9:47 AM, Matthew Knepley <knepley(a)gmail.com>wrote: >>> >>>> I think we just need a small converter from AIJ: >>>> >>>> >>>> https://bitbucket.org/poulson/clique/src/dc417c7e9403/tests/DistSparseMatri… >>>> >>>> Matt >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >> >

1 0

Re: [petsc-dev] Jack has produced a scalable sparse direct solver
by Hong Zhang 23 Jul '12

23 Jul '12

Jack: > > I'm getting started on creating documentation right now; it should end up > looking very similar to Elemental's. > The detailed instruction will be very helpful, especially the sparse matrix data structures. Notify us when the documentation is out. Is this a sparse Cholesky direct solver? Currently, only mumps supports parallel sparse Cholesky direct solver. Yours will be a good addition to the petsc external solvers. > > Please keep in mind that there is not _yet_ any support for pivoting, as I > needed Clique's functionality for indefinite complex symmetric matrices > which are nice enough to be factored accurately without pivoting. Petsc users likely use it as a preconditioner. In case of zero pivot, you may provide a routine/option for adding a small shift? > > Clique's nested dissection is built on top of a nodal graph bisection > routine which uses a custom interface to parmetis; I suspect that the graph > partitioning is currently the least scalable part of the black-box > interface due to KLFM refinement being so hard to parallelize. Xuan has almost finished the petsc-elemental interface (see petsc-dev/src/mat/impls/elemental/). We would appreciate if you can take a look at it and give us your comments/suggests. Hong > > > On Mon, Jul 23, 2012 at 10:12 AM, Hong Zhang <hzhang(a)mcs.anl.gov> wrote: > >> Xuan will take a look at this. >> Hong >> >> On Mon, Jul 23, 2012 at 9:47 AM, Matthew Knepley <knepley(a)gmail.com>wrote: >> >>> I think we just need a small converter from AIJ: >>> >>> >>> https://bitbucket.org/poulson/clique/src/dc417c7e9403/tests/DistSparseMatri… >>> >>> Matt >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> >

1 0

Re: [petsc-dev] Jack has produced a scalable sparse direct solver
by Jack Poulson 23 Jul '12

23 Jul '12

Hi Hong, I'm getting started on creating documentation right now; it should end up looking very similar to Elemental's. Please keep in mind that there is not _yet_ any support for pivoting, as I needed Clique's functionality for indefinite complex symmetric matrices which are nice enough to be factored accurately without pivoting. Clique's nested dissection is built on top of a nodal graph bisection routine which uses a custom interface to parmetis; I suspect that the graph partitioning is currently the least scalable part of the black-box interface due to KLFM refinement being so hard to parallelize. Jack On Mon, Jul 23, 2012 at 10:12 AM, Hong Zhang <hzhang(a)mcs.anl.gov> wrote: > Xuan will take a look at this. > Hong > > On Mon, Jul 23, 2012 at 9:47 AM, Matthew Knepley <knepley(a)gmail.com>wrote: > >> I think we just need a small converter from AIJ: >> >> >> https://bitbucket.org/poulson/clique/src/dc417c7e9403/tests/DistSparseMatri… >> >> Matt >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > >

1 0

Re: [petsc-dev] Jack has produced a scalable sparse direct solver
by Matthew Knepley 23 Jul '12

23 Jul '12

On Mon, Jul 23, 2012 at 10:12 AM, Hong Zhang <hzhang(a)mcs.anl.gov> wrote: > Xuan will take a look at this. > He has added nice variables for the build as well http://poulson.github.com/Elemental/build.html#testing-the-installation Matt > Hong > > On Mon, Jul 23, 2012 at 9:47 AM, Matthew Knepley <knepley(a)gmail.com>wrote: > >> I think we just need a small converter from AIJ: >> >> >> https://bitbucket.org/poulson/clique/src/dc417c7e9403/tests/DistSparseMatri… >> >> Matt >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Jack has produced a scalable sparse direct solver
by Hong Zhang 23 Jul '12

23 Jul '12

Xuan will take a look at this. Hong On Mon, Jul 23, 2012 at 9:47 AM, Matthew Knepley <knepley(a)gmail.com> wrote: > I think we just need a small converter from AIJ: > > > https://bitbucket.org/poulson/clique/src/dc417c7e9403/tests/DistSparseMatri… > > Matt > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener >

1 0

Jack has produced a scalable sparse direct solver
by Matthew Knepley 23 Jul '12

23 Jul '12

I think we just need a small converter from AIJ: https://bitbucket.org/poulson/clique/src/dc417c7e9403/tests/DistSparseMatri… Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Scalability of removing null spaces
by Jed Brown 22 Jul '12

22 Jul '12

On Sun, Jul 22, 2012 at 1:50 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > I think we have to figure out how to do this pipelined thing in a > mechanical way (not necessarily automated, but > done without thinking). You would think this had been figured out by the > architecture guys. > Bah, those guys don't use mathematical invariants so when they see an instruction sequence like a Krylov method, they say there is a data dependency and the pipeline stalls. The pipelined Krylov methods use the structure and the choices you make affect stability. In the case of GMRES, I think we just add an extra VecMDotBegin/End pair in with projection against the prior Krylov space (before normalization).

1 0

Re: [petsc-dev] Scalability of removing null spaces
by Matthew Knepley 22 Jul '12

22 Jul '12

On Sun, Jul 22, 2012 at 1:19 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > We recommend to people that they just call MatSetNullSpace() and solve the > singular system. What if reductions are very expensive? Are there Krylov > methods that accommodate a pipelined or deferred null space removal? > Shouldn't be hard to do, but I haven't seen papers on it. I think we have to figure out how to do this pipelined thing in a mechanical way (not necessarily automated, but done without thinking). You would think this had been figured out by the architecture guys. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Scalability of removing null spaces
by Jed Brown 22 Jul '12

22 Jul '12

We recommend to people that they just call MatSetNullSpace() and solve the singular system. What if reductions are very expensive? Are there Krylov methods that accommodate a pipelined or deferred null space removal? Shouldn't be hard to do, but I haven't seen papers on it.

1 0

Re: [petsc-dev] MatMultHermitianTranspose
by Barry Smith 20 Jul '12

20 Jul '12

Jose, Looks ok to me for you to push for 3.3 and dev. Barry On Jul 20, 2012, at 4:20 AM, Jose E. Roman wrote: > > El 18/07/2012, a las 17:45, Jed Brown escribió: > >> On Wed, Jul 18, 2012 at 10:22 AM, Jose E. Roman <jroman(a)dsic.upv.es> wrote: >> El 18/07/2012, a las 15:05, Jed Brown escribió: >> >>> MatMultHermitianTranspose could have a default implementation in terms of the others. Then it would work in all cases, right? >>> >> >> Do you mean VecConjugate(x) plus MatMultTranspose(A,x,y) plus VecConjugate(y) ?? >> I guess this would work. >> >> In the PETSc implementation, right? >> >> > > How about the attached patch? > If you agree, should I push to petsc-3.3? or petsc-dev only? > > Jose > > <patch-hermitian-transpose>

1 0