May 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Multigrid is confusing
by Barry Smith 25 May '12

25 May '12

On May 24, 2012, at 10:35 PM, Jed Brown wrote: > On Thu, May 24, 2012 at 10:19 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > I think I've fixed PCPre/PostSolve_Eisenstat() to work properly with kspest but seems to be with ex19 a much worse smoother than SOR (both with Cheby). I'm not sure why. With SOR we are essentially doing a few Chebyshev iterations with > (U+D)^-1 (L+D)^-1 A x = b while with Eisenstat it is (L+D)^-1 A (U + D)^-1 y = (L+D)^-1 b. Well no time to think about it now. > > Mark, > > Has anyone looked at using Cheby Eisentat smoothing as opposed to Cheby SOR smoothing? > > Aren't they supposed to be the same? Well one is left preconditioning while the other is "split" preconditioning so iterates could be different. There might also some difference with diagonal scalings? Barry > > > Barry > > On May 24, 2012, at 1:20 PM, Jed Brown wrote: > > > On Wed, May 23, 2012 at 2:52 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > > On Wed, May 23, 2012 at 2:26 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > > > Note that you could use -pc_type eisenstat perhaps in this case instead. Might save lots of flops? I've often wondered about doing Mark's favorite chebyshev smoother with Eisenstat, seems like it should be a good match. > > > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > > [0]PETSC ERROR: No support for this operation for this object type! > > [0]PETSC ERROR: Cannot have different mat and pmat! > > > > Also, I'm having trouble getting Eisenstat to be more than very marginally faster than SOR. > > > > > > I think we should later be getting the eigenvalue estimate by applying the preconditioned operator to a few random vectors, then orthogonalizing. The basic algorithm is to generate a random matrix X (say 5 or 10 columns), compute > > > > Y = (P^{-1} A)^q X > > > > where q is 1 or 2 or 3, then compute > > > > Q R = Y > > > > and compute the largest singular value of the small matrix R. The orthogonalization can be done in one reduction and all the MatMults can be done together. Whenever we manage to implement a MatMMult and PCMApply or whatever (names inspired by VecMDot), this will provide a very low communication way to get the eigenvalue estimates. > > > > > > I want to turn off norms in Chebyshev by default (they are very wasteful), but how should I make -mg_levels_ksp_monitor turn them back on? I'm already tired of typing -mg_levels_ksp_norm_type unpreconditioned. > >

1 0

Re: [petsc-dev] Multigrid is confusing
by Mark F. Adams 25 May '12

25 May '12

On May 24, 2012, at 11:35 PM, Jed Brown wrote: > On Thu, May 24, 2012 at 10:19 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > I think I've fixed PCPre/PostSolve_Eisenstat() to work properly with kspest but seems to be with ex19 a much worse smoother than SOR (both with Cheby). I'm not sure why. With SOR we are essentially doing a few Chebyshev iterations with > (U+D)^-1 (L+D)^-1 A x = b while with Eisenstat it is (L+D)^-1 A (U + D)^-1 y = (L+D)^-1 b. Well no time to think about it now. > > Mark, > > Has anyone looked at using Cheby Eisentat smoothing as opposed to Cheby SOR smoothing? > > Aren't they supposed to be the same? I thought you just ignore the work that you know is 0.0 in E. So yes, they should be the same AFAIK. And, I've never seen Gauss-Siedel used with Cheby because G-S has the correct damping properties, as is, for the Laplacian. Note, G-S is not symmetric and Cheby for unsymmetric is a different can of worms. So if A is symmetric then maybe try SSOR. Mark > > > Barry > > On May 24, 2012, at 1:20 PM, Jed Brown wrote: > > > On Wed, May 23, 2012 at 2:52 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > > On Wed, May 23, 2012 at 2:26 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > > > Note that you could use -pc_type eisenstat perhaps in this case instead. Might save lots of flops? I've often wondered about doing Mark's favorite chebyshev smoother with Eisenstat, seems like it should be a good match. > > > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > > [0]PETSC ERROR: No support for this operation for this object type! > > [0]PETSC ERROR: Cannot have different mat and pmat! > > > > Also, I'm having trouble getting Eisenstat to be more than very marginally faster than SOR. > > > > > > I think we should later be getting the eigenvalue estimate by applying the preconditioned operator to a few random vectors, then orthogonalizing. The basic algorithm is to generate a random matrix X (say 5 or 10 columns), compute > > > > Y = (P^{-1} A)^q X > > > > where q is 1 or 2 or 3, then compute > > > > Q R = Y > > > > and compute the largest singular value of the small matrix R. The orthogonalization can be done in one reduction and all the MatMults can be done together. Whenever we manage to implement a MatMMult and PCMApply or whatever (names inspired by VecMDot), this will provide a very low communication way to get the eigenvalue estimates. > > > > > > I want to turn off norms in Chebyshev by default (they are very wasteful), but how should I make -mg_levels_ksp_monitor turn them back on? I'm already tired of typing -mg_levels_ksp_norm_type unpreconditioned. > >

1 0

Re: [petsc-dev] Multigrid is confusing
by Jed Brown 24 May '12

24 May '12

On Thu, May 24, 2012 at 10:19 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > I think I've fixed PCPre/PostSolve_Eisenstat() to work properly with > kspest but seems to be with ex19 a much worse smoother than SOR (both with > Cheby). I'm not sure why. With SOR we are essentially doing a few Chebyshev > iterations with > (U+D)^-1 (L+D)^-1 A x = b while with Eisenstat it is (L+D)^-1 A (U + > D)^-1 y = (L+D)^-1 b. Well no time to think about it now. > > Mark, > > Has anyone looked at using Cheby Eisentat smoothing as opposed to > Cheby SOR smoothing? > Aren't they supposed to be the same? > > Barry > > On May 24, 2012, at 1:20 PM, Jed Brown wrote: > > > On Wed, May 23, 2012 at 2:52 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > > On Wed, May 23, 2012 at 2:26 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > > > Note that you could use -pc_type eisenstat perhaps in this case > instead. Might save lots of flops? I've often wondered about doing Mark's > favorite chebyshev smoother with Eisenstat, seems like it should be a good > match. > > > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > > [0]PETSC ERROR: No support for this operation for this object type! > > [0]PETSC ERROR: Cannot have different mat and pmat! > > > > Also, I'm having trouble getting Eisenstat to be more than very > marginally faster than SOR. > > > > > > I think we should later be getting the eigenvalue estimate by applying > the preconditioned operator to a few random vectors, then orthogonalizing. > The basic algorithm is to generate a random matrix X (say 5 or 10 columns), > compute > > > > Y = (P^{-1} A)^q X > > > > where q is 1 or 2 or 3, then compute > > > > Q R = Y > > > > and compute the largest singular value of the small matrix R. The > orthogonalization can be done in one reduction and all the MatMults can be > done together. Whenever we manage to implement a MatMMult and PCMApply or > whatever (names inspired by VecMDot), this will provide a very low > communication way to get the eigenvalue estimates. > > > > > > I want to turn off norms in Chebyshev by default (they are very > wasteful), but how should I make -mg_levels_ksp_monitor turn them back on? > I'm already tired of typing -mg_levels_ksp_norm_type unpreconditioned. > >

1 0

Re: [petsc-dev] Multigrid is confusing
by Barry Smith 24 May '12

24 May '12

I think I've fixed PCPre/PostSolve_Eisenstat() to work properly with kspest but seems to be with ex19 a much worse smoother than SOR (both with Cheby). I'm not sure why. With SOR we are essentially doing a few Chebyshev iterations with (U+D)^-1 (L+D)^-1 A x = b while with Eisenstat it is (L+D)^-1 A (U + D)^-1 y = (L+D)^-1 b. Well no time to think about it now. Mark, Has anyone looked at using Cheby Eisentat smoothing as opposed to Cheby SOR smoothing? Barry On May 24, 2012, at 1:20 PM, Jed Brown wrote: > On Wed, May 23, 2012 at 2:52 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Wed, May 23, 2012 at 2:26 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Note that you could use -pc_type eisenstat perhaps in this case instead. Might save lots of flops? I've often wondered about doing Mark's favorite chebyshev smoother with Eisenstat, seems like it should be a good match. > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: No support for this operation for this object type! > [0]PETSC ERROR: Cannot have different mat and pmat! > > Also, I'm having trouble getting Eisenstat to be more than very marginally faster than SOR. > > > I think we should later be getting the eigenvalue estimate by applying the preconditioned operator to a few random vectors, then orthogonalizing. The basic algorithm is to generate a random matrix X (say 5 or 10 columns), compute > > Y = (P^{-1} A)^q X > > where q is 1 or 2 or 3, then compute > > Q R = Y > > and compute the largest singular value of the small matrix R. The orthogonalization can be done in one reduction and all the MatMults can be done together. Whenever we manage to implement a MatMMult and PCMApply or whatever (names inspired by VecMDot), this will provide a very low communication way to get the eigenvalue estimates. > > > I want to turn off norms in Chebyshev by default (they are very wasteful), but how should I make -mg_levels_ksp_monitor turn them back on? I'm already tired of typing -mg_levels_ksp_norm_type unpreconditioned.

1 0

Re: [petsc-dev] Multigrid is confusing
by Jed Brown 24 May '12

24 May '12

On Thu, May 24, 2012 at 10:05 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > I think its wrong, but not for the obvious reason. You can do very elegant > things with macros, but > they always end up being thrown away when you have to maintain them and > have others understand > them. > Perhaps, I'm just not convinced that a separate DSL/code generation system is easier to work with or educate people about.

1 0

Re: [petsc-dev] Multigrid is confusing
by Matthew Knepley 24 May '12

24 May '12

On Thu, May 24, 2012 at 10:25 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Thu, May 24, 2012 at 9:17 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > >> Sarcasm on -- >> Yes, of course. It is essentially done, write it up and I want it on my >> desk by 8 AM Friday. >> -- Sarcasm off >> >> So what if we know the primatives. How does it give us the "language" >> to express these kernels and the tools to do the processing? >> > > I hate C++ templates as much as the next guy, but I think they can do this > without insanity. I think we could also do it with a handful of macro hacks > and C inline functions. > > Sure, we might be able to make a cool DSL for expressing this stuff, but > I'm afraid we'd end up spending more time and debugging confusion making it > play with everything else than we'd gain by having the DSL in the first > place. > > I could be totally wrong. > I think its wrong, but not for the obvious reason. You can do very elegant things with macros, but they always end up being thrown away when you have to maintain them and have others understand them. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Need a CFLAGS that does is *not* included in the link
by Jed Brown 24 May '12

24 May '12

On Thu, May 24, 2012 at 9:50 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > According to autoconf, CFLAGS goes with both. CPPFLAGS is only for the > compiler. Cool, where do I add to CPPFLAGS based on the type of compiler? It would be natural in compilerOptions.py, but it's not clear to me where to put it.

1 0

Re: [petsc-dev] Need a CFLAGS that does is *not* included in the link
by Matthew Knepley 24 May '12

24 May '12

On Thu, May 24, 2012 at 10:09 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Building PETSc with clang++ produces a warning about compiling *.c as C++ > being deprecated. To silence the warning, we would need to pass "-x c++" to > the compiler, but NOT to the linker. CFLAGS is currently also passed to the > linker. Is this something we want to fix? According to autoconf, CFLAGS goes with both. CPPFLAGS is only for the compiler. Matt > Clang used to SEGV when "-x c++" is passed to the linker. Now (latest SVN) > it just interprets the object file as C++ source (which obviously produces > a ton of garbage). > > http://llvm.org/bugs/show_bug.cgi?id=12924 > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Need a CFLAGS that does is *not* included in the link
by Chetan Jhurani 24 May '12

24 May '12

> So in general, CFLAGS must be passed to the linker (it contains optimization and debugging stuff that should be consistent). The -x option is very special in that it cannot be passed to the linker. Oh yes, I didn’t describe my fix properly, but you’re right… From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Jed Brown Sent: Thursday, May 24, 2012 7:34 PM To: For users of the development version of PETSc Subject: Re: [petsc-dev] Need a CFLAGS that does is *not* included in the link On Thu, May 24, 2012 at 9:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: I faced a similar problem when using -x argument with nvcc during petsc configuration, which could be fixed if CFLAGS was not passed to the linker. So sending this info in case it is useful. So in general, CFLAGS must be passed to the linker (it contains optimization and debugging stuff that should be consistent). The -x option is very special in that it cannot be passed to the linker. [jhurani@enrico ~]$ cat a.c int main() { } [jhurani@enrico ~]$ nvcc a.c -x c++ -c [jhurani@enrico ~]$ nvcc a.o -x c++ a.o:1: error: stray '\177' in program a.o:1: error: stray '\2' in program a.o:1: error: stray '\1' in program a.o:1: error: stray '\1' in program a.o:1:8: warning: null character(s) ignored a.o:1: error: stray '\1' in program a.o:1:18: warning: null character(s) ignored and many more lines… Chetan From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Jed Brown Sent: Thursday, May 24, 2012 7:09 PM To: For users of the development version of PETSc Subject: [petsc-dev] Need a CFLAGS that does is *not* included in the link Building PETSc with clang++ produces a warning about compiling *.c as C++ being deprecated. To silence the warning, we would need to pass "-x c++" to the compiler, but NOT to the linker. CFLAGS is currently also passed to the linker. Is this something we want to fix? Clang used to SEGV when "-x c++" is passed to the linker. Now (latest SVN) it just interprets the object file as C++ source (which obviously produces a ton of garbage). http://llvm.org/bugs/show_bug.cgi?id=12924

1 0

Re: [petsc-dev] Need a CFLAGS that does is *not* included in the link
by Jed Brown 24 May '12

24 May '12

On Thu, May 24, 2012 at 9:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com>wrote: > I faced a similar problem when using -x argument with nvcc during**** > > petsc configuration, which could be fixed if CFLAGS was not passed**** > > to the linker. So sending this info in case it is useful. > So in general, CFLAGS must be passed to the linker (it contains optimization and debugging stuff that should be consistent). The -x option is very special in that it cannot be passed to the linker. > **** > > ** ** > > [jhurani@enrico ~]$ cat a.c**** > > int main()**** > > {**** > > }**** > > ** ** > > [jhurani@enrico ~]$ nvcc a.c -x c++ -c**** > > [jhurani@enrico ~]$ nvcc a.o -x c++**** > > a.o:1: error: stray '\177' in program**** > > a.o:1: error: stray '\2' in program**** > > a.o:1: error: stray '\1' in program**** > > a.o:1: error: stray '\1' in program**** > > a.o:1:8: warning: null character(s) ignored**** > > a.o:1: error: stray '\1' in program**** > > a.o:1:18: warning: null character(s) ignored**** > > ** ** > > and many more lines…**** > > ** ** > > Chetan**** > > ** ** > > ** ** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Jed Brown > *Sent:* Thursday, May 24, 2012 7:09 PM > *To:* For users of the development version of PETSc > *Subject:* [petsc-dev] Need a CFLAGS that does is *not* included in the > link**** > > ** ** > > Building PETSc with clang++ produces a warning about compiling *.c as C++ > being deprecated. To silence the warning, we would need to pass "-x c++" to > the compiler, but NOT to the linker. CFLAGS is currently also passed to the > linker. Is this something we want to fix?**** > > ** ** > > ** ** > > Clang used to SEGV when "-x c++" is passed to the linker. Now (latest SVN) > it just interprets the object file as C++ source (which obviously produces > a ton of garbage).**** > > ** ** > > http://llvm.org/bugs/show_bug.cgi?id=12924**** >

1 0