February 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Merge sets
by Satish Balay 18 Feb '12

18 Feb '12

On Sat, 18 Feb 2012, Barry Smith wrote: > 2) If the rebase implementation has been fixed from the hacky > versions that fucked unnecessarily with my file system I'll be happy > to start using rebase. Is it fixed? The issue is: >> When file 'a.c' is locally commited, and a pull potentially has changes to 'b.c', 'pull --rebase' also changes the timestamp on a.c [whereas 'pull; merge; commit' does not change timestamp on a.c] << Latest hg [version 2.1] still updates the timestamp of a.c on rebase. Rebase is doing state changes [wrt commit states] - in this process I think removes local changesets and recommits them - so the locally changed files get new timestamps in this process. The fact that 'make' relies on timestamps messes things up [causing unnecessary recompiles]. Reseting timestamps on unchanged files would be a good optimziation - which is what Barry wants.. Satish

1 0

Re: [petsc-dev] Merge sets
by Barry Smith 18 Feb '12

18 Feb '12

I've installed the latest hg and will try rebasing for a while (Until/if it screws me and then I'll report problems). Barry On Feb 18, 2012, at 9:44 PM, Matthew Knepley wrote: > On Sat, Feb 18, 2012 at 9:28 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Feb 18, 2012, at 3:18 PM, Matthew Knepley wrote: > > > I have lost the BB battle, however can we at least start using > > > > hg pull --rebase > > > > so that we avoid this > > > > https://bitbucket.org/petsc/petsc-dev/changeset/ad9064ecab66 > > > 1) I don't like the idea of a pull doing anything locally on my machine. I want it to only get the stuff from the remote repository and bring it to my machine. This reeks of svn > > I see it as a replacement for multiple repositories and carefully orchestrated pulling, like Linus used to do, in order > to keep change sets clean. It is not giving up on the concept of change sets like svn. > > 2) If the rebase implementation has been fixed from the hacky versions that fucked unnecessarily with my file system I'll be happy to start using rebase. Is it fixed? > > I cannot identify the behavior from that description. However, I have been using it for almost a year now, and the > nice thing is that if anything goes wrong (merges with MacHg can screw up badly), I just do > > hg rollback > hg revert --all > > You might be talking about the need to revert. > > Matt > > Barry > > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Fwd: Undelivered Mail Returned to Sender
by Matthew Knepley 18 Feb '12

18 Feb '12

Fix the mail headers for this thing. My message is below. Matt ---------- Forwarded message ---------- From: Mail Delivery System <MAILER-DAEMON(a)mailrelay.anl.gov> Date: Sat, Feb 18, 2012 at 9:44 PM Subject: Undelivered Mail Returned to Sender To: knepley(a)gmail.com This is the mail system at host mailrelay.anl.gov. I'm sorry to have to inform you that your message could not be delivered to one or more recipients. It's attached below. For further assistance, please send mail to postmaster. If you do so, please include this problem report. You can delete your own text from the attached returned message. The mail system <petsc-dev(a)zimbra.anl.gov>: host zimbra.anl.gov[130.202.101.12] said: 550 5.1.1 <petsc-dev(a)zimbra.anl.gov>: Recipient address rejected: zimbra.anl.gov(in reply to RCPT TO command) Final-Recipient: rfc822; petsc-dev(a)zimbra.anl.gov Original-Recipient: rfc822;petsc-dev(a)zimbra.anl.gov Action: failed Status: 5.1.1 Remote-MTA: dns; zimbra.anl.gov Diagnostic-Code: smtp; 550 5.1.1 <petsc-dev(a)zimbra.anl.gov>: Recipient address rejected: zimbra.anl.gov ---------- Forwarded message ---------- From: Matthew Knepley <knepley(a)gmail.com> To: petsc-dev(a)zimbra.anl.gov, For users of the development version of PETSc <petsc-dev(a)mcs.anl.gov> Cc: Date: Sat, 18 Feb 2012 21:44:32 -0600 Subject: Re: [petsc-dev] Merge sets On Sat, Feb 18, 2012 at 9:28 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Feb 18, 2012, at 3:18 PM, Matthew Knepley wrote: > > > I have lost the BB battle, however can we at least start using > > > > hg pull --rebase > > > > so that we avoid this > > > > https://bitbucket.org/petsc/petsc-dev/changeset/ad9064ecab66 > > > 1) I don't like the idea of a pull doing anything locally on my machine. > I want it to only get the stuff from the remote repository and bring it to > my machine. This reeks of svn > I see it as a replacement for multiple repositories and carefully orchestrated pulling, like Linus used to do, in order to keep change sets clean. It is not giving up on the concept of change sets like svn. > 2) If the rebase implementation has been fixed from the hacky versions > that fucked unnecessarily with my file system I'll be happy to start using > rebase. Is it fixed? > I cannot identify the behavior from that description. However, I have been using it for almost a year now, and the nice thing is that if anything goes wrong (merges with MacHg can screw up badly), I just do hg rollback hg revert --all You might be talking about the need to revert. Matt > Barry > > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Merge sets
by Matthew Knepley 18 Feb '12

18 Feb '12

On Sat, Feb 18, 2012 at 9:28 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Feb 18, 2012, at 3:18 PM, Matthew Knepley wrote: > > > I have lost the BB battle, however can we at least start using > > > > hg pull --rebase > > > > so that we avoid this > > > > https://bitbucket.org/petsc/petsc-dev/changeset/ad9064ecab66 > > > 1) I don't like the idea of a pull doing anything locally on my machine. > I want it to only get the stuff from the remote repository and bring it to > my machine. This reeks of svn > I see it as a replacement for multiple repositories and carefully orchestrated pulling, like Linus used to do, in order to keep change sets clean. It is not giving up on the concept of change sets like svn. > 2) If the rebase implementation has been fixed from the hacky versions > that fucked unnecessarily with my file system I'll be happy to start using > rebase. Is it fixed? > I cannot identify the behavior from that description. However, I have been using it for almost a year now, and the nice thing is that if anything goes wrong (merges with MacHg can screw up badly), I just do hg rollback hg revert --all You might be talking about the need to revert. Matt > Barry > > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Merge sets
by Barry Smith 18 Feb '12

18 Feb '12

On Feb 18, 2012, at 3:18 PM, Matthew Knepley wrote: > I have lost the BB battle, however can we at least start using > > hg pull --rebase > > so that we avoid this > > https://bitbucket.org/petsc/petsc-dev/changeset/ad9064ecab66 1) I don't like the idea of a pull doing anything locally on my machine. I want it to only get the stuff from the remote repository and bring it to my machine. This reeks of svn 2) If the rebase implementation has been fixed from the hacky versions that fucked unnecessarily with my file system I'll be happy to start using rebase. Is it fixed? Barry > > Matt > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] Mac OSX complex BLAS problem
by Barry Smith 18 Feb '12

18 Feb '12

This is a known problem. Intel changed their complex dot a while ago causing grief. Perhaps Apple follows the Intel blas libraries since they are in bed with Intel. Anyways, yes a proper set of configure tests is needed for different versions of complex (double and single) dot and then the PETSc source code needs adjusting to handle the situations. It is likely this means ugly ifdef crap in our source. Thanks Jack we really appreciate it. Barry From: Alexander Grayver <agrayver(a)gfz-potsdam.de> Subject: Re: [petsc-users] MKL BLAS interface inconsistency Date: January 19, 2012 10:55:21 AM CST To: PETSc users list <petsc-users(a)mcs.anl.gov> Reply-To: PETSc users list <petsc-users(a)mcs.anl.gov> This (dirty) patch solves the problem: diff -r a3e9ca59ab58 src/vec/vec/impls/seq/bvec2.c --- a/src/vec/vec/impls/seq/bvec2.c Tue Jan 17 22:04:05 2012 -0600 +++ b/src/vec/vec/impls/seq/bvec2.c Thu Jan 19 17:28:39 2012 +0100 @@ -232,12 +232,13 @@ PetscErrorCode ierr; PetscInt n = xin->map->n; PetscBLASInt one = 1, bn = PetscBLASIntCast(n); + PetscScalar cnorm; PetscFunctionBegin; if (type == NORM_2 || type == NORM_FROBENIUS) { ierr = VecGetArrayRead(xin,&xx);CHKERRQ(ierr); - *z = BLASdot_(&bn,xx,&one,xx,&one); - *z = PetscSqrtReal(*z); + zdotc(&cnorm,&bn,xx,&one,xx,&one); + *z = PetscSqrtReal(PetscAbsScalar(cnorm)); ierr = VecRestoreArrayRead(xin,&xx);CHKERRQ(ierr); ierr = PetscLogFlops(PetscMax(2.0*n-1,0.0));CHKERRQ(ierr); } else if (type == NORM_INFINITY) { The same is applied to mpi vector implementation from /petsc-dev/src/vec/vec/impls/mpi/pvec2.c Of course it works only if one uses Intel MKL BLAS/LAPACK and double complex arithmetics. Which is my case. On Feb 18, 2012, at 5:47 PM, Matthew Knepley wrote: > On Sat, Feb 18, 2012 at 5:34 PM, Jack Poulson <jack.poulson(a)gmail.com> wrote: > I think I mentioned this to several PETSc folks two years ago. It is a bad idea to return complex numbers from Fortran, as there is no portable solution as far as I know; some compilers need you to declare it as > > complex zdotc_( <regular args> ) > > some as > > void zdotc_( <regular args>, complex* z ) > > and I think I ran into problems with both on OSX (but don't quote me on it). I ended up settling on writing my own complex dotc and dotu routines. > > That is exactly it. Scipy has a patch for it > > http://projects.scipy.org/scipy/attachment/ticket/1321/scipy-r6856-isolve-v… > > So what are we going to do in PETSc folks? It sounds like we need a configure test for complex dot, and then > and then an #define wrapper for ones that return rather than taking an argument. > > MAtt > > > Jack > > > On Sat, Feb 18, 2012 at 5:28 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > We have a problem with complex dot product on OSX. Here is the easiest way to see it (I think): > > knepley:/PETSc3/petsc/petsc-dev$ ./arch-complex-fftw-debug/lib/ex5-obj/ex5 -snes_monitor > ./arch-complex-fftw-debug/lib/ex5-obj/ex5 -snes_monitor > 0 SNES Function norm 0.000000000000e+00 > > With the debugger: > > Breakpoint 1, VecNorm_Seq (xin=0x1022fc170, type=NORM_2, z=0x7fff5fbfea10) at bvec2.c:238 > 238 ierr = VecGetArrayRead(xin,&xx);CHKERRQ(ierr); > (gdb) n > Current language: auto; currently c++ > 239 *z = BLASdot_(&bn,xx,&one,xx,&one); > (gdb) p bn > $1 = 16 > (gdb) p xx[0]@16 > $2 = {{ > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = -0.10378182131158753 + 0 * I > }, { > _M_value = -0.10378182131158753 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = -0.10378182131158753 + 0 * I > }, { > _M_value = -0.10378182131158753 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }} > (gdb) p one > $3 = 1 > (gdb) n > 240 *z = PetscSqrtReal(*z); > (gdb) p *z > $4 = 0 > (gdb) p zdotc > $5 = {<text variable, no debug info>} 0x7fff863e71e2 <zdotc_> > > It appears that the dot product is producing nothing. Didn't we have a similar problem a little while ago? > > Matt > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] Mac OSX complex BLAS problem
by Matthew Knepley 18 Feb '12

18 Feb '12

On Sat, Feb 18, 2012 at 5:34 PM, Jack Poulson <jack.poulson(a)gmail.com>wrote: > I think I mentioned this to several PETSc folks two years ago. It is a bad > idea to return complex numbers from Fortran, as there is no portable > solution as far as I know; some compilers need you to declare it as > > complex zdotc_( <regular args> ) > > some as > > void zdotc_( <regular args>, complex* z ) > > and I think I ran into problems with both on OSX (but don't quote me on > it). I ended up settling on writing my own complex dotc and dotu routines. That is exactly it. Scipy has a patch for it http://projects.scipy.org/scipy/attachment/ticket/1321/scipy-r6856-isolve-v… So what are we going to do in PETSc folks? It sounds like we need a configure test for complex dot, and then and then an #define wrapper for ones that return rather than taking an argument. MAtt > > Jack > > > On Sat, Feb 18, 2012 at 5:28 PM, Matthew Knepley <knepley(a)gmail.com>wrote: > >> We have a problem with complex dot product on OSX. Here is the easiest >> way to see it (I think): >> >> knepley:/PETSc3/petsc/petsc-dev$ >> ./arch-complex-fftw-debug/lib/ex5-obj/ex5 -snes_monitor >> ./arch-complex-fftw-debug/lib/ex5-obj/ex5 -snes_monitor >> 0 SNES Function norm 0.000000000000e+00 >> >> With the debugger: >> >> Breakpoint 1, VecNorm_Seq (xin=0x1022fc170, type=NORM_2, >> z=0x7fff5fbfea10) at bvec2.c:238 >> 238 ierr = VecGetArrayRead(xin,&xx);CHKERRQ(ierr); >> (gdb) n >> Current language: auto; currently c++ >> 239 *z = BLASdot_(&bn,xx,&one,xx,&one); >> (gdb) p bn >> $1 = 16 >> (gdb) p xx[0]@16 >> $2 = {{ >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = -0.10378182131158753 + 0 * I >> }, { >> _M_value = -0.10378182131158753 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = -0.10378182131158753 + 0 * I >> }, { >> _M_value = -0.10378182131158753 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }, { >> _M_value = 0 + 0 * I >> }} >> (gdb) p one >> $3 = 1 >> (gdb) n >> 240 *z = PetscSqrtReal(*z); >> (gdb) p *z >> $4 = 0 >> (gdb) p zdotc >> $5 = {<text variable, no debug info>} 0x7fff863e71e2 <zdotc_> >> >> It appears that the dot product is producing nothing. Didn't we have a >> similar problem a little while ago? >> >> Matt >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Mac OSX complex BLAS problem
by Jack Poulson 18 Feb '12

18 Feb '12

I think I mentioned this to several PETSc folks two years ago. It is a bad idea to return complex numbers from Fortran, as there is no portable solution as far as I know; some compilers need you to declare it as complex zdotc_( <regular args> ) some as void zdotc_( <regular args>, complex* z ) and I think I ran into problems with both on OSX (but don't quote me on it). I ended up settling on writing my own complex dotc and dotu routines. Jack On Sat, Feb 18, 2012 at 5:28 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > We have a problem with complex dot product on OSX. Here is the easiest way > to see it (I think): > > knepley:/PETSc3/petsc/petsc-dev$ ./arch-complex-fftw-debug/lib/ex5-obj/ex5 > -snes_monitor > ./arch-complex-fftw-debug/lib/ex5-obj/ex5 -snes_monitor > 0 SNES Function norm 0.000000000000e+00 > > With the debugger: > > Breakpoint 1, VecNorm_Seq (xin=0x1022fc170, type=NORM_2, z=0x7fff5fbfea10) > at bvec2.c:238 > 238 ierr = VecGetArrayRead(xin,&xx);CHKERRQ(ierr); > (gdb) n > Current language: auto; currently c++ > 239 *z = BLASdot_(&bn,xx,&one,xx,&one); > (gdb) p bn > $1 = 16 > (gdb) p xx[0]@16 > $2 = {{ > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = -0.10378182131158753 + 0 * I > }, { > _M_value = -0.10378182131158753 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = -0.10378182131158753 + 0 * I > }, { > _M_value = -0.10378182131158753 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }, { > _M_value = 0 + 0 * I > }} > (gdb) p one > $3 = 1 > (gdb) n > 240 *z = PetscSqrtReal(*z); > (gdb) p *z > $4 = 0 > (gdb) p zdotc > $5 = {<text variable, no debug info>} 0x7fff863e71e2 <zdotc_> > > It appears that the dot product is producing nothing. Didn't we have a > similar problem a little while ago? > > Matt > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener >

1 0

Mac OSX complex BLAS problem
by Matthew Knepley 18 Feb '12

18 Feb '12

We have a problem with complex dot product on OSX. Here is the easiest way to see it (I think): knepley:/PETSc3/petsc/petsc-dev$ ./arch-complex-fftw-debug/lib/ex5-obj/ex5 -snes_monitor ./arch-complex-fftw-debug/lib/ex5-obj/ex5 -snes_monitor 0 SNES Function norm 0.000000000000e+00 With the debugger: Breakpoint 1, VecNorm_Seq (xin=0x1022fc170, type=NORM_2, z=0x7fff5fbfea10) at bvec2.c:238 238 ierr = VecGetArrayRead(xin,&xx);CHKERRQ(ierr); (gdb) n Current language: auto; currently c++ 239 *z = BLASdot_(&bn,xx,&one,xx,&one); (gdb) p bn $1 = 16 (gdb) p xx[0]@16 $2 = {{ _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = -0.10378182131158753 + 0 * I }, { _M_value = -0.10378182131158753 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = -0.10378182131158753 + 0 * I }, { _M_value = -0.10378182131158753 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }, { _M_value = 0 + 0 * I }} (gdb) p one $3 = 1 (gdb) n 240 *z = PetscSqrtReal(*z); (gdb) p *z $4 = 0 (gdb) p zdotc $5 = {<text variable, no debug info>} 0x7fff863e71e2 <zdotc_> It appears that the dot product is producing nothing. Didn't we have a similar problem a little while ago? Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Merge sets
by Satish Balay 18 Feb '12

18 Feb '12

On Sat, 18 Feb 2012, Jed Brown wrote: > On Sat, Feb 18, 2012 at 16:18, Matthew Knepley <knepley(a)gmail.com> wrote: > > > I have lost the BB battle, however can we at least start using > > > > hg pull --rebase > > > > so that we avoid this > > > > https://bitbucket.org/petsc/petsc-dev/changeset/ad9064ecab66 > > > > This is a visualization issue. Yes - I just use 'hg view' - and if its a merge commit - I go to the parent to check. But i do use 'hg pull --rebase' for of my [trivial] commits though.. Satish > > $ hg diff -r 63b68cc62c3d..ad9064ecab66 > diff --git a/src/ksp/pc/impls/mg/mg.c b/src/ksp/pc/impls/mg/mg.c > --- a/src/ksp/pc/impls/mg/mg.c > +++ b/src/ksp/pc/impls/mg/mg.c > @@ -1026,17 +1026,22 @@ > . -pc_mg_type <additive,multiplicative,full,cascade> - multiplicative is > the default > . -pc_mg_log - log information about time spent on each level of the > solver > . -pc_mg_monitor - print information on the multigrid convergence > -. -pc_mg_galerkin - use Galerkin process to compute coarser operators > -- -pc_mg_dump_matlab - dumps the matrices for each level and the > restriction/interpolation matrices > +. -pc_mg_galerkin - use Galerkin process to compute coarser operators, > i.e. Acoarse = R A R' > +. -pc_mg_multiplicative_cycles - number of cycles to use as the > preconditioner (defaults to 1) > +. -pc_mg_dump_matlab - dumps the matrices for each level and the > restriction/interpolation matrices > to the Socket viewer for reading from MATLAB. > +- -pc_mg_dump_binary - dumps the matrices for each level and the > restriction/interpolation matrices > + to the binary output file called binaryoutput > > Notes: By default this uses GMRES on the fine grid smoother so this > should be used with KSPFGMRES or the smoother changed to not use GMRES > > + When run with a single level the smoother options are used on that > level NOT the coarse grid solver options > + > Level: intermediate > > Concepts: multigrid/multilevel > > -.seealso: PCCreate(), PCSetType(), PCType (for list of available types), > PC, PCMGType, PCEXOTIC > +.seealso: PCCreate(), PCSetType(), PCType (for list of available types), > PC, PCMGType, PCEXOTIC, PCGAMG, PCML, PCHYPRE > PCMGSetLevels(), PCMGGetLevels(), PCMGSetType(), > PCMGSetCycleType(), PCMGSetNumberSmoothDown(), > PCMGSetNumberSmoothUp(), PCMGGetCoarseSolve(), > PCMGSetResidual(), PCMGSetInterpolation(), > PCMGSetRestriction(), PCMGGetSmoother(), PCMGGetSmootherUp(), > PCMGGetSmootherDown(), >

1 0