Jed There are some additional issues when the GMRES runs for zero iterations in computing eigenvalues for Cheby. static PetscErrorCode KSPChebyshevComputeExtremeEigenvalues_Private(KSP kspest,PetscReal *emin,PetscReal *emax) { PetscErrorCode ierr; PetscInt n,neig; PetscReal *re,*im,min,max; PetscFunctionBegin; ierr = KSPGetIterationNumber(kspest,&n);CHKERRQ(ierr); ierr = PetscMalloc2(n,&re,n,&im);CHKERRQ(ierr); ierr = KSPComputeEigenvalues(kspest,n,re,im,&neig);CHKERRQ(ierr); min = PETSC_MAX_REAL; max = PETSC_MIN_REAL; for (n=0; n<neig; n++) { min = PetscMin(min,re[n]); max = PetscMax(max,re[n]); } ierr = PetscFree2(re,im);CHKERRQ(ierr); *emax = max; *emin = min; * thread #1: tid = 0x3c16cf, 0x00000001041b6777 libpetsc.3.5.dylib`KSPSolve_Chebyshev(ksp=0x00007fb85197ac60) + 1495 at cheby.c:378, queue = 'com.apple.main-thread', stop reason = step over frame #0: 0x00000001041b6777 libpetsc.3.5.dylib`KSPSolve_Chebyshev(ksp=0x00007fb85197ac60) + 1495 at cheby.c:378 375 cheb->emin = cheb->tform[0]*min + cheb->tform[1]*max; 376 cheb->emax = cheb->tform[2]*min + cheb->tform[3]*max; 377 -> 378 cheb->estimate_current = PETSC_TRUE; 379 } 380 381 ksp->its = 0; (lldb) p cheb->emin (PetscReal) $11 = -1.7976931348623158E+307 (lldb) p cheb->emax (PetscReal) $12 = -Inf This puts giberish into the eigenvalue estimates. I am not sure how you want to handle this? The zero right hand side zero, initial guess case is a great corner case for testing complicated solvers :-) I’d forgotten about it for many years Barry
On Oct 21, 2014, at 6:47 PM, Barry Smith <[email protected]> wrote:
Mark,
Thanks for figuring it out.
It is legitimate to malloc 0 size memory in PETSc and to have arrays of zero length. So
1) the asserts PetscValidScalarPointer(r,3); PetscValidScalarPointer(c,4);
are over zealous. They should actually be if (n) PetscValidScalarPointer(r,3);
BTW: we should have an additional assert that n >=0
2) that said I do not know if all the later code will function correctly in this case. I see some bad stuff
PetscErrorCode KSPComputeEigenvalues_CG(KSP ksp,PetscInt nmax,PetscReal *r,PetscReal *c,PetscInt *neig) { KSP_CG *cgP = (KSP_CG*)ksp->data; PetscScalar *d,*e; PetscReal *ee; PetscErrorCode ierr; PetscInt j,n = ksp->its;
PetscFunctionBegin; if (nmax < n) SETERRQ(PetscObjectComm((PetscObject)ksp),PETSC_ERR_ARG_SIZ,"Not enough room in work space r and c for eigenvalues"); *neig = n;
ierr = PetscMemzero(c,nmax*sizeof(PetscReal));CHKERRQ(ierr); if (!n) { *r = 0.0; ^^^^ assumes there is space in r for something even if the array dimension is zero which is not good.
This likely requires some more work to insure the code can run through properly in this case. We don’t want to simply short circuit the process.
Barry
On Oct 21, 2014, at 5:53 PM, Mark Adams <[email protected]> wrote:
I am getting this in the output:
[0] VecNormalize(): Vector of zero norm can not be normalized; Returning only the zero norm [0] KSPGMRESCycle(): Converged due to zero residual norm on entry [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3 [0]PETSC ERROR: PETSC: Attaching /usr/bin/gdb to ./test1 of pid 50632 on display /tmp/launch-ikNsM1/org.macosforge.xquartz:0 on machine Macintosh-6.local
and in gdb I see this (appended). The problem is that KSPComputeEigenvalues is called with NULL outputs. This because KSPChebyshevComputeExtremeEigenvalues_Private get the number of iterations and it is zero, it mallocs zero, and this is an error.
Should there be a check like:
326 ierr = KSPGetIterationNumber(kspest,&n);CHKERRQ(ierr); if (!n) bail out 327 ierr = PetscMalloc2(n,&re,n,&im);CHKERRQ(ierr);
or is something going wrong upstream?
Mark
#5 0x000000010fb86057 in PetscAttachDebuggerErrorHandler (comm=1140850689, line=119, fun=0x1115b0a25 "KSPComputeEigenvalues", file=0x1115b09b9 "/Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c", num=85, p=PETSC_ERROR_INITIAL, mess=0x7fff50102290 "Null Pointer: Parameter # 3", ctx=0x0) at /Users/markadams/Codes/petsc/src/sys/error/adebug.c:462 #6 0x000000010fb88d30 in PetscError (comm=1140850689, line=119, func=0x1115b0a25 "KSPComputeEigenvalues", file=0x1115b09b9 "/Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c", n=85, p=PETSC_ERROR_INITIAL, mess=0x1115518ca "Null Pointer: Parameter # %d") at /Users/markadams/Codes/petsc/src/sys/error/err.c:378 #7 0x0000000110bd3464 in KSPComputeEigenvalues (ksp=0x7fe92d813e60, n=0, r=0x0, c=0x0, neig=0x7fff50102c14) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:119 #8 0x0000000110b6a915 in KSPChebyshevComputeExtremeEigenvalues_Private (kspest=0x7fe92d813e60, emin=0x7fff50102ca0, emax=0x7fff50102ca8) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:328 #9 0x0000000110b69043 in KSPSolve_Chebyshev (ksp=0x7fe92c869a60) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:373 #10 0x0000000110bd81ce in KSPSolve (ksp=0x7fe92c869a60, b=0x7fe92d002860, x=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:460 #11 0x0000000110a8e4b8 in PCMGMCycle_Private (pc=0x7fe92c862e60, mglevelsin=0x7fe92c868a60, reason=0x0) at /Users/markadams/Codes/petsc/src/ksp/pc/impls/mg/mg.c:19 #12 0x0000000110a9e1d5 in PCApply_MG (pc=0x7fe92c862e60, b=0x7fe92d002860, x=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/pc/impls/mg/mg.c:337 #13 0x0000000110ad6f2c in PCApply (pc=0x7fe92c862e60, x=0x7fe92d002860, y=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/pc/interface/precon.c:440 #14 0x0000000110affbd2 in KSP_PCApply (ksp=0x7fe92c833a60, x=0x7fe92d002860, y=0x7fe92d007c60) at kspimpl.h:230 #15 0x0000000110afcd31 in KSPSolve_CG (ksp=0x7fe92c833a60) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cg/cg.c:139 #16 0x0000000110bd81ce in KSPSolve (ksp=0x7fe92c833a60, b=0x0, x=0x0) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:460 #17 0x000000010fafea7d in main () (gdb) frame 8 #8 0x0000000110b6a915 in KSPChebyshevComputeExtremeEigenvalues_Private (kspest=0x7fe92d813e60, emin=0x7fff50102ca0, emax=0x7fff50102ca8) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:328 328 ierr = KSPComputeEigenvalues(kspest,n,re,im,&neig);CHKERRQ(ierr); Current language: auto; currently minimal (gdb) p re $1 = (PetscReal *) 0x0 (gdb) list 323 PetscReal *re,*im,min,max; 324 325 PetscFunctionBegin; 326 ierr = KSPGetIterationNumber(kspest,&n);CHKERRQ(ierr); 327 ierr = PetscMalloc2(n,&re,n,&im);CHKERRQ(ierr); 328 ierr = KSPComputeEigenvalues(kspest,n,re,im,&neig);CHKERRQ(ierr); 329 min = PETSC_MAX_REAL; 330 max = PETSC_MIN_REAL; 331 for (n=0; n<neig; n++) { 332 min = PetscMin(min,re[n]);
On Tue, Oct 21, 2014 at 6:29 PM, Mark Adams <[email protected]> wrote: Humm, GAMG works fine form (like you said), but with MG (with or without Galerkin) on 1 or 4 procs I get:
[0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3
Can you tell me exactly what you get with:
./test1 -pc_type mg
I get:
[0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3 [0]PETSC ERROR: PETSC: Attaching /usr/bin/gdb to ./test1 of pid 50433 on display /tmp/launch-ikNsM1/org.macosforge.xquartz:0 on machine Macintosh-6.local
And, it seems that the RHS is all zeros on one processor.
I am using a debug build. Try to do the same.
On Tue, Oct 21, 2014 at 2:50 AM, Filippo Leonardi <[email protected]> wrote: Here,
run this e.g. with mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin
In Debug mode it crashes, in standard mode it fails. Using Petsc current.
Run without mg and works!
Hope it's not me being stupid and doing something terribly wrong!
Best, Filippo
On Monday 20 October 2014 07:29:04 Barry Smith wrote:
Can you send us the code: to [email protected] or [email protected] ? Or something that reproduces the problem?
Barry
On Oct 20, 2014, at 3:31 AM, Filippo Leonardi <[email protected]> wrote:
Hi,
I have a very specific problem that I cannot figure out with PCMG and multiple solves. I got a linear system that I solve many times, with same matrix but different RHS. I can successfully solve the system with standard techniques, such as default solver or LU or PCGAMG. Even MG works if I destroy the ksp each time or I recompute the matrices at each time. But when I try and go to MG and not recomputing the matrices each time the solver fails. Any idea?
Here some detail: the setup: ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr);
Then I solve as usual, for a large number of time steps: ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); ierr = KSPGetSolution(ksp, &phi);
The solver converges and does that in a reasonable number of iterations: Linear solve converged due to CONVERGED_RTOL iterations 7 And ksp_view and ksp_monitor do not show any weird stuff.
- Weirdly enough using any solver (for instance cg+bjacobi or gamg) everything works (So the matrix and RHS are working fine). - But the problem persists with Galerkin matrices (-pc_mg_galerkin) (So is not a ComputeMatrix problem). - If I do: ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); CHKERRQ(ierr); between each solve or destroy the ksp entirely each time the solution is also perfect (So is not a boundary scaling or other stuff problem). - If I run MG with only 2 levels (so just coarse) I also get a fine result.
Setup RHS is called each time as expected, setup matrix is called just once, also as expected.
The only thing I can think is that MG does not update some value that actually needs to be recomputed.
Any idea?
The solution is not that different from: http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ ex29.c.html
Best, Filippo