On Tue, Apr 10, 2012 at 04:47, Alexander Grayver <agrayver@gfz-potsdam.de> wrote:
Barry,

Well, this is not easy to apply bisection here since "good" petsc-dev had bugs that I patched manually (including Intel MKL BLAS incompatibility) and only after that I was able to run solver successfully. "Bad" petsc-dev has all these issues patched and names of some routines have been changed as well (MatCreateXXX)... However TFQMR doesn't work at all for any system I tried.

Unfortunately, the shortest time gap between "good" and "bad" I can give so far is:

HG revision: a4dae06b4d05ef5b78a9a9aa2cbc31b2b158834f HG Date: Wed Feb 08 11:29:52 2012 -0600 --- GOOD
HG revision: 59ca71b85d784abbe22f12c34ffcb2ccc6dc68b0 HG Date: Tue Mar 20 01:05:13 2012 -0500 --- BAD

I tested them like that:
./ex10 -f A.dat -rhs RHS.dat -ksp_type tfqmr -pc_type sor -ksp_monitor_true_residual -ksp_max_it 500

 95 KSP preconditioned resid norm 2.117351054162e-18 true resid norm 1.981564630346e-15 ||r(i)||/||b|| 3.893342156134e-05   ---- GOOD

500 KSP preconditioned resid norm 3.000274090666e-13 true resid norm 4.577554424627e-11 ||r(i)||/||b|| 8.993895702653e-01   ---- BAD

$ ./ex10 -f ~/dl/A.dat -rhs ~/dl/RHS.dat -ksp_type tfqmr -pc_type sor -ksp_monitor_true_residual
[0]PETSC ERROR: --------------------- Error Message ------------------------------------
[0]PETSC ERROR: Arguments are incompatible!
[0]PETSC ERROR: Zero diagonal on row 44!
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Petsc Development HG revision: 21fb71e3b67102b07ab64b3779f68e8719c17739  HG Date: Mon Mar 19 08:24:25 2012 -0500
[0]PETSC ERROR: See docs/changes/index.html for recent updates.
[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[0]PETSC ERROR: See docs/index.html for manual pages.
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: ./ex10 on a mpich named batura by jed Tue Apr 10 07:09:45 2012
[0]PETSC ERROR: Libraries linked from /home/jed/petsc/mpich/lib
[0]PETSC ERROR: Configure run at Mon Mar 19 09:50:34 2012
[0]PETSC ERROR: Configure options --download-ams --download-blacs --download-chaco --download-hypre --download-metis --download-ml --download-parmetis --download-spai --download-spooles --download-sundials --download-superlu --download-superlu_dist --download-triangle --with-blas-lapack=/usr --with-c2html --with-cholmod-dir=/usr --with-lgrind --with-mpi-dir=/opt/mpich2 --with-pcbddc --with-pthreadclasses --with-shared-libraries --with-single-library=0 --with-sowing --with-umfpack-dir=/usr --with-x -PETSC_ARCH=mpich
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: MatInvertDiagonal_SeqAIJ() line 1315 in /home/jed/petsc/src/mat/impls/aij/seq/aij.c
[0]PETSC ERROR: MatSOR_SeqAIJ() line 1348 in /home/jed/petsc/src/mat/impls/aij/seq/aij.c
[0]PETSC ERROR: MatSOR() line 3671 in /home/jed/petsc/src/mat/interface/matrix.c
[0]PETSC ERROR: PCApply_SOR() line 35 in /home/jed/petsc/src/ksp/pc/impls/sor/sor.c
[0]PETSC ERROR: PCApply() line 384 in /home/jed/petsc/src/ksp/pc/interface/precon.c
[0]PETSC ERROR: KSPInitialResidual() line 64 in /home/jed/petsc/src/ksp/ksp/interface/itres.c
[0]PETSC ERROR: KSPSolve_TFQMR() line 41 in /home/jed/petsc/src/ksp/ksp/impls/tfqmr/tfqmr.c
[0]PETSC ERROR: KSPSolve() line 434 in /home/jed/petsc/src/ksp/ksp/interface/itfunc.c
[0]PETSC ERROR: main() line 327 in src/ksp/ksp/examples/tutorials/ex10.c
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0

There is no space allocated at (44,44); note that counting is 0-based. This consistency check has been in PETSc for many years, so I don't know why you didn't trip it in earlier runs. Do you have some options in PETSC_OPTIONS or .petscrc?
 

If it is of any help files are here:
http://dl.dropbox.com/u/60982984/A.dat
http://dl.dropbox.com/u/60982984/RHS.dat

Is this possible to look up changes in one file only?

$ hg log src/ksp/ksp/impls/tfqmr/

But the problem was clearly not in that file.