Garth, I have found the bug and it is ours. In SuiteSparse.py we set ulong_max = 'ULONG_MAX' which is totally wrong because SuiteSparse is looking for the largest SIGNED long. So if you change the ULONG_MAX to LONG_MAX in SuiteSparse.py and rerun the ./configure (after deleting your $PETSC_DIR/$PETSC_ARCH directory) the 64 bit index version will run the same as the 32 bit version. I have fixed this in the branches maint and master Barry

On Dec 16, 2014, at 2:52 PM, Garth N. Wells <[email protected]> wrote:

Attached is test code and matrix/vector files to reproduce a problem I'm seeing with Umfpack and 64-bit indices. It works fine with Umfpack and 32-bit indices. The matrix comes from a mixed formulation of the Poisson equation. The matrix is indefinite. I've computed all eigenvalues and they're nonzero.

Running the test program with:

./test -pc_factor_mat_solver_package umfpack

the output is

Solution L2 norm: -nan Solution L2 norm should be: 2.96824

Running the test program with

./test -pc_factor_mat_solver_package superlu_dist

the output is

Solution L2 norm: 2.968238 Solution L2 norm should be: 2.96824

My PETSc configure line is

--with-64-bit-indices --download-hypre=yes --download-suitesparse=1 --download-parmetis --download-metis --with-debugging=no COPTFLAGS="-O3 -march=native" FOPTFLAGS="-O3 -march=native" CXXOPTFLAGS="-O3 -march=native" --prefix=/home/garth/local/packages/petsc-64 --download-ptscotch --download-superlu_dist

Git log is 7fbfed63fbc5325cf1eefdb967627c4417715c46

(message copied to petsc-maint so the attachment doesn't get scrubbed)

Garth

On Mon, 15 Dec, 2014 at 9:49 PM, Garth N. Wells <[email protected]> wrote:

...

...

On 15 Dec 2014, at 21:42, Barry Smith <[email protected]> wrote:

...

On Dec 15, 2014, at 3:16 PM, Garth N. Wells <[email protected]> wrote: It's possible to configure PETSc with the options --with-64-bit-indices --download-mumps and compile and run, but it doesn't look like the MUMPS interface supports 64 bit indices. Should the above combination throw an error at configure time? Yes, there is commented code in the consistency check, I cannot remember why it was commented. # if self.double and not self.scalartypes.precision.lower() == 'double': # raise RuntimeError('Cannot use '+self.name+' withOUT double precision numbers, it is not coded for this capability') # if not self.complex and self.scalartypes.scalartype.lower() == 'complex': # raise RuntimeError('Cannot use '+self.name+' with complex numbers it is not coded for this capability') # if self.libraryOptions.integerSize == 64 and self.requires32bitint: # raise RuntimeError('Cannot use '+self.name+' with 64 bit integers, it is not coded for this capability') (On a side note, SuperLU_dist and Umfpack are flaky with 64 bit indices, which seems largely related to calling ParMETIS/METIS). Hmm, never heard such reports. Are you using the --download-parmetis and metis? You should be. Is it reproducible problem? We'd like to fix it. Yes, I’m using --download-parmetis and --download-metis. Yes, it’s reproducible for a given matrix but hard to know which matrices will be a problem, and for which solver (SuperLU_dist/Umfpack). I’ll try to package up a couple of examples that fail or give wrong results. Garth Barry Garth <petsc_lu_test.tgz>