December 2014 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] MPICH no longer reliable
by Satish Balay 16 Dec '14

16 Dec '14

I'll report with the current details.. Satish On Tue, 16 Dec 2014, Barry Smith wrote: > > Are the MPICH folks aware that MPICH doesn't successfully run on over 5 % of the systems users have today? Do they care? > > Barry > >

1 0

MPICH no longer reliable
by Barry Smith 16 Dec '14

16 Dec '14

Are the MPICH folks aware that MPICH doesn't successfully run on over 5 % of the systems users have today? Do they care? Barry

1 0

Re: [petsc-dev] MUMPS and 64 bit indices
by Garth N. Wells 16 Dec '14

16 Dec '14

Attached is test code and matrix/vector files to reproduce a problem I'm seeing with Umfpack and 64-bit indices. It works fine with Umfpack and 32-bit indices. The matrix comes from a mixed formulation of the Poisson equation. The matrix is indefinite. I've computed all eigenvalues and they're nonzero. Running the test program with: ./test -pc_factor_mat_solver_package umfpack the output is Solution L2 norm: -nan Solution L2 norm should be: 2.96824 Running the test program with ./test -pc_factor_mat_solver_package superlu_dist the output is Solution L2 norm: 2.968238 Solution L2 norm should be: 2.96824 My PETSc configure line is --with-64-bit-indices --download-hypre=yes --download-suitesparse=1 --download-parmetis --download-metis --with-debugging=no COPTFLAGS="-O3 -march=native" FOPTFLAGS="-O3 -march=native" CXXOPTFLAGS="-O3 -march=native" --prefix=/home/garth/local/packages/petsc-64 --download-ptscotch --download-superlu_dist Git log is 7fbfed63fbc5325cf1eefdb967627c4417715c46 (message copied to petsc-maint so the attachment doesn't get scrubbed) Garth On Mon, 15 Dec, 2014 at 9:49 PM, Garth N. Wells <gnw20(a)cam.ac.uk> wrote: > >> On 15 Dec 2014, at 21:42, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >> >> >>> On Dec 15, 2014, at 3:16 PM, Garth N. Wells <gnw20(a)cam.ac.uk> >>> wrote: >>> >>> It's possible to configure PETSc with the options >>> >>> --with-64-bit-indices --download-mumps >>> >>> and compile and run, but it doesn't look like the MUMPS interface >>> supports 64 bit indices. Should the above combination throw an >>> error at configure time? >> >> Yes, there is commented code in the consistency check, I cannot >> remember why it was commented. >> >> # if self.double and not self.scalartypes.precision.lower() == >> 'double': >> # raise RuntimeError('Cannot use '+self.name+' withOUT >> double precision numbers, it is not coded for this capability') >> # if not self.complex and self.scalartypes.scalartype.lower() >> == 'complex': >> # raise RuntimeError('Cannot use '+self.name+' with complex >> numbers it is not coded for this capability') >> # if self.libraryOptions.integerSize == 64 and >> self.requires32bitint: >> # raise RuntimeError('Cannot use '+self.name+' with 64 bit >> integers, it is not coded for this capability') >> >>> >>> (On a side note, SuperLU_dist and Umfpack are flaky with 64 bit >>> indices, which seems largely related to calling ParMETIS/METIS). >> >> Hmm, never heard such reports. Are you using the >> --download-parmetis and metis? You should be. Is it reproducible >> problem? We'd like to fix it. >> > > Yes, I’m using --download-parmetis and --download-metis. > > Yes, it’s reproducible for a given matrix but hard to know which > matrices will be a problem, and for which solver > (SuperLU_dist/Umfpack). I’ll try to package up a couple of examples > that fail or give wrong results. > > Garth > > >> >> Barry >> >>> >>> Garth >>> >> >

1 0

Broken example ?
by Pierre Jolivet 16 Dec '14

16 Dec '14

When trying to run snes/examples/tutorials/ex55 with the command line given in the source code, i.e. $ ./ex55 -ksp_type fgmres -pc_type fieldsplit -pc_fieldsplit_detect_saddle_point -pc_fieldsplit_type schur -pc_fieldsplit_schur_precondition self -fieldsplit_1_ksp_type fgmres -fieldsplit_1_pc_type lsc -snes_vi_monitor -ksp_monitor_true_residual -fieldsplit_ksp_monitor -fieldsplit_0_pc_type hypre -da_grid_x 65 -da_grid_y 65 -snes_atol 1.e-11 -ksp_rtol 1.e-8 I end up with a crash with these lines in the log: [0]PETSC ERROR: --------------------- Stack Frames ------------------------------------ [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [0]PETSC ERROR: INSTEAD the line number of the start of the function [0]PETSC ERROR: is given. [0]PETSC ERROR: [0] PCFieldSplitSetDefaults line 321petsc/src/ksp/pc/impls/fieldsplit/fieldsplit.c [0]PETSC ERROR: [0] PCSetUp_FieldSplit line 484 petsc/src/ksp/pc/impls/fieldsplit/fieldsplit.c [0]PETSC ERROR: [0] KSPSetUp line 220 petsc/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: [0] SNESSolve_VINEWTONRSLS line 346 petsc/src/snes/impls/vi/rs/virs.c [0]PETSC ERROR: [0] SNESSolve line 3696 petsc/src/snes/interface/snes.c This is happening on two computers (a MBP and a Linux, on the master branch), both have —download-hypre, am I the only one having this error ? Please let me know if so, I’ll send you both configure.log if need be. Thank you, Pierre

1 0

Re: [petsc-dev] MUMPS and 64 bit indices
by Barry Smith 15 Dec '14

15 Dec '14

> On Dec 15, 2014, at 5:57 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > > On Mon, Dec 15, 2014 at 4:17 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > I cannot fix this. There is a cycle in the dependencies if I introduce libraryOptions inside package.py so package.py cannot do the consistency check, but it should so it is totally screwed. > > Matt could fix it but he'd probably have to fuck up the nice simplicity of the code that is there now :-(! > > We were already using a mechanism to get around this circularity. Comments on the aesthetic import welcome. I think the trauma of seeing the provides interface destroyed part of my brain years ago, hence I didn't realize it could be used here. So, yah, if this works its fine and the way to go. Thanks Barry > > https://bitbucket.org/petsc/petsc/branch/knepley/fix-configure-package-check > > Matt > > > Barry > > > On Dec 15, 2014, at 3:42 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > > > > >> On Dec 15, 2014, at 3:16 PM, Garth N. Wells <gnw20(a)cam.ac.uk> wrote: > >> > >> It's possible to configure PETSc with the options > >> > >> --with-64-bit-indices --download-mumps > >> > >> and compile and run, but it doesn't look like the MUMPS interface supports 64 bit indices. Should the above combination throw an error at configure time? > > > > Yes, there is commented code in the consistency check, I cannot remember why it was commented. > > > > # if self.double and not self.scalartypes.precision.lower() == 'double': > > # raise RuntimeError('Cannot use '+self.name+' withOUT double precision numbers, it is not coded for this capability') > > # if not self.complex and self.scalartypes.scalartype.lower() == 'complex': > > # raise RuntimeError('Cannot use '+self.name+' with complex numbers it is not coded for this capability') > > # if self.libraryOptions.integerSize == 64 and self.requires32bitint: > > # raise RuntimeError('Cannot use '+self.name+' with 64 bit integers, it is not coded for this capability') > > > >> > >> (On a side note, SuperLU_dist and Umfpack are flaky with 64 bit indices, which seems largely related to calling ParMETIS/METIS). > > > > Hmm, never heard such reports. Are you using the --download-parmetis and metis? You should be. Is it reproducible problem? We'd like to fix it. > > > > > > Barry > > > >> > >> Garth > >> > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] MUMPS and 64 bit indices
by Matthew Knepley 15 Dec '14

15 Dec '14

On Mon, Dec 15, 2014 at 4:17 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > > I cannot fix this. There is a cycle in the dependencies if I introduce > libraryOptions inside package.py so package.py cannot do the consistency > check, but it should so it is totally screwed. > > Matt could fix it but he'd probably have to fuck up the nice simplicity > of the code that is there now :-(! We were already using a mechanism to get around this circularity. Comments on the aesthetic import welcome. https://bitbucket.org/petsc/petsc/branch/knepley/fix-configure-package-check Matt > > Barry > > > On Dec 15, 2014, at 3:42 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > > > > >> On Dec 15, 2014, at 3:16 PM, Garth N. Wells <gnw20(a)cam.ac.uk> wrote: > >> > >> It's possible to configure PETSc with the options > >> > >> --with-64-bit-indices --download-mumps > >> > >> and compile and run, but it doesn't look like the MUMPS interface > supports 64 bit indices. Should the above combination throw an error at > configure time? > > > > Yes, there is commented code in the consistency check, I cannot > remember why it was commented. > > > > # if self.double and not self.scalartypes.precision.lower() == > 'double': > > # raise RuntimeError('Cannot use '+self.name+' withOUT double > precision numbers, it is not coded for this capability') > > # if not self.complex and self.scalartypes.scalartype.lower() == > 'complex': > > # raise RuntimeError('Cannot use '+self.name+' with complex > numbers it is not coded for this capability') > > # if self.libraryOptions.integerSize == 64 and > self.requires32bitint: > > # raise RuntimeError('Cannot use '+self.name+' with 64 bit > integers, it is not coded for this capability') > > > >> > >> (On a side note, SuperLU_dist and Umfpack are flaky with 64 bit > indices, which seems largely related to calling ParMETIS/METIS). > > > > Hmm, never heard such reports. Are you using the --download-parmetis > and metis? You should be. Is it reproducible problem? We'd like to fix it. > > > > > > Barry > > > >> > >> Garth > >> > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] MUMPS and 64 bit indices
by Barry Smith 15 Dec '14

15 Dec '14

I cannot fix this. There is a cycle in the dependencies if I introduce libraryOptions inside package.py so package.py cannot do the consistency check, but it should so it is totally screwed. Matt could fix it but he'd probably have to fuck up the nice simplicity of the code that is there now :-(! Barry > On Dec 15, 2014, at 3:42 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > >> On Dec 15, 2014, at 3:16 PM, Garth N. Wells <gnw20(a)cam.ac.uk> wrote: >> >> It's possible to configure PETSc with the options >> >> --with-64-bit-indices --download-mumps >> >> and compile and run, but it doesn't look like the MUMPS interface supports 64 bit indices. Should the above combination throw an error at configure time? > > Yes, there is commented code in the consistency check, I cannot remember why it was commented. > > # if self.double and not self.scalartypes.precision.lower() == 'double': > # raise RuntimeError('Cannot use '+self.name+' withOUT double precision numbers, it is not coded for this capability') > # if not self.complex and self.scalartypes.scalartype.lower() == 'complex': > # raise RuntimeError('Cannot use '+self.name+' with complex numbers it is not coded for this capability') > # if self.libraryOptions.integerSize == 64 and self.requires32bitint: > # raise RuntimeError('Cannot use '+self.name+' with 64 bit integers, it is not coded for this capability') > >> >> (On a side note, SuperLU_dist and Umfpack are flaky with 64 bit indices, which seems largely related to calling ParMETIS/METIS). > > Hmm, never heard such reports. Are you using the --download-parmetis and metis? You should be. Is it reproducible problem? We'd like to fix it. > > > Barry > >> >> Garth >> >

1 0

Re: [petsc-dev] MUMPS and 64 bit indices
by Sean Farley 15 Dec '14

15 Dec '14

Barry Smith writes: >> On Dec 15, 2014, at 3:16 PM, Garth N. Wells <gnw20(a)cam.ac.uk> wrote: >> >> It's possible to configure PETSc with the options >> >> --with-64-bit-indices --download-mumps >> >> and compile and run, but it doesn't look like the MUMPS interface supports 64 bit indices. Should the above combination throw an error at configure time? > > Yes, there is commented code in the consistency check, I cannot remember why it was commented. > > # if self.double and not self.scalartypes.precision.lower() == 'double': > # raise RuntimeError('Cannot use '+self.name+' withOUT double precision numbers, it is not coded for this capability') > # if not self.complex and self.scalartypes.scalartype.lower() == 'complex': > # raise RuntimeError('Cannot use '+self.name+' with complex numbers it is not coded for this capability') > # if self.libraryOptions.integerSize == 64 and self.requires32bitint: > # raise RuntimeError('Cannot use '+self.name+' with 64 bit integers, it is not coded for this capability') I think that's from when I added patches to suitesparse, mumps, and superlu for working with 64-bit ints. I wouldn't be surprised if something wasn't working. I believe, to date, only Sherry has incorporated my patches. That will learn me to try to contribute patches to academics.

1 0

Re: [petsc-dev] MUMPS and 64 bit indices
by Garth N. Wells 15 Dec '14

15 Dec '14

> On 15 Dec 2014, at 21:42, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > >> On Dec 15, 2014, at 3:16 PM, Garth N. Wells <gnw20(a)cam.ac.uk> wrote: >> >> It's possible to configure PETSc with the options >> >> --with-64-bit-indices --download-mumps >> >> and compile and run, but it doesn't look like the MUMPS interface supports 64 bit indices. Should the above combination throw an error at configure time? > > Yes, there is commented code in the consistency check, I cannot remember why it was commented. > > # if self.double and not self.scalartypes.precision.lower() == 'double': > # raise RuntimeError('Cannot use '+self.name+' withOUT double precision numbers, it is not coded for this capability') > # if not self.complex and self.scalartypes.scalartype.lower() == 'complex': > # raise RuntimeError('Cannot use '+self.name+' with complex numbers it is not coded for this capability') > # if self.libraryOptions.integerSize == 64 and self.requires32bitint: > # raise RuntimeError('Cannot use '+self.name+' with 64 bit integers, it is not coded for this capability') > >> >> (On a side note, SuperLU_dist and Umfpack are flaky with 64 bit indices, which seems largely related to calling ParMETIS/METIS). > > Hmm, never heard such reports. Are you using the --download-parmetis and metis? You should be. Is it reproducible problem? We'd like to fix it. > Yes, I’m using --download-parmetis and --download-metis. Yes, it’s reproducible for a given matrix but hard to know which matrices will be a problem, and for which solver (SuperLU_dist/Umfpack). I’ll try to package up a couple of examples that fail or give wrong results. Garth > > Barry > >> >> Garth >> >

1 0

Re: [petsc-dev] MUMPS and 64 bit indices
by Barry Smith 15 Dec '14

15 Dec '14

> On Dec 15, 2014, at 3:16 PM, Garth N. Wells <gnw20(a)cam.ac.uk> wrote: > > It's possible to configure PETSc with the options > > --with-64-bit-indices --download-mumps > > and compile and run, but it doesn't look like the MUMPS interface supports 64 bit indices. Should the above combination throw an error at configure time? Yes, there is commented code in the consistency check, I cannot remember why it was commented. # if self.double and not self.scalartypes.precision.lower() == 'double': # raise RuntimeError('Cannot use '+self.name+' withOUT double precision numbers, it is not coded for this capability') # if not self.complex and self.scalartypes.scalartype.lower() == 'complex': # raise RuntimeError('Cannot use '+self.name+' with complex numbers it is not coded for this capability') # if self.libraryOptions.integerSize == 64 and self.requires32bitint: # raise RuntimeError('Cannot use '+self.name+' with 64 bit integers, it is not coded for this capability') > > (On a side note, SuperLU_dist and Umfpack are flaky with 64 bit indices, which seems largely related to calling ParMETIS/METIS). Hmm, never heard such reports. Are you using the --download-parmetis and metis? You should be. Is it reproducible problem? We'd like to fix it. Barry > > Garth >

1 0