April 2011 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] DACreate3d should work for 1 node in Z
by Ethan Coon 19 Apr '11

19 Apr '11

Ah, the 2D case provides an error message for this case, but not the 3d. [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Argument out of range! [0]PETSC ERROR: Local y-width of domain y 1 is smaller than stencil width s 2! [0]PETSC ERROR: The problem is less about the 1-node-wide domain with 1 proc in that dimension, and more about the case where there are more than one procs in that dimension but less than the stencil size local unknowns. To handle this, the scatter would have to look two proc neighbors away, not just one. This has never been handled by PETSc. Consider the z-case with local size zl and stencil size s... I supposed one could check for the special case where P, the number of procs, in the z-dimension is 1 and s > zl. In that case, DMDA_BOUNDARY_GHOSTED and DMDA_BOUNDARY_NONE both work (as evidenced by the 3D code). Then, it need only error if (s > zl) and ((P > 1) or (zwrap == DMDA_BOUNDARY_PERIODIC)) Alternatively, DMDA_BOUNDARY_PERIODIC could be made to work with more if statements to set up the indices for the scatter, doing the wrap around of the domain multiple times. Not sure how much work this would be. At any rate, I'll fix the error message in 2D and add it to 3D to only error in the cases that actually currently break. Doing DMDA_BOUNDARY_PERIODIC with P=1 and s > zl (the multiple-wraps around the domain case) will take more thought... Ethan On Tue, 2011-04-19 at 21:25 +0200, Jed Brown wrote: > On Tue, 19 Apr 2011 09:18:44 -0600, Ethan Coon <ecoon(a)lanl.gov> wrote: > > Can you shoot me your example? I run 1-node z direction both with and > > without periodic BCs, and with and without ghost nodes, without a > > valgrind complaint in DMDACreate3D. > > See attached very simple test case. It only crashes for stencil width > greater than 1 (the original support request needs stencil width of 2 > for TVD FV methods), > > mpiexec -n 1 ./a.out # stencil width 2 in code > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: Argument out of range! > [0]PETSC ERROR: Negative index -100 at 0 location! > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development HG revision: 335e836c7798747e6242df554ae58fe4db9f803f HG Date: Fri Apr 15 16:04:07 2011 -0500 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: ./a.out on a ompi named kunyang by jed Tue Apr 19 21:22:07 2011 > [0]PETSC ERROR: Libraries linked from /home/jed/petsc/ompi/lib > [0]PETSC ERROR: Configure run at Sat Apr 16 13:36:39 2011 > [0]PETSC ERROR: Configure options --download-blacs --download-chaco --download-hypre=externalpackages/hypre-2.7.0b.tar.gz --download-ml --download-mum > ps --download-parms --download-scalapack --download-scotch --download-spai --download-spooles --download-sundials --download-superlu --download-superl > u_dist --with-blas-lapack-dir=/usr --with-c2html --with-cholmod-dir=/usr --with-clib-autodetect=0 --with-cxxlib-autodetect=0 --with-fortranlib-autodet > ect=0 --with-hdf5-dir=/usr --with-java-dir=/usr --with-lgrind --with-mpi-dir=/usr --with-parmetis-dir=/usr --with-shared-libraries --with-single-libra > ry=0 --with-sowing --with-umfpack-dir=/usr -LIBS="-L/usr/lib/openmpi -lmpi_f90 -lmpi_f77 -lmpi_cxx -lstdc++ -lgfortran" PETSC_ARCH=ompi > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: VecScatterCheckIndices_Private() line 26 in /home/jed/petsc/src/vec/vec/utils/vscat.c > [0]PETSC ERROR: VecScatterCreate() line 1106 in /home/jed/petsc/src/vec/vec/utils/vscat.c > [0]PETSC ERROR: DMSetUp_DA_3D() line 953 in /home/jed/petsc/src/dm/impls/da/da3.c > [0]PETSC ERROR: DMSetUp_DA() line 22 in /home/jed/petsc/src/dm/impls/da/dareg.c > [0]PETSC ERROR: DMSetUp() line 189 in /home/jed/petsc/src/dm/interface/dm.c > [0]PETSC ERROR: DMDACreate3d() line 1366 in /home/jed/petsc/src/dm/impls/da/da3.c > [0]PETSC ERROR: main() line 14 in "unknowndirectory/"da3d.c > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > with errorcode 63. > > > I still see Valgrind noise for stencil width of 1: > > mpiexec -n 2 valgind ./a.out # stencil width 1 in code > > ==26044== Invalid read of size 2 > ==26044== at 0x4C26C2C: memcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==26044== by 0x71D8D01: opal_convertor_pack (in /usr/lib/openmpi/libmpi.so.1.0.1) > ==26044== by 0x117600C8: ??? (in /usr/lib/openmpi/openmpi/mca_btl_sm.so) > ==26044== by 0x10F421B7: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) > ==26044== by 0x10F30C0C: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) > ==26044== by 0x7127650: PMPI_Isend (in /usr/lib/openmpi/libmpi.so.1.0.1) > ==26044== by 0x6C5A103: PetscGatherMessageLengths (mpimesg.c:127) > ==26044== by 0x697A90A: VecScatterCreate_PtoS (vpscat.c:1623) > ==26044== by 0x69564FD: VecScatterCreate (vscat.c:1334) > ==26044== by 0x7CB339A: DMSetUp_DA_3D (da3.c:953) > ==26044== by 0x7D044BF: DMSetUp_DA (dareg.c:22) > ==26044== by 0x7C76A6C: DMSetUp (dm.c:189) > ==26044== Address 0xdfeda02 is 2 bytes inside a block of size 8 alloc'd > ==26044== at 0x4C23918: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==26044== by 0x6CC9DEF: PetscMallocAlign (mal.c:30) > ==26044== by 0x697A5EA: VecScatterCreate_PtoS (vpscat.c:1603) > ==26044== by 0x69564FD: VecScatterCreate (vscat.c:1334) > ==26044== by 0x7CB339A: DMSetUp_DA_3D (da3.c:953) > ==26044== by 0x7D044BF: DMSetUp_DA (dareg.c:22) > ==26044== by 0x7C76A6C: DMSetUp (dm.c:189) > ==26044== by 0x7CB6D8D: DMDACreate3d (da3.c:1366) > ==26044== by 0x400938: main (in /home/jed/lang/c/petsc/a.out) > > C source code attachment (da3d.c) > #include <petscdmda.h> > > int main(int argc, char *argv[]) > { > DM da; > PetscErrorCode ierr; > > PetscInitialize(&argc,&argv,0,0); > ierr = DMDACreate3d(PETSC_COMM_WORLD, > DMDA_BOUNDARY_PERIODIC, DMDA_BOUNDARY_PERIODIC, DMDA_BOUNDARY_PERIODIC, > DMDA_STENCIL_STAR, > 10, 10, 1, > PETSC_DECIDE, PETSC_DECIDE, PETSC_DECIDE, 1, 1, /* Stencil width of 2 in original request */ > PETSC_NULL, PETSC_NULL, PETSC_NULL, &da);CHKERRQ(ierr); > ierr = DMDestroy(da);CHKERRQ(ierr); > PetscFinalize(); > return 0; > } -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------

1 0

Re: [petsc-dev] DACreate3d should work for 1 node in Z
by Jed Brown 19 Apr '11

19 Apr '11

On Tue, Apr 19, 2011 at 21:47, Ethan Coon <ecoon(a)lanl.gov> wrote: > Hmm, crap. I don't get your valgrind error > On further analysis, this looks like an Open MPI bug since it does not occur when I run the same thing with MPICH2. Unfortunately, I don't have time to reduce it to submit a good bug report. > but I do get the crash with > stencil size 2. > Thanks for looking into this.

1 0

Re: [petsc-dev] DACreate3d should work for 1 node in Z
by Ethan Coon 19 Apr '11

19 Apr '11

Hmm, crap. I don't get your valgrind error but I do get the crash with stencil size 2. Will get back to you, thanks, Ethan On Tue, 2011-04-19 at 21:25 +0200, Jed Brown wrote: > On Tue, 19 Apr 2011 09:18:44 -0600, Ethan Coon <ecoon(a)lanl.gov> wrote: > > Can you shoot me your example? I run 1-node z direction both with and > > without periodic BCs, and with and without ghost nodes, without a > > valgrind complaint in DMDACreate3D. > > See attached very simple test case. It only crashes for stencil width > greater than 1 (the original support request needs stencil width of 2 > for TVD FV methods), > > mpiexec -n 1 ./a.out # stencil width 2 in code > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: Argument out of range! > [0]PETSC ERROR: Negative index -100 at 0 location! > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development HG revision: 335e836c7798747e6242df554ae58fe4db9f803f HG Date: Fri Apr 15 16:04:07 2011 -0500 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: ./a.out on a ompi named kunyang by jed Tue Apr 19 21:22:07 2011 > [0]PETSC ERROR: Libraries linked from /home/jed/petsc/ompi/lib > [0]PETSC ERROR: Configure run at Sat Apr 16 13:36:39 2011 > [0]PETSC ERROR: Configure options --download-blacs --download-chaco --download-hypre=externalpackages/hypre-2.7.0b.tar.gz --download-ml --download-mum > ps --download-parms --download-scalapack --download-scotch --download-spai --download-spooles --download-sundials --download-superlu --download-superl > u_dist --with-blas-lapack-dir=/usr --with-c2html --with-cholmod-dir=/usr --with-clib-autodetect=0 --with-cxxlib-autodetect=0 --with-fortranlib-autodet > ect=0 --with-hdf5-dir=/usr --with-java-dir=/usr --with-lgrind --with-mpi-dir=/usr --with-parmetis-dir=/usr --with-shared-libraries --with-single-libra > ry=0 --with-sowing --with-umfpack-dir=/usr -LIBS="-L/usr/lib/openmpi -lmpi_f90 -lmpi_f77 -lmpi_cxx -lstdc++ -lgfortran" PETSC_ARCH=ompi > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: VecScatterCheckIndices_Private() line 26 in /home/jed/petsc/src/vec/vec/utils/vscat.c > [0]PETSC ERROR: VecScatterCreate() line 1106 in /home/jed/petsc/src/vec/vec/utils/vscat.c > [0]PETSC ERROR: DMSetUp_DA_3D() line 953 in /home/jed/petsc/src/dm/impls/da/da3.c > [0]PETSC ERROR: DMSetUp_DA() line 22 in /home/jed/petsc/src/dm/impls/da/dareg.c > [0]PETSC ERROR: DMSetUp() line 189 in /home/jed/petsc/src/dm/interface/dm.c > [0]PETSC ERROR: DMDACreate3d() line 1366 in /home/jed/petsc/src/dm/impls/da/da3.c > [0]PETSC ERROR: main() line 14 in "unknowndirectory/"da3d.c > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > with errorcode 63. > > > I still see Valgrind noise for stencil width of 1: > > mpiexec -n 2 valgind ./a.out # stencil width 1 in code > > ==26044== Invalid read of size 2 > ==26044== at 0x4C26C2C: memcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==26044== by 0x71D8D01: opal_convertor_pack (in /usr/lib/openmpi/libmpi.so.1.0.1) > ==26044== by 0x117600C8: ??? (in /usr/lib/openmpi/openmpi/mca_btl_sm.so) > ==26044== by 0x10F421B7: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) > ==26044== by 0x10F30C0C: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) > ==26044== by 0x7127650: PMPI_Isend (in /usr/lib/openmpi/libmpi.so.1.0.1) > ==26044== by 0x6C5A103: PetscGatherMessageLengths (mpimesg.c:127) > ==26044== by 0x697A90A: VecScatterCreate_PtoS (vpscat.c:1623) > ==26044== by 0x69564FD: VecScatterCreate (vscat.c:1334) > ==26044== by 0x7CB339A: DMSetUp_DA_3D (da3.c:953) > ==26044== by 0x7D044BF: DMSetUp_DA (dareg.c:22) > ==26044== by 0x7C76A6C: DMSetUp (dm.c:189) > ==26044== Address 0xdfeda02 is 2 bytes inside a block of size 8 alloc'd > ==26044== at 0x4C23918: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==26044== by 0x6CC9DEF: PetscMallocAlign (mal.c:30) > ==26044== by 0x697A5EA: VecScatterCreate_PtoS (vpscat.c:1603) > ==26044== by 0x69564FD: VecScatterCreate (vscat.c:1334) > ==26044== by 0x7CB339A: DMSetUp_DA_3D (da3.c:953) > ==26044== by 0x7D044BF: DMSetUp_DA (dareg.c:22) > ==26044== by 0x7C76A6C: DMSetUp (dm.c:189) > ==26044== by 0x7CB6D8D: DMDACreate3d (da3.c:1366) > ==26044== by 0x400938: main (in /home/jed/lang/c/petsc/a.out) > > C source code attachment (da3d.c) > #include <petscdmda.h> > > int main(int argc, char *argv[]) > { > DM da; > PetscErrorCode ierr; > > PetscInitialize(&argc,&argv,0,0); > ierr = DMDACreate3d(PETSC_COMM_WORLD, > DMDA_BOUNDARY_PERIODIC, DMDA_BOUNDARY_PERIODIC, DMDA_BOUNDARY_PERIODIC, > DMDA_STENCIL_STAR, > 10, 10, 1, > PETSC_DECIDE, PETSC_DECIDE, PETSC_DECIDE, 1, 1, /* Stencil width of 2 in original request */ > PETSC_NULL, PETSC_NULL, PETSC_NULL, &da);CHKERRQ(ierr); > ierr = DMDestroy(da);CHKERRQ(ierr); > PetscFinalize(); > return 0; > } -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------

1 0

Re: [petsc-dev] DACreate3d should work for 1 node in Z
by Jed Brown 19 Apr '11

19 Apr '11

On Tue, 19 Apr 2011 09:18:44 -0600, Ethan Coon <ecoon(a)lanl.gov> wrote: > Can you shoot me your example? I run 1-node z direction both with and > without periodic BCs, and with and without ghost nodes, without a > valgrind complaint in DMDACreate3D. See attached very simple test case. It only crashes for stencil width greater than 1 (the original support request needs stencil width of 2 for TVD FV methods), mpiexec -n 1 ./a.out # stencil width 2 in code [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Argument out of range! [0]PETSC ERROR: Negative index -100 at 0 location! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: 335e836c7798747e6242df554ae58fe4db9f803f HG Date: Fri Apr 15 16:04:07 2011 -0500 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./a.out on a ompi named kunyang by jed Tue Apr 19 21:22:07 2011 [0]PETSC ERROR: Libraries linked from /home/jed/petsc/ompi/lib [0]PETSC ERROR: Configure run at Sat Apr 16 13:36:39 2011 [0]PETSC ERROR: Configure options --download-blacs --download-chaco --download-hypre=externalpackages/hypre-2.7.0b.tar.gz --download-ml --download-mum ps --download-parms --download-scalapack --download-scotch --download-spai --download-spooles --download-sundials --download-superlu --download-superl u_dist --with-blas-lapack-dir=/usr --with-c2html --with-cholmod-dir=/usr --with-clib-autodetect=0 --with-cxxlib-autodetect=0 --with-fortranlib-autodet ect=0 --with-hdf5-dir=/usr --with-java-dir=/usr --with-lgrind --with-mpi-dir=/usr --with-parmetis-dir=/usr --with-shared-libraries --with-single-libra ry=0 --with-sowing --with-umfpack-dir=/usr -LIBS="-L/usr/lib/openmpi -lmpi_f90 -lmpi_f77 -lmpi_cxx -lstdc++ -lgfortran" PETSC_ARCH=ompi [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: VecScatterCheckIndices_Private() line 26 in /home/jed/petsc/src/vec/vec/utils/vscat.c [0]PETSC ERROR: VecScatterCreate() line 1106 in /home/jed/petsc/src/vec/vec/utils/vscat.c [0]PETSC ERROR: DMSetUp_DA_3D() line 953 in /home/jed/petsc/src/dm/impls/da/da3.c [0]PETSC ERROR: DMSetUp_DA() line 22 in /home/jed/petsc/src/dm/impls/da/dareg.c [0]PETSC ERROR: DMSetUp() line 189 in /home/jed/petsc/src/dm/interface/dm.c [0]PETSC ERROR: DMDACreate3d() line 1366 in /home/jed/petsc/src/dm/impls/da/da3.c [0]PETSC ERROR: main() line 14 in "unknowndirectory/"da3d.c -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 63. I still see Valgrind noise for stencil width of 1: mpiexec -n 2 valgind ./a.out # stencil width 1 in code ==26044== Invalid read of size 2 ==26044== at 0x4C26C2C: memcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==26044== by 0x71D8D01: opal_convertor_pack (in /usr/lib/openmpi/libmpi.so.1.0.1) ==26044== by 0x117600C8: ??? (in /usr/lib/openmpi/openmpi/mca_btl_sm.so) ==26044== by 0x10F421B7: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) ==26044== by 0x10F30C0C: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) ==26044== by 0x7127650: PMPI_Isend (in /usr/lib/openmpi/libmpi.so.1.0.1) ==26044== by 0x6C5A103: PetscGatherMessageLengths (mpimesg.c:127) ==26044== by 0x697A90A: VecScatterCreate_PtoS (vpscat.c:1623) ==26044== by 0x69564FD: VecScatterCreate (vscat.c:1334) ==26044== by 0x7CB339A: DMSetUp_DA_3D (da3.c:953) ==26044== by 0x7D044BF: DMSetUp_DA (dareg.c:22) ==26044== by 0x7C76A6C: DMSetUp (dm.c:189) ==26044== Address 0xdfeda02 is 2 bytes inside a block of size 8 alloc'd ==26044== at 0x4C23918: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==26044== by 0x6CC9DEF: PetscMallocAlign (mal.c:30) ==26044== by 0x697A5EA: VecScatterCreate_PtoS (vpscat.c:1603) ==26044== by 0x69564FD: VecScatterCreate (vscat.c:1334) ==26044== by 0x7CB339A: DMSetUp_DA_3D (da3.c:953) ==26044== by 0x7D044BF: DMSetUp_DA (dareg.c:22) ==26044== by 0x7C76A6C: DMSetUp (dm.c:189) ==26044== by 0x7CB6D8D: DMDACreate3d (da3.c:1366) ==26044== by 0x400938: main (in /home/jed/lang/c/petsc/a.out)

1 0

Re: [petsc-dev] DACreate3d should work for 1 node in Z
by Ethan Coon 19 Apr '11

19 Apr '11

Can you shoot me your example? I run 1-node z direction both with and without periodic BCs, and with and without ghost nodes, without a valgrind complaint in DMDACreate3D. I don't have a snes though, and it's done in C, not C++. On Tue, 2011-04-19 at 17:09 +0200, Jed Brown wrote: > On Tue, Apr 19, 2011 at 16:56, Ethan Coon <ecoon(a)lanl.gov> wrote: > This is in 3.1 I assume? > > Yeah, it's a problem in petsc-dev too (only tested with all periodic > BCs). I get a seg-fault in > > > [0]PETSC ERROR: [0] VecScatterBegin_1 line > 22 /home/jed/petsc/include/../src/vec/vec/utils/vpscat.h > [0]PETSC ERROR: [0] VecScatterBegin line > 1546 /home/jed/petsc/src/vec/vec/utils/vscat.c > [0]PETSC ERROR: [0] DMGlobalToLocalBegin_DA line > 15 /home/jed/petsc/src/dm/impls/da/dagtol.c > [0]PETSC ERROR: [0] DMGlobalToLocalBegin line > 630 /home/jed/petsc/src/dm/interface/dm.c > [0]PETSC ERROR: [0] pci_function line 58 > "unknowndirectory/"divclean.cpp > [0]PETSC ERROR: [0] SNES user function line 0 unknownunknown > [0]PETSC ERROR: [0] SNESComputeFunction line > 1152 /home/jed/petsc/src/snes/interface/snes.c > [0]PETSC ERROR: [0] SNESSolve_LS line > 142 /home/jed/petsc/src/snes/impls/ls/ls.c > [0]PETSC ERROR: [0] SNESSolve line > 2385 /home/jed/petsc/src/snes/interface/snes.c > > > > > earlier valgrind issues > > > ==14160== Invalid read of size 2 > ==14160== at 0x4C26C2C: memcpy > (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==14160== by 0x7589D01: opal_convertor_pack > (in /usr/lib/openmpi/libmpi.so.1.0.1) > ==14160== by 0x114B90C8: ??? > (in /usr/lib/openmpi/openmpi/mca_btl_sm.so) > ==14160== by 0x10C9B1B7: ??? > (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) > ==14160== by 0x10C89C0C: ??? > (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) > ==14160== by 0x74D8650: PMPI_Isend > (in /usr/lib/openmpi/libmpi.so.1.0.1) > ==14160== by 0x700B103: PetscGatherMessageLengths (mpimesg.c:127) > ==14160== by 0x6D2B90A: VecScatterCreate_PtoS (vpscat.c:1623) > ==14160== by 0x6D074FD: VecScatterCreate (vscat.c:1334) > ==14160== by 0x5C5939A: DMSetUp_DA_3D (da3.c:953) > ==14160== by 0x5CAA4BF: DMSetUp_DA (dareg.c:22) > ==14160== by 0x5C1CA6C: DMSetUp (dm.c:189) > ==14160== Address 0x12addbb2 is 2 bytes inside a block of size 16 > alloc'd > ==14160== at 0x4C23918: memalign > (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==14160== by 0x707ADEF: PetscMallocAlign (mal.c:30) > ==14160== by 0x6D2B5EA: VecScatterCreate_PtoS (vpscat.c:1603) > ==14160== by 0x6D074FD: VecScatterCreate (vscat.c:1334) > ==14160== by 0x5C5939A: DMSetUp_DA_3D (da3.c:953) > ==14160== by 0x5CAA4BF: DMSetUp_DA (dareg.c:22) > ==14160== by 0x5C1CA6C: DMSetUp (dm.c:189) > ==14160== by 0x5C5CD8D: DMDACreate3d (da3.c:1366) > ==14160== by 0x406608: multi_init(int, char**) > (multi_petsc.cpp:107) > ==14160== by 0x403A67: main (mhd_main.cpp:351) > > > ==14160== Invalid read of size 8 > ==14160== at 0x6D15283: Pack_1 (vpscat.c:498) > ==14160== by 0x6D1BFC2: VecScatterBegin_1 (vpscat.h:100) > ==14160== by 0x6D0A483: VecScatterBegin (vscat.c:1575) > ==14160== by 0x5C62E24: DMGlobalToLocalBegin_DA (dagtol.c:19) > ==14160== by 0x5C1FFE3: DMGlobalToLocalBegin (dm.c:631) > ==14160== by 0x40B408: pci_function(_p_SNES*, _p_Vec*, _p_Vec*, > void*) (divclean.cpp:61) > ==14160== by 0x4E8D9B5: SNESComputeFunction (snes.c:1162) > ==14160== by 0x4E74BB1: SNESSolve_LS (ls.c:158) > ==14160== by 0x4E9A27C: SNESSolve (snes.c:2407) > ==14160== by 0x40AB6F: divclean() (divclean.cpp:27) > ==14160== by 0x403AFD: main (mhd_main.cpp:365) > ==14160== Address 0x12af1570 is 0 bytes after a block of size 800 > alloc'd > ==14160== at 0x4C23918: memalign > (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==14160== by 0x707ADEF: PetscMallocAlign (mal.c:30) > ==14160== by 0x6CD46A4: VecCreate_MPI_Private (pbvec.c:409) > ==14160== by 0x6CD4BF7: VecCreate_MPI (pbvec.c:453) > ==14160== by 0x6C8FB00: VecSetType (vecreg.c:53) > ==14160== by 0x6CD4F46: VecCreate_Standard (pbvec.c:482) > ==14160== by 0x6C8FB00: VecSetType (vecreg.c:53) > ==14160== by 0x5C738AB: DMCreateGlobalVector_DA (dadist.c:34) > ==14160== by 0x5C1D2D4: DMCreateGlobalVector (dm.c:277) > ==14160== by 0x5C732FE: VecDuplicate_MPI_DA (dadist.c:17) > ==14160== by 0x6C81665: VecDuplicate (vector.c:485) > ==14160== by 0x40C565: pci_init() (divclean.cpp:177) > > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------

1 0

Re: [petsc-dev] DACreate3d should work for 1 node in Z
by Jed Brown 19 Apr '11

19 Apr '11

On Tue, Apr 19, 2011 at 16:56, Ethan Coon <ecoon(a)lanl.gov> wrote: > This is in 3.1 I assume? Yeah, it's a problem in petsc-dev too (only tested with all periodic BCs). I get a seg-fault in [0]PETSC ERROR: [0] VecScatterBegin_1 line 22 /home/jed/petsc/include/../src/vec/vec/utils/vpscat.h [0]PETSC ERROR: [0] VecScatterBegin line 1546 /home/jed/petsc/src/vec/vec/utils/vscat.c [0]PETSC ERROR: [0] DMGlobalToLocalBegin_DA line 15 /home/jed/petsc/src/dm/impls/da/dagtol.c [0]PETSC ERROR: [0] DMGlobalToLocalBegin line 630 /home/jed/petsc/src/dm/interface/dm.c [0]PETSC ERROR: [0] pci_function line 58 "unknowndirectory/"divclean.cpp [0]PETSC ERROR: [0] SNES user function line 0 unknownunknown [0]PETSC ERROR: [0] SNESComputeFunction line 1152 /home/jed/petsc/src/snes/interface/snes.c [0]PETSC ERROR: [0] SNESSolve_LS line 142 /home/jed/petsc/src/snes/impls/ls/ls.c [0]PETSC ERROR: [0] SNESSolve line 2385 /home/jed/petsc/src/snes/interface/snes.c earlier valgrind issues ==14160== Invalid read of size 2 ==14160== at 0x4C26C2C: memcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==14160== by 0x7589D01: opal_convertor_pack (in /usr/lib/openmpi/libmpi.so.1.0.1) ==14160== by 0x114B90C8: ??? (in /usr/lib/openmpi/openmpi/mca_btl_sm.so) ==14160== by 0x10C9B1B7: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) ==14160== by 0x10C89C0C: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) ==14160== by 0x74D8650: PMPI_Isend (in /usr/lib/openmpi/libmpi.so.1.0.1) ==14160== by 0x700B103: PetscGatherMessageLengths (mpimesg.c:127) ==14160== by 0x6D2B90A: VecScatterCreate_PtoS (vpscat.c:1623) ==14160== by 0x6D074FD: VecScatterCreate (vscat.c:1334) ==14160== by 0x5C5939A: DMSetUp_DA_3D (da3.c:953) ==14160== by 0x5CAA4BF: DMSetUp_DA (dareg.c:22) ==14160== by 0x5C1CA6C: DMSetUp (dm.c:189) ==14160== Address 0x12addbb2 is 2 bytes inside a block of size 16 alloc'd ==14160== at 0x4C23918: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==14160== by 0x707ADEF: PetscMallocAlign (mal.c:30) ==14160== by 0x6D2B5EA: VecScatterCreate_PtoS (vpscat.c:1603) ==14160== by 0x6D074FD: VecScatterCreate (vscat.c:1334) ==14160== by 0x5C5939A: DMSetUp_DA_3D (da3.c:953) ==14160== by 0x5CAA4BF: DMSetUp_DA (dareg.c:22) ==14160== by 0x5C1CA6C: DMSetUp (dm.c:189) ==14160== by 0x5C5CD8D: DMDACreate3d (da3.c:1366) ==14160== by 0x406608: multi_init(int, char**) (multi_petsc.cpp:107) ==14160== by 0x403A67: main (mhd_main.cpp:351) ==14160== Invalid read of size 8 ==14160== at 0x6D15283: Pack_1 (vpscat.c:498) ==14160== by 0x6D1BFC2: VecScatterBegin_1 (vpscat.h:100) ==14160== by 0x6D0A483: VecScatterBegin (vscat.c:1575) ==14160== by 0x5C62E24: DMGlobalToLocalBegin_DA (dagtol.c:19) ==14160== by 0x5C1FFE3: DMGlobalToLocalBegin (dm.c:631) ==14160== by 0x40B408: pci_function(_p_SNES*, _p_Vec*, _p_Vec*, void*) (divclean.cpp:61) ==14160== by 0x4E8D9B5: SNESComputeFunction (snes.c:1162) ==14160== by 0x4E74BB1: SNESSolve_LS (ls.c:158) ==14160== by 0x4E9A27C: SNESSolve (snes.c:2407) ==14160== by 0x40AB6F: divclean() (divclean.cpp:27) ==14160== by 0x403AFD: main (mhd_main.cpp:365) ==14160== Address 0x12af1570 is 0 bytes after a block of size 800 alloc'd ==14160== at 0x4C23918: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==14160== by 0x707ADEF: PetscMallocAlign (mal.c:30) ==14160== by 0x6CD46A4: VecCreate_MPI_Private (pbvec.c:409) ==14160== by 0x6CD4BF7: VecCreate_MPI (pbvec.c:453) ==14160== by 0x6C8FB00: VecSetType (vecreg.c:53) ==14160== by 0x6CD4F46: VecCreate_Standard (pbvec.c:482) ==14160== by 0x6C8FB00: VecSetType (vecreg.c:53) ==14160== by 0x5C738AB: DMCreateGlobalVector_DA (dadist.c:34) ==14160== by 0x5C1D2D4: DMCreateGlobalVector (dm.c:277) ==14160== by 0x5C732FE: VecDuplicate_MPI_DA (dadist.c:17) ==14160== by 0x6C81665: VecDuplicate (vector.c:485) ==14160== by 0x40C565: pci_init() (divclean.cpp:177)

1 0

Re: [petsc-dev] DACreate3d should work for 1 node in Z
by Ethan Coon 19 Apr '11

19 Apr '11

This is in 3.1 I assume? Ethan On Tue, 2011-04-19 at 16:13 +0200, Jed Brown wrote: > This would make it easier to solve 2D problems with a 3D code. It > currently crashes or produces NaN because the scatter is set up > incorrectly: > > > ==10402== Invalid read of size 2 > ==10402== at 0x4C26C2C: memcpy > (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==10402== by 0x6D87D01: opal_convertor_pack > (in /usr/lib/openmpi/libmpi.so.1.0.1) > ==10402== by 0xF33D0C8: ??? > (in /usr/lib/openmpi/openmpi/mca_btl_sm.so) > ==10402== by 0xEB1F1B7: ??? > (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) > ==10402== by 0xEB0DC0C: ??? > (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) > ==10402== by 0x6CD6650: PMPI_Isend > (in /usr/lib/openmpi/libmpi.so.1.0.1) > ==10402== by 0x551F85A: PetscGatherMessageLengths (mpimesg.c:128) > ==10402== by 0x580D5AC: VecScatterCreate_PtoS (vpscat.c:1621) > ==10402== by 0x5823C9F: VecScatterCreate (vscat.c:1316) > ==10402== by 0x518A547: DACreate_3D (da3.c:957) > ==10402== by 0x51D1ACA: DASetType (dareg.c:48) > ==10402== by 0x5194FA7: DASetTypeFromOptions_Private > (dacreate.c:65) > ==10402== Address 0xbc33562 is 498 bytes inside a block of size 516 > alloc'd > ==10402== at 0x4C23918: memalign > (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==10402== by 0x53E3D3B: PetscMallocAlign (mal.c:30) > ==10402== by 0x5566432: PetscTrMallocDefault (mtr.c:192) > ==10402== by 0x580D2B4: VecScatterCreate_PtoS (vpscat.c:1601) > ==10402== by 0x5823C9F: VecScatterCreate (vscat.c:1316) > ==10402== by 0x518A547: DACreate_3D (da3.c:957) > ==10402== by 0x51D1ACA: DASetType (dareg.c:48) > ==10402== by 0x5194FA7: DASetTypeFromOptions_Private > (dacreate.c:65) > ==10402== by 0x5195A81: DASetFromOptions (dacreate.c:131) > ==10402== by 0x518FFF5: DACreate3d (da3.c:1742) > ==10402== by 0x40669A: multi_init(int, char**) > (multi_petsc.cpp:115) > ==10402== by 0x403AF5: main (mhd_main.cpp:351) > > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------

1 0

DACreate3d should work for 1 node in Z
by Jed Brown 19 Apr '11

19 Apr '11

This would make it easier to solve 2D problems with a 3D code. It currently crashes or produces NaN because the scatter is set up incorrectly: ==10402== Invalid read of size 2 ==10402== at 0x4C26C2C: memcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==10402== by 0x6D87D01: opal_convertor_pack (in /usr/lib/openmpi/libmpi.so.1.0.1) ==10402== by 0xF33D0C8: ??? (in /usr/lib/openmpi/openmpi/mca_btl_sm.so) ==10402== by 0xEB1F1B7: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) ==10402== by 0xEB0DC0C: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) ==10402== by 0x6CD6650: PMPI_Isend (in /usr/lib/openmpi/libmpi.so.1.0.1) ==10402== by 0x551F85A: PetscGatherMessageLengths (mpimesg.c:128) ==10402== by 0x580D5AC: VecScatterCreate_PtoS (vpscat.c:1621) ==10402== by 0x5823C9F: VecScatterCreate (vscat.c:1316) ==10402== by 0x518A547: DACreate_3D (da3.c:957) ==10402== by 0x51D1ACA: DASetType (dareg.c:48) ==10402== by 0x5194FA7: DASetTypeFromOptions_Private (dacreate.c:65) ==10402== Address 0xbc33562 is 498 bytes inside a block of size 516 alloc'd ==10402== at 0x4C23918: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==10402== by 0x53E3D3B: PetscMallocAlign (mal.c:30) ==10402== by 0x5566432: PetscTrMallocDefault (mtr.c:192) ==10402== by 0x580D2B4: VecScatterCreate_PtoS (vpscat.c:1601) ==10402== by 0x5823C9F: VecScatterCreate (vscat.c:1316) ==10402== by 0x518A547: DACreate_3D (da3.c:957) ==10402== by 0x51D1ACA: DASetType (dareg.c:48) ==10402== by 0x5194FA7: DASetTypeFromOptions_Private (dacreate.c:65) ==10402== by 0x5195A81: DASetFromOptions (dacreate.c:131) ==10402== by 0x518FFF5: DACreate3d (da3.c:1742) ==10402== by 0x40669A: multi_init(int, char**) (multi_petsc.cpp:115) ==10402== by 0x403AF5: main (mhd_main.cpp:351)

1 0

Re: [petsc-dev] please add a faq for using bisection
by Satish Balay 19 Apr '11

19 Apr '11

added http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#bisect satish On Mon, 18 Apr 2011, Barry Smith wrote: > > Satish, > > Please add a FAQ question: How do I determine what update to PETSc broke my code? and explain how to do the bisection and how to update the BuildSystem to match the petsc-dev version etc. Let me know when it is ready. > > Thanks > > Barry > > > >

1 0

please add a faq for using bisection
by Barry Smith 18 Apr '11

18 Apr '11

Satish, Please add a FAQ question: How do I determine what update to PETSc broke my code? and explain how to do the bisection and how to update the BuildSystem to match the petsc-dev version etc. Let me know when it is ready. Thanks Barry

1 0