Re: [petsc-dev] uni
Mark, The mpiuni MPI_Comm_create() didn't increase the reference count for the comm hence freed too soon. Also I removed your use of PetscCommDuplicate(), there is no reason to call that before passing the communicator to create a PETSc object, you can just pass any old MPI_Comm and PETSc does the right thing. Only reason to call PetscCommDuplicate() is if you are going to do MPI message passing on the communicator and it may not be a PETSc communicator. Let me know if there are additional problems, Barry On Nov 23, 2011, at 5:47 PM, Mark F. Adams wrote:
On Nov 23, 2011, at 6:10 PM, Barry Smith wrote:
On Nov 23, 2011, at 3:14 PM, Mark F. Adams wrote:
I fixed my code error with valgrind and now I see the same thing (appended) as before (eg, valgrind does not seem to help).
I'm suspecting that MatConvert is messed, but I will try not deleting this matrix and see what happens.
==69836== Conditional jump or move depends on uninitialised value(s) ==69836== at 0x1E8A5DE: ATL_dgemvT_a1_x1_b0_y1 (in /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib) ==69836== ==69836== Conditional jump or move depends on uninitialised value(s) ==69836== at 0x1EBAC7A: ATL_dger1_a1_x1_yX (in /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib) ==69836== [0]
Yes it pisses me off to no end that Apple got suckered into using that ATLAS crap and it is buggy with basic memory errors and Jack doesn't give a shit.
Wow ....
So I fixed the problem. The errors happened when calling the repartitioning code w/o MPI. THe code works fine on one processor with the MPI version. So I think there is a bug in PETSc having to do with MatConvert or Partitioning code w/o MPI.
I have pushed fix today, but if you really want to debug it ... I'm sure that any no-MPI test using GAMG would trip this. All that needs to change is on line 191 of gamg.c:
if( s_avoid_repart || npe==1 ) {
change it to
if( s_avoid_repart ) {
This will run the partitioning code even on one processor.
Let me know if I can help.
Thanks, Mark
It you can tell us how to run a code (or push an example and tell use it) that reproduces this we can also debug it to see what the problem is.
Barry
On Nov 23, 2011, at 2:01 PM, Barry Smith wrote:
On Nov 23, 2011, at 10:31 AM, Mark F. Adams wrote:
Yep, need to keep the extra include with uni, but not the lib anymore.
Now I get this error, any ideas? I need a uni regression test apparently.
After you do the "make all test" on PETSc do "make alltests" and it should run all the tests that run on one process.
Something is wrong with your reference counting likely here. Does this work with --with-debugging=0? I would run debug version with valgrind first. Then debug version in debugger.
Barry
[0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Corrupt argument: see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind! [0]PETSC ERROR: Inner MPI_Comm does not have expected reference to outer comm! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: 3dda30872eeec1a2ea5c130f267935c5e0b2534a HG Date: Mon Nov 21 11:19:39 2011 -0800 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./viscousTensorSolve2d.Darwin.g++-4.gfortran.DEBUG.OPT.PETSC.ex on a arch-maco named madams-macbk-3.local by markadams Wed Nov 23 11:25:30 2011 [0]PETSC ERROR: Libraries linked from /Users/markadams/Codes/petsc-dev/arch-macosx-gnu-seq/lib [0]PETSC ERROR: Configure run at Tue Nov 22 12:41:46 2011 [0]PETSC ERROR: Configure options CXX=/sw/lib/gcc4.4/bin/g++-4 CC=/sw/lib/gcc4.4/bin/gcc-4 FC=/sw/lib/gcc4.4/bin/gfortran --with-x=0 --with-clanguage=c++ --with-debugging=0 --with-mpi=0 PETSC_ARCH=arch-macosx-gnu-seq [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc_DelComm() line 448 in /Users/markadams/Codes/petsc-dev/src/sys/objects/pinit.c [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Corrupt argument: see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind! [0]PETSC ERROR: MPI_Comm does not have tag/name counter nor does it have inner MPI_Comm! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: 3dda30872eeec1a2ea5c130f267935c5e0b2534a HG Date: Mon Nov 21 11:19:39 2011 -0800 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./viscousTensorSolve2d.Darwin.g++-4.gfortran.DEBUG.OPT.PETSC.ex on a arch-maco named madams-macbk-3.local by markadams Wed Nov 23 11:25:30 2011 [0]PETSC ERROR: Libraries linked from /Users/markadams/Codes/petsc-dev/arch-macosx-gnu-seq/lib [0]PETSC ERROR: Configure run at Tue Nov 22 12:41:46 2011 [0]PETSC ERROR: Configure options CXX=/sw/lib/gcc4.4/bin/g++-4 CC=/sw/lib/gcc4.4/bin/gcc-4 FC=/sw/lib/gcc4.4/bin/gfortran --with-x=0 --with-clanguage=c++ --with-debugging=0 --with-mpi=0 PETSC_ARCH=arch-macosx-gnu-seq [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PetscCommDestroy() line 229 in /Users/markadams/Codes/petsc-dev/src/sys/objects/tagm.c [0]PETSC ERROR: PetscHeaderDestroy_Private() line 110 in /Users/markadams/Codes/petsc-dev/src/sys/objects/inherit.c [0]PETSC ERROR: MatDestroy() line 1045 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c [0]PETSC ERROR: partitionLevel() line 347 in /Users/markadams/Codes/petsc-dev/src/ksp/pc/impls/gamg/gamg.c [0]PETSC ERROR: PCSetUp_GAMG() line 599 in /Users/markadams/Codes/petsc-dev/src/ksp/pc/impls/gamg/gamg.c [0]PETSC ERROR: PCSetUp() line 819 in /Users/markadams/Codes/petsc-dev/src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 261 in /Users/markadams/Codes/petsc-dev/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: solveprivate() line 191 in "unknowndirectory/"PetscLinearSolverI.H
On Nov 22, 2011, at 3:19 PM, Barry Smith wrote:
You have som other MPI include file hanging around that is getting intercepted before the MPI Uni one.
Barry
On Nov 22, 2011, at 12:00 PM, Mark F. Adams wrote:
Scorched earth did not work:
"VecCreate(ompi_communicator_t*, _p_Vec**)", referenced from: PetscLinearSolver<LevelData<FArrayBox> >::solveprivate(LevelData<FArrayBox>&, LevelData<FArrayBox> const&) in libamrelliptic2d.Darwin.g++-4.gfortran.DEBUG.OPT.PETSC.a(PetscLinearSolver.o) "_ompi_mpi_comm_self", referenced from: _ompi_mpi_comm_self$non_lazy_ptr in libamrelliptic2d.Darwin.g++-4.gfortran.DEBUG.OPT.PETSC.a(PetscLinearSolver.o)
<configure.log.gz>
On Nov 22, 2011, at 11:47 AM, Satish Balay wrote:
Something is strange.
I'll suggeset
rm -rf externalpackages arch-macosx-gnu-seq
and then doing a fresh build.
And then verifying the build with 'make test'
If 'make test' works - and the application build is not working- I'd like to see the complete compile log thats producing this error.
wrt libmpiuni.a - all the relavent stuff is added to libpetsc.a and a separate libmpiuni.a is not created.
Satish
On Tue, 22 Nov 2011, Mark F. Adams wrote:
I'm trying to build a non MPI code and get this error:
Undefined symbols: "_ompi_mpi_comm_self", referenced from: _ompi_mpi_comm_self$non_lazy_ptr in libamrelliptic2d.Darwin.g++-4.gfortran.DEBUG.OPT.PETSC.a(PetscLinearSolver.o) (maybe you meant: _ompi_mpi_comm_self$non_lazy_ptr)
In the past I have linked libmpiuni.a but that did not seem to get created. Did libmpiuni.a go away or did my make fail ...
Mark
participants (1)
-
Barry Smith