July 2008 - petsc-dev - mailman3.cels.anl.gov

Enabling OpenMP in Petsc
by Albert Hartono 31 Jul '08

31 Jul '08

Hi, Could someone explain how to build Petsc to enable OpenMP? I want to include OpenMP directives in src/mat/impls/aij/seq/inode.c, but I cannot figure out how to build it correctly so that OpenMP parallelization can be used correctly on the BGP surveyor machine. What I did is this: 1. I added #include <omp.h> in src/mat/impls/aij/seq/inode.c 2. I added a simple OpenMP test (shown below) in one of the functions in src/mat/impls/aij/seq/inode.c ierr=PetscPrintf(PETSC_COMM_WORLD,"***************************\n"); omp_set_num_threads(4); int nthreads, tid; #pragma omp parallel private(nthreads, tid) { tid = omp_get_thread_num(); ierr=PetscPrintf(PETSC_COMM_WORLD,"Hello from thread = %d\n",tid); if (tid == 0) { nthreads = omp_get_num_threads(); ierr=PetscPrintf(PETSC_COMM_WORLD,"Number of threads = %d\n",nthreads); } } ierr=PetscPrintf(PETSC_COMM_WORLD,"***************************\n"); 3. To activate the OpenMP test, I used snes/examples/tutorials/ex27.c. I modified the makefile in the tutorials directory as follows. line 6: CFLAGS = -qsmp=omp line 124: ex27: ex27.o chkopts -${CLINKER} -o ex27 ex27.o ${PETSC_SNES_LIB} -qsmp=omp ${RM} ex27.o 4. Then I was able to rebuild the inode.c and ex27 with no errors. And then run ex27 executable on BGP surveyor machine. And below is the results. ... *************************** Hello World from thread = 0 Number of threads = 1 *************************** *************************** Hello World from thread = 0 Number of threads = 1 *************************** ... The results showed the OpenMP problem: only ONE thread was used (although it was explicitly set to use four threads). Note that I didn't include -qsmp=omp in the makefile in src/mat/impls/aij/seq/ since it caused a multiple-definition error problem (the link was done twice). It may have been linked statically there. Helps will be greatly appreciated. Thanks, --Albert

1 0

Check it out
by Matthew Knepley 29 Jul '08

29 Jul '08

http://www.selenic.com/mercurial/wiki/index.cgi/ForestExtension Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: New approach for external direct solvers in PETSc
by Barry Smith 25 Jul '08

25 Jul '08

On Jul 25, 2008, at 5:47 PM, Lisandro Dalcin wrote: > Perhaps yor are missing in petscmat.h a line like the one below?? In > the docsyou mention it, but it is not listed alongside the other > packages. > > #define MAT_SOLVER_PETSC "petsc" > > Yup, I keep forgetting to put it in. Just pushing now. > BTW, Have you ever considered the posibility of not using the 'Mat' > class for factored matrices?? I believe that normal matrices and > factored matrices share very few in common as to being instances of > the same class. Looking at the source code in src/mat, there are many > checks like 'if (mat-->factor)' or 'if (!mat->factor)' that seems to > support my concern. What do you think? > This is a very valid point. The OOP fanboy would certainly make it another class and likely have a whole hierarchy of classes for this stuff :-). From the point of view of PETSc users I work VERY hard to limit the number of classes (I already think we have too many). Also given our crude "C classes" it is difficult to have nice hierarchy of classes so for now I will leave it as is. It would be reasonable to make other choices. Barry > > > On Fri, Jul 25, 2008 at 7:02 PM, Barry Smith <bsmith(a)mcs.anl.gov> > wrote: >> >> petsc-dev users, >> >> I have finally completed my changes to PETSc for a new approach to >> accessing external direct solvers >> in PETSc like Spooles, MUMPS etc. I will be pushing it to petsc-dev >> sometime >> the middle of next week. >> If you are using the direct solvers you might consider cloning from >> >> http://petsc.cs.iit.edu/hg/petsc/petsc-dev-new-solvers or >> >> ssh://[email protected]//hg/petsc/petsc-dev-new-solvers >> >> and trying it out before then. Please report problem to >> petsc-maint(a)mcs.anl.gov as you hit them. >> >> >> Barry >> >> From the changes file >> >> The "parallel direct solver" matrix types like >> MATAIJSPOOLES are ALL gone. Now you use -pc_factor_mat_solver_package >> spooles etc or PCFactorSetMatSolverPackage() or if working directly >> with >> matrices, MatGetFactor(A,MATSPOOLES,...) >> >> > > > > -- > Lisandro Dalcï¿½n > --------------- > Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) > Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) > Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) > PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 >

1 0

Re: New approach for external direct solvers in PETSc
by Lisandro Dalcin 25 Jul '08

25 Jul '08

Perhaps yor are missing in petscmat.h a line like the one below?? In the docsyou mention it, but it is not listed alongside the other packages. #define MAT_SOLVER_PETSC "petsc" BTW, Have you ever considered the posibility of not using the 'Mat' class for factored matrices?? I believe that normal matrices and factored matrices share very few in common as to being instances of the same class. Looking at the source code in src/mat, there are many checks like 'if (mat-->factor)' or 'if (!mat->factor)' that seems to support my concern. What do you think? On Fri, Jul 25, 2008 at 7:02 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > petsc-dev users, > > I have finally completed my changes to PETSc for a new approach to > accessing external direct solvers > in PETSc like Spooles, MUMPS etc. I will be pushing it to petsc-dev sometime > the middle of next week. > If you are using the direct solvers you might consider cloning from > > http://petsc.cs.iit.edu/hg/petsc/petsc-dev-new-solvers or > > ssh://[email protected]//hg/petsc/petsc-dev-new-solvers > > and trying it out before then. Please report problem to > petsc-maint(a)mcs.anl.gov as you hit them. > > > Barry > > From the changes file > > The "parallel direct solver" matrix types like > MATAIJSPOOLES are ALL gone. Now you use -pc_factor_mat_solver_package > spooles etc or PCFactorSetMatSolverPackage() or if working directly with > matrices, MatGetFactor(A,MATSPOOLES,...) > > -- Lisandro Dalcï¿½n --------------- Centro Internacional de Mï¿½todos Computacionales en Ingenierï¿½a (CIMEC) Instituto de Desarrollo Tecnolï¿½gico para la Industria Quï¿½mica (INTEC) Consejo Nacional de Investigaciones Cientï¿½ficas y Tï¿½cnicas (CONICET) PTLC - Gï¿½emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

1 0

New approach for external direct solvers in PETSc
by Barry Smith 25 Jul '08

25 Jul '08

petsc-dev users, I have finally completed my changes to PETSc for a new approach to accessing external direct solvers in PETSc like Spooles, MUMPS etc. I will be pushing it to petsc-dev sometime the middle of next week. If you are using the direct solvers you might consider cloning from http://petsc.cs.iit.edu/hg/petsc/petsc-dev-new-solvers or ssh://[email protected]//hg/petsc/petsc-dev-new-solvers and trying it out before then. Please report problem to petsc-maint(a)mcs.anl.gov as you hit them. Barry From the changes file The "parallel direct solver" matrix types like MATAIJSPOOLES are ALL gone. Now you use -pc_factor_mat_solver_package spooles etc or PCFactorSetMatSolverPackage() or if working directly with matrices, MatGetFactor(A,MATSPOOLES,...)

1 0

Re: problem with PetscSynchronizedFPrintf (fwd)
by Matthew Knepley 21 Jul '08

21 Jul '08

On Mon, Jul 21, 2008 at 3:03 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Matt, > > You have to be careful here. You use PetscMalloc() here, are you positive > this > routine will never be called BEFORE PetscMalloc() is setup for (sometime in > PetscInitialize), or after PetscMalloc() > is taken down (somewhere in PetscFinalize())? Is there someway you can make > sure that > it is safe? In a few places on setup I have to use a raw malloc instead of > PetscMalloc(), I don't > want you to have to do that here. > > I'm not saying to change back the code, just to be careful and check it. I looked at all the code. There are no calls in PetscInitialize() and I cannot find anything else that might be called before PetscMalloc is set. Matt > Barry > > On Jul 21, 2008, at 2:47 PM, Matthew Knepley wrote: > >> I made this change because synchronized printing was cutting off my >> stuff. Now the queue is dynamically allocated so it can accomodate >> any size string. >> >> I tried ex7 and it runs fine for me, even under valgrind. >> >> Shi, can you reproduce this with a Sieve example? >> >> Matt >> >> On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: >>> >>> Looks like this could be related to changeset 9e3fbe04a926 >>> >>> --> mprint.h <-- >>> >>> - char string[QUEUESTRINGSIZE]; >>> + char *string; >>> >>> So now strings needs to use malloc()/free()? >>> >>> Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c >>> >>> Satish >>> >>> On Mon, 21 Jul 2008, Satish Balay wrote: >>> >>>> >>>> ---------- Forwarded message ---------- >>>> Date: Sun, 20 Jul 2008 22:39:13 -0700 (PDT) >>>> From: Shi Jin <jinzishuai(a)yahoo.com> >>>> Subject: problem with PetscSynchronizedFPrintf >>>> To: petsc-dev(a)mcs.anl.gov >>>> >>>> Hi, >>>> >>>> I recently updated my petsc-dev and found my previously working code is >>>> having trouble running in parallel. I've narrowed it down to the calls to >>>> PetscSynchronizedFPrintf. Even if I don't have those, I will have errors at >>>> the end of my MPI runs like: >>>> [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c >>>> [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in >>>> src/sys/viewer/impls/ascii/filev.c >>>> [1]PETSC ERROR: PetscMemoryShowUsage() line 367 in src/sys/memory/mtr.c >>>> [1]PETSC ERROR: PetscFinalize() line 675 in src/sys/objects/pinit.c >>>> [1]PETSC ERROR: completeSection() line 163 in main.cxx >>>> [0]0:Return code = 0, signaled with Interrupt >>>> [0]1:Return code = 85 >>>> >>>> Since the same code was running fine before the update. I am wondering >>>> if it is something recently introduced. >>>> Thanks a lot. >>>> >>>> -- >>>> Shi Jin, PhD >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: problem with PetscSynchronizedFPrintf (fwd)
by Matthew Knepley 21 Jul '08

21 Jul '08

Fixed. Matt On Mon, Jul 21, 2008 at 3:32 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Matt, > > While you are at it could you please fix the buggy code in mprint.c? > > make PETSC_DIR=/Users/bsmith/Src/petsc-dev-for-fixes > PETSC_ARCH=arch-icc-ifort > /Users/bsmith/Src/petsc-dev-for-fixes/arch-icc-ifort/bin/mpicc -o mprint.o > -c -wd1572 -Qoption,cpp,--extended_float_type -g > -I/Users/bsmith/Src/petsc-dev-for-fixes > -I/Users/bsmith/Src/petsc-dev-for-fixes/arch-icc-ifort/include > -I/Users/bsmith/Src/petsc-dev-for-fixes/include -I/usr/X11/include > -I/Users/bsmith/Src/petsc-dev-for-fixes/arch-icc-ifort/include > -D__SDIR__='"src/sys/fileio/"' mprint.c > mprint.c(350): warning #592: variable "fullLength" is used before its value > is set > while(fullLength >= next->size) { > ^ > > > On Jul 21, 2008, at 3:30 PM, Matthew Knepley wrote: > >> Pushed a fix. I missed the Viewer when I corrected the raw call. That >> should >> not be separate code. >> >> Matt >> >> On Mon, Jul 21, 2008 at 2:54 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: >>> >>> Matt - Perhaps ex7 is a different Problem. But it does crash for me. >>> And I noticed the following code: >>> >>>>>>>>>>>>>> >>> ierr = PetscNew(struct _PrintfQueue,&next);CHKERRQ(ierr); >>> if (queue) {queue->next = next; queue = next;} >>> else {queuebase = queue = next;} >>> queuelength++; >>> string = next->string; >>> ierr = PetscMemzero(string,QUEUESTRINGSIZE);CHKERRQ(ierr); >>> <<<<<<<<< >>> >>> i.e 'next' is allocated - here is no malloc for 'next->string'. But >>> its being zeroed in PetscMemzero() - hence the crash with ex7 for me. >>> >>> Satish >>> >>> --------------------------------------------- >>> >>> asterix:/home/balay/spetsc/src/dm/ao/examples/tests>mpiexec -n 2 ./ex7 >>> Number of elements in ordering 10 >>> PETSc->App App->PETSc >>> 0 0 0 0 >>> 1 5 1 2 >>> 2 1 2 4 >>> 3 6 3 6 >>> 4 2 4 8 >>> 5 7 5 1 >>> 6 3 6 3 >>> 7 8 7 5 >>> 8 4 8 7 >>> 9 9 9 9 >>> [1]PETSC ERROR: --------------------- Error Message >>> ------------------------------------ >>> [1]PETSC ERROR: Null argument, when expecting valid pointer! >>> [1]PETSC ERROR: Trying to zero at a null pointer! >>> [1]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [1]PETSC ERROR: Petsc Development Version 2.3.3, Patch 13, unknown HG >>> revision: unknown >>> [1]PETSC ERROR: See docs/changes/index.html for recent updates. >>> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>> [1]PETSC ERROR: See docs/index.html for manual pages. >>> [1]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [1]PETSC ERROR: ./ex7 on a asterix64 named asterix by balay Mon Jul 21 >>> 14:51:02 2008 >>> [1]PETSC ERROR: Libraries linked from /home/balay/spetsc/lib >>> [1]PETSC ERROR: Configure run at Mon Jul 21 10:38:52 2008 >>> [1]PETSC ERROR: Configure options >>> --with-mpi-dir=/home/balay/soft/linux64/mpich2-1.0.7 --with-shared=1 >>> PETSC_ARCH=asterix64 >>> [1]PETSC ERROR: ----------------------------[0] Number of indices in >>> (stride) set 5 >>> [0] 0 0 >>> [0] 1 2 >>> [0] 2 4 >>> [0] 3 6 >>> [0] 4 8 >>> -------------------------------------------- >>> [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c >>> [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in >>> src/sys/viewer/impls/ascii/filev.c >>> [1]PETSC ERROR: ISView_Stride() line 231 in >>> src/vec/is/impls/stride/stride.c >>> [1]PETSC ERROR: ISView() line 398 in src/vec/is/interface/index.c >>> [1]PETSC ERROR: main() line 31 in src/dm/ao/examples/tests/ex7.c >>> application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1[cli_1]: >>> aborting job: >>> application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1 >>> [0]0:Return code = 0, signaled with Interrupt >>> [0]1:Return code = 85 >>> asterix:/home/balay/spetsc/src/dm/ao/examples/tests> >>> >>> >>> >>> >>> On Mon, 21 Jul 2008, Matthew Knepley wrote: >>> >>>> I made this change because synchronized printing was cutting off my >>>> stuff. Now the queue is dynamically allocated so it can accomodate >>>> any size string. >>>> >>>> I tried ex7 and it runs fine for me, even under valgrind. >>>> >>>> Shi, can you reproduce this with a Sieve example? >>>> >>>> Matt >>>> >>>> On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: >>>>> >>>>> Looks like this could be related to changeset 9e3fbe04a926 >>>>> >>>>> --> mprint.h <-- >>>>> >>>>> - char string[QUEUESTRINGSIZE]; >>>>> + char *string; >>>>> >>>>> So now strings needs to use malloc()/free()? >>>>> >>>>> Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c >>>>> >>>>> Satish >>> >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: problem with PetscSynchronizedFPrintf (fwd)
by Barry Smith 21 Jul '08

21 Jul '08

Matt, While you are at it could you please fix the buggy code in mprint.c? make PETSC_DIR=/Users/bsmith/Src/petsc-dev-for-fixes PETSC_ARCH=arch- icc-ifort /Users/bsmith/Src/petsc-dev-for-fixes/arch-icc-ifort/bin/mpicc -o mprint.o -c -wd1572 -Qoption,cpp,--extended_float_type -g -I/Users/ bsmith/Src/petsc-dev-for-fixes -I/Users/bsmith/Src/petsc-dev-for-fixes/ arch-icc-ifort/include -I/Users/bsmith/Src/petsc-dev-for-fixes/include -I/usr/X11/include -I/Users/bsmith/Src/petsc-dev-for-fixes/arch-icc- ifort/include -D__SDIR__='"src/sys/fileio/"' mprint.c mprint.c(350): warning #592: variable "fullLength" is used before its value is set while(fullLength >= next->size) { ^ On Jul 21, 2008, at 3:30 PM, Matthew Knepley wrote: > Pushed a fix. I missed the Viewer when I corrected the raw call. > That should > not be separate code. > > Matt > > On Mon, Jul 21, 2008 at 2:54 PM, Satish Balay <balay(a)mcs.anl.gov> > wrote: >> Matt - Perhaps ex7 is a different Problem. But it does crash for me. >> And I noticed the following code: >> >>>>>>>>>>>>> >> ierr = PetscNew(struct _PrintfQueue,&next);CHKERRQ(ierr); >> if (queue) {queue->next = next; queue = next;} >> else {queuebase = queue = next;} >> queuelength++; >> string = next->string; >> ierr = PetscMemzero(string,QUEUESTRINGSIZE);CHKERRQ(ierr); >> <<<<<<<<< >> >> i.e 'next' is allocated - here is no malloc for 'next->string'. But >> its being zeroed in PetscMemzero() - hence the crash with ex7 for me. >> >> Satish >> >> --------------------------------------------- >> >> asterix:/home/balay/spetsc/src/dm/ao/examples/tests>mpiexec -n 2 ./ >> ex7 >> Number of elements in ordering 10 >> PETSc->App App->PETSc >> 0 0 0 0 >> 1 5 1 2 >> 2 1 2 4 >> 3 6 3 6 >> 4 2 4 8 >> 5 7 5 1 >> 6 3 6 3 >> 7 8 7 5 >> 8 4 8 7 >> 9 9 9 9 >> [1]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [1]PETSC ERROR: Null argument, when expecting valid pointer! >> [1]PETSC ERROR: Trying to zero at a null pointer! >> [1]PETSC ERROR: >> ------------------------------------------------------------------------ >> [1]PETSC ERROR: Petsc Development Version 2.3.3, Patch 13, unknown >> HG revision: unknown >> [1]PETSC ERROR: See docs/changes/index.html for recent updates. >> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [1]PETSC ERROR: See docs/index.html for manual pages. >> [1]PETSC ERROR: >> ------------------------------------------------------------------------ >> [1]PETSC ERROR: ./ex7 on a asterix64 named asterix by balay Mon Jul >> 21 14:51:02 2008 >> [1]PETSC ERROR: Libraries linked from /home/balay/spetsc/lib >> [1]PETSC ERROR: Configure run at Mon Jul 21 10:38:52 2008 >> [1]PETSC ERROR: Configure options --with-mpi-dir=/home/balay/soft/ >> linux64/mpich2-1.0.7 --with-shared=1 PETSC_ARCH=asterix64 >> [1]PETSC ERROR: ----------------------------[0] Number of indices >> in (stride) set 5 >> [0] 0 0 >> [0] 1 2 >> [0] 2 4 >> [0] 3 6 >> [0] 4 8 >> -------------------------------------------- >> [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c >> [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in >> src/sys/viewer/impls/ascii/filev.c >> [1]PETSC ERROR: ISView_Stride() line 231 in src/vec/is/impls/stride/ >> stride.c >> [1]PETSC ERROR: ISView() line 398 in src/vec/is/interface/index.c >> [1]PETSC ERROR: main() line 31 in src/dm/ao/examples/tests/ex7.c >> application called MPI_Abort(MPI_COMM_WORLD, 85) - process >> 1[cli_1]: aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1 >> [0]0:Return code = 0, signaled with Interrupt >> [0]1:Return code = 85 >> asterix:/home/balay/spetsc/src/dm/ao/examples/tests> >> >> >> >> >> On Mon, 21 Jul 2008, Matthew Knepley wrote: >> >>> I made this change because synchronized printing was cutting off my >>> stuff. Now the queue is dynamically allocated so it can accomodate >>> any size string. >>> >>> I tried ex7 and it runs fine for me, even under valgrind. >>> >>> Shi, can you reproduce this with a Sieve example? >>> >>> Matt >>> >>> On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay <balay(a)mcs.anl.gov> >>> wrote: >>>> Looks like this could be related to changeset 9e3fbe04a926 >>>> >>>> --> mprint.h <-- >>>> >>>> - char string[QUEUESTRINGSIZE]; >>>> + char *string; >>>> >>>> So now strings needs to use malloc()/free()? >>>> >>>> Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c >>>> >>>> Satish >> >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener >

1 0

Re: problem with PetscSynchronizedFPrintf (fwd)
by Matthew Knepley 21 Jul '08

21 Jul '08

I will check all the occurences of SynchronizedPrintf() in the source. Matt On Mon, Jul 21, 2008 at 3:03 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Matt, > > You have to be careful here. You use PetscMalloc() here, are you positive > this > routine will never be called BEFORE PetscMalloc() is setup for (sometime in > PetscInitialize), or after PetscMalloc() > is taken down (somewhere in PetscFinalize())? Is there someway you can make > sure that > it is safe? In a few places on setup I have to use a raw malloc instead of > PetscMalloc(), I don't > want you to have to do that here. > > I'm not saying to change back the code, just to be careful and check it. > > Barry > > On Jul 21, 2008, at 2:47 PM, Matthew Knepley wrote: > >> I made this change because synchronized printing was cutting off my >> stuff. Now the queue is dynamically allocated so it can accomodate >> any size string. >> >> I tried ex7 and it runs fine for me, even under valgrind. >> >> Shi, can you reproduce this with a Sieve example? >> >> Matt >> >> On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: >>> >>> Looks like this could be related to changeset 9e3fbe04a926 >>> >>> --> mprint.h <-- >>> >>> - char string[QUEUESTRINGSIZE]; >>> + char *string; >>> >>> So now strings needs to use malloc()/free()? >>> >>> Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c >>> >>> Satish >>> >>> On Mon, 21 Jul 2008, Satish Balay wrote: >>> >>>> >>>> ---------- Forwarded message ---------- >>>> Date: Sun, 20 Jul 2008 22:39:13 -0700 (PDT) >>>> From: Shi Jin <jinzishuai(a)yahoo.com> >>>> Subject: problem with PetscSynchronizedFPrintf >>>> To: petsc-dev(a)mcs.anl.gov >>>> >>>> Hi, >>>> >>>> I recently updated my petsc-dev and found my previously working code is >>>> having trouble running in parallel. I've narrowed it down to the calls to >>>> PetscSynchronizedFPrintf. Even if I don't have those, I will have errors at >>>> the end of my MPI runs like: >>>> [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c >>>> [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in >>>> src/sys/viewer/impls/ascii/filev.c >>>> [1]PETSC ERROR: PetscMemoryShowUsage() line 367 in src/sys/memory/mtr.c >>>> [1]PETSC ERROR: PetscFinalize() line 675 in src/sys/objects/pinit.c >>>> [1]PETSC ERROR: completeSection() line 163 in main.cxx >>>> [0]0:Return code = 0, signaled with Interrupt >>>> [0]1:Return code = 85 >>>> >>>> Since the same code was running fine before the update. I am wondering >>>> if it is something recently introduced. >>>> Thanks a lot. >>>> >>>> -- >>>> Shi Jin, PhD >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: problem with PetscSynchronizedFPrintf (fwd)
by Matthew Knepley 21 Jul '08

21 Jul '08

Pushed a fix. I missed the Viewer when I corrected the raw call. That should not be separate code. Matt On Mon, Jul 21, 2008 at 2:54 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: > Matt - Perhaps ex7 is a different Problem. But it does crash for me. > And I noticed the following code: > >>>>>>>>>>>> > ierr = PetscNew(struct _PrintfQueue,&next);CHKERRQ(ierr); > if (queue) {queue->next = next; queue = next;} > else {queuebase = queue = next;} > queuelength++; > string = next->string; > ierr = PetscMemzero(string,QUEUESTRINGSIZE);CHKERRQ(ierr); > <<<<<<<<< > > i.e 'next' is allocated - here is no malloc for 'next->string'. But > its being zeroed in PetscMemzero() - hence the crash with ex7 for me. > > Satish > > --------------------------------------------- > > asterix:/home/balay/spetsc/src/dm/ao/examples/tests>mpiexec -n 2 ./ex7 > Number of elements in ordering 10 > PETSc->App App->PETSc > 0 0 0 0 > 1 5 1 2 > 2 1 2 4 > 3 6 3 6 > 4 2 4 8 > 5 7 5 1 > 6 3 6 3 > 7 8 7 5 > 8 4 8 7 > 9 9 9 9 > [1]PETSC ERROR: --------------------- Error Message ------------------------------------ > [1]PETSC ERROR: Null argument, when expecting valid pointer! > [1]PETSC ERROR: Trying to zero at a null pointer! > [1]PETSC ERROR: ------------------------------------------------------------------------ > [1]PETSC ERROR: Petsc Development Version 2.3.3, Patch 13, unknown HG revision: unknown > [1]PETSC ERROR: See docs/changes/index.html for recent updates. > [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [1]PETSC ERROR: See docs/index.html for manual pages. > [1]PETSC ERROR: ------------------------------------------------------------------------ > [1]PETSC ERROR: ./ex7 on a asterix64 named asterix by balay Mon Jul 21 14:51:02 2008 > [1]PETSC ERROR: Libraries linked from /home/balay/spetsc/lib > [1]PETSC ERROR: Configure run at Mon Jul 21 10:38:52 2008 > [1]PETSC ERROR: Configure options --with-mpi-dir=/home/balay/soft/linux64/mpich2-1.0.7 --with-shared=1 PETSC_ARCH=asterix64 > [1]PETSC ERROR: ----------------------------[0] Number of indices in (stride) set 5 > [0] 0 0 > [0] 1 2 > [0] 2 4 > [0] 3 6 > [0] 4 8 > -------------------------------------------- > [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c > [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c > [1]PETSC ERROR: ISView_Stride() line 231 in src/vec/is/impls/stride/stride.c > [1]PETSC ERROR: ISView() line 398 in src/vec/is/interface/index.c > [1]PETSC ERROR: main() line 31 in src/dm/ao/examples/tests/ex7.c > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1[cli_1]: aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1 > [0]0:Return code = 0, signaled with Interrupt > [0]1:Return code = 85 > asterix:/home/balay/spetsc/src/dm/ao/examples/tests> > > > > > On Mon, 21 Jul 2008, Matthew Knepley wrote: > >> I made this change because synchronized printing was cutting off my >> stuff. Now the queue is dynamically allocated so it can accomodate >> any size string. >> >> I tried ex7 and it runs fine for me, even under valgrind. >> >> Shi, can you reproduce this with a Sieve example? >> >> Matt >> >> On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: >> > Looks like this could be related to changeset 9e3fbe04a926 >> > >> > --> mprint.h <-- >> > >> > - char string[QUEUESTRINGSIZE]; >> > + char *string; >> > >> > So now strings needs to use malloc()/free()? >> > >> > Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c >> > >> > Satish > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0