May 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] GTC 2012 recap (NVIDIA propaganda) + MAGMA plagarism
by Barry Smith 18 May '12

18 May '12

1 0

Re: [petsc-dev] GTC 2012 recap (NVIDIA propaganda) + MAGMA plagarism
by Jack Poulson 18 May '12

18 May '12

1 0

Re: [petsc-dev] GTC 2012 recap (NVIDIA propaganda) + MAGMA plagarism
by Jed Brown 18 May '12

18 May '12

1 0

Re: [petsc-dev] horrid bug
by Mark F. Adams 18 May '12

18 May '12

Thanks, I think I found it and pushed to solution. The problem was that the row block sizes where used to set the scatter maps in MatMult where column block size is needed. This splitting the row and column block sizes will probably cause more problems. This bug was only a fatal problem when the two block sizes where not multiples of each other (so my 3D problems worked fine). Mark On May 18, 2012, at 1:50 PM, Dmitry Karpeev wrote: > The problem is most likely in the creation routine or, rather, in the arguments passed into it: > is it possible that the Vecs fed into VecScatterCreate have different block sizes on different procs? > > Dmitry. > > On Fri, May 18, 2012 at 7:20 AM, Mark F. Adams <mark.adams(a)columbia.edu> wrote: > I've got a bug with block sizes in multigrid that is getting harry and wanted to see if I could get some help. > > The crux of the problem seems to be that different versions of VecScatterEnd are being called. I've added a print statement in: > > PetscErrorCode PETSCMAP1(VecScatterEnd)(VecScatter ctx,Vec xin,Vec yin,InsertMode addv,ScatterMode mode) > > in vpscat.h. and with a two processor run see this: > > *****[0]VecScatterEnd_3 start, N=27 Xin.bs=3 yin.bs=1 > *****[1]VecScatterEnd_3 start, N=27 Xin.bs=3 yin.bs=1 > *****[0]VecScatterEnd_2 start, N=27 Xin.bs=3 yin.bs=1 > *****[1]VecScatterEnd_1 start, N=27 Xin.bs=3 yin.bs=1 > [1]PETSC ERROR: VecScatterEnd_1() line 217 in /Users/markadams/Codes/petsc-dev/include/../src/vec/vec/utils/vpscat.h > [1]PETSC ERROR: VecScatterEnd() line 1620 in /Users/markadams/Codes/petsc-dev/src/vec/vec/utils/vscat.c > [1]PETSC ERROR: MatMultAdd_MPIAIJ() line 1118 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c > [1]PETSC ERROR: MatMultAdd() line 2372 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c > [1]PETSC ERROR: MatInterpolateAdd() line 7439 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c > [1]PETSC ERROR: PCMGMCycle_Private() line 52 in /Users/markadams/Codes/petsc-dev/src/ksp/pc/impls/mg/mg.c > > > This is a two level solve and the coarse grid is all on proc 0 and has a block size of 3. The fine grid has a block size of 2. THis error, as you can see, comes from the interpolation from coarse to fine. You can see from this output that the code called VecScatterEnd_3 successfully (the coarse grid) but when it does the interpolation proc 0 calls VecScatterEnd_2 (the destination is block size 2) and proc 1 calls VecScatterEnd_1 -- clearly wrong. > > Anyway, I'll keep digging, but any ideas would be welcome. > > Mark > >

1 0

Re: [petsc-dev] horrid bug
by Dmitry Karpeev 18 May '12

18 May '12

The problem is most likely in the creation routine or, rather, in the arguments passed into it: is it possible that the Vecs fed into VecScatterCreate have different block sizes on different procs? Dmitry. On Fri, May 18, 2012 at 7:20 AM, Mark F. Adams <mark.adams(a)columbia.edu>wrote: > I've got a bug with block sizes in multigrid that is getting harry and > wanted to see if I could get some help. > > The crux of the problem seems to be that different versions of > VecScatterEnd are being called. I've added a print statement in: > > PetscErrorCode PETSCMAP1(VecScatterEnd)(VecScatter ctx,Vec xin,Vec > yin,InsertMode addv,ScatterMode mode) > > in vpscat.h. and with a two processor run see this: > > *****[0]VecScatterEnd_3 start, N=27 Xin.bs=3 yin.bs=1 > *****[1]VecScatterEnd_3 start, N=27 Xin.bs=3 yin.bs=1 > *****[0]VecScatterEnd_2 start, N=27 Xin.bs=3 yin.bs=1 > *****[1]VecScatterEnd_1 start, N=27 Xin.bs=3 yin.bs=1 > [1]PETSC ERROR: VecScatterEnd_1() line 217 in > /Users/markadams/Codes/petsc-dev/include/../src/vec/vec/utils/vpscat.h > [1]PETSC ERROR: VecScatterEnd() line 1620 in > /Users/markadams/Codes/petsc-dev/src/vec/vec/utils/vscat.c > [1]PETSC ERROR: MatMultAdd_MPIAIJ() line 1118 in > /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c > [1]PETSC ERROR: MatMultAdd() line 2372 in > /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c > [1]PETSC ERROR: MatInterpolateAdd() line 7439 in > /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c > [1]PETSC ERROR: PCMGMCycle_Private() line 52 in > /Users/markadams/Codes/petsc-dev/src/ksp/pc/impls/mg/mg.c > > > This is a two level solve and the coarse grid is all on proc 0 and has a > block size of 3. The fine grid has a block size of 2. THis error, as you > can see, comes from the interpolation from coarse to fine. You can see > from this output that the code called VecScatterEnd_3 successfully (the > coarse grid) but when it does the interpolation proc 0 calls > VecScatterEnd_2 (the destination is block size 2) and proc 1 calls > VecScatterEnd_1 -- clearly wrong. > > Anyway, I'll keep digging, but any ideas would be welcome. > > Mark > >

1 0

horrid bug
by Mark F. Adams 18 May '12

18 May '12

I've got a bug with block sizes in multigrid that is getting harry and wanted to see if I could get some help. The crux of the problem seems to be that different versions of VecScatterEnd are being called. I've added a print statement in: PetscErrorCode PETSCMAP1(VecScatterEnd)(VecScatter ctx,Vec xin,Vec yin,InsertMode addv,ScatterMode mode) in vpscat.h. and with a two processor run see this: *****[0]VecScatterEnd_3 start, N=27 Xin.bs=3 yin.bs=1 *****[1]VecScatterEnd_3 start, N=27 Xin.bs=3 yin.bs=1 *****[0]VecScatterEnd_2 start, N=27 Xin.bs=3 yin.bs=1 *****[1]VecScatterEnd_1 start, N=27 Xin.bs=3 yin.bs=1 [1]PETSC ERROR: VecScatterEnd_1() line 217 in /Users/markadams/Codes/petsc-dev/include/../src/vec/vec/utils/vpscat.h [1]PETSC ERROR: VecScatterEnd() line 1620 in /Users/markadams/Codes/petsc-dev/src/vec/vec/utils/vscat.c [1]PETSC ERROR: MatMultAdd_MPIAIJ() line 1118 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c [1]PETSC ERROR: MatMultAdd() line 2372 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c [1]PETSC ERROR: MatInterpolateAdd() line 7439 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c [1]PETSC ERROR: PCMGMCycle_Private() line 52 in /Users/markadams/Codes/petsc-dev/src/ksp/pc/impls/mg/mg.c This is a two level solve and the coarse grid is all on proc 0 and has a block size of 3. The fine grid has a block size of 2. THis error, as you can see, comes from the interpolation from coarse to fine. You can see from this output that the code called VecScatterEnd_3 successfully (the coarse grid) but when it does the interpolation proc 0 calls VecScatterEnd_2 (the destination is block size 2) and proc 1 calls VecScatterEnd_1 -- clearly wrong. Anyway, I'll keep digging, but any ideas would be welcome. Mark

1 0

release preparation
by Barry Smith 17 May '12

17 May '12

petsc-dev folks, Recall that we are in a phase of a code freeze that means do not push random new code to petsc-dev, push only bug fixes and clean up to petsc-dev. By all means keep on with code development, just don't push the new stuff to petsc-dev. We hope to make the release repository (and then tarball) next Wednesday May 23. So please do be checking the nightly builds and making fixes to petsc-dev. Barry Note: threaded code and GPU code will not be supported with the release (users should continue to use petsc-dev after the release) so fixes/extensions to those things are not needed and shouldn't be pushed to petsc-dev at the moment.

1 0

Re: [petsc-dev] patch for block size in VecLoad_HDF5
by Matthew Knepley 16 May '12

16 May '12

On Wed, May 16, 2012 at 6:27 PM, Blaise Bourdin <bourdin(a)lsu.edu> wrote: > Hi, > > ex19 in src/vec/vec/examples/tutorials is still broken, since changing > block size of a Vec is no longer supported. > the mechanism in VecLoad_HDF5 was to rely on an undocumented flag > (-vecload_block_size) > > Shall we require that the block size be set _before_ calling VecLoad (in > which case, I will push a patch to VecLoad_HDF5), or should the block size > be obtained from the hdf5 files and set? That is horrible. Require that the input vector have that blocksize and get rid of that option. The user can just as easily use that option to set the blocksize before entry. Matt > > Blaise > > -- > Department of Mathematics and Center for Computation & Technology > Louisiana State University, Baton Rouge, LA 70803, USA > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 > http://www.math.lsu.edu/~bourdin > > > > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

patch for block size in VecLoad_HDF5
by Blaise Bourdin 16 May '12

16 May '12

Hi, ex19 in src/vec/vec/examples/tutorials is still broken, since changing block size of a Vec is no longer supported. the mechanism in VecLoad_HDF5 was to rely on an undocumented flag (-vecload_block_size) Shall we require that the block size be set _before_ calling VecLoad (in which case, I will push a patch to VecLoad_HDF5), or should the block size be obtained from the hdf5 files and set? Blaise -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin

1 0

Re: [petsc-dev] please review this patch
by Satish Balay 16 May '12

16 May '12

They were added in the following commit http://petsc.cs.iit.edu/petsc/petsc-dev/rev/97e68eb8972d Satish On Wed, 16 May 2012, Barry Smith wrote: > > Looks good for me. I cannot explain why those lines are there. > > Barry > > On May 16, 2012, at 10:51 AM, Lisandro Dalcin wrote: > > > Could you please help me to review this patch? These lines disable the > > usage of MatSolve_SeqAIJ_NaturalOrdering_inplace that is set a few > > lines above. Am I missing something? > > > > > > diff -r 7ea93c40c86e src/mat/impls/aij/seq/aijfact.c > > --- a/src/mat/impls/aij/seq/aijfact.c Sun May 13 21:38:46 2012 -0500 > > +++ b/src/mat/impls/aij/seq/aijfact.c Wed May 16 12:34:43 2012 -0300 > > @@ -759,8 +759,6 @@ > > ierr = PetscInfo2(A,"number of shift_nz tries %D, shift_amount > > %G\n",sctx.nshift,sctx.shift_amount);CHKERRQ(ierr); > > } > > } > > - (C)->ops->solve = MatSolve_SeqAIJ_inplace; > > - (C)->ops->solvetranspose = MatSolveTranspose_SeqAIJ_inplace; > > ierr = Mat_CheckInode(C,PETSC_FALSE);CHKERRQ(ierr); > > PetscFunctionReturn(0); > > } > > > > > > -- > > Lisandro Dalcin > > --------------- > > CIMEC (INTEC/CONICET-UNL) > > Predio CONICET-Santa Fe > > Colectora RN 168 Km 472, Paraje El Pozo > > 3000 Santa Fe, Argentina > > Tel: +54-342-4511594 (ext 1011) > > Tel/Fax: +54-342-4511169 > >

1 0