June 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] error reading binary file
by Jed Brown 12 Jun '12

12 Jun '12

On Tue, Jun 12, 2012 at 5:05 PM, Mark F. Adams <mark.adams(a)columbia.edu>wrote: > On Jun 12, 2012, at 5:52 PM, Jed Brown wrote: > > Okay, every matrix type except MPIAIJ respects -matload_block_size. Can > you fix that? > > > Can you fix that? > > > I'll take a shot at it Friday. And I assume I need to add > '-matload_block_size 3' to the info files. > Yeah, add that to MatView_MPIAIJ_Binary and MatView_SeqAIJ_Binary, then read it back in MatLoad_MPIAIJ_Binary. Look at the BAIJ stuff to see how to read and write the attribute from the info file.

1 0

Re: [petsc-dev] error reading binary file
by Mark F. Adams 12 Jun '12

12 Jun '12

On Jun 12, 2012, at 5:52 PM, Jed Brown wrote: > Okay, every matrix type except MPIAIJ respects -matload_block_size. Can you fix that? > > > Can you fix that? I'll take a shot at it Friday. And I assume I need to add '-matload_block_size 3' to the info files. Mark > > On Tue, Jun 12, 2012 at 4:52 PM, Mark F. Adams <mark.adams(a)columbia.edu> wrote: > Mine works in serial but not in parallel. > > It looks like vecs are created in MatLoad via MatMPIAIJSetPreallocation_MPIAIJ and they are not split wrt blocks. here is the stack trace again: > > [0]PETSC ERROR: PetscSplitOwnership() line 93 in /Users/markadams/Codes/petsc-dev/src/sys/utils/psplit.c > [0]PETSC ERROR: PetscLayoutSetUp() line 147 in /Users/markadams/Codes/petsc-dev/src/vec/vec/impls/mpi/pmap.c > [0]PETSC ERROR: MatMPIAIJSetPreallocation_MPIAIJ() line 3194 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatMPIAIJSetPreallocation() line 3928 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatLoad_MPIAIJ() line 3442 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatLoad() line 881 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c > [0]PETSC ERROR: main() line 109 in src/ksp/ksp/examples/tutorials/ex10.c > > > > On Jun 12, 2012, at 5:43 PM, Jed Brown wrote: > >> On Tue, Jun 12, 2012 at 4:45 PM, Mark F. Adams <mark.adams(a)columbia.edu> wrote: >> Satish beat me to it. I still get the same error ... did you use -mat_block_size 3? >> >> I will look in to further. >> >> ~/petsc/src/ksp/ksp/examples/tutorials$ ./ex10 -f0 binaryoutput-sym -ksp_monitor -pc_type gamg -mat_block_size 3 >> [0]PCSetData_AGG bs=3 MM=89373 >> 0 KSP Residual norm 4.854898290598e-03 >> 1 KSP Residual norm 1.748948970672e-03 >> 2 KSP Residual norm 1.098105829371e-03 >> 3 KSP Residual norm 7.267836867590e-04 >> 4 KSP Residual norm 4.752091560022e-04 >> 5 KSP Residual norm 3.152074302240e-04 >> 6 KSP Residual norm 1.998139269819e-04 >> 7 KSP Residual norm 1.142295077940e-04 >> 8 KSP Residual norm 6.107885553654e-05 >> 9 KSP Residual norm 2.903395117932e-05 >> 10 KSP Residual norm 1.748667302607e-05 >> 11 KSP Residual norm 9.864255584167e-06 >> 12 KSP Residual norm 5.077178570024e-06 >> 13 KSP Residual norm 2.996102603244e-06 >> 14 KSP Residual norm 2.293954251541e-06 >> 15 KSP Residual norm 1.977076000505e-06 >> 16 KSP Residual norm 1.729762967197e-06 >> 17 KSP Residual norm 1.422352992308e-06 >> 18 KSP Residual norm 1.063366577331e-06 >> 19 KSP Residual norm 7.479518697501e-07 >> 20 KSP Residual norm 4.861597752713e-07 >> 21 KSP Residual norm 2.996075502312e-07 >> 22 KSP Residual norm 1.962056634115e-07 >> 23 KSP Residual norm 1.216036063867e-07 >> 24 KSP Residual norm 8.556036439634e-08 >> 25 KSP Residual norm 5.911194128322e-08 >> 26 KSP Residual norm 3.784597093891e-08 >> Number of iterations = 26 >> Residual norm 7.51457e-10 > >

1 0

Re: [petsc-dev] error reading binary file
by Mark F. Adams 12 Jun '12

12 Jun '12

Mine works in serial but not in parallel. It looks like vecs are created in MatLoad via MatMPIAIJSetPreallocation_MPIAIJ and they are not split wrt blocks. here is the stack trace again: [0]PETSC ERROR: PetscSplitOwnership() line 93 in /Users/markadams/Codes/petsc-dev/src/sys/utils/psplit.c [0]PETSC ERROR: PetscLayoutSetUp() line 147 in /Users/markadams/Codes/petsc-dev/src/vec/vec/impls/mpi/pmap.c [0]PETSC ERROR: MatMPIAIJSetPreallocation_MPIAIJ() line 3194 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: MatMPIAIJSetPreallocation() line 3928 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: MatLoad_MPIAIJ() line 3442 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: MatLoad() line 881 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c [0]PETSC ERROR: main() line 109 in src/ksp/ksp/examples/tutorials/ex10.c On Jun 12, 2012, at 5:43 PM, Jed Brown wrote: > On Tue, Jun 12, 2012 at 4:45 PM, Mark F. Adams <mark.adams(a)columbia.edu> wrote: > Satish beat me to it. I still get the same error ... did you use -mat_block_size 3? > > I will look in to further. > > ~/petsc/src/ksp/ksp/examples/tutorials$ ./ex10 -f0 binaryoutput-sym -ksp_monitor -pc_type gamg -mat_block_size 3 > [0]PCSetData_AGG bs=3 MM=89373 > 0 KSP Residual norm 4.854898290598e-03 > 1 KSP Residual norm 1.748948970672e-03 > 2 KSP Residual norm 1.098105829371e-03 > 3 KSP Residual norm 7.267836867590e-04 > 4 KSP Residual norm 4.752091560022e-04 > 5 KSP Residual norm 3.152074302240e-04 > 6 KSP Residual norm 1.998139269819e-04 > 7 KSP Residual norm 1.142295077940e-04 > 8 KSP Residual norm 6.107885553654e-05 > 9 KSP Residual norm 2.903395117932e-05 > 10 KSP Residual norm 1.748667302607e-05 > 11 KSP Residual norm 9.864255584167e-06 > 12 KSP Residual norm 5.077178570024e-06 > 13 KSP Residual norm 2.996102603244e-06 > 14 KSP Residual norm 2.293954251541e-06 > 15 KSP Residual norm 1.977076000505e-06 > 16 KSP Residual norm 1.729762967197e-06 > 17 KSP Residual norm 1.422352992308e-06 > 18 KSP Residual norm 1.063366577331e-06 > 19 KSP Residual norm 7.479518697501e-07 > 20 KSP Residual norm 4.861597752713e-07 > 21 KSP Residual norm 2.996075502312e-07 > 22 KSP Residual norm 1.962056634115e-07 > 23 KSP Residual norm 1.216036063867e-07 > 24 KSP Residual norm 8.556036439634e-08 > 25 KSP Residual norm 5.911194128322e-08 > 26 KSP Residual norm 3.784597093891e-08 > Number of iterations = 26 > Residual norm 7.51457e-10

1 0

Re: [petsc-dev] error reading binary file
by Jed Brown 12 Jun '12

12 Jun '12

Okay, every matrix type except MPIAIJ respects -matload_block_size. Can you fix that? Can you fix that? On Tue, Jun 12, 2012 at 4:52 PM, Mark F. Adams <mark.adams(a)columbia.edu>wrote: > Mine works in serial but not in parallel. > > It looks like vecs are created in MatLoad > via MatMPIAIJSetPreallocation_MPIAIJ and they are not split wrt blocks. > here is the stack trace again: > > [0]PETSC ERROR: PetscSplitOwnership() line 93 in > /Users/markadams/Codes/petsc-dev/src/sys/utils/psplit.c > [0]PETSC ERROR: PetscLayoutSetUp() line 147 in > /Users/markadams/Codes/petsc-dev/src/vec/vec/impls/mpi/pmap.c > [0]PETSC ERROR: MatMPIAIJSetPreallocation_MPIAIJ() line 3194 in > /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatMPIAIJSetPreallocation() line 3928 in > /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatLoad_MPIAIJ() line 3442 in > /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatLoad() line 881 in > /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c > [0]PETSC ERROR: main() line 109 in src/ksp/ksp/examples/tutorials/ex10.c > > > > On Jun 12, 2012, at 5:43 PM, Jed Brown wrote: > > On Tue, Jun 12, 2012 at 4:45 PM, Mark F. Adams <mark.adams(a)columbia.edu>wrote: > >> Satish beat me to it. I still get the same error ... did you >> use -mat_block_size 3? >> >> I will look in to further. >> > > ~/petsc/src/ksp/ksp/examples/tutorials$ ./ex10 -f0 binaryoutput-sym > -ksp_monitor -pc_type gamg -mat_block_size 3 > [0]PCSetData_AGG bs=3 MM=89373 > 0 KSP Residual norm 4.854898290598e-03 > 1 KSP Residual norm 1.748948970672e-03 > 2 KSP Residual norm 1.098105829371e-03 > 3 KSP Residual norm 7.267836867590e-04 > 4 KSP Residual norm 4.752091560022e-04 > 5 KSP Residual norm 3.152074302240e-04 > 6 KSP Residual norm 1.998139269819e-04 > 7 KSP Residual norm 1.142295077940e-04 > 8 KSP Residual norm 6.107885553654e-05 > 9 KSP Residual norm 2.903395117932e-05 > 10 KSP Residual norm 1.748667302607e-05 > 11 KSP Residual norm 9.864255584167e-06 > 12 KSP Residual norm 5.077178570024e-06 > 13 KSP Residual norm 2.996102603244e-06 > 14 KSP Residual norm 2.293954251541e-06 > 15 KSP Residual norm 1.977076000505e-06 > 16 KSP Residual norm 1.729762967197e-06 > 17 KSP Residual norm 1.422352992308e-06 > 18 KSP Residual norm 1.063366577331e-06 > 19 KSP Residual norm 7.479518697501e-07 > 20 KSP Residual norm 4.861597752713e-07 > 21 KSP Residual norm 2.996075502312e-07 > 22 KSP Residual norm 1.962056634115e-07 > 23 KSP Residual norm 1.216036063867e-07 > 24 KSP Residual norm 8.556036439634e-08 > 25 KSP Residual norm 5.911194128322e-08 > 26 KSP Residual norm 3.784597093891e-08 > Number of iterations = 26 > Residual norm 7.51457e-10 > > >

1 0

Re: [petsc-dev] error reading binary file
by Mark F. Adams 12 Jun '12

12 Jun '12

Split ownership is not respecting blocks: [0]PETSC ERROR: PetscSplitOwnership() line 93 in /Users/markadams/Codes/petsc-dev/src/sys/utils/psplit.c On Jun 12, 2012, at 5:33 PM, Jed Brown wrote: > On Tue, Jun 12, 2012 at 4:35 PM, Mark F. Adams <mark.adams(a)columbia.edu> wrote: > And I now get the following error when I try to run this in parallel. > > Note, Blaise, I do not have coordinates, so perhaps you could check that you still get divergence w/o coordinates. (I don't need anything else form you) > > Did you apply the patch? I ran that code in parallel using ./ex10 -f0 binaryoutput-sym

1 0

Re: [petsc-dev] error reading binary file
by Mark F. Adams 12 Jun '12

12 Jun '12

On Jun 12, 2012, at 5:33 PM, Jed Brown wrote: > On Tue, Jun 12, 2012 at 4:35 PM, Mark F. Adams <mark.adams(a)columbia.edu> wrote: > And I now get the following error when I try to run this in parallel. > > Note, Blaise, I do not have coordinates, so perhaps you could check that you still get divergence w/o coordinates. (I don't need anything else form you) > > Did you apply the patch? I ran that code in parallel using ./ex10 -f0 binaryoutput-sym Satish beat me to it. I still get the same error ... did you use -mat_block_size 3? I will look in to further.

1 0

Re: [petsc-dev] error reading binary file
by Jed Brown 12 Jun '12

12 Jun '12

On Tue, Jun 12, 2012 at 4:45 PM, Mark F. Adams <mark.adams(a)columbia.edu>wrote: > Satish beat me to it. I still get the same error ... did you > use -mat_block_size 3? > > I will look in to further. > ~/petsc/src/ksp/ksp/examples/tutorials$ ./ex10 -f0 binaryoutput-sym -ksp_monitor -pc_type gamg -mat_block_size 3 [0]PCSetData_AGG bs=3 MM=89373 0 KSP Residual norm 4.854898290598e-03 1 KSP Residual norm 1.748948970672e-03 2 KSP Residual norm 1.098105829371e-03 3 KSP Residual norm 7.267836867590e-04 4 KSP Residual norm 4.752091560022e-04 5 KSP Residual norm 3.152074302240e-04 6 KSP Residual norm 1.998139269819e-04 7 KSP Residual norm 1.142295077940e-04 8 KSP Residual norm 6.107885553654e-05 9 KSP Residual norm 2.903395117932e-05 10 KSP Residual norm 1.748667302607e-05 11 KSP Residual norm 9.864255584167e-06 12 KSP Residual norm 5.077178570024e-06 13 KSP Residual norm 2.996102603244e-06 14 KSP Residual norm 2.293954251541e-06 15 KSP Residual norm 1.977076000505e-06 16 KSP Residual norm 1.729762967197e-06 17 KSP Residual norm 1.422352992308e-06 18 KSP Residual norm 1.063366577331e-06 19 KSP Residual norm 7.479518697501e-07 20 KSP Residual norm 4.861597752713e-07 21 KSP Residual norm 2.996075502312e-07 22 KSP Residual norm 1.962056634115e-07 23 KSP Residual norm 1.216036063867e-07 24 KSP Residual norm 8.556036439634e-08 25 KSP Residual norm 5.911194128322e-08 26 KSP Residual norm 3.784597093891e-08 Number of iterations = 26 Residual norm 7.51457e-10

1 0

Re: [petsc-dev] error reading binary file
by Mark F. Adams 12 Jun '12

12 Jun '12

And I now get the following error when I try to run this in parallel. Note, Blaise, I do not have coordinates, so perhaps you could check that you still get divergence w/o coordinates. (I don't need anything else form you) Mark [1]PETSC ERROR: --------------------- Error Message ------------------------------------ [1]PETSC ERROR: Nonconforming object sizes! [1]PETSC ERROR: Sum of local lengths 29784 does not equal global length 29791, my local length 2482 likely a call to VecSetSizes() or MatSetSizes() is wrong. See http://www.mcs.anl.gov/petsc/documentation/faq.html#split! [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: Petsc Development HG revision: 73637941c41b51aa3e638c914e6cb05248ece8b0 HG Date: Mon Jun 11 14:55:24 2012 -0400 [1]PETSC ERROR: See docs/changes/index.html for recent updates. [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [1]PETSC ERROR: See docs/index.html for manual pages. [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: ./ex10 on a arch-maco named madams-macbk-3.local by markadams Tue Jun 12 17:31:55 2012 [1]PETSC ERROR: Libraries linked from /Users/markadams/Codes/petsc-dev/arch-macosx-gnu-g/lib [1]PETSC ERROR: Configure run at Mon Jun 11 17:03:36 2012 [1]PETSC ERROR: Configure options CXX=/sw/lib/gcc4.4/bin/g++-4 CC=/sw/lib/gcc4.4/bin/gcc-4 CFLAGS=-malign-double CXXFLAGS=-malign-double FC=/sw/lib/gcc4.4/bin/gfortran --download-hypre=1 --download-parmetis=1 --download-metis=1 --download-triangle=1 --with-x=0 --with-debugging=1 --download-mpich=1 PETSC_ARCH=arch-macosx-gnu-g [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: PetscSplitOwnership() line 93 in /Users/markadams/Codes/petsc-dev/src/sys/utils/psplit.c [1]PETSC ERROR: PetscLayoutSetUp() line 147 in /Users/markadams/Codes/petsc-dev/src/vec/vec/impls/mpi/pmap.c [1]PETSC ERROR: MatMPIAIJSetPreallocation_MPIAIJ() line 3194 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c On Jun 12, 2012, at 5:19 PM, Blaise Bourdin wrote: > Hi, > > Sorry, I should have mentioned that the .h5 and xml files are not for public eyes. I am uploading the binary files created with -U_ksp_view_binary, but my internet connection sucks. > > Jed, I put the file on your account on schur. can you access them? The datafiles.tgz archive contains the the result files in hdf5 format. ksp_view_binary.tgz contains only the binary output of ksp_view_binary > > If you give me until tonight, I can try to set a prefix one the vecs. > > Thanks a lot for your help. > > Blaise > > On Jun 12, 2012, at 4:06 PM, Mark F. Adams wrote: > >> >> On Jun 12, 2012, at 4:56 PM, Jed Brown wrote: >> >>> On Tue, Jun 12, 2012 at 3:54 PM, Mark F. Adams <mark.adams(a)columbia.edu> wrote: >>> How is what? The ex10 code or "I set the block size for matrices before SetFromOptions". >>> >>> This how I create matrices with blocks sizes: >>> >>> ierr = MatCreate( wcomm, &Prol ); CHKERRQ(ierr); >>> ierr = MatSetSizes(Prol,nloc*bs,nLocalSelected*col_bs,PETSC_DETERMINE,PETSC_DETERMINE); >>> CHKERRQ(ierr); >>> ierr = MatSetBlockSizes( Prol, bs, col_bs ); CHKERRQ(ierr); >>> >>> Sorry, I thought you were talking about vectors because the error is coming from a vector load. >>> >> >> There is a vector load, at least ex10 loads a vec and its seems to work (so I assume Blaise put one in there). >> >>> Where is this input file? I'll grab it sort out the issue. >> >> You can get an archive containing the KSP and the data files at https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/52338Q0at2Y >> >> These are just standard "binaryoutput" files I think, >> >> Mark > > -- > Department of Mathematics and Center for Computation & Technology > Louisiana State University, Baton Rouge, LA 70803, USA > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin > > > > > > >

1 0

Re: [petsc-dev] error reading binary file
by Satish Balay 12 Jun '12

12 Jun '12

On Tue, 12 Jun 2012, Jed Brown wrote: > On Tue, Jun 12, 2012 at 4:06 PM, Mark F. Adams <mark.adams(a)columbia.edu>wrote: > > > There is a vector load, at least ex10 loads a vec and its seems to work > > (so I assume Blaise put one in there). > > > Fixed here, someone can merge it to petsc-dev (I have a dirty tree and > don't want to mess with it). > > http://petsc.cs.iit.edu/petsc/releases/petsc-3.3/rev/9e1563b28611 Its now merged into petsc-dev Satish

1 0

Re: [petsc-dev] error reading binary file
by Jed Brown 12 Jun '12

12 Jun '12

On Tue, Jun 12, 2012 at 4:35 PM, Mark F. Adams <mark.adams(a)columbia.edu>wrote: > And I now get the following error when I try to run this in parallel. > > Note, Blaise, I do not have coordinates, so perhaps you could check that > you still get divergence w/o coordinates. (I don't need anything else form > you) > Did you apply the patch? I ran that code in parallel using ./ex10 -f0 binaryoutput-sym

1 0