September 2011 - petsc-users - mailman3.cels.anl.gov

Matricize a vector
by elam＠cs.toronto.edu 06 Sep '11

06 Sep '11

Hello, Is it possible to reshape a vector into a matrix in PETSc? I need to do this to compute a matrix vector multiplication (Ax) where the matrix A can be decomposed into Kronecker tensor format. For example, A = A1 kron A2 kron A3, where A1, A2, A3 are smaller sparse matrices. Essentially, I need to matricize the vector x and perform a series of matrix matrix multiplications to compute Ax. In Matlab the function to perform the matricization is reshape. Is there a similar function in PETSc? I could implement the (tensor product) matrix vector multiplication by creating temporary matrices to store each of the intermediate matrix matrix multiplication results. But this seems inefficient. I was wondering if there is a more efficient way to implement this in PETSc? Can I simply reset the size of a matrix (as long as the number of elements remain the same) to reshape it? Any suggestions would be appreciated. Thank you for your help. Best regards, Elizabeth

3 4

(no subject)
by Kristen Eisenberg 05 Sep '11

05 Sep '11

[petsc-users] SLEPc generalized eigenvalue problem question? I create 2 matrices using: MatCreateSeqDense(PETSC_COMM_SELF, n, n, Ka, &A); MatCreateSeqDense(PETSC_COMM_SELF, n, n, Kb, &B); These matrices are 99% zeros ( 16,016,004 entries and 18660 non-zeros). They are symmetric and real. Their tri-diagonal elements are non-zero plus a few other entries. I tried to use ex7 for the generalized eigenvalue problem: ./ex7.exe -f1 k.dat -f2 m.dat -eps_gen_hermitian -eps_smallest_real > x.out 2>&1 without specifying an EPS and get: Generalized eigenproblem stored in file. Reading REAL matrices from binary files... Number of iterations of the method: 500 Number of linear iterations of the method: 4009 Solution method: krylovschur Number of requested eigenvalues: 1 Stopping condition: tol=1e-07, maxit=500 Number of converged approximate eigenpairs: 0 Is krylovschur inappropriate for this problem or have I set up the problem incorrectly by using MatCreateSeqDense(...) to create the matrix input files in PETSc binary form? Kristen Eisenberg Billige Flüge Marketing GmbH Emanuelstr. 3, 10317 Berlin Deutschland Telefon: +49 (33) 5310967 Email: utebachmeier at gmail.com Site: http://flug.airego.de - Billige Flüge vergleichen

2 1

Write complex matrix/vector from Matlab
by amrit poudel 04 Sep '11

04 Sep '11

Hello everyone, Does anyone here know how one can write a complex or real, matrix or vector (matrix could be sparse or full) in MATLAB to a binary file such that it can be read correctly by PETSc? I tried using PetscBinaryWrite.m that came with PETSc ( /bin/matlab) to write a simple real (or complex), full (or sparse) matrix in binary file named 'matrix.dat' and tried reading it in PETSc, but it absolutely does not work :( I did PetscBinaryWrite('matrix.dat', C) to write a matrix C in an empty file named 'matrix.dat' in MATLAB. Then I tried reading 'matrix.dat' file in PETSc using the following command : PetscViewer binviewer ; Mat C ; char fileC[PETSC_MAX_PATH_LEN] ; ierr = PetscOptionsGetString(PETSC_NULL, "-fC", fileC, PETSC_MAX_PATH_LEN-1, PETSC_NULL) ; CHKERRQ(ierr) ; ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, fileC, FILE_MODE_READ, &binviewer) ; CHKERRQ(ierr) ; ierr = MatLoad(binviewer, MATMPIAIJ, &C) ; CHKERRQ(ierr) ; I must add that I compiled PETSc with complex numbers. I plan on assembling my matrix (very large square sparse matrix, size greater than 1e+8) in MATLAB, which does not take too long to assemble in MATLAB at all. Then I will write the matrix to a binary file and read that in PETSc and solve a very large linear system of equations in parallel in PETSc using MPI. Hence, I really need to be able to write the matrix in MATLAB to a binary file such that PETSc can correctly read the matrix. I am really surprised to find out that PETSc cannot read/write a matrix or vector, full or sparse, real or complex in MATRIX MARKET FORMAT. I have gone though the manual, googled around quite a bit, but I just have not found anything useful. I will truly appreciate any help or a pointer in this regard. Thank you. Amrit

2 3

src/ksp/ksp/examples/tutorials/ex22.c
by Алексей Рязанов 04 Sep '11

04 Sep '11

Hello! Help me please to solve my tiny problem with dmmg. In the example src/ksp/ksp/examples/tutorials/ex22.c, which is discussed on the 111th page of users manual, we use function (line 36): DACreate3d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_STAR,-3,-3,-3,PETSC_DECIDE,PETSC_DECIDE,PETSC_DECIDE,1,1,0,0,0,&da); to "*create the DA that stores information about the coarsest grid you wish to use*". We can see that negative dimension values (-M, -N, -P) are used "*to indicate that it may be set to a different value from the command line with -da_grid_x <M> -da_grid_y <N> -da_grid_z <P>". * The question is: when I use -da_grid_ M,N,P options as the size of the coarsest matrix, what's the size of the finest (most accurate, original problem) matrix? I've tried to clear this up, using VecVew on the residual vector and figured out, that when i'm using M,N,P as input parameters (so define the coarsest grid as field of M*N*P dots), I get the residual vector of size (4*M-3)*(4*N-3)*(4*P-3). Is my result correct? If yes, could you please explain why it happens in this way. Sorry if my question is incorrect. I could mix up everything at all. Thanks in advance! -- Best regards, Alexey Ryazanov ______________________________________ Nuclear Safety Institute of Russian Academy of Sciences

3 2

PETSc 3.2 release candidate
by Barry Smith 03 Sep '11

03 Sep '11

You can find the PETSc 3.2 release candidate via Mecurial at: > > ssh://[email protected]//hg/petsc/releases/petsc-3.2 or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 > ssh://[email protected]//hg/petsc/releases/BuildSystem-3.2 or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 If you find any problems please let us know. We'll be making the tarball in a couple of days. Thanks Barry If you fix anything, please make the fix in petsc-3.2 not petsc-dev > >

4 5

ERROR: sum weight for constraint 0 is zero
by Dominik Szczerba 03 Sep '11

03 Sep '11

I am occasionally and not reproducibly getting this error: ERROR: sum weight for constraint 0 is zero It is emitted by parmetis. My code passed valgrind reasonably well. Well, the input mesh could be invalid in some way or another of course (even when I do not think so), but then the error should be reproducible, while it is not. Does anyone have an idea what can cause it and how to prevent it? Many hanks for any hints, Dominik

1 0

Re: [petsc-users] about MatTranspose
by Likun Tan 02 Sep '11

02 Sep '11

Thank you very much. With the command: for(j=0; j<n; j++) { for(i=0; i<M; i++) { MatSetValues(B, 1, &i, 1, &j, &Value, INSERT_VALUES); } } This only defines the columns from 0 to n-1? How exactly MPI_Scan() getting used here? And when i try to print out the result with PetscViewer viewer; PetscViewerSetFormat(viewer, PETSC_VIEWER_ASCII_MATLAB); MatView(B, viewer); I can only see the columns from 0 to n-1. Maybe there is still problem in computing B. Many thanks, Likun On Thu, September 1, 2011 1:54 pm, Jed Brown wrote: > On Thu, Sep 1, 2011 at 12:45, Likun Tan <likunt(a)andrew.cmu.edu> wrote: > > >> I still have some confusions. When computing B before MPI_Scan(), could >> i compute the values in parallel? After using MPI_Scan(), does that mean >> the columns of B will be gathered in one processor? >> > > No, the scan is just computing the start column for a given process. > > > >> >> Here are main steps i took based on your suggestion, >> >> >> PetscSplitOwnership(PETSC_COMM_WORLD, &n, &N); >> >> > > Add these two lines here: > > > MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); > cstart -= n; > > >> MatCreateSeqDense(PETSC_COMM_SELF, M, n, PETSC_NULL, &B); >> MatCreateSeqDense(PETSC_COMM_SELF, M, M, PETSC_NULL, &A); >> >> > > Good > > > >> MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &x); >> >> > > Replace with > > > MatDuplicate(B,MAT_DO_NOT_COPY_VALUES,&x); > > > >> for(j=0; j<N; j++) >> > > change the loop bounds here: > > for(j=0; j<n; j++) > > >> { >> for(i=0; i<M; i++) { >> >> > > Good, now compute value as the value that goes in (i,cstart+j). > > > MatSetValues(B, 1, &i, 1, &j, &value, INSERT_VALUES); > >> } >> } >> MatAssemblyBegin(...); >> MatAssemblyEnd(...) >> >> > > This part is correct. > > > >> MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); >> cstart -= n; >> > > We did this already. > > > >> >> MatConvert(...); >> MatCholeskyFactor(...); >> MatMatSolve(...); >> >> > > Yes. > > > > You can gather the matrix x onto all processes if you need the whole > result everywhere, but for performance reasons if you scale further, you > should avoid it if possible. >

2 2

Neumann BC in src/ksp/ksp/example/tutorial/ex29.c
by Alan Wei 02 Sep '11

02 Sep '11

Dear All, I hope you're having a nice day. I'm still working on this Poisson solver example, and I still have a little confusion on the boundary condition. I wonder if I have a 17*17 grid. (index i = 0~16 and j = 0~16, solving for variable 'u') 1) If a Dirichlet BC applied on the left boundary, i.e. u= 1, is that all values with u[i= 0][] = 1 or with u[i= -1][] = 1 (where, u[i=-1][] are ghost grids left next to grid of u[i= 0][]). 2) If a Neumann BC applied on the right boundary, i.e. du/dx = 0, is that (u[i=16][]-u[i=15][])/dx= or (u[i=17][]-u[i=16][])/dx = 0 (where u[i =17][] are ghost grids right next to grid of u[i = 16][]). Thank you so much. best, Alan

3 5

Re: [petsc-users] Mumps Error
by Max Rudolph 01 Sep '11

01 Sep '11

I'm not sure if you figured out a solution yet, but I think that you might want to run with -mat_mumps_icntl_14 100 Max >> Dear Users, >> >> I'm having trouble figuring out why the MUMPS solver is failing on a specific range of one of my parameters. When using the PETSc direct solver on a single processor I have no issues. There error is: >> >> [0]PETSC ERROR: --------------------- Error Message ------------------------------------ >> [0]PETSC ERROR: Error in external library! >> [0]PETSC ERROR: Error reported by MUMPS in numerical factorization phase: INFO(1)=-9, INFO(2)=13 >> ! >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 CDT 2011 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./cntor on a complex-c named hpc-1-14.local by abyrd Wed Aug 31 10:53:42 2011 >> [0]PETSC ERROR: Libraries linked from /panfs/storage.local/scs/home/abyrd/petsc-3.1-p8/complex-cpp-mumps/lib >> [0]PETSC ERROR: Configure run at Mon Jul 11 15:28:42 2011 >> [0]PETSC ERROR: Configure options PETSC_ARCH=complex-cpp-mumps --with-cc=mpicc --with-fc=mpif90 --with-blas-lapack-dir=/usr/lib64 --with-shared --with-clanguage=c++ --with-scalar-type=complex --download-mumps=1 --download-blacs=1 --download-scalapack=1 --download-parmetis=1 --with-cxx=mpicxx >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: MatFactorNumeric_MUMPS() line 517 in src/mat/impls/aij/mpi/mumps/mumps.c >> [0]PETSC ERROR: MatLUFactorNumeric() line 2587 in src/mat/interface/matrix.c >> [0]PETSC ERROR: PCSetUp_LU() line 158 in src/ksp/pc/impls/factor/lu/lu.c >> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >> [0]PETSC ERROR: InvertHamiltonian() line 102 in WDinvert.h >> >> >> I suspect it has something to do with the preconditioning or setup of the matrix I am trying to invert. The matrix becomes singular at energy = 0 eV, and is nearly singular for values close to that, but the code is failing on energies relatively far from that point. The affected energy interval is [-0.03095, 0.03095]. >> >> Is anyone able to point in the right direction to figure out what I'm not setting up properly? >> >> Respectfully, >> Adam Byrd >> <PETScCntor.zip> > > > > > _______________________________________________ > petsc-users mailing list > petsc-users(a)mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/petsc-users

3 2

Re: [petsc-users] about MatTranspose
by Likun Tan 01 Sep '11

01 Sep '11

Thank you for your patience. I still have some confusions. When computing B before MPI_Scan(), could i compute the values in parallel? After using MPI_Scan(), does that mean the columns of B will be gathered in one processor? Here are main steps i took based on your suggestion, PetscSplitOwnership(PETSC_COMM_WORLD, &n, &N); MatCreateSeqDense(PETSC_COMM_SELF, M, n, PETSC_NULL, &B); MatCreateSeqDense(PETSC_COMM_SELF, M, M, PETSC_NULL, &A); MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &x); for(j=0; j<N; j++) { for(i=0; i<M; i++) { MatSetValues(B, 1, &i, 1, &j, &value, INSERT_VALUES); } } MatAssemblyBegin(...); MatAssemblyEnd(...) MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); cstart -= n; MatConvert(...); MatCholeskyFactor(...); MatMatSolve(...); best, Likun On Thu, September 1, 2011 11:59 am, Jed Brown wrote: > On Thu, Sep 1, 2011 at 10:46, Likun Tan <likunt(a)andrew.cmu.edu> wrote: > > >> I tried to use >> PetscSplitOwnership(PETSC_COMM_WORLD,&n,&N); >> MatCreateSeqDense(PETSC_COMM_SELF,M,n,PETSC_NULL,&B); >> >> >> But it seems not working. This command is make B store in N/n >> processors with n columns in each processor? >> > > B (in the code) is a different matrix on each process. You should fill > the columns so that each process has part of B (the logical thing you are > working with). > > >> I try to calculate the N/n parts in >> parallel, but when i apply MatGetOwnershipRange(), i found the matrix >> will be automatically partitioned by row. >> > > MatGetOwnershipRange() just gives you the rows from 0 to 26. You need to > sum the column partition that was computed earlier. The most explicit way > to do that is > > PetscInt cstart; > MPI_Scan(&n,&cstart,1,MPIU_INT,MPI_SUM,PETSC_COMM_WORLD); > cstart -= n; > > Now each process should compute columns of the logical B in the range > [cstart,cstart+n). The B object in your code will indexed [0,n) on each > process, but these columns correspond to [cstart,cstart+n). > > >> That's why i go back to use MatCreate() to create a 200*27 matrix and >> try to get the tranpose of it.I used MatCreateSeqDense(PETSC_COMM_SELF, >> M, N, PETSC_NULL, &C) >> to create C, there is no partition of C, i guess this is the reason of >> the error. >> > > Don't go down this route, it doesn't do what you want. > >

2 1