December 2010 - petsc-users - mailman3.cels.anl.gov

Set number of iterations
by Jakub Pola 11 Dec '10

11 Dec '10

Hi again, I was searching trought documentation but without success. I would like to know how to set the number of iterations during the KSPSolve(). Is it possible to set the iteration numbers to 100 for example? Kuba

2 1

optimizing repeated calls to KSPsolve?
by Luke Bloy 11 Dec '10

11 Dec '10

Hi I'm new to Petsc so excuse me if this question is naive. I'm trying to solve the following system A x = b for x. A is a sparse square matrix (2000000 by 2000000 with ~45,000,000 nonzero elements) I'm currently using ex1 as the basis for solving the system and it is working quite well. My problem is that i have a large number (~500,000) of b vectors that I would like to find solutions for. My plan is to call KSPsolve repeatedly with each b. However I wonder if there are any solvers or approaches that might benefit from the fact that my A matrix does not change. Are there any decompositions that might still be sparse that would offer a speed up? Thanks for any suggestions. Luke

4 8

global index distributed arrays
by Mohamad M. Nasr-Azadani 10 Dec '10

10 Dec '10

Hi guys, I was wondering if there is an easy way of accessing the global index of a node which is not within the local and ghost node regions on every processor for DA. To be more more specific, I am trying to setup a matrix based on a three-dimensional DA. (Star stencil, width=1). For some special nodes, I need to insert nonzero values which do not fit in the local plus the ghost regions of the DA. I know that I can not use MatSetValuesStencil anymore, but I still can use MatSetValues. But I need to know the global index of those nodes. I tried to use DAGetGlobalIndices(), but that would again return the global indices of the local plus ghost nodes on current processor. I know that I could use some MPI commands to pass those indices among processors, but I was wondering if there a clean and neat way with which each processor can have access to the global index for any given 3-d index. Thank, Mohamad

3 11

optimizing repeated calls to KSPsolve?
by Luke Bloy 10 Dec '10

10 Dec '10

Hi I'm new to Petsc so excuse me if this question is naive. I'm trying to solve the following system A x = b for x. A is a sparse square matrix (2000000 by 2000000 with ~45,000,000 nonzero elements) I'm currently using ex1 as the basis for solving the system and it is working quite well. My problem is that i have a large number (~500,000) of b vectors that I would like to find solutions for. My plan is to call KSPsolve repeatedly with each b. However I wonder if there are any solvers or approaches that might benefit from the fact that my A matrix does not change. Are there any decompositions that might still be sparse that would offer a speed up? Thanks for any suggestions. Luke

1 0

KSPBICG on GPU
by Jakub Pola 10 Dec '10

10 Dec '10

Hi Does anyone have a benchmark of KSPBICG solver which gives me an information about loop time, GFLOPS, memory transfer bandwidth ? Thanks in advance

1 0

pets-3.1-p6 with CUDA: Unknown vector type: cuda!
by Jakub Pola 10 Dec '10

10 Dec '10

Hello, I was trying to use pets library with my GPU (GTX 460 ) and see how it works. Unfortunatelly I cant run any example with -vec_type cuda. Thisis my log from executing ./ex2 -vec_type cuda [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type seehttp://www.mcs.anl.gov/petsc/petsc-as/documentation/installation.html#ex…! [0]PETSC ERROR: Unknown vector type: cuda! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 6, Tue Nov 16 17:02:32 CST 2010 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./ex2 on a linux-gnu named desktop by kuba Thu Dec 9 20:20:38 2010 [0]PETSC ERROR: Libraries linked from /home/kuba/External/petsc-3.1-p6/linux-gnu-c-debug/lib [0]PETSC ERROR: Configure run at Thu Dec 9 19:50:31 2010 [0]PETSC ERROR: Configure options --with-cc=gcc --with-fc=gfortran --download-f-blas-lapack=1 --download-mpich=1 --with-cuda=1 --with-debug=no [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: VecSetType() line 46 in src/vec/vec/interface/vecreg.c [0]PETSC ERROR: VecSetTypeFromOptions_Private() line 1335 in src/vec/vec/interface/vector.c [0]PETSC ERROR: VecSetFromOptions() line 1372 in src/vec/vec/interface/vector.c [0]PETSC ERROR: main() line 127 in src/ksp/ksp/examples/tutorials/ex2.c application called MPI_Abort(MPI_COMM_WORLD, 86) - process 0[unset]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 86) - process 0 I configured my pets with following command: petsc-3.1-p6$ ./config/configure.py --with-cc=gcc --with-fc=gfortran --download-f-blas-lapack=1 --download-mpich=1 --with-cuda=1 --with-debug=no Everything went ok. Then I compiled the library with make PETSC_DIR=/home/kuba/External/petsc-3.1-p6 PETSC_ARCH=linux-gnu-c-debug all and after that tests make PETSC_DIR=/home/kuba/External/petsc-3.1-p6 PETSC_ARCH=linux-gnu-c-debug test Tests were performed successfully. I have cuda configured in LD_LIBRARY_PATH: ldconfig -p | grep cuda libicudata.so.42 (ELF) => /usr/lib/libicudata.so.42 libcusparse.so.3 (libc6) => /usr/local/cuda/lib/libcusparse.so.3 libcusparse.so (libc6) => /usr/local/cuda/lib/libcusparse.so libcurand.so.3 (libc6) => /usr/local/cuda/lib/libcurand.so.3 libcurand.so (libc6) => /usr/local/cuda/lib/libcurand.so libcufft.so.3 (libc6) => /usr/local/cuda/lib/libcufft.so.3 libcufft.so (libc6) => /usr/local/cuda/lib/libcufft.so libcudart.so.3 (libc6) => /usr/local/cuda/lib/libcudart.so.3 libcudart.so (libc6) => /usr/local/cuda/lib/libcudart.so libcuda.so.1 (libc6) => /usr/lib/nvidia-current/libcuda.so.1 libcuda.so (libc6) => /usr/lib/nvidia-current/libcuda.so libcublas.so.3 (libc6) => /usr/local/cuda/lib/libcublas.so.3 libcublas.so (libc6) => /usr/local/cuda/lib/libcublas.so and I have cuda directories in PATH: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/kuba/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/usr/local/cuda/lib:/usr/local/cuda/lib64" Could you please help me with that. My final destination is to run CG and BICGSTAB solvers and measure performance using sparse matrices. Thank you in advance for your help. Best regards, Jakub.

5 11

MatMAIJ tests
by Nunion 07 Dec '10

07 Dec '10

Hello all, What would be a fix to get around the issue with complex numbers for ex100.c: Tests vatious routines in MatMAIJ format<http://w3.pppl.gov/m3d/petsc-dev/src/mat/examples/tests/ex100.c.html>in the src/mat/examples/tests/ directory. Currently it is not written for complex numbers. Thanks, Mark

2 2

Is PCMG a generic PC object ?
by Vijay S. Mahadevan 07 Dec '10

07 Dec '10

Hi all, I was wondering whether the MG preconditioner object is generic enough to work out of the box like say ILU or SOR. To elaborate on this, if I can provide the number of levels, restriction and prolongation operators for each level and the system operators along with vectors allocated for solution and rhs, would it work as a preconditioner for my given problem and a prescribed rhs at the finest level of PCMG. Or does it need some knowledge of the fine and coarser meshes to perform the MG operations correctly ? All the examples I've seen using MG in petsc involve the DA and DMMG objects and since I use my own mesh and corresponding discretization code for an elliptic system, I'm curious about this usage. It would not be terribly difficult to write my own framework to do a simple V-cycle with my existing framework but since petsc already provides this functionality along with different types of MG solves (with verified code!), I really want to use it for my system. Any help and/or pointers are welcome. Thanks, vijay

6 21

SuperLU_4.0 with petsc-3.1-p3 - flipped output in KSPSolve.
by Chetan Jhurani 07 Dec '10

07 Dec '10

Hello, I have a small code that solves a 3x3 system with and without SuperLU. If SuperLU is used, vectors x and b (in A x = b) come out flipped after KSPSolve. Am I doing something stupid? The two outputs, the code, and the configure command are pasted below. Thanks, Chetan -------------------------------------------------------------------------- Run with SuperLU: ~> lu_test -pc_factor_mat_solver_package superlu x before solve -10 -10 -10 b before solve 200 200 200 x after solve 200 200 200 b after solve 0.2 0.2 0.2 -------------------------------------------------------------------------- Run without SuperLU (default KSP, PC options) ~> lu_test x before solve -10 -10 -10 b before solve 200 200 200 x after solve 0.2 0.2 0.2 b after solve 200 200 200 -------------------------------------------------------------------------- Program: int main(int argc, char* argv[]) { PetscErrorCode ierr; ierr = PetscInitialize(&argc, &argv, PETSC_NULL, PETSC_NULL); CHKERRQ(ierr); Mat A; ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, 3, 3, 1, PETSC_NULL, &A); CHKERRQ(ierr); ierr = MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); ierr = MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); ierr = MatShift(A, 1000); CHKERRQ(ierr); Vec x, b; ierr = VecCreateSeq(PETSC_COMM_SELF, 3, &x); CHKERRQ(ierr); ierr = VecShift(x, -10); CHKERRQ(ierr); ierr = VecCreateSeq(PETSC_COMM_SELF, 3, &b); CHKERRQ(ierr); ierr = VecShift(b, 200); CHKERRQ(ierr); KSP ksp; ierr = KSPCreate(PETSC_COMM_SELF, &ksp); CHKERRQ(ierr); ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); printf("x before solve\n"); ierr = VecView(x, PETSC_VIEWER_STDOUT_SELF); CHKERRQ(ierr); printf("b before solve\n"); ierr = VecView(b, PETSC_VIEWER_STDOUT_SELF); CHKERRQ(ierr); ierr = KSPSetOperators(ksp, A, A, SAME_NONZERO_PATTERN); CHKERRQ(ierr); ierr = KSPSolve(ksp, b, x); CHKERRQ(ierr); printf("x after solve\n"); ierr = VecView(x, PETSC_VIEWER_STDOUT_SELF); CHKERRQ(ierr); printf("b after solve\n"); ierr = VecView(b, PETSC_VIEWER_STDOUT_SELF); CHKERRQ(ierr); ierr = PetscFinalize(); CHKERRQ(ierr); } -------------------------------------------------------------------------- configure command: python ./config/configure.py -with-clanguage=C++ -with-debugging=no --with-gnu-compilers=1 --with-mpi=1 --with-umfpack=1 --with-superlu=1 --with-hypre=1 --download-umfpack=1 --download-superlu=1 --download-hypre=1 --with-mumps=1 --download-mumps=1 --with-parmetis=1 --download-parmetis=1 --with-scalapack=1 --download-scalapack=1 --with-blacs=1 --download-blacs=1 -with-c-support=1 --------------------------------------------------------------------------

3 3

run direct linear solver in parallel
by Xiangdong Liang 03 Dec '10

03 Dec '10

Hi everyone, I am wondering how I can run the direct solver in parallel. I can run my program in a single processor with direct linear solver by ./foo.out -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package spooles However, when I try to run it with mpi: mpirun.openmpi -np 2 ./foo.out -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package spooles I got error like this: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: No support for this operation for this object type! [0]PETSC ERROR: Matrix type mpiaij symbolic LU! [0]PETSC ERROR: Libraries linked from /home/hazelsct/petsc-2.3.3/lib/linux-gnu-c-opt [0]PETSC ERROR: Configure run at Mon Jun 30 14:37:52 2008 [0]PETSC ERROR: Configure options --with-shared --with-dynamic --with-debugging=0 --useThreads 0 --with-mpi-dir=/usr/lib/openmpi --with-mpi-shared=1 --with-blas-lib=-lblas --with-lapack-lib=-llapack --with-umfpack=1 --with-umfpack-include=/usr/include/suitesparse --with-umfpack-lib="[/usr/lib/libumfpack.so,/usr/lib/libamd.so]" --with-superlu=1 --with-superlu-include=/usr/include/superlu --with-superlu-lib=/usr/lib/libsuperlu.so --with-spooles=1 --with-spooles-include=/usr/include/spooles --with-spooles-lib=/usr/lib/libspooles.so --with-hypre=1 --with-hypre-dir=/usr --with-babel=1 --with-babel-dir=/usr [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatLUFactorSymbolic() line 2174 in src/mat/interface/matrix.c [0]PETSC ERROR: PCSetUp_LU() line 257 in src/ksp/pc/impls/factor/lu/lu.c ------------------------------------------------------- Would you like to tell me where I am doing wrong? I appreciate your help. Xiangdong

4 3