Hi everyone,

I use PETSC (version 3.1-p5) to solve a linear problem Ax=b. The matrix A and right hand vector b are read from files. The dimension of A is 1.2Million*1.2Million. I am pretty sure the matrix A and vector b have been read correctly.

I compiled the program with optimized version (--with-debugging=0), tested the speed up performance on two servers, and I have found that the performance is very poor.

For the two servers, one is 4 cpus * 4 cores per cpu, i.e., with a total 16 cores. And the other one is 4 cpus * 12 cores per cpu, with a total 48 cores.

On each of them, with the increasing of computing cores k from 1 to 8 (mpiexec –n  k ./Solver_MPI -pc_type jacobi -ksp-type gmres), the speed up will increase from 1 to 6, but when the computing cores k increase from 9 to 16(for the first server) or 48 (for the second server), the speed up decrease firstly and then remains a constant value 5.0 (for the first server) or 4.5(for the second server).

Actually, the program LAMMPS speed up excellently on these two servers.

Any comments are very appreciated! Thanks!

--------------------------------------------------------------------------------------------------------------------------

PS: the related codes are as following,

//firstly read A and b from files

...

//then

              ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);

              ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);

              ierr = VecAssemblyBegin(b); CHKERRQ(ierr);

              ierr = VecAssemblyEnd(b); CHKERRQ(ierr);

              ierr = MatSetOption(A,MAT_SYMMETRIC,PETSC_TRUE); CHKERRQ(ierr);

              ierr = MatGetRowUpperTriangular(A); CHKERRQ(ierr);

              ierr = KSPCreate(PETSC_COMM_WORLD,&ksp);CHKERRQ(ierr);

              ierr = KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN);CHKERRQ(ierr);

              ierr = KSPGetPC(ksp,&pc);CHKERRQ(ierr);

              ierr = KSPSetTolerances(ksp,1.e-7,PETSC_DEFAULT,PETSC_DEFAULT,PETSC_DEFAULT);CHKERRQ(ierr);

              ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr);

              ierr = KSPSolve(ksp,b,x);CHKERRQ(ierr);

              ierr = KSPView(ksp,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr);

              ierr = KSPGetSolution(ksp, &x);CHKERRQ(ierr);

              ierr = VecAssemblyBegin(x);CHKERRQ(ierr);

              ierr = VecAssemblyEnd(x);CHKERRQ(ierr);

...