On Wed, Dec 7, 2022 at 8:15 AM 김성익 <[email protected]> wrote:
Following your comments, I used below command mpirun -np 4 ./app -ksp_type preonly -pc_type mpi -mpi_linear_solver_server -mpi_pc_type lu -mpi_pc_factor_mat_solver_type mumps -mpi_mat_mumps_icntl_7 5 -mpi_ksp_view
so the output is as below
KSP Object: (mpi_) 1 MPI process type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: (mpi_) 1 MPI process type: lu out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: external factor fill ratio given 0., needed 0. Factored matrix follows: Mat Object: (mpi_) 1 MPI process type: mumps rows=192, cols=192 package used to perform factorization: mumps total: nonzeros=17334, allocated nonzeros=17334 MUMPS run parameters: Use -mpi_ksp_view ::ascii_info_detail to display information for all processes RINFOG(1) (global estimated flops for the elimination after analysis): 949441. RINFOG(2) (global estimated flops for the assembly after factorization): 18774. RINFOG(3) (global estimated flops for the elimination after factorization): 949441. (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0.,0.)*(2^0) INFOG(3) (estimated real workspace for factors on all processors after analysis): 17334 INFOG(4) (estimated integer workspace for factors on all processors after analysis): 1724 INFOG(5) (estimated maximum front size in the complete tree): 96 INFOG(6) (number of nodes in the complete tree): 16 INFOG(7) (ordering option effectively used after analysis): 5 INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100 INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 17334 INFOG(10) (total integer space store the matrix factors after factorization): 1724 INFOG(11) (order of largest frontal matrix after factorization): 96 INFOG(12) (number of off-diagonal pivots): 0 INFOG(13) (number of delayed pivots after factorization): 0 INFOG(14) (number of memory compress after factorization): 0 INFOG(15) (number of steps of iterative refinement after solution): 0 INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 1 INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 1 INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 1 INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 1 INFOG(20) (estimated number of entries in the factors): 17334 INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 1 INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 1 INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1 INFOG(25) (after factorization: number of pivots modified by static pivoting): 0 INFOG(28) (after factorization: number of null pivots encountered): 0 INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 17334 INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 0, 0 INFOG(32) (after analysis: type of analysis done): 1 INFOG(33) (value used for ICNTL(8)): 7 INFOG(34) (exponent of the determinant if determinant is requested): 0 INFOG(35) (after factorization: number of entries taking into account BLR factor compression - sum over all processors): 17334 INFOG(36) (after analysis: estimated size of all MUMPS internal data for running BLR in-core - value on the most memory consuming processor): 0 INFOG(37) (after analysis: estimated size of all MUMPS internal data for running BLR in-core - sum over all processors): 0 INFOG(38) (after analysis: estimated size of all MUMPS internal data for running BLR out-of-core - value on the most memory consuming processor): 0 INFOG(39) (after analysis: estimated size of all MUMPS internal data for running BLR out-of-core - sum over all processors): 0 linear system matrix = precond matrix: Mat Object: 1 MPI process type: seqaij rows=192, cols=192 total: nonzeros=9000, allocated nonzeros=36864 total number of mallocs used during MatSetValues calls=0 using I-node routines: found 64 nodes, limit used is 5
Is it correct that I successfully computed with mumps by using metis(icntl_7 5)?
Yes, first you know it is MUMPS Mat Object: (mpi_) 1 MPI process type: mumps Second you know METIS was used INFOG(7) (ordering option effectively used after analysis): 5 I would not expect much speedup on 2 processes with such a small matrix rows=192, cols=192 Thanks Matt
Thanks, Hyung Kim
2022년 12월 7일 (수) 오후 8:41, Matthew Knepley <[email protected]>님이 작성:
On Wed, Dec 7, 2022 at 6:15 AM 김성익 <[email protected]> wrote:
I think I don't understand the meaning of -pc_type mpi
This option says to use the PCMPI preconditioner. This allows you to parallelize the solver in what is otherwise a serial code.
-mpi_pc_type lu
This tells the underlying solver in PCMPI to use the LU preconditioner.
What's the exact meaning of -pc_type mpi and -mpi_pc_type lu?? Is this difference coming from 'mpi_linear_solver_server' option??
Please use -ksp_view as I asked to look at the entire solver. Send it anytime you mail about solver questions.
Thanks
Matt
Thanks, Hyung Kim
2022년 12월 7일 (수) 오후 8:05, Matthew Knepley <[email protected]>님이 작성:
On Wed, Dec 7, 2022 at 5:13 AM 김성익 <[email protected]> wrote:
I want to use METIS for ordering. I heard the MUMPS has good performance with METIS ordering.
However there are some wonder things. 1. With option "-mpi_linear_solver_server -ksp_type preonly -pc_type mpi -mpi_pc_type lu " the MUMPS solving is slower than with option "-mpi_linear_solver_server -pc_type mpi -ksp_type preonly". Why does this result happen?
You are probably not using MUMPS. Always always always use -ksp_view to see exactly what solver you are using.
2. (MPIRUN case (actually, mpi_linear_solver_server case))) In my code, there is already has "PetscCall(PCSetType(pc,PCLU))" . However, to use METIS by using "-mpi_mat_mumps_icntl_7 5" I must append this option "-mpi_pc_type pu". If I don't apply "-mpi_pc_type lu", the metis option ("-mpi_mat_mumps_icntl_7 5"). Can I get some information about this?
Again, it seems like the solver configuration is not what you think it is.
Thanks,
Matt
Thanks, Hyung Kim
2022년 12월 7일 (수) 오전 12:24, Barry Smith <[email protected]>님이 작성:
On Dec 6, 2022, at 5:15 AM, 김성익 <[email protected]> wrote:
Hello,
I have some questions about pc and mumps_icntl.
1. What’s the difference between adopt preconditioner by code (for example, PetscCall(PCSetType(pc,PCLU)) and option -pc_type lu?? And also, What’s the priority between code pcsettype and option -pc_type ??
2. When I tried to use METIS in MUMPS, I adopted metis by option (for example, -mat_mumps_icntl_7 5). In this situation, it is impossible to use metis without pc_type lu. However, in my case pc type lu makes the performance poor. So I don’t want to use lu preconditioner. How can I do this?
The package MUMPS has an option to use metis in its ordering process which can be turned on as indicated while using MUMPS. Most preconditioners that PETSc can use do not use metis for any purpose hence there is no option to turn on its use. For what purpose do you wish to use metis? Partitioning, ordering, ?
Thanks,
Hyung Kim
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>