Following your comments,
I used below commandmpirun -np 4 ./app -ksp_type preonly -pc_type mpi -mpi_linear_solver_server -mpi_pc_type lu -mpi_pc_factor_mat_solver_type mumps -mpi_mat_mumps_icntl_7 5 -mpi_ksp_viewso the output is as belowKSP Object: (mpi_) 1 MPI process
type: gmres
restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
happy breakdown tolerance 1e-30
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
PC Object: (mpi_) 1 MPI process
type: lu
out-of-place factorization
tolerance for zero pivot 2.22045e-14
matrix ordering: external
factor fill ratio given 0., needed 0.
Factored matrix follows:
Mat Object: (mpi_) 1 MPI process
type: mumps
rows=192, cols=192
package used to perform factorization: mumps
total: nonzeros=17334, allocated nonzeros=17334
MUMPS run parameters:
Use -mpi_ksp_view ::ascii_info_detail to display information for all processes
RINFOG(1) (global estimated flops for the elimination after analysis): 949441.
RINFOG(2) (global estimated flops for the assembly after factorization): 18774.
RINFOG(3) (global estimated flops for the elimination after factorization): 949441.
(RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0.,0.)*(2^0)
INFOG(3) (estimated real workspace for factors on all processors after analysis): 17334
INFOG(4) (estimated integer workspace for factors on all processors after analysis): 1724
INFOG(5) (estimated maximum front size in the complete tree): 96
INFOG(6) (number of nodes in the complete tree): 16
INFOG(7) (ordering option effectively used after analysis): 5
INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100
INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 17334
INFOG(10) (total integer space store the matrix factors after factorization): 1724
INFOG(11) (order of largest frontal matrix after factorization): 96
INFOG(12) (number of off-diagonal pivots): 0
INFOG(13) (number of delayed pivots after factorization): 0
INFOG(14) (number of memory compress after factorization): 0
INFOG(15) (number of steps of iterative refinement after solution): 0
INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 1
INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 1
INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 1
INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 1
INFOG(20) (estimated number of entries in the factors): 17334
INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 1
INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 1
INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0
INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1
INFOG(25) (after factorization: number of pivots modified by static pivoting): 0
INFOG(28) (after factorization: number of null pivots encountered): 0
INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 17334
INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 0, 0
INFOG(32) (after analysis: type of analysis done): 1
INFOG(33) (value used for ICNTL(8)): 7
INFOG(34) (exponent of the determinant if determinant is requested): 0
INFOG(35) (after factorization: number of entries taking into account BLR factor compression - sum over all processors): 17334
INFOG(36) (after analysis: estimated size of all MUMPS internal data for running BLR in-core - value on the most memory consuming processor): 0
INFOG(37) (after analysis: estimated size of all MUMPS internal data for running BLR in-core - sum over all processors): 0
INFOG(38) (after analysis: estimated size of all MUMPS internal data for running BLR out-of-core - value on the most memory consuming processor): 0
INFOG(39) (after analysis: estimated size of all MUMPS internal data for running BLR out-of-core - sum over all processors): 0
linear system matrix = precond matrix:
Mat Object: 1 MPI process
type: seqaij
rows=192, cols=192
total: nonzeros=9000, allocated nonzeros=36864
total number of mallocs used during MatSetValues calls=0
using I-node routines: found 64 nodes, limit used is 5Is it correct that I successfully computed with mumps by using metis(icntl_7 5)?
Thanks,Hyung Kim2022년 12월 7일 (수) 오후 8:41, Matthew Knepley <knepley@gmail.com>님이 작성:On Wed, Dec 7, 2022 at 6:15 AM 김성익 <ksi2443@gmail.com> wrote:I think I don't understand the meaning of
-pc_type mpiThis option says to use the PCMPI preconditioner. This allows you to parallelize thesolver in what is otherwise a serial code.-mpi_pc_type luThis tells the underlying solver in PCMPI to use the LU preconditioner.What's the exact meaning of -pc_type mpi and -mpi_pc_type lu??Is this difference coming from 'mpi_linear_solver_server' option??Please use -ksp_view as I asked to look at the entire solver. Send it anytime you mail about solver questions.ThanksMattThanks,Hyung Kim2022년 12월 7일 (수) 오후 8:05, Matthew Knepley <knepley@gmail.com>님이 작성:On Wed, Dec 7, 2022 at 5:13 AM 김성익 <ksi2443@gmail.com> wrote:I want to use METIS for ordering.
I heard the MUMPS has good performance with METIS ordering.However there are some wonder things.1. With option "-mpi_linear_solver_server -ksp_type preonly -pc_type mpi -mpi_pc_type lu " the MUMPS solving is slower than with option "-mpi_linear_solver_server -pc_type mpi -ksp_type preonly".
Why does this result happen?You are probably not using MUMPS. Always always always use -ksp_view to see exactly what solver you are using.2. (MPIRUN case (actually, mpi_linear_solver_server case))) In my code, there is already has "PetscCall(PCSetType(pc,PCLU))" . However, to use METIS by using "-mpi_mat_mumps_icntl_7 5" I must append this option "-mpi_pc_type pu".
If I don't apply "-mpi_pc_type lu", the metis option ("-mpi_mat_mumps_icntl_7 5"). Can I get some information about this?Again, it seems like the solver configuration is not what you think it is.Thanks,MattThanks,Hyung Kim2022년 12월 7일 (수) 오전 12:24, Barry Smith <bsmith@petsc.dev>님이 작성:On Dec 6, 2022, at 5:15 AM, 김성익 <ksi2443@gmail.com> wrote:Hello,
I have some questions about pc and mumps_icntl.
1. What’s the difference between adopt preconditioner by code (for example, PetscCall(PCSetType(pc,PCLU)) and option -pc_type lu??
And also, What’s the priority between code pcsettype and option -pc_type ??
2. When I tried to use METIS in MUMPS, I adopted metis by option (for example, -mat_mumps_icntl_7 5). In this situation, it is impossible to use metis without pc_type lu. However, in my case pc type lu makes the performance poor. So I don’t want to use lu preconditioner. How can I do this?
The package MUMPS has an option to use metis in its ordering process which can be turned on as indicated while using MUMPS. Most preconditioners that PETSc can use do not use metis for any purpose hence there is no option to turn on its use. For what purpose do you wish to use metis? Partitioning, ordering, ?Thanks,
Hyung Kim
--What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener--What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener