Question About Assembly matrix and declaration of KSP & pc
Hello, I’m working on FEM using PETSc. As everyone knows, it is necessary to repeatedly solve Ax=B. Regarding this, I have 4 questions. 1. There are many steps for preparing KSPSolve. For example KSPcreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions… In Nonlinear FEM, there are repeatedly kspsolve for getting answer vector. Is it correct to do all of the aforementioned processes (KSPcreate, KSPSetOperators ~~~) for each KSPSolve? Or should I declare it only once at the beginning and not call it again? 2. If the answer to question 1 is that it must be repeated every time, should this work be done right before kspsolve, that is, when the global matrix assembly is finished, or is it irrelevant to performance at any time? 3. When performing FEM, local matrices are often scattered in global matrices depending on connectivity. In this case, which is better in terms of performance: adding the values one by one with MatSetValue or adding them all at once with MatSetValues even if they are scattered? 4. I would like to measure the time of each section of the process. Which method is recommended? Thank you for your help. Hyung Kim
On Wed, Nov 30, 2022 at 5:08 AM 김성익 <[email protected]> wrote:
Hello,
I’m working on FEM using PETSc.
As everyone knows, it is necessary to repeatedly solve Ax=B.
Regarding this, I have 4 questions.
1. There are many steps for preparing KSPSolve. For example KSPcreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions… In Nonlinear FEM, there are repeatedly kspsolve for getting answer vector. Is it correct to do all of the aforementioned processes (KSPcreate, KSPSetOperators ~~~) for each KSPSolve? Or should I declare it only once at the beginning and not call it again?
You just do these once at setup but for nonlinear problems KSPSetOperators tells the solver that you have a new matrix and so "matrix setup" work needs to be done.
2. If the answer to question 1 is that it must be repeated every time, should this work be done right before kspsolve, that is, when the global matrix assembly is finished, or is it irrelevant to performance at any time?
KSPSetOperators should be set after the new matrix values are set but it might work before. It just sets a pointer to the matrix and flags it as not setup.
3. When performing FEM, local matrices are often scattered in global matrices depending on connectivity. In this case, which is better in terms of performance: adding the values one by one with MatSetValue or adding them all at once with MatSetValues even if they are scattered?
You want to add one element matrix at a time, generally.
4. I would like to measure the time of each section of the process. Which method is recommended?
PETSc methods are timed separateluy, but setup gets folded into KSPSolve unless you call SNESSetUp before the SNES[KSP]Solve. You can add you own timers also https://petsc.org/release/docs/manualpages/Profiling/PetscLogEventRegister/ Mark
Thank you for your help.
Hyung Kim
Thank you for your comments. However, I have more questions. 1. Generally, (KSPCreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions ) above functions are should be called after each "MatassemblyEnd??" 2. Though reading the user guide, I don't fully understand under what circumstances the functions mentioned above should be called again. Can you explain when each function should be called? Thanks, Hyung Kim 2022년 11월 30일 (수) 오후 8:37, Mark Adams <[email protected]>님이 작성:
On Wed, Nov 30, 2022 at 5:08 AM 김성익 <[email protected]> wrote:
Hello,
I’m working on FEM using PETSc.
As everyone knows, it is necessary to repeatedly solve Ax=B.
Regarding this, I have 4 questions.
1. There are many steps for preparing KSPSolve. For example KSPcreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions… In Nonlinear FEM, there are repeatedly kspsolve for getting answer vector. Is it correct to do all of the aforementioned processes (KSPcreate, KSPSetOperators ~~~) for each KSPSolve? Or should I declare it only once at the beginning and not call it again?
You just do these once at setup but for nonlinear problems KSPSetOperators tells the solver that you have a new matrix and so "matrix setup" work needs to be done.
2. If the answer to question 1 is that it must be repeated every time, should this work be done right before kspsolve, that is, when the global matrix assembly is finished, or is it irrelevant to performance at any time?
KSPSetOperators should be set after the new matrix values are set but it might work before. It just sets a pointer to the matrix and flags it as not setup.
3. When performing FEM, local matrices are often scattered in global matrices depending on connectivity. In this case, which is better in terms of performance: adding the values one by one with MatSetValue or adding them all at once with MatSetValues even if they are scattered?
You want to add one element matrix at a time, generally.
4. I would like to measure the time of each section of the process. Which method is recommended?
PETSc methods are timed separateluy, but setup gets folded into KSPSolve unless you call SNESSetUp before the SNES[KSP]Solve. You can add you own timers also https://petsc.org/release/docs/manualpages/Profiling/PetscLogEventRegister/
Mark
Thank you for your help.
Hyung Kim
On Wed, Nov 30, 2022 at 7:51 AM 김성익 <[email protected]> wrote:
Thank you for your comments. However, I have more questions.
1. Generally, (KSPCreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions ) above functions are should be called after each "MatassemblyEnd??"
KSPCreate is called once. You do not need PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions more than once, unless you want to change the solver type. KSPSetOperators is called if you want to change the system matrix. KSPSolve is called when you want to change the rhs. Thanks, Matt
2. Though reading the user guide, I don't fully understand under what circumstances the functions mentioned above should be called again. Can you explain when each function should be called?
Thanks,
Hyung Kim
2022년 11월 30일 (수) 오후 8:37, Mark Adams <[email protected]>님이 작성:
On Wed, Nov 30, 2022 at 5:08 AM 김성익 <[email protected]> wrote:
Hello,
I’m working on FEM using PETSc.
As everyone knows, it is necessary to repeatedly solve Ax=B.
Regarding this, I have 4 questions.
1. There are many steps for preparing KSPSolve. For example KSPcreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions… In Nonlinear FEM, there are repeatedly kspsolve for getting answer vector. Is it correct to do all of the aforementioned processes (KSPcreate, KSPSetOperators ~~~) for each KSPSolve? Or should I declare it only once at the beginning and not call it again?
You just do these once at setup but for nonlinear problems KSPSetOperators tells the solver that you have a new matrix and so "matrix setup" work needs to be done.
2. If the answer to question 1 is that it must be repeated every time, should this work be done right before kspsolve, that is, when the global matrix assembly is finished, or is it irrelevant to performance at any time?
KSPSetOperators should be set after the new matrix values are set but it might work before. It just sets a pointer to the matrix and flags it as not setup.
3. When performing FEM, local matrices are often scattered in global matrices depending on connectivity. In this case, which is better in terms of performance: adding the values one by one with MatSetValue or adding them all at once with MatSetValues even if they are scattered?
You want to add one element matrix at a time, generally.
4. I would like to measure the time of each section of the process. Which method is recommended?
PETSc methods are timed separateluy, but setup gets folded into KSPSolve unless you call SNESSetUp before the SNES[KSP]Solve. You can add you own timers also https://petsc.org/release/docs/manualpages/Profiling/PetscLogEventRegister/
Mark
Thank you for your help.
Hyung Kim
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
In your comments, KSPSetOperators is called if you want to change the system matrix. "change the system matrix" means the components of matrix are changed? I mean the values of some components of matrix are changed. Thanks, Hyung Kim 2022년 11월 30일 (수) 오후 10:00, Matthew Knepley <[email protected]>님이 작성:
On Wed, Nov 30, 2022 at 7:51 AM 김성익 <[email protected]> wrote:
Thank you for your comments. However, I have more questions.
1. Generally, (KSPCreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions ) above functions are should be called after each "MatassemblyEnd??"
KSPCreate is called once.
You do not need PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions more than once, unless you want to change the solver type.
KSPSetOperators is called if you want to change the system matrix.
KSPSolve is called when you want to change the rhs.
Thanks,
Matt
2. Though reading the user guide, I don't fully understand under what circumstances the functions mentioned above should be called again. Can you explain when each function should be called?
Thanks,
Hyung Kim
2022년 11월 30일 (수) 오후 8:37, Mark Adams <[email protected]>님이 작성:
On Wed, Nov 30, 2022 at 5:08 AM 김성익 <[email protected]> wrote:
Hello,
I’m working on FEM using PETSc.
As everyone knows, it is necessary to repeatedly solve Ax=B.
Regarding this, I have 4 questions.
1. There are many steps for preparing KSPSolve. For example KSPcreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions… In Nonlinear FEM, there are repeatedly kspsolve for getting answer vector. Is it correct to do all of the aforementioned processes (KSPcreate, KSPSetOperators ~~~) for each KSPSolve? Or should I declare it only once at the beginning and not call it again?
You just do these once at setup but for nonlinear problems KSPSetOperators tells the solver that you have a new matrix and so "matrix setup" work needs to be done.
2. If the answer to question 1 is that it must be repeated every time, should this work be done right before kspsolve, that is, when the global matrix assembly is finished, or is it irrelevant to performance at any time?
KSPSetOperators should be set after the new matrix values are set but it might work before. It just sets a pointer to the matrix and flags it as not setup.
3. When performing FEM, local matrices are often scattered in global matrices depending on connectivity. In this case, which is better in terms of performance: adding the values one by one with MatSetValue or adding them all at once with MatSetValues even if they are scattered?
You want to add one element matrix at a time, generally.
4. I would like to measure the time of each section of the process. Which method is recommended?
PETSc methods are timed separateluy, but setup gets folded into KSPSolve unless you call SNESSetUp before the SNES[KSP]Solve. You can add you own timers also https://petsc.org/release/docs/manualpages/Profiling/PetscLogEventRegister/
Mark
Thank you for your help.
Hyung Kim
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On Wed, Nov 30, 2022 at 8:25 AM 김성익 <[email protected]> wrote:
In your comments, KSPSetOperators is called if you want to change the system matrix.
"change the system matrix" means the components of matrix are changed? I mean the values of some components of matrix are changed.
If you just change values in the matrix, you do not have to call it again. Thanks, Matt
Thanks, Hyung Kim
2022년 11월 30일 (수) 오후 10:00, Matthew Knepley <[email protected]>님이 작성:
On Wed, Nov 30, 2022 at 7:51 AM 김성익 <[email protected]> wrote:
Thank you for your comments. However, I have more questions.
1. Generally, (KSPCreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions ) above functions are should be called after each "MatassemblyEnd??"
KSPCreate is called once.
You do not need PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions more than once, unless you want to change the solver type.
KSPSetOperators is called if you want to change the system matrix.
KSPSolve is called when you want to change the rhs.
Thanks,
Matt
2. Though reading the user guide, I don't fully understand under what circumstances the functions mentioned above should be called again. Can you explain when each function should be called?
Thanks,
Hyung Kim
2022년 11월 30일 (수) 오후 8:37, Mark Adams <[email protected]>님이 작성:
On Wed, Nov 30, 2022 at 5:08 AM 김성익 <[email protected]> wrote:
Hello,
I’m working on FEM using PETSc.
As everyone knows, it is necessary to repeatedly solve Ax=B.
Regarding this, I have 4 questions.
1. There are many steps for preparing KSPSolve. For example KSPcreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions… In Nonlinear FEM, there are repeatedly kspsolve for getting answer vector. Is it correct to do all of the aforementioned processes (KSPcreate, KSPSetOperators ~~~) for each KSPSolve? Or should I declare it only once at the beginning and not call it again?
You just do these once at setup but for nonlinear problems KSPSetOperators tells the solver that you have a new matrix and so "matrix setup" work needs to be done.
2. If the answer to question 1 is that it must be repeated every time, should this work be done right before kspsolve, that is, when the global matrix assembly is finished, or is it irrelevant to performance at any time?
KSPSetOperators should be set after the new matrix values are set but it might work before. It just sets a pointer to the matrix and flags it as not setup.
3. When performing FEM, local matrices are often scattered in global matrices depending on connectivity. In this case, which is better in terms of performance: adding the values one by one with MatSetValue or adding them all at once with MatSetValues even if they are scattered?
You want to add one element matrix at a time, generally.
4. I would like to measure the time of each section of the process. Which method is recommended?
PETSc methods are timed separateluy, but setup gets folded into KSPSolve unless you call SNESSetUp before the SNES[KSP]Solve. You can add you own timers also https://petsc.org/release/docs/manualpages/Profiling/PetscLogEventRegister/
Mark
Thank you for your help.
Hyung Kim
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On Wed, Nov 30, 2022 at 8:31 AM Matthew Knepley <[email protected]> wrote:
On Wed, Nov 30, 2022 at 8:25 AM 김성익 <[email protected]> wrote:
In your comments, KSPSetOperators is called if you want to change the system matrix.
"change the system matrix" means the components of matrix are changed? I mean the values of some components of matrix are changed.
If you just change values in the matrix, you do not have to call it again.
You do need to call KSPSetOperators before the solve if you want the KSP to redo the (PC) setup. In GAMG this is substantial. If the matrix does not change much you can try not doing this and experiment.
Thanks,
Matt
Thanks, Hyung Kim
2022년 11월 30일 (수) 오후 10:00, Matthew Knepley <[email protected]>님이 작성:
On Wed, Nov 30, 2022 at 7:51 AM 김성익 <[email protected]> wrote:
Thank you for your comments. However, I have more questions.
1. Generally, (KSPCreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions ) above functions are should be called after each "MatassemblyEnd??"
KSPCreate is called once.
You do not need PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions more than once, unless you want to change the solver type.
KSPSetOperators is called if you want to change the system matrix.
KSPSolve is called when you want to change the rhs.
Thanks,
Matt
2. Though reading the user guide, I don't fully understand under what circumstances the functions mentioned above should be called again. Can you explain when each function should be called?
Thanks,
Hyung Kim
2022년 11월 30일 (수) 오후 8:37, Mark Adams <[email protected]>님이 작성:
On Wed, Nov 30, 2022 at 5:08 AM 김성익 <[email protected]> wrote:
Hello,
I’m working on FEM using PETSc.
As everyone knows, it is necessary to repeatedly solve Ax=B.
Regarding this, I have 4 questions.
1. There are many steps for preparing KSPSolve. For example KSPcreate, KSPSetOperators, KSPGetPC, PCSetType, PCFactorSetMatSolverType, KSPSetFromOptions… In Nonlinear FEM, there are repeatedly kspsolve for getting answer vector. Is it correct to do all of the aforementioned processes (KSPcreate, KSPSetOperators ~~~) for each KSPSolve? Or should I declare it only once at the beginning and not call it again?
You just do these once at setup but for nonlinear problems KSPSetOperators tells the solver that you have a new matrix and so "matrix setup" work needs to be done.
2. If the answer to question 1 is that it must be repeated every time, should this work be done right before kspsolve, that is, when the global matrix assembly is finished, or is it irrelevant to performance at any time?
KSPSetOperators should be set after the new matrix values are set but it might work before. It just sets a pointer to the matrix and flags it as not setup.
3. When performing FEM, local matrices are often scattered in global matrices depending on connectivity. In this case, which is better in terms of performance: adding the values one by one with MatSetValue or adding them all at once with MatSetValues even if they are scattered?
You want to add one element matrix at a time, generally.
4. I would like to measure the time of each section of the process. Which method is recommended?
PETSc methods are timed separateluy, but setup gets folded into KSPSolve unless you call SNESSetUp before the SNES[KSP]Solve. You can add you own timers also https://petsc.org/release/docs/manualpages/Profiling/PetscLogEventRegister/
Mark
Thank you for your help.
Hyung Kim
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
participants (3)
-
Mark Adams -
Matthew Knepley -
김성익