October 2024 - petsc-users - mailman3.cels.anl.gov

Re: [petsc-users] Multigrid Preconditioned CG on cell-centered Poisson with DMStag
by Barry Smith 02 Oct '24

02 Oct '24

Use -mg_coarse_pc_type svd Verify that when you apply the restriction to a constant vector (all ones) it returns a constant vector, maybe scaled Verify that when you apply the interpolation to a constant vector it returns a constant vector > On Oct 2, 2024, at 12:29 PM, Gautam Luhana <gkluhana(a)cs.ubc.ca> wrote: > > GMRES is similar i.e. converges but iteration counts go +1-2 with every doubling of the mesh. > > I am attaching the output from a cg and gmres run for a single stokes iteration with a velocity and pressure subdomain solves, subsequent iterations are similar. The pressure subdomain ksp is titled pre_mgcg. > > Adding more information in case it is relevant to isolating the issue: > > -> The coarse grid operators are formed by setting PCMGSetGalerkin to PC_MG_GALERKIN_BOTH. > > -> The interpolation / restriction operators are formed using the DMCreateInterpolation and DMCreateRestriction functions on coarsened DMs (DMStag implementation) and sent to PCMGSetInterpolation/PCMGSetRestriction. > > Thanks for the help! > > -Gautam > > > On 2024-10-02 7:42 a.m., Barry Smith wrote: >> [CAUTION: Non-UBC Email] >> >> Using >> >> PetscCall(MatNullSpaceCreate(PetscObjectComm((PetscObject)dmPressure), PETSC_TRUE, 0, nullptr, &matNullSpace)); >> >> should be fine (the other approach is equivalent). >> >> Any change if you use -ksp_type gmres? >> >> Please send all the output from a sample run with -ksp_monitor_true_residual -ksp_view >> >> >> >> >> >> >> >>> On Oct 1, 2024, at 6:42 PM, Gautam Luhana <gkluhana(a)cs.ubc.ca> wrote: >>> >>> I did think that I accounted for that but now I see that I am getting the same number of iterations in the pressure space with and without the call to MatSetNullspace so maybe I am not doing that correctly and that's where the problem is. >>> >>> I tried it in the following way first: >>> >>> PetscCall(MatNullSpaceCreate(PetscObjectComm((PetscObject)dmPressure), PETSC_TRUE, 0, nullptr, &matNullSpace)); >>> >>> PetscCall(MatSetNullSpace(*A, matNullSpace)); >>> >>> and then also as: >>> >>> PetscCall(DMGetGlobalVector(dmPressure, &constantPressure)); >>> PetscCall(VecSet(constantPressure, 1.0)); >>> PetscCall(VecNorm(constantPressure, NORM_2, &nrm)); >>> PetscCall(VecScale(constantPressure, 1.0 / nrm)); >>> PetscCall(DMCreateGlobalVector(dmPressure, &basis)); >>> PetscCall(VecCopy(constantPressure, basis)); >>> PetscCall(MatNullSpaceCreate(PetscObjectComm((PetscObject)dmPressure), PETSC_FALSE, 1, &basis, &matNullSpace)); >>> PetscCall(MatSetNullSpace(*A, matNullSpace)); >>> >>> Is this the right way? are there other options that need to be set? >>> >>> -Gautam >>> >>> On 2024-10-01 12:47 p.m., Barry Smith wrote: >>>> [CAUTION: Non-UBC Email] >>>> >>>> Are you handling the null space? Like with MatSetNullSpace()? >>>> >>>> Can you try src/ksp/ksp/tutorials/ex32.c it solves the same problem but with a simpler DMDA. I don't see any reason why using DMStag with the cell-centered values should be different. >>>> >>>> Barry >>>> >>>> >>>>> On Oct 1, 2024, at 3:37 PM, Gautam Luhana <gkluhana(a)cs.ubc.ca> wrote: >>>>> >>>>> Hello, >>>>> >>>>> I am solving the stokes equations on the MAC grid using DMStag with an approximate Schur complement based preconditioner. The preconditioner needs a subdomain solve of the Poisson equation on both the velocity as well as the pressure space. I am using multigrid preconditioned CG for both the subdomain solves. While the subdomain solve on the velocity space converges in 4-5 iterations and is independent of mesh-size, the pressure solve is not mesh-size independent. Relevant information: >>>>> >>>>> Pressure space operator has Neumann boundary conditions. >>>>> >>>>> Coarse-grid operators are being generated using the Galerkin approach (A = P*Af*Pt) for both of the subdomains. >>>>> >>>>> For the pressure space, the jacobi and sor smoothers were tried with several omega and damping parameter choices. >>>>> >>>>> Symmetric smoothing (same number of pre- and post-smoothing iterations) >>>>> >>>>> Has anyone tried solving the poisson operator on the cell-centered pressure space with DMStag? Or know what might be going wrong with the mg-cg convergence? >>>>> >>>>> Thanks! >>>>> >>>>> -Gautam >>>>>

1 0

Re: [petsc-users] Multigrid Preconditioned CG on cell-centered Poisson with DMStag
by Barry Smith 02 Oct '24

02 Oct '24

Using PetscCall(MatNullSpaceCreate(PetscObjectComm((PetscObject)dmPressure), PETSC_TRUE, 0, nullptr, &matNullSpace)); should be fine (the other approach is equivalent). Any change if you use -ksp_type gmres? Please send all the output from a sample run with -ksp_monitor_true_residual -ksp_view > On Oct 1, 2024, at 6:42 PM, Gautam Luhana <gkluhana(a)cs.ubc.ca> wrote: > > I did think that I accounted for that but now I see that I am getting the same number of iterations in the pressure space with and without the call to MatSetNullspace so maybe I am not doing that correctly and that's where the problem is. > > I tried it in the following way first: > > PetscCall(MatNullSpaceCreate(PetscObjectComm((PetscObject)dmPressure), PETSC_TRUE, 0, nullptr, &matNullSpace)); > > PetscCall(MatSetNullSpace(*A, matNullSpace)); > > and then also as: > > PetscCall(DMGetGlobalVector(dmPressure, &constantPressure)); > PetscCall(VecSet(constantPressure, 1.0)); > PetscCall(VecNorm(constantPressure, NORM_2, &nrm)); > PetscCall(VecScale(constantPressure, 1.0 / nrm)); > PetscCall(DMCreateGlobalVector(dmPressure, &basis)); > PetscCall(VecCopy(constantPressure, basis)); > PetscCall(MatNullSpaceCreate(PetscObjectComm((PetscObject)dmPressure), PETSC_FALSE, 1, &basis, &matNullSpace)); > PetscCall(MatSetNullSpace(*A, matNullSpace)); > > Is this the right way? are there other options that need to be set? > > -Gautam > > On 2024-10-01 12:47 p.m., Barry Smith wrote: >> [CAUTION: Non-UBC Email] >> >> Are you handling the null space? Like with MatSetNullSpace()? >> >> Can you try src/ksp/ksp/tutorials/ex32.c it solves the same problem but with a simpler DMDA. I don't see any reason why using DMStag with the cell-centered values should be different. >> >> Barry >> >> >>> On Oct 1, 2024, at 3:37 PM, Gautam Luhana <gkluhana(a)cs.ubc.ca> wrote: >>> >>> Hello, >>> >>> I am solving the stokes equations on the MAC grid using DMStag with an approximate Schur complement based preconditioner. The preconditioner needs a subdomain solve of the Poisson equation on both the velocity as well as the pressure space. I am using multigrid preconditioned CG for both the subdomain solves. While the subdomain solve on the velocity space converges in 4-5 iterations and is independent of mesh-size, the pressure solve is not mesh-size independent. Relevant information: >>> >>> Pressure space operator has Neumann boundary conditions. >>> >>> Coarse-grid operators are being generated using the Galerkin approach (A = P*Af*Pt) for both of the subdomains. >>> >>> For the pressure space, the jacobi and sor smoothers were tried with several omega and damping parameter choices. >>> >>> Symmetric smoothing (same number of pre- and post-smoothing iterations) >>> >>> Has anyone tried solving the poisson operator on the cell-centered pressure space with DMStag? Or know what might be going wrong with the mg-cg convergence? >>> >>> Thanks! >>> >>> -Gautam >>> >>>

1 0

ISView() in PETSc 3.22
by Adrian Croucher 01 Oct '24

01 Oct '24

hi, I am testing my (Fortran) code on PETSc 3.22 and have got it to build. However I am getting some unusual new behaviour when I write an IS to an HDF5 file using ISView(). The attached minimal example shows the issue. It creates a simple 10-element IS and writes it to HDF5. With previous versions of PETSc this would give me a 10x1 dataset containing the values 0 - 9, as expected. When I run it with PETSc 3.22 (in serial), I again get the expected values written on stdout, so it looks like the IS itself is correct. But in the HDF5 file I get a 1x3 dataset containing the values (10,1,0). Has something changed here? - Adrian -- Dr Adrian Croucher Senior Research Fellow Department of Engineering Science Waipapa Taumata Rau / University of Auckland, New Zealand email: a.croucher(a)auckland.ac.nz tel: +64 (0)9 923 4611

2 4

Multigrid Preconditioned CG on cell-centered Poisson with DMStag
by Gautam Luhana 01 Oct '24

01 Oct '24

Hello, I am solving the stokes equations on the MAC grid using DMStag with an approximate Schur complement based preconditioner. The preconditioner needs a subdomain solve of the Poisson equation on both the velocity as well as the pressure space. I am using multigrid preconditioned CG for both the subdomain solves. While the subdomain solve on the velocity space converges in 4-5 iterations and is independent of mesh-size, the pressure solve is not mesh-size independent. Relevant information: Pressure space operator has Neumann boundary conditions. Coarse-grid operators are being generated using the Galerkin approach (A = P*Af*Pt) for both of the subdomains. For the pressure space, the jacobi and sor smoothers were tried with several omega and damping parameter choices. Symmetric smoothing (same number of pre- and post-smoothing iterations) Has anyone tried solving the poisson operator on the cell-centered pressure space with DMStag? Or know what might be going wrong with the mg-cg convergence? Thanks! -Gautam

2 1