On Tue, Feb 21, 2012 at 3:36 AM, Dave May <dave.mayhem23@gmail.com> wrote:
Max,

>
> The test case that I am working with is isoviscous convection, benchmark
> case 1a from Blankenbach 1989.
>

Okay, I know this problem.
An iso viscous problem, solved on a uniform grid using  dx=dy=dz discretised
via FV should be super easy to precondition.

>
>
> I think that this is the problem. The (2,2) slot in the LHS matrix is all
> zero (pressure does not appear in the continuity equation), so I think that
> the preconditioner is meaningless. I am still confused as to why this choice
> of preconditioner was suggested in the tutorial, and what is a better choice
> of preconditioner for this block? Should I be using one of the Schur
> complement methods instead of the additive or multiplicative field split?
>

No, you need to define an appropriate stokes preconditioner
You should assemble this matrix
 B = ( K,B ; B^T, -1/eta* I )
as the preconditioner for stokes.
Here eta* is a measure of the local viscosity within each pressure
control volume.
Unless you specify to use the real diagonal

Pass this into the third argument in KSPSetOperators() (i.e. the Pmat variable)
http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSetOperators.html

Not sure how you represent A and B, but if you really want to run just
additive with fieldsplit, you don't need the off diagonal blocks, so
 B = ( K,0 ; 0, -1/eta* I )
would yield the same result. Depending on your matrix representation,
this may save you some memory.

PCFieldsplit will use the B(1,1) and B(2,2) to build the stokes
preconditioner unless you ask for it to use the real diagonal - but
for the stokes operator A, this makes no sense.

This is the right thing to do (as Matt states).
Try it out, and let us know how it goes.


Cheers,
 Dave

Dave and Matt,
Thanks for your help. I had some time to work on this a little more. I now have a stokes operator A that looks like this:
A=(K B; B^T 0) and a matrix from which the preconditioner is generated P=(K B; B^T -1/eta*I)

I verified that I can solve this system using the default ksp and pc settings in 77 iterations for the first timestep (initial guess zero) and in 31 iterations for the second timestep (nonzero initial guess).

I adopted your suggestion to use the multiplicative field split as a starting point. My reading of the PETSc manual suggests to me that the preconditioner formed should then look like:

B = (ksp(K,K) 0;-B^T*ksp(K,K)*ksp(0,-1/eta*I)    ksp(0,-1/eta*I))

My interpretation of the output suggests that the solvers within each fieldsplit are converging nicely, but the global residual is not decreasing after the first few iterations. Given the disparity in residual sizes, I think that there might be a problem with the scaling of the pressure variable (I scaled the continuity equation by eta/dx where dx is my grid spacing). I also scaled the (1,1) block in the preconditioner by this scale factor. Thanks again for all of your help.

Max


Options used:
    -stokes_pc_fieldsplit_0_fields 0,1 -stokes_pc_fieldsplit_1_fields 2 \
    -stokes_pc_type fieldsplit -stokes_pc_fieldsplit_type multiplicative \
    -stokes_fieldsplit_0_pc_type ml \
    -stokes_fieldsplit_0_ksp_type gmres \
    -stokes_fieldsplit_0_ksp_monitor_true_residual \
    -stokes_fieldsplit_0_ksp_norm_type UNPRECONDITIONED \
    -stokes_fieldsplit_0_ksp_max_it 3 \
    -stokes_fieldsplit_0_ksp_type gmres \
    -stokes_fieldsplit_0_ksp_rtol 1.0e-4 \
    -stokes_fieldsplit_0_mg_levels_ksp_type gmres \
    -stokes_fieldsplit_0_mg_levels_pc_type bjacobi \
    -stokes_fieldsplit_0_mg_levels_ksp_max_it 4 \
    -stokes_fieldsplit_1_pc_type jacobi \
    -stokes_fieldsplit_1_ksp_type preonly \
    -stokes_fieldsplit_1_ksp_max_it 3 \
    -stokes_fieldsplit_1_ksp_monitor_true_residual \
    -stokes_ksp_type gcr \
    -stokes_ksp_monitor_blocks \
    -stokes_ksp_monitor_draw \
    -stokes_ksp_view \
    -stokes_ksp_atol 1e-6 \
    -stokes_ksp_rtol 1e-6 \
    -stokes_ksp_max_it 100 \
    -stokes_ksp_norm_type UNPRECONDITIONED \
    -stokes_ksp_monitor_true_residual \

Output:

  0 KSP Component U,V,P residual norm [ 0.000000000000e+00, 1.165111661413e+06, 0.000000000000e+00 ]
  Residual norms for stokes_ solve.
  0 KSP unpreconditioned resid norm 1.165111661413e+06 true resid norm 1.165111661413e+06 ||r(i)||/||b|| 1.000000000000e+00
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.165111661413e+06 true resid norm 1.165111661413e+06 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 3.173622513625e+05 true resid norm 3.173622513625e+05 ||r(i)||/||b|| 2.723878421898e-01
    2 KSP unpreconditioned resid norm 5.634119635158e+04 true resid norm 1.725996376799e+05 ||r(i)||/||b|| 1.481399967026e-01
    3 KSP unpreconditioned resid norm 1.218418968344e+03 true resid norm 1.559727441168e+05 ||r(i)||/||b|| 1.338693528546e-01
  1 KSP Component U,V,P residual norm [ 5.763380362961e+04, 1.154490085631e+05, 3.370358145704e-12 ]
  1 KSP unpreconditioned resid norm 1.290353784783e+05 true resid norm 1.290353784783e+05 ||r(i)||/||b|| 1.107493665644e-01
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.290353784783e+05 true resid norm 1.290353784783e+05 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 1.655137188235e+04 true resid norm 1.655137188235e+04 ||r(i)||/||b|| 1.282700301076e-01
    2 KSP unpreconditioned resid norm 1.195941831181e+03 true resid norm 4.554417355181e+03 ||r(i)||/||b|| 3.529588093508e-02
    3 KSP unpreconditioned resid norm 8.479547025398e+01 true resid norm 3.817072778396e+03 ||r(i)||/||b|| 2.958159865466e-02
  2 KSP Component U,V,P residual norm [ 2.026983725663e+03, 2.531521226429e+03, 3.419060873106e-12 ]
  2 KSP unpreconditioned resid norm 3.243032954498e+03 true resid norm 3.243032954498e+03 ||r(i)||/||b|| 2.783452489493e-03
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 3.243032954498e+03 true resid norm 3.243032954498e+03 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 1.170090628031e+02 true resid norm 1.170090628031e+02 ||r(i)||/||b|| 3.608013376517e-02
    2 KSP unpreconditioned resid norm 9.782830529900e+00 true resid norm 1.741722174777e+01 ||r(i)||/||b|| 5.370658267168e-03
    3 KSP unpreconditioned resid norm 6.886950142735e-01 true resid norm 1.636749336722e+01 ||r(i)||/||b|| 5.046971028932e-03
  3 KSP Component U,V,P residual norm [ 7.515013854917e+01, 7.515663601801e+01, 3.418919176066e-12 ]
  3 KSP unpreconditioned resid norm 1.062829396540e+02 true resid norm 1.062829396540e+02 ||r(i)||/||b|| 9.122124786317e-05
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.062829396540e+02 true resid norm 1.062829396540e+02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.373578062042e+01 true resid norm 5.373578062042e+01 ||r(i)||/||b|| 5.055917797846e-01
    2 KSP unpreconditioned resid norm 1.199305097134e+00 true resid norm 3.492111756827e+01 ||r(i)||/||b|| 3.285674792393e-01
    3 KSP unpreconditioned resid norm 9.508597255523e-02 true resid norm 3.452079362567e+01 ||r(i)||/||b|| 3.248008922038e-01
  4 KSP Component U,V,P residual norm [ 7.495897679790e+01, 7.527868410560e+01, 3.418919160091e-12 ]
  4 KSP unpreconditioned resid norm 1.062343093509e+02 true resid norm 1.062343093509e+02 ||r(i)||/||b|| 9.117950911420e-05
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.062343093509e+02 true resid norm 1.062343093509e+02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.419252803207e+01 true resid norm 5.419252803207e+01 ||r(i)||/||b|| 5.101226558840e-01
    2 KSP unpreconditioned resid norm 1.431134174522e+00 true resid norm 3.339055236737e+01 ||r(i)||/||b|| 3.143104386088e-01
    3 KSP unpreconditioned resid norm 9.760479467902e-02 true resid norm 3.304522520358e+01 ||r(i)||/||b|| 3.110598205561e-01
  5 KSP Component U,V,P residual norm [ 7.491128585963e+01, 7.523275560552e+01, 3.418919008441e-12 ]
  5 KSP unpreconditioned resid norm 1.061681132221e+02 true resid norm 1.061681132221e+02 ||r(i)||/||b|| 9.112269384837e-05
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.061681132221e+02 true resid norm 1.061681132221e+02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.343215079492e+01 true resid norm 5.343215079492e+01 ||r(i)||/||b|| 5.032787074508e-01
    2 KSP unpreconditioned resid norm 1.288069736759e+00 true resid norm 3.308925591301e+01 ||r(i)||/||b|| 3.116684935691e-01
    3 KSP unpreconditioned resid norm 9.505248953960e-02 true resid norm 3.281875055845e+01 ||r(i)||/||b|| 3.091205971589e-01
  6 KSP Component U,V,P residual norm [ 7.481188568118e+01, 7.527346267608e+01, 3.418918860626e-12 ]
  6 KSP unpreconditioned resid norm 1.061268694649e+02 true resid norm 1.061268694649e+02 ||r(i)||/||b|| 9.108729487455e-05
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.061268694649e+02 true resid norm 1.061268694649e+02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.300383444945e+01 true resid norm 5.300383444945e+01 ||r(i)||/||b|| 4.994384053416e-01
    2 KSP unpreconditioned resid norm 1.118785004087e+00 true resid norm 3.282090953364e+01 ||r(i)||/||b|| 3.092610730828e-01
    3 KSP unpreconditioned resid norm 9.758015489979e-02 true resid norm 3.259718081014e+01 ||r(i)||/||b|| 3.071529479244e-01
  7 KSP Component U,V,P residual norm [ 7.475024970669e+01, 7.530858268154e+01, 3.418918784089e-12 ]
  7 KSP unpreconditioned resid norm 1.061083524362e+02 true resid norm 1.061083524362e+02 ||r(i)||/||b|| 9.107140195255e-05
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.061083524362e+02 true resid norm 1.061083524362e+02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.296981668051e+01 true resid norm 5.296981668051e+01 ||r(i)||/||b|| 4.992049679820e-01
    2 KSP unpreconditioned resid norm 9.379451887610e-01 true resid norm 3.378466967056e+01 ||r(i)||/||b|| 3.183978348066e-01
    3 KSP unpreconditioned resid norm 9.102580142867e-02 true resid norm 3.360853440947e+01 ||r(i)||/||b|| 3.167378781957e-01
  8 KSP Component U,V,P residual norm [ 7.464535615814e+01, 7.537007679541e+01, 3.418918790515e-12 ]
  8 KSP unpreconditioned resid norm 1.060781677449e+02 true resid norm 1.060781677449e+02 ||r(i)||/||b|| 9.104549482946e-05
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.060781677449e+02 true resid norm 1.060781677449e+02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.281972737642e+01 true resid norm 5.281972737642e+01 ||r(i)||/||b|| 4.979321240109e-01
    2 KSP unpreconditioned resid norm 9.224594814880e-01 true resid norm 3.351285171891e+01 ||r(i)||/||b|| 3.159260046751e-01
    3 KSP unpreconditioned resid norm 9.143100662935e-02 true resid norm 3.329269756083e+01 ||r(i)||/||b|| 3.138506091177e-01
  9 KSP Component U,V,P residual norm [ 7.451688471900e+01, 7.544516987344e+01, 3.418918860847e-12 ]
  9 KSP unpreconditioned resid norm 1.060412172952e+02 true resid norm 1.060412172952e+02 ||r(i)||/||b|| 9.101378074496e-05
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.060412172952e+02 true resid norm 1.060412172952e+02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.275132249899e+01 true resid norm 5.275132249899e+01 ||r(i)||/||b|| 4.974605520805e-01
    2 KSP unpreconditioned resid norm 7.755381284769e-01 true resid norm 3.453933285011e+01 ||r(i)||/||b|| 3.257161105001e-01
    3 KSP unpreconditioned resid norm 7.298768665179e-02 true resid norm 3.435179316160e+01 ||r(i)||/||b|| 3.239475558447e-01
  10 KSP Component U,V,P residual norm [ 7.451431102619e+01, 7.544762349626e+01, 3.418918857322e-12 ]
 10 KSP unpreconditioned resid norm 1.060411544587e+02 true resid norm 1.060411544587e+02 ||r(i)||/||b|| 9.101372681321e-05
    Residual norms for stokes_fieldsplit_0_ solve.
    0 KSP unpreconditioned resid norm 1.060411544587e+02 true resid norm 1.060411544587e+02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.276103518337e+01 true resid norm 5.276103518337e+01 ||r(i)||/||b|| 4.975524403961e-01
    2 KSP unpreconditioned resid norm 7.777079890360e-01 true resid norm 3.454373663425e+01 ||r(i)||/||b|| 3.257578325186e-01
    3 KSP unpreconditioned resid norm 7.356028471071e-02 true resid norm 3.435584054266e+01 ||r(i)||/||b|| 3.239859158269e-01
  11 KSP Component U,V,P residual norm [ 7.438335197779e+01, 7.553731959735e+01, 3.418918856471e-12 ]