Matt and Lisandro, Let me explain a little more about new API for mat-mat operations, which include MatMaMult (AB), MatPtAP (PtAP), MatMatMatMult (ABC), with a total 6 matproduct types. 1) MatProduct is logically similar to MatFactor, not a single Mat. Calling MatProductCreate(A,B,NULL,&D); MatProductSetType(D, MATPRODUCT_AB); MatDestory(&A); // A is gone MatDestory(&B); // B is gone MatProductSymbolic(D); is equivalent to calling MatMatMult(null, null, ..., &D) or MatGetFactor(null, ...,&F). If you do this, you'll get segfault (I'll add additional error message for this use case). 2) New API is an incremental improvement over the old API, e.g., MatMatMultSymbolic(A,B, fill,&C); MatMatMultNumeric(A,B,C) are replaced by MatProductCreate(A,B,NULL,&D); MatProductSetType(D, MATPRODUCT_AB); MatProductSetFill(D,fill); MatProductSetFromOptions(D); MatProductSymbolic(D): MatProductNumeric(D); There are at least two reasons we decided to introduce new API. a) there are multiple implementations (algorithms) for some product types and matrix types, e.g., A*B and PtAP for mpiaij_mpiaij matrices. Developers are adding more and more special implementations, e.g, for gpu etc. Previously, the options of algorithms are selected in MatMatMultSymbolic_xxx(...,&D) with random option names. However, some options need to be selected before D is setup, and attached to D. For this reason, we introduced new API, and add MatProductSetFromOptions(D) to handle all the options. Note: the options can only be determined at the level of producttype and mattype are known, thus the options are enabled in MatProductSetFromOptions_mattype_producttype(). b) Reuse of MatProduct D = A*B means same A and B with updated values. However, some users change A or B, some use different D without knowing original D might contain internal data structure that is associated with A and B. The new API forces reuse same A, B, unless user calls MatProductReplaceMats(A,B,C,D). 3) For MatProduct, I consider MatProductSetFromOptions() is logically equivalent to MatSetFromOptions(), MatProductSymbolic() is logically equivalent to MatSetUp(), and MatProductNumeric() assembles the matproduct D repeatedly when A or B are updated. I'm willing to incorporate your thoughts and comments into this new API. Hong On Mon, Mar 23, 2020 at 4:29 PM Matthew Knepley <[email protected]> wrote:

On Mon, Mar 23, 2020 at 5:12 PM Lisandro Dalcin <[email protected]> wrote:

...

On Mon, 23 Mar 2020 at 17:57, [email protected] <[email protected]> wrote:

...

Lisandro:

...

MatProductSetFromOptions() is not really setting any options. It looks like the correct name for this API is MatProductSetUp(). In any case, it looks like both SetFromOptions() and SetUp() should be provided.

Not all product and matrix types enable algorithmic options. For those who do have options, MatProductSetFromOptions_mattype_producttype() enables it. See MatProductSetFromOptions_MPIAIJ_AB().

I have to insist, you should separate SetFromOptions() from SetUp(). For many types, SetFromOptions may be empty (you can even use a NULL pointer to handle these cases), but the rest of the code that does setup stuff (that is, preparing the data structures) should go SetUp() routines. This design pattern is used everywhere in PETSc. That's the proper way to do it, IMHO. This way is the one that gives maximum flexibility to power users/developers, because some times developers may want to configure objects with some parameters, and do a SetUp() without calling SetFromOptions() to prevent users to mess up with the parameters the developer carefully chose.

I agree here. We should not be using options in SetUp or setting up in SetFromOptions() since it might not be called. I am tying to fix all this in DM.

Matt

...

-- Lisandro Dalcin ============ Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/

-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>