Manav,

The conversion MATAIJ -> MATIS is suboptimal if you can generate the matrices (e.g. with libMesh).

For best performance of these methods, you really need to assemble the subdomain Neumann problems.
This has been done for other libraries (e.g. FEniCS, MFEM, FreeFem++, I also did it for deal.II at some point), and it can be done from libMesh too. I’m not aware of the low level details of the library, but I can tell you how the MATIS object should be properly setup.

This is the sequence of calls (they don’t need to be in the same function call, but the order should be respected)

1) MatCreate
2) MatSetSizes
3) MatSetType
4) MatSetLocalToGlobalMapping
5) MatISSetPreallocation

Then, either MatSetValues or MatSetValuesLocal can be called and the Neumann problems will be automatically assembled.
Most of the matrix operations are supported (e.g. MatMult, MatZeroRows, etc)

The crucial part is 4); all distributed memory FEM  libraries have this information hidden somewhere, as it is needed to properly assemble the matrix anyway in a standard local loop on the elements owned by the MPI Process.
It is a list of indices in the global matrix that are owned by the local MPI process (e.g. corresponding to the dofs of the elements that are in the interior of the local part of the mesh) or shared (e.g. does of the elements lying on the boundary of the local mesh).
Probably, libMesh experts are reading this message and they can tell you where to find the information you need.

Some notes:

- 2) and 3) can be swapped
- 5) MatXAIJSetPreallocation can be called instead

Let me know if you need further clarifications
Stefano

On Dec 28, 2018, at 9:05 PM, Manav Bhatia via petsc-users <petsc-users@mcs.anl.gov> wrote:

Hi, 

   I am currently accessing PETSc through libMesh where the matrices of type MPIAIJ are being initialized using the MatSetSizes, MatSetFromOptions, and MatMPIAISetPreallocation calls. 

   I am interested in accessing the BDDC solver and have looked through the PDF manual that talks about BDDC setup through MATIS. However, it is not clear what would be the most efficient route to accessing BDDC for my unstructured FE code. 

   Example 72 (https://www.mcs.anl.gov/petsc/petsc-3.10/src/ksp/ksp/examples/tutorials/ex72.c.html) seems to be calling MatConvert to make the conversion to MATIS. However, the documentation of MATIS says that MatSetLocalToGlobalMapping should be called before initializing MATIS, which is not done in Example 72. I am also not sure if I need to setup any DM/DA constructs to aid in this process. 

   I would appreciate if someone could provide an outline of steps required to make the conversion of MatMPIAIJ in order to use BDDC/FETI preconditioners. 

Thanks,
Manav