On Thu, Feb 21, 2013 at 1:10 PM, Matthew Knepley <knepley@gmail.com> wrote:

Yes, he needs the local to global scatters. Rich, I think the easiest way to do this now may be
to define your data layout as a PetscSection (these are all local) using DMSetDefaultSection(),

and record who owns which point using a PetscSF with DMSetPointSF().

Using this information, the DM will automatically construct a scatter between local and global,
and the CreateLocal/GlobalVector() functions will also work right. This is what its is designed for.

Does this make sense?

Rich, this will work fine if you don't mind having PETSc do the communication. It will be done using PetscSFBcastBegin/End, which should have similar performance to VecScatterBegin/End. (I should do a performance test of that.) If you want to use your own internal communication routines, then we need to add the DMShell functions to get your functions called.