Your FormFunction is not doing any communication of x values, thus the function evaluation can only use local values (same with the Jacobian). So these functions cannot be correct. If you look at the DMDA version you will see DMGlobalToLocal() that manages the communication inside FormFunction. If you do not want to use DMDA then you need to manage the needed communication yourself. This means figuring out what needs to be communicated (which depends on the type of grid and discretization you want to use) and setting up the communication; this can be done with VecScatterCreate(). Barry
On Apr 8, 2021, at 8:08 AM, Suwun Suwunnarat <[email protected]> wrote:
To whom it may concern,
I would like to modify the code in SNES's ex2.c to run in parallel.
https://www.mcs.anl.gov/petsc/petsc-3.6.4/src/snes/examples/tutorials/ex2.c.... <https://www.mcs.anl.gov/petsc/petsc-3.6.4/src/snes/examples/tutorials/ex2.c.html>
I modified my ex2.c, as attached here. Even though ex3.c is the parallel version, I would like my implementation to "not contain" DMDA.
(Despite version 3.6.4, it works for version 3.15.0, which I use in my debugging purpose.)
The "crucial" parts of ex2.c that I modified are (apart from deletion of comments, exact solution verification, etc):
- (line 63-64) matrix preallocation - use MatGetOwnershipRange on a PETSc matrix jac (in FormJacobian) - use VecGetOwnershipRange on every PETSc vector in the program
I try to run the program with mpiexec. For n = 1 it works fine for -pc_type lu -snes_type newtonls. However, when I increase n to be n = 2, the code does not iterate further. (It does not go into the FormJacobian for after the first iteration.) I try several different pc_type and snes_type, but still do not get the result as obtained with n = 1.
I attached the terminal printout also.
Since the manual (3.15.0) said that (p81, section 2.4, 1st paragraph)
"Also, the SNES interface is identical for the uniprocess and parallel cases; the only difference in the parallel version is that each process typically forms only its local contribution to various matrices and vectors"
Therefore, I think that by modified only the vectors and matrices part, I should be able to get parallelism.
Regards, Suwun
PS1: I compile using mpicc and use mpiexec found in the directory ${PETSC_DIR}/arch-mumps-opt/bin/ to run the code. PS2: I attached the install.sh (I install with MUMPS) that I use to install PETSc. Note that I also turn on --with-debuggin=yes PS3: my OS is Ubuntu 20.04.2. Intel CPU. My environment should have everything necessary (gfortran, gcc, g++, valgrind, cmake). <terminal_printout.html><ex2.c><install.sh>