On Thu, Apr 1, 2021 at 6:24 PM Junchao Zhang <junchao.zhang@gmail.com> wrote:
Matt,
It is easy for me to know it is not an SF bug, since the code passes a null pointer (as leafdata) to an SF with 9 roots and leaves. But I could not figure out what the root reason is. 

Junchao, thanks so much for looking at this. With what you show below, I think I can figure it out now.

  Thanks,

     Matt
 
Here is my finding.
In DMSetUp(forest) >> DMSetUp_p4est >> DMPforestGetPlex >> DMConvert_pforest_plex >> .. >> DMShareDiscretization,  we have
     dmB->sectionSF = dmA->sectionSF;
Which makes the two DMs,  forest, and  ((DM_Forest_pforest*) ((DM_Forest*) forest->data)->data)->plex, share the same sectionSF. But in the test, there are two places to populate the sectionSF based on different DMs.

1) In DMGlobalToLocal(forest, g, INSERT_VALUES, l), the SF is built based on forest and its local/globalSection.
2) In VecView(g, viewer), which does
             ierr = VecGetDM(vec,&dm);CHKERRQ(ierr);
  ierr = DMPforestGetPlex(dm,&plex);CHKERRQ(ierr);
  ierr = VecSetDM(vec,plex);CHKERRQ(ierr);
  ierr = VecView_Plex(vec,viewer);CHKERRQ(ierr);
  ierr = VecSetDM(vec,dm);CHKERRQ(ierr);
VecView_Plex() calls VecView_Plex_HDF5_Internal(), which builds the SF based on plex's local/globalSection. 

Depending on which function is called first, we get different SFs. The crashed one did 1) first and then 2). The 'good'  one did 2) and then 1).  But  even the good one is wrong, since it gives an empty SF (thus not crashing the code).

--Junchao Zhang


On Tue, Mar 30, 2021 at 5:44 AM Matthew Knepley <knepley@gmail.com> wrote:
On Mon, Mar 29, 2021 at 11:05 PM Junchao Zhang <junchao.zhang@gmail.com> wrote:
Matt,
  I can reproduce the error. Let me see what is wrong.

Thanks! It might be a bug in Plex or Forest as well, but it is hard for me to tell.

   Matt
 
  Thanks.
--Junchao Zhang


On Mon, Mar 29, 2021 at 2:16 PM Matthew Knepley <knepley@gmail.com> wrote:
Junchao,

I have an SF problem, which I think is a caching bug, but it is hard to see what is happening in the internals. I have made a small example which should help you see what is wrong. It is attached.

If you run without arguments, you get

master *:~/Downloads/tmp/Salac$ ./forestHDF
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Null argument, when expecting valid pointer
[0]PETSC ERROR: Trying to copy to a null pointer
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.14.5-879-g03cacdc99d  GIT Date: 2021-03-22 01:02:08 +0000
[0]PETSC ERROR: ./forestHDF on a arch-master-debug named MacBook-Pro.fios-router.home by knepley Mon Mar 29 15:14:16 2021
[0]PETSC ERROR: Configure options --PETSC_ARCH=arch-master-debug --download-bamg --download-chaco --download-ctetgen --download-egads --download-eigen --download-exodusii --download-fftw --download-hpddm --download-libpng --download-metis --download-ml --download-mumps --download-netcdf --download-opencascade --download-p4est --download-parmetis --download-pnetcdf --download-scalapack --download-slepc --download-suitesparse --download-superlu_dist --download-triangle --with-cmake-exec=/PETSc3/petsc/apple/bin/cmake --with-ctest-exec=/PETSc3/petsc/apple/bin/ctest --with-hdf5-dir=/PETSc3/petsc/apple --with-mpi-dir=/PETSc3/petsc/apple --with-shared-libraries --with-slepc --with-zlib --download-tetgen
[0]PETSC ERROR: #1 PetscMemcpy() at /PETSc3/petsc/petsc-dev/include/petscsys.h:1798
[0]PETSC ERROR: #2 UnpackAndInsert_PetscReal_1_1() at /PETSc3/petsc/petsc-dev/src/vec/is/sf/impls/basic/sfpack.c:426
[0]PETSC ERROR: #3 ScatterAndInsert_PetscReal_1_1() at /PETSc3/petsc/petsc-dev/src/vec/is/sf/impls/basic/sfpack.c:426
[0]PETSC ERROR: #4 PetscSFLinkScatterLocal() at /PETSc3/petsc/petsc-dev/src/vec/is/sf/impls/basic/sfpack.c:1248
[0]PETSC ERROR: #5 PetscSFBcastBegin_Basic() at /PETSc3/petsc/petsc-dev/src/vec/is/sf/impls/basic/sfbasic.c:193
[0]PETSC ERROR: #6 PetscSFBcastWithMemTypeBegin() at /PETSc3/petsc/petsc-dev/src/vec/is/sf/interface/sf.c:1493
[0]PETSC ERROR: #7 DMGlobalToLocalBegin() at /PETSc3/petsc/petsc-dev/src/dm/interface/dm.c:2565
[0]PETSC ERROR: #8 VecView_Plex_HDF5_Internal() at /PETSc3/petsc/petsc-dev/src/dm/impls/plex/plexhdf5.c:251
[0]PETSC ERROR: #9 VecView_Plex() at /PETSc3/petsc/petsc-dev/src/dm/impls/plex/plex.c:385
[0]PETSC ERROR: #10 VecView_p4est() at /PETSc3/petsc/petsc-dev/src/dm/impls/forest/p4est/pforest.c:4922
[0]PETSC ERROR: #11 VecView() at /PETSc3/petsc/petsc-dev/src/vec/vec/interface/vector.c:613
[0]PETSC ERROR: #12 main() at /Users/knepley/Downloads/tmp/Salac/forestHDF.c:53
[0]PETSC ERROR: PETSc Option Table entries:
[0]PETSC ERROR: -malloc_debug
[0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint@mcs.anl.gov----------
application called MPI_Abort(MPI_COMM_SELF, 53001) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=53001

If you run with

  ./forestHDF -write_early

or

  ./forestHDF -no_g2l


Then it is fine. Thus it appears to me that if you run a G2L at the wrong time, something is incorrectly cached.

  Thanks,

    Matt

--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener



--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener



--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener