Yes, the two files are just petsc headers. --Junchao Zhang On Fri, Aug 13, 2021 at 5:17 PM Mark Adams <[email protected]> wrote:

I seem to be getting Kokkos includes in my install but there is no kokkos in the configure and I started with a clean PETSc arch directory and install directory. Does this make sense?

18:12 main= /gpfs/alpine/csc314/scratch/adams/petsc$ ll /gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray/include/ | grep -i kokko -rw-rw-r-- 1 adams adams 7074 May 27 06:57 petscdmda_kokkos.hpp -rw-rw-r-- 1 adams adams 1636 May 27 06:57 petscvec_kokkos.hpp 18:12 main= /gpfs/alpine/csc314/scratch/adams/petsc$

On Fri, Aug 13, 2021 at 2:27 PM Mark Adams <[email protected]> wrote:

...

On Fri, Aug 13, 2021 at 2:01 PM Matthew Knepley <[email protected]> wrote:

...

On Fri, Aug 13, 2021 at 1:49 PM Mark Adams <[email protected]> wrote:

...

I was building on Spock a few weeks ago, but am getting these errors now. I have a setup with this environment and get this error.

Any ideas?

Here is the error:

Executing: cc -o /tmp/petsc-6izfgpkr/config.setCompilers/conftest -L/opt/rocm-4.2.0/lib -lhsa-runtime64 -L${ROCM_PATH}/lib -lamdhip64 -lhsa-runtime64 /tmp/petsc-6izfgpkr/config.setCompilers/conftest.o Possible ERROR while running linker: exit code 1 stderr: ld.lld: error: unable to find library -lmpi_gtl_hsa clang-12: error: linker command failed with exit code 1 (use -v to see invocation) Linker output before filtering:

ld.lld: error: unable to find library -lmpi_gtl_hsa clang-12: error: linker command failed with exit code 1 (use -v to see invocation) : Linker output after filtering:

ld.lld: error: unable to find library -lmpi_gtl_hsa clang-12: error: linker command failed with exit code 1 (use -v to see invocation): Error testing C compiler: Cannot compile/link C with cc. Deleting "CC"

That -l look like it is put in directly by 'cc'. How does anything work on that machine?

It looks like this comes from an env variable that I have:

export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa"

...

Matt

...

Thanks, Mark

module load craype-accel-amd-gfx908 module load rocm module load emacs module load zlib module load autoconf automake libtool

## These must be set before compiling so the executable picks up GTL export PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib" export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa"

## These must be set before running export MPIR_CVAR_GPU_EAGER_DEVICE_MEM=0 export MPICH_GPU_SUPPORT_ENABLED=1 export MPICH_SMP_SINGLE_COPY_MODE=CMA

-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>