[petsc-dev] Petsc compilation issues with Xcode 15 on macOS Sonoma

13 Nov 2023

      Dear Petsc devs,

I have been having issues lately when compiling petsc on MacOS with various
libraries. This is probably due to the updated linker in MacOS and
unfortunately the newer operating systems do not allow using older
toolchains.

on arm64, only 3.20 and up can be configured

My configure command usually looks like this:
'sudo ./configure  COPTFLAGS=-O3 -g CXXOPTFLAGS=-O3 -g FOPTFLAGS=-O3 -g
--with-cxx-dialect=C++11 --with-mpi-dir=/opt/openfpm/dep_clang/MPI
--with-64-bit-indices  --with-parmetis-dir=/opt/openfpm/dep_clang/PARMETIS
--with-metis-dir=/opt/openfpm/dep_clang/METIS --with-boost=yes
--with-boost-dir=/opt/openfpm/dep_clang/BOOST
--with-suitesparse-dir=/opt/openfpm/dep_clang/SUITESPARSE
--download-superlu_dist --download-hypre
--prefix=/opt/openfpm/dep_clang/PETSC --download-scalapack --download-mumps
--with-debugging=0 --with-clean
'

There are two main issues:

1) Duplicate 'LC_Paths' when compiling code with apple clang and gfortran.
If I enable download_scalapack, the the cmake findPetsc fails with the
following error on both x86 and arm64:
"""
 kind: "try_run-v1"
    backtrace:
      -
"/opt/homebrew/Cellar/cmake/3.27.7/share/cmake/Modules/Internal/CheckSourceRuns.cmake:93
(try_run)"
      -
"/opt/homebrew/Cellar/cmake/3.27.7/share/cmake/Modules/CheckCSourceRuns.cmake:52
(cmake_check_source_runs)"
      - "cmake_modules/FindPackageMultipass.cmake:97 (check_c_source_runs)"
      - "cmake_modules/FindPETSc.cmake:284 (multipass_source_runs)"
      - "cmake_modules/FindPETSc.cmake:318 (petsc_test_runs)"
      - "CMakeLists.txt:69 (find_package)"
    checks:
      - "Performing Test MULTIPASS_TEST_4_petsc_works_all"
    directories:
      source:
"/Users/absingh/openfpm_pdata/build/CMakeFiles/CMakeScratch/TryCompile-LJcmJB"
      binary:
"/Users/absingh/openfpm_pdata/build/CMakeFiles/CMakeScratch/TryCompile-LJcmJB"
    cmakeVariables:
      CMAKE_C_FLAGS: ""
      CMAKE_EXE_LINKER_FLAGS: ""
      CMAKE_MODULE_PATH: "/Users/absingh/openfpm_pdata/cmake_modules/"
      CMAKE_OSX_ARCHITECTURES: ""
      CMAKE_OSX_DEPLOYMENT_TARGET: ""
      CMAKE_OSX_SYSROOT:
"/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk"
    buildResult:
      variable: "MULTIPASS_TEST_4_petsc_works_all_COMPILED"
      cached: true
      stdout: |
        Change Dir:
'/Users/absingh/openfpm_pdata/build/CMakeFiles/CMakeScratch/TryCompile-LJcmJB'

        Run Build Command(s): /opt/homebrew/Cellar/cmake/3.27.7/bin/cmake
-E env VERBOSE=1 /usr/bin/make -f Makefile cmTC_e0165/fast
        /Library/Developer/CommandLineTools/usr/bin/make  -f
CMakeFiles/cmTC_e0165.dir/build.make CMakeFiles/cmTC_e0165.dir/build
        Building C object CMakeFiles/cmTC_e0165.dir/src.c.o
        /Library/Developer/CommandLineTools/usr/bin/cc
-DMULTIPASS_TEST_4_petsc_works_all -I/opt/openfpm/dep_clang/PETSC/include
-I/opt/openfpm/dep_clang/SUITESPARSE/include
-I/opt/openfpm/dep_clang/PARMETIS/include
-I/opt/openfpm/dep_clang/METIS/include
-I/opt/openfpm/dep_clang/BOOST/include -I/opt/openfpm/dep_clang/MPI/include
-arch arm64 -isysroot
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk -MD -MT
CMakeFiles/cmTC_e0165.dir/src.c.o -MF CMakeFiles/cmTC_e0165.dir/src.c.o.d
-o CMakeFiles/cmTC_e0165.dir/src.c.o -c
/Users/absingh/openfpm_pdata/build/CMakeFiles/CMakeScratch/TryCompile-LJcmJB/src.c
        Linking C executable cmTC_e0165
        /opt/homebrew/Cellar/cmake/3.27.7/bin/cmake -E cmake_link_script
CMakeFiles/cmTC_e0165.dir/link.txt --verbose=1
        /Library/Developer/CommandLineTools/usr/bin/cc  -arch arm64
-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk
-Wl,-search_paths_first -Wl,-headerpad_max_install_names
CMakeFiles/cmTC_e0165.dir/src.c.o -o cmTC_e0165
 -Wl,-rpath,/opt/openfpm/dep_clang/PETSC/lib
-Wl,-rpath,/opt/openfpm/dep_clang/SUITESPARSE/lib
/opt/openfpm/dep_clang/PETSC/lib/libpetsc.dylib
/opt/openfpm/dep_clang/PETSC/lib/libHYPRE.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libspqr.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libumfpack.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libklu.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libcholmod.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libbtf.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libccolamd.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libcolamd.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libcamd.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libamd.dylib
/opt/openfpm/dep_clang/SUITESPARSE/lib/libsuitesparseconfig.dylib
/opt/openfpm/dep_clang/PETSC/lib/libdmumps.a
/opt/openfpm/dep_clang/PETSC/lib/libmumps_common.a
/opt/openfpm/dep_clang/PETSC/lib/libpord.a
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/lib/libpthread.tbd
/opt/openfpm/dep_clang/PETSC/lib/libscalapack.dylib
/opt/openfpm/dep_clang/PETSC/lib/libsuperlu_dist.dylib
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/lib/liblapack.tbd
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/lib/libblas.tbd
/opt/openfpm/dep_clang/PARMETIS/lib/libparmetis.a
/opt/openfpm/dep_clang/METIS/lib/libmetis.dylib
/opt/openfpm/dep_clang/MPI/lib/libmpi_usempif08.dylib
/opt/openfpm/dep_clang/MPI/lib/libmpi_usempi_ignore_tkr.a
/opt/openfpm/dep_clang/MPI/lib/libmpi_mpifh.dylib
/opt/openfpm/dep_clang/MPI/lib/libmpi.dylib
/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/libgfortran.dylib
/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/gcc/aarch64-apple-darwin23/13/libemutls_w.a
/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/libquadmath.dylib
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/lib/libc++.tbd
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/lib/libgcc_s.1.tbd

        ld: warning: duplicate LC_RPATH are deprecated
('/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/gcc/aarch64-apple-darwin23/13')
        ld: warning: duplicate LC_RPATH are deprecated
('/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/gcc')
        ld: warning: duplicate LC_RPATH are deprecated
('/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current')
        ld: warning: duplicate LC_RPATH are deprecated
('/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/gcc/aarch64-apple-darwin23/13')
        ld: warning: duplicate LC_RPATH are deprecated
('/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/gcc')
        ld: warning: duplicate LC_RPATH are deprecated
('/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current')
        ld: duplicate LC_RPATH
'/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/gcc/aarch64-apple-darwin23/13'
in '/opt/openfpm/dep_clang/PETSC/lib/libscalapack.2.2.1.dylib'
        clang: error: linker command failed with exit code 1 (use -v to see
invocation)
        make[1]: *** [cmTC_e0165] Error 1
        make: *** [cmTC_e0165/fast] Error 2

      exitCode: 2
    runResult:
      variable: "MULTIPASS_TEST_4_petsc_works_all_EXITCODE"
      cached: true
"""

Second, on arm64, I am unable to use the option
"with-metis-dir=/path/to/metis". The installation works fine but when I run
make check, I get the error that 'libmetis.dylib' is not found. This
behavior continues even when I export 'DYLD_LIBRARY_PATH' to have the
libmetis. CMake findPetsc also fails with the same error.

Any advice would be great.

Thank you!
Abhinav

[petsc-dev] Petsc compilation issues with Xcode 15 on macOS Sonoma

Abhinav Singh