Thanks! Are you able to use cmake from this spack install? I get this error: [a04201@a33-5211c ~]$ spack load cmake ==> Error: cmake matches multiple packages. Matching packages: ioepv7h [email protected]%[email protected] arch=linux-rhel8-skylake_avx512 bae6gkh [email protected]%[email protected] arch=linux-rhel8-a64fx Use a more specific spec. [a04201@a33-5211c ~]$ spack load [email protected]%[email protected] [WARN] xos LPG 2002 - Failed to map HugeTLBfs for data/bss: /bin/bash The e_type of elf header must be ET_EXEC when using libmpg. You can check it on your load module by readelf -h command. Satish On Fri, 16 Apr 2021, Sreepathi, Sarat via petsc-dev wrote:
For GNU in general, -Ofast -march=armv8.2-a+sve -msve-vector-bits=512
You may want to use -O3 if you run into FP exceptions etc. or look into turning on/off specific optimizations. -Sarat.
-----Original Message----- From: Satish Balay <[email protected]> Sent: Friday, April 16, 2021 12:41 PM To: Sreepathi, Sarat <[email protected]> Cc: Mark Adams <[email protected]>; petsc-dev <[email protected]> Subject: Re: [petsc-dev] [EXTERNAL] Re: building on Fugaku
Sarat,
I can give this a try. I see the instructions say "Execute the following commands on the command line in an interactive job"
What optimization flags do you use here?
Satish
On Fri, 16 Apr 2021, Sreepathi, Sarat via petsc-dev wrote:
Hi Mark,
I couldn’t follow all the issues in the other thread. Anyway, you are in luck.
I have pushed for GNU compiler support with Fujitsu MPI since November after hitting multiple issues. Finally, got an update from Fugaku helpdesk earlier this week. As you are aware, you will probably get better performance with GNU than Fujitsu on A64FX. My latest experiments confirmed that *again* for climate workloads.
See "Users Guide - Language and development environment Ver. 1.02" - Chapter 4 for details. https://www.fugaku.r-ccs.riken.jp/doc_root/en/user_guides/lang_1.02/GC C_MPI/index.html
Additionally, please note that you will need to `spack load [email protected]`<mailto:[email protected]%60> to load the latest GNU compiler. To summarize, you need to do the following on a compute node to get the requisite environment. ``` . /vol0004/apps/oss/spack/share/spack/setup-env.sh spack load fujitsu-mpi%gcc spack load [email protected]<mailto:[email protected]> ```
Hope this helps (cc’ing petsc-dev), Sarat.
From: Mark Adams <[email protected]> Sent: Friday, April 16, 2021 10:27 AM To: Sreepathi, Sarat <[email protected]> Subject: Re: [EXTERNAL] Re: building on Fugaku
Sarat, I am making progress. Kokkos is working w/o OMP, but Kokkos Kernels + OMP is barfing and Kokkos people suggested GNU. But one of my third party libraries needs MPI. Do you know of some sort of MPI library on Fugaku? It just has to work with one process. Thanks, Mark
On Wed, Apr 14, 2021 at 3:21 PM Sreepathi, Sarat <[email protected]<mailto:[email protected]>> wrote: Hi Mark,
From earlier notes, I used the Kokkos “develop” branch.
cmake -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=${KOKKOS_SRC_DIR}/install \ -DBUILD_TESTING=OFF \ -DKokkos_ENABLE_LIBDL=OFF \ -DKokkos_ENABLE_OPENMP=ON \ -DKokkos_ENABLE_SERIAL=ON \ -DKokkos_ENABLE_AGGRESSIVE_VECTORIZATION=ON \ .. -Sarat.
From: Mark Adams <[email protected]<mailto:[email protected]>> Sent: Wednesday, April 14, 2021 3:09 PM To: Sreepathi, Sarat <[email protected]<mailto:[email protected]>> Cc: Richard Tran Mills <[email protected]<mailto:[email protected]>> Subject: [EXTERNAL] Re: building on Fugaku
Sarat,
I have made some progress but am getting a C++ error:
CMake Warning at cmake/kokkos_test_cxx_std.cmake:47 (MESSAGE): C++14 is not supported as a compiler feature. We will choose custom flags for now, but this behavior has been deprecated. Please open an issue at https://github.com/kokkos/kokkos/issues reporting that Fujitsu failed for 14, preferrably including your CMake command.
I am using:
04:08 adams/landau-gpu-opt= ~/petsc$ mpiFCC --version FCC (FCC) 4.5.0 20210304 simulating gcc version 6.1
Do you have any ideas?
Thanks, Mark
On Wed, Apr 14, 2021 at 7:55 AM Mark Adams <[email protected]<mailto:[email protected]>> wrote: Hi Sarat,
My colleague, Rich Mills, and I are coming back to Fugaku after a while, and I wanted to ask you how you build, with Kokkos, on Fugaku. I have tried Arm and Fugitzu compilers and was able to get Fugitzu to work a few months ago, but I wanted to see what you use. Especially for Kokkos. Can you share with us your build details?
Thanks, Mark