November 2019 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Optimizing runtime for 4 mpiexec.hydra jobs
by Joachim Protze 13 Nov '19

13 Nov '19

Hi Doug, in general, using hyperthreads only improves execution time, if you do not utilize the core with a single process/thread. I.e., if you see ~100% cpu utilization per process in "top" for the single job execution, doubling the execution time from 2 to 4 mpi jobs sounds reasonable. If your application is mostly calculating (as an MPI application hopefully does), the two processes/threads running on the same core share the execution time of the core and will finally end up with double execution time. Depending on your application, additional processes/threads also might increase the pressure on the memory bus and therefore slow down the other application by making it wait for memory accesses. This might also explain the execution time increase from one to two mpi jobs. All this depends on the cpu/memory configuration of this machine. Best Joachim On 11/13/19 5:39 PM, Douglas Dommermuth via discuss wrote: > I am running Ubuntu 18.04.3 with MPICH 3.3~a2-4 and GFortran > 4:7.4.0-1ubuntu2.3 and GCC 4:7.4.0-1ubuntu2.3CC on dual AMD EPYC 7742 > processors with hyper threading enabled. My codes are written in MPI > and Fortran. The dual AMD processors have 128 cores and 256 threads. > I want to optimize the runtime for 4 mpi jobs running concurrently > with 64 threads each. Some timings are provided here: > > 1. One mpi job with mpiexec.hydra -n 64 myprog => 57.32s > 2. One mpi job with mpiexec.hydra -bind-to numa -n 64 => 50.52s > 3. Two mpi jobs with mpiexec.hydra -n 64 myprog => 99.77s > 4. Two mpi jobs with mpiexec.hydra -bind-to numa -n 64 => 72.23s > 5. Four mpi jobs with mpiexec.hydra -bind-to numa -n 64 => 159.2s > > The option "-bind-to numa" helps, but even so, running four mpi > jobs concurrently with 64 threads each is considerably slower than > running one mpi job with 64 threads. I can almost run four mpi jobs > sequentially and match the time for running four mpi jobs concurrently. > How can I improve on the result for running 4 mpi jobs concurrently? > Thanks, Doug. > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Dipl.-Inf. Joachim Protze IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0

Re: [mpich-discuss] Optimizing runtime for 4 mpiexec.hydra jobs
by Douglas Dommermuth 13 Nov '19

13 Nov '19

Hi Hui, Here is the updated list with your request: 1. One mpi job with mpiexec.hydra -n 64 myprog => 57.32s 2. One mpi job with mpiexec.hydra -bind-to numa -n 64 myprog => 50.52s 3. Two mpi jobs with mpiexec.hydra -n 64 myprog => 99.77s 4. Two mpi jobs with mpiexec.hydra -bind-to numa -n 64 myprog => 72.23s 5. Four mpi jobs with mpiexec.hydra -bind-to numa -n 64 myprog => 159.2s 6. Four mpi jobs with mpiexec.hydra -n 64 myprog => 159.9s Case 6 is the same speed as Case 5. The code is a CFD code based on a subdomain approach. There is communication between neighboring subdomains. The machine is a supermicro server with 512GB of memory. These small test cases use about 3% of the available memory. I want to run 4 larger MPI jobs concurrently. ________________________________ From: Zhou, Hui <zhouh(a)anl.gov> Sent: Wednesday, November 13, 2019 8:55 AM To: discuss(a)mpich.org Cc: Douglas Dommermuth Subject: Re: [mpich-discuss] Optimizing runtime for 4 mpiexec.hydra jobs Hi Doug, What is your number running 4 mpi jobs without -bind-to numa? - Hui Zhou On Nov 13, 2019, at 10:39 AM, Douglas Dommermuth via discuss <discuss(a)mpich.org<mailto:[email protected]>> wrote: I am running Ubuntu 18.04.3 with MPICH 3.3~a2-4 and GFortran 4:7.4.0-1ubuntu2.3 and GCC 4:7.4.0-1ubuntu2.3CC on dual AMD EPYC 7742 processors with hyper threading enabled. My codes are written in MPI and Fortran. The dual AMD processors have 128 cores and 256 threads. I want to optimize the runtime for 4 mpi jobs running concurrently with 64 threads each. Some timings are provided here: 1. One mpi job with mpiexec.hydra -n 64 myprog => 57.32s 2. One mpi job with mpiexec.hydra -bind-to numa -n 64 => 50.52s 3. Two mpi jobs with mpiexec.hydra -n 64 myprog => 99.77s 4. Two mpi jobs with mpiexec.hydra -bind-to numa -n 64 => 72.23s 5. Four mpi jobs with mpiexec.hydra -bind-to numa -n 64 => 159.2s The option "-bind-to numa" helps, but even so, running four mpi jobs concurrently with 64 threads each is considerably slower than running one mpi job with 64 threads. I can almost run four mpi jobs sequentially and match the time for running four mpi jobs concurrently. How can I improve on the result for running 4 mpi jobs concurrently? Thanks, Doug. _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Optimizing runtime for 4 mpiexec.hydra jobs
by Congiu, Giuseppe 13 Nov '19

13 Nov '19

Hi Doug, When you do -map-by numa, processes should be bound to available numas using a round robin mapping. Thus if you are running only one MPI job with -bind-to numa and you have two numas, you are binding rank 0 to numa0:core0:hwthread0, rank 1 to numa1:core64:hwthread128, rank 2 to numa0:core0:hwthread1, and so on. Which is what you were expecting. However, if you run multiple MPI jobs with the same bind policy, these are independent from each other and will all map to the same core and hwthreads. Thus if you have 256 MPI processes running over 4 separate jobs you are still using half of the cores and a forth of the hwthreads. If you want to map to different hwthreads you can use a custom policy different for every job: mpiexec -bind-to --help -bind-to: Process-core binding type to use Binding type options: Default: none -- no binding (default) Architecture unaware options: rr -- round-robin as OS assigned processor IDs user:0+2,1+4,3,2 -- user specified binding The user policy allows you to do exactly that. There you define what hwthreads you want to bind your ranks to (+N) means your MPI process can move across hwthreads skipping every N hwthreads. So if you have 32 hwthreads and one process and request -bind-to user:0+2 you get your process moving around across hwthreads: 0, 2, 4, 6, 8, … If you want to see what the binding is for your processes you can export HYDRA_TOPO_DEBUG=1 and use /bin/true as simple example code to inspect the binding you get for different options. If you don’t do any binding MPI processes will be re-scheduled over the course of the runtime on different cores, hwthreads (losing cached data) and possibly even different numas. In Linux memory is allocated on first touch policy by default. Which means that physical memory is allocated on the numa your process is running when it accessed it first. If your process gets moved to another numa later on during the runtime it will be accessing its memory across numa domains which has a bigger impact on latency. So if your application is latency sensitive you may get a big hit on performance. Hope this helps. Best, Giuseppe On Nov 13, 2019, at 10:39 AM, Douglas Dommermuth via discuss <discuss(a)mpich.org<mailto:[email protected]>> wrote: 256

1 0

Re: [mpich-discuss] Optimizing runtime for 4 mpiexec.hydra jobs
by Zhou, Hui 13 Nov '19

13 Nov '19

Hi Doug, What is your number running 4 mpi jobs without -bind-to numa? — Hui Zhou On Nov 13, 2019, at 10:39 AM, Douglas Dommermuth via discuss <discuss(a)mpich.org<mailto:[email protected]>> wrote: I am running Ubuntu 18.04.3 with MPICH 3.3~a2-4 and GFortran 4:7.4.0-1ubuntu2.3 and GCC 4:7.4.0-1ubuntu2.3CC on dual AMD EPYC 7742 processors with hyper threading enabled. My codes are written in MPI and Fortran. The dual AMD processors have 128 cores and 256 threads. I want to optimize the runtime for 4 mpi jobs running concurrently with 64 threads each. Some timings are provided here: 1. One mpi job with mpiexec.hydra -n 64 myprog => 57.32s 2. One mpi job with mpiexec.hydra -bind-to numa -n 64 => 50.52s 3. Two mpi jobs with mpiexec.hydra -n 64 myprog => 99.77s 4. Two mpi jobs with mpiexec.hydra -bind-to numa -n 64 => 72.23s 5. Four mpi jobs with mpiexec.hydra -bind-to numa -n 64 => 159.2s The option "-bind-to numa" helps, but even so, running four mpi jobs concurrently with 64 threads each is considerably slower than running one mpi job with 64 threads. I can almost run four mpi jobs sequentially and match the time for running four mpi jobs concurrently. How can I improve on the result for running 4 mpi jobs concurrently? Thanks, Doug. _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Optimizing runtime for 4 mpiexec.hydra jobs
by Douglas Dommermuth 13 Nov '19

13 Nov '19

I am running Ubuntu 18.04.3 with MPICH 3.3~a2-4 and GFortran 4:7.4.0-1ubuntu2.3 and GCC 4:7.4.0-1ubuntu2.3CC on dual AMD EPYC 7742 processors with hyper threading enabled. My codes are written in MPI and Fortran. The dual AMD processors have 128 cores and 256 threads. I want to optimize the runtime for 4 mpi jobs running concurrently with 64 threads each. Some timings are provided here: 1. One mpi job with mpiexec.hydra -n 64 myprog => 57.32s 2. One mpi job with mpiexec.hydra -bind-to numa -n 64 => 50.52s 3. Two mpi jobs with mpiexec.hydra -n 64 myprog => 99.77s 4. Two mpi jobs with mpiexec.hydra -bind-to numa -n 64 => 72.23s 5. Four mpi jobs with mpiexec.hydra -bind-to numa -n 64 => 159.2s The option "-bind-to numa" helps, but even so, running four mpi jobs concurrently with 64 threads each is considerably slower than running one mpi job with 64 threads. I can almost run four mpi jobs sequentially and match the time for running four mpi jobs concurrently. How can I improve on the result for running 4 mpi jobs concurrently? Thanks, Doug.

1 0

Re: [mpich-discuss] Compilation with AOCC - clang-8 errors unsupported options '-soname', etc.
by Je Iaa 11 Nov '19

11 Nov '19

1 0

Re: [mpich-discuss] Compilation with AOCC - clang-8 errors unsupported options '-soname', etc.
by Je Iaa 08 Nov '19

08 Nov '19

1 0

Re: [mpich-discuss] Compilation with AOCC - clang-8 errors unsupported options '-soname', etc.
by Zhou, Hui 08 Nov '19

08 Nov '19

With `—disable-shared` you didn’t solve the problem, you avoided the problem. From your `./libtool —tag=FC —config`, apparently libtool has no idea about `flang`, so it doesn’t know how to pass on the linker flags. It is strange that they took a blank default for `wl`. `flang` is quite new, until libtool gets updated, the only solution is to patch libtool. To do it manually, if you open the libtool script (after configure), if you see `wl=‘’`, then it probably should be changed into `wl=‘-Wl,’`. — Hui Zhou On Nov 8, 2019, at 6:04 AM, Je Iaa via discuss <discuss(a)mpich.org<mailto:[email protected]>> wrote: Hi Thomas, I thought I had "solved" the problem by using --disable-shared --enable-static, but then I have other issues after (with hdf5, netcdf, etc.), so back to square one... You will find below what the ./libtool --tag=FC --config command gives, is that of any help? Regards, J ./libtool --tag=FC --config # Which release of libtool.m4 was used? macro_version=2.4.6 macro_revision=2.4.6 # Whether or not to build shared libraries. build_libtool_libs=yes # Whether or not to build static libraries. build_old_libs=yes # What type of objects to build. pic_mode=default # Whether or not to optimize for fast installation. fast_install=yes # Shared archive member basename,for filename based shared library versioning on AIX. shared_archive_member_spec= # Shell to use when invoking shell scripts. SHELL="/bin/sh" # An echo program that protects backslashes. ECHO="printf %s\\n" # The PATH separator for the build system. PATH_SEPARATOR=":" # The host system. host_alias= host=x86_64-unknown-linux-gnu host_os=linux-gnu # The build system. build_alias= build=x86_64-unknown-linux-gnu build_os=linux-gnu # A sed program that does not truncate output. SED="/usr/bin/sed" # Sed that helps us avoid accidentally triggering echo(1) options like -n. Xsed="$SED -e 1s/^X//" # A grep program that handles long lines. GREP="/usr/bin/grep" # An ERE matcher. EGREP="/usr/bin/grep -E" # A literal string matcher. FGREP="/usr/bin/grep -F" # A BSD- or MS-compatible name lister. NM="/usr/bin/nm -B" # Whether we need soft or hard links. LN_S="ln -s" # What is the maximum length of a command? max_cmd_len=1572864 # Object file suffix (normally "o"). objext=o # Executable file suffix (normally ""). exeext= # whether the shell understands "unset". lt_unset=unset # turn spaces into newlines. SP2NL="tr \\040 \\012" # turn newlines into spaces. NL2SP="tr \\015\\012 \\040\\040" # convert $build file names to $host format. to_host_file_cmd=func_convert_file_noop # convert $build files to toolchain format. to_tool_file_cmd=func_convert_file_noop # An object symbol dumper. OBJDUMP="objdump" # Method to check whether dependent libraries are shared objects. deplibs_check_method="pass_all" # Command to use when deplibs_check_method = "file_magic". file_magic_cmd="\$MAGIC_CMD" # How to find potential files when deplibs_check_method = "file_magic". file_magic_glob="" # Find potential files using nocaseglob when deplibs_check_method = "file_magic". want_nocaseglob="no" # DLL creation program. DLLTOOL="false" # Command to associate shared and link libraries. sharedlib_from_linklib_cmd="printf %s\\n" # The archiver. AR="ar" # Flags to create an archive. AR_FLAGS="cr" # How to feed a file listing to the archiver. archiver_list_spec="@" # A symbol stripping program. STRIP="strip" # Commands used to install an old-style archive. RANLIB="ranlib" old_postinstall_cmds="chmod 644 \$oldlib~\$RANLIB \$tool_oldlib" old_postuninstall_cmds="" # Whether to use a lock for old archive extraction. lock_old_archive_extraction=no # A C compiler. LTCC="clang" # LTCC compiler flags. LTCFLAGS=" -O2" # Take the output of nm and produce a listing of raw symbols and C names. global_symbol_pipe="sed -n -e 's/^.*[ ]\$[ABCDGIRSTW][ABCDGIRSTW]*\$[ ][ ]*\$[_A-Za-z][_A-Za-z0-9]*\$\$/\\1 \\2 \\2/p' | sed '/ __gnu_lto/d'" # Transform the output of nm in a proper C declaration. global_symbol_to_cdecl="sed -n -e 's/^T .* \$.*\$\$/extern int \\1();/p' -e 's/^[ABCDGIRSTW][ABCDGIRSTW]* .* \$.*\$\$/extern char \\1;/p'" # Transform the output of nm into a list of symbols to manually relocate. global_symbol_to_import="" # Transform the output of nm in a C name address pair. global_symbol_to_c_name_address="sed -n -e 's/^: \$.*\$ .*\$/ {\"\\1\", (void *) 0},/p' -e 's/^[ABCDGIRSTW][ABCDGIRSTW]* .* \$.*\$\$/ {\"\\1\", (void *) \\&\\1},/p'" # Transform the output of nm in a C name address pair when lib prefix is needed. global_symbol_to_c_name_address_lib_prefix="sed -n -e 's/^: \$.*\$ .*\$/ {\"\\1\", (void *) 0},/p' -e 's/^[ABCDGIRSTW][ABCDGIRSTW]* .* \$lib.*\$\$/ {\"\\1\", (void *) \\&\\1},/p' -e 's/^[ABCDGIRSTW][ABCDGIRSTW]* .* \$.*\$\$/ {\"lib\\1\", (void *) \\&\\1},/p'" # The name lister interface. nm_interface="BSD nm" # Specify filename containing input files for $NM. nm_file_list_spec="@" # The root where to search for dependent libraries,and where our libraries should be installed. lt_sysroot= # Command to truncate a binary pipe. lt_truncate_bin="/usr/bin/dd bs=4096 count=1" # The name of the directory that contains temporary libtool files. objdir=.libs # Used to examine libraries when file_magic_cmd begins with "file". MAGIC_CMD=file # Must we lock files when doing compilation? need_locks="no" # Manifest tool. MANIFEST_TOOL=":" # Tool to manipulate archived DWARF debug symbol files on Mac OS X. DSYMUTIL="" # Tool to change global to local symbols on Mac OS X. NMEDIT="" # Tool to manipulate fat objects and archives on Mac OS X. LIPO="" # ldd/readelf like tool for Mach-O binaries on Mac OS X. OTOOL="" # ldd/readelf like tool for 64 bit Mach-O binaries on Mac OS X 10.4. OTOOL64="" # Old archive suffix (normally "a"). libext=a # Shared library suffix (normally ".so"). shrext_cmds=".so" # The commands to extract the exported symbol list from a shared archive. extract_expsyms_cmds="" # Variables whose values should be saved in libtool wrapper scripts and # restored at link time. variables_saved_for_relink="PATH LD_LIBRARY_PATH LD_RUN_PATH" # Do we need the "lib" prefix for modules? need_lib_prefix=no # Do we need a version for libraries? need_version=no # Library versioning type. version_type=linux # Shared library runtime path variable. runpath_var=LD_RUN_PATH # Shared library path variable. shlibpath_var=LD_LIBRARY_PATH # Is shlibpath searched before the hard-coded library search path? shlibpath_overrides_runpath=no # Format of library name prefix. libname_spec="lib\$name" # List of archive names. First name is the real one, the rest are links. # The last name is the one that the linker finds with -lNAME library_names_spec="\$libname\$release\$shared_ext\$versuffix \$libname\$release\$shared_ext\$major \$libname\$shared_ext" # The coded name of the library, if different from the real name. soname_spec="\$libname\$release\$shared_ext\$major" # Permission mode override for installation of shared libraries. install_override_mode="" # Command to use after installation of a shared archive. postinstall_cmds="" # Command to use after uninstallation of a shared archive. postuninstall_cmds="" # Commands used to finish a libtool library installation in a directory. finish_cmds="PATH=\\\"\\\$PATH:/sbin\\\" ldconfig -n \$libdir" # As "finish_cmds", except a single script fragment to be evaled but # not shown. finish_eval="" # Whether we should hardcode library paths into libraries. hardcode_into_libs=yes # Compile-time system search path for libraries. sys_lib_search_path_spec="/opt/modules/src/aocc-compiler-2.0.0/lib/clang/8.0.0 /usr/lib/gcc/x86_64-redhat-linux/4.8.5 /usr/lib64 /lib64 /usr/lib /opt/modules/src/aocc-compiler-2.0.0/lib /lib " # Detected run-time system search path for libraries. sys_lib_dlsearch_path_spec="/usr/lib64//bind9-export /usr/lib64/dyninst /usr/lib64/mysql /lib /lib64 /lib/sse2 /lib64/sse2 /lib64/tls " # Explicit LT_SYS_LIBRARY_PATH set during ./configure time. configure_time_lt_sys_library_path="" # Whether dlopen is supported. dlopen_support=unknown # Whether dlopen of programs is supported. dlopen_self=unknown # Whether dlopen of statically linked programs is supported. dlopen_self_static=unknown # Commands to strip libraries. old_striplib="strip --strip-debug" striplib="strip --strip-unneeded" # The linker used to build libraries. LD="/usr/bin/ld -m elf_x86_64" # How to create reloadable object files. reload_flag=" -r" reload_cmds="\$LD\$reload_flag -o \$output\$reload_objs" # Commands used to build an old-style archive. old_archive_cmds="\$AR \$AR_FLAGS \$oldlib\$oldobjs~\$RANLIB \$tool_oldlib" # A language specific compiler. CC="clang" # Is the compiler the GNU compiler? with_gcc=yes # Compiler flag to turn off builtin functions. no_builtin_flag=" -fno-builtin -fno-rtti -fno-exceptions" # Additional compiler flags for building library objects. pic_flag=" -fPIC -DPIC" # How to pass a linker flag through the compiler. wl="-Wl," # Compiler flag to prevent dynamic linking. link_static_flag="" # Does compiler simultaneously support -c and -o options? compiler_c_o="yes" # Whether or not to add -lc for building shared libraries. build_libtool_need_lc=no # Whether or not to disallow shared libs when runtime libs are static. allow_libtool_libs_with_static_runtimes=no # Compiler flag to allow reflexive dlopens. export_dynamic_flag_spec="\$wl--export-dynamic" # Compiler flag to generate shared objects directly from archives. whole_archive_flag_spec="\$wl--whole-archive\$convenience \$wl--no-whole-archive" # Whether the compiler copes with passing no objects directly. compiler_needs_object="no" # Create an old-style archive from a shared archive. old_archive_from_new_cmds="" # Create a temporary old-style archive to link instead of a shared archive. old_archive_from_expsyms_cmds="" # Commands used to build a shared archive. archive_cmds="\$CC -shared \$pic_flag \$libobjs \$deplibs \$compiler_flags \$wl-soname \$wl\$soname -o \$lib" archive_expsym_cmds="echo \\\"{ global:\\\" > \$output_objdir/\$libname.ver~ cat \$export_symbols | sed -e \\\"s/\\\$.*\\\$/\\\\1;/\\\" >> \$output_objdir/\$libname.ver~ echo \\\"local: *; };\\\" >> \$output_objdir/\$libname.ver~ \$CC -shared \$pic_flag \$libobjs \$deplibs \$compiler_flags \$wl-soname \$wl\$soname \$wl-version-script \$wl\$output_objdir/\$libname.ver -o \$lib" # Commands used to build a loadable module if different from building # a shared archive. module_cmds="" module_expsym_cmds="" # Whether we are building with GNU ld or not. with_gnu_ld="yes" # Flag that allows shared libraries with undefined symbols to be built. allow_undefined_flag="" # Flag that enforces no undefined symbols. no_undefined_flag="" # Flag to hardcode $libdir into a binary during linking. # This must work even if $libdir does not exist hardcode_libdir_flag_spec="\$wl-rpath \$wl\$libdir" # Whether we need a single "-rpath" flag with a separated argument. hardcode_libdir_separator="" # Set to "yes" if using DIR/libNAME$shared_ext during linking hardcodes # DIR into the resulting binary. hardcode_direct=no # Set to "yes" if using DIR/libNAME$shared_ext during linking hardcodes # DIR into the resulting binary and the resulting library dependency is # "absolute",i.e impossible to change by setting $shlibpath_var if the # library is relocated. hardcode_direct_absolute=no # Set to "yes" if using the -LDIR flag during linking hardcodes DIR # into the resulting binary. hardcode_minus_L=no # Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR # into the resulting binary. hardcode_shlibpath_var=unsupported # Set to "yes" if building a shared library automatically hardcodes DIR # into the library and all subsequent libraries and executables linked # against it. hardcode_automatic=no # Set to yes if linker adds runtime paths of dependent libraries # to runtime path list. inherit_rpath=no # Whether libtool must link a program against all its dependency libraries. link_all_deplibs=unknown # Set to "yes" if exported symbols are required. always_export_symbols=no # The commands to list exported symbols. export_symbols_cmds="\$NM \$libobjs \$convenience | \$global_symbol_pipe | \$SED 's/.* //' | sort | uniq > \$export_symbols" # Symbols that should not be listed in the preloaded symbols. exclude_expsyms="_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*" # Symbols that must always be exported. include_expsyms="" # Commands necessary for linking programs (against libraries) with templates. prelink_cmds="" # Commands necessary for finishing linking programs. postlink_cmds="" # Specify filename containing input files. file_list_spec="" # How to hardcode a shared library path into an executable. hardcode_action=immediate # The directories searched by this compiler when creating a shared library. compiler_lib_search_dirs="" # Dependencies to place before and after the objects being linked to # create a shared library. predep_objects="" postdep_objects="" predeps="" postdeps="" # The library search path used internally by the compiler when linking # a shared library. compiler_lib_search_path="" # ### BEGIN LIBTOOL TAG CONFIG: FC # The linker used to build libraries. LD="/usr/bin/ld -m elf_x86_64" # How to create reloadable object files. reload_flag=" -r" reload_cmds="\$LD\$reload_flag -o \$output\$reload_objs" # Commands used to build an old-style archive. old_archive_cmds="\$AR \$AR_FLAGS \$oldlib\$oldobjs~\$RANLIB \$tool_oldlib" # A language specific compiler. CC="flang" # Is the compiler the GNU compiler? with_gcc=no # Compiler flag to turn off builtin functions. no_builtin_flag="" # Additional compiler flags for building library objects. pic_flag="" # How to pass a linker flag through the compiler. wl="" # Compiler flag to prevent dynamic linking. link_static_flag="" # Does compiler simultaneously support -c and -o options? compiler_c_o="yes" # Whether or not to add -lc for building shared libraries. build_libtool_need_lc=no # Whether or not to disallow shared libs when runtime libs are static. allow_libtool_libs_with_static_runtimes=no # Compiler flag to allow reflexive dlopens. export_dynamic_flag_spec="\$wl--export-dynamic" # Compiler flag to generate shared objects directly from archives. whole_archive_flag_spec="\$wl--whole-archive\$convenience \$wl--no-whole-archive" # Whether the compiler copes with passing no objects directly. compiler_needs_object="no" # Create an old-style archive from a shared archive. old_archive_from_new_cmds="" # Create a temporary old-style archive to link instead of a shared archive. old_archive_from_expsyms_cmds="" # Commands used to build a shared archive. archive_cmds="\$CC -shared \$pic_flag \$libobjs \$deplibs \$compiler_flags \$wl-soname \$wl\$soname -o \$lib" archive_expsym_cmds="echo \\\"{ global:\\\" > \$output_objdir/\$libname.ver~ cat \$export_symbols | sed -e \\\"s/\\\$.*\\\$/\\\\1;/\\\" >> \$output_objdir/\$libname.ver~ echo \\\"local: *; };\\\" >> \$output_objdir/\$libname.ver~ \$CC -shared \$pic_flag \$libobjs \$deplibs \$compiler_flags \$wl-soname \$wl\$soname \$wl-version-script \$wl\$output_objdir/\$libname.ver -o \$lib" # Commands used to build a loadable module if different from building # a shared archive. module_cmds="" module_expsym_cmds="" # Whether we are building with GNU ld or not. with_gnu_ld="yes" # Flag that allows shared libraries with undefined symbols to be built. allow_undefined_flag="" # Flag that enforces no undefined symbols. no_undefined_flag="" # Flag to hardcode $libdir into a binary during linking. # This must work even if $libdir does not exist hardcode_libdir_flag_spec="\$wl-rpath \$wl\$libdir" # Whether we need a single "-rpath" flag with a separated argument. hardcode_libdir_separator="" # Set to "yes" if using DIR/libNAME$shared_ext during linking hardcodes # DIR into the resulting binary. hardcode_direct=no # Set to "yes" if using DIR/libNAME$shared_ext during linking hardcodes # DIR into the resulting binary and the resulting library dependency is # "absolute",i.e impossible to change by setting $shlibpath_var if the # library is relocated. hardcode_direct_absolute=no # Set to "yes" if using the -LDIR flag during linking hardcodes DIR # into the resulting binary. hardcode_minus_L=no # Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR # into the resulting binary. hardcode_shlibpath_var=unsupported # Set to "yes" if building a shared library automatically hardcodes DIR # into the library and all subsequent libraries and executables linked # against it. hardcode_automatic=no # Set to yes if linker adds runtime paths of dependent libraries # to runtime path list. inherit_rpath=no # Whether libtool must link a program against all its dependency libraries. link_all_deplibs=unknown # Set to "yes" if exported symbols are required. always_export_symbols=no # The commands to list exported symbols. export_symbols_cmds="\$NM \$libobjs \$convenience | \$global_symbol_pipe | \$SED 's/.* //' | sort | uniq > \$export_symbols" # Symbols that should not be listed in the preloaded symbols. exclude_expsyms="_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*" # Symbols that must always be exported. include_expsyms="" # Commands necessary for linking programs (against libraries) with templates. prelink_cmds="" # Commands necessary for finishing linking programs. postlink_cmds="" # Specify filename containing input files. file_list_spec="" # How to hardcode a shared library path into an executable. hardcode_action=immediate # The directories searched by this compiler when creating a shared library. compiler_lib_search_dirs="/usr/lib/gcc/x86_64-redhat-linux/4.8.5 /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64 /lib/../lib64 /usr/lib/../lib64 /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../.. /opt/modules/src/aocc-compiler-2.0.0/bin/../lib /lib /usr/lib" # Dependencies to place before and after the objects being linked to # create a shared library. predep_objects="/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbeginS.o" postdep_objects="/usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtendS.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crtn.o" predeps="" postdeps="-lflangmain -L/usr/lib -L/usr/lib64 -L/opt/modules/src/aocc-compiler-2.0.0/lib -L/opt/modules/src/aocc-compiler-2.0.0/lib32 -L. -lpgmath -lflang -lflangrti -lompstub -lm -lrt -lpthread -lgcc_s -lgcc -lc -lgcc_s -lgcc" # The library search path used internally by the compiler when linking # a shared library. compiler_lib_search_path="-L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../.. -L/opt/modules/src/aocc-compiler-2.0.0/bin/../lib -L/lib -L/usr/lib" # ### END LIBTOOL TAG CONFIG: FC Sent: Friday, November 08, 2019 at 12:02 PM From: "Thomas Jahns via discuss" <discuss(a)mpich.org<mailto:[email protected]>> To: discuss(a)mpich.org<mailto:[email protected]> Cc: "Thomas Jahns" <jahns(a)dkrz.de<mailto:[email protected]>> Subject: Re: [mpich-discuss] Compilation with AOCC - clang-8 errors unsupported options '-soname', etc. On 11/7/19 6:37 PM, Zhou, Hui via discuss wrote: > The attached log does not show libtool options. Is it possible to > reproduce and obtain the output of `./libtool —config`? I think ./libtool --tag=FC --config might be more interesting? Thomas > > — > Hui Zhou > > > > > > > > > >> On Nov 7, 2019, at 11:22 AM, Je Iaa <jeiaa(a)gmx.com<mailto:[email protected]> >> <mailto:[email protected]>> wrote: >> >> Hi Hui, >> Thank you for looking into this. >> You will find the log attached. >> Regards, >> J >> *Sent:* Thursday, November 07, 2019 at 5:03 PM >> *From:* "Zhou, Hui" <zhouh(a)anl.gov<mailto:[email protected]> <mailto:[email protected]>> >> *To:* "discuss(a)mpich.org<mailto:[email protected]> <mailto:[email protected]>" >> <discuss(a)mpich.org<mailto:[email protected]> <mailto:[email protected]>> >> *Cc:* "Je Iaa" <jeiaa(a)gmx.com<mailto:[email protected]> <mailto:[email protected]>> >> *Subject:* Re: [mpich-discuss] Compilation with AOCC - clang-8 errors >> unsupported options '-soname', etc. >> Those options are libtool options that supposed to be passed down to >> linker. It looks like the libtool is misconfigured. Could you do `make >> V=1` and attach a more verbose log? >> — >> Hui Zhou >> >> On Nov 7, 2019, at 9:53 AM, Je Iaa via discuss <discuss(a)mpich.org<mailto:[email protected]> >> <mailto:[email protected]>> wrote: >> Hi, >> I am trying without success to install mpich3.3.1 on virtual >> machine with AMD EPYC Processors and the AOCC2.0.0 compiler. >> This is what I did and the error I get: >> export PREFIX=/opt/modules/packages >> export MPICH_HOME=$PREFIX/mpich/3.3.1-aocc2.0.0 >> export CC=clang >> export CXX=clang++ >> export FC=flang >> export F77=flang >> >> mkdir -p $MPICH_HOME >> tar xvf mpich-3.3.1.tar.gz >> cd mpich-3.3.1 >> ./configure --prefix=$MPICH_HOME >> >> make >> Failed: >> clang-8: error: unsupported option '--whole-archive' >> clang-8: error: unsupported option '--no-whole-archive' >> clang-8: error: unknown argument: '-soname' >> clang-8: error: no such file or directory: 'libmpi.so.12' >> make[2]: *** [lib/libmpi.la<http://libmpi.la> <http://libmpi.la/>] Error 1 >> make[2]: Leaving directory `/opt/modules/src/mpich-3.3.1' >> make[1]: *** [all-recursive] Error 1 >> make[1]: Leaving directory `/opt/modules/src/mpich-3.3.1' >> make: *** [all] Error 2 >> Any idea how to sort this out? >> Regards, >> J >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org<mailto:[email protected]> <mailto:[email protected]> >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> <m.txt> > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org<mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Thomas Jahns HPC-Gruppe Abteilung Anwendungssoftware Deutsches Klimarechenzentrum GmbH Bundesstraße 45a • D-20146 Hamburg • Germany Phone: +49 40 460094-151 Fax: +49 40 460094-270 Email: Thomas Jahns <jahns(a)dkrz.de<mailto:[email protected]>> URL: www.dkrz.de<http://www.dkrz.de/> Geschäftsführer: Prof. Dr. Thomas Ludwig Sitz der Gesellschaft: Hamburg Amtsgericht Hamburg HRB 39784 _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Compilation with AOCC - clang-8 errors unsupported options '-soname', etc.
by Je Iaa 08 Nov '19

08 Nov '19

1 0

Re: [mpich-discuss] Compilation with AOCC - clang-8 errors unsupported options '-soname', etc.
by Thomas Jahns 08 Nov '19

08 Nov '19

On 11/7/19 6:37 PM, Zhou, Hui via discuss wrote: > The attached log does not show libtool options. Is it possible to > reproduce and obtain the output of `./libtool —config`? I think ./libtool --tag=FC --config might be more interesting? Thomas > > — > Hui Zhou > > > > > > > > > >> On Nov 7, 2019, at 11:22 AM, Je Iaa <jeiaa(a)gmx.com >> <mailto:[email protected]>> wrote: >> >> Hi Hui, >> Thank you for looking into this. >> You will find the log attached. >> Regards, >> J >> *Sent:* Thursday, November 07, 2019 at 5:03 PM >> *From:* "Zhou, Hui" <zhouh(a)anl.gov <mailto:[email protected]>> >> *To:* "discuss(a)mpich.org <mailto:[email protected]>" >> <discuss(a)mpich.org <mailto:[email protected]>> >> *Cc:* "Je Iaa" <jeiaa(a)gmx.com <mailto:[email protected]>> >> *Subject:* Re: [mpich-discuss] Compilation with AOCC - clang-8 errors >> unsupported options '-soname', etc. >> Those options are libtool options that supposed to be passed down to >> linker. It looks like the libtool is misconfigured. Could you do `make >> V=1` and attach a more verbose log? >> — >> Hui Zhou >> >> On Nov 7, 2019, at 9:53 AM, Je Iaa via discuss <discuss(a)mpich.org >> <mailto:[email protected]>> wrote: >> Hi, >> I am trying without success to install mpich3.3.1 on virtual >> machine with AMD EPYC Processors and the AOCC2.0.0 compiler. >> This is what I did and the error I get: >> export PREFIX=/opt/modules/packages >> export MPICH_HOME=$PREFIX/mpich/3.3.1-aocc2.0.0 >> export CC=clang >> export CXX=clang++ >> export FC=flang >> export F77=flang >> >> mkdir -p $MPICH_HOME >> tar xvf mpich-3.3.1.tar.gz >> cd mpich-3.3.1 >> ./configure --prefix=$MPICH_HOME >> >> make >> Failed: >> clang-8: error: unsupported option '--whole-archive' >> clang-8: error: unsupported option '--no-whole-archive' >> clang-8: error: unknown argument: '-soname' >> clang-8: error: no such file or directory: 'libmpi.so.12' >> make[2]: *** [lib/libmpi.la <http://libmpi.la/>] Error 1 >> make[2]: Leaving directory `/opt/modules/src/mpich-3.3.1' >> make[1]: *** [all-recursive] Error 1 >> make[1]: Leaving directory `/opt/modules/src/mpich-3.3.1' >> make: *** [all] Error 2 >> Any idea how to sort this out? >> Regards, >> J >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> <m.txt> > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Thomas Jahns HPC-Gruppe Abteilung Anwendungssoftware Deutsches Klimarechenzentrum GmbH Bundesstraße 45a • D-20146 Hamburg • Germany Phone: +49 40 460094-151 Fax: +49 40 460094-270 Email: Thomas Jahns <jahns(a)dkrz.de> URL: www.dkrz.de Geschäftsführer: Prof. Dr. Thomas Ludwig Sitz der Gesellschaft: Hamburg Amtsgericht Hamburg HRB 39784

1 0