May 2020 - Darshan-users - mailman3.cels.anl.gov

darshan-job-summary.pl question
by Jeffrey Layton 20 May '20

20 May '20

Good morning, I really apologize for being a pest :) I've been able to profile an extremely simple C code and I'm ready to post-process. I wanted to experiment with darshan-job-summary.pl. When I run it, I get the following message: $ darshan-job-summary.pl /usr/local/darshan-logs/2020/5/19/laytonjb_ex1_id4630_5-19-28572-4707187774811817098_1589889373.darshan Can't locate Pod/LaTeX.pm in @INC (you may need to install the Pod::LaTeX module) (@INC contains: /usr/local/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/local/lib/TeX/Encode.pm line 10. BEGIN failed--compilation aborted at /usr/local/lib/TeX/Encode.pm line 10. Compilation failed in require at /usr/local/bin/darshan-job-summary.pl line 12. BEGIN failed--compilation aborted at /usr/local/bin/darshan-job-summary.pl line 12. I installed the dependencies for the pdf output: gnuplot, pdflatex, epstopdf (no errors during darshan-utils build and I can run these commands). Am I missing a Perl module? Thank you very much! Jeff

2 5

Cannot create log directory
by Jeffrey Layton 19 May '20

19 May '20

Good afternoon, I have a small problem when I build Darshan. It doesn't create the log directory, so I used the script darshan-mk-log-dirs.pl. However, I get an error: # ./darshan-mk-log-dirs.pl Error: could not mkdir /usr/local/darshan-logs//2020. This is an Ubuntu system I used sudo to run the script and I also tried as root. In either case, I get the same result. While I can follow the Perl script (it's been a few years since I last coded in Perl), I want to make sure I track down the cause of this error. Thanks! Jeff

2 4

Trouble building 3.2.0
by Jeffrey Layton 18 May '20

18 May '20

Good morning, I've been trying to build version 3.2.0 using the Ubuntu 18.04 gcc-7.5.0 compiler set along with mpich 3.3.2. My configuration command is, ./configure --with-log-path=/darshan-logs --with-jobid-env=NONE CC=mpicc --enable-mmap-logs --enable-group-readable-logs --without-mpi The configure seems to run fine, but when I use "make", I get the following errors: $ make mpicc -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I. -I./../ -g -O2 -D_LARGEFILE64_SOURCE -shared -fpic -DPIC -DDARSHAN_PRELOAD -c lib/darshan-core.c -o lib/darshan-core.po lib/darshan-core.c: In function ‘darshan_init_mmap_log’: lib/darshan-core.c:781:5: warning: implicit declaration of function ‘PMPI_Bcast’ [-Wimplicit-function-declaration] PMPI_Bcast(&logmod, 1, MPI_UINT64_T, 0, core->mpi_comm); ^~~~~~~~~~ lib/darshan-core.c:781:28: error: ‘MPI_UINT64_T’ undeclared (first use in this function); did you mean ‘__UINT64_C’? PMPI_Bcast(&logmod, 1, MPI_UINT64_T, 0, core->mpi_comm); ^~~~~~~~~~~~ __UINT64_C lib/darshan-core.c:781:28: note: each undeclared identifier is reported only once for each function it appears in lib/darshan-core.c:781:49: error: ‘struct darshan_core_runtime’ has no member named ‘mpi_comm’ PMPI_Bcast(&logmod, 1, MPI_UINT64_T, 0, core->mpi_comm); ^~ Makefile:120: recipe for target 'lib/darshan-core.po' failed make: *** [lib/darshan-core.po] Error 1 I've tried a few alternatives: - CC=gcc instead of CC=mpicc - not using the "--with-out-mpi" option and using CC=mpicc - Running "make clean" a few times I'm curious why, despite the "--without-mpi" option, it appears to want/need an MPI implementation (but I may be way off base here). Thanks! Jeff

3 4

Darshan 3.2.1 bugfix release available
by Snyder, Shane 15 May '20

15 May '20

Darshan users, Due to a reported bug in last week’s 3.2.0 release of Darshan, we have decided to quickly release Darshan 3.2.1 for our users. It is available for download here<ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-3.2.1.tar.gz>. This bugfix is somewhat critical, particularly in production environments, as it is can lead to corrupted Darshan log file data and, potentially, application crashes (though we have not triggered any crashes in our testing). The issue was originally detected by noticing bogus values in the COMMON_ACCESS counters reported by the POSIX, MPIIO, and H5 modules. In any case, we highly recommend any 3.2.0 users upgrade to this version to avoid any potential for crashes or corrupted Darshan log file data. Please report any additional questions, issues, or concerns using this mailing list, or by opening an issue on the Darshan GitLab page<https://xgitlab.cels.anl.gov/darshan/darshan>. Thanks! --Darshan team

1 0

trace files can not be created with Darshan
by Ayşe Arslan 12 May '20

12 May '20

Hello, I can not create trace files under darshan log directory, I built in that way, ---------------------------------------- DARSHAN_LOGPATH=/lustre/nec/ws2/ws/tuner/logs ./configure --with-mem-align=8 --with-log-path-by-env=DARSHAN_LOGPATH --with-jobid-env=NONE CC=mpicc --prefix=/lustre/nec/ws2/ws/tuner/darshan-3.1.7-ompi make make install --------------------------------------- #! /bin/bash #PBS -N a #PBS -l nodes=1:ppn=24 #PBS -l walltime=00:40:00 #PBS -o a.txt cd $PBS_O_WORKDIR module load compiler/gnu/8.3.0 module load mpi/openmpi/4.0.1-gnu-8.3.0 module load tools/hdf5/1.10.5-openmpi-4.0.1-gnu-8.3.0 export DARSHAN_LOGPATH=/lustre/nec/ws2/ws/tuner/logs cd /lustre/nec/ws2/tuner/benchmarks/Generators/mkTcTS LD_PRELOAD=/lustre/nec/ws2/ws/tuner/darshan-3.1.7-ompi/lib/libdarshan.so mpirun -n 24 ../../../src/MarDyn config.xml ------------------------------------------------------------- I have 2020/5/12 folders under log directory, configuration was successful, I got no error after running script, but there is no trace file (.darshan files) Any idea? Thanks, Ayse

2 1

Darshan version 3.2.0 is now officially available
by Snyder, Shane 04 May '20

04 May '20

Darshan-users, Darshan 3.2.0 is now available for download here<ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-3.2.0.tar.gz>. This release contains a number of new features, bug fixes, and other changes to Darshan. Some of the more notable changes that may be of interest to users: * Added detailed instrumentation of HDF5 file (H5F) and dataset (H5D) interfaces. * Must be explicitly enabled by passing “–enable-hdf5-mod=/path/to/hdf5/install” when configuring Darshan. * Due to ABI incompatibility from HDF5 version 1.8.x -> 1.10.x, special care must be taken to ensure users do not link applications with HDF5 versions that are incompatible with the version the Darshan library was built with (i.e., both HDF5 library versions must be either >=1.10 or <1.10). Using two incompatible HDF5 versions will lead to either link or runtime failures. * Support only intended for HDF5 versions 1.8.0+. * Added new feature allowing for instrumentation of non-MPI applications. * Darshan no longer strictly requires that instrumented applications use MPI, extending coverage to a breadth of new contexts. * Note that this feature is only functional in dynamic linking use cases. * Thanks to Glenn Lockwood (NERSC) for his help in implementing/testing this feature. * Added MPI-IO offset information to Darshan’s DXT tracing mechanism. * Updated Darshan compiler wrappers and Cray software modules to transparently and uniformly support dynamic and static linking cases. These methods previously only supported static linking uses cases. * Re-implemented Darshan’s PMPI/MPI wrappers to help avoid deadlock with other monitoring tools that rely on PMPI. * Added new “–log-path” option to darshan-config utility to allow users to more easily query the directory Darshan logs are stored in. Please review darshan-runtime<https://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html> and darshan-util<https://www.mcs.anl.gov/research/projects/darshan/docs/darshan-util.html> documentation for more details on the new HDF5 instrumentation module and the experimental non-MPI instrumentation mechanism. Additionally, consult the ChangeLog in the top-level of the source for a full list of changes associated with this release. Note that we are currently aware of and looking into a couple of issues related to Lustre file systems that have been reported by Darshan users: * Crashes in Darshan’s Lustre module in newer Lustre versions (2.11.x in one reported case). Typically results in additional errors stating: “using old ioctl(LL_IOC_LOV_GETSTRIPE)”. * If you experience this problem with Darshan, a temporary workaround would be to just disable the Lustre module — this can only be done at configure time by passing “–disable-lustre-mod”. * Floating point exceptions or other warnings related to dividing by zero when writing Darshan log to a Lustre file system (at Darshan shutdown time). * We are still working out what combinations of MPI and Lustre libraries exhibit this problem, but a simple workaround in the time being is to run the command “export DARSHAN_LOGHINTS=” before running your application. We hope to resolve these bugs quickly and intend to release an updated version of Darshan once they are. Please report any additional questions, issues, or concerns using this mailing list, or by opening an issue on the Darshan GitLab page<https://xgitlab.cels.anl.gov/darshan/darshan>. Thanks! --Darshan team

1 0