- Darshan-users - mailman3.cels.anl.gov

Darshan 3.5.0 release is now available
by Wei-Keng Liao 20 Oct '25

20 Oct '25

1 0

announcement: new collection collection of logs available to the public
by Carns, Phil 06 Jun '25

06 Jun '25

The Darshan team is pleased to announce the public availability of the ALCF Polaris Darshan Log Repository<https://urldefense.us/v3/__https://doi.org/10.5281/zenodo.15052603__;!!G_uC… >. This is an anonymized, continuously updated collection of all production logs captured on the 560 node Polaris system<https://www.alcf.anl.gov/polaris > at the ALCF<https://www.alcf.anl.gov/ >. We hope that continuous publication of this data helps the computer science community to better understand current production workloads. As of this writing (June 5, 2025) the repository contains over 1.2 million log files and is growing at an average rate of roughly 3,000 logs per day, though coverage rates and job workloads vary considerably from day to day. See the Zenodo report<https://urldefense.us/v3/__https://doi.org/10.5281/zenodo.15052603__;!!G_uC… > for more information about how to download, analyze, and acknowledge use of the data in publications. See the CUG 2025 presentation<https://www.mcs.anl.gov/research/projects/darshan/wp-content/uploads/sites/… > (corresponding paper pending proceedings publication) for more information about how the repository was created, examples of analyses that can be performed on it, and links to tools that can be used to reproduce those examples. thanks, * Darshan team

1 0

Summary report of several darshan logfiles of a non-MPI job ?
by OGER Niels 30 May '25

30 May '25

Hello, I'm trying to use Darshan for the first time to get the I/O profil of several kinds of jobs (Fortran and Python). For the Fortran jobs I have no issue, I get 1 darshan logfile per execution and I can build a report with this command python -m darshan summary "log.darshan". For the Python jobs, I use export DARSHAN_ENABLE_NONMPI=1 before running the job. My Python script uses multiprocessing to read several files in parallel. I get one darshan logfile for each sub-process at the end of the execution. I have tried to use " python -m darshan summary " to get a report merging all the logfiles but did not manage to get the right syntax. I am wondering if what I want to do is even possible with Pydarshan. I tried using the "--include_names" argument but it seems to be a filter on the files accessed by the job and not on the log files themselves. Can someone confirm me if it's possible to 'merge' several logfiles in one report or not ? best regards, Niels -- ----- Météo-France ----- OGER NIELS DSI/D - Chef de projet Calcul Intensif niels.oger(a)meteo.fr Fixe : +33 561078198

2 1

Darshan 3.4.7 release is now available
by Snyder, Shane 13 May '25

13 May '25

Darshan users, Darshan version 3.4.7 is now officially available for download HERE<https://www.mcs.anl.gov/research/projects/darshan/download/ >. This point release most notably includes new modules enabling detailed instrumentation of DAOS storage systems (e.g., on ALCF Aurora): * There are 2 new modules that instrument the DAOS file system interface (DFS) and native object interfaces (DAOS). * Darshan’s heatmap module also now supports capture of I/O activity across both of these new DAOS modules. We have also released PyDarshan 3.4.7.0 on PyPI, which includes the following new capabilities: * New PyDarshan CLI tools for extracting statistics from one or more input log files: * job_stats: Overall job-level I/O statistics for each input Darshan log * file_stats: Overall file-level I/O statistics for application files accessed across each input Darshan log * Each tool offers command line options to control how jobs/files are sorted, statistics output format (CSV or Rich-style printing), Darshan module to analyze, etc. * Support for reading DAOS module data, as well as integrations of DAOS data into PyDarshan summary reports. * Support for PyDarshan ReportObject name filtering, enabling users to provide record name patterns to match when reading data from Darshan logs. * Available via filter_patterns and filter_mode arguments to various ReportObject routines: * filter_patterns: list of regexes to match * filter_mode: “exclude” or “include” records matching provided regexes Documentation for Darshan and PyDarshan is available HERE<https://www.mcs.anl.gov/research/projects/darshan/documentation/ >. Please report any questions, issues, or concerns with this release on our Slack instance, using this mailing list, or by opening an issue on our GitHub<https://urldefense.us/v3/__https://github.com/darshan-hpc/darshan__;!!G_uCf… >. Thanks! --Darshan team

1 0

Using Darshan with non-MPI applications (e.g., AI/ML frameworks)
by Snyder, Shane 12 Mar '25

12 Mar '25

Darshan users, We wanted to make you aware of a new article on the Darshan website describing how to use Darshan to instrument generic non-MPI applications, like AI/ML frameworks, workflow systems, etc. You can find the article here: https://www.mcs.anl.gov/research/projects/darshan/2025/03/11/using-darshan-… This has traditionally been a blind spot for Darshan, but recent improvements and experiences with Darshan have given us a better idea on how Darshan can provide instrumentation in these contexts. This article outlines some of the Darshan changes made to support this, as well as general best practice for using Darshan in generic non-MPI applications/frameworks. Please let us know if you have any questions, comments, or issues with this. Thanks! --Darshan team

1 0

Darshan trace file not created for WRF
by Wadud Miah 26 Nov '24

26 Nov '24

Hi, I am trying to profile the I/O of WRF and have the following settings: export DARSHAN_LOGPATH=/home/miahw/WRF_WADUD_PRE-COMPILED LD_PRELOAD=/home/miahw/openmpi-4.1-darshan-3.4.5/lib/libdarshan.so mpirun -n $SLURM_NTASKS /apps/ku/gcc-9_4/openmpi-4_1/wrf/4.0/WRF/run/wrf.exe However, no trace file gets created and the job stdout and stderr does not contain any Darshan warning/error messages. Can anyone suggest a way to increase debug level to investigate this further? Regards, Dr. Wadud Miah Scientific Computing Support Senior Specialist Scientific Computing Support [cid:[email protected]] wadud.miah(a)ku.ac.ae T : +971 2 312 5531 ku.ac.ae<https://urldefense.us/v3/__http://www.ku.ac.ae/__;!!G_uCfscf7eWS!aYrOJexVEU… > PO. Box 127788, Abu Dhabi, UAE Khalifa University [cid:[email protected]]<https://urldefense.us/v3/__http://www.ku.ac.ae/social__;!!G_uCfscf7eWS!aYrO… > "Disclaimer: This email and any attachments are intended solely for the use of the recipient(s) and may contain confidential or legally privileged information. If you are not the intended recipient, please notify the sender immediately and delete this email. Any unauthorized review, use, or distribution is prohibited. The views expressed in this email are those of the sender and may not reflect the official policies or position of Khalifa University of Science and Technology."

2 4

darshan_library_warning
by Zoya Masih 18 Nov '24

18 Nov '24

Hello, I have installed Darshan version 3.4.6 on an HPC cluster with the following configuration: ./configure --enable-mmap-logs --with-log-path=/mnt/lustre-grete/usr/u11509/DARSHAN/darshan-logs --with-jobid-env=PBS_JOBID --prefix=/path/to/DARSHAN/ CC=mpicc The installation of darshan-runtime and darshan-util is correctly done. Then I followed the instructions: *1- export **LD_PRELOAD=/**path/to**/lib/libdarshan.s**o* *2- export DARSHAN_ENABLE_NONMPI=1* *3- python Hello_world*.py I get this error: darshan_library_warning: unable to create log file /path/to/darshan-logs/2024/11/18/u11509_python_id1736997-1736997_11-18-34797-17835898535848440029.darshan_partial. The code is siply a hello world code, and python package doesn't have any problem. The permissions are also met. Would you please help me understand the issue? Sincerely Zoya Masih

2 1

Darshan 3.4.6 release is now available
by Snyder, Shane 17 Oct '24

17 Oct '24

Darshan-users, Darshan version 3.4.6 is now officially available for download HERE<https://urldefense.us/v3/__https://www.mcs.anl.gov/research/projects/darsha… >. This point release includes a couple of important new capabilities and bug fixes: * Added enhancements to Darshan's Lustre instrumentation module to capture more extensive details on Lustre striping configurations * Lustre file records now composed of potentially multiple component counter sets (e.g., LUSTRE_COMP1_*, LUSTRE_COMP2_*, etc.) * Allows full characterization of newer Lustre striping features, including progressive file layouts , data-on-metadata, file-level redundancy, and self-extending layouts * Fixed bugs in Darshan's log compression/decompression routines that are triggered when using the zlib-ng software package, a new implementation of the zlib compression library * darshan-runtime bug fix corrects problematic log compression strategy * darshan-util bug fix corrects logs already generated with this issue * Fixed bug leading to hangs when parsing improperly formatted Darshan runtime library configuration settings We have also released PyDarshan 3.4.6.0 on PyPI, though this is just to track the 3.4.6 darshan-util library. There are no modifications to PyDarshan functionality. Documentation for Darshan and PyDarshan is available HERE<https://urldefense.us/v3/__https://www.mcs.anl.gov/research/projects/darsha… >. Please report any questions, issues, or concerns with this release on our Slack instance, using this mailing list, or by opening an issue on our GitHub. Thanks! --Darshan team

1 0

trouble running pyDarshan on laptop
by Robinson, John-Paul 17 Jul '24

17 Jul '24

Hi I'm trying to use pyDardshan on my laptop to analyze logs from Polaris. I've successfully built and installed darshan-util and the pydarshan from a cloned repo. ./prepare.sh cd darshan-util/ ./configure --prefix ~/.local --enable-pydarshan make make install Followed by installing the module into my virtual environment. cd pydarshon pip install . When I import darshan and then try to run "report = darshan.DarshanReport(filename, read_all=True)" where filename is one of my Polaris darshan logs I get an error message about APMPI (one of the modules used to generate the darshan log). File ~/projects/presto/venv/lib/python3.10/site-packages/darshan/report.py:753, in DarshanReport.mod_read_all_apmpi_records(self, mod, dtype, warnings) 749 self.counters[mod] = {} 751 # fetch records 752 # fetch header record --> 753 rec = backend.log_get_apmpi_record(self.log, mod, "HEADER", dtype=dtype) 754 while rec != None: 755 self.records[mod].append(rec) AttributeError: module 'darshan.backend.cffi_backend' has no attribute 'log_get_apmpi_record' I assume this is because the module wasn't included during the darshan-util build. I'm trying to build this module into darshan-util and have added the autoperf submodule to the darshan repo. When I add --enable-apmpi-mod to the configure step above i get an error that it can't find the header. checking for GNU default python prefix... ${prefix} checking for GNU default python exec_prefix... ${exec_prefix} checking for python script directory (pythondir)... ${PYTHON_PREFIX}/lib/python3.10/site-packages checking for python extension module directory (pyexecdir)... ${PYTHON_EXEC_PREFIX}/lib/python3.10/site-packages checking for /home/jpr/projects/presto/darshan/darshan-util/../modules/autoperf/apmpi/darshan-apmpi-log-format.h... no configure: error: The autoperf MPI module is not present Looking at the config.log, it seems the test is just trying to confirm the include of the header using the same path, which does exist on my laptop. I'm a bit stuck on how to proceed. Any hints appreciated. Thanks.

2 4

darshan-3.4.5 is now available
by Snyder, Shane 03 May '24

03 May '24

Darshan users, Darshan version 3.4.5 is now officially available for download here: https://urldefense.us/v3/__https://www.mcs.anl.gov/research/projects/darsha… . This point release includes a couple of important new capabilities and bug fixes: * Added capability for Darshan's runtime library to properly shutdown in non-MPI applications that call _exit() directly * This behavior has been commonly observed in the Python multiprocessing package, which has traditionally prevented Darshan from properly instrumenting applications that use it (e.g., the PyTorch DataLoader) * Added optional integration with the LDMS data/metrics collection system, allowing realtime analysis of Darshan instrumented I/O operations * For more details, see https://urldefense.us/v3/__https://ovis-hpc.readthedocs.io/en/latest/ldms/l… * Contributed by Sara Walton and the LDMS team * Fixed bug preventing instrumentation of fscanf() calls on some systems * Fixed bug in HDF5 module causing any call to HDF5's H5Pset_fapl_mpio() routine to fail We have also released PyDarshan 3.4.5.0 on PyPI, though this is just to track the 3.4.5 darshan-util library. There are no modifications to PyDarshan functionality. Documentation for Darshan and PyDarshan is available here: https://urldefense.us/v3/__https://www.mcs.anl.gov/research/projects/darsha… . Please report any questions, issues, or concerns with this release using this mailing list or by opening an issue on our GitHub: https://urldefense.us/v3/__https://github.com/darshan-hpc/darshan__;!!G_uCf… . Thanks! --Darshan team

1 0