February 2017 - Darshan-users - mailman3.cels.anl.gov

Profiling Python C/C++ extensions with Darshan
by Ivan Raikov 06 Mar '17

06 Mar '17

Hello, I have a Python extension written in C++ which calls the HDF library, and I would like to use Darshan to profile an MPI-based Python application which invokes that extension. However, compiling the extension and running the application on Blue Waters (Cray-based system) with Darshan 2.3 results in no profile data being generated. Using LD_PRELOAD to load Darshan dynamically results in some kind of socket error when (I think) the MPI process manager is launching the Python executable. I have also tried the LD_PRELOAD mechanism with Darshan 3.1.3 built by the HDF group, but I get the same error. I was wondering if anyone here has done profiling with dynamically linked Python extensions, or if anyone has any advice how to approach this. I invoke the application with LD_PRELOAD thus: export LD_PRELOAD=/path/to/darshan/3.1.3/lib/libdarshan.so aprun -n 4096 python main.py ... or aprun -n 4096 -e LD_PRELOAD=/path/to/darshan/3.1.3/.../lib/libdarshan.so The error I get with LD_PRELOAD is the following: [PE_3344]:inet_listen_socket_setup:inet_setup_listen_socket: bind failed port 1371 listen_sock = 88 Address already in use [PE_3344]:_pmi_inet_listen_socket_setup:socket setup failed [PE_3344]:_pmi_init:_pmi_inet_listen_socket_setup (full) returned -1 [PE_208]:inet_listen_socket_setup:inet_setup_listen_socket: bind failed port 1371 listen_sock = 74 Address already in use [PE_208]:_pmi_inet_listen_socket_setup:socket setup failed [PE_208]:_pmi_init:_pmi_inet_listen_socket_setup (full) returned -1 Thanks and best regards,

3 4

Darshan-Ruby for Darshan 3.1.3
by Dorier, Matthieu 21 Feb '17

21 Feb '17

Hi all, Darshan-Ruby has been updated to match Darshan 3.1.3, including its new DXT module. https://xgitlab.cels.anl.gov/darshan/darshan-ruby Matthieu

1 0

DXT parser issues
by James Dickson 20 Feb '17

20 Feb '17

Hi, I'm having some issues parsing the output file generated with DXT enabled. When using the darshan-parser utility, I'm getting a seg fault upon reaching the DXT_POSIX module, which is coming from line 482 in the darshan-parser code according to gdb. However, if I use the darshan-dxt-parser it gets through the file no problem. Is this a problem with the parser or some quirk of my trace file? Many thanks, James

2 1

Darshan "I/O" pattern
by Wadud Miah 20 Feb '17

20 Feb '17

Hello, I have the attached "I/O pattern" bar chart from Darshan which I don't understand. My question is what do the "Sequential" and "Consecutive" labels mean? Any help will be greatly appreciated. Regards, Wadud. ----------------------------------- Dr. Wadud Miah Computational Scientist Numerical Algorithms Group 01865 518035 ________________________________ The Numerical Algorithms Group Ltd is a company registered in England and Wales with company number 1249803. The registered office is: Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. This e-mail has been scanned for all viruses by Microsoft Office 365. ________________________________

2 1

Re: [Darshan-users] Darshan-users Digest, Vol 64, Issue 3
by Jean Bez 15 Feb '17

15 Feb '17

Hi, > Are you actually performing any read or write operations to a file in > the application you are linking with Darshan? DXT only traces reads and > writes, so if you are just opening a file (for instance), it shouldn't > produce any DXT data. > Yes, but after looking at the application I was testing it used some functions (fwrite for instance) that were not traced by Darshan DxT, although they appeared on the default Darshan report. So I looked at the souce-code and those calls is not handled yet. If your application is doing reads/writes, there should be DXT data if > you enabled it properly. Did you run 'export DXT_ENABLE_IO_TRACE='? Note > that you need to include the trailing equal sign, even if you aren't > setting a value for the amount of memory to use. For example: > 'export DXT_ENABLE_IO_TRACE=' will give you the default 4MiB buffer per > process for tracing > 'export DXT_ENABLE_IO_TRACE=8' will instead allocate 8MiB buffers per > process > Yes, I set the variable via the MPICH -env option and now it worked ok for the MPI-IO requests. Thanks! Jean > > Thanks, > --Shane > > On 02/13/2017 11:03 AM, Jean Bez wrote: > > Hi! > > > > I was just trying to use the new DXT module and although I have set > > the environment variable (DXT_ENABLE_IO_TRACE) to activate it, when I > > attempt to parse the .darshan file with the darshan-dxt-parser I > > always get the message: > > > > "no DXT module data available for this Darshan log." > > > > Is there something else we need to do to enable DXT? > > > > Thanks! > > > > Best regards, > > Jean > > > > > > _______________________________________________ > > Darshan-users mailing list > > Darshan-users(a)lists.mcs.anl.gov > > https://lists.mcs.anl.gov/mailman/listinfo/darshan-users > >

2 1

Enable DXT
by Jean Bez 13 Feb '17

13 Feb '17

Hi! I was just trying to use the new DXT module and although I have set the environment variable (DXT_ENABLE_IO_TRACE) to activate it, when I attempt to parse the .darshan file with the darshan-dxt-parser I always get the message: "no DXT module data available for this Darshan log." Is there something else we need to do to enable DXT? Thanks! Best regards, Jean

2 1

Darshan pre-release 3.1.3-pre1 is now available
by Shane Snyder 13 Feb '17

13 Feb '17

Darshan users, A new Darshan pre-release (3.1.3-pre1) is now available for download here <ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-3.1.3-pre1.tar.gz>. Please be aware that this pre-release is experimental and *not* recommended for full-time use in production yet. We hope to have the official 3.1.3 release ready some time next week. Here is the list of new features and bug fixes in this release: * New DxT (Darshan eXtended Tracing) instrumentation modules that provide fine-grained tracing data of read/write operations at the POSIX & MPI-IO layers - This tracing module is not enabled by default, and it is envisioned that it be enabled selectively at runtime for applications of interest - This code was contributed by Cong Xu and Intel's High Performance Data Division (HPDD) * Darshan now captures command line arguments for Fortran applications - This fix was contributed by Cristian Simarro * Fix potential deadlock when using the Cray hugepages module and Darshan - Reported by Glenn Lockwood & Cristian Simarro * Fix potential segmentation fault when Darshan exceeds the maximum number of tracked access sizes/strides - Reported by Glenn Lockwood * Disable Darshan's HDF5 module by default to avoid potential ABI compatibility issues with HDF5 - New configure switches are provided to enable Darshan support for specific HDF5 library versions - Problem report and initial patch provided by Karl-Ulrich Bamberg * Add whitelist to Darshan's list of file path exclusions to prevent I/O to/from Cray's Datawarp service from being filtered out - Contributed by Glenn Lockwood Updated documentation related to using the DxT tracing modules can be found in the source tree ('darshan-runtime/docs' for the Darshan runtime library & darshan-util/docs for Darshan's utilities). To summarize, DxT tracing can be enabled at runtime by exporting the 'DXT_ENABLE_IO_TRACE' environment variable. DxT also has a corresponding parser utility named 'darshan-dxt-parser' that can be used to parse trace data out of Darshan log files. As always, let us know if you have any questions, issues, or concerns using this mailing list, or by opening an issue on the Darshan GitLab page <https://xgitlab.cels.anl.gov/darshan/darshan>. Thanks! --Darshan team

1 1