May 2014 - Darshan-users - mailman3.cels.anl.gov

Re: [Darshan-users] darshan-runtime.pc
by Phil Carns 28 May '14

28 May '14

The new release is out now. JG, you can continue using 2.2.9 with the fix to the configure script. I just wanted to put out a new release to avoid confusion for future users. thanks, -Phil On 05/27/2014 01:30 PM, Carns, Philip H. wrote: > Thanks JG. That's a mistake on my part; it looks like I failed to > update the version number in the darshan-runtime portion of the package > for the 2.2.9 release. The proper fix is to update the AC_INIT line at > the top of configure.in, run ./prepare, and then do the build/install as > usual. > > We may need to put out a quick point release to fix this. I'll look > into it. > > thanks, > -Phil > > On 05/27/2014 08:36 AM, jgp wrote: >> Hi, >> >> This is probably a very minor issue but >> darshan-runtime/lib/pkgconfig/darshan-runtime.pc >> from >> /apps/daint/darshan/src/WGET/darshan-2.2.9.tar.gz >> incorrectly sets >> Version: 2.2.9-pre2 >> instead of >> Version: 2.2.9 ? >> >> Regards, jg. >> _______________________________________________ >> Darshan-users mailing list >> Darshan-users(a)lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users > _______________________________________________ > Darshan-users mailing list > Darshan-users(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/darshan-users

1 0

Darshan 2.3.0 release
by Phil Carns 28 May '14

28 May '14

Darshan 2.3.0 is a minor update from Darshan 2.2.9. It fixes incorrect version numbering in the darshan-runtime component. Changelog: darshan-2.3.0 ============= * Fix incorrect version numbering in darshan-runtime component of Darshan 2.2.9, reported by Jean-Guillaume Piccinali thanks, - Darshan development team

1 0

file count summary
by jgp 28 May '14

28 May '14

Hi, I am reading 1 (103G) file with parallel-hdf5 using 8, 64 and 512 cores. Reports from darshan/229 (see attached) looks good for 8 and 64 cores: > 8cores * 13G/core ~= 103G: ok > 64cores * 1.6G/core ~= 103G: ok but > 512cores * 0.13G/core ~= 67G != 103G Does it mean only 67G are being read ? I am attaching the logs for reference. Regards, jg.

2 1

Re: [Darshan-users] darshan-runtime.pc
by Phil Carns 27 May '14

27 May '14

Thanks JG. That's a mistake on my part; it looks like I failed to update the version number in the darshan-runtime portion of the package for the 2.2.9 release. The proper fix is to update the AC_INIT line at the top of configure.in, run ./prepare, and then do the build/install as usual. We may need to put out a quick point release to fix this. I'll look into it. thanks, -Phil On 05/27/2014 08:36 AM, jgp wrote: > Hi, > > This is probably a very minor issue but > darshan-runtime/lib/pkgconfig/darshan-runtime.pc > from > /apps/daint/darshan/src/WGET/darshan-2.2.9.tar.gz > incorrectly sets > Version: 2.2.9-pre2 > instead of > Version: 2.2.9 ? > > Regards, jg. > _______________________________________________ > Darshan-users mailing list > Darshan-users(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/darshan-users

1 0

darshan-runtime.pc
by jgp 27 May '14

27 May '14

Hi, This is probably a very minor issue but darshan-runtime/lib/pkgconfig/darshan-runtime.pc from /apps/daint/darshan/src/WGET/darshan-2.2.9.tar.gz incorrectly sets Version: 2.2.9-pre2 instead of Version: 2.2.9 ? Regards, jg.

1 0

Graphics generated by darshan-job-summary.pl
by Thomas Bouguet 16 May '14

16 May '14

Hello, I use Darshan in the context of an internship. I generated pdf summaries (using darshan-job-summary.pl) and looked at the different graphics. I would like to understand more specifically which counters were used for each of these graphs ? Thank you, Thomas Bouguet .

2 1

Darshan 2.2.9 release
by Phil Carns 06 May '14

06 May '14

The official Darshan 2.2.9 release is now available on the web site. This is primarily a maintenance and bug fix release. It also includes support for versions 2.0 and newer of the Cray Programming Environment. The full change log since 2.2.8 is listed below: * Bug fixes: - Fix mnt table overflow if a large number of file systems are mounted,reported by David Shrader. - Fix argument parsing for darshan-convert, reported by Mouhamed Gueye. - Fix metadata annotation overflow in darshan-convert, reported byMouhamed Gueye. - Fix const-correctness in dynamic library when built against MPI 3.xlibraries. - Fix "undefined symbol: dlsym" error when using preloaded dynamic libraryon some platforms, reported by Florin Isaila. - Normalize timestamps to always be relative to MPI_Init(). - Better library name matching in compiler wrappers to handle more MPICH variations on Blue Gene systems. - Fixed header dependency problems when using darshan-logutils.h in externalprograms (Shane Snyder). * Enhancements: - Add support (both in documentation and in provided module files) forCray PE 2.x. - Honor CC variable to allow darshan-util to be built with other compilersbesides gcc. - Update documentation to clarify the interaction between rpath and LD_PRELOAD when instrumenting dynamic libraries, issue reported andinvestigated by Davide Del Vento. - Ability to disable shared-file reduction by setting the DARSHAN_DISABLE_SHARED_REDUCTION environment variable. - More thorough output from darshan-parser --perf, suggested by Huong Luu. - Increased metadata annotation room from 64 bytes to 1KiB in header. - CP_F_{FASTEST/SLOWEST}_RANK_TIME counters now take MPI-IO time intoaccount, not just POSIX time, issue reported by Huong Luu. - Better handling of systems with many mounted file systems (after whichpoint Darshan will assume file resides on / file system), issue reportedand investigated by David Shrader: - Track up to 64 rather than 32 mounted file systems at runtime. - Increase header space available for storing mount point information in log file from approximately 1 KiB to approximately 3 KiB. - Prioritize storing information about non-NFS volumes over NFS volumes if too many file systems are mounted to record them all. - Added darshan-util pkgconfig file (Shane Snyder). - Added --enable-shared configure option to darshan-util to build anda shared library version of libdarshan-util thanks, -Darshan development team

1 0

Darshan pre-release: 2.2.9-pre2
by Phil Carns 05 May '14

05 May '14

Darshan 2.2.9-pre2 is now available for download. The changes since pre1 are: - bug fixes to the Cray PE 2.x environment support (thanks to testing from Yushu Yao) - --enable-shared option for darshan-util (suggested by Matthieu Dorier) We'll be doing some cross-platform testing with 2.2.9-pre2 this week to try to wrap up a final release. thanks, -Phil On 04/14/2014 04:36 PM, Carns, Philip H. wrote: > I also forgot to mention that there are some documentation updates in > this release. The updated documentation is not available on the web > site, but you can find it in the darshan-util/doc/ and > darshan-runtime/doc/ subdirectories in the source code. > > thanks, > -Phil > > On 04/14/2014 04:34 PM, Carns, Philip H. wrote: >> There is a new experimental version of Darshan available for download >> today. It includes fixes for several bugs as well as support for >> versions 2.0 and newer of the Cray Programming Environment. The full >> change log is listed below. >> >> Feedback is always welcome! We will do more thorough cross-platform >> testing before the official 2.2.9 release; please just use this version >> for testing and evaluation. >> >> thanks! >> -Darshan team >> >> darshan-2.2.9-pre1 >> ============= >> * Bug fixes: >> - Fix mnt table overflow if a large number of file systems are mounted, >> reported by David Shrader. >> - Fix argument parsing for darshan-convert, reported by Mouhamed Gueye. >> - Fix metadata annotation overflow in darshan-convert, reported by >> Mouhamed Gueye. >> - Fix const-correctness in dynamic library when built against MPI 3.x >> libraries. >> - Fix "undefined symbol: dlsym" error when using preloaded dynamic >> library >> on some platforms, reported by Florin Isaila. >> - Normalize timestamps to always be relative to MPI_Init(). >> - Better library name matching in compiler wrappers to handle more MPICH >> variations on Blue Gene systems. >> - Fixed header dependency problems when using darshan-logutils.h in >> external >> programs (Shane Snyder). >> * Enhancements: >> - Add support (both in documentation and in provided module files) for >> Cray PE 2.x. >> - Honor CC variable to allow darshan-util to be built with other >> compilers >> besides gcc. >> - Update documentation to clarify the interaction between rpath and >> LD_PRELOAD when instrumenting dynamic libraries, issue reported and >> investigated by Davide Del Vento. >> - Ability to disable shared-file reduction by setting the >> DARSHAN_DISABLE_SHARED_REDUCTION environment variable. >> - More thorough output from darshan-parser --perf, suggested by Huong >> Luu. >> - Increased metadata annotation room from 64 bytes to 1KiB in header. >> - CP_F_{FASTEST/SLOWEST}_RANK_TIME counters now take MPI-IO time into >> account, not just POSIX time, issue reported by Huong Luu. >> - Better handling of systems with many mounted file systems (after which >> point Darshan will assume file resides on / file system), issue >> reported >> and investigated by David Shrader: >> - Track up to 64 rather than 32 mounted file systems at runtime. >> - Increase header space available for storing mount point >> information in >> log file from approximately 1 KiB to approximately 3 KiB. >> - Prioritize storing information about non-NFS volumes over NFS volumes >> if too many file systems are mounted to record them all. >> - Added darshan-util pkgconfig file (Shane Snyder). >> >> _______________________________________________ >> Darshan-users mailing list >> Darshan-users(a)lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users > _______________________________________________ > Darshan-users mailing list > Darshan-users(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/darshan-users

1 0

Re: [Darshan-users] Compiling Darshan with -fPIC
by Phil Carns 05 May '14

05 May '14

On 05/05/2014 03:09 PM, Carns, Philip H. wrote: > On 05/05/2014 09:00 AM, Matthieu Dorier wrote: >> Hi, >> >> While installing Darshan-Ruby on a new plateforme (Debian with Linux >> kernel 3.2.0 amd64 and gcc 4.7), I encountered a compilation error, >> basically asking me to recompile Darshan with -fPIC. >> I tried to use CFLAGS=-fPIC but, I don't know why, it didn't work. I >> ended up changing the Makefile.in of Darshan, line 59 and 60, to the >> following: >> >> >> darshan-logutils.o: darshan-logutils.c >> $(CC) $(CFLAGS) -fPIC -c $< -o $@ >> >> >> Maybe it could be useful to add the options of compiling Darshan with >> or without PIC (with PIC enabled by default), or even some kind of >> --enable-shared option to produce a libdarshan-util.so? >> >> Best, >> >> Matthieu Dorier >> PhD student at ENS Rennes >> http://people.irisa.fr/Matthieu.Dorier > > Thanks for the bug report Matthieu. Which version of Darshan was this? > > Is the PIC option required for Darshan-Ruby to use the libdarshan-util > library correctly, or did you need to add it for libdarshan-util to > build cleanly at all on this platform? > > I'll definitely look into this further. At a minimum we should fix > the CFLAGS issue so that you can pass in custom flags. > I wasn't able to reproduce the CFLAGS problem, but that may be something that has changed in trunk (I didn't look back at previous releases). The --enable-shared option is pretty straightforward, though, so I went ahead and added that. It will be available in 2.2.9-pre2 and later. thanks, -Phil

1 0

Re: [Darshan-users] Compiling Darshan with -fPIC
by Phil Carns 05 May '14

05 May '14

On 05/05/2014 09:00 AM, Matthieu Dorier wrote: > Hi, > > While installing Darshan-Ruby on a new plateforme (Debian with Linux > kernel 3.2.0 amd64 and gcc 4.7), I encountered a compilation error, > basically asking me to recompile Darshan with -fPIC. > I tried to use CFLAGS=-fPIC but, I don't know why, it didn't work. I > ended up changing the Makefile.in of Darshan, line 59 and 60, to the > following: > > > darshan-logutils.o: darshan-logutils.c > $(CC) $(CFLAGS) -fPIC -c $< -o $@ > > > Maybe it could be useful to add the options of compiling Darshan with > or without PIC (with PIC enabled by default), or even some kind of > --enable-shared option to produce a libdarshan-util.so? > > Best, > > Matthieu Dorier > PhD student at ENS Rennes > http://people.irisa.fr/Matthieu.Dorier Thanks for the bug report Matthieu. Which version of Darshan was this? Is the PIC option required for Darshan-Ruby to use the libdarshan-util library correctly, or did you need to add it for libdarshan-util to build cleanly at all on this platform? I'll definitely look into this further. At a minimum we should fix the CFLAGS issue so that you can pass in custom flags. thanks, -Phil

1 0