March 2013 - Darshan-users - mailman3.cels.anl.gov

how many compilers and MPI libraries can a single build of Darshan support?
by David Shrader 07 Mar '13

07 Mar '13

Hello All, I've been playing around a lot with Darshan the last couple of days and I am currently trying to find out how much a single build of Darshan can support in terms of other compilers and MPI libraries. The documentation has this statement if Darshan's CC is based on a GNU compiler, " Once Darshan has been installed, it should be capable of instrumenting executables built with GNU, Intel, and PGI compilers." I have found that not only will Darshan work with GNU, Intel, and PGI compilers, it seems to work with any version of these compilers (well, any versions that I have in production right now) as well as Pathscale compilers. Is this to be expected? For example, I have built Darshan with Open MPI 1.6.4 that was built using the GNU system compilers (v4.4.6). This single build of Darshan is able to operate on a binary that was built with Open MPI 1.6.4 based on the following compilers: gcc 4.6.1, gcc 4.7.0, gcc 4.7.2, intel 11.1.072, intel 12.1.2, intel 12.1.5, pgi 9.0-3, pgi 10.9, pgi 12.6, pgi 12.10 and pathscale 4.0. Meaning, I have builds of Open MPI 1.6.4 compatible with each of these compilers and I use them to build my application. No matter what version of Open MPI 1.6.4 I use to compile my application, I can get Darshan output from running it through libdarshan.so (that was built using Open MPI 1.6.4 based on the system GNU compilers). I have done a similar test with mvapich2 1.8. This is absolutely amazing to me and I would like to know if it is expected. A related question to that is what can a single Darshan build support in terms of MPI libraries? I have found that my Open MPI 1.6.4 build will not support mvapich2 compiled binaries and vice versa. But, I have found that my mvapich2 1.8 build supports mvapich2 1.7, 1.8 and 1.9a2 compiled binaries. Is this expected as well? Since only Open MPI 1.6.4 runs well with Darshan out of the box, I don't have a similar test there. In short, I am amazed at the field that a single build of Darshan seems to be able to support. I'm curious to know if there is any guidance on what the expected edges of that field are. Again, thank you for all of the help! David -- David Shrader SICORP, Inc 1350 Central Ave Suite 104 Los Alamos, NM 87544 David.Shrader(a)SICORP.com LANL contact information: LANL #: 505-664-0996 LANL email: dshrader(a)lanl.gov

2 1

DARSHAN and Open MPI
by David Shrader 06 Mar '13

06 Mar '13

Hello All, I have DARSHAN working with our mvapich2 1.8 GNU installation and am trying to get it to work with a GNU build of Open MPI 1.6.3. The building of DARSHAN completes without a problem when I point it to a mpicc that is owned by Open MPI 1.6.3, but when I try to run an application compiled with Open MPI 1.6.3, I get a seg fault. I know, from reading the documentation, that DARSHAN seems to work easier with MPICH derivatives; that being said, are there any known issues and/or workarounds with trying to use Open MPI? Right now I am using a TOSS2 machine with dynamic linking (I currently can't do static linking due to a missing static version of a system library). Thank you! David -- David Shrader SICORP, Inc 1350 Central Ave Suite 104 Los Alamos, NM 87544 David.Shrader(a)SICORP.com LANL contact information: LANL #: 505-664-0996 LANL email: dshrader(a)lanl.gov

2 2

guidance on using --with-mem-align
by David Shrader 05 Mar '13

05 Mar '13

Hello All, I am new to Darshan and am working on making it available on our clusters. I am having some trouble figuring out what to give the --with-mem-align configure parameter. I have read through the documentation where I see 8 and 16 used. Is this referring only to the minimum read and write size of physical memory on a node (not disk, but RAM)? Are there any resources for trying to figure out what this parameter should be set to for a particular system? My Google searches have come up with some research topics and programs for exposing the differences in timing for aligned vs non-aligned writes, but I'm not sure how to apply this information to what Darshan is looking for. For example, the recipe for IBM Blue Gene/Q sets it to 16 partly because "that is the optimal alignment for the network transport used between compute nodes and I/O nodes in the system." This statement makes it seem like not only is physical memory important, but also network transport. Does anyone have any resources or guidance on how to think about this parameter that they can point me to? Any and all help is appreciated! David -- David Shrader SICORP, Inc 1350 Central Ave Suite 104 Los Alamos, NM 87544 David.Shrader(a)SICORP.com LANL contact information: LANL #: 505-664-0996 LANL email: dshrader(a)lanl.gov

2 2