July 2014 - Darshan-users - mailman3.cels.anl.gov

Re: [Darshan-users] Number of non-consecutive or non-sequential reads
by Phil Carns 23 Jul '14

23 Jul '14

Hi Matthieu, Those CONSEC and SEQ counters refer to POSIX operations. The total number of POSIX operations can be found by adding CP_POSIX_READS and CP_POSIX_FREADS. The SPLIT/COLL/NB/INDEP counters refer to MPI-IO operations, which would be tricky to directly relate to the POSIX CONSEC and SEQ fields. Sorry for the confusing naming scheme here; its hard to tell which things are POSIX vs MPI-IO on some of these counters. -Phil On 07/22/2014 07:47 AM, Matthieu Dorier wrote: > Hi, > > I see that Darshan has CP_CONSEC_READS and CP_SEQ_READS counters to > count the number of consecutive and sequential reads. > I would like to compute the number of non-consecutive and > non-sequential reads but it is unclear to me which counters I should > use to compute the total number of reads from which to substract the > number of consecutive/sequential. For instance are CP_SPLIT_READS > already counted in CP_COLL_READS? Are CP_NB_READS already counted in > CP_INDEP_READS? etc. > > Can you help? > Thanks! > > Matthieu Dorier > PhD student at ENS Rennes > http://people.irisa.fr/Matthieu.Dorier

1 0

dynamic/static linking
by Salmon, Rene 22 Jul '14

22 Jul '14

Hi, I am trying to link in darshan to my simple HDF5 application to gather some performance data. I am able to gather some data but not all. Most of my graphs are empty and that seems to be because darshan is not collecting some of the data to be able to generate the graphs. I might be missing somethings and I am hoping I can get a hint as to what. My application uses the intel compilers and it statically links in the intel MPI libraries as well as the HDF5 Libraries but the rest is left dynamic so not a fully static executable as you can see. ldd h5testcode linux-vdso.so.1 => (0x00007fff9939c000) libz.so.1 => /lib64/libz.so.1 (0x00002acc4dafc000) libdl.so.2 => /lib64/libdl.so.2 (0x00002acc4dd12000) libm.so.6 => /lib64/libm.so.6 (0x00002acc4df16000) librt.so.1 => /lib64/librt.so.1 (0x00002acc4e190000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002acc4e399000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002acc4e5b6000) libc.so.6 => /lib64/libc.so.6 (0x00002acc4e7cd000) /lib64/ld-linux-x86-64.so.2 (0x00002acc4d8db000) nm h5testcode | grep darsha 0000000000721a90 T darshan_condense 00000000007233d0 t darshan_file_by_fd 0000000000724aa0 T darshan_file_by_handle 000000000071a110 T darshan_file_by_name 0000000000723280 t darshan_file_by_name_setfd 0000000000724950 T darshan_file_by_name_sethandle 0000000000724b40 T darshan_file_closehandle 0000000000459e70 t darshan_file_reduce 0000000000724940 T darshan_finalize 000000000045afd0 T darshan_get_exe_and_mounts 0000000000d435f0 B darshan_global_job 0000000000724ff0 T darshan_hash 0000000000724c40 T darshan_initialize 0000000000d1e7c4 d darshan_mem_alignment 000000000045b7c0 T darshan_mnt_id_from_path 000000000045b890 T darshan_mpi_initialize 0000000000723460 T darshan_search_bench 0000000000451a10 T darshan_shutdown 0000000000721d80 T darshan_shutdown_bench 000000000045ae50 T darshan_walk_file_accesses The little application runs and generates a log file and I can process it to generate a PDF. But the pdf is mostly empty. If I run darshan-parser to see the data I can see that many of the values are zero. darshan-parser filename.gz > log.txt grep BYTES log.txt # CP_BYTES_*: total bytes read and written. # CP_*_RANK_BYTES: fastest, slowest and variance of bytes transfer. -1 753191520991278725 CP_BYTES_READ 0 ...mr0/SDS_chnk.h5 /lustre02 lustre -1 753191520991278725 CP_BYTES_WRITTEN 0 ...mr0/SDS_chnk.h5 /lustre02 lustre -1 753191520991278725 CP_FASTEST_RANK_BYTES 0 ...mr0/SDS_chnk.h5 /lustre02 lustre -1 753191520991278725 CP_SLOWEST_RANK_BYTES 0 ...mr0/SDS_chnk.h5 /lustre02 lustre -1 753191520991278725 CP_F_VARIANCE_RANK_BYTES 0.000000 ...mr0/SDS_chnk.h5 /lustre02 lustre I know I am writting data and the application works. I have a valid HDF5 file with data after the application is done. So it can’t be writing zero bytes. I am attaching the compressed log file if that helps. Thanks in advanced for any help on this. Rene

2 2

Number of non-consecutive or non-sequential reads
by Matthieu Dorier 22 Jul '14

22 Jul '14

Hi, I see that Darshan has CP_CONSEC_READS and CP_SEQ_READS counters to count the number of consecutive and sequential reads. I would like to compute the number of non-consecutive and non-sequential reads but it is unclear to me which counters I should use to compute the total number of reads from which to substract the number of consecutive/sequential. For instance are CP_SPLIT_READS already counted in CP_COLL_READS? Are CP_NB_READS already counted in CP_INDEP_READS? etc. Can you help? Thanks! Matthieu Dorier PhD student at ENS Rennes http://people.irisa.fr/Matthieu.Dorier

1 0

Re: [Darshan-users] error in darshan-job-summary.pl
by Phil Carns 11 Jul '14

11 Jul '14

Thanks Matthieu! I just committed the fix to trunk so that it will be included in the next release. -Phil On 07/11/2014 09:57 AM, Matthieu Dorier wrote: > Hi, > > My intern spotted a potential error in darshan-job-summary.pl. Line > 732 you have > $cumul_write_bytes_shared + $cumul_write_bytes_shared > I suppose it should be > $cumul_write_bytes_shared + $cumul_write_bytes_indep > > Best, > > Matthieu Dorier > PhD student at ENS Rennes > http://people.irisa.fr/Matthieu.Dorier

1 0

error in darshan-job-summary.pl
by Matthieu Dorier 11 Jul '14

11 Jul '14

Hi, My intern spotted a potential error in darshan-job-summary.pl. Line 732 you have $cumul_write_bytes_shared + $cumul_write_bytes_shared I suppose it should be $cumul_write_bytes_shared + $cumul_write_bytes_indep Best, Matthieu Dorier PhD student at ENS Rennes http://people.irisa.fr/Matthieu.Dorier

1 0