Darshan-users

Download

[email protected]

February 2019

2 participants
2 discussions

About HPE MPI (MPT) support in Darshan (MPT ERROR: PMI2_Init)
by pramod kumbhar 12 Aug '19

12 Aug '19

Dear All, I was trying to use Darshan (3.1.6) with HPE MPI (MPT) <https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-a00037728en_us…> on our system and seeing below error: LD_PRELOAD=/gpfs/some_path/lib/libdarshan.so srun -n 1 /gpfs/some_another_path/bin/ior -a MPIIO -b 1G -t 4M -c -i 3 MPT ERROR: PMI2_Init With other MPI (e.g. intel-mpi), everything is working fine. Do you have any suggestions / workaround? Please let me know and I could help to debug/test the issue. Regards, Pramod

3 5

Extracting wallclock time from darshan-parser?
by Cox, Samuel (Dr.) 22 Feb '19

22 Feb '19

Hi, I’m running Darshan 3.1.6. Is there any way to extract the wallclock time of operations from darshan-parser? Let’s say I have sample program running on 16 MPI threads: part of it uses serial I/O, and part of it MPI-IO. > time mpirun -np 16 -host localhost ./benchio.x I then analyse the results with darshan-parser --total and filter out certain lines: > /cm/shared/apps/darshan/intel/3.1.6/bin/darshan-parser --total ~/dc-cox1_benchio.x_id15025_2-18-43921-17740490755319864674_1.darshan | grep -i "nprocs:\|run time\|POSIX_F_READ_TIME\|POSIX_F_WRITE_TIME\|POSIX_F_META_TIME\|MPIIO_F_READ_TIME\|MPIIO_F_WRITE_TIME\|MPIIO_F_META_TIME" # nprocs: 16 # run time: 83 total_POSIX_F_READ_TIME: 0.000000 total_POSIX_F_WRITE_TIME: 57.435211 total_POSIX_F_META_TIME: 8.595520 total_MPIIO_F_READ_TIME: 0.000000 total_MPIIO_F_WRITE_TIME: 471.718718 total_MPIIO_F_META_TIME: 5.261146 I also analyse with darshan-parser --base --perf --file-list and filter similarly: > /cm/shared/apps/darshan/intel/3.1.6/bin/darshan-parser --base --perf --file-list ~/dc-cox1_benchio.x_id15025_2-18-43921-17740490755319864674_1.darshan | grep -i "nprocs:\|run time\|POSIX_F_READ_TIME\|POSIX_F_WRITE_TIME\|POSIX_F_META_TIME\|MPIIO_F_READ_TIME\|MPIIO_F_WRITE_TIME\|MPIIO_F_META_TIME" # nprocs: 16 # run time: 83 POSIX 0 14271214572185876165 POSIX_F_READ_TIME 0.000000 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/serial.dat /lustre/dirac lustre POSIX 0 14271214572185876165 POSIX_F_WRITE_TIME 39.276790 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/serial.dat /lustre/dirac lustre POSIX 0 14271214572185876165 POSIX_F_META_TIME 4.394738 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/serial.dat /lustre/dirac lustre POSIX -1 4647831539574040701 POSIX_F_READ_TIME 0.000000 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/mpiio.dat /lustre/dirac lustre POSIX -1 4647831539574040701 POSIX_F_WRITE_TIME 18.158421 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/mpiio.dat /lustre/dirac lustre POSIX -1 4647831539574040701 POSIX_F_META_TIME 4.200782 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/mpiio.dat /lustre/dirac lustre MPI-IO -1 4647831539574040701 MPIIO_F_READ_TIME 0.000000 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/mpiio.dat /lustre/dirac lustre MPI-IO -1 4647831539574040701 MPIIO_F_WRITE_TIME 471.718718 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/mpiio.dat /lustre/dirac lustre MPI-IO -1 4647831539574040701 MPIIO_F_META_TIME 5.261146 /lustre/dirac/home/dc-cox1/benchio/shared-file/source/benchio_files/mpiio.dat /lustre/dirac lustre Comparing the two outputs, it’s clear that it’s CPU time being measured. But as a high-level profiling aid I’d like to extract the associated wallclock time instead. I think this would be 39.276+(18.15/16) and 4.39+(4.2/16) for the POSIX_F_WRITE/META_TIME. I’d rather not have to write a separate script to filter against the rank column from the --base --perf --file-list output. Thanks in advance, Sam

2 1