March 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPI I/O vs Sequential I/O
by Rajeev Thakur 20 Mar '13

20 Mar '13

fwrite() does buffered I/O, which will perform better for small writes. A better comparison would be to use write(), which is what MPI-IO uses internally. Rajeev On Mar 18, 2013, at 3:49 PM, Muhammad Zulfikar Handana wrote: > hello, i have some questions about MPI-I/O. > > 1) when i trying execute my program using MPI-I/O. why when iam execute > with 1 proses, execute time more good than when iam execute with 2 proses. > am i wrong? this is my source code. > MPI_File_open (MPI_COMM_WORLD, file_akhir, MPI_MODE_CREATE | > MPI_MODE_WRONLY, MPI_INFO_NULL, &hasil); > int jumlah1 = (((block-1)*16)+pad); > int jumlah = jumlah1 / size; > > > MPI_File_set_view(hasil, rank * jumlah * > sizeof(char),MPI_CHAR, MPI_CHAR, "native", MPI_INFO_NULL); > MPI_File_write (hasil, output, jumlah1, MPI_CHAR, > MPI_STATUS_IGNORE); > > MPI_File_close(&hasil); > > 2) and after that i have compared that source using MPI - I/O and with > sequential I/O, apparently, sequential i / o is better than the parallel i > / o. how it could be happen? this is my sequential source. > if (rank==0) { > > hasil = fopen (file_akhir, "wb"); > int test = (((block-1)*16)+pad); > fwrite (output, sizeof(char), test, hasil ); > fclose (hasil); > } > > thanks a lot for your attention _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI I/O vs Sequential I/O
by Rob Latham 20 Mar '13

20 Mar '13

On Tue, Mar 19, 2013 at 03:49:09AM +0700, Muhammad Zulfikar Handana wrote: > 1) when i trying execute my program using MPI-I/O. why when iam execute > with 1 proses, execute time more good than when iam execute with 2 proses. > am i wrong? this is my source code. In general, with one process doing i/o the file system can cache the heck out of requests. When a second process comes along, the caching needs to be disabled or invalidated, causing a "performance crash" that only recovers at much higher levels of parallelism. Let's look specifically at your code, though: MPI_File_open (MPI_COMM_WORLD, file_akhir, MPI_MODE_CREATE | MPI_MODE_WRONLY, MPI_INFO_NULL, &hasil); int jumlah1 = (((block-1)*16)+pad); int jumlah = jumlah1 / size; MPI_File_set_view(hasil, rank * jumlah * sizeof(char), MPI_CHAR, MPI_CHAR, "native", MPI_INFO_NULL); MPI_File_write (hasil, output, jumlah1, MPI_CHAR, MPI_STATUS_IGNORE); MPI_File_close(&hasil); - Are you writing a large amount of data or a small amount of data here? - What file system are you using? -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-discuss] running out of communicators
by Ryan Crocker 19 Mar '13

19 Mar '13

I just checked, i get this error in serial too. Also, it is after a long run, 10-20 thousand iterations. I ran with Valgrind and i didn't see a problem there, but i'm re-running it with gdb to see what shows up when it actually crashes. -Ryan On Mar 19, 2013, at 8:18 PM, Jim Dinan wrote: > I don't recall the exact limit on the number of communicators, but is certainly much more than 21. Any chance that you're forgetting to close one of those files somewhere? Can you attach a debugger to confirm that the error is happening in MPI_File_open? > > ~Jim. > > On 3/19/13 8:50 PM, Ryan Crocker wrote: >> I'm opening 21 files and i do have the close calls after they are finished being written. >> >> On Mar 19, 2013, at 6:29 PM, Jim Dinan wrote: >> >>> Hi Ryan, >>> >>> MPI_File_open does call MPI_Comm_dup internally. How many files are you opening? And are you closing them when you're finished with them? >>> >>> ~Jim. >>> >>> On 3/19/13 8:20 PM, Ryan Crocker wrote: >>>> Hi Jim, >>>> >>>> That's the thing, i'm not sure where they are being created. I initialize my MPI environment, then that program never gets called again. The only think i can think of is that it's something to do with my MPI IO, but after every open file, i just double checked, there is a corresponding close file call. I also do not call MPI_Comm_dup, unless that is called by another mpi call that i am not aware of. >>>> >>>> -Ryan >>>> >>>> On Mar 19, 2013, at 6:13 PM, Jim Dinan wrote: >>>> >>>>> Hi Ryan, >>>>> >>>>> Every time you call MPI_Comm_dup a new communicator is created. Are you ever freeing these, using MPI_Comm_free? >>>>> >>>>> Also, what are you trying to achieve by using multiple communicators, and why does it require so many? >>>>> >>>>> Best, >>>>> ~Jim. >>>>> >>>>> On 3/19/13 7:57 PM, Ryan Crocker wrote: >>>>>> I realized i forgot to attach the error: >>>>>> >>>>>> Fatal error in PMPI_Comm_dup: Other MPI error, error stack: >>>>>> PMPI_Comm_dup(176)............: MPI_Comm_dup(comm=0x84000000, new_comm=0x7fff5fbfe9a4) failed >>>>>> PMPI_Comm_dup(161)............: >>>>>> MPIR_Comm_dup_impl(55)........: >>>>>> MPIR_Comm_copy(967)...........: >>>>>> MPIR_Get_contextid(521).......: >>>>>> MPIR_Get_contextid_sparse(752): Too many communicators >>>>>> >>>>>> Hi all, >>>>>> >>>>>> So i can't seem to find the answer to this question. I keep getting the failure "Too many communicators". Could someone explain what calls "use up" communicators, and could i be calling/using more than the default total?. I'm trying to debug my code so i can free up the communicators and not just repeatedly create them. I'm a bit perplexed with this, probably a lack of in depth knowledge, but i only run any of my MPI initializations one time then all my calls with mpi are either sums, max, min, allreduce, alltoall. >>>>>> >>>>>> thanks >>>>>> >>>>>> Ryan Crocker >>>>>> University of Vermont, School of Engineering >>>>>> Mechanical Engineering Department >>>>>> rcrocker(a)uvm.edu >>>>>> 315-212-7331 >>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> Ryan Crocker >>>> University of Vermont, School of Engineering >>>> Mechanical Engineering Department >>>> rcrocker(a)uvm.edu >>>> 315-212-7331 >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> Ryan Crocker >> University of Vermont, School of Engineering >> Mechanical Engineering Department >> rcrocker(a)uvm.edu >> 315-212-7331 >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department rcrocker(a)uvm.edu 315-212-7331

1 0

Re: [mpich-discuss] running out of communicators
by Jim Dinan 19 Mar '13

19 Mar '13

I don't recall the exact limit on the number of communicators, but is certainly much more than 21. Any chance that you're forgetting to close one of those files somewhere? Can you attach a debugger to confirm that the error is happening in MPI_File_open? ~Jim. On 3/19/13 8:50 PM, Ryan Crocker wrote: > I'm opening 21 files and i do have the close calls after they are finished being written. > > On Mar 19, 2013, at 6:29 PM, Jim Dinan wrote: > >> Hi Ryan, >> >> MPI_File_open does call MPI_Comm_dup internally. How many files are you opening? And are you closing them when you're finished with them? >> >> ~Jim. >> >> On 3/19/13 8:20 PM, Ryan Crocker wrote: >>> Hi Jim, >>> >>> That's the thing, i'm not sure where they are being created. I initialize my MPI environment, then that program never gets called again. The only think i can think of is that it's something to do with my MPI IO, but after every open file, i just double checked, there is a corresponding close file call. I also do not call MPI_Comm_dup, unless that is called by another mpi call that i am not aware of. >>> >>> -Ryan >>> >>> On Mar 19, 2013, at 6:13 PM, Jim Dinan wrote: >>> >>>> Hi Ryan, >>>> >>>> Every time you call MPI_Comm_dup a new communicator is created. Are you ever freeing these, using MPI_Comm_free? >>>> >>>> Also, what are you trying to achieve by using multiple communicators, and why does it require so many? >>>> >>>> Best, >>>> ~Jim. >>>> >>>> On 3/19/13 7:57 PM, Ryan Crocker wrote: >>>>> I realized i forgot to attach the error: >>>>> >>>>> Fatal error in PMPI_Comm_dup: Other MPI error, error stack: >>>>> PMPI_Comm_dup(176)............: MPI_Comm_dup(comm=0x84000000, new_comm=0x7fff5fbfe9a4) failed >>>>> PMPI_Comm_dup(161)............: >>>>> MPIR_Comm_dup_impl(55)........: >>>>> MPIR_Comm_copy(967)...........: >>>>> MPIR_Get_contextid(521).......: >>>>> MPIR_Get_contextid_sparse(752): Too many communicators >>>>> >>>>> Hi all, >>>>> >>>>> So i can't seem to find the answer to this question. I keep getting the failure "Too many communicators". Could someone explain what calls "use up" communicators, and could i be calling/using more than the default total?. I'm trying to debug my code so i can free up the communicators and not just repeatedly create them. I'm a bit perplexed with this, probably a lack of in depth knowledge, but i only run any of my MPI initializations one time then all my calls with mpi are either sums, max, min, allreduce, alltoall. >>>>> >>>>> thanks >>>>> >>>>> Ryan Crocker >>>>> University of Vermont, School of Engineering >>>>> Mechanical Engineering Department >>>>> rcrocker(a)uvm.edu >>>>> 315-212-7331 >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> Ryan Crocker >>> University of Vermont, School of Engineering >>> Mechanical Engineering Department >>> rcrocker(a)uvm.edu >>> 315-212-7331 >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > Ryan Crocker > University of Vermont, School of Engineering > Mechanical Engineering Department > rcrocker(a)uvm.edu > 315-212-7331 > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] running out of communicators
by Ryan Crocker 19 Mar '13

19 Mar '13

I'm opening 21 files and i do have the close calls after they are finished being written. On Mar 19, 2013, at 6:29 PM, Jim Dinan wrote: > Hi Ryan, > > MPI_File_open does call MPI_Comm_dup internally. How many files are you opening? And are you closing them when you're finished with them? > > ~Jim. > > On 3/19/13 8:20 PM, Ryan Crocker wrote: >> Hi Jim, >> >> That's the thing, i'm not sure where they are being created. I initialize my MPI environment, then that program never gets called again. The only think i can think of is that it's something to do with my MPI IO, but after every open file, i just double checked, there is a corresponding close file call. I also do not call MPI_Comm_dup, unless that is called by another mpi call that i am not aware of. >> >> -Ryan >> >> On Mar 19, 2013, at 6:13 PM, Jim Dinan wrote: >> >>> Hi Ryan, >>> >>> Every time you call MPI_Comm_dup a new communicator is created. Are you ever freeing these, using MPI_Comm_free? >>> >>> Also, what are you trying to achieve by using multiple communicators, and why does it require so many? >>> >>> Best, >>> ~Jim. >>> >>> On 3/19/13 7:57 PM, Ryan Crocker wrote: >>>> I realized i forgot to attach the error: >>>> >>>> Fatal error in PMPI_Comm_dup: Other MPI error, error stack: >>>> PMPI_Comm_dup(176)............: MPI_Comm_dup(comm=0x84000000, new_comm=0x7fff5fbfe9a4) failed >>>> PMPI_Comm_dup(161)............: >>>> MPIR_Comm_dup_impl(55)........: >>>> MPIR_Comm_copy(967)...........: >>>> MPIR_Get_contextid(521).......: >>>> MPIR_Get_contextid_sparse(752): Too many communicators >>>> >>>> Hi all, >>>> >>>> So i can't seem to find the answer to this question. I keep getting the failure "Too many communicators". Could someone explain what calls "use up" communicators, and could i be calling/using more than the default total?. I'm trying to debug my code so i can free up the communicators and not just repeatedly create them. I'm a bit perplexed with this, probably a lack of in depth knowledge, but i only run any of my MPI initializations one time then all my calls with mpi are either sums, max, min, allreduce, alltoall. >>>> >>>> thanks >>>> >>>> Ryan Crocker >>>> University of Vermont, School of Engineering >>>> Mechanical Engineering Department >>>> rcrocker(a)uvm.edu >>>> 315-212-7331 >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> Ryan Crocker >> University of Vermont, School of Engineering >> Mechanical Engineering Department >> rcrocker(a)uvm.edu >> 315-212-7331 >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department rcrocker(a)uvm.edu 315-212-7331

1 0

Re: [mpich-discuss] running out of communicators
by Jim Dinan 19 Mar '13

19 Mar '13

Hi Ryan, MPI_File_open does call MPI_Comm_dup internally. How many files are you opening? And are you closing them when you're finished with them? ~Jim. On 3/19/13 8:20 PM, Ryan Crocker wrote: > Hi Jim, > > That's the thing, i'm not sure where they are being created. I initialize my MPI environment, then that program never gets called again. The only think i can think of is that it's something to do with my MPI IO, but after every open file, i just double checked, there is a corresponding close file call. I also do not call MPI_Comm_dup, unless that is called by another mpi call that i am not aware of. > > -Ryan > > On Mar 19, 2013, at 6:13 PM, Jim Dinan wrote: > >> Hi Ryan, >> >> Every time you call MPI_Comm_dup a new communicator is created. Are you ever freeing these, using MPI_Comm_free? >> >> Also, what are you trying to achieve by using multiple communicators, and why does it require so many? >> >> Best, >> ~Jim. >> >> On 3/19/13 7:57 PM, Ryan Crocker wrote: >>> I realized i forgot to attach the error: >>> >>> Fatal error in PMPI_Comm_dup: Other MPI error, error stack: >>> PMPI_Comm_dup(176)............: MPI_Comm_dup(comm=0x84000000, new_comm=0x7fff5fbfe9a4) failed >>> PMPI_Comm_dup(161)............: >>> MPIR_Comm_dup_impl(55)........: >>> MPIR_Comm_copy(967)...........: >>> MPIR_Get_contextid(521).......: >>> MPIR_Get_contextid_sparse(752): Too many communicators >>> >>> Hi all, >>> >>> So i can't seem to find the answer to this question. I keep getting the failure "Too many communicators". Could someone explain what calls "use up" communicators, and could i be calling/using more than the default total?. I'm trying to debug my code so i can free up the communicators and not just repeatedly create them. I'm a bit perplexed with this, probably a lack of in depth knowledge, but i only run any of my MPI initializations one time then all my calls with mpi are either sums, max, min, allreduce, alltoall. >>> >>> thanks >>> >>> Ryan Crocker >>> University of Vermont, School of Engineering >>> Mechanical Engineering Department >>> rcrocker(a)uvm.edu >>> 315-212-7331 >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > Ryan Crocker > University of Vermont, School of Engineering > Mechanical Engineering Department > rcrocker(a)uvm.edu > 315-212-7331 > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] running out of communicators
by Ryan Crocker 19 Mar '13

19 Mar '13

Hi Jim, That's the thing, i'm not sure where they are being created. I initialize my MPI environment, then that program never gets called again. The only think i can think of is that it's something to do with my MPI IO, but after every open file, i just double checked, there is a corresponding close file call. I also do not call MPI_Comm_dup, unless that is called by another mpi call that i am not aware of. -Ryan On Mar 19, 2013, at 6:13 PM, Jim Dinan wrote: > Hi Ryan, > > Every time you call MPI_Comm_dup a new communicator is created. Are you ever freeing these, using MPI_Comm_free? > > Also, what are you trying to achieve by using multiple communicators, and why does it require so many? > > Best, > ~Jim. > > On 3/19/13 7:57 PM, Ryan Crocker wrote: >> I realized i forgot to attach the error: >> >> Fatal error in PMPI_Comm_dup: Other MPI error, error stack: >> PMPI_Comm_dup(176)............: MPI_Comm_dup(comm=0x84000000, new_comm=0x7fff5fbfe9a4) failed >> PMPI_Comm_dup(161)............: >> MPIR_Comm_dup_impl(55)........: >> MPIR_Comm_copy(967)...........: >> MPIR_Get_contextid(521).......: >> MPIR_Get_contextid_sparse(752): Too many communicators >> >> Hi all, >> >> So i can't seem to find the answer to this question. I keep getting the failure "Too many communicators". Could someone explain what calls "use up" communicators, and could i be calling/using more than the default total?. I'm trying to debug my code so i can free up the communicators and not just repeatedly create them. I'm a bit perplexed with this, probably a lack of in depth knowledge, but i only run any of my MPI initializations one time then all my calls with mpi are either sums, max, min, allreduce, alltoall. >> >> thanks >> >> Ryan Crocker >> University of Vermont, School of Engineering >> Mechanical Engineering Department >> rcrocker(a)uvm.edu >> 315-212-7331 >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department rcrocker(a)uvm.edu 315-212-7331

1 0

Re: [mpich-discuss] running out of communicators
by Jim Dinan 19 Mar '13

19 Mar '13

Hi Ryan, Every time you call MPI_Comm_dup a new communicator is created. Are you ever freeing these, using MPI_Comm_free? Also, what are you trying to achieve by using multiple communicators, and why does it require so many? Best, ~Jim. On 3/19/13 7:57 PM, Ryan Crocker wrote: > I realized i forgot to attach the error: > > Fatal error in PMPI_Comm_dup: Other MPI error, error stack: > PMPI_Comm_dup(176)............: MPI_Comm_dup(comm=0x84000000, new_comm=0x7fff5fbfe9a4) failed > PMPI_Comm_dup(161)............: > MPIR_Comm_dup_impl(55)........: > MPIR_Comm_copy(967)...........: > MPIR_Get_contextid(521).......: > MPIR_Get_contextid_sparse(752): Too many communicators > > Hi all, > > So i can't seem to find the answer to this question. I keep getting the failure "Too many communicators". Could someone explain what calls "use up" communicators, and could i be calling/using more than the default total?. I'm trying to debug my code so i can free up the communicators and not just repeatedly create them. I'm a bit perplexed with this, probably a lack of in depth knowledge, but i only run any of my MPI initializations one time then all my calls with mpi are either sums, max, min, allreduce, alltoall. > > thanks > > Ryan Crocker > University of Vermont, School of Engineering > Mechanical Engineering Department > rcrocker(a)uvm.edu > 315-212-7331 > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

running out of communicators
by Ryan Crocker 19 Mar '13

19 Mar '13

I realized i forgot to attach the error: Fatal error in PMPI_Comm_dup: Other MPI error, error stack: PMPI_Comm_dup(176)............: MPI_Comm_dup(comm=0x84000000, new_comm=0x7fff5fbfe9a4) failed PMPI_Comm_dup(161)............: MPIR_Comm_dup_impl(55)........: MPIR_Comm_copy(967)...........: MPIR_Get_contextid(521).......: MPIR_Get_contextid_sparse(752): Too many communicators Hi all, So i can't seem to find the answer to this question. I keep getting the failure "Too many communicators". Could someone explain what calls "use up" communicators, and could i be calling/using more than the default total?. I'm trying to debug my code so i can free up the communicators and not just repeatedly create them. I'm a bit perplexed with this, probably a lack of in depth knowledge, but i only run any of my MPI initializations one time then all my calls with mpi are either sums, max, min, allreduce, alltoall. thanks Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department rcrocker(a)uvm.edu 315-212-7331

1 0

running out of communicators
by Ryan Crocker 19 Mar '13

19 Mar '13

Hi all, So i can't seem to find the answer to this question. I keep getting the failure "Too many communicators". Could someone explain what calls "use up" communicators, and could i be calling/using more than the default total?. I'm trying to debug my code so i can free up the communicators and not just repeatedly create them. I'm a bit perplexed with this, probably a lack of in depth knowledge, but i only run any of my MPI initializations one time then all my calls with mpi are either sums, max, min, allreduce, alltoall. thanks Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department rcrocker(a)uvm.edu 315-212-7331

1 0