So MPICH doesn't fail when I try this on my Mac, but it would be good
to know if I have a reasonable expectation of doing this in general.
Thanks,
Jeff
jrhammon-mac01:ENDPOINTS jrhammon$ mpiexec -n 4 ./a.out
PMI says I am 2 of 4
PMI says I am 0 of 4
PMI says I am 1 of 4
PMI says I am 3 of 4
MPI says I am 0 of 4 (provided=1)
MPI says I am 2 of 4 (provided=1)
MPI says I am 3 of 4 (provided=3)
MPI says I am 1 of 4 (provided=3)
jrhammon-mac01:ENDPOINTS jrhammon$ cat pmi.c
#include <stdio.h>
#include <stdlib.h>
/*#include <pmi.h>*/
int PMI_Init( int *spawned );
int PMI_Finalize( void );
int PMI_Get_size( int *size );
int PMI_Get_rank( int *rank );
#include <mpi.h>
int main(int argc, char * argv[])
{
int has_parent;
PMI_Init(&has_parent);
int rank, size;
PMI_Get_size(&size);
PMI_Get_rank(&rank);
printf("PMI says I am %d of %d \n", rank, size);
fflush(stdout);
int requested = (rank%2==0) ? MPI_THREAD_FUNNELED : MPI_THREAD_MULTIPLE;
int provided;
MPI_Init_thread(&argc, &argv, requested, &provided);
int wsize, wrank;
MPI_Comm_size(MPI_COMM_WORLD, &wsize);
MPI_Comm_rank(MPI_COMM_WORLD, &wrank);
printf("MPI says I am %d of %d (provided=%d)\n", wrank, wsize, provided);
fflush(stdout);
MPI_Finalize();
//PMI_Finalize(); /* required? */
return 0;
}
On Tue, Apr 21, 2015 at 9:09 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote:
> Is it valid in _MPICH_ to make calls to PMI before MPI is initialized?
> I am interested in PMI_Init, PMI_Get_size and PMI_Get_rank.
>
> The motivating use case is to initialize e.g. half my ranks with
> MPI_THREAD_FUNNELED and the other half with MPI_THREAD_MULTIPLE but
> not use MPMD launching.
>
> I have two reasons for wanting to do it this way. First, it is
> otherwise unnecessary to use MPMD since I can branch at the top of my
> code and meet the requirements of FUNNELED and MULTIPLE. Second, I
> might be interested in running my code on a supercomputer with an
> MPICH-based MPI implementation that does not support MPMD launching in
> an unrestricted manner.
>
> If MPICH can handle this, then I will worry about whether or not the
> aforementioned supercomputer can. It is my assumption that, if MPICH
> cannot handle this, then the MPICH derivative in question cannot
> either.
>
> Thanks,
>
> Jeff
>
> --
> Jeff Hammond
> jeff.science(a)gmail.com
> http://jeffhammond.github.io/
--
Jeff Hammond
jeff.science(a)gmail.com
http://jeffhammond.github.io/
Is it valid in _MPICH_ to make calls to PMI before MPI is initialized?
I am interested in PMI_Init, PMI_Get_size and PMI_Get_rank.
The motivating use case is to initialize e.g. half my ranks with
MPI_THREAD_FUNNELED and the other half with MPI_THREAD_MULTIPLE but
not use MPMD launching.
I have two reasons for wanting to do it this way. First, it is
otherwise unnecessary to use MPMD since I can branch at the top of my
code and meet the requirements of FUNNELED and MULTIPLE. Second, I
might be interested in running my code on a supercomputer with an
MPICH-based MPI implementation that does not support MPMD launching in
an unrestricted manner.
If MPICH can handle this, then I will worry about whether or not the
aforementioned supercomputer can. It is my assumption that, if MPICH
cannot handle this, then the MPICH derivative in question cannot
either.
Thanks,
Jeff
--
Jeff Hammond
jeff.science(a)gmail.com
http://jeffhammond.github.io/
>> Oh, but then - <bam> - we see on slide 33, "Hide communication inside
>> "distributed array". OMG it's a high-level abstraction layer on top
>> of MPI that hides the supposed assembly language of parallel
>> programming and makes the programmer productive.
>>
>> But maybe Baidu is exceptional and most HPC programmers who deal with
>> array are doomed to tedious MPI programming. Let's see if there any
>> distributed array abstraction layers out there that hide MPI. It took
>> me about 42 microseconds to find http://libelemental.org/ and
>> https://www.emsl.pnl.gov/docs/global/, and those are just the ones
>> that I have a personal relationship with. And that is just the tip of
>> the MPI library iceberg.
>>
>> So yeah, most people that criticize MPI do so in a manner that is
>> logically equivalent to "Linux is way better than Fortran".
>
>
> This is but one of the points made, but I'm glad you brought it up. Parallel
> HDF5 and Parallel-NetCDF layer on top of MPI-IO routines, as well. so you
> and I know MPI meets one of its goals: be a good foundation for parallel
> libraries.
>
> Can you speak to Dursi's point about Gasnet and Charm++ 's relationship to
> MPI? It was you, not the GASNET folks, that developed the MPI transport
> layer for ARMCI, after all, but I am not familiar with those groups thought
> processes.
You are the zen master, Rob Latham. I was having so much fun being
mad, and you managed to ignore all my sarcasm and find something
constructive. Your talents are clearly wasted on computing; have you
considered a career in peace negotiation? :-)
MPI as a foundation for parallel libraries and thus good software
engineering in general is fundamental to its success. The IO
libraries you note do far more than MPI can or should while at the
same time subtracting nothing from the usability of MPI. I am not
aware of equivalent capability in e.g. Chapel or UPC, either in the
standard IO library of the language or in third-party code. And if it
exists in UPC, I believe that it is constrained by the lack of
layering/scoping capability equivalent to MPI comm, win or file
objects.
To be clear, ARMCI-MPI exists because of Pavan Balaji and Jim Dinan.
When they started that project, I was working on my own non-MPI port
of ARMCI, which I ultimately abandoned upon realizing that the
microseconds I could save with a non-portable implementation were not
worth the lack of portability relative to ARMCI over MPI-2 RMA, which
at that time was standard and supported on Blue Gene, Cray and
InfiniBand systems. Remodeling and optimizing ARMCI-MPI for MPI-3 was
something that I did due to the obvious practical advantages for
running NWChem and to validate the MPI Forum effort to support Global
Arrays effectively in MPI-3.
GASNet exists in part because of the shortcomings of MPI-2 RMA, which
are described in
http://upc.lbl.gov/publications/bonachea-duell-mpi.pdf. GASNet
remains relevant because of implementation shortcomings in some MPI
libraries, which still exist but are monotonically decreasing in
magnitude, and because - like MPI - there is a software ecosystem
build around it that users cannot immediately abandon. My hope is
that GASNet continues to exist, but that the default conduit is one
based upon the latest and greatest RMA and thread-oriented features of
MPI-3 and eventually MPI-4. But the MPI-3 direct port of UPC should
be faster since, unlike GASNet, MPI-3 exposes a rich set of atomics
that obviate the need for active-messages in many cases.
As for Charm++, Argonne has done some work to show that the unexpected
message performance issue can be mitigated with simple parameter
tuning. My hope is MPI implementations provide the necessary features
to make Charm++ run as fast as possible relative to lower-level
conduits and that the portability of MPI is recognized of greater
value than maintaining Charm++ backends for MPI, TCP/UDP/IP, IBV,
PAMI, uGNI, etc. etc. But at extreme scale, the NAMD people probably
need to shave as many microseconds off the comm time as possible,
which may require abandoning MPI. But such efforts should always be a
means of last resort and, in any case, such events do not support
Dursi's thesis.
Best,
Jeff
--
Jeff Hammond
jeff.science(a)gmail.com
http://jeffhammond.github.io/
>> Let's look at
>> http://on-demand.gputechconf.com/gtc/2014/webinar/gtc-express-acoates-webin….
>> The author works for Baidu's Silicon Valley AI Lab. Baidu is the
>> Chinese Google. About as Silicon Valley as it gets.
>
> we can find silicon valley using MPI, but it's not widespread. Another
> instance that comes to mind is this not heavily cited 2007 work from yahoo
> using mpi for indexing:
>
> http://link.springer.com/chapter/10.1007%2F978-3-540-75416-9_21
True, but the reason it is widespread is not for the reasons that
Dursi states. The primary reasons are sociological and economic,
rather than technical.
Jeff
--
Jeff Hammond
jeff.science(a)gmail.com
http://jeffhammond.github.io/
On 04/09/2015 09:45 AM, Jeff Hammond wrote:
> www.hammond.us/programming-model-discourse-is-dying-and-false-dichotomies-a…
>
> Let's look at http://on-demand.gputechconf.com/gtc/2014/webinar/gtc-express-acoates-webin….
> The author works for Baidu's Silicon Valley AI Lab. Baidu is the
> Chinese Google. About as Silicon Valley as it gets.
we can find silicon valley using MPI, but it's not widespread. Another
instance that comes to mind is this not heavily cited 2007 work from
yahoo using mpi for indexing:
http://link.springer.com/chapter/10.1007%2F978-3-540-75416-9_21
==rob
> Slide 30 says:
>
> OTS HPC Sotware Infrastructure
> � Infiniband (�IB�): Use MPI
> � MPI = Message Passing Interface
> � Standard mid-level API usually supporing IB.
>
> Slide 32 says "Enables message passing, but this is pretty unnatural."
> I guess Dursi is right. MPI sucks so hard. And it's definitely MPI
> alone that uses send-recv semantics. It's not like the Internet is
> based upon that or anything. And one definitely must switch to UPC to
> get one-sided semantics.
>
> Oh, but then - <bam> - we see on slide 33, "Hide communication inside
> "distributed array". OMG it's a high-level abstraction layer on top
> of MPI that hides the supposed assembly language of parallel
> programming and makes the programmer productive.
>
> But maybe Baidu is exceptional and most HPC programmers who deal with
> array are doomed to tedious MPI programming. Let's see if there any
> distributed array abstraction layers out there that hide MPI. It took
> me about 42 microseconds to find http://libelemental.org/ and
> https://www.emsl.pnl.gov/docs/global/, and those are just the ones
> that I have a personal relationship with. And that is just the tip of
> the MPI library iceberg.
>
> So yeah, most people that criticize MPI do so in a manner that is
> logically equivalent to "Linux is way better than Fortran".
>
> http://arxiv.org/abs/cs/0109017 is a must-read, for anyone that hasn't already.
>
> Jeff
>
>
> On Thu, Apr 9, 2015 at 7:16 AM, Rob Latham <robl(a)mcs.anl.gov> wrote:
>> I (and it seems quite a few others) found this yesterday:
>>
>> http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/
>>
>> The premise is a couple things:
>> - spark, hadoop, and the "silicon valley stack" has tremendous mindshare
>> - MPI is too hard for application writers
>> - and too challenging for library writers
>> - and HPC has "not invented here" blinders keeping it from adopting new
>> techonolgy
>>
>> now, when someone writes "X is dying" it's more of an opening position
>> statement for a discussion than a statement of fact. But it does bother
>> me immensely that there is an "HPC stack" and a "silicon valley stack".
>> Convergence is happening, but not fast enough.
>>
>> ==rob
>>
>> --
>> Rob Latham
>> Mathematics and Computer Science Division
>> Argonne National Lab, IL USA
>> _______________________________________________
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/mailman/listinfo/devel
>
>
>
--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA
On 04/09/2015 09:45 AM, Jeff Hammond wrote:
> Oh, but then - <bam> - we see on slide 33, "Hide communication inside
> "distributed array". OMG it's a high-level abstraction layer on top
> of MPI that hides the supposed assembly language of parallel
> programming and makes the programmer productive.
>
> But maybe Baidu is exceptional and most HPC programmers who deal with
> array are doomed to tedious MPI programming. Let's see if there any
> distributed array abstraction layers out there that hide MPI. It took
> me about 42 microseconds to find http://libelemental.org/ and
> https://www.emsl.pnl.gov/docs/global/, and those are just the ones
> that I have a personal relationship with. And that is just the tip of
> the MPI library iceberg.
>
> So yeah, most people that criticize MPI do so in a manner that is
> logically equivalent to "Linux is way better than Fortran".
This is but one of the points made, but I'm glad you brought it up.
Parallel HDF5 and Parallel-NetCDF layer on top of MPI-IO routines, as
well. so you and I know MPI meets one of its goals: be a good foundation
for parallel libraries.
Can you speak to Dursi's point about Gasnet and Charm++ 's relationship
to MPI? It was you, not the GASNET folks, that developed the MPI
transport layer for ARMCI, after all, but I am not familiar with those
groups thought processes.
==rob
>
> http://arxiv.org/abs/cs/0109017 is a must-read, for anyone that hasn't already.
>
> Jeff
>
>
> On Thu, Apr 9, 2015 at 7:16 AM, Rob Latham <robl(a)mcs.anl.gov> wrote:
>> I (and it seems quite a few others) found this yesterday:
>>
>> http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/
>>
>> The premise is a couple things:
>> - spark, hadoop, and the "silicon valley stack" has tremendous mindshare
>> - MPI is too hard for application writers
>> - and too challenging for library writers
>> - and HPC has "not invented here" blinders keeping it from adopting new
>> techonolgy
>>
>> now, when someone writes "X is dying" it's more of an opening position
>> statement for a discussion than a statement of fact. But it does bother
>> me immensely that there is an "HPC stack" and a "silicon valley stack".
>> Convergence is happening, but not fast enough.
>>
>> ==rob
>>
>> --
>> Rob Latham
>> Mathematics and Computer Science Division
>> Argonne National Lab, IL USA
>> _______________________________________________
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/mailman/listinfo/devel
>
>
>
--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA
www.hammond.us/programming-model-discourse-is-dying-and-false-dichotomies-a…
Let's look at http://on-demand.gputechconf.com/gtc/2014/webinar/gtc-express-acoates-webin….
The author works for Baidu's Silicon Valley AI Lab. Baidu is the
Chinese Google. About as Silicon Valley as it gets.
Slide 30 says:
OTS HPC Sotware Infrastructure
• Infiniband (“IB”): Use MPI
– MPI = Message Passing Interface
• Standard mid-level API usually supporing IB.
Slide 32 says "Enables message passing, but this is pretty unnatural."
I guess Dursi is right. MPI sucks so hard. And it's definitely MPI
alone that uses send-recv semantics. It's not like the Internet is
based upon that or anything. And one definitely must switch to UPC to
get one-sided semantics.
Oh, but then - <bam> - we see on slide 33, "Hide communication inside
"distributed array". OMG it's a high-level abstraction layer on top
of MPI that hides the supposed assembly language of parallel
programming and makes the programmer productive.
But maybe Baidu is exceptional and most HPC programmers who deal with
array are doomed to tedious MPI programming. Let's see if there any
distributed array abstraction layers out there that hide MPI. It took
me about 42 microseconds to find http://libelemental.org/ and
https://www.emsl.pnl.gov/docs/global/, and those are just the ones
that I have a personal relationship with. And that is just the tip of
the MPI library iceberg.
So yeah, most people that criticize MPI do so in a manner that is
logically equivalent to "Linux is way better than Fortran".
http://arxiv.org/abs/cs/0109017 is a must-read, for anyone that hasn't already.
Jeff
On Thu, Apr 9, 2015 at 7:16 AM, Rob Latham <robl(a)mcs.anl.gov> wrote:
> I (and it seems quite a few others) found this yesterday:
>
> http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/
>
> The premise is a couple things:
> - spark, hadoop, and the "silicon valley stack" has tremendous mindshare
> - MPI is too hard for application writers
> - and too challenging for library writers
> - and HPC has "not invented here" blinders keeping it from adopting new
> techonolgy
>
> now, when someone writes "X is dying" it's more of an opening position
> statement for a discussion than a statement of fact. But it does bother
> me immensely that there is an "HPC stack" and a "silicon valley stack".
> Convergence is happening, but not fast enough.
>
> ==rob
>
> --
> Rob Latham
> Mathematics and Computer Science Division
> Argonne National Lab, IL USA
> _______________________________________________
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/devel
--
Jeff Hammond
jeff.science(a)gmail.com
http://jeffhammond.github.io/
I (and it seems quite a few others) found this yesterday:
http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/
The premise is a couple things:
- spark, hadoop, and the "silicon valley stack" has tremendous mindshare
- MPI is too hard for application writers
- and too challenging for library writers
- and HPC has "not invented here" blinders keeping it from adopting new
techonolgy
now, when someone writes "X is dying" it's more of an opening position
statement for a discussion than a statement of fact. But it does
bother me immensely that there is an "HPC stack" and a "silicon valley
stack". Convergence is happening, but not fast enough.
==rob
--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA
