Jeff, I am not trying to benchmark performance of MPI or beat up on poor old bulk-synchronous MPI-1. My goal is to understand which API functions are most commonly used in programs that are representative of MPI programs. I am doing a Ph.D. on systematic testing on concurrent software. In my experience, software developers use concurrency for the sake of performance, but are unaware of the quantitative implications their designs have on the ability to thoroughly test the code they write. Recently, I have been involved in a project that explores the performance-testability trade-off in the context of multi-threaded programs and I am interested in doing the same in the context of MPI programs. As a start I would like to get a better sense of how people use MPI. I hope this puts any worries you might have about unfair performance comparisons to rest and sheds some light on what I mean by a "macro-benchmark". Best, --Jiri On Fri, Jul 5, 2013 at 1:01 PM, Jeff Hammond <[email protected]> wrote:
What is a macrobenchmark? The more complexity you add to a benchmark, the harder it is to see the details. This is why most MPI benchmarks are microbenchmarks. As soon as you add complexity in the usage of MPI, you run into issues with algorithmic variation. For example, all of the "macrobenchmarks" I have seen (e.g. NAS PB) are written to MPI-1. They are also quite simple in their use of MPI, relying upon barrier synchronization when it may not be necessary or appropriate. You may see papers that show UPC beating MPI for NAS PB, but these comparisons are apples-to-oranges; if one rewrote the code to use nonblocking collectives, RMA or other less synchronous approach, the performance would likely be as good if not better than UPC. In this respect, one is not so much evaluating MPI performance (which is the point of an MPI benchmark, no?) but rather evaluating the use of a simplistic and perhaps overly synchronous programming model that happens to use MPI-1.
What is it you really want to do? Do you have some new-fangled programming model (NFPM) and you want to beat up on poor old bulk-synchronous MPI-1? If so, just don't do that. Write your own comparison between NFMP and MPI-3 using all the same algorithmic tricks that NFMP does. If you want to understand the performance of MPI in detail when employed by nontrivial applications, then you should try to find a nontrivial application that uses MPI very effectively. Unfortunately, many apps use MPI in a simplistic way because they are not communication limited in the way that strong-scaled macrobenchmarks are, hence no exotic use of MPI is required. I find that better algorithms are usually a more effective way to improve application performance than going OCD on my use of MPI...
It is entirely possible that there does not exist anything out there that you can reuse. However, this is clearly no problem for you since you are now well-rested and can complete the project by yourself :-) I am, of course, referencing http://www.cs.cmu.edu/~jsimsa/images/dilbert.jpg on your home page :-)
Best,
Heff
On Fri, Jul 5, 2013 at 11:22 AM, Pavan Balaji <[email protected]> wrote:
On 07/05/2013 10:38 AM, Jiri Simsa wrote:
Could anyone point me in the direction of programs that are representative of MPI programs? I am looking for something like PARSEC (http://parsec.cs.princeton.edu/) for MPI. In other words, I am interested in macro-bechnmarks not micro-benchmarks. Thank you.
You could try the Sequoia suite. There's also the NAS parallel
benchmarks
and Graph500 benchmarks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
-- Jeff Hammond [email protected]