- devel - mailman3.cels.anl.gov

Re: [mpich-devel] Second review on test suite change
by Latham, Robert J. 04 Feb '14

04 Feb '14

On Tue, 2014-02-04 at 19:36 +0000, Dave Goodell (dgoodell) wrote: > On Feb 4, 2014, at 1:06 PM, "Latham, Robert J." <robl(a)mcs.anl.gov> wrote: > > > but there is one fix that's in the "so simple how could it have ever > > worked before" category that i'd like to run by a wider audience: > > > > commit b4c9f88fa1aa6a957d2a1df8d9a33561e1d68ab8 > > I clearly wrote the code in question, see http://git.mpich.org/mpich.git/commitdiff/4aba817fda95da257258befed3fa34167…. Sorry for the bug. > > To me the fix looks good. We probably never noticed the bug before because it's somewhat rare for the MPICH test suite to have build failures when built against the matching version of MPICH itself. Most failures are run-time issues in that case. > Dave! thanks for parachuting in. In this case the test that was failing was a new test. We wouldn't expect those to fail either but we're only human... ==rob

1 0

Re: [mpich-devel] Second review on test suite change
by Dave Goodell (dgoodell) 04 Feb '14

04 Feb '14

On Feb 4, 2014, at 1:06 PM, "Latham, Robert J." <robl(a)mcs.anl.gov> wrote: > but there is one fix that's in the "so simple how could it have ever > worked before" category that i'd like to run by a wider audience: > > commit b4c9f88fa1aa6a957d2a1df8d9a33561e1d68ab8 I clearly wrote the code in question, see http://git.mpich.org/mpich.git/commitdiff/4aba817fda95da257258befed3fa34167…. Sorry for the bug. To me the fix looks good. We probably never noticed the bug before because it's somewhat rare for the MPICH test suite to have build failures when built against the matching version of MPICH itself. Most failures are run-time issues in that case. -Dave

1 0

Second review on test suite change
by Latham, Robert J. 04 Feb '14

04 Feb '14

Bill and I have been working on some datatype changes. A few weeks back, that work broke the MPICH-1 tests run out of cron, so we reverted. Bill's got a series of patches to introduce the MPICH1 tests into the mpich tests (getting them into the jenkins reports as well). I think Jenkins is happy with the patches: https://jenkins.mpich.org/view/mpich-review/job/mpich-review/660/ but there is one fix that's in the "so simple how could it have ever worked before" category that i'd like to run by a wider audience: commit b4c9f88fa1aa6a957d2a1df8d9a33561e1d68ab8 Author: William Gropp <wgropp(a)illinois.edu> Date: Sun Feb 2 22:34:37 2014 -0600 Fix bug in marking unbuilt tests as failed The xfail additions to the runtests code had an incorrect test on xfail that caused runtests to stop recording a test that could not be built as failed - thus, the test script would report success for all tests when some tests were not run. Signed-off-by: Rob Latham <robl(a)mcs.anl.gov> diff --git a/test/mpi/runtests.in b/test/mpi/runtests.in index ad15bb55..91157e94 100644 --- a/test/mpi/runtests.in +++ b/test/mpi/runtests.in @@ -492,7 +492,7 @@ sub RunList { $progEnv, $mpiexecArgs, $xfail ); } } - elsif ($xfail ne '') { + elsif ($xfail eq '') { # We expected to run this program, so failure to build # is an error $found_error = 1;

1 0

Re: [mpich-devel] MPICH per-obj build issues
by Archer, Charles J 23 Jan '14

23 Jan '14

Ok. The patch isn’t quite right, I’m going through all the thread options to test my current patch. You definitely need to protect the prototypes with (my first patch will break the single thread build): #if defined(MPICH_IS_THREADED) && !defined(MPID_DEVICE_DEFINES_THREAD_CS) -C From: Wesley Bland [mailto:[email protected]] Sent: Thursday, January 23, 2014 12:44 PM To: Archer, Charles J Cc: Pavan Balaji; William Gropp; devel(a)mpich.org Subject: Re: MPICH per-obj build issues Hi Charles, I know little about the threading inside MPICH at this point and even less about per object builds. Though I intend to now learn more about this, I’m CC’ing the mpich-devel mailing list in an effort to bring in the right people more quickly. Wesley On Jan 23, 2014, 2:03:48 PM, Archer, Charles J <charles.j.archer(a)intel.com<mailto:[email protected]>> wrote: ________________________________ Hi, Wesley, I’m playing around with per-obj builds, using per-object/tls configure options. pamid defined it’s own critical sections so it doesn’t see this build error, but I’m trying to use default per-obj builds and I hit this: mpiimplthreadpost.h:27: error: static declaration of 'MPIU_Thread_CS_enter_lockname_impl_' follows non-static declaration My solution for me was to prototype the functions in src/include/mpiimplthread.h: diff --git a/src/include/mpiimplthread.h b/src/include/mpiimplthread.h index f0141c6..c9af705 100644 --- a/src/include/mpiimplthread.h +++ b/src/include/mpiimplthread.h @@ -1039,5 +1039,20 @@ do { \ #define MPIU_THREAD_MPI_OBJ_FINALIZE(objptr_) do {} while (0) #endif + +#if 0 +static MPIU_DBG_INLINE_KEYWORD void +MPIU_Thread_CS_enter_lockname_impl_(enum MPIU_Nest_mutexes kind, + const char *lockname, + MPID_Thread_mutex_t *mutex); + + +static MPIU_DBG_INLINE_KEYWORD void +MPIU_Thread_CS_exit_lockname_impl_(enum MPIU_Nest_mutexes kind, + const char *lockname, + MPID_Thread_mutex_t *mutex); +#endif + + #endif /* !defined(MPIIMPLTHREAD_H_INCLUDED) */ …. Can you guys add in the prototypes? This patch isn’t complete, I wanted to talk to you first. Is this the right solution? Also related (reported by Bill): https://trac.mpich.org/projects/mpich/ticket/1937

1 0

Re: [mpich-devel] MPICH per-obj build issues
by Wesley Bland 23 Jan '14

23 Jan '14

Hi Charles, I know little about the threading inside MPICH at this point and even less about per object builds. Though I intend to now learn more about this, I’m CC’ing the mpich-devel mailing list in an effort to bring in the right people more quickly. Wesley On Jan 23, 2014, 2:03:48 PM, Archer, Charles J <charles.j.archer(a)intel.com> wrote: Hi, Wesley, I’m playing around with per-obj builds, using per-object/tls configure options. pamid defined it’s own critical sections so it doesn’t see this build error, but I’m trying to use default per-obj builds and I hit this: mpiimplthreadpost.h:27: error: static declaration of 'MPIU_Thread_CS_enter_lockname_impl_' follows non-static declaration My solution for me was to prototype the functions in src/include/mpiimplthread.h: diff --git a/src/include/mpiimplthread.h b/src/include/mpiimplthread.h index f0141c6..c9af705 100644 --- a/src/include/mpiimplthread.h +++ b/src/include/mpiimplthread.h @@ -1039,5 +1039,20 @@ do { \ #define MPIU_THREAD_MPI_OBJ_FINALIZE(objptr_) do {} while (0) #endif + +#if 0 +static MPIU_DBG_INLINE_KEYWORD void +MPIU_Thread_CS_enter_lockname_impl_(enum MPIU_Nest_mutexes kind, + const char *lockname, + MPID_Thread_mutex_t *mutex); + + +static MPIU_DBG_INLINE_KEYWORD void +MPIU_Thread_CS_exit_lockname_impl_(enum MPIU_Nest_mutexes kind, + const char *lockname, + MPID_Thread_mutex_t *mutex); +#endif + + #endif /* !defined(MPIIMPLTHREAD_H_INCLUDED) */ …. Can you guys add in the prototypes? This patch isn’t complete, I wanted to talk to you first. Is this the right solution? Also related (reported by Bill): https://trac.mpich.org/projects/mpich/ticket/1937

1 0

Re: [mpich-devel] P states and MPI
by Rountree, Barry L. 17 Jan '14

17 Jan '14

The released version of Adagio is here: [email protected]:scalability-llnl/Adagio.git The algorithm described is the most trivial of the three we looked at in Rountree ICS 2009. We called it "Fermata": predict how long the MPI rank would wait in a blocking MPI call, and if the expected delay was long enough, drop down to the lowest CPU clock frequency and pop back up just before the call was expected to complete. We saw significant (>10%) total system energy savings will <1% execution time overhead. The best algorithm, though, slowed down computation to consume slack time that was generated by blocking calls, and only if any slack remained we dropped into the lowest frequency. That got us 20% total system energy savings on ParaDis with <1% execution time overhead. This was very much a research project and I expect bitrot has set in by now. Rewriting from scratch wouldn't be difficult (now that I have 5 years more experience writing PMPI tools), but there are a few complications that would need to be addressed before this could be made into a production tool (OpenMP, nondeterminism caused by turbo mode, processor performance inhomogeneity, etc). I think the more interesting question is how to optimize performance under a power bound; solving that problem has the side effect of achieving near-optimal energy use (at least in the common case). We're working on redoing Adagio to solve thing problem, using Intel Running Average Power Limit (RAPL) technology. Best, Barry On 1/17/14 8:25 AM, "Bronis R. de Supinski" <bronis(a)llnl.gov> wrote: > >Jeff: > >Adagio has similar characteristics. Barry has recently >released it so he should comment on its availability. > >Bronis > > >On Fri, 17 Jan 2014, Jeff Hammond wrote: > >> Thanks Bill. I agree with your assessment. If this is the type of >> thing that pays off in practice, then it seems to argue for the >> necessary policy changes. >> >> Bronis - What impresses me about this is the ease at which it can >> deployed. For the codes for which it reduces power consumption, it's >> free joules. If it doesn't, there is nothing lost. If your code is >> similarly portable and easy to deploy, then I'll have to evaluate >> which one is more effective for the relevant workload. >> >> Best, >> >> Jeff >> >> >> On Fri, Jan 17, 2014 at 8:46 AM, Bronis R. de Supinski >><bronis(a)llnl.gov> wrote: >>> >>> Um, I would say the Dell approach has no novelty and is >>> rather rudimentary. We have done a lot more in Adagio: >>> >>> http://dl.acm.org/citation.cfm?id=1542340 >>> >>> @inproceedings{Rountree:2009:AMD:1542275.1542340, >>> author = {Rountree, Barry and Lownenthal, David K. and de Supinski, >>>Bronis >>> R. and Schulz, Martin and Freeh, Vincent W. and Bletsch, Tyler}, >>> title = {Adagio: Making DVS Practical for Complex HPC Applications}, >>> booktitle = {Proceedings of the 23rd International Conference on >>> Supercomputing}, >>> series = {ICS '09}, >>> year = {2009}, >>> isbn = {978-1-60558-498-0}, >>> location = {Yorktown Heights, NY, USA}, >>> pages = {460--469}, >>> numpages = {10}, >>> url = {http://doi.acm.org/10.1145/1542275.1542340}, >>> doi = {10.1145/1542275.1542340}, >>> acmid = {1542340}, >>> publisher = {ACM}, >>> address = {New York, NY, USA}, >>> keywords = {dvfs, dvs, energy, mpi, runtime}, >>> >>> } >>> >>> >>> >>> On Fri, 17 Jan 2014, William Gropp wrote: >>> >>>> There has been some work on this for parallel programs; Wu Feng at >>>> Virginia Tech has done some, for example. I don't recall seeing any >>>>work >>>> that used the profiling interface. Some of the issues raised, >>>>especially >>>> the required privs, mean that this is only available for experiments, >>>>not >>>> production, until that changes. Also, its sometimes possible to make >>>> predictions about wait times based on previous iterations; that could >>>>be >>>> used to refine such control. >>>> >>>> Bill >>>> >>>> William Gropp >>>> Director, Parallel Computing Institute >>>> Deputy Director for Research >>>> Institute for Advanced Computing Applications and Technologies >>>> Thomas M. Siebel Chair in Computer Science >>>> University of Illinois Urbana-Champaign >>>> >>>> >>>> >>>> >>>> On Jan 16, 2014, at 9:36 PM, Jeff Hammond wrote: >>>> >>>>> I was impressed by >>>>> >>>>> >>>>>http://www.hpcadvisorycouncil.com/events/2013/Spain-Workshop/pdf/5_Del >>>>>l.pdf >>>>> / >>>>> >>>>>http://www.bsc.es/sites/default/files/public/mare_nostrum/2013hpcac-05 >>>>>.pdf. >>>>> I wonder if anyone else has seen this, done anything similar, is >>>>> interested in doing something similar, etc. I can imagine that a >>>>>more >>>>> integrated implementation has some performance advantages, albeit at >>>>> much greater maintenance cost. >>>>> >>>>> Thanks, >>>>> >>>>> Jeff >>>>> >>>>> -- >>>>> Jeff Hammond >>>>> jeff.science(a)gmail.com >>>> >>>> >>>> >>> >> >> >> >> -- >> Jeff Hammond >> Argonne Leadership Computing Facility >> University of Chicago Computation Institute >> jhammond(a)anl.gov / jhammond(a)uchicago.edu / (630) 252-5381 >> http://www.linkedin.com/in/jeffhammond >> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond >>

1 0

Re: [mpich-devel] P states and MPI
by Bronis R. de Supinski 17 Jan '14

17 Jan '14

Jeff: Adagio has similar characteristics. Barry has recently released it so he should comment on its availability. Bronis On Fri, 17 Jan 2014, Jeff Hammond wrote: > Thanks Bill. I agree with your assessment. If this is the type of > thing that pays off in practice, then it seems to argue for the > necessary policy changes. > > Bronis - What impresses me about this is the ease at which it can > deployed. For the codes for which it reduces power consumption, it's > free joules. If it doesn't, there is nothing lost. If your code is > similarly portable and easy to deploy, then I'll have to evaluate > which one is more effective for the relevant workload. > > Best, > > Jeff > > > On Fri, Jan 17, 2014 at 8:46 AM, Bronis R. de Supinski <bronis(a)llnl.gov> wrote: >> >> Um, I would say the Dell approach has no novelty and is >> rather rudimentary. We have done a lot more in Adagio: >> >> http://dl.acm.org/citation.cfm?id=1542340 >> >> @inproceedings{Rountree:2009:AMD:1542275.1542340, >> author = {Rountree, Barry and Lownenthal, David K. and de Supinski, Bronis >> R. and Schulz, Martin and Freeh, Vincent W. and Bletsch, Tyler}, >> title = {Adagio: Making DVS Practical for Complex HPC Applications}, >> booktitle = {Proceedings of the 23rd International Conference on >> Supercomputing}, >> series = {ICS '09}, >> year = {2009}, >> isbn = {978-1-60558-498-0}, >> location = {Yorktown Heights, NY, USA}, >> pages = {460--469}, >> numpages = {10}, >> url = {http://doi.acm.org/10.1145/1542275.1542340}, >> doi = {10.1145/1542275.1542340}, >> acmid = {1542340}, >> publisher = {ACM}, >> address = {New York, NY, USA}, >> keywords = {dvfs, dvs, energy, mpi, runtime}, >> >> } >> >> >> >> On Fri, 17 Jan 2014, William Gropp wrote: >> >>> There has been some work on this for parallel programs; Wu Feng at >>> Virginia Tech has done some, for example. I don't recall seeing any work >>> that used the profiling interface. Some of the issues raised, especially >>> the required privs, mean that this is only available for experiments, not >>> production, until that changes. Also, its sometimes possible to make >>> predictions about wait times based on previous iterations; that could be >>> used to refine such control. >>> >>> Bill >>> >>> William Gropp >>> Director, Parallel Computing Institute >>> Deputy Director for Research >>> Institute for Advanced Computing Applications and Technologies >>> Thomas M. Siebel Chair in Computer Science >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> >>> On Jan 16, 2014, at 9:36 PM, Jeff Hammond wrote: >>> >>>> I was impressed by >>>> >>>> http://www.hpcadvisorycouncil.com/events/2013/Spain-Workshop/pdf/5_Dell.pdf >>>> / >>>> http://www.bsc.es/sites/default/files/public/mare_nostrum/2013hpcac-05.pdf. >>>> I wonder if anyone else has seen this, done anything similar, is >>>> interested in doing something similar, etc. I can imagine that a more >>>> integrated implementation has some performance advantages, albeit at >>>> much greater maintenance cost. >>>> >>>> Thanks, >>>> >>>> Jeff >>>> >>>> -- >>>> Jeff Hammond >>>> jeff.science(a)gmail.com >>> >>> >>> >> > > > > -- > Jeff Hammond > Argonne Leadership Computing Facility > University of Chicago Computation Institute > jhammond(a)anl.gov / jhammond(a)uchicago.edu / (630) 252-5381 > http://www.linkedin.com/in/jeffhammond > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond >

1 0

Re: [mpich-devel] P states and MPI
by Jeff Hammond 17 Jan '14

17 Jan '14

Thanks Bill. I agree with your assessment. If this is the type of thing that pays off in practice, then it seems to argue for the necessary policy changes. Bronis - What impresses me about this is the ease at which it can deployed. For the codes for which it reduces power consumption, it's free joules. If it doesn't, there is nothing lost. If your code is similarly portable and easy to deploy, then I'll have to evaluate which one is more effective for the relevant workload. Best, Jeff On Fri, Jan 17, 2014 at 8:46 AM, Bronis R. de Supinski <bronis(a)llnl.gov> wrote: > > Um, I would say the Dell approach has no novelty and is > rather rudimentary. We have done a lot more in Adagio: > > http://dl.acm.org/citation.cfm?id=1542340 > > @inproceedings{Rountree:2009:AMD:1542275.1542340, > author = {Rountree, Barry and Lownenthal, David K. and de Supinski, Bronis > R. and Schulz, Martin and Freeh, Vincent W. and Bletsch, Tyler}, > title = {Adagio: Making DVS Practical for Complex HPC Applications}, > booktitle = {Proceedings of the 23rd International Conference on > Supercomputing}, > series = {ICS '09}, > year = {2009}, > isbn = {978-1-60558-498-0}, > location = {Yorktown Heights, NY, USA}, > pages = {460--469}, > numpages = {10}, > url = {http://doi.acm.org/10.1145/1542275.1542340}, > doi = {10.1145/1542275.1542340}, > acmid = {1542340}, > publisher = {ACM}, > address = {New York, NY, USA}, > keywords = {dvfs, dvs, energy, mpi, runtime}, > > } > > > > On Fri, 17 Jan 2014, William Gropp wrote: > >> There has been some work on this for parallel programs; Wu Feng at >> Virginia Tech has done some, for example. I don't recall seeing any work >> that used the profiling interface. Some of the issues raised, especially >> the required privs, mean that this is only available for experiments, not >> production, until that changes. Also, its sometimes possible to make >> predictions about wait times based on previous iterations; that could be >> used to refine such control. >> >> Bill >> >> William Gropp >> Director, Parallel Computing Institute >> Deputy Director for Research >> Institute for Advanced Computing Applications and Technologies >> Thomas M. Siebel Chair in Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> >> On Jan 16, 2014, at 9:36 PM, Jeff Hammond wrote: >> >>> I was impressed by >>> >>> http://www.hpcadvisorycouncil.com/events/2013/Spain-Workshop/pdf/5_Dell.pdf >>> / >>> http://www.bsc.es/sites/default/files/public/mare_nostrum/2013hpcac-05.pdf. >>> I wonder if anyone else has seen this, done anything similar, is >>> interested in doing something similar, etc. I can imagine that a more >>> integrated implementation has some performance advantages, albeit at >>> much greater maintenance cost. >>> >>> Thanks, >>> >>> Jeff >>> >>> -- >>> Jeff Hammond >>> jeff.science(a)gmail.com >> >> >> > -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)anl.gov / jhammond(a)uchicago.edu / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [mpich-devel] P states and MPI
by Bronis R. de Supinski 17 Jan '14

17 Jan '14

I guess I should add that in particular we predict wait times on a call site basis in order to exploit consistent imbalance (or even slowly evolving imbalance, see the results for ParaDiS, for which process work changes but not rapidly, which is typical of real apps) On Fri, 17 Jan 2014, Bronis R. de Supinski wrote: > > Um, I would say the Dell approach has no novelty and is > rather rudimentary. We have done a lot more in Adagio: > > http://dl.acm.org/citation.cfm?id=1542340 > > @inproceedings{Rountree:2009:AMD:1542275.1542340, > author = {Rountree, Barry and Lownenthal, David K. and de Supinski, Bronis > R. and Schulz, Martin and Freeh, Vincent W. and Bletsch, Tyler}, > title = {Adagio: Making DVS Practical for Complex HPC Applications}, > booktitle = {Proceedings of the 23rd International Conference on > Supercomputing}, > series = {ICS '09}, > year = {2009}, > isbn = {978-1-60558-498-0}, > location = {Yorktown Heights, NY, USA}, > pages = {460--469}, > numpages = {10}, > url = {http://doi.acm.org/10.1145/1542275.1542340}, > doi = {10.1145/1542275.1542340}, > acmid = {1542340}, > publisher = {ACM}, > address = {New York, NY, USA}, > keywords = {dvfs, dvs, energy, mpi, runtime}, > } > > > > On Fri, 17 Jan 2014, William Gropp wrote: > >> There has been some work on this for parallel programs; Wu Feng at >> Virginia Tech has done some, for example. I don't recall seeing any work >> that used the profiling interface. Some of the issues raised, especially >> the required privs, mean that this is only available for experiments, not >> production, until that changes. Also, its sometimes possible to make >> predictions about wait times based on previous iterations; that could be >> used to refine such control. >> >> Bill >> >> William Gropp >> Director, Parallel Computing Institute >> Deputy Director for Research >> Institute for Advanced Computing Applications and Technologies >> Thomas M. Siebel Chair in Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> >> On Jan 16, 2014, at 9:36 PM, Jeff Hammond wrote: >> >> > I was impressed by >> > http://www.hpcadvisorycouncil.com/events/2013/Spain-Workshop/pdf/5_Dell.pdf >> > / >> > http://www.bsc.es/sites/default/files/public/mare_nostrum/2013hpcac-05.pdf. >> > I wonder if anyone else has seen this, done anything similar, is >> > interested in doing something similar, etc. I can imagine that a more >> > integrated implementation has some performance advantages, albeit at >> > much greater maintenance cost. >> > >> > Thanks, >> > >> > Jeff >> > >> > -- >> > Jeff Hammond >> > jeff.science(a)gmail.com >> >> >

1 0

Re: [mpich-devel] P states and MPI
by Bronis R. de Supinski 17 Jan '14

17 Jan '14

Um, I would say the Dell approach has no novelty and is rather rudimentary. We have done a lot more in Adagio: http://dl.acm.org/citation.cfm?id=1542340 @inproceedings{Rountree:2009:AMD:1542275.1542340, author = {Rountree, Barry and Lownenthal, David K. and de Supinski, Bronis R. and Schulz, Martin and Freeh, Vincent W. and Bletsch, Tyler}, title = {Adagio: Making DVS Practical for Complex HPC Applications}, booktitle = {Proceedings of the 23rd International Conference on Supercomputing}, series = {ICS '09}, year = {2009}, isbn = {978-1-60558-498-0}, location = {Yorktown Heights, NY, USA}, pages = {460--469}, numpages = {10}, url = {http://doi.acm.org/10.1145/1542275.1542340}, doi = {10.1145/1542275.1542340}, acmid = {1542340}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {dvfs, dvs, energy, mpi, runtime}, } On Fri, 17 Jan 2014, William Gropp wrote: > There has been some work on this for parallel programs; Wu Feng at Virginia Tech has done some, for example. I don't recall seeing any work that used the profiling interface. Some of the issues raised, especially the required privs, mean that this is only available for experiments, not production, until that changes. Also, its sometimes possible to make predictions about wait times based on previous iterations; that could be used to refine such control. > > Bill > > William Gropp > Director, Parallel Computing Institute > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Thomas M. Siebel Chair in Computer Science > University of Illinois Urbana-Champaign > > > > > On Jan 16, 2014, at 9:36 PM, Jeff Hammond wrote: > >> I was impressed by >> http://www.hpcadvisorycouncil.com/events/2013/Spain-Workshop/pdf/5_Dell.pdf >> / http://www.bsc.es/sites/default/files/public/mare_nostrum/2013hpcac-05.pdf. >> I wonder if anyone else has seen this, done anything similar, is >> interested in doing something similar, etc. I can imagine that a more >> integrated implementation has some performance advantages, albeit at >> much greater maintenance cost. >> >> Thanks, >> >> Jeff >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com > >

1 0