March 2011 - Esc-team - mailman3.cels.anl.gov

Plan status and ToDo
by Pete Beckman 28 Mar '11

28 Mar '11

Status and ToDo. Look for your name. Further instructions below: LaTeX work: Pavan, Kamil is out today, please make 4_SoftwareDevl, 5_SoftwareStack separate documents with title pages. Also, check the makefile dependencies for all the makefiles, and correct them if needed. Narrative: * Intro: Paul Messina is working on some text we will drop in later * ESC Intro: Pete has finished first draft, Charlie, Marc, and others are reading and making suggestions * Mgmt Plan: Pete has finished first draft, Charlie, Marc, and others are reading and making suggestions * CoDesign: Al Geist is working on that section * GapAnalysis/Inventory: needs high-level overview for narrative * SoftwareStack: needs high-level overview for narrative * QA, Integration, Support: needs high-level overview for narrative * Outreach: Terry Moore is working on the text * Budget & Milestones: Jason and Mike Software Development Document: * Terry Quinn, Greg Pope, Bill Gropp, and Mike Heroux have finished initial draft. It is time for editing and extension. Ray Bair, Bill Kramer, and Kumar are looking at it. Software Stack: * The document will be version locked this afternoon. If you have any remaining bits to get in, do it by this afternoon. After that, I'm pruning sections not complete, until after the review. Then we can see if we can fit them back in, if there is room in the budget. Also, I'll be doing an overall balancing pass on effort and technologies. There are pieces that have to be tied together with more effort, and some places where effort must be adjusted. I hope to complete that pass and chat with area leads in the next day or so. Project management pieces and milestones and staffing plans are being prepared by Jason and Mike, and we hope to have something out later this week. Bill Harrod has provided us a bit more time, so the entire package is not due until after IESP, which Bill is attending. However, we need to have a draft suitable for sharing with Bill on April 1st so we can discuss and adjust as needed while at IESP. -pete

1 0

NNSA Meeting
by Pete Beckman 22 Mar '11

22 Mar '11

For those of us at the NNSA meeting. Please take notes on where we need to adjust our plans for what we missed, key directions, etc. Also, if you discover key organizational or planning ideas for how we organize ESC's plan, please share them. -Pete

6 6

Re: [Esc-team] NNSA Meeting
by Dosanjh, Sudip S 22 Mar '11

22 Mar '11

Hi Ron, I would like to see a research component in ESC as well. I don't have a percentage in mind but getting to Exascale won't be evolutionary (or if it is, we don't need an initiative). Regards, Sudip On 3/22/11 10:48 AM, "ronald minnich" <rminnich(a)sandia.gov> wrote: >My biggest single concern about what I've seen on the ESC roadmap, and I >really very reluctant to say this because of the hard work done to date, >is that it seems very evolutionary, and I just can't tell how that's >going to play out on the new big machines. Is there room for revolution >in the ESC? If so, what fraction of the spend plan? I think it's a >problem if it's under 60% > >But am I being unfair? If so, just tell me! > >ron >_______________________________________________ >Esc-team mailing list >Esc-team(a)lists.mcs.anl.gov >https://lists.mcs.anl.gov/mailman/listinfo/esc-team >

2 3

Missing parts of 5_SoftwareStack
by Kamil Iskra 21 Mar '11

21 Mar '11

Dear all, Pete would like to have a fairly complete draft of 5_SoftwareStack by Tuesday evening (yes, tomorrow). Make sure that your components have some milestones in the Deliverables section. Don't go overboard with it; one-two milestones per year for the first three years for each component will be plenty, we mostly want them there so that the reviewers can't complain about a lack of them. Besides, Pete wants the whole part 5 trimmed down to 20 pages (it's currently at 34, and a number of things are still missing, so, short of using a font size 6, it will take a ruthless trimmer to squeeze it; any volunteers?). Speaking of missing things, here's what I noticed after a quick browse through the document: Operating System and Runtime - Linux Integration (Kamil): incomplete Deliverables - Advanced OS Functionality (Ron): missing Deliverables, Collaboration, Support - Qthreads (Ron): missing - I/O Forwarding (Kamil): missing Deliverables, Support - Simulation (Ron): missing Tools - Measurement and Analysis (John, Bronis, Al): complete? (other than the milestones) - Underlying Infrastructure (Bronis, John, Jeff): complete? (other than the milestones) - Tool Integration (Bronis, Al, John, Jeff): mostly complete Programming Models - Advanced Languages (Barb, John): complete? (the deliverables almost read like milestones) - MPI (Rajeev, Rusty): missing Deliverables, Collaboration, Support - Plum (Pavan): missing - Resilience and FT (Al): complete (it even has actual milestones -- for five years!) Data - In-Situ Analysis (Jim, Hank): complete - Checkpoint (Rob): missing Deliverables, Collaboration, Support - Hierarchical Storage (Lee): incomplete Deliverables - I/O Libraries (Gary): complete? - Low Level Storage (Galen): incomplete Deliverables - Metadata (Rob): incomplete Deliverables and Support, missing Collaboration Math (Jack, Mike) - Used a different formula than the rest (one giant component spanning over four pages, with multiple tasks), but other than the milestones, complete? Note: I made some formatting changes to the section headers in *all* component.tex files, so be sure to update your checked out SVN tree before modifying anything to avoid merging conflicts down the road... There is now a top-level Makefile in "5_SoftwareStack" that can be used to build the whole part 5. Feel free to change the order of groups/components as it's currently mostly alphabetical; the relevant files are software-stack.tex (for the top level) and group.tex in each group subdirectory. Kamil

1 0

Short overview of software stack
by Pete Beckman 16 Mar '11

16 Mar '11

Thanks for everyone's work editing together a very short summary of the current SW plans (the longer plans are in the component.tex sheets). This is still a draft, and things may change, but represents our initial priorities. Please continue to keep these documents within this group, since more editing will happen. Also, if there are obvious dependencies in the SW development or integration with Europe or Japan, please let me know. ESC will likely want to put into place MOUs with Europeans and Japanese for SW components, rather than fund them directly, since EESI (in Europe) and the "Post Peta" group in Japan are looking for ways to collaborate. The file is also in the repo: ./esc/review/5_SoftwareStack -Pete

1 0

Subversion server down tomorrow
by Kamil Iskra 16 Mar '11

16 Mar '11

Dear all, Our subversion server, lewis.ci.uchicago.edu, will be down tomorrow morning (Wednesday, March 16th) for hardware maintenance. The work will begin at about 9:30am central and last for no more than two hours, barring anything unexpected. I'll send an all-clear to the list when services have been restored. Kamil

1 1

Complete component sheets and introductory material
by Pete Beckman 11 Mar '11

11 Mar '11

it is critically important that every group complete their initial software component sheets, including addressing the points in the templates. We are trying to prepare an overview for the program offices on the technical priorities we currently have in the plan, and I will be creating a single 1 page summary of the planned technical investments. Don't worry about wordsmithing now, just get the plan into the repo. -Pete

1 0

Meeting Tomorrow
by Pete Beckman 09 Mar '11

09 Mar '11

Tomorrow we have "Balboa #1" reserved. Since our group is too large for a dinner reservation, let's meet after dinner. We will start promptly at 7:30 in the Balboa #1 room, and go over the plans for the ESC materials, and who has what... -Pete

1 1

Programming Models Components:
by Pete Beckman 07 Mar '11

07 Mar '11

Each of the 5 areas are putting together their plan. We want to have the draft component sections checked into the repo by Wednesday night. You can use this as a reference for the programming models work. See the svn templates for details: esc/review/5_SoftwareStack --- Overall Lead for programming models: Barbara Chapman and Rajeev Thakur * Adv-lang: $25M=2FTE, $40M=4FTE Lead: Barbara Chapman Scope: OpenMP, UPC, CAF initially. Resources roughly 1/2 on OpenMP, and 1/4 on each of UPC and CAF. Plan will be to develop advanced language features for OpenMP primarily (since all exascale applications require OpenMP) and extend UPC and CAF into exascale so they can be composed with OpenMP and multithreaded programs. Focus will be on pushing forward scalability features and threading and dynamic behavior needed to address mountain of parallelism and heterogeneous multicore. Features will be tested and then transferred to vendors in co-design process. * Fault tolerance: $25M=2FTE, $40M=3FTE Lead: Al Geist Scope: Refactor concepts and/or core software from CIFTS, using what was successful and reinventing what is needed for exascale, provide a common way for run-time programming model components to share fault information within a user application and pushing notifications to other system software. Create exascale fault model / taxonomy that can be shared across all exascale pieces. * MPI: $25M=2.25FTE, $40M=4FTE Lead: Rajeev Thakur Scope: Extend MPI implementations to scale to exascale systems, with reduced memory consumption and performance scaling. Support new MPI-3 features and future versions of MPI standard. Efficient composition with UPC/CAF/OpenMP and other programming models. Use Plum as common runtime. Support MPI-3 fault tolerance and interoperate with the fault tolerance component of ESC. * Plum: $25M=3FTE, $40M=5FTE Lead: Pavan Balaji Scope: Build a common unified low-level runtime library on top of which multiple programming models (MPI, UPC, CAF, OpenMP, Global Arrays, Chapel, Madness, etc) can be layered as well as integrated. Key design issues include exascale performance, interoperation of multiple higher-level programming models, and support for dynamic parallelism. --- PS. Support for ROSE is required for the Programming Models work, but will be written up only once, in the Tools section.

1 0

Attendees to ASCR Exascale San Diego workshop
by Pete Beckman 07 Mar '11

07 Mar '11

Hello, If you will be in San Diego for the ASCR exascale workshop, please reply to this email with a "yes" -Pete

4 4