- Hpc-admins - mailman3.cels.anl.gov

Meet today?
by John Valdes 12 Sep '12

12 Sep '12

Do folks want to meet today? I myself don't have anything on the agenda. John

1 0

no meeting today
by John Valdes 05 Sep '12

05 Sep '12

Ti's out sick today, and Craig's at home recovering, so I vote for no HPC meeting today unless others have issues they'd like to discuss. John

3 2

Meeting today
by Craig Stacey 22 Aug '12

22 Aug '12

Hunter will stop by and show off check_mk monitoring tool. 3:00, in 5172. If you want to attend by Adobe Connect, let me know and I'll make sure to fire it up. -- Craig

2 2

Re: [HPC-Admins] Cancelled: Weekly HPC Sync
by John Valdes 15 Aug '12

15 Aug '12

Craig is out having fun at the dentist today, but if there's interest, we could still meet. Or we could just "meet" over email. Anyone have anything you want to bring up to the group? Only questions/issues I have: 1) For those who attended yesterday's NetApp presentation, what did you think of their offerings? Do any of you have any interest in their E-Series stuff? 2) Did my parallel Matlab comments clear things up or just make things more confusing? :) John

3 2

NetApp E series
by John Valdes 14 Aug '12

14 Aug '12

Here's a video of the NetApp E5460 (E5400 controller with DE6600 disk enclosure) showing how the drive tray slides out and the drives are removed, plus the internal "bicycle chain" cable management: https://communities.netapp.com/videos/2036 I'm interested in testing one of these out as a potential replacement for LCRC's DDN 9550. Anyone else interested in this? If so, we should coordinate our efforts. John

1 0

Netapp meetings
by Craig Stacey 13 Aug '12

13 Aug '12

These will be held in 2172. -- Craig

1 1

Fwd: Lunch for tomorrow
by Craig Stacey 13 Aug '12

13 Aug '12

Objections only, otherwise this is the plan. -- Craig Begin forwarded message: > From: "Mark Tuggle" <mtuggle(a)usicorp.com> > Subject: Lunch for tomorrow > Date: August 13, 2012 2:38:26 PM CDT > To: "'Craig Stacey'" <stace(a)mcs.anl.gov> > > Hi Craig, > > I made reservation for 10 – 11 folks at La Notte Due Restaurant. 8123 South Cass Ave. > > It has proximity going for it, food has always been good, buffet or menu ordering, and should absorb our gang easily. > > If you have other restaurant preference, I am open. > > > > Best regards, > Mark > > _________________________________________________ > Mark P. Tuggle > > USI Corporation > Office 708-784-1500 x222 > Cell 708-606-9711 > Fax 708-784-0500 > USI Corporation 600 Hillgrove Ave, Unit 4, Western Springs, IL 60558 > www.usicorp.com > Reliable Products. > Responsive Service. > Proven Innovators. >

1 0

NetApp Meetings on 8/14.
by Craig Stacey 13 Aug '12

13 Aug '12

Just a reminder, if you're interested in the morning E-Series deep dive at 10AM or the Cluster Mode deep dive at 1:30 PM (with a lunch offsite provided by USI), please let me know. So far, I have: E series meeting in the morning: Craig Stacey <stace(a)mcs.anl.gov>; Brad Baker (Brad.Baker(a)netapp.com) Hunter Matthews <hunter(a)mcs.anl.gov>; Linda Winkler <winkler(a)mcs.anl.gov>; Dan Olson <dolson(a)mcs.anl.gov>; Joseph Urbanski <jurbanski(a)ci.uchicago.edu>; Max Trefonides <maxadam(a)mcs.anl.gov>; Ti Leggett <leggett(a)mcs.anl.gov>; Wally Nowakowski <wnowakowski(a)anl.gov> Cluster mode deep dive: Craig Stacey <stace(a)mcs.anl.gov>; Brad Baker (Brad.Baker(a)netapp.com) Hunter Matthews <hunter(a)mcs.anl.gov>; Linda Winkler <winkler(a)mcs.anl.gov>; Dan Olson <dolson(a)mcs.anl.gov>; Ken Raffenetti <raffenet(a)mcs.anl.gov>; Joseph Urbanski <jurbanski(a)ci.uchicago.edu>; Max Trefonides <maxadam(a)mcs.anl.gov>; Ti Leggett <leggett(a)mcs.anl.gov>; Wally Nowakowski <wnowakowski(a)anl.gov> Also, taking suggestions on where to go for lunch. -- Craig

2 3

parallel matlab
by John Valdes 09 Aug '12

09 Aug '12

Parallel Matlab came up briefly in yesterday's HPC meeting. At the bottom of this email are some of the costs I got from MathWorks when I inquired about it last year. I didn't remember many of the details yesterday, but digging into this anew, my memory is starting to page back in. There are two products oriented towards running Matlab in "parallel", the Parallel Computing Toolbox and the Distributed Computing Server. The Parallel Computing Toolbox, http://www.mathworks.com/products/parallel-computing/ , appears, from my reading, to be a Matlab Toolbox that provides parallel functions (eg, parallel for loops) and enables parallel capabilities in other Toolboxes. To run in parallel in this mode, the Toolbox provides a mechanism for starting "workers" which you can either control interactively in your main Matlab instance or using some sort of batch script, which you likewise control from a Matlab instance but which run asynchronously. The Parallel Computing Toolbox only allows you to run with workers on the same computer as your Matlab instance, so it seems to be targeted for users of a single, multicore desktop, for example. The Distributed Computing Server, http://www.mathworks.com/products/distriben/index.html, appears to extend the Parallel Computing Toolbox by allowing the workers to run on other computers, eg, compute nodes in a cluster. The use model appears to be that a person runs a main Matlab instance on his desktop or a cluster login node (call this the head instance), and from within that Matlab instance, he can then start workers on other nodes, either as jobs submitted to a scheduler (eg, torque) or perhaps directly on the nodes for clusters running w/o a scheduler. It's not entirely clear from Matlab's website what licenses are required for all this, but from my reading, it looks like one needs at least a license for the Parallel Computing Toolbox for each user running a head Matlab instance, and then a suitable number of Distributed Computing Server worker licenses for each worker that is run on the other nodes. It sounds like the worker licenses effectively "proxy" for the core Matlab and Toolbox licenses, so you don't need to have core matlab and toolbox licenses for each worker. The following example should illustrate how this all works, if I understand it all correctly myself. :) Say you have two Matlab users that have been running Matlab "serially" up til now. In addition to the core Matlab functions, one user also uses the Image Processing Toolbox while the other uses the Optimization Toolbox. For their serial work then, you would need 2 core MATLAB licenses, 1 Image_Toolbox license, and 1 Optimization_Toolbox license. Say now that each user wants to run his Matlab code in parallel, each across 32 workers (say 8 cores on 4 compute nodes in a cluster). In that case, the required license are then still the 2 core MATLAB licenses, 1 Image toolbox license & 1 Optimization toolbox license, but in addition, you would now also need 2 Parallel Toolbox licenses (1 for each user), and 1 Distributed Computing Server licensed for 64 workers. So basically, in addition to the existing licenses, you just have to purchase a Parallel Computing Toolbox license for each user who wants to run code in parallel, and buy enough workers in your Distributed Computing license to cover them all. The Parallel toolbox currently supplies 12 workers (only on one computer), but I don't know if any of those worker "licenses" can be applied towards the count of licensed Distributed Computing workers (eg, continuing with the example, whether you could get by with only 40 Distributed Computing works given that you have 24 workers from the two Parallel Toolbox licenses). The MathWorks sales guy could probably answer that. All that said, if you don't actually have to run parallel code in Matlab (eg, code that processes a shared array across multiple cores) and are only interested in running multiple Matlab instances concurrently on a single node, then one thing that I've noticed that might reduce your license requirements is that Matlab seems to grant licenses on a user+host basis; that is, one user can start multiple instances of Matlab on the same host and only consume 1 Matlab license. The instances can't share data with each other (at least not using any of the built-in Matlab parallel functions), but they can run simultaneously. As a result, on an 8-core compute node, one could in princple start 8 Matlab "jobs" and only use up only 1 Matlab license. Again, all this is mainly just my impression from reading the info on the Matlab website and may not be 100% accurate. Hope this helps nonetheless. John ----- Forwarded message from John Valdes <valdes(a)mcs.anl.gov> ----- Date: Thu, 24 Feb 2011 16:42:49 -0600 From: John Valdes <valdes(a)mcs.anl.gov> To: MCS Systems via RT <systems(a)mcs.anl.gov> Subject: Re: [mcs-systems #64081] matlab On Fri, Feb 11, 2011 at 04:14:10PM -0600, I wrote: > > Also, has anyone looked into licensing the Parallel Computing Toolbox > for Matlab, http://www.mathworks.com/products/parallel-computing/ ? > If so, do you know how much it costs? If not, could you ask our > Matlab sales contact about it or give me their contact info so I can > ask them? A group from the APS is apparently interested in using > parallel Matlab on fusion. I spoke w/ a sales guy at Matlab, and FTR, below are the costs of the Parallel Computing Toolbox (which allows up to 8 workers on the computer running Matlab) and the Distributed Computing Server (which allows you to run parallel Matlab on remote systems such as a cluster or cloud). It looks fairly expensive, so we (LCRC) will need to decide whether to we want to buy this and/or maybe have APS kick in some $$. John ---------------------------------------- From: Robert Segal <Robert.Segal(a)mathworks.com> To: "valdes(a)mcs.anl.gov" <valdes(a)mcs.anl.gov> Subject: Parallel Computing and MATLAB Distributed Computing Server I want to thank you for taking a few minutes to speak with me today. To add Parallel Computing Toolbox to your existing MATLAB license it will cost $4000 per seat. Here is the cost breakdown for MATLAB Distributed Computing Server: 8 workers ..............................................................6,000 16 workers ..........................................................11,500 32 workers ..........................................................21,000 64 workers ..........................................................36,500 96 workers ..........................................................50,000 128 workers ........................................................65,000 160 workers ........................................................77,500 192 workers ........................................................92,500 224 workers ......................................................105,000 256 workers ......................................................115,000 Robert W. Segal Senior Account Manager MathWorks 508-647-7615 508-652-8775 (fax) robert.segal(a)mathworks.com ----- End forwarded message -----

1 0

September Syncs
by Craig Stacey 03 Aug '12

03 Aug '12

You'll get Zimbra notices I'm canceling the syncs that happen in September. I'll be on medical leave at that time - by all means continue to meet in my absence if you feel so inclined. (just be sure to reserve the location). -- Craig

1 0