On 02/12/2013 06:40 PM, Jed Brown wrote:

On Tue, Feb 12, 2013 at 6:06 PM, Tim Tautges <[email protected] <mailto:[email protected]>> wrote:

I'm kind of surprised at the > 10k element crossover myself. For the strong scaling cases, at high core counts, that's not terribly far from the number of DOFS per processor, is it? I guess CPUs will be slower than the Xeon in most cases (BGx), or fewer (Titan), but still.

Which crossover are you referring to? The CPU versus GTX285 at about 20k dofs, but with only very small gains for another order of magnitude?

Yeah, that's what I meant. There seems to be little benefit to both Phis and true GPUs until you get well above 20k DOFs per core, but for strong scaling studies, I wonder how many applications are really that large. Maybe they are on Titan-class machines (fewer CPU cores, more memory/core), but maybe those cores are faster than the Xeon 55xx (not sure about that assertion), but then the crossover would be at even larger DOF counts. If you take into account the added complexity of building those codes and running those machines, I still don't see the payoff to those machines. Anybody know whether that effect showed up in the INCITE proposals? Since users get charged for the GPUs whether they use them or not, I'd expect to see requests skewed toward Mira. - tim

For 2D Laplace, we expect to see strong scaling peter out around a couple thousand dofs per core. It can go a little further on Blue Gene because the network is much faster and the cores are a bit slower.

Titan has a lot of (premium price) GPUs that you have to use to utilize the machine well, but it's unclear whether the architecture is delivering a science/dollars advantage, even if you ignore development costs to port and re-tune codes and the environment costs (it's more complicated to build and run, so it takes people longer to get running). I think the main justification is speculation about what future hardware will look like, not cost-effectiveness today.

-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Adjunct Professor, Engr. Physics ([email protected]) University of Wisconsin-Madison gphone: (608) 354-1459 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706