On Wed, Dec 22, 2010 at 5:54 PM, Satish Balay <[email protected]> wrote:
On Wed, 22 Dec 2010, Yongjun Chen wrote:
On Wed, Dec 22, 2010 at 5:40 PM, Barry Smith <[email protected]> wrote:
Processors: 4 CPUS * 4Cores/CPU, with each core 2500MHz
Memories: 16 *2 GB DDR2 333 MHz, dual channel, data width 64 bit, so the memory Bandwidth for 2 memories is 64/8*166*2*2=5.4GB/s.
Wait a minute. You have 16 cores that share 5.4 GB/s???? This is not enough for iterative solvers, in fact this is absolutely terrible for iterative solvers. You really want 5.4 GB/s PER core! This machine is absolutely inappropriate for iterative solvers. No package can give you good speedups on this machine.
Barry, there are 16 memories, every 2 memories make up one dual channel, thus in this machine there are 8 dual channel, each dual channel has the memory bandwidth 5.4GB/s.
What hardware is this? [processor/chipset?]
By dmidecode, it shows the processor is Handle 0x0010, DMI type 4, 40 bytes Processor Information Socket Designation: CPU 4 Type: Central Processor Family: Quad-Core Opteron Manufacturer: AMD ID: 06 05 F6 40 74 03 E8 3D Signature: Family 5, Model 0, Stepping 6 Flags: DE (Debugging extension) TSC (Time stamp counter) MSR (Model specific registers) PAE (Physical address extension) CX8 (CMPXCHG8 instruction supported) APIC (On-chip APIC hardware supported) CLFSH (CLFLUSH instruction supported) DS (Debug store) ACPI (ACPI supported) MMX (MMX technology supported) FXSR (Fast floating-point save and restore) SSE2 (Streaming SIMD extensions 2) SS (Self-snoop) HTT (Hyper-threading technology) TM (Thermal monitor supported) Version: Quad-Core AMD Opteron(tm) Processor 8360 SE Voltage: 1.5 V External Clock: 200 MHz Max Speed: 4600 MHz Current Speed: 2500 MHz Status: Populated, Enabled Upgrade: Other L1 Cache Handle: 0x0011 L2 Cache Handle: 0x0012 L3 Cache Handle: 0x0013 Serial Number: N/A Asset Tag: N/A Part Number: N/A Core Count: 4 Core Enabled: 4 Characteristics: 64-bit capable
From what you say - it looks like each chip has 4cores, and 2 dual-channel memory controllers for each of them.
The question is - does the hardware provide scalable memory-bandwidth per core? Most machines don't.
This point is not clear for me right now.
I.e the same 5.4*2GB/s is avilable for 1 core run as well as the 4 core run.
So if the algorithm is able to use 5.4GB/s [or more] for 1 threads, 10.8 [or more] for 2 threads - you would just see scalable performance from 1 to 2, and 3, 4 would perhaps be slightly incremental to the 2-core performance.
Satish