Richard Tran Mills <[email protected]> writes:

Folks,

I am wondering how PETSc's BuildSystem currently chooses the memory alignment to use. In the example file I provide in the repo for the Cori KNL nodes, I specify '--with-memalign=64' to match the cache line width. If I don't do this, then configure chooses a 16 byte alignment. Do people think that we should try to make a better effort to choose a more appropriate alignment?

I believe that all modern x86 CPUs use a 64 byte cache width, so should we be defaulting to that? I don't know how much this matters on a Xeon processor, but it can be important on Xeon Phi.

The preference for 16 was for SSE2, prior to AVX and AVX512. With AVX, I believe there is no particular reason to prefer more than 32-byte alignment -- it just ensures that objects cannot share cache lines. Harmless with large arrays, but not ideal for smaller allocations like nodes in a linked list. (PETSc doesn't use these very often, but it would be nice if control structures don't soak up more cache than they need.)