On Thu, 2014-02-27 at 14:30 +0100, Matthieu Dorier wrote:
Hi,
Simple questions out of curiosity: I see some counters like, for instance, CP_ACCESS[1-4]_ACCESS (described as "4 most common access sizes"). Does it mean that for each file accessed, Darshan will keep in memory a full histogram of all the access sizes until the end of the program, to be able to get the most frequent ones when writing the log file? If so, isn't it memory-consuming in case of a large number of accesses with different sizes? Besides, why 4? Is it motivated by some analysis that showed 4 to be good enough for most applications?
Darshan does keep a histogram, as you've seen, but the space for that histogram is bounded by the number of buckets -- and the buckets are fixed. The ACCESS counters come from simply a Most Frequently Used list with four slots. Again, bounded by the number of slots no matter how crazy the access pattern might be. ==rob