Periodically flushing tags to disk?
Happy 4th of July everybody! We are currently running a simulation, which creates new tags at runtime and only at the end writes the entire MOAB to disk (via DMMoabOutput). This method is wasteful and dangerous because we might run out of memory and, in case of premature termination, end up with nothing (on disk). We were wondering if there was a way to flush tags to disk periodically and limit our memory footprint? Ideally, this would not require writing the mesh again. We are also not wedded to the PETSc MOAB interface. If there is an MOAB-native way, that would be fine as well. Thanks, G.
Hello, AFAIK, Moab does not offer a way to write only tags on a file. There is a way to write only selected mesh sets and selected tags virtual ErrorCode write_file( const char* file_name, const char* file_type = 0, const char* options = 0, const EntityHandle* output_sets = 0, int num_output_sets = 0, const Tag* tag_list = 0, int num_tags = 0 ); If output_sets is 0, complete mesh is written if tag_list is 0, all tags pertinent to the mesh entities in "output_sets" are written If tags_list is not NULL, and contains an array of tags, only those will be written to the file After writing this file, you may delete the tags to free up memory Not sure if this helps your scenario Thanks, Iulian ________________________________________ From: [email protected] [[email protected]] on behalf of Gerd Heber [[email protected]] Sent: Thursday, July 02, 2015 9:36 AM To: MOAB dev Subject: [MOAB-dev] Periodically flushing tags to disk? Happy 4th of July everybody! We are currently running a simulation, which creates new tags at runtime and only at the end writes the entire MOAB to disk (via DMMoabOutput). This method is wasteful and dangerous because we might run out of memory and, in case of premature termination, end up with nothing (on disk). We were wondering if there was a way to flush tags to disk periodically and limit our memory footprint? Ideally, this would not require writing the mesh again. We are also not wedded to the PETSc MOAB interface. If there is an MOAB-native way, that would be fine as well. Thanks, G.
I spent a lot of time thinking about that, and came to the following conclusions: a) there were enough examples of file formats which duplicate the mesh for each time slice (actually, these look more like dumping a new file for each time) that taking that approach was not unique to MOAB; furthermore, there is tool-level support in paraview for reading time data as a series of individual files, including mesh in each file, such that a trivial amount of effort is required to implement a time-varying reader given a single-time reader. b) in the in-memory database itself, it is unlikely that a code would choose to write a new tag for every timestep; rather, it would choose to have n different tags, for saving n timesteps worth of data, and cycle between them. For those reasons, I eventually decided it wasn't a high enough priority to develop both individual-timestep/tag writing as well as time-dependent tags in MOAB. I understood/stand that there would be a significant execution time advantage for doing that, but there always seemed to be bigger fish to fry (like getting an honest to goodness simulation code written based on MOAB). Gerd, if you've gotten there now, maybe it is time to consider writing that kind of writer (and reader, since you'll also have to post-process the results of course). - tim On 07/02/2015 09:36 AM, Gerd Heber wrote:
Happy 4th of July everybody!
We are currently running a simulation, which creates new tags at runtime and only at the end writes the entire MOAB to disk (via DMMoabOutput). This method is wasteful and dangerous because we might run out of memory and, in case of premature termination, end up with nothing (on disk). We were wondering if there was a way to flush tags to disk periodically and limit our memory footprint? Ideally, this would not require writing the mesh again. We are also not wedded to the PETSc MOAB interface. If there is an MOAB-native way, that would be fine as well.
Thanks, G.
-- Timothy J. Tautges Manager, Directed Meshing, CD-adapco Phone: (631) 629-3139 (internal: x29130) Mobile/Gvoice: (608) 354-1459 [email protected]
participants (3)
-
Gerd Heber -
Grindeanu, Iulian R. -
Tim Tautges