duplicate dataloop directory
What would it take to be able to get rid of one of the two copies of the dataloop code that we have in the mpich2 source tree? My understanding is that we use the src/mpid/common copy and the src/mpi/ romio/common copy is unused except by others like OpenMPI that use our MPI-IO implementation separately. The two copies have diverged over time through developer error and this has the potential to cause some pretty frustrating errors down the road. Possible solutions: 1) Use the svn:externals feature to cause the code to be checked out into the ROMIO directory automatically. Unfortunately this svn feature seems a bit fragile and I'm not sure how we would make it play nicely with the two different build systems. 2) Delete the copy in the romio dir. Add a "make romio-export" target or similar that copies the code into the romio tree that is used whenever someone is trying to incorporate ROMIO into their non- mpich2 codebase. 3) Delete the copy in the romio dir outright and just make anyone who is trying to use ROMIO separately copy it by hand. 4) Make the romio dir code the canonical code and get rid of the mpid/ common code. I like this because ROMIO will almost always be present in MPICH2. It has problems wrt what happens if someone configures with --disable-romio and probably something else that I'm not thinking of. 5) Pull ROMIO out into a separate library of some sort and make MPICH2 depend on it in a more traditional fashion, rather than building it into the library. This could work well for folks like OpenMPI as well, since they might have less work to do to incorporate it into their build system. OTOH, this is probably harder than it appears to me at first glance. 6) Change to some revision control system that can handle hard-links within the repository. This would be a high-effort change and I don't know off the top of my head whether there even are any systems that handle hard-links properly. I favor something like #2, although I'm not exactly familiar with all of the issues here. Thoughts? -Dave
On Mon, Aug 25, 2008 at 05:14:16PM -0500, Dave Goodell wrote:
My understanding is that we use the src/mpid/common copy and the src/mpi/romio/common copy is unused except by others like OpenMPI that use our MPI-IO implementation separately.
Not exactly. src/mpid/common is used in MPICH2 and src/mpi/romio/common is not used by anything (but could and probably should be one day).
1) Use the svn:externals feature to cause the code to be checked out into the ROMIO directory automatically. Unfortunately this svn feature seems a bit fragile and I'm not sure how we would make it play nicely with the two different build systems.
Yeah, I don't like this either.
2) Delete the copy in the romio dir. Add a "make romio-export" target or similar that copies the code into the romio tree that is used whenever someone is trying to incorporate ROMIO into their non-mpich2 codebase.
Nobody needs to worry about romio's dataloop dir yet. Maybe we can just leave it alone altogether?
3) Delete the copy in the romio dir outright and just make anyone who is trying to use ROMIO separately copy it by hand.
I'll have to defer to robr.. when he's back from the wilderness. He knows best what MPICH2-isms had to be removed for ROMIO's use.
4) Make the romio dir code the canonical code and get rid of thek mpid/ common code. I like this because ROMIO will almost always be present in MPICH2. It has problems wrt what happens if someone configures with --disable-romio and probably something else that I'm not thinking of.
Intriguing. Will probably require robr's input though..
5) Pull ROMIO out into a separate library of some sort and make MPICH2 depend on it in a more traditional fashion, rather than building it into the library. This could work well for folks like OpenMPI as well, since they might have less work to do to incorporate it into their build system. OTOH, this is probably harder than it appears to me at first glance.
We probably should have done this at the dawn of MPICH2, but this wouldn't be terribly hard to do right now. We make a pass at this from time to time. There are a lot of MPICH2 features in ROMIO, and 5 years of ROMIO assuming it would be part of MPICH2 the vast vast majority of the time. Here again, a ROMIO test suite would be terrific.
6) Change to some revision control system that can handle hard-links within the repository. This would be a high-effort change and I don't know off the top of my head whether there even are any systems that handle hard-links properly.
hah. veto! ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
The code in romio/common is incomplete and is not used by anyone. It does not even get compiled at present. Is there some way to let it be there but not get checked out say? Rajeev
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Dave Goodell Sent: Monday, August 25, 2008 5:14 PM To: [email protected] Subject: [mpich2-core] duplicate dataloop directory
What would it take to be able to get rid of one of the two copies of the dataloop code that we have in the mpich2 source tree? My understanding is that we use the src/mpid/common copy and the src/mpi/ romio/common copy is unused except by others like OpenMPI that use our MPI-IO implementation separately. The two copies have diverged over time through developer error and this has the potential to cause some pretty frustrating errors down the road.
Possible solutions: 1) Use the svn:externals feature to cause the code to be checked out into the ROMIO directory automatically. Unfortunately this svn feature seems a bit fragile and I'm not sure how we would make it play nicely with the two different build systems. 2) Delete the copy in the romio dir. Add a "make romio-export" target or similar that copies the code into the romio tree that is used whenever someone is trying to incorporate ROMIO into their non- mpich2 codebase. 3) Delete the copy in the romio dir outright and just make anyone who is trying to use ROMIO separately copy it by hand. 4) Make the romio dir code the canonical code and get rid of the mpid/ common code. I like this because ROMIO will almost always be present in MPICH2. It has problems wrt what happens if someone configures with --disable-romio and probably something else that I'm not thinking of. 5) Pull ROMIO out into a separate library of some sort and make MPICH2 depend on it in a more traditional fashion, rather than building it into the library. This could work well for folks like OpenMPI as well, since they might have less work to do to incorporate it into their build system. OTOH, this is probably harder than it appears to me at first glance. 6) Change to some revision control system that can handle hard-links within the repository. This would be a high-effort change and I don't know off the top of my head whether there even are any systems that handle hard-links properly.
I favor something like #2, although I'm not exactly familiar with all of the issues here. Thoughts?
-Dave
Well, we could just delete it. One could always get it back from the repo at a later point if needed. But I'll let robl make the call on doing this. -d On 08/25/2008 05:45 PM, Rajeev Thakur wrote:
The code in romio/common is incomplete and is not used by anyone. It does not even get compiled at present. Is there some way to let it be there but not get checked out say?
Rajeev
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Dave Goodell Sent: Monday, August 25, 2008 5:14 PM To: [email protected] Subject: [mpich2-core] duplicate dataloop directory
What would it take to be able to get rid of one of the two copies of the dataloop code that we have in the mpich2 source tree? My understanding is that we use the src/mpid/common copy and the src/mpi/ romio/common copy is unused except by others like OpenMPI that use our MPI-IO implementation separately. The two copies have diverged over time through developer error and this has the potential to cause some pretty frustrating errors down the road.
Possible solutions: 1) Use the svn:externals feature to cause the code to be checked out into the ROMIO directory automatically. Unfortunately this svn feature seems a bit fragile and I'm not sure how we would make it play nicely with the two different build systems. 2) Delete the copy in the romio dir. Add a "make romio-export" target or similar that copies the code into the romio tree that is used whenever someone is trying to incorporate ROMIO into their non- mpich2 codebase. 3) Delete the copy in the romio dir outright and just make anyone who is trying to use ROMIO separately copy it by hand. 4) Make the romio dir code the canonical code and get rid of the mpid/ common code. I like this because ROMIO will almost always be present in MPICH2. It has problems wrt what happens if someone configures with --disable-romio and probably something else that I'm not thinking of. 5) Pull ROMIO out into a separate library of some sort and make MPICH2 depend on it in a more traditional fashion, rather than building it into the library. This could work well for folks like OpenMPI as well, since they might have less work to do to incorporate it into their build system. OTOH, this is probably harder than it appears to me at first glance. 6) Change to some revision control system that can handle hard-links within the repository. This would be a high-effort change and I don't know off the top of my head whether there even are any systems that handle hard-links properly.
I favor something like #2, although I'm not exactly familiar with all of the issues here. Thoughts?
-Dave
Dave and I talked about this yesterday. I think the decision was to tag with a "last-version-with-robrs-unused-dataloop-junk" or something and then kill it? -- Rob On Aug 25, 2008, at 5:54 PM, Darius Buntinas wrote:
Well, we could just delete it. One could always get it back from the repo at a later point if needed. But I'll let robl make the call on doing this.
-d
On 08/25/2008 05:45 PM, Rajeev Thakur wrote:
The code in romio/common is incomplete and is not used by anyone. It does not even get compiled at present. Is there some way to let it be there but not get checked out say? Rajeev
-----Original Message----- From: [email protected] [mailto:[email protected] ] On Behalf Of Dave Goodell Sent: Monday, August 25, 2008 5:14 PM To: [email protected] Subject: [mpich2-core] duplicate dataloop directory
What would it take to be able to get rid of one of the two copies of the dataloop code that we have in the mpich2 source tree? My understanding is that we use the src/mpid/common copy and the src/ mpi/ romio/common copy is unused except by others like OpenMPI that use our MPI-IO implementation separately. The two copies have diverged over time through developer error and this has the potential to cause some pretty frustrating errors down the road.
Possible solutions: 1) Use the svn:externals feature to cause the code to be checked out into the ROMIO directory automatically. Unfortunately this svn feature seems a bit fragile and I'm not sure how we would make it play nicely with the two different build systems. 2) Delete the copy in the romio dir. Add a "make romio-export" target or similar that copies the code into the romio tree that is used whenever someone is trying to incorporate ROMIO into their non- mpich2 codebase. 3) Delete the copy in the romio dir outright and just make anyone who is trying to use ROMIO separately copy it by hand. 4) Make the romio dir code the canonical code and get rid of the mpid/ common code. I like this because ROMIO will almost always be present in MPICH2. It has problems wrt what happens if someone configures with --disable-romio and probably something else that I'm not thinking of. 5) Pull ROMIO out into a separate library of some sort and make MPICH2 depend on it in a more traditional fashion, rather than building it into the library. This could work well for folks like OpenMPI as well, since they might have less work to do to incorporate it into their build system. OTOH, this is probably harder than it appears to me at first glance. 6) Change to some revision control system that can handle hard- links within the repository. This would be a high-effort change and I don't know off the top of my head whether there even are any systems that handle hard-links properly.
I favor something like #2, although I'm not exactly familiar with all of the issues here. Thoughts?
-Dave
I'll tag it as "last_unused_romio_dataloop_code" and then delete it when I get back to the States. -Dave On Sep 9, 2008, at 9:32 AM, Rob Ross wrote:
Dave and I talked about this yesterday. I think the decision was to tag with a "last-version-with-robrs-unused-dataloop-junk" or something and then kill it? -- Rob
On Aug 25, 2008, at 5:54 PM, Darius Buntinas wrote:
Well, we could just delete it. One could always get it back from the repo at a later point if needed. But I'll let robl make the call on doing this.
-d
participants (5)
-
Darius Buntinas -
Dave Goodell -
Rajeev Thakur -
Rob Ross -
Robert Latham