Hey Guys, Reposting my last message, just so it's archived. Something else to note, to start this has a count of 17 repos. 2-3 of these at least will be removed for ACME (all MPAS components). And anything that we don't need (do we need dead_share) can also be removed. So, I've been trying to setup a "release" version of CESM in git (as a starting place for ACME). And I just thought I'd share some of my experiences so far. Currently I've been trying to preserve all of the history from CESM's repo. One of the problems I've run into however, is that CESM is currently a mess of svn externals that point all over the root of the repository. It could be possible to convert all of the history to git and have it work properly, but it would either take a robust (and / or hacky) script or a lot of time. Also, several of these repositories are nested, and should probably be joined into a repo (i.e. all of the scripts and tools should be joined together). What might be a workable solution would be to pick the latest public release of CESM, and start the repos from there. Then we can test setting things up in a correct manner. After that, we should be able to easily incorporate history modifications from the svn repo (assuming we remember the revision number we want). It seems like from here: http://www.cesm.ucar.edu/models/cesm1.2/ we can safely start from 1.2. And then we can setup branches as needed. Here is a rough example of what I'm planning to setup for the multiple repo case, at least to begin with. https://gist.github.com/douglasjacobsen/89322ebcc746dfe51d40 Any comments or questions would be more than welcome. Thanks, Doug
Douglas Jacobsen <[email protected]> writes:
Also, several of these repositories are nested, and should probably be joined into a repo (i.e. all of the scripts and tools should be joined together).
Agreed.
What might be a workable solution would be to pick the latest public release of CESM, and start the repos from there. Then we can test setting things up in a correct manner. After that, we should be able to easily incorporate history modifications from the svn repo (assuming we remember the revision number we want).
It seems like from here: http://www.cesm.ucar.edu/models/cesm1.2/ we can safely start from 1.2. And then we can setup branches as needed.
Is there a reason not to do subtree merges of whichever components we want to consolidate as of the 1.2 release? The UI for git-blame and such is not so good, but at least it would be in the history. I talked to Sean Santos on Friday afternoon. He thought that the fundamental resistance to switching to Git was the effort and expertise, but that CSEG will be watching to see what we come up with. He says they will be busy over the summer, but that in the fall (e.g., September), it might be possible to decide whether to switch. He agrees that excessive privacy is detrimental/expensive, and would like to see a way that we can at least share non-science code changes at lower latency than paper publication. He pointed out some funny svn external relationships, such as scripts and machines, which are not uniquely placed relative to each other when looking at CESM versus CAM. As an experiment, I stripped the handful of binary files (mostly NetCDF) out of the repository, in which case a shallow clone of all of CESM will have a 16 MB .git directory. I figure this is an acceptable size for a minimal clone. Having a single CESM repository (modulo independent projects like MPAS) would simplify the scripts and avoid the need for various mini-CESM implementations. He says that many people work with several clones of the svn repository because they are either making source changes or because a run is in the queue. Current runs depend on some files found in the source tree. (He thinks it's feasible to copy those into case directories, in which case it would be safe to switch branches and work on a different case after queuing a job.) Of course each of the svn clones uses double storage because it includes a copy of everything in .svn. (Duplicate git clones are cheaper because they use hard links and the data is compressed.) Anyway, it's worth ensuring that it's easy and reliable to switch branches in the same clone or in different clones.
On 05/19/2014 01:44 PM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Also, several of these repositories are nested, and should probably be joined into a repo (i.e. all of the scripts and tools should be joined together). Agreed.
What might be a workable solution would be to pick the latest public release of CESM, and start the repos from there. Then we can test setting things up in a correct manner. After that, we should be able to easily incorporate history modifications from the svn repo (assuming we remember the revision number we want).
It seems like from here: http://www.cesm.ucar.edu/models/cesm1.2/ we can safely start from 1.2. And then we can setup branches as needed. Is there a reason not to do subtree merges of whichever components we want to consolidate as of the 1.2 release? The UI for git-blame and such is not so good, but at least it would be in the history.
I'm just trying to setup with as many submodules as possible to give the opposite end of the spectrum from what Jim was testing (related to the telecon 2 fridays ago).
I talked to Sean Santos on Friday afternoon. He thought that the fundamental resistance to switching to Git was the effort and expertise, but that CSEG will be watching to see what we come up with. He says they will be busy over the summer, but that in the fall (e.g., September), it might be possible to decide whether to switch. He agrees that excessive privacy is detrimental/expensive, and would like to see a way that we can at least share non-science code changes at lower latency than paper publication.
He pointed out some funny svn external relationships, such as scripts and machines, which are not uniquely placed relative to each other when looking at CESM versus CAM. As an experiment, I stripped the handful of binary files (mostly NetCDF) out of the repository, in which case a shallow clone of all of CESM will have a 16 MB .git directory. I figure this is an acceptable size for a minimal clone. Having a single CESM repository (modulo independent projects like MPAS) would simplify the scripts and avoid the need for various mini-CESM implementations.
He says that many people work with several clones of the svn repository because they are either making source changes or because a run is in the queue. Current runs depend on some files found in the source tree. (He thinks it's feasible to copy those into case directories, in which case it would be safe to switch branches and work on a different case after queuing a job.) Of course each of the svn clones uses double storage because it includes a copy of everything in .svn. (Duplicate git clones are cheaper because they use hard links and the data is compressed.) Anyway, it's worth ensuring that it's easy and reliable to switch branches in the same clone or in different clones. Yeah, I keep describing to people here that CESM is a mess of svn externals. It has externals that point to externals that point to externals. Which is one of the reasons I'm thinking we might want to just abandon the old history.
In terms of CESM usage with git, you can also have separate working directories. Where you have a single clone but can work on multiple branches. Something I learned as well with the current SVN usage (even though the CSEG people don't like to admit this), is that some of the CESM developer / users really do use SourceMods in place of branches. Essentially, they will develop in SourceMods, tar up their changes, and ship them off to CSEG for them to make a branch with the changes. But the original develop won't maintain the branch at all. Anyway, I'll point you guys to what I have as soon as I complete it. Doug
Douglas Jacobsen <[email protected]> writes:
On 05/19/2014 01:44 PM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
It seems like from here: http://www.cesm.ucar.edu/models/cesm1.2/ we can safely start from 1.2. And then we can setup branches as needed. Is there a reason not to do subtree merges of whichever components we want to consolidate as of the 1.2 release? The UI for git-blame and such is not so good, but at least it would be in the history.
I'm just trying to setup with as many submodules as possible to give the opposite end of the spectrum from what Jim was testing (related to the telecon 2 fridays ago).
Okay, so the notation "Repo" should be equated with "submodule" in your Gist?
In terms of CESM usage with git, you can also have separate working directories. Where you have a single clone but can work on multiple branches.
Normally this is done via a local clone. GIT_DIR has only one HEAD and only one index, so you really have separate clones with object sharing behind the scenes (multiple ways to do that). Unless you use "plumbing" commands, which we really don't want to encourage.
Something I learned as well with the current SVN usage (even though the CSEG people don't like to admit this), is that some of the CESM developer / users really do use SourceMods in place of branches. Essentially, they will develop in SourceMods, tar up their changes, and ship them off to CSEG for them to make a branch with the changes. But the original develop won't maintain the branch at all.
So this is certainly no harder in Git than in Svn. Also, for developers that only ever make linear commits in their branch (no merges), it looks to me like GitHub's svn support would be adequate.
Okay, so the notation "Repo" should be equated with "submodule" in your Gist? That is definitely a fair assessment. Some of them can be consolidated and might not live within the ACME project, but we'll see.
Normally this is done via a local clone. GIT_DIR has only one HEAD and only one index, so you really have separate clones with object sharing behind the scenes (multiple ways to do that). Unless you use "plumbing" commands, which we really don't want to encourage. Yeah, I use https://gist.github.com/douglasjacobsen/6e35783d5ff0dab48436 for my development (I didn't write the script, it's packaged with git). It helps me have separate working directories with a shared history.
So this is certainly no harder in Git than in Svn. Also, for developers that only ever make linear commits in their branch (no merges), it looks to me like GitHub's svn support would be adequate.
Agreed. It's just as easy in git. However, I think it's something we largely want to discourage. Within git, it's a lot easier for someone to just commit to their local repo. And then when they want to share it, push it to some "shared repo" (aka. fork). I would rather see a developers changed be tied to the developer rather than the "support" person they emailed them too. Also, it's not clear we'll have the resources to support this type of development. Mariana made it seem like they had already decided Github's svn support was insufficient for their needs, but I'm not sure what made them come to that conclusion. As you said, it should be easy enough to support svn usage within the repo if someone "requires" it. Doug
Here you guys go. https://github.com/ACME-Climate/release I don't think this will build yet, but only because of PIO. PIO has a funky directory structure, and I just copied the whole thing. It would be easy enough to fix, but for now I doubt it will work as expected. Doug On 05/19/2014 02:21 PM, Douglas Jacobsen wrote:
Okay, so the notation "Repo" should be equated with "submodule" in your Gist? That is definitely a fair assessment. Some of them can be consolidated and might not live within the ACME project, but we'll see.
Normally this is done via a local clone. GIT_DIR has only one HEAD and only one index, so you really have separate clones with object sharing behind the scenes (multiple ways to do that). Unless you use "plumbing" commands, which we really don't want to encourage. Yeah, I use https://gist.github.com/douglasjacobsen/6e35783d5ff0dab48436 for my development (I didn't write the script, it's packaged with git). It helps me have separate working directories with a shared history.
So this is certainly no harder in Git than in Svn. Also, for developers that only ever make linear commits in their branch (no merges), it looks to me like GitHub's svn support would be adequate.
Agreed. It's just as easy in git. However, I think it's something we largely want to discourage. Within git, it's a lot easier for someone to just commit to their local repo. And then when they want to share it, push it to some "shared repo" (aka. fork). I would rather see a developers changed be tied to the developer rather than the "support" person they emailed them too. Also, it's not clear we'll have the resources to support this type of development.
Mariana made it seem like they had already decided Github's svn support was insufficient for their needs, but I'm not sure what made them come to that conclusion. As you said, it should be easy enough to support svn usage within the repo if someone "requires" it.
Doug _______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Just fyi, I changed the repo name to follow what it's intended purpose is. https://github.com/ACME-Climate/multiple_repo_example Doug On 05/19/2014 02:43 PM, Douglas Jacobsen wrote:
Here you guys go.
https://github.com/ACME-Climate/release
I don't think this will build yet, but only because of PIO. PIO has a funky directory structure, and I just copied the whole thing. It would be easy enough to fix, but for now I doubt it will work as expected.
Doug
On 05/19/2014 02:21 PM, Douglas Jacobsen wrote:
Okay, so the notation "Repo" should be equated with "submodule" in your Gist? That is definitely a fair assessment. Some of them can be consolidated and might not live within the ACME project, but we'll see.
Normally this is done via a local clone. GIT_DIR has only one HEAD and only one index, so you really have separate clones with object sharing behind the scenes (multiple ways to do that). Unless you use "plumbing" commands, which we really don't want to encourage. Yeah, I use https://gist.github.com/douglasjacobsen/6e35783d5ff0dab48436 for my development (I didn't write the script, it's packaged with git). It helps me have separate working directories with a shared history.
So this is certainly no harder in Git than in Svn. Also, for developers that only ever make linear commits in their branch (no merges), it looks to me like GitHub's svn support would be adequate.
Agreed. It's just as easy in git. However, I think it's something we largely want to discourage. Within git, it's a lot easier for someone to just commit to their local repo. And then when they want to share it, push it to some "shared repo" (aka. fork). I would rather see a developers changed be tied to the developer rather than the "support" person they emailed them too. Also, it's not clear we'll have the resources to support this type of development.
Mariana made it seem like they had already decided Github's svn support was insufficient for their needs, but I'm not sure what made them come to that conclusion. As you said, it should be easy enough to support svn usage within the repo if someone "requires" it.
Doug _______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash. Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
On 05/20/2014 07:59 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
https://github.com/ACME-Climate/multiple_repo_example Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash.
Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
The dashes vs. underscores doesn't matter to me much. I can change it to use dashes if preferred though. I haven't started working on a workflow at all, I mostly just wanted to get the repos up so we could play with them and see how they work for different use cases. It would probably useful if all of us tried to explore how to perform some actions. Maybe starting from the list you mentioned, we could each take a few actions and try to figure out how they work both in this setup, and in the one Jim is working on. That way we can compare and contrast the two setups for the different workflow options. Currently I hadn't planned no adding scripting layers, but if it would make it easier to work in the parent repo it might not be that hard. That might be something we could revisit if it's something we think we need. Either way, the repos should all mostly be in a state we can play around with different options for setting them up. For now we have 1 more repo we can setup. So, in order to test a different workflow / setup we'll need to trash one, or combine some. Doug
Douglas Jacobsen <[email protected]> writes:
The dashes vs. underscores doesn't matter to me much. I can change it to use dashes if preferred though.
I just like consistency.
I haven't started working on a workflow at all, I mostly just wanted to get the repos up so we could play with them and see how they work for different use cases. It would probably useful if all of us tried to explore how to perform some actions.
Maybe starting from the list you mentioned, we could each take a few actions and try to figure out how they work both in this setup, and in the one Jim is working on. That way we can compare and contrast the two setups for the different workflow options.
Currently I hadn't planned no adding scripting layers, but if it would make it easier to work in the parent repo it might not be that hard. That might be something we could revisit if it's something we think we need.
Automating the easy case is not that hard, but making it robust is very hard (as evidenced by many people trying and nobody producing something good). I would rather avoid such scripting, especially if the failure modes are not predictable.
Automating the easy case is not that hard, but making it robust is very hard (as evidenced by many people trying and nobody producing something good). I would rather avoid such scripting, especially if the failure modes are not predictable. I agree, I think we should avoid scripting unless it's necessary. I think it's going to make it harder for people to work with the repo in general.
Though, if the repo has minimal people working in it, it might not be a bad idea to have some scripts that can be used to make their lives easier. If the majority of ACME is going to be working in a single repo, scripts are going to cause us headaches. At least, that's my opinion... Doug
Douglas Jacobsen <[email protected]> writes:
Though, if the repo has minimal people working in it, it might not be a bad idea to have some scripts that can be used to make their lives easier. If the majority of ACME is going to be working in a single repo, scripts are going to cause us headaches. At least, that's my opinion...
I don't think single-repo vs multi-repo is relevant. Work should be confined to a branch and there may be multiple forks. With multi-repo, the script has to do something about the relationship between multiple repos. If the script is capable of pushing, then a mistake could have consequences in several repos, which is surely more complicated to fix. Anyway, if adopting a script, expect that the author gets support mail every time the script fails. Add to that education to use the script (which won't be useful with any other Git projects) and the script would have to be very easy to use and reliable to justify its existence. I have no problem with scripts used by SE/integrators, especially those people that could have more-or-less written the script themselves.
I don't think single-repo vs multi-repo is relevant. Work should be confined to a branch and there may be multiple forks. With multi-repo, the script has to do something about the relationship between multiple repos. If the script is capable of pushing, then a mistake could have consequences in several repos, which is surely more complicated to fix.
I mostly mean that in a multi-repo case, the script needs to deal with "synchronizing" the different repos. While in a single-repo case the script needs to deal with propagating changes to external repos. Either way, these are details we hopefully will be able to explore in these test repos.
Anyway, if adopting a script, expect that the author gets support mail every time the script fails. Add to that education to use the script (which won't be useful with any other Git projects) and the script would have to be very easy to use and reliable to justify its existence.
Yes, I completely agree.
Douglas Jacobsen <[email protected]> writes:
I mostly mean that in a multi-repo case, the script needs to deal with "synchronizing" the different repos. While in a single-repo case the script needs to deal with propagating changes to external repos.
What does "propagating changes to external repos" mean? "Single-repo" to me means that development of all the components in the single repository happens in that repository, so we wouldn't need "external repos" containing each component separately. (What is the value in having an external repo if it can't be used/tested on its own?)
On 05/20/2014 08:49 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
I mostly mean that in a multi-repo case, the script needs to deal with "synchronizing" the different repos. While in a single-repo case the script needs to deal with propagating changes to external repos. What does "propagating changes to external repos" mean?
"Single-repo" to me means that development of all the components in the single repository happens in that repository, so we wouldn't need "external repos" containing each component separately. (What is the value in having an external repo if it can't be used/tested on its own?) All MPAS components require having a separate repo. So, if we want to do development in a single repo the changes to MPAS components will need to be propagated back to the external repos.
Douglas Jacobsen <[email protected]> writes:
All MPAS components require having a separate repo. So, if we want to do development in a single repo the changes to MPAS components will need to be propagated back to the external repos.
I thought we agreed that MPAS should be a submodule, since they want to do development independently and make "releases" to ACME? My comment was referring to components like CLM, CAM, the data implementations, etc.
On 05/20/2014 09:11 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
All MPAS components require having a separate repo. So, if we want to do development in a single repo the changes to MPAS components will need to be propagated back to the external repos. I thought we agreed that MPAS should be a submodule, since they want to do development independently and make "releases" to ACME? My comment was referring to components like CLM, CAM, the data implementations, etc.
Well, right now the multi-repo setup I'm working on is assuming CAM, CLM, the data implementation, etc are all separate repos (as per the telecon ~2 weeks ago). I think the understanding from that telecon was that we would likely use a hybrid approach (mix between multi-repo and single-repo). Also, after last week's meeting I think some people decided we needed to have and atmosphere developer contributing to this task to ensure all of their requirements were met as well. CAM was the main concern for both ACME and CSEG last week, since it seems like they have quite a few shared developments in the short term that need to be communicated between both repos. I'm not sure CLM falls into this category, but it's good to think about. Also, it's still rather unclear how and to what level we'll need to support long term communication with the CESM repo. In my mind, this is much more difficult with a single repo than multiple repos, but I'd be more than happy to be proven wrong about it. It seems like this topic is something none of us have really explored up until getting involved in ACME, so it's good to scope options. Assuming CSEG is not willing to change (as they said last week) for at least roughly 3-6 months and there are some shared developments that we need to incorporate from the CESM repo (I think both CLM and CAM have these, but at a minimum CAM does), this is mostly going to be something we need to accomadate if it's something we want. I'm not super familiar with hooks, but could some of the issues we have be dealt with via hooks? i.e. maybe a pre-commit hook could check if submodules have uncommitted changes and fail if that's the case? Then we could have a single script that could be used to clone/setup the project (including some repo that contains our custom hooks). This way we might avoid the need to use any non-git commands (for typical developers) but catch the places we think a problem might arise. Doug
Douglas Jacobsen <[email protected]> writes:
Well, right now the multi-repo setup I'm working on is assuming CAM, CLM, the data implementation, etc are all separate repos (as per the telecon ~2 weeks ago). I think the understanding from that telecon was that we would likely use a hybrid approach (mix between multi-repo and single-repo). Also, after last week's meeting I think some people decided we needed to have and atmosphere developer contributing to this task to ensure all of their requirements were met as well. CAM was the main concern for both ACME and CSEG last week, since it seems like they have quite a few shared developments in the short term that need to be communicated between both repos. I'm not sure CLM falls into this category, but it's good to think about.
Well, let's invite some people rather than speculate in private about what they might think.
Also, it's still rather unclear how and to what level we'll need to support long term communication with the CESM repo. In my mind, this is much more difficult with a single repo than multiple repos,
Why do you say this? Is it about dealing with their current svn repository layout or something more fundamental about the Git development model?
but I'd be more than happy to be proven wrong about it. It seems like this topic is something none of us have really explored up until getting involved in ACME, so it's good to scope options. Assuming CSEG is not willing to change (as they said last week) for at least roughly 3-6 months and there are some shared developments that we need to incorporate from the CESM repo (I think both CLM and CAM have these, but at a minimum CAM does), this is mostly going to be something we need to accomadate if it's something we want.
What are we allowed to have from CESM? If all we can do is cherry-pick changes, I don't see how it matters. If we can merge branches, it's nice in the short term to make those branches correspond, but let's not compromise the long-term model for a (hopefully) temporary state.
I'm not super familiar with hooks, but could some of the issues we have be dealt with via hooks? i.e. maybe a pre-commit hook could check if submodules have uncommitted changes and fail if that's the case?
That's possible, though you have to install hooks manually (they are not part of the repo, for security reasons). You can have the build system install or upgrade them, but be sure to notify and avoid doing it in a destructive way.
Then we could have a single script that could be used to clone/setup the project (including some repo that contains our custom hooks). This way we might avoid the need to use any non-git commands (for typical developers) but catch the places we think a problem might arise.
Well, let's invite some people rather than speculate in private about what they might think.
This was mostly discussions that occurred as of last week. It was my understanding that we would be trying to get an atmosphere developer who is part of T7 to contribute to this discussion. But I haven't heard anymore about it.
Also, it's still rather unclear how and to what level we'll need to support long term communication with the CESM repo. In my mind, this is much more difficult with a single repo than multiple repos, Why do you say this? Is it about dealing with their current svn repository layout or something more fundamental about the Git development model?
I'm assuming you're asking why I think it's more difficult with a single repo than a multiple repo. This is purely due to their current svn layout and setup. Essentially the coupling of their directory structure with their very liberal use of svn externals make it difficult if not impossible to track something that closely resembles what our single repo would be.
What are we allowed to have from CESM? If all we can do is cherry-pick changes, I don't see how it matters. If we can merge branches, it's nice in the short term to make those branches correspond, but let's not compromise the long-term model for a (hopefully) temporary state.
This also is not clear. I'm not sure which state you're considering as temporary, but I agree with your sentiment. My understanding is that currently (in the short term, while they are unwilling to change their workflow) the best we can do is cherry-pick, unless we can come up with an easy way to transfer branches between the two, which might not be that difficult (on a component by component basis). Longer term, if they change their workflow to be a more "git friendly" workflow, we should be able to easily merge branches between the two.
That's possible, though you have to install hooks manually (they are not part of the repo, for security reasons). You can have the build system install or upgrade them, but be sure to notify and avoid doing it in a destructive way.
Sure, I was thinking we could have a repo that represents hooks for the ACME repo, and then the only script we need it one that clones the parent repo (which make have parts baked in) and then "install" the ACME hooks repo into the proper place. Then everyone gets the same hooks, though they can modify them.
Douglas Jacobsen <[email protected]> writes:
I'm assuming you're asking why I think it's more difficult with a single repo than a multiple repo. This is purely due to their current svn layout and setup. Essentially the coupling of their directory structure with their very liberal use of svn externals make it difficult if not impossible to track something that closely resembles what our single repo would be.
Fine, but below you say that we're only cherry-picking and the changes are only going one way (from CESM to us). As long as they stick with subversion, the other direction will just appear as huge blobs anyway. And note that if possible, we should always merge from them before creating the big patch for them to apply, that way someone's history (ours) contains the original patches for debugging/audit purposes.
What are we allowed to have from CESM? If all we can do is cherry-pick changes, I don't see how it matters. If we can merge branches, it's nice in the short term to make those branches correspond, but let's not compromise the long-term model for a (hopefully) temporary state.
This also is not clear. I'm not sure which state you're considering as temporary,
"temporary", as in the workflow that they currently use and won't change before their next release. After that, technical (and political) rationale will be considered.
but I agree with your sentiment. My understanding is that currently (in the short term, while they are unwilling to change their workflow) the best we can do is cherry-pick, unless we can come up with an easy way to transfer branches between the two, which might not be that difficult (on a component by component basis). Longer term, if they change their workflow to be a more "git friendly" workflow, we should be able to easily merge branches between the two.
That's possible, though you have to install hooks manually (they are not part of the repo, for security reasons). You can have the build system install or upgrade them, but be sure to notify and avoid doing it in a destructive way.
Sure, I was thinking we could have a repo that represents hooks for the ACME repo, and then the only script we need it one that clones the parent repo (which make have parts baked in) and then "install" the ACME hooks repo into the proper place. Then everyone gets the same hooks, though they can modify them.
Minor point, but I'd rather have an orphan branch than a separate repo, so that it gets updated automatically on pull.
As Doug said, a new requirement has appeared that we be able to pull in upcoming changes to CAM from NCAR SVN repository over the next year or so. (I am trying to acclimatize myself to this mode of working, where we find out requirements as a reaction to decisions/progress that we make.)
From the discussions here, I think we are all on the same page. A Single repo is best for workflow, branching, ability to use Google for help and debugging. Some limited number of separate repos (CAM, MPAS), perhaps git submodules, will make life easier for merging work done both in ACME and in a standalone repo.
A single repo will require a simple/robust way to merge work in an ACME subdirectory into a separate stand-alone repo. A multiple repo will require one of these: scripts that recursively run on all sub-repos submodule commands running git commands manually in each repo For this "scripts to make a multiple repo act as a single repo" case: The CASL project has 5-10 repos and uses a python script to propagate a git command to all repos. There is a file that lists the repos and the python command runs git on each one of them. I agree with Jed in the drawbacks of home-grown scripts -- however I still think this is a strong candidate for our optimum. I hope someone can take some time and look at Jim's single repo with established subtree-merge with external protocols. The nice part may be that the ugliness is only seen by SE gurus and not experienced by the whole team (like the scripts I was suggesting). Keep up the good work. Andy On 5/20/14 9:24 AM, "Douglas Jacobsen" <[email protected]> wrote:
On 05/20/2014 09:11 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
All MPAS components require having a separate repo. So, if we want to do development in a single repo the changes to MPAS components will need to be propagated back to the external repos. I thought we agreed that MPAS should be a submodule, since they want to do development independently and make "releases" to ACME? My comment was referring to components like CLM, CAM, the data implementations, etc.
Well, right now the multi-repo setup I'm working on is assuming CAM, CLM, the data implementation, etc are all separate repos (as per the telecon ~2 weeks ago). I think the understanding from that telecon was that we would likely use a hybrid approach (mix between multi-repo and single-repo). Also, after last week's meeting I think some people decided we needed to have and atmosphere developer contributing to this task to ensure all of their requirements were met as well. CAM was the main concern for both ACME and CSEG last week, since it seems like they have quite a few shared developments in the short term that need to be communicated between both repos. I'm not sure CLM falls into this category, but it's good to think about.
Also, it's still rather unclear how and to what level we'll need to support long term communication with the CESM repo. In my mind, this is much more difficult with a single repo than multiple repos, but I'd be more than happy to be proven wrong about it. It seems like this topic is something none of us have really explored up until getting involved in ACME, so it's good to scope options. Assuming CSEG is not willing to change (as they said last week) for at least roughly 3-6 months and there are some shared developments that we need to incorporate from the CESM repo (I think both CLM and CAM have these, but at a minimum CAM does), this is mostly going to be something we need to accomadate if it's something we want.
I'm not super familiar with hooks, but could some of the issues we have be dealt with via hooks? i.e. maybe a pre-commit hook could check if submodules have uncommitted changes and fail if that's the case?
Then we could have a single script that could be used to clone/setup the project (including some repo that contains our custom hooks). This way we might avoid the need to use any non-git commands (for typical developers) but catch the places we think a problem might arise.
Doug
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Two notes about some of the fragile issues related to submodules. Here is a super simple pre-commit hook I wrote that tests if there are uncommitted changes in submodules before allowing you to commit. https://gist.github.com/douglasjacobsen/14b93d122a54d764d34e Also, git push (as of 1.7.7.) has a --recurse-submodules=check option, that will check to see if commits are pushed. It should be pretty easy to write a pre-push hook as well that tests this so we don't have to rely on people using --recurse-submodules. Aside from this, I'm curious if there are any other issues we need to deal with related to submodules. Thanks! Doug On 05/20/2014 07:59 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
https://github.com/ACME-Climate/multiple_repo_example Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash.
Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
Douglas Jacobsen <[email protected]> writes:
Two notes about some of the fragile issues related to submodules.
Here is a super simple pre-commit hook I wrote that tests if there are uncommitted changes in submodules before allowing you to commit. https://gist.github.com/douglasjacobsen/14b93d122a54d764d34e
Also, git push (as of 1.7.7.) has a --recurse-submodules=check option, that will check to see if commits are pushed. It should be pretty easy to write a pre-push hook as well that tests this so we don't have to rely on people using --recurse-submodules.
The issue of whether submodules are dirty/have been pushed is relatively straightforward and not the "fundamental complexity" that I'm worried about.
Aside from this, I'm curious if there are any other issues we need to deal with related to submodules.
Merge at top-level involving cross-cutting changes that produce conflicts in submodules. What is the workflow to find and resolve conflicts? Do merge strategies work properly? Can the merge always be aborted cleanly. Is the rerere cache fully functional?
The issue of whether submodules are dirty/have been pushed is relatively straightforward and not the "fundamental complexity" that I'm worried about.
Ok, I was under the impression that these were some of the things that cause submodules to be referred to as fragile.
Merge at top-level involving cross-cutting changes that produce conflicts in submodules. What is the workflow to find and resolve conflicts? Do merge strategies work properly? Can the merge always be aborted cleanly. Is the rerere cache fully functional?
A couple things. First, I don't have a good use case for cross-cutting changes. I don't really see a case where anyone is going to be making large cross-cutting changes within the ACME repo. Largely changes will be restricted to a single component, until they are stable. This might require making some changes in the "shared bits of code" (code that all components use), but otherwise I don't see how this is going to be an issue. The same is true for finding / resolving conflicts (in my mind). The merges of a new feature would be done at the component level, so within a submodule. I don't think I can answer "Do merge strategies work properly? Can the merge always be aborted cleanly." though, because I don't have a good test case for this. Maybe I can try to make some changes to each repo that would cause conflicts and see what I have to do to resolve them. And related to the rerere cache, I don't see why it wouldn't be fully functional aside from each submodule having it's own rerere cache. The only real issue I see currently is having different branches in a submodule than the super project. But at the same time, this is one of the things I think is nice about the submodules. The ability to drill into a component, develop, and not have it impact the super project at all. Then when you want to incorporate those changes into the super project, you can. Doug
Douglas Jacobsen <[email protected]> writes:
Merge at top-level involving cross-cutting changes that produce conflicts in submodules. What is the workflow to find and resolve conflicts? Do merge strategies work properly? Can the merge always be aborted cleanly. Is the rerere cache fully functional?
A couple things. First, I don't have a good use case for cross-cutting changes. I don't really see a case where anyone is going to be making large cross-cutting changes within the ACME repo. Largely changes will be restricted to a single component, until they are stable. This might require making some changes in the "shared bits of code" (code that all components use), but otherwise I don't see how this is going to be an issue.
Change the coupler interface or the I/O interface, for example.
The same is true for finding / resolving conflicts (in my mind). The merges of a new feature would be done at the component level, so within a submodule.
Only if the external interface is unchanged.
I don't think I can answer "Do merge strategies work properly? Can the merge always be aborted cleanly." though, because I don't have a good test case for this. Maybe I can try to make some changes to each repo that would cause conflicts and see what I have to do to resolve them.
And related to the rerere cache, I don't see why it wouldn't be fully functional aside from each submodule having it's own rerere cache.
So what exactly was committed? Did you start committing merges is one submodule, then move to a different and realize that the whole thing is a catastrophe and a different approach is needed? Do you remember to invalidate rr-cache for the submodules you've already committed, so that they don't automerge to the wrong thing? (This gets off in the weeds, but it's the sort of thing that is just enough more complicated/error-prone with submodules to make hard-to-notice problems later.)
The only real issue I see currently is having different branches in a submodule than the super project. But at the same time, this is one of the things I think is nice about the submodules. The ability to drill into a component, develop, and not have it impact the super project at all.
What's wrong with doing this in branches within a fork? If I want to work entirely within CAM, what's wrong with $ git clone github:cam-maintainer/acme and ignoring acme-climate/acme. My changes may eventually be merged by cam-maintainer, who later makes a "release" (merge into acme-climate/acme). I have some source lying around that I'm not using in uncoupled tests, but I actually needed a sizable fraction of the total code for testing anyway. How is this more constraining than working in a submodule and making sure that the people testing my patches wrap the correct version of ACME around it?
Changing the coupling interface and or the I/O interface only affects components that are inherently tied into the coupler. Also, there's no reason changing theses (in the coupler and in the component) can't be done in separate repos, i.e. the component updates to a new interface based on some shared document. At the same time, the coupler developers update the interface within the coupler. After which they are sync'd together. I really down we'd expect someone making these modifications to be doing them alone. Along the same lines, last week we discussed how MPAS-CICE is going to be coupled into ACME (i.e. who is responsible). Rob and I discussed that it would likely be a combination of two people, a component developer and a coupler developer. They would work together on the two portions of code relevant to getting MPAS-CICE coupled in. The reason this breaks for models like CAM and CLM right now is they are inherently tied into the coupler. But the larger problem is they are tied into the CESM coupler, not the ACME coupler. If we make changes to the coupler interface or the I/O interface (for example) currently some of our scientific developments have to remain able to make their way back into CESM and/or we need to be able to incorporate scientific developments from the CESM repo. In this case, the interface changes required from ACME have to be separated from the scientific developments required by ACME. Honestly, about the rerere cache, I doubt most developers are going to want to understand what it is. Maybe maintainers will know what it is, but given the difficulties with introducing some developers to git in the first place, rerere caches are something we'll probably have to gloss over and ignore mostly. I'm not trying to say they aren't useful, or that we should discourage the use of them in any way. I just think it's one more complication that will make some of the people in this field get a head ache and not pay attention. Maybe you could make a fork of some of the repos I have put up and show some cases where a workflow would break. That might help push the conversation along. Related to your last point about developing in CAM. The only thing that's not clear to me from that is how we would deal with contributing modifications to / from the CESM repo. But similarly, that might be something we can deal with once the single repo setup is live to play with. Doug ________________________________________ From: Jed Brown [[email protected]] Sent: Wednesday, May 21, 2014 4:10 PM To: Jacobsen, Douglas W; [email protected] Subject: Re: [ACME-SE] ACME Git setup Douglas Jacobsen <[email protected]> writes:
Merge at top-level involving cross-cutting changes that produce conflicts in submodules. What is the workflow to find and resolve conflicts? Do merge strategies work properly? Can the merge always be aborted cleanly. Is the rerere cache fully functional?
A couple things. First, I don't have a good use case for cross-cutting changes. I don't really see a case where anyone is going to be making large cross-cutting changes within the ACME repo. Largely changes will be restricted to a single component, until they are stable. This might require making some changes in the "shared bits of code" (code that all components use), but otherwise I don't see how this is going to be an issue.
Change the coupler interface or the I/O interface, for example.
The same is true for finding / resolving conflicts (in my mind). The merges of a new feature would be done at the component level, so within a submodule.
Only if the external interface is unchanged.
I don't think I can answer "Do merge strategies work properly? Can the merge always be aborted cleanly." though, because I don't have a good test case for this. Maybe I can try to make some changes to each repo that would cause conflicts and see what I have to do to resolve them.
And related to the rerere cache, I don't see why it wouldn't be fully functional aside from each submodule having it's own rerere cache.
So what exactly was committed? Did you start committing merges is one submodule, then move to a different and realize that the whole thing is a catastrophe and a different approach is needed? Do you remember to invalidate rr-cache for the submodules you've already committed, so that they don't automerge to the wrong thing? (This gets off in the weeds, but it's the sort of thing that is just enough more complicated/error-prone with submodules to make hard-to-notice problems later.)
The only real issue I see currently is having different branches in a submodule than the super project. But at the same time, this is one of the things I think is nice about the submodules. The ability to drill into a component, develop, and not have it impact the super project at all.
What's wrong with doing this in branches within a fork? If I want to work entirely within CAM, what's wrong with $ git clone github:cam-maintainer/acme and ignoring acme-climate/acme. My changes may eventually be merged by cam-maintainer, who later makes a "release" (merge into acme-climate/acme). I have some source lying around that I'm not using in uncoupled tests, but I actually needed a sizable fraction of the total code for testing anyway. How is this more constraining than working in a submodule and making sure that the people testing my patches wrap the correct version of ACME around it?
Hi guys, It's probably too late to coordinate our message for tomorrow and for that I apologize. I assume Doug will discuss his investigation of submodules and I can talk about the single-repo investigation I've been doing. On a related note, I've attached the script on which I've been working. The intended use is to be run daily by a maintainer to sync the sub-repo within the super-repo with the independent repo. It seems to work well so far, but admittedly my testing has been very light. -Jim On 5/21/14 3:30 PM, "Douglas Jacobsen" <[email protected]> wrote:
Two notes about some of the fragile issues related to submodules.
Here is a super simple pre-commit hook I wrote that tests if there are uncommitted changes in submodules before allowing you to commit. https://gist.github.com/douglasjacobsen/14b93d122a54d764d34e
Also, git push (as of 1.7.7.) has a --recurse-submodules=check option, that will check to see if commits are pushed. It should be pretty easy to write a pre-push hook as well that tests this so we don't have to rely on people using --recurse-submodules.
Aside from this, I'm curious if there are any other issues we need to deal with related to submodules.
Thanks! Doug
On 05/20/2014 07:59 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
https://github.com/ACME-Climate/multiple_repo_example Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash.
Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
"Foucar, James G" <[email protected]> writes:
Hi guys,
It's probably too late to coordinate our message for tomorrow and for that I apologize. I assume Doug will discuss his investigation of submodules and I can talk about the single-repo investigation I've been doing.
On a related note, I've attached the script on which I've been working. The intended use is to be run daily by a maintainer to sync the sub-repo within the super-repo with the independent repo. It seems to work well so far, but admittedly my testing has been very light.
# Do merge, this is complicated because it may require user to resovle conflicts stat = run_cmd("git merge --no-edit %s/%s" % (subrepo, branch))[0] return stat == 0 So after a semantic conflict (e.g., B adds a new use of interface foo() while C changes the foo() interface) is committed and merged both ways (because this script doesn't ensure that regression tests pass and such), who is responsible for clean-up and what will they do?
Hi Jed, The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo. -Jim On 5/22/14 6:48 PM, "Jed Brown" <[email protected]> wrote:
"Foucar, James G" <[email protected]> writes:
Hi guys,
It's probably too late to coordinate our message for tomorrow and for that I apologize. I assume Doug will discuss his investigation of submodules and I can talk about the single-repo investigation I've been doing.
On a related note, I've attached the script on which I've been working. The intended use is to be run daily by a maintainer to sync the sub-repo within the super-repo with the independent repo. It seems to work well so far, but admittedly my testing has been very light.
# Do merge, this is complicated because it may require user to resovle conflicts stat = run_cmd("git merge --no-edit %s/%s" % (subrepo, branch))[0] return stat == 0
So after a semantic conflict (e.g., B adds a new use of interface foo() while C changes the foo() interface) is committed and merged both ways (because this script doesn't ensure that regression tests pass and such), who is responsible for clean-up and what will they do?
"Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo.
The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side. If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it.
Here are some related questions for the broader group... How many SE/Integrators are we going to have? Where do they sit physically and relative to the components / coupler? Do we have resources available for this model (i.e. are people funded to take on these responsibilities)? It seems like the answers to these questions are something like: O(Components + 1 for couplers) = ~5, Maybe 1 per lab / 1 per component + coupler at a minimum, and no. Since I'm assuming the last one is no, that makes an integrator workflow more difficult. Any thoughts on that? Thanks, Doug ________________________________________ From: Jed Brown [[email protected]] Sent: Friday, May 23, 2014 11:33 AM To: Foucar, James G; Jacobsen, Douglas W; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup "Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo.
The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side. If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it.
It’s been pretty interesting trying to get any budget information out of anyone on my end. I can’t even technically charge any time for doing any ACME-related work now. When things do “start up,” I’m only slated for 1/3 time. Now it’s probably a different allotment for other folks, but if we’re all at that level it’s hard to envision how we’re going to be providing integration support AND doing anything else. Jeff On May 23, 2014, at 10:41 AM, Jacobsen, Douglas W <[email protected]> wrote:
Here are some related questions for the broader group...
How many SE/Integrators are we going to have? Where do they sit physically and relative to the components / coupler? Do we have resources available for this model (i.e. are people funded to take on these responsibilities)?
It seems like the answers to these questions are something like: O(Components + 1 for couplers) = ~5, Maybe 1 per lab / 1 per component + coupler at a minimum, and no.
Since I'm assuming the last one is no, that makes an integrator workflow more difficult. Any thoughts on that?
Thanks, Doug
________________________________________ From: Jed Brown [[email protected]] Sent: Friday, May 23, 2014 11:33 AM To: Foucar, James G; Jacobsen, Douglas W; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
"Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo.
The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side.
If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it. _______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Yeah, those are good points. It's odd that you're only funded 1/3 time, since one of the ACME wide requirements was no one could be funded less than 1/2 time... I'm mostly asking these questions to get a feel for how many / who we would expect to be using these scripts. Thanks, Doug ________________________________________ From: Jeffrey Johnson [[email protected]] Sent: Friday, May 23, 2014 11:48 AM To: Jacobsen, Douglas W Cc: Jed Brown; Foucar, James G; [email protected] Subject: Re: [ACME-SE] [EXTERNAL] Re: ACME Git setup It’s been pretty interesting trying to get any budget information out of anyone on my end. I can’t even technically charge any time for doing any ACME-related work now. When things do “start up,” I’m only slated for 1/3 time. Now it’s probably a different allotment for other folks, but if we’re all at that level it’s hard to envision how we’re going to be providing integration support AND doing anything else. Jeff On May 23, 2014, at 10:41 AM, Jacobsen, Douglas W <[email protected]> wrote:
Here are some related questions for the broader group...
How many SE/Integrators are we going to have? Where do they sit physically and relative to the components / coupler? Do we have resources available for this model (i.e. are people funded to take on these responsibilities)?
It seems like the answers to these questions are something like: O(Components + 1 for couplers) = ~5, Maybe 1 per lab / 1 per component + coupler at a minimum, and no.
Since I'm assuming the last one is no, that makes an integrator workflow more difficult. Any thoughts on that?
Thanks, Doug
________________________________________ From: Jed Brown [[email protected]] Sent: Friday, May 23, 2014 11:33 AM To: Foucar, James G; Jacobsen, Douglas W; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
"Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo.
The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side.
If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it. _______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Hi Jed, That's a good point. I'm going to test what happens when there's a merge conflict and see how the tool works. I've also noticed that the rebase on line 244 is causing duplication of commits in the super-repo. -Jim On 5/23/14 11:33 AM, "Jed Brown" <[email protected]> wrote:
"Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo.
The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side.
If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it.
Hey Jim, I actually really like the setup you have. I like how the model is more than just a version of external repos, it's a complete package itself. Things I'd like to explore at some point which you might already have workflow descriptions for is the communication issues related to bringing an external change into this and pushing a change from this back out to the external. One other concern I have is that I'm not 100% sure this layout will work for what NCAR wants. Since their model is to have more flexibility with components. Though, I guess I don't see a big reason why we can't have more flexibility with components, just making sure they each how their own directories in models. A final question is, is it possible to so the sub-tree merge with a subdirectory of the external repo. For example with the pio repo, we don't really want the whole repo in models/utils/pio, we want the directory that is currently models/utils/pio/pio to be models/utils/pio (if that makes any sense). It would be nice to be able to pull in just the subdirectory (preserving the relevant history) and push back out to the subdirectory. Then external projects can maintain whatever higher level interfaces they need that are not relevant for the coupled project. This is a detail and we can get around it without the subtree merges, but I just thought this would make it nicer too. Thanks, Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Friday, May 23, 2014 11:44 AM To: Jed Brown; Jacobsen, Douglas W; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup Hi Jed, That's a good point. I'm going to test what happens when there's a merge conflict and see how the tool works. I've also noticed that the rebase on line 244 is causing duplication of commits in the super-repo. -Jim On 5/23/14 11:33 AM, "Jed Brown" <[email protected]> wrote:
"Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo.
The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side.
If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it.
Hi Doug,
Things I'd like to explore at some point which you might already have workflow descriptions for is the communication issues related to bringing an external change into this and pushing a change from this back out to the external.
I envisioned a scenario where the single-repo-helper script was run daily by a code maintainer. If commits brought in from the independent repo conflicted with commits that were made in the single repo, then the maintainer would potentially have to talk to the author of the commit to get help, otherwise no communication would be needed. Hopefully, most development of a product would occur in one place or the other, so conflicts should be rare.
A final question is, is it possible to so the sub-tree merge with a subdirectory of the external repo. For example with the pio repo, we don't really want the whole repo in models/utils/pio, we want the directory that is currently models/utils/pio/pio to be models/utils/pio (if that makes any sense). It would be nice to be able to pull in just the subdirectory (preserving the relevant history) and push back out to the subdirectory. Then external projects can maintain whatever higher level interfaces they need that are not relevant for the coupled project. This is a detail and we can get around it without the subtree merges, but I just thought this would make it nicer too.
I'm actually not using subtree merge in the single-repo-helper script because it mangled the history too much. I think I outlined my problems with subtree merge in a previous email. What I'm doing is fetching a remote branch, rewriting the remote branch with filter-branch, and merging the remote branch into the current branch. You can see the details if you look at the update_subrepo_impl function in single-repo-helper. To answer your question, yes, this technique will work for what you describe. Basically, anything that filter-branch can support, which includes just grabbing commits from a subdirectory, we can support. -Jim On 5/23/14 12:42 PM, "Jacobsen, Douglas W" <[email protected]> wrote:
Hey Jim,
I actually really like the setup you have. I like how the model is more than just a version of external repos, it's a complete package itself.
Things I'd like to explore at some point which you might already have workflow descriptions for is the communication issues related to bringing an external change into this and pushing a change from this back out to the external.
One other concern I have is that I'm not 100% sure this layout will work for what NCAR wants. Since their model is to have more flexibility with components. Though, I guess I don't see a big reason why we can't have more flexibility with components, just making sure they each how their own directories in models.
A final question is, is it possible to so the sub-tree merge with a subdirectory of the external repo. For example with the pio repo, we don't really want the whole repo in models/utils/pio, we want the directory that is currently models/utils/pio/pio to be models/utils/pio (if that makes any sense). It would be nice to be able to pull in just the subdirectory (preserving the relevant history) and push back out to the subdirectory. Then external projects can maintain whatever higher level interfaces they need that are not relevant for the coupled project. This is a detail and we can get around it without the subtree merges, but I just thought this would make it nicer too.
Thanks, Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Friday, May 23, 2014 11:44 AM To: Jed Brown; Jacobsen, Douglas W; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
Hi Jed,
That's a good point. I'm going to test what happens when there's a merge conflict and see how the tool works.
I've also noticed that the rebase on line 244 is causing duplication of commits in the super-repo.
-Jim
On 5/23/14 11:33 AM, "Jed Brown" <[email protected]> wrote:
"Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo.
The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side.
If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it.
Hey Jim,
I envisioned a scenario where the single-repo-helper script was run daily by a code maintainer. If commits brought in from the independent repo conflicted with commits that were made in the single repo, then the maintainer would potentially have to talk to the author of the commit to get help, otherwise no communication would be needed. Hopefully, most development of a product would occur in one place or the other, so conflicts should be rare.
Who are the maintainers? And how would you handle the case where an independent repo doesn't want to be "brought in" (i.e. allow development on the component within the single repo)? Here maybe a better way to phrase this question is, could the script be implementation agnostic, i.e. could some of the subrepos be submodules, while others are actually included? The maintainers issue isn't one you necessarily need to answer, but it is one we need to think about. Mostly how many there are, what their roles are, who they are, what they primarily do, and how they interact with the coupler and component teams. This answer could guide what we can and cannot do with repos in the future.
I'm actually not using subtree merge in the single-repo-helper script because it mangled the history too much. I think I outlined my problems with subtree merge in a previous email. What I'm doing is fetching a remote branch, rewriting the remote branch with filter-branch, and merging the remote branch into the current branch. You can see the details if you look at the update_subrepo_impl function in single-repo-helper. To answer your question, yes, this technique will work for what you describe. Basically, anything that filter-branch can support, which includes just grabbing commits from a subdirectory, we can support.
Sure, so filter branch works fine for pulling in history. How do it work for pushing history back out? I've been thinking a lot more about how CLM and CAM don't have any stand alone capability, and don't necessarily plan to implement such a thing. Something I was thinking about in order to solve this, was to setup a "single repo" that actually contains all of the "tagged" versions of CESM, up to whatever are considered publicly available versions. These can be converted into git repos pretty easily I think. This could be the public "upstream". Then we could setup orgs for both CAM and CLM that have public forks of this upstream, and can have their own access lists. These lists could be co-manged by DOE and NCAR, but the repo look the same, in that CLM and CAM both have access to every model that was in the tag they branched from. Their development can happen in this fork in branches where they only modify the relevant component. i.e. CAM developers shouldn't mess with CLM, and vice versa. They can then contribute back to the upstream via pull requests from the CAM / CLM repos to the "CESM" repo. Then for ACME, we can "fork" (i.e. duplicated) the "CESM" repo as a private development repo called ACME or something. And our internal developers could work in forks that have branches that only touch CAM or CLM, the same way they would under the CAM / CLM development repos. The only thing that's not clear is how to deal with independent repos in this case. Though the only real way I can see doing this is via submodules. Which might not be a big deal in this case, since the components are independent. This might be identical to what some people were thinking earlier, so I apologize if this is where some people wanted to be to begin with. But, let me know if anyone has any comments on this setup. It wouldn't require any additional scripts, but we might be able to clean up the submodule issues with hooks. Thanks, Doug
-Jim
On 5/23/14 12:42 PM, "Jacobsen, Douglas W" <[email protected]> wrote:
Hey Jim,
I actually really like the setup you have. I like how the model is more than just a version of external repos, it's a complete package itself.
Things I'd like to explore at some point which you might already have workflow descriptions for is the communication issues related to bringing an external change into this and pushing a change from this back out to the external.
One other concern I have is that I'm not 100% sure this layout will work for what NCAR wants. Since their model is to have more flexibility with components. Though, I guess I don't see a big reason why we can't have more flexibility with components, just making sure they each how their own directories in models.
A final question is, is it possible to so the sub-tree merge with a subdirectory of the external repo. For example with the pio repo, we don't really want the whole repo in models/utils/pio, we want the directory that is currently models/utils/pio/pio to be models/utils/pio (if that makes any sense). It would be nice to be able to pull in just the subdirectory (preserving the relevant history) and push back out to the subdirectory. Then external projects can maintain whatever higher level interfaces they need that are not relevant for the coupled project. This is a detail and we can get around it without the subtree merges, but I just thought this would make it nicer too.
Thanks, Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Friday, May 23, 2014 11:44 AM To: Jed Brown; Jacobsen, Douglas W; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
Hi Jed,
That's a good point. I'm going to test what happens when there's a merge conflict and see how the tool works.
I've also noticed that the rebase on line 244 is causing duplication of commits in the super-repo.
-Jim
On 5/23/14 11:33 AM, "Jed Brown" <[email protected]> wrote:
"Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo. The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side.
If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it.
Hi Doug,
Here maybe a better way to phrase this question is, could the script be implementation agnostic, i.e. could some of the subrepos be submodules, while others are actually included?
Yes, I think that would be pretty easy since submodules are so much easier than what I'm trying to do with the "inlined" repositories. I have no idea how many resources we can allocate towards maintainers. I think it could be a pretty easy job as long as there wasn't heavy development going on simultaneously in the single and independent repos.
Sure, so filter branch works fine for pulling in history. How do it work for pushing history back out?
I found that subtree spilt worked better than filter-branch for pushing history back out. However, even subtree split is not able to reproduce exactly what's in the independent repository. For example, say you have independent repository foo. We fetch foo/master, filter it, and merge it into our single repository. We then run subtree split on foo to pull it back out. I've found that I don't get the exact history of foo/master out that was put in. This is what forces me to rebase work that happened in the single repo which doesn't appear to be an acceptable solution. I may need to switch to some other filtering mechanism. The repo setup you've described sounds reasonable to me. -Jim On 5/27/14 1:33 PM, "Douglas Jacobsen" <[email protected]> wrote:
Hey Jim,
I envisioned a scenario where the single-repo-helper script was run daily by a code maintainer. If commits brought in from the independent repo conflicted with commits that were made in the single repo, then the maintainer would potentially have to talk to the author of the commit to get help, otherwise no communication would be needed. Hopefully, most development of a product would occur in one place or the other, so conflicts should be rare.
Who are the maintainers? And how would you handle the case where an independent repo doesn't want to be "brought in" (i.e. allow development on the component within the single repo)?
Here maybe a better way to phrase this question is, could the script be implementation agnostic, i.e. could some of the subrepos be submodules, while others are actually included?
The maintainers issue isn't one you necessarily need to answer, but it is one we need to think about. Mostly how many there are, what their roles are, who they are, what they primarily do, and how they interact with the coupler and component teams.
This answer could guide what we can and cannot do with repos in the future.
I'm actually not using subtree merge in the single-repo-helper script because it mangled the history too much. I think I outlined my problems with subtree merge in a previous email. What I'm doing is fetching a remote branch, rewriting the remote branch with filter-branch, and merging the remote branch into the current branch. You can see the details if you look at the update_subrepo_impl function in single-repo-helper. To answer your question, yes, this technique will work for what you describe. Basically, anything that filter-branch can support, which includes just grabbing commits from a subdirectory, we can support.
Sure, so filter branch works fine for pulling in history. How do it work for pushing history back out?
I've been thinking a lot more about how CLM and CAM don't have any stand alone capability, and don't necessarily plan to implement such a thing.
Something I was thinking about in order to solve this, was to setup a "single repo" that actually contains all of the "tagged" versions of CESM, up to whatever are considered publicly available versions. These can be converted into git repos pretty easily I think. This could be the public "upstream". Then we could setup orgs for both CAM and CLM that have public forks of this upstream, and can have their own access lists. These lists could be co-manged by DOE and NCAR, but the repo look the same, in that CLM and CAM both have access to every model that was in the tag they branched from.
Their development can happen in this fork in branches where they only modify the relevant component. i.e. CAM developers shouldn't mess with CLM, and vice versa. They can then contribute back to the upstream via pull requests from the CAM / CLM repos to the "CESM" repo.
Then for ACME, we can "fork" (i.e. duplicated) the "CESM" repo as a private development repo called ACME or something. And our internal developers could work in forks that have branches that only touch CAM or CLM, the same way they would under the CAM / CLM development repos.
The only thing that's not clear is how to deal with independent repos in this case. Though the only real way I can see doing this is via submodules. Which might not be a big deal in this case, since the components are independent.
This might be identical to what some people were thinking earlier, so I apologize if this is where some people wanted to be to begin with. But, let me know if anyone has any comments on this setup. It wouldn't require any additional scripts, but we might be able to clean up the submodule issues with hooks.
Thanks, Doug
-Jim
On 5/23/14 12:42 PM, "Jacobsen, Douglas W" <[email protected]> wrote:
Hey Jim,
I actually really like the setup you have. I like how the model is more than just a version of external repos, it's a complete package itself.
Things I'd like to explore at some point which you might already have workflow descriptions for is the communication issues related to bringing an external change into this and pushing a change from this back out to the external.
One other concern I have is that I'm not 100% sure this layout will work for what NCAR wants. Since their model is to have more flexibility with components. Though, I guess I don't see a big reason why we can't have more flexibility with components, just making sure they each how their own directories in models.
A final question is, is it possible to so the sub-tree merge with a subdirectory of the external repo. For example with the pio repo, we don't really want the whole repo in models/utils/pio, we want the directory that is currently models/utils/pio/pio to be models/utils/pio (if that makes any sense). It would be nice to be able to pull in just the subdirectory (preserving the relevant history) and push back out to the subdirectory. Then external projects can maintain whatever higher level interfaces they need that are not relevant for the coupled project. This is a detail and we can get around it without the subtree merges, but I just thought this would make it nicer too.
Thanks, Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Friday, May 23, 2014 11:44 AM To: Jed Brown; Jacobsen, Douglas W; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
Hi Jed,
That's a good point. I'm going to test what happens when there's a merge conflict and see how the tool works.
I've also noticed that the rebase on line 244 is causing duplication of commits in the super-repo.
-Jim
On 5/23/14 11:33 AM, "Jed Brown" <[email protected]> wrote:
"Foucar, James G" <[email protected]> writes:
Hi Jed,
The merge would be done by whoever did the "single-repo-helper update-subrepo" command, presumably a code maintainer. That's assuming the conflict was caused by work that happened on the independent repo with work that happened in the super-repo. The conflict could be "caused" by either side. In these cases, usually nobody is at fault, just a lack of out-of-band communication or an attempt to integrate features developed at different times. Committing the merge for an incompatible state is wrong. (Well, not wrong if you fix it before pushing.) But even if one side is "at fault", the best way to fix it could be on the other side.
If the integrator notices the problem before pushing and if they understand all the semantics handled by the script in the friendly case, they can manually repair. But now they don't have the support/consistency of the tool and it's easy to forget things. If they don't notice the problem, someone else needs to be able to diagnose what went wrong and either fix it or report and wait for the SE/integrator to fix it.
"Foucar, James G" <[email protected]> writes:
I found that subtree spilt worked better than filter-branch for pushing history back out. However, even subtree split is not able to reproduce exactly what's in the independent repository. For example, say you have independent repository foo. We fetch foo/master, filter it, and merge it into our single repository. We then run subtree split on foo to pull it back out. I've found that I don't get the exact history of foo/master out that was put in.
The commits really are different. Presumably you can verify that the trees are indeed the same.
This is what forces me to rebase work that happened in the single repo which doesn't appear to be an acceptable solution.
Yeah, note that merge history will cause a problem for rebase.
Was a conclusion reached on single vs. multiple repos? (given that "single" still means separate repos for HOMME, MPAS, etc.) What will likely happen is that some in-progress development will continue on existing svn branches in the CESM repo. But where does testing and merging to a trunk/master happen? Rob On 5/27/14 4:44 PM, Jed Brown wrote:
"Foucar, James G" <[email protected]> writes:
I found that subtree spilt worked better than filter-branch for pushing history back out. However, even subtree split is not able to reproduce exactly what's in the independent repository. For example, say you have independent repository foo. We fetch foo/master, filter it, and merge it into our single repository. We then run subtree split on foo to pull it back out. I've found that I don't get the exact history of foo/master out that was put in.
The commits really are different. Presumably you can verify that the trees are indeed the same.
This is what forces me to rebase work that happened in the single repo which doesn't appear to be an acceptable solution.
Yeah, note that merge history will cause a problem for rebase.
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Hey Rob, I'm going to give my opinions, which may or may not mirror other peoples opinions. So take them with a grain of salt for now. I've spent a large portion of my time recently trying to figure out how best to handle our requirements related to setting up the ACME repo. The main two requirements I've been focusing on are the follows: 1) Allowing the ability to communicate with the CESM repo in some capacity 2) Trying to preserve (at a minimum) the history of the trunk of each individual component. So far, I haven't satisfied 2, but I have an idea how this could be satisfied. Granted, the way I'm planning to satisfy it will produce an extremely messy history in the past, and will largely be unusable. It will only be usable from a "it exists, and I can look at it" point of view, but anyone wanting to test a previous version of a component within the coupled model will have to put in some effort (granted, it won't be much effort, but it will still be some). Also, I don't have a good idea for where to "end" the old histories. But I'll try to come up with something. I would propose using: https://github.com/ACME-Climate/CESM_versions as the ACME repo. I set it up in a way I think could work for a public "neutral" for both ACME and CESM. CAM and CLM could both work in this, as branches of the main repo. This would reduce issues with duplication of the coupler "framework". I was thinking that CAM and CLM could also go one step further and setup organizations where they can maintain the developer lists for both components independently from the CESM / ACME developer lists. Of course some components are still in separate repos (MPAS, HOMME?, PIO, MCT, etc...). Right now, there are two options. The first is submodules, and the second is subtree merges. Subtree merges could cause some complicated history, and submodules will cause a bit more complicated workflow when dealing with these components. Note, both complications only occur with components in these separate setups. In terms of merging. I think tracking branches can make that fairly painless. Worst case scenario would be merging via patch exports, which again isn't that terrible, if you know the starting place. The main thing we'll need to know to bring in changes would be knowing what tag they branched off of to begin with. The over use of SVN_EXTERNAL_DIRECTORIES files within the CESM repo has cause an abundance of headaches related to this conversion. Hopefully that answers your very few questions, in an overly verbose way. Sorry this email turned in to be a lot longer than I had hoped. Doug On 06/05/2014 11:14 AM, Robert Jacob wrote:
Was a conclusion reached on single vs. multiple repos? (given that "single" still means separate repos for HOMME, MPAS, etc.)
What will likely happen is that some in-progress development will continue on existing svn branches in the CESM repo. But where does testing and merging to a trunk/master happen?
Rob
On 5/27/14 4:44 PM, Jed Brown wrote:
"Foucar, James G" <[email protected]> writes:
I found that subtree spilt worked better than filter-branch for pushing history back out. However, even subtree split is not able to reproduce exactly what's in the independent repository. For example, say you have independent repository foo. We fetch foo/master, filter it, and merge it into our single repository. We then run subtree split on foo to pull it back out. I've found that I don't get the exact history of foo/master out that was put in.
The commits really are different. Presumably you can verify that the trees are indeed the same.
This is what forces me to rebase work that happened in the single repo which doesn't appear to be an acceptable solution.
Yeah, note that merge history will cause a problem for rebase.
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
On 6/5/14 1:03 PM, Douglas Jacobsen wrote:
Hey Rob,
I'm going to give my opinions, which may or may not mirror other peoples opinions. So take them with a grain of salt for now.
I've spent a large portion of my time recently trying to figure out how best to handle our requirements related to setting up the ACME repo. The main two requirements I've been focusing on are the follows:
1) Allowing the ability to communicate with the CESM repo in some capacity What are you thinking is required here? I think this is more an issue of keeping the same directory layout then svn/git choice. That is, if I have a git clone of the ACME repo and an svn checkout of the CESM repo on the same system, I'm happy "communicating" by copying files from one to the other as necessary.
2) Trying to preserve (at a minimum) the history of the trunk of each individual component. This is more of CSEG requirement. I'm not sure this is a requirement for ACME.
I would propose using: https://github.com/ACME-Climate/CESM_versions as the ACME repo. I set it up in a way I think could work for a public "neutral" for both ACME and CESM.
*The* ACME repo should probably not have "CESM" in the name. Also note that we can't really go public until ACME v1 is released in 2 years. Our base repo will be some set of development tags from the current SVN repo. The only thing keeping us from identifying those now and making a clean break is that some DOE development is still going on in svn branches and have not made it back to a trunk tag yet.
Of course some components are still in separate repos (MPAS, HOMME?, PIO, MCT, etc...). Right now, there are two options. The first is submodules, and the second is subtree merges. Subtree merges could cause some complicated history, and submodules will cause a bit more complicated workflow when dealing with these components. Note, both complications only occur with components in these separate setups.
I'm leaning towards submodules. Rob
In terms of merging. I think tracking branches can make that fairly painless. Worst case scenario would be merging via patch exports, which again isn't that terrible, if you know the starting place.
The main thing we'll need to know to bring in changes would be knowing what tag they branched off of to begin with.
The over use of SVN_EXTERNAL_DIRECTORIES files within the CESM repo has cause an abundance of headaches related to this conversion.
Hopefully that answers your very few questions, in an overly verbose way. Sorry this email turned in to be a lot longer than I had hoped.
Doug
On 06/05/2014 11:14 AM, Robert Jacob wrote:
Was a conclusion reached on single vs. multiple repos? (given that "single" still means separate repos for HOMME, MPAS, etc.)
What will likely happen is that some in-progress development will continue on existing svn branches in the CESM repo. But where does testing and merging to a trunk/master happen?
Rob
On 5/27/14 4:44 PM, Jed Brown wrote:
"Foucar, James G" <[email protected]> writes:
I found that subtree spilt worked better than filter-branch for pushing history back out. However, even subtree split is not able to reproduce exactly what's in the independent repository. For example, say you have independent repository foo. We fetch foo/master, filter it, and merge it into our single repository. We then run subtree split on foo to pull it back out. I've found that I don't get the exact history of foo/master out that was put in.
The commits really are different. Presumably you can verify that the trees are indeed the same.
This is what forces me to rebase work that happened in the single repo which doesn't appear to be an acceptable solution.
Yeah, note that merge history will cause a problem for rebase.
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
1) Allowing the ability to communicate with the CESM repo in some capacity What are you thinking is required here? I think this is more an issue of keeping the same directory layout then svn/git choice. That is, if I have a git clone of the ACME repo and an svn checkout of the CESM repo on the same system, I'm happy "communicating" by copying files from one to the other as necessary.
Agreed. I think that's fine for now. I'm thinking communication in both the short and long terms. Short term it could be exactly what you describe, and long term it could remain that. But if CSEG switches to using git, then I mean setting up some repository that would be useful for them as well.
2) Trying to preserve (at a minimum) the history of the trunk of each individual component. This is more of CSEG requirement. I'm not sure this is a requirement for ACME.
Agreed again. But I figured if I had to set it up for them (in terms of some repo they want), it might not be a bad idea to keep it for ourselves as well.
I would propose using: https://github.com/ACME-Climate/CESM_versions as the ACME repo. I set it up in a way I think could work for a public "neutral" for both ACME and CESM.
*The* ACME repo should probably not have "CESM" in the name. Also note that we can't really go public until ACME v1 is released in 2 years.
Yeah, I agree again. I just named it CESM_versions as a starting place. Also, this one doesn't have submodules setup, so it's not *exactly* what we are looking for.
Our base repo will be some set of development tags from the current SVN repo. The only thing keeping us from identifying those now and making a clean break is that some DOE development is still going on in svn branches and have not made it back to a trunk tag yet.
Well, v0 is a modified version of CESM 1.2. So at a minimum our development repo could start from that. Changes from svn could still be merged in. The only thing we really need to easily be able to merge the changes from those branches is to know the CESM tag that is the base of the changes.
I'm leaning towards submodules.
Me too. For all the trouble they can cause, I think they are more flexible and we can easily get around the potential issues. Doug
On 6/5/14 3:11 PM, Douglas Jacobsen wrote
Our base repo will be some set of development tags from the current SVN repo. The only thing keeping us from identifying those now and making a clean break is that some DOE development is still going on in svn branches and have not made it back to a trunk tag yet.
Well, v0 is a modified version of CESM 1.2. So at a minimum our development repo could start from that. Changes from svn could still be merged in.
So you would "replay" the changes since v0 into our repo to form the true starting point? Is that possible when that history is in svn? Rob
Well, v0 is going to be a permanent branch (at least that was my plan). It would branch off of 1.2, and live forever but never have any commits on it beyond what is required to get the v0 runs complete. ACME would branch off of something (maybe 1.2 as well) and continue with development. Then, if we have something that's a branch in SVN. Let's say it is a CAM branch, and we know the CAM branch came off of a CAM trunk tag, and we can figure out which it was. We can figure out which CESM "release" tag that came from, and make a new branch there. Then replay the patches on top of that branch to "re-create" the history (if we want to go through that much work). Alternatively, we could just diff the final version with the base, and apply that as a patch. Either way, patches are just about the only way to do it because of the overuse of svn externals in the CESM repo. We won't really have a branch that looks like the CAM or CLM trunk, just because those look like mini CESM's. Doug ________________________________________ From: Robert Jacob [[email protected]] Sent: Thursday, June 05, 2014 7:57 PM To: Jacobsen, Douglas W; [email protected] Subject: Re: [ACME-SE] [EXTERNAL] Re: ACME Git setup On 6/5/14 3:11 PM, Douglas Jacobsen wrote
Our base repo will be some set of development tags from the current SVN repo. The only thing keeping us from identifying those now and making a clean break is that some DOE development is still going on in svn branches and have not made it back to a trunk tag yet.
Well, v0 is a modified version of CESM 1.2. So at a minimum our development repo could start from that. Changes from svn could still be merged in.
So you would "replay" the changes since v0 into our repo to form the true starting point? Is that possible when that history is in svn? Rob
Hey Jim I won't really have much to discuss tomorrow. Aside from having set up the repos and the experience I can share related to syncing commits / work between git and svn (ACME and CESM). In terms of use, I haven't had enough time to actually test work flows in the repositories. It would probably be good if we each took a usecase, and tried to test how it worked in each repo. Hopefully this will be useful tomorrow, but I'm sure just discussing requirements with everyone will be beneficial. Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Thursday, May 22, 2014 6:18 PM To: Jacobsen, Douglas W; Jed Brown; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup Hi guys, It's probably too late to coordinate our message for tomorrow and for that I apologize. I assume Doug will discuss his investigation of submodules and I can talk about the single-repo investigation I've been doing. On a related note, I've attached the script on which I've been working. The intended use is to be run daily by a maintainer to sync the sub-repo within the super-repo with the independent repo. It seems to work well so far, but admittedly my testing has been very light. -Jim On 5/21/14 3:30 PM, "Douglas Jacobsen" <[email protected]> wrote:
Two notes about some of the fragile issues related to submodules.
Here is a super simple pre-commit hook I wrote that tests if there are uncommitted changes in submodules before allowing you to commit. https://gist.github.com/douglasjacobsen/14b93d122a54d764d34e
Also, git push (as of 1.7.7.) has a --recurse-submodules=check option, that will check to see if commits are pushed. It should be pretty easy to write a pre-push hook as well that tests this so we don't have to rely on people using --recurse-submodules.
Aside from this, I'm curious if there are any other issues we need to deal with related to submodules.
Thanks! Doug
On 05/20/2014 07:59 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
https://github.com/ACME-Climate/multiple_repo_example Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash.
Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Hi guys, This is my first time calling in to the telecon without Andy. When/how were you assigned an access code for readytalk? I'm looking at step 3: 3. Enter your 7-digit Access Code, followed by ³7308085#² -Jim On 5/22/14 7:22 PM, "Jacobsen, Douglas W" <[email protected]> wrote:
Hey Jim
I won't really have much to discuss tomorrow. Aside from having set up the repos and the experience I can share related to syncing commits / work between git and svn (ACME and CESM).
In terms of use, I haven't had enough time to actually test work flows in the repositories.
It would probably be good if we each took a usecase, and tried to test how it worked in each repo.
Hopefully this will be useful tomorrow, but I'm sure just discussing requirements with everyone will be beneficial.
Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Thursday, May 22, 2014 6:18 PM To: Jacobsen, Douglas W; Jed Brown; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
Hi guys,
It's probably too late to coordinate our message for tomorrow and for that I apologize. I assume Doug will discuss his investigation of submodules and I can talk about the single-repo investigation I've been doing.
On a related note, I've attached the script on which I've been working. The intended use is to be run daily by a maintainer to sync the sub-repo within the super-repo with the independent repo. It seems to work well so far, but admittedly my testing has been very light.
-Jim
On 5/21/14 3:30 PM, "Douglas Jacobsen" <[email protected]> wrote:
Two notes about some of the fragile issues related to submodules.
Here is a super simple pre-commit hook I wrote that tests if there are uncommitted changes in submodules before allowing you to commit. https://gist.github.com/douglasjacobsen/14b93d122a54d764d34e
Also, git push (as of 1.7.7.) has a --recurse-submodules=check option, that will check to see if commits are pushed. It should be pretty easy to write a pre-push hook as well that tests this so we don't have to rely on people using --recurse-submodules.
Aside from this, I'm curious if there are any other issues we need to deal with related to submodules.
Thanks! Doug
On 05/20/2014 07:59 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
https://github.com/ACME-Climate/multiple_repo_example Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash.
Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Hey Jim, As far as I know "7308085" is the access code. Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Friday, May 23, 2014 8:33 AM To: Jacobsen, Douglas W; Jed Brown; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup Hi guys, This is my first time calling in to the telecon without Andy. When/how were you assigned an access code for readytalk? I'm looking at step 3: 3. Enter your 7-digit Access Code, followed by ³7308085#² -Jim On 5/22/14 7:22 PM, "Jacobsen, Douglas W" <[email protected]> wrote:
Hey Jim
I won't really have much to discuss tomorrow. Aside from having set up the repos and the experience I can share related to syncing commits / work between git and svn (ACME and CESM).
In terms of use, I haven't had enough time to actually test work flows in the repositories.
It would probably be good if we each took a usecase, and tried to test how it worked in each repo.
Hopefully this will be useful tomorrow, but I'm sure just discussing requirements with everyone will be beneficial.
Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Thursday, May 22, 2014 6:18 PM To: Jacobsen, Douglas W; Jed Brown; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
Hi guys,
It's probably too late to coordinate our message for tomorrow and for that I apologize. I assume Doug will discuss his investigation of submodules and I can talk about the single-repo investigation I've been doing.
On a related note, I've attached the script on which I've been working. The intended use is to be run daily by a maintainer to sync the sub-repo within the super-repo with the independent repo. It seems to work well so far, but admittedly my testing has been very light.
-Jim
On 5/21/14 3:30 PM, "Douglas Jacobsen" <[email protected]> wrote:
Two notes about some of the fragile issues related to submodules.
Here is a super simple pre-commit hook I wrote that tests if there are uncommitted changes in submodules before allowing you to commit. https://gist.github.com/douglasjacobsen/14b93d122a54d764d34e
Also, git push (as of 1.7.7.) has a --recurse-submodules=check option, that will check to see if commits are pushed. It should be pretty easy to write a pre-push hook as well that tests this so we don't have to rely on people using --recurse-submodules.
Aside from this, I'm curious if there are any other issues we need to deal with related to submodules.
Thanks! Doug
On 05/20/2014 07:59 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
https://github.com/ACME-Climate/multiple_repo_example Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash.
Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Hey Jim, I still don't see the single repo example on github. I was curious if you could push it up there so I can take a look at it. Thanks! Doug ________________________________________ From: [email protected] [[email protected]] on behalf of Jacobsen, Douglas W [[email protected]] Sent: Friday, May 23, 2014 8:38 AM To: Foucar, James G; Jed Brown; [email protected] Subject: Re: [ACME-SE] [EXTERNAL] Re: ACME Git setup Hey Jim, As far as I know "7308085" is the access code. Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Friday, May 23, 2014 8:33 AM To: Jacobsen, Douglas W; Jed Brown; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup Hi guys, This is my first time calling in to the telecon without Andy. When/how were you assigned an access code for readytalk? I'm looking at step 3: 3. Enter your 7-digit Access Code, followed by ³7308085#² -Jim On 5/22/14 7:22 PM, "Jacobsen, Douglas W" <[email protected]> wrote:
Hey Jim
I won't really have much to discuss tomorrow. Aside from having set up the repos and the experience I can share related to syncing commits / work between git and svn (ACME and CESM).
In terms of use, I haven't had enough time to actually test work flows in the repositories.
It would probably be good if we each took a usecase, and tried to test how it worked in each repo.
Hopefully this will be useful tomorrow, but I'm sure just discussing requirements with everyone will be beneficial.
Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Thursday, May 22, 2014 6:18 PM To: Jacobsen, Douglas W; Jed Brown; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
Hi guys,
It's probably too late to coordinate our message for tomorrow and for that I apologize. I assume Doug will discuss his investigation of submodules and I can talk about the single-repo investigation I've been doing.
On a related note, I've attached the script on which I've been working. The intended use is to be run daily by a maintainer to sync the sub-repo within the super-repo with the independent repo. It seems to work well so far, but admittedly my testing has been very light.
-Jim
On 5/21/14 3:30 PM, "Douglas Jacobsen" <[email protected]> wrote:
Two notes about some of the fragile issues related to submodules.
Here is a super simple pre-commit hook I wrote that tests if there are uncommitted changes in submodules before allowing you to commit. https://gist.github.com/douglasjacobsen/14b93d122a54d764d34e
Also, git push (as of 1.7.7.) has a --recurse-submodules=check option, that will check to see if commits are pushed. It should be pretty easy to write a pre-push hook as well that tests this so we don't have to rely on people using --recurse-submodules.
Aside from this, I'm curious if there are any other issues we need to deal with related to submodules.
Thanks! Doug
On 05/20/2014 07:59 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
https://github.com/ACME-Climate/multiple_repo_example Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash.
Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Hi Doug, I pushed it a few days ago. It's name is "single-repo-example". Have you refreshed the page? Do I need to do something special for others to see it? -Jim On 5/23/14 10:43 AM, "Jacobsen, Douglas W" <[email protected]> wrote:
Hey Jim,
I still don't see the single repo example on github. I was curious if you could push it up there so I can take a look at it.
Thanks! Doug ________________________________________ From: [email protected] [[email protected]] on behalf of Jacobsen, Douglas W [[email protected]] Sent: Friday, May 23, 2014 8:38 AM To: Foucar, James G; Jed Brown; [email protected] Subject: Re: [ACME-SE] [EXTERNAL] Re: ACME Git setup
Hey Jim,
As far as I know "7308085" is the access code.
Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Friday, May 23, 2014 8:33 AM To: Jacobsen, Douglas W; Jed Brown; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
Hi guys,
This is my first time calling in to the telecon without Andy. When/how were you assigned an access code for readytalk?
I'm looking at step 3: 3. Enter your 7-digit Access Code, followed by ³7308085#²
-Jim
On 5/22/14 7:22 PM, "Jacobsen, Douglas W" <[email protected]> wrote:
Hey Jim
I won't really have much to discuss tomorrow. Aside from having set up the repos and the experience I can share related to syncing commits / work between git and svn (ACME and CESM).
In terms of use, I haven't had enough time to actually test work flows in the repositories.
It would probably be good if we each took a usecase, and tried to test how it worked in each repo.
Hopefully this will be useful tomorrow, but I'm sure just discussing requirements with everyone will be beneficial.
Doug ________________________________________ From: Foucar, James G [[email protected]] Sent: Thursday, May 22, 2014 6:18 PM To: Jacobsen, Douglas W; Jed Brown; [email protected] Subject: Re: [EXTERNAL] Re: [ACME-SE] ACME Git setup
Hi guys,
It's probably too late to coordinate our message for tomorrow and for that I apologize. I assume Doug will discuss his investigation of submodules and I can talk about the single-repo investigation I've been doing.
On a related note, I've attached the script on which I've been working. The intended use is to be run daily by a maintainer to sync the sub-repo within the super-repo with the independent repo. It seems to work well so far, but admittedly my testing has been very light.
-Jim
On 5/21/14 3:30 PM, "Douglas Jacobsen" <[email protected]> wrote:
Two notes about some of the fragile issues related to submodules.
Here is a super simple pre-commit hook I wrote that tests if there are uncommitted changes in submodules before allowing you to commit. https://gist.github.com/douglasjacobsen/14b93d122a54d764d34e
Also, git push (as of 1.7.7.) has a --recurse-submodules=check option, that will check to see if commits are pushed. It should be pretty easy to write a pre-push hook as well that tests this so we don't have to rely on people using --recurse-submodules.
Aside from this, I'm curious if there are any other issues we need to deal with related to submodules.
Thanks! Doug
On 05/20/2014 07:59 AM, Jed Brown wrote:
Douglas Jacobsen <[email protected]> writes:
Just fyi, I changed the repo name to follow what it's intended purpose is.
https://github.com/ACME-Climate/multiple_repo_example Minor nit, but should we agree on dashes "multiple-repo-example" versus underscores? "ACME-Climate" uses a dash.
Are you proposing a workflow to go with this? What does a cross-cutting change look like? How about a conflicted merge? Is your intent to add scripting layers so that people can operate in the parent repo? (I think that is inherently fragile; many have tried and failed.)
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
"Foucar, James G" <[email protected]> writes:
Hi Doug,
I pushed it a few days ago. It's name is "single-repo-example".
Have you refreshed the page? Do I need to do something special for others to see it?
I don't see it listed here: https://github.com/ACME-Climate/ What address did you push to?
I see it as the very first repo on that list. On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
I don't see it either. Is it somehow private only to you? Rob On 5/23/14 12:25 PM, Foucar, James G wrote:
I see it as the very first repo on that list.
On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
OK, so clearly I've done something wrong. What happens when you guys go here: https://github.com/ACME-Climate/single-repo-example -Jim On 5/23/14 11:28 AM, "Robert Jacob" <[email protected]> wrote:
I don't see it either. Is it somehow private only to you?
Rob
On 5/23/14 12:25 PM, Foucar, James G wrote:
I see it as the very first repo on that list.
On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
404 error for me. Jeff On May 23, 2014, at 10:28 AM, Foucar, James G <[email protected]> wrote:
OK, so clearly I've done something wrong.
What happens when you guys go here: https://github.com/ACME-Climate/single-repo-example
-Jim
On 5/23/14 11:28 AM, "Robert Jacob" <[email protected]> wrote:
I don't see it either. Is it somehow private only to you?
Rob
On 5/23/14 12:25 PM, Foucar, James G wrote:
I see it as the very first repo on that list.
On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Do I need to transfer ownership to the ACME-Climate org? On 5/23/14 11:28 AM, "Foucar, James G" <[email protected]> wrote:
OK, so clearly I've done something wrong.
What happens when you guys go here: https://github.com/ACME-Climate/single-repo-example
-Jim
On 5/23/14 11:28 AM, "Robert Jacob" <[email protected]> wrote:
I don't see it either. Is it somehow private only to you?
Rob
On 5/23/14 12:25 PM, Foucar, James G wrote:
I see it as the very first repo on that list.
On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
OK, I just added ACME-Climate/admin as a collaborator for the repo. Hopefully that will do it. -Jim On 5/23/14 11:30 AM, "Foucar, James G" <[email protected]> wrote:
Do I need to transfer ownership to the ACME-Climate org?
On 5/23/14 11:28 AM, "Foucar, James G" <[email protected]> wrote:
OK, so clearly I've done something wrong.
What happens when you guys go here: https://github.com/ACME-Climate/single-repo-example
-Jim
On 5/23/14 11:28 AM, "Robert Jacob" <[email protected]> wrote:
I don't see it either. Is it somehow private only to you?
Rob
On 5/23/14 12:25 PM, Foucar, James G wrote:
I see it as the very first repo on that list.
On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
Works for me! Jeff On May 23, 2014, at 10:32 AM, Foucar, James G <[email protected]> wrote:
OK, I just added ACME-Climate/admin as a collaborator for the repo. Hopefully that will do it.
-Jim
On 5/23/14 11:30 AM, "Foucar, James G" <[email protected]> wrote:
Do I need to transfer ownership to the ACME-Climate org?
On 5/23/14 11:28 AM, "Foucar, James G" <[email protected]> wrote:
OK, so clearly I've done something wrong.
What happens when you guys go here: https://github.com/ACME-Climate/single-repo-example
-Jim
On 5/23/14 11:28 AM, "Robert Jacob" <[email protected]> wrote:
I don't see it either. Is it somehow private only to you?
Rob
On 5/23/14 12:25 PM, Foucar, James G wrote:
I see it as the very first repo on that list.
On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
I bet what happened is when you created the repo it only added it to the owners team. That meant only you and Andy could see it. I see it now though. Thanks! Doug ________________________________________ From: Jeffrey Johnson [[email protected]] Sent: Friday, May 23, 2014 11:32 AM To: Foucar, James G Cc: [email protected]; Jed Brown; Jacobsen, Douglas W; [email protected] Subject: Re: [ACME-SE] [EXTERNAL] Re: ACME Git setup Works for me! Jeff On May 23, 2014, at 10:32 AM, Foucar, James G <[email protected]> wrote:
OK, I just added ACME-Climate/admin as a collaborator for the repo. Hopefully that will do it.
-Jim
On 5/23/14 11:30 AM, "Foucar, James G" <[email protected]> wrote:
Do I need to transfer ownership to the ACME-Climate org?
On 5/23/14 11:28 AM, "Foucar, James G" <[email protected]> wrote:
OK, so clearly I've done something wrong.
What happens when you guys go here: https://github.com/ACME-Climate/single-repo-example
-Jim
On 5/23/14 11:28 AM, "Robert Jacob" <[email protected]> wrote:
I don't see it either. Is it somehow private only to you?
Rob
On 5/23/14 12:25 PM, Foucar, James G wrote:
I see it as the very first repo on that list.
On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
OK, great! Sorry for the confusionŠ never used GitHub before ACME. -Jim On 5/23/14 11:32 AM, "Jeffrey Johnson" <[email protected]> wrote:
Works for me!
Jeff
On May 23, 2014, at 10:32 AM, Foucar, James G <[email protected]> wrote:
OK, I just added ACME-Climate/admin as a collaborator for the repo. Hopefully that will do it.
-Jim
On 5/23/14 11:30 AM, "Foucar, James G" <[email protected]> wrote:
Do I need to transfer ownership to the ACME-Climate org?
On 5/23/14 11:28 AM, "Foucar, James G" <[email protected]> wrote:
OK, so clearly I've done something wrong.
What happens when you guys go here: https://github.com/ACME-Climate/single-repo-example
-Jim
On 5/23/14 11:28 AM, "Robert Jacob" <[email protected]> wrote:
I don't see it either. Is it somehow private only to you?
Rob
On 5/23/14 12:25 PM, Foucar, James G wrote:
I see it as the very first repo on that list.
On 5/23/14 11:24 AM, "Jed Brown" <[email protected]> wrote:
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
_______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
On 5/19/14 3:21 PM, Douglas Jacobsen wrote:
Mariana made it seem like they had already decided Github's svn support was insufficient for their needs, but I'm not sure what made them come to that conclusion. As you said, it should be easy enough to support svn usage within the repo if someone "requires" it.
This is based on experience. When doing a checkout of CESM, it will sometimes hang at the point it tries to get the external of MCT by doing an "svn co" of the MCT repo on Github. I'm not sure of recent problems but it happened often enough to leave a bad impression of Github's svn support. Rob
Doug _______________________________________________ ACME-SE mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/acme-se
This is based on experience. When doing a checkout of CESM, it will sometimes hang at the point it tries to get the external of MCT by doing an "svn co" of the MCT repo on Github. I'm not sure of recent problems but it happened often enough to leave a bad impression of Github's svn support.
It's weird that they had this issue. We've never run into this issue using MPAS-O within CESM, and it's linked in the exact same way. Doug
participants (7)
-
Douglas Jacobsen -
Foucar, James G -
Jacobsen, Douglas W
-
Jed Brown -
Jeffrey Johnson -
Robert Jacob -
Salinger, Andrew