On Fri, 10 Oct 2014, Dave Nystrom wrote:
Thanks for your detailed reply. Suppose I start working with wdn_mods_master instead of wdn_mods_next where wdn_mods_master is master + my local changes to master. Now, suppose I find a bug which I report. With the current petsc workflow, it seems like that bug would get fixed on next and then merged to master after some amount of time. Is that correct?
Nope - the bug is fixed in a corresponding 'feature' branch [where the bug was originally introduced] or a new 'bug-fix' branch [created off latest master/maint] or directly in master/maint (if we think its a trivial fix that wont' break stuff). And then this branch is merged to next for further testing. Note: We don't develop (or make direct commits) on 'next'. It is an integration branch where feature/bugfix branches are merged into - and from where nightly tests are done. We find bugs with this testing - or we realize (very rarely) some of these feature branches won't graduate and make it to master [or need to be redone]. Hence its discard after every release. You are correct in the sense that most branches [sometimes bugfixes] have to stay in next for some time - before they get merged to master. And this 'time' 1 day or more [due to the way some of this testing is done - party the shedule - partly the amount of testing - and partly how we manually interpret test results and match them with these branches]
If so, then I have to wait for that amount of time before I can have the fix. Or I would need to figure out how to pull it into my wdn_mods_master from next.
You can temporarily switch to the 'bug-fix' branch and use it - or merge it [but then be prepared to revert it - if this branch is rebased or discarded]. Yeah - its not ideal. But one of the points of git is to use/discard branches as you need. [trying to stick to one branch to track multi-branch development can be difficult]
And then most of the petsc features I am most interested in show up in next before master - like the stuff with cuda, threads and ViennaCL, etc. So I'm not sure how to really dispense with my use of and reliance on next. But I will give it more thought.
Yes - next is where the features show up first. But thats different then 'use next for bugfixes' - as 'next' is likely to have more new bugs [due to these new features] than bugfixes.
And I will study the details of your reply below and see if I can upgrade my git skills. But I've never had to use the rebase, reset or cherry picking features of git before. So it will take take some study.
And based on the issue you are having with 'git reset --hard next' (the 'next' part - not the 'git reset' part) - an understanding of git branches is perhaps needed. Without it - its not easy to use git [or communicate about it] for eg: - understaning origin/branchname vs branchname [and how fetch/pull (and other basic operations) differ on origin/branchname vs branchname - use of commit-id vs branchname in various git commands - branchname vs tag - what is a 'new branch' - and what does 'delete branch' mean etc.. Satish
Satish Balay writes:
I believe the workflow used by petsc is same as what most projects with git do - so there is nothing special about it.
Try 'man gitworkflows'
I'll check this out.
I think your reason for using next is flawed (you get more bugs than bugfixes on next - as its an integration branch - and features are merged here for testing purposes). But if thats what you want to do - thats fine. We can benefit with the extra testing that you are now providing on next. And I believe git gives you tools to manage whatever workflow you choose.
Generally we don't branch off next. New branches are created from master (or maint). This works well for new features workflow (or bug fix workflow).
In your case - you'd like to maintain 'wdn_mods_next' branch - which is 'next' + local fixes that are specific to next (and not shared across other branches - like master). And these fixes are next specific - hence you want this branch to start off next. Presumably this is low over head stuff - and don't need permanant history for this (as we usually do with feature branches)
It is low over head and I don't need a permanent history of petsc changes.
For the above requirement - currently you do:
a. branch wdn_mods_next from next b. add local commits in this branch c. merge latest next into wdn_mods_next. (as needed)
Right.
Alternative is to use 'git rebase' to manage your local commits. This is not critical - but it lets you keep track of them more sanely.
<rebase local commits over latest next> git checkout next git pull git checkout wdn_mods_next git rebase next
[also check 'git rebase -i']
OK, I don't really know what rebase does - so I will have to study the documentation and my git book.
In the present situation - when next is unwound - you have to unwind and recreate wdn_mods_next. For this you'd like to extract your current modifications from wdn_mods_next - and apply to a new wdn_mods_next branch.
<First step is to backup current wdn_mods_next. i.e:> git checkout wdn_mods_next git checkout -b wdn_mods_next-oct-2014
<Now reset wdn_mods_next to current next> git checkout wdn_mods_next git reset --hard next # assuming you've already reset 'next' to latest
At the moment, I'm not sure the above git reset command will work - based on trying it and getting a git error message.
<check previous local modifications> git log origin/next-oct-2014..wdn_mods_next-oct-2014 git diff origin/next-oct-2014..wdn_mods_next-oct-2014
Now you can 'git cherrypick' the commits you need [from the above list] or grab the above diff - and apply to dn_mods_next(new) branch. for eg:
git diff origin/next-oct-2014..wdn_mods_next-oct-2014 > my_changes patch -Np1 < my_changes <resolve patch conflicts manually> git commit
This seems like a complicated procedure but maybe it will seem simpler and more intuitive if I study this more.
Thanks,
Dave
Satish
On Fri, 10 Oct 2014, Nystrom, William David wrote:
Jed,
I don't think your reply below really does what I want to do. My wdn_mods_next branch is a branch that I maintain that was derived from the next branch. I do not merge changes from wdn_mods_next into the next branch. The next branch is supposed to be a mirror of the official petsc next branch and I only update the next branch in my petsc clones by first performing a "git remote -v update origin" in my bare mirror repo which mirrors the petsc maint, master and next branches as well as containing any additional branches that I create.
So, I update wdn_mods_next from the latest copy of next that I have by using the following command:
git pull origin next
in the petsc directory where I have wdn_mods_next checked out. Generally this is easy and has no conflicts to resolve but on a rare occasion, I might have to resolve a conflict. This is because my wdn_mods_next branch does not differ much from next - just a few hacks I have had to resort to.
So here is one way I think I could accomplish my goal. In my bare, mirror repo execute
git remote -v update origin
At this point my bare mirror repo now has a copy of the new unwound next branch - if I am understanding correctly.
Then in a clone of my mirror repo execute
git checkout wdn_mods_next git reset --hard next git push origin +wdn_mods_next
If I understand correctly, then my wdn_mods_next branch would be identical to the next branch and I would not have the changes I made to next that resulted in wdn_mods_next.
Then I could untar a file of those changes on top of my clone with wdn_mods_next checked out and recommit those changes. This is about half a dozen files. But this seems like a hacky way to do business that is only reasonable because I have a small number of files.
Isn't there a better way to do this?
Perhaps you or Satish might ask why I am using next instead of master. The reason is that I am forever uncovering some bug in petsc that you guys fix on the next branch and I don't want to wait for several days before it gets pulled into master and I can resume productive work.
So maybe there is some flaw or weakness in my git work flow. I have been using git for several years now but am not a power user. I like git but do not really want to have to become a git guru. However, this petsc policy of periodically unwinding the next branch seems to always disrupt my workflow - so I have not found a robust way to deal with it yet.
Dave
________________________________________ From: Jed Brown [[email protected]] Sent: Thursday, October 02, 2014 4:39 PM To: Satish Balay; Nystrom, William David Cc: petsc-users; [email protected] Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound!
Satish Balay <[email protected]> writes:
You would have to delete both 'next' and 'wdn_mods_next' branches from all your clones - and recreate them.
Do your normal "git remote update" (or git fetch) in your "master" clone (confusing to use the same name to refer to a repository and a branch). I'll call the repository Mirror. Anyway, you should see that all three branches are updated. 'maint' and 'master' should fast-forward, while 'next' will say "forced update".
After that, recreate 'next' on the clones just like in Satish's original email.