On Fri, 10 Oct 2014, Dave Nystrom wrote:
> Thanks for your detailed reply. Suppose I start working with wdn_mods_master
> instead of wdn_mods_next where wdn_mods_master is master + my local changes
> to master. Now, suppose I find a bug which I report. With the current petsc
> workflow, it seems like that bug would get fixed on next and then merged to
> master after some amount of time. Is that correct?
Nope - the bug is fixed in a corresponding 'feature' branch [where the
bug was originally introduced] or a new 'bug-fix' branch [created off
latest master/maint] or directly in master/maint (if we think its a
trivial fix that wont' break stuff). And then this branch is merged to
next for further testing.
Note: We don't develop (or make direct commits) on 'next'. It is an
integration branch where feature/bugfix branches are merged into - and
from where nightly tests are done. We find bugs with this testing - or
we realize (very rarely) some of these feature branches won't graduate
and make it to master [or need to be redone]. Hence its discard after
every release.
You are correct in the sense that most branches [sometimes bugfixes]
have to stay in next for some time - before they get merged to
master. And this 'time' 1 day or more [due to the way some of this
testing is done - party the shedule - partly the amount of testing -
and partly how we manually interpret test results and match them with
these branches]
> If so, then I have to
> wait for that amount of time before I can have the fix. Or I would need to
> figure out how to pull it into my wdn_mods_master from next.
You can temporarily switch to the 'bug-fix' branch and use it - or
merge it [but then be prepared to revert it - if this branch is
rebased or discarded]. Yeah - its not ideal. But one of the points of
git is to use/discard branches as you need. [trying to stick to one
branch to track multi-branch development can be difficult]
> And then most of the petsc features I am most interested in show up in next
> before master - like the stuff with cuda, threads and ViennaCL, etc. So I'm
> not sure how to really dispense with my use of and reliance on next. But I
> will give it more thought.
Yes - next is where the features show up first. But thats different
then 'use next for bugfixes' - as 'next' is likely to have more new
bugs [due to these new features] than bugfixes.
> And I will study the details of your reply below and see if I can upgrade my
> git skills. But I've never had to use the rebase, reset or cherry picking
> features of git before. So it will take take some study.
And based on the issue you are having with 'git reset --hard next'
(the 'next' part - not the 'git reset' part) - an understanding of git
branches is perhaps needed. Without it - its not easy to use git [or
communicate about it]
for eg:
- understaning origin/branchname vs branchname [and how fetch/pull
(and other basic operations) differ on origin/branchname vs
branchname
- use of commit-id vs branchname in various git commands
- branchname vs tag
- what is a 'new branch' - and what does 'delete branch' mean
etc..
Satish
>
> Satish Balay writes:
> > I believe the workflow used by petsc is same as what most projects
> > with git do - so there is nothing special about it.
> >
> > Try 'man gitworkflows'
>
> I'll check this out.
>
> > I think your reason for using next is flawed (you get more bugs than
> > bugfixes on next - as its an integration branch - and features are
> > merged here for testing purposes). But if thats what you want to do -
> > thats fine. We can benefit with the extra testing that you are now
> > providing on next. And I believe git gives you tools to manage
> > whatever workflow you choose.
> >
> > Generally we don't branch off next. New branches are created from
> > master (or maint). This works well for new features workflow (or bug
> > fix workflow).
> >
> > In your case - you'd like to maintain 'wdn_mods_next' branch - which
> > is 'next' + local fixes that are specific to next (and not shared
> > across other branches - like master). And these fixes are next
> > specific - hence you want this branch to start off next. Presumably
> > this is low over head stuff - and don't need permanant history for
> > this (as we usually do with feature branches)
>
> It is low over head and I don't need a permanent history of petsc changes.
>
> > For the above requirement - currently you do:
> >
> > a. branch wdn_mods_next from next
> > b. add local commits in this branch
> > c. merge latest next into wdn_mods_next. (as needed)
>
> Right.
>
> > Alternative is to use 'git rebase' to manage your local commits. This
> > is not critical - but it lets you keep track of them more sanely.
> >
> > <rebase local commits over latest next>
> > git checkout next
> > git pull
> > git checkout wdn_mods_next
> > git rebase next
> >
> > [also check 'git rebase -i']
>
> OK, I don't really know what rebase does - so I will have to study the
> documentation and my git book.
>
> > In the present situation - when next is unwound - you have to unwind
> > and recreate wdn_mods_next. For this you'd like to extract your
> > current modifications from wdn_mods_next - and apply to a new
> > wdn_mods_next branch.
> >
> > <First step is to backup current wdn_mods_next. i.e:>
> > git checkout wdn_mods_next
> > git checkout -b wdn_mods_next-oct-2014
> >
> > <Now reset wdn_mods_next to current next>
> > git checkout wdn_mods_next
> > git reset --hard next # assuming you've already reset 'next' to latest
>
> At the moment, I'm not sure the above git reset command will work - based on
> trying it and getting a git error message.
>
> > <check previous local modifications>
> > git log origin/next-oct-2014..wdn_mods_next-oct-2014
> > git diff origin/next-oct-2014..wdn_mods_next-oct-2014
> >
> > Now you can 'git cherrypick' the commits you need [from the above
> > list] or grab the above diff - and apply to dn_mods_next(new) branch. for eg:
> >
> > git diff origin/next-oct-2014..wdn_mods_next-oct-2014 > my_changes
> > patch -Np1 < my_changes
> > <resolve patch conflicts manually>
> > git commit
>
> This seems like a complicated procedure but maybe it will seem simpler and
> more intuitive if I study this more.
>
> Thanks,
>
> Dave
>
> > Satish
> >
> > On Fri, 10 Oct 2014, Nystrom, William David wrote:
> >
> > > Jed,
> > >
> > > I don't think your reply below really does what I want to do. My wdn_mods_next branch is
> > > a branch that I maintain that was derived from the next branch. I do not merge changes
> > > from wdn_mods_next into the next branch. The next branch is supposed to be a mirror of
> > > the official petsc next branch and I only update the next branch in my petsc clones by first
> > > performing a "git remote -v update origin" in my bare mirror repo which mirrors the petsc maint,
> > > master and next branches as well as containing any additional branches that I create.
> > >
> > > So, I update wdn_mods_next from the latest copy of next that I have by using the following
> > > command:
> > >
> > > git pull origin next
> > >
> > > in the petsc directory where I have wdn_mods_next checked out. Generally this is easy and
> > > has no conflicts to resolve but on a rare occasion, I might have to resolve a conflict. This is
> > > because my wdn_mods_next branch does not differ much from next - just a few hacks I have
> > > had to resort to.
> > >
> > > So here is one way I think I could accomplish my goal. In my bare, mirror repo execute
> > >
> > > git remote -v update origin
> > >
> > > At this point my bare mirror repo now has a copy of the new unwound next branch - if I am
> > > understanding correctly.
> > >
> > > Then in a clone of my mirror repo execute
> > >
> > > git checkout wdn_mods_next
> > > git reset --hard next
> > > git push origin +wdn_mods_next
> > >
> > > If I understand correctly, then my wdn_mods_next branch would be identical to the next
> > > branch and I would not have the changes I made to next that resulted in wdn_mods_next.
> > >
> > > Then I could untar a file of those changes on top of my clone with wdn_mods_next checked
> > > out and recommit those changes. This is about half a dozen files. But this seems like a
> > > hacky way to do business that is only reasonable because I have a small number of files.
> > >
> > > Isn't there a better way to do this?
> > >
> > > Perhaps you or Satish might ask why I am using next instead of master. The reason is that
> > > I am forever uncovering some bug in petsc that you guys fix on the next branch and I don't
> > > want to wait for several days before it gets pulled into master and I can resume productive
> > > work.
> > >
> > > So maybe there is some flaw or weakness in my git work flow. I have been using git for
> > > several years now but am not a power user. I like git but do not really want to have to
> > > become a git guru. However, this petsc policy of periodically unwinding the next branch
> > > seems to always disrupt my workflow - so I have not found a robust way to deal with it
> > > yet.
> > >
> > > Dave
> > >
> > > ________________________________________
> > > From: Jed Brown [jed(a)jedbrown.org]
> > > Sent: Thursday, October 02, 2014 4:39 PM
> > > To: Satish Balay; Nystrom, William David
> > > Cc: petsc-users; petsc-dev(a)mcs.anl.gov
> > > Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound!
> > >
> > > Satish Balay <balay(a)mcs.anl.gov> writes:
> > >
> > > > You would have to delete both 'next' and 'wdn_mods_next' branches from
> > > > all your clones - and recreate them.
> > >
> > > Do your normal "git remote update" (or git fetch) in your "master" clone
> > > (confusing to use the same name to refer to a repository and a branch).
> > > I'll call the repository Mirror. Anyway, you should see that all three
> > > branches are updated. 'maint' and 'master' should fast-forward, while
> > > 'next' will say "forced update".
> > >
> > > After that, recreate 'next' on the clones just like in Satish's original
> > > email.
> > >
>
