April 2016 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] plans for PETSc release
by Satish Balay 24 Apr '16

24 Apr '16

On Sun, 24 Apr 2016, Jed Brown wrote: > Satish Balay <balay(a)mcs.anl.gov> writes: > > > On Sun, 24 Apr 2016, Jed Brown wrote: > > > >> Satish Balay <balay(a)mcs.anl.gov> writes: > >> > >> > Master currently doesn't work as you describe. > >> > >> Why doesn't it work that way? That was the philosophy when we adopted > >> this branching model years ago, it works reliably for many other > >> projects, and I thought it worked for us when we used it that way. Did > >> something change? > > > > I was thinking about the number of times master was broken in the last month. > > That's a workflow problem independent of releasing. But if you feel > like 'master' is not stable enough for the promise we try to make about > 'master', putting that instability in 'maint' is pretty much the most > reckless thing possible. You are infering something I did not say. I did not merge instable stuff into maint. > The reason for a feature freeze on 'master' is > to bring its stability up to that of 'maint'. I don't think thats enforceable. As it turns out - its not even enforceable on maint - as this thread demonstrates. > > >> > To me - currently we are at RC [yeah - witout a change in > >> > petscversion.h or a tag] - and RC to RELEASE should be via maint > >> > workflow - hence this update to maint. > >> > >> Why should RC to release "be via maint workflow"? > > > > The thought is - one you want a release in the next few days - if a > > fix is critical then it should go in. If a fix can't be in 3.7.1 then > > it shouldn't go in. And if a fix can be in 3.7.1 - then mostlikely it > > can wait till 3.7.1. > > Presumably at least one of those is supposed to be 3.7.0. But since you > haven't tagged v3.7, you're basically making 'maint' the new 'master', > which doesn't make any sense to me. I'm not making 'maint' the new master. I'm enforcing bugfix only policy. which is a maint policy. > > Anyway, if we allow 3.7.0 to ship with no more stability than a first > RC, Again you are misinterpreting my statement. I see you are interpreting" RC=>buggy - so RC should never be in maint". To me its equivalent of saying "3.6.3 is buggy - so it should have never be released - so only 3.6.4 shold have been released as 3.6" > we're basically telling users they shouldn't even bother trying > until *.*.1 releases. I think that's a cop out. Having a feature > freeze, clearing out 'next', and encouraging users to test 'master' is a > good way to make sure the *.*.0 releases are better. Barry made an anouncement on encouraging users to test a few weeks back. I don't think that works.. We are attempting to release what we know is stable. > > [there have been requests for TC testing - I was hoping we could be in > > this RC mode for a week - like a patch release test mode] > > TC? RC > > I think we should have a week of feature freeze (no new features to > 'master' or 'next') prior to tagging a release. But tag the release on > 'master'. > > >> I'd also argue we're clearly not at RC because new features (currently > >> in 'next') are still being merged. > > > > I was trying to avoid that. > > Well, there were several feature branches in 'next' before you > fast-forwarded 'maint' so it's not all a surprise. > > >> After changing 'maint', you still > >> advised Lisandro to merge new features to 'next' so that they could > >> later graduate for the release. > > > > If they can go into 3.7.1 - then they should be merged by then. [We've > > added new features in patch releases before - if they didn't modify > > exisiting API]. > > Normally the new features are more minor than a TS overhaul, but it can > be done. > > >> As for pending merges, what is the status of these branches (in 'next', > >> but not yet in 'master')? > > [...] > > > The same 'maint' criteria. If they can't go into a patch release - > > they shouldn't be merged to maint. If they can - then they can be > > merged [if they can be tested in maint again - or wait for next patch > > release] > > Normally we rewind 'next' when making a release. These features either > need to be merged or are abandoned for now (can try again for 3.8). If > they are 'maint'-eligible, then they should be merged for 3.7. Cleaning > up those loose ends is one of the things supposed to happen during the > freeze. You can formulate all this stuff now. We never had a proper freeze policy. I was attempting that. Satish > > I was trying to avoid this last minuite push to merge features - just > > before the release. > > Of course, which is why I prefer a feature freeze of about a week before > tagging a release. > > > Barry can override me on this - [If have to spin a tarball on monday] > > I won't merge anything into maint - unless they are tested fixes. And > > deferer other valid things to patch1 > > I know that when target dates slip, it's annoying and tempting to just > declare good enough. But why not spin an RC on Monday and tag the > release later in the week?

1 0

Re: [petsc-dev] plans for PETSc release
by Jed Brown 24 Apr '16

24 Apr '16

Lisandro Dalcin <dalcinl(a)gmail.com> writes: > On 24 April 2016 at 16:06, Jed Brown <jed(a)jedbrown.org> wrote: >> Okay, but I think we should never do a release this way again. We >> should finish all merging to 'master', tag the release on 'master', then >> fast-forward 'maint' and rewind 'next'. > > I'm with Jed. IMHO, that's the right workflow to make a new release. FWIW, here's gitworkflows(7): Branch management for a release Assuming you are using the merge approach discussed above, when you are releasing your project you will need to do some additional branch management work. A feature release is created from the master branch, since master tracks the commits that should go into the next feature release. The master branch is supposed to be a superset of maint. If this condition does not hold, then maint contains some commits that are not included on master. The fixes represented by those commits will therefore not be included in your feature release. To verify that master is indeed a superset of maint, use git log: Example 5. Verify master is a superset of maint git log master..maint This command should not list any commits. Otherwise, check out master and merge maint into it. Now you can proceed with the creation of the feature release. Apply a tag to the tip of master indicating the release version: Example 6. Release tagging git tag -s -m "Git X.Y.Z" vX.Y.Z master You need to push the new tag to a public Git server (see "DISTRIBUTED WORKFLOWS" below). This makes the tag available to others tracking your project. The push could also trigger a post-update hook to perform release-related items such as building release tarballs and preformatted documentation pages. Similarly, for a maintenance release, maint is tracking the commits to be released. Therefore, in the steps above simply tag and push maint rather than master. Maintenance branch management after a feature release After a feature release, you need to manage your maintenance branches. First, if you wish to continue to release maintenance fixes for the feature release made before the recent one, then you must create another branch to track commits for that previous release. To do this, the current maintenance branch is copied to another branch named with the previous release version number (e.g. maint-X.Y.(Z-1) where X.Y.Z is the current release). Example 7. Copy maint git branch maint-X.Y.(Z-1) maint The maint branch should now be fast-forwarded to the newly released code so that maintenance fixes can be tracked for the current release: Example 8. Update maint to new release · git checkout maint · git merge --ff-only master If the merge fails because it is not a fast-forward, then it is possible some fixes on maint were missed in the feature release. This will not happen if the content of the branches was verified as described in the previous section.

1 0

Re: [petsc-dev] Open PRs to be merged for release 3.7
by Jed Brown 24 Apr '16

24 Apr '16

Satish Balay <balay(a)mcs.anl.gov> writes: > I was hoping for a feature freeze - by my maint branch merge. > [and Barry agreed to that :). See the text of the post: "[petsc-dev] plans for PETSc release"] I think the text of that email inadvertently (or intentionally?) changed the release process in a rather fundamental way that I disagree with. We never had a petsc-dev saying that a feature freeze will be in effect starting Xday and dumping 'master' into 'maint' without a feature freeze is reckless because we're knowingly advertising more stability (via the 'maint' label) than is actually present. > Irrespective the maint issue - the changes are not yet tested in > next. Even if its tested in tonights next - they would be untested in > maint/master. > > So I can't do both - merge them tomorrow and spin a tarball tomorrow. My preference would be to push the release date back by a few days so we can ensure a more stable v3.7. This irrespective of whether Lisandro's changes are merged for v3.7.

1 0

Re: [petsc-dev] plans for PETSc release
by Jed Brown 24 Apr '16

24 Apr '16

Satish Balay <balay(a)mcs.anl.gov> writes: > On Sun, 24 Apr 2016, Jed Brown wrote: > >> Satish Balay <balay(a)mcs.anl.gov> writes: >> >> > Master currently doesn't work as you describe. >> >> Why doesn't it work that way? That was the philosophy when we adopted >> this branching model years ago, it works reliably for many other >> projects, and I thought it worked for us when we used it that way. Did >> something change? > > I was thinking about the number of times master was broken in the last month. That's a workflow problem independent of releasing. But if you feel like 'master' is not stable enough for the promise we try to make about 'master', putting that instability in 'maint' is pretty much the most reckless thing possible. The reason for a feature freeze on 'master' is to bring its stability up to that of 'maint'. >> > To me - currently we are at RC [yeah - witout a change in >> > petscversion.h or a tag] - and RC to RELEASE should be via maint >> > workflow - hence this update to maint. >> >> Why should RC to release "be via maint workflow"? > > The thought is - one you want a release in the next few days - if a > fix is critical then it should go in. If a fix can't be in 3.7.1 then > it shouldn't go in. And if a fix can be in 3.7.1 - then mostlikely it > can wait till 3.7.1. Presumably at least one of those is supposed to be 3.7.0. But since you haven't tagged v3.7, you're basically making 'maint' the new 'master', which doesn't make any sense to me. Anyway, if we allow 3.7.0 to ship with no more stability than a first RC, we're basically telling users they shouldn't even bother trying until *.*.1 releases. I think that's a cop out. Having a feature freeze, clearing out 'next', and encouraging users to test 'master' is a good way to make sure the *.*.0 releases are better. > [there have been requests for TC testing - I was hoping we could be in > this RC mode for a week - like a patch release test mode] TC? I think we should have a week of feature freeze (no new features to 'master' or 'next') prior to tagging a release. But tag the release on 'master'. >> I'd also argue we're clearly not at RC because new features (currently >> in 'next') are still being merged. > > I was trying to avoid that. Well, there were several feature branches in 'next' before you fast-forwarded 'maint' so it's not all a surprise. >> After changing 'maint', you still >> advised Lisandro to merge new features to 'next' so that they could >> later graduate for the release. > > If they can go into 3.7.1 - then they should be merged by then. [We've > added new features in patch releases before - if they didn't modify > exisiting API]. Normally the new features are more minor than a TS overhaul, but it can be done. >> As for pending merges, what is the status of these branches (in 'next', >> but not yet in 'master')? [...] > The same 'maint' criteria. If they can't go into a patch release - > they shouldn't be merged to maint. If they can - then they can be > merged [if they can be tested in maint again - or wait for next patch > release] Normally we rewind 'next' when making a release. These features either need to be merged or are abandoned for now (can try again for 3.8). If they are 'maint'-eligible, then they should be merged for 3.7. Cleaning up those loose ends is one of the things supposed to happen during the freeze. > I was trying to avoid this last minuite push to merge features - just > before the release. Of course, which is why I prefer a feature freeze of about a week before tagging a release. > Barry can override me on this - [If have to spin a tarball on monday] > I won't merge anything into maint - unless they are tested fixes. And > deferer other valid things to patch1 I know that when target dates slip, it's annoying and tempting to just declare good enough. But why not spin an RC on Monday and tag the release later in the week?

1 0

Re: [petsc-dev] Open PRs to be merged for release 3.7
by Satish Balay 24 Apr '16

24 Apr '16

On Sun, 24 Apr 2016, Lisandro Dalcin wrote: > On 24 April 2016 at 18:38, Satish Balay <balay(a)mcs.anl.gov> wrote: > > On Sun, 24 Apr 2016, Lisandro Dalcin wrote: > > > >> Satish, I have two open PRs ready to be merged into master (already > >> merged into next). As you are in charge of making the release, I did > >> not want to step on you, so I'll not make the merge into master until > >> you acknowledge. Or just merge them yourself through the Bitbucket > >> interface. > > > > Lisandro, > > > > Looks like I won't be able to merge then for 3.7. > > > > If the API reviewers [Jed, Barry] says that API isn't changing - then > > I'll schedule them for 3.7.1. [If not - they can be 'master' features] > > > > Barry agreed in petsc-dev ML to get these new features in. This is > because of your premature update of 'maint' branch? I was hoping for a feature freeze - by my maint branch merge. [and Barry agreed to that :). See the text of the post: "[petsc-dev] plans for PETSc release"] Irrespective the maint issue - the changes are not yet tested in next. Even if its tested in tonights next - they would be untested in maint/master. So I can't do both - merge them tomorrow and spin a tarball tomorrow. > PS: These two PRs are forward compatible: they add new APIs, but do > not modify previous ones in any way. Then there should be no problem with scheduling them for 3.7.1 one issue : does Jed consider changes to include/petsc/private/tsimpl.h as API/ABI change and object.. Satish

1 0

Re: [petsc-dev] plans for PETSc release
by Jed Brown 24 Apr '16

24 Apr '16

Satish Balay <balay(a)mcs.anl.gov> writes: > On Sun, 24 Apr 2016, Satish Balay wrote: > >> > As for pending merges, what is the status of these branches (in 'next', >> > but not yet in 'master')? >> > >> > barry/fix-some-clang-warnings >> > knepley/feature-fe-nonaffine >> > knepley/fix-mem-init >> > pr329/master/Fande-Kong/matpartitioning-hierarch >> > pr359/master/Fande-Kong/fix-mat-convert-nest-aij >> > tisaac/dmp4est-feature-mapped-coordinates >> > tisaac/fix-ex3-coords > > BTW: I see some discrepancy here > > balay@asterix /home/balay/petsc (maint=) > $ git fetch -p > balay@asterix /home/balay/petsc (maint=) > $ git branch -r |grep pr359 > balay@asterix /home/balay/petsc (maint=) > $ git branch -r |grep pr329 Oh, those branches must have been abandoned; I hadn't pruned recently. You have the correct list. > balay@asterix /home/balay/petsc (maint=) > $ comm -12 <(git branch -r --merged origin/next | sort) <(git branch -r --no-merged origin/master | sort) > origin/barry/fix-some-clang-warnings > origin/dalcinl/ts-2nd-order-systems > origin/dalcinl/tsbdf > origin/knepley/feature-fe-nonaffine > origin/next > origin/tisaac/dmp4est-feature-mapped-coordinates > origin/tisaac/fix-ex3-coords > balay@asterix /home/balay/petsc (maint=) > $ > > > Satish

1 0

Re: [petsc-dev] Open PRs to be merged for release 3.7
by Lisandro Dalcin 24 Apr '16

24 Apr '16

On 24 April 2016 at 18:38, Satish Balay <balay(a)mcs.anl.gov> wrote: > On Sun, 24 Apr 2016, Lisandro Dalcin wrote: > >> Satish, I have two open PRs ready to be merged into master (already >> merged into next). As you are in charge of making the release, I did >> not want to step on you, so I'll not make the merge into master until >> you acknowledge. Or just merge them yourself through the Bitbucket >> interface. > > Lisandro, > > Looks like I won't be able to merge then for 3.7. > > If the API reviewers [Jed, Barry] says that API isn't changing - then > I'll schedule them for 3.7.1. [If not - they can be 'master' features] > Barry agreed in petsc-dev ML to get these new features in. This is because of your premature update of 'maint' branch? PS: These two PRs are forward compatible: they add new APIs, but do not modify previous ones in any way. -- Lisandro Dalcin ============ Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 0109 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459

1 0

Re: [petsc-dev] Open PRs to be merged for release 3.7
by Satish Balay 24 Apr '16

24 Apr '16

On Sun, 24 Apr 2016, Lisandro Dalcin wrote: > Satish, I have two open PRs ready to be merged into master (already > merged into next). As you are in charge of making the release, I did > not want to step on you, so I'll not make the merge into master until > you acknowledge. Or just merge them yourself through the Bitbucket > interface. Lisandro, Looks like I won't be able to merge then for 3.7. If the API reviewers [Jed, Barry] says that API isn't changing - then I'll schedule them for 3.7.1. [If not - they can be 'master' features] Satish

1 0

Re: [petsc-dev] plans for PETSc release
by Satish Balay 24 Apr '16

24 Apr '16

On Sun, 24 Apr 2016, Satish Balay wrote: > > As for pending merges, what is the status of these branches (in 'next', > > but not yet in 'master')? > > > > barry/fix-some-clang-warnings > > knepley/feature-fe-nonaffine > > knepley/fix-mem-init > > pr329/master/Fande-Kong/matpartitioning-hierarch > > pr359/master/Fande-Kong/fix-mat-convert-nest-aij > > tisaac/dmp4est-feature-mapped-coordinates > > tisaac/fix-ex3-coords BTW: I see some discrepancy here balay@asterix /home/balay/petsc (maint=) $ git fetch -p balay@asterix /home/balay/petsc (maint=) $ git branch -r |grep pr359 balay@asterix /home/balay/petsc (maint=) $ git branch -r |grep pr329 balay@asterix /home/balay/petsc (maint=) $ comm -12 <(git branch -r --merged origin/next | sort) <(git branch -r --no-merged origin/master | sort) origin/barry/fix-some-clang-warnings origin/dalcinl/ts-2nd-order-systems origin/dalcinl/tsbdf origin/knepley/feature-fe-nonaffine origin/next origin/tisaac/dmp4est-feature-mapped-coordinates origin/tisaac/fix-ex3-coords balay@asterix /home/balay/petsc (maint=) $ Satish

1 0

Re: [petsc-dev] plans for PETSc release
by Satish Balay 24 Apr '16

24 Apr '16

On Sun, 24 Apr 2016, Jed Brown wrote: > Satish Balay <balay(a)mcs.anl.gov> writes: > > > Master currently doesn't work as you describe. > > Why doesn't it work that way? That was the philosophy when we adopted > this branching model years ago, it works reliably for many other > projects, and I thought it worked for us when we used it that way. Did > something change? I was thinking about the number of times master was broken in the last month. > > > To me - currently we are at RC [yeah - witout a change in > > petscversion.h or a tag] - and RC to RELEASE should be via maint > > workflow - hence this update to maint. > > Why should RC to release "be via maint workflow"? The thought is - one you want a release in the next few days - if a fix is critical then it should go in. If a fix can't be in 3.7.1 then it shouldn't go in. And if a fix can be in 3.7.1 - then mostlikely it can wait till 3.7.1. [there have been requests for TC testing - I was hoping we could be in this RC mode for a week - like a patch release test mode] > > I'd also argue we're clearly not at RC because new features (currently > in 'next') are still being merged. I was trying to avoid that. > After changing 'maint', you still > advised Lisandro to merge new features to 'next' so that they could > later graduate for the release. If they can go into 3.7.1 - then they should be merged by then. [We've added new features in patch releases before - if they didn't modify exisiting API]. > If any of those changes affect the API, > then people on 'maint' have gotten an API that will never be released. > This breaks our promises with respect to 'maint'. The whole point of > that branch is to have something stable that users can update > worry-free. > > > I didn't intend to be rude to our 'maint' users. Ideally there should > > have been a tracking branch maint-3.6 always - so users who whish to > > time the switch from 3.6 to 3.7 do this on their terms [and not > > implicitly via maint]. > > > > I'll make an annoucement to maint users about switching to maint-3.6 > > branch. > > Okay, but I think we should never do a release this way again. We > should finish all merging to 'master', tag the release on 'master', then > fast-forward 'maint' and rewind 'next'. > > As for pending merges, what is the status of these branches (in 'next', > but not yet in 'master')? > > barry/fix-some-clang-warnings > knepley/feature-fe-nonaffine > knepley/fix-mem-init > pr329/master/Fande-Kong/matpartitioning-hierarch > pr359/master/Fande-Kong/fix-mat-convert-nest-aij > tisaac/dmp4est-feature-mapped-coordinates > tisaac/fix-ex3-coords The same 'maint' criteria. If they can't go into a patch release - they shouldn't be merged to maint. If they can - then they can be merged [if they can be tested in maint again - or wait for next patch release] I was trying to avoid this last minuite push to merge features - just before the release. Barry can override me on this - [If have to spin a tarball on monday] I won't merge anything into maint - unless they are tested fixes. And deferer other valid things to patch1 Satish

1 0