Bot to automatically generate cherry-picking PRs

[brettcannon]

As pulled from PEP 512:

Since the decision has been made to work with cherry-picks instead of forward merging of branches, it would be convenient to have a bot that would generate pull requests based on cherry-picking for any pull requests that affect multiple branches. The most likely design is a bot that monitors merged pull requests with key labels applied that delineate what branches the pull request should be cherry-picked into. The bot would then generate cherry-pick pull requests for each label and remove the labels as the pull requests are created (this allows for easy detection when automatic cherry-picking failed).

[soltysh]

I'm interested in helping with this one 😄

[brettcannon]

Yeah, I think this will be one of those high-impact improvements that will definitely make PR acceptance more pleasurable compared to the hg workflow. I think this should be number 2 on the TODO list once #6 is dealt with (as I see the NEWS file being the stickiest point in accepting PRs and it also makes this issue easier by lowering potential merge conflicts).

[berkerpeksag]

Note that I already have a working bot (tested it on https://github.com/fayton/cpython), but I didn't had time to clean it up and make it open source in the past months.

My plan is to work on bug fixes for 3.5.3, 3.6.1 and python.org over the holiday. I can help you after mid-January if you had time to create a prototype during the holiday (or you can help me to clean up my bot if none of us can work on a prototype before mid-January -- I codenamed it "Duke Nukem Forever" :))

[soltysh]

👍 @berkerpeksag we can sync and I'd love to help with that.

[berkerpeksag]

Just an FYI, I've finally cleaned-up my bot and set up on my test repository. There is a missing feature though: it doesn't create a new pull request; it automatically pushes the cherry-picked commit into maintenance branches.

See https://github.com/fayton/cpython/pull/2/#issuecomment-278313431 for an example. Note that it also automatically fills the committer field by inspecting the "core developers" team created on the organization.

[brettcannon]

What about if the cherry-pick fails either due to testing or the code not patching cleanly? And how do you keep track of what has (not) successfully been cherry-picked (I'm assuming some label on the original PR)?

[Carreau]

What about if the cherry-pick fails either due to testing or the code not patching cleanly?

The other case I have seen is auto backport of an already backport patch. So the 2 regular failure of one of my bot is either:

• empty commit after cherry-picking.
• conflict.
• Other (remote commit disappeared in the meantime/ GitHub down/ API rate limited.).

Seeing that now with github you can edit in the UI, you could try to commit and push the conflict on another branch, and resolve it in the GitHub UI as well.

[berkerpeksag]

What about if the cherry-pick fails either due to testing or the code not patching cleanly?

It reverts to the current state of upstream branch and adds a comment to the original pull request.

And how do you keep track of what has (not) successfully been cherry-picked (I'm assuming some label on the original PR)?

I'm using a simple sqlite database to keep track of the state [1], but we can add a label to the original PR too.

[1] This way it's also easy to quit early if the commit has already been cherry-picked.

[WhyNotHugo]

Since the decision has been made to work with cherry-picks instead of forward merging of branches, it would be convenient to have a bot that would generate pull requests based on cherry-picking for any pull requests that affect multiple branches.

Out of curiosity, why was this decision taken? PEP-0512 doesn't seem to explain any of the benefits of doing this.

[brettcannon]

@hobarrera couple of reasons. One is people instinctively create PRs against master on GitHub and merge-forward wouldn't like that. It also allows you to accept commits on master while any discussion about backporting happens. Finally it makes the 2.7 no longer the odd-ball branch that never merges anywhere.

[brettcannon]

Just so no one feels too much pressure for getting a bot up, we could also write a Python script to automate this locally as a hold-over. I would expect a CLI like backport --commit <hash> --versions [comma-separated Python versions] [--upstream <remote for python/cpython>] [--fork <remote to personal fork>]. This would do:

git fetch {upstream}
for version in versions:
    git branch {version} backport-{version}-{commit}
    if git cherry-pick ...:
        git push {fork} backport-{version}-{commit}
    else:
        print(f"cherry-pick for {version} failed", file=sys.stderr)

Could even get really fancy and if someone provides GitHub credentials have the PR also be automatically created.

[Carreau]

https://github.com/minrk/ghpro may be of help:

backport-pr apply 4.x 1234

Made to work with merge commit, but I'm guessing that shouldn't be hard to change.