ratchetFrom 'origin/main' does not fetch if not present

[JLLeitschuh]

If you specify rachetFrom and the branch doesn't exist locally, it should be fetched by the ratchet infrastructure.

https://github.com/JLLeitschuh/spotless/runs/1200916598?check_suite_focus=true#step:5:56

  [2020-10-02 21:07:06] [autobuild] * What went wrong:
  [2020-10-02 21:07:06] [autobuild] A problem occurred evaluating project ':ide'.
  [2020-10-02 21:07:06] [autobuild] > Could not create task ':lib:spotlessJava'.
  [2020-10-02 21:07:06] [autobuild]    > No such reference 'origin/main'

EDIT: Summary of workarounds from below

• GitHub Actions: add fetch-depth: 0 to <action>.yml
• GitLab CI: add GIT_DEPTH: 0 under the variables: section of .gitlab-ci.yml
• BitBucket Pipelines: clone: depth: full
• Travis: git: depth: false

[nedtwigg]

Interesting, good idea! I think GitHub is doing clone --depth=1. Doing a fetch origin main right after sort of defeats the purpose of the shallow clone, and the user is probably better off performance-wise doing something like fetch-depth: 0 (code for deep clone) , which does a full clone in the first place.

Even so, I still think it's better to print a warning like "This is silly, you should just do a full clone to start. No worries though, we'll do the fetch ourselves" and just fix it ourselves, as you suggest.

The clever thing would be to do fetch --depth=5 for both origin main and HEAD. 5 might not be enough though, we're actually formatting based on the merge head. But you could do an doubling thing 5,10,20,40,80,etc. until you find the merge head. It's probably not worth the complexity. Happy to take a PR for either approach. The code in question is this:

spotless/lib-extra/src/main/java/com/diffplug/spotless/extra/GitRatchet.java

Lines 201 to 213 in df3ee6b

	ObjectId commitSha = repo.resolve(reference);
	if (commitSha == null) {
	throw new IllegalArgumentException("No such reference '" + reference + "'");
	}
	RevCommit ratchetFrom = revWalk.parseCommit(commitSha);
	RevCommit head = revWalk.parseCommit(repo.resolve(Constants.HEAD));
	revWalk.setRevFilter(RevFilter.MERGE_BASE);
	revWalk.markStart(ratchetFrom);
	revWalk.markStart(head);
	RevCommit mergeBase = revWalk.next();

[JLLeitschuh]

I spent ~ 10 min poking at the code and the JGit API. I really don't feel like I have the depth of knowledge here of how Git works to appropriately fix this issue.

The reason that I opened this PR was mostly to try to support this pull request #711. I actually don't need this for my own uses of spotless.

[nedtwigg]

Fair enough :). Can you fix #711 with fetch-depth: 0 (note that the link is to github actions documentation).

[JLLeitschuh]

Fair enough :). Can you fix #711 with fetch-depth: 0 (note that the link is to github actions documentation).

Done! Would you like to keep this issue open in case some enterprising individual comes along and wants to resolve this, or should we close it out?

[nedtwigg]

Let's keep it open, I still think it's a good idea.

[prudhvir3ddy]

😅 Guys, facing same and i have other issues to fix.

TBH ratchet from got no use if it doesn't work on CI ?

bcoz usually we check lints at ci.

or do we have something like baseline file options where that file can store current lint errors and only errors that's not their in the file is reported ?

[nedtwigg]

It works in CI, but it doesn't work on a bare or --depth=1 checkout. Most CI systems (for sure GitHub and GitLab) allow for full checkout, so that's the workaround. Implementing this is only a performance improvement, and only for some CI systems.

[Xendar]

I had the same issue with my CI (Jenkins) using Bitbucket for Git.
For multibranch pipelines Jenkins will fetch only the branch associated to the current build. So I added a manual fetch of my main branch in my pipeline to be sure it's there

[JLLeitschuh]

This sounds like an optimization that is implemented by-default into a majority of CI tools.

[nedtwigg]

is implemented by-default into a majority of CI tools.

I agree, but I still think it would still be nice if Spotless fixed it automatically. If the branch that Spotless needs isn't there, it can fetch it as well as anybody.

[driv]

I might be able to take a look.

What is the idea here then?

• The CI tool has to do a proper fetch with history. This is not going to be spotless responsibility.
• If the base branch is not present, a fetch is tried.

[nedtwigg]

Spotless is looking for the merge base. So it needs the history of ratchetFrom, and it also needs the history of HEAD, at least deep enough to find the intersection of those two things.

The idea is that if any history is missing, whether of HEAD or of ratchetFrom, it would be better for Spotless to just fix it with a fetch rather than fail, regardless of whether it is HEAD or ratchetFrom (or both) which is missing.

The place where code will need to be changed is described in #710 (comment)