Cleanup libgit2 build

[phatblat]

Fixing the libgit2 iOS build and removing unused defines.

• Correct spelling of LIBSSH2_INCLUDE_DIRS (fixes SSH functionality is disabled #481)
• Disable CURL on iOS
• Remove obsolete OPENSSL_* defines (libgit2 now uses SecureTransport on Apple platforms)
• Add tests for git_libgit2_features flags

[pietbrauer]

So we don't need OpenSSL no more? Then we could as well get rid of the target and enable Travis again because OpenSSL uses 15 of the 20 minutes build time.

[phatblat]

libssh2 still needs OpenSSL, it's just libgit2 that has dropped it as a dependency. So the iOS build still has to deal with that huge compile time. I hope libssh2 drops it too.

[pietbrauer]

I will test and review this.

[pietbrauer]

Tests are green, cloning is working and carthage build --no-skip-current also produces a valid archive 👍

[phatblat]

💚

[pietbrauer]

@phatblat @joshaber What do you guys think of a new release? I could volunteer to do it if @joshaber no one else finds time.

[joshaber]

Awesome work 👍 I'll cut a new release now.

[joshaber]

The Mac tests currently failing:

expect(@(git_features & GIT_FEATURE_SSH)).to(beTruthy());

[pietbrauer]

Stupid question: Did you remove External/libgit2.a so that it rebuilds it?

On 10 Jul 2015, at 15:52, Josh Abernathy notifications@github.com wrote:

The Mac tests currently failing:

expect(@(git_features & GIT_FEATURE_SSH)).to(beTruthy());
—
Reply to this email directly or view it on GitHub #483 (comment).

[phatblat]

Yep. That file's presence causes update_libgit2 script to skip building.

Sent from my giant iPhone

On Jul 10, 2015, at 7:55 AM, Piet Brauer notifications@github.com wrote:

Stupid question: Did you remove External/libgit2.a so that it rebuilds it?

On 10 Jul 2015, at 15:52, Josh Abernathy notifications@github.com wrote:

The Mac tests currently failing:

expect(@(git_features & GIT_FEATURE_SSH)).to(beTruthy());
—
Reply to this email directly or view it on GitHub #483 (comment).

—
Reply to this email directly or view it on GitHub.

[joshaber]

Yup, removed and rebuilt but it's still failing.

[pietbrauer]

I just cloned again and they are still green Xcode 6.4 on 10.10.4 😢

[phatblat]

@joshaber is your brew --prefix set to /opt/boxen? I mistakenly hard-coded the path to libssh2.h

[joshaber]

Nah, it's /usr/local. I'm looking into this. Not sure what's happening 😦

[phatblat]

Try removing LIBSSH2_INCLUDE_DIRS from update_libgit2. That may not be necessary in the Mac build.

[joshaber]

No joy unfortunately.

[phatblat]

Also try blowing away External/libgit2/build as the CMakeCache.txt file caches the arguments passed to cmake.

[joshaber]

Even if I give CMake the explicit path to libssh2, it can't seem to find it:

  -DCMAKE_INCLUDE_PATH:PATH=/usr/local/Cellar/libssh2/1.6.0/include/ \
  -DCMAKE_LIBRARY_PATH:PATH=/usr/local/Cellar/libssh2/1.6.0/lib/ \

-- LIBSSH2 not found. Set CMAKE_PREFIX_PATH if it is installed outside of the default search path.

😢

[joshaber]

A fresh clone also fails.

Where do you all have libssh2 installed?

[pietbrauer]

Via brew:

$ brew info libssh2
libssh2: stable 1.6.0 (bottled), HEAD
C library implementing the SSH2 protocol
http://www.libssh2.org/
/usr/local/Cellar/libssh2/1.6.0 (178 files, 1.3M) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/libssh2.rb
==> Dependencies
Recommended: openssl ✔
Optional: libressl ✘
==> Options
--with-libressl
    build with LibreSSL instead of OpenSSL
--without-openssl
    Build without openssl support
--HEAD
    Install HEAD version

./script/update_libgit2 output:

$ ./script/update_libgit2
-- The C compiler identification is AppleClang 6.1.0.6020053
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.28") 
-- Found Security /System/Library/Frameworks/Security.framework
-- Found CoreFoundation /System/Library/Frameworks/CoreFoundation.framework
-- checking for module 'libcurl'
--   found libcurl, version 7.30.0
-- Could NOT find HTTP_Parser (missing:  HTTP_PARSER_INCLUDE_DIR HTTP_PARSER_LIBRARY) 
-- http-parser was not found or is too old; using bundled 3rd-party sources.
-- Found ZLIB: /usr/lib/libz.dylib (found version "1.2.5") 
-- checking for module 'libssh2'
--   found libssh2, version 1.6.0
-- Looking for libssh2_userauth_publickey_frommemory in ssh2
-- Looking for libssh2_userauth_publickey_frommemory in ssh2 - found
-- Found Iconv: -L/usr/lib -liconv
-- Performing Test IS_WNO-MISSING-FIELD-INITIALIZERS_SUPPORTED
-- Performing Test IS_WNO-MISSING-FIELD-INITIALIZERS_SUPPORTED - Success
-- Performing Test IS_WSTRICT-ALIASING=2_SUPPORTED
-- Performing Test IS_WSTRICT-ALIASING=2_SUPPORTED - Success
-- Performing Test IS_WSTRICT-PROTOTYPES_SUPPORTED
-- Performing Test IS_WSTRICT-PROTOTYPES_SUPPORTED - Success
-- Performing Test IS_WDECLARATION-AFTER-STATEMENT_SUPPORTED
-- Performing Test IS_WDECLARATION-AFTER-STATEMENT_SUPPORTED - Success
-- Performing Test IS_WNO-UNUSED-CONST-VARIABLE_SUPPORTED
-- Performing Test IS_WNO-UNUSED-CONST-VARIABLE_SUPPORTED - Success
-- Performing Test IS_WNO-UNUSED-FUNCTION_SUPPORTED
-- Performing Test IS_WNO-UNUSED-FUNCTION_SUPPORTED - Success
-- Performing Test IS_WNO-DEPRECATED-DECLARATIONS_SUPPORTED
-- Performing Test IS_WNO-DEPRECATED-DECLARATIONS_SUPPORTED - Success
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - not found
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/pietbrauer/Documents/Projects/objective-git/External/libgit2/build
Scanning dependencies of target git2
[  0%] Building C object CMakeFiles/git2.dir/src/annotated_commit.c.o
[  1%] Building C object CMakeFiles/git2.dir/src/attr.c.o
[  2%] Building C object CMakeFiles/git2.dir/src/attr_file.c.o
[  3%] Building C object CMakeFiles/git2.dir/src/attrcache.c.o
[  3%] Building C object CMakeFiles/git2.dir/src/blame.c.o
[  4%] Building C object CMakeFiles/git2.dir/src/blame_git.c.o
[  5%] Building C object CMakeFiles/git2.dir/src/blob.c.o
[  6%] Building C object CMakeFiles/git2.dir/src/branch.c.o
[  6%] Building C object CMakeFiles/git2.dir/src/buf_text.c.o
[  7%] Building C object CMakeFiles/git2.dir/src/buffer.c.o
[  8%] Building C object CMakeFiles/git2.dir/src/cache.c.o
[  9%] Building C object CMakeFiles/git2.dir/src/checkout.c.o
[ 10%] Building C object CMakeFiles/git2.dir/src/cherrypick.c.o
[ 10%] Building C object CMakeFiles/git2.dir/src/clone.c.o
[ 11%] Building C object CMakeFiles/git2.dir/src/commit.c.o
[ 12%] Building C object CMakeFiles/git2.dir/src/commit_list.c.o
[ 13%] Building C object CMakeFiles/git2.dir/src/config.c.o
[ 13%] Building C object CMakeFiles/git2.dir/src/config_cache.c.o
[ 14%] Building C object CMakeFiles/git2.dir/src/config_file.c.o
[ 15%] Building C object CMakeFiles/git2.dir/src/crlf.c.o
[ 16%] Building C object CMakeFiles/git2.dir/src/curl_stream.c.o
[ 17%] Building C object CMakeFiles/git2.dir/src/date.c.o
[ 17%] Building C object CMakeFiles/git2.dir/src/delta-apply.c.o
[ 18%] Building C object CMakeFiles/git2.dir/src/delta.c.o
[ 19%] Building C object CMakeFiles/git2.dir/src/describe.c.o
[ 20%] Building C object CMakeFiles/git2.dir/src/diff.c.o
[ 20%] Building C object CMakeFiles/git2.dir/src/diff_driver.c.o
[ 21%] Building C object CMakeFiles/git2.dir/src/diff_file.c.o
[ 22%] Building C object CMakeFiles/git2.dir/src/diff_patch.c.o
[ 23%] Building C object CMakeFiles/git2.dir/src/diff_print.c.o
[ 24%] Building C object CMakeFiles/git2.dir/src/diff_stats.c.o
[ 24%] Building C object CMakeFiles/git2.dir/src/diff_tform.c.o
[ 25%] Building C object CMakeFiles/git2.dir/src/diff_xdiff.c.o
[ 26%] Building C object CMakeFiles/git2.dir/src/errors.c.o
[ 27%] Building C object CMakeFiles/git2.dir/src/fetch.c.o
[ 27%] Building C object CMakeFiles/git2.dir/src/fetchhead.c.o
[ 28%] Building C object CMakeFiles/git2.dir/src/filebuf.c.o
[ 29%] Building C object CMakeFiles/git2.dir/src/fileops.c.o
[ 30%] Building C object CMakeFiles/git2.dir/src/filter.c.o
[ 31%] Building C object CMakeFiles/git2.dir/src/fnmatch.c.o
[ 31%] Building C object CMakeFiles/git2.dir/src/global.c.o
[ 32%] Building C object CMakeFiles/git2.dir/src/graph.c.o
[ 33%] Building C object CMakeFiles/git2.dir/src/hash.c.o
[ 34%] Building C object CMakeFiles/git2.dir/src/hashsig.c.o
[ 34%] Building C object CMakeFiles/git2.dir/src/ident.c.o
[ 35%] Building C object CMakeFiles/git2.dir/src/ignore.c.o
[ 36%] Building C object CMakeFiles/git2.dir/src/index.c.o
[ 37%] Building C object CMakeFiles/git2.dir/src/indexer.c.o
[ 37%] Building C object CMakeFiles/git2.dir/src/iterator.c.o
[ 38%] Building C object CMakeFiles/git2.dir/src/merge.c.o
[ 39%] Building C object CMakeFiles/git2.dir/src/merge_file.c.o
[ 40%] Building C object CMakeFiles/git2.dir/src/message.c.o
[ 41%] Building C object CMakeFiles/git2.dir/src/mwindow.c.o
[ 41%] Building C object CMakeFiles/git2.dir/src/netops.c.o
[ 42%] Building C object CMakeFiles/git2.dir/src/notes.c.o
[ 43%] Building C object CMakeFiles/git2.dir/src/object.c.o
[ 44%] Building C object CMakeFiles/git2.dir/src/object_api.c.o
[ 44%] Building C object CMakeFiles/git2.dir/src/odb.c.o
[ 45%] Building C object CMakeFiles/git2.dir/src/odb_loose.c.o
[ 46%] Building C object CMakeFiles/git2.dir/src/odb_mempack.c.o
[ 47%] Building C object CMakeFiles/git2.dir/src/odb_pack.c.o
[ 48%] Building C object CMakeFiles/git2.dir/src/oid.c.o
[ 48%] Building C object CMakeFiles/git2.dir/src/oidarray.c.o
[ 49%] Building C object CMakeFiles/git2.dir/src/openssl_stream.c.o
[ 50%] Building C object CMakeFiles/git2.dir/src/pack-objects.c.o
[ 51%] Building C object CMakeFiles/git2.dir/src/pack.c.o
[ 51%] Building C object CMakeFiles/git2.dir/src/path.c.o
[ 52%] Building C object CMakeFiles/git2.dir/src/pathspec.c.o
[ 53%] Building C object CMakeFiles/git2.dir/src/pool.c.o
[ 54%] Building C object CMakeFiles/git2.dir/src/posix.c.o
[ 55%] Building C object CMakeFiles/git2.dir/src/pqueue.c.o
[ 55%] Building C object CMakeFiles/git2.dir/src/push.c.o
[ 56%] Building C object CMakeFiles/git2.dir/src/rebase.c.o
[ 57%] Building C object CMakeFiles/git2.dir/src/refdb.c.o
[ 58%] Building C object CMakeFiles/git2.dir/src/refdb_fs.c.o
[ 58%] Building C object CMakeFiles/git2.dir/src/reflog.c.o
[ 59%] Building C object CMakeFiles/git2.dir/src/refs.c.o
[ 60%] Building C object CMakeFiles/git2.dir/src/refspec.c.o
[ 61%] Building C object CMakeFiles/git2.dir/src/remote.c.o
[ 62%] Building C object CMakeFiles/git2.dir/src/repository.c.o
[ 62%] Building C object CMakeFiles/git2.dir/src/reset.c.o
[ 63%] Building C object CMakeFiles/git2.dir/src/revert.c.o
[ 64%] Building C object CMakeFiles/git2.dir/src/revparse.c.o
[ 65%] Building C object CMakeFiles/git2.dir/src/revwalk.c.o
[ 65%] Building C object CMakeFiles/git2.dir/src/settings.c.o
[ 66%] Building C object CMakeFiles/git2.dir/src/sha1_lookup.c.o
[ 67%] Building C object CMakeFiles/git2.dir/src/signature.c.o
[ 68%] Building C object CMakeFiles/git2.dir/src/socket_stream.c.o
[ 68%] Building C object CMakeFiles/git2.dir/src/sortedcache.c.o
[ 69%] Building C object CMakeFiles/git2.dir/src/stash.c.o
[ 70%] Building C object CMakeFiles/git2.dir/src/status.c.o
[ 71%] Building C object CMakeFiles/git2.dir/src/stransport_stream.c.o
[ 72%] Building C object CMakeFiles/git2.dir/src/strmap.c.o
[ 72%] Building C object CMakeFiles/git2.dir/src/submodule.c.o
[ 73%] Building C object CMakeFiles/git2.dir/src/sysdir.c.o
[ 74%] Building C object CMakeFiles/git2.dir/src/tag.c.o
[ 75%] Building C object CMakeFiles/git2.dir/src/thread-utils.c.o
[ 75%] Building C object CMakeFiles/git2.dir/src/tls_stream.c.o
[ 76%] Building C object CMakeFiles/git2.dir/src/trace.c.o
[ 77%] Building C object CMakeFiles/git2.dir/src/transaction.c.o
[ 78%] Building C object CMakeFiles/git2.dir/src/transport.c.o
[ 79%] Building C object CMakeFiles/git2.dir/src/tree-cache.c.o
[ 79%] Building C object CMakeFiles/git2.dir/src/tree.c.o
[ 80%] Building C object CMakeFiles/git2.dir/src/tsort.c.o
[ 81%] Building C object CMakeFiles/git2.dir/src/util.c.o
[ 82%] Building C object CMakeFiles/git2.dir/src/vector.c.o
[ 82%] Building C object CMakeFiles/git2.dir/src/zstream.c.o
[ 83%] Building C object CMakeFiles/git2.dir/src/transports/auth.c.o
[ 84%] Building C object CMakeFiles/git2.dir/src/transports/auth_negotiate.c.o
[ 85%] Building C object CMakeFiles/git2.dir/src/transports/cred.c.o
[ 86%] Building C object CMakeFiles/git2.dir/src/transports/cred_helpers.c.o
[ 86%] Building C object CMakeFiles/git2.dir/src/transports/git.c.o
[ 87%] Building C object CMakeFiles/git2.dir/src/transports/http.c.o
[ 88%] Building C object CMakeFiles/git2.dir/src/transports/local.c.o
[ 89%] Building C object CMakeFiles/git2.dir/src/transports/smart.c.o
[ 89%] Building C object CMakeFiles/git2.dir/src/transports/smart_pkt.c.o
[ 90%] Building C object CMakeFiles/git2.dir/src/transports/smart_protocol.c.o
[ 91%] Building C object CMakeFiles/git2.dir/src/transports/ssh.c.o
[ 92%] Building C object CMakeFiles/git2.dir/src/transports/winhttp.c.o
[ 93%] Building C object CMakeFiles/git2.dir/src/xdiff/xdiffi.c.o
[ 93%] Building C object CMakeFiles/git2.dir/src/xdiff/xemit.c.o
[ 94%] Building C object CMakeFiles/git2.dir/src/xdiff/xhistogram.c.o
[ 95%] Building C object CMakeFiles/git2.dir/src/xdiff/xmerge.c.o
[ 96%] Building C object CMakeFiles/git2.dir/src/xdiff/xpatience.c.o
[ 96%] Building C object CMakeFiles/git2.dir/src/xdiff/xprepare.c.o
[ 97%] Building C object CMakeFiles/git2.dir/src/xdiff/xutils.c.o
[ 98%] Building C object CMakeFiles/git2.dir/src/unix/map.c.o
[ 99%] Building C object CMakeFiles/git2.dir/src/unix/realpath.c.o
[100%] Building C object CMakeFiles/git2.dir/deps/http-parser/http_parser.c.o
Linking C static library libgit2.a
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libgit2.a(auth_negotiate.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libgit2.a(winhttp.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libgit2.a(auth_negotiate.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libgit2.a(winhttp.c.o) has no symbols
[100%] Built target git2
libgit2.a -> ../../libgit2.a
libgit2 has been updated.

[phatblat]

Trying on another Mac and I'm now able to reproduce the SSH feature test failure. Odd thing is that Xcode (6.4), OS X (10.10.4), libssh2 (1.6.0) and brew --prefix all match between the two Macs. Only discrepancy I've found so far is the cmake version: 3.2.3 on Mac without issue, 3.2.2 on Mac with failing test.

[joshaber]

Aha! I didn't have pkg-config. With that installed it seems to work 🤘

I wonder if we should add that to REQUIRED_TOOLS in script/bootstrap.

[phatblat]

Found the cause of the failures on my other Mac. I used to use Boxen on it and both libssh2 and cmake were still installed from the old homebrew location.

$ ll `which cmake`
lrwxr-xr-x  1 ben  staff    29B Feb 10 12:16 /usr/local/bin/cmake -> /opt/boxen/homebrew/bin/cmake

$ brew doctor

Warning: You have unlinked kegs in your Cellar
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:

    cmake
    libssh2

Re-linking these from /usr/local resolved my issues (I already had pkg-config installed) and now the Mac tests are all green.

[joshaber]

🆒

I'm in the process of making the necessary changes to our internal CI. Will TravisCI support this?

[phatblat]

Looks like it. The TravisCI VMs have homebrew installed

[pietbrauer]

Now that we've found the root cause we could build a new version \o/

[joshaber]

Just tagged a new release: https://github.com/libgit2/objective-git/releases/tag/0.8.6

Thanks all!