Skip to content

ci: Use clang-8 for fuzzing to run on aarch64 ci systems - #17470

Merged
laanwj merged 2 commits into
bitcoin:masterfrom
maflcko:1911-ciFuzzClang8
Nov 18, 2019
Merged

ci: Use clang-8 for fuzzing to run on aarch64 ci systems#17470
laanwj merged 2 commits into
bitcoin:masterfrom
maflcko:1911-ciFuzzClang8

Conversation

@maflcko

@maflcko maflcko commented Nov 13, 2019

Copy link
Copy Markdown
Member

Ubuntu bionic clang is clang version 6, which does not come with libfuzzer. So the ci system breaks down when run on aarch64.

Fix that by using clang-8

For reference, the previous error on my ci system was:

/usr/bin/ld: cannot find /usr/lib/llvm-6.0/lib/clang/6.0.0/lib/linux/libclang_rt.fuzzer-aarch64.a: No such file or directory

@maflcko maflcko added the Tests label Nov 13, 2019
@maflcko

maflcko commented Nov 13, 2019

Copy link
Copy Markdown
Member Author

Excerpt from my ci system:

Options used to compile and link:
  with wallet   = no
  with gui / qt = no
  with zmq      = no
  with test     = yes
    with prop   = no
    with fuzz   = yes
  with bench    = no
  with upnp     = no
  use asm       = yes
  sanitizers    = fuzzer,address,undefined
  debug enabled = no
  gprof enabled = no
  werror        = no

  target os     = linux
  build os      = 

  CC            = /usr/bin/ccache clang-8
  CFLAGS        = -g -O2
  CPPFLAGS      =   -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments  -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS
  CXX           = /usr/bin/ccache clang++-8 -std=c++11
  CXXFLAGS      =   -Wstack-protector -fstack-protector-all  -Wall -Wextra -Wformat -Wvla -Wswitch -Wformat-security -Wthread-safety-analysis -Wrange-loop-analysis -Wredundant-decls  -Wno-unused-parameter -Wno-self-assign -Wno-unused-local-typedef -Wno-deprecated-register -Wno-implicit-fallthrough   -g -O2
  LDFLAGS       = -pthread  -Wl,-z,relro -Wl,-z,now -pie  
  ARFLAGS       = cr

travis_fold:end:configure

travis_fold:start:build
Making install in src
make[1]: Entering directory '/root/workspace/bitcoin-core/build/bitcoin-aarch64-unknown-linux-gnu/src'
make[2]: Entering directory '/root/workspace/bitcoin-core/build/bitcoin-aarch64-unknown-linux-gnu/src'
  CXX      test/fuzz/test_fuzz_address_deserialize-fuzz.o
  CXX      test/util/test_fuzz_address_deserialize-setup_common.o

@practicalswift

Copy link
Copy Markdown
Contributor

ACK fa6e01b -- diff looks correct

@fanquake fanquake left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK fa6e01b

@maflcko

maflcko commented Nov 14, 2019

Copy link
Copy Markdown
Member Author

Bumped the timeout in a new commit, because it turns out that just running over the existing seeds takes 5 minutes for me:

root@odroid:~/workspace/bitcoin-core/build/bitcoin-aarch64-unknown-linux-gnu# time ./test/fuzz/test_runner.py ../../qa-assets/fuzz_seed_corpus/
Fuzz targets found: ['address_deserialize', 'addrman_deserialize', 'banentry_deserialize', 'bech32', 'block_deserialize', 'blockheader_deserialize', 'blocklocator_deserialize', 'blockmerkleroot', 'blocktransactions_deserialize', 'blocktransactionsrequest_deserialize', 'blockundo_deserialize', 'bloomfilter_deserialize', 'coins_deserialize', 'descriptor_parse', 'diskblockindex_deserialize', 'eval_script', 'inv_deserialize', 'messageheader_deserialize', 'netaddr_deserialize', 'parse_iso8601', 'script', 'script_flags', 'service_deserialize', 'spanparsing', 'transaction', 'txoutcompressor_deserialize', 'txundo_deserialize']
Fuzz targets selected: ['blocktransactions_deserialize', 'bloomfilter_deserialize', 'parse_iso8601', 'txoutcompressor_deserialize', 'blocklocator_deserialize', 'diskblockindex_deserialize', 'service_deserialize', 'banentry_deserialize', 'address_deserialize', 'inv_deserialize', 'script_flags', 'coins_deserialize', 'descriptor_parse', 'blockheader_deserialize', 'block_deserialize', 'addrman_deserialize', 'messageheader_deserialize', 'blocktransactionsrequest_deserialize', 'txundo_deserialize', 'blockmerkleroot', 'eval_script', 'bech32', 'blockundo_deserialize', 'script', 'netaddr_deserialize', 'spanparsing', 'transaction']

real	5m5.762s
user	4m9.200s
sys	0m8.390s

@laanwj

laanwj commented Nov 18, 2019

Copy link
Copy Markdown
Member

ACK fa2ec9f

laanwj added a commit that referenced this pull request Nov 18, 2019
fa2ec9f fuzz: Bump timeout in test_runner to accomodate for slow arm64 CPUs (MarcoFalke)
fa6e01b ci: Use clang-8 for fuzzing to run on aarch64 ci systems (MarcoFalke)

Pull request description:

  Ubuntu bionic clang is clang version 6, which does not come with libfuzzer. So the ci system breaks down when run on aarch64.

  Fix that by using clang-8

  For reference, the previous error on my ci system was:

  ```
  /usr/bin/ld: cannot find /usr/lib/llvm-6.0/lib/clang/6.0.0/lib/linux/libclang_rt.fuzzer-aarch64.a: No such file or directory

ACKs for top commit:
  laanwj:
    ACK fa2ec9f

Tree-SHA512: 4954dbc36c444d1ae145290115eea6291753c9810c92003ab8d75433c3fe3bfee439d3a99dc394418275527157a8b89f04038c8b16e08c69ec9ded50fb869e70
@laanwj
laanwj merged commit fa2ec9f into bitcoin:master Nov 18, 2019
@maflcko
maflcko deleted the 1911-ciFuzzClang8 branch November 18, 2019 13:38
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Oct 27, 2020
Summary:
Backport of core [[bitcoin/bitcoin#17470 | PR17470]].

Only this commit is relevant:
bitcoin/bitcoin@fa2ec9f

Test Plan:
  ninja bitcoin-fuzzers
  ./test/fuzz/test_runner.py <path_to_corpus>

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D8139
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants