Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

Conversation

@Zha0q1
Copy link
Contributor

@Zha0q1 Zha0q1 commented Sep 18, 2020

This pr forces static link to libopenblas.a on all pipelines to avoid the name clashing issue (except for windows where visual studio does not support static link to openblas according to this https://github.com/xianyi/OpenBLAS/wiki/How-to-use-OpenBLAS-in-Microsoft-Visual-Studio see bottom). On ubuntu and centos7 we changed from apt get/ yum openblas and lapack to building openblas(blas+lapack) from source. Also we add a flag to support LAPACKE api for non-mkl builds. The original description is below:

In mxnet we wrap lapack functions in c_lapack_api.h so that we hide the differences in the underlying blas/lapack libraries such as mkl, openblas, atlas, and accelerate.

For mkl we use/wrap the LAPACKE (https://www.netlib.org/lapack/lapacke.html) c interfaces which are the cleanest. For the rest of the libraries we wrap the old CLAPACK interfaces.

Openblas is by default built with lapack support and the binary will contain a full set of LAPACKE functions. Thus this poc adds a build option with which we can use the LAPACKE apis even for openblas.

This change will make ilp64 blas/lapack support easier as now we have the same wrapping logic or both mkl and openblas with USE_LAPACKE_INTERFACE = ON. Support for ilp64 mkl #19067 will automatically mean support for ilp64 openblas.

Known issue: linux distros ship openblas binaries WITHOUT lapack so this option can only work if we build openblas from source. This is not a problem for our mxnet binary distributions as we always use our own openblas build there. However, For users building mxnet from source they must be advised against using this option unless they also build openblas from source

@Zha0q1 Zha0q1 requested review from leezu and szha as code owners September 18, 2020 01:18
@mxnet-bot
Copy link

Hey @Zha0q1 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [unix-gpu, windows-cpu, unix-cpu, website, edge, centos-cpu, centos-gpu, clang, sanity, miscellaneous, windows-gpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@Zha0q1 Zha0q1 changed the title Make OpenBLAS use the LAPACKE apis [poc] Make OpenBLAS use the LAPACKE apis Sep 18, 2020
@Zha0q1 Zha0q1 changed the title [poc] Make OpenBLAS use the LAPACKE apis [poc] Add option for OpenBLAS to use the LAPACKE apis Sep 18, 2020
@lanking520 lanking520 added pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 9, 2020
@Zha0q1
Copy link
Contributor Author

Zha0q1 commented Nov 10, 2020

@mxnet-bot run ci [centos-cpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [centos-cpu]

@lanking520 lanking520 added pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels Nov 10, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-review PR is waiting for code review pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels Dec 4, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 4, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress pr-awaiting-review PR is waiting for code review and removed pr-awaiting-review PR is waiting for code review pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels Dec 4, 2020
Copy link
Contributor

@leezu leezu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for spending the effort to make the feature easy to use!

@leezu leezu merged commit b1b5021 into apache:master Dec 4, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

pr-awaiting-review PR is waiting for code review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants