Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[FEATURE]Migrating all CD pipelines to Ninja build + fix cu112 CD pipeline#19974

Merged
Zha0q1 merged 19 commits intoapache:v1.xfrom
access2rohit:cd_ninja
Mar 12, 2021
Merged

[FEATURE]Migrating all CD pipelines to Ninja build + fix cu112 CD pipeline#19974
Zha0q1 merged 19 commits intoapache:v1.xfrom
access2rohit:cd_ninja

Conversation

@access2rohit
Copy link
Copy Markdown
Contributor

@access2rohit access2rohit commented Mar 3, 2021 •
edited
Loading

Description

Migrates all cd build to ninja from make
fixes CUDA11.2 pipeline for v1.x.
updates docker test images to ubuntu18 and fixes depedencies.
Fixes build issues with cmake
provides correct path for some make builds
Makes CD generated binaries compliant with Apache

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Testing

CD pipeline is working: https://jenkins.mxnet-ci.amazon-ml.com/job/restricted-mxnet-cd/job/mxnet-cd-pipeline-1.x-rohit/

@mxnet-bot
Copy link
Copy Markdown

mxnet-bot commented Mar 3, 2021

Hey @access2rohit , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [sanity, miscellaneous, windows-gpu, edge, centos-cpu, unix-gpu, centos-gpu, windows-cpu, unix-cpu, website, clang]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@access2rohit access2rohit changed the base branch from master to v1.x March 3, 2021 02:58
@access2rohit access2rohit changed the title Cd ninja Migrating all CD pipelines to Ninja build + fix cu112 pipeline Mar 3, 2021
@access2rohit access2rohit changed the title Migrating all CD pipelines to Ninja build + fix cu112 pipeline [DO NOT MERGE]Migrating all CD pipelines to Ninja build + fix cu112 pipeline Mar 3, 2021
@access2rohit access2rohit changed the title [DO NOT MERGE]Migrating all CD pipelines to Ninja build + fix cu112 pipeline [DO NOT MERGE][v1.x]Migrating all CD pipelines to Ninja build + fix cu112 pipeline Mar 3, 2021
@access2rohit access2rohit reopened this Mar 3, 2021
@access2rohit access2rohit changed the title [DO NOT MERGE][v1.x]Migrating all CD pipelines to Ninja build + fix cu112 pipeline [DO NOT MERGE][v1.x]Migrating all CD pipelines to Ninja build + fix cu112 CD pipeline Mar 3, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Mar 4, 2021
leezu
leezu reviewed Mar 4, 2021
View reviewed changes
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Mar 4, 2021
@access2rohit access2rohit force-pushed the cd_ninja branch 2 times, most recently from f323a76 to eb14659 Compare March 5, 2021 00:11
@lanking520 lanking520 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Mar 5, 2021
Zha0q1
Zha0q1 approved these changes Mar 10, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@Zha0q1 Zha0q1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thank you!

@access2rohit
Copy link
Copy Markdown
Contributor Author

access2rohit commented Mar 10, 2021

Latest CD successful build: https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/restricted-mxnet-cd%2Fmxnet-cd-release-job-1.x-rohit/detail/mxnet-cd-release-job-1.x-rohit/58/pipeline

@access2rohit access2rohit changed the title [DO NOT MERGE][WIP][FEATURE]Migrating all CD pipelines to Ninja build + fix cu112 CD pipeline [FEATURE]Migrating all CD pipelines to Ninja build + fix cu112 CD pipeline Mar 10, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Mar 11, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Mar 11, 2021
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Mar 11, 2021
@access2rohit
Copy link
Copy Markdown
Contributor Author

access2rohit commented Mar 11, 2021 •
edited
Loading

Apparently, unable to download MNIST dataset from http://yann.lecun.com/exdb/mnist/ so added backup URL todata.sh to stablize unix-gpu CI.

@lanking520 lanking520 added pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels Mar 11, 2021
@access2rohit
Copy link
Copy Markdown
Contributor Author

access2rohit commented Mar 11, 2021

@mxnet-bot run ci [unix-gpu]

@mxnet-bot
Copy link
Copy Markdown

mxnet-bot commented Mar 11, 2021

Jenkins CI successfully triggered : [unix-gpu]

@access2rohit
Copy link
Copy Markdown
Contributor Author

access2rohit commented Mar 11, 2021 •
edited
Loading

Will disable flaky tests #20011 in this PR too if the current CI run fails.

Zha0q1
Zha0q1 approved these changes Mar 12, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@Zha0q1 Zha0q1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

@access2rohit access2rohit mentioned this pull request Mar 12, 2021
Merged
4 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@aaronmarkham aaronmarkham Awaiting requested review from aaronmarkham aaronmarkham is a code owner

@eric-haibin-lin eric-haibin-lin Awaiting requested review from eric-haibin-lin

@gigasquid gigasquid Awaiting requested review from gigasquid

@iblislin iblislin Awaiting requested review from iblislin

@marcoabreu marcoabreu Awaiting requested review from marcoabreu marcoabreu is a code owner

@nswamy nswamy Awaiting requested review from nswamy nswamy is a code owner

@sergeykolychev sergeykolychev Awaiting requested review from sergeykolychev

@szha szha Awaiting requested review from szha szha is a code owner

@thirdwing thirdwing Awaiting requested review from thirdwing

@yzhliu yzhliu Awaiting requested review from yzhliu

2 more reviewers

@leezu leezu leezu approved these changes

@Zha0q1 Zha0q1 Zha0q1 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

pr-awaiting-review PR is waiting for code review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@access2rohit @mxnet-bot @leezu @Zha0q1 @lanking520