Improve CI Efficiency

[akarki2005]

Proposed Changes

This change seeks to improve the average-case runtime of the CI config in two ways:

1. Run parallel jobs when resolving dependencies to reduce worst-case CI runtime.
2. Use partial dependency caching to reduce the probability of worst-case CI runs occuring (see https://circleci.com/docs/guides/optimize/caching-strategy/ for details).

Type of Change

(Write an X or a brief description next to the type or types that best describe your changes.)

Type	Applies?
🚨 Breaking change (fix or feature that would cause existing functionality to change)
✨ New feature (non-breaking change that adds functionality)
🐛 Bug fix (non-breaking change that fixes an issue)
🎨 User interface change (change to user interface; provide screenshots)
♻️ Refactoring (internal change to codebase, without changing functionality)
🚦 Test update (change that only adds or modifies tests)
📦 Dependency update (change that updates a dependency)
🔧 Internal (change that only affects developers or continuous integration)	X

Checklist

(Complete each of the following items for your pull request. Indicate that you have completed an item by changing the [ ] into a [x] in the raw text, or by clicking on the checkbox in the rendered description on GitHub.)

Before opening your pull request:

• I have performed a self-review of my changes.
  • Check that all changed files included in this pull request are intentional changes.
  • Check that all changes are relevant to the purpose of this pull request, as described above.
• [N/A] I have added tests for my changes, if applicable.
  • This is required for all bug fixes and new features.
• [N/A] I have updated the project documentation, if applicable.
  • This is required for new features.
• [N/A] If this is my first contribution, I have added myself to the list of contributors.
• I have updated the project Changelog (this is required for all changes).

After opening your pull request:

• I have verified that the CircleCI checks have passed.
• I have requested a review from a project maintainer.

[coveralls]

Pull Request Test Coverage Report for Build fdb21652-b67b-4323-bcf7-7fa63a2b819b

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 55.118%

---

Totals
Change from base Build 4f01acc7-ca46-45b8-bc50-95514d40c87a:	0.0%
Covered Lines:	2217
Relevant Lines:	3948

---

💛 - Coveralls

[akarki2005]

Hi David, in this PR I've added partial dependency caching as well as parallelism during the build process after a cache miss. However, I'm not sure how we can (or if we can) test these changes without actually merging the PR and seeing it work in production. Since the changes would require either a partial or full cache miss to test (which is not happening with this PR), I wanted to ask for your feedback before proceeding further.

Specifically, with my parallel jobs change, I looked here and saw that with our resource_class field set to large in our config, we can run at max 4 jobs concurrently, and so I was thinking to first increment the -j flag to 4, test that out, and decrement until we reach a value that does not exceed the 8GB RAM allowance provided with our CI config.

[david-yz-liu]

@akarki2005 thanks for doing this. I'm fine with merging in these changes for now, and then please do open a new PR to test these changes.