[WIP] Newton accelaration and stabilization for adjoint runs (including multiphysics)

[oleburghardt]

Proposed Changes

Similar to the quasi-Newton acceleration provided by CQuasiNewtonInvLeastSquares, the proposed toolbox class CNewtonUpdateOnSubspace performs a correction on adjoint solution updates by keeping track of past iterates. Here, they are used to identify eigenvectors with large eigenvalues which slow down the overall convergence and to form a (small) subspace basis from them. Each fixed-point solution update is then projected onto that subspace and the projected part will be additionally corrected via a "real" Newton step (a.k.a. "RPM") using the computational graph for precise derivative information.

This preliminary result with one basis vector indicates that the procedure can be effective at very low additional cost (for n basis vectors, it mainly reduces to n scalar products of the current fixed-point iterate with the basis vectors, the Newton step is cheap as n is small). The residuals are obtained from a fully coupled adjoint CHT problem (2d heated cylinder in laminar flow) with a heatflux objective function.

(Purple: No correction applied, Green/Blue: Usage of 1-dim. basis after extraction of one basis vector at around 500 iterations, Yellow: MPI run with two processes using the manual but MPI-compatible Gram-Schmidt-QR-decomposition for identification of basis vectors.)

The code is WIP. It needs a lot of additions, changes and testing, though the overall structure should be clear from what's already implemented.

ToDo's:

• Remove Eigen dependencies, not needed anymore
• Add a meaningful testcase
• Add eigenvalue estimation / QR criterion values to screen and history output fields
• Implement a basisMaintenance()-routine
• Figure out a proper base class for both CQuasiNewtonInvLeastSquares and CNewtonUpdateOnSubspace
• Figure out further algorithmic strategies for multiphysics problems, now the default is to create the basis in the outer loop, and to apply it for corrections in the inner

PR Checklist

Put an X by all that apply. You can fill this out after submitting the PR. If you have any questions, don't hesitate to ask! We want to help. These are a guide for you to know what the reviewers will be looking for in your contribution.

• I am submitting my contribution to the develop branch.
• My contribution generates no new compiler warnings (try with the '-Wall -Wextra -Wno-unused-parameter -Wno-empty-body' compiler flags, or simply --warnlevel=2 when using meson).
• My contribution is commented and consistent with SU2 style.
• I have added a test case that demonstrates my contribution, if necessary.
• I have updated appropriate documentation (Tutorials, Docs Page, config_template.cpp) , if necessary.

[pcarruscag]

Is Stroff the reference for this? Or are there modifications to the original RPM?

[oleburghardt]

It is very close to the original paper by Shroff. They didn't have access to AD derivatives of course, but apart from that, it's an implementation of the RPM algorithm proposed there (forgot to mention "RPM" above, thanks for the hint).

As a next step, I'd like to try out how keeping some "slow" basis vectors across outer loops works out, so that the whole multiphysics thing will hopefully have any gain from identifying them. (Doing it for inner iterations won't help as much, as it takes some iterations before a proper subspace can be formed.)

[pcarruscag]

👍 I recently found out I should have named the other one "Anderson Acceleration"... 1965 such a hipster method.

[oleburghardt]

:O Great. There's even a Wiki article on that.. the chapter on relations to other classes of methods is quite interesting, too. Potentially this becomes a corrector toolbox family soon :)

[lgtm-com]

This pull request introduces 1 alert when merging 418aa7a into 295485c - view on LGTM.com

new alerts:

• 1 for Expression has no effect

[lgtm-com]

This pull request introduces 1 alert when merging 56f4893 into 295485c - view on LGTM.com

new alerts:

• 1 for Expression has no effect

[oleburghardt]

@pcarruscag Coming back to our discussion a while ago, it should do MPI now (at least for my testcases it does). I'll clean it up and probably remove Eigen completely.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is still a relevant issue please comment on it to restart the discussion. Thank you for your contributions.

[oleburghardt]

WIP

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is still a relevant issue please comment on it to restart the discussion. Thank you for your contributions.