charts/gundeck: add prestop hook

[jschaul]

We currently get a few 500s sometime when gundeck restarts, where some current requests seem to get aborted mid-request. This is possibly due to terminating pods still getting some traffic.

https://wearezeta.atlassian.net/browse/WPB-19694

Checklist

• Add a new entry in an appropriate subdirectory of changelog.d
• Read and follow the PR guidelines

[supersven]

Unfortunately, I'm not an K8s expert: Could you please explain why we need the sleep command? Why isn't it good enough to only increase terminationGracePeriodSeconds? 🤔

[lwille]

the default terminationGracePeriodSeconds is 30 seconds, which is plenty.

What this sleep would do is delaying the time between the pod entering the Terminating state, and thus being removed from any Service that sends traffic there.
However, gundeck will still accept any in-flight requests and process them, since it doesn't know about being terminated.

Shamelessly adapted from SO to visualize and explain the process:

The full sequence is :

1. pod deletion is requested (state: Terminating)
2. preStop hook kicks in and terminationGracePeriodSeconds countdown starts :
   • when preStop hook completes, kubelet sends a SIGTERM to the container
   • if preStop hook isn't finished within terminationGracePeriodSeconds countdown, kubelet sends SIGKILL to the container

→ this means:

• we delay the shutdown (SIGTERM) to gundeck to allow for the k8s API to remove it from its Service and process
• once the 10s counter is finished, the regular SIGTERM signal is sent, causing gundeck to gracefully shutdown
• if the whole thing takes longer than terminationGracePeriodSeconds, it's simply killed.