fix: duplicated keys in annotations caused by airfowPodAnnotations value being overwritten by safeToEvict value of worker

[rzjfr]

as a result of f9db9c9952 commit, value of .Values.airflowPodAnnotations would be changed for all the other services, this would result in the services except workers, to have duplicate cluster-autoscaler.kubernetes.io/safe-to-evict as the first one would point to the safeToEvict value of the service itself and the second to .Values.airflowPodAnnotations which is overwritten by the pod template.

closes: #40553

---

[hussein-awala]

Could you add a test that fails without your fix?

[rzjfr]

Could you add a test that fails without your fix?

Hi @hussein-awala,
Thank you for reviewing the PR,

It's bit tricky to write a test to catch the key duplication as render_chart returns a dictionary. But since the in the order of annotations, .Values.airflowPodAnnotations comes after for example .Values.dagProcessor.safeToEvict (here) then the last one which is worker's safeToEvict value overwrites service's own safeToEvict .

So that is the only valid test which comes to my mind. It fails without the change in this PR.

I just added that in this commit, Please let me know if you had a better proposal.

[rzjfr]

I realized probably I should add bit of an explanation on the solution as well.

merge or mergeOverwrite actually combine the given parameters from right to left into the left most parameter. So what is actually happening is that pod-template is changing .Values.airflowPodAnnotations and since .Values.airflowPodAnnotations is being used in other services like scheduler, triggerer and dag-processor it causes issue. Since .Values.airflowPodAnnotations is being added separately after services' own safeToEvict annotation the result would have the safe to evict annotation at least twice. Actually from the two the one that matters most (the second one) is going to be the incorrect one (only if for example .Values.triggerer.podAnnotations is not set) as .Values.airflowPodAnnotations evaluated after services' own safe to evict annotation and .Values.airflowPodAnnotations would have workers' safe to evict value because pod-template has set it.

For example here you can see value of a (the left most variable) in the output.

in pod-template if we add a copy of Values.airflowPodAnnotations instead, the issue would be resolved.