There should be a k8s metric of 'cluster utilization'/'packing metric' the autoscaler from k8s should reschedule pods but rescheduling only exists for node drain, Priority and resource issues but not for packing a cluster.
You can only do this with Deschedule, which is not part of the k8s distri itself...
Thats it about my generic k8s rant (i do love k8s nonetheless).
Whats missing is an annotation on the specs to clarify how aggressive an autoscaler can handle certain pods (like do not move this stateful workload ever, reschedule this type of pod at night or look at this metric to see when its less active).
From an algorithm standpoint, its probably the same issue as java has with old and young generation: there are plenty of stable pods, which just need to run always. The base load, easy to pack, easy to keep aligned, seldom repackaking needed. Than you have the young generation: unclear how long the workload is required. Might mean that you have a big node running nearly empthy for an additional x hours just for the pod to finish its task or for the load to scale down again. If you know more about the type of workload, you could decide to spin up a big node or a lot more smaller nodes from a number of node pools. If you know that this job runs for 20 hours but is gone after, put it on a small node.
You could also do a strategy (depending on the cluster size) to only have one big node between 1-99% and make sure that all other nodes are always packed.
The project itself is tbh. shitty in describing how it works. I was neither able to read in the README or in the linked blogpost HOW they are doing it. The key information is missing. What algorithm does it use to utilize it?
I do love to have more autoscalers though.