The trick is knowing what the optimum settings are to use.. with h.265 as you lower the bitrate it smooths more and more and you lose detail. h.264 does blocking instead, so there is an image quality difference.
As for final bitrate, maybe we need to talk more about use case here. Because for very small encodes (often around 250kbps), I never cared about moment to moment bitrate, just the final file size. And if that's too far off I change the factor and run it again.
For things I intend to stream I usually have a bitrate limit on top of the CRF setting, but that's the only optional flag and it doesn't kick in very often. The result is quite high quality out of 2-3Mbps AV1, without any flags that affect the details of the video encoder, so I don't see a need for knowing and understanding optimum settings. And the same setup worked with h.264 at a moderately higher bitrate.
The best thing you can do for the encoder is give it time to work.