There's a temptation to get caught up on quantifiable scores when working on SEO because so much of it can't be quantified, everyone is mostly guessing what Google wants... however, scores are ultimately meaningless and you should be measuring the impact on your business metrics. For example, if your website is designed to generate sales, did reducing your CWV to 800ms improve revenue? SEO is a means to an end, you can spend thousands of human hours to rank #1 for a keyword but if it doesn't improve a metric that matters (e.g: your revenue) it's a complete waste of time.
More broadly, prioritisation isn't about whether a piece of work will achieve its aim, it's about understanding which options are the best use of limited resources. For example, you may be able to say confidently that reducing CWV from 800ms to 600ms will increase your traffic from Google search by 5% but that's immaterial until it's compared against other options -- there's an opportunity cost associated with all work.
Personally, I would be surprised if reducing CWV from 800ms to 600ms is the best use of your resources, unless your business is one of the few that has a strong organic search strategy with organic search accounting for a meaningful volume of revenue -- nowadays, most companies find paid ads are much more effective.