The new best practice is to use the RUN --mount cache options. Making the removal of intermediate files unnecessary and speeds up the builds too. Surprised to see so few mentions of it.
Externally managed caches don't have a lifecycle controlled or invalidated by changes in Dockerfiles, or by docker cache-purging commands like "system prune".
That means you have to keep track of those caches yourself, which can be a pain in complex, multi-contributor environments with many layers and many builds using the same caches (intentionally or by mistake).
Example: in the last few years docker compose files have gone from version 2 to version 3 (missing tons of great v2 features in the name of simplification) to the newest, unnamed unnumbered version which is a merging of versions 2 and 3.