Not if you use CLISP! It seems like bytecode vs. native compilation is a large contributor to memory usage (Racket uses Chez Scheme which compiles to native; same story for Node/Deno and V8).
I have a low traffic web service and I wasn't happy with ~125 MB of my 512 MB VM's memory being used up by SBCL, but I was able to just drop in CLISP and memory usage dropped to ~25 MB (and for reference: a "hello world" is under 10 MB with CLISP, similar to Python).