That is part of the problem; if you have 4 cores, your program should be using 4 OS-threads. Your programming language's runtime should take care of distributing your 4000 lightweight/green threads to the 4 actual threads.
This is what e.g. Haskell does, and Go as well, I think.