Just run the GitHub cache action on your build directory and then run make inside it?