The way jj tracks changes isn't a feature you can turn off AFAIK. Maybe you can, but if you can, you shouldn't. This is fundamental to the jj internals and workflow.

As someone who also kept a lot of files in their repo that went untracked, it was a small adjustment. Add to my .gitignore, untrack the files, and done.