People get hung up on this, because the nonce looks like it could serve as a key; if you keep the key hidden, an attacker can't brute-force your hash at all. The obvious response to that is: if you can do keep your nonce secret like that, just get rid of the passwords, and key your system with actual keys. Or, store your passwords in the super-secure place you store the nonces.
These discussions quickly rabbithole into analyses of the varying levels of security between filesystems, HSMs, program memory, networked filesystems, the kernel, VM boundaries, and the difficulty for an attacker of assembling all these components at once. It's all pretty silly. But the answer to your question is simple: a salt (or nonce) isn't a key; that's not the purpose it serves in the design. If you really want to key your password hash, you don't need to muck with the salt to do that.
Edit: The defaults for the argon2 crate are here[1]. They seem to prioritize time cost over memory cost. Some random searching online suggests that a time cost of 3 corresponds to roughly ~2 seconds on modern hardware, so running a 100k dictionary with a time cost of 3 would require ~27.7 hours for the amortized find (50k) or ~55.5 for the worst case find. So, this isn't a very good scheme for a motivated (or parallel) attacker and an exceptionally weak password.
Let's imagine that you increased the time by 10x. That's 277 hours for a password. That's not very long at all - 12 days. Even if you increased by 100x, 120 days is not crazy, and presumably attackers can go way faster than your assumption.
A KDF isn't going to be enough to save you if you're using a top 100k password and the attacker can bruteforce offline.
Of course, users won't do this for themselves, which is why tools that do passphrase encryption should generate passwords by default, and accept user-provided passwords only as a non-default option.
Passphrases still have value, even when they're long strings of words: they're easy to write down, easy to repeat aloud, and easier than a random string to "recognize" visually.
Further, argon2id incorporates strategies to make GPU parallelization less effective.
Obviously this won't protect you against something like a dictionary attack, there's nothing that can magically protect you if you choose a low-entropy password, just something that can make the process more difficult.
Ignoring the extra time for decryption there's no difference between a unique low-entropy password that takes 2 years to bruteforce and a high-entropy password that takes 2 years to bruteforce.
If we're talking a more random but still short password (for example, just 8 random alphanumeric characters is log2(262+10)8 = ~48 bits), then the KDF becomes very attractive to help skyrocket the brute forcing cost to something more similar to trying to brute force the 256 bit key instead.
I will add a conclusion with an explanation of the security of the system.
TL;DR: You can play with Argon2's parameters to use the resources that fit your requirements.