Two questions:
1. Does your machine use a spinning disk for its system drive, or SSD? If the former, that's slowing you down more any anything else. STRONGLY recommend moving to SSD system drive if so. 2. Have you tried `powershell /noprofile` from Run dialog or Cmd? If you're still seeing PowerShell take > 600ms to load and start, likely something else is up.
If you're expecting PowerShell to load faster than ~600ms, your expectations are likely a little out of whack - Cmd loads so fast because it's tiny and - relatively speaking - as dumb as a stump. PowerShell is a .NET language, and so loads the CLR, so incurs some overhead vs. Cmd, but it does A TON more that'll save you more time and effort over-all than any amount of time saving vs. starting Cmd.
HTH.