Latency doesn't affect throughput. As soon as you fire off a call to an external vendor, your server can work on processing the next request. It doesn't matter if that call takes 10 ms or 5 seconds. Even if it took 30 seconds, that means you need to be able to handle 12k concurrent requests (almost all of which are asleep), which is nothing.

I'm not, otherwise something like 0.1% would be a reasonable upper bound. Even if you figure each order involves 20 async steps, that would be ~8k steps per second, which is nothing for a modern server. My old i5 can do 70k simple requests/second (the first type you describe) on the JVM with the app and database on the same computer (so contending for CPU time). A modern server CPU is ~15x more capable than mine without getting into multi-socket.

Put another way, a dual socket Xeon platinum can give you 120 cores. To do 400 orders/second on such a machine, you need to do less than 300 ms of core-time per order, which is an insanely large number. The things you mentioned spend almost all of their time waiting.

Why even have a whole system for it? Just expose an FTP server and let users edit an “orders.txt” file

1. Yes, of course.

2. How many is "many" magnitudes? The word suggests at least 4, but if so, it'd make more sense to brag about that.

And anything is cacheable if it doesn't affect transactional logic. If the user is looking at a page for 10 seconds, are they going to care that the inventory count is 3 seconds old? At your many orders of magnitude, caching for a couple seconds would give you hits almost all of the time for popular pages.

Also, aren't inventory counts that are displayed to users typically completely made up? Why would you make something like that dynamic and accurate in a high load system? The only reason to have it is to create FOMO or for b2b bulk ordering, and the latter is a whole different thing.