I want something like std::vector, but I don't want exceptions. How does C++ help in this case?
What I'm saying is that you can use C++ without having to use STL and you can use C++ just fine to implement your own exceptionless container if you want (and many such implementations exist already).
And you can also compile with exceptions disabled. Should your code throw it will terminate the process.
The C++ community at its finest.
You can use which bits of C++ that you want - that's its great strength, permitting low-level behaviour whilst supporting high level abstractions (and as simple or as complex as you like).
Windows actually provides a stronger gurantee here. But if any of your dependencies are not OOM safe then neither are you. And much of windows userland is not OOM safe.