I'm pretty sure that we've not seen the end of The Expanse's universe.
I don't know why they would include it if they don't have plans on continuing the show in some form in the future. It's not like there was any shortage of content in the mainline story to cover.
This is not the mark of a show that they plan to renew. :-(
Amazon doesn't own the TV/movie rights of The Expanse (beyond the seasons they funded), they're owned by Alcon Entertainment. So basically, whatever Amazon does or doesn't plan to do is kind of irrelevant and there isn't really anything stopping from the show appearing on, say, Netflix, or the BBC in a few years time. This is also why it transferred from SyFy to Amazon relatively easily.