How would you defend it against competition?
* Competition includes customers building the thing themselves and third party competitors.
* Don't worry about product market fit, distribution channels, audience targeting, MVP features, or any other very valid concerns for early-stage startups. This post only concerns moat.
Maybe compute would be a moat...can you run it in inference cheaper? Conditional computation, very quantized models. Probably training compute is not a moat however.
Not sure what SOTA is on callouts to oracles (e.g. calling out to Mathematica or some database) during inference, but maybe if you could use.
I would look at BloombergGPT for inspiration on a SOTA vertical GPT [3].
Probably speed to deployment & integration & sales would be a moat too, as once you have customers using your stuff your competitors will have more trouble selling to the same folks.
[1] In the recent interview on Lex, Sam Altman referred to "data from partnerships". [2] Probably very reviewed by human raters. [3] https://arxiv.org/abs/2303.17564