If request latency is your top concern and you don’t need it to be as smart as GPT-4, why are you using OpenAI and not a model hosted by Groq or similar?
llama3-groq-70b-8192-tool-use-preview llama3-groq-8b-8192-tool-use-preview
Sonnet3.5 also has tool usage, and is slightly cheaper than 4o ($3 input vs $5).
If you _only_ need tool usage, then 4o-mini or others will be cheaper...
mistal's Mixtral 8x22B has function calling for $2/$6, which is a bit cheaper than 4o.