> Does that 8K limit, limit your context length, i.e. history you have per chat thread with the tool?
Yes, the token limit for an LLM limits the combination of the prompt (which normally includes the whole conversation history, as the LLM itself has no memory) and response.
There's tricks to have a longer conversation without completely forgetting the past (summarization, offloading parts to a database, usually indexed by embedding vectors, and using search to recall relevant history, etc.) but the base case is everything has to fit into context.