- Request 1 is a post that generates a "todo" with a random id "5y22"
- Request 2 is a "get" for /todo/5y22.
That works in production, but on replay of the traffic:
- Request 1 generates a different random id, "86jj".
- Request 2 is still a replayed "get" for /todo/5y22, which is now 404.
How does your tooling handle this nondeterminism in replays?
So if you mean that we want to "test" TODO, our analyzer looks for data in subsequent requests that was provided by the system-under-test (SUT) in previous responses. A common example of this is an HTTP cookie. The SUT gives us a session id through the Set-Cookie header response. So in a subsequent request we use the cookie from the app, not the one that was recorded. This has been done in general way to look for tokens.
Of course nobody is perfect so we'd love to see your real world app and test our algorithms against it. :)
Non-determinism would be, for example, something that's time-sensitive. If some result varies by time, then the only way to test it is to include time as a parameter. This can be complicated if the time variable plays into asynchronous updates (e.g. you want to test that a POST update worked, but it's actually eventually consistent). Caching (e.g. through Varnish or a CDN) would be another thing to make such tests much more complicated.
Another example is an API that has side effects. For example, a stock trading API might read real time quotes from another service. A stock trade then alters the next quote.