anarkafkas

Thanks for your interest! We currently support 7 models from OpenAI and Anthropic including gpt-3.5 and gpt-4. You can definitely configure the agents to the specific needs of your application including dynamic configuration that changes for each user. The fine-tuning of base models is currently not supported but our early customers have not really needed this. Happy to help you with your specific use case! Feel free to reach out to me at anar@proficientai.com or our Discord channel. Also you should find much of this information in documentation.

Are you sure about this? Not sure what you mean by "offset" but we are passing the last document of the current batch to the .startAfter() method which ensures that the next batch only contains the docs that come after that. So there shouldn't be any doc that is processed twice. But as I said earlier, the new docs won't be traversed which is expected. I'm working on a different type of traverser that will fix this.

I'll actually write up some tests to confirm that we don't process any docs twice!

Nothing unexpected. You're charged N reads where N is the number of docs traversed.

Thank you! Yes, that's correct. The limit for a batch migrator must be 500 because it uses Firestore write batches internally.

I'm currently writing another migrator that won't be using Firestore batches, it'll just use the good old Promise.all(). I'm planning to add more capabilities soon like error-resilient traversers using different traversal strategies, the ability to re-traverse the docs that couldn't be migrated the first time etc.

Great question! Since traversing the entire collection may take a while, it's definitely possible that a new doc has been added in the meantime. Whether that new doc will be traversed or not depends on its order/index within the collection. It definitely won’t be traversed twice.

If it's positioned before all the docs in the current batch then it won't be traversed. If it's positioned after the current batch then it will be. So obviously that also depends on whether you’re traversing a plain collection or a Query.

Catching all the new docs that were added requires implementing a different strategy like adding a temporary field to all the traversed docs and then querying the ones that don’t have that field. It’s definitely something that we can implement soon!