openquery

Hey!

> Is there a way to guarantee a specific set of test data each time

Absolutely. You can seed the model so that the data you get each time is completely reproducible

> For this testing it's not as important that the data is repeatable, but the testers have a few different scenarios they want to test, so I'd need a way to make a low-data, medium-data, and high-data test set where the backing data fit within some ranges.

This is a great use-case for Synth. With the upcoming Firehose API you can point it at an existing database and specify how much synthetic data you want to generate and pump into your db.

For now you can either create a database and write the ETL, or do `synth model sample <model-id> --ouput <some-directory> --sample-size <number-of-rows>` to sample directly from the model into a directory of CSV files and use that to load your database

Feel free to get in touch if you would like to learn more :)

Thanks for the feedback. This is exactly what we're looking for.

> It is 100% easier for me to export a little production data to test on (and maybe sanitize), or to write a small script to generate a few users and those things I need to test.

In your case it may very well be. But when you are an organization with a schema which has 100+ tables and these tables have scattered sensitive information this can become a nightmare to manage. I've seen this first hand. Furthermore if you are trying to generate more than 'a little' data this can get more complex as you have to create factories and write a lot of code to make the whole thing coherent and tell a story. I think undertaking the added complexity of Synth is a trade-off one should consider depending on the sophistication of the testing data they require.

> If I did use it, i'd prefer it to be much easier to use

I think this misconception may be attributed to the fact that we use machine learning under the hood. We've spent a lot of time abstracting the developer away from this. In fact you can run the whole lifecycle with 1 line of code:

`synth model new --from-database <database-uri> --train --deploy`

> I would still be concerned about whether this is deterministically creating data or if its random?

At this point you can choose. You can either pick a seed with which the whole generation process starts (this may not be in production yet) or elect to randomly seed it.

Thanks for the great questions :)

Thanks for your comment :)

These are all very good points. We are in the process of figuring out a natural way to express user-specified semantic types. We have some ideas but more on this coming soon!

Yes - you're absolutely right in that data is a messy business.

Even in the early days we've seen crazy data types and constraints that makes our job of completely automating the process hard. However, every instance of this makes the product better and this transfers to the next customer.

> You really need an analysis phase that can tell a customer whether their data will work or not

This is part of the roadmap, it's a non-trivial piece of engineering. In the meantime you can try it for free and see if it works for you :)

Thanks for the question!

For now text data will be marked as `categorical` or `text`. When you have sensitive data you want to use `text` which will provide a lorem-ipsum type generator.

If the model has classified that column with the semantic type `text`, no information from the column should be leaked :)

Hey - good question.

Not right now, but it shouldn't be hard to implement. Is there something some specific use-case this would address?

Hey - great question!

We've been careful to design Synth such that the model doesn't contain any sensitive information. That being said I completely understand where you're coming from.

We do offer the enterprise version for on-prem deployments. Basically, if you have a Kubernetes cluster you can run Synth on-prem :)

Thanks and great questions!

> First, we’re using Postgres and some of our tables use JSON...

We've seen this before when we were talking to a company we were considering to pilot pre-launch - it's on our roadmap. Currently the JSON text would be treated as a string, i.e. it is classified as a categorical type or text.

What we would want is for the classifier to traverse the JSON object instead of treating it like text. This feature is going to be implemented when we extend to NoSQL databases.

> Second, I’m concerned about giving Synth access to my data as much of it is sensitive.

Absolutely. This has been one of the guiding principles in building Synth. We've built it so that our servers never have to see any sensitive information. (Hence why you can use Synth via a CLI tool instead of an API)

Also:

1) The CLI is soon to be OSS giving full visibility into exactly what's happening when you use it. (Really it's OSS now since you can just take a look at the source code running in the container, we just haven't had the time to make our repo public)

2) The models are designed to be transparent. You can inspect them by running `synth model inspect <model-id>`. This gives you visibility into exactly what the model looks like. (Looking at the data which has been sampled is still a WIP)

3) If something goes wrong and sensitive information is uploaded to the Synth platform, you can easily purge all traces of it using `synth model rm <model-id>`

Absolutely! We spent a bunch of time in the data privacy space before pivoting to Synth. Synth has utility as a dev tool but really does address exactly this issue.

This also ties into GDPR and CCPA compliance - we think that as regulations tighten (which seems almost inevitable) this sort of tooling will empower developers to go quicker and focus on their applications instead of compliance.

Hey!

So we are soon introducing the Firehose API. Basically this allows you to point at an arbitrary database and fill it up with as much data as you need from the model.

The Firehose should work for use-case and be much more cost effective.

A more hacky solution for right now, you can spin up a database and run a `select * ...`.