HNHacker News
TopNewBestAskShowJobs

n4atki

1 karma · joined February 27, 2024

submissionscomments
n4atki··on Synthetic data generation for tabular data
Curious what a good end-to-end solution looks like for you? Is it more about ease-of-use (import/export with minimal effort) or is there a privacy layer that's missing?

I see it in 4 steps: 1. Connect to a source db to import your data 2. Train a Gen AI using the source data 3. Use it create synthetic data 3. Export synthetic data into a new db

The SDV team is working on business solutions to cover the full use case. You can use the public SDV to validate steps 2 and 3.

n4atki··on Synthetic data generation for tabular data
SDV does offer a CTGANSynthesizer, which is a GAN-based generative approach. Could be worth a try, though CTGAN specifically may require customization (tweaking some parameters).

That being said, synthetic data definitely isn't a magic pill for all use cases. I have found it particularly useful for things like QA, performance testing, etc. -- where alternative tools for test data creation aren't sufficient.

For the use case of imbalanced classification: May be worth asking what is it about existing solutions (SMOTE) that doesn't work well?