Hacker News Reader

Comment by nerdponx | Hacker News Reader

So you ask OpenAI to generate a schema and computer code that will clean input data to conform to that schema, and then run the user's data through that program? Is it possible for users to obtain and audit the generated code for correctness, performance, etc.? How do you prevent things like the AI from generating catastrophically-backtracking regex, O(N!) algorithms, or outright mistakes?

View on HN

@nerdponx Open AI only generate a validation schema.We have our own schema generation engine for any custom validation which can be used where Open AI is not able to understand or generate correct schema.Yes you are absolutely right.Open AIs output is not always right.We have integrated a JSON parser which validate Open AI output and currently developing a regex parser which validates Open AIs output.Hope it answers your query..Happy to understand your pain point more.

Interesting, thanks for explaining how it works. Would it also be possible to construct one of these JSON schemas by hand, without OpenAI? The core data cleaning system sounds like just as interesting a piece of technology as the AI schema generation.

@nerdponx Yes YoBulk provides a way to write JSON schemas by hand..We have added some custom keyword like validate which is not defined in JSON standard..But our validation engine can understand that..You can pass a javascript function through validate keyword..It's a game changer.. "first_name":{ "type":"string", "format": "first_name", "validate": "(x) => {\r\n let regex = new RegExp(\"([a-z][a-zA-Z]*)\");\r\n return regex.test(x);\r\n }" },

This is a very interesting combination of technologies. Thank you again for explaining how they work! I tend to prefer open-source solutions for my own work, but I can see this being highly useful and valuable for many businesses.

Thanks keep following YoBulk..We will keep on doing some path breaking work in CSV importing space.

Reply on news.ycombinator.com