HNHacker News
TopNewBestAskShowJobs

prasoonds

631 karma · joined April 7, 2023

ml/fintech

prasoon [at] getlumen [dot] dev

submissionscomments
prasoonds··on Claude 3.5 Sonnet
This is amazing - I far prefer the personality of Claude to GPT-4 series models. Also, with coding tasks, Claude-3-Opus and been far better for me vs gpt-4-turbo and gpt-4o both. Looking forward to giving it a spin.

Seems like it's doing better than GPT-4o in most benchmarks though I'd like to see if its speed is comparable or not. Also, eagerly awaiting the LMSYS blind comparison results!

prasoonds··on Safe Superintelligence Inc.
Are you claiming Ilya Sutskever is a grifter?
prasoonds··on What Is ChatGPT Doing and Why Does It Work? (2023)
Like the sibling comment says, you typically have to pay for the premium model (and believe me, they are miles ahead of the free stuff - free copilot uses gpt3.5 which is a full 2 generations behind).

The other way is to get an API key from OpenAI or Anthropic and then, you can pay per token which is incredibly cheap - load it up with $5 and it can easily last you months if you're not using it much.

As a dev, I cannot recommend these better models enough. The ROI on my $10-$20 that I spend every month is easily in hundreds to thousands of dollars.

prasoonds··on What Is ChatGPT Doing and Why Does It Work? (2023)
People have tried this! And for not-complex problems, the newer models do give the right answer and yet usually fail on the complex tasks.

Prompting does help though - if you ask the model to check it's own answer, it can often catch errors it made and improve. But the interesting part is precisely how does the model succeed at all at any spatial reasoning task? Surely it hasn't seen all spatial reasoning tasks in its training set. So, what's the other hypothesis? As you say, people suggest that there could be a world model that is constructed internally by the LLM.

However, as you can see right here in this thread that people disagree that you need a world model for solving such tasks.

prasoonds··on What Is ChatGPT Doing and Why Does It Work? (2023)
You're right of course. Let me rephrase. What I was trying to say was: If the model solves some reasoning tasks that may imply it's building some inner world model which is fundamental to reasoning. That in-turn might mean that the model is reasoning.

Others have argued against the "inner world model" theory and suggested that solving reasoning tasks is merely an extension of the "stochastic parrot" scenario - i.e., they claim that no such world model exists and that the model has rote memorized these reasoning scenarios.

prasoonds··on What Is ChatGPT Doing and Why Does It Work? (2023)
This is an interesting thought exercise! The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world.

Another interesting bit of experiment people did when the GPT-4 class models launched were to test out spatial awareness. For eg, you could describe with words a construction made of blocks, spheres and so on and then ask questions about the stability of the structure. The newer models more often than not got it right.

Of course, that hasn't stopped people from making tired old claims of "stochastic parrot" and so forth but I think if your model can solve such reasoning problem (spatial or otherwise) then there's something really interesting to discover. I'm glad that folks at Anthropic are really trying to create a better understanding of these models (https://www.anthropic.com/research/mapping-mind-language-mod...)

prasoonds··on Chat Control Must Be Stopped – Now
The criminals absolutely will move away to something that is outside government control. Many such apps already exist and you can run what you like on Android phones with custom ROMs.

Think about it - if you're a criminal, and you know about chat control, why would you risk your chats being leaked at all? Why wouldn't you use a different app that you know to be more secure (this already happens for any serious crime already btw)

It's precisely the law-abiding people whose privacy will be invaded for no-gain

prasoonds··on Meta won't train AI on Euro posts after all, as watchdogs put their paws down
The article very clearly mentions training on your public posts.
prasoonds··on EU: Users who refuse scanning to be prevented from sharing photos and links
Not sure how to cross-post here but I think it's a losing game. EC can keep proposing the same law once every year and at some point they will get an EP that approves it. Repealing is impossible unilaterally by the EP. It's an every tightening rachet.

See my comment here: https://news.ycombinator.com/item?id=40560873

prasoonds··on EU: Users who refuse scanning to be prevented from sharing photos and links
EP is a fundamentally a weaker body than its name suggests. The European Council can keep proposing pretty much the same law over and over again until they get a pliant EP that will accept it.

Once accepted, it's almost impossible to repeal the law by the EP because they will need the approval of the EC and Council of the EU, both of which have members that are not directly elected by the people. EC and Council of the EU is also that body which so far has proven to be far more authoritarian in it's approach.

So, the parent is largely correct.

prasoonds··on EU: Users who refuse scanning to be prevented from sharing photos and links
I love how Europe with all it's GDPR laws (even binding on the government in many cases) and specifically Germany and it's Datenschutz-fixated citizens is sleepwalking into DDR era authoritarianism. Stasi's wet dreams couldn't think about this much control
prasoonds··on Scaling will never get us to AGI
I’m not sure if I’m parsing your sentence right but surely you can’t be implying that autonomous driving isn’t possible or that it is some undermined time away?

You can come down to SF and take a waymo today. It works great.

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Hey! We added support for SQL blocks btw!
prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Thanks!

We're different in a couple of ways:

- You don't need SQL to transform and manipulate data in Pretzel - we have visual blocks where you can construct a transform chain (though we do support manual SQL blocks now!)

- We have better CSV parsing (I spent a fair bit of time on this!). See one of our test case CSVs here that breaks (https://imgur.com/a/O8XMfET)

- We allow plotting!

- We support PRQL - which IMO is a better way to build data pipelines (but we also support SQL)

- We have AI Blocks! You can use your API keys (or our server) to get SQL from text which you can edit

- We're currently adding support for Python via Pyodide (pyodide.org) so you can do complex data transforms, even train simple ML models right inside the browser!

- Lastly, we'll be adding a ton more features to Pretzel over the coming months (many of them have been asked for in this thread actually!)

Support for multiple files is crucial - we'll be adding that soon. Right now, we're rethinking our whole task execution architecture so any major features will have to wait until then.

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Thanks! Yes, we're going to build the make modern data tool with a low barrier to entry (so non-devs, non-data folks can use it) but that also works great for power users (a way better jupyter notebook, with modern IDE like capabilities - pair programming, copilot, extension, out-of-the-box linting, testing etc).

It's almost a meta thing but I think dev-tools have been getting better and better over the last 10-20 years while data tools seem to be stuck in the past. Devs, when then see friction, go build something to solve it while this tendency is much less common in data scientists, for eg.

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Hey, I left a comment in another thread about the differences:

https://news.ycombinator.com/item?id=39732135

Didn't know about OpenRefine - but one major difference is Pretzel is web-based meaning no installation necessary (important for work laptops, for eg). We're also going to be adding a ton of features to this - think Jupyter notebooks crossed with Metabase crossed with Datasette/Pretzel.

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Datasette was definitely a huge influence for this and I'm a big of @simonw and his work.

However, there's a couple of major differences:

- Datasette - to me - is a tool for devs - it takes a fair bit of tech know-how to understand how to load new data in datasette, install plugins and deploy an instance. I wanted Pretzel to have a "low-ceiling, high-floor" - meaning the tool should work well for non-technical folks but also shouldn't hinder technical folks and power users

- Datasette has a unique way to explore data based on "facets" where you can dig through data. It's a powerful way to explore but I personally don't think Datasette is the best tool for complex data transforms - its strengths lie in exploring datasets instead.

- For accessibility and speed, we wanted something browser local and fast - hence the focus on DuckDB and canvas rendered tables. Datasette requires setting up an instance and that's additional friction for non-tech folks and isn't particularly designed for large datasets (is my impression - I might be wrong)

- Lastly, we have ton of future plans - including in-browser python support via WASM (Pyodide), local LLM support, ability to connect databases etc.

I think there's space for both tools!

prasoonds··on Y Combinator's chief startup whisperer is demoting himself
Product Market Fit. Basically, can you find through iteration something that people really want and are willing to pay for
prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Wow, Graphic Walker looks incredible! AND they vega-lite - I'm a BIG altair fan - this look like fun - I'll take a look!
prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Yes! I think it was TDS viewer or something like it - I loved using it so thank you for building it :)

Tad viewer looks perfect for embedding in Pretzel! Is it easy to embed it in web apps? I’m on mobile right now and did a quick search and didn’t find anything. I’ll definitely be in touch!

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
that sounds like a fun project :) maybe I'll take a stab at it!
prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
DuckDB supports SQLite through extensions (https://duckdb.org/docs/extensions/sqlite.html)

So, it should be pretty straightforward to let folks drop a SQLite file instead of a CSV file! I haven't been able to get extensions to work on WASM so far though but I will definitely take a look to see if we can make this work!

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Thanks! Maple looks really cool - really interesting demo video, too!

This is quite interesting - we've not explored really large files so far and being honest, we haven't thought that far either. Didn't know about CBOR! I will have to look deeper into how this can save on memory. I was wondering though, since WASM memory is limited to 4GB, if I have sufficiently large memory on my compute device, at least one tab should be able to handle 1gb+ files too, correct?

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Replying as I couldn't edit:

Alright, did a bit of digging and the flamegraph points to a problem with the Table component. I'd hoped that BlueprintJS tables would be performant enough for our usecase but apparently not!

We'll try to move to canvas based rendering ASAP - that should fix any lags in the filter UI

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Right - I used to work in GS and they had this really great internal table display tool where you could simply drag a column from the top to the left side and it would Pivot + Collapse all fields on it (plus allow multi level pivots). Then, you could look at the Pivot table OR do a drill-down to see the root-cause why a value was so high.

I really liked that interface and haven't really found anything near as useable yet. Perspective JS (a free library by JP Morgan, you can try it here: https://perspective.finos.org/block/) has some really cool functionality in this direction but it has its one data processing engine, query language, rendering engine etc so we couldn't have used that for this project.

But, this is an interesting thought - I definitely would want to see this functionality in Pretzel - the only question is how to prioritize this. If you know of any performant table libraries that support collapsible pivots out-of-the-box, I'd love to integrate that. Alternatively, we'll write our own!

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Thank you! Yes, one of the items in the Roadmap is support for Pyodide (https://github.com/pyodide/pyodide) for running in-browser python on the results of each of the code blocks! This should allow most ML libs to be usable in-browser! This is pretty high-up on our priority list.
prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Hey, absolutely - that's a perfectly fair question. I can give you my personal reasons:

- I don't have excel on a Mac so I need to use Google Sheets. Google Sheets crashes on any file over 30-40 MB (at least on my macbook air)

- I think this type of chained data transform is a much better (more code-like?) way of doing transforms. You can see exactly how you got to your end-result. Vs, for example, in Google Sheets, you're modifying data in-place and your transform history is only accessible via Undo and Redo. This is far more reproducible short of using a python script.

(I've worked as a data scientist for a while and I suppose that influences how I think about chained data transforms vs the in-place Excel way too!)

But this is just for now - the long term vision is: Being able to easily switch between visual no-code blocks, SQL and Python in the same browser-local "notebook"/"workbook" that's easily shareable. I think it doesn't take much convincing that such a tool would be more powerful than Excel for certain workflows!

prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Oh hey! I remember seeing your HN post several weeks ago, I think - it's a great tool, thanks for building this! It was definitely an inspiration for us :)
prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
This echoes my thoughts exactly. Right now, we're actually more limited by the JS UI so a couple 100 MBs is the most you can do in a browser otherwise the UI becomes really slow. There's a lot of room for improvement - we're using React and that's causing a bunch of un-needed re-renders right now that we don't need. We probably need to create our own DAG based task management system and use Canvas to render everything - with all that, workflows on much larger files will hopefully become usable.
prasoonds··on Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
Hey Max, love what you all doing with PRQL! It's a wonderful tool.

That makes sense. I ran into some old issues about Pivot but it seems it's not been implemented yet. And yes, s-strings wouldn't work - the DuckDB pivot statement looks like this:

  PIVOT ⟨dataset⟩ 
  ON ⟨columns⟩
  USING ⟨values⟩ 
  GROUP BY ⟨rows⟩
So, we'd have to pass the `dataset` so far into the pivot statement. This is where CTEs come in handy but happy to hear if there's a better solution :)
← PreviousPage 3 of 4Next →