HNHacker News
TopNewBestAskShowJobs

prasoonds

631 karma · joined April 7, 2023

ml/fintech

prasoon [at] getlumen [dot] dev

submissionscomments
prasoonds··on Artificial intelligence gives weather forecasters a new edge
This has not really been my experience in my information bubble. But I admit I haven't kept up too well with the latest models and their failure modes.

Can you have provide any examples of the scenario you described?

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
I'm not sure if you're referring to us or any other projects but in case you're referring to us, I'm sorry to hear that you came away with that impression.

Not sure what more we could have said to make it absolutely clear that Pretzel is a fork of Jupyter but here's the first line of our Show HN post today:

> We’ve forked Jupyter Lab

Here's the first line of our README

> Pretzel is a fork of Jupyter with the goal to improve Jupyter's capabilities

Here's the first line of our LICENCE file

> Pretzel is a fork of Project Jupyter, which is licensed under the terms of the Modified BSD License.

I'm happy to add this to more places if you think it would make it clearer for people.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Hey Mike, just filed a github issue for this: https://github.com/pretzelai/pretzelai/issues/111

If you wouldn't mind, can you please comment on this issue so we can ask a few follow-up questions to help in debugging this issue?

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Hey, we've actually mentioned jupyter-ai in the original post above. In my experience, the UX we've created is much better than jupyter-ai and AFAICT jupyter-ai doesn't have an inline code completion replacement like GitHub Copilot (that we're supporting with Codestral)

If you've used jupyter-ai, I'd really appreciate if you can try out Pretzel and tell us if you're missing anything. We definitely are behind on the number of integrations and as you say, jupyter-ai codebase is more mature but I really think we've a made a much more usable tool.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Yes - I've replied here: https://news.ycombinator.com/item?id=40857807

I'll also copy what I said in another comment: In short, for now, yes might have been able to implement this functionality as an extension (though there are several other changes that we've made to core Jupyter behavior as well). But our roadmap calls for much more sweeping changes to the code execution model itself and that simply cannot be an extension.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Hey, that's a fair question. I've responded to another question about why fork here: https://news.ycombinator.com/item?id=40857807

In short, for now, yes might have been able to implement this functionality as an extension (though there are several other changes that we've made to core Jupyter behavior as well). But our roadmap calls for much more sweeping changes to the code execution model itself and that simply cannot be an extension.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Thank you for the links! We'll take a look.

At the moment, the system prompt is hardcoded in code. I don't think it should bias analyses because our goal is usually returning code that does something specific (for eg "calculate normalized rates for column X grouped by column Y") as opposed to something generic ("why is our churn rate going up"). So, the idea would be that the operator is responsible for asking the right questions - the AI merely acts as a facilitator to generate and understand code.

Also tangentially, we do want to allow users some king of local prompt management and allow easy access, potentially via slash commands. In our testing so far, the existing prompt largely works (took us a fair bit of hit-and-trial to find something that mostly works for common data usecases) but we're hitting limitations of a fixed system prompt so these links will be useful.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
That's a fair concern and one I've been through myself. I think what we've tried to do is a little bit the opposite honestly - hundreds of thousands of people already use Jupyter for data work and we started with the idea to go where the users are precisely so that they don't have to switch tools.

By making a fork that can be installed in one line of code, we're hoping that we don't make Jupyter user's go through decision fatigue for yet another dev tool. Instead, the idea is to simply make their existing tool better.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Ah sorry about this. This Show HN was targeted towards folks who have a passing familiarity with Jupyter Notebooks but I should have explained a bit more.

Jupyter is a web application that lets a user execute python code in either a local or remote "kernel" - which is just a Python process with a communication layer.

You can then do literate programming (meaning run some code and immediately see results including plots and tables) within this web interface. Basically a much fancier REPL. The code can run either locally on your machine or remotely in the aforementioned python "kernel".

Here's a quick introduction to Notebooks that I found on YouTube: https://www.youtube.com/watch?v=jZ952vChhuI

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Hey Mathias! Was fun chatting about Livebook the other day and yes, I'm definitely looking at it for inspiration! Alas it's an Elixir only notebook and so far as I know, there's very few data folks using Elixir so might be a hard sell.
prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
It's true - VSC family of editors (VSCode, Codeium and Cursor) all let you use AI autocomplete, question answering etc with your notebook code through various extensions. However, lots of Data Scientists and Analysts prefer using Jupyter Lab or Colab or in general notebook like interfaces. Plus, this is just a start - we're going to be adding way more features that will make the differentiation clearer (see my other comment earlier today: https://news.ycombinator.com/item?id=40858509)

As of now though - there are plenty of people who do use Jupyter and we hope Pretzel - as it stands today - can already be of help to them.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Thanks Marc! We'll check them out. So far, with our limited experience with local models, we'd simply been thinking to use Ollama so thanks for the heads up!
prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
You're right, that's not how it would work! If you look at our license - all the Jupyter code stays BSD3. If we modify any BSD3 code, the modified code stays BSD3. All the new code we write - in separate files - is AGPLv3 however and is clearly delineated in the repo in the file headers.
prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Interesting! We have a inline autocomplete as well (that's the one that uses Codestral). So in your example, you'd see a autocompletion prompt after you type pd.read and hitting tab will autocomplete the line (or many lines if there's enough context to generate that much text).

The AI mode/box with Cmd + K is for more complex prompts, multi-line code chunks and such. We've tried to make everything completely keyboard accessible btw, including the sidebar so that you never have to use the mouse :)

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Oh cool! I'll definitely try to make it in one of the meetings :)
prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
So, we were tipped off to Codestral being really good because of continue.dev - for reference, that's a VSCode extension that gives you similar features to Cursor. After we trialled it out head-to-head against GPT-4o for fill-in-the-middle completion, in my experience (purely vibe based), it produced better completions maybe twice as fast as GPT-4o.

We haven't tried vs Sonnet 3.5 yet - my hunch is that on the speed/quality/cost space, Sonnet will end up doing better than Codestral for some folks.

Against general purpose local models (taking Llama-70B as a high-water mark), Codestral does better far better on code related tasks while being less than 1/3rd the size (22B!). That said, I'm definitely exited to try out DeepSeek Coder v2 - by all reports, it's amazing model for code completion and will likely also beat out Codestral.

I don't think we're planning to fine-tune Codestral though (or any model for that matter). The latest models keep on becoming faster, better and cheaper AND they already work quite well. My thinking at this time is that waiting it out and having a big AI lab make a more capable general model is a better strategy.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Yeah PyCharm and VSCode are definitely great options (though PyCharm is paid and VSCode AI extensions aren't notebook tailored). If you ever get a chance, I'd love to get your feedback on Pretzel - I think Codestral is a better and faster inline completion model than GH Copilot's GPT4 class model plus I think we might do context-relevant questions better :)
prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
That's a cool blogpost! I'm mostly using Cursor now (just waiting until someone makes a kick-ass Emacs package so I can switch back!) so I can definitely see your perspective.

I'd be curious to hear a bit more about the kind of work you do that made you switch. Also if there's anything you miss in VSC/Cursor vs Jupyter. If you don't mind a small email exchange, let me know and I'll drop you a message :)

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Agree with Daksh in the sibling comment. I think it's like you said - different people have different workflows and some might prefer using VSCode. IME though, most data scientists (and all data analysts) I worked with preferred using the company hosted internal Jupyter instance for their work.

Also, as we build more features, we're definitely going in the direction of more analytics workloads (live collaboration, leaving comments, google-doc type versioning, fully AI driven analyses similar to OpenAI Interpreter mode etc) and with these features, I think there will be a clear divergence of feature-set in VSCode/PyCharm vs Pretzel.

If I may ask, are you more on the engineering side (MLE) or more on the data side (Data Analyst)? EDIT: Just saw your other comment!

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
That's fair, I was mostly responding to "but it's similar to what you're concerned about" because I didn't think the concerns are the same (but I can see your perspective on it too! While we are "giving away" the code, we're definitely posing some limitation).

Thanks for the kind words, we're exited to be building this :)

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Thank you for the list - I think I've come across all of these in my research! I'll try highlight the differences for each.

- https://noteable.io/ - as you say, it doesn't exist anymore

- https://deepnote.com - Deepnote is closed source sadly - you can't run it locally, you can't tweak it, you need to learn a new interface and switch to it

- https://github.com/jupyterlab/jupyter-ai - I actually mentioned this in the post but in my experience, the UX and features are far behind what we've built already. I'd love to hear from anyone who's tried jupyter-ai to give us a shot and let me know what we're missing! The plus side of jupyter-ai is of course that it supports way more models and the codebase is a lot more hackable than what we've built.

- https://colab.google/ - closed-source, similar challenges as with Deepnote. Another big challenge is that if you want to use Colab as a company, AFAICT, you need use their enterprise version (so that you can have native data collectors, support guarantees etc) and that only works with GCP so if you're an AWS shop, this might be a deal-breaker.

- https://cocalc.com - hadn't used it so far but congrats on a great project! Will check it out. Didn't look in detail but first impressions makes it look like a fairly different interface from Jupyter. One of our goals was to go to where the users already are - that meant Jupyter. So that's definitely a major difference.

- VSCode - as I've mentioned elsewhere, we're targeting a more of an analytics usecase with the features we're building. VSCode has AI features of course! But we'll look quite different once we build more items on the roadmap :)

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
This is true. All new added code is licensed under AGPLv3. But I fail to see how it's the same thing as modifying and open-source tool and re-selling it as a closed source tool. This is what AGPL gives us - anyone can use Pretzel however they want. They can even re-package and re-sell it if they want so long as they too open-source their modifications and improvements.

Selecting the license for an open-source tool backed by a company is tricky. You want your code to be open-source for it's benefits (for eg, for us one benefit is building trust with people working with sensitive data). But, the history of open-source tools is full of tools that another company just started reselling without doing any of the work (sentry, mongodb etc). So, you need to find a balance. AGPLv3 strikes the right balance for us.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Thanks for the kind words. Keeping Pretzel open-source was important to us - partly for trust reasons. When most people use Jupyter, they do so with sensitive data. Making a closed source tool simply wouldn't work. I wouldn't have trusted a closed source jupyter alternative with my company's data unless the counterparty was huge and well-known.

To your second point - completely agree that most users would expect these feature from their IDEs today. But, only two IDEs support Jupyter Notebooks: VSCode and PyCharm. You can certainly use them for notebook work but most AI extensions written for VSCode wouldn't be optimized for notebook work (for eg, GH Copilot apparently has difficulty completing code across different cells as per a friend). Secondly, to your point, this is just a start - we're going to be building a lot more Data Analysis specific features that don't exist in any IDE. I think there's a decent space for a tool a like this.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
It's as others say - in VSCode there support for Copilot but most Data Scientists and specially analysts who don't spend most of their day in a text editor still use Jupyter Lab (or Notebook - I mean the software, not the file format) and with Codestral, we've found similarly good completions (sometimes better than Copilot) but at a much better speed and cost.
prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Yeah, this is definitely a good way to access AI code completion (inline or otherwise) in Jupyter notebooks. In fact, I know some data folks who've been using Jupyter from day 1 switching to VSCode simply because their company buys a copilot license for everyone and they really miss it their Jupyter workflow.
prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
So we discuss this briefly on our FAQ but let me try to expand on it.

Our goal is to make a modern literate programming tool. On a surface level, a tool like that would end up looking very similar to Jupyter, though with better features. We've mentioned some things we'd like to have in this final tool in our README and also in the post above.

Our first thought was to make a tool from scratch. The challenge was, it's very hard to get people to switch and so, we had to go where people already are - that meant Jupyter.

We could've made this one feature an extension with some difficulty (in-fact, our early experiments, we started by making an extension). It would have some downsides - we wouldn't have granular control over certain core Jupyter behaviours like we do right now (for eg, we wanted to allow creating hidden folders to store some files). But we probably could have made a 95% working version of Pretzel work as a jupyter extension.

The bigger reason we chose to fork was because down the line, we want to completely change the code execution model to being DAG based to allow for reproducible notebooks (similar to https://plutojl.org/ for eg). Similarly, we want to completely remove Codemirror and replace it with Monaco (the core editor engine in VSCode) to provide a more IDE like experience in Jupyter. These things simply couldn't have been done as extensions.

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
Agree. We actually tried getting GitHub Copilot to work with Jupyter but GH doesn't have an official API. We actually took some time to reverse engineer an implementation from neovim GH Copilot extension [1] and from Zed [2] but found it too flaky and too much trouble in the end.

Meanwhile, we also found a better speed/quality tradeoff with Codestral (since it has a fill-in-the-middle version, unlike a general LLM) so we decided to go with Codestral. This is inspired by continue.dev using Codestral for tab completion :)

[1] https://github.com/github/copilot.vim [2] https://zed.dev/blog/copilot

prasoonds··on Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks
We're just a fork of Jupyter so everything - notebook files, keybindings, extensions, settings should just work.

We pull all of your config from the ~/.jupyter folder so you should be able to switch between Jupyter and Pretzel from different python environments (though you might see some warnings)

prasoonds··on Waymo One is now open to everyone in San Francisco
I highly recommend everyone try it out if you're in SF. It's an incredibly smooth and sure ride. The cars are really nice too (Jaguar I-Pace electric cars), clean and spacious.

The first time you ride in one, it feels truly sci-fi. But within 5 minutes, you're almost bored of it - that's how good it is. If I had to choose between an Uber of questionable cleanliness and driver temperament and a Waymo with a slightly longer wait and slightly more fare, I'd choose the Waymo every time.

(I have no affiliation with Waymo, Google or any related industry - it's just an amazing service!)

prasoonds··on Claude 3.5 Sonnet
100% agree here. Claude is especially good at larger context sizes and retains coherence way longer than GPT-4 series of models
← PreviousPage 2 of 4Next →