Neuron, a new VS Code extension for data science | Hacker News Reader

Neuron, a new VS Code extension for data science

blogs.msdn.microsoft.com

259 pointsstarbugs59 comments

Hey great to hear the excitement for this. In fact, we (the Python VSCode extension team) were so excited about it, that we've already begun integrating it into the python VSCode extension itself! It's a little rough yet for a full release but it's highly functional. If you want to play with it, just build the extension from https://github.com/Microsoft/vscode-python. Suggestions are welcome, but remember, it's still undergoing changes.

Unfortunately this extension has already died in activity. I forked the extension here: https://github.com/pavanagrawal123/VSNotebooks . A lot of bug fixes and usability improvements have been put in!

I am curious as to why Microsoft would release a post just two days ago about a dead plugin written by one of the developers of said plugin?

See my note elsewhere in the thread. This was summer student project. Not sure why the blog was posted so late...

In what way has it died in activity when there are tens of commits around June and July? That's not even four months.

3 months can feel like a long time without any commits in the age of corporate-funded projects like Vue and React.

This is not a big project like Vue or React. It's just an add-on. It is very common that popular add-ons are not maintained for months.

More relevantly, if people actually read the blog post, they'd see that this has been merged into the core Python extension, and is being worked on there.

Why not just contribute to the existing extension?

This is fantastic stuff, thank you.

No problem, if anyone has any feedback, please leave it in the issues. I want to make this extension great, and I would love the community's input!

Just to make sure it's seen, the Python extension team at Microsoft has an upcoming release that will integrate this functionality. The lastest github repo for this extension has the code we're working on.

oh interesting I did not know this... Is the code being integrated from Neuron?

No, Neuron was the inspiration, but the code was redesigned and reimplemented. Additional features have been added as well, such as defining a cell via #%% in code rather than having to select text.

Ahh I see... I've been looking at the PR and the implementation is really nice, I'm excited to see what it turns into :)

FYI - This was one of the two Cambridge University projects that was done with our team last summer. I'm not sure why the report came out this late. Both were well done, especially considering the learning curve and duration.

The 1st one was the data science plugin for vscode mentioned in the blog. The other one which I haven't seen published yet was a C# kernel for Jupyter which can be found here along w a video: https://github.com/MohamedEihab/icsharp/tree/Roslyn_2

[disclaimer - work at microsoft]

This looks awesome. I'm continually surprised by the improvements made to VS Code. I love both Jupyter and VS Code, and having both merged seems great for my use case (and anyone else doing exploratory code).

Props!

Anything that resembles RStudio for python makes me really excited to shift working from R to python for projects running in production.

Spyder was pretty nice coming from RStudio. I last used it a few years ago, when it came with the anaconda distribution.

https://www.spyder-ide.org

In that case you should check out Yhat's Rodeo IDE. Essentially built as "RStudio for Python".

https://rodeo.yhat.com/

It was a fun project but it's no longer supported.

https://github.com/yhat/rodeo/issues/655

https://github.com/yhat/rodeo/issues/631

Ah, right you are. Even looks like the download links are broken.

Any Emacs users looking for something similar, see EIN [0] and ob-ipython [1].

[0] https://github.com/millejoh/emacs-ipython-notebook

[1] https://github.com/gregsexton/ob-ipython

ESS [0] is fantastic, which I am using with R.

[0] https://ess.r-project.org/

The next generation of Jupyter, JupyterLab is turning into a whole IDE/platform, not just a notebook. They are also trying to incorporate the monaco editor which drives VS Code. Wouldn't that be a better solution?

https://github.com/jupyterlab/jupyterlab-monaco

that won't bring the rich support of extensions VSCode has.

VSCode's magic isn't just the monaco editor, IMO it's really the ecosystem and really nice Intellisense the other extensions provide.

though I do agree, this might be the better solution in the long run.

I spent time reading through Joel Grus' "I hate notebooks" slide deck and he makes a really compelling why they (Jupyter notebooks) are not the right tool for many reasons.

https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUh...

It was eye opening.

His YouTube live coding on building a neural net from scratch really shows the power of using VSCode to do AI work and complements his assertions.

https://www.youtube.com/watch?v=o64FV-ez6Gw&feature=youtu.be

Both are well worth the time. All the reading I've been doing on ML/AI really came together after watching and taking notes in that YouTube video.

You have to give credit to JupyterCon for inviting him! Most of his issues relate to Jupyter, he has/had not seen JupyterLab. You can trust that many of these issues will get worked out in the long run. I have nothing against VS Code but Jupyter(Lab) didn't win the ACM Software System award (Nobel price for software) for nothing.

This looks a lot like the Jupyter extension [0], which is made by a Microsoft employee and was last updated before the summer. I really like the cell format of Jupyter (inspired by Matlab I presume), I don't really want to select the code before I run it. Will Microsoft support both extensions? I would like support for actual notebooks as well, for when I want to mix text, code, and figures but without having to work in a browser.

[0] https://marketplace.visualstudio.com/items?itemName=donjayam...

> (inspired by Matlab I presume)

I think Jupyter is inspired by Mathematica (and Macsyma) notebooks, not Matlab... Source: Fernando Perez started IPython, very much inspired by Mathematica (he was switching from Mathematica to Python). Then Alex Clemesha and I created the Sage notebook in 2007, partly inspired by discussions and demos from IPython developers, and also very much inspired by Mathematica (Alex Clemesha, a physics student, was also a heavy Mathematica user). Alex then made knoboo in 2008 https://github.com/knoboo/knoboo), which is another notebook that inspired the Jupyter notebook implementation.

I was referring to the cell mode, not notebooks. In Matlab you create a new cell by starting a line with two comment characters (%%), in the Jupyter extension you use #%%. You can then run the current cell with ctrl+enter, no need for selecting anything.

https://blogs.mathworks.com/community/2008/01/07/ive-got-som...

I think Spyder is inspired by Matlab https://www.spyder-ide.org/

in my fork here: https://github.com/pavanagrawal123/VSNotebooks I'm almost done with cell support.

This runs Jupyter in the background.

So it retains some of the issues with it - e.g. notebook diff,etc.

I think (and this isn't just personal experience) most of the appeal of Jupyter is the fractioned persistent state that allows you to write incrementally but not re-run increasingly long scripts. Inline plots are just gravy.

(Yes yes write documents export LaTeX. Not sure people care about this at all)

I've been developing a style where development "cells" are cached with joblib.Memory so re-running a script from command line doesn't take idiotic amounts of time. Further, I have litte context managers such that

   with Figure('filename.png'): plotting_logic()

exports a file that immediately shows in the folder navigation bar of Atom. It's overall a better experience.

What I would really like is being able to use joblib.Memory not as a decorator (which requires bunching up your "cells" in functions and calling them separately) but as a context manager.

---

To the tune of the "Portlandia" song: the dream of MUMPS state is alive in joblib...

You can actually do this as a context manager, I have done something very similar in the past. I created a dummy context manager with a unique name and used an importhook to do an AST rewrite to the body of the context manager. It's a bit of a hack (okay, a lot of a hack), but allows for some really neat patterns that context managers didn't allow.

It actually looks like it is creating the notebook format on the fly from user’s selections, so it probably doesn’t suffer the notebook json diff problem. Other issues you have though, probably are present.

any exported notebook will suffer from the same issue, but the source file is just a normal python file, so diffs will be fine for those

You're confusing the Jupyter backend (which manages the Python kernels) with the front-end (which maintains the JSON document that reflects the notebook). You can have different front ends that use different formats than .ipynb.

So when I saw my "Advantages of Using R Notebooks Instead of Jupyter Notebooks" post was trending on HN this morning, I remembered Neuron is a strong idea to work outside both technologies, easily complemented by the multitude of VS Code extensions/theming.

There are still a few important missing features unfortunately (see the issues page: https://github.com/lorenzo2897/vscode-ipe/issues), but there's a lot of promise!

Looking through the issues list you linked to, lack of conda support [0] looks like a pretty big deal for most people doing heavy analysis in Python.

[0]https://github.com/lorenzo2897/vscode-ipe/issues/162

This is a big issue, and fixed in my fork here: https://github.com/pavanagrawal123/VSNotebooks. Unfortunately, neuron already seems to be dead :( so I created a fork with a bunch of new features and additions!

What problem does conda try to solve?

In general, it is a package manager which supports different environments. It is similar to pipenv, but existed before. I believe there are still some differences, for instance conda is language agnostic, and it particularly was useful for data science where you would want the compiled version of certain dependencies (eg. MKL) for performance reasons. This SO answer seems like a good overview: https://stackoverflow.com/q/20994716/1060781

Can anyone comment on strengths or weaknesses of this compared to Hydrogen?

(disclaimer, i'm working on a fork with better feature support)

Strengths: VSCode support

Weaknesses:

Bad code quality for the extension

No variable explorer

No extensibility

No custom env (fixed in my fork)

No cell support (almost fixed in my fork)

This looks really cool. Does anything similar exist for exploration on a SQL database?

I would love to see some notebook like interface for a SQL database with all the context awareness of Datagrips/IntelliJ’s editor but with similar visualisations like this tool.

Not exactly your answer, but microsoft is developing an open-source tool, that's visually similar to VSCode, to work on databases [1]. Sadly it only supports SQL Server, Azure SQL DB and SQL DW for now, but people are asking for additional database engine [2].

[1] https://github.com/Microsoft/azuredatastudio [2] https://github.com/Microsoft/azuredatastudio/issues/56

this is a pretty interesting idea. I feel like this should just be implemented at a Jupyter (kernel) level and not in an editor though....

Does this have a variable viewer like spyder (for data frames, lists, etc.)?

VSCode already provides that in the debug view

Neuron doesn't support this, it won't work natively with the notebook.

looks similar to Hydrogen for Atom. I'd recommend that (since it is actively maintained) too

Reply on news.ycombinator.com