I guess someone needs to build a Solara app running in the Jupyter Notebook to interface with Jupiter (running on Solana).
334 karma · joined February 16, 2017
It supports streamlit, dash, virzo and solara. Frameworks like shiny, gradio and panel are planned (maybe even fasthtml).
It's also really nice to integrate this into your documentation: https://mkdocs.py.cafe/
This also uses pyodide, but supports more frameworks out of the box.
* Solara will not continuously re-execute your script as Streamlit does.
* Solara will re-execute components instead, only what needs to. * State in Solara is separate from the UI components, unlike streamlit, where they are strongly linked.
* State can be on the application level (global) for simplicity or on the component level (local) for creating reusable components.
* Solara should not block the render loop. Long-running functions should be executed in a thread using use_thread.
GitHub: https://github.com/widgetti/wanderlust App: https://huggingface.co/spaces/solara-dev/wanderlust X: https://twitter.com/maartenbreddels/status/17223244907077020... Solara: https://github.com/widgetti/solara/
The follow up of that is Solara: "NextJS, but in Python" :)
Let’s take that question to GitHub or discord not to go too much off topic here ok? Thanks!
I think what sets Solara apart from the rest is that we use a 10 year proven paradigm, React, but on the Python side, although with a more Pythonic API. For instance making reusable components in Python is a real benefit. Pynecone uses ReactJS in the front end, but on the Python side it’s their own API.
I do like pynecone, their API is pretty. I do wonder with many of these frameworks. Does the paradigm works for more complex apps, or does it end in complexity hell once you go past the hello world examples. At least React has proven itself.
Mercury, in my view, seems really interesting for quick demos and reports. It's somewhat akin to Streamlit, making it ideal for notebook authors.
On the other end of the spectrum (and pardon my shameless plug here), there's Solara (https://github.com/widgetti/solara/). Solara is a fully-fledged web framework for Python, specifically designed to handle app sizes that surpass what Streamlit and Dash can manage. It's worth noting that Solara can render ipywidgets, using a pure Python React-like framework. This can be particularly useful after the exploration phase, when you're looking to transition a project from the notebook into production.
We helped many clients get their prototypes into production and identifier some friction doing that.
We now have a way to make more complex apps using the same stack, and get better app startup performance.
While there are many Python web frameworks out there, most are designed for small data apps or use paradigms unproven for larger scale. Code organization, reusability, and state tend to suffer as apps grow in complexity, resulting in either spaghetti code or offloading to a React application.
Solara addresses this gap. Using a React-like API, we don't need to worry about scalability. React has already proven its ability to support the world's largest web apps.
Solara uses a pure Python implementation of React (Reacton), creating ipywidget-based applications. These apps work both inside the Jupyter Notebook and as standalone web apps with frameworks like FastAPI. This paradigm enables component-based code and incredibly simple state management.
By building on top of ipywidgets, we automatically leverage an existing ecosystem of widgets and run on many platforms, including JupyterLab, Jupyter Notebook, Voilà, Google Colab, DataBricks, JetBrains Datalore, and more.
We look forward to your thoughts and feedback!
Indeed, for small data there is not much to gain, at least this is not the focus of this article. Although even with small amounts of the, the automatic pipelines are useful https://vaex.io/blog/ml-impossible-train-a-1-billion-sample-...
The first commit was Jan 2014, when it between 50-80% of my time until 2018 I think where I mostly developed it myself. After that, it is more difficult to say how much time was spend on it, and it wasn't only my time. Although I do most of the development, ideas and discussions with data scientists can be more important than just pure dev time. But it took some time :)
We are also working on GraphQL support, with a Hasura-like API: https://docs.vaex.io/en/latest/example_graphql.html
I think GraphQL is easier in combinations with front end development, and you can tab-complete your way out. Early days for this sub-project, but I think very promising.
I think they solve a different problem, smaller data, data/relational integrity, data mutations. Dataframes can take shortcuts here.
If you want to do a serious benchmark, feel free to contact us. Github: https://github.com/vaexio/vaex/issues Email: (I'm easy to google).
Xarray is more about nd-arrays, less about ~tabular data.
Vaex is built from the ground up with the idea that you can never copy the dataset (1TB dataset was quite common). We also never needed distributed computing, because it was always fast enough, and thus never had to use dask (although we're eager to support it fully).
Also, vaex is lazy but tries to hide it from you. For instance, if you add a new column to your dataframe, it will only be computed when needed (taking up 0 memory). However, in practice, you're not really aware of that. This means it feels more like pandas (immediate results) than dask.dataframe (no .compute()/.persist() needed).
I would say they are all complementary, with small amount of overlap. Small data: use Pandas. Out of memory error: move to vaex. Crazy amount of data (100TB?) that will never fit onto 1 computer: dask.dataframe, or help us implement full dask support.
Vaex is much smarter with memory, it memory maps the data, nothing is allocated, all the memory is left to the OS for disk cache. This means you can have 10 users regularly restarting/running their program, without any (or minimal) harddrive activity. So I think it is a fair statement, and actually very conservative (see my other comments). An important message is to make people aware that working with a dataset this large is a pip install away, no need to spin up a cluster (yet).