HNHacker News
TopNewBestAskShowJobs

pplonski86

7,520 karma · joined March 9, 2016

New generation of data science tools

https://mljar.com

https://github.com/mljar/mercury

https://github.com/mljar/mljar-supervised

https://runmercury.com

meet.hn/city/pl-Łapy

submissionscomments
pplonski86··on RStudio Is Becoming Posit
Quatro is a tool for converting notebooks into shareable formats (like PDF, HTML, Docx). Notebooks are very hard to share. You can't send it to your stakeholders, because Python code will scare them. You can use Quatro to convert notebooks into other formats and easily hide the code. It is like nbconvert on steroids.

I'm working on solution for notebooks sharing with non-technical users. Similar to Quatro it has YAML header, but it can also add widgets to the notebook and serve notebook as interactive web app. My framework is called https://github.com/mljar/mercury

pplonski86··on Ask HN: How to sell an open-source software company?
What are the numbers bebhind the busniess? What is ARR, number of customers, team size. My friend is looking for a business with templates.
pplonski86··on Ask HN: Can I see your cheatsheet?
Jupyter Notebook shortcuts https://github.com/mljar/data-science-cheat-sheets/tree/main...
pplonski86··on Automated PDF Reports with Python Notebooks
The Mercury uses `nbconvert` internally to convert notebooks, so you don't need to execute them manually. It also adds additional features to make it more user-friendly:

- you can add widgets (parameters) to the notebook, without rewriting the notebook,

- you can easily schedule the notebook,

- you can convert notebook to PDF with mouse click,

- you can serve interactive HTML notebooks with a Django-based server.

pplonski86··on Automated PDF Reports with Python Notebooks
Mercury has a different architecture. The Voila keeps the live kernel and connection to UI (using Tornado framework). To use Voila you need to add widgets (with ipywidgets) to the notebook (mix UI code with analytics code). The Mercury generates UI based on the YAML header (very simple, no need to mix UI with analytics code). When user tweaks widgets values, the whole notebook is executed with new parameters and converted with nbconvert (using Django + Celery). Mercury can serve multiple notebooks to multiple users on one server. It has option to export notebook to PDF or HTML. You can schedule the notebook execution with crontab string (for example `schedule: '30 8 * * 1-5'`) and add email notifications. You can easily add authentication to notebooks. It was designed to make notebook sharing fast and easy.

What is more, I'm thinking about including Voila into Mercury. So you will be able to serve Voila apps with Mercury. The end-goal is to make notebooks sharing easy.

pplonski86··on Automated PDF Reports with Python Notebooks
What do you hate the most about jupyter notebooks?
pplonski86··on Automated PDF Reports with Python Notebooks
Right now it is flowing the notebook html. It should be possible to add some layout library (Python library that will do HTML+CSS to get nice layout).

Recently I did similar for displaying numbers in the notebook as a good looking boxes. I created a small Python package that takes the number and creates a box with borders with HTML+CSS. It is pretty handy for building dashboards in Python. The package name is Bloxs https://github.com/mljar/bloxs

pplonski86··on Automated PDF Reports with Python Notebooks
Hey! Author here. I'm working on open-source framework called Mercury. I'm building it to make notebooks sharing easy, especially with non-technical users. The Mercury can turn Python notebook to web application, dashboard, presentation, REST API or report. It has option to easily hide the code, schedule automatic execution, convert to PDF, and send email notifications. The Github repo: https://github.com/mljar/mercury
pplonski86··on Automated PDF Reports with Python Notebooks
There are many ways to build PDF reports with Python. The notebook approach gives some WYSIWYG while report building. Additionally, when used with Mercury framework you get scheduling and email notifications.
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
It might be a solution but not all python packages work with WebAssembly. The next problem might be connection to external data sources. You cant keep secrets in frontend code.
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
The Mercury is available with AGPL license (OSS version) and with commercial license with additional features and dedicated support (Pro version).
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
The REST API functionality allows to execute the notebook by POST request. It is powered by DRF. You can read more in the docs https://mercury-docs.readthedocs.io/en/latest/notebook-as-re...
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
The demo is hosted on free Heroku dyno with 0.5 GB RAM. This is version 1.0 and still many things can be improved in the Mercury framework.
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
With Mercury you can:

- host multiple notebooks,

- you can easily hide code,

- you can schedule notebooks,

- you can download notebooks as PDF,

- you can easily add authentication to the notebooks.

It all depends on your needs and project requirements.

pplonski86··on How to create a dashboard in Python with Jupyter Notebook
My email is piotr at mljar dot com
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
Thanks! Mercury's goal is to make notebook sharing as easy as possible for creators (developers) and viewers (non technical users).
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
I would say it depends on data type and size. There are data sizes that would not fit in spreadsheets and need to be presented in a different way.

BTW, I love spreadsheets. Spreadsheets + Python is very powerful connection :)

pplonski86··on How to create a dashboard in Python with Jupyter Notebook
- What is your project? is it from data science space?

- Selling OSS might be a great idea. Didn't consider it. I thought the Personal Pro version ($199/y) will be good for persons that would like to support the project or just need a support/guidance.

- Yes, I do consulting services. I don't treat support hours in similar way as consulting. The support is for product that I love working on and I love supporting people using it.

- Django - I'm pretty good at Django. What is more, I wanted to use Celery for processing background tasks and scheduling. This gives an architecture that can handle large loads. The frontend is in React with TypeScript.

- Speed can be improved. There are many ways to better handle that (it is version 1.0). You can add Plotly charts to make presentation more interactive. The advantage of Mercury is that you can recompute all slides if needed.

pplonski86··on How to create a dashboard in Python with Jupyter Notebook
I love your attitude!
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
Mercury works very well with Plotly (here is a demo https://mercury.mljar.com/app/7). If you need to choose between Flask+Plotly and Mercury I would strongly recommend Mercury because of the speed of development.

What do you mean by dynamic data? Do you need automatically fetch new data in time intervals? (Mercury has scheduling and auto-update built-in with 1 minute lowest time interval).

pplonski86··on How to create a dashboard in Python with Jupyter Notebook
R Shiny is fantastic solution. To be honest, the Mercury is heavily inspired by R Markdown.
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
Yes, that's the solution but only if you share with people that understand Python and if you are allowed to upload code + data to the cloud.
pplonski86··on How to create a dashboard in Python with Jupyter Notebook
You are lucky! There are many people that don't want to touch any code. Setting up a Python environment locally might be a big pain. That's a huge blocker for notebooks sharing.

I treat dashboard as visualization part. What is more, visualized data is used to trigger further actions - for example, send alert email. Yes, I found them useful.

pplonski86··on How to create a dashboard in Python with Jupyter Notebook
Mercury has a different architecture. The Voila keeps the live kernel and connection to UI (using Tornado framework). To use Voila you need to add widgets (with ipywidgets) to the notebook (mix UI code with analytics code).

The Mercury generates UI based on the YAML header (very simple, no need to mix UI with analytics code). When user tweaks widgets values, the whole notebook is executed with new parameters and converted with nbconvert (using Django + Celery). Mercury can serve multiple notebooks to multiple users on one server. It has option to export notebook to PDF or HTML. You can schedule the notebook execution with crontab string (for example `schedule: '30 8 * * 1-5'`) and add email notifications. You can easily add authentication to notebooks. It was designed to make notebook sharing fast and easy.

What is more, I'm thinking about including Voila into Mercury. So you will be able to serve Voila apps with Mercury. The end-goal is to make notebooks sharing easy.

Mercury github repository https://github.com/mljar/mercury

pplonski86··on How to create a dashboard in Python with Jupyter Notebook
The Mercury framework was designed to make notebooks sharing easy. You don't need to build UI. Just define the widgets in the YAML header. Widgets are directly connected to variables in Python code. You can share notebook with non-technical users and hide the code (show-code: False). The user can tweak widgets and execute the notebook.

Additional features for sharing:

- user can download notebook as PDF or HTML,

- you can schedule notebook execution, and send executed notebook as PDF attachment in email notification,

- adding login view to notebooks is simple, just add `share: private` to YAML

pplonski86··on Show HN: Reddit search engine for startup founders
Is there a similar service based on StackOverflow?
pplonski86··on 1x Engineer
there is a button at the top of the website to switch the font
pplonski86··on Surprising ways how to use Jupyter Notebook
Thank you! I think that the full potential of computational documents created with Jupyter is not used. The idea of mixing prose in Markdown and code (Python, Julia, R, ...) is very appealing.

I wish that Jupyter Notebook and Python was as easy to use as Excel.

pplonski86··on Donald Knuth was framed (2020)
Why? Isnt enough if you mix markdown with code? What are other tools for LP?
pplonski86··on Donald Knuth was framed (2020)
The literate programming is very under rated in Jupyter Notebook. It is mainly used for experiments. I hope that in the future there will be more applications in notebook for example in automation. Can you imagine notebooks replacing services like zapier?
← PreviousPage 6 of 13Next →