21 karma · joined June 3, 2016
This is exactly how I would describe my experience. When I talk to others about polars now I usually quickly mention its fast up front, but then mostly talk about the API, its composability, small surface area, etc. are really what make it great to work with. Having these same semantics backed by eager execution, query optimized lazy API, streaming engine, GPU engine, and now distributed auto-magical ephemeral boxes in the sky engine just make it that much better of a tool.
df = (
pl.scan_parquet('hf://datasets/minimaxir/mtg-embeddings/mtg_embeddings.parquet')
.filter(
pl.col("type").str.contains("Sorcery"),
pl.col("manaCost").str.contains("B"),
)
.collect()
)Polars is awesome to use, would highly recommend. Single node it is excellent at saturating CPUs, if you need to distribute the work put it in a Ray Actor with some POLARS_MAX_THREADS applied depending on how much it saturates a single node.
Me (2 months later): Here it is.
Them:
This was captured well in their company announcement blogpost [0]:
> A strict, consistent and composable API. Polars gives you the hangover up front and fails fast, making it very suitable for writing correct data pipelines.
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Config doesn't have to be very complicated, I've used this for a while.
set -g mouse on # allow mouse
set -g history-limit 999999999 # unlimited history
set -sg escape-time 0 # vim esc response fasterWe ran into many unsolvable or 30m+ solvable envs with conda that mamba handled quickly.
The underlying solver can be used with conda directly as well, but I have not done that (https://www.anaconda.com/blog/a-faster-conda-for-a-growing-c...)
Will definitely try it out.
Jupyter Notebook: https://github.com/lambdalisue/jupyter-vim-binding Jupyter Lab: https://github.com/jwkvam/jupyterlab-vim
I’ve been tracking the Munx VR [0] app which is similar, but unable to try it out without a VR headset. They just announced desktop version support so going to give that a go soon along with this Nototo app.
~50 hair tie packets, ~5 beanies, a scarf, and some makeup.
My wife sometimes wears the light blue beanie they sent and the scarf. I still hear her say "I need a hair tie. Can't find one anywhere" every couple of days. Keep 'em coming guys.