HNHacker News
TopNewBestAskShowJobs

quodlibetor

94 karma · joined October 28, 2013

submissionscomments
quodlibetor··on Mise: Monorepo Tasks
yeah artifact caching is the obvious interpretation of caching when you're used to being compared to bazel, but the conversation was conflating "cache artifacts" and "cache should-run?" features.
quodlibetor··on Mise: Monorepo Tasks
Mise does do local-only Make-similar task caching, if you specify sources and outputs: https://mise.jdx.dev/tasks/task-configuration.html#sources

If you specify sources but not "outputs" then mise will auto-track whether sources have been modified.

I requested the auto-track feature to speed up Docker builds a pretty long time ago, and it's been fantastic.

quodlibetor··on Polars Cloud and Distributed Polars now available
Thanks for the insight!
quodlibetor··on Polars Cloud and Distributed Polars now available
> Not much polars can do about that in Rust

I'm ignorant about the exact situation in Polars, but it seems like this is the same problem that web frameworks have to handle to enable registering arbitrary functions, and they generally do it with a FromRequest trait and macros that implement it for functions of up to N arguments. I'm curious if there are were attempts that failed for something like FromDataframe to enable at least |c: Col<i32>("a"), c2: Col<f64>("b")| {...}

https://github.com/tokio-rs/axum/blob/86868de80e0b3716d9ef39...

https://github.com/tokio-rs/axum/blob/86868de80e0b3716d9ef39...

quodlibetor··on Hurricane category 6 could be introduced under new storm severity scale
And the following criteria:

(a) The final category can never be lower than the highest hazard-based category;

(b) The TCSS should adequately reflect the case of high potential risk of two or more hazards. We consider a hazard of high risk when its respect- ive category is classified as 3 or higher (equal to the definition for a Major Hurricane on the SSHWS). Whenever (at least) two high risk haz- ards have the same category value and the third hazard has a lower category value, the final category should increment the highest hazard- based category. This implies that a TC scoring a Category 3 on both wind and storm surge, and a Category 1 on rainfall, will be classified as a Category 4.

(c) To warn the general public for an event with multiple extreme hazards, a high-risk TC can be classified as a Category 6 when either 1. at least two of the hazard-based categories are of Cat- egory 5; or 2. two categories are of Category 4, and one of Category 5.

quodlibetor··on The anti-abundance critique on housing is wrong
My understanding is that vacancy means available units for rent. So, plausibly, if you say 50 of the 100 units in your building aren't available for rent because you say they're being painted then they don't contribute to the vacancy of your building.

That's almost the exact opposite of your definition, but I agree that a 1.4% vacancy rate means there's almost nothing available for rent.

I'm having trouble finding an official definition from a source that reports them, but my definition matches things that I can find online, eg https://www.brickunderground.com/rent/vacancy-rate-what-does...

quodlibetor··on The anti-abundance critique on housing is wrong
Yeah I know it's among the lowest in the world, it's still an ~order of magnitude higher than a few tenths of a percent, which would be shocking for the reasons you mention.

My point though was just that I've seen arguments that these numbers can be manipulated, and the city's own data doesn't make sense by itself: either the 1.4% number is wrong or the slowly recovering population estimate is wrong. Especially considering the 60,000 housing units (representing 2% growth) created.

quodlibetor··on The anti-abundance critique on housing is wrong
> a few tenths of a percentage point of NYC

Feb 2024 (last year there's data, I think) was a record low and it was 1.4% empty, according to NYC[1].

But I don't really know the methodology, and according to other nyc gov data it's surprising, since we still haven't recovered our population from COVID[2].

The first statistic (housing pressure) is based on population growth, but the NYC population statistics suggest still meaningful population loss since 2020.

I have seen articles in the past that suggest that apartment vacancy rates in NYC are self-reported and misleading at best, but I don't really understand how that would work and I can't find any sources on that now.

It's also my understanding that some classes of landlords can mark empty apartments as income losses, basically or partially making up for the loss of revenue in tax rebates. But that's also not something I understand well, just something I have seen asserted.

[1]: https://www.nyc.gov/site/hpd/news/007-24/new-york-city-s-vac... [2]: https://s-media.nyc.gov/agencies/dcp/assets/files/pdf/data-t...

quodlibetor··on S5cmd: Parallel S3 and local filesystem execution tool
I recently wrote a similar tool focused more on optimizing the case of exploring millions or billions of objects when you know a few aspects of the path: https://github.com/quodlibetor/s3glob

It supports glob patterns like so, and will do smart filtering at every stage possible: */2025-0[45]-*/user*/*/object.txt

I haven't done real benchmarks, but it's parallel enough to hit s3 parallel request limits/file system open file limits when downloading.*

quodlibetor··on Some of us like "interdiff" code review
I have been chasing the gerrit code review high since I left a company that used it almost 5 years ago.

Stacked pull requests are usually what people point to to get this back, but this article points out that _just_ stacked pull requests don't handle it correctly. Specifically with github, you can't really see the differences in response to code review comments, you just get a new commit. Additionally, often github loses conversations on lines that have disappeared due to force pushes.

That said, I have a couple scripts that make it easier to to work with stacks of PRs (the git-*stack scripts in[1]) and a program git-instafix[2] that makes amending old commits less painful. I recently found ejoffe/spr[3] which seems like a tool that is similar to my scripts but much more pleasant for working with stacked PRs.

There's also spacedentist/spr[4] which gets _much_ closer to gerrit-style "treat each commit like a change and make it easier for people to review responses" with careful branch and commit management. Changes don't create new commits locally, they only create new commits in the PR that you're working on. It's, unfortunately, got many more rough edges than ejoffe/spr and is less maintained.

[1]: https://github.com/quodlibetor/dotfiles/tree/main/dot_local/... [2]: https://github.com/quodlibetor/git-instafix/ [3]: https://github.com/ejoffe/spr [4]: https://github.com/spacedentist/spr

quodlibetor··on Show HN: Inshellisense – IDE style shell autocomplete
I'm pretty sure that the scripts generated by inshellisense are CRLF, and the carriage returns aren't recognized by unix shells.

You should be able to fix it with:

    vi $HOME/.inshellisense/key-bindings.zsh -c "set ff=unix" -c ":wq"
quodlibetor··on Testing the memory safe Rust implementation of Sudo/Su
The dependency list[0] looks pretty reasonable, AFAICT the overwhelming majority of that line-of-code count comes from autogenerated Windows API methods.

edit actual counts:

  $ cargo vendor && cd vendor && {
      for p in * ; do
      echo -n $p
      tokei $p | rg '^\s+Rust'
  done } | sort -n -k 4 | tabulate
  ----------------------------  ----  ---  ------  ------  ----  ----
  errno-dragonfly               Rust    2       9       8     0     1
  windows_aarch64_gnullvm       Rust    2      11       9     0     2
  windows_aarch64_msvc          Rust    2      11       9     0     2
  windows_i686_gnu              Rust    2      11       9     0     2
  windows_i686_msvc             Rust    2      11       9     0     2
  windows_x86_64_gnu            Rust    2      11       9     0     2
  windows_x86_64_gnullvm        Rust    2      11       9     0     2
  windows_x86_64_msvc           Rust    2      11       9     0     2
  winapi-i686-pc-windows-gnu    Rust    2      25      13    12     0
  winapi-x86_64-pc-windows-gnu  Rust    2      25      13    12     0
  windows-targets               Rust    1      54      46     3     5
  output_vt100                  Rust    2      67      55     0    12
  cfg-if                        Rust    2     164     131    16    17
  instant                       Rust    4     316     260     6    50
  countsctor                    Rust    2     331     254    21    56
  errno                         Rust    5     375     280    41    54
  diff                          Rust    4     561     485     9    67
  autocfg                       Rust    9     702     558    41   103
  yansi                         Rust    7     741     627     3   111
  signal-hook-registry          Rust    3     818     566   150   102
  fastrand                      Rust    4     830     710    16   104
  hermit-abi                    Rust    4     847     601     5   241
  pretty_assertions             Rust    5    1231    1072    33   126
  glob                          Rust    2    1589    1291   113   185
  bitflags                      Rust   20    1715    1373   105   237
  unicode-ident                 Rust   11    1794    1697    36    61
  signal-hook                   Rust   17    1969    1520   147   302
  tempfile                      Rust   15    2367    1928   102   337
  quote                         Rust   17    2458    1979   148   331
  redox_syscall                 Rust   23    3595    2996    83   516
  log                           Rust    9    3635    2970    97   568
  io-lifetimes                  Rust   15    4218    3605    80   533
  proc-macro2                   Rust   17    5286    4514   139   633
  cc                            Rust   13    5861    4767   488   606
  rustix                        Rust  236   39927   33837  1467  4623
  syn                           Rust   92   51956   48946   493  2517
  libc                          Rust  224  109836   99688  2073  8075
  linux-raw-sys                 Rust   61  145628  145455    84    89
  winapi                        Rust  405  179933  176630  3299     4
  windows-sys                   Rust  281  497624  497608     4    12
  ----------------------------  ----  ---  ------  ------  ----  ----
[0]: https://github.com/memorysafety/sudo-rs/blob/60985b2f5f7ffa8...
quodlibetor··on Correct Git commits with git-autofixup
If you start doing a commit (via `c` in the magit status buffer, with the standard semantics of "you're going to commit everything that's currently staged") you can press capital F for an instant fixup, or capital S for instant squash.

When you press either of those, magit pops up a commit picker which shows the current git log. Selecting a commit will then instantaneously apply your staged changes to the selected commit. It's much simpler than any of the other workflows I've seen in response to your question.

The gif in this repo (for a tool I made that simulates this behavior as a cli tool for some jealous coworkers) tries to show the workflow: https://github.com/quodlibetor/git-fixup

That said, this _doesn't_ support the "automatically figure out which commits to apply hunks to" workflow. I personally find that I use both workflows depending on the nature of my changes.

quodlibetor··on Materialize Raises a $32M Series B
This is a fantastic comment!

One small thing: we do now have tables[1]! At the moment they are ephemeral and only support inserts -- no update/delete. We will remove both of those limitations over time, though!

[1]: https://materialize.com/docs/sql/create-table/

quodlibetor··on Materialize Raises a $32M Series B
Ha! Your blog post was one of the reasons that I trusted in the future of Materialize enough to decide to work here!

I agree, that is exactly the problem that I, in particular, think we are solving.

quodlibetor··on Dijo: A terminal-based habit tracker written in Rust
If this is all this is it actually doesn't look like a crazy amount of work to make cursive look good: https://github.com/NerdyPepper/dijo/blob/8b91a7c0b3d9bd4fac3...

I've avoided looking too deeply into cursive in the past because I naively assumed it would be difficult to make it look like like anything other than a late-90s BIOS, but this is exciting.

quodlibetor··on Eventual Consistency isn’t for Streaming
I didn't downvote you, but as an outside observer I can see that folks might take issue with the fact that you start with a tl;dr, suggesting that you are summarizing the whole article, followedan in-depth analysis, and then ending by stating that you didn't read the whole thing.
quodlibetor··on Recently minted database technologies that I find intriguing
I'm not sure exactly how fast FAST REFRESH ON COMMIT is, but we at materialize are very fast, and getting faster. Once data is in materialized we can process streams to multiple downstream views with millisecond-level latencies. I'm working on improving our benchmarking capabilities so that we can provide less qualified answers to the question of "how fast is materialized?"

Much more interesting than our speed, though, in my opinion, is the fact that you can use materialized as the place where you do joins _across_ databases and file formats. It's particularly interesting in a microservices environment, where you may have a postgres db that was set up by one team and a mysql db that was set up by another team and the only thing you care about is doing some join or aggregate across the two: with materialized (and debezium) you can just stream the data into materialized and have continuously up to date views across services. Combine this with a graphql or rest postgres api layer (like hasura) and a large amount of CRUD code -- entire CRUD services that I've worked on in the past -- just disappears.

quodlibetor··on Ask HN: What startup/technology is on your 'to watch' list?
This is exactly what we do! This is a walkthrough of connecting a db (these docs are for mysql, but postgres works and is almost identical) via debezium and defining views in materialize: https://materialize.io/docs/demos/business-intelligence/
quodlibetor··on How Netflix uses Druid for realtime insights
> if you give it a query that only requires certain result rows from one of its mat views, then Materialize is only going to compute the intermediate rows

This is absolutely correct!

> You can just have a bunch of “the same” Materialize node (i.e. every node just freestanding clone of a template node, with exactly the same sources and matviews) and then hit them with the parts of a map-reduce query

This should work, but we have been thinking about it/testing it differently internally. In general you should be able to create materialized views on different "shards" that have different `where` conditions, allowing you to control memory that way. This technique does require data that is actually partitionable in this way, same as it must be partitionable in mapreduce.

> this is all irrelevant the moment you write a query that needs a pure reduce

Of course, with materialize's sinks you can spin up a bunch of `materialized`s and connect them for a final reduce after data has gone through e.g. kafka or shared files. Being able to write joins and aggregates across heterogenous sources makes this kind of workload actually pretty pleasant.

quodlibetor··on How Netflix uses Druid for realtime insights
> materialize has to be able to keep all of its state in memory

For now. We have a pretty good idea of what needs to be done to shed state to disk, and have designed to be able to implement it. We expect it to "just" be a matter of putting in the engineering effort.

quodlibetor··on Reasons Python Sucks
> I've come across a few cases, when pushing python's type annotations to their limits, that force you to put the type names in string quotes

With pep-0563, python3.7, and `from __future__ import annotations` this should no longer be necessary. Those are some pretty detailed caveats, but I have been using it in places where I can and it is so nice.

quodlibetor··on Rust Ownership Explained with Python
You're probably thinking of http://pythontutor.com/ for python. It's one of my absolute favorite instructional resources for any language.

Birdseye[1] looks like it is trying to be a similar kind of thing for debugging Python, which is amazing.

I don't know of anything for anything similar for Rust or anything with similar characteristics (C/++).

[1] https://github.com/alexmojaki/birdseye

quodlibetor··on Rust's 2018 roadmap
I'm not experienced in this space, but you might be interested in the Real-Time For the Masses[1] project. The author of that is the lead of the embedded devices working group.

1: http://blog.japaric.io/rtfm-v2/

quodlibetor··on Why writing a linked list in safe Rust is so damned hard
Linked lists are in the standard library[1]. The source code is linked to from that page (`src`, on the top-right corner), although the implementation is optimized so it's probably not a great learning resource.

If, for some reason, you actually need a linked list there are tons of implementations available on crates.io[2].

It's not that there aren't linked lists, and it's not that they can't be implemented as efficiently as in C, it's that implementing them is an inherently tricky problem where the obvious implementation can't be proved by Rust's type system so they're an extremely unpleasant way to learn Rust.

1: https://doc.rust-lang.org/std/collections/struct.LinkedList.... 2: https://crates.io/search?q=linked%20list

quodlibetor··on Why is Rust difficult?
The nice thing about Rust not having Async IO built into the language is that arbitrary third-party implementations are possible on a level playing field with the async framework being developed by the core team.

For example, there is the May[1] concurrency library. It provides alternative implementations of the standard library's IO interface, but does Go's automatic suspend/resume, so it still looks like blocking code, which is nice. From what I can tell it's still early days on a one-person project, but it is interesting, at least.

[1]: https://blog.zhpass.com/2017/12/23/may-announcement/

quodlibetor··on The Death of Microservice Madness in 2018
I believe this is a paraphrase of Conway's law. The Wikipedia article lists three studies in favor of it, so there's that.

https://en.wikipedia.org/wiki/Conway%27s_law

quodlibetor··on Show HN: Promptui – Rich UI elements for cli prompts in Go
prompt_toolkit is, IMO, the standard by which all of these types of libraries should be measured.

It's incredibly easy to get started with, and lets you build crazy wonderful things like pt_python and pgcli/mycli.

quodlibetor··on Pytest 3.2.0 released
Interesting. pytest's handling of `assert foo == bar` is one of my top two favorite things about it. My other favorite thing being fixtures.
quodlibetor··on Pytest 3.2.0 released
especially in combination with pytest-mock.
Page 1 of 2Next →