HNHacker News
TopNewBestAskShowJobs

michalc

816 karma · joined April 7, 2016

https://charemza.name/
submissionscomments
michalc··on Write Better Commits, Build Better Projects
I guess since the workflow (very almost) requires the use of force pushing (with or without lease), it feels like the article hasn't presented a complete workflow people can follow, and so is a bit incomplete.
michalc··on Write Better Commits, Build Better Projects
Interestingly it doesn't seem to mention force pushing, which I think is needed if you've pushed a feature branch, and only then want to re-arrange its commits, which I think is quite a common workflow (at least for me).
michalc··on Ws4sqlite: Query SQLite via HTTP
And Python - https://github.com/michalc/sqlite-s3-query

(although this is specific to S3 rather than generic HTTP)

(shameless plug - made mostly by me)

michalc··on Additional Checksum Algorithms for Amazon S3
This might be the first real-world use of HTTP trailers I've ever seen
michalc··on Is your code too complicated?
I’ve become increasingly skeptical of both cognitive and cyclomatic complexity metrics in recent years.

I think for 2 reasons.

Firstly, they don’t account for complexity you need to solve your problem. And secondly, they seem to suggest that splitting up the same functionality over more functions, so each has a lower complexity, is “better” - this isn’t always the case, at least for me. Often I find following the flow/logic of loads of small functions more difficult than through lines in the same function.

The examples I’ve seen in articles such as this are always fairly clear cut “well yes of course these small re-written functions are clearer”, but they don’t really seem to consider the wider context of complex requirements?

I don’t really have an alternative metric to suggest though… I wish I did!

michalc··on SQLite Release 3.37.0
I have been particularly looking forward to that. Although I have to admit that I'm a bit... disappointed(?)... that previous versions of SQLite won't be able to open such tables (unless they use PRAGMA writable_schema=ON).

I wonder if SQLite didn't store the schema for a table as a "CREATE TABLE..." string, but instead something a bit more... structured, then such things could be added in a more backwards compatible way?

michalc··on Python – Writing large ZIP archives without memory inflation
Looks good! I've been thinking about making a writable version of https://github.com/uktrade/stream-unzip, but looks like you beat me to it!

(Full disclosure: I'm the main developer of stream-unzip)

michalc··on Certificate-Based Authentication for S3 with MinIO
Semi-related: I really like minio, and there is only so much I can complain about since it _is_ free…

… but I do wish they would implement being able to store objects at arbitrary keys. Some combinations of keys are not possible to store at the same time. And I guess to me it seems… odd… and maybe even frustrating?, that they say it was a “blunder” on AWS’s part to implement such a thing. [1]

For example, as far as I know, in minio you can’t have objects at both of the below keys:

red

red/blue

while in AWS S3, this is fine

[1] https://github.com/minio/minio/issues/10160

michalc··on Show HN: SQLite-s3vfs – Read and write SQLite databases on S3
Thanks!

(In fact, that project was one of the sources of inspiration for this one. I've now added that fact to the README)

michalc··on Show HN: SQLite-S3-query – Python function to query a SQLite database on S3
I could do this, and it certainly crossed my mind. But depending on the project, it could significantly increase the amount of code overall. It feels a feels a bit... heavy to me.

I can understand the AWS auth code... while there seem to be so many layers in boto3/botocore. There's a lot of code to add a few HTTP headers to each request. (It was a few years ago, but I posted about not using boto3 https://charemza.name/blog/posts/aws/python/you-might-not-ne...)

I might be suffering a little bit from not-invented-here syndrome admittedly... I don't know. But one of the things I really like about S3 is how much of its API is quite simple: it's just HTTP (I briefly wrote about _that_ as well https://charemza.name/blog/posts/http/s3/the-core-s3-api-is-...). My instinct is that in many cases, it's better to use a simple API directly rather than (what I see as) a more complex code base on top of it.

michalc··on Show HN: SQLite-S3-query – Python function to query a SQLite database on S3
There is certainly overlap with what's available in AWS/S3, so in many situations, if you're architecting a system, some of the things you suggest may be completely reasonable. In fact, in another project I'm involved in, we use S3's S3 Select to query CSVs extensively.

But I think there is a, or could be, a place for this:

- I don't think (but very happy to be corrected) that the things you mention can query SQLite files directly. So, there would have to be both a conversion of data, and then the query language used would not be SQLite's flavour of SQL. So, if you're using, or pro, using SQLite and wanting to keep the moving parts and languages down, then I _think_ there is a good argument for sqlite-s3-query to be used (with the caveat that it's a _very_ early version!)

- If you want, "light" dependency on AWS. There are a number of S3-compatible services that this could work on. Or even, the code could be tweaked to just use HTTP range headers, remove the dependency on versionId, which I think is only feature that makes this quite S3-specific. (Although, this would allow the case of errors/corruption if the database is replaced half way through a query...)

michalc··on Show HN: SQLite-S3-query – Python function to query a SQLite database on S3
You are right. I hold my hands up: I did not give credit to what made me think of doing this. I think I saw this (or at least, their blog post https://phiresky.github.io/blog/2021/hosting-sqlite-database...) on HN a few weeks back.

I have now given them credit.

michalc··on Show HN: Stream-unzip – stream unZIP files in Python
To check, do you mean Deflate64?

I think Zip64 _is_ supported by Python, and doesn't really involve anything at the zlib level(?)

(The reason I'm asking: a recent answer at https://stackoverflow.com/a/67579486/1319998 mentions Deflate64 and Windows)

michalc··on Faster web fonts
I actually think there are cases for this. Say, you have a small amount of CSS and/or the user will in all probability just look at the one page on the site.
michalc··on Faster web fonts
I've done this a few times as well. Another benefit of this is that there is neither FOIT nor FOUT.
michalc··on Ask HN: Solo developer, what's your recipe to start working?
I suspect depends a bit on what you're doing... but for me and the things I tend to work on, I tend to get scared by "big" projects that won't be "done" until some far off point in the future after loads of work, so my motivation is kept up by:

- Getting something "out there" + "working" as soon as possible. Even if _really_ rough, buggy, ugly, can't scale, only works in 1 case, needs a code change to work in any other etc...

- Keep it working with every change

- Every change is small. Potentially, teeny-tiny

Examples:

- Forking a project from GitHub. Have something working straight away! I've done a few project that literally have no lines of code in common with the original, except perhaps the license [so not appropriate for all projects]

- Releasing a website with whatever it is really early. Even if unstyled, no documentation, but does a tiny sliver of what you want it to do eventually. Then iterate, releasing the new version every single change.

- I think some people suggest avoiding tests in early stages of projects, but for me... some high-level tests on the "public facing" behaviour really helps give me the feeling that it's working as changes are made, and I'm never stepping backwards.

Things to avoid:

- Long running feature branches where lots of things are broken

- Refactoring without some sort of externally-facing feature in mind

- Low level tests of implementation details

- Working too much on "behind the scenes" stuff. You probably don't need a really fancy CI system that supports 100 developers deploying every day... [But: enough of a system for _you_ to release often is good]

michalc··on Ask HN: Anyone know any funny programming jokes?
My favourite:

"What do we want?"

"Now!"

"When do we want it?"

"Fewer race conditions!"

michalc··on The True Size Of Countries: Mercator projection
A few years ago I made

http://projections.charemza.name/

... click and drag a Mercator map to rotate the world "before" the Mercator projection is applied, to see a whole range of "equivalent" projections. None of them look like the world to me, but they're just as valid.

michalc··on Show HN: A simple app to make Ctrl+V paste unformatted text by default (Windows)
For me, CMD + Shift + V usually works
michalc··on Show HN: Parallelise pipelines of Python async iterables
Indirectly: the initial version was a single function, so could have been used in a decorator easily enough.

The issue is what happens on exceptions to avoid tasks hanging around forever. Exceptions propagate “forward” in the pipeline fairly automatically. However, getting the ones “behind” the exception to notice: that was tricky. The best I came up with was the bit of state wrapped up in buffered_pipeline that notices the exceptions and cancels the tasks.

michalc··on Show HN: Parallelise pipelines of Python async iterables
Hi, author here,

While yes, running a coroutine in a new task is in the standard library, that is only a component of what this library does, which is:

- given an async iterable,

- iterating over it in a new task, into a buffer,

- and then returning an iterable that yields values from this buffer as it's filled.

I don't _think_ this logic is in the standard library, at least not at https://docs.python.org/3/library/asyncio-task.html#asyncio.... from what I can see.

michalc··on Show HN: Parallelise pipelines of Python async iterables
Hi, author here,

I was slightly torn on the usage of the word "parallelise", in that yes, CPU-wise, no tasks will progress in parallel.

However, the point of the library is to allow certain parts of a pipeline to progress at the same time in the real-world. In the example, they are calls to asyncio.sleep, but in real cases, they could be HTTP requests. As in, the bytes of the request/response will be going across the wire at the same time, so the total wall-clock time of (certain) pipelines will go down.

So, I claim "parallelise" _is_ appropriate.

michalc··on Guide to Concurrency in Python with Asyncio
Shameless plug for what is essentially my own much shorter intro to asyncio: https://charemza.name/blog/posts/python/asyncio/I-like-pytho...
michalc··on Ask HN: What are good distance learning Bachelor's/Master's degrees one can do?
I’m currently enrolled on the Data Science, Technology and Innovation distance learning MSc at the University of Edinburgh.

So far, I like it: a variety of modules, and each I have taken so far has sparked potential dissertation ideas :-)

https://www.ed.ac.uk/bayes/about-us/our-work/education/data-...

michalc··on Mini HTTP Guide for Developers
Author here: thanks!

I was hoping to get across exactly that meaning of leaky: that HTTP shows information from, and allows control of, parts of the otherwise-abstracted-away lower-levels, e.g. TCP/IP. Will see if I can make that clearer.

michalc··on Why databases use ordered indexes but programming uses hash tables
Nit-pick / pet-peeve regarding Big O notation!

> Hash tables provide constant time O(1) access for single values, while trees provide logarithmic time O(log n) access. For single value lookups, this means hash tables are faster,

> for small n, the hash table is better, but for large n, the cost of that rare scan dominates, and the tree is better

The Big O notation describes how the number of operations scales as the size of the input tends to infinity: the asymptotic complexity. This means that an O(1) algorithm can include some fixed number of operations, even enough to add _years_ onto the runtime, and still be classed as O(1).

Essentially I don’t think it’s really enough to consider the asymptotic complexity of algorithms to compare practical runtimes. You need to know what the smaller-order / constant terms are.

michalc··on What Mac OS’s case-insensitive filenames teaches us
I think it also teaches that Postel’s robustness principle isn’t always a good thing.

If the MacOS filesystem was not case-insensitive, i.e. it was _less_ liberal with what it accepted, the reported issue wouldn’t have happened.

michalc··on Uber’s self-driving car could not detect pedestrians outside of a crosswalk
As a UK-resident, I find the omission of considering a pedestrian in the road quite unexpectedly, and unfortunately, overly car-centric.

There isn’t really the term “jaywalking” here. It’s just “crossing the road”. I’m not sure on exactly who has what legal responsibility, but it certainly feels like pedestrians should look out for cars when crossing, and drivers should look out for pedestrians.

michalc··on Simulated infant program in highschools doubled teen pregnancy rate (2016) [pdf]
“Parent or guardian consent forms had a tea bag stapled to the top of the form to enhance uptake.”

I couldn’t help but smile.

michalc··on Show HN: Mobius3 – Continual Sync to and from S3
Thanks!
← PreviousPage 3 of 4Next →