sa46

I also switched from Lastpass to 1Password. I did a mildly deep technical investigation into why Lastpass is slow on the browser. I found LastPass delays all page rendering by about 70ms. https://joe.schafer.dev/passing-lastpass/

We tried pg_upgrade going from Postgres 10 to Postgres 12 and it didn't work. An individual instance was about 8 TiB in size. We left it running for over a day to see if it would complete. Instead, we used an approach similar to logical replication described in the article.

To be fair, our use case was probably close to pathological for pg_upgrade. We had lots of TOAST data and dozens to hundreds of indexes per table.

If there's even a rough order to the underlying data, I'll buy their claim. On ordered data, a Postgres block-range index (BRIN) is often several orders of magnitude smaller than a B-tree index.

If the data is random, I suspect you're right and the PGM index is no-better than a B-tree index. Most data does have an order and would probably see similar gains.

I agree with most of the list. It's a good distillation of software development principles. In the spirit of HN, here's what I disagree with:

> Designing scalable systems when you don't need to makes you a bad engineer.

Creating overly scalable solutions is useful for learning, especially early on. I've done this.

> Despite being called "engineers," most decision are pure cargo-cult with no backing analysis, data, or numbers.

This is true in almost every human endeavour. It's not a useful opinion. Most decisions don't matter that much so adding data is a waste of time.

> Code coverage has absolutely nothing to do with code quality

I don't buy there's no correlation between code coverage and code quality. I'd much rather take code with >80% coverage than 0 coverage. My revised take is that the value of code coverage rapidly diminishes past 80% (maybe log scale-ish). I also think code coverage metrics are not especially useful outside of library code.

> 90% – maybe 93% – of project managers, could probably disappear tomorrow to either no effect or a net gain in efficiency.

Google tried getting rid of managers. It didn't work. Establishing lines of communications in large orgs is hard.

GPL2 doesn't prevent a closed fork of Linux. It requires that if you distribute your closed fork outside of your organization, you must also distribute the modified source under the GPL2 license. Large tech companies run their own forks of Linux without providing the source.

I’m with you in trying to avoid shell scripts. That said, shell scripts are especially convenient for simple glue scripts. No external dependencies and shell will run basically anywhere. Long isn’t necessarily bad for pipelines but anything over 100 lines is suspect.

Since you’re on btrfs, a copy-on-write file system, you should also investigate disabling full page writes [1] for Postgres WAL. I disabled it for our Postgres cluster on ZFS and got a nice reduction in write volume.

[1]: https://www.2ndquadrant.com/en/blog/on-the-impact-of-full-pa...

Since you’re on btrfs, a copy-on-write file system, you should also investigate disabling full page writes for Postgres WAL. I disabled it for our cluster on ZFS and got a nice reduction in write volume.

The helmet in the West Point emblem is the helmet of the Greek goddess Athena. I suspect any Spartan influence is incidental and superficial.

> The committee began with the creation of an emblem that consisted of a sword, a universal symbol of war, and the helmet of Pallas Athena. Athena, a fully armed mythological goddess, is associated with the arts of war, and her helmet signifies wisdom and learning.

http://usheraldicregistry.com/index.php?n=Registrations.2008...

Are there similar offerings for other clouds?

Your first sentence is quite tricky to parse due to the triple negative. I parsed it as:

Most people that email you will care if you ignore them.

Ironically, nickel was the name of the Google-internal, ill-fated replacement for GCL. I think it just lives on in blueprint files.

I also miss GCL, as quirky as it was. YAML composes poorly.

Can anyone recommend a good intro to debugging network problems assuming competent knowledge of most Linux sysadmin tasks?

I can't speak to doctors, but I know a few nurse practitioners and jobs will auto-reject if you don't have the correct specialty. A quick search reveals there's at least 13 specialties [1]. "We need an adult gerontology nurse practitioner, not orthopedics."

I thought this was interesting because a common programmer hiring model assumes you can train an exceptional programmer to your specific stack regardless of their existing specialty. I'd argue programming specialties are still broad, maybe frontend, backend, iOS, android, and embedded. Perhaps as the software development matures, we'll see more specialized fragmentation.

[1]: https://nursinglicensemap.com/advanced-practice-nursing/nurs...

Interesting,the Google results for "search engine" are heavily weighted by recency compared to Algolia.

Google looks like it's much better at non-title searches:

https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

https://cse.google.com/cse?cx=37e5b152f44307b8a&q=%22high-ca...

I think Datadog's trace_id is 64 bits which I thought was a good balance between efficiency and uniqueness. Pretty sure Dapper used 64 bits as well.

I also don't understand the purpose of the 16 bytes for the trace_id in the spec. That's a huge range of numbers. Anyone know the rationale?

I looked into diffy but the license was changed from Apache to AGPL about a year ago in what appeared to be a monetization scheme based on the links to the support company in the REAMDME. https://github.com/opendiffy/diffy/commit/0848f9888f59aded4d...

Does PGO support reproducible builds? If you pass the same profile at compile time do you get the same binary?

Presumably:

> This second one is instructive: the EU is taking the view that Apple has a monopoly of payment on the iPhone. Market definition is everything. I-am-not-a-lawyer, but I don’t see how Apple can win on Spotify (or Kindle), and I don’t think it should.

And the final sentence:

> Apple finally won the argument with users’ wallets, and that means it’s not niche anymore - Apple has become the navy, and different rules apply.

> 15x native compression

That is a rather remarkable compression ratio. Do you have a link to technical details? The lightweight compression outlined in the Gorilla TSDB and implemented in Uber's m3db is 11x which I thought was best in class: https://cs.brown.edu/courses/csci2270/papers/gorilla.pdf

Sure they are. Sum types usually mean a pattern matching syntax, destructuring, and a different declaration syntax. Rust’s match keyword, for example, is a powerful construct covering guards, destructuring, and different types of binding. Non-nullability requires generics for a result type which is a huge source of complexity. Generics impact parsing, code gen and runtime efficiency.

Interesting, I hadn't come across Kusto. Looks like a competitor to Snowflake, perhaps with stronger analysis capabilities. Do you know if there's an academic paper? I found a white paper: https://azure.microsoft.com/mediahandler/files/resourcefiles...

Does anyone know of generative art for gothic style [drop caps]? I thought it would be neat to generate the curves from a letter instead of relying on a font. I know Gwern.net uses some neat tricks to load a subset of the drop-caps fonts.

[drop caps]: https://en.wikipedia.org/wiki/Initial

[gwern]: https://www.gwern.net/About#implementation-details

> Generally we provide both row and column based access despite being column store. So anything a traditional database does we can do too.

Does quest DB use the same physical data layout for the row based access? My understanding was providing an entire row of data from a columnar file is expensive because you have to assemble the row from num_columns different slices of the columnar file.

How timely! I've done a deep dive into column store databases for the past couple of weeks. Reading through the Quest docs, I'd give it the following characteristics. Are these accurate?

- single node database, not [yet] distributed

- primary focus is time-series data, specifically in-order time series data (the `designated timestamp` extension)

- physical data layout is an append-only column store

- Implements a small subset of SQL with some affordances for time series (LATEST BY, SAMPLE BY).

- Doesn't support explicit GROUP BY or HAVING clauses. Instead, questdb implicitly assumes GROUP BY or HAVING based on presence of aggregation functions in the select clause.

- Small standard library of functions: only 4 text functions.

Based on these characteristics it seems the quest db is well positioned against Influx. It's probably faster than Timescale DB but significantly less flexible given that Timescale has all of Postgres behind it. Quest DB might eventually compete with clickhouse but it's long ways out given that it's not distributed and implements a much smaller subset of SQL.

I'd love to get any insight into join performance. Quite a few column stores handle large joins poorly (clickhouse, druid).

Not main-less but you can run go scripts using a shebang with some elbow grease. https://blog.cloudflare.com/using-go-as-a-scripting-language...

> I have to use LastPass.

I recently switched from LastPass to 1Password because of the added latency from Lastpass. Lastpass adds about 70ms to first contentful paint on example.com. 1Password, on the other hand, runs after the painting is done so it doesn't block rendering. I polished up a blog draft I had lying around about switching to 1Password: https://joe.schafer.dev/passing-lastpass/

> I'd be much more interested to see how extensions like Evernote or LastPass increase the time it takes for a real webpage (e.g. "nytimes.com") to finish painting the viewport not including ads.

I reinstalled Lastpass to test on nytimes.com. It takes 58ms to evaluate onloadwff.js (the Lastpass entry point) before any content is rendered.

> Ansible has its own set of features that are largely abstracted out by its YAML playbook syntax.

I don't understand what YAML has to do with abstraction. The example from pyinfra is pretty much the same as the ansible equivalent. Whether the feature set is comparable is a different question (I agree with you, it's probably nowhere close to ansible).

    apt.packages(
        {'Install iftop'},
        'iftop',
        sudo=True,
        update=True,
    )

> If you feel YAML is too complicated for you

Ansible YAML is a poor substitute for a real language once you add conditional logic to the mix. Another huge downside is that composition in YAML is much harder than in any programming language. A Python (or dhall, or cue) library that compiled to ansible would be pretty swell.

Speaking of high cardinality metrics, what are good options that aren’t as custom as map reduce jobs and a bit more real time?

We killed our influx cluster numerous times with high cardinality metrics. We migrated to Datadog which charges based on cardinality so we actively avoid useful tags that have too much cardinality. I’m investigating Timescale since our data isn’t that big and btrees are unaffected by cardinality.

> The specific profile was a Samsung Galaxy S4 over 3G.

1.7 seconds is pretty reasonable for a 7 year-old phone on 3G.