sa46

Chrome dev tools are an engineering marvel; the downside is it’s quite hard to find the perfectly shaped tool.

I spent about a week tracking down a memory leak in our web app. About 5 days in I discovered you could compare heap snapshots which led to the root cause of a chrome bug with hidden classes. The fix was to move object spread to the bottom of an object literal.

Here’s the bug. It was a fun one to hunt: https://crbug.com/v8/13303

His BillG story is a really fun read: https://www.joelonsoftware.com/2006/06/16/my-first-billg-rev...

Threads can certainly run in parallel with one another if the OS schedules them on different cores. I did a quick experiment and the main thread and worker threads run in parallel.

https://github.com/jschaf/web-worker-experiment/

> You don't get extra calculation performance with web workers

The primary purpose of web workers is extra calculation performance. From MDN:

> Workers are mainly useful for allowing your code to perform processor-intensive calculations without blocking the user interface thread

I thought the primary purpose of web workers was that the browser can run the workers in parallel to the main thread. As the spec says:

> [Web workers] allow long tasks to be executed without yielding to keep the page responsive

Most languages include unstructured logging libraries in the standard library, including Go. Structured logging is usually provided by third party libraries.

> Yeah, TIMESTAMPTZ expressions don’t index nicely because they’re nondeterministic

Could you expand on why it's not deterministic? I thought timestamptz was 8-byte microseconds since the epoch. Is the problem that the timestamptz uses the server timezone?

Joins, however, scale exponentially if you only have full table scans.

There was an effort within Google SRE to use Go instead of bash for all the typical usages of bash. [1]

Cloudflare uses Go similarly as a scripting language. [2]

[1]: https://go.dev/solutions/google/sitereliability [2]: https://blog.cloudflare.com/using-go-as-a-scripting-language...

For my specific setup and a single row lookup, the ORDER BY ... LIMIT 1 is faster (0.1 ms vs. 1.2 ms).

The ORDER BY ... LIMIT 1 one is the same as the DISTINCT ON query, but the DISTINCT ON can return more than one resource.

Ah, I've had this same use case: I store many versions of a resource from an external API, but I typically only want the most recent version. I've used the following techniques:

Select the ID with the max sync_token. Easiest for Postgres to optimize, assuming you have a primary key of (id, sync_token).

    SELECT *
    FROM external_api
    WHERE id = :my_id 
      AND sync_token = (SELECT max(sync_token) FROM ext_api WHERE id = :my_id)

Define a view using DISTINCT ON. Convenient for ad-hoc querying. Postgres usually figures out it can use the primary key to avoid a full-table scan.

    SELECT DISTINCT ON (id) *
    FROM external_api
    WHERE id = :my_id 
      AND sync_token = (SELECT max(sync_token) FROM ext_api WHERE id = :my_id)
    ORDER BY id, sync_token DESC

For tricky predicates, I use a trigger to track the most recent resource in a separate table. This is a hacky version of incremental view maintenance. [1]

[1]: https://wiki.postgresql.org/wiki/Incremental_View_Maintenanc...

What does “Ceci n'est pas une pipe” mean in this context? I ended up on https://en.m.wikipedia.org/wiki/The_Treachery_of_Images but I’m not sure what it means here.

All the way! I was 1-505 PIR.

> dumb Joe that keeps erasing the codes out of the radio by turning on the truck without putting it on standby

Oh man, I have nightmares about comms. Juggling frequencies while traversing areas of operation, trying to get satcom working in mountainous terrain with jammers, and depending on the finicky windows-tablet encryption device, was not a fun time.

Around 2014 or so, we (light infantry) had field telephones on the books, but we only used them once for a field exercise to practice assuming all modern tech was down.

Some additional techniques for triggers I've found helpful:

- Triggers for validation are awesome. Avoid triggers for logic if you can help it -- harder to debug and update than a server sending SQL and easier than you might think to cause performance problems and cascading triggers.

- Use custom error codes in validation triggers and add as much context as possible to the message when raising the exception. Future you will thank you.

    RAISE EXCEPTION USING
      ERRCODE = 'SR010',
      MESSAGE = 'cannot add a draft invoice ' || new.invoice_id || ' to route ' || new.route_id;

- Postgres exceptions abort transactions, so if using explicit transactions, make sure you have a defer Rollback() so you don't return an aborted transaction to the server connection pool.

- For better trigger performance, prefer statement-level triggers [1] or conditional before-row-based triggers.

[1]: https://www.postgresql.org/docs/current/trigger-definition.h...

I explicitly want single-threaded execution within a tenant. After writing the original post, I figured out how to parallelize across tenants. The trick is to limit the query to only look at the next pending job for each tenant, which then allows for using SKIP LOCKED to process tenants in parallel:

    SELECT job.job_id, rm.data
    FROM qbo.transmit_job job
      JOIN resource_mutation rm USING (tenant_id, resource_mutation_id)
    WHERE job.state = 'pending'
      AND job.job_id = (
         SELECT qj2.job_id
         FROM qbo.transmit_job qj2
         WHERE job.tenant_id = qj2.tenant_id
           AND qj2.state = 'pending'
         ORDER BY qj2.create_time
         LIMIT 1
      )
    ORDER BY job.create_time
    LIMIT 1 FOR NO KEY UPDATE OF job SKIP LOCKED

> you should just use Temporal

Do you mean the Airflow-esque DAG runner? I prefer Postgres because it's one less moving part, I like transactional guarantees, my volume is tiny, and I can tweak the queue logic like above with a simple predicate change.

Here are a couple of tips if you want to use postgres queues:

- You probably want FOR NO KEY UPDATE instead of FOR UPDATE so you don't block inserts into tables that have a foreign key relationship with the job table. [1]

- If you need to process messages in order, you don't want SKIP LOCKED. Also, make sure you have an ORDER BY clause.

My main use-case for queues is syncing resources in our database to QuickBooks. The overall structure looks like:

    BEGIN; -- start a transaction

    SELECT job.job_id, rm.data
    FROM qbo.transmit_job job
      JOIN resource_mutation rm USING (tenant_id, resource_mutation_id)
    WHERE job.state = 'pending'
    ORDER BY job.create_time
    LIMIT 1 FOR NO KEY UPDATE OF job NOWAIT;

    -- External API call to QuickBooks.

    -- If successsful:
    UPDATE qbo.transmit_job
    SET state = 'transmitted'
    WHERE job_id = $1;

    COMMIT;

This code will serialize access to the transmit_job table. A more clever approach would be to serialize access by tenant_id. I haven't figured out how to do that yet (probably lock on a tenant ID first, then lock on the job ID).

Somewhat annoyingly, Postgres will log an error if another worker holds the row lock (since we're not using SKIP LOCKED). It won't block because of NOWAIT.

CrunchyData also has a good overview of Postgres queues: [2]

[1]: https://www.migops.com/blog/2021/10/05/select-for-update-and...

[2]: https://blog.crunchydata.com/blog/message-queuing-using-nati...

Looks super snazzy:

Can I point Retool Database to an existing Postgres database? If not, do y'all have any plans to add support?

How's it work under the hood? Custom extensions?

Here's a fun, related paper by Okasaki:

Even higher-order functions for parsing or Why would anyone ever want to use a sixth-order function? [1]

[1]: https://www.cambridge.org/core/services/aop-cambridge-core/c...

You omitted a piece of the definition:

> Malware is software that is installed on a computer without the user's consent and that performs malicious actions.

https://www.mcafee.com/en-us/antivirus/malware.html#:~:text=....

Erroring on appending to a slice would require checking every call to append for an error. I'd find it more surprising for append to error on resize since append implies a growable array.

> I can't do in TS with the help of a library or two.

Libraries like io-ts and zod embed the type information into JavaScript, so the information is available at runtime but only for wrapped types. If TypeScript provided runtime type information, such libraries wouldn't be necessary.

> Types are absolutely available at runtime

TypeScript types are not available at runtime. As the name implies, runtime type information requires runtime support. Since TypeScript compiles to JavaScript, the only type information available is that provided by JavaScript.

Jotai may be small and focused, but is definitely not a single file: https://github.com/pmndrs/jotai/tree/main/src/

In terms of confidentiality, availability, and integrity: I'll bet LastPass would gladly trade availability right now to regain confidentiality.

> I think people get hemmed up talking about what FP is

You've defined purely functional programming, which seems to be the increasingly common meaning of FP. The original definitions only explicitly refer to constructing programs by combining functions:

A History of Lisp http://jmc.stanford.edu/articles/lisp/lisp.pdf

> LISP is characterized by [...] composition of functions as a tool for forming more complex functions.

Can programming be liberated from the von Neumann style? https://dl.acm.org/doi/pdf/10.1145/359576.359579

> informal description of a class of simple applicative programming systems called functional programming (FP) systems, in which "programs" are simply functions without variables.

However, Backus defines his functional programming system mathematically, which could reasonably be construed as referentially transparent.

Super cool. If I understand correctly:

- There's no IDL file, like a .proto file. Instead, weaver looks for marker interfaces by embedding weaver.Implements[T].

- Weaver interfaces must be serializable, which explicitly support protobufs. Was the intent to be able to port gRPC services to weaver?

- Weaver maintains a list of deployments, and each deployment has components. Components may only communicate with components belonging to the same deployment. Sounds like a way to implement atomic deploys.

- Named listeners are mapping over a net.Listener?

Sidenote: I'm going to steal the metrics implementation. I haven't found a lightweight metrics implementation for GCP.

An opportunity arises for a Kafkaesque headline: "In Codd's Shadow, Redpanda Rises."

Fun historical quote:

> In the fall of 1972 President Nixon announced that the rate of increase of inflation was decreasing. This was the first time a sitting president used the third derivative to advance his case for reelection.

https://www.ams.org/notices/199610/page2.pdf

Homebrew does have Postgres 15. https://formulae.brew.sh/formula/postgresql@15

Nice article. I've used composite types with success for similar problems. Some tips:

- Composite types are useful for data validation. Money types and unit-based types are good for composite types.

- Avoid over-using composite types. Most of the time, regular columns are better. Not many tools interact well with composite types, like reporting tools or database libraries.

- Like the article notes, avoid using composite types for data types that may change.

- A JSONB domain type is a good alternative for data types that change often. Note that if you put the validation into a helper function, Postgres will not revalidate the domain if the function definition changes.

- Using composite types is mildly annoying since you must wrap the composite column in parenthesis to access the field.

    -- Bad 
    SELECT package_semver.major FROM package_version;
    -- Errors with `missing FROM-clause entry for table "package_semver"`
    
    -- Good
    SELECT (package_semver).major FROM package_version;

- When defining a domain type, separate checks into named constraints with ALTER DOMAIN. The Postgres error message for check validation failures is lackluster and provides little beyond "it failed."

    CREATE DOMAIN item_delta_node AS jsonb NOT NULL;
    
    ALTER DOMAIN item_delta_node
      ADD CONSTRAINT item_delta_node_is_object
        CHECK (coalesce(jsonb_typeof(value), '') = 'object');
    
    ALTER DOMAIN item_delta_node
      ADD CONSTRAINT item_delta_node_item_delta_id_null_or_num
        CHECK (coalesce(jsonb_typeof(value['item_delta_id']), 'number') IN ('null', 'number'));