peter_l_downs

Great question! pgtestdb requires that you somehow run a postgres server. It then connects to that server and handles all of the database creation. The README goes into more detail here, but basically you should be easily able to point pgtestdb at a dockertest-managed postgres server.

One massive advantage of pgtestdb compared to using dockertest on its own is that it handles running migrations for you in a very efficient way that amortizes the cost to zero as the number of tests trends to infinity. This should be much faster than naively creating a new database and re-running the migrations for each test.

I happen to recommend using docker-compose over dockertest out of some slight personal preference that is one part separation of concerns (it's weird to me to have the test code handle the state of the postgres container), one part developer-experience related (it's nice for devs to be able to easily manage their postgres server instance with existing docker tools outside of golang code), one part infra related (it's nice to be able to use any method to provide and manage the postgres server, particularly in CI), and one part totally arbitrary.

Agreed with most of this but I’m skeptical of the rsc.io/script dsl approach. I’ll try it, though, because Russ is often right.

shameless advert: do you wish testify was implemented with generics and go-cmp, and had a more understandable surface area? Check out my small library, testy https://github.com/peterldowns/testy

shameless advert: do you want to write tests against your postgres database, but each new test adds seconds to your test suite? Check out pgtestdb, the marginal cost of each test is measured in tens of milliseconds, and each test gets a unique and isolated postgres instance — with all your migrations applied. https://github.com/peterldowns/pgtestdb

I’m a fan of passkeys and bullish on continued UX improvements but completely agree with you. Even basic things like “when I’m on a website, show all my passkeys for it and where they’re stored (browser, 1password, macos keychain)” or “when I’m adding a passkey for a site, properly choose where it’s stored and know what other devices it will be synced to or accessible from” are hard or impossible or confusing. Improvements needed.

If I were maintaining an auth system right now, I’d make sure webauthn was supported as both a login and 2fa method, make sure i didnt require hardware-scoped keys, and leave them off by default. Advanced users can opt in and support themselves. 2fa defaults to the standard qr code. Logging in allows “send me a magic link” but defaults to email and password. No usernames. I feel like thats reasonable?

The apps discussed in the article do not allow cashing out. Sorry to hear about your friend’s difficulties.

Does an equivalent linter exist for Python, or for Golang, or for Scala? I'm sold, it would be great to have something like this.

EDIT:

- pylint: using-constant-test (W0125) https://docs.pylint.org/features.html#id13

- golangci-lint: revive's constant-logical-expression https://golangci-lint.run/usage/linters/#revive

That's on the long list of blog posts that I've never written. Feel free to start with these and go forward! Make sure to get a physical copy of Shape, it's one of the most baffling textbooks I've ever tried to read.

This is great, thank you. Do you actually use this software? I’m very curious to hear about real world applications of these generative systems.

Strongly recommend reading this, as well as looking up Homakov's exploits. My opinion is that OAuth 2.0 is a very bad spec because in practice people do not implement it correctly. As to why? Maybe it's that the spec is split across multiple RFCs, and defines many insecure behaviors as compliant, and authorization is just hard.

When Facebook, Twitter, Github, etc. all got burned at various points in time with their OAuth implementations, you have to start looking for the common denominator.

Hard to believe this isn't common practice at this point. Way back when I was just getting started in my career I implemented an OAuth 2.0 server and it was already accepted that auth code grants were the only reasonable way to do things. Here's what I wrote in the docs for that project:

    The decisions that are most important to the security of your application are:
    
    - The authorization endpoint will only return authorization codes, which can later be exchanged for access tokens.

    - Password credentials grants, implicit grants, client credentials grants, and all extension grants are not supported.

    - Public clients are not supported.

    - Every client is required to register its redirect_uri.

    - All authorization, token, and API requests are required to use TLS encryption in order to prevent credentials from being leaked to a third-party. In addition, the registered redirect_uri must also be secured with TLS.

    - Clients are required to CSRF-protect their redirection endpoints.

https://djoauth2.readthedocs.io/en/latest/overview.html#what...

The migra maintainer has abandoned it for greener pastures. If you’re interested in sponsoring an alternative, send me an email. My pgmigrate tool has all the groundwork necessary to make this possible but I have held off implementing this because I am not personally interested in using it.

If you’re seriously willing to pay money for this, send me an email — I’m considering implementing this in my pgmigrate tool but not sure if it’s worth the development time. I have all the groundwork done, probably achievable in about 10 hours of concerted effort.

I believe Migra can generate those changes for you via diffing, not sure how well it handles dependent views/functions/etc

Flyway is generally the most popular. Django/alembic in the python world.

I think most migration tools are missing a lot of features that would make them safer and easier to use. Modern teams merge a lot and deploy frequently, but most migration tools are built for a db admin manually running commands.

I wrote a migration cli tool (and golang library) that I think is more suited to modern workflows, if you check it out I’d appreciate any feedback!

https://github.com/peterldowns/pgmigrate

Terrible interface but awesome animations, thanks for posting this.

For more information about DCAT-US, click through to

https://github.com/DOI-DO/dcat-us/wiki/What-is-DCAT%E2%80%90...

> DCAT-US is the metadata standard associated with the requirements for enterprise data inventories in the OMB M-13-13 open data policy and the Foundations for Evidence-Based Policymaking Act Title II, OPEN Government Data Act (Evidence Act). The Evidence Act applies to all agencies. These federal policies do not apply to state and local governments which may have their own policies. However, state and local governments are welcome voluntarily to contribute their metadata to Data.gov. To do so, they must publish their metadata using the DCAT-US standard while omitting any federal-specific metadata elements as noted in the documentation.

I think the idea behind breaking down the task into a composable pipeline is that you then replace the LLM steps in a pipeline with supervised models that are much faster. So you end up with a pipeline of non-LLM models, which are faster and more explainable.

I came here to write this. Routing and networking is still confusing for me and all the writing about it is usually very "abstract" to me. A hands-on example like this one is really appreciated. Nice work, OP. I'll try to do it myself and follow along.

EDIT: one of the only other posts about this stuff that has made much sense to me is this one from Tailscale. It contains lots of "worked out examples" that really make it clear how everything fits together.

https://tailscale.com/blog/how-nat-traversal-works/

I just learned that the Assistive Technology act provides funding for all sorts of cool, free, assistive technology for Americans. For instance, this is a library of helpful 3d-printed assistance devices that you can request for free if you live in Pennsylvania!

Yes, I was inspired by djrobstep's work and many of the analysis queries that pgmigrate uses to generate a schema dump come directly from schemainspect (the library that migra uses underneath the hood). Migra targets a different kind of development flow but it is also in the migration space. My tool, pgmigrate, would be a good way to apply migrations created by Migra.

It is unfortunate that the Migra project is abandoned, I believe djrobstep has gotten into political advocacy and spends most of his efforts on things other than programming at this point. I say unfortunate because he is extremely talented and I have learned a lot reading and working with his code. Hopefully he comes back at some point to help transition Migra to some other maintainers and fix the website and such :)

Out of curiosity, how has it been working with the pganalyze team? It seems like Supabase has been able to collaborate really effectively with other teams in the Postgres space (including pgAdmin and pgmigra, for the `diff` migrations tooling), and it's wonderful to see!

That's an interesting idea, thank you, I'll read through the post.

EDIT: I found the post interesting but unfortunately these projects are in no way load-bearing. I'm happy to hear Filippo has made it work for himself but as he points out, the projects he works on are simply much better suited to sponsorship and retainer/consulting agreements. I wonder what he would recommend for newer or less-load-bearing projects.

I asked this question in a discord for this kind of stuff and the answer I got was "not possible." I also spoke with the developer of OrbStack, and they suggested just not being open source and charging for it, which is what they're going to do. When it comes to dev tools (particularly those that are involved in operations/sre flows like pgmigrate) I consider open source a huge benefit, and I'm sad to think that there is no way to get paid for an open source project without shoehorning in a bad experience or unnecessary features (like Baldur points out)

Pretty good overview from Baldur — I don't always agree with everything he writes but this seems relatively correct.

One question I'd ask him (and anyone else reading) is: what are some other options for monetization?

Over the last few weeks I had three different VCs reach out to me about some of the open source projects I've been releasing, and ask me if I'd thought about making a business out of them. I told them that no, based on the problem the software was solving, I didn't see how I could adopt open-core or companion-saas business models, and I wasn't sure how else it could be done while keeping the code open source.

Can anyone suggest a viable business model that would allow:

* Code remains at least source available, ideally open source for non-commercial use.

* I can charge for commercial use.

* Actually doing the licensing is reasonable, ie no spyware or phoning home from the tool.

Wouldn't need to be perfect, I understand that if the code is open source a company could easily fork and use it without paying me. The idea would be to make it zero-headache to pay me for a license if the code is being used by a funded team.

The projects:

* https://github.com/peterldowns/localias

* https://github.com/peterldowns/pgmigrate

He totally does! I grew up reading the comics and still consider myself a fan, thanks for teaching me something new about Tintin. Very cool.

Looks pretty cool, but could you explain why I might want this? I’ve worked with postgres a lot but I can’t think of a time where I wanted or needed this. That doesn’t mean it’s not useful, just curious about how I could use it!

Yes, it is unfortunate and difficult and I did not realize reading a prepared script off a clipboard would have helped me get my point across better. I could only see it causing them to view me as stonewalling them or cause them further confusion.

I may have to try this for myself though, I am not a doctor and I haven’t tried it before.

This is the most “that didn’t happen” story I’ve seen in a while. No way the concerned family responded positively to the doctor reading a script to them off a clipboard, forgot about it, and then responded positively to other nurses/doctors reading the same script to them again.

If it's a secret, it's only because the godoc format is so strangely obtuse about some things. It's sort of like markdown, but sort of not. You can do lists, kind of. You can do links, kind of. What a pain!

How does this actually work?

If you're classifying "banned" / "sensitive" text strings, and intercepting network requests on all employees computers, and then showing the "banned"/"sensitive" text strings to admins in a web interface... aren't the employees leaking a ton of information to you, a third party?

Since you say you've found it invaluable for your own projects, can you explain why? What bugs has it found for you that you wouldn't immediately notice by running code or unit tests?

I ask because in the example, you ask it to find bugs in a script that immediately fails when you run it. So I'm not sure what the value add is here.

    $ ipython
    Python 3.11.3 (main, Apr  7 2023, 21:05:46) [Clang 14.0.0 (clang-1400.0.29.202)]
    Type 'copyright', 'credits' or 'license' for more information
    IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

    In [1]: def greet(name, age):
       ...:     print("hello, my name is {} and I am {} years old".format(name))
       ...:
    
    In [2]: greet("John", 25)
    ---------------------------------------------------------------------------
    IndexError                                Traceback (most recent call last)
    Cell In[2], line 1
    ----> 1 greet("John", 25)
    
    Cell In[1], line 2, in greet(name, age)
          1 def greet(name, age):
    ----> 2     print("hello, my name is {} and I am {} years old".format(name))
    
    IndexError: Replacement index 1 out of range for positional args tuple

EDIT: For anyone else who is wondering, this is just:

system prompt:

    Find bugs in the user's code.
    - Format results as JSON like so:
     [{"line":1,"reason":"short reason", "type":"type"},...]
    - It is better to return an empty array than a mistake (not really a bug).

user prompt:

    Here is my code:

    ${source_code}

    JSON Result:

@funerr why would you even create this and post it here? Is it just to try to market your newsletter? Who do you think would look at this and be impressed?