matrss

Until the "published cookbooks" market is flooded with AI generated bullshit. Related: https://news.ycombinator.com/item?id=41269514

> particularly because it also allows for non-Python dependencies (one of my recent projects involved a browser automation solution, and I chose to install Firefox in a conda env

You could just as well install it either globally with pixi global, or as a project dependency in a pixi project. In this case, the latter.

> I also have a few shared envs, for quick tests

Fair, I just use temporary directories with pixi projects for that and thus no longer see a point in conda envs for this. It has the added benefit that all my temporary directories are in the same location and easy to get rid of.

> general utils

I would want those to be available at all times, so conda envs aren't fit for that purpose. Instead, I use pixi's global subcommand.

> Poetry for project management.

That limits you to lock files for python dependencies only, unfortunately. In a project relying on browser automation, I would want the browser version to be recorded in the lock file as well. Pixi does that.

---

I still don't really see why you found pixi lacking. It addresses all of your problems, maybe with the exception of globally installing multiple and non-CLI packages. But again, conda isn't any better as you are generally advised not to install stuff into the base env and conda has no other facility for globally installing general utilities.

Pixi is installable under the same conditions and uses the same repositories as conda, so unless you are specifically allowed to use conda and forbidden to use anything else this isn't an argument for conda.

What are you using each of them for in that workflow?

I've found that there really only are two kinds of packages I want to install: those I want globally (e.g. CLI tools like git, git-annex, DataLad, whatever shell enhancing you want, etc.) and project-specific dependencies (anything required to get going with it on a freshly installed system with the smallest amount of assumptions possible).

The former is sufficiently addressed by `pixi global` and other package managers, the latter by pixi projects. Notably, conda environments are a bad fit for both (not global, not really updatable, not automatically tied to a project, not rebuildable due to missing lock files, ...).

For the future, please pick a package manager that can give you a lock file alongside your code, so that you have a definitive record of the dependencies.

Even if you have all versions as of the time of the last modification to the code, you don't know if the dependency resolution happened at that point in time, or if the environment was set up years prior and never updated.

Nevertheless, this is what you are looking for: https://pypi.org/project/pypi-timemachine/

Yes, the entire workflow of conda/mamba is a liability. Having environments detached from directories encourages bad practices (reusing environments for unrelated things, installing everything into the same environment and getting very broken dependency resolutions, that stuff) and it doesn't even have a lock file format, so environments can't be saved short of dumping its entire directory.

Iff it must be the conda ecosystem, pixi (https://pixi.sh/latest/) is a much better pick.

But conda-style packages (or anything with install time dependency resolution really) also have the issue of being completely untestable. That makes them unsuitable at least for user-facing package management, if you care about your packages not randomly breaking without warning.

I'd rather see every language converge on a single package manager that implements functional package management, i.e. guix or nix. One can dream...

There definitely doesn't need to be truncation. A upper limit yes, to avoid the potential for DoS, but no truncation. That limit should be reasonably large, 1000 characters or something maybe.

I've seen a service whose registration form accepted my 64 characters long randomly generated password, but then the login form apparently truncated the password to something smaller, silently, and just said my password was incorrect. I then had to guess what was happening and reset my password to something shorter, which worked. If you must have a stupid password policy at least make it explicit (well, and consistent).

> What happens when it's no longer fun, but 150,000 people depend on it for their job?

Then someone else can take over, or not, depending on how important it really is to those 150000.

From the perspective of a business, using open source is strictly safer than closed source. With closed source, if the entity owning the code goes bankrupt, increases licensing fees substantially, just axes the project (incidentally what Google is also known for) or looses interest in some other way, you are out of luck. Go find something similar and hope there is a migration path. With open source, the equivalent is the maintainer walking away because they are bored, and you have the code right there to take over, or pay someone else to do so.

> I question the validity of open-sourcing anything for fun unless you design the licence, and more, to enable you to walk away and/or get bored.

Every open source license is already designed so that you can just walk away for whatever reason, e.g. when you are bored.

> That said, I usually prefer the changing bit at the end. So something like GlobalScopeForWorker, GlobalScopeForWorklet. But then that's clunky, so we're back at WorkerGlobalScope and WorkletGlobalScope again.

I wouldn't necessarily call "GlobalScopeForWorker" more clunky than "WorkerGlobalScope", just a bit longer, but also more descriptive.

Using the languages namespacing features might also make it more obvious, e.g. "Worker::GlobalScope" and "Worklet::GlobalScope" or the inverted version "GlobalScope::{Worker,Worklet}".

Looking at it from a functional programming perspective, I also like approaches of the form "GlobalScopeFor({Worker,Worklet})", i.e. a function returning the respective thing.

Naming things is hard, but the possibilities are endless...

> In the same vein, let's only write tests for the places where bugs will actually be.

Sounds like a good idea, actually. These places being the entire surface of your code that is exposed to the real world. So not unlike what I have been suggesting.

I mean, if you really want to start every function with multiple `assert true == true` statements, feel free to do so. But I will question its usefulness.

Do you have any particular example in mind that is not superfluous and really can not be exercised by the public API? I have a hard time thinking of any.

> In a good program, most of those checks are not coverable from the public interfaces.

If they aren't then you are defending against situations that can never occur. In that case the check seems unnecessary, and I at least wouldn't make it a priority to cover it. Also, those checks are usually simple enough that one can reason about them easily, again making it less of a priority to have tests cover them.

> The other problem is that finding tests that cover everything from the public interfaces can be extremely difficult. Most test suites don't achieve full coverage even of theoretically reachable points in the code.

Yes, testing is hard. That is not a reason to write more detrimental tests though.

I don't fully agree. Yes, employ as many testing approaches as possible (e2e tests, property-based testing, golden tests, whatever), but only test for behavior that you expose and want to guarantee will stay as-is. Otherwise you will get into the situation where every refactor will require a test change.

Unit testing is fine, if you do it for your public interface. If you are writing a math library then sure, unit test that `add(1, 2) == 3`. But if you just have an internal helper function for that, then think about if you really want to lock its existence and behavior into place, or if that would just hinder future architectural changes.

You can always test the exposed functionality that uses the helper and achieve full coverage of it that way. If you can't, then you have dead code.

Of course this is all a bit more nuanced. Past a certain size it might make sense to e.g. consider one modules interface to be public for the rest of your application and test it. But you can definitely overdo it and testing every single function you write (as I've seen people unironically suggest) is very likely detrimental.

This is good advice not just for compilers, but for any kind of software. Test the behavior of the public interface, not of the internals. Doing the latter too much will just unnecessarily lock you into one implementation of your software, without much gain.

As examples: If you are creating a web API, only test against the exposed public routes of that API, instead of writing tests for internal helpers. If you are creating a GUI application, programmatically exercise the GUI instead of starting your tests halfway into the innards that respond to the GUI buttons.

Tests lock behavior into place, so you should only write tests for behavior that needs to be locked.

So, contrary to your own point, you did indeed put in some time and learned one of the ways you can fix a "fucked" repo, because it is more convenient to just 'git reset --hard main' than to delete and re-clone.

There are some simple next steps to go from there:

  - notice that you don't have to reset to the main branch, you can put in any branch name
  - it doesn't even have to be a branch name, it can be any kind of reference to a commit
  - 'git reflog' is useful to find out a reference to a commit you recently were on, that you can then give to reset
  - you don't have to use --hard, I personally like using --keep, and sometimes you might want to keep local changes so you don't supply those options at all
  - ...

What you already did is what Zambyte suggested you should do. Of course there is diminishing returns at some point, but that point comes way after 'git commit -am $msg && git push'.

Its worth keeping this (from their readme) in mind though:

> However, at present SSHFS does not have any active, regular contributors, and there are a number of known issues (see the bugtracker).

Not that it is unusable or anything, it is still in widespread use, but I'd guess many assume it to be part of openssh and maintained with it, when it isn't.

An interesting alternative might be https://rclone.org/, which can speak SFTP and can mount all (of the many) protocols it speaks.

I find it to be a good default when writing a script, but yes, it can get noisy and potentially leak stuff, if that is what your script deals with. That hasn't been a concern in the settings I usually use it, though.

I am not sure how it could break anything though, unless you are parsing stderr of your script in a subsequent step, which would seem unusual anyway.

Oh wow, I didn't know that. I am pretty sure I had a sh trip up on it just recently, so I thought it still was bash only. Yes, use it everywhere possible.

I can recommend starting every bash script with

    set -euxo pipefail

The "u" has basically the same effect as the question mark, but for every variable usage.

The solver might be deterministic if you take some sufficiently large global state into account, but the process of

    apt install bar

from a users view is not. It can have a different result depending on what is already installed, or even if all other system state is the same it can change with time.

My point is that a package "bar v3.2.1" that depends on "libfoo v1.2.3" needs to be considered a different package than "bar v3.2.1" that depends on "libfoo v1.2.4" or even "libfoo v1.2.3 compiled with -O3 instead of -O2", because the resulting package can behave differently due to the variation.

But as soon as you have that there is no need for install time dependency resolution anymore, since it is all done at build time.

That is an orthogonal issue. Guix has first-class support for versions (where "version" means the version string of the source code), while nixpkgs uses the same naming-based approach as Debian.

The important part is being able to install two packages in isolation, irrespective of if they represent the same software in different versions or if they are completely unrelated. Apt couldn't install two unrelated packages if they contain files that have the same path, to my knowledge.

As soon as one foregos putting everything into a single filesystem structure (like in FHS, which most distros follow to some extent) the logical conclusion will be to prefix each packages directory with some identifier that meaningfully describes its content (which could be considered an "extended version" of this package). With nix this is a hash of all the inputs that went into the package (its own source code, compiler flags, build settings, dependencies, etc. and recursively the same for all of its dependencies). This means that if the compiler flags of e.g. a five levels deep transitive dependency are changed, then the package that depends on it changes as well, which is only reasonable since its behavior could change.

At that point all dependency resolution has happened at build time and none is required at runtime. Being able to install different versions of the same package is merely a by-product of this approach.

I think install time dependency resolution was a bad idea and needs to be removed entirely. Packages for package managers that do it are untestable by design, because their runtime behavior cannot be known at build time.

"Functional package management", as implemented in nix and guix, is a much saner approach.

> think Ansible, but imperative

For basically all purposes ansible is imperative.

I agree with you, your declarative abstraction has to have an imperative implementation underneath that will do all the dirty work. Ansible presents this declarative interface at the module level (if the module is implemented properly, most aren't), and a playbook is an imperative list of declarations to be applied. Roles also combine a list of imperative steps into a declarative interface.

Since apparently (I try to avoid ansible, so I might be missing something) playbooks are the go-to approach of using ansible this means that most uses of ansible are imperative (in the context of configuring a system), unless you only ever give a system a singular role and then you are probably defining your role in imperative steps.

A system like NixOS on the other hand presents the entirety of a system configuration in a single declarative interface that is applied in one go, while applying such a configuration to a system can be a thought of as an imperative step (although it is usually a singular, unconditional step). So it is declarative at a higher abstraction level.

> I’m not sure how “I don’t have to actually think about any of the separation” meshes with the fact that you explicitly setup multiple users and configured file and group permissions accordingly. You clearly put a lot of thought into it.

That's the thing, with NixOS you usually don't have to explicitly setup users and permissions. For most simple services, the entire setup is a single line of code in your NixOS configuration. E.g.

    services.uptime-kuma.enable = true;

will make sure that your system is running an uptime-kuma instance, with its own user and all.

Some more complex software might require more configuration, but most of the time user and group setup is not part of that.

Yes, this is debating state management. For full declarativity some form of state management for the parts of the system that should be under declarative control (like terraform) or a stateless but very holistic view of the system (like NixOS, I guess also Guix System) are needed.

Given that ansible has neither it can't be much better then what it is. I disagree that that is the right choice though. As it is I see not much more value in ansible than in some sort of SSH over xargs contraption combined with a list of servers. The guarantees they give are the same.

> Do you know every package that should be installed because it comes out of the box in the distro? Do you want to [...]?

No, I don't want to. Thankfully, with NixOS I don't need to, since the pre-installed packages are automatically part of the declared state of my NixOS systems (i.e. I declare the wanted state in the same way in which the defaults are also declared, which makes it easy to merge both).

Ansible playbooks are usually a list of steps to execute in order (imperative). Those steps may try to present a declarative interface to what they are supposed to do, but many fail to fulfill the definition of "declarative" that you have linked. E.g. with the built-in modules it is impossible to declare a desired set of installed packages, only a set of packages to be installed _in addition to all already installed packages_. This means it is impossible to remove an installed package again by removing it from the declaration, you have to specify a second step (imperative) that explicitly removes the package. This makes it impossible to declare a final state for "installed packages" with ansible.

Or you go the functional package management route and strip out these global paths entirely. nix and guix do this. Every package that needs python is built against a specific version of python (where a "version" is defined by its source code, compiler flags, toolchain versions, bootstrap toolchain versions, etc.; basically anything that can make the build result behave differently) and will only ever use that. Different packages can use different python versions. A system configuration can also choose to make a specific version available in the PATH, but it doesn't have to. A user environment can do the same, as can a project specific environment. All in one package management system.

> And finally finally, replace android with GrapheneOS.

The irony of which is that it only runs on Google's phones.

I find it useful to keep in mind that LLMs are essentially "just" random generators based on the probability distribution of some language (or more accurately a sample of that distribution in the form of a finite amount of text, a dataset).

You don't expect a random generator to spit out facts, you expect to get random nonsense. But it can be very good at replicating the fitted probability distribution in its output, i.e. generate convincingly coherent language.