weavejester

It certainly could be a blub situation, as aside from some toy applications, I don't have much experience with Rust. Obviously in Clojure, like most languages, you can restrict a resource to a particular scope - but it sounds like you mean something more than that.

> Right; the difference is that by the destructing time you have lost the type information about that ‘discarded’ data, meaning you no longer know how to discard it. In Clojure this is not a problem, because the only values you're allowed to have are those that can be garbage collected

Yes, this is very true. Languages like Rust can certainly manage memory and data structures more precisely. Not only is Clojure garbage collected, by virtue of being based on the JVM and being dynamically typed, but in idiomatic usage it encourages data to be built from a small number of basic types.

> On this I absolutely agree. The only thing I'd be careful to distinguish is not simply that Rust doesn't have the machinery but that the machinery makes sense only given a fairly onerous set of additional assumptions about your values that Rust doesn't make

I guess only I'd disagree here that the additional assumptions are "fairly onerous" - but that's just an opinion. You may be doing work that butts into those assumptions far more often than I do (which is almost never).

> No, you're right: I think I was thrown off by your assertion that these are uncoupled from the type system when we were talking originally of ‘conceptual’ types.

I don't think I expressed myself very clearly, and indeed Clojure's type system isn't very clear either, since it's bolted onto the JVM and so can't entirely do its own thing. So we have "types" which refer to Java classes/interfaces, and "schema" or "specs" which refer to Clojure's own runtime validation ideas.

> These two types aren't typing the same data, though: in the latter case (in Rust) you've actually thrown away some of the data, and the type reflects that.

In Clojure the data is thrown away too, just at a slightly later point. The data is parsed then destructured then processed, and it is at the destructuring stage that irrelevant data is discarded and marked for GC.

Just as the Rust function avoids coupling by using a more narrow type, the Clojure function avoids coupling by using a more narrow binding.

> I think this isn't a great comparison: the usual vehicle for this thing in Rust is the trait, which manages to abstract over the representation statically without introducing a bunch of runtime machinery for it

But you have to create and implement those traits. I'm not saying this is impossible in Rust; just a lot more onerous because you don't get all the machinery Clojure has that makes it trivial.

> These are just predicates; I don't think they have much to do with this discussion?

This might be why we're talking past each other. When I say "type", I mean a set of possible values. I don't think this is an uncommon definition; the first sentence of the "data type" Wikipedia page pretty much defines it the same way.

Given this definition, you perhaps see why we can narrow a runtime type by composing it with a predicate.

My understanding is that you view types differently, as more of an interpretation of some sequence of bits, rather than a set of data values. Is that correct?

> As we get better at writing compilers we're increasingly seeing a move towards mechanisms like Rust's traits or C++'s concepts that allow the programmer to actually abstract over the representation of a type (as opposed to forcing a uniform representation as is common in dynamically typed languages).

I agree that's there's no reason in principle that you couldn't statically type it with a sufficiently advanced compiler.

> I'm not totally sure how to interpret this question. If I already have a type that includes all the values I want as a subset, I can restrict that type by limiting the values it can take and restricting or removing the operations on it to guarantee they never produce any of the forbidden values.

Sorry, perhaps I'm being a little obtuse. What I'm ultimately trying to get at is that sometimes its useful to have different types for the same data under different circumstances. A subset of data might benefit from a narrower type.

For example, suppose we have some CSV file that contains a bunch of patient information, including average white blood cell count and birthdate. This CSV file might contain bad data! So perhaps we type it as:

    struct Patient {
        avg_wbc_count: Either<f64, String>,
        birthdate: Either<Date, String>,
    }

This covers all our bases, but it's also somewhat annoying to work with. Perhaps we only want to find patients with WBC counts outside a certain range, and discard those lines where the data is invalid. In which case, we could write a narrower type, and simply not parse the CSV rows that are invalid:

    struct PatientWithKnownWbcCount {
       avg_wbc_count: <f64>,
    }

So in this case we have a couple of options for types. The latter type decouples the birthdate (it's never even parsed), but adds the restriction that the WBC count needs to be numerical; while the former type is a more accurate representation of the CSV file overall, but has greater coupling.

Obviously which we use depends on the nature of our program, but what I'm trying to get at is that a type isn't set in stone, but something we choose. Ideally we choose the most restrictive and decoupled type for the circumstances, but that might result in having many hundreds of different variations of the same type, so there's a tension between the number of distinct types and how specific or narrow they are.

> In the Rust case you need to write down these things explicitly so the compiler can make sure you don't pass a value that doesn't satisfy them. In Clojure, the reason you don't need to write them down is that every value in the language is restricted to only be able to represent such values, so those constraints are implicitly on every function that you write.

Granted, but Clojure's approach can result in greater decoupling with less effort, particularly when dealing with imperfect data.

In the previous examples I presented a scenario where we might want to think carefully about how exactly we store data that might contain invalid fields. But in Clojure we don't care - we can quite easily have a data structure that's partially invalid or unparsed - the entire problem of how we represent this data in memory is sidestepped.

We can still define what constitutes valid fields:

    (s/def :patient/birthdate     inst?)
    (s/def :patient/avg-wbc-count float?)

But these schema definitions are independent of the type system (i.e. decoupled), so we can apply them selectively or not at all. We can say "this function requires patient data with valid birthdates and WBC counts, but we don't care about any other field".

Could you do the same in Rust or other similar languages? Sure, it's ultimately just maps and predicates. But many languages aren't geared up to make that pleasant to use.

> Nothing (ish: you have to be careful to describe what happens to all the integer operations when restricted to your type) — that's a totally valid way to define a type.

Then at which point does a type become equivalent to a restriction that could be decoupled from the underlying type by choosing a broader type?

> No, you just have to be explicit about the possibility of a birthdate (or other fields) being of a wide type

Yes, you could recreate Clojure's semantics in Rust. More accurately, I'd say you'd be looking at defining it like so:

    struct WbcCount {
      wbc_count: Vec<u64>
    }

    struct BirthDate {
      birth_date: Box<dyn Any>
    }

Since we want to be able to reason about these keypairs individually. You might have a structure that has a WbcCount but not a BirthDate, or one with a BirthDate but not a WbcCount, or one with neither.

We'd then be faced with the challenge of creating a type that could contain an arbitrary number of unique structs, and to be able to pull a struct out of that set by its type. Realistically we'd probably just use a HashMap at that point and discard all static typing.

Alternatively, we could create a mega struct that contains every possible field we could want to use, whether or not we know they are related, and use this data structure to represent all structured data in the application. The "if everything is coupled, nothing is" approach.

But both of those options are difficult or unidiomatic to write.

Languages like Rust, Java, etc. encourage coupling of data because it's more convenient and space efficient to group data together in records/structs. In Clojure, there's no need to do so; we can couple only when necessary. If we don't need to know the birthdate to calculate the white blood cell count, then we can exclude that field from the input type checking. Most statically typed languages find this difficult, with TypeScript being one of the few exceptions in this regard.

> Here we disagree. Unsigned 8-bit integers† and signed 64-bit integers, while both conveniently notated with Arabic numerals, are actually different values that support different operations, for example negation.

What about a 32 bit unsigned integer and a 64 bit unsigned integer? Are they still separate values?

You appear to be saying that the values of a type cannot be a subset of another type; that is, there is no '1', only a '1' that is an integer, a '1' that is a short, and so forth, and every '1' is distinct.

Fine, that's a possible way of looking at it, but why is that more valid or consistent than a model that allows subtypes? That, for example, the value '1' could be both a Number, an Integer, and a NaturalNumber?

Further, what's the practical difference between a type defined as consisting of the numbers [1 2 3], and an integer that's restricted to those values?

> This works in Clojure only because Clojure couples some semantics (data semantics plus operations on runtime type information) into every value, i.e. it restricts what values are even representable in the language in order to ensure that this function will always be safe to write.

Even if we view a Clojure value as a coupling between type and data, that's only a coupling between two things, and it ensures we can avoid further coupling caused by large record types. On net, we reduce the amount of coupling a statically typed language that uses closed record types would require.

For instance:

    (defn average-wbc-count [{:patient/keys [wbc-counts]}]
      (/ (apply + wbc-counts) (count wbc-counts)))

This function is coupled to only one key/value pair. Any other information in the map is irrelevant, which is why an invalid :patient/birthdate doesn't cause the function to fail. There's no coupling between :patient/birthdate and average-wbc-count.

Conversely:

    fn average_wbc_count(patient: &Patient) -> f64 {
      patient.wbc_counts.iter().sum::<f64>() / patient.wbc_counts.len() as f64
    }

This function requires patient to be a Patient struct, and therefore the function is implicitly coupled to every field in the struct, regardless of whether that field is ever actually used. I need to ensure that a patient has some birthdate that's an anticipated type (even if that's an error type) before I can call the function.

> To reiterate: a type doesn't limit the values but specifies the values (or rather, more generally, the meaningful operations on a value of that type).

But a value can have more than one possible type. The number 1 could come from an unsigned byte, or a signed long, for example. These are different types that support the same numerical operations, but differ in cardinality. So we can't say that a type's only purpose is to specify the meaningful operations on a value, as we might have two types that are identical in that regard.

> The real type of `:patient/birthdate` there is just the (discriminated) union of the date type and the string type.

Yes, in this particular instance that would be the case, but that's not necessarily something you know ahead of time. The point is that you may not have anticipated that not everyone would know their date of birth, and the data you receive is invalid according to your earlier assumptions.

In Clojure this results in a more graceful failure condition. Functions that don't require the date of birth will continue to work with no change required. If I want the average white blood cell count of a patient, I don't care what the date of birth is, and therefore the output for that particular operation isn't affected.

> In fact the opposite is true: raw values have no valid operations and types allow you to add operations that make sense for those values.

I think you're using 'value' to mean something slightly different to the way I meant it. So to get us onto the same page, by 'value', I mean something independent of how its stored; that is, the number 1 is a value, whether it's stored as as 00000001 or 0000000000000001.

A type limits both which values it is possible to represent (i.e. a u8 limits us to representing the integers 0 to 255), and also determines how it is encoded in memory (in this case 8 bits).

> Most statically-typed languages are actually very loose about how values are structured at runtime

Yes, but ultimately the compiler needs to be able to map a sequence of bits to the value it represents, even if there's not a strict one-to-one mapping.

> I'll have to do some guesswork here, but I imagine when people make arguments like this they are significantly imagining a situation in which, say, all the values are expected to be in the range [5, 100] and some befuddled experimenter or piece of machinery gives you the value 2

That's a common assumption, but the reality can be much more messy. It might be that we expect an integer between 5 and 100, but receive a string of UTF-8 characters instead.

For example, suppose you want to record a patient's date of birth. In Clojure, we might represent that as a map that connects a :patient/birthdate key with an encoded date object:

    {:patient/birthdate #date "1972-04-08"}

But what if the patient doesn't know their exact birthdate? Perhaps they immigrated when they were a young child from a less developed country and don't know the exact year they were born in. However, they tell the doctor that they do know they were born before 1980, because that's when they first arrived in the country.

In this case, the doctor might record the data as:

    {:patient/birthdate "before 1980"}

Even though the data doesn't match the type we expect (a date), it's important that it still be recorded as it could affect the medicine that the patient is given. These sorts of messy entries are not uncommon in areas where it's more important to accurately record the data than precisely type it.

Types are used to both define how data is represented in memory and to restrict which values are allowed. For instance, an enum might be internally represented as a byte, but the compiler further restricts the permitted values to those the enum represents, and limits what operations are available (e.g. we can multiple a u8 by a u8, but not an enum by an enum).

In this sense, most statically-typed languages conflate how data is structured with how it is restricted. Some overlap is unavoidable, as anything represented by a single byte is always going to be restricted to at most 256 values, but Clojure tends to take the view that the more decoupling (or decomplecting) you can achieve the better.

This can be useful when dealing with data that is in some sense invalid. You might receive data that's outside expected bounds or even of a different type, and it might make sense to handle it in some fashion. This is a common necessity in pharmaceutical trials, for example.

Regardless of whether the type information is static or dynamic, you're still coupling some type to some data. The type is still implicit even after compilation; there still exists a structure to the data, even if that structure isn't easily discerned without the source. Or to put it another way: just because there's no runtime type information, doesn't mean that the data now is entirely decoupled from the type.

There's a parallel between welfare and passive gains from diversified investments; yet as you point out, government welfare is tracked and very visible, while private investments are far more opaque.

Part of the wealth that workers create is paid in taxes to the government; another part is paid to the shareholders of the company they work for, to the bank they get their mortgage from, and so forth. The latter is equivalent to an hidden tax, part of which goes toward a form of private welfare that's proportional to an individual's wealth.

People get quite angry about government welfare, particularly if the recipient seems undeserving of it. Yet they have no problem with the passive income from billionaires, even if it ultimately amounts to the same thing.

Forgive me if this is an ignorant question, but does your use of the Mainline DHT mean that Bittorrent clients will be responding to P2P address lookups from Iroh?

Yes, a company needs to extract more value than it pays its employees, if only to cover its other costs. The problem is when employees are significantly underpaid compared to what they produce.

Negotiation clearly doesn't work in the general case, otherwise we wouldn't have billionaires. There's too much of a power difference between an employer and employee, and companies have a clear incentive to keep it that way.

There's an argument that if someone agrees to a bad deal, that's their own fault. Where I think it becomes unethical is where there's a significant power imbalance that disadvantages one side.

Suppose I buy a painting from a flea market for $100, get it evaluated by a specialist, and then discover it's actually worth $100,000. In this example I have no inherent advantage over the seller; neither of us knew the value of the painting at the time it was sold.

Now suppose a famous TV antique dealer stumbled across that painting instead, and immediately realizes its true value. The seller recognizes the dealer, and the antique dealer offers to buy the painting for $25. The seller, trusting the antique dealer's judgement, agrees to the discount.

Would you say in both examples everyone acted ethically? This is a genuine question, as I can certainly see the argument that using the assets you possess to secure yourself the best deal possible is just business, and yet I would personally see the antique dealer in the second example as being exploitative.

When it comes to companies there's a similar disparity in power. An employee requires money to live, while someone founding or investing in a company often has enough of a financial safety net that they won't starve if the venture fails. Equally, any would-be billionaire is explicitly looking for employees who generate vastly more value than their cost. You don't get rich by paying people what they're worth; you get rich by underpaying them and pocketing the difference.

The other problem, and one you've touched on, is how do we assess the value of an individual employee? This is obviously not easy, and businesses also have no incentive to work it out or reveal that information to their employees even if they knew. On the contrary it benefits employers to keep their employees as much in the dark as possible.

Aside from the ethical problems there's a practical one. The very existence of billionaires implies that a significant number of people are undervaluing their work. It's a pricing problem that the market isn't solving, and is only getting worse.

Not everyone can afford to take the financial risk of being a founder, and not every business type can be started with low initial capital.

That's not quite what I'm saying. You may very well have been paid fairly for each job you've taken, assuming that the value you generated for the business was not substantially higher than your salary.

But hiring people who are compensated fairly does not make someone a billionaire. If you generate $300,000 of value per year and I pay you $200,000, then I'm only making $100,000 profit off your work. I could hire more employees, but value does not scale linearly indefinitely. Doubling my number of employees does not guarantee I double my profits.

No, if I want to become a billionaire within my lifetime, I need an asset that generates far more money than it costs to buy and maintain it. In other words, I need employees who will generate millions for every thousand I pay them.

Now you might well argue that I'm taking a risk. How do I know if an asset or an employee or a team of employees is undervalued? Not every bet is going to pay dividends. However, while this is true, I don't think this makes it ethical. If I'm a venture capitalist looking to make it rich (or richer), the fact that I'm taking a risk doesn't change the fact that ultimately I'm looking for people who I can pay far less than they're worth.

How many businesses are there that are worth at least $1 billion and employ no-one but the founders?

When people say that it's not possible to earn a billion dollars, they're talking about the discrepancy between the wealth gained by those employed by the company versus the shareholders of the company. For example, when WhatsApp was sold to Meta for $19 billion, how many of WhatsApp's 55 employees walked away with hundreds of millions of dollars?

The fundamental problem is that it's possible for an employee to generate a hundreds of millions of value for a business, and yet be compensated for a vanishingly small fraction of that. Even if the employees agreed to a particular salary, is it ethical to pay them so little in comparison to the worth they generate, or is it exploitative?

Most, if not all billionaires, reach that status by paying people far less than the value they generate. If you want to become a billionaire, you need to find people who are willing to be paid thousands or tens of thousands of times less than they're worth. You need employees who will generate you $100 million in exchange for being given $100 thousand.

> You effectively have to keep the types of all the things involved in your head and/or trace them to ensure that you don't run into a crash.

You make this sound difficult, but in practice type errors are rare in Clojure and generally caught in the REPL or by tests, since the moment you go down a branch with a type error an exception is thrown.

Contrast this to errors caused via mutable state, which are usually far harder to track down, because the failure condition is more specific.

> This is trivial in TypeScript.

In the example you give you're omitting assoc entirely, which defeats the point. I'm using assoc as a minimal example, but the same principle applies to more complex functions, so replacing assoc with the equivalent expression doesn't tell us whether or not we can effectively type a function that deals with maps.

So lets try doing this properly. At minimum we need something like this:

    type Assoc<M extends object, K extends string, V> =
        Omit<M, K> & Record<K, V>;

    function assoc<M extends object, K extends string, V>(
        m: M, k: K, v: V): Assoc<M, K, V> {
      return { ...m, [k]: v } as Assoc<M, K, V>;
    }

(Note that we need to perform an explicit cast in order to inform TypeScript of the type of the key.)

However, this produces some rather messy types consisting of nested Assocs. In order to get back to something a human can read, we can use an additional Simplify type to force the type system to reduce it back down into an typed object:

    type Simplify<T> = {[K in keyof T]: T[K]} & {};

    type Assoc<M extends object, K extends string, V> =
        Simplify<Omit<M, K> & Record<K, V>>;

    function assoc<M extends object, K extends string, V>(
        m: M, k: K, v: V): Assoc<M, K, V> {
      return { ...m, [k]: v } as Assoc<M, K, V>;
    }

(The empty `& {}` intersection forces normalization, providing a cleaner reported type.)

We're still not done, though, as if we want the same type checking that a class has, we need to ensure that a key cannot be overwritten with a value of a differing type. So we'll type the value argument as well to ensure it matches the type of an existing value within the map:

    type Simplify<T> = {[K in keyof T]: T[K]} & {};

    type Assoc<M extends object, K extends string, V> =
        Simplify<M & Record<K, V>>;

    type AssocValue<M extends object, K extends string, V> =
        K extends keyof M ? (V extends M[K] ? V : never) : V;

    function assoc<M extends object, K extends string, V>(
        m: M, k: K, v: AssocValue<M, K, V>): Assoc<M, K, V> {
      return { ...m, [k]: v } as Assoc<M, K, V>;
    }

So this is possible to type in TypeScript (to its credit), but is it "trivial"? And is this type signature significantly less complex than one might find in Haskell?

I don't see an asymmetry in the abstraction. Both vectors and maps are associative structures - you can assign a key to a value - the only difference is that vectors have a more constrained keyspace (i.e. ordered, consecutive integers starting from zero).

But that wasn't really my point. Even if we limit `assoc` solely to maps it would still be difficult to type effectively.

For instance, suppose we have some code like:

    (let [m* (assoc m :number 3)]
      (:number m*))

We can see that the return type of this expression is obviously an integer, but what is the type of m*? How do we type m* such that (:number m*) can be inferred to be an integer by the compiler?

Most statically typed languages sidestep this problem: instead of using an open data structure like a map, a closed structure like a record or class is used instead, and these structures must be explicitly typed by the user.

The problem with this approach is that now every record is specific and bespoke. You lose access to all the general-purpose functions that operate on generic data structures, and as records and classes are closed, you also lose the ability to extend them.

This is the ultimate problem with static type systems: you're trading capability for safety. If you're programming within a static type system, there are options that are simply not available or feasible to use.

I too agree that "until you get better" isn't a good take. To err is human, and even the most experienced developers make mistakes.

That said, you don't get static typing for free. As with many things it's a trade-off: you catch some errors at compile time in exchange for working within the confines of the type system. The ultimate hope is that the time you spend fiddling with types is going to be less than the time you spend debugging type errors.

> There is nothing you can do with dynamic typing that you cannot do with a sufficiently powerful static type system - and it doesn't have to be something absurd like Haskell's. You basically just need structural typing and type inference and some type-level programming constructs.

Haskell doesn't have a complex type system for no reason; it's necessary to encompass everything it wishes to do, and even then it's not as flexible as a dynamically typed language.

For instance, how would you statically type Clojure's `assoc` function? It's not at all trivial if you want to retain the type information of the keys and values.

I'm not sure I agree. Certainly there are differences other than syntax, but that doesn't mean syntax is irrelevant. For instance, would Clojure programmers use maps as much if there was no syntax for map literals?

Syntax determines what parts of a language are within easy reach, and therefore affects how programmers use the language. Tools that a syntax make easy are used often; tools that syntax makes hard are used infrequently. This indirectly impacts how a piece of software is designed.

But syntax must necessarily include what it's representing, no? For instance, `{:a 1}` represents an immutable map in Clojure, in the same way that `42` represents an immutable integer in Java.

The programming language informs the design of the system. As I said in my earlier comment, an idiomatic Java codebase is going to be designed very differently to an idiomatic Clojure codebase, even if they both intend to solve the same problem.

To be clear, I'm not questioning your choice of runtime or language. I'm just curious why you think that "Programming language syntax scarcely matters", as to me that seems the same as saying "How a codebase is architectured and designed scarcely matters".

> Programming language syntax scarcely matters. It does to some extent but we the programmers tend to over-romanticize it. The runtime and its properties are the much better thing to optimize for.

I'm not sure I understand this argument. Java and Clojure share a runtime, but an idiomatic Java codebase is going to have a very different architecture and design to an idiomatic Clojure codebase. Conversely, a codebase written in Go may end up looking very similar to a codebase written in Java, despite using different runtimes.

In the UK there's been a recent spate of nationalist flag flying. Given the artist and location, "blinded by nationalism" is the most likely intended meaning.

Hype around switching from Windows servers?

I'm surprised the article didn't also mention Rich Hickey's metric of complexity; that is, complexity being a measure of how interconnected code is.

https://www.youtube.com/watch?v=SxdOUGdseq4

Typically you're either deploying via a container, in which case there's no more overhead than any other container deployment, or you're deploying directly to some Linux machine, in which case all you need is a JVM - hardly an arcane ritual.

I'll add a note to the cljfmt README to tell people about these commands, as your experience shows that it might not be obvious to people that they likely already have access to cljfmt in Emacs as a result of using LSP or CIDER.