adtac

There's definitely some lossy compression when you snap it to the nearest known vector: enumerating every word ever written in human history wouldn't even come close to the 2^(16*D) representable points for a D-dimensional float16 embedding vector. In fact, even adding two float16 values is a form of lossy compression for most additions.

But I'd be surprised if either of those were the primary reason. The words "sea" and "ocean" are different vectors but they'll be very close to each other. salt + water = sea and salt + water = ocean both sound correct to me so the problem is more about whether the v_salt + v_water can even get to the vicinity of either v_sea or v_ocean.

Addition won't work for things that depend on the order of operations. If salt + water is ocean and water + fire is steam, what's salt + water + fire? Is it salt + steam or ocean + fire?

Associativity and commutivity in vector addition doesn't translate well to semantic meaning. Extrapolating your example, it'd also mean:

    v_king  - v_queen   ~= v_father  - v_mother
    v_king  - v_royalty ~= v_father  - v_parent
    v_king  - v_father  ~= v_royalty - v_parent ~= v_queen - v_mother

I don't see why those should all be true. Intuitively, trying to satisfy O(N^2) semantic pairings with vectors that are optimised for a very specific and different numerical operation (cosine similarity) feels like something that won't work. I'd imagine errors get amplified with 3+ operands.

>Competing interests: None, apart from owning a bicycle as well as a unicycle.

Rare earth metals aren't actually rare. It's a misnomer. Some of them are pretty common. It's just that they're hard to extract and refine.

It's funny you mentioned Google's internal infra because my motivation for this was to hack together something to emulate the kind of static fat binaries deployed on Borg.

Of course! Bash script is Turing complete so it should be possible to implement everything in it :)

The only upside to having an executable Dockerfile is that it's still a valid Dockerfile that you can use with docker build, docker-compose, etc. in addition to being able to execute it.

As you can imagine, it wasn't a fun developer experience building this incrementally without build logs. This was the only way I could find to have the cake (logs) and it eat it too (sha).

I'm probably getting banned for committing this war crime but have you considered a

    #!/usr/bin/env -S bash -c "docker run --privileged ..."

    FROM docker
    
    RUN <<EOF cat >/other.Dockerfile
      #!/usr/bin/env -S bash -c "docker run ..."
      FROM debian:buster
    EOF
    RUN chmod +x /other.Dockerfile
    
    CMD bash -c "/other.Dockerfile &; ./main"

I'd take it even further: make peer review public and open to all members of that community. Imagine a forum-like discussion anybody can anonymously review any submission and all reviews are public.

Becoming a reviewer should still be invite-only and the system should keep track of the reviewer's identity behind the scenes to monitor for abuse, of course. The review can include coarse-grained reputation signals like "has reviewed 100+ papers in the last 5 years".

It might be worth embargoing reviews with a fixed time delay before making them public to prevent bandwagon effects and disincentivise review plagiarism tho. The reviewer's identity should be deanonymised too after something along the same time scale as when the paper author's identity is revealed.

Colour e-ink displays exist. We don't live in spaceships yet but this is just as good.

> In September an average of 1,581 shares a day of eis were sold short, representing 17% of the daily total trading volume. On October 2nd, five days before the attacks, a whopping 227,820 shares were shorted

It’s misleading to compare an average with an extreme, especially since October 2nd is the first Monday of the month and the fourth quarter (probably books being balanced). It would be better to compare with the maximum in the whole of September or July.

[tangential to this really cool demo] JPEG images being the only possible interface to GPT-4 feels wasteful. the human eye works the delta between "frames", not the image itself. I wonder if the next big step that would allow real-time video processing at high resolutions is to have the model's internal state operate on keyframes and deltas similar to how video codecs like MPEG work.

Is "it's just processing frames" the new "it's just predicting the next token"?

WebRTC isn't great for non-realtime streams where quality is much more important than latency. I hope p2p WebTransport gets standardised soon.

I agree that it's very unlikely for someone to learn Go in a week and start writing flawless code.

But Go's real strength is in its readability, not writability. I think it's very much possible to learn Go in a week, then read clean Go code like the standard library and understand exactly what's going on. At least that's my interpretation of what it means for a new grad to be productive in Go in less than a week. Nobody is expecting someone new to write production-grade libraries with intricate concurrency bits in their first week, but they're already productive if they can read and understand it.

As a rule of thumb we spend 10x more time reading code than we do writing it (code reviews, debugging, refactors). So why not optimise for it?

That's what the `func foo() (*T, error)` pattern is for. It's actually better than syntactic sugar for optional values because now you also have a descriptive reason for why the value is nil.

But if you really cannot afford to return more than one bit of information, do `func foo() (*T, bool)`.

> Nevertheless, they are frustrating and hamper the learning process

But that is the learning process. What else is there to learn in a language if the syntax doesn't count? They're all Turing complete and all of them can do everything. All we need to do is learn the exact magic words.

I'm sorry but nearly all of them are along the lines of "I came from language X and in X we did it this way, but Go's syntax is different". That's not a footgun.

You know what's a footgun? Uncaught exceptions popping up in places far away from where they were created at which point you have very little context to deal with it robustly. Use after frees. FactoryFactoryFactories.

I'm not sure if that was the best example to showcase NilAway. I understand there's a lot of context omitted to focus on NilAway's impact, but why is foo returning a channel to bar if bar is just going to block on it anyway? Why not just return a *U? If foo's function signature was func foo() (*U, error) {}, this wouldn't be a problem to begin with.

Imagine a multidimensional matrix with various payment methods, local regulations, cloud providers, third party dependencies, web/mobile platforms, etc. Then also add more dimensions for internal things like accounting, hiring, payroll, promotions, compliance, security, monitoring, etc. Then double it for Uber Eats or whatever.

There's a lot of overlap and some invalid combinations, but you're still left with a huge number of combinations where Uber must simply work. And every time you add a new thing to this list, the total number of combinations grows polynomially.

(Also, Go is slightly more verbose than most languages. I think that's a feature and not a bug, but it's one more reason.)

Maybe the real scaling hypothesis was the friends we made along the way

We, uh, fixed the glitch^W^W all bugs

Certificate pinning is easy to work around if you know which bytes to change, so that's what adblocking will be

This is slightly tangential, but has anyone recently used ChatGPT for non-trivial programming tasks where you start typing a precise description of what you need from the system and by the time you finish writing it you realise there's a simple way to do it? It has happened to me several times and it was as if serialising the problem was the hard part, like that famous Einstein quote.

(1) Twitter's data is accurately timestamped, (2) there's new data constantly flowing in talking about recent events. There's no other source like that in English other than Reddit.

AFAIU neither of those are relevant to GPT-like architectures but it's not inconceivable to think there might be a model architecture in the future that takes advantage of those. Purely from a information theoretic POV, there's non-zero bits of information in the timestamp and relative ordering of tweets.

The malware will then rely on actual ptrace behaviour as a check. You could instead use seccomp_unotif and let the target ptrace itself as much as it wants: https://man.archlinux.org/man/seccomp_unotify.2.en

Do you remember when ISPs would redirect unresolved DNS A record queries to their “search engine”?

The Schwarzschild radius [0], which defines the radius at which the escape velocity equals the speed of light, is given by 2GM/c^2. I don't know what the impact parameter is either, but given these two expressions, it sounds like b is dimensionless.

I don't know how fast the radius of curvature drops off as a function of the Schwarzschild radius, but I'd imagine it's at least R^-1. So assuming a spherical cow^H^H^H oven with gravitational lensing dying out at ~100x the event horizon, we need a Schwarzschild radius the size of a tennis ball in order to still see the curvature a few metres out. The oven needs to weigh ~4x the Earth's mass for that.

[0] https://en.wikipedia.org/wiki/Schwarzschild_radius

    b, err := os.ReadFile(path)
    if err != nil {
      return nil, fmt.Errorf("read %s: %w", path, err)
    }

is so much better than

    b := Must!(os.ReadFile(path))

because when things go wrong, I have exactly the right amount of information I want. Assigning to err magically (it's not even mentioned in the source code) is exactly the kind of thing that'll turn out to be the cause of a subtle bug 6 months later. Why not spend the a couple of extra lines thinking about the error when the context is still in your head?

Additionally I like how error handling acts as a visual delimiter, especially when you use meaningful fmt.Errorfs as strings are usually highlighted with a different colour, making it easy to quickly jump through code.

I consider the stdlib to be Go's best feature too. As a somewhat contrived example, can you name any language that lets you write a HTTP/2 TLS endpoint that computes the HMAC of a PNG file's pixels without any dependencies? And if something is still missing, it's probably in golang.org/x (which is basically stdlib)!

After that, I probably consider readability at 3am [1], defer statements, explicit error handling and fast compile time to be the most important.

[1] Readability of not just your code: being able to go-to-definition into stdlib and immediately understanding it without having to grok a million unrelated decorators/FactoryFactoryFactory/std::_Vector_iterator<std::_Vector_val<std::_Simple_types<<block>>> is incredible