ntonozzi

People bike and ski that fast every day with no protection other than a helmet.

I wish the writer would let us know if the total number of activations went up or down.

> Residents interested in adding vacant lots or side lots to their properties objected that those additions would have increased their tax bills as if there were a house on the added land, even if it was just an empty lot.

From https://www.washingtonexaminer.com/the-short-life-of-pennsyl....

By adding the features that those new age startups launch: https://www.elastic.co/guide/en/elasticsearch/reference/curr...

Building a classic text search engine is way harder than building a KNN engine, and bolting a KNN engine into a term search engine is easier than the other way around.

Yes! Check out this paper describing how Linear Transformers are secretly Fast Weight Programmers: https://arxiv.org/abs/2102.11174.

What are the most impactful and likely features that you'd like to see merged into CPython?

How reliable is surgery?

It uses Claude from anthropic which has a 100k context window. It typically stuffs the 15 most relevant files into the prompt to spit out the output. It’s shockingly fast.

I really like it for figuring out how something in my code base is done, like figuring out where exactly rate limits happen and what service we use to track them.

Insulin resistance causes weight gain, and insulin can cause insulin resistance right? It's complicated but not implausible.

Even with this setting enabled (for Safari), it still asks me every week or two where I want to open the link and tries to get me to download Chrome. I doubt they do this if you have Chrome enabled.

The most effective architecture for LLMs without tokenization is Meta's MEGABYTE architecture: https://arxiv.org/abs/2305.07185. The sacrifice here is that they have two different types of layers, instead of the simple uniform transformer layers.

The episode focuses on Utah’s housing first program, not California. They also bring up that it wasn’t a panacea and in practice it took more time than expected before the cost to the public went down.

Why do people think that Meta released their model in order to get open source coders to improve their models? They will get absolutely no competitive advantage from this. Every other team developing a closed source LLM can easily copy the innovations that open source coders have applied to Llama on their own, closed source models.

There's no advantage here. Meta just spent $10 million on releasing fun chaos into the world and increasing their recruiting power.

Why did they say they got this working in the browser? They should be more clear about which runtimes support which system calls.

How do they get networking and sockets working in the browser? As far as I know, no browsers allow raw network access. Some apps can get around this by using things like a websocket proxy, but this relies on routing your network traffic through a third party. It'd be great to see this solved and to let users host ephemeral network servers from their browsers.

It’s great for prototyping and seeing what is possible but for running in production you’ll likely need to write it yourself, and it will just take a few minutes.

I am not an expert, but I think that they are structurally identical only in decoder only transformers like GPT. The original transformers were used for translation, and so the encoder-decoder layers use Q from the decoder layer and K from the encoder layer. The attention is all you need paper has an explanation:

> In "encoder-decoder attention" layers, the queries come from the previous decoder layer, and the memory keys and values come from the output of the encoder. This allows every position in the decoder to attend over all positions in the input sequence. This mimics the typical encoder-decoder attention mechanisms in sequence-to-sequence models such as...

How does this work? I've seen a cool project about forcing Llama to output valid JSON: https://twitter.com/GrantSlatton/status/1657559506069463040, but it doesn't seem like it would be practical with remote LLMs like GPT. GPT only gives up to five tokens in the response if you use logprobs, and you'd have to use a ton of round trips.

I requested access on March 14th or 15th and got it on March 20th.

Have you tried VS Code or WebStorm? They are both independent debuggers. They are quite good for debugging in my experience. I don't know that they offer profiling capabilities.

WebStorm and VSCode both have OK debuggers, but in my experience they are even more finicky than the Chrome one, which as you noted does have issues. Firefox and Safari are likewise fine, but not as good as Chrome.

More importantly though, if you are that distrustful of Chrome, you probably should not be using Node because it is based on v8, the Chrome Javascript runtime.

I’m not saying it’s easy but it’s much better than tokens IMO. I think bytes would be understandable too.

Why would that be annoying? It’s much easier to understand, predict and truncate appropriately than having to explain all of these different tokenization schemes to devs.

That article is so weird:

> Of course, there is a selection bias in who comes to me. There are people out there doing just fine on 5000 units of Vitamin D daily. I only see the ones who develop high calcium levels. But I see enough of them to know that this is not an exceptionally rare occurrence. I have been to lectures in which physicians have claimed that Vitamin D toxicity almost never occurs. In my experience, this is false. I have seen many cases of Vitamin D toxicity in people who were taking the recommended dose from an over-the-counter bottle.

She is an extremely specialized professional who is referred and treats people with this condition. Of course it's not rare in her experience! For all we know, she could be one of a hundred parathyroid specialists in the country, and only 1 in 100000 people who take Vitamin D get Vitamin D toxicity. The author should really provide some evidence on how common this is.

And toothbrushes too, did you know in some countries it is legal to brush your teeth and buy a toothbrush even if you didn’t see a dentist in the last year?

Not soon enough, the iOS transcription is terrible compared to even the small whisper model. I really hope Apple revamps voice recognition this fall, but I'm not putting money on it being Whisper quality.

Thanks for the helpful explanation. What is the context length in this explanation?

Nuclear waste from power plants is not a legitimate issue: https://thebreakthrough.org/issues/energy/the-boring-truth-a..., https://zionlights.substack.com/p/everything-i-believed-abou....