h_mirin

I wonder if bots get any pleasure from karma farming.

I can be fairly confident that when Google says they don’t see your data, they don’t.

I am also fairly confident that nobody sees the moderation decisions either.

The 2027 incident report will explain how the agents exchanged Morse code through the shared L3 cache.

I'm Japanese, and this is the first I've heard of Gakutensoku. If you asked people in Japan for an old Japanese "robot", I think most would come up with this tea-carrying karakuri doll. https://en.wikipedia.org/wiki/Karakuri_puppet

That one predates the word "robot" entirely. The design is documented in a 1796 book called "Karakuri Zui", so Edo period, shoguns still in charge.

Previously in my ~/Downloads:

linear_algebra_done_right.pdf, 0 pages read, July 2023

linear_algebra_done_right (1).pdf, 0 pages read, Oct 2023

linear_algebra_done_right (2).pdf, 0 pages read, Sept 2024

Downloading (3) now.

This is really an argument about test-time scaling, even though the post never uses the term.

These days "test-time scaling" mostly means letting the model talk to itself for longer, but the first genuinely surprising results came from plain sampling. Google's AlphaCode generated millions of candidate programs and filtered them down to a handful of submissions, which beat the average human programmer in 2022, before ChatGPT even showed up.

Sampling is what AI is good at. Making examples and doing LeetCode are similar in that verification is clear and cheap. Compared to that, "proof" is still a vague concept, except where Lean works. See the fuss over the ABC conjecture. So humans are still needed.

The interesting question to me is what happens after enough learning from "sampling." Isn't AlphaGo's move 37 an AI's nose? If that happens in mathematics, we may end up with results that are correct, machine checkable, and not explainable in any way we find satisfying.

In Japan, these 22GB 2080 Tis have been showing up on flea-market apps for months now, and I suspect many of them may be coming from China. I’ve been wondering whether buyers realize they’re modified cards, not factory models.

Every time I see this kind of story, two things bother me.

First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining your fine-tune.

Second, the $500 training bill is the cheapest line item in this story. The expensive parts are creating the data and maintaining the model afterwards. How many use cases can actually produce 177k scored episodes? Here they had to generate them synthetically from Amazon Berkeley Objects. To me, that dataset is the strongest evidence in the article of how hard fine-tuning is to apply: if the data existed naturally, nobody would need to manufacture it.

Business Source License https://www.infoq.com/news/2023/08/hashicorp-adopts-bsl/

Interesting! It would be great to have some example outputs in README.md maybe.

Are you all discussing web development? Then, I assumed that most web developers use ORM. So, I'm purely surprised by the number of people here who are against it.

I agree that abstraction can be leaky, and it's true that you often end up writing some SQL-like code.

However, isn't writing raw SQL potentially dangerous, considering aspects like security, access control etc?

I want to learn your thoughts!

I like fly.io because it's just (much) better Heroku.

I also liked Heroku, so it's sad to see their status now.

Most models in recent years are not made with/for word level representation. Therefore, as you suggest, you need to use older word level emebedding models such as word2vec etc. You can just do "cow" - "animal" and search for the words similar to the vector, using libralies such as gensim in Python.

The link you provided seems to be a joke repo! However, your idea to look into the open source alternative implementations is indeed insightful. I'll take a look at them. Thanks.

Especially in recent years, public infrastructure investment is evaluated by ROI (though not just from an economic perspective). Therefore, if the population declines, the number of projects meeting the ROI threshold may also decline.

Also, I live in Japan and can perhaps provide an example:

In Japan:

- Public infrastructure investment reached its peak in the mid-1990s.

- The birth rate peaked in the mid-1970s.

I'm purely surprised by the activeness of CMake development. I didn't know there was still room for improvement. A debugger is surely nice, considering the complexity of the CMakeLists...

Even in 2021, Google Workspace had more than 3 billion users. I believe that 99% of these individuals had no clue what Markdown was. If they had, perhaps third-party apps that can connect to Google Drive, such as StackEdit, would be much more popular now.

I'm the author of this project. Yes, yet another terminal AI assistant with OpenAI APIs!

I know there are plenty of great projects for terminal AI assistants already out there. But, in my experience, none of these tools completely meet the criteria I consider essential:

Terminal Reading: Most tools are unable to read your terminal, which means you have to manually copy and paste the terminal output elsewhere.

Simple API: Who wants to remember a plethora of complex commands and options? This is the very reason I use these tools in the first place.

Multi-turn Capability: Most tools are designed for single-turn queries due to their lack of memory, significantly limiting their utility.

Direct Command Execution on zsh/bash: Some tools execute commands in a wrapped environment or REPLs to capture the terminal output. This approach can break some commands, which I find undesirable.

Personally, I use this tool very conveniently. I open chat.openai.com less often because I can get enough answers from GPT-3.5 or GPT-4 without leaving the terminal!

Any feedback or comment is appreciated!

In principle, citations should give credible sources of information to the reader and let them reproduce the result ultimately. However, this is impossible for ChatGPT because it generates answers probabilistically and its random number generator cannot be controlled. (Though perhaps this could be partially achieved by setting the temperature to zero through the API?)

I hope open-sourced models will achieve better performance soon.

I didn't know that GPTs have such duplicated tokens! The image in the article means that there're three davids ("David", " david", "david") in the tokens, right?

While I agree with your point, I think it ultimately depends on the timescale we're looking at. In other words, if we're looking at a timescale of several years, it's conceivable that we've been experiencing such leaps several times over the previous rate (or more?) for about the past decade. However, if we're looking at a matter of a few weeks, it may seem like we're stagnating. The point is that we cannot predict if these leaps will continue to occur, and when they will happen.

What might be seen as exponential progress could actually be the accumulation of a few tremendous leaps, I believe. For instance, within the architecture realm, we've witnessed leaps such as:

Stacked neural networks (deep learning) / CNN / Transformer

From the application perspective, there are also areas that have seen significant progress:

AlphaGo (An application of reinforcement learning) / AlphaFold (An application for biological sequences) / ChatGPT (An application of Large Language Models / Reinforcement Learning from Human Feedback)

You may pick different ideas. However, what I want to say is that these great advancements happened approximately once every two or three years. It may take some time before we witness another major leap.

In truth, the majority of research and development does not contribute to exponential progress. It's sad. However, this does not render such effort meaningless. Although it may seem as though there's too much competition within a narrow field, this may not necessarily be detrimental to humanity.

Around 200 readings? Incredible.