jerpint

If they’re launching a cloud-based service, doesn’t this effectively remove the risk of running it locally ?

I wonder if this has to do with catastrophic forgetting to some extent; fine tuning on a large enough dataset to make the RLHF go away. Genius to add the “negative” code intent to the mix

Just tried but it returned an error

I think the big thing overlooked is how much the human steering the models matters. If you know what you’re doing and what changes you need, cursor and other tools make you so productive.

If you don’t know what you’re doing, these things can sometimes produce good code, and sometimes produce things that don’t work at all

you're right - I hadn't noticed! I fixed it now, thanks for pointing it out

will probably focus on getting the text out of the papers first, figures might be a good next step after that

whoa - i haven't yet played with MCP - might be a good first project!

For now , yes - abstracts and other metadata

The other day I tried an open source deep research implementation, and a ton of links returned 403s because I was using an agent. But it is for legitimate purposes. I think we need better ways of identifying legitimate agents working on my behalf vs spam bots

The ability to add watermarks to text is really interesting. Obviously it could be worked around , but could be a good way to subtly watermark e.g. LLM outputs

I think a good analogy is being able to drive a car vs understanding the engine of a car.

Both are useful, but you wouldn’t hire a mechanic drive you around

> In 50% and 90% experimental trials, they succeed in creating a live and separate copy of itself respectively

I mean any half decent coding LLM can literally do

``` from transformers import SomeLLM

model = SomeLLM.from_pretrained(“…”) ```

The hard part will be provisioning the GPUs and actual orchestration at scale to do any kind of actual damage

Similar feeling with Gemini g suite integration

I wonder if this is a good alternative for running potentially malicious tools like random fine tunes of LLMs or coding agent outputs with little to no risk

I really wonder how much the self-supervised data flywheel will be enough to reproduce this model or not.

There are also so many different tweaks that could be made to try to get these models to perform better, kudos to the team reproducing it all in the open

Browser use supports gemini via langchain

Thank you for all of your incredible contributions!

It’s funny because I actually use vim mostly when I don’t want LLM assisted code. Sometimes it just gets in the way.

If I do, I load up cursor with vim bindings.

> This code repository and the model weights are licensed under the MIT License. DeepSeek-R1 series support commercial use, allow for any modifications and derivative works, including, but not limited to, distillation for training other LLMs.

Wow. They’re really trying to undercut closed source LLMs

You can still game a test set without training on it, that’s why you usually have a validation set and a test set that you ideally seldom use. Routinely running an evaluation on the test set can get the humans in the loop to overfit the data

just today using ffmpeg , I was thinking how useful it would be to have an LLM in the logs, explaining what the command you just ran will do

There are so many opportunities for google to improve their services.

For example, I found myself asking Claude about places to see in a city I’m visiting while switching back and forth to gmaps. This would have been a much better experience integrated directly with gmaps knowledge graph

So basically canibalizing Siri ?

Airbnb still has better selection than most platforms, especially for longer term rental. also having comments and ratings of hosts is useful when booking overseas.

It is always possible to contact the host directly and bypass the platform, but only when breadcrumbs of info are left by the hosts (e.g. a logo to a website, reverse image search, etc) however making payments is much more difficult from past experience and not all hosts prefer this option

Of course, if you already are in the city and not pressed for time, you can typically find much better deals in person, especially in cities that are not very tech savvy

I think there will be lessons learned here as well for better agentic systems writing code more generally; instead of “committing” to code as of the first token generated, first generate overall structure of code base, with abstractions, and only then start writing code.

I usually instruct Claude/chatGPT/etc not to generate any code until I tell it to, as they are eager to do so and often box themselves in a corner early on

I should specify that unfortunately I didn’t store the raw output from the LLM, just the parsed code snippets they produced, but the code to reproduce it is all there

Author here: the point of the article was only to evaluate zero-shot capabilities. I’m certain that had I used LLMs I would have definitely gotten more stars on AoC (got 41/50 without). Because I chose to solve this year without LLMs, I was simply curious to see how the opposite setup would do, using basically zero human intervention to solve AoC.

That said, if I cared only to produce the best results I would 100% pair up with an LLM

The code they generated and the outputs (including trace back errors) is all included, you can view them on the post itself or from hf space:

https://huggingface.co/spaces/jerpint/advent24-llm

Author here: all the code to reproduce this is actually all on the huggingface space here [1]

https://huggingface.co/spaces/jerpint/advent24-llm/tree/main

It looks much better than Sora but still kind of in uncanny valley