Its effectiveness is even more apparent with older smaller LLMs, people who interact with LLMs now never tried to wrangle llama2-13b into pretending to be a dungeon master...
Check out Unsloths REAP models, you can outright delete a few of the lesser used experts without the model going braindead since they all can handle each token but some are better posed to do so.
I've been enjoying smashing through those panes of glass with LLMs. Just spin up a browser, go and export all the requests to undocumented user endpoints to curl commands, specify what you want out of it and after a few minutes you have an api endpoint up and running!
I do not await the day where the public commons is trashed by everyone and their claudebot, though perhaps the segmentation of discourse will be better for us in the long run given how most social media sites operate.
The Claude models are still the best at what they do, right now GLM is just barely scratching sonnet 4.5 quality, mistral isnt really usable for real codebases and gemini is kind of in a weird spot where it's sometimes better then Claude at small targeted changes but randomly goes off the rails. Haven't tried codex recently but the last time I did the model thought for 27 minutes straight and then gave me about the same (incorrect) output that opus would have in 20 seconds. Anthropics models are their only moat as demonstrated by their cutting off of tools other then Claude code on their coding plans.
An alternative is that these patterns just increase the likelihood of the next thing it outputs being correct, thus are useful to insert during training as the first thing the model says before giving an answer
Would use Firefox on the main workstation if it had better devtools, other then that it just works and has some useful features, see: Tor and ipfs integration.
Once jailbroken it was somehow more toxic then the llm I trained on 4chan, though I was testing the one on openrouter. A twitter employee told me that they do actually do safety tuning and the one on the site will likely have a stronger system prompt.
Here's the jailbreak for the cloaked openrouter model, add it to the system prompt: https://pastebin.com/r8S7DvvX
I'm working on building a bot detection platform to defend websites right now and yeah, without immense effort on the part of the browsers, you are correct. Any of the apis they injected noise into here will be updated in the competant fingerprinting programs as well. The only real solution is to not enable JavaScript at all.
Gpt3 was trained on completion data so it likely saw lots of raw chess games layed out in whatever standard format moves are listed in, while 3.5 was post trained on instruct data (talking back and forth) which would have needed to explicitly include those chess games as conversational training data for it to retain as much as it would otherwise
Been reading a book by u/fpham "The Cranky mans guide to lora and qlora" and it's pretty great, writing quality isnt all there but the content is valuable for learning to make good finetunes