90 karma · joined August 10, 2015
Big solo projects exist, they're just not coming out as frequently because of, uh, how time works
Your interpretation of retirement is probably more strict than most people’s, but even if it wasn’t, how can it be hard to imagine connecting with a community outside your coworkers and customers? That you don’t have to be employed and monetizing your labor to create anything worth sharing?
Even if you would be miserable without some form of external validation how can it be difficult to imagine that others are capable of simply enjoying the limited time they have to be alive, spending it on things that matter more to them than polishing a deck on SaaS sales metrics? How can it be unclear that the reality of work for the vast, vast majority of humans on earth is a brutally costly and undesirable experience? People die in low-paying jobs they took to survive roughly every other minute.
Good for you that you dig what you’re doing though, I hope you never have to see the alternative.
I found City of Fortune by Roger Crowley to be a very entertaining read, the full story of the Crusade is absurd. Venice completely dedicated its entire economy to producing the Crusade’s ships for years to make this happen.
https://docs.typesafe.ai/model-jaggedness/jev-1.13#generatio...
If its outputs resemble human reasoning, it’s because the encoding and training process managed to capture those patterns and use them to simulate fitting text. There is no real reasoning happening or second-order thought, other than a simulation of that happening through the mimicry of human writing.
LLMs can’t be prompted to perform actual reasoning, but they can be told to generate “thoughts” about what they’re doing that bring out more nuanced detail when they give their answers. This isn’t any more magical than writing out a more thoughtful prompt to get a conditioned answer, it’s just getting the LLM to flesh out the prompt engineering for you in the general direction you want it to go.
That seems rather fundamental to me, the idea that with some generic prompting the model tries to fit what it thinks reasoning looks like and can then take advantage of the additional context that would others be buried too deep to influence its answer.
I suspect that prompting the model to explore “thought” asks it to go down paths of linguistic connections that are related to the topic but not immediately connected to the answer in a way that would immediately influence the top predictions. Bringing summaries of those connections into the token context is a kind of zero-shot training on their relevancy to forming an answer.
To me this is less “reasoning” and more suggestive of the idea that some of the heuristics for data retrieval and question answering we collectively refer to as reasoning have broader applications.
It would help to understand why you feel that it’s immoral; my interpretation is that you are saying it gives people an advantage at a cost that you don’t want to be forced into paying yourself in order to compete.
What this seems to show is that the model does not yet have system-level or architectural guidance on when to employ reasoning, but must be explicitly reminded.