Well, I resigned my postdoc and decided to pursue "Science 1" as he terms it. Let's see how successful I am in a year or two...
1,676 karma · joined September 26, 2023
Prompting an LLM not to confabulate won't actually prevent it from doing so. It's so disappointing to see an organization like this, that's mission is to inform the public, used AI not understanding the limitations and then making a claim like this.
It certainly doesn't help that the pay is atrocious — I make less in inflation adjusted dollars than my first job straight out of school nearly 20yrs ago, in a higher cost of living place.
Needless to say, I've only focused on roles that fit that narrow profile. One of the recruiters that contacted me didn't even know I worked in games, despite it making up the bulk of my work experience (including as a lead developer).
Considering how closely I match this narrow profile, and the number of people that likely do, it's weird how low my callback ratio has been.
I wonder if they are optimizing for batching, which might introduce even more delays.
For example, searching for climbing comp videos and getting a completely unrelated video about some new tech gadget released within the last couple days from a random popular content creators makes no sense.
Clearly, it works for Google (content creators intentionally make click-baity thumbnails and titles because Google encourages it), but it's user hostile: it's designed to suck you into a vortex, which is not what the user was intending in the first place.
That said, all content platforms do this right now, so my intention isn't singling out Google. It's frustrating nonetheless.
This is patently false. The content is not theirs, users grant Reddit a license to their content. See the TOS [1] you spoke of:
> You retain any ownership rights you have in Your Content, but you grant Reddit the following license to use that Content:
This isn't just some bit of semantics either. If you can contact the users, you could request a license for the content yourself. So really the only bastion Reddit has is technical measures for preventing scraping, since publicly accessible data is allowed to be scraped in the US due to legal precedent. That is, nothing in the TOS prevents other parties from displaying the data in other formats (as long as they don't use the API). The thing is, all the apps did make use of the API because they were not adversarial with Reddit, as opposed to webapps like libreddit that circumvent the API altogether.
Since OpenAI did not specify what exactly is in their reasoning trace, it's not clear what if any difference there is between the approaches. They could be vastly different, or they could be slight variations of each other. Without details from OpenAI, it's not currently possible to tell.
EDIT: Note this was run over a dataset of short stories rather than the novels since the API errors out with very long contexts like novels.
Here's the updates to the model config on huggingface:
https://huggingface.co/mattshumer/Reflection-Llama-3.1-70B/c...
That said, I'm withholding judgment on how likely the claims are. A friend who developed NoCha [1] is running the model on that benchmark, which will really stress test its ability to reason over full novels. I'll reserve judgement until then.
It is now firmly established in the case law that Section 230 cannot act as a shield whenever a website is in any way complicit in the creation or development of illegal content.
...
In FTC v. Accusearch,[69] the Tenth Circuit Court of Appeals held that a website’s mere posting of content that it had no role whatsoever in creating — telephone records of private individuals — constituted “development” of that information, and so deprived it of Section 230 immunity. Even though the content was wholly created by others, the website knowingly transformed what had previously been private information into a publicly available commodity. Such complicity in illegality is what defines “development” of content, as distinguished from its creation.
In writing Section 230, Rep. Wyden and I, and ultimately the entire Congress, decided that these legal rules should continue to apply on the internet just as in the offline world. Every business, whether operating through its online facility or through a brick-and-mortar facility, would continue to be responsible for all of its own legal obligations.
EDIT: Added quote about additional legal obligations.
[1]: https://jolt.richmond.edu/2020/08/27/the-origins-and-origina...
Reminds me of the research that shows that 87% of people in the US can be uniquely identified with only three pieces of information: date of birth, gender, and zip code [1].
[1]: https://dataprivacylab.org/projects/identifiability/paper1.p...
I think the ultimate problem is that social media is not unbiased — it curates what people are shown. In that role they are no longer an impartial party merely hosting content. It seems this ruling is saying that the curation being algorithmic does not absolve the companies from liability.
In a very general sense, this ruling could be seen as a form of net neutrality. Currently social media platforms favor certain content, while down weighting others. Sure, it might be at a different level than peer agreements between ISPs and websites, but it amounts to a similar phenomenon when most people interact on social media through the feed.
Honestly, I think I'd love to see what changes this ruling brings about. HN is quite literally the only social media site (loosely interpreted) I even have an account on anymore, mainly because of how truly awful all the sites have become. Maybe this will make social media more palatable again? Maybe not, but I'm inclined to see what shakes out.
[1]: https://dojoteef.com/papers/virtual_gm_wordplay_2024.pdf
By the time I was in undergrad I had no motivation to study. I'd skip classes and cram the night before for any class that was memorization based. I didn't even buy the books, just went to the library and checked the book out for a couple hours.
Fast forward a bit and I ended up dropping some classes because I was paying so little attention, I didn't even know when the tests were supposed to be. Came in one day and the prof said, "You missed the test last week, I presume you want to retake it?"
That said, I've done well enough having gone back later in life to get a PhD, but I do wonder sometimes if I would have accomplished more if I was forced to push the boundaries of my ability, thus leading me to develop a work ethic earlier on.
Despite my PhD being in NLP, they still seemed incredulous at my assertion. I truly think the general public have not been adequately warned that they cannot trust the outputs of LLMs. I wonder how long until that fact becomes commonplace.
The website comes off as a marketing strategy rather than a fun one-day hackathon project. I think that's why it's getting the reaction you're seeing.
It also supports complex structures like union types, lists, etc. I used it to create cresset [2], a package that allows building Pytorch models directly from config files.
[1]: https://pypi.org/project/confactory/ [2]: https://pypi.org/project/cresset/
It's certainly jarring to hear major events like the assassination attempt from friends and family first, but it's very reminiscent of my experience before the rise of internet news.
Sure, the exact form using the made up word gronk might not be in the training data, but the general form of that reasoning problem definitely exists, quite frequently in fact.
This problem is pervasive and effects everything from food supply (planting genetically identical seeds rather than diversified "heirloom" crops) to businesses across the board buying and gutting their competitors thus reducing consumer choice.
It's a tough problem akin to a multi-armed bandit: exploit a known strategy or "waste" some effort exploring alternatives in the hopes of better returns. The more efficient you are (exploitation), the higher the likelihood of catastrophic failure in weird edge cases.