choxi

Any suggestions for creating training data? Did you just manually create your own dataset or did you use any synthetic methods?

If anyone’s looking for alternatives, Lemmy is federated and nice to use: http://lemmy.world/

It’s really only lacking the size of Reddit for me, there isn’t a ton of activity in niche subs, but they’re growing.

40% of 3% is 1.2%, where’d you get 0.9% from?

How much is that? The grand prize for the Vesuvius Challenge is $700k

you might be thinking of Elon’s claim that the CyberTruck would be semi-amphibious

But there are a lot of selection mechanisms that filter out the bad generated content. For one, people will publish a fraction of the content they generate, and it most likely will go through some more editing and selection steps by real people. Then the Internet itself becomes like a decentralized labeling service, where there are various signals you could detect to identify different qualities of content. What ends up being crawled for the next training iteration is a societally processed version of the raw model output that might have contributed to generating it.

It's kind of interesting to think we might all be contributing to the training data of AGI not just through generating content but also what we choose to share or upvote.

I watched Hoffman’s Lex Fridman interview, it sounds like his idea is that our perceptions are like a user interface that abstracts reality so we don’t perceive reality as it actually is.

Isn’t that well accepted already? We know we don’t sense everything, and our brains give us a highly compressed version of the senses we do have. But none of that matters when we use tools like cameras or microphones because they’re not constrained the way our senses are.

Is there anything more to his idea than that?

Isn’t the measurement problem still potentially linked to consciousness? We don’t know why we experience one thing when reality is actually in a superposition of possibilities. It could be that our measurement devices cause quantum collapse, or they too exist in a superposition until we measure the measuring device.

What’s the “fingerprint idea”? That’s not mentioned in the article.

I don’t see why this theory contradicts the one in the article, in your own examples you’re using bodily states as the input to emotional processes.

Where is the name "Wonderlust" coming from? I don't see it anywhere on the page

I’ve been meditating casually for almost 10 years, I couldn’t sit still for a minute when I started and can sit regularly for an hour now.

I still have a lot of skepticism about the sensations that other meditators describe feeling, like jhanas and profound states of tranquility and whatnot. I have experienced deeper calmness and have made some insights about my brain (eg thoughts are like a sensor organ not unlike a constantly generating LLM), but I haven’t experienced anything like the sometimes superhuman claims that come from people in this community. Naval Ravikant has claimed he can experience psychedelic states just through meditation, Nick Cammarata at OpenAI makes claims about achieving perfect equanimity and being able to feel jhanas that are better than sex but also doesn’t feel desire for it because of aforementioned equanimity. There are many others on social media who make similar claims. From the outside, all of these people seem like normal humans, maybe calmer and wiser than usual, but who still seek material comfort and the validation of others just like anyone else does.

I wonder, what do others make of these superhuman meditation claims? How does one verify that their sensations during meditation are real reflections of how the brain works and not just increasingly subtle hallucinations? If you convinced everyone that the brain has a secret mechanism to feel perfect happiness, wouldn’t a lot of people hallucinate that they’ve experienced it even if it wasn’t real?

I had flashbacks to Backbone, Angular, Ember, Angular 2, and Vue when the author said React was early to the scene

Would a doctorate work? Here’s how Shaq did it: https://academiainsider.com/does-shaq-have-a-phd/

There’s always been fuzziness between data and code, it’s kind of the core concept of Turing machines.

“Somewhere between the janitor and the CEO, reasons stop mattering.”

The cycle of ripe innovation to ossified industry is complete

That’s some expensive free marketing

Did you consider just using LLMs for your product? Even with LLMs it doesn’t seem trivial to format the survey data, design the prompts, validate them across a large dataset, and present the results with nice charts and links back to the original text etc.

With LLMs I’d imagine you could do even more interesting things with analyzing survey data. It seems like the core concept of your product is still useful, and could potentially do more now. Are you actually losing customers to GPT or just feel the end is near?

Also would love to read that future blog post! Thanks

I don't understand or appreciate what the mystique of creativity is. The inside of the black box could just be your brain randomly remixing concepts and evaluating the variations. Most of them are probably nonsensical (“purple monkey dishwasher”) but some of them are interesting (“monkey dishwashers”) and then you take that idea and keep mixing it. The ideas that gets mixed are just whatever you’re able to pull into context, so ideas you’re familiar with are more likely to get mixed. Sometimes you struggle to come up with creative ideas, and it’s because you’ve run out of new ideas to mix with so changing your environment helps by injecting new ideas.

But at its core creativity is, at a practical level, just remixing ideas and applying some value metric against them.

Regarding the article, you can apply creative remixing at multiple levels so a creative nonfiction writer may start with fixed facts but can still remix the order, presentation, format, where to begin/end, etc.

You can even get a full self driving autopilot automatic automobile and still need someone to drive it

Agreed, there’s a lot more design space to explore with prompt chains/flows. But a single prompt requires design too especially if it’s dynamically generated. You have to choose what information to provide, frame the question or task you want it to complete in a way that provides good answers and is also extractable from the completion, and fit within the token limits (32k tokens is a lot but expensive and still small for some tasks).

This article only covers the simplest type of prompting you might do.

I was curious because it doesn’t sound like a typical Indian name, apparently it’s a nickname for Hyderabad because there’s a large tech industry there

This is possibly my favorite PG essay, the metaphor about gliding seems very relevant to your question: http://www.paulgraham.com/hs.html

My two cents: If you’re interested in engineering and physics, you should explore that. Does your university have a CompE or EE program? At mine a lot of the classes between different eng programs overlapped so it wasn’t difficult to switch.

But you could also just pivot because no one in the tech industry really cares if you have a CS degree, and it sounds like you’d still enjoy coding anyway so you’ll keep learning on your own. The CS curriculum is going to feel painfully old fashioned compared to what you’re learning on your own, though there are some useful fundamentals in there.

I believe it’s that Bloom’s study implies that 1-on-1 pacing and teaching produces the best learning results, but that Bloom thought it was infeasible to scale to everyone: it’s “too costly for most societies to bear on a large scale”

Data truly is the new oil. When it’s all done the compute costs and code will be cheap or free. There’s a lot hinging on how we interpret copyright laws or what kind of data rights laws we enact.

Also keep in mind that his reference point for “overhype” is people claiming AGI within 2-3 years.

3 and 4 seem somewhat valid. 1 and 2 only apply to a minority of people so Walker’s statements are good advice for most people. 5 is just ridiculous, it’s reasonable to average 7-9 as a recommendation of 8 hours.

This seems incredibly nit picky. My summary of a few points below:

#1: Walker claims sleep deprivation decreases life span, but so does sleeping too much.

#2: Walker says sleep deprivation is always bad, but sometimes it helps people with depression.

#3: Walker says that lack of sleep alone may kill you because it does for people who get "fatal familial insomnia" (FFI), but insomnia is a symptom and not the root cause of FFI.

#4: Walker says the WHO claims we have a "sleep epidemic", but Walker's citation does not actually show that and there is conflicting research showing we may actually be sleeping more.

#5: Walker said the WHO recommends 8 hours of sleep, but it's actually the National Sleep Foundation and they recommend 7-9

Here's Walker's response to some of this: https://sleepdiplomat.wordpress.com/2019/12/19/why-we-sleep-...

EDIT: I added points #3 and #4 after just to be more charitable and give a full summary instead of just the ones I felt were most nit picky.

The earlier GPT-3 models aren’t really like that, I wonder if their army of supervised trainers for InstructGPT were just instructed to write like that?

the answer was (paraphrasing) “not six decades, not five decades. Maybe in a couple decades we’ll have a functioning plant”