computerdork

Interesting. There is of course be a line between condoned use and unwanted stealing. And now that you’re mentioning it, think there is a higher tolerance for humans using a person‘s ideas versus AI. Think this is not only because a human is a real person and typically is also associated with music who has studied it to some degree, but also because when AI uses your ideas, you’re basically giving free access to them for any of the millions of users of the model. Hmm, Useful to think about

Agreed:)

Agreed... and am realizing, especially with themes/melodies, sometimes it's not always easy to modify them enough to the point that it's unrecognizable, but even then, you could be said to quoting their theme - although, am of course not talking about quoting popular music melodies, no one wants a lawsuit:)

Actually, didn't hear about this, but makes sense, Brahms clearly builds off of Beethoven.

But, have had heard the same thing that was said in the page you linked, that one of the reasons Brahms took so long in his life before he wrote a symphony is that he had to compete with Beethoven, and didn't feel it was ready until much older. Yeah, what an immense weight he must have felt to have to follow up what Beethoven had done!

Am an aspiring composer. We often say we have no qualms about "stealing" from other composers. Yeah, if we see an idea or technique used in piece of music we like, we won't hesitate to try it out in our own pieces. Of course, we don't actually steal to the point of plagarism, but it's pretty common to do, and the ideas we take are often modified for your own needs or just taken in parts.

Seems similar to what is happening with software developers.

Very useful info. Didn't realize model releases often reused the same pretrained base-model and just enhanced the post-training RL. And also that contractors are creating tasks and solutions for this post-training. Super interesting and makes a lot of sense.

... And yeah, agree with your statement that this is not emergence. But to me, it's still impressive, because the more post-training tasks they create, the more the model can do, and this set of abilities is only going to keep growing and growing.

... Also, if you wouldn't mind one last question, have read that the AI companies have started to do self-training, where the models are starting to train themselves. Might this lead to even faster and faster increase in model capabilities? ... although, based on what you're saying, maybe it just leads to a model reaching a its max plateau for existing capabilities rather than an explosion in new ones. Because, as you mentioned, for a model to learn an ability, it still needs a human to create a training dataset for it. Is this your understanding?

haha! Speak for yourself, but software never judges you or cancels on you at the last minute when you're supposed to meet up at the bar. "Friends," sheesh! :)

Also interesting. Do you happen to know if once they've used those thousands of contractors to teach the model something like Blender (or some similar app) using RL, when they training a new model, do they need to use same contractors again to teach that new model the same behaviors?

The reason I ask is even though it can take hundreds or thousands of contractors to teach a model a certain behavior, wonder if they really only need to do it once for each desired behavior? (of course, future models might expand and refine this previous training) Because if that is the case, then wow, then future models can really expand their capabilities really very fast.

...Am wondering if they somehow record a training session so they can play it back whenever they need to train a new model with the same info? Or maybe the new models can just use distillation from the old model to relearn the old behaviors?

Interesting. Have heard about the need to create synthetic data for LLM's, but didn't realize it was such an important factor. Although, how big a factor synthetic data is the next question, but guess there is no way to truly verify how much difference it makes with these closed models.

Ah, I see, LLM's aren't able to do incredible things and you were just commenting on that. Sounds reasonable

eh, they are also used extensively in medical research using alpha fold, and in math, and in legal for one thing. And this is of course only the beginning, and IMHO, it'd take a lack of foresight to not start putting layers and layers of safeguards now.

Yeah, to me the hugging face hacking is a harbinger of what's to come, not just in the world of tech but eventually to other fields too.

Ah, didn't think of this. Some rogue militia group might try to use this LLM (or create their own LLM based on this work) to help them create biological weapons or to do a mass hacking the infrastructure of targeted country.

Wonder what safeguards Kolibri uses to prevent this? Or if they even can

:-)

Haha, am not trying to criticize you here, but feel like your comment is very ironic, to the point of being squarely in the camp of hypocritical (haha, you know, “performative cynicism”). It’s all good:)

For me personally, when doing development with LLM's, you've won me over towards using safer languages rather than looser ones. Because for one thing, the developer working with an LLM still has to remember to ask it to check for things like memory leaks and security issues. And (as I understand it), LLM's are statistically in the way they work and some level of randomness is always apart of their answer, so there is always a chance they will miss something. Interesting

Yeah, in my personal opinion, seems like Boshalfoshal's conclusion was too strong for me too, but I haven't done development enough with LLM's to make my own determination.

Interesting. Have heard many times that Google has become risk adverse. Maybe fomr something like Android, they just don't want to rock the profitable boat that it has become by switching out the core OS? ... Also now that we're talking, maybe it's just too much work to do so. Backwards compatibility seems like it'd be a nightmare, especially with all the device drivers.

Had the same thought. On the wikipedia, it only mentions Fuschia used on the Google Nest Hub, which probably means it's used on a decent number of devices, but would think it was such a great OS, they would have used it for something like the upcoming GoogleBook.

Hmm, I don't disagree with you that LLM's remove the need for type-safe languages, but as the blog mentioned, Google is porting their C++/C code to rust. Does this mean the port is waste of time and that they should just rely on the LLM's to catch memory errors?

Agreed, the murder scene is very realistic in its "make it up as you go along" feel to it. Really also liked the scene where Raskolnikov, after days of carrying the tremendous burden of guilt and shame of these murders, finally confesses what he did to Sonya. You could feel this enormous weight being lifted off his shoulders. Both great scenes.

Re-read Crime and Punishment - read this as a teenager in high school, and (not suprisingly) think at the time had missed the main point behind it (and warning, if you haven't read it, spoilers ahead!).

When I was young, thought that Raskolnikov was an example of an intelligent, moral person who was struggling with higher concepts which led him to murder. And moreover back then, thought he was a overall a good protagnist who was just in an impossible situation.

Now, realized the novel is more about psychology and how people who are overall good and others who are overall bad make choices, and then they have to mentally deal with these decisions. Yeah, think Dostoevsky is saying Raskolnikov was actually very misguided in the reasoning behind his murders. Eventhough his intentions were in many respects actually good, they were egotistically and overly idealistic. Eventually, Raskonikov realized he had deluded himself and that he was hyperinflating his own merits and value as a person over other people. And also, at a more basically level, he struggled with his conscience on whether to avoid being caught, commit suicide or turn himself in. He himself didn't believe he was an immoral person, and when he realized he had deluded himself, he struggled in the end on what course of action to take.

One downside of the book is by today's standards, it is very long-winded and often take several pages to go through something that would take a single page by many writers today. But think this also makes it more realistic, because most people in actual life don't get to the point that quickly. Really fascinating.

Agree with your take on it, the struggles of the characters are so true to life. The world feels entirely real, while still having a sense of beauty. She's an amazing writer

Middlemarch is so amazing. And for me at least, really like George Eliot's writing, maybe even more than Austen. I heard it said that George Eliot is the one author that writes for adults, think this is true. As much as I enjoy Austen, Eliot's world and characters are more real and true to life. The characters are more balanced, almost all with strengths and weaknesses, and in many cases, these weaknesses either leading to their downfall, or something they mightly struggle to overcome.

Also, Eliot's language is incredible. On one hand, it's a lot more difficult to read than Austen, but on the other, it's so descriptive while still often being laced with a sense of beauty. Really wonderful prose.

Ah, didn't know the term for this, but vaguely remembered articles multiple models being trained together. Good to know:)

Umm, as a man myself, I've got to say that I am always well behaved and have never gratified my sexual needs, ever! Sheesh, you neanderthal (haha:)

Eh, think you might not have read my comment well enough, because actually I partially agree with you in that two models may not be the best way to train an LLM. Yeah, I did say "In AI, maybe two models should be developed and trained in conjunction," meaning this is just some random idea based on the two system idea that might be useful.

Was just implying we don't know all the useful ideas (not just in tech but in medicine, genetics, evolutionary bioligy, mathematics...) that might be developed because we now know that the brain is two separate systems. It's all good though.

Ah, I stand corrected:) Maybe a better diagram might be to merge our ideas:

Fish Brain = brain stem + pallium + other parts of brain[1]

      |
 (evolved into)
      |
      V

Human Brain = brain stem + gigantic cortex(which directly evolved from the pallium) + other parts of brain[1]

Good to know:)

[1] Am using "other parts of the brain" to cover any parts of the brain not covered by the brain stem and cortex

am no expert in brain development, still have read about evolution. I like aspects of the way you mentioned this, but seems that putting cortex separately from fish brain isn't quite right. As you mentioned, the cortex is simply a part of the fish brain that evolved much (much, much) larger than a fish's. Maybe:

Fish Brain = brain stem + small cortex (+ other parts of brain)

      |
 (evolved into)
      |
      V

Human Brain = brain stem + gigantic cortex (+ other parts of brain)

Sorry, know this is splitting hairs, but just thought it a useful distinction to make.

Interesting, but for me at least, this new research did not seem to promote the dispelled hypothesis about the reptile brain. Because the reptile brain hypothesis says a primitive reptile-brain evolved first (lower brain), then a separate part of the brain (upper brain) evolved later that was responsible for complex thought.

Actually, the upper and lower brains evolved together and most animals have both, including reptiles.

Didn't read it thoroughly, but your [2] reference seems to be saying this too.