polygamous_bat

> What are your thoughts on diffusion policy line of work?

It's been interesting, for sure! More robust than ACT as long as you are not using action chunking (has the same problem as ACT). Downside is, it can be too slow/require too much data to train. I have a sneaking suspicion that at that scale of data even visual nearest neighbor stuff would be similarly robust.

Of course, no one has figured out an iota of useful generalization, which is sad across the field.

> I don't think ACT (unlike MPC) have an explicit use case for different embodiment. Or what do you mean by two different instances?

I mean two different Aloha setups (2 x 2x WidowX) in the same lab. This is the very minimal level of "generalization" you may expect, barely enough to be called generalization, but still, the learned policy seems to overfit to the individual robot's joint space quirks.

> in the field of BC (Not IRL) what have been the more robust method for your case?

Different things for different definitions of robust. What's your definition?

> A PhD in ML is worth less to me than day to day operational product engineering experience utilizing the fast changing ML tooling landscape

This is how you get a four-week transition from "I can fix twitter search in an internship!" to "twitter search is unfixable." Ideally, a PhD teaches you to bust your behind doing a deep dive on a seemingly unsolvable problem for three-five years.

If only we could recalibrate ourselves to stop the search for easy money and remind ourselves that some problems are just genuinely hard.

> would geohot have been better to have gone to school to learn this and deferred all the work he has done?

Could you remind me what he has done since his jailbreaking day, i.e. in the last ten years, that can't be described as "setting VC money on fire?" Because my opinion would be, that yes, the world would have been better off if he did go to school.

Bigger issue with offline RL in the real world (I.e. not Atari video games) has been the assumption of reward labeling. Who’s giving you reward labels at scale? In my opinion that’s why we haven’t seen any large scale real world success stories using offline RL.

You may enjoy this paper: https://arxiv.org/abs/2307.14619

I am talking about the accumulated error part near the end of the article, so your (uncharitable) assumption is not correct.

That is not the only problem however, because if it were self driving would be solved by 2018.

The problem is central to using ML tricks for anything: when you use maximum likelihood estimates for training anything, you get no guarantees for unlikely situations. That’s why you get self driving cars dragging someone stuck beneath them or LLMs just straight up spewing fiction.

The problem with ACT is very similar to the problem with MPC: if your forward model of the world is not great, it will come back to bite you. So much so that afaik ACT policies can have difficulty transferring between two instances of the same robot.

As someone who is doing robotics and behavior cloning for a while, it’s incredibly funny to see the self-taught genius rediscover what’s been known in the field for 30 years and can be found by walking in to the first two lectures of any decision making/control theory class.

In practice, deep learning based behavior cloning works when there is enough data for the model to interpolate to the next “frame”, the proof of which is GPT and other LLMs whose primary training objective is practically behavior cloning. However, adding a neural network means there are some hallucinations, which can range for silly (for LLMs) to murderous (for self driving). However, this presents a huge opportunity as well, in that anyone who can solve hallucinations can make headway into not one but two billion dollar problems at once.

Expiry of near zero interest rate.

Here’s a question for you: why do we punish people who have already committed a crime?

As a deterrence. This may as well be making an example of a whistleblower to signal potential future whistleblowers to stay silent.

TLDR is its fair use when Google takes your data and stealing when you reverse engineer their “intellectual property” /s

Wasn’t the tweet recommendation system “open sourced” as well? Does this guy know the difference between open source and “open source”?

> This is commonly refered to as a market working as intended.

Is it? Seems like market is unable to separate wheat from the chaff and is just throwing money around hoping to hit the jackpot. While AI has massive chance of affecting our lives, the investment market paints a pretty similar picture to what happened during the crypto boom.

Perhaps a cure for venture capitalist FOMO for not having invested in AI?

You've never seen a high school or a college newspaper?

This reminds me of a quip I heard on twitter and found quite funny.

“Elon is the stupid people’s fantasy of what being smart is like, just like Trump can be a poor person’s fantasy of what being rich is like.”

Don’t agree entirely in the poor part, but the Elon part I find quite accurate.

A great way to be transparent would be to admit that some enormous egos prevented work that should be open from being open, and counterfactually opening them up. Sure, it may piss off microsoft, but statistically things that piss off microsoft also has been great for the world at large.

But that will never happen, will it?

Once again, it’s a literary device; you’re doing the same thing when you call it “doomsaying”. The goal of it is to evoke a certain emotion out of the reader, which is fine in a discussion. Otherwise any conversation would be dry as a doctor’s report for gastric issues.

So many of us know about Goodhart’s law but willfully ignorant about how that applies to modern economies. We made money to be the metric we judge things by, and now we get to enjoy or suffer the results of overoptimization on that metric.

Hyperbole is often (successfully) used as a literary device to express sentiments. Is there a sentiment expressed in the parent comment you don’t agree with? Then let’s talk about that instead of the form of the argument.

When I think about falling trust in so called “mainstream” media, pieces like these come to mind. Used to be that newspapers were the fourth estate, existing independently and keeping powers in check by being a source of transparency. Now in the age of web driven advertisement that business model doesn’t work anymore, and what we get is the shells of the old behemoths used as corporate mouthpieces and the crumbling democracy and freedom that entails. What an absolute shame.

Since log(xy) = log(x) + log(y), you can simply calculate sum(log2(i) for i in range(1, 53)) :) might be a nicer formulation for when you don’t have arbitrary precision support.

Even if you’re talking to your next door landlord and coming up with a joint price, it’s collusion.

Thanks for braving the crowd here, you will unfortunately only find hard questions.

Anyway, about my second question: why are the videos only half second ish long? Does the model unravel after that?

Also

> This is the first version of something that is now possible and will only improve with scale.

11b params is already pretty large considering the stable diffusion and LLM scale. How much higher do we need to scale until we get something useful beyond simple setups?

Who knows, maybe it will be web4 or web5 even, this time around.

What is the video resolution, 64x64? And even then it becomes blurry. Seems like another Google flag-plant-y paper filled with hot air that we will never see the source code or model for because it will expose how poor its capabilities are relative to competitors.

The internal politics at these places must be exhausting. Industry research was supposed to be free from the publish or perish mindset, but it seems like it just got replaced by a different kind of need for posturing.

Firstly, do these models learn a good physics grounding for nonsense actions? Like keep pressing down even when you are in the ground? Or will they phase you through the ground?

Secondly, why are all videos like half a second long? I thought video generation came much farther than this. My guess would be that the world models unravel at any length longer than that, which is (and has always been) the problem with models such as these. Minus the video generation part, we had pretty good world models for games already, see Dreamer line of work: https://danijar.com/project/dreamerv3/

Are elderly or disabled people "normal" in your book? Do you see yourself or your loved ones growing old someday?