Hacker News Reader

klowreyon Model Predictive Control in the Browser with WebAssembly

Here's a (hacky) demo of MPC using MuJoCo in the browser: https://klowrey.github.io/mujoco_wasm/

I hacked it using MPPI and it only works on the cartpole model so as to not have to dwell in Javascript too long; just click the 'MPPI Controller' button and you can perturb the model and see it recover.

klowreyon Ask HN: Discuss ADHD and your use of medication

Vyvanse is dextroamphetamine, while Adderall and Adzenys are mixed dextro- and Levo-.

While there are now generic versions of Vyvanse, they’re unlikely to be as effective until other developers work out the manufacturing kinks.

However, generic extended release dextroamphetamine is generally not in short supply and 10x cheaper than Vyvanse. Might be worth a shot if that’s what worked best for you.

klowreyon Exact binary vector search for RAG in 100 lines of Julia

Combined with broadcasting its fast.

Under the hood it’s doing the same thing with a vector of ints (64 bits for bitvectors) and all the bulk manipulation is handled that way so SIMD in inherent as well. Worth a shot.

klowreyon Exact binary vector search for RAG in 100 lines of Julia

Why not use the built in BitVector type that has specialized code for things like xor?

https://docs.julialang.org/en/v1/base/arrays/#Base.BitArray

klowreyon An electric new era for Atlas

Woah we can’t ask for the impossible. We just want the robots to _make_ all the things that end up as garbage in the average American home.

klowreyon An electric new era for Atlas

https://www.youtube.com/watch?v=xNeZWP5Mx9s

different configuration, but electric motors are fine if you get momentum on your side. Humans use their entire range of motion get build up velocity to jump; this is motion control thing.

klowreyon An electric new era for Atlas

Static holds. Once you pressurize the cylinder to make it move to a certain position, it can hold that position without using more energy.

This makes sense for quasi-static systems but obviously is a limiting factor for dynamic robots.

klowreyon An electric new era for Atlas

Hydraulics shouldn't have any give, as the working fluid is considered "incompressible". Of course in the real world the tubing can expand slightly and there are friction losses, but the reason they went with hydraulics in the first place is they can set a position and not have to use more energy to hold it there (since the cylinders are pressurized).

If the gear ratio on these motors is high, then there can only be faked compliance in the tuned force-torque controllers you mentioned. MIT's little cheetah robot, on the other hand, deliberately used low-gear ratios to keep things naturally squishy if needed. This is the way to go; putting elastic tendons or spring elements seems like a good idea but then you can't actually model the non-linearity well (the 1st order motor becomes a 2nd or higher order system).

klowreyon Google DeepMind's Aloha Unleashed is pushing the boundaries of robot dexterity

We're building software for neuromorphic cameras specifically for robotics. If robots could actually understand motion in completely unconstrained situations, then both optimal control and modern ML techniques would easily see uplift in capability (i.e. things work great in simulation, but you can't get good positions and velocities accurately and at high enough rate in the real world). Robots already have fast, accurate motors, but their vision systems are like seeing the world through a strobe light.

klowreyon Google DeepMind's Aloha Unleashed is pushing the boundaries of robot dexterity

The detail about end-effector frame is pretty critical as doing this BC with joint angles would not be tractable. You can tell there was a big shift from the RL approaches trying to do very generalizing algorithms to more recent works that are heavily focused on this arms/manipulators because end-effector control enables more flashy results.

Another limiting factor is that data collection is a big problem: not only will you never be sure you've collected enough data, they're collecting data of a human trying to do this work through a janky teleoperation rig. The behavior they're trying to clone is of a human working poorly, which isn't a great source of data! Furthermore limiting the data collection to (typically) 10Hz means that the scene will always have to be quasi-static, and I'm not sure these huge models will speed up enough to actually understand velocity as a 'sufficient statistic' of the underlying dynamics.

Ultimately, it's been frustrating to see so much money dumped into the recent humanoid push using teleop / BC. It's going to hamper the folks actually pursing first-principles thinking.

klowreyon Spiking Neural Networks

Have any links you would recommend?

klowreyon Julia 1.7 Highlights

I manage the MuJoCo.jl wrapper, and having the source code wont (naively) help with AD. Internally, there's a number of iterative algorithms that you wouldn't want to automatically differentiate, and if analytical derivatives were apparent they would be in there (the MuJoCo creator was my advisor). As it stands, finite differencing is the best way to extract derivative/gradient information from the underlying physics model of MuJoCo, and is the suggested method.

We had a paper that included MuJoCo (with custom defined adjoints) within Julia's DiffEq framework to learn continuous control policies (https://arxiv.org/abs/2012.06684) which revealed a whole host of other issues, namely that gradients are not all that helpful when your optimization landscape is insanely non-linear. But that's a different problem.

Open to chatting about; Julia is definitely the tool for this kind of work.

klowreyon Open-sourcing the MuJoCo physics simulator for robotics

There was a free license for individual users, with the non-free license applicable to those who were receiving financial support (i.e. academic / industry researchers, etc.). The individual license was very popular and heavily utilized.

klowreyon Effects of cat ownership on the gut microbiota of owners

It may be frowned upon, but we communicated to our cats that meowing at the door was not acceptable by discouraging them with a vacuum cleaner. Flip the switch on the vacuum and run the cord under the door to your bed side. Upon meow, quietly plug the vacuum in to activate -- if the cat senses that you activated the device, the process doesn't work. If "the environment" doesn't want them to meow at the door, they learn quickly.

Of course give them lots of love in the morning and before bed.

klowreyon Standardizing OpenAI’s deep learning framework on PyTorch

For folks interested in Julia and RL, I've been involved in https://www.lyceum.ml/ a set of tools for continuous control problems like robotics.

It's pretty quick.

klowreyon Ask HN: What is your ML stack like?

After wrapping the Mujoco physics engine in Julia, I did a few reinforcement learning / robotics control projects, combining it with the Flux.jl library for neural nets.

While I never made my research code public, https://openreview.net/forum?id=SyxytxBFDr documents a similar setup to what I use.

klowreyon Ask HN: What is your ML stack like?

Full stack Julia for RL (and trajectory optimization)

klowreyon Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

https://arxiv.org/abs/1703.02660

Disclosure: I contributed to the linked work.

To do simple tasks, you don’t even need neural networks, in short. If you want to do complex tasks, training the neural net isn’t the problem, but a NN can indeed represent more than say a linear policy. The issue is getting the NN to do the complex thing correctly: it’s more of an exploration problem than function approximation problem.