HNHacker News
TopNewBestAskShowJobs

angusturner

434 karma · joined September 9, 2020

ML Engineer

Interested in generative models, programming, music and philosophy of mind.

https://github.com/angusturner/

submissionscomments
angusturner··on We should measure well-being instead of failing to measure economic productivity
Only tangentially related, but there is some cool work on using reinforcement learning to learn an optimal tax rate. See: “AI Economist”.
angusturner··on Stable Diffusion is a big deal
That just feels like such a pessimistic forecast to me. Of course, the current trajectory of improvements in model efficiency and better commercial GPUs / ML-accelerators may hit a wall.

But I would not be surprised if this was trainable on a commercial GPU at home within that time. But I think another important trend that we are seeing is that you don't need to train these models from scratch.

Open-source "foundation models" means that you can usually get away with the much easier task of fine-tuning, as to not throw away / re-learn everything that these large models have already fit.

Edit: I initially said 2-5 years, but on more reflection this does seem optimistic (for training from scratch).

angusturner··on Stable Diffusion is a big deal
A few months ago this task was virtually impossible. Then it was possible, but extremely expensive and pay-walled behind the "Open"AI' website.

As of about a week ago this tech runs on consumer GPUs. The weights have been downloaded 100s of thousands of times, and fine-tuning / modifying is possible.

Training from scratch is about $500k still, but it will only get cheaper and easier.

angusturner··on Stable Diffusion Is the Most Important AI Art Model Ever
It seems pretty well-agreed that the HN moderation works because of dedicated human moderators and community guidelines etc.

I think spaces that effectively moderate AI art content will be successful (or not) based on these same factors.

It won't depend on some brittle technology for predicting if something is harmful or NSFW. (Which, incidentally, people will use to optimize/find NSFW content specifically, as they already do with Stable Diffusion).

angusturner··on Stable Diffusion Is the Most Important AI Art Model Ever
Developing models that can predict if stuff is harmful ironically makes it easier for people to optimize for harm.

e.g. the one line of code in Stable Diffusion that predicts if stuff is NSFW, can be inverted to generate only NSFW stuff.

I tend to agree with OP that there is no technical solution to this problem.

angusturner··on Ask HN: DALL-E was trained on watermarked stock images?
Relevant earlier discussion about this issue: https://news.ycombinator.com/item?id=32436203
angusturner··on Copyright infringement in AI art
My bad, I misunderstood you! I agree with you.
angusturner··on Copyright infringement in AI art
Where I think this analogy breaks down is that these models are not simply copying / memorizing the training data. They are using the data to fit a model, highly compressing the specifics while holding on to general patterns and abstractions.

I really feel like "reading lots of books" is a better analogy. (Of course its not perfect, since these systems can scale way past the number of books a human can read).

angusturner··on Copyright infringement in AI art
These are fair points. I suppose I am more optimistic about all the potential ways this tech could augment or enhance human creativity.

Like, isn’t it kind of wild to think that the average person could have access to models that have learned from all art ever made…

I’m also encouraged by the fact that some of the best versions of these systems are open source and run on a commercial GPU.

angusturner··on Copyright infringement in AI art
I don’t think it makes sense to talk about these systems as though they have any agency (and I think I was careful not to).

Of course there is nothing that necessitates letting these systems do what humans do.

What I am saying is that I am allowed as a human to ingest copyrighted material, learn from it, form mental abstractions and use those learnings to generate new art. Why then, would I be forbidden from externalising some of that process into an artificial system that operates on similar principles.

The trained model obviously does not have agency. It is a creative and computational tool, much like any other digital tool or product.

angusturner··on Copyright infringement in AI art
Feel like people would have said the same with images 6 months ago tbh. (I mean sure, you could get a fake photograph of a person, but the text control that clip has unlocked is just wild…)
angusturner··on Copyright infringement in AI art
The question that fascinates me about this whole debate is on what basis (if any) one draws the line between a machine learning from copyrighted data vs a human learning from copyrighted data.

And I do think “learning” is the correct metaphor here. There are good reasons to believe that the brain is doing something conceptually similar to generative statistical models. i.e compressing inputs (/percepts) into some kind of abstract representation space. Even if we don’t understand the specific details yet.

As far as I’m concerned, it makes no sense to prevent training on copyright data. If I use a trained model to substantially reproduce parts of a copyrighted work, feels like the issue is largely with me and what I do with the content.

These models are just tools after all.

angusturner··on Google has banned the training of deepfakes in Colab
Good on them. As a researcher it is so disheartening when GPUs get hoovered up by people participating in proof of waste pyramid schemes.

Likewise, if google decides it wants to preserve its service for students and researchers, then I hope that people chewing up resources for malicious/questionable use-cases are booted off.

angusturner··on The problem with Bitcoin miners
Isn't this just proof of work?
angusturner··on My lizard brain is no match for infinite scroll
This is awesome. I use the "self control" app for Mac OS X, which is pretty effective, but obviously doesn't stop me using my phone.

Would be interested to learn more about this approach.

angusturner··on Web3? I have my DAOts
A lot of crypto trade (especially on shadier exchanges without KYC or the option to use fiat) is denominated in tether.

Tether keeps printing billions of dollars with no oversight or regulation. They lied about it being backed by cash reserves, and now make a much softer claim that its backed by "some cash" and other assets (other assets include more crypto IIRC).

Basically, it feels like tether is inflating the price of BTC, and the exchanges all have a vested interested in continuing to pretend tether is worth something. If there was a run on tether (i.e. everyone tries to exchange for fiat) it could be catastrophic for all of crypto.

(Disclaimer: I am not a financial expert, and have just picked this up from reading about tether over the last few months).

angusturner··on My friends Instagram was hacked and deep-fake videos posted in less than 6 hours
This is actually quite good for Bitcoin.
angusturner··on ConstitutionDAO
This is such a good point. This is my first thought whenever I see a new "blockchain for x", where x involves some real-world product or activity.

Suddenly the promise of "trust-less" makes no sense, and we are just left with poor performance and immutability of human errors.

angusturner··on Rich Harris joins Vercel to work on Svelte full time
As someone familiar with the old react API, I agree 100% about useEffect being super arcane.

There's just too much magic going on, which is the same thing that makes me reluctant to write Vue or Svelte (after reading this)

angusturner··on Discord Is Integrating Ethereum
Wow, a lot of people are upset about this. Curious what HN thinks of the argument that Layer 2 scaling + POS will solve / mitigate the environmental problems with ETH at the moment.
angusturner··on Diffusion Models as a Kind of Hierarchical VAE
New blog post exploring diffusion models from the perspective of variational inference, along with PyTorch code.
angusturner··on Square Buys $170M More Bitcoin, Deepening Crypto Bet
I am honestly astounded that the whole tether and bitfinex controversy has not had a greater fallout.

For the first time in my life I have non-technical friends asking me how to invest in cryptocurrency, and I have to explain to them the real risk that there is a huge speculative bubble right now.

I am seeing the DeFi people reinventing all sorts of synthetics, derivatives and highly leveraged trading instruments in the absence of any regulatory oversight and can't help but wonder when there will be a reality check.

Even more worrying is the prospect that we don't get a price correction soon, lulling more retail investors and institutions to expose themselves to BTC. And when the fallout does happen it will hurt the real economy too (you know, the one that actually produces goods and services with value).

Crazy.

angusturner··on The Computational Power of Biological Dendritic Trees
I can't really comment on the novelty of this work, but I don't think the connectivity structure makes much sense.

I mean, it does in the sense that local pixels are strongly correlated and a binary tree will captures this. In fact if you add weight-sharing to the K-tree model you can recover 1D convolution with a stride and kernel of 2.

But is this really the right operation for images? Why fixed kernel of 2? I think capsules or some other vector-based operation would make more sense. Perhaps with a learned or dynamic connectivity pattern.

← PreviousPage 4 of 4