HNHacker News
TopNewBestAskShowJobs

heyitsguay

1,081 karma · joined January 5, 2017

submissionscomments
heyitsguay··on Ultra-efficient machine learning transistor cuts AI energy use by 99%
You have to go from this article to the Northwestern press release to the source research paper to learn that this is for simple SVMs in wearable devices like for healthtech.

Neat, but it doesn't really have any bearing on the deep learning and generative AI trends.

heyitsguay··on Ask HN: Is making a self taught transition to AI/ML related fields possible?
I talk to a bunch of engineers with this question at an SF AI meetup, and I'm really curious to hear what the community consensus is but from my perspective, it's tough and getting tougher.

The issue isn't that it can't be done -- in fact, the greatest need right now is for engineers who can come in and build rock solid real world applications on top of commodified neural network architectures and weights, not PhD scientists. Your business might not even use its own ML model! You might just be calling an API.

The challenge is that for a few reasons, it's a very crowded market right now. A lot of people want to make a move into AI, yet for all the hype, the space of viable commercial applications that will survive without indefinite VC funding remains kinda small. Look how AVs are doing after billions and billions in funding chasing one of the most lucrative commercial possibilities imaginable. There's really cool stuff happening industry-wide, and commercial potential is growing, but nowhere near as fast as the cultural hype that has infected certain parts of tech space these past 12 months or so. Plus, many experienced ML engineers and scientists have been dumped back into the job market due to layoffs. So from the hiring side right now, for every AI posting there are tons of applications that have the cool portfolio, and then also a relevant degree and/or prior experience.

That's what you're competing against, so if you're going on portfolio alone it's got to be really outstanding. Way beyond doing the homework for a free course. Learning how to build an ML service that solves an actual problem in the real world reliably enough that you can actually use it should be the goal.

If you happen to be employed at a company where there is a need for an ML engineer in some capacity but no availability (hiring is expensive!), you can try stepping up to help out. Hiring challenges aside, it is absolutely possible to learn on the job the engineering skills needed to, say, build ML infra or work as part of an MLOps team. I recognize that's sort of just up to circumstance though. If you look for a new job, be a little wary of any that want to hire you for a more mundane task (like data entry/cleaning/labeling) with a promise of getting to do the ML engineering stuff, too, "eventually". Such roles do exist but it's also a bait-and-switch tactic.

Anyway, that's what I've got as someone who has been thinking about how to help people looking to do what you're doing, but i hope this thread turns up more ideas too.

heyitsguay··on In the Battle Between Bots and Comedians, A.I. Is Killing
Less glibly, I feel like the comment OP made is looking at first-order consequences of new generative technologies, but that it misses a deeper truth about art and creation. The creation of the camera didn't "solve" painting. By and large the art we find exceptional and noteworthy is that where an artist takes a collection of tools and invests a large amount of time in combining them in a new, rare, and/or exceedingly detailed and refined way. nobody is going to create enduring, culturally significant art off of 60 seconds of effort entering one prompt into Midjourney. There will be artists who do amazing things using, among other tools, generative AI, but it will be because they put lots of time into understanding the new creative possibilities they unlock and refining their composition into something aesthetically significant and uncommon, even given the new generative status quo.
heyitsguay··on In the Battle Between Bots and Comedians, A.I. Is Killing
What happens when anyone can create photorealistic images of their surroundings without spending years getting trained in portraiture? Painting evolved, plus we got a whole new medium too.
heyitsguay··on Why host your own LLM?
It might be tough to make more efficient, but $15k seems exactly about the price of "stick 6 4090s in a decent box and throw in a couple grand for my troubles", versus any revolutionary hardware configuration. The way it advertises running fp16 Llama 70B feels a bit contrived too, given the prevelance of quantizing to 8 bit at minimum.
heyitsguay··on Fooocus: OSS for image generation by ControlNet author
It was easy for me to set up on Ubuntu with CUDA 11.7. I cloned the repo, created a new python venv, and installed the requirements in requirements_versions.txt, but looking at launch.py, that wasn't even necessary. Just create a new venv for the project, activate it, and run `python launch.py`.
heyitsguay··on Artificial General Intelligence – A gentle introduction
Wager: None of the theory on this page will have any bearing on the implementation of a real world AI system that solves any problem of interest.

Not that there aren't problems to solve regarding AI, just that this line of inquiry won't be relevant to solving them. It'll be complicated boring work dealing with power structures, economics, and social movements, not thought experiments about omnipotent Others.

heyitsguay··on Do Machine Learning Models Memorize or Generalize?
https://bernstein-network.de/wp-content/uploads/2021/03/Lect... this has an awesome overview of the current understanding of neural encoding mechanisms.
heyitsguay··on Artificial General Intelligence – A gentle introduction
This is one of those conversations that happens so much here it's too exhausting to repeat in its entirety, but -- if you're looking to learn about the real questions and challenges faced by real modern AI in real world systems, this is not a good resource. It will leave you with the wrong impression of the issues faced in the field.
heyitsguay··on An early look at HNSW performance with pgvector
This is pretty cool, but what are the scaling limits of a pgvector-based embeddings storage solution? 1 TB of embeddings? 100 TB? Is pgvector suitable for large scale installations?
heyitsguay··on We’re closer to ‘engineering’ blood vessels
Seeking cofounder for a disruptive, high-margin healthcare startup providing new treatments with deep historical roots.
heyitsguay··on Experts starting to doubt AI ‘hallucinations’ will go away: ‘This isn’t fixable’
Actually you're right, that was my mistake. Current-gen autoregressive token predictor "hallucinations" probably can't be fixed, but yeah, the way forward will involve new architectures and I would place my bets that something like world models will be involved to ground generative output.
heyitsguay··on Experts starting to doubt AI ‘hallucinations’ will go away: ‘This isn’t fixable’
A meta point as someone who has been working in mathematical signal processing and ML professionally for a while -- at the time of writing, every single comment in this thread displayed a serious (yet unique!) misconception about AI, either as it works today or in general. I understand why it's gotten so buzzy in popular discourse, but it's sadly all but ruined discussion on social media (including HN).

Are there any good, maybe more gatekept forums for discussing AI news? It is great to have a community to discuss with, the field is so large and moving so fast, and there's so much to learn from other practitioners, but the SNR here (where i often find news) has just gotten so low.

heyitsguay··on Unconfirmed video showing potential LK-99 sample exhibiting the Meissner effect
It is, the same author posted a rotated version in the replies: https://nitter.net/667keos/status/1686490898480173056#m
heyitsguay··on First-principles study on the electronic structure of Pb10−xCux(PO4)6O (x=0, 1)
I think this conversation demonstrates a misunderstanding of what GPT4 can and cannot do. You can get useful background information about DFT and molecular simulation, but definitely no first-principles reasoning about molecular properties.
heyitsguay··on Sam Altman’s Worldcoin Crypto Project Officially Launches
Love seeing the posters for this, plastered across the decaying facades of abandoned buildings on Mission St, amidst the ragged people and "street vendors". It seems like a fitting place to advertise the project.
heyitsguay··on Running Open Source LLMs in Jupyter Python Notebook as an Alternative to ChatGPT
This is blogspam, but in case people come here because the title sounds interesting, check out text-generation-webui (https://github.com/oobabooga/text-generation-webui). It's a better solution to local LLMs than running in a Jupyter Notebook.
heyitsguay··on Minigpt4 Inference on CPU
I know it's not the main point of this, but... so many multimodal models now that take frozen vision encoders and language decoders and weld them together with a projection layer! I wanna grab the EVA02-CLIP-E image encoder and the Llama-2 33B model and do the same, I bet that'd be fun :D
heyitsguay··on Exploring Linear A
Is this site not just a handy visual catalog of known artifacts and transcriptions? Is there some speculative decipherment implied in the phoneticizations?
heyitsguay··on OpenOrca: open source dataset and instruct-tuned LLMs
Absolute best case in the cloud for the kind of GPUs this needs? ~$1/GPU/hr, but maybe up to $5/GPU/hr depending on provider and configuration. But companies or other organizations with extra capacity on their in-house hardware might also be able to just run their training script for a while, at which point the cost is more like electricity + opportunity cost.
heyitsguay··on Declassified U.S. report on origins of Covid-19 virus [pdf]
tl;dr probably from a sick animal, maybe from a lab, no smoking gun either way. Probably not engineered or lab-adapted.
heyitsguay··on Meta says its new speech-generating AI model is too dangerous for public release
Compared to OpenAI's antics I think this is actually a very reasonable move, and a good example of the less glamorous but more concrete reality of actual AI risk management. There's a concrete harm -- disruptive ease of impersonation over common voice communication channels. Tools for this will diffuse out anyway, because at the heart of the problem is a bad cryptography-style security assumption. We assumed voices were expensive to clone in real time with high fidelity, turns out maybe they aren't. In time, societal expectations will shift, like they did with photographs, but it sounds like Meta is making the responsible choice to not accelerate the cats ahead of the mice.
heyitsguay··on My AI Wife Just Got Even Better [video]
Oh no did i get woosh'd? I guess I'm feeling a little trigger happy after seeing all these "AI dating app" posters come up in the Mission. I don't think they're parodies...

(Though now I'm questioning that too :D)

heyitsguay··on My AI Wife Just Got Even Better [video]
I'm saying contemporary chat bots, even with GPT-4, are incredibly crude simulacra of human intelligence and agency, so to see someone become emotionally invested in one is pitiful on the level of a scared monkey clinging to a piece of cloth it's confused with a mother.

If the author doesn't like real humans disagreeing with them, they're free to make more chatbots to tell them what a great idea this is.

heyitsguay··on My AI Wife Just Got Even Better [video]
Depressing as hell. Reminds me of those photos of the monkey clinging to its cloth mother surrogate.
heyitsguay··on Attention is Turing complete (2021) [pdf]
I'm not going to engage with this directly, but for any other readers passing through - this is nonsense. One more drop in the flood of uninformed AI noise that's been drowning out the signal.

Choose the sources you trust very carefully, and look to the people actually working on real-world AI systems, not the storytellers and hangers-on.

heyitsguay··on Attention is Turing complete (2021) [pdf]
It won't, it just means this is another pointless theoretical study seeking to interpret Transformers in a framework that has no explanatory value for AI.
heyitsguay··on LLM.int8(): 8-Bit Matrix Multiplication for Transformers at Scale (2022)
Sounds like the idea is that "outlier features" effectively give the model an inhibitory system for other features, and transformer models converge to consistent outlier channel selection per layer at around the 6.7B param mark?

It's interesting, biological neural networks are known to use inhibitory systems for signal gating and sharpening/sparsification, but it's typically handled by distinct neuron types that are strictly inhibitory.

I wonder if one could boost smaller transformer performance with an analogous construction - add on a small additional module constructed for channel inhibition in the rest of the model.

heyitsguay··on MeZO: Fine-Tuning Language Models with Just Forward Passes
It's not. That's part of the uninformed AI hype train that will consistently be posting first on AI stuff for the next 6 months.

Right now the main bottleneck for LLM size is GPU memory (VRAM). Training requires much more VRAM than inference, which limits the ability for entities that aren't Google or OpenAI-scale to finetune models (aka do a little more training on your custom dataset).

The paper here suggests that one can actually finetune LLMs with inference-sized VRAM usage instead of training-sized VRAM usage. If true, it will be possible to fine tune larger models on smaller (though still expensive) GPUs -- like a single 3090 instead of 1x or 8xA100s. So, more people can create more customized models.

heyitsguay··on AI: Nvidia Is Taking All the Money
Yeah but the challenge lately has been that efficiency gains have also made the capabilities of large compute clusters even more powerful. Transformers were introduced as an efficient alternative to RNNs and conv nets, then it turned out that however much they supercharged a single GPU, they were dramatically more powerful deployed at datacenter scale.
← PreviousPage 4 of 15Next →