HNHacker News
TopNewBestAskShowJobs

loveparade

2,076 karma · joined December 30, 2022

Meet Me At The Love Parade
submissionscomments
loveparade··on The rise and rise of e-sports
I disagree not with your overall point but with the extent to which game knowledge is necessary. I have never played baseball, I barely even know the rules, and watching a baseball match is incredibly boring to me. I can't appreciate it at all. It's literally just people randomly moving around to me. There is no difference to videogames here.

It certainly depends on the game, but you don't need to be an expert to appreciate some of the plays. A few days or weeks of gaming are usually enough to understand what's going on in professional matches and start enjoying them. That's the same as for traditional sports.

loveparade··on The rise and rise of e-sports
How many people watching professional football do you believe regularly get off the couch to get some real exercise? I don't see the difference.
loveparade··on OpenAI’s policies hinder reproducible research on language models
Given how small the time window between the successive releases was it's extremely unlikely that there were any big changes to the model. Most likely it's just better preprocessed training data, more training data, trained for longer, performance optimizations for attention, or a few changes to layer sizes.
loveparade··on OpenAI’s policies hinder reproducible research on language models
A headstart doesn't matter unless you can keep it. The point is that there are many mort smart people and resources outside of OpenAI and there are inside of OpenAI. If they focus their efforts, they will easily catch up.

A headstart is not a competitive moat like network effects are. Go and try to raise money for your startup from a VC and tell them "well, everyone is doing the same as us, but we started 6 months earlier!!" - nobody cares.

loveparade··on OpenAI’s policies hinder reproducible research on language models
Leaving the legal aspects of crawling aside, I think there is an important distinction here between 1. "can you procure it" and 2. "do you have enough money to process it all"

1. Yes, I think almost anyone can write code to procure the training corpus, in theory, and test it on a small scale

2. No, only the biggest labs and universities have enough resources to process such huge amounts of data and iterate on models with that scale. But that's just a matter of resources that can be overcome with partnerships between industry and academia that are common anyway. All the big labs already have huge efforts underway to reproduce GPT-X and it's just a matter of time before they catch up.

loveparade··on OpenAI’s policies hinder reproducible research on language models
What's most surprising to me is that OpenAI really seems to believe that not publishing details will save them from competition. Everyone knows how these models work, and while I'm sure there is a bunch of "secret sauce" that OpenAI has built for training and fine-tuning, it's ridiculous to believe that the research community and competitors like Google and Facebook can't figure out the same. They just haven't really tried until recently because the capabilities and ROI of these models weren't obvious. No matter who you are, most of the smartest people work for someone else.

The only competitive advantage that OpenAI has here is a headstart of 6-12 months from all the infrastructure investment into training these kinds of models. Now that everyone wants to build competing models with the same capabilities, this advantage is going to disappear very quickly.

loveparade··on MRSK vs. Fly.io
I'll never understand these arguments against Kubernetes and its complexity that are so prevalent on HN. Yup, k8s has a learning curve, just like any new technology. You'll need to spend a couple days understanding it. But once you've grasped the abstractions it's actually quite easy to setup, operate, and manage, even for small side projects. I'd pick it any day over "third party magic" that is a black box and I have no control over.

It feels like many developers these days are so spoiled by magic services that they are unwilling to even spend a few days going deep into something. Everything has to work in minutes. Next, what inevitably happens is that services shut down, pricing changes, or something stops working, and they have no way to debug it or move off it. And then we get customer support complaints and posts on HN about it. These developers look for the next shiny 3rd party service that solves the problem immediately and repeat the cycle, without ever learning anything that helps in the long term.

loveparade··on Coinbase suspending USDC:USD conversions over the weekend
No, in many countries you cannot buy USD directly. You also cannot store USD in a local bank account. And in various places it's explicitly forbidden to hold USD as savings because it would devalue local currency if everyone started transacting in USD instead.

So, the "C" makes USD exposure available to anyone worldwide, assuming you have e.g. an Ethereum wallet.

loveparade··on Coinbase suspending USDC:USD conversions over the weekend
Or maybe you just don't understand it? The whole point of USDC is to have exposure to USD and the US banking system, which is supposed to be stable compared to other banking systems around the world. Could that maybe be why it's called USDC? Hmm.

If the US banking system starts failing, it absolutely should affect USD stable coins backed by exactly this banking system.

loveparade··on Five years of indie hacking
> With the launch of my second business last July (a programmatic SEO tool called PageFactory

Great read, but it's a bit disappointing to see that the author was only able to make money with yet another SEO spam tool to generate low-quality content. The idea of the original Fantasy Congress was much more interesting, too bad that didn't work out.

loveparade··on Pandas 2.0 and the Arrow revolution
There is also Polars [0], which is backed by arrow and a great alternative to pandas.

[0] https://www.pola.rs/

loveparade··on Language Is Not All You Need: Aligning Perception with Language Models
I think the models would be too biased if the training data doesn't have legs.
loveparade··on Ask HN: Is this Generative AI hype healthy?
> Nowadays, whenever I see new content online, the first thing that I think about is whether it was created by an AI or a Human

That's just going to be the new normal. It means there will be more services that rely on reputation and trust, e.g. verified accounts and credentials. This isn't great news for HN. Unless there are some big changes, I think HN is going to drown in generated content that's indistinguishable from low to medium quality human-written content.

loveparade··on Learn Vim (2021)
Helix itself looks amazing, but the keybindings make me stay away from it. Even if they are better than the original, I don't want to learn another set of "non-standard" keybindings that I need to switch between only to use an editor.
loveparade··on Physics and Mathematics Self-Study Project
Hats off, I think it takes some real dedication to stick with it.

> One of the things I found useful here was to get immediate practice once I knew just enough. I think this is one of the reasons project-based learning can be so powerful: you have a problem and research your way into it versus researching your way into it and then seeing what to apply.

This. Perhaps it varies from person to person, but this is also how I learn best. I want to start with a problem, struggle to solve it, and then learn how to actually solve it. That way the material immediately clicks and sticks with me longer. Why don't schools teach this way?

loveparade··on Fastbook: Listen to audiobooks faster (2020)
I love listening to audiobooks at 1.5x-2x speed while I'm walking outside. I can get through 100 books a year this way. Of course I don't take notes or think deeply about the content. Who needs that? In fact, I have no idea which books I've read this year! Or last year! All I want is cross stuff off my vanity to-do list consisting of books collected from HN and get a feeling of accomplishment. And I can tell others about it! 100 books a year! /s
loveparade··on A Spiritual Successor to Emacs
- Emacs buffer, window and project management. I wasted so much time in vscode navigating across projects and windows

- That I don't need to switch out of emacs for git (magit is great), complex file operations (dired), and other things like repl/shell/complex rgreps/etc

- Vim keybindings. I know this isn't related to emacs, but with Doom emacs I forced myself for the first time to rely much more on keybindings. I could probably do something similar with vscode, but it's not built for this and doesn't encourage it. It's painful in the beginning, but over time you're much faster with these keybindings.

- In general, I feel like I can customize it. I feel like I have control. For example, I had a painful time managing nested projects using multiple languages in vscode. The LSP extension's "defaults that work most of the time" get in the way when you're doing something that's a bit esoteric. With emacs I can easily customize it. This is just one example, but the point applies in general.

loveparade··on A Spiritual Successor to Emacs
> You can start climbing the levels of Emacs proficiency, a ladder that has no end

I switched from VSCode to (Doom) emacs and I think this point is important. With VSCode, my productivity had been constant over time. From the first time I opened the editor, it was intuitive, and my workflow barely changed over the years. It has amazing defaults that just work. Most people don't ever think about making significant changes to these defaults. Extensions do lots of magic and I just cross my fingers that they work as advertised.

With Emacs, I was less productive the first 2-3 months after switching. Then I got to about the same level of productivity I had with vscode. A couple more months later, I think I am now more productive than I was with vscode.

loveparade··on Show HN: Sora, Personal Publishing Platform
You want me to sign up and own my content/audience? No, thanks. These times are long over and there are hundreds of similar services. For publishing online, I would only ever consider self-hosted options where I truly own both my content and audience. I'm tired of these platforms.
loveparade··on Ask HN: What would it take for an AI to convince us it is conscious?
Nobody knows what conscious even means. All definitions of consciousness involve some fuzziness and "subjective experience" - it's a meaningless question. If you could define consciousness in terms of test, like a Turing test, it would be easy to train a model to pass that specific test, but that model would still fail at all kinds of other tests and the needle of what constitutes consciousness would move, just like it has been with language understanding and reasoning.
loveparade··on Jupyterlab Desktop
Jupyter is the main reason I come back to writing Python. I really wish there was a Jupyter environment for any language. I'm aware there exist kernels for other languages, but many of them are unstable, slow, outdated, or miss important features. I have tried multiple, and every time I come back to Python, not because I want to write Python, but because Jupyter works so well.
loveparade··on Debunking the Microsoft/OpenAI/Google hype and over-reaction
Unfortunately that's how tech and media works. OpenAI itself was lucky as well. Before the LLM success, they were just another research lab that did the same stuff as everyone else and were mostly focused on RL. A few researchers randomly stumbled upon [0] the fact huge LLMs work unexpectedly well, but this might as well have happened at any other lab like Google or Meta because everyone was doing the same kind of research anyway. Anyone who is acting as if any of this was planned is just spinning a narrative.

[0] https://news.ycombinator.com/item?id=34754355

loveparade··on I gained a new appreciation for Test Driven Development
TDD can be great if you know exactly what you're building and what you must test for, but it gets in the way when you're rapidly prototyping and exploring.

Just don't be dogmatic about practices like TDD. It's always a tradeoff. Being a good engineer means understanding when the tradeoff is worth it and when it isn't. Be wary of development teams, or people, that force you to practice certain methodologies regardless of context.

loveparade··on Understanding Large Language Models – A Transformative Reading List
I worked in one of the big labs when the first large models came out and I can pretty confidently say that nobody in the field predicted this. Sure, there were always people who said "let's make models bigger because why not, we have the infra and it'll be a good paper" but nobody expected them to become this good just by being bigger and using more data. The consensus was that they'd hit a ceiling of what they can do much sooner.
loveparade··on Understanding and coding the self-attention mechanism of large language models
There's a bit more to it, but you can partly view (self-)attention as a trick that makes optimization easier, that is, it improves gradient flow similar to skip connections to make learning easier. That was more obvious to see when we used to use attention with RNNs, where attention can be viewed as being equivalent to "dynamic skip connections"

While fully connected layers can in theory learn anything, that's a very hard optimization problem in terms of gradient flow. Attention adds inductive biases (prior/domain knowledge) about what you want the network to learn, which makes the optimization of that specific aspect easier for the optimization algorithm.

In general, you can view almost anything in ML/DL as improving either optimization or generalization, and while it's a spectrum, attention falls more into the optimization category.

loveparade··on Ask HN: How do you deal with information and internet addiction?
The following trick has worked for me: After listening to a podcast, reading an article or watching a video, I ask myself: What did I get out of this? Have I gained any insight or learned something applicable?

95% of the time, the answer is no. I didn't get anything out of the consumed information. As this keeps happening, I'm starting to feel much more comfortable ignoring informational content.

loveparade··on Trustfall: How to Query (Almost) Everything
Isn't this the same as converting websites into a GraphQL API? Aren't there already dozens of projects and services that do the "convert websites into an API", at scale? What exactly is the innovation?
loveparade··on Bard and new AI features in Search
> So my question is, how hard of a problem is that with LLMs? I get the sense that LLMs are trained on a very large data set all at once, but that it is difficult to incrementally update them with new data. Is that a true assessment?

That's not quite correct. It's trivial to update the model with new information. What's not trivial is to encode priorities or time into the model. Fundamentally, the model doesn't understand about the concept of time, so you can't easily condition it on "give me only information from last year". You can get there and nudge it in the right direction by generating training datasets for such queries and fine-tuning the model on them, by oversampling more recent data in your training procedure, or by adjusting your gradients for more recent data, but all of these are still "fuzzy" - the model may or may not do exactly what you want during prediction time and could still give you old results.

However, I would assume that Google is doing something more sophisticated and that's it's not just a plain LLM like GPT3. It's probably a more complex architecture with external systems and data sources around it that tries to solve the above using heuristics.

That being said, the current Google search is extremely bad at time relevance too...

loveparade··on Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
You are getting downvoted because this list if from ChatGPT, but as a researcher in the field, this list is actually really good, except for perhaps the SARSA and GLUE papers, which are less generally relevant. I would add WaveNet, the Seq2Seq paper, GANs, some optimizer papers (e.g. Adam), diffusion models, and some of the newer Transformer variants.

I'm very confident that this is pretty much what any researcher, including Ilya, would recommend. It really isn't hard to find those resources, they are simply the most cited papers. Of course you can go deeper into any of the subfields if you desire.

loveparade··on John Carmack’s ‘Different Path’ to Artificial General Intelligence
Perhaps not directly, but from the very start OpenAI was considered "Elon's AI company" and people wanted to work there because they wanted to work with Elon. That had a huge effect on what kind of talent OpenAI was able to hire.

Yes, I'm aware that Ilya had much more day-to-day impact, but if you're part of the ML research community and have worked with Ilya, or heard rumors of working with Ilya, that may have had more of a negative effect...

← PreviousPage 11 of 13Next →