HNHacker News
TopNewBestAskShowJobs

radarsat1

4,157 karma · joined May 28, 2011

Info: https://sinclairs.gitlab.io

Email: <myusername>@gmail.com

submissionscomments
radarsat1··on Planes in 3D Space
Agreed.
radarsat1··on Planes in 3D Space
Even though it's well intended, I feel like pasting in responses from ChatGPT should be against HN guidelines, if it's not yet. Reason is, if I want that type of explanation, I can go and get it myself. I come here for people's personal comments, not to read them regurgitate what they had an LLM generate. And I use ChatGPT all the time so don't get me wrong, I just think it doesn't add a lot to the discussion here and takes up a lot of room.
radarsat1··on Gitlab Duo
If my code is included in a training set which then helps someone else solve a problem, then I am helping other people.

Why should I care if some 3rd party is the middleman that builds the system that performs the transformation? I put out my code so that people can use it. The existence of said 3rd party doesn't restrict that in the slightest, in fact it enhances the reusability of my work which seems like a win-win.

radarsat1··on Gitlab Duo
But it's not forking or linking to any of that software. It's "using it" only in the sense of reading the code as examples of how to do things after mixing it with a million other sources. Sorry but I just don't see the problem with that. Very different, categorically, in my mind from what is intended by software licenses when they talk about derivative works.
radarsat1··on Gitlab Duo
They take publicly accessible code and learn to predict good token strings from it to solve related or unrelated problems. I am not convinced this is not morally right.

What they are not doing, as far as I can tell, at least not intentionally, is just copying code and removing the license and attribution. Even if you don't see the AI output as legitimately "creative", it's certainly transforming and remixing existing solutions in ways that are transformative enough not to be just copies of anything except the most bog standard boilerplate, which is usually not a copyright issue.

People seem upset because there is money to be made, if there was no money involved I don't think anyone would see any issue here.

It's not about the greater good, and I don't mean to be an apologist, but I just legitimately don't see a problem with doing it, or how it's not fair use. It really does seem like fair use to me.

radarsat1··on Self-reasoning tokens: teaching models to think ahead
Seems very similar to the Think Before You Speak paper.

https://arxiv.org/abs/2310.02226

radarsat1··on Medium bans AI-generated content from its paid Partner Program
> Nobody is getting banned for the "Gray-AI" of assisted writing, where you get help with an outline

Maybe not but it's certainly worth worrying that this is what will happen just because the line has to be drawn somewhere and that's an easier way to draw it. Baby, bathwater, etc.

radarsat1··on ULIDs and Primary Keys (2022)
I've found ULID to be really useful in my API backend that runs on AWS Lambda and writes to DynamoDB. This is a use case where I need Lambda's parallelism, meaning no sharing some common sequential integer. Meanwhile I can roughly order items by creation time which makes pagination and selecting items by time range possible, right on the sort key instead of as a filter. In other words items might end up out of order by a few milliseconds but in my use case that's not a big deal, and I get the benefit of communication-free parallelism.
radarsat1··on Kobo announces color e-readers
I want something like this but just to hang on the wall to display graphs. Why are e-ink displays only ever focused on ebooks, I think there are lots of cool non-ebook applications for this technology but to use it you either have to build something with a soldering iron and a raspberry pi or hack an ebook's OS. I don't understand why there aren't more special purpose e-ink devices for other uses like monitoring, price displays & restaurant menus, etc.
radarsat1··on She slept with a violin on her pillow
There's actually lots of research on that topic, if you search for "Stradivarius blind listening" on Google Scholar you'll find some. Can't recommend a specific article unfortunately, but it's an interesting topic. My current understanding is that there is no perceptual difference that has been proven, and even later models are blindly judged to be better.
radarsat1··on Kobold letters: HTML emails are a risk
Sure, and how many people use it? Available != adopted.
radarsat1··on Kobold letters: HTML emails are a risk
The other day I was discussing the design for an "update" email that our designer was putting together and showing me the size of this stupid graphical header he put at the top and I was showing him how with that you can't even see the text of the title of the email without scrolling down, and then he forwards some other version to me for more feedback and suddenly started getting all disturbed and upset because something looked different to him.. like the fonts were a bit smaller.. and then he said, oh no, when you forward it, it transforms it to the desktop version, instead of the mobile version! And I'm sitting there in disbelief, just staring at him, like,... this is EMAIL we're talking about, what do you mean "desktop" version and "mobile" version? How is that even a thing? What do you mean it "transforms" it? It's literally just copying it! The fact that it "breaks" when you do that is evidence of how stupid this is. My god man, why is CSS in my email at all?

The fact that we haven't adopted something much simpler to just be able to express italics or whatever, like markdown, is just bonkers to me. It just shows how little anyone cares about actually improving the situation. And all just to cater to the bizarre corporate need to put logos and banners everything. HTML email is just ridiculous.

radarsat1··on LLaMA now goes faster on CPUs
Depending on how many individual tweaks are necessary for hardware variants of course... but at this level of code & complexity it actually seems pretty reasonable to write 3 or 4 versions of things for different vendors. More work yes, but not pointless.
radarsat1··on Black Holes Are Even Weirder Than You Imagined
But there are black holes in our universe. Can black holes be inside black holes?
radarsat1··on Emad Mostaque resigned as CEO of Stability AI
(Can't edit anymore, but I meant "automatic transcription" above..)
radarsat1··on Implementation of mixture of experts language model in a single file of PyTorch
Ah ok, I was wondering if there was some theory here that I wasn't aware of but if it's just experimentation no problem ;) good to know in any case!

I find it a bit difficult to find resources describing the properties of various options for this topic of discrete choices and clustering, apart from a few papers & blogs describing the idea.

radarsat1··on Emad Mostaque resigned as CEO of Stability AI
We use Whisper for automatic translation, supposedly SotA, but we have to fix its output, I would say, very often. It repeats things, translates things for no reason, has trouble with numbers.. it's improved in leaps and bounds but I'd say that speech recognition doesn't seem to be there yet.
radarsat1··on Implementation of mixture of experts language model in a single file of PyTorch

    Adding scaled unit gaussian noise to the logits
        noise = torch.randn_like(logits)*F.softplus(noise_logits)
        noisy_logits = logits + noise
Question, if you changed this Gaussian normal for Gumbel noise you would get something like Gumbel softmax, right? I'm curious why not use it? Isn't it a usual way to implement differentiable discrete selection? My curiosity is about the effectiveness of Gumbel softmax since I have had some trouble using it in practice so I'm curious why it's not used here and if there are downsides to it compared to other methods. Honestly just adding normal noise like this seems simpler anyway.
radarsat1··on 8 years later: A world Go champion's reflections on AlphaGo
I think AlphaZero does use MCTS?
radarsat1··on Show HN: Not sure you're talking to a human? Create a human check
Why does it ask for a name?
radarsat1··on Google Scholar search: "certainly, here is" -chatgpt -llm
Ironically? Are you saying that using an LLM to assist writing is plagiarism?
radarsat1··on Ask HN: Why can't image generation models spell?
> Text models receive tokens(akin to morpheme chunks) and don't have access to the underlying letters

Heh, this gives me the idea of training an image generator not on tokens but on rendered text of the prompt in some 8x8 font, that would be a fun experiment ;)

radarsat1··on Improving cursor rendering on Wayland
Doesn't the CPU/GPU bottleneck which is already assumed to be slow actually provide the perfect opportunity for abstraction over a network protocol? Sending "what to draw" and "how" (shaders) over the wire infrequently and issuing cheap draw commands on demand? I think GPUs provide a better situation for a network first model than was available when X was designed.
radarsat1··on Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
This is true but you have to also consider the autoregressive component. In your example, it's 50 steps per iteration of the model, where the model is executed once for each token in the output.

So practically speaking it's a bit more complicated to calculate how much the model can "think". Of course once a token is output it is committed to that (in the most basic scenario), but that doesn't mean it is not still "thinking" as it produces subsequent tokens.

> perhaps a notepad

Exactly, the context and previously output tokens can be considered such a notepad since they are input for the next steps of the model.

radarsat1··on Large Language Models Are Neurosymbolic Reasoners
Yes, this is one view of machine learning, the idea that you are training some function to map input to output, similar to "looking up" what output is addressed by some input.

And that's why the concept of generalization is so important on machine learning, and as a consequence, why the internal representation of that "lookup" matters.

By definition a lookup table can only store data it is given. However, the idea of ML systems is actually to predict values of inputs that are similar to but not given in their training data.

Interpolation and extrapolation, key components to applying ML systems to new data and therefore critical for actual usage, are enabled by internal representations that allow for modeling the space between and around data points. It so happens that multilayer neural networks accomplish this by general and smoothed (due to regularization tricks and inductive biases) iterative warpings of the representation (embedding) space.

Due to the manifold hypothesis, we can interpret this as determining underlying and semantically meaningful subspaces, and unfolding them to perform generalized operations such as logical manipulations and drawing classification boundaries in some relatively smooth semantic space, then refolding things to drive some output representation (pixels, classes, etc.)

Another view on this is that these manipulations allow a kind of compression by optimizing the representation to make manipulations easier, in other words they re-express the data in a form that allows algorithmic evaluation of some input program. This gives the chance of modeling intrinsic relationships such as infinite sequences as vector programs. (Here I mean things like mathematical recursions, etc.) When this is accomplished, and it happens due to the pressure to optimally compress data, you could say that "understanding" emerges, and the result is a program that extrapolates to unseen values of such sequences. At this point you could say that while the input-output relationship is like a lookup table, functionally it is not the same thing because the need to compress these input-output relationships has led to some representation which allows for extrapolation, aka "intelligence" by some definitions.

The fact that these systems are still very dumb sometimes is simply due to not developing these representations as well as we would like them to, for a variety of reasons. But theoretically this is the idea behind why emergence might occur in an NN but not in a lookup table.

radarsat1··on Opus 1.5 released: Opus gets a machine learning upgrade
This is an important question. However, I'd like to point out that similar biases can easily exist for non-ML, hand-tuned algorithms. Even in the latter case test sets and often even "training" and "validation" sets are used for finding good parameters. Any of these can be a source of bias, as can the ears of evaluators making these decisions.

It's true that bias questions often come up in ML context because fundamentally these algorithms do not work without data, but _all_ algorithms are designed by people, and _many_ can involve data in setting their parameters. Both of which can be sources of bias. ML is more known for it, I believe, because the _inductive_ biases are less than in traditional algorithms, and therefore are more keen to adopt biases present in the dataset.

radarsat1··on Vibrating vests translate music for deaf concertgoers (2023)
Reminds me of Cutaneous Grooves (2003)

https://www.tandfonline.com/doi/abs/10.1076/jnmr.32.4.369.18...

Although that idea was more about using haptics as a medium to compose explicitly for rather than translate existing music to.

radarsat1··on ChatGPT and Google Gemini are both doomed
When debating the benefits of fine tuning people always focus on "culture" but what about spelling mistakes?

I think we too easily forget about all the other reasons for fine tuning. Without it I assume using ChatGPT would be more like chatting with an average reddit comment or Twitter post, full of typos and emojis and random gif links.

I'd much rather chat with something that gives me reasonably "cleaned up" answers even if it takes a little work to get it to do what you want.

It's obviously rather hard to hit that fine-tuning sweet spot though, no doubt.

radarsat1··on Let me clear a huge misunderstanding
The main thing I'm confused about by these comments is that as far as I understand, the Sora model (and many others like it) performs the diffusion process in latent space, and then translates this to pixels.

So it's strange to me to claim that it doesn't have an abstract representation.

But, maybe the latent space of a diffusion-VQVAE pipeline is fundamentally different from that of JEPA, I haven't read the relevant papers for that. Curious if someone could explain if they are different ideas of representation.

radarsat1··on Orgzly Revived: a community-maintained version of Orgzly
Same here, I use it for todo lists on my phone and didn't know it wasn't being maintained. Although, maybe that's because i haven't encountered any issues. But my use of it is really minimal, as I prefer typing on a keyboard.
← PreviousPage 8 of 34Next →