HNHacker News
TopNewBestAskShowJobs

radarsat1

4,157 karma · joined May 28, 2011

Info: https://sinclairs.gitlab.io

Email: <myusername>@gmail.com

submissionscomments
radarsat1··on Generative AI's Act Two
> Is there room for generative AI in science?

I would love a system like ChatGPT but targeted specifically at exploring existing literature. A system that can recommend papers to read, that you can chat with about your problem and can recommend approaches that have worked for others and tell you why. That you can prompt and refine and go into detail with while it helps you figure out what do next based on previous work. That can link you to actual papers to read.

With ChatGPT, I get some of this, but I have to be veeery careful how I use it. It is generally good at discussing points at a high level and helping you sort out some ideas, but when you get into detail it is very easy to catch it making mistakes, and when you ask it for references to read further, it almost always makes up some or all of them. Forget asking it to give you actual links. Maybe some of the other GPT systems targeted at the search space are better at this, I don't know I haven't tried them. But I would love a system that actually does this kind of thing well.

Not just for scientific research -- I've used ChatGPT to help understand some legal things, to help understand some government application procedures (cut through the "consulate speak" and explain some steps to me in my son's visa application in plain language. Of course I double-checked everything it told me.)

Having a system that has "read everything" and can explain it back to you after you ask some questions is just fantastic. I just want it to be more reliable.

radarsat1··on Neurons in Large Language Models: Dead, N-Gram, Positional
> There is no mechanism to say "oh these neurons will never activate, let's prune them"

Not at training time. But after training you can evaluate which activations rarely fire and simply remove their columns from the weight matrix.

Or you could re-randomize them and continue training.

radarsat1··on Q-Transformer: Scalable Reinforcement Learning via Autoregressive Q-Functions
Honestly the best way is starting with implementing a Q table for some small grid-world problem. You get a lot of knowledge from doing that. Then a bit more work on understanding various approaches, e.g policy learning, world models. Then, reading text books, blogs tutorials, etc.

But "getting" the idea of Q learning for a small state space is fundamental and surprisingly approachable.

radarsat1··on Add extra stuff to a “standard” encoding? Sure, why not
Not protobuf, but liblo, a library I maintain for Open Sound Control, ended up supporting two different packetization standards for stream-based transmission, because there was a lack of this in the original specification.

Originally OSC was intended to be "transport independent" and in practice only used for UDP transmission, so packetization was left as a problem for implementations to figure out. Cue the predictable problem of different incompatible ideas.

In the specification update it was suggested to use SLIP encoding for packetization, but prior to this the library had already implemented exactly this kind of length-prefix encoding. So now the library allows to select one or the other for sending, and for backwards compatibility on reception the library tries to sniff which packetization protocol is in use. (Fortunately OSC has a predictable first character, and 99% of the time this does not line up with the first byte of the length prefix, so when it starts with that character, the software switches to SLIP mode and scans for the end-of-packet code, otherwise it assumes there is a length prefix.)

It's not ideal, but it works. But it would have been simpler if the original specification had mentioned stream-based transmission. I still get the occasional question about "what are these extra bytes" at the beginning of the message when TCP is used.

Overall I find that a lot of standards have had to be hacked a bit to support packetized transmission, including JSON [0], etc. It's an odd thing to leave out, but admittedly it is indeed a transport problem and most often not considered part of the "file format". So I don't know what the best solution is, but I guess it would be good if data format specifications say something about how to handle multiple serialized instances in a stream if no other standards apply.

Interestingly YAML seemed to have thought about this with the "---" and "..." symbols, even in version 1.0 of the spec [1].

[0]: https://jsonlines.org/

[1]: https://yaml.org/spec/1.0/index.html

radarsat1··on Q-Transformer: Scalable Reinforcement Learning via Autoregressive Q-Functions
Cool to see an approach to using transformers that sticks closer to traditional RL than the decision transformer. The action dimension trick here is clever. Curious to see where this can be taken, game playing, multiagent, etc.
radarsat1··on Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion
I think it depends on how you define that. People were quite happy with HMM-based MIDI generators that could generate Beethoven- or Mozart-like sequences 10, maybe even 15 or 20 years ago. But of course other people pointed out the problems of it being boring eventually. Then LSTMs improved long-term dependencies and people were impressed by the improved quality of generating whole musical pieces. But still others thought it was not good enough. Then the goalposts moved again with transformers and neural vocoders and now we want top-40 direct audio generation. And these latest systems can kind of sort of do it! But still there are people who demand better. And so on, things will continue to improve.

Progress only moves as fast as expectations, and expectations move with technology. Music is not special in this respect. So you could say at any given time in the past that some people "see meaningfully high quality" and others are disappointed. You see exactly both these sides of the spectrum even now with text-to-image and text-to-audio technology.

radarsat1··on The trouble with supermarket self checkouts
I don't mean it as any kind of rule of course, it's a massive generalization. But in general in Europe we are more used getting on the train without a barrier and then later having to show our ticket to the conductor who walks through the train, compared to having to pass an impenetrable barrier by showing our ticket just to get onto the platform.

Obviously both systems exist in both places, but I think the former exists more in Europe. It is just a comparison of course, take it as you will. I do find someone checking my bags randomly bit annoying but I prefer it to fighting an automated system that manages a poor approximation of figuring out what you put in your bags. At least in the Netherlands I have not seen that latter kind of system here, and I suspect it's precisely because people would not like it.

radarsat1··on The trouble with supermarket self checkouts
Albert Heijn in the Netherlands interestingly seems to have none of the mentioned problems.

You swipe or scan your items and place them in your bags however you want. If you scan a beer bottle they let you keep going and someone watching sees that you are an adult just remotely ticks a box before you get to the checkout stage.

There is no scale checking the weight of your bags, the only annoyance is that occasionally a worker comes over before you complete your transaction to randomly check what you put in your bags. Aoart from that it's all very fluid.

On the other hand I have experienced all the mentioned problems in Canada at a Provigo in Quebec.

So I think all this is really just a matter of implementation and actually caring about user experience. All if these issues are solvable, the store just has to decide it cares about these issues and not accept a suboar solution.

One difference I'll note, of course a bit of a wild guess, is maybe one of culture. In North America people really don't want to be bothered by random checks, they feel it's a violation of their privacy or personal space. So everything, including security, has to be automated as much as possible. In Europe people are used to the idea of random checks maybe because it's part of the experience of taking public transport.

radarsat1··on Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion
> Generative models can certainly create midi, but no one has done it yet.

Note sequence generation from statistical models has a long history, at least as long if not longer than text generation.

Have a look at section 2.1 of this survey paper [0] that cites a paper from 1957 as the first work that applies Markov models to music generation.

And, of course, plenty of follow-up work 6 decades later on GANs, LSTMs, and transformers.

[0]: https://www.researchgate.net/publication/345915209_A_Compreh...

radarsat1··on Emoji Generator with AI
Interested to know what can be done with the up/down votes, some kind of RLHF for image generation?

Unfortunately it seems some "tiled images" must have made it into the dataset as half the time it generates an array of tiny images instead of a single emoji. Out of about 5 or 6 tries I got one good one that wasn't centered correctly.

radarsat1··on Gaussian splatting is pretty cool
My point is a bit different though. It's not about culture wars or being anti or pro AI. It's about how silly it is to call one optimization function AI and another "not AI" and then make a whole curfuffel about bucking trends, when what you've done is actually something very similar. (Maybe very interesting in its own right and a good solution though, with its own desirable properties.. again, I'm not commenting on the paper itself, just this one off-hand comment in the blog.)

In my view calling a solution "neural" and "AI" because you are optimizing a stack of relus, and then turning around and solving the same (or almost the same) problem by optimizing a bunch of Gaussian model parameters on the same data, and then saying that is "not AI" and "at least it's not neural!" is just a very specious and unnecessary way to delineate things imho. They're just two solutions to the same problem, both data driven, both generating "models".. they have differences, just not differences that I think warranted such a comment in the blog, and moreover I thought it weird to attribute some kind of normative evaluation based on how you categorize the solution.

radarsat1··on Gaussian splatting is pretty cool
Ah gotcha, thanks for pointing that out.
radarsat1··on Gaussian splatting is pretty cool
> This is using gradient descent and “differentiable rendering” and all the other things that are way over my head.

then..

> And finally, they have resisted the temptation to do “neural” anything ;)

So, they are doing something similar to NeRF, in a sense, but using a different basis function, and a slightly different target. The whole "neural" or not "neural" is just about what you are optimizing, but it's not that different, conceptually -- they are optimizing a data-driven approximator of a 3d scene-related variable based on images.

The big difference of course, based on reading this, is that NeRF models the whole light transmission function ("radiance field") whereas this seems to model only the boundary conditions (what the light "hits"). So given the different modelling target, then yes, a different, and perhaps simpler, representation basis definitely seems warranted, and is shown here to give good results. But the comment, "they avoided neural anything" feels a bit smug, as if "avoiding the hype" is a laudable goal on its own merits. As if people are using neural networks for no reason but because they're cool, and not because they're actually an appropriate solution that are shown to work really well in practice. They surely are cool, don't get me wrong, but the hype is so often justified, because neural networks are really good function approximators. They are also not just one thing -- the choice and breadth of architectures that we happen to call "neural networks" is huge, so explicitly avoiding this huge potential solution space for some reason without good reason feels pretty specious; heck you could even consider having a Gaussian "output layer" to a neural network, which is called an RBF, being something like this "splat" approach. (This is in response to the blog post, not the paper -- I'm sure the paper has plenty of justification for modeling choices.) Having said that, of course it's interesting to also explore simpler, perhaps more efficient approaches, but I don't see "resisting using neural anything" as a good thing, just because they are popular. It ignores that they are popular for a reason.

(nb: This assessment is just based on the blog post and giving my impression from the tone of the introduction, I haven't read the paper yet, so might not be accurate with respect to the paper contents. I just found this off-hand comment in the post worth reflecting on.)

radarsat1··on Gaussian splatting is pretty cool
> (Although I am suprised that there's not more attention being paid to the extra spatial relationships that can be inferred from merely knowing something is a video. Surely it could be used as an extra constraint when inferring camera poses - you know the camera can only move in certain ways from one frame to the next?)

Can I ask why you say this? Camera motion models including Kalman filters of various kinds and constraints on derivatives etc, are absolutely used in SLAM and photogrammetry afaik, and have been for a long time.

radarsat1··on The Battle over Books3
People have somehow conflated artificial intelligence with "oracle that knows everything" and so keeping models up to date with recent knowledge has become a must. Of course, that could be done by fine tuning techniques and better architectures that can outsource knowledge retrieval to tools, all very interesting ongoing work on that topic, but even these approaches require to not be "blocked" in their data retrieval tasks for them to work well.

I'm sure you can do lots of interesting research using outdated datasets but for companies creating products this will not be sufficient.

radarsat1··on The problem of creeping commercial surveillance (2022)
> The loyalty card still works

huh? at least in my supermarket it doesn't work, unless you activate it, the new thing being that you can't do that anymore without installing the app. Not sure if that was unclear in my story.

radarsat1··on The problem of creeping commercial surveillance (2022)
Supermarkets used to have coupons that you had to cut out of the paper to get discounts on certain items. That was already bullshit -- if you can sell it at that price, please sell it at that price! Why the extra hoop of cutting up a magazine first?

Then supermarkets discovered bar codes. If you can convince people to carry around a barcode, suddenly you can track their usage patterns in exchange for a small discount.

Okay fine, but at least it's anonymous, right?

Then supermarkets realized they could get you to "activate" your barcode by going to a website. Hey, this way not only do we get their usage patterns, but we get data about them too! Demographics, etc. And people do it! Because of discounts.

Sure, some of us give fake info, but plenty of people just give their real names, addresses, and emails. Fantastic.

Well, lately I accidentally broke my barcode keychain thingy while locking up my bike (don't ask) and so next time I was in the supermarket I picked up a new one. Tried to activate it on my phone at the automatic cash but couldn't figure out how to log in to the site. Saw a link to the app but figured I'd look more into it later.

So later, I did that.. and discovered you can't log in to the site, because you can only activate the barcode by installing the app!

So, anyway, that's the story about when I gave up getting discounts at the supermarket.

radarsat1··on GIL removal and the Faster CPython project
Your JournalParser sounds like it could be implemented using normal Python threads or by an asyncio event loop without much performance problems. If I understand all it's doing is watching for events and posting a signal somewhere, so it doesn't sound like the kind of application that is CPU-bound.
radarsat1··on Git-appraise – Distributed Code Review for Git
There was an attempt at some point to build decentralized issue tracking into git: https://github.com/duplys/git-issues

I was always a bit disappointed that it didn't catch on, but I realize it would require a UI and probably buy-in from companies like Github and Gitlab, whose whole business is based around the lack of these features in git, so it was DoA, unfortunately.

Update: looks like that link above was a separate attempt from [1] and [2]

[1]: https://github.com/dspinellis/git-issue

[2]: https://github.com/MichaelMure/git-bug

radarsat1··on Down and Out in the Magic Kingdom (2003)
Interesting, yeah I think that aspect of the novel definitely went over my head.
radarsat1··on Down and Out in the Magic Kingdom (2003)
I read this a while ago. I don't know if it's because I've never been to disney world or know enough about it, but I didn't really "get" the book. Some of the concepts were interesting like dead heading and backing up but not really things I haven't read by other authors of the same decade. Although, maybe some of these ideas stem from this book so not sure. But overall I found it hard to follow, mostly because i didn't understand the importance of the various venues or why the various characters were in that particular location in the first place.

I think the point was more about the kind of social credit system, in that sense it seemed not to be able to decide if it was depicting a utopia or dystopia. But please tell me if I'm way off, interested in hearing what people felt about this book.

radarsat1··on Llama: Add grammar-based sampling
This reminds me of something i did once using a 1d CRF at the output of a sequence model. I set "impossible" transitions to -inf so that transitions between certain state pairs were simply never predicted. I could imagine executing something like this that is conditional on the current grammatical context.

On fact it's a bit surprising to me how little I see CRFs mentioned in the context of language models. They are useful whenever you want to model or learn transition probabilities.

radarsat1··on Introduction to ActivityPub (2021)
It's possible to download parts of torrents though.
radarsat1··on JDF.jl – Julia DataFrames serialization format
> I think there’s even an Excel plug-in for parquet these days.

While CSV comes with plenty of known problems, its one advantage, which apparently remains a significant one, is that it doesn't require any plugins or special software support at all, just a text editor. It seems that's a big enough feature that no attempt to fully replace it has really worked out. Amazing when you think about it, the power of plain text encoding.

Of course I agree that it's often not the best choice but somehow it remains incredibly useful and can even be hard to make arguments against it on a team project because it just kind or works well enough in a lot of cases and is simply a lowest common denominator when people can't agree on what format to use.

radarsat1··on Geddit: Open-source, Reddit client for Android without using API
> Neat. I wasn't aware that there's an endpoint you could use to retrieve JSON data without an API key.

Call me naive but in fact I always thought that's what the apps were using. One thing that took me by surprise in the whole fiasco was that the apps were actually using their own API keys -- I always thought I was just logged in to reddit in my own account. I figured it might be using some backend service for storing some app metadata but I didn't realize it was needing to communicate with reddit using the app developer's account on my behalf.

radarsat1··on Emacs GUI Library
> behind the scenes it is all text parsing. There's no "node" structure.

But I feel like this is one of the strongest things about org mode and something that keeps it really apart from similar tools.

Rather than being forced to think in "org" and manipulate its "objects", you just manipulate text, and if you happen to shape that text into something org recognizes, then it knows how to work with it.

This is amazingly inviting because you never feel "stuck" like you converted something to the wrong type of thing and now can't get it back, or whatever. Instead if you make a mistake you just.. change it to what it should be. It feels very fluid and you learn it quickly because you're not presented with trying to fit your thoughts into someone else's abstractions. You just write what you want and progressively learn more and more how to naturally shape your work into something that org can help you with.

radarsat1··on Classifying customer messages with LLMs vs traditional ML
It comes from this paper [0], and I believe the idea is that the LLM was not trained on the task in question, but is able to do it with only instructions (zero shot) or with one or a few examples (few shot). The paper rightfully points out the unexpected fact that the model is only trained to predict the next word and yet can follow arbitrary instructions and perform tasks that it was not explicitly trained to do.

[0]: https://arxiv.org/abs/2109.01652

radarsat1··on AI weights are not open “source”
> Weights probably aren't copyrightable in the US. ... is understood to require human authorship.

Are you arguing here that because the weights come from an optimization program, they are not "human authored"? If so I find that to be a strange assertion. If I'm working every day on my model and training algorithm to ensure it produces the best weights possible to solve my problem, I would be very surprised for someone to tell me I have no ownership over those weights because they are generated from a program I wrote and data that I own.

radarsat1··on Everything that uses configuration files should report where they're located
It should be easier to track file opening history of any running program. There is strace, but grepping/parsing the output for this purpose gets very messy.
radarsat1··on The 'fuck you' pattern (2021)
It's actually not that bad a name for this if you think about the concept of 'fuck you' money, ie. that some people have so much money that 'fuck you' they'll do whatever they want.

Similarly Instagram can only get away with this behavior because of their level of following as a social media platform. So many people use it that people want to use it to follow others, and will consequently put up with this bullshit, and consequently to they can get away with whatever they want because 'fuck you', people will keep coming anyway.

Unfortunate, but not sure what lessons can be taken from it for other players in the market, since (fortunately) few other platforms are in a position to get away with this kind of thing.

← PreviousPage 11 of 34Next →