HNHacker News
TopNewBestAskShowJobs

versteegen

1,857 karma · joined May 8, 2015

Independent AI researcher (neuro-symbolic NLU, data compression), FLOSS contributor, Generalist & Luddite.

rver017@aucklanduni.ac.nz

submissionscomments
versteegen··on Niklaus Wirth has died
> I ask because I've found `async/await` to just be an irritating overhead

Then what you want are coroutines[1], which are strictly more flexible than async/await. Languages like Lua and Squirrel have coroutines. I and plenty of other people thing it's tragic that Python and Javascripts added async/await instead, but I assume the reason wasn't to make them easier to reason about, but rather to make them easier to implement without hacks in existing language interpreters not designed around them. Though Stackless Python is a CPython fork that adds real coroutines, also available as the greenlet module in standard CPython [2], amazing that it works.

[1] Real coroutines, not what Python calls "coroutines with async syntax". See also nearby comment about coroutines vs coop multitasking https://news.ycombinator.com/item?id=38859828

[2] https://greenlet.readthedocs.io/en/latest/

versteegen··on Lazarus IDE 3.0
Does FPC avoid using versioned symbols from glibc that would prevent it from running on distros without a recent glibc, unlike any C/C++ program compiled using system libc headers?

This is a particular problem with recent Slackware-current: I used to compile portable x86/64 linux binaries (from C+C++ sources or from FreeBASIC compiled to C) by patching out those versioned symbols using ld --wrap= flags, but since glibc 2.34 even that no longer works: https://news.ycombinator.com/item?id=32479545

versteegen··on Show HN: Heynote – A dedicated scratchpad for developers
Just like emacs itself, it's hard to describe what org-mode is, since it has so many features, most of which any individual probably won't use or even know about. You didn't mention tables, hyperlinks, tagging, exporting documents...

See the manual's table of contents: https://orgmode.org/org.html

versteegen··on Word2Vec received 'strong reject' four times at ICLR2013
I haven't seen a manuscript that could not made better, but peer review isn't the best way to improve it.

For a start, even for a journal, the fact that there are multiple reviewers means that none of them feel much responsibility to help improve the paper. They (and I, honestly) oft seem content to read part of it and just provide comments/criticism about that. Especially at conferences the possible verdicts are "reject" or "can't find a reason to reject". I've submitted at least one paper where I wanted the reviewers to point out flaws and reject, and I was left sorely disappointed (it was rejected of course, but not for good reasons) and still in the dark about whether the research had merit.

That said, OpenReview reviews are on average far better than the ones I've received, I think it's fantastic.

versteegen··on Do large language models need all those layers?
Because the training data/model size/compute tradeoff derived from that paper is highly suboptimal (too many parameters) compared to the ones from the later Deepmind scaling laws [1]. And then Meta researchers recommended using even smaller models, to trade-off training- and inference-time compute [2] (which I thought was pretty obvious if you care about more than just benchmarks).

[1] https://arxiv.org/abs/2203.15556 Training Compute-Optimal Large Language Models

[2] https://arxiv.org/abs/2302.13971 LLaMA: Open and Efficient Foundation Language Models

versteegen··on AMD's CDNA 3 Compute Architecture
Wow, a very nice reimplementation.

To go on a tangent, I note your custom 'BCML1' 5bit per weight compression codec and your optimised hand-coded AVX2 to encode it... was that really needed? Are the weights encoded on every startup? Why not do it once and save to disk?

versteegen··on Do large language models need all those layers?
Maybe that's why the Phi models do so well for their size. I'm guessing they may have been trained close to convergence, but the loss graphs aren't published. Phi-1.5 (1.3B parameters) was trained on 150B tokens (5 epochs), yet phi-1.5-web was trained on 300B so they didn't stop for lack of compute. Phi-2 (2.7B params) was trained on 1.4T tokens, epochs unknown.
versteegen··on Do large language models need all those layers?
Mixtral 8x7B actually has 46.7B total parameters, not 8*7B = 56B. The reason being that not all parameters are multiplied 8x.

Also it uses 12.9B parameters per token, not quite comparable to 7B models.

versteegen··on How to Leave the House of Lords
> The monarch is ... entitled to vote, no court would ever deny them such rights as they really derive from shared freedoms given to all UK nationals.

You seem to be correct that the monarch is technically not barred from voting (although they don't and shouldn't), but the Lords are not entitled to vote, so I don't think the reason you give is correct.

versteegen··on Python Expertise Level – Self-Assessment
You forgot the walrus operator!

  accum = initial
  series = [accum := bifun(accum, value) for value in seq]
  # result = series[-1]
To be honest, this is the first time I've used the walrus operator.
versteegen··on Dagor Engine and Tools source code from Gaijin Games KFT
They fill two different niches, daScript is statically typed, AoT compiled and attempting to outperform LuaJIT, Quirrel is dynamically typed and better for writing game logic.

Note that Quirrel is a fork of Squirrel (in early 2019), which is a language very heavily inspired by Lua (you could say Lua-done-over) with a very similar embedding API, but much more like a typical OO language rather than quirky (which makes it very similar to Javascript). Squirrel was most famous used by many Valve games like Portal 2 and L4D2, and I recommend it. Quirrel attempts to make Squirrel more Python-like, easier, add tooling like static analysis and tests, and to continue development, as Squirrel has only received the odd bugfix for years and been neglected. I've been watching Quirrel for years, and development has been picking up. They seem to have about 1.5-2 people working on it fulltime. The language is diverging more and more from Squirrel (and further towards Python) and getting bigger.

versteegen··on Chrome Removing Theora Support
Well I'm looking for the same thing, for a lightweight 2D game engine supporting decades old hardware (the game player is 2MB zipped, 1MB for a minimised build). Animations are likely to be low resolution and seconds long, so a huge decoding library would be pointless. Sure, most users don't care about file sizes, but some (including me) still do. Especially for the wasm and Win95 ports.

I considered libtheora, the library size is good but the compression/visual quality is awful compared to the alternatives.

versteegen··on Write more "useless" software
> Nothing can recreate the first ten or hundred hours of gameplay, when you still keep discovering small and big stuff ...

Well, I played a lot of Minecraft back in the day, and I have never even heard of a single thing you just mentioned, which seems to disprove what you said there: In fact, as the devs keep adding stuff to the game, it can apparently stay fresh.

versteegen··on First word discovered in unopened Herculaneum scroll by CS student
Actually it's much worse than that. Only a very small fraction of the first of the two scanned scrolls has been segmented/unwrapped after 5 months, and it's the easiest parts that are done -- about 1000cm^2 across something like 100 layers of papyrus 10cm wide. Only 50cm^2 of scroll 2 is done. Where the sheets are right against each other is much harder.
versteegen··on First word discovered in unopened Herculaneum scroll by CS student
Other things like hlb-gpt? :)

I'd studied the problem to work on it, but didn't get as far as you. I agree that intuitively (not backed up by experiment) I expect that preprocessing to further flatten the segments and other hand-crafted features based on desired invariances should work well. It seems that a lot of people really have just used 2D convolutions, applied to just one or a few surface layers and then combined. So I'd also be interested in your code.

> I surmised that that 'crackling' effect actually is just of course slices of the data over different rifts in the parchment, and that the data of the ink lay on the manifold of that crackling and bending.

I'm afraid I can't follow this.

versteegen··on One game, by one man, on six platforms: The good, the bad and the ugly
What about calling navigator.storage.persist() to prompt the user to allow persistent storage?

Wow, thanks for the warning about that itch.io problem. It's discussed here: https://itch.io/t/2636350/localstorage-doesnt-work-for-html5... and behaviour of Safari's default-on "Prevent Cross-site Tracking" here: https://stackoverflow.com/questions/63922558/safari-localsto...

So this is happening because itch.io hosts the game on a CDN subdomain such as v6p9d9t4.ssl.hwcdn.net which may change at any time? If it doesn't change, it sounds to me like there should be no problem, but I suspect I misunderstand. And sounds like it's Safari, not just iOS?

I'm about to start letting users export their games to html5, almost all of which will end up on itch.io, so really need to solve this too.

versteegen··on One game, by one man, on six platforms: The good, the bad and the ugly
That's horrible and dastardly, but at least it's far easier for users to bypass SmartScreen on Windows than the block on Macs. I wonder how many Mac users actually know how to.

If you just get a regular (cheaper) code signing certificate I realise SmartScreen will still block you anyway until enough people have installed it, but how many is "enough"?

Also, previously: "Microsoft Defender SmartScreen is hurting independent developers" https://news.ycombinator.com/item?id=23392404

versteegen··on First word discovered in unopened Herculaneum scroll by CS student
These models certainly have found letters, although mostly they produce unreadable partial letters. Look at the images from the "What’s next?" section of the article! [1] They certainly seem better than human annotation, and more importantly don't hallucinate whole letters. Casey Handmer made a submission for this First Letters prize [2] based solely on hand-annotation and wasn't awarded it, because it's really unconvincing. His letters [3] don't look at all like the computer annotations.

[1] https://scrollprize.org/img/firstletters/youssef-new.png

[2] https://caseyhandmer.wordpress.com/2023/08/05/reading-ancien...

[3] https://lh6.googleusercontent.com/C-vKV4SdsyH961w6KPwD6rypt0...

versteegen··on First word discovered in unopened Herculaneum scroll by CS student
Unfortunately some people are taking you seriously. The whole scrolls are a huge amount of data, but for this First Letters prize using more than one GPU would be overkill. The amount of training data available is actually too small for deep learning to even work well.
versteegen··on First word discovered in unopened Herculaneum scroll by CS student
Can't speak for the video, but this is a bit misleading actually. What cracked this was actually visual inspection looking for patterns which could then be used as better training data, which so far apparently hasn't found very many letters that were too hard to see. Read the OP describing the iterative process of hand-annotation guided by output of a model, then retraining the model with the additional data, it's a fascinating technique! Simply using deep learning on the initially available ground truths without knowing what features the models should be looking for actually pretty much didn't work!

Also, so far the process of virtually unrolling the scrolls is mostly manual and extremely labour intensive.

versteegen··on First word discovered in unopened Herculaneum scroll by CS student
The lettering was found by looking for 'crackle' texture on papyrus segments from the CT scans which obviously were in the shape of Greek letters, and annotating those as training data. Unfortunately such crackle texture isn't visible, at least by eye, on most of the papyrus. Probably it's only that visible where the ink was very thick. You can easily see the difference in texture in this electron microscope image [1] (far higher resolution than the CT scans) but especially on the very edge of the inked area (the narrow strip in the left image; I think the whole right image is inked) where the ink was pushed to. I'm surprised the crackle was discovered only after the Kaggle Ink Detection contest. Looking at the CT-scanned fragments with infrared ground truths, which were used in the Kaggle contest, Casey Handmer wrote [2]:

> The ongoing apparent failure of deep-learning based ink detection based on the fragments indicated to me that direct inspection of the actual data would be more fruitful, as it has been here.

> ...

> I found similar “cracked mud” and “flake” textures corresponding to known character ink, but only for perhaps 10% of the known characters. It’s been a long day, I can probably find more on closer inspection, but that does make one wonder about automated ink detection and what that is seeing.

These new images are much better than I hoped for, but still only in one small area, so I'm still pessimistic about more than an odd sentence being readable.

[1] https://scrollprize.org/img/tutorials/sem.png

[2] https://caseyhandmer.wordpress.com/2023/08/05/reading-ancien...

versteegen··on Ask HN: Has anyone gotten complete, permanent relief from tinnitus?
I think my tinnitus is so easy to ignore because it's perfectly even and at extremely high frequencies I probably can't even hear any more. Which is what I assume the cause is.

Honestly I didn't think I had real tinnitus because I don't notice it for more than a moment for weeks at a time, at least not enough to be bothered, although I've have it as long as I can remember. It's not all that loud, but after concentrating on it for a while it's been there constantly and consciously for a good 10min already and it's not going away!

versteegen··on What we learned making a plastic injection mold with a Chinese mold maker
Addendum: I asked my dad and he said what I overlooked is that it takes far less thought and skill to make a mold or other tool today not just because of CNC (to cut paths you can't by manually turning knobs), but because of the algorithms that plan out how to make it.
versteegen··on What we learned making a plastic injection mold with a Chinese mold maker
My dad was (amongst many other things) a toolmaker who made plastic injection molds. These things can be hideously complicated, especially if the designer doesn't understand how it will actually be built, and as a result he once spent a year on one. In his days everything was manual, no CNC, no 3D models, no simulations, and probably no contact with the designer to make improvements.

He had to cut milling bits with custom profiles to cut custom tools such as a series of graphite and copper spark eroder forms to cut just some of the corners and shapes in the (paper) technical drawings, then repeat for the next ill-advised bevel or interior sharp angle or even lettering which is too close together to get a milling bit inbetween two letters. Figuring that out was his responsibility alone. Instead of computer control, he'd have to do something like first create a scale model of part of the tool out of a block of plastic, then use a pantograph (a mechanical linkage allowing you to trace a shape with a pen and have the movement magnified/minified) to copy the shapes on a milling machine -- even shapes that seem simple. And then maybe throw away (actually, take home as a souvenir) a month or two's work if you cut slightly too deep -- these were high precision parts. Not because the plastic had to be so precise, but because the completed dies are made out of many steel parts that have to fit tightly together and slide past each other, as you can see in the article.

Considering the ~two months quoted in the article for this mold, I'd think a lot of that complexity is still essential despite modern computerized machining. I'm sure a great deal of ingenuity and many steps are still needed.

versteegen··on Ask HN: Degree Finished, What Now?
Yeah, I find it unfair and have complained. dang told me the spam filters "are tuned a bit more strictly for new accounts" but looking those new user posts that do get marked dead (quite a large %) I typically can't find even a keyword to suggest why.
versteegen··on Ask HN: Degree Finished, What Now?
I don't have a real answer for you, except to say every employer/work environment is different, so don't assume the worst and get discouraged by mere assumptions and impressions. Also I just saw this recent thread which might have some useful advice, though I feel bad linking something so discouraging, and there's surely better threads if you look for them:

"Ask HN: I have a computer science degree, how to I get a software developer job?" https://news.ycombinator.com/item?id=36876485

Also, this post was marked dead, because you're a new account (they've made the filter really aggressive), until I just vouched for it. But it still doesn't even show on the 'ask' tab. You might have to try posting again.

versteegen··on 1.3B Worldcat scrape and data science mini-competition
Good news, such a site was launched very recently: https://www.thenose.cc/

There's plenty of discussion about it by 'nostril' here on HN: https://news.ycombinator.com/threads?id=nostril

versteegen··on Changes to Hosting on Replit
I have a debit card but not a credit card (and can't get one). I'm locked out of many hosting services, which don't want to allow me to pay up-front. It's very American-centric and annoying.
versteegen··on UK Parliament undermined the privacy, security, freedom of all internet users
We can instead look at the statistics for scanning of communications that aren't yet end-to-end encrypted, which will provide a good indication for the effect of bypassing encryption.

[Edit:] I know this is not a great source, but from https://www.gov.uk/government/news/home-secretary-urges-meta...

> Currently, 800 predators a month are arrested by UK law enforcement agencies and up to 1,200 children are safeguarded from child sexual abuse following information provided by social media companies.

There are also some interesting NCMEC statistics which provide a sense of scale, at https://www.missingkids.org/gethelpnow/cybertipline/cybertip...

> The CyberTipline receives reports from the public and online electronic service providers (ESPs). ... U.S. based ESPs are legally required to report instances of child sexual abuse material (CSAM) or “apparent child pornography” to the CyberTipline when they become aware of them, but there are no legal requirements regarding proactive efforts to detect CSAM

(Reports are also in various other categories aside from CSAM.)

> In 2022, NCMEC staff escalated over 49,000 reports to law enforcement as the reported incident was urgent in nature or there was information that a child was in imminent danger. ... NCMEC designated 49% of the reports from the tech industry as “actionable” when referring them to law enforcement in 2022

(49000 is a heavily filtered fraction of the 32 million total reports.)

versteegen··on Circles do not exist
> simply adding more nodes improves accuracy exponentially

According to another comment, it actually improves at O(n^6).

← PreviousPage 10 of 25Next →