HNHacker News
TopNewBestAskShowJobs

gillesjacobs

1,969 karma · joined January 29, 2018

[ my public key: https://keybase.io/gillesjacobs; my proof: https://keybase.io/gillesjacobs/sigs/naKVxhthsAZCj70Vsuk8ooWuF4gl1Ugl39KGf77zvrU ]
submissionscomments
gillesjacobs··on Ask HN: What is the state of AI data annotation of pdf documents using LLM
Everything is an engineering trade-off between classification performance, data requirements (quality and quantity for supervised fine-tuning/training), dev time and inference speed/resources.

You don't always have to do full document extraction. For a pdf form information extraction task, I once implemented a pipelined approach on images that extracts handwritten fields by using a trained segmentation model and a extraction model. You get error percolation but performance was between 85-90 F1, and we had human-in-the-loop for uncertain preds.

However, if top performance is a concern (and you're not dealing with handwritten stuff here) I would look into SotA document extraction models for extracting text, tables, graphs etc like LayoutLMv3 [1]. It's probably something like LayoutLM that OpenAI is using themselves. Every major cloud provider has its own document AI services that have similar performance, so that might get you going faster.

On the extracted text of scientific papers, you will run in to context size issues, so take it into account if using LLM zero-shot approach.

You could go with a chunked topic model approach for this too (BerTopic is a nice starting point).

In any case, it all depends on the specific task (amount of topics to classify), and the engineering trade-offs. Only you have all information to figure that one out.

1. https://huggingface.co/docs/transformers/model_doc/layoutlmv...

2. https://maartengr.github.io/BERTopic/index.html

gillesjacobs··on LLM leaderboard focusing on assessing their biases
The yes-or-no reference answer test is a really bad way to go about this. Maybe take a note out of RAGAS evaluation templates and use an LLM to iteratively summarise the nuanced category.
gillesjacobs··on LLM leaderboard focusing on assessing their biases
They have invested the most in preference alignment with special attention to DEI (for better or for worse).
gillesjacobs··on Google Scholar PDF Reader
Ever since the big UX app overhaul there is not much to like regarding functionality. They removed the pdf renaming and some other major automated file mgmt features from mendeley.

Other than that the commenting and note reader UI was pretty good. And overall UI/UX felt more modern than Zotero, also free (as in beer) cloud backup.

Today I had to do some literature review, and I reinstalled Zotero 7beta because I am not happy with the removed functionality from Mendeley.

gillesjacobs··on Google Scholar PDF Reader
Mendeley beat Zotero for me with automatic pdf renaming, organising and its highlight and note taking tools in the reader.

The Elsevier account integration is disgusting though, and I hate the idea of using all Elsevier product.

gillesjacobs··on Show HN: Causal 2.0 – Modern Financial Planning for Startups
Looks great, but I have my reservations about confidentiality of the data.

The financial model, expenses, hiring/ salary plan and basically all thin ledger financials of most SMBs/startups are highly sensitive.

If leaked or sold to third parties, it gives competitors (or maybe your own investors) insights in the technology, development expenses, and direction of the company.

How do you handle data confidentiality? How is the data security?

gillesjacobs··on Nanofont3x4: Smallest readable 3x4 font with lowercase (2015)
> Once the novelty wears off a "practical" example would be rendering "in-game book pages" that don't look like complete gibberish, or an "accurate print preview" with real text instead of blurry placeholder pixels that don't even look close to being the glyphs scaled down.

Low-res or LoD video game textures are a bit of a stretch but could be cool.

gillesjacobs··on Nanofont3x4: Smallest readable 3x4 font with lowercase (2015)
The lowercase is nigh unreadable but the all uppercase example is not too bad.

Would be cool to integrate in my split keyboard OLED screens in ZMK firmware.

gillesjacobs··on Grok
I believe it was Llama 1 that notoriously got leaked with a torrent on 4chan.
gillesjacobs··on Google Scholar search: "certainly, here is" -chatgpt -llm
Yes exactly the whole claim is fundamentally flawed.

Better queries would include proper timeframing and all LLM-preference-tuning phrases from https://github.com/AlpinDale/gptslop

But even then, I don't think this is the gotcha OP thinks it is: LLM generation and summarizing is a legit tool in the paper writing process, especially for conclusions.

In my academic experience, the paper introduction and conclusion is often written last-minute, close to the submission deadline. While a good intro and conclusion are important for the reader, very little time is made available for it since researchers spend maximal time on expanding material findings and experiments. No wonder that assistive writing tools are used to speed things up. As long as proper post-editing and proof-reading is applied.

gillesjacobs··on Speech and Language Processing (3rd ed. draft)
Of course BERT is an LM, I never claimed it wasn't. It is just much smaller than what is now termed "LLM" which is typically an extremely large generative decoder-only transformer for general multi-language (often instruction and chat tuned). I was training and finetuned dozens of LMs back when BERT-large was considered a GPU memory issue and I have finetuned hundreds LMs over my NLP research and engineering career. I have even implemented Hidden Markov Model LMs from scratch (as an exercise mostly, performance was mostly bad even in 2015).

When I used "specific task", I meant specialized, domain specific tasks like financial sentiment and event extraction in which I hold a PhD. As a matter of fact, for Fiqa SA finetuned Roberta scores 88% F1 while BloombergGPT scores 75% F1. [1] Still very impressive for zero/few shot learner, but depending on data availability, performance targets and inference cost tradeoffs, it might not need your meds.

My point was "small" masked encoder transformer LMs like BERT can still hold their own on narrow domain tasks. And what OP claims that all NLP is solved by prompting a general purpose LLM service is simply inaccurate.

1. https://arxiv.org/pdf/2310.12664.pdf

gillesjacobs··on Speech and Language Processing (3rd ed. draft)
LLMs typically do worse on specific extractive and classification tasks than a finetuned BERT-large model, so no you actually can't replace everything with LLM calls with similar performance.

(Cf. BloombergGPT paper all financial benchmark tasks).

And that's not even taking into account inference cost, but that is a business case issue.

gillesjacobs··on JetBrains IDE new Terminal Interface
Commonly used plugins preconfigured (I don't install my own anymore because it covers my bases) and the fastest possible plugin loading out of all zsh frameworks.
gillesjacobs··on JetBrains IDE new Terminal Interface
Obviously I mean "the terminal application installed on your local host system where you develop".
gillesjacobs··on Stonelifting Etiquette
Had my friend read the etiquette page, he probably wouldn't have lifted the stone without asking permission. So no, OP's website is beneficial wrt educating people to avoid the purported obnoxious behavior. Additionally the hobbyists that travel to lift stones probably already know not to lift monuments.

But I guess you mainly wanted to complain about obnoxious tourists in Japan, so fair enough. I highly doubt tourists in general are more disrespectful in Japan vs. elsewhere.

gillesjacobs··on JetBrains IDE new Terminal Interface
In almost a decade of using Pycharm I have not once used the builtin shell. (Debug console is another matter.)

It can never beat the system terminal+shell setup you pick yourself (for me Alacrity+z4h zsh distro). I can only see it as a nice-to-have for when you for some reason cannot install your own terminal. The UI-UX divergence from the system-wide shell setup will always be too wide and incur a learning-curve/feature gap.

gillesjacobs··on JetBrains IDE new Terminal Interface
On linux the default shell is simply in the $SHELL env var.
gillesjacobs··on Stonelifting Etiquette
You're really dedicated to the sport. They're not on your site yet, do you regularly add new ones or are monuments like these not in-scope?

Know any in the Benelux? The region seems pretty empty on the site, can't say there is a strong stone lifting or strongman culture here.

gillesjacobs··on Stonelifting Etiquette
Common sense, really. I got angry with my beefcake friend when we were are at an anthropological history site in Japan and he decided spontaneously to lift the ancestral lifting stone.

Edit: I recall the name: Hida Folk Village (Hida no Sato). There is a historic lifting stone on the side of a path, which you obviously are not allowed to pick up or would require permission. Exact location: https://maps.app.goo.gl/oxKc4M2djkk76s8y5

gillesjacobs··on OLMo: Accelerating the Science of Language Models [pdf]
It's more common than you think. I did the same for one of my research papers.
gillesjacobs··on The Rust project has a burnout problem
The things that cause burnout like high expectations, pressure and endless bureaucratic project management slog sound like any software development job. Except the devs aren't getting paid which exacerbates the issue.
gillesjacobs··on Chess-GPT's Internal World Model
To be fair, the term "world model" does not presume scientific understanding, factfullness or causality.

In an ideal AI model this would be the aim though.

gillesjacobs··on Orientable Dense Cyclic Infill for Anisotropic Appearance Fabrication
Project page: https://xavierchermain.github.io/fdm_aa/

Code: https://github.com/mfx-inria/anisotropic_appearance_fabricat...

gillesjacobs··on Turing awardees republished key methods and ideas without credit
> In general, the closer the name is to either end of the author list, the more significant their contributions likely were.

In ML, NLP, and many Humanities too, the supervisor (supervising professor/postdoc, lab head, PI) is put last regardless of contribution. The rest of the author list is ranked in descending order for contribution. Often the last author's contributions are very limited to obtaining funding or proof-reading.

Now this practice is controversial with many venues stating that obtaining funding and only supervising is not a valid reason for authorship, but in my experience this practice doesn't die out.

gillesjacobs··on Turing awardees republished key methods and ideas without credit
Within the field his name has become a punchline to joke about self-aggrandizing.

But yeah, you honestly have to wonder what it is like day-to-day working with someone with this level of delusions of grandeur.

gillesjacobs··on Nebula Genomics – First to offer consumer anonymous sequencing
I tried Nebula's kit but refunded after two attempts. They took on average 3 months to attempt to sequence my sample (due to sample batching) and they both failed. It costs me a fair bit to send the sample to Georgia (US) every time so the cost and wait times were too much.

It's a matter of luck whether your sequencing fails though so I will try again when they offer truly anonymous sequencing (my last attempts were eponymous).

Their website has been saying for at least two years that Anonymous Sequencing is coming. So take their product launch timeline with a grain of salt. In any case true anonymity and the price of 175usd are really impressive if they iron out the logistics.

gillesjacobs··on Insta360's Ace Pro combines Leica engineering with computational photography
Does this pivot from 360 cameras towards traditional action cams means that 360 cameras are too niche a market?

Too bad, even for traditional action camera use cases 360 cameras have several advantages, but I guess the higher cost limits the market.

gillesjacobs··on Ask HN: Is paid ChatGPT Plus worth it?
I bought it for the first three months, cancelled because I didn't use it frequent enough. Now I resubscribed but mainly because I needed image assets and DALLE3+GPT4 is really cool way to generate il image ideas.

I do believe that it might be cheaper for many developers to pay per request using the API platform and use one of the many frontend clones with your API key.

gillesjacobs··on OpenAI's board has fired Sam Altman
Turns out tribal nepotism doesn't make for good governance. Who could've known!
gillesjacobs··on Berlin's famed nightclubs, losing customers, face an uncertain future
You will be hard pressed to find a more pretentious music scene than (European) techno. And I am saying this as someone who enjoys the music.
← PreviousPage 4 of 14Next →