They are normal scrapers nothing specific to LLM as they are not yet used for training an LLM, unless I miss something from their architecture. So I don't get why they would be called LLM crawlers, when they are search engine crawlers. At least they could be called RAG crawlers for better nuance. The article linked in the post first sentence is more precise as it deals with scrapers: https://techcrunch.com/2025/08/04/perplexity-accused-of-scra...
Some people may be ok with search engines but not LLM training so it's not the same deal.
The first sentence of the article is literally wrong as it conflates LLM and the search part of a RAG (retrieval augmented generation, when you mix a web search and an LLM).
Blocking bots cuts you off from the next-generation search, because it cuts you off from search at all. So far, blocking LLM simply prevents you from being part of the training dataset, which is not the same thing.
Please stop upvoting such bad content it really makes Hackernews a terrible place for staying informed on LLMs.
On the electron part, it's common to (ironically) not support Linux. There are pretty annoying bugs with windows management (window will stay stuck in the background), build process are always OS specific, etc.
So often not worth the maintainance.
Hi, regarding streams interoperability I've documented how to handle file streams a while ago, after experimenting with Next.js old system (Node.js based) and new system (web based) : https://www.ericburel.tech/blog/nextjs-stream-files#2024-upd....
It sums up as "const stream = fileHandle.readableWebStream()" to produce a web stream using Node.js fs, rather than creating a Node.js stream.
Author's profile on Huggingface: https://huggingface.co/hesamation
HN mods suggested me to repost after a less successful share. I especially liked this article because the author goes through different types of embeddings rather than sticking to the definition.
For instance this one is a great, valuable read about AI and developers productivity: https://news.ycombinator.com/item?id=44526912
Post that are just opinions doesn't really fuel discussions as they don't bring enough raw material to build on. Some opinions are still worthy because they are original, but here I see nothing new.
Can we collectively stop upvoting low quality 300 words random opinions about llm based dev? As a professional in the field I find the qualify of what's posted on HN really super low in this area, compared to posts eg related to cloud infrastructure or software engineering. There are better posts and sources than that.
TL;DR: article means traditional websites with server navigation (now dubbed MPAs, multi page applications) have evolved positively with better handling of page transitions. But they choose to frame this idea by trashing SPA, while both MPA and SPA have a role to play.
It's nice to see the French engineers collecting quantitative data to back future decisions thanks to our solid environment agency (the Ademe) work and public-private cooperation.
That's the problem with raws large models, it should always be coupled with satellite small models and logic. It's (probably) easier to detect hallucinations using a traditional ML/DL model that can catch mismatches (it's easy to build a synthetic dataset for this) than transcribing. And the simplest piece of code can detect a silence and that it should match no text.
The article would probably be more complete with references to similar terms "linear program" or "integer program", unless this is also a slightly different meaning?
This means I believe we currently underuse LLM capabilities and their empirical nature makes it difficult to assess their limitations without trying.
I've been studying LLMs from various angles during a few months before coming to this conclusion, as an experienced software engineer and consultant. I must admit it is however biased towards my experience as an SME and in my local ecosystem.
This is a bit developer centric, I am much more impressed by the opportunities I see in consulting rather than applying LLMs to dev tasks.
And I am still impressed by the code it can output eventhough we are still in the funny intern stage in this area.
They try to get value at their scale, which is tough. Your local SME definitely sees value in an embedding-based semantic search engine over their 20 years of weird unstructured data.
In an agentic setup the value is half the prompts half how you plug them together. I am opposing for instance a big prompt that is supposed to write a dissertation vs a smart web scraper that builds a knowledge graph out of sources and outputs a specialized search engine for your task. The former is a free funny intern, the latter is growth percentage visible in the economy.
I don't have a subscription not even an Open AI account (mostly cause they messed up their google account system). You can't extract value of an LLM by just using the official UI, you just scratch the surface of how they work. And yet there aren't much developers able to actually build an actual agent architecture that does deliver some value.
I don't include the "thousands" of startups that are clearly suffer from a signaling bias: they don't exist in the economy and I don't care about them like at all in my reasonning.
I am talking about actual LLM developers that you can recruit locally the same way you recruit a web developer today, and that can make sense out of "frontier" LLM garbage talk by using proper architectures. These devs are not there yet.
Most of them are a bunch of prompts and don't even have actual developers. For the good reason that there is no training system yet and the wording of how you call the people that build these system isn't even there or clearly defined. Local companies haven't even setup a proper internal LLM or at least a contract with a provider.
I am in France so probably lagging behind USA a bit especially NY/SF but the word "LLM developer" is just arriving now and mostly under the pressure of isolated developers and companies like me. This feel really really early stage.
Developers haven't even started extracting the value of LLMs with agent architectures yet. Using an LLM UI like open ai is like we just figured fire and you use it to warm you hands (still impressive when you think about it, but not worth the burns), while LLM development is about building car engines (here is you return on investment).
My experience is different, so far I've defended R&D that I believed to be eligible to tax credits, in order for companies to be competitive with other countries that also subsidize R&D and innovation, namely USA and Canada.
You can't generalize a 7 billion tax cut system based on one journalist work (the same and the same is quoted again...), opinions based on a few rotten fruits in the basket, and an anti-startup trend that amplifies this hatred for political and ideological reasons.
Hi, CIR expert here, it's well and alive. There has been a communication push against it last year but relatively over.
It's 30% of R&D expenses as a tax cut.
Update: I think the 50% you mention is related to non salary expenses CII = a smaller similar system for innovation, which we differentiate from R&D. CII used to cover non salary expenses with a 50% forfait but this part has been removed indeed. It still covers 20% of salary expenses.
The complexity of an agent may range from something relatively simple to whatever level of complexity you want. So your projet sounds doable but you'll have to run some exploration to get proper answers. Regarding reliability, quality and security, it is as important to learn how to observe an agent system than learning how to implement an agent system. An agent/LLM-based solution is proven to work only if you observe that it actually works, experiments, tests and monitoring are not optional like eg in web development. As for security concerns, you'd want to take a look at the OWASP top 10 for LLMs: https://owasp.org/www-project-top-10-for-large-language-mode...
LLMs/agents indeed have their own new set of vulnerabilities.
I don't want to be the European in the room, yet I am wondering if you can prove the AI Act conformance of such a system.
You'd need to prove that it doesn't evolve into a problematic behaviour which sounds difficult.
I guess for the "backend" part it may be true, but there is a new "frontend" skillset of being able to actually implement a complex agentic workflow, and put that into a functioning product, which can be a lot of code. So closer to web dev in the end.
I have nothing against button clicking if it does work, as it may be a sign of maturity for a technical field. I click on a button in my car and turn a wheel and it usually go where I want, feels great. The equivalent would be the Microsoft ecosystem here.
Hi, I am working in making the term "llm developer" more popular in France and train people to this new job.
We will need a bunch of them in the months/years to come to implement advanced AI systems after companies manage to properly pick and set up their AI platforms. Currently people would tend to involve data scientists into this job, but data scientists are often less versed into the software engineering aspect, eg when they work more on notebooks than web apps.
The job is akin to being a web developer, so a "normal" developer but specialized in a certain field. Knowing the internals of LLMs is a big bonus, but you can start your journey with treating them as black box tools and still craft relevant solutions. You'll need to learn about running systems with databases (vector, graph, relational and nosql are all useful) and plugging multiple services together (docker, kubernetes, cloud hosting).
You'd want to make this tool domain specific - language, type of docs, perhaps target a specific documentation framework/format that is common and standardized enough. I fail to buy a content-agnostic summarization method, though I recognize it could be better than nothing.
Also benchmark or it doesn't exist.
A lot of criticism of LLMs is certainly legit and relevant, but are targeting very naive uses of the tools. Post-processing the output with an improvement prompt isn't even mentioned here. I'd rather read a criticism of agents, systems that produce SEO optimized articles etc. The debates would be more interesting.