HNHacker News
TopNewBestAskShowJobs

amilios

650 karma · joined January 14, 2014

submissionscomments
amilios··on LLM Inference Using Julia's Transformers.jl
At this point I doubt any language will be able to oust Python as the go-to language for machine learning. The Python ecosystem is just too strong, the value-add for the new language would have to be extremely high, I just don't think it's possible
amilios··on Thought Propagation: An analogical approach to complex reasoning with LLMs
Ah I see. Yeah the criticism seems fair, gotta at least prepend "Abstract:" or something haha.
amilios··on Thought Propagation: An analogical approach to complex reasoning with LLMs
The abstract of a paper without quotations? No, I don't think so. What do you mean exactly? Someone posted the abstract as a text post directly?
amilios··on Outlier Weighed Layerwise Sparsity: A Missing Secret Sauce for Pruning LLMs
Paper abstract: Large Language Models (LLMs), renowned for their remarkable performance across diverse domains, present a challenge due to their colossal model size when it comes to practical deployment. In response to this challenge, efforts have been directed toward the application of traditional network pruning techniques to LLMs, uncovering a massive number of parameters can be pruned in one-shot without hurting performance. Building upon insights gained from pre-LLM models, particularly BERT-level language models, prevailing LLM pruning strategies have consistently adhered to the practice of uniformly pruning all layers at equivalent sparsity levels, resulting in robust performance. However, this observation stands in contrast to the prevailing trends observed in the field of vision models, where non-uniform layerwise sparsity typically yields substantially improved results. To elucidate the underlying reasons for this disparity, we conduct a comprehensive analysis of the distribution of token features within LLMs. In doing so, we discover a strong correlation with the emergence of outliers, defined as features exhibiting significantly greater magnitudes compared to their counterparts in feature dimen- sions. Inspired by this finding, we introduce a novel LLM pruning methodology that incorporates a tailored set of non-uniform layerwise sparsity ratios specif- ically designed for LLM pruning, termed as Outlier Weighed Layerwise sparsity (OWL). The sparsity ratio of OWL is directly proportional to the outlier ratio observed within each layer, facilitating a more effective alignment between layer- wise weight sparsity and outlier ratios. Our empirical evaluation, conducted across the LLaMA-V1 family and OPT, spanning various benchmarks, demonstrates the distinct advantages offered by OWL over previous methods. For instance, our approach exhibits a remarkable performance gain, surpassing the state-of-the-art Wanda and SparseGPT by 61.22 and 6.80 perplexity at a high sparsity level of 70%, respectively. Codes are available at https://github.com/luuyin/OWL.
amilios··on DPO fine-tuned Mistral 7B beats Llama 70B on MT Bench
Can anyone corroborate this anecdotally? I.e. has anyone actually looked at the output of the two models side-by-side for common tasks? There's lots of talks these days about academic benchmarks being pretty "broken" for modern LMs, and not really properly showcasing the differences between models. I wonder if that's the case here or if the model is genuinely better.
amilios··on ToolEmu: Identifying the Risks of LM Agents with an LM-Emulated Sandbox
Paper abstract: Recent advances in Language Model (LM) agents and tool use, exemplified by applications like ChatGPT Plugins, enable a rich set of capabilities but also amplify potential risks - such as leaking private data or causing financial losses. Identifying these risks is labor-intensive, necessitating implementing the tools, manually setting up the environment for each test scenario, and finding risky cases. As tools and agents become more complex, the high cost of testing these agents will make it increasingly difficult to find high-stakes, long-tailed risks. To address these challenges, we introduce ToolEmu: a framework that uses an LM to emulate tool execution and enables the testing of LM agents against a diverse range of tools and scenarios, without manual instantiation. Alongside the emulator, we develop an LM-based automatic safety evaluator that examines agent failures and quantifies associated risks. We test both the tool emulator and evaluator through human evaluation and find that 68.8% of failures identified with ToolEmu would be valid real-world agent failures. Using our curated initial benchmark consisting of 36 high-stakes tools and 144 test cases, we provide a quantitative risk analysis of current LM agents and identify numerous failures with potentially severe outcomes. Notably, even the safest LM agent exhibits such failures 23.9% of the time according to our evaluator, underscoring the need to develop safer LM agents for real-world deployment.
amilios··on Thought Propagation: An analogical approach to complex reasoning with LLMs
Abstract: Large Language Models (LLMs) have achieved remarkable success in reasoning tasks with the development of prompting methods. However, existing prompting approaches cannot reuse insights of solving similar problems and suffer from accumulated errors in multi-step reasoning, since they prompt LLMs to reason from scratch. To address these issues, we propose Thought Propagation (TP), which explores the analogous problems and leverages their solutions to enhance the complex reasoning ability of LLMs. These analogous problems are related to the input one, with reusable solutions and problem-solving strategies. Thus, it is promising to propagate insights of solving previous analogous problems to inspire new problem-solving. To achieve this, TP first prompts LLMs to propose and solve a set of analogous problems that are related to the input one. Then, TP reuses the results of analogous problems to directly yield a new solution or derive a knowledge-intensive plan for execution to amend the initial solution obtained from scratch. TP is compatible with existing prompting approaches, allowing plug-and-play generalization and enhancement in a wide range of tasks without much labor in task-specific prompt engineering. Experiments across three challenging tasks demonstrate TP enjoys a substantial improvement over the baselines by an average of 12% absolute increase in finding the optimal solutions in Shortest-path Reasoning, 13% improvement of human preference in Creative Writing, and 15% enhancement in the task completion rate of LLM-Agent Planning.
amilios··on 41% of French pop in favour of limiting everyone to 4 flights for entire life
Suffice to say I think this would only work in places like Europe where everything is relatively densely connected by rail. For me personally I live in North American and I visit my family in Europe once a year. Cutting me off from them would suck and effectively force me back to Europe :) this seems like it's coming from a place of "privilege" per se of everything being nearby.
amilios··on Intelligence from 'Five Eyes' nations helped Canada link India to killing
...good? Isn't this kind of the point of Five Eyes? Seems like it's working as intended. In this case Canada is extremely justified to use whatever intelligence they have at their disposal I think, this was an extrajudicial killing of a Canadian citizen on Canadian soil by a foreign government. Still blows my mind. At least with the UK Novichok case it was a foreign national, this was a Canadian national.
amilios··on Declarative GUI for C#
Are people still using C# outside of like, corporate settings? I have the impression that especially with the recent Unity scare market share for C# must be trending downwards...
amilios··on Low-Code Programming Models
Not sure if this counts, but many successful games have been made with Game-maker-style tools, which I think would be classified as "low-code". E.g. Undertale
amilios··on Mistral 7B
Oh yeah that's true! Just block the stop token. But yes, my thought is that there are scenarios where it can get "stuck" as you said. I'll look at the PR, thanks!
amilios··on Mistral 7B
How does this actually work though, since the model could e.g. abruptly end the generation giving something syntactically invalid? Doesn't it need to look at the whole output at some stage?
amilios··on Mistral 7B
Huh! Nevermind then! I take it back. Would be interesting to see what kind of tuning they did/pit the model head-to-head with LLaMA-2-7B-chat. Seems like they did just instruction tuning but not RLHF? So I assume Mistral won't be refusing to answer etc., probably doesn't have many safety guardrails (I guess that's desirable for some!)
amilios··on Mistral 7B
Not a fair comparison as Mistral is not instruction tuned. You need to write a "continuation-style" prompt or few-shot it, rather than just giving it a direct order. Honestly this is one of the reasons imo that Mistral won't see wide acceptance, LLaMA-2-7b-chat is RLHF and instruction tuned and is just so much easier to use as a result.
amilios··on I made a transformer to predict a simple sequence manually
The technical reason it can't be done (or would be very difficult to do) is that weights are typically very uninterpretable. There aren't specific clusters of neurons that map to one concept or another, everything kind of does everything.
amilios··on Claude and LLaMa get basic human anatomy wrong
I mean, it's a well known fact that LLMs do badly in the "long tail" of the distribution: anything they haven't seen many many times (e.g. relatively obscure specialist knowledge), they are definitely very likely to get wrong. The more "popular" the thing you're asking from them is, the more likely it is they'll answer correctly.
amilios··on A BERT for laptops, from scratch
To clarify for people familiar with the names but not what corresponds to what:

BERT - Encoder-only - embeddings for downstream tasks

GPT/OPT/etc - Decoder-only - language generation

T5/T0 - Encoder-decoder. Kind of does both?

amilios··on An evaluation of risks associated with relying on Fortran
Very interesting. I'll admit I really enjoyed the contrasting reviewer judgements, having the counter-perspective right beside the report text. Is this standard practice for US Gov reports? If not it absolutely should be!
amilios··on CheatGPT – The hidden wave of employees using AI on the sly
I see no issue with this to be honest. Would we consider someone using VSCode in a company where everyone is still using nano as "cheating", because they decided to use a superior tool to augment their productivity? Is using Github Copilot to 2X your programming productivity "cheating"? If the answer to both of these is no, then using ChatGPT to write texts cannot be cheating.
amilios··on Is AI’s hallucination problem fixable?
One way to improve this issue is to focus on retrieval-augmented models, i.e. models that include an explicit retrieval step from a corpus of documents and try to "cite" their sources as they produce text. However then oftentimes you get a more advanced form of the hallucination problem, where the model writes a piece of text, cites a document, but the document actually has nothing to do with what was written. It's a tricky problem for sure
amilios··on Depthboot – Your Chromebook, your way
Anyone able to give a quick rundown how this works without modifying the firmware?
amilios··on List of Generation Z Slang
In terms of the origin of a lot of the slang being from marginalized communities (LGBT) or communities of color, it's really interesting to think how much this has been driven by the internet, and the increased range that people producing content on the internet have. On the other hand, this kind of appropriation of slang can be seen as a new version of something very old (see: Elvis appropriating black music), so maybe not that much has really changed. The slang being taken from sexual minorities does feel new though (someone correct me if I'm wrong)
amilios··on How to Fine-Tune Llama2
It's incredible how far we've come with quantization and parameter-efficient fine-tuning a la QLora. It's legitimately possible to fine-tune multi-billion parameter models on Google Colab or a consumer GPU. I can see how something like this service can be useful to simplify things for those that don't want to deal directly with hardware, but if you do have your own hardware it's amazing how much more you can do with it than before.
amilios··on Members of US Congress on Mastodon
Would be wonderful if Congress or the US Gov did something like the EU: https://social.network.europa.eu/ A common ActivityPub server for all EU institutions.
amilios··on Why More Travelers Are Choosing Buses over Flights [video]
I feel like a big part of this is just affordability -- at a time when more and more people are struggling financially, busses are a more attractive option in terms of cost than flights in lots of cases. Simple as that
amilios··on Ask HN: What frameworks do people use
For? Web? ML? It's like asking what vehicle people use, depends on the purpose!!
amilios··on Elon Musk took x Twitter handle from its user without compensation
Is there any law or anything else binding that requires Twitter to compensate users for confiscating their handles? If not, is this really surprising?
amilios··on Risk of Myocarditis After Sequential Doses of Covid Vaccine and CoV-2 Infection
The thing is, even if the risk of myocarditis specifically is higher in young men with the vacccine vs. with infection, the vaccine protects against much more than just myocarditis. So even if the myocarditis risk goes up (relatively slightly it seems), if the other risks are all greatly lowered, as an overall package it still is attractive to be vaccinated. Looking just at myocarditis feels like being unable to see the forest for the trees.
amilios··on Stable Diffusion XL 1.0
Hmm this is a good point, diffusion requires several (many?) inference passes as you refine the noise into an image, right? Makes sense that this is more expensive to scale up. Thanks for the explanation!
← PreviousPage 3 of 7Next →