HNHacker News
TopNewBestAskShowJobs

popinman322

606 karma · joined May 29, 2015

Software Dev. Interested in UX, reliability, and performance as paths to user adoption.
submissionscomments
popinman322··on Hi everyone yes, I left OpenAI yesterday
For the tasks my group is considering, even a 7B model is adequate.

Sufficiently accurate responses can be fed into other systems downstream and cleaned up. Even code responses can benefit from this by restricting output tokens using the grammar of the target language, or iterating until the code compiles successfully.

And for a decent number of LLM-enabled use cases the functionality unlocked by these models is novel. When you're going from 0 to 1 people will just be amazed that the product exists.

popinman322··on StreamingLLM: tiny tweak to KV LRU improves long conversations
Previous discussion, on a link to the implementation: https://news.ycombinator.com/item?id=37740932
popinman322··on Posthog is closing their Slack community in favor of forum
I don't have my phone number attached to my GitHub account, if that's any indication
popinman322··on Posthog is closing their Slack community in favor of forum
As someone who participates in relatively high information density Discord servers (mostly ML related), search is rarely a problem for me. Yes, you need to do keyword search, but it's not the end of the world-- whenever I've needed to find a message 1-3 years back, I've found it successfully. Even in servers that I'm relatively new to.
popinman322··on 01-AI/Yi: A series of large language models trained from scratch
It's valuable data sometimes. Those types of comments show you how other people perceived something.

In this case my lens is colored by my interests; I've been thinking a lot on RL for LLMs and it slipped out here. For me I rely on phrases to quickly filter for (subjectively) interesting work. And assuming we value that subjective filtering of content, we make it easier for others to filter consistently when we use consistent language.

popinman322··on 01-AI/Yi: A series of large language models trained from scratch
Among arxiv publications there are 217 results that contain "large language model" in the full text and "from scratch" in the title or abstract.

There are 2873 results that contain "large language model" in the full text and use "pretrained" in the title or abstract. A 10x difference in publication count does make one feel more common than the other?

I'd need to get into more involved queries to break down the semantic categories of those papers.

popinman322··on 01-AI/Yi: A series of large language models trained from scratch
When I saw "from scratch" instead of "pretrained" I assumed this was trained using some novel RL setup-- there's value to using the same verbiage as everyone else.
popinman322··on Does Market Timing Work?
Half an hour of time recovered using a system not specialized to the work at hand. With better tooling and integrations that could scale to several hours a day spent doing more valuable work.
popinman322··on Meta pitches EU to charge a $10 subscription for ad-free Facebook and Instagram
YouTube is interesting because most large creators, who take a majority of views, also have ad space inside their videos
popinman322··on OpenAI predicted to generate over $1B
AWS offers optimized inference and custom infrastructure (https://aws.amazon.com/machine-learning/inferentia/).
popinman322··on Can a worker-owned restaurant work?
Everyone is different and has different motivations. We don't all think the same way, either.

Some workers just want a stable, sane work environment and are willing to compromise on pay for other benefits. These same people are typically willing to let others do more work for more pay because it makes their own lives easier.

One possible approach: you could do an exercise where everyone gets $N imaginary dollars and they can bid to not do certain tasks. The bid against each task is then the compensation for that task.

popinman322··on Show HN: LLMs can generate valid JSON 100% of the time
Does this work in tandem with beam search or does it do greedy sampling?
popinman322··on Show HN: LLMs can generate valid JSON 100% of the time
It should hopefully be a quick change to llama.cpp to add a mask per grammar state to bring it in line with your generation method; I don't think the two are incompatible, thankfully.

I do wonder how much you win here by masking the tokens? You still need to iterate along the output vector to apply the mask. Masking on the accelerator still requires filtering on the CPU side? Compared to running the language model, the cost of iterating over the edges in the grammar seems small.

popinman322··on LLaMA2 Chat 70B outperformed ChatGPT
Was this with or without fine-tuning?
popinman322··on GPT-4 API General Availability
I've done exactly this for another project. I'd recommend grabbing an open source model and fine-tuning on some augmented data in your domain. For example: I grabbed tech blog posts, turned each post into a collection of phonemes, reconstructed the phonemes into words, added filler words, and removed punctuation+capitalization.
popinman322··on Patterns of Distributed Systems (2022)
The correlation between IQ and job performance is typically weak[0] (weaker than the correlation between conscientiousness+agreeableness on job performance in some studies[1]) with a more modest correlation for "high complexity jobs".

Interesting excerpt from [0]:

> Finally, it seems that even the weak IQ-job performance correlations usually reported in the United States and Europe are not universal. For example, Byington and Felps (2010) found that IQ correlations with job performance are “substantially weaker” in other parts of the world, including China and the Middle East, where performances in school and work are more attributed to motivation and effort than cognitive ability.

[0]: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4557354/ [1]: https://www.academia.edu/download/50754745/The_Interactive_E...

Sorry about the PDF link to [1]. The APA link has a paywall otherwise I'd link there.

popinman322··on Alphabet selling Google Domains assets to Squarespace
Active Directory is actually pretty good I think. Having worked with both Azure and AWS, AzureAD makes AWS IAM permissions feel half baked.

Granted my experience was inside Azure so we had the benefit of vertical integration.

popinman322··on AI Is Catapulting Nvidia Toward the $1 Trillion Club
1xH100 is faster than 8xA100 for a work-in-progress architecture I'm iterating on. Meanwhile the code doesn't work on TPUs right now because it hangs before initialization. (This is with PyTorch for what it's worth) All that to say Nvidia's hardware work and software evangelization has really paid off-- CUDA just works™ and performance continues to increase.

TPUs are good hardware, but TPUs are not available outside of GCP. There's not as much of an incentive for other companies to build software around TPUs like there is with CUDA. The same is likely true of chips like Cerebras' wafer scale accelerators as well.

Nvidia's won a stable lead on their competition that's probably not going to disappear for the next 2-5 years and could compound over that time.

popinman322··on One million cancel broadband as living costs rise
> That rising inequality is leading to poverty is a failure of government, not corporations or capitalism. Why introduce a layer of indirection by blaming the rich or corporations, when the rules are made by government? Blame the government directly.

This implies that capitalism is not capable of self-regulation and therefore requires government intervention. It follows that this is a joint failure of capitalism and governments to reduce negative externalities that lead to high wealth inequality.

> Let reddit handle the economic ignorance.

Please don't include implicit ad hominem jabs at other posters. You could have omitted your first sentence with no loss to your argument.

popinman322··on LLMs are not greater than the sum of their parts: researchers
There was a discussion about the paper[0] this was based on 7 days ago: https://news.ycombinator.com/item?id=35768824

[0] https://arxiv.org/abs/2304.15004

popinman322··on Effective Immediately: A central hub for people who've been recently laid off (2020)
Tangentially related: A flat tech co-op model could be good. Build tools to make starting tech companies much easier, then let members start companies with those tools for either some regular payment in either labor (eg. allocate headcount for the co-op) or cash.

The co-op owns office space, (maybe) helps fund companies, provides vetted staff to reduce interview overhead, etc..

popinman322··on Beyond automatic differentiation
I also liked the graph that showed only training loss vs Adam. Would have been nice to see validation loss as well; are we just helping Adam overfit the data?
popinman322··on Tesla workers shared sensitive images recorded by customer cars
Both violate user privacy and imply a need for better data privacy protections.

One was targeted and the other was not, but they are in the same class of issues even if the severity is different.

popinman322··on Ask HN: Who is hiring? (April 2023)
As someone who's applied before, you really need to know C. The first phase of the process is essentially a code comprehension exercise followed by a simple programming exercise. I think they're looking primarily for candidates with kernel/wine programming experience.
popinman322··on Saying Goodbye to GitHub
> I googled "fair use copyright US" and have now decided that US copyright law is stupid.

Fair use helps artists, journalists, and (indirectly) the general public. It prevents censorship of critical or opposing views, among a lot of other uses that are beneficial to society (see sibling comment). US copyright law is backwards in a lot of ways but this isn't one of them.

popinman322··on Launch HN: Flower (YC W23) – Train AI models on distributed or sensitive data
I didn't read it as a move towards centralizing data, but instead as working with companies to federate over their userbase or between a collection of companies.
popinman322··on Zero-1-to-3: Zero-shot One Image to 3D Object
NeRFs are a form of inverse renderer; this paper uses Score Jacobian Chaining[0] instead. Model reconstruction from NeRFs is also an active area of research. Check out the "Model Reconstruction" section of Awesome NeRF[1].

From the SJC paper:

> We introduce a method that converts a pretrained 2D diffusion generative model on images into a 3D generative model of radiance fields, without requiring access to any 3D data. The key insight is to interpret diffusion models as function f with parameters θ, i.e., x = f (θ). Applying the chain rule through the Jacobian ∂x/∂θ converts a gradient on image x into a gradient on the parameter θ.

> Our method uses differentiable rendering to aggregate 2D image gradients over multiple viewpoints into a 3D asset gradient, and lifts a generative model from 2D to 3D. We parameterize a 3D asset θ as a radiance field stored on voxels and choose f to be the volume rendering function.

Interpretation: they take multiple input views, then optimize parameters (a voxel grid in this case) to a differentiable renderer (the volume rendering function for voxels) such that they can reproduce the input views.

[0]: https://pals.ttic.edu/p/score-jacobian-chaining [1]: https://github.com/awesome-NeRF/awesome-NeRF

popinman322··on Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
Is there something I'm missing? ChatLlama doesn't reference any human feedback datasets.

> You can of course also cross train it using actual ChatGPT.

You mean train it on ChatGPT's output? That's against OpenAI's terms of service.

popinman322··on Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
You're likely right that applying RLHF (+ fine-tuning with instructions) to LLaMA 7b would produce results similar to ChatGPT, but I think you're implying that that would be feasible today.

RLHF requires a large amount of human feedback data and IIRC there's no open data set for that right now.

popinman322··on Getting work done with PowerShell on Linux
> It's a huge minus. The fact that all shell tools produce text is a basic tenet of shell programming which underlies the composability and interoperability of them all.

PowerShell is essentially a cut down .NET environment with some nice syntax-sugar for typical shell activities. The objects in PowerShell are marshalled to/from text the same as in any other programming language.

So you can still write your tools as normal. Any PowerShell specific objects would just be a wrapper, or extra metadata. If we're being honest we already have that for other shells to make them more convenient: autocompletions. PowerShell objects give you autocompletions, type information, and interoperability with .NET libraries _but for data in addition to commands_.

The shell to programming language continuum has enough space to permit richer interpretations of the base data tools emit. We don't lose composability.

> With something like PowerShell - you need a debugger, data viewers etc.

Oh, what I would give for better shell script debugging. This is right up there with better CMake debugging for me.

← PreviousPage 2 of 6Next →