HNHacker News
TopNewBestAskShowJobs

PheonixPharts

4,161 karma · joined April 7, 2022

submissionscomments
PheonixPharts··on Return to Office Is Bullshit and Everyone Knows It
> But not being in-person for collaboration/morale is miles apart.

I do agree that collaboration is much more productive when remote.

Nearly all of my exciting office "collaborations" end up feeling great then they're happening, but in retrospect, almost always fizzle out after a small window of time. I like onsites for the energy and socialization, but even then I always find most of the work happens back in the hotel room at night, and real planning happens after everyone is back at home. I can't imagine working that way all the time.

Whereas all of my remote collaborations are well documented while they're happening. Typically we have some kind of shared note taking, and writing code collaboratively, etc. Doesn't feel as social sometimes, but tends to be a much larger impact.

Likewise Open Source software has been largely written by remote people since long before the advent of even video conferencing. Git was originally written with the design intention that kernel hackers could work on a plane (per airplane wifi) without requiring a centralized server to communicate with.

> Video conf still sucks. It's literally the same as 10 years ago.

I still marvel at this. I've been working remote for nearly 15 years now and I honestly don't feel that video conferencing has improved noticeably.

PheonixPharts··on Fine, I'll run a regression analysis but it won't make you happy
> If Long Covid isn't real

I'm not arguing that long covid isn't real (in fact I explicitly state "To be clear, I'm not doubting that long-covid exists"), just that I'm not convinced that we are "we are so, so fucked" (to quote parent).

Your anecdata is appreciated since I share your lost faith in data about public health.

> prevalence of fatigue and impaired cognition from some other source

I personally think anxiety and unspoken stress regarding climate collapse and other forms of society degradation we're experiencing and seeing accelerate are responsible for a good deal of this. Everyone I know certainly seems more tired, more stressed out, and with fuzzier thinking, but I don't think that is a byproduct of long covid. People can't even openly talk about this seriously, even though nearly everyone is clearly feeling it. There are also plenty of people in aggressive denial about this which makes it even harder to discuss.

PheonixPharts··on Fine, I'll run a regression analysis but it won't make you happy
> None of this is over, and we are so, so fucked.

I love doom more than the average person, by what I would say is a quite significant margin... but I don't see this at all.

I've had covid at least twice, including some awful symptoms a month after recovery that had me convinced I had long covid as well... a few months later (now years) not so much.

While I do know many people of who have lost loved ones from Covid, I don't know anyone, or know anyone who knows anyone who has any serious "long-covid" symptoms.

To be clear, I'm not doubting that long-covid exists. But I'm very skeptical, based on my own observations, that "we are so, so fucked" and covid of all things is the source of that "fucking". Personally I think a lot of the post-covid strangeness is because, for a variety of other reasons (not the least of which is rapidly progressing climate change), people are under tremendous stress and at the same time can't quite articulate what that stress is, nor find relief from it.

A lot of people (not me) did believe there was going to be a return to normal post-covid, and the increasingly obvious impossibility of this is causing people to have tremendous mental health problems. But long-covid itself being the source is something that I just don't see anywhere.

And, as I said, I'm not afraid of being labeled a "doomer", so if you have some good sources to read up on, I'm all ears (er, eyes).

PheonixPharts··on Programming Language Inventor or Serial Killer? (2003)
I'm just glad the game isn't "File system creator or killer"
PheonixPharts··on Cannabis use disorder and subsequent risk of depression and bipolar disorder
No because you can never know for sure that there isn't a latent factor C which is the real causal factor and A is merely correlating with it (if we presume causality to exist at all, which is a pretty standard assumption).

We can compose DAGs that control for the factors we do know about, but it's impossible to exhaust or even know all possible latent processes that are impacting the outcome.

PheonixPharts··on Cannabis use disorder and subsequent risk of depression and bipolar disorder
It's important to recognize that we've never successfully answered Hume's general skepticism about the existence of causality as something that is real outside of our own minds. So causality can't be positively confirmed to exist at all, let alone be detected and confirmed through statistics.

Before you dismiss that is philosophical non-sense, causality is closely related to the "Arrow of Time" which is considered an unsolved problem in physics [0]. From what we've observed time appears to be the only asymmetric physical process, largely due to entropy (that is you can immediately tell if a video of a jar breaking is being played backwards because we don't expected a jar to "fall together"). There are Quantum processes, for example, that are time symmetric, that is playing a video of these processes would look the same reversed and forward.

That said, as someone who does a lot of statistics, in practice what we do is model the causal process with a directed acyclic graph and see how well our models behave under causal assumptions. These work okay for answering practical questions about does A cause B. By controlling for correlating variables we can see the impact on what we believe to be a causal variable when we condition on these other correlating factors.

Worth mention that nearly all of the work is done by linear models in practice.

0. https://en.wikipedia.org/wiki/Arrow_of_time

PheonixPharts··on Against LLM Maximalism
> why not use LLMs for dataset creation or augmentation in order to train a classic supervised model?

Or, as I mentioned in another comment, just use the embeddings directly. This also does a lot to remove the "cost (money and latency)" part of the problem since you can batch queries to be lightening fast and the dollar cost of generating the embeddings is effectively zero (~3000 pages of text per $1) for most traditional NLP tasks that require a vector representation.

PheonixPharts··on Against LLM Maximalism
I personally still think most people (not necessarily the author) miss out on the biggest improvement LLMs have to offer: powerful embeddings for text representation for text classification.

All of the prompting stuff is, of course, incredible, but the use of these models to create text embeddings of virtually any text document (from a sentence to a news paper article) allows for incredibly fast iteration on many traditional ML text classification problems.

Multiple times I've taken cases where I have ~1,000 text documents with labels, run them through ada-002, and stuck that in a logistic model and gotten wildly superior performance to anything I've tried in the past.

If you have an old NLP classification problem that you couldn't quite solve satisfactorily enough a few years ago, it's worth just mindlessly running it through the OpenAI embeddings API and sticking using those embeddings on your favorite off the shelf classifier.

Having done NLP work for many years, it is insane to me to consider how many countless hours I spent doing tricky feature engineering to try to squeeze the most information I could out of the limited text data available, to realize it can now be replaced with about 10 minutes of programming time and less than a dollar.

An even better improvement is the trivial ability to scale to real documents. It wasn't long ago that the best document models were just sums/averages of word embeddings.

PheonixPharts··on The Airbnb Bubble Popping Will Pop the Housing Bubble
I think the main difference is that, at least in the US, a lot of people that bought multiple homes in the last few years are fairly heavily leveraged. It's not "very affluent class owning the properties", but people that took out a another mortgage because they thought real estate was a sure bet.

The real issue in the US is going to come down to how many people can continue to afford multiple homes which is where rising interest rates and inflation both come into play. Anyone that bought a second home using an ARM may be forced into paying much more than they thought, and rising interest rates are going to put additional pressure on people with investment properties as they may need more liquidity sooner than originally expected to cover routine expenses.

PheonixPharts··on Extreme El Niño weather switched off South American's carbon sink
I think it's far more far to say the opposite: science is an ever evolving model of the world, but science is always just a model.

F = ma is a model.

And you may try to limit your argument to only "statistical models = not science" but then you would be dismissing essentially all social sciences but also ecology, geology, climate science, not to mention every single aspect of statistical mechanics.

Science is essentially hypothesis testing where hypothesis represent models of how the world works. Didn't you even take high school physics where you spend considerable time running experiments and writing up why they don't exactly match the formula's prediction in the book?

Would you mind explaining what model-free science looks like? Is it just cataloging observations?

PheonixPharts··on I built a plane spotter for my son in minutes
Anyone remember the unlimited number of "build Twitter/blog/todo list in 15 minutes using Ruby on Rails" tutorials?

It's great that we've got it down to 2 minutes.

However anyone who has worked in a rails shop has experienced that somehow that initial set up seems to make everything else down the line get a bit more time consuming.

In fact, I would posit that there might be an inverse correlation between how quickly you can get a blog up using framework X (including an LLM) and the long term scalability of that project as you start adding features and engineers.

Legacy rails apps can be terrifying, I can only image the horrors that will be legacy LLM apps if they can even manage the structural cohesion to become legacy apps.

PheonixPharts··on Vector Search with OpenAI Embeddings: Lucene Is All You Need
You should write a paper: "all you need considered harmful"

I don't think referencing well known papers has ever (well maybe not, ever) implied that the authors feel their work is on par with original. It's a pretty common practice in some academic circles when there is an impactful paper with a catchy name and you simply want to pay a bit homage and have a less boring title than you might have otherwise.

PheonixPharts··on Vector Search with OpenAI Embeddings: Lucene Is All You Need
I'm guessing they choose to use OpenAI embeddings since this is the dominant use case for most people.

The embeddings are just the "data" that's in the database. Swapping out getting embeddings from OpenAI with Llama is as trivial as putting information about your own customers in your own database as opposed to using info on someone else's customers.

PheonixPharts··on Fine-tuning GPT-3.5-turbo for natural language to SQL
I think you have fallen into the trap of mistaking interpolation for generalization.

Working with these models every day, it's clear that they can certainly interpolate between points in latent space and generate sensible answers to unseen questions, but it's pretty clear that they don't generalize. I've seen far to many examples of models failing to display any sense of generalization to believe otherwise.

That's not to say that interpolation in a rich latent space of text isn't very useful. But it's not the same level of abstraction that comes from true generalization in this space.

PheonixPharts··on Python Type Hints – *args and **kwargs (2021)
> anything more than half a dozen modules/classes can turn into a bloody minefield very fast

I don't disagree with this, but it's worth noting that there are many data scientists I've worked with who have never written a python class or module, and yet produced large amounts of valuable work.

For quick prototyping and exploratory work, both domains where Python sees a lot of success, it's often the case that you don't really know what types you're working with and are iterating very quickly, such that not being able to quickly change all of the types your working with can be a time sink.

So I think you're imagining writing software that is starts more well defined than most optimal use cases for Python.

> And the supposed speed of iteration is negated and reversed by having to dig through the untyped codebase trying to very inefficiently determine the types manually

I do understand this feeling, but this is applying the logic/patterns of statically typed programming dynamic programming. The dynamic answer to "how do I write robust code" is not to keep the types in your head, it's write tests. Well written tests in turn become documentation for your code.

Again, this style of programming works well when you don't really even know how you want your programming to behave. You naturally write tests when writing this kind of code, even if it's just manual tests. Get in the habit of starting your manual tests as unit tests and you're already doing TDD.

Dynamic types works best when you are writing code in a very interactive and exploratory way.

To be clear: I'm not advocating for dynamic languages over statically typed ones. I do believe, whenever possible, production software should be written in strongly typed languages. I think anytime you know the behavior of your program before you start writing you should use statically typed languages.

PheonixPharts··on Python Type Hints – *args and **kwargs (2021)
"my current project" type problems are where Python is great. Types remain "nice to have" (if you like them) and aren't really essential compared to the ease of prototyping new ideas and building a PoC. You're choosing Python because the benefit of libraries outweighs your personal preference for types.

Most of my work is in machine learning/numeric computing, so I'm very familiar with the benefits of Python's ecosystem. Basically all of AI/ML work is about prototyping ideas rapidly, where access to libraries and iterating fast greatly trumps the need for type safety.

At nearly every place I've worked, Python is the tool for building models quickly but shipping them to production and integrating them with the core product almost always involves another language, typically with types, better suited for large engineering teams working on a large code base where you really want some sort of type checking in place. Most of the companies I know that do serious ML in production typically take models from python and then implement them in either C++ or Scala for the actual production serving.

It's worth pointing out that the vast majority of those libraries you use were initially developed without any consideration, or need, for types. Great, reliable, software can be written without types. Dynamic typing is a great language choice, and there's no need to fight the language itself by trying to bolt types on.

Where types are important is where you have a complex, rapidly changing code base with a large number of developers of differing skill levels releasing frequently. If that's the environment you're in, I would strongly recommend against using Python in prod, even if it means you have to implement the features of some libraries internally.

PheonixPharts··on Python Type Hints – *args and **kwargs (2021)
As someone who has written Python for nearly 20 years now, and also has plenty of experience with strongly and statically typed languages (including a fair bit of Haskell), I think type hints in Python should at most remain just that, hints.

A language being statically typed or dynamically typed is a design decision with implications for what the language can do. There are benefits to each method of programming.

Trying to strap type checking on to Python is born out of some misplaced belief that static type is just better. Using Python as a dynamically typed language allows you for certain programming patterns that cannot be done in a statically typed language. There are some great examples in SICP of Scheme programs that could not exist (at least with as much elegance) in a typed language. Dynamic typing allows a language to do things that you can't do (as easily/elegantly) in a statically typed language.

Some may argue that these type of programming patterns are bad for production systems. For most of these arguments I strongly agree. But that means for those systems Python is probably a poor choice. I also think metaprogramming is very powerful, but also a real potential footgun. It would be ridiculous to attempt to strip metaprogramming from Ruby to make it "better", just use a language that depends less on metaprogramming if you don't like it.

This is extra frustrating because in the last decade we've seen the options for well designed, statically typed languages explode. It's no longer Python vs Java/C++. TypeScript and Go exist, have great support and are well suited for most of the domains that Python is. Want types? Use those languages.

PheonixPharts··on Nvidia announces financial results for second quarter fiscal 2024
> This is what having a monopoly looks like !

As someone who has been in the AI/ML space for over a decade, and even had an AMD/Radeon card for more than half of that, I can't help but feel that this is partially AMD's own fault.

For many, many years it seemed to me that AMD just didn't take AI/ML seriously whereas, for all it's faults, NVIDIA seemed to catch on very early that ML presented a tremendous potential market.

To this day getting things like Stable Diffusion to run on an AMD card requires extra work. At least from my perspective it seems like dedicating a few engineers to getting ROCm working on all major OSes with all major scientific computing/deep learning libraries would have been a pretty good investment.

Is there some context I'm missing for why AMD never caught up in this space?

PheonixPharts··on Why did people in the past look so much older?
I think this has it backwards: Why are adults today more like children?

Adults, including myself to some extent if I'm honest, in multiple ways act much more like children than any other age in history. This is a very recent phenomena, and I'm continually shocked by how unquestioned it is by most "adults" I know.

My suspicions is that our culture of neoteny is a byproduct of late capitalism. The ideal consumer is a mental and emotional toddler with the income of an adult professional: constantly wanting a new shiny toy or tasty treat to occupy their time, without the need for parental supervision to limit their ability to mean those desires.

When I look around I increasingly see the definition of "adult" being pushed back further and further in age (in another post people are going on about the myth that "mental maturity" starts at 25!)."Adults" that refuse to outgrow the wants of a child leads us to a world where we have dessert with every meal and Christmas is everyday (Amazon makes that possible). It also means a world were individuals refuse to take any responsibility for the reality they participate in.

PheonixPharts··on We need scientific dissidents
> The incentive structures in science encourage novelty

Former academic here. This may be superficially true, but it's always been "novelty within the established orthodoxy". Kuhn documents this quite well, but it's become even more true since the funding motivated push for peer review that started in the 1970s.

Academic publishing today strongly demands conformity for survival. Geoffrey Hinton has a great quote, that I can't seem to dig up, about having any truly ground breaking work dismissed by juniors who have no idea what you're talking about being a major problem in moving science forward.

The bigger issue is that we have very little space for actual dissidenters. For example there is lots to legitimately question about mainstream medicine, but it's hard to walk too far down that path without immediately getting thrown in with "vaccines cause autism!!!".

PheonixPharts··on Was Y Combinator worth it?
Funnily enough you can't actually implement the proper y-combinator in Haskell because the type system won't allow it. Which is a real shame because in most Lisps you can only implement the applicative order y-combinator due to a lack of lazy evaluation.
PheonixPharts··on To opt out of Grammarly AI training requires business account with 500+ users
Kinda chisels away at their official "We're not a keylogger!" argument:

> Is Grammarly a keylogger?

> No. A keylogger records every keystroke, sends data to a third party for the benefit of that party, and does so without the user’s knowledge. Grammarly’s product doesn’t fit any of these descriptions. [0].

Certainly the data being sent must be primarily for their own benefit. Clearly if it was for the benefit of the user then they would have not need to make turning it off a premium feature. Maybe Grammarly will create an AI that makes their product better, but they're still using user data to make that product in the first place.

[0]. https://support.grammarly.com/hc/en-us/articles/360003816032...

PheonixPharts··on Ocean Heat Record Broken
> only begins sometime in the 1980s is very poor

The observation that "we only have N years of data so we don't know anything!" creeps up frequently in climate denial arguments.

But I am sincerely curious, do you earnestly believe that, ignoring the other methods of estimating these values we have, that if we had satellite measurements going back 100s of years they would reveal that this level of heating is in fact quite common?

Of course we do have data that suggests, at least as far back as 1880 [0] that we are in an era of record sea surface temperatures [0]. But even if we didn't, we have countless other observations that strongly suggest the planet is heating up in multiple ways. We also have a fairly good hypothesis as well that nicely explains all of these observations (CO2 released by human industrialization is capturing more solar heat, causing the planet to warm). Yes, when we try to get too granular with our predictions the models aren't as great, but overall we have a nice hypothesis that seems to explain a wide range of observed phenomena over a large range of time scales for different data sets.

When you take into account all of this information, while it is important to always remain open to skepticism, it seems remarkably unlikely that what we're observing is in fact common in recent human history and therefore not particularly worth of note.

0. https://www.epa.gov/climate-indicators/climate-change-indica...

PheonixPharts··on Tor’s shadowy reputation will only end if we all use it
> but I don't want to get anywhere near illegal stuff.

Then you should probably stop using the web altogether.

I'm seriously confused how using Tor places you closer to "illegal stuff" then browsing as you do? Could you clarify?

Even if we're going to draw the distinction between .onion sites and the plain web (and there's nothing about Tor that requires you to visit or interact with .onion sites) I'm nearly certain that there are many orders of magnitude more "illegal stuff" being shared on traditional websites than the "dark web". Plenty of drugs are purchased through Venmo, and Tumblr and Twitter have had pretty high incidents of child exploitation materials being shared on there (and, despite having been a heavy user of those sites at some point, never came across any content close to that).

My experience has been that, barring 4chan 10 years ago, it is extremely rare that you'll ever come across any "illegal stuff" unless you are looking for it.

PheonixPharts··on Shopify employee breaks NDA to reveal firm replacing laid off workers with AI
Personally I find the fact that Keynes's 15-hour work week never materialized despite massive improvements in automation to be fairly strong evidence in favor of Marx's Labor Theory of Value.

One of the ways that Marx posited capitalist extract surplus value is to have the increase in quality of life always lagging the increase in efficiency. Our quality of life does increase, but only if capitalists are able to increase greater surplus value from our labor than that quality of life increase accounts for.

> don't forget that we're going to take down the world economy with it

Interestingly enough this is one of the major focuses of Capital vol III: the tendency of the rate of profit to fall.

Marx argued that eventually the Capitalist pressure to increase efficiency through automation means that while production continues to be more efficient, precisely because of this the ratio of profit to capital expended decreases. For Marx this was the ultimate threat to Capitalism itself.

PheonixPharts··on Bayesian methods to provide probablistic solution for the Drake equation (2019)
> but does that mean xenoarchaeology

My favorite "crazy theory" is that the Paleocene–Eocene Thermal Maximum (PETM) was caused by a previous industrial civilization that, like us, found a bunch of easy to access hydrocarbons, naively dug them up and reached the same climate change problem that we did, ultimately wiping them out.

Of course there is 0 evidence for this at all, but it was also more than 50 million years ago. If you look into it basically nothing about an advanced industrial civilization would likely survive to make it into the geological record for us to observe. The only reason to even entertain this hypothesis is that the PETM also experience a very rapid increase in atmospheric CO2 and we aren't entirely sure why.

Unfortunately this means that there's probably not much chance for a rich field of xenoarchaeology to exist since it's not even possible to do this on our own planet.

What is an interesting thought experiment is: Suppose we realized we as a civilization were doomed and wanted to sent a message to future industrial civilization on Earth warning them about being too aggressive with hydrocarbon usage. To my knowledge there is no known method to ensure a message could be sent that far in the future, but it's fun to try to think of ways we could send a message to a future, essentially, alien civilization here on Earth.

PheonixPharts··on A glitch in the SEO matrix
This is strangely where I've found the most personal benefit of LLMs. As long as it's not too current or depend on recent information (like the weather), searching for information in LLMs reminds me a lot of Google 10 years ago.

Yea sometimes LLMs hallucinate, but sometimes websites are just wrong. For a variety of topics LLMs are just a better form of search.

PheonixPharts··on A stationary jet stream is fueling intense heat worldwide
You can take it further and even argue that complex life is ultimately just a tool for increasing entropy (which the universe seems to enjoy doing).

Digging that oil up from the ground and the burning it does a tremendous amount to free up all that energy stored in a low-entropy system.

The book Into the Cool makes an interesting case regarding this overall concept.

PheonixPharts··on Ask HN: How are you improving your use of LLMs in production?
After an initial first rush of getting features to prod, at this point I would say we're doing "Evaluation Driven Development". That is new features are built by building out the evaluations of those results as the starting point.

At least from the people I've talked with, how important evaluations/tests are to your team seems to be the major differentiator between the people rushing out hot garbage and those seriously building products in this space.

More specially: we're running hundreds/thousands of evaluations on a wide range of scenarios for prompts we're using. We can very quickly see if there are regressions, and have a small team of people working on improving this functionality.

PheonixPharts··on A stationary jet stream is fueling intense heat worldwide
The problem with climate change, in particular global warming (i.e. adding energy to the system), is that it increases the variance of the system.

This means that the system becomes harder to predict, and we're not looking at a mere "redistribution" of weather. It's very possible for weather to get worse (i.e. disruptive to the current functioning of the local ecosystem) everywhere

I would not at all be surprised if some areas suffering from extreme heat right now become unlivably cold in the coming decades.

The problem with climate change is that it does not, in a human life time, lead to a new normal. If these new weather patterns were here to stay, we could adapt. what we'll get instead is constant changing which becomes increasingly resource intensive to survive.

← PreviousPage 7 of 15Next →