Arctic_fly

Interesting post. Did the difference in wall clock training time take the reduction in cold start time into account? Seems like that could be a significant factor for small jobs and negligible for large ones.

Does the IRS get to decide that? Seems like something that the a judge should rule on.

Seems relatively benign, seeing as he'll continue leading design on Apple Watch for some time. Also, I had no idea that COOs did design work.

I hover somewhere between AI doomer and boomer. For a long time I thought all the economic gains and power produced by AI would be concentrated in a company like OpenAI and that the quality of our lives would hinge on decisions made by its stakeholders. Fortunately, that future no longer appears inevitable.

This is an excellent question. My very unscientific suspicion is that the decreases in average attention span and ability to concentrate zero out the theoretical possible increases in productivity that computers allow.

Definitely not the case for coding. I'm a capable senior engineer, and I know many other very experienced senior engineers who are all benefitting immensely from AI, both in the code editor and chat interfaces.

My company just redid our landing page. It would probably have taken a decent developer two weeks to build it out. Using AI to create the initial drafts, it took two days.

New PII redaction library that's SOTA and runs locally on a variety of devices. Kind of slow when I tested on my M3 Max macbook, but works pretty well!

I don't believe they used an LLM to gather or transform information ahead of generating this map (used https://wikinearby.toolforge.org, a database of geotagged articles), but the ease of processing the necessary information to create something like this using AI is part of the reason the future could be great.

We've looked into this tool a couple times, haven't adopted it yet. The main issue is compatibility with prisma, which we use for defining our db's schema and automating migrations. If there were a slick integration between the two, that would go a long way to convincing us to adopt. In the meantime, we'll go with convenience and (slightly, almost immeasurably) lower reliability. We haven't had downtime due to a migration in months!

> What’s the benefit of generating embeddings for such large chunks?

Not an expert, but I believe now that we can fit more tokens into an LLM's context window, we can avoid a number of problems by providing additional context around any chunk of text that might be useful to the LLM. Solves the problem of misinterpretation of the important bit by the LLM.

> But in 2015 there is a stark discontinuity, where the number of stories (with text) shoots up by >10x, and the average score drops by 5x! Is this some kind of eternal September?

Based on the later analysis in the post (which I agree with), the total score of a comment is disproportionately tied to whether it hits the front page, and of course how long it stays there. Regardless of the quality of the average post starting in 2015, the sheer quantity would make it impossible for all but a few to stay on the front page for very long. Hacker News got more popular, so each story got less prime time.

I remember hearing about Dragon when I was in elementary school. It's cool to reflect on how far things have progressed in the last decade and a half.

Mastodon is a good example. Facebook could become less opaque by revealing their decision-making process for what posts are shown/suppressed, making analyses like the one linked unnecessary. It’s a bad sign when third parties are necessary to explain which content is shown and hidden on a platform.

> These are the realities of war.

The proportion of civilian casualties in Israel’s invasion of Gaza is not normal for the 21st century, and indicates a below-average regard for civilian life.

https://www.axios.com/2023/11/27/gaza-civilian-deaths-israel...

Perhaps the feminists in question believe that their voices are more likely to lead to a change in the stance of the US and the behavior of Israel than a change in the past behavior of Hamas. What Hamas _did_ was wrong. What Israel is _doing_ is wrong, and it’s probably only going to stop in the face of real external pressure from democracies, which have historically shown an interest in protests. I can’t speak for these feminists. But you certainly can’t either.

This argument is logical and I’m glad you laid it out. Stating obvious truths is a good strategy for revealing willfully ignorant posters.

Thanks for the source. Shakir’s perspective makes sense. Stanford shouldn’t be supporting Israel’s strikes on Palestinian civilians.

Thanks for the source.

If HRW has lost all credibility, then giving Meta the benefit of the doubt and ignoring this report makes sense. However, if you think there’s a chance that HRW might be telling the truth, then it’s worth examining your existing assumptions on how Meta decides what content to allow on its platform.

Fantastic, thanks.

I want to believe that the Russian economy is suffering negative effects as a result of their leaders’ decision to invade Ukraine, but the authors come off as overly politically motivated. I’m also surprised by how limited even the claimed effects are.

I do appreciate, and was previously unaware, that sectors such as food and health were not targeted by sanctions in an effort to avoid undue punishment of Russian innocents.

I find concerted efforts to silence political speech disturbing in general, but restricting reporting and commenting on the deaths of thousands of recognizably innocent people (children not old enough to participate in the conflict) is downright heartless. I do not want to live in a world in which opaque companies control public discourse.

Curse you for being interesting enough to make me get on my desktop.

It was bound to happen. I think we'll have to start determining the credibility of photographic evidence from the source that claims to have captured it. Then we have to hope there's enough backlash whenever a source lies that they stop doing it.

Same. I try to rent exclusively on the top-floor (next to impossible in a big city) because of a memorable experience I had with a an overweight upstairs neighbor who favored jumping jacks as his primary form of exercise.

Those devious academics probably inspired it

Very high-tech launch mechanism

Yep! The linked notebook includes an example of exactly that (fine-tuning a 7b model to match the syntax of GPT-4 function call responses): https://github.com/OpenPipe/OpenPipe/blob/main/examples/clas...

Currently OpenPipe allows you to capture input/output from a powerful model and use it to fine-tune a much smaller one, then offers you the option to host through OpenPipe or download it and host it elsewhere. Models hosted on OpenPipe enjoy a few benefits, like data drift detection and automatic reformatting of output to match the original model you trained against (think extraction "function call" responses from a purely textual Llama 2 response) through the sdk.

Longer-term, we'd love to expand the selection of base models to include specialized LLMs that are particularly good at a certain task, e.g. language translation, and let you train off of those as well. Providing a ton of specialized starting models will decrease the amount of training data you need, and increase the number of tasks at which fine-tuned models can excel.

I haven't had much trouble with GPT 3.5 or 4 function calls returning in an undesirable format recently. I did get a few bad syntax responses when OpenAI first rolled it out, but not for the past few months.

Llama 2 can also pick the function call format up, given sufficient training data that contains function call responses, though you'll then have to parse the returned object out of the text-based response.