celestialcheese

At least with the native tribes in Alaska, many of their land holdings are managed under corporations[1], so I think it's fair to consider it land ownership rather than just "nations", although there are much more restrictions to the shares of these corporations than what is typical. [2]

1 - https://www.ncai.org/tribal-directory/alaska-native-corporat... 2 - https://www.propublica.org/article/what-are-alaska-native-co...

I don't get this. I find "passive" advertising like banners, mid-roll, etc so much easier to ignore/block/

The type of advertising that's invasive for me is the "sponsored content" or paid product placements, especially in movies. It feels like every big-budget movie I see these days is littered with "ads" for some car or electronics company. There's no way to block this, and sometimes its hard to tell if something is paid or just someones honest recommendation.

Alcoa had a pretty large aluminum smelter on the west coast in Ferndale WA, but it shut down near the beginning of covid. The bonneville power contracts with Alcoa were pretty generous, somewhere around $0.035/kWh for ~300MW, and even then, they couldn't make it work.

A PE firm is trying to buy it up, but the bonneville administration isn't playing ball and giving them the same rates, so it'll likely sit empty.

I use it for filtering and summarization tasks on huge contexts. Specifically for extracting data from raw HTML in scraping tasks. It works surprisingly well.

Usually those types of display ads show up when targeting isn't possible, or your anonymized such that the CPMs for your session are low enough that it's cheap enough for these types of ads to make money.

There are really only 3 buyers of banner/display ads when no targeting is available:

1) Huge multinationals - think geico, coke, etc. Broad awareness campaigns

2) Scams, body horror, YMYL, click-trick, etc. This is what you're seeing. Info products, garbage cosmetics and scam courses have such huge margins that they can spend on broad-targeted placements for very cheap CPMs, with aggressive creatives that are A/B tested to death to get people to click. As long as the CPM is cheap, and the CTRs are high enough, the numbers work.

3) "Inexperienced" ad buyers

I'm not commenting or interested in debating the ethics of tracking and the internet data economy, but a side effect of removing 3rd party targeting is that as attribution goes away, buyers of ads pay less since they can't attribute dollars to campaigns, so you have to move to attribution methods more common in radio and TV. And as costs fall, it gives opportunities to the slimey shit you've been seeing.

This. I'll still use langchain for token/cost counting and some nice abstractions on top of the LLM, and the document loader system is semi-useful, but all of the retriever/chain stuff abstracts away the most important part - the prompt.

The problem that these bills are trying to solve isn't just consumer, it's End of Life management at the recycler level. Glued-in, Li-ion batteries over the years have caused so much headache for ewaste processors. Apple has improved over the years with their glue tabs, but it's still a big problem in the market as a whole.

https://www.popsci.com/energy/lithium-ion-batteries-recyclin... https://www.waste360.com/business/li-ion-battery-fires-unfai...

It's pretty impressive - but too aligned to be anything more than a customer service bot. It just feels like a corporate drone that continuously is friendly and upbeat. Which, maybe that's their goal.

Those service links are intermittent - getting 504's on S3 console

My site is down as well - us-east-1

Civitai is a good place to get these models if you don't want to train your own. NSFW partially

While the APIs make it significantly easier to ingest for LLM training, scraping will still work.

Unless they put 100% of content behind a login-gate, it's currently legal* to scrape and use for derivative works, as long as you have the money and the chutzpah to deal with lawsuits that may or may not come.

- https://blog.ericgoldman.org/archives/2022/12/hello-youve-be...

> “I don’t have those documents” is an acceptable response to a subpoena, so it’s not like he has to come up with stuff he doesn’t already have.

Speaking from experience, better be sure you word that response right, or you might accidentally give them reason to push harder/further. You need a lawyer to fight against good lawyers.

> Molson also has rights. The discovery process isn’t supposed to be used to overwhelm or harass third parties, so he should move to quash if he really is being overwhelmed

While you're right, the process of even doing this will cost you a minimum of $10k USD if you're in a wealthy area in the coastal US. Which for a corporation is nothing, you likely have in-house counsel you're already paying for that can do this legal boilerplate.

But as an individual it's terrifying and incredibly stressful.

GPT-4 does a much better job at paying attention to details in prompts

> [1] Is a summary of [2], by one of its authors, not a separate source.

Yes. Your point? I included both because I found them both interesting. The paper is the source, the 137 emergent behaviours page is one of the authors continuing the work, and [3] is a journalist talking about this, so I included it as it's a unique perspective.

I used the word "emergent" because that's what the SME used when describing this. From 5.1 in the paper linked:

> Although there are dozens of examples of emergent abilities, there are currently few compelling explanations for why such abilities emerge in the way they do.

You say this "should be surprising to no one", yet the authors disagree.

Additionally, in the GPT-4 system card - "Emergent" appears 15 times, specificly section 2.9 is interesting https://cdn.openai.com/papers/gpt-4-system-card.pdf So it's not just a word used callously by one group of researchers at Google.

Yet there are emergent behaviours from these LLMs that are both surprising and not immediately understood. [1][2][3] Everyone has theories, of course, but still pretty "magic" considering these behaviours weren't theorised in papers prior to observation.

1 - https://www.jasonwei.net/blog/emergence 2 - https://arxiv.org/pdf/2206.07682.pdf 3 - https://www.quantamagazine.org/the-unpredictable-abilities-e...

This tutorial is a good overview of the rough systems behind most of this "Chat your data" application you're seeing now.

https://www.pinecone.io/learn/langchain-retrieval-augmentati...

Langchain / Llama indexes are both toolboxes that abstract away a lot of the plumbing for doing this kind of thing, and Pinecone is one of dozens of vector databases.

Personally, i'd try out langchain and chromadb and go through some of the examples langchain has in their docs, then be prepared to completely abandon langchain and just work with the LLM APIs directly. Start with openai, get on the waitlist for GPT-4 tokens, and also get on Anthropics Claude waitlist for the 100k-1.3. It's _very_ good for knowledge retrieval.

Langchain tries to do too much in extracting away the prompts, and the prompts are really what matter in getting interesting stuff out of your own data. Use langchain, llamaindex pieces but build from scratch for most things as your tinkering.

It's really not hard if you have a background in programming, and it's _so_ much fun. You'll feel like you have superpowers once you get a scraping interface hooked into an LLM. All of a sudden you can automate some really complex pipelines very quickly

Simple things that don't seem impressive to you, but are impressive to others, may be because you haven't experienced the first hand difficulty of doing that thing pre-transformers/GPT.

For example, I get _unbelievably_ excited with knowledge extraction and question answering demos on PDFs. Why? Because i've built similar systems for over a decade and know how difficult it is to build on top of messy archival data. Now, with very little code, i'm getting SOTA results.

If you didn't have experience with this, you'd probably thing "Huh, that's not impressive, XYZ does this already". But it's the moving of the baseline that's what's really impressive.

===

AI hype-beasts aside of course - the breathless pontificating of "influencers" and former crypto bros is cringe.

US advertising CPMs for this type of placement will be 100x more valuable than Southeast Asia and 50x LATAM. The economics don’t work anywhere else. Even Europe

If you’ve never built PDF or archive document parsing systems, you don’t know true pain.

I see it as incredible. Most PDFs that i see are basically just thin wrappers around image scans of documents that don’t exist anywhere anymore. Archives from estates, manuals, etc.

These techniques of using LLMs to clean ocr output is game changing because best in class before was human-in-the-loop systems that required huge amounts of rewriting to get useable output.

Now LLMs are unlocking for significantly cheaper previously difficult data sources for relatively cheap.

~$0.45

I don't think this will work, because it seems like the average person prefers being lied too, or put another way, people prefer to live in blissful ignorance.

A $300 TV gives you the illusion of privacy, sure you see some ads, but you always have seen ads with your paid cable subscription, so what's the difference.

With this, a free TV immediately makes me skeptical and defensive. "Nothing in life is free". Plus, Telly is doing a hostage situation - Here's the $300 tv for free, but if you fuck with anything, we'll charge you $2k (or whatever it is).

I don't see this going well for Telly.

exactly. Just sent raw tesseract output, no formatting or "fix the OCR text" step. So the data looked like:

``` col1col2col3\nrow label\tdatapoint1\tdatapoint2... ``` Very messy.

I don't think this is generalizable with the same 100% accuracy across any OCR output (they can be _really_ bad). I'm still planning on doing a first pass with a better Table OCR system like Textract, DocumentAI, PaddPaddle Table, etc which should improve accuracy.

Claude 100k 1.3 blew me away.

Giving it a task of extracting a specific column of information, using just the table header column text, from a table inside a PDF, with text extracted using tesseract, no extra layers on top. (for those that haven't tried extracting tables with OCR, it's a non-trivial problem, and the output is a mess)

> 40k tokens in context, it performed at extracting the data, at 100% accuracy.

Changing the prompt to target a different column from the same table, worked perfectly as well. Changing a character in the table in the OCR context to test if it was somehow hallucinating, also accurately extracted the new data.

One of those "Jaw to the floor" moments for me.

Did the same task in GPT-4 (just limiting the context window to just 8k tokens), and it worked, but at ~4x more expensive, and without being able to feed it the whole document.

I couldn't disagree more.

The majority of my usage, and the usage of _many_ others based on the view count for DIY stuff, is watching people fix or build stuff. Usually because I'm in the middle of a project trying to figure out how to do something, or because I'm thinking about a project and wanting to get an idea of how difficult it'll be. Youtube University basically.

I pay for premium because it saves so much time, and I hate ads when I'm under a car trying to remember what bolt goes where.

Also, from an economic model, Stratechery has a great article last week discussing this - worth a read. https://stratechery.com/2023/the-unified-content-business-mo...

Hola VPN, one of the largest free ones, is operated by BrightData. They make money with their residential and mobile proxy service. By using Hola, you act as an edge node in what is essentially a legal botnet, and Bright Data charges like $10/gb to clients who want to use it for scraping,etc.

Even worse - john deere uses open source code, and doesn't honor its licenses.

https://sfconservancy.org/blog/2023/mar/16/john-deere-gpl-vi...

> Where needed you can make models deterministic by setting temperature to 0 with a fixed model version

This isn't guaranteed - you'll still get differing responses to the same input with 0 temp. Here's one explanation [1]

Likewise, for certain tasks, the magic of the LLMs doesn't come through unless temp > 0. More specifically, with text cleanup tasks from OCR output, if temp=0, i've found that GPT-3.5/4 doesn't do as good of a job fixing really broken output.

That being said, you can mitigate this with proper evals and good old fashioned output validation.

1 - https://twitter.com/goodside/status/1608525976702525440?lang...

This is the best summary of the current state of scraping laws - https://blog.ericgoldman.org/archives/2022/12/hello-youve-be...

It's complicated is the short answer.