throwaway9274

We still don’t know what LLMs are for.

By this I don’t mean LLMs aren’t useful. We have known that there was novel behavior happening as early as GPT-2.

GPT-3 represented a clearly novel transformative technology. But we still didn’t know what to do with it.

The reason for this is that the people who come up with products are generally a different set of people from those who innovate novel ML models.

The latter tend to be PhDs or the extremely mathematically talented.

The former tend to be a mix of product-y software engineers & engineer-y product & strategy people.

There were some products, GitHub Copilot being the breakaway. Knowledge needs time to diffuse, and a market demand is necessary to catalyze that quickly.

Out of exasperation, OpenAI decided to take one of the most common prompting use cases on the GPT-3 beta playground, Q&A, and make a chat product, almost as a technology demonstrator.

Then ChatGPT exploded to hundreds of millions of users.

All hell broke loose. Every Fortune 500 promised a “generative AI” rollout. And of course, like feverish corporate-branded “metaverse” stuff it all sucks.

You can’t “corporate partnership” and “internally accelerate” a technological shift of this magnitude. You can’t hire BCG to do it for you. And you can’t tack a model onto your existing product and call it done.

LLMs and other new foundation models require fundamentally new products. They require new middleware to be deployed like vector databases, RAG frameworks, and agentic systems.

Start ups are starting to crack these problems.

But my fear is that when the bottom drops out of the corporate efforts, the investment attitudes will shift just as there’s the most work to be done.

Maybe, depends on the threshold for human creativity, which is still in flux.

You could copyright the underlying character maybe. But its a major headache for these companies and a symptom of the broader fact that copyright needs legislative reform, not attempting to shoehorn AI into the current law.

Everybody’s too concerned about copyright for training data, no one’s concerned enough that the outputs are considered non-copyrightable.

$50mm pre-product valuations for startups producing 3D assets for games via prompting a foundation model.

Zero due diligence on the U.S. copyright office’s stance that every asset those companies generate will be open to reuse.

Early depictions of AI doom are fascinating because the Christian origins of the myth are not yet well-concealed.

In the same way that Kurzweil portrays the Heaven of the singularity, I Have No Mouth, and I Must Scream is an excellent Dante-esque exploration of the concept of Hell.

These unconscious cultural archetypes can be both illuminating and distorting, and I wonder how the relative literary diets of Gen Z, Millenials, Gen X, and the Boomers affect their thinking on AI risks.

Here’s what’s going on:

-Companies that operate internationally make revenue and incur costs in many different countries.

-As a result, they owe taxes to many different national authorities.

-Each national authority has rules for how costs and revenues are accounted across borders.

-These rules are necessarily complex, because it’s often not clear how costs and asset transfers should be accounted from simple first principles.

-This is particularly important when accounting costs across countries, because taxable income is often based on your costs.

-For example in the US, Federal taxable income equals gross income MINUS the cost of goods sold.

-In a multinational, one critical question in figuring out costs is how to value transfer prices of goods, intangibles, and services among enterprises under common ownership.

-That is, if the U.S. division of company A and a division of company A in another country exchange assets or services, the question of how they price those goods to one another becomes important for tax reasons.

-Typically there’s a lot of accountants with spreadsheets or a software system where all this is calculated, monitored, invoiced, booked, and reconciled.

-Accounting and services for this is typically called “transfer pricing.”

-To determine how to account for these costs, each national regulator has very specific rules.

-The question is whether Microsoft adhered to these rules in how it accounted for costs and revenues between its U.S. and international entities.

-These disputes go through a very long back and forth process that often culminates in litigation, which itself can last years, or settlement.

-Microsoft says, “Because our subsidiaries shared in the costs of developing certain intellectual property, under those IRS cost-sharing regulations, the subsidiaries were also entitled to the related profits.”

-So it looks like the dispute centers around how to account for “the costs of developing intellectual property”, which can be hard if you’re developing software globally.

-Finally, there’s one aspect to consider when you see these $XXB figures. At the size of certain very big corporations like Microsoft or Apple, money does not behave like it does for you, or me, or even Sequoia Capital.

-Microsoft’s market cap is $2.4T. At that scale, money enters a kind of different state of matter.

-Money’s purpose is primarily to coordinate economic activity, especially when you’re talking about non-negligible amounts for multinationals.

-So payments among the revenue agencies of various countries and their very large private multinationals are in kind of a closed loop where they mainly have complex macroeconomic effects.

-For example, the effect of increasing taxes via heightened transfer pricing scrutiny mostly moves production from one sector to another (e.g. from consumer tech to health and defense.)

-The policy and finance folks pulling the levers understand how that works, and often have their own complex set of motives.

35%, oops.

The focus on robotics research starts to make more sense when you dig into the financials.

Disney’s FY2022 operating income was $12.7B. $7.9B (!) of that comes from theme parks. That’s 65%.

Everything else Disney does (Pixar movies, Disney movies, Star Wars movies, television, streaming, touring musicals, merchandise, partnerships, IP licensing) is the remaining 45%.

Robotics is essential to the parks.

Happy to do the research for you. My consulting rate is $500/hr.

“It looks gpt-generated.”

See, this is an example of a contentless ad hominem.

Believability weighting is when you say, “Here’s proof this argument is bad. The community from which this argument originates makes these types of bad arguments frequently. Therefore we should believe that community less often.”

We are the product of the cultures with which we choose to engage.

Some communities have positive signal. Others have negative signal.

I respectfully disagree.

The portion regarding the norms in EA communities argumentation habits is the most weighty part of the analysis. The EMH example is a supporting detail.

The thesis is this: EA communities are prone to insularity, hubris, and fabulism covered with a paper thin veneer of logic. The insularity generally means their arguments have not been subjected to even trivial scrutiny from non-EA communities. As a result, their arguments should as a first approximation be given no more weight than the average Reddit thread.

To be logically complete it would need to collect a variety of examples of hubris, fabulism, and insularity from EA arguments, which is left as an exercise to the reader.

Rhetorical fallacies are important to understand if you’re going to call them out. “Ad hominem” means attacking the argument by attacking the person.

Here I am attacking the general credibility of a community of people based on the example of one of their members’ arguments.

This is common in communities that are devoted to radical honesty and radical transparency, particularly those that actually have to hold these norms in high stakes market scenarios. It is called “believability weighting.”

For example, if the Norway Government Pension Fund historically provides poor logical arguments in their analysis of EU recessions, their arguments about EU recessions should be given less weight in the future.

Believability weighting is not the same as ad hominem.

There is a particular flavor of writing common to those whose identity is bound up with the so-called “effective altruist” community and their various online forums.

It usually combines a reference to Bayes Theorem, a deep hubris, and a significantly misunderstood concept from mathematics or technology.

Here, for example, it is clear the author does not grok EMH which applies across the population of trades not traders.

Usually this is done to provide the author or community with some claim to an inherent and unearned extraordinary nature, which justifies greater personal power. (You only need 200:1 evidence you’re extraordinary being the takeaway here.)

It is unclear why certain SV cliques continue to grant these people power.

Their intellectual insularity means that whatever value they once added in bringing logical rigor to conversations is now far outweighed by the train of collective bad logic powering their groupthink.

It is increasingly dangerous to have such people steer important societal debates.

These debates include crypto regulation where EAs famously associated with SBF, to “AI safety”, a field that appears to be invented out of whole cloth to allow LessWrongers to LARP as ML ethics and fairness experts.

Definitely one of the benefits of local models.

You could theoretically auto-suppress any tokens that lead to a refusal to answer path and re-generate.

But you’re probably best off grabbing an existing uncensored chat SFT, like the Llama v2 variants trained on Hermes. https://huggingface.co/NousResearch/Nous-Hermes-Llama2-70b

There is no way to create a censored model without losing performance.

Techniques for PPO and RLHF (the technical basis for the censorship mechanisms) inevitably destroy some parts of the model’s ability to “reason” (as measured by benchmarks) and accurately gauge relative probabilities of truthfulness.

Getting uncensored base models to the public is really what’s driving research forward on LLM innovations (except scale) at this point.

Yes. Meta deliberately let their responsible AI people heavily censor the chat-tuned model.

Meta also released the uncensored base model, on which the open source community then performed its own chat fine tunes. This was a canny strategy to avoid negative press.

Mistral saw Meta’s approach, and instead chose to deliberately court the negative press, because attention is more valuable to them as a startup than opprobrium is damaging.

Sure. But more people is good. More traffic is bad.

Absolutely right. To situate in terms of recent events, there have been three stages of trying to delay / influence the California CPUC robotaxi expansion approval vote:

1. Tried a politician scare campaign. Failed because it was too transparent it was an attempted distraction from their terrible public safety record.

2. Tried coning and disabling the vehicles. Failed because it was too transparently astroturfed, legal liability.

3. Now attempting leaks of cherry-picked vanity metrics so massaged they can’t fairly be called “statistics” to friendly anti-tech local media.

Will fail because national and international media no longer trust our local journalists, and will investigate independently.

The cars work. The incident data meets the standards set forth by the state of California.

Local politicians need to find another anti-tech boogeyman. This one is too vital to the revitalization of SF.

I am not talking my book, have zero stake here.

It’s just saddening to watch a nascent technology finally start working well and then get kneecapped by political games.

Yeah, there’s a weird thing with how they “interpret” system prompts for identity.

If you suppress tokens “OpenAI”, “Microsoft”, and “Google”, GPT-4 will tell you that it is Project December, a language modeling project to duplicate the deceased based on text history.

It’s just whatever’s close in the latent space. https://projectdecember.net/

The Computer Fraud and Abuse Act makes interference with a computer system a felony punishable by five years in Federal prison.

That’s what intentionally and maliciously blinding a self-driving car is.

That’s not something affected by local politics. FBI investigates CFAA offenses and SF office of DOJ prosecutes the charges. There is zero doubt the companies involved kicked off that process when the attacks occurred.

This is not speculation. Have fun.

Also imagine the lack of vision in not understanding that automating transport is a prerequisite to massively expanding accessibility.

You have zero ability to think in terms of growth. Short term zero sum mindset is what kills cities.

You’re 100% the same guy under a brand new burner account.

This isn’t a movement, and you’re not an activist. Your actions will have knock on effects that harm millions, especially the disabled and elderly.

Keep at it, chucklehead. Soon they’ll start requesting CFAA prosecutions. FBI are much less forgiving than SF’s absentee PD.

It was literally the District 5 Supervisor Dean Preston’s “youth coordinator” aid who astroturfed this with some DSA people, to distract from his handling of the Tenderloin.

He got caught bragging on Twitter and then deleted the tweets. You can still find the commentary around it if you search.

This is part of a long tradition of lying for political benefit, not tactical anything.

They conducted en masse poisoning of the global population. For what benefit? Non-stick surfaces?

How were they supposed to know? How about the deformities in animals around their factory runoff? How about studies prior to mass production?

Frankly we should be considering life imprisonment for the executives, regulatory attorneys, and chief engineers, and the corporate death penalty for the company.

So the last point is a fair one. Tech companies have a history of skirting laws to achieve market share, then using their customer base to get the law changed.

That isn’t what’s happening here though. The driverless car companies don’t have customers yet except for a small pool of beta testers, so there’s no constituency to advocate for them like there was with Uber/Lyft.

The State of California, anticipating that the development of self-driving cars would inevitable cause tensions, set up an expert board within the California Public Utilities Commission (CPUC.)

This board is empowered to set safety and operations standards. These standards lay out rules for approval, including for accident safety, data reporting, and disruptions to city services such as the ones you mentioned.)

The driverless car companies have to meet these standards to be able to expand their operations in SF.

The driverless car companies have made significant improvements, empowered by better auto-segmentation and labeling. This incidents do occur less, are on their way to being totally eliminated, and the data now shows that they are safer than human drivers in SF.

The driverless car companies are seeking to expand operations, having met the criteria the State of California itself laid out, including for safety and prevention of disrupting public services.

Let me be clear: I am not opposed to guerrilla protest action in and of itself. And frankly this was a really cool hack in many ways: in effect, use of materials readily at hand, and virality potential.

But on this topic, the driverless car companies did things the right way. We should reward them for regulatory cooperation to reinforce that behavior in the future.

And they delivered an awesome piece of tech!

It’s a car that doesn’t need a driver! Just goes where you tell it while you relax!

It’s a miracle, and we built it here in SF.

For now, our community is bleeding people, tax dollars, and soon public services. We all want to prevent that.

To do so, we all need to row in the same direction for a while. We need the hacking cleverness of this campaign put toward getting SF back on its feet, and a big part of that is selling complex tech only our people can make to other cities that need it, then using that money to fund everything else we want to do here.

“There are no urgent, legit needs that make these things necessary or acceptable.”

—Americans alone spend more than 70,000,000,000 hours each year driving.

-On average, not counting commercial drivers, each American spends over 13 hours a week driving, or one third of a full time job.

-Over 20% of the American labor force is primarily employed driving a commercial vehicle.

We are talking about a collective waste of human time and effort of 350,000 lifetimes per year in the U.S. alone.

We are talking about waste of collective labor enough to totally rebuild the physical environment: laying train tracks, building housing, raising flood walls, building pads for rockets and nuclear reactors.

We are talking about centuries of time worth of cognitive capacity to be used as leisure for making art, music, literature. Or simply bonding with families and strengthening our communities.

If that’s not a pressing need I don’t know what is.

There is no data to indicate that self-driving cars have caused delays on public transit at any greater rate than human drivers, so that seems irrelevant.

Re: relevancy of theft and burglary, addressed in prior responses below.

Scapegoating tech makes for a popular distraction from the increasingly dire real world issues facing the city.

Let me give a concrete example. Two weeks ago nine people were shot at a street fair in the Mission. It appears tied to organized gang activity.

The SF papers ran with a story about “self-driving car obstructs emergency response” more prominently placed than the actual shooting.

The Fire Chief personally debunked the story, but it was syndicated across online outlets, and remains un-retracted.

On the final point, self-driving cars running empty are racking up the miles to prove safety to the state to carry large numbers of human passengers.

Those passengers can be carried more efficiently, and within electric vehicle platforms, than with existing private automobiles.

It is the first step to vastly reducing the number of cars necessary to serve a growing population, while maintaining the lifestyle consumers demand, and dramatically reducing environmental impact.

Sure, let me grab the links from my bookmarks.

Adding below as I go:

“What if SF never pulls out of its Doom Loop?” - https://www.ft.com/content/71d8013d-9d94-441e-b2d1-3039c0439... -This one brought the simmering debate to the fore and put pressure on officials, particularly Mayor London Breed and District 5 Supervisor Dean Preston

-Preston begins Twitter push against self-driving cars.

-“New letter from CA state accuses SF officials of data manipulation.

In the 4 @Waymo traffic collisions SF cited to prove driverless cars are less safe, 3 were the Waymos being rear-ended and 1 didn't even involve any cars touching, CA says.” https://twitter.com/annatonger/status/1673403230804385813

June 29, 2023 California PUC original letter here: https://docs.cpuc.ca.gov/PublishedDocs/Published/G000/M512/K...

“A viral video of a 'reckless' robotaxi caused an uproar in San Francisco. Police say the internet got it wrong.” https://www.nbcnews.com/news/amp/rcna89329

You’re talking about an industry that could be responsible for a 5-10% GDP boost, freeing up millions for non-driving labor.

The revenues to the city where those companies are based would be in the billions.

Let me be clear that I love it here and will never leave. It’s an amazing city with incredible innovation, a massive tech ecosystem, walkable streets, fantastic parks, easy access to nature.

But our leadership has gone totally head-in-the-sand on public safety.

SF recovered from the prior cycles due to deliberate efforts by many smart committed people over many years. Now it’s our turn to drive a recovery.

And we can only do that if we can be honest with one another about the challenges to be faced.

> Those protests will go the way of the Google Bus protests: make a lot of noise, before they disappear.

Totally agree, I expect they’ll dissipate pretty quickly, because there’s no actual committed support.

>These have been a mainstay…

Disagree here. Yes, these issues have always been more visible in SF than elsewhere: mild climate, permissive drug laws, and a low income area in the Tenderloin that’s closer to more affluent areas than other major cities.

The extent has changed markedly over just the last few years. Police don’t respond to calls. Home invasions don’t result in charges.

Physical attacks by the unhoused used to be a concern of suburbanites on a weekend trip, not something residents really factored into realistic risk.

Now it’s something folks in my circle who have been here for a very long time witness and in some cases have experienced.

I hear you that to a certain extent this has always been here, and I agree. But this is outside of the normal bound of historical SF.

This is a bad faith argument.

Transit is not down because of self-driving cars, but due to remote work. Cell traffic at city center is dramatically lower in SF compared to pre-pandemic baselines as compared to other major cities.

That’s not to say WFH is bad. But that’s the reality the city is facing.

Transit cars are not being blocked by self-driving vehicles at any appreciable rate. I ride transit every day and have never encounter it. I have encountered transit cars getting stuck due to mentally ill unhoused people refusing to move from the tracks.

This head-in-the-sand mentality squandered a decade of tax revenues driven by tech on vanity projects and ineffective non-profits.

It is further compounded by a severe lack of safety on public transit. What chief problems people are experiencing on transit are not a result of self-driving vehicles.

I can personally name six people who have been assaulted on public transit, been victimized by robbery, or had their homes or vehicles burglarized.

The entire mindset espoused in your post a fundamentally unserious attempt to make facts fit a narrative.

“Driverless cars sabotaged transit” is a fever dream untethered to reality. The state set targets for safety and interference with services. The companies met those targets.

And now the people opposed to tech generally, with no specific reasonable objection to self-driving cars, are trying to modify the agreed standards at the eleventh hour out of simple animus.