1,261 karma · joined March 15, 2018
Documentation of unknown quality is useless noise.
People don't understand the unfathomable amount of garbage that's going to be generated in light of all these models. Doesn't matter how accurate they are, the lack of understanding for that remainder percent of inaccuracy is going to cause false confidence and cause errors to compound like mad.
It’s the same problem that caused the replication crisis.
You start with something small, like a department for a new field, a random paper, etc. You explore and hit some interesting valuable knowledge. You share that, and it blows up and makes everyone’s lives vastly and unexpectedly better. So you pour a whole bunch of resources into getting more students, pushing more papers, creating more procedures, and building your little department into a giant machine. It works amazingly for a while, and then you think the machine is what’s generating the value and leave it on autopilot. Researchers that fit the machine remain, those that put more energy into actually exploring rather than fitting the machines criteria for explorer fade away. Knowledge slowly turns to nonsense through self reference as the true value, the researchers lucky enough to stumble on insight, get drowned out and leave.
Our perception of the physical world is itself an evolved construct. The wider thing that perception is fitting is something I don’t think we can fit a machine to. I think we can only fit it to the construct.
I get where you’re coming from/appreciate the argument, and you may be right, but I‘ve come to appreciate the ubiquity of hidden context more and more/lean that way.
Gödel proved it’s impossible for any deterministic system to determine its own correctness close to a hundred years ago. Whatever we’re doing when we say an output A is “right” and output B is “wrong” is fundamentally different than what a propositional deterministic system does, regardless of how large or chaotic or well fitted to expected output it is.
What qualifies as “true intelligence” is the core issue here, yes, and transformers don’t qualify. That doesn’t mean they aren’t valuable or can’t give correct results to things much faster and better than humans, but I think anything we could deliberately design inevitably ends up being deterministic and subject to the introspective limitations of any such system, no matter how big or all encompassing.
I think we’re going to create better and more comprehensive “checkpoint”/“savepoints” for the results of collective intelligence, but that’s different than the intelligence itself.
However I do think it’s quite possible and likely that we’ll create mimics that convince most people they’re intelligent. But I think that’s a weird type of reflection/anything we can make is limited to mirroring our observable past and existing collective thought and can never be truly introspective, because true introspection requires evolutionary context and hidden motivation you can’t transfer.
Imagine an amoeba that can only detect light and dark. Its observable universe exists on a simple gradient. But the cells that are used in its perceptual system cannot be described on a gradient. I think that probably scales up indefinitely.
We confuse what we see for all that exists because of our recent history and scientific advances. The triumph of science comes from focusing solely on what we can see and reason about and improving our vision because thats what we have control over, but it doesn’t mean we see everything. I think true intelligence originates from something we can’t see. You can call it a soul, or call it a scaled up amoeba cell, but I personally think the origin of true intelligence and qualia is from some weird very very old evolutionarily created thing we can’t see (and I think other animals have similar invisible perceptual machinery somewhere, I don’t think it’s just a human thing).
Essentially him and his crowd have been trying to create a rationally grounded purely logical non contradictory moral framework for action from axiomatic first principles (he’s not the first or the last to do this), and they believe such a system is essential if AGI comes to be.
I personally think AGI is impossible, his purely logical moral system is impossible, and think what he’s working on is not applicable to current AI (I think he knows this, but his conclusion seems to be that the path to AGI is out of the bag/was made incredibly foolishly and should have been constructed differently with more ability to add guardrails)
But I do think the high level abstract way he thinks is fruitful/very valuable, and a good moderating counterbalance to the frenzied “harder better faster stronger” ML stampede that’s not really thinking about the ramifications of what they’re doing with any sort of discipline.
It’s like a weird parrot. But I think a parrot “understands” more because of the shared embedded evolutionary context it has.
That evolutionary history has the key to true intelligence somewhere, but personally I think it’s inevitably hidden/I don’t think we’ll ever understand how intuition and truly non-derivative, non propositional human thought works.
I also don’t think any of what I’m saying negates the value of these models. These models are fantastic autofill generators for a huge swath of different applications and can vastly improve productivity. I’m saying all this in a lot of threads where it comes up because it seems clear there’s going to be too much enthusiastic adoption, which is going to effectively destroy a lot of value of the internet.
The internet is the best tool for finding genuinely creative and novel ideas you were unexposed to that has ever existed. But it is increasingly dominated by derivative unoriginal content that drowns out what I would argue it was designed to help you find. I have no problem with derivative unoriginal content when it’s properly understood as such. I have a problem with how good these things seem to be at tricking people into following something derivative and blind, which seems very very dangerous.
Most of these ML projects are essentially just creating a best fit line/shape connecting a huge number of points in multiple dimensions and giving you output coordinates on that line/shape based on your input coordinates (as I understand it). The more supervision, the more you’re negating the value, as you’re basically telling it to make a shape more like something you already understand (instead of something new/actually generative, which requires interesting/novel human input)
I’m not an ML expert either, and if one wants to chime in about how this picture I’m painting is wrong or what else is going on that would be welcome. I’m not trying to belittle how impressive progress has been (I have no idea how the parameters are determined and have a huge amount of respect for people able to handle a hyper-dimensional best fit optimization problem). But I don’t see how all the value isn’t inevitably downstream of high quality human generated digital content, which seems likely to decrease rapidly as more automated content floods the internet and lowers incentives for creators.
The thesis of the web revolution was “the world is too big to be understood by any one person”. The response was “let us all contribute to a shared map to compensate”
The thesis of the AI revolution is “the world seems compressible”. The response has been “let us condense and contain all the meaningful parts in a single generative pattern that can understand everything”
The first thesis is correct. The second thesis is wrong.
Not only is it wrong, but it depends on the fruits of the first thesis to create enough compressible material to trick you into thinking it may be true. The more society leans into AI the less genuine content will exist and the worse it will work.
I understand the desire, and clever search indexing allegedly solves this, but it's fundamentally an impossible tradeoff. Organization is essential to finding things.
I'm tired of people pretending search and AI can solve that problem. It can help, but it can't tell you what x or y thing you want is. I find myself typing nonsense like "what's the thing that's important in speaking" if I watched a video about how to give a good talk and am wanting to watch a part of it again. I haven't given it a title relative to how I think about it, I haven't thought about how to categorize it, and I haven't let a "place" for it settle in my head, because I didn't bookmark it or add it to a playlist.
The nitty gritty punch card programming was something stereotypically done by middle aged/older women. Many early computers were massive and housed in what could be considered temples in big universities surrounded by gothic architecture. They were expensive to replace and kept around for long periods of time because of how difficult it was to upgrade (some large mainframe systems still exist today for that reason). Mechanical computers like the Babbage difference engine would last a lifetime. The idea of an average Joe spending all day using expensive compute time was insane. Those who focused on them did so for academic reasons, and spent most of their time meditating on computation, math, and the structure of language and meaning.
It was the most amazing, refreshing, delicious water ever. While there was almost definitely a psychological component to the initial taste (we were used to iodine flavor, were tired after about 3 days of hiking, and the area was beautiful/it looked like something out of a commercial for mountain water), we were there long enough and it continued to taste good enough that I'm positive there was something else going on. I think it had something to do with the rocks, a light amount of healthy bacteria(?) and it being fresh snowmelt.
I'm in no way advocating for shipping water across the planet, and I don't think you can bottle that up or recreate whatever was going on there at scale, but there's definitely a legitimate, big range of difference in water quality. I'd bet good money most double blind taste testers could easily distinguish between legitimately fresh cold mountain water and refrigerated water that's been sitting in an aquifer and then a plastic bottle for a while, even if you matched the salt content. But you'd have to go up a mountain to do that test properly. I think it has to be like right out of the stream to make the biggest difference.
I'm also joking, but only partially. It could be a fun gimmick. Check all your electronics in at the door, go through the doorway with an EMP zapper, and then enjoy a distraction free camera free evening. You get banned if you get caught sneaking in electronics and you might wreck your expensive phone, so why risk it.
We could also just stop non consensual surveillance of everyone and everything. I prefer that option.
A lot of performative outrage seems to be propping up a very large white collar industry of sensitivity training and related roles, and a lot of it might go away if there were more alternative, stable white collar industries that were related to skills taught in college and did not require constant learning and new skill acquisition.
I also think people who have a hard time learning new skills at the ever increasing pace technology demands lean on performative outrage as a way to scare employers and increase job security. If people felt more stable in their jobs, another chunk of performative outrage would probably disappear.
And I also think a lack of social belonging and attention from things like family and neighbors you’d get from a middle class household experience leads people to use performative outrage to get a fix of attention they’re missing.
I combined a lot of those ideas/what I’m saying may not have been clear, but those are all side effects of a shrinking middle class and pathways to the stable mass achievable paychecks that enable it.
I’m not arguing that there’s no legitimate level of complaint or job for helping employees relate to each other better, or that people shouldn’t strive to relate to each other naturally on each others’ terms, but it seems like there’s been a steep rise in bureaucracy that is less about actually encouraging productive cooperation and happy employees and a lot more about job stability for the bureaucracy.
That can be true despite there being a majority of good faith participation in that bureaucracy and a lot of employees who believe in the mission. If you get a job and you’re a decent person, which I believe the majority of people are, you want to do a decent job. But if it’s hard to find other comparable jobs, the need to feed and house yourself usually overrides considering whether or not your job is actually exacerbating the problem you’re tasked with solving.
I don’t think having people dedicated to monitoring employee communication for insensitive interactions and training people in the way that is currently done typically helps, I think it typically makes employees more paranoid and prone to overanalyzing interactions due to increased focus on sensitivities. Which is tragic, because again, I think most employees who get involved in jobs to help people communicate better legitimately want to be making a positive impact.
Consider the population of rapists. Who are you more likely feeling safe complaining about: 1) A professor 2) A cop 3) A powerful politician 4) A gangbanger 5) An unhinged uncle
Consider the population of false rape accusers. Are they more likely to be: 1) An immature college student relatively new to determining sexual boundaries marinating in ideological literature that considers looking at someone the wrong way rape 2) A drug addict, prostitute, or other person a cop typically runs into 3) A regular person that knows the abuser has relationships to prosecutors 4) A person with good reason to fear for their life if they report 5) A person who has to deal with family betrayal if they report.
Between the administrative bureaucracy obsessed with student safety, the often rich and well connected protective parents propping up the school, and the often nerdy and agreeable nature of professors, professors are also probably some of the safest people to accuse of rape (rightly or wrongly) on the planet.
That right there is the real problem. I’m convinced a huge motivation for a lot of the current performative outrage is a combination of conscious and unconscious anxiety about having very little to no real skill based leverage or ownership/stewardship of things that anchor you to society, like housing and kids. That’s the sympathetic angle. The unsympathetic angle is laziness and greed. Both are applicable.
Everything is so damn complicated and fast paced and fewer and fewer people seem to know how anything actually works. That means fewer and fewer people know what a reliable career track will be. A lot of people can’t pivot their skillset nearly as fast as tech is demanding and they’re worried they’re going to be made obsolete and left without any viable alternative path if the one they’re on dries up.
I think a lot of the performative outrage would disappear if the middle class regained stability and people felt like they owned something rather than felt like a replaceable cog in an unintelligible bureaucratic machine.
Academic types and teachers are some of the most agreeable and easily bullied people on the planet. They’ve already been severely gouged out of payment for the vast majority of the value they provide by administrators. The only real reason for that is personality profile. Colleges and schools get virtually all of their real value from high quality teachers. If teachers were less agreeable and less easily bullied by bureaucrats they’d have kicked the vast majority of the admins out by now.
Professors give their lives and turn down lots of money they could be earning on different high intelligence career tracks to teach students in classes where you’re lucky if only 50% are ungrateful, disrespectful and/or incompetent. They do that because they genuinely love teaching and mentoring and research/learning, even if only a fraction of their students absorb it. A lot of them also do it because they have a kind hearted but soft-spined personality profile that’s never left the Alma Mater.
The profession doesn’t have a power tripping cop like problem, at least for the majority of teachers. And the types of professors that do go on power trips often turn classes into cults and use their students as weapons. Look at Evergreen.
These kinds of anonymous complaints are just another tool for the types of bullies that have already destroyed a lot of the value of the university to get student followers to use against people they don’t like, and a tool for immature students (which colleges are full of, since the main impetus of college is to guide young adults to intellectual maturity) to attack their professors on a whim with zero consequence.
I don’t see any world in which they don’t do way more harm than good.
This article is from 2017, and I don't know about you, but I increasingly feel like software is more and more broken year after year. I think that has a lot to do with "delegating complexity" and building things without really understanding what they're doing. I think the correlation between human understanding of the fine details and underlying logic and desired outcome and software quality is pretty tight. That doesn't mean "software 2.0" neural net stuff doesn't fit in, you just need a human to plug it in right that really understands its benefits and its limitations.
The author mentions the downsides, but I think they underestimate them. If you lean on AI too heavily and don't ever translate to a traditional language, you've basically liquified your logic/there's no garbage dump to salvage from when things go bad and zero understanding of implicit context. If you use it to generate ostensibly human readable code you get "documentation" with no guarantee of accuracy, which makes it worse than having no documentation (depending on how high the error bars are). While that's not a new problem either, if it's autogenerated that means it's easy to create way more of it than human generated code, which means it'll probably be an ever larger portion of what gets sucked up into later AI models. If they become too self referential they'll become increasingly detached from human judgement about whether the code is doing what it should and error bars will grow.
I'm still convinced these things are virtually all going to end up in a fancy autocomplete suggestion and compression niche after a lot of pain. But that's still a big deal/I don't think the limitations of these means they don't have a big future place. The sheer number of things you can have autosuggest for now with these AI models are amazing, and that expansion is boosting productivity and creating a large number of new products that are going to become essential tools. That being said, every time this type of thread comes up I'm like "woah woah woah, pump the breaks, these things have no understanding of what they're doing. You can't just stop thinking about stuff and let a machine do it, bad bad bad idea."