Does this mean that transformers are most akin to rule-based expert systems with opaque (currently) statistically-devised rules?
499 karma · joined June 20, 2022
For every 20 paragraph thesis about the world around us, how often do you think someone meaningfully innovates and changes meaningfully our understanding of the world around us, and it's fundamentals?
Consider, like, classical logic. It's pretty much done. You have the universal gate sets, and any novel functions are just going to be an mxn-mux. Most other such general concepts are essentially figured out and have had their applications recorded, explained, demonstrated.
Sure, biographical information and history changes on a whim, but no one is requesting artificial clairvoyant intelligence.
I suppose you could have the model pontificate and assess the truth of its own argumentation (ask it to generate outer-product concepts, ask it to reassess assumptions in current models) but it will quickly learn its own language that will be mutually unintelligible with the corpus of current human knowledge it has been trained on.
Why does the underlying model need to change on a whim, in your opinion?... on what whim?
... not that I don't disagree with the rest of what you think you've said here, but I think that this part of your claim is egregiously paucious.
Thank you for completing your job application for the [not available] position!Check with your school's career center and seek an internship. Success at an internship and maintaining a quality career track is one way a B.S. is extremely valuable.
The other way a B.S. is valuable is networking and making groundbreaking moves -- startups with talented peers or research with guidance from talented professors -- while you're pursuing the degree.
Think of the degree at the end as a hedge for your access to any of those tracks and you'll be better served.
If you're looking forward to porting to a more performant language, dict has parity with struct, and vtable dereferencing and polymorphism has performance penalties in say, C++. rust and golang don't really have a robust object implementation so flatter structuring is preferred.
Apparently copious unfulfilling underpaid labor may lead to escapism. Simultaneously we're exporting inflation to our southern neighbors who have a robust, lucrative industry which may lead to conditions favoring that fate.
If banks pick winners, everything becomes a bank.
The question no one asked is if everything should be stockpiled.
I think there's no question that we should make things better or more plentiful.
I mean if your intent is speculation, this is how the rest of the internet has been built.
TCP and OS security are so bad they ripped the internet a whole Cloudflare, but they all still exist and find their uses in real systems.
No comment on regulatory or legal frameworks in place since I have no idea what's gone on in the courts on that front. It's like entropy though and I have doubts that there's any long term value in trying to stop it.
Banks beget banks, because banks only invest in diversified quasi-banks, basically stockpiling non-cash assets, feast or famine. A reasonable structure for much of human history, if you like (often) dry, safe, grain/peas in the winter. Before global transport was fast and risk-free. In vogue right now: "all the PCs, all the business software, all the electrification efforts/grid modernization, all the goods manufacturers storefronts, all the films, all the personal sites, all the web pages." Maybe not such a relevant capability (large stockpiles) outside of, well, contingency.
Bitcoin, the machine, only invests in progress, cheaper more plentiful energy and faster processors or you lose. It has scarcity and inherits supporting stockpiling.
Ethereum? Presumably cooperation and transparent software, also good things! Perhaps in the next century proof of health, proof of satiety.
That's not even a whole city's worth of people "demonstrating", and it's a global thing. It's a quarter of the speculative value of AAPL.
It's also concurrent and passive. Very few people have dropped everything to do the crypto influencer dance, and now influencer+crypto is seeing SEC crackdowns.
Crypto is at most slacktivism right now. Nothing like BLM, MAGA, which are no doubt viewed by powerful economic planners as wasteful.
You can't generalize the whole thing as spectacle just because a few of the congressmen choose to grandstand / push an agenda / be generally tonedeaf, I can almost guess that in 6 hours at least some of the concerns are legitimate and merit public discourse even if they are nuanced, possibly quite boring, and have more to do with how OS maintainers and parents should behave.
Everything is in the crosshairs and nothing material is sacred, and it has been that way for as long as I can remember[0]. That's why we hire these people.
If individuals aren't being "excessively enriched" by conditions it's unlikely to become another spectacular market wide blow-up.
Social causes are a proxy for froth. A short Friday memo probably isn't.
Exuberance is in the hands of price gougers right now, and even that's mostly confined to California,... and eggs.
Sure we're back to "grocery delivery" (which marked the height of the .com bubble) but it's the biggest players involved, the technology is practically free these days, and the immunocompromised benefit!
I can remember being in high school, still in the age of minority, and feeling joy when one of the louder, dumber cooks finally kicked off.
I kind of suspect Beijing and the CCP of not being so colorful at the federal level, and rather just being a giant executive bureaucracy with principle, law, and regulation intermixed freely, minimal federation, which leaves them largely ignored by society and festering until they're in your face enforcing lock downs -- like an authoritarian military state.
I unfortunately don't read a lick of Mandarin, and I'm not sure where to get a more optimistic view about what goes on over there, but I'm curious.
This looks like splitting, or you're deliberately failing to understand how lawmaking works and the importance of making informed decisions at the federal level, in the long term.
Congress constantly grills CEOs, the elite, and victims because it is understood that they reshape society, sometimes for the worse, not always on purpose.
Is a law needed? How do we tax it? Who is being harmed? What is the harm? Can a law be enforced?
The transparency is by default. Not a "spectacle".
"Policymakers have tools to solve banking crises [...]"
"The economy is in a much different place"
"The magnitude of the problem is, so far, much smaller"
In conclusion, "we still don't see it coming" avoiding the term, "subprime" or acknowledging that predatory lending ever occurred, and this weird typo:
"[...] the primary facility *though* which banks can borrow [...]"If you think of it as a statistically-derived processor/algorithm, unless there's some great task of short-term memory/memorization it's forced to undergo it's going to be biased towards more direct responses.
Another thing to consider, regarding the fine-tuning, is that "instruction" may be biased towards sentence ordering, so that the actual order with which you present your prompt can be important.
For example Prompt: <detailed situational specification>. <task to be performed> might perform considerably worse than <task to be performed>. <detailed situational specification>.
I suspect it's just a short-mid-term resource constraint though since:
- demonstrating extremely sharp memory performance conversationally isn't that common
- total bout / token width isn't particularly wide in training
- presumably fewer "transistors" than humans -- basically even 200B parameters isn't all that many
Another concern that I'm not sure how to consider is that 4-letter tokenization could be an obvious reason ChatGPT is such a poor speller.