455 karma · joined March 28, 2016
Interested in: - epistemology - social/financial structures. - startups
However, in this article I contend that those limitations have posed little adversity in the field given the success of the latest models. As a result, it may be a bit premature to be concerned about it.
I wanted to avoid my experience but I worked at FAANG and helped create a multi-billion dollar corporation (from a handful of people to 1,500 people).
The self-employed is now able to rapidly develop their ideas while employed reducing their risk and cost.
Per VC, that's right and secondarily it's more difficult to justify earlier funding when one no longer needs to hire a substantial team. I expect the current major VC firms to get even bigger.
In this world the neurodivergent is empowered, they no longer are charged with persuasion of their ideas to a team or corporation. They can build their ideas themselves and form a limited partnership with someone with the talents they lack.
Though I make it a point to describe the new economic order without regard to its desirability. However, other authors on the subject (e.g. Lysander Spooner, Rothbard, etc) would be pleased by the development in terms of its social welfare.
> This is irrelevant for AI, because people throw more hardware at bigger problems
GAI is a fixed problem which is Solomonoff Induction. Further Amdahl's law is a limitation on neither software nor a super computer.
Both inference and training rely on parallelization, LLM inference has multiple serialization points per layer. Vegh et al 2019 quantifies how Amdahl's law limits success in neural networks[1]. He further states:
"A general misconception (introduced by successors of Amdahl) is to assume that Amdahl’s law is valid for software only". It would apply to a neural network as it does equally to the problem of self-driving cars.
> These two sentences contradict each other
There is no contradiction only a misunderstanding of what "eviscerates" means and even with that incorrect definition resulting in your threshold test, it still remains applicable.
1. https://pmc.ncbi.nlm.nih.gov/articles/PMC6458202/
Further reading on Amdahl's law w.r.t LLM:
2. https://medium.com/@TitanML/harmonizing-multi-gpus-efficient...
3. https://pages.cs.wisc.edu/~sinclair/papers/spati-iiswc23-tot...
If we assume they are beneficial, this also doesn't mean the government should subsidize it less but instead more to continue to allow the production of drugs. If we artificially require a price, many of these drugs would simply not be developed.
I'm under the persuasion that the future of healthcare is a voucher system to allow the competitive bidding of drugs/insurance to remain as well as the maintenance of the quality and quantity of medical research. Many countries purport to have better medical systems however we find that the American payers subsidize the rest of the world––with the USA producing more medical research than the rest of the world combined.
In a single healthcare system, the prices of drugs deemed essential will skyrocket while others will simply be excluded by the bureau. This then creates a cost on the tax payers through the inflation/excess tax required by the national budget to afford it. While at the same time the single healthcare system has little incentive to improve its efficiency of administration until it's fully deteriorated and governmental intervention promises a revamp which may still be ineffective.
The voucher system allows many insurance companies to exist, allowing the quality of insurance provided treatments to remain through that competition on their varied administrative strategies. While at the same time consumers with varying degrees of medicinal concerns can choose the right programme for themselves––opting to even pay beyond the sum provided by the voucher. This way it's no longer only affordable through a corporate benefit and the buyer group is available instead through a national benefit.
Indeed even theoretically we can do 86% multi and 99% with quantum dots——in the current state of technology these remain science fiction considering the commercial manufacturing processes available.
"even if space is composed of nothing but relations between observers and events, it would be conceptually possible for all observers to agree on their measurements, whereas relativity implies they will disagree" [0]
Developers will therefore always maximize resource usage as more becomes available.