725 karma · joined November 18, 2021
A more relevant example is video games, imagine if the only viable ones were top end AAA games whose completed versions could only be accessed by cloud gaming?
A lot of deployed deep learning is in Python so I doubt this will matter. Looking at the docs, it looks like the main speech to text is python heavy, it leverages elasticsearch and calls libraries that are bindings to code written in C++. As none of that overhead will ultimately matter for anything approaching the fidelity you get with cloud services, cost benefit trade off favors Java in this scenario.
> lower profile computers
When it comes to deep learning, this just isn't the direction things are headed. If you want the best results with low latency, your home server is going to have to be fairly beefy with a decent GPU and lots of memory.
To be fair, a lot of creative work requires plenty of trial and error. And since no problems are solved from scratch, all things considered, the most immediate contributors to your result and you might have iterated through tens of dozens of possibilities.
My advantage as a human is I can often tell you why I am eliminating this branch of the search space. The catch is my reasoning can be flawed. But we do ok.
> just copying previous solutions with slight adjustments.
It's not just doing that, Copilot can do a workable job providing suggestions for an invented DSL. A better analogy than autocomplete is inpainting missing or corrupted details based on a surrounding context. Except instead of a painting we are probabilistically filling in patterns common in solutions to leetcode style problems. Novelty beyond slight adjustments comes in when constraints are insufficient to pin down a problem to a known combination of concepts. The intelligence of the model is then how appropriate its best guesses are.
The limitations to GPT3 codex and AlphaCode seems to be they're relatively weak at selection and that they require problem spaces with enough data to distill a sketch of and how to inpaint well in them. Leetcode style puzzles are constructed to be soluble in a reasonable number of lines, are not open ended and have a trick to them. One can complain that while we're closer to real world utility, we're still restricted to the closed worlds of verbose apis, games and puzzles.
While lots of commenters seem concerned about jobs, I look forward to having the dataset oliphaunt and ship computer from Fire Upon Deep someday soon.
Recording is not knowing, at best it is memorization of largely useless trivia. Sherlock has a good discussion on this (Sherlock solar system). To know something you have to have processed and at the very least achieved some compression of it.
Notes help with processing and thinking things through, aiding conscious and active understanding.
Unless recordings feed into one's process, they're a waste of space. Sturgeon's Law: The vast majority of what we encounter is junk, much of it even self-contradictory. The more you let into your corpus, the harder the problem of retrieval relevance.
My clippings are actively curated, carefully sized to fit in working memory while also independently comprehensible and guaranteed to have met my minimal standards on factuality. Because I have read them carefully before, I expand my mental canvas but am not slowed down too much by processing demands. The throttling on note growth means an easier time for information retrieval and much fewer irrelevant results to ignore.
Nuance requires representing uncertainty and is higher complexity from accounting for multiple special cases. Placing more cognitive load and attentional demands on receiver.
Human communication is lossy and decoding can be non-trivial inference. Given that everyone comes with differing priors, the more finer grained and complex details are required in reconstructing message intent, the more likely it is to be misconstruected. Failing to attend to a single core detail can rend meaning.
At scale the chance for errors to propagate without correction increases.
Getting rid of fine grained details and communicating a lower entropy message inline with a crowd's biases also ensures it's more likely to survive in a form close to original intent. A good manipulative orator focuses less on truth or content and more on minimizing mismatch between receiver mental states and orator's directional preferences.
The history of astronomy spans thousands of years, a high number of unusual observations will sure to have accumulated over time. But finding an unpredicted aspect of a previously unobserved class of objects is an exciting event carrying a relatively high amount of surprisal. The involved astronomers are clearly excited and being reserved with language is to be expected of formal papers. While not an astronomer, I find the resulting changes in observation patterns exciting to read about and look forward to learning about the unusual radiative properties of such magnetars.
> We find that the source pulses every 18.18 min, an unusual periodicity that has, to our knowledge, not been observed previously.
The mysterious (as yet unexplained) aspect of it will require updating models's physics to account for its unusual emission properties. The lead researcher herself distinguished it from known types of objects and states "completely unexpected [object]...nobody thought would be able to produce this kind of emission".
They're estimated to live between 250 and 500 years.
My thoughts on this is as tools for thought, there can never be a one size fits all solution for the reason that there are many thinking styles and personalities. Todo lists, project planning and time tracking, programming languages, note taking apps, UI frameworks, game engines all fit the pattern of tools meant to organize complex mental processes and artifacts and where everyone settles for what best matches them though never fully satisfied. Meanwhile those who can find the time and ability to make something better suited to their taste do so, also bringing something to those most aligned to their tastes.
The word list for the initial guess can be precomputed (I handle this simply as well, top 16 words with 3+ vowels, no duplicated letters and maximized pairwise similarity). The easiest first step that goes a long way on its own is simply maintaining constraints. Having applied constraints, the next issue is selecting from narrowed word lists. To do that I compare every word to every other word by spelling, idea being highest scoring word will be most informative, but also constrained to maximize difference from previous guess (although this step is rarely needed).
As available guesses are already significantly whittled down even by second guess, it runs in << 1s and completes in 3-4 guesses.
A less cynical take would shift towards the perspective of Gustafson's law. Modern systems attend to more stuff and have more capabilities and affordancies than older trimmer but also simpler software.
On Jevon's paradox, a countervailing force is ephemeralization or dematerialization, of which a large aspect is pushing away from resource intensive builds to more energy and information processing intense arrangements.
F I L L S (0 green, 0 yellow)
W H E R E (0 green, 0 yellow)
D O U B T (0 green, 0 yellow)
M A N N A (3 green, 0 yellow)
C A N N Y (4 green, 0 yellow)
N A N N Y (5 green, 0 yellow)
A L L O Y (0 green, 0 yellow)
F E T C H (0 green, 1 yellow)
I N D E X (0 green, 2 yellow)
P R I Z E (3 green, 0 yellow)
B R I B E (3 green, 0 yellow)
G R I M E (5 green, 0 yellow)I would prefer if we could have more certainty than that. I have a list of questions.
1) Does no one else have access to this codex? Has anyone tried to replicate this? It should be easy.
2) If it was not fine-tuned, there is still the issue of an over-fit question set, as you say. How easy is it to break? How general is it?
3) Are there really so many extensive examples on the use of sympy? Some of my googling (I know the math but not sympy) did not support this but it could be my lack of familiarity of the library ecosystem.
4) And the biggest issue where when it gave answers to certain limit problems without any computation, which should not have been possible unless it computed it internally, complexity of which kinda contradicts the entire exercise.
Depending on how generalizable this result is, it could be anything from merely impressive to mind blowing and revolutionary. Imagine if the language is Coq or Lean instead of sympy!
My biggest gripe with this paper is how unclear it is on its methods. How many rewrites per question? How did they select the solved questions? They state they used a model trained on language and fine-tuned on code, but fine-tuning could describe either their or codex's process.
The biggest hint in favor of fine-tuning is that AFAICT, their dataset adds up to 205 questions but they've solved 235 questions. In which case I'd suspect overfitting on question form. In intro level math, problems are usually in template form and solving them boils down to matching on the correct template and slot filling the answer.
To prove whether it's been fine-tuned, people with davinci codex access should try to see if the can replicate this.
To prove it's not overfit, authors should release their training dataset if there is one and allow people to test with it.
How many parameters does davinci codex have? The original codex was 12 biliion IIRC and certainly wasn't this capable.
---
Some of the answers look wrong or incomplete.
Table 221 seems to be missing a factorial, such leniency suggests they must not be automatically scoring this.
Not sure what's going on in Table 144. Table 44 too.
In 42 and 47, particularly 42, the solution program seems incomplete.
211 is impressive, it also writes code for LU decomposition, permutations, runge kutta, card combinatorics and other clever stuff.
Even though for most of the more challenging problems, hard part was pre-digested or worse, it was hand fed the answer's algorithm, there were a few where the network had to have had a solid understanding of the math and code requirements. A program like they claim would revolutionize and hugely simplify mathematically and algorithmically involved programming.
The biggest cause for worry that it might have overfit is that it works with no sampling at all and gets a perfect score (they claim).
They had electric telegraphs, telephone and radio, so I'd explain it as miniaturized wireless telephones and typewriters combined, additionally with capabilities of paperless networked teleprinters of a kind. Some of the stations would be libraries with expert librarians on hand to tirelessly answer your questions.
> then Twitter and Facebook
I believe this is already covered. Anyone can now own a small personal broadcast station.
> try to explain how a cryptocurrency works.
I'd use the analogy of a network of tabulating machines with accumulators controlled by inputs according to something quite like point to point teleprinters instead of by punch cards. Records with a copy on each machine, would store on a very flexible type of automatically configurable control panel board with a special arrangement of relay circuits capable of acting as a memory, rather than in boxes of cards. Such a machine could track some quantity, addable and subtractable like a currency. I would then bring up the difficulty of protecting against liars and double spending. One solution might be to connect each machine to automatic slot machines and have them play till there was a winner who then got to broadcast an append to the record with the longest history in their collection of records. I'd also note that the more slot machines there were, the more slots were magically added to each. The idea being to make it costly for any one view to dominate the network.
Far from perfect but the gist could be communicated to anyone with a bit of imagination. Modern technologies have had a large impact on life but can be connected to 1920s tech. This is unlike the gap from the 1820s to 1920s that saw electromagnetism, statistical mechanics, quantum mechanics, special and general relativity, radio broadcasting, telephone, electricity and electric engineering, atomic physics, flight, automobiles and more.
But you can't be friendly if foundation is strewn with fear and insecurity. It's also important to be able to recognize if rivalry is causing too much stress and revisit your goals and priorities. Sometimes it's fine to explore and go at your own pace.
As advanced as Ancient Greek mathematics was, its heavy emphasis on geometric formulations made many developments more difficult. The Hindu-Arabic numeral system, algebra, invention of a zero number concept (Cardan's proof of the cubic and quartic would have gone significantly easier if he had utilized zero as a number) and Descartes's Analytic geometry were important developments on the way to calculus. By the time of Barrow's and Fermat's contributions, nearly all essential components of calculus existed, just requiring some genius to grab, synthesize and streamline them.
When I searched I could not find anything that could be pointed at to justify his being labelled racist. The example you point out is an instance of the just-so type stories I mentioned. It would have been more helpful if the SciAm article had gone into detail with examples like that, pointing out relevant and topical flaws and gaps in his reasoning.
The criticism section of the plato page I linked to goes into some detail and ideas of his relying on group selection can be instantly marked as suspect. It's a shame because if the article had been more on-topic, resulting conversation could have been more substantial.
What I could piece together:
People didn't like that he supported the idea that genetics has a large influence on personality, behavior or intellect, a genetic leash as he called it (he also argued against free will). I suppose the fact that Evolutionary Psychology (which is susceptible to just-so stories) derives from Sociobiology is also enough for current people to unfairly accuse him of promoting racism.
Something surprising I learned from all this is cancellations, de-platforming and campus protests of certain ideas already existed way back in the 60s and 70s! The only difference with today is increased connectivity and mass.
> In the ’60s and ’70s it became almost dogma — it was a dogma — to believe that the human brain was a tabula rasa, a blank slate. I don’t think scholars in this generation, even those of middle age, can appreciate how stern was the prohibition against believing that human behavior was influenced by genes in any manner whatsoever.
> But I really was upset at being called a racist, promoting racism and sexism. I was accused of trying to reintroduce a retrograde, outmoded, dangerous philosophy. There was nothing in “Sociobiology” to suggest such a thing. The words had to be taken out of context and tweaked.
> On one occasion, I had a little mob in Harvard Square parading and protesting and holding placards demanding that Harvard dismiss me. On another occasion, when I was to give a lecture at the Science Center, a crowd of protesters gathered at the entrance with signs and shouts and chants and so on. I was ushered in through the rear by University police. My class was disrupted at least once.
---
The SciAm article spent only a few words on Wilson. It instead seemed to really want to air disagreements on how health studies are carried out and IMO, was sloppy in its attempt to shoehorn Wilson in. It would probably have worked better as a stand alone piece on how poverty impacts on health outcomes are ignored.
---
Sources:
https://news.harvard.edu/gazette/story/2014/04/search-until-...
https://plato.stanford.edu/entries/sociobiology
https://old.reddit.com/r/math/comments/1bwoeg/e_o_wilson_vs_...
Covers Quantum Field Theory and General relativity in as approachable a manner as I imagine is possible while still going deep into detail: https://www.youtube.com/c/viascience/videos
Theoretical machine learning, Information theory and probabilistic inference: https://www.youtube.com/user/mathematicalmonk/videos
Heavy emphasis on analysis and measure theory: https://www.youtube.com/c/brightsideofmaths/videos
While ScienceClic's coverage of differential geometry topics is mostly a thin vertical slice of the subject, it covers it with incredible visualizations and pedagogical approach: https://www.youtube.com/watch?v=xodtfM1r9FA&list=PLu7cY2CPiR....
In my country, this also describes the day to day situation. You can't trust doctors because they make up diagnoses, people pay taxes but it never shows up in infrastructure, schools or anywhere; daily alerts by banks warning against some new clever form of fraud, until recently there was a fair chance new banks would vanish with customer money (I still only trust a few reputable banks), actual family destroying pyramid schemes, ransoms attached to child kidnappings not data, politicians stealing from their constituents, police not trusted because of lack of transparency and corruption, bribes as a defacto tax.
The combination of high levels of poverty and lack of regulation is what leads to the proliferation of scams and dishonest behaviors. Poverty lowers the threshold of criminality, lack of regulation and lack of an enforced justice system enables its flourishing. Poverty also catalyzes more susceptible victims of such crimes.
-------
> only in the cryptocurrency debate do defenders have to offer up a disclaimer
Financial activity in crypto lacks regulation, which attracts predators, who create scams that attract the poor who are not necessarily gullible, just gambles from desperation.
> Android or iPhone
In Android vs iPhone or Windows vs Mac, people do argue about vulnerability to malware and the merits and demerits of locked down platforms.
I'd also argue your examples are not appropriate, they're more clashes of belief systems and less people freely interacting in market like scenarios. Consider that kickstarter is rife with vaporware, hype and scams. Scams, fraud and hacks in Roblox and Entropia. Crime, speculation and hacks in Habbo Hotel. Western financial systems discriminate against certain professions and citizens of entire countries due to some cost benefit calculations related to fraud.
If you have a system where money can exchange hands, you'll find psychopaths preying on the poor and careless. Oh and this isn't just poor countries and virtual markets, psychopaths preying on the poor and desperate also occurs for developed countries too (you just give them fancy names like debt bubbles and mortgage crises).
The combination of these features: a distributed, open, permissionless, minimal subjectivity, practical double spend immunity, and malicious actor or sybil resistance log of transactions, are what make it as it is and difficult to improve upon without weakening one of those aspects. To answer your why questions would be to defend those properties.
My experience is people from oppressed corruption filled countries tend to be sympathetic of those properties and those from wealthy stable countries tend to think them not worth the cost. It probably can't be used as a global currency but can (in theory) be there for scenarios where trust and subjectivity tolerance are minimal.
> isn't this a ponzi scheme
I would say no, it's more like a speculative bubble and casino that breeds pump and dump schemes. If most participants are uninformed then it's a scam, if most are informed it's senseless gambling. There are some legit uses such as funding civil disobedience or hedging against serious inflation in countries with constrained access to dollars (this usecase is why it fails to be a ponzi scheme, you don't want value to increase, stability is preferred).
Or a tad bit more risky, you could email it.
Despite barely enforced laws, people are mostly good and largely follow rules. Human trafficking, armed robberies and kidnappings are a lot higher than in wealthier countries but it's still only a minority who succumb to that. It's mostly scams (including having a business but not paying employees) that a significant number participate in.
With property, those that have less respect each other and those that have much more also have gated compounds and private security (among which we can count having police on call). It sounds weird but in disputes, even absent an actual justice system, a verifiable paper trail or record still carries a great deal of weight.
Having suffered a lot from it, I think a lot about corruption. I've also read a fair few number of papers on the topic, from the sociological to the game theoretical.
Lack of accountability and lack of transparency are major enablers. Public blockchains are of interest because of their (pseudonymous) transparency and virtually tamper-proof nature. Not a solution but of interest. A fully transparent and neutral system or layer of payment of last resort, where the powerful cannot forge records, for use where trust is non-existent would be useful.