jointpdf

A condo in GR is a possibility for sure, though you have to admit the market there has gotten ridiculous in recent years. Most of the decent houses near downtown get multiple offers instantly (same in Kalamazoo).

Is $1250/mo a realistic amount to charge a roommate in GR?

Indeed, my shortlist is Chicago (and maybe Grand Rapids) at this point. I’ve always been in love with Chicago—such a gorgeous city. Housing wise, I’m enchanted by the industrial brick and timber loft condos that there seem to be a lot of for decent prices.

Can you recommend other neighborhoods besides Lakeview/Lincoln Park? Those are the only areas I’ve spent significant time in aside from downtown.

By “research” I really meant “searching, then searching again...and again”, not actual research. Word choice is important in HN comments because you all are too sharp.

I haven’t looked at that, but I have wondered how universal satellite internet coverage and the more permanent (?) move towards remote work will affect rural areas. It is an interesting trend to keep an eye on.

Housing (and construction) prices have been ballooning out of control everywhere for the last several years. Both major cities (DC, Boston, Portland, Seattle, Chicago, LA, Denver, San Diego) and secondary cities (Boise ID, Grand Rapids MI, many more) are all totally unaffordable. I’ve spent hundreds of hours researching different areas over the last year. Looking at sold homes from just 2-3 years ago, many seem like a steal compared to what’s available now. It’s just extremely bleak.

Meanwhile, my landlord bragged about buying the house I live in with no money down and no income check during the subprime mortgage era, and has been renting it out since. Now he is selling it during a pandemic and I’m about to become a vagabond.

I do think the vacuuming up of affordable housing by real estate investors (perhaps many of whom accumulated wealth via the surging stock market) is at least part of the problem. Landowners have strong incentives to keep the housing stock low (and the rents high), or they get washed.

The term shovelware reminds me of the E.T. incident. A game so bad that Atari literally got out the shovels:

https://en.m.wikipedia.org/wiki/Atari_video_game_burial

I’m a huge fan of UMAP, but this [0] paper suggests that t-SNE can be tuned to produce UMAP-like results (the algorithms are extremely similar—you can recover t-SNE with certain UMAP parameter choices). One of the insights is to use PCA first to better preserve the global structure.

For example, see figure 9 in the paper: the plot on the left is the typical result of default t-SNE (distance between global structures not well-represented, since everything is jammed together), and the plot on the right is very UMAPish.

Basically, there are a lot of preprocessing and parameter choices involved in producing these embedding plots, so it’s advisable to try to understand the effects of these choices regardless of which algorithm you choose.

[0]: https://www.nature.com/articles/s41467-019-13056-x

I did feel genuinely bad for my comment, since it was a few shades nastier than I was going for and I try hard to be positive and not cut people down. It was mean spirited and I apologize.

My point was that intelligent humans can and often do make mistakes in logic and computation (arithmetic) in ways that machines typically do not. One reason may be colliding or incomplete representations of certain concepts, and (relatedly) the fact that we are relying on language. I think of neural networks as fuzzy representation composers, so it seems they also fail for similar reasons. Basically, it (GPT) does have some layered representation of the concept of numbers and how they are used in different contexts which gives it some faculty at carrying out common operations, but it doesn’t “add up” to a reliable system of logic (that would allow it to extend addition to say 100-digit numbers, the way even a sharp and/or patient 2nd grader could do, generalizing from the simpler cases).

I think accuracy is sometimes the correct measure, and in this instance it seems fine—at baseline, we should expect ~0% accuracy since it is generating output from essentially the space of all possible text (texts <= 2048 tokens). I agree that it would be interesting to probe the model with better tests, and understanding when/why it fails on certain arithmetic problems or types of reasoning.

I liked what you wrote about finding heuristics, though I disagree with your conclusion that heuristic finding does not qualify as learning—it is just somewhere along the spectrum between a randomized model and an ALU (neither of which can be said to have learned anything) in terms of its ability to perform arithmetic.

Of course, we already have better models for solving proofs and such, so I generally think the way toward more complete AI models is to return to the system design view of AI (meta-learning, integration of different models, etc) rather than trying to evolve one colossal model to rule them all. That is, a meta-model that recognizes what sort of problem it is facing, then selecting a model/program to solve or generate possible solutions to that problem, while revealing or explaining as much of this process as possible to the user.

In any case, I have definitely have more to read on the subject and am mostly musing at this point. Thanks for the references and the conversation.

I’m not attacking you, which is why I immediately offered you a salve for the burn by marking my comment as snarky. It did have a real point though—that learning (even something inherently logical like arithmetic) is not a binary outcome, as fossuser also pointed out.

The claim that “GPT-3’s performance on arithmetic tasks is solely due to memorization / data leakage—it has no generalization ability on this type of task”, is easily attackable by...well, doing arithmetic and applying common sense.

There are 2(10^5)^2 = 20 billion possible 5 digit problems (both addition and subtraction). The accuracy on those tasks is about 10%, so roughly 2 billion* 5-digit addition and subtraction problems would need to be represented in the training data (Common Crawl + books as you said). Each problem is at minimum 5 tokens (e.g. 99999 + 11111 = 111110). So is ~2.5% of the training corpus 5-digit addition and subtraction problems that eluded their filtration process? (assuming it’s ~400B tokens like you said). Seems exceedingly unlikely, so much so that memorization ceases to be the simplest explanation.

That said, yes it is surprising that a language model can generalize in this way—that’s the point of the paper. How exactly this happens seems like a valuable thread to pull. Your critiques may help, but writing the results off as impossible magic does not.

I’m just gonna leave these here and see if anyone has relevant experiences to share:

> ”There is something about the core of this experience that opens people up to the great mystery of what it is that we don’t know.”

https://www.scientificamerican.com/article/psilocybin-a-jour...

> ”Of the participants who received the high dose in the second study, 83% reported feeling significantly less depression and 58% reported less anxiety after 7 weeks. Only 14% of those who received niacin reported less anxiety and less depression.”

https://www.sciencemag.org/news/2016/12/hallucinogenic-drugs...

Your reply is misleading, sorry. You didn’t offer any actual evidence of your own to support the memorization claim. You didn’t even do your own arithmetic problem correctly. Since your performance on this task was <100% accurate, I can only assume you do not know the rules of arithmetic.

> How many "x + y" questions can be formulated where x and y are both single-digit numbers? The answer is 2^10, or 100.

Less snarkily, if there’s (10^4)^2 = 100 million combinations of 4 digit addition problems, and GPT-3 is reaching 25.5% accuracy on those problems (vs. 0.4% in the 13B parameter model). For 3 digit problems, it’s even better: 1 million combinations and 80.4% accuracy. Clearly, there is more happening than simple memorization—the training set does not contain 800k 3-digit addition problems. Thus, it’s fair to say that the model has at least a partial grasp of how to perform arithmetic operations (but probably not fair to say that it has synthesized the entire system of arithmetic).

Also, the paper does say that they scrubbed exact examples from the training set to avoid memorization, a fact you left out:

> (pg. 23): ”To spot-check whether the model is simply memorizing specific arithmetic problems, we took the 3-digit arithmetic problems in our test set and searched for them in our training data in both the forms "<NUM1> + <NUM2> =" and "<NUM1> plus <NUM2>". Out of 2,000 addition problems we found only 17 matches (0.8%) and out of 2,000 subtraction problems we found only 2 matches (0.1%), suggesting that only a trivial fraction of the correct answers could have been memorized. In addition, inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”, suggesting it is actually attempting to perform the relevant computation rather than memorizing a table.”

This (GPT-3’s performance on arithmetic tasks) is covered in the original paper (https://arxiv.org/abs/2005.14165)

Pages: 21-23, 63

So this article doesn’t cite the original source, but I think it’s this (HUD and the Census Bureau are part of Commerce): https://www.census.gov/construction/nrc/pdf/newresconst.pdf

This puts everything in much better context, especially the time series plot. It seems home starts / permits were already increasing from July 2019 - Dec 2019, matching the July 2020 levels. So the better question might be: why that surge?

Thanks for this, it was interesting and helpful. My comment was mostly appreciating the sardonic and dry humor in the poster (and the concept of the study itself).

It was my first time hearing about this, and it struck me as the type of funny anecdote that could be slipped into a talk/lecture (I find it takes quite a bit of finesse to keep non-technical people engaged when talking about statistics and data science, so humor and relatable examples always help). So, it’s good to know the broader context.

> Subject: One mature Atlantic Salmon (Salmo salar) participated in the fMRI study. The salmon was approximately 18 inches long, weighed 3.8 lbs, and was not alive at the time of scanning.

> Task: The task administered to the salmon involved completing an open-ended mentalizing task. The salmon was shown a series of photographs depicting human individuals in social situations with a specified emotional valence. The salmon was asked to determine what emotion the individual in the photo must have been experiencing.

This is just so, so good. Thank you to this salmon for participating in science.

I’ve heard (way) more than one story about MongoDB mysteriously annihilating all the data. Launched into oblivion.

The worst imaginable outcome for a database.

I should have made it more clear that I was trying to recreate the mental model that my teacher’s explanation gave me, at the time. I realized after posting it that my model was wrong—I know it’s really the other way around.

This is presumably only one aspect of “long Covid”, but a study [0] that was making the rounds a couple weeks ago found this (quoted from the accompanying editorial)[1]:

> ”In 100 recovering patients included in the study, 67% of whom recovered at home, evaluated a mean of 71 days after confirmed COVID-19 diagnosis, 78% had demonstrable cardiac involvement via cardiac magnetic resonance imaging, 76% had detectable high-sensitivity troponin, and 60% had evidence of active myocardial inflammation by abnormal native T1 and T2.”

> ”...if this high rate of risk is confirmed, the pathologic basis for progressive left ventricular dysfunction is validated, and especially if longitudinal assessment reveals new-onset heart failure in the recovery phase of COVID-19, then the crisis of COVID-19 will not abate but will instead shift to a new de novo incidence of heart failure and other chronic cardiovascular complications.”

In other words, even mild cases of Covid-19 are very frequently resulting in heart inflammation and other forms of cardiac injury.

[0]: https://jamanetwork.com/journals/jamacardiology/fullarticle/...

[1]: https://jamanetwork.com/journals/jamacardiology/fullarticle/...

[2]: (another editorial, easier read) https://www.statnews.com/2020/07/27/covid19-concerns-about-l...

So is the claim by Elon Musk that current iterations of Tesla vehicles have all of the sensors and compute power needed to be fully autonomous (Level 4+ I guess?) in the future, via software updates only, a specious one?

Admittedly, my understanding of thermodynamics has not progressed much beyond the 2nd grade level (aside from a class on PDEs giving me at least some enhanced intuition). My takeaway from the PDE class was that anything beyond a simple textbook problem needs to be simulated and/or observed experimentally, rather than solved for analytically.

But other things being equal, wouldn’t cold air permeate a less dense barrier (say a layer of insulation in a home) more easily than a dense barrier? Just as water (or cold air) would permeate soil more readily than rock.

I have a vivid memory of being taught this in 2nd grade, probably because it seemed (and still seems) extremely counterintuitive. I think the intuition given was that hot water is less dense than cold water, so the coldness can more easily permeate through the liquid (I imagine a vein-like pattern). Something like that.

At least according to Wikipedia, it seems that it’s a bit more complex: https://en.m.wikipedia.org/wiki/Mpemba_effect

Anyways, I’ve been repeating this fact for 2.5 decades, so shout out to my teacher for blowing my young mind.

That stood out to me as well. I think it meant to say: 2% of children who were diagnosed with the multisystem inflammatory syndrome (MIS-C) died from it.

> 80% of the children who developed the condition required intensive care, 20% required mechanical ventilation, and four children, or 2%, died. [https://www.statnews.com/2020/06/29/nejm-inflammation-childr...]

How would you interpret the work of Brian Eno, or other algorithmic/generative artists? Steve Reich’s compositions are algorithmic (but not stochastic), and are widely considered to be masterpieces. Another example: Eno’s Reflection, which exists as both a 54 minute version and an endless generative ambient composition.

> The creation of a piece of music like this falls into three stages: the first is the selection of sonic materials and a musical mode – a constellation of musical relationships. These are then patterned and explored by a system of algorithms which vary and permutate the initial elements I feed into them, resulting in a constantly morphing stream (or river) of music. The third stage is listening. Once I have the system up and running I spend a long time – many days and weeks in fact – seeing what it does and fine-tuning the materials and sets of rules that run the algorithms. It’s a lot like gardening: you plant the seeds and then you keep tending to them until you get a garden you like.

> Moving the composition into software allowed an extra opportunity; the rules themselves could change with the time of day. The harmony is brighter in the morning, transitioning gradually over the afternoon to reach the original key by evening. As the early hours draw in, newly introduced conditions thin the notes out and slow everything down. [https://pitchfork.com/news/70448-brian-eno-details-generativ...]

To me, crafting a system that itself crafts endlessly novel music or art requires significant level of intention, understanding, and meaning.

The most disruptive errors are those that could seemingly be cleaned up with some simple rules (e.g. if the word is in the dictionary, then don’t change it). This doesn’t seem like a “state of ML” issue—the Gmail text completion is annoyingly good, and you-know-who can compose soliloquies and smut with uncanny skill.

And no, I don’t often discuss The Rock, but if I did (“Do you smell what the r...”) I'd expect it to finish the alley-oop. I do not want to go Rock climbing.

iMessage is software that is used <Carl Sagan voice> billions and billions of times a day, and half the time it feels like verbal jiu-jitsu (“jiu-hursuit” according to autocorrect just now) with Dory the fish.

Setting aside all the other work Apple is doing in this area, can someone explain why iMessage autocorrect is unacceptably bad (to put it diplomatically) in the era of weapons-grade language models?

Its behavior is totally bizarre. It’s like an underpowered chess engine that makes flagrant blunders: capitalizing random words in the middle of a sentence (rock -> Rock), the context sensitivity of an actual rock, forcing the same correction multiple times (i.e. you go back to fix its error, and it defiantly repeats it), contraction mixups, blindness to off-by-one-keystroke errors (consentuallt -> <no action>), and of course the occasional random word substitution. Only martial artists want to duck people (and it—no joke—just now substituted [duck -> suck]. You had one job autocorrect.)

Is this just me? Is it actually 2020? What is going on?

I have (semi-)frequently gone “off-grid” far away from established trails, without carrying a PLB. Sometimes I’ve gone days without seeing another human. I have extensive training in wilderness navigation, and have always reviewed my itinerary with the rangers ahead of time.

In hindsight, I can see how this was a selfish decision.

This feeling has been nagging at me for a while, since I’ve had close calls in remote areas that would have jeopardized the inevitable SAR crews.

So, I’ll acquire a PLB and carry it on future trips. Thank you for the reminder.

That said, I think some of my hesitation in the past was rooted in this belief that PLBs were unreliable and mostly a nuisance due to the abuse by inexperienced hikers. Also, if you are in cell range (like on the majority of the AT), I believe that a phone is the superior device since you can communicate the severity of your emergency to emergency services and they can respond accordingly. I’d like to read more about this if you have some links:

> ”now rescue organizations all over the planet are asking you to please...carry a goddamn PLB”

Corroborating the sister comment here—you are unlikely to be charged unless you do something grossly negligent—something like ignoring explicit warnings not to climb Mt. Whitney in adverse conditions and going unpermitted.

I can’t speak for what the policies are worldwide, but at least in the US (in particular California), you are very unlikely to be charged for the rescue services (medical costs are a different story). Usually the county is responsible for the SAR operation, which is typically carried out by volunteers. If you’re in a National Park, then it’s NPS. Again, they could fine you, but my understanding is that this is rare. Volunteers will even come looking for you for years after you’ve disappeared (https://www.climbing.com/people/vanished-the-mysterious-disa...).

Anecdote: I knew a (European) PCT hiker who was airlifted out of the Sierra. They were diabetic and had suffered a cut that for a healthy person would have been non-threatening. They were not charged anything. It would have been a shame if they hesitated due to fear of getting slapped with a bill.

Old but useful threads (and great forums in general—the HNs of backcountry travel):

https://backpackinglight.com/forums/topic/14464/

http://www.highsierratopix.com/community/viewtopic.php?t=893...

> The vast majority of searches I have been on have no medical component whatsoever - in fact, none of the searches in the past several years, whether I have been on them or not, have involved medical intervention. A few have involved finding remains, which allow families closure and to make claims on life insurance - a community service that makes creeping through manzanita picking up bone shards worthwhile. If my family was depending on the income from life insurance and I vanished - I'd hope search teams found what was left of me. Some searches are related to law enforcement activities and locate evidence. Some searches are urban, and locate Alzheimer's sufferers or small children lost in town. None of them costs anyone a dime, and we don't appreciate the perpetuation of the myth that anyone is charged because we don't want to see anyone delay getting help when it's clear someone is missing.”

> ”Which is why our search team has never and will never charge anything. We are a self supporting nonprofit organization. We buy our own gear, pay for our own training, and raise the funds to replace team gear. I just wrote a grant proposal to get us new gps units and will shortly submit another. Madera, Merced, Fresno, Tulare, Inyo and other counties are all independent agencies, nonprofits, and do not charge a cent.”

You might enjoy Oliver Knill’s lecture notes on undergrad math topics, which are similarly concise: http://people.math.harvard.edu/~knill/teach/index.html

Stop deliberately misrepresenting the truth and make an actual argument. “It will never happen” is not a logical argument. Clearly, the level of action is below what is needed—but that does not imply it is impossible. The US is the major domino that needs to fall (and has been for decades).

Here are some facts from (https://openknowledge.worldbank.org/bitstream/handle/10986/3...):

* “As of April 1, 2019, 57 carbon pricing initiatives have been implemented, or are scheduled for implementation. This consists of 28 ETSs, spread across national and subnational jurisdictions, and 29 carbon taxes, primarily implemented on a national level. In total, as of 2019, national and 28 subnational jurisdictions are putting a price on carbon”

* “Of the 185 Parties that have submitted their Nationally Determined Contributions (NDCs) to the Paris Agreement, 96—representing 55 percent of global GHG emissions—have stated that they are planning or considering the use of carbon pricing as a tool to meet their commitments.”

Initiatives implemented or scheduled for implementation:

* National ETSs: Australia, Austria, Belgium, Bulgaria, China, Croatia, Cyprus, Czech Republic, Germany, Greece, Hungary, Italy, Kazakhstan, Lithuania, Luxembourg, Malta, the Netherlands, New Zealand, the Republic of Korea, Romania, and Slovakia.

* National carbon taxes: Argentina, Chile, Colombia, Japan, Mexico, Singapore, South Africa, and Ukraine. Both national ETSs and carbon taxes: Canada, Denmark, Estonia, Finland, France, Iceland, Ireland, Latvia, Liechtenstein, Norway, Poland, Portugal, Slovenia, Spain, Sweden, Switzerland, and the United Kingdom. Subnational ETSs: Beijing, California, Chongqing, Connecticut, Delaware, Fujian, Guangdong, Hubei, Maine, Maryland, Massachusetts, New Hampshire, New York, Nova Scotia, Québec, Rhode Island, Saitama, Saskatchewan, Shanghai, Shenzhen, Tianjin, Tokyo, Vermont, and Washington State.

* Subnational carbon tax: Prince Edward Island.

* Both subnational ETSs and carbon taxes: Alberta, British Columbia, Newfoundland and Labrador.

Note that an ETS (emissions trading scheme, aka cap-and-trade), is effectively equivalent to a carbon tax in terms of economic theory, though their implementations obviously differ.

Completely pointless comment. It is already happening in many ways.

https://www.vox.com/energy-and-environment/2017/6/15/1579620...

Could you take (embeddings of) frames from your query video and measure the pairwise similarity to frames from the reference videos, then rank by some summary function of that list of similarity scores?

What are the bottlenecks besides performance?