phowon

My roommate has a cat who mostly hangs out in the living room. I never had pets growing up, so I never interact with it. Every so often, when I'm walking past it to get around the apartment, it swipes at/scratches me, even drawing blood at times.

Did I also have it coming?

India is generally not considered to be in South-East Asia.

As someone who aggressively uses conda, I do think one of its downsides is how heavy it is. I agree that it's a good one-stop-shop if you want to get an environment up and running with no issues. But if you're not doing any scientific computing for a given project, it might help to use something more lightweight.

And here is a tweet thread on why using NLP models to "fill in" the redacted portions is a horrendously terrible idea.

https://twitter.com/emilymbender/status/1119081131234611201

If anything, I've found Airpods to be extremely visible given their stark white color. As opposed to darker bluetooth headpieces that I often missed.

>I've definitely noticed a much stronger emphasis on wowing visual effects in today's RPG's and a much weaker focus on the story and plot lines as was the case in past games.

This basically started in the PS3 era, where it appeared that the casual consumer's bar of visual graphics were raised high enough that pushed up development costs, to the point where developer couldn't cheaply churn out interesting, experimental titles.

Many people will cite the SNES and PS1 era as the golden age of JRPGs, but I think the PS2 era is under-appreciated. The SNES/PS1 era had a lot of classics, but the PS2 era was absolutely flooded with great, 8/10 JRPGs across all manner of series, benefiting from the gameplay/UI/UX refinements learned from the SNES/PS1 era and the improved hardware capabilities of the PS2. You had participation across all manner of series: from your popular Final Fantasies and Dragon Quests, to Tales, Persona, Star Ocean, Suikoden, Wild Arms, Breath of Fire, Arc the Lad and so on. You had a whole generation of new entrants like Radiata Stories, Shadow Hearts, Atelier, Dark Cloud, Rogue Galaxy, Xenosaga, .Hack, and many others. Not all of these were amazing, but most of these were at least very good, and in particular they were diverse while also being streamlined as some of the visual/gameplay "language" JRPGs become more firmly established. To me, this was the last great age of JRPGs.

>As an example, there are many people who like grinding in JRPGs and some even consider it as a defining element of JRPGs (in that a JRPG is not real JRPG if it doesn't have grinding)

I am one such person who is happy to argue for the fact that ability to grind is one of the (maybe two?) defining traits of a JRPG. The option to meaningfully strengthen your player avatar that is not tied to story/game progression is the heart of JRPGs.

I know this talking point has been brought up by different people, but it's worth pointing out that Transformers were already covered in the class in 2018. https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1184...

The success of Transformers aside, I'm not sure you should be relying on model titles for anything, lest we forget papers like "One Model To Learn Them All" [1].

[1] https://arxiv.org/abs/1706.05137

Aren't causal convolutions basically CNNs with masking?

Putting aside the whole Schmidhuber debate - where are people getting this idea that causal convolutions are anywhere near the prominence of RNNS/LSTMs?

As far as I'm aware, causal convolutions were used in WaveNet (and subsequent models) and a small number of NLP applications. Meanwhile, LSTM-based models are used in just about every NLP paper, and at least a baseline in the newer ones more dominated by Transformers.

The Hug of Death seems to have killed the page so I can't really tell what it's about, but here's an example of inserting new trainable layers in BERT: https://arxiv.org/abs/1902.00751

There is no constraint that companies must maximize shareholder value. Companies can be set up for any number of reasons (including non-profit reasons).

What is it generally used for though? To me, a "k.v" store sounds like a very generic but nice thing to have, but I still don't have a good sense of what it is and what people think of it.

The one place I've run into it is in web development where it's used for caching? In some tutorials I've read.

As someone who is fairly tech literate but not familiar with this tech stack - in practical terms, what is Redis, and what is it used for?

The BERT paper also introduced BERT Base, with is 12 layers with approximately the same number of parameters as GPT, but still outperforms GPT on GLUE.

With ELMo, the pretrained weights are frozen. Only the scalars for ELMo layers are tuned (as well as the additional top-level model, of course).

It's relatively small modification of BERT with multi-task fine-tuning and slightly different output heads. It should be easy for any NLP researcher to replicate.

Put another way, even if you take the strict definition of a rational (optimal) agent, in probably any non-trivial or non-contrived corner case, for any arbitrary set of actions, there exists some utility function and information constraint that can justify that set of arbitrary functions.

Have a set of "irrational actions"? Add in information and computational constraints and the necessity for heuristics, the exact right kind of risk-aversion / novelty-seeking behavior, (or play with the second or higher derivatives of your utility function), add in social dynamics like signaling, repeated games (or incorporate mental burdens that negate the impact of repeated games) etc. There exists some formalized universe where your agent is indeed rational.

I am aware of the origin of the word. I am distinguishing between the generic notion of a meme (which as you pointed out simply means "transmittable idea", which is an incredibly general concept) and the specific notion of an Internet Meme (and, as in OP, 'meme culture', which surely refers to Internet meme culture, and not the culture of transmittable ideas in general).

The later misuse of the word meme to refer only to image macros is also a separate shift.

That's not what's being discussed here. We're talking about when the word 'meme' was repurposed to refer to "Internet trend".

Outsiders to 4chan would probably also be surprised at the site-wide outpouring of genuine sadness, nostalgia and gratitude when moot announced that he was stepping down.

Yes! I needed someone to confirm that people used to call it "fads" before memes.

>For example, in jointly learning to align and translate, the attention is certainly not invariant to number of context vectors. You train the attention to take in a fixed number of context vectors and produce a distribution over the fixed number of context vectors

That's not true.

You compute an attention weight across however many context steps you have by computing an interaction between some current decoder hidden state and every encoder hidden state, and normalizing over all of them via a softmax. There is no constraint whatsoever on a fixed context length or a fixed number of context vectors. See section 3.1 in the paper.

I will be happy to discuss and clarify over email.

So I'm curious - is this considered blackmail if the object isn't money?

It really is underappreciated how much 4chan has shaped Internet, and now mainstream, culture.

I'm not sure if you're understanding me correctly.

Attention is generally length invariant. You take some transformation on the hidden representations (/+ inputs) at that each time step, and then you normalize over all the transformed values to get weights that sum to one. No part of this is constrained by length.

For CNNs, any network that has pooling has the potential to be length/dimension invariant. Whether it actually is is a combination of the architectural design and an implementation detail (e.g. some implementations when trying to pool will specifically define a pooling operation over, say, a 9x9 window. You could define the same pooling operation over a variable-dimension window).

The length/dimension invariance aren't a special or novel property. In the case of attention it's built in. In the case of CNNs, the convolutions are not length invariant, but depending on the architecture, the pooling operations are (or can be modified to be).

ULMFiT is LSTM-based. https://arxiv.org/pdf/1801.06146.pdf

And convolution-based models still find use in all sorts of cool applications in language, such as: https://arxiv.org/abs/1805.04833

With regards to adversarial discussions, it's one thing to argue about whether method A or method B gives better results in a largely empirical and experimental field. But giving a very misleading characterization of a model is actively detrimental especially when it would give casual readers the impression that the Transformer is a "convolution-based" model, which no one in the field would do.

Like I said - pooling.

You can take the mean over 3 elements or 10 elements just the same. Pooling is lossy, but it seems that if you have the right architecture the model can still learn what it needs to.

It's worth noting that the attention mechanism (at least in RNNs) has always been invariant to inputs lengths. It's a weighted sum with weights computed per element, so there's no length constraint at all.